-- Logs begin at Sat 2022-05-28 20:01:13 UTC, end at Sat 2022-05-28 22:29:27 UTC. -- May 28 20:40:21.370274 ncdv-master-1 systemd[1]: Starting Devstack devstack@keystone.service... May 28 20:40:21.378936 ncdv-master-1 devstack@keystone.service[103091]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 28 20:40:21.380922 ncdv-master-1 devstack@keystone.service[103091]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat May 28 20:40:21 2022] *** May 28 20:40:21.380922 ncdv-master-1 devstack@keystone.service[103091]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 28 20:40:21.380922 ncdv-master-1 devstack@keystone.service[103091]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 28 20:40:21.380922 ncdv-master-1 devstack@keystone.service[103091]: nodename: ncdv-master-1 May 28 20:40:21.380922 ncdv-master-1 devstack@keystone.service[103091]: machine: x86_64 May 28 20:40:21.380922 ncdv-master-1 devstack@keystone.service[103091]: clock source: unix May 28 20:40:21.380922 ncdv-master-1 devstack@keystone.service[103091]: pcre jit disabled May 28 20:40:21.380922 ncdv-master-1 devstack@keystone.service[103091]: detected number of CPU cores: 4 May 28 20:40:21.380922 ncdv-master-1 devstack@keystone.service[103091]: current working directory: / May 28 20:40:21.380922 ncdv-master-1 devstack@keystone.service[103091]: detected binary path: /usr/bin/uwsgi-core May 28 20:40:21.380922 ncdv-master-1 devstack@keystone.service[103091]: your processes number limit is 47680 May 28 20:40:21.380922 ncdv-master-1 devstack@keystone.service[103091]: your memory page size is 4096 bytes May 28 20:40:21.380922 ncdv-master-1 devstack@keystone.service[103091]: detected max file descriptor number: 2048 May 28 20:40:21.381659 ncdv-master-1 devstack@keystone.service[103091]: lock engine: pthread robust mutexes May 28 20:40:21.381659 ncdv-master-1 devstack@keystone.service[103091]: thunder lock: enabled May 28 20:40:21.381659 ncdv-master-1 devstack@keystone.service[103091]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 28 20:40:21.381659 ncdv-master-1 devstack@keystone.service[103091]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 28 20:40:21.430472 ncdv-master-1 devstack@keystone.service[103091]: Python main interpreter initialized at 0x5622faaad9c0 May 28 20:40:21.430472 ncdv-master-1 devstack@keystone.service[103091]: python threads support enabled May 28 20:40:21.430472 ncdv-master-1 devstack@keystone.service[103091]: your server socket listen backlog is limited to 100 connections May 28 20:40:21.430472 ncdv-master-1 devstack@keystone.service[103091]: your mercy for graceful operations on workers is 90 seconds May 28 20:40:21.430803 ncdv-master-1 devstack@keystone.service[103091]: mapped 403077 bytes (393 KB) for 2 cores May 28 20:40:21.430879 ncdv-master-1 devstack@keystone.service[103091]: *** Operational MODE: preforking *** May 28 20:40:21.431126 ncdv-master-1 devstack@keystone.service[103091]: *** uWSGI is running in multiple interpreter mode *** May 28 20:40:21.431126 ncdv-master-1 devstack@keystone.service[103091]: spawned uWSGI master process (pid: 103091) May 28 20:40:21.431317 ncdv-master-1 systemd[1]: Started Devstack devstack@keystone.service. May 28 20:40:21.432877 ncdv-master-1 devstack@keystone.service[103091]: spawned uWSGI worker 1 (pid: 103103, cores: 1) May 28 20:40:21.432877 ncdv-master-1 devstack@keystone.service[103091]: spawned uWSGI worker 2 (pid: 103104, cores: 1) May 28 20:40:21.432877 ncdv-master-1 devstack@keystone.service[103091]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 28 20:40:27.230523 ncdv-master-1 devstack@keystone.service[103104]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 20:40:27.230523 ncdv-master-1 devstack@keystone.service[103104]: from cryptography.utils import int_from_bytes May 28 20:40:27.231297 ncdv-master-1 devstack@keystone.service[103104]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 20:40:27.231297 ncdv-master-1 devstack@keystone.service[103104]: from cryptography.utils import int_from_bytes May 28 20:40:27.231433 ncdv-master-1 devstack@keystone.service[103103]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 20:40:27.231433 ncdv-master-1 devstack@keystone.service[103103]: from cryptography.utils import int_from_bytes May 28 20:40:27.231433 ncdv-master-1 devstack@keystone.service[103103]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 28 20:40:27.231433 ncdv-master-1 devstack@keystone.service[103103]: from cryptography.utils import int_from_bytes May 28 20:40:27.280348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 28 20:40:27.280547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 28 20:40:27.280728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 28 20:40:27.280829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] command line args: [] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 28 20:40:27.280915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 28 20:40:27.280989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 28 20:40:27.281106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] command line args: [] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 28 20:40:27.281185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ================================================================================ {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 28 20:40:27.281382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 28 20:40:27.281464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] admin_token = **** {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.281570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ================================================================================ {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 28 20:40:27.281652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] config_dir = [] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.281813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.281924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] admin_token = **** {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.282029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] config_source = [] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.282152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] config_dir = [] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.282249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] debug = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.282334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.282408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.282644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] config_source = [] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.282644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] debug = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.282763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.282824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.282911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] insecure_debug = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.283155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.283240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.283313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] insecure_debug = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.283379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.283438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.283609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] list_limit = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.283700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.283836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] log_config_append = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.283929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] list_limit = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.283997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.284062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] log_config_append = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.284126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] log_dir = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.284293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.284374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] log_file = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.284442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] log_dir = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.284508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.284572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] log_file = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.284750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.284831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.284998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.285595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.285782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.285912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.285999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.286064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.286138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.286312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.286427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.286508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.286626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.286718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.286868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.286947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.287108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.287202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.287344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.287435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.287579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.287657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.287817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.287939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.288063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] notification_format = cadf {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.288149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.288310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] notification_format = cadf {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.288394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.288589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.288821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.289007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] publish_errors = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.289135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.289244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.289332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] publish_errors = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.289477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.289562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.289641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.289758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.289896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.289985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.290190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.290276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.290348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] standard_threads = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.290457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.291258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] strict_password_check = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.291377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] standard_threads = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.291464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.291542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] strict_password_check = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.291614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] use_eventlog = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.291685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.291763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] use_journal = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.291836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] use_eventlog = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.291906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] use_json = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.291975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] use_journal = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.292061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] use_stderr = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.292139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] use_json = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.292222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] use_syslog = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.292301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] use_stderr = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.292377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] watch_log_file = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.292448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] use_syslog = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.292518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.292595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] watch_log_file = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:40:27.292673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.292749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.292821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.292890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.292964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.293043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.293121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.293198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.293269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.293344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.293493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.293585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.293734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.293813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.293995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.294076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.294247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.294336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.294493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.294576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.294727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.294806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.294964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.295117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.295199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.295270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.295440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.295519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.295686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.295772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.295924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.296007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.296162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.296240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.296421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.296508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.296666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.296744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.296905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.296982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.297141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.297276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.297397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.297476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.297651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.297728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.297926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.298023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.298168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.298254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.298398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.298484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.298628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.298716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.298857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.298934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.299095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.299174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.299342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.299421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.299576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.299656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.299807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.299884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.300041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.300121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.300289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.300374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.300525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.300602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.300838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.301459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.301588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.301677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.301823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.301965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.302086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.302168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.302323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.302402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.302556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.302635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.302788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.302869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.303019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.303108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.303247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.303331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.303485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.303573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.303717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.303806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.303952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.304030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.304183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.304271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.304423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.304663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.304741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.304909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.304993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.305140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.305221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.305382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.305461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.305617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.305695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.305866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.305950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.306108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.306188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.306343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.306427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.306575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.306658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.306809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.306887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.307040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.307185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.307270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.307343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.307492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.307581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.307738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.307816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.307961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.308039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.308194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.308274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.308456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.308665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.308877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.308971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.309122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.309204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.309365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.309445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.309605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.309687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.309856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.309936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.310103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.310183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.310340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.310423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.310588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.enabled = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.310674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.310824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.310904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.311060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.311189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.311291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.311372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.311528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.enabled = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.311608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.311763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.311849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.311998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.312126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.312232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.312312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.312466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.312544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.312703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.312793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.312938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.313026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.313175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.313260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.313411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.313492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.313654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.313734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.313915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.313998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.314158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.314239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.314395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.314473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.314623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.314701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.314853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.314936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.315085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.315164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.315317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.315403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.315545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.315624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.315774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.315854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.316005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.316093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.316242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.316323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.316496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.316575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.316740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.316829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.316977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.317061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.317228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.317307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.317462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.317549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] auth.external = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.317693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] auth.mapped = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.317772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.317992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.318074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.318237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.318328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.318468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] auth.password = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.318572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] auth.external = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.318698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] auth.mapped = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.318779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] auth.token = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.318937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.319017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.319175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] catalog.caching = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.319266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.319425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.319505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] auth.password = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.319653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] auth.token = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.319731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.319894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.319990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.320130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] catalog.caching = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.320215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.320360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.320440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.320597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.320684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] credential.caching = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.320841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] credential.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.320921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.321085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.321166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.321317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.321469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.321547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] credential.caching = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.321619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.321784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.321885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] credential.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.322046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.322140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.322305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.322386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.322544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.322624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.322783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.322871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.323015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.323094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.323277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.323357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.323531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.323620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.323768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.323849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.324019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.324099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.324278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.324367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.324517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] federation.caching = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.324598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.324756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.324843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.325004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] federation.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.325094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.325252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.325332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.325486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.325568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] federation.caching = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.325722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.325810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.325984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.326069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] federation.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.326232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.326313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.326464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.326546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.326714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.326795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.326951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.327030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.327199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.327282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.327434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.327513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] identity.caching = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.327681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.327759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.327915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.327996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.328155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.328234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.328388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] identity.caching = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.328467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.328633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] identity.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.328721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.328869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.328949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.329104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.329183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.329343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.329425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.329589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] identity.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.329669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.329822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.329970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.330062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.330137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.330299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.330381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.330547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.330634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.330778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.330866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.331011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.331091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.331244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.331325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.331487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.331575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.331738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.331819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.332020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.332098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.332261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.332412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.332501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.332585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.332719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.332801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.332953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.333037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.333190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.333275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.333429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.333510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.333665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.333746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.333928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.334013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.334161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.334253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.334399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.334478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.334639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.334719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.334873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.334963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.335107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.335186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.335355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.335434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.335581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.335668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.335820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.335901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.336051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.password = **** {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.336131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.336288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.336376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.336516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.336595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.336768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.336877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.336982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.password = **** {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.337071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.337216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.337295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.337448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.337526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.337689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.337778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.337942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.338025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.338174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.338254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.338420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.338511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.338666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.338745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.338894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.338972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.339123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.339212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.339356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.339436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.339588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.339670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.339826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.339913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.340061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.340143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.340296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.340375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.340527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.340615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.340766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.340846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.341010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.341088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.341245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.341333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.341476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.341556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.341711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.341790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.341967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.342061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.342207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.342288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.342436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.342516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.342669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.342756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.342905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.342985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.343141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.343220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.343370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.343462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.343618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.343698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.343844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.343923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.344092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.344250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.344334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.344408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.344554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.344635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.344787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.344875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.345020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.345099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.345257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.345335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.345498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.345591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.345733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.345811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.345990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.346071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] policy.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.346231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.346313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.346475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.346554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.346710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.346795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.346946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] receipt.caching = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.347029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] policy.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.347178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.347257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.347422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.347501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.348182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.348268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.348421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] receipt.caching = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.348501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.348657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.348767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.348939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.349021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] resource.caching = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.349189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.349267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.349422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.349500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] resource.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.349660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.349742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.349924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] resource.caching = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.350010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.350176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.350255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.350416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] revoke.caching = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.350499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] resource.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.350652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.350730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.350891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.350970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.351132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] role.cache_time = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.351214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.351367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] revoke.caching = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.351445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] role.caching = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.351612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] role.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.351691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.351846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] role.list_limit = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.351934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.352084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] role.cache_time = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.352166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.352319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] role.caching = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.352406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.352566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] role.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.352648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.352797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] role.list_limit = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.352876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.353044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.353122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.353292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.353381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.353539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.353618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.353782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.353914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.354050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.354133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.354297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.354376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.354544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.354629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.354797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.354881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.355028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.355107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.355277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.355425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.355515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.355663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.355744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.355916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.355996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.356168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.356377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.356457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.356614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.356694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.356867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.356956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.357118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.357197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.357364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.357461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.357601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.357689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.357866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.357980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.358100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.358198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.358341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.358435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.358590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.358670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.358826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.358907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.359060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.359149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.359293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.359374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.359526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.359605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.359759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] token.cache_time = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.359848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.359995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] token.caching = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.360077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.360234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.360313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.360468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.360557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] token.provider = fernet {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.360701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] token.cache_time = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.360782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.360938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] token.caching = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.361018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.361176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.361265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.361414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.361494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] token.provider = fernet {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.361648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.361726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.361911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.362008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.362162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] trust.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.362244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.362399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.362477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.362635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.362723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.362871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.362951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.363113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] trust.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.363210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.363908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.364008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.364151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.364232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.364387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.364467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.364636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.364725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.364886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.364988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.connection = **** {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.365118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.365235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.365361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.365484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.365654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.365737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.365943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.connection = **** {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.366105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.366193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.366269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.366428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.366516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.366665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.366745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.366899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.366978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.367133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.367221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.367370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.367451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.367603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.367682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.367840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.367932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.368090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.368169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.368325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.368404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.368571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.368660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.368802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.368882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.369046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.369127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.369281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.369368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.369443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=103104) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 28 20:40:27.369515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.369745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.370012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.370251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:40:27.370421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=103103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 28 20:40:27.400236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.400468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.400699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.400936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.401172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.401402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.409227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.410707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.438372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.438604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.439110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.439344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.492844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.493626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.542240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.542473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.542562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.542797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.607865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.608044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.608186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.608274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.608419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.608503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.608670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.608772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.608964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.609045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.609120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.609306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.634117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.634346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.634524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.634611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.634758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.634842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.634969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.635054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.635181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.635263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.635392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.635483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.635609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.635823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.647332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.647697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.659869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.660129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.660895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.660985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.661112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.661197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.661325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.661411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.661535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.661645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.661729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=103104) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.661891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.662107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.662318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=103103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 28 20:40:27.678772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.678981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 28 20:40:27.679079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.679225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 28 20:40:27.679308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.679494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 28 20:40:27.679576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.679741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 28 20:40:27.679931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.680175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.680259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.680427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.680507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.680681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.680765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.680937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.691461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.691732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.691993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.692112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.692265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.692385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.692647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.692921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.700323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.701002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.703727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.704003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.704280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.704374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.704554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.704638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.704826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.704909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.705079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.705171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.705340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.705424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.705606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.705690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.705888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.705977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.706156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.706238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.706417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.706510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.706690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.707455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.733724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.734787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.735350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.735626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.736388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.736661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.740882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.741156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.741436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.741947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.742220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.742494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.748893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.749170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.749829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.750127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.753620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.753944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.754226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.754505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.754590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.754772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.754857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.755036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.755129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.755297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.755379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.755556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.755637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.755898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.756162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.756425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.774934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.775205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.775805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.776078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.779749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.780625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.783402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.783693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.784268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.784558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.789412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.790215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.793039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.793772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.796692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.796994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.797273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.797401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.797555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.797700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.797823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.798008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.798102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.798297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.798571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.798844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.817129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.817407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.817668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.817947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.818094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.818373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.818636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.818899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.825825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.826752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.827524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.827795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.828441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.828714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.830863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.831144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.831534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.831782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.831868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.832591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.832677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.832996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.833281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.833559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.845765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.846065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.846342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.846626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.846722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.846902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.847011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.847171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.847291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.847440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.847616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.848413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.848667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.848936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.865953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.866762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.869130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.870017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.872498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.873318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.874173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.874975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.876892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.877172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.877455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.877697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.878010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.878299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.884717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.885563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.886342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.887193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.889532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.889818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.890126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.890399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.890482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.890699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.890978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.891258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.900356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.900646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.901154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.901291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.901571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.901862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.909698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=103103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.910009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.911186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.911464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.912257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.912528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.912797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.913078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.913353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.913726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.914022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.914285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.914555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.914635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=103104) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 28 20:40:27.914822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=103103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.914916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.915174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.915446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.915714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.916102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.916470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.916931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.917417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.918102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.918545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.919010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.919521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.919994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=103104) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 28 20:40:27.952708 ncdv-master-1 devstack@keystone.service[103103]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5622faaad9c0 pid: 103103 (default app) May 28 20:40:27.956089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39a26a-f102-45f0-8839-3d7343ef5a02 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:27.956390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39a26a-f102-45f0-8839-3d7343ef5a02 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:27.956671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39a26a-f102-45f0-8839-3d7343ef5a02 None None] PATH_INFO: `/v3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:27.961417 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Sat May 28 20:40:27 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:27.961512 ncdv-master-1 devstack@keystone.service[103104]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5622faaad9c0 pid: 103104 (default app) May 28 20:40:36.978761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7815d41-ba30-4e17-bf83-5c70cfba62f1 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:36.979335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7815d41-ba30-4e17-bf83-5c70cfba62f1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:36.979335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7815d41-ba30-4e17-bf83-5c70cfba62f1 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:36.980748 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2/2] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:40:36.988443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6738ae-f145-41bb-aea4-d82c972c5861 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:36.988931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6738ae-f145-41bb-aea4-d82c972c5861 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:36.989033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6738ae-f145-41bb-aea4-d82c972c5861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:37.062076 ncdv-master-1 devstack@keystone.service[103104]: INFO dbcounter [None req-7e6738ae-f145-41bb-aea4-d82c972c5861 None None] Registered counter for database keystone May 28 20:40:37.082912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG oslo_db.sqlalchemy.engines [None req-7e6738ae-f145-41bb-aea4-d82c972c5861 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103104) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 20:40:37.088992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writer thread running {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 20:40:37.368378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG passlib.handlers.bcrypt [None req-7e6738ae-f145-41bb-aea4-d82c972c5861 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=103104) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 28 20:40:37.368791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG passlib.handlers.bcrypt [None req-7e6738ae-f145-41bb-aea4-d82c972c5861 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=103104) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 28 20:40:37.424925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7e6738ae-f145-41bb-aea4-d82c972c5861 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:37.426556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7e6738ae-f145-41bb-aea4-d82c972c5861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:37.546482 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1/3] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 561 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 28 20:40:37.741720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7815d41-ba30-4e17-bf83-5c70cfba62f1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:37.815434 ncdv-master-1 devstack@keystone.service[103103]: INFO dbcounter [None req-d7815d41-ba30-4e17-bf83-5c70cfba62f1 None None] Registered counter for database keystone May 28 20:40:37.835128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG oslo_db.sqlalchemy.engines [None req-d7815d41-ba30-4e17-bf83-5c70cfba62f1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=103103) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 20:40:37.841275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writer thread running {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 20:40:37.961488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7815d41-ba30-4e17-bf83-5c70cfba62f1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:37.965816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ef5a4-0864-4168-b9aa-0e940393a8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:37.966496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ef5a4-0864-4168-b9aa-0e940393a8be admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:37.966781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ef5a4-0864-4168-b9aa-0e940393a8be admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:37.967061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ef5a4-0864-4168-b9aa-0e940393a8be admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:37.976309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ef5a4-0864-4168-b9aa-0e940393a8be admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:38.131419 ncdv-master-1 devstack@keystone.service[103103]: WARNING keystone.common.rbac_enforcer.enforcer [None req-550ef5a4-0864-4168-b9aa-0e940393a8be admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 28 20:40:38.132056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ef5a4-0864-4168-b9aa-0e940393a8be admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:38.142284 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3/4] 10.222.0.33 () {58 vars in 1324 bytes} [Sat May 28 20:40:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 20:40:38.148829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6738ae-f145-41bb-aea4-d82c972c5861 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:38.163667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6738ae-f145-41bb-aea4-d82c972c5861 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:38.168056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e57886-096c-4779-9f3a-d6bd00f9ff74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:38.169154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e57886-096c-4779-9f3a-d6bd00f9ff74 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:38.169439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e57886-096c-4779-9f3a-d6bd00f9ff74 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:38.169718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e57886-096c-4779-9f3a-d6bd00f9ff74 admin admin] PATH_INFO: `/v3/projects/a7d6b819de6044b98f2c78098814b8f2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:38.173409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e57886-096c-4779-9f3a-d6bd00f9ff74 admin admin] RBAC: Authorizing `identity:get_project(project_id=a7d6b819de6044b98f2c78098814b8f2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:38.224092 ncdv-master-1 devstack@keystone.service[103104]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d4e57886-096c-4779-9f3a-d6bd00f9ff74 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 28 20:40:38.224567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e57886-096c-4779-9f3a-d6bd00f9ff74 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:38.229162 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2/5] 10.222.0.33 () {56 vars in 1207 bytes} [Sat May 28 20:40:38 2022] GET /identity/v3/projects/a7d6b819de6044b98f2c78098814b8f2 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:40.454862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139ccdb-6476-4ae3-9e4b-f2c0e239c672 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:40.455362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139ccdb-6476-4ae3-9e4b-f2c0e239c672 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:40.455362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139ccdb-6476-4ae3-9e4b-f2c0e239c672 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:40.460026 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4/6] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:40:40.465414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d993e-c53e-4822-9a88-26b44b20890b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:40.465817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d993e-c53e-4822-9a88-26b44b20890b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:40.466000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d993e-c53e-4822-9a88-26b44b20890b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:40.484932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8c4d993e-c53e-4822-9a88-26b44b20890b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:40.486469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8c4d993e-c53e-4822-9a88-26b44b20890b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:40.494172 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3/7] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 28 20:40:40.517413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139ccdb-6476-4ae3-9e4b-f2c0e239c672 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:40.529791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6139ccdb-6476-4ae3-9e4b-f2c0e239c672 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:40.534046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b821d8-df94-480e-9f4c-f0bb287c58d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:40.534673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b821d8-df94-480e-9f4c-f0bb287c58d3 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:40.534957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b821d8-df94-480e-9f4c-f0bb287c58d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:40.535235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b821d8-df94-480e-9f4c-f0bb287c58d3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:40.543470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b821d8-df94-480e-9f4c-f0bb287c58d3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:40.545250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b821d8-df94-480e-9f4c-f0bb287c58d3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:40.553894 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 5/8] 10.222.0.33 () {58 vars in 1324 bytes} [Sat May 28 20:40:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 20:40:40.560247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d993e-c53e-4822-9a88-26b44b20890b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:40.569174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d993e-c53e-4822-9a88-26b44b20890b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:40.573469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8312a921-b67c-45b7-bb73-d33b953752d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:40.574348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312a921-b67c-45b7-bb73-d33b953752d7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:40.574632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312a921-b67c-45b7-bb73-d33b953752d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:40.574906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8312a921-b67c-45b7-bb73-d33b953752d7 admin admin] PATH_INFO: `/v3/users/3e371762e9974f2484669486a37339c7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:40.576266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312a921-b67c-45b7-bb73-d33b953752d7 admin admin] RBAC: Authorizing `identity:get_user(user_id=3e371762e9974f2484669486a37339c7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:40.578134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8312a921-b67c-45b7-bb73-d33b953752d7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:40.579681 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4/9] 10.222.0.33 () {56 vars in 1198 bytes} [Sat May 28 20:40:40 2022] GET /identity/v3/users/3e371762e9974f2484669486a37339c7 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:42.890967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80339b7d-4da8-4cf2-9648-4a205116c7e2 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:42.891774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80339b7d-4da8-4cf2-9648-4a205116c7e2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:42.891774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80339b7d-4da8-4cf2-9648-4a205116c7e2 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:42.893153 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 6/10] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:40:42.898977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d49d98-95af-444b-8f34-4e7bac5cc0f2 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:42.899379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d49d98-95af-444b-8f34-4e7bac5cc0f2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:42.899528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d49d98-95af-444b-8f34-4e7bac5cc0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:42.916473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-31d49d98-95af-444b-8f34-4e7bac5cc0f2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:42.917634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-31d49d98-95af-444b-8f34-4e7bac5cc0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:42.952898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9463182-0d8c-4925-aeb6-96c3cf050bbe None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:42.953189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9463182-0d8c-4925-aeb6-96c3cf050bbe None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:42.953478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9463182-0d8c-4925-aeb6-96c3cf050bbe None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:42.954862 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 7/11] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:40:42.960354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dfe0c-4bca-47ec-8a02-965ab8b852bf None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:42.960646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dfe0c-4bca-47ec-8a02-965ab8b852bf None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:42.960934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dfe0c-4bca-47ec-8a02-965ab8b852bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:42.998353 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 5/12] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:40:43.000334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG passlib.handlers.bcrypt [None req-d73dfe0c-4bca-47ec-8a02-965ab8b852bf None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=103103) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 28 20:40:43.000783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG passlib.handlers.bcrypt [None req-d73dfe0c-4bca-47ec-8a02-965ab8b852bf None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=103103) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 28 20:40:43.025228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d49d98-95af-444b-8f34-4e7bac5cc0f2 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:43.048029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d73dfe0c-4bca-47ec-8a02-965ab8b852bf None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:43.048891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:43.049726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d73dfe0c-4bca-47ec-8a02-965ab8b852bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:43.127799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d49d98-95af-444b-8f34-4e7bac5cc0f2 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:43.151400 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 8/13] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:40:43.164609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89532ccb-e982-459e-b32d-3b2bf9e7a2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:43.165406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89532ccb-e982-459e-b32d-3b2bf9e7a2a7 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:43.165687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89532ccb-e982-459e-b32d-3b2bf9e7a2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:43.165994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89532ccb-e982-459e-b32d-3b2bf9e7a2a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:43.174123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dfe0c-4bca-47ec-8a02-965ab8b852bf None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:43.177011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89532ccb-e982-459e-b32d-3b2bf9e7a2a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:40:43.188142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-89532ccb-e982-459e-b32d-3b2bf9e7a2a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:40:43.188142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:40:43.188142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:40:43.188142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:40:43.188142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:40:43.188142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:40:43.188142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:40:43.188142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:40:43.188142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:40:43.188142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:40:43.188142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:40:43.188142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:40:43.188142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:40:43.188142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:40:43.188142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:40:43.188142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:40:43.189264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:40:43.189264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:40:43.189264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:40:43.189264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:40:43.189264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:40:43.189264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:40:43.189264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:40:43.189264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:40:43.189264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:40:43.189264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:40:43.189264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:40:43.189264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:40:43.189264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:40:43.189264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:40:43.189264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:40:43.189264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:40:43.190142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:40:43.190142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:40:43.190142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:40:43.190142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:40:43.190142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:40:43.190142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:40:43.190142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:40:43.190142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:40:43.190142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:40:43.190142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:40:43.190142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:40:43.190142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:40:43.190142 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 6/14] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:40:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:40:43.195493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89532ccb-e982-459e-b32d-3b2bf9e7a2a7 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:43.275158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:43.276818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dfe0c-4bca-47ec-8a02-965ab8b852bf None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:43.279182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:43.292995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89532ccb-e982-459e-b32d-3b2bf9e7a2a7 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:43.314652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57133685-a594-4a84-b661-899e1b329eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:43.315539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57133685-a594-4a84-b661-899e1b329eee None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:43.315826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57133685-a594-4a84-b661-899e1b329eee None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:43.316103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57133685-a594-4a84-b661-899e1b329eee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:43.327348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea34478-9886-4e56-b148-9ddb94a872ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:43.328008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea34478-9886-4e56-b148-9ddb94a872ae None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:43.328296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea34478-9886-4e56-b148-9ddb94a872ae None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:43.328574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea34478-9886-4e56-b148-9ddb94a872ae None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:43.329324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea34478-9886-4e56-b148-9ddb94a872ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:40:43.329630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea34478-9886-4e56-b148-9ddb94a872ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:43.331067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea34478-9886-4e56-b148-9ddb94a872ae None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:43.342588 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 7/15] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:40:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:40:43.347998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea34478-9886-4e56-b148-9ddb94a872ae None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:43.384469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57133685-a594-4a84-b661-899e1b329eee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:43.386064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57133685-a594-4a84-b661-899e1b329eee None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:43.444814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea34478-9886-4e56-b148-9ddb94a872ae None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:43.479111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a1b68-62a9-4fa8-a0f6-6971a3e46d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:43.479972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a1b68-62a9-4fa8-a0f6-6971a3e46d5c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:43.480259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a1b68-62a9-4fa8-a0f6-6971a3e46d5c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:43.480548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576a1b68-62a9-4fa8-a0f6-6971a3e46d5c None admin] PATH_INFO: `/v3/users/3e371762e9974f2484669486a37339c7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:43.481756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a1b68-62a9-4fa8-a0f6-6971a3e46d5c None admin] RBAC: Authorizing `identity:get_user(user_id=3e371762e9974f2484669486a37339c7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:43.483243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576a1b68-62a9-4fa8-a0f6-6971a3e46d5c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:43.484494 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 9/16] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:40:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 20:40:43.485995 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 8/17] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:40:43 2022] GET /identity/v3/users/3e371762e9974f2484669486a37339c7 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:40:43.490067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57133685-a594-4a84-b661-899e1b329eee None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:43.491029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a1b68-62a9-4fa8-a0f6-6971a3e46d5c None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:43.493355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:43.505006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:43.587201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57133685-a594-4a84-b661-899e1b329eee None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:43.587485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576a1b68-62a9-4fa8-a0f6-6971a3e46d5c None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:43.620909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112db25-34e9-4aae-89fc-29c4621823e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:43.621764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112db25-34e9-4aae-89fc-29c4621823e3 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:43.622081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112db25-34e9-4aae-89fc-29c4621823e3 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:43.626946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf9fd3-734f-401e-a2ba-5f30e44989f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:43.627066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112db25-34e9-4aae-89fc-29c4621823e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:43.627150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf9fd3-734f-401e-a2ba-5f30e44989f8 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:43.627150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf9fd3-734f-401e-a2ba-5f30e44989f8 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:43.627150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf9fd3-734f-401e-a2ba-5f30e44989f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:43.627150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf9fd3-734f-401e-a2ba-5f30e44989f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:43.627150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf9fd3-734f-401e-a2ba-5f30e44989f8 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:43.627973 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 9/18] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:40:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:40:43.633883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf9fd3-734f-401e-a2ba-5f30e44989f8 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:43.642744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112db25-34e9-4aae-89fc-29c4621823e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:43.644229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112db25-34e9-4aae-89fc-29c4621823e3 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:43.666022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-6112db25-34e9-4aae-89fc-29c4621823e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:40:43.666022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:40:43.666022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:40:43.666022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:40:43.666022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:40:43.666022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:40:43.666022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:40:43.666022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:40:43.666022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:40:43.666022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:40:43.666022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:40:43.666022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:40:43.666022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:40:43.666022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:40:43.666022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:40:43.667070 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:40:43.667070 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:40:43.667070 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:40:43.667070 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:40:43.667070 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:40:43.667070 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:40:43.667070 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:40:43.667070 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:40:43.667070 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:40:43.667070 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:40:43.667070 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:40:43.667070 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:40:43.667070 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:40:43.667070 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:40:43.667070 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:40:43.667070 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:40:43.667070 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:40:43.668031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:40:43.668031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:40:43.668031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:40:43.668031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:40:43.668031 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 10/19] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:40:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:40:43.674436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112db25-34e9-4aae-89fc-29c4621823e3 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:43.716730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:43.734250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:43.736298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf9fd3-734f-401e-a2ba-5f30e44989f8 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:43.771777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f228d-f2e7-4972-ab59-55782676e8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:43.772436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f228d-f2e7-4972-ab59-55782676e8b6 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:43.772716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f228d-f2e7-4972-ab59-55782676e8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:43.772991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f228d-f2e7-4972-ab59-55782676e8b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:43.773819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19f228d-f2e7-4972-ab59-55782676e8b6 None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.domain.id=default, user.id=3e371762e9974f2484669486a37339c7) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:40:43.775822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19f228d-f2e7-4972-ab59-55782676e8b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:43.776652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19f228d-f2e7-4972-ab59-55782676e8b6 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:43.779883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112db25-34e9-4aae-89fc-29c4621823e3 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:43.783304 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 10/20] 10.222.0.33 () {56 vars in 1322 bytes} [Sat May 28 20:40:43 2022] GET /identity/v3/role_assignments?user.id=3e371762e9974f2484669486a37339c7&scope.domain.id=default&role.id=7805b93b32f1472f8365936d4d57d49a => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:40:43.816376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4105c47-b651-48af-ad41-88f5e5a3dd97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:43.817060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4105c47-b651-48af-ad41-88f5e5a3dd97 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:43.817350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4105c47-b651-48af-ad41-88f5e5a3dd97 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:43.817629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4105c47-b651-48af-ad41-88f5e5a3dd97 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:43.818926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4105c47-b651-48af-ad41-88f5e5a3dd97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:40:43.819239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4105c47-b651-48af-ad41-88f5e5a3dd97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:43.820682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4105c47-b651-48af-ad41-88f5e5a3dd97 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:43.841107 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 11/21] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:40:46.287174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90900a1a-6aeb-4bae-a2b6-48b7f10b67e5 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:46.287670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90900a1a-6aeb-4bae-a2b6-48b7f10b67e5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:46.287768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90900a1a-6aeb-4bae-a2b6-48b7f10b67e5 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:46.289258 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 11/22] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:40:46.294434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e241384-11f1-4b64-85bc-bb2a2a884a01 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:46.294835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e241384-11f1-4b64-85bc-bb2a2a884a01 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:46.295006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e241384-11f1-4b64-85bc-bb2a2a884a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:46.315068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3e241384-11f1-4b64-85bc-bb2a2a884a01 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:46.316192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3e241384-11f1-4b64-85bc-bb2a2a884a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:46.412458 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 12/23] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:40:46.451294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90900a1a-6aeb-4bae-a2b6-48b7f10b67e5 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:46.579571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90900a1a-6aeb-4bae-a2b6-48b7f10b67e5 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:46.584100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca22da-385e-427d-addb-452a3baf50b5 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:46.588511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca22da-385e-427d-addb-452a3baf50b5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:46.588511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca22da-385e-427d-addb-452a3baf50b5 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:46.594232 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 13/24] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:46 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:40:46.600256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce07960-ea99-498d-b423-1be4bb79ce79 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:46.600546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce07960-ea99-498d-b423-1be4bb79ce79 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:46.600828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce07960-ea99-498d-b423-1be4bb79ce79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:46.626821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b7396-6b5e-46f9-ae77-9fb9c29f3058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:46.627626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b7396-6b5e-46f9-ae77-9fb9c29f3058 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:46.627907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b7396-6b5e-46f9-ae77-9fb9c29f3058 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:46.628237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b7396-6b5e-46f9-ae77-9fb9c29f3058 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:46.639255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6ce07960-ea99-498d-b423-1be4bb79ce79 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:46.640674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6ce07960-ea99-498d-b423-1be4bb79ce79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:46.669023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7b7396-6b5e-46f9-ae77-9fb9c29f3058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:40:46.676719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-4d7b7396-6b5e-46f9-ae77-9fb9c29f3058 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:40:46.676719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:40:46.676719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:40:46.676719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:40:46.676719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:40:46.676719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:40:46.676719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:40:46.676719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:40:46.676719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:40:46.676719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:40:46.676719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:40:46.676719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:40:46.676719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:40:46.676719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:40:46.676719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:40:46.676719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:40:46.677612 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:40:46.677612 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:40:46.677612 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:40:46.677612 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:40:46.677612 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:40:46.677612 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:40:46.677612 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:40:46.677612 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:40:46.677612 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:40:46.677612 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:40:46.677612 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:40:46.677612 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:40:46.677612 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:40:46.677612 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:40:46.677612 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:40:46.677612 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:40:46.678441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:40:46.678441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:40:46.678441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:40:46.678441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:40:46.678441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:40:46.678441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:40:46.678441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:40:46.678441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:40:46.678441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:40:46.678441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:40:46.678441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:40:46.678441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:40:46.678441 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 12/25] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:40:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:40:46.683995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b7396-6b5e-46f9-ae77-9fb9c29f3058 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:46.694941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:46.756333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:46.847419 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 14/26] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:40:46.885170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce07960-ea99-498d-b423-1be4bb79ce79 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:46.903736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b7396-6b5e-46f9-ae77-9fb9c29f3058 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:46.978357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc3edb-a992-4490-87b5-aa19e7ca184f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:46.979401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc3edb-a992-4490-87b5-aa19e7ca184f None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:46.979882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc3edb-a992-4490-87b5-aa19e7ca184f None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:46.980352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc3edb-a992-4490-87b5-aa19e7ca184f None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:46.981320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc3edb-a992-4490-87b5-aa19e7ca184f None admin] RBAC: Adding query filter params (name=admin) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:40:46.981857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc3edb-a992-4490-87b5-aa19e7ca184f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:46.983535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc3edb-a992-4490-87b5-aa19e7ca184f None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:46.999277 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 13/27] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:40:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:47.012336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc3edb-a992-4490-87b5-aa19e7ca184f None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:47.091997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce07960-ea99-498d-b423-1be4bb79ce79 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:47.100528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:47.137690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc3edb-a992-4490-87b5-aa19e7ca184f None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:47.179229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483bcba-db02-40cb-aa8a-f8418c78fa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:47.180132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483bcba-db02-40cb-aa8a-f8418c78fa55 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:47.180416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483bcba-db02-40cb-aa8a-f8418c78fa55 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:47.180692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483bcba-db02-40cb-aa8a-f8418c78fa55 None admin] PATH_INFO: `/v3/users/3e371762e9974f2484669486a37339c7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:47.182350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483bcba-db02-40cb-aa8a-f8418c78fa55 None admin] RBAC: Authorizing `identity:get_user(user_id=3e371762e9974f2484669486a37339c7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:47.183869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483bcba-db02-40cb-aa8a-f8418c78fa55 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:47.187050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecdd672-179a-4d21-b3a8-c3f8c553d1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:47.187866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdd672-179a-4d21-b3a8-c3f8c553d1a9 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:47.188146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdd672-179a-4d21-b3a8-c3f8c553d1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:47.188422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecdd672-179a-4d21-b3a8-c3f8c553d1a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:47.191714 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 14/28] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:40:47 2022] GET /identity/v3/users/3e371762e9974f2484669486a37339c7 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:47.193756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6606f3a-0e44-45ec-80f4-0b4b2761bf9d None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:47.194069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6606f3a-0e44-45ec-80f4-0b4b2761bf9d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:47.194386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6606f3a-0e44-45ec-80f4-0b4b2761bf9d None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:47.195937 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 15/29] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:40:47.196462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6606f3a-0e44-45ec-80f4-0b4b2761bf9d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:47.213287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecdd672-179a-4d21-b3a8-c3f8c553d1a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:47.214902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecdd672-179a-4d21-b3a8-c3f8c553d1a9 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:47.247990 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-3ecdd672-179a-4d21-b3a8-c3f8c553d1a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:40:47.247990 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:40:47.247990 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:40:47.247990 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:40:47.247990 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:40:47.247990 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:40:47.247990 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:40:47.247990 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:40:47.247990 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:40:47.247990 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:40:47.247990 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:40:47.247990 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:40:47.247990 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:40:47.247990 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:40:47.247990 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:40:47.248835 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:40:47.248835 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:40:47.248835 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:40:47.248835 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:40:47.248835 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:40:47.248835 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:40:47.248835 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:40:47.248835 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:40:47.248835 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:40:47.248835 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:40:47.248835 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:40:47.248835 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:40:47.248835 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:40:47.248835 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:40:47.248835 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:40:47.248835 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:40:47.248835 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:40:47.249631 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:40:47.249631 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:40:47.249631 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:40:47.249631 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:40:47.249631 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 15/30] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:40:47.251905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac7b3a-523b-4323-966f-5f07ce21f885 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:47.252154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac7b3a-523b-4323-966f-5f07ce21f885 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:47.254625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac7b3a-523b-4323-966f-5f07ce21f885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:47.282182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c9ac7b3a-523b-4323-966f-5f07ce21f885 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:47.283705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c9ac7b3a-523b-4323-966f-5f07ce21f885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:47.309050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:47.358505 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 16/31] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:40:47.359559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac7b3a-523b-4323-966f-5f07ce21f885 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:47.411357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6606f3a-0e44-45ec-80f4-0b4b2761bf9d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:47.469217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c747cf-281e-4c82-89bc-41a45963ac35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:47.470234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c747cf-281e-4c82-89bc-41a45963ac35 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:47.470524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c747cf-281e-4c82-89bc-41a45963ac35 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:47.471300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c747cf-281e-4c82-89bc-41a45963ac35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:47.472791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c747cf-281e-4c82-89bc-41a45963ac35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:47.475847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c747cf-281e-4c82-89bc-41a45963ac35 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:47.477508 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 16/32] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:40:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:47.482666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c747cf-281e-4c82-89bc-41a45963ac35 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:47.489042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:47.517504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac7b3a-523b-4323-966f-5f07ce21f885 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:47.565680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb2b41-9fe3-4ffd-a029-4798095d25f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:47.566389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb2b41-9fe3-4ffd-a029-4798095d25f3 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:47.566674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb2b41-9fe3-4ffd-a029-4798095d25f3 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:47.566950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb2b41-9fe3-4ffd-a029-4798095d25f3 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:47.568275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb2b41-9fe3-4ffd-a029-4798095d25f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:40:47.568598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb2b41-9fe3-4ffd-a029-4798095d25f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:47.570046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb2b41-9fe3-4ffd-a029-4798095d25f3 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:47.586272 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 17/33] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:40:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:47.587620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759c22f-0180-4874-8bb5-e60e0ad033fe None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:47.587923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759c22f-0180-4874-8bb5-e60e0ad033fe None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:47.588208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759c22f-0180-4874-8bb5-e60e0ad033fe None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:47.590242 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 18/34] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:40:47.591113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759c22f-0180-4874-8bb5-e60e0ad033fe None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:47.603809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c747cf-281e-4c82-89bc-41a45963ac35 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:47.618470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:47.650532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb6b2e-d940-4b3a-a70b-b266d7604f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:47.651808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb6b2e-d940-4b3a-a70b-b266d7604f49 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:47.652097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb6b2e-d940-4b3a-a70b-b266d7604f49 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:47.652380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb6b2e-d940-4b3a-a70b-b266d7604f49 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:47.654361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb6b2e-d940-4b3a-a70b-b266d7604f49 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:47.655795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb6b2e-d940-4b3a-a70b-b266d7604f49 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:47.711895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759c22f-0180-4874-8bb5-e60e0ad033fe None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:47.749936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327c602-0c57-4e36-973f-8368a24e20ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:47.750877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327c602-0c57-4e36-973f-8368a24e20ea None admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:47.751167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327c602-0c57-4e36-973f-8368a24e20ea None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:47.751463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327c602-0c57-4e36-973f-8368a24e20ea None admin] PATH_INFO: `/v3/domains/default/users/3e371762e9974f2484669486a37339c7/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:47.754243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327c602-0c57-4e36-973f-8368a24e20ea None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=3e371762e9974f2484669486a37339c7, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:47.756520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327c602-0c57-4e36-973f-8368a24e20ea None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:47.868560 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 17/35] 10.222.0.33 () {60 vars in 1131 bytes} [Sat May 28 20:40:47 2022] POST /identity/v3/roles => generated 219 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:40:47.870183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb6b2e-d940-4b3a-a70b-b266d7604f49 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:47.943945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:47.965909 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 19/36] 10.222.0.33 () {58 vars in 1361 bytes} [Sat May 28 20:40:47 2022] PUT /identity/v3/domains/default/users/3e371762e9974f2484669486a37339c7/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:40:47.967377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687479b-213c-4dd4-9c69-32bc4f7db8ed None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:47.967811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687479b-213c-4dd4-9c69-32bc4f7db8ed None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:47.968237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8687479b-213c-4dd4-9c69-32bc4f7db8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:47.996264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb6b2e-d940-4b3a-a70b-b266d7604f49 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:48.009438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8687479b-213c-4dd4-9c69-32bc4f7db8ed None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:48.011193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8687479b-213c-4dd4-9c69-32bc4f7db8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:48.074417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bb88f6-a2b9-4271-b40b-dc7a0c96ffbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:48.075116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb88f6-a2b9-4271-b40b-dc7a0c96ffbc None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:48.075404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb88f6-a2b9-4271-b40b-dc7a0c96ffbc None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:48.075687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb88f6-a2b9-4271-b40b-dc7a0c96ffbc None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:48.077878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb88f6-a2b9-4271-b40b-dc7a0c96ffbc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:48.078161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bb88f6-a2b9-4271-b40b-dc7a0c96ffbc None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:48.146444 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 20/37] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:40:48.148068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327d6fe-5148-4f3a-b057-7fee8fa54972 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:48.148363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327d6fe-5148-4f3a-b057-7fee8fa54972 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:48.148697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327d6fe-5148-4f3a-b057-7fee8fa54972 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:48.153307 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 21/38] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:40:48.159267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3eb427-ab57-49ef-bc60-baea558d7db0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:48.159561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3eb427-ab57-49ef-bc60-baea558d7db0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:48.159850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3eb427-ab57-49ef-bc60-baea558d7db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:48.183921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1c3eb427-ab57-49ef-bc60-baea558d7db0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:48.185236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1c3eb427-ab57-49ef-bc60-baea558d7db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:48.195063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:48.211034 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 18/39] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 20:40:47 2022] POST /identity/v3/projects => generated 301 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:40:48.212483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15231de0-c59a-492d-b241-03d24da040db None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:48.212785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15231de0-c59a-492d-b241-03d24da040db None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:48.213073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15231de0-c59a-492d-b241-03d24da040db None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:48.221705 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 19/40] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:48 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:40:48.222516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15231de0-c59a-492d-b241-03d24da040db None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:48.280931 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 22/41] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:40:48.282563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5c697-c9fb-401e-88b8-4da3e66fa20a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:48.283032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5c697-c9fb-401e-88b8-4da3e66fa20a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:48.283537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5c697-c9fb-401e-88b8-4da3e66fa20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:48.306060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0dd5c697-c9fb-401e-88b8-4da3e66fa20a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:48.307309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0dd5c697-c9fb-401e-88b8-4da3e66fa20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:48.371766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15231de0-c59a-492d-b241-03d24da040db None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:48.390498 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 23/42] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:40:48.391920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5c697-c9fb-401e-88b8-4da3e66fa20a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:48.392845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:48.419285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4f8b1-1aa5-4d9e-b97c-eeba5c1a7de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:48.421058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4f8b1-1aa5-4d9e-b97c-eeba5c1a7de9 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:48.421546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4f8b1-1aa5-4d9e-b97c-eeba5c1a7de9 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:48.422140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4f8b1-1aa5-4d9e-b97c-eeba5c1a7de9 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:48.423346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4f8b1-1aa5-4d9e-b97c-eeba5c1a7de9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:48.425895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c4f8b1-1aa5-4d9e-b97c-eeba5c1a7de9 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:48.480630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:48.514281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5c697-c9fb-401e-88b8-4da3e66fa20a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:48.549726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e83d1-c026-488a-94be-5fe5916fe2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:48.550556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e83d1-c026-488a-94be-5fe5916fe2f5 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:48.550847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e83d1-c026-488a-94be-5fe5916fe2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:48.551130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e83d1-c026-488a-94be-5fe5916fe2f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:48.552363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e83d1-c026-488a-94be-5fe5916fe2f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:48.553993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e83d1-c026-488a-94be-5fe5916fe2f5 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:48.555609 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 24/43] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:40:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:48.556348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e83d1-c026-488a-94be-5fe5916fe2f5 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:48.607581 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 20/44] 10.222.0.33 () {60 vars in 1131 bytes} [Sat May 28 20:40:48 2022] POST /identity/v3/roles => generated 225 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:40:48.608251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4f8b1-1aa5-4d9e-b97c-eeba5c1a7de9 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:48.680252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e83d1-c026-488a-94be-5fe5916fe2f5 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:48.726933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4f8b1-1aa5-4d9e-b97c-eeba5c1a7de9 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:48.727123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddc069-8b34-4a5c-ac1e-96dfe94a5cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:48.727661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddc069-8b34-4a5c-ac1e-96dfe94a5cab None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:48.727948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddc069-8b34-4a5c-ac1e-96dfe94a5cab None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:48.728224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddc069-8b34-4a5c-ac1e-96dfe94a5cab None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:48.729914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddc069-8b34-4a5c-ac1e-96dfe94a5cab None admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:48.731419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddc069-8b34-4a5c-ac1e-96dfe94a5cab None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:48.739946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:48.771336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8fc320-c603-40f4-b037-a34361fec7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:48.771916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fc320-c603-40f4-b037-a34361fec7bc None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:48.772117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fc320-c603-40f4-b037-a34361fec7bc None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:48.772389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8fc320-c603-40f4-b037-a34361fec7bc None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:48.773121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8fc320-c603-40f4-b037-a34361fec7bc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:48.774646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8fc320-c603-40f4-b037-a34361fec7bc None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:48.801538 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 25/45] 10.222.0.33 () {60 vars in 1131 bytes} [Sat May 28 20:40:48 2022] POST /identity/v3/roles => generated 223 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:40:48.888445 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 21/46] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 20:40:48 2022] POST /identity/v3/projects => generated 312 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:40:50.463827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff70c-8ca8-4d13-83f3-e362bab80e4d None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:50.464321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff70c-8ca8-4d13-83f3-e362bab80e4d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:50.464321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eff70c-8ca8-4d13-83f3-e362bab80e4d None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:50.465697 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 26/47] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:40:50.471053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1152830-a3c2-4345-8771-7b7f36694361 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:50.471475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1152830-a3c2-4345-8771-7b7f36694361 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:50.471623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1152830-a3c2-4345-8771-7b7f36694361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:50.488891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d1152830-a3c2-4345-8771-7b7f36694361 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:50.490442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d1152830-a3c2-4345-8771-7b7f36694361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:50.510659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:50.557061 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 22/48] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:40:50.582531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eff70c-8ca8-4d13-83f3-e362bab80e4d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:50.682673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eff70c-8ca8-4d13-83f3-e362bab80e4d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:50.716195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54ea9d-dff1-44c9-9ee6-38e2c6df0c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:50.716918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54ea9d-dff1-44c9-9ee6-38e2c6df0c06 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:50.717206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54ea9d-dff1-44c9-9ee6-38e2c6df0c06 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:50.717486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54ea9d-dff1-44c9-9ee6-38e2c6df0c06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:50.728924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54ea9d-dff1-44c9-9ee6-38e2c6df0c06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:40:50.737711 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-5d54ea9d-dff1-44c9-9ee6-38e2c6df0c06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:40:50.737711 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:40:50.737711 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:40:50.737711 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:40:50.737711 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:40:50.737711 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:40:50.737711 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:40:50.737711 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:40:50.737711 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:40:50.737711 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:40:50.737711 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:40:50.737711 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:40:50.737711 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:40:50.737711 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:40:50.737711 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:40:50.737711 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:40:50.738793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:40:50.738793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:40:50.738793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:40:50.738793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:40:50.738793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:40:50.738793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:40:50.738793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:40:50.738793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:40:50.738793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:40:50.738793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:40:50.738793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:40:50.738793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:40:50.738793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:40:50.738793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:40:50.738793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:40:50.738793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:40:50.739533 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:40:50.739533 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:40:50.739533 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:40:50.739533 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:40:50.739533 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:40:50.739533 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:40:50.739533 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:40:50.739533 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:40:50.739533 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:40:50.739533 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:40:50.739533 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:40:50.739533 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:40:50.739533 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 27/49] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:40:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:40:50.744903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1152830-a3c2-4345-8771-7b7f36694361 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:50.844321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1152830-a3c2-4345-8771-7b7f36694361 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:50.882004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee8184-2bfc-464a-bcc3-e9ba053e5fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:50.882683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee8184-2bfc-464a-bcc3-e9ba053e5fc7 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:50.882974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee8184-2bfc-464a-bcc3-e9ba053e5fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:50.883257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee8184-2bfc-464a-bcc3-e9ba053e5fc7 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:50.884019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee8184-2bfc-464a-bcc3-e9ba053e5fc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:40:50.884332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee8184-2bfc-464a-bcc3-e9ba053e5fc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:50.885762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee8184-2bfc-464a-bcc3-e9ba053e5fc7 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:50.894219 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 23/50] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:40:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:50.899794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54ea9d-dff1-44c9-9ee6-38e2c6df0c06 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:50.957181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:51.005186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54ea9d-dff1-44c9-9ee6-38e2c6df0c06 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:51.039477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1534d5-15d6-4737-953a-894e1bfb5956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:51.040358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1534d5-15d6-4737-953a-894e1bfb5956 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:51.040648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1534d5-15d6-4737-953a-894e1bfb5956 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:51.040928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1534d5-15d6-4737-953a-894e1bfb5956 None admin] PATH_INFO: `/v3/users/3e371762e9974f2484669486a37339c7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:51.042167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1534d5-15d6-4737-953a-894e1bfb5956 None admin] RBAC: Authorizing `identity:get_user(user_id=3e371762e9974f2484669486a37339c7)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:51.043639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1534d5-15d6-4737-953a-894e1bfb5956 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:51.045273 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 28/51] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:40:50 2022] GET /identity/v3/users/3e371762e9974f2484669486a37339c7 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:51.050811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee8184-2bfc-464a-bcc3-e9ba053e5fc7 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:51.070627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:51.150005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee8184-2bfc-464a-bcc3-e9ba053e5fc7 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:51.180686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ab0f3-ea98-45f7-9937-dca66f730817 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:51.180990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ab0f3-ea98-45f7-9937-dca66f730817 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:51.181280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ab0f3-ea98-45f7-9937-dca66f730817 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:51.182722 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 29/52] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:40:51.183490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744085c-e8a6-4843-ab88-79adb76267eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:51.184282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744085c-e8a6-4843-ab88-79adb76267eb None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:51.184571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744085c-e8a6-4843-ab88-79adb76267eb None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:51.184850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5744085c-e8a6-4843-ab88-79adb76267eb None admin] PATH_INFO: `/v3/domains/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:51.186086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744085c-e8a6-4843-ab88-79adb76267eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:51.187566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7c585-0d27-410f-aba0-83f4428ea81a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:51.187665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5744085c-e8a6-4843-ab88-79adb76267eb None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:51.187887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7c585-0d27-410f-aba0-83f4428ea81a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:51.188173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7c585-0d27-410f-aba0-83f4428ea81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:51.189276 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 24/53] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:40:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:51.194766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744085c-e8a6-4843-ab88-79adb76267eb None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:51.204799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8fe7c585-0d27-410f-aba0-83f4428ea81a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:51.206053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8fe7c585-0d27-410f-aba0-83f4428ea81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:51.272078 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 30/54] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:40:51.278804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:51.293022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5744085c-e8a6-4843-ab88-79adb76267eb None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:51.297102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7c585-0d27-410f-aba0-83f4428ea81a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:51.327727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35e073-933e-4b7a-823b-63ff2cc34adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:51.328393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35e073-933e-4b7a-823b-63ff2cc34adc None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:51.328684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35e073-933e-4b7a-823b-63ff2cc34adc None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:51.328977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c35e073-933e-4b7a-823b-63ff2cc34adc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:51.329819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35e073-933e-4b7a-823b-63ff2cc34adc None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.domain.id=default, user.id=3e371762e9974f2484669486a37339c7) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:40:51.330170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35e073-933e-4b7a-823b-63ff2cc34adc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:51.331656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c35e073-933e-4b7a-823b-63ff2cc34adc None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:51.338599 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 25/55] 10.222.0.33 () {56 vars in 1322 bytes} [Sat May 28 20:40:51 2022] GET /identity/v3/role_assignments?user.id=3e371762e9974f2484669486a37339c7&scope.domain.id=default&role.id=7805b93b32f1472f8365936d4d57d49a => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:40:51.351983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:51.400704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7c585-0d27-410f-aba0-83f4428ea81a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:51.437640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d310d3f-7984-40c5-9e7f-d79589cb93de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:51.438503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d310d3f-7984-40c5-9e7f-d79589cb93de None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:51.438795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d310d3f-7984-40c5-9e7f-d79589cb93de None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:51.439079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d310d3f-7984-40c5-9e7f-d79589cb93de None admin] PATH_INFO: `/v3/domains/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:51.440319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d310d3f-7984-40c5-9e7f-d79589cb93de None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:51.441810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d310d3f-7984-40c5-9e7f-d79589cb93de None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:51.443533 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 31/56] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:40:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:40:51.449251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35e073-933e-4b7a-823b-63ff2cc34adc None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:51.554462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c35e073-933e-4b7a-823b-63ff2cc34adc None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:51.589717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac6067-fce9-473f-ab56-301648a18fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:51.590460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac6067-fce9-473f-ab56-301648a18fd5 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:51.590749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac6067-fce9-473f-ab56-301648a18fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:51.591029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac6067-fce9-473f-ab56-301648a18fd5 None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:51.591743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac6067-fce9-473f-ab56-301648a18fd5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:51.593233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac6067-fce9-473f-ab56-301648a18fd5 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:51.726308 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 26/57] 10.222.0.33 () {60 vars in 1140 bytes} [Sat May 28 20:40:51 2022] POST /identity/v3/projects => generated 298 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:40:53.974764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8293751-1164-4d5a-9ae2-47f88bbe1657 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:53.975303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8293751-1164-4d5a-9ae2-47f88bbe1657 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:53.975303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8293751-1164-4d5a-9ae2-47f88bbe1657 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:53.976691 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 32/58] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:40:53.981769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102c18f-c0fa-4ae6-91a2-2cb299c94575 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:53.982218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102c18f-c0fa-4ae6-91a2-2cb299c94575 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:53.982377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102c18f-c0fa-4ae6-91a2-2cb299c94575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:53.987443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:53.999149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8102c18f-c0fa-4ae6-91a2-2cb299c94575 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:54.007984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8102c18f-c0fa-4ae6-91a2-2cb299c94575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:54.066000 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 27/59] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:40:54.093232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8293751-1164-4d5a-9ae2-47f88bbe1657 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:54.191706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8293751-1164-4d5a-9ae2-47f88bbe1657 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:54.213714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:54.227593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25146c97-c2f2-4512-9c31-eb7e9d69e108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:54.227593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25146c97-c2f2-4512-9c31-eb7e9d69e108 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:54.227593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25146c97-c2f2-4512-9c31-eb7e9d69e108 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:54.227593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25146c97-c2f2-4512-9c31-eb7e9d69e108 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:54.227909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25146c97-c2f2-4512-9c31-eb7e9d69e108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:54.229058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25146c97-c2f2-4512-9c31-eb7e9d69e108 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:54.230679 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 33/60] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:40:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:54.236216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102c18f-c0fa-4ae6-91a2-2cb299c94575 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:54.331625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102c18f-c0fa-4ae6-91a2-2cb299c94575 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:54.359716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:54.364429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f7b41-beec-4f50-8c0c-e7b8fd3f15ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:54.374761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f7b41-beec-4f50-8c0c-e7b8fd3f15ab None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:54.374761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f7b41-beec-4f50-8c0c-e7b8fd3f15ab None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:54.374761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f7b41-beec-4f50-8c0c-e7b8fd3f15ab None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:54.374761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f7b41-beec-4f50-8c0c-e7b8fd3f15ab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:54.374761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f7b41-beec-4f50-8c0c-e7b8fd3f15ab None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:54.473944 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 28/61] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:40:54 2022] POST /identity/v3/users => generated 275 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:40:57.897740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456e7bf-ee3f-4a3d-84cb-f58807312a67 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:57.898799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456e7bf-ee3f-4a3d-84cb-f58807312a67 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:57.899302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456e7bf-ee3f-4a3d-84cb-f58807312a67 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:57.900902 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 34/62] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:40:57.906673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e9c7f-e1f0-4700-9011-60b2120d0fa4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:57.907092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e9c7f-e1f0-4700-9011-60b2120d0fa4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:57.907308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e9c7f-e1f0-4700-9011-60b2120d0fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:57.930066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-450e9c7f-e1f0-4700-9011-60b2120d0fa4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:57.931279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-450e9c7f-e1f0-4700-9011-60b2120d0fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:58.037481 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 29/63] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:40:58.063358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456e7bf-ee3f-4a3d-84cb-f58807312a67 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:58.217748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456e7bf-ee3f-4a3d-84cb-f58807312a67 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:58.235241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d4305-1949-4fa4-9bcf-b82e75c29c5d None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:58.235895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d4305-1949-4fa4-9bcf-b82e75c29c5d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:58.236377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d4305-1949-4fa4-9bcf-b82e75c29c5d None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:58.242216 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 30/64] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:40:58.248172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c30cea-29b9-4915-ac49-8282bced4351 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:58.248795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c30cea-29b9-4915-ac49-8282bced4351 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:58.249285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c30cea-29b9-4915-ac49-8282bced4351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:58.269978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85becf-e9c8-4f55-8a13-ef2deebfbeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:58.271755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85becf-e9c8-4f55-8a13-ef2deebfbeb7 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:58.272290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85becf-e9c8-4f55-8a13-ef2deebfbeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:58.272769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85becf-e9c8-4f55-8a13-ef2deebfbeb7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:58.284657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d5c30cea-29b9-4915-ac49-8282bced4351 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:58.287445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d5c30cea-29b9-4915-ac49-8282bced4351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:58.295658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85becf-e9c8-4f55-8a13-ef2deebfbeb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:40:58.304105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-bf85becf-e9c8-4f55-8a13-ef2deebfbeb7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:40:58.304105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:40:58.304105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:40:58.304105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:40:58.304105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:40:58.304105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:40:58.304105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:40:58.304105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:40:58.304105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:40:58.304105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:40:58.304105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:40:58.304105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:40:58.304105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:40:58.304105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:40:58.304105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:40:58.304105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:40:58.305016 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:40:58.305016 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:40:58.305016 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:40:58.305016 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:40:58.305016 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:40:58.305016 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:40:58.305016 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:40:58.305016 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:40:58.305016 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:40:58.305016 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:40:58.305016 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:40:58.305016 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:40:58.305016 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:40:58.305016 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:40:58.305016 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:40:58.305016 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:40:58.305772 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:40:58.305772 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:40:58.305772 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:40:58.305772 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:40:58.305772 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:40:58.305772 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:40:58.305772 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:40:58.305772 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:40:58.305772 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:40:58.305772 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:40:58.305772 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:40:58.305772 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:40:58.308595 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 35/65] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:40:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:40:58.310287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20cd57-b38d-4b0b-832e-7f7d0b0e790d None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:58.310806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20cd57-b38d-4b0b-832e-7f7d0b0e790d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:58.311284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20cd57-b38d-4b0b-832e-7f7d0b0e790d None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:58.312861 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 36/66] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:40:58.319896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4dd6c8-3707-4588-88b7-f05671a9adb1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:58.320439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4dd6c8-3707-4588-88b7-f05671a9adb1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:58.320918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4dd6c8-3707-4588-88b7-f05671a9adb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:58.339414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fc4dd6c8-3707-4588-88b7-f05671a9adb1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:58.341017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fc4dd6c8-3707-4588-88b7-f05671a9adb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:58.371567 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 31/67] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:40:58.379002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c30cea-29b9-4915-ac49-8282bced4351 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:58.397154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:58.419723 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 37/68] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:40:58.429241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4dd6c8-3707-4588-88b7-f05671a9adb1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:58.451396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:58.505962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c30cea-29b9-4915-ac49-8282bced4351 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:58.543103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86234b-dc8d-4b2f-abd8-0a5e7e622696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:58.543778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86234b-dc8d-4b2f-abd8-0a5e7e622696 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:58.544068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86234b-dc8d-4b2f-abd8-0a5e7e622696 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:58.544350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db86234b-dc8d-4b2f-abd8-0a5e7e622696 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:58.544916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4dd6c8-3707-4588-88b7-f05671a9adb1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:58.545109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86234b-dc8d-4b2f-abd8-0a5e7e622696 None admin] RBAC: Adding query filter params (name=member) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:40:58.545421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86234b-dc8d-4b2f-abd8-0a5e7e622696 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:58.546876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db86234b-dc8d-4b2f-abd8-0a5e7e622696 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:58.554916 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 32/69] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:40:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:58.555710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86234b-dc8d-4b2f-abd8-0a5e7e622696 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:58.579976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d09570-cd6b-4200-b56e-46c518029dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:58.581191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d09570-cd6b-4200-b56e-46c518029dc0 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:58.581479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d09570-cd6b-4200-b56e-46c518029dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:58.581761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d09570-cd6b-4200-b56e-46c518029dc0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:58.592737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d09570-cd6b-4200-b56e-46c518029dc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:40:58.601091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-d3d09570-cd6b-4200-b56e-46c518029dc0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:40:58.601091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:40:58.601091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:40:58.601091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:40:58.601091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:40:58.601091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:40:58.601091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:40:58.601091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:40:58.601091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:40:58.601091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:40:58.601091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:40:58.601091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:40:58.601091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:40:58.601091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:40:58.601091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:40:58.601091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:40:58.601978 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:40:58.601978 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:40:58.601978 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:40:58.601978 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:40:58.601978 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:40:58.601978 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:40:58.601978 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:40:58.601978 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:40:58.601978 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:40:58.601978 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:40:58.601978 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:40:58.601978 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:40:58.601978 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:40:58.601978 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:40:58.601978 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:40:58.601978 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:40:58.602739 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:40:58.602739 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:40:58.602739 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:40:58.602739 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:40:58.602739 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:40:58.602739 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:40:58.602739 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:40:58.602739 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:40:58.602739 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:40:58.602739 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:40:58.602739 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:40:58.602739 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:40:58.602739 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 38/70] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:40:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:40:58.603780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8010825-6e5e-49f7-b7db-046d3c4482b9 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:58.603944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8010825-6e5e-49f7-b7db-046d3c4482b9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:58.604232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8010825-6e5e-49f7-b7db-046d3c4482b9 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:58.605651 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 39/71] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:40:58.606681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ba873-e8e5-459d-af3a-308e0df63ceb None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:58.607061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ba873-e8e5-459d-af3a-308e0df63ceb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:58.607419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ba873-e8e5-459d-af3a-308e0df63ceb None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:58.608652 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 40/72] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:40:58.609351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ba873-e8e5-459d-af3a-308e0df63ceb None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:58.653318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:58.666824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db86234b-dc8d-4b2f-abd8-0a5e7e622696 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:58.677460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:58.701236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a5aaf-03d9-4f60-8668-7829deca42db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:58.702058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a5aaf-03d9-4f60-8668-7829deca42db None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:58.702345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a5aaf-03d9-4f60-8668-7829deca42db None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:58.702626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a5aaf-03d9-4f60-8668-7829deca42db None admin] PATH_INFO: `/v3/roles/member` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:58.713004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ba873-e8e5-459d-af3a-308e0df63ceb None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:58.713200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a5aaf-03d9-4f60-8668-7829deca42db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:40:58.725818 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-473a5aaf-03d9-4f60-8668-7829deca42db None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:40:58.725818 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:40:58.725818 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:40:58.725818 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:40:58.725818 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:40:58.725818 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:40:58.725818 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:40:58.725818 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:40:58.725818 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:40:58.725818 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:40:58.725818 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:40:58.725818 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:40:58.725818 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:40:58.725818 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:40:58.725818 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:40:58.725818 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:40:58.726696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:40:58.726696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:40:58.726696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:40:58.726696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:40:58.726696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:40:58.726696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:40:58.726696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:40:58.726696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:40:58.726696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:40:58.726696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:40:58.726696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:40:58.726696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:40:58.726696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:40:58.726696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:40:58.726696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:40:58.726696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:40:58.727512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:40:58.727512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:40:58.727512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:40:58.727512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:40:58.727512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:40:58.727512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:40:58.727512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:40:58.727512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:40:58.727512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:40:58.727512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:40:58.727512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:40:58.727512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:40:58.727512 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 33/73] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:40:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:40:58.728140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a5aaf-03d9-4f60-8668-7829deca42db None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:58.750812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83605d0e-7dec-4a9e-a946-f983fd6ec4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:58.751684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83605d0e-7dec-4a9e-a946-f983fd6ec4bb None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:58.751979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83605d0e-7dec-4a9e-a946-f983fd6ec4bb None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:58.752254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83605d0e-7dec-4a9e-a946-f983fd6ec4bb None admin] PATH_INFO: `/v3/users/6edbb2f56e344bc2b039e08db2a25dbc` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:58.767824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83605d0e-7dec-4a9e-a946-f983fd6ec4bb None admin] RBAC: Authorizing `identity:get_user(user_id=6edbb2f56e344bc2b039e08db2a25dbc)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:58.769240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83605d0e-7dec-4a9e-a946-f983fd6ec4bb None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:58.771668 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 41/74] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:40:58 2022] GET /identity/v3/users/6edbb2f56e344bc2b039e08db2a25dbc => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:58.772640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1123b-7a46-4f15-9772-34f8db51ef8c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:58.772962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1123b-7a46-4f15-9772-34f8db51ef8c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:58.773307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1123b-7a46-4f15-9772-34f8db51ef8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:58.789073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5bc1123b-7a46-4f15-9772-34f8db51ef8c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:58.790324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5bc1123b-7a46-4f15-9772-34f8db51ef8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:58.830252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a5aaf-03d9-4f60-8668-7829deca42db None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:58.855934 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 42/75] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:40:58.857321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82310e-c59f-4525-b2e2-649959521298 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:58.857738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82310e-c59f-4525-b2e2-649959521298 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:58.858168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82310e-c59f-4525-b2e2-649959521298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:58.863624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5b1f9-0d00-432c-b6fd-89018f11d10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:58.864315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5b1f9-0d00-432c-b6fd-89018f11d10e None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:58.864611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5b1f9-0d00-432c-b6fd-89018f11d10e None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:58.864898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5b1f9-0d00-432c-b6fd-89018f11d10e None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:58.865662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5b1f9-0d00-432c-b6fd-89018f11d10e None admin] RBAC: Adding query filter params (name=admin) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:40:58.866013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5b1f9-0d00-432c-b6fd-89018f11d10e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:58.867440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5b1f9-0d00-432c-b6fd-89018f11d10e None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:58.876620 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 34/76] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:40:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:58.877505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5b1f9-0d00-432c-b6fd-89018f11d10e None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:58.881204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9d82310e-c59f-4525-b2e2-649959521298 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:40:58.882587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9d82310e-c59f-4525-b2e2-649959521298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:40:58.909335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:58.910448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:58.954257 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 43/77] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:40:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:40:58.955091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82310e-c59f-4525-b2e2-649959521298 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:58.984014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5b1f9-0d00-432c-b6fd-89018f11d10e None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:59.017038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24794733-4e5c-4304-921e-5fd89bc1c650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:59.017712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24794733-4e5c-4304-921e-5fd89bc1c650 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:59.018085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24794733-4e5c-4304-921e-5fd89bc1c650 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:59.018315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24794733-4e5c-4304-921e-5fd89bc1c650 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:59.019072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24794733-4e5c-4304-921e-5fd89bc1c650 None admin] RBAC: Adding query filter params (name=member) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:40:59.019380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24794733-4e5c-4304-921e-5fd89bc1c650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:59.021487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24794733-4e5c-4304-921e-5fd89bc1c650 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:59.029374 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 35/78] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:40:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:59.035469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24794733-4e5c-4304-921e-5fd89bc1c650 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:59.055902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82310e-c59f-4525-b2e2-649959521298 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:59.093320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52687d90-6f9b-419c-8390-6dc42f4c9cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:59.094006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52687d90-6f9b-419c-8390-6dc42f4c9cbb None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:59.094290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52687d90-6f9b-419c-8390-6dc42f4c9cbb None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:59.094602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52687d90-6f9b-419c-8390-6dc42f4c9cbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:59.120653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:59.132553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:59.139156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24794733-4e5c-4304-921e-5fd89bc1c650 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:59.162910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52687d90-6f9b-419c-8390-6dc42f4c9cbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:59.164411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52687d90-6f9b-419c-8390-6dc42f4c9cbb None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:59.172362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c0cc4-01f4-4e65-b6e0-9c25d03fb8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:59.173241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c0cc4-01f4-4e65-b6e0-9c25d03fb8f3 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:59.173551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c0cc4-01f4-4e65-b6e0-9c25d03fb8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:59.173830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c0cc4-01f4-4e65-b6e0-9c25d03fb8f3 None admin] PATH_INFO: `/v3/users/3e371762e9974f2484669486a37339c7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:59.175052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c0cc4-01f4-4e65-b6e0-9c25d03fb8f3 None admin] RBAC: Authorizing `identity:get_user(user_id=3e371762e9974f2484669486a37339c7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:59.176548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c0cc4-01f4-4e65-b6e0-9c25d03fb8f3 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:59.178221 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 36/79] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:40:59 2022] GET /identity/v3/users/3e371762e9974f2484669486a37339c7 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:59.178976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c0cc4-01f4-4e65-b6e0-9c25d03fb8f3 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:59.266415 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 44/80] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:40:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:40:59.267255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52687d90-6f9b-419c-8390-6dc42f4c9cbb None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:59.276793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c0cc4-01f4-4e65-b6e0-9c25d03fb8f3 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:59.310295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbc34c-2d9c-4d5e-a48a-c93f439a8b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:59.311087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbc34c-2d9c-4d5e-a48a-c93f439a8b06 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:59.311381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbc34c-2d9c-4d5e-a48a-c93f439a8b06 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:59.311665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dbc34c-2d9c-4d5e-a48a-c93f439a8b06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:59.312867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dbc34c-2d9c-4d5e-a48a-c93f439a8b06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:59.314386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dbc34c-2d9c-4d5e-a48a-c93f439a8b06 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:59.316000 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 37/81] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:40:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:59.316751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbc34c-2d9c-4d5e-a48a-c93f439a8b06 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:59.323627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:59.346885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:59.364797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52687d90-6f9b-419c-8390-6dc42f4c9cbb None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:59.399158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118ca25-a4af-4207-96ba-e5b1596d42e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:59.400028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118ca25-a4af-4207-96ba-e5b1596d42e5 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:59.400256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118ca25-a4af-4207-96ba-e5b1596d42e5 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:59.400538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118ca25-a4af-4207-96ba-e5b1596d42e5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:59.410385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7118ca25-a4af-4207-96ba-e5b1596d42e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:40:59.415586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dbc34c-2d9c-4d5e-a48a-c93f439a8b06 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:59.419290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-7118ca25-a4af-4207-96ba-e5b1596d42e5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:40:59.419290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:40:59.419290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:40:59.419290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:40:59.419290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:40:59.419290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:40:59.419290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:40:59.419290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:40:59.419290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:40:59.419290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:40:59.419290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:40:59.419290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:40:59.419290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:40:59.419290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:40:59.419290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:40:59.419290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:40:59.420199 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:40:59.420199 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:40:59.420199 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:40:59.420199 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:40:59.420199 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:40:59.420199 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:40:59.420199 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:40:59.420199 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:40:59.420199 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:40:59.420199 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:40:59.420199 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:40:59.420199 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:40:59.420199 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:40:59.420199 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:40:59.420199 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:40:59.420199 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:40:59.420988 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:40:59.420988 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:40:59.420988 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:40:59.420988 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:40:59.420988 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:40:59.420988 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:40:59.420988 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:40:59.420988 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:40:59.420988 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:40:59.420988 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:40:59.420988 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:40:59.420988 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:40:59.420988 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 45/82] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:40:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:40:59.421757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118ca25-a4af-4207-96ba-e5b1596d42e5 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:59.448910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dc5d9-2610-4865-ac19-07804954ab03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:59.449784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dc5d9-2610-4865-ac19-07804954ab03 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:59.450106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dc5d9-2610-4865-ac19-07804954ab03 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:59.450386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482dc5d9-2610-4865-ac19-07804954ab03 None admin] PATH_INFO: `/v3/users/6edbb2f56e344bc2b039e08db2a25dbc` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:59.451568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482dc5d9-2610-4865-ac19-07804954ab03 None admin] RBAC: Authorizing `identity:get_user(user_id=6edbb2f56e344bc2b039e08db2a25dbc)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:59.453016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482dc5d9-2610-4865-ac19-07804954ab03 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:59.454754 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 38/83] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:40:59 2022] GET /identity/v3/users/6edbb2f56e344bc2b039e08db2a25dbc => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:59.457549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dc5d9-2610-4865-ac19-07804954ab03 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:59.524552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118ca25-a4af-4207-96ba-e5b1596d42e5 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:59.541013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:59.552648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:59.556529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482dc5d9-2610-4865-ac19-07804954ab03 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:59.557237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564980a5-0fff-41e8-9e3e-45a2a344209c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:59.557923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564980a5-0fff-41e8-9e3e-45a2a344209c None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:59.558210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564980a5-0fff-41e8-9e3e-45a2a344209c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:59.558496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564980a5-0fff-41e8-9e3e-45a2a344209c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:59.590412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f6b19-c168-41cd-af76-fe21c9908046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:59.591279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f6b19-c168-41cd-af76-fe21c9908046 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:59.591569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f6b19-c168-41cd-af76-fe21c9908046 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:59.591850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f6b19-c168-41cd-af76-fe21c9908046 None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:59.593044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f6b19-c168-41cd-af76-fe21c9908046 None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:59.594575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f6b19-c168-41cd-af76-fe21c9908046 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:59.596272 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 39/84] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:40:59 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:59.597017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f6b19-c168-41cd-af76-fe21c9908046 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:59.629623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564980a5-0fff-41e8-9e3e-45a2a344209c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:59.631675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564980a5-0fff-41e8-9e3e-45a2a344209c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:59.696761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f6b19-c168-41cd-af76-fe21c9908046 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:59.730677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe0403-8145-4474-93a9-b00f4c288331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:59.731345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe0403-8145-4474-93a9-b00f4c288331 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:59.731635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe0403-8145-4474-93a9-b00f4c288331 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:59.731914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe0403-8145-4474-93a9-b00f4c288331 None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:59.732629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe0403-8145-4474-93a9-b00f4c288331 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:59.734082 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 46/85] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:40:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:40:59.734197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe0403-8145-4474-93a9-b00f4c288331 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:59.734914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564980a5-0fff-41e8-9e3e-45a2a344209c None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:59.753108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:59.806225 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 40/86] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:40:59 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:40:59.807074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe0403-8145-4474-93a9-b00f4c288331 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:59.839455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:40:59.848404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564980a5-0fff-41e8-9e3e-45a2a344209c None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:59.888702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed249f77-da6a-46e6-b787-90579184879b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:59.889403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed249f77-da6a-46e6-b787-90579184879b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:59.889697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed249f77-da6a-46e6-b787-90579184879b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:59.890142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed249f77-da6a-46e6-b787-90579184879b None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:40:59.891207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed249f77-da6a-46e6-b787-90579184879b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:40:59.891525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed249f77-da6a-46e6-b787-90579184879b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:40:59.892981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed249f77-da6a-46e6-b787-90579184879b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:40:59.908384 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 47/87] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:40:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:40:59.909188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed249f77-da6a-46e6-b787-90579184879b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:40:59.929066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe0403-8145-4474-93a9-b00f4c288331 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:40:59.967887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d09de-2177-46df-a6a3-9f455543ae6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:40:59.968674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d09de-2177-46df-a6a3-9f455543ae6b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:40:59.968964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d09de-2177-46df-a6a3-9f455543ae6b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:40:59.969245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54d09de-2177-46df-a6a3-9f455543ae6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:00.007804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:00.031792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed249f77-da6a-46e6-b787-90579184879b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:00.057538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d09de-2177-46df-a6a3-9f455543ae6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:00.059135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54d09de-2177-46df-a6a3-9f455543ae6b None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:00.086361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab73bd7-8f72-4cc2-94e5-86e2e51440d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:00.087291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab73bd7-8f72-4cc2-94e5-86e2e51440d1 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:00.087904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab73bd7-8f72-4cc2-94e5-86e2e51440d1 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:00.088435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab73bd7-8f72-4cc2-94e5-86e2e51440d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:00.089592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab73bd7-8f72-4cc2-94e5-86e2e51440d1 None admin] RBAC: Adding query filter params (role.id=c0c09ba97c694896ae14fceb6982b1a5, scope.project.id=e872f1acf4e544f6963d85f6afc13e1c, user.id=6edbb2f56e344bc2b039e08db2a25dbc) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:00.089719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:00.089936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab73bd7-8f72-4cc2-94e5-86e2e51440d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:00.091833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab73bd7-8f72-4cc2-94e5-86e2e51440d1 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:00.103904 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 48/88] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:40:59 2022] GET /identity/v3/role_assignments?user.id=6edbb2f56e344bc2b039e08db2a25dbc&scope.project.id=e872f1acf4e544f6963d85f6afc13e1c&role.id=c0c09ba97c694896ae14fceb6982b1a5 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:00.104768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab73bd7-8f72-4cc2-94e5-86e2e51440d1 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:00.213114 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 41/89] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:40:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:00.214050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d09de-2177-46df-a6a3-9f455543ae6b None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:00.233606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab73bd7-8f72-4cc2-94e5-86e2e51440d1 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:00.315584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a574489-485c-4404-8fbb-24062d5318c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:00.316413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a574489-485c-4404-8fbb-24062d5318c0 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:00.316707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a574489-485c-4404-8fbb-24062d5318c0 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:00.316996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a574489-485c-4404-8fbb-24062d5318c0 None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:00.318803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a574489-485c-4404-8fbb-24062d5318c0 None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:00.320280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a574489-485c-4404-8fbb-24062d5318c0 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:00.325954 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 49/90] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:00 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:00.326828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a574489-485c-4404-8fbb-24062d5318c0 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:00.343364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54d09de-2177-46df-a6a3-9f455543ae6b None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:00.347505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:00.374678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:00.388534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16714cf-19f3-461a-b7b3-bbd14b58d328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:00.389470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16714cf-19f3-461a-b7b3-bbd14b58d328 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:00.389747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16714cf-19f3-461a-b7b3-bbd14b58d328 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:00.390054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16714cf-19f3-461a-b7b3-bbd14b58d328 None admin] PATH_INFO: `/v3/users/6edbb2f56e344bc2b039e08db2a25dbc` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:00.395294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16714cf-19f3-461a-b7b3-bbd14b58d328 None admin] RBAC: Authorizing `identity:get_user(user_id=6edbb2f56e344bc2b039e08db2a25dbc)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:00.402058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16714cf-19f3-461a-b7b3-bbd14b58d328 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:00.403770 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 42/91] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:00 2022] GET /identity/v3/users/6edbb2f56e344bc2b039e08db2a25dbc => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:00.405541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16714cf-19f3-461a-b7b3-bbd14b58d328 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:00.494742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a574489-485c-4404-8fbb-24062d5318c0 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:00.593769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f0a8b-c176-4e70-8def-3b0459fa55b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:00.595021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f0a8b-c176-4e70-8def-3b0459fa55b4 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:00.595499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f0a8b-c176-4e70-8def-3b0459fa55b4 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:00.595968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f0a8b-c176-4e70-8def-3b0459fa55b4 None admin] PATH_INFO: `/v3/projects/b8c5ffd284744fbf8e8eda63ea09cd3e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:00.600401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f0a8b-c176-4e70-8def-3b0459fa55b4 None admin] RBAC: Authorizing `identity:get_project(project_id=b8c5ffd284744fbf8e8eda63ea09cd3e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:00.605754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f0a8b-c176-4e70-8def-3b0459fa55b4 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:00.607664 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 50/92] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:00 2022] GET /identity/v3/projects/b8c5ffd284744fbf8e8eda63ea09cd3e => generated 312 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:00.608455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f0a8b-c176-4e70-8def-3b0459fa55b4 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:00.634980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16714cf-19f3-461a-b7b3-bbd14b58d328 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:00.672364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb2a2b-0ba3-4f0c-9de5-ed1e3ad86788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:00.674601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb2a2b-0ba3-4f0c-9de5-ed1e3ad86788 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:00.675776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb2a2b-0ba3-4f0c-9de5-ed1e3ad86788 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:00.676251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb2a2b-0ba3-4f0c-9de5-ed1e3ad86788 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:00.677826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb2a2b-0ba3-4f0c-9de5-ed1e3ad86788 None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=e872f1acf4e544f6963d85f6afc13e1c, user.id=3e371762e9974f2484669486a37339c7) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:00.678928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb2a2b-0ba3-4f0c-9de5-ed1e3ad86788 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:00.680750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb2a2b-0ba3-4f0c-9de5-ed1e3ad86788 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:00.696162 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 43/93] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:41:00 2022] GET /identity/v3/role_assignments?user.id=3e371762e9974f2484669486a37339c7&scope.project.id=e872f1acf4e544f6963d85f6afc13e1c&role.id=7805b93b32f1472f8365936d4d57d49a => generated 259 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:00.697291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb2a2b-0ba3-4f0c-9de5-ed1e3ad86788 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:00.782224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:00.813390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:00.819243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f0a8b-c176-4e70-8def-3b0459fa55b4 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:00.899955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb2a2b-0ba3-4f0c-9de5-ed1e3ad86788 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:00.903106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adab813-6e70-4e2d-9100-be81b887efc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:00.903786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adab813-6e70-4e2d-9100-be81b887efc9 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:00.904077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adab813-6e70-4e2d-9100-be81b887efc9 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:00.904363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adab813-6e70-4e2d-9100-be81b887efc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:00.944136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d2317-e07a-40cd-8fb8-42008096f5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:00.944871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d2317-e07a-40cd-8fb8-42008096f5c5 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:00.945183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d2317-e07a-40cd-8fb8-42008096f5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:00.945454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d2317-e07a-40cd-8fb8-42008096f5c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:00.946336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d2317-e07a-40cd-8fb8-42008096f5c5 None admin] RBAC: Adding query filter params (role.id=c0c09ba97c694896ae14fceb6982b1a5, scope.project.id=b8c5ffd284744fbf8e8eda63ea09cd3e, user.id=6edbb2f56e344bc2b039e08db2a25dbc) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:00.946650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d2317-e07a-40cd-8fb8-42008096f5c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:00.948201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d2317-e07a-40cd-8fb8-42008096f5c5 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:00.962297 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 44/94] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:41:00 2022] GET /identity/v3/role_assignments?user.id=6edbb2f56e344bc2b039e08db2a25dbc&scope.project.id=b8c5ffd284744fbf8e8eda63ea09cd3e&role.id=c0c09ba97c694896ae14fceb6982b1a5 => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:01.030270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adab813-6e70-4e2d-9100-be81b887efc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:01.031835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adab813-6e70-4e2d-9100-be81b887efc9 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:01.131337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:01.142154 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 51/95] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:01.148175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d2317-e07a-40cd-8fb8-42008096f5c5 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:01.308920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d2317-e07a-40cd-8fb8-42008096f5c5 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:01.335951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:01.356212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244c31cb-016f-4cf2-8999-9c7c7fda4b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:01.357091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c31cb-016f-4cf2-8999-9c7c7fda4b20 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:01.357428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c31cb-016f-4cf2-8999-9c7c7fda4b20 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:01.357712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244c31cb-016f-4cf2-8999-9c7c7fda4b20 None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:01.358971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c31cb-016f-4cf2-8999-9c7c7fda4b20 None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:01.360453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244c31cb-016f-4cf2-8999-9c7c7fda4b20 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:01.362240 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 45/96] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:01 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:01.368724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adab813-6e70-4e2d-9100-be81b887efc9 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:01.498658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adab813-6e70-4e2d-9100-be81b887efc9 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:01.549775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14654703-cecc-484a-9145-a22f2d5819ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:01.550465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14654703-cecc-484a-9145-a22f2d5819ac None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:01.550753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14654703-cecc-484a-9145-a22f2d5819ac None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:01.551049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14654703-cecc-484a-9145-a22f2d5819ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:01.552592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14654703-cecc-484a-9145-a22f2d5819ac None admin] RBAC: Adding query filter params (role.id=92b887e453ad4fa9aa5c49f7594a7e53, scope.project.id=e872f1acf4e544f6963d85f6afc13e1c, user.id=6edbb2f56e344bc2b039e08db2a25dbc) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:01.552914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14654703-cecc-484a-9145-a22f2d5819ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:01.554420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14654703-cecc-484a-9145-a22f2d5819ac None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:01.566802 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 52/97] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:41:01 2022] GET /identity/v3/role_assignments?user.id=6edbb2f56e344bc2b039e08db2a25dbc&scope.project.id=e872f1acf4e544f6963d85f6afc13e1c&role.id=92b887e453ad4fa9aa5c49f7594a7e53 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:03.232224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e3886-ca6e-480b-9363-5d04961c6075 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:03.232777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e3886-ca6e-480b-9363-5d04961c6075 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:03.232777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e3886-ca6e-480b-9363-5d04961c6075 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:03.234339 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 46/98] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:03.240292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960f398-90e3-4007-8b6c-6ccb34ec2979 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:03.240702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960f398-90e3-4007-8b6c-6ccb34ec2979 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:03.240842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e960f398-90e3-4007-8b6c-6ccb34ec2979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:03.267073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e960f398-90e3-4007-8b6c-6ccb34ec2979 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:03.268517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e960f398-90e3-4007-8b6c-6ccb34ec2979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:03.318992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:03.331703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dd0d4-1686-4db9-98fb-fa963935a815 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:03.345355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dd0d4-1686-4db9-98fb-fa963935a815 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:03.345355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dd0d4-1686-4db9-98fb-fa963935a815 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:03.345355 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 47/99] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:41:03.345355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbab479-3ccf-4306-8bdb-2ddd5b641ec9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:03.345355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbab479-3ccf-4306-8bdb-2ddd5b641ec9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:03.345355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbab479-3ccf-4306-8bdb-2ddd5b641ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:03.361573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-afbab479-3ccf-4306-8bdb-2ddd5b641ec9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:03.364065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-afbab479-3ccf-4306-8bdb-2ddd5b641ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:03.372820 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 53/100] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:03.418785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960f398-90e3-4007-8b6c-6ccb34ec2979 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:03.465718 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 48/101] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:41:03.515713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbab479-3ccf-4306-8bdb-2ddd5b641ec9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:03.607588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e960f398-90e3-4007-8b6c-6ccb34ec2979 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:03.661986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbab479-3ccf-4306-8bdb-2ddd5b641ec9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:03.667908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbefda60-ea19-4c4a-b70b-31c2eafad4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:03.668343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefda60-ea19-4c4a-b70b-31c2eafad4ac None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:03.668638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefda60-ea19-4c4a-b70b-31c2eafad4ac None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:03.668921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefda60-ea19-4c4a-b70b-31c2eafad4ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:03.670205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbefda60-ea19-4c4a-b70b-31c2eafad4ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:03.671687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbefda60-ea19-4c4a-b70b-31c2eafad4ac None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:03.673485 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 54/102] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:03.679442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:03.683215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c53f4-3c6a-40bc-88cb-16f8f5dc6188 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:03.683517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c53f4-3c6a-40bc-88cb-16f8f5dc6188 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:03.683866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c53f4-3c6a-40bc-88cb-16f8f5dc6188 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:03.685200 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 55/103] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:03 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:03.685994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c53f4-3c6a-40bc-88cb-16f8f5dc6188 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:03.703093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef48f331-010e-4d76-aab8-b293bcf11bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:03.703903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48f331-010e-4d76-aab8-b293bcf11bb1 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:03.704186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48f331-010e-4d76-aab8-b293bcf11bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:03.704462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef48f331-010e-4d76-aab8-b293bcf11bb1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:03.715911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef48f331-010e-4d76-aab8-b293bcf11bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:03.724344 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-ef48f331-010e-4d76-aab8-b293bcf11bb1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:03.724344 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:03.724344 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:03.724344 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:03.724344 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:03.724344 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:03.724344 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:03.724344 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:03.724344 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:03.724344 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:03.724344 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:03.724344 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:03.724344 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:03.724344 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:03.724344 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:03.724344 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:03.725352 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:03.725352 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:03.725352 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:03.725352 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:03.725352 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:03.725352 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:03.725352 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:03.725352 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:03.725352 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:03.725352 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:03.725352 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:03.725352 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:03.725352 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:03.725352 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:03.725352 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:03.725352 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:03.726233 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:03.726233 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:03.726233 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:03.726233 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:03.726233 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:03.726233 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:03.726233 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:03.726233 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:03.726233 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:03.726233 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:03.726233 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:03.726233 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:03.726233 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 49/104] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:41:03.726911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e544ed-d1a0-4c62-941c-8588a81c2baa None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:03.727952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e544ed-d1a0-4c62-941c-8588a81c2baa None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:03.727952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e544ed-d1a0-4c62-941c-8588a81c2baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:03.756160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f4e544ed-d1a0-4c62-941c-8588a81c2baa None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:03.757318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f4e544ed-d1a0-4c62-941c-8588a81c2baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:03.793397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:03.871369 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 50/105] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:03.872257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e544ed-d1a0-4c62-941c-8588a81c2baa None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:03.895125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c53f4-3c6a-40bc-88cb-16f8f5dc6188 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:03.963518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ff8f7-c93d-4434-8f7f-5b710642edc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:03.964589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ff8f7-c93d-4434-8f7f-5b710642edc9 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:03.965090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ff8f7-c93d-4434-8f7f-5b710642edc9 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:03.965579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ff8f7-c93d-4434-8f7f-5b710642edc9 None admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:03.966537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ff8f7-c93d-4434-8f7f-5b710642edc9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:03.968198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ff8f7-c93d-4434-8f7f-5b710642edc9 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:03.984720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e544ed-d1a0-4c62-941c-8588a81c2baa None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:03.992421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:04.021530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf23e8e-a486-48f8-8bb4-3c4ceb419189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:04.022598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf23e8e-a486-48f8-8bb4-3c4ceb419189 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:04.023107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf23e8e-a486-48f8-8bb4-3c4ceb419189 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:04.023595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf23e8e-a486-48f8-8bb4-3c4ceb419189 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:04.024549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf23e8e-a486-48f8-8bb4-3c4ceb419189 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:04.025063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf23e8e-a486-48f8-8bb4-3c4ceb419189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:04.026740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf23e8e-a486-48f8-8bb4-3c4ceb419189 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:04.036157 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 51/106] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:04.041609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf23e8e-a486-48f8-8bb4-3c4ceb419189 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:04.080141 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 56/107] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 20:41:03 2022] POST /identity/v3/projects => generated 302 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:41:04.081498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ff8f7-c93d-4434-8f7f-5b710642edc9 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:04.151113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf23e8e-a486-48f8-8bb4-3c4ceb419189 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:04.191596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e364b-2212-41f1-b7f4-9b5ffb69ec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:04.192987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e364b-2212-41f1-b7f4-9b5ffb69ec10 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:04.193279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e364b-2212-41f1-b7f4-9b5ffb69ec10 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:04.193560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e364b-2212-41f1-b7f4-9b5ffb69ec10 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:04.221000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e364b-2212-41f1-b7f4-9b5ffb69ec10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:04.230181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-b76e364b-2212-41f1-b7f4-9b5ffb69ec10 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:04.230181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:04.230181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:04.230181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:04.230181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:04.230181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:04.230181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:04.230181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:04.230181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:04.230181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:04.230181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:04.230181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:04.230181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:04.230181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:04.230181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:04.230181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:04.231038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:04.231038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:04.231038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:04.231038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:04.231038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:04.231038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:04.231038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:04.231038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:04.231038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:04.231038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:04.231038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:04.231038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:04.231038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:04.231038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:04.231038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:04.231038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:04.231790 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:04.231790 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:04.231790 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:04.231790 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:04.231790 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:04.231790 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:04.231790 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:04.231790 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:04.231790 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:04.231790 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:04.231790 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:04.231790 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:04.238267 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 52/108] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:04.244422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e364b-2212-41f1-b7f4-9b5ffb69ec10 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:04.294264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:04.310600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ff8f7-c93d-4434-8f7f-5b710642edc9 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:04.316773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:04.386581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7afd58-412a-4964-a38b-67aa87076f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:04.388639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7afd58-412a-4964-a38b-67aa87076f4c None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:04.388934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7afd58-412a-4964-a38b-67aa87076f4c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:04.389235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7afd58-412a-4964-a38b-67aa87076f4c None admin] PATH_INFO: `/v3/users/a1e90098a71445c086e97d571c6cf4bc` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:04.422559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7afd58-412a-4964-a38b-67aa87076f4c None admin] RBAC: Authorizing `identity:get_user(user_id=a1e90098a71445c086e97d571c6cf4bc)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:04.424033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7afd58-412a-4964-a38b-67aa87076f4c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:04.427888 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 57/109] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:04 2022] GET /identity/v3/users/a1e90098a71445c086e97d571c6cf4bc => generated 289 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:04.433817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0b0b4-6109-4bb6-a62c-08030b7cfdb8 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:04.434176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0b0b4-6109-4bb6-a62c-08030b7cfdb8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:04.434412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0b0b4-6109-4bb6-a62c-08030b7cfdb8 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:04.435998 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 58/110] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:04.443722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13252242-d597-423b-8770-a04a347536fc None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:04.449119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13252242-d597-423b-8770-a04a347536fc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:04.449119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13252242-d597-423b-8770-a04a347536fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:04.474190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e364b-2212-41f1-b7f4-9b5ffb69ec10 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:04.478233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-13252242-d597-423b-8770-a04a347536fc None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:04.480292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-13252242-d597-423b-8770-a04a347536fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:04.547317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399f2f4-9640-4bc5-9d93-bdb4e3af95ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:04.547994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399f2f4-9640-4bc5-9d93-bdb4e3af95ae None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:04.548280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399f2f4-9640-4bc5-9d93-bdb4e3af95ae None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:04.548558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399f2f4-9640-4bc5-9d93-bdb4e3af95ae None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:04.549322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399f2f4-9640-4bc5-9d93-bdb4e3af95ae None admin] RBAC: Adding query filter params (name=member) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:04.549632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399f2f4-9640-4bc5-9d93-bdb4e3af95ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:04.551265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399f2f4-9640-4bc5-9d93-bdb4e3af95ae None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:04.560682 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 53/111] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:04.561406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399f2f4-9640-4bc5-9d93-bdb4e3af95ae None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:04.579672 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 59/112] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:04.581019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc070e8-324b-47b8-90e8-62306cef0e66 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:04.581315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc070e8-324b-47b8-90e8-62306cef0e66 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:04.581720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc070e8-324b-47b8-90e8-62306cef0e66 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:04.583667 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 60/113] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:04.584276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc070e8-324b-47b8-90e8-62306cef0e66 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:04.688662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:04.691041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:04.707064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399f2f4-9640-4bc5-9d93-bdb4e3af95ae None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:04.717074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc070e8-324b-47b8-90e8-62306cef0e66 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:04.743127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb7854-6e89-4d21-836c-9170eef8ba39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:04.743902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb7854-6e89-4d21-836c-9170eef8ba39 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:04.744259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb7854-6e89-4d21-836c-9170eef8ba39 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:04.744613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb7854-6e89-4d21-836c-9170eef8ba39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:04.757193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34950fd2-ba58-4914-9f2f-daf4daea3624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:04.758250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34950fd2-ba58-4914-9f2f-daf4daea3624 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:04.759894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34950fd2-ba58-4914-9f2f-daf4daea3624 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:04.760169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34950fd2-ba58-4914-9f2f-daf4daea3624 None admin] PATH_INFO: `/v3/users/6edbb2f56e344bc2b039e08db2a25dbc` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:04.761692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34950fd2-ba58-4914-9f2f-daf4daea3624 None admin] RBAC: Authorizing `identity:get_user(user_id=6edbb2f56e344bc2b039e08db2a25dbc)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:04.763313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34950fd2-ba58-4914-9f2f-daf4daea3624 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:04.765250 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 61/114] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:04 2022] GET /identity/v3/users/6edbb2f56e344bc2b039e08db2a25dbc => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:04.766809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e264630-21d4-4a34-9393-1aa32d30abb0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:04.767105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e264630-21d4-4a34-9393-1aa32d30abb0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:04.767382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e264630-21d4-4a34-9393-1aa32d30abb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:04.787228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2e264630-21d4-4a34-9393-1aa32d30abb0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:04.788810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2e264630-21d4-4a34-9393-1aa32d30abb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:04.817259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb7854-6e89-4d21-836c-9170eef8ba39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:04.819413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb7854-6e89-4d21-836c-9170eef8ba39 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:04.865471 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 62/115] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:04.871465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e264630-21d4-4a34-9393-1aa32d30abb0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:04.920353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:04.932053 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 54/116] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:04.933102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb7854-6e89-4d21-836c-9170eef8ba39 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:04.964152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:04.983175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e264630-21d4-4a34-9393-1aa32d30abb0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:05.019935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c91a00-1d6a-4832-bc84-545239c74caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:05.021421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c91a00-1d6a-4832-bc84-545239c74caa None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:05.021712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c91a00-1d6a-4832-bc84-545239c74caa None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:05.022023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c91a00-1d6a-4832-bc84-545239c74caa None admin] PATH_INFO: `/v3/roles/member` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:05.036724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c91a00-1d6a-4832-bc84-545239c74caa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:05.045161 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-61c91a00-1d6a-4832-bc84-545239c74caa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:05.045161 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:05.045161 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:05.045161 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:05.045161 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:05.045161 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:05.045161 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:05.045161 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:05.045161 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:05.045161 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:05.045161 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:05.045161 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:05.045161 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:05.045161 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:05.045161 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:05.045161 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:05.046031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:05.046031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:05.046031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:05.046031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:05.046031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:05.046031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:05.046031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:05.046031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:05.046031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:05.046031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:05.046031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:05.046031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:05.046031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:05.046031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:05.046031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:05.046031 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:05.051898 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:05.051898 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:05.051898 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:05.051898 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:05.051898 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:05.051898 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:05.051898 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:05.051898 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:05.051898 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:05.051898 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:05.051898 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:05.051898 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:05.051898 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 63/117] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:05.052527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb7854-6e89-4d21-836c-9170eef8ba39 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:05.052596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435183c-d0be-413c-89f7-5c1bfaae3c24 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:05.052596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435183c-d0be-413c-89f7-5c1bfaae3c24 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:05.052596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c435183c-d0be-413c-89f7-5c1bfaae3c24 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:05.052596 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 64/118] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:05.054307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435183c-d0be-413c-89f7-5c1bfaae3c24 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:05.093124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423df6b-ab9c-4a9e-8964-a05245f6f26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:05.094024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423df6b-ab9c-4a9e-8964-a05245f6f26a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:05.094313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423df6b-ab9c-4a9e-8964-a05245f6f26a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:05.094598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423df6b-ab9c-4a9e-8964-a05245f6f26a None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:05.095771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423df6b-ab9c-4a9e-8964-a05245f6f26a None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:05.097218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423df6b-ab9c-4a9e-8964-a05245f6f26a None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:05.099471 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 55/119] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:04 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:05.099734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423df6b-ab9c-4a9e-8964-a05245f6f26a None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:05.163842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:05.171812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c435183c-d0be-413c-89f7-5c1bfaae3c24 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:05.207105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6bf46-1e1b-4067-9ced-118d744babda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:05.207923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6bf46-1e1b-4067-9ced-118d744babda None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:05.208218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6bf46-1e1b-4067-9ced-118d744babda None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:05.208496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6bf46-1e1b-4067-9ced-118d744babda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:05.208648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423df6b-ab9c-4a9e-8964-a05245f6f26a None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:05.216653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:05.220316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c6bf46-1e1b-4067-9ced-118d744babda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:05.228079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-e8c6bf46-1e1b-4067-9ced-118d744babda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:41:05.228079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:05.228079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:05.228079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:05.228079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:05.228079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:05.228079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:05.228079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:05.228079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:05.228079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:05.228079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:05.228079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:05.228079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:05.228079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:05.228079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:05.228079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:05.229108 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:05.229108 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:05.229108 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:05.229108 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:05.229108 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:05.229108 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:05.229108 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:05.229108 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:05.229108 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:05.229108 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:05.229108 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:05.229108 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:05.229108 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:05.229108 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:05.229108 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:05.229108 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:05.230045 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:05.230045 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:05.230045 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:05.230045 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:05.230045 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:05.230045 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:05.230045 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:05.230045 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:05.230045 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:05.230045 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:05.230045 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:41:05.230045 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:05.230045 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 65/120] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:41:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:05.230687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6bf46-1e1b-4067-9ced-118d744babda None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:05.246389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d005b244-73cc-4c63-935a-832440f12b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:05.247251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005b244-73cc-4c63-935a-832440f12b95 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:05.247535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005b244-73cc-4c63-935a-832440f12b95 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:05.247811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d005b244-73cc-4c63-935a-832440f12b95 None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:05.248976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d005b244-73cc-4c63-935a-832440f12b95 None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:05.250460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d005b244-73cc-4c63-935a-832440f12b95 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:05.252178 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 56/121] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:05 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:05.253294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c59740-2f08-497a-b96c-27abf4de7708 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:05.253578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c59740-2f08-497a-b96c-27abf4de7708 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:05.253880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c59740-2f08-497a-b96c-27abf4de7708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:05.274312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b9c59740-2f08-497a-b96c-27abf4de7708 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:05.275507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b9c59740-2f08-497a-b96c-27abf4de7708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:05.341907 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 57/122] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:05.343002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c59740-2f08-497a-b96c-27abf4de7708 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:05.356229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c6bf46-1e1b-4067-9ced-118d744babda None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:05.392699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579f13d-6b34-40a0-89e9-6dafb74d08cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:05.393384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579f13d-6b34-40a0-89e9-6dafb74d08cb None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:05.393676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579f13d-6b34-40a0-89e9-6dafb74d08cb None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:05.393997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579f13d-6b34-40a0-89e9-6dafb74d08cb None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:05.395128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579f13d-6b34-40a0-89e9-6dafb74d08cb None admin] RBAC: Adding query filter params (name=member) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:05.395951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579f13d-6b34-40a0-89e9-6dafb74d08cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:05.397664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579f13d-6b34-40a0-89e9-6dafb74d08cb None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:05.406060 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 66/123] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:05.410299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:05.412998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579f13d-6b34-40a0-89e9-6dafb74d08cb None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:05.454859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c59740-2f08-497a-b96c-27abf4de7708 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:05.484628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:05.492847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a18d4-677b-4a15-9c00-4f568acc773b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:05.493749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a18d4-677b-4a15-9c00-4f568acc773b None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:05.494068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a18d4-677b-4a15-9c00-4f568acc773b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:05.494420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3a18d4-677b-4a15-9c00-4f568acc773b None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c/users/6edbb2f56e344bc2b039e08db2a25dbc/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:05.496375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a18d4-677b-4a15-9c00-4f568acc773b None admin] RBAC: Authorizing `identity:create_grant(project_id=e872f1acf4e544f6963d85f6afc13e1c, user_id=6edbb2f56e344bc2b039e08db2a25dbc, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:05.497858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3a18d4-677b-4a15-9c00-4f568acc773b None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:05.523281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579f13d-6b34-40a0-89e9-6dafb74d08cb None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:05.556319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4d474-8596-49c6-a6ab-81002e0395e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:05.556995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4d474-8596-49c6-a6ab-81002e0395e9 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:05.557285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4d474-8596-49c6-a6ab-81002e0395e9 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:05.557569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4d474-8596-49c6-a6ab-81002e0395e9 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:05.558359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4d474-8596-49c6-a6ab-81002e0395e9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:05.558675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4d474-8596-49c6-a6ab-81002e0395e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:05.560077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b4d474-8596-49c6-a6ab-81002e0395e9 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:05.568885 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 67/124] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:05.569543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4d474-8596-49c6-a6ab-81002e0395e9 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:05.591025 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 58/125] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:41:05 2022] PUT /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c/users/6edbb2f56e344bc2b039e08db2a25dbc/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:41:05.592098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a18d4-677b-4a15-9c00-4f568acc773b None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:05.713186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3a18d4-677b-4a15-9c00-4f568acc773b None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:05.731786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:05.734016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4d474-8596-49c6-a6ab-81002e0395e9 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:05.752555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bbec5-734e-4066-a43d-ef62495ca726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:05.753504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bbec5-734e-4066-a43d-ef62495ca726 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:05.753888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bbec5-734e-4066-a43d-ef62495ca726 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:05.754293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bbec5-734e-4066-a43d-ef62495ca726 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:05.764651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:05.767791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380bbec5-734e-4066-a43d-ef62495ca726 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:05.777712 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-380bbec5-734e-4066-a43d-ef62495ca726 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:05.777712 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:05.777712 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:05.777712 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:05.777712 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:05.777712 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:05.777712 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:05.777712 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:05.777712 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:05.777712 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:05.777712 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:05.777712 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:05.777712 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:05.777712 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:05.777712 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:05.777712 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:05.778641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:05.778641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:05.778641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:05.778641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:05.778641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:05.778641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:05.778641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:05.778641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:05.778641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:05.778641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:05.778641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:05.778641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:05.778641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:05.778641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:05.778641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:05.778641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:05.779464 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:05.779464 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:05.779464 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:05.779464 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:05.779464 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:05.779464 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:05.779464 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:05.779464 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:05.779464 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:05.779464 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:05.779464 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:05.779464 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:05.779464 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 59/126] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:41:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:05.780098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bbec5-734e-4066-a43d-ef62495ca726 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:05.786030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bbf62-d2dd-42f7-9e96-0d747554a8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:05.786969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bbf62-d2dd-42f7-9e96-0d747554a8e4 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:05.787412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bbf62-d2dd-42f7-9e96-0d747554a8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:05.787848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bbf62-d2dd-42f7-9e96-0d747554a8e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:05.788849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53bbf62-d2dd-42f7-9e96-0d747554a8e4 None admin] RBAC: Adding query filter params (role.id=19a96330fa9044a28cedef74491c0d4c, scope.project.id=e872f1acf4e544f6963d85f6afc13e1c, user.id=a1e90098a71445c086e97d571c6cf4bc) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:05.789327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53bbf62-d2dd-42f7-9e96-0d747554a8e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:05.791231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53bbf62-d2dd-42f7-9e96-0d747554a8e4 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:05.798480 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 68/127] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:41:05 2022] GET /identity/v3/role_assignments?user.id=a1e90098a71445c086e97d571c6cf4bc&scope.project.id=e872f1acf4e544f6963d85f6afc13e1c&role.id=19a96330fa9044a28cedef74491c0d4c => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:05.799586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bbf62-d2dd-42f7-9e96-0d747554a8e4 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:05.893730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bbec5-734e-4066-a43d-ef62495ca726 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:05.936425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc60de4-4ec4-40a7-865a-1436a12af631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:05.937214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bbf62-d2dd-42f7-9e96-0d747554a8e4 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:05.937459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc60de4-4ec4-40a7-865a-1436a12af631 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:05.937810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc60de4-4ec4-40a7-865a-1436a12af631 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:05.938206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc60de4-4ec4-40a7-865a-1436a12af631 None admin] PATH_INFO: `/v3/users/6edbb2f56e344bc2b039e08db2a25dbc` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:05.940768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc60de4-4ec4-40a7-865a-1436a12af631 None admin] RBAC: Authorizing `identity:get_user(user_id=6edbb2f56e344bc2b039e08db2a25dbc)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:05.942984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc60de4-4ec4-40a7-865a-1436a12af631 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:05.944925 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 60/128] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:05 2022] GET /identity/v3/users/6edbb2f56e344bc2b039e08db2a25dbc => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:05.945623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc60de4-4ec4-40a7-865a-1436a12af631 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:06.004447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05f003-db7b-41fc-bf79-c2caf6d94cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:06.005856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05f003-db7b-41fc-bf79-c2caf6d94cbf None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:06.006149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05f003-db7b-41fc-bf79-c2caf6d94cbf None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:06.006429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e05f003-db7b-41fc-bf79-c2caf6d94cbf None admin] PATH_INFO: `/v3/users/3e371762e9974f2484669486a37339c7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:06.008651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e05f003-db7b-41fc-bf79-c2caf6d94cbf None admin] RBAC: Authorizing `identity:get_user(user_id=3e371762e9974f2484669486a37339c7)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:06.010179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e05f003-db7b-41fc-bf79-c2caf6d94cbf None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:06.012085 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 69/129] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:05 2022] GET /identity/v3/users/3e371762e9974f2484669486a37339c7 => generated 247 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:06.012709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05f003-db7b-41fc-bf79-c2caf6d94cbf None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:06.034608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:06.105164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc60de4-4ec4-40a7-865a-1436a12af631 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:06.144286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d9c61-ec21-4a8c-9985-9d10500ce9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:06.145350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d9c61-ec21-4a8c-9985-9d10500ce9bd None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:06.145901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d9c61-ec21-4a8c-9985-9d10500ce9bd None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:06.146405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8d9c61-ec21-4a8c-9985-9d10500ce9bd None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:06.147724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d9c61-ec21-4a8c-9985-9d10500ce9bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:06.148259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d9c61-ec21-4a8c-9985-9d10500ce9bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:06.150394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8d9c61-ec21-4a8c-9985-9d10500ce9bd None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:06.159894 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 61/130] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:41:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:06.161447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d9c61-ec21-4a8c-9985-9d10500ce9bd None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:06.162064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:06.250252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e05f003-db7b-41fc-bf79-c2caf6d94cbf None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:06.313773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91ae64-621c-47a1-89e4-c12d29a57fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:06.314757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91ae64-621c-47a1-89e4-c12d29a57fe2 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:06.315103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91ae64-621c-47a1-89e4-c12d29a57fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:06.315389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e91ae64-621c-47a1-89e4-c12d29a57fe2 None admin] PATH_INFO: `/v3/projects/b8c5ffd284744fbf8e8eda63ea09cd3e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:06.316657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91ae64-621c-47a1-89e4-c12d29a57fe2 None admin] RBAC: Authorizing `identity:get_project(project_id=b8c5ffd284744fbf8e8eda63ea09cd3e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:06.318659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e91ae64-621c-47a1-89e4-c12d29a57fe2 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:06.323424 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 70/131] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:06 2022] GET /identity/v3/projects/b8c5ffd284744fbf8e8eda63ea09cd3e => generated 312 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:06.324229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91ae64-621c-47a1-89e4-c12d29a57fe2 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:06.368799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8d9c61-ec21-4a8c-9985-9d10500ce9bd None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:06.380599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:06.436045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004f3f1-5390-40ad-b19d-2bda6e7dd9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:06.437390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004f3f1-5390-40ad-b19d-2bda6e7dd9d0 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:06.437901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004f3f1-5390-40ad-b19d-2bda6e7dd9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:06.438371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a004f3f1-5390-40ad-b19d-2bda6e7dd9d0 None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:06.440411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004f3f1-5390-40ad-b19d-2bda6e7dd9d0 None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:06.442126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a004f3f1-5390-40ad-b19d-2bda6e7dd9d0 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:06.444260 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 62/132] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:06 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:06.447017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004f3f1-5390-40ad-b19d-2bda6e7dd9d0 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:06.485485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e91ae64-621c-47a1-89e4-c12d29a57fe2 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:06.538243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:06.564001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635110ed-65e5-445d-82b1-6fb6e1a24447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:06.564831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635110ed-65e5-445d-82b1-6fb6e1a24447 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:06.565125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635110ed-65e5-445d-82b1-6fb6e1a24447 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:06.565406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635110ed-65e5-445d-82b1-6fb6e1a24447 None admin] PATH_INFO: `/v3/users/6edbb2f56e344bc2b039e08db2a25dbc` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:06.570600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635110ed-65e5-445d-82b1-6fb6e1a24447 None admin] RBAC: Authorizing `identity:get_user(user_id=6edbb2f56e344bc2b039e08db2a25dbc)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:06.572104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635110ed-65e5-445d-82b1-6fb6e1a24447 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:06.577984 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 71/133] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:06 2022] GET /identity/v3/users/6edbb2f56e344bc2b039e08db2a25dbc => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:06.577984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635110ed-65e5-445d-82b1-6fb6e1a24447 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:06.601049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a004f3f1-5390-40ad-b19d-2bda6e7dd9d0 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:06.639216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14462f8-715e-40c8-aca0-059a95377537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:06.641957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14462f8-715e-40c8-aca0-059a95377537 None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:06.641957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14462f8-715e-40c8-aca0-059a95377537 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:06.641957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14462f8-715e-40c8-aca0-059a95377537 None admin] PATH_INFO: `/v3/projects/b8c5ffd284744fbf8e8eda63ea09cd3e/users/6edbb2f56e344bc2b039e08db2a25dbc/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:06.644302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14462f8-715e-40c8-aca0-059a95377537 None admin] RBAC: Authorizing `identity:create_grant(project_id=b8c5ffd284744fbf8e8eda63ea09cd3e, user_id=6edbb2f56e344bc2b039e08db2a25dbc, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:06.646707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14462f8-715e-40c8-aca0-059a95377537 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:06.743933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635110ed-65e5-445d-82b1-6fb6e1a24447 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:06.746506 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 63/134] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:41:06 2022] PUT /identity/v3/projects/b8c5ffd284744fbf8e8eda63ea09cd3e/users/6edbb2f56e344bc2b039e08db2a25dbc/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:41:06.747715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14462f8-715e-40c8-aca0-059a95377537 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:06.803874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33b1fe-052b-4f15-a4dc-4e3c861fb73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:06.804864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33b1fe-052b-4f15-a4dc-4e3c861fb73c None admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:06.805548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33b1fe-052b-4f15-a4dc-4e3c861fb73c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:06.805881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33b1fe-052b-4f15-a4dc-4e3c861fb73c None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c/users/3e371762e9974f2484669486a37339c7/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:06.813859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33b1fe-052b-4f15-a4dc-4e3c861fb73c None admin] RBAC: Authorizing `identity:create_grant(project_id=e872f1acf4e544f6963d85f6afc13e1c, user_id=3e371762e9974f2484669486a37339c7, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:06.813859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33b1fe-052b-4f15-a4dc-4e3c861fb73c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:06.843445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:06.901608 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 72/135] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:41:06 2022] PUT /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c/users/3e371762e9974f2484669486a37339c7/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:41:06.903221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222829c6-9777-49e1-9370-dae0fbc52dec None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:06.903523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222829c6-9777-49e1-9370-dae0fbc52dec None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:06.904822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222829c6-9777-49e1-9370-dae0fbc52dec None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:06.905222 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 73/136] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:06.912280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f8e73-9c9b-464b-bf60-cf020bf449c4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:06.912594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f8e73-9c9b-464b-bf60-cf020bf449c4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:06.913892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f8e73-9c9b-464b-bf60-cf020bf449c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:06.946795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-941f8e73-9c9b-464b-bf60-cf020bf449c4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:06.948917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-941f8e73-9c9b-464b-bf60-cf020bf449c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:06.963904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14462f8-715e-40c8-aca0-059a95377537 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:07.035301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81e876-6264-4b39-b105-9212d3eff258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:07.036366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81e876-6264-4b39-b105-9212d3eff258 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:07.036656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81e876-6264-4b39-b105-9212d3eff258 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:07.036950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f81e876-6264-4b39-b105-9212d3eff258 None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:07.038670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81e876-6264-4b39-b105-9212d3eff258 None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:07.039786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:07.041036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f81e876-6264-4b39-b105-9212d3eff258 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:07.047616 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 64/137] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:06 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:07.049240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81e876-6264-4b39-b105-9212d3eff258 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:07.095563 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 74/138] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:07.139604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f8e73-9c9b-464b-bf60-cf020bf449c4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:07.220155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f81e876-6264-4b39-b105-9212d3eff258 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:07.222158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:07.285477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a22cc4-d08e-4532-ba55-e38d7e82facb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:07.286600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a22cc4-d08e-4532-ba55-e38d7e82facb None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:07.287009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a22cc4-d08e-4532-ba55-e38d7e82facb None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:07.287391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a22cc4-d08e-4532-ba55-e38d7e82facb None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c/users/6edbb2f56e344bc2b039e08db2a25dbc/roles/92b887e453ad4fa9aa5c49f7594a7e53` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:07.290210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f8e73-9c9b-464b-bf60-cf020bf449c4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:07.294685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a22cc4-d08e-4532-ba55-e38d7e82facb None admin] RBAC: Authorizing `identity:create_grant(project_id=e872f1acf4e544f6963d85f6afc13e1c, user_id=6edbb2f56e344bc2b039e08db2a25dbc, role_id=92b887e453ad4fa9aa5c49f7594a7e53)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:07.296325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a22cc4-d08e-4532-ba55-e38d7e82facb None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:07.329786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7962ab-41bd-4792-aa4f-b06fa48f5b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:07.331023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7962ab-41bd-4792-aa4f-b06fa48f5b49 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:07.331469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7962ab-41bd-4792-aa4f-b06fa48f5b49 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:07.331888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7962ab-41bd-4792-aa4f-b06fa48f5b49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:07.333875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7962ab-41bd-4792-aa4f-b06fa48f5b49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:07.335531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7962ab-41bd-4792-aa4f-b06fa48f5b49 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:07.337428 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 75/139] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:07.342725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7962ab-41bd-4792-aa4f-b06fa48f5b49 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:07.389028 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 65/140] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:41:07 2022] PUT /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c/users/6edbb2f56e344bc2b039e08db2a25dbc/roles/92b887e453ad4fa9aa5c49f7594a7e53 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:41:07.413666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:07.474431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7962ab-41bd-4792-aa4f-b06fa48f5b49 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:07.516832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e462ba9-fbd1-4f81-bccb-7ba9fd8515cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:07.517520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e462ba9-fbd1-4f81-bccb-7ba9fd8515cc None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:07.517807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e462ba9-fbd1-4f81-bccb-7ba9fd8515cc None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:07.518113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e462ba9-fbd1-4f81-bccb-7ba9fd8515cc None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:07.518889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e462ba9-fbd1-4f81-bccb-7ba9fd8515cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:07.520956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e462ba9-fbd1-4f81-bccb-7ba9fd8515cc None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:07.698542 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 76/141] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:41:07 2022] POST /identity/v3/users => generated 283 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:41:08.626555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bebfbe-8c7c-4e5e-9efa-3a193f0d7bb5 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:08.627626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bebfbe-8c7c-4e5e-9efa-3a193f0d7bb5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:08.628122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bebfbe-8c7c-4e5e-9efa-3a193f0d7bb5 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:08.630338 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 66/142] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:08.637515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b334dda-7030-4ded-a400-d6eadb96860a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:08.638001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b334dda-7030-4ded-a400-d6eadb96860a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:08.638219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b334dda-7030-4ded-a400-d6eadb96860a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:08.658330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6b334dda-7030-4ded-a400-d6eadb96860a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:08.659540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6b334dda-7030-4ded-a400-d6eadb96860a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:08.733451 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 77/143] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:08.779965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d38b8e-6930-45f8-850b-91b966563a35 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:08.780525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b334dda-7030-4ded-a400-d6eadb96860a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:08.780806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d38b8e-6930-45f8-850b-91b966563a35 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:08.781310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d38b8e-6930-45f8-850b-91b966563a35 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:08.782920 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 67/144] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:08.788328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98351b-7ee5-42c1-9f1d-79918e031094 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:08.788869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98351b-7ee5-42c1-9f1d-79918e031094 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:08.789384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98351b-7ee5-42c1-9f1d-79918e031094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:08.811472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2d98351b-7ee5-42c1-9f1d-79918e031094 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:08.812983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2d98351b-7ee5-42c1-9f1d-79918e031094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:08.831010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:08.957161 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 68/145] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 28 20:41:08.981739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b334dda-7030-4ded-a400-d6eadb96860a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:08.995776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98351b-7ee5-42c1-9f1d-79918e031094 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:09.042580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3fb51-2c06-476b-b023-d9f759276c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:09.045245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3fb51-2c06-476b-b023-d9f759276c09 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:09.045245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3fb51-2c06-476b-b023-d9f759276c09 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:09.045245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3fb51-2c06-476b-b023-d9f759276c09 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:09.062599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3fb51-2c06-476b-b023-d9f759276c09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:09.077680 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-2ac3fb51-2c06-476b-b023-d9f759276c09 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:09.077680 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:09.077680 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:09.077680 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:09.077680 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:09.077680 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:09.077680 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:09.077680 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:09.077680 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:09.077680 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:09.077680 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:09.077680 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:09.077680 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:09.077680 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:09.077680 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:09.077680 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:09.078585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:09.078585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:09.078585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:09.078585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:09.078585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:09.078585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:09.078585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:09.078585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:09.078585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:09.078585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:09.078585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:09.078585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:09.078585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:09.078585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:09.078585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:09.078585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:09.079331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:09.079331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:09.079331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:09.079331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:09.079331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:09.079331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:09.079331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:09.079331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:09.079331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:09.079331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:09.079331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:09.079331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:09.079331 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 78/146] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:41:09.089087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3fb51-2c06-476b-b023-d9f759276c09 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:09.212788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:09.219176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3fb51-2c06-476b-b023-d9f759276c09 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:09.230873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:09.256485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98351b-7ee5-42c1-9f1d-79918e031094 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:09.294254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67445dfc-e514-4c75-b9c2-46e322807bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:09.297004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67445dfc-e514-4c75-b9c2-46e322807bcc None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:09.297004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67445dfc-e514-4c75-b9c2-46e322807bcc None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:09.297004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67445dfc-e514-4c75-b9c2-46e322807bcc None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:09.297004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67445dfc-e514-4c75-b9c2-46e322807bcc None admin] RBAC: Adding query filter params (name=reader) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:09.297004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67445dfc-e514-4c75-b9c2-46e322807bcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:09.298800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67445dfc-e514-4c75-b9c2-46e322807bcc None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:09.313592 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 79/147] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:09.321451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67445dfc-e514-4c75-b9c2-46e322807bcc None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:09.333974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c37b8-097f-41d6-b9eb-d1fcc0f8264e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:09.335171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c37b8-097f-41d6-b9eb-d1fcc0f8264e None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:09.335634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c37b8-097f-41d6-b9eb-d1fcc0f8264e None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:09.336092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c37b8-097f-41d6-b9eb-d1fcc0f8264e None admin] PATH_INFO: `/v3/roles/member` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:09.355732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c37b8-097f-41d6-b9eb-d1fcc0f8264e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:09.368945 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-e43c37b8-097f-41d6-b9eb-d1fcc0f8264e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:09.368945 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:09.368945 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:09.368945 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:09.368945 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:09.368945 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:09.368945 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:09.368945 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:09.368945 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:09.368945 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:09.368945 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:09.368945 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:09.368945 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:09.368945 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:09.368945 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:09.368945 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:09.369859 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:09.369859 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:09.369859 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:09.369859 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:09.369859 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:09.369859 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:09.369859 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:09.369859 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:09.369859 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:09.369859 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:09.369859 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:09.369859 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:09.369859 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:09.369859 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:09.369859 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:09.369859 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:09.370638 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:09.370638 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:09.370638 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:09.370638 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:09.370638 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:09.370638 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:09.370638 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:09.370638 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:09.370638 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:09.370638 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:09.370638 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:09.370638 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:09.373116 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 69/148] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:09.379523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c37b8-097f-41d6-b9eb-d1fcc0f8264e None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:09.577286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c37b8-097f-41d6-b9eb-d1fcc0f8264e None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:09.600086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67445dfc-e514-4c75-b9c2-46e322807bcc None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:09.647322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ab2ce-2f23-4fb1-911b-47d83bbf0e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:09.648721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ab2ce-2f23-4fb1-911b-47d83bbf0e92 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:09.649159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ab2ce-2f23-4fb1-911b-47d83bbf0e92 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:09.649580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ab2ce-2f23-4fb1-911b-47d83bbf0e92 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:09.650514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ab2ce-2f23-4fb1-911b-47d83bbf0e92 None admin] RBAC: Adding query filter params (name=member) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:09.651354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ab2ce-2f23-4fb1-911b-47d83bbf0e92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:09.653132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732ab2ce-2f23-4fb1-911b-47d83bbf0e92 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:09.676358 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 70/149] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:09.682568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ab2ce-2f23-4fb1-911b-47d83bbf0e92 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:09.686552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d70f07-f492-4d53-9958-ae436f72728b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:09.687467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d70f07-f492-4d53-9958-ae436f72728b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:09.687759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d70f07-f492-4d53-9958-ae436f72728b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:09.688042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d70f07-f492-4d53-9958-ae436f72728b None admin] PATH_INFO: `/v3/users/a1e90098a71445c086e97d571c6cf4bc` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:09.693885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d70f07-f492-4d53-9958-ae436f72728b None admin] RBAC: Authorizing `identity:get_user(user_id=a1e90098a71445c086e97d571c6cf4bc)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:09.693885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d70f07-f492-4d53-9958-ae436f72728b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:09.693885 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 80/150] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:09 2022] GET /identity/v3/users/a1e90098a71445c086e97d571c6cf4bc => generated 289 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:09.698830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d70f07-f492-4d53-9958-ae436f72728b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:09.722060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:09.746823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:09.857081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d70f07-f492-4d53-9958-ae436f72728b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:09.903717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ab2ce-2f23-4fb1-911b-47d83bbf0e92 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:09.922580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1f598-13a7-4232-a920-f16b76014972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:09.923479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1f598-13a7-4232-a920-f16b76014972 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:09.923767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1f598-13a7-4232-a920-f16b76014972 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:09.924051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf1f598-13a7-4232-a920-f16b76014972 None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:09.925314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf1f598-13a7-4232-a920-f16b76014972 None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:09.927496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf1f598-13a7-4232-a920-f16b76014972 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:09.929528 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 81/151] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:09 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:09.935111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1f598-13a7-4232-a920-f16b76014972 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:09.959530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e2424-8713-45c7-b895-aebbcee7ed47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:09.960429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e2424-8713-45c7-b895-aebbcee7ed47 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:09.960713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e2424-8713-45c7-b895-aebbcee7ed47 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:09.960992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e2424-8713-45c7-b895-aebbcee7ed47 None admin] PATH_INFO: `/v3/users/6edbb2f56e344bc2b039e08db2a25dbc` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:09.966593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e2424-8713-45c7-b895-aebbcee7ed47 None admin] RBAC: Authorizing `identity:get_user(user_id=6edbb2f56e344bc2b039e08db2a25dbc)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:09.968073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e2424-8713-45c7-b895-aebbcee7ed47 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:09.969744 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 71/152] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:09 2022] GET /identity/v3/users/6edbb2f56e344bc2b039e08db2a25dbc => generated 275 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:09.975671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e2424-8713-45c7-b895-aebbcee7ed47 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:10.087600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf1f598-13a7-4232-a920-f16b76014972 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:10.090127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:10.130847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:10.148173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e97dba-ef73-4c90-8a59-241b0c1abdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:10.149089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e97dba-ef73-4c90-8a59-241b0c1abdb7 None admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:10.149379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e97dba-ef73-4c90-8a59-241b0c1abdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:10.149662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e97dba-ef73-4c90-8a59-241b0c1abdb7 None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c/users/a1e90098a71445c086e97d571c6cf4bc/roles/19a96330fa9044a28cedef74491c0d4c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:10.151535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e2424-8713-45c7-b895-aebbcee7ed47 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:10.155216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e97dba-ef73-4c90-8a59-241b0c1abdb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e872f1acf4e544f6963d85f6afc13e1c, user_id=a1e90098a71445c086e97d571c6cf4bc, role_id=19a96330fa9044a28cedef74491c0d4c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:10.156710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e97dba-ef73-4c90-8a59-241b0c1abdb7 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:10.201060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec88b2e-4b67-47f1-9e68-1027bd5c658c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:10.201715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec88b2e-4b67-47f1-9e68-1027bd5c658c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:10.202026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec88b2e-4b67-47f1-9e68-1027bd5c658c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:10.202311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec88b2e-4b67-47f1-9e68-1027bd5c658c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:10.228641 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 82/153] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:41:09 2022] PUT /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c/users/a1e90098a71445c086e97d571c6cf4bc/roles/19a96330fa9044a28cedef74491c0d4c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:41:10.299654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec88b2e-4b67-47f1-9e68-1027bd5c658c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:10.301152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec88b2e-4b67-47f1-9e68-1027bd5c658c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:10.410548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:10.435045 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 72/154] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 20:41:10.445326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e97dba-ef73-4c90-8a59-241b0c1abdb7 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:10.621818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e97dba-ef73-4c90-8a59-241b0c1abdb7 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:10.668370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e732e-8c45-4f4b-8b15-91fb24b78f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:10.669164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e732e-8c45-4f4b-8b15-91fb24b78f2c None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:10.669457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e732e-8c45-4f4b-8b15-91fb24b78f2c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:10.669740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e732e-8c45-4f4b-8b15-91fb24b78f2c None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:10.670999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e732e-8c45-4f4b-8b15-91fb24b78f2c None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:10.672468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e732e-8c45-4f4b-8b15-91fb24b78f2c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:10.674232 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 83/155] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:10 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:10.680249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec88b2e-4b67-47f1-9e68-1027bd5c658c None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:10.713593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afc02c-1fae-4b78-a3ed-b68c61c1a465 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:10.714112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afc02c-1fae-4b78-a3ed-b68c61c1a465 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:10.714183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9afc02c-1fae-4b78-a3ed-b68c61c1a465 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:10.715571 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 84/156] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:41:10.728713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd16b08-ba9e-4a00-a0d3-c24259ba8a64 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:10.728995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd16b08-ba9e-4a00-a0d3-c24259ba8a64 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:10.729285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd16b08-ba9e-4a00-a0d3-c24259ba8a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:10.762194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3bd16b08-ba9e-4a00-a0d3-c24259ba8a64 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:10.763412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3bd16b08-ba9e-4a00-a0d3-c24259ba8a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:10.794225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:10.825950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec88b2e-4b67-47f1-9e68-1027bd5c658c None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:10.872991 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 85/157] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:41:10.892695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c5beb-1b89-467d-9a4e-b8f57cb25e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:10.893366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c5beb-1b89-467d-9a4e-b8f57cb25e55 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:10.893651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c5beb-1b89-467d-9a4e-b8f57cb25e55 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:10.893955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c5beb-1b89-467d-9a4e-b8f57cb25e55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:10.894800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c5beb-1b89-467d-9a4e-b8f57cb25e55 None admin] RBAC: Adding query filter params (role.id=c0c09ba97c694896ae14fceb6982b1a5, scope.project.id=e872f1acf4e544f6963d85f6afc13e1c, user.id=6edbb2f56e344bc2b039e08db2a25dbc) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:10.895114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c5beb-1b89-467d-9a4e-b8f57cb25e55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:10.897160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602c5beb-1b89-467d-9a4e-b8f57cb25e55 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:10.909457 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 73/158] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:41:10 2022] GET /identity/v3/role_assignments?user.id=6edbb2f56e344bc2b039e08db2a25dbc&scope.project.id=e872f1acf4e544f6963d85f6afc13e1c&role.id=c0c09ba97c694896ae14fceb6982b1a5 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:10.932983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd16b08-ba9e-4a00-a0d3-c24259ba8a64 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:10.988653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705ff2c-8ab0-4063-82d6-61756da99b39 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:10.988944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705ff2c-8ab0-4063-82d6-61756da99b39 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:10.989271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705ff2c-8ab0-4063-82d6-61756da99b39 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:10.991451 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 74/159] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:11.009379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e6d82-4be3-4f67-aed5-6899f06416e6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:11.009709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e6d82-4be3-4f67-aed5-6899f06416e6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:11.009987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e6d82-4be3-4f67-aed5-6899f06416e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:11.047089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-241e6d82-4be3-4f67-aed5-6899f06416e6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:11.048339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-241e6d82-4be3-4f67-aed5-6899f06416e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:11.072621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd16b08-ba9e-4a00-a0d3-c24259ba8a64 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:11.137874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00d62a-8f9e-4b73-b4bf-6b6b61e31273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:11.137874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00d62a-8f9e-4b73-b4bf-6b6b61e31273 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:11.137874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00d62a-8f9e-4b73-b4bf-6b6b61e31273 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:11.137874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd00d62a-8f9e-4b73-b4bf-6b6b61e31273 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:11.140312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:11.167612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd00d62a-8f9e-4b73-b4bf-6b6b61e31273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:11.183380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-cd00d62a-8f9e-4b73-b4bf-6b6b61e31273 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:41:11.183380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:11.183380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:11.183380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:11.183380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:11.183380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:11.183380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:11.183380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:11.183380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:11.183380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:11.183380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:11.183380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:11.183380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:11.183380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:11.183380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:11.183380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:11.184362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:11.184362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:11.184362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:11.184362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:11.184362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:11.184362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:11.184362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:11.184362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:11.184362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:11.184362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:11.184362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:11.184362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:11.184362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:11.184362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:11.184362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:11.184362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:11.185158 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:11.185158 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:11.185158 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:11.185158 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:11.185158 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:11.185158 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:11.185158 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:11.185158 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:11.185158 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:11.185158 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:11.185158 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:41:11.185158 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:11.186497 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 86/160] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:41:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:11.193772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00d62a-8f9e-4b73-b4bf-6b6b61e31273 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:11.225346 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 75/161] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:41:11.234259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:11.258580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e6d82-4be3-4f67-aed5-6899f06416e6 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:11.352723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd00d62a-8f9e-4b73-b4bf-6b6b61e31273 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:11.396128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e6d82-4be3-4f67-aed5-6899f06416e6 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:11.413194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225da314-dd30-4686-839c-ea7aa49524ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:11.414416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225da314-dd30-4686-839c-ea7aa49524ef None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:11.414953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225da314-dd30-4686-839c-ea7aa49524ef None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:11.415427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225da314-dd30-4686-839c-ea7aa49524ef None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:11.416400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225da314-dd30-4686-839c-ea7aa49524ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:11.416902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225da314-dd30-4686-839c-ea7aa49524ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:11.418644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225da314-dd30-4686-839c-ea7aa49524ef None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:11.432805 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 87/162] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:11.438156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225da314-dd30-4686-839c-ea7aa49524ef None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:11.438389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7354b56-8239-414f-8e74-8c76cf24a79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:11.439210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7354b56-8239-414f-8e74-8c76cf24a79a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:11.439493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7354b56-8239-414f-8e74-8c76cf24a79a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:11.439839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7354b56-8239-414f-8e74-8c76cf24a79a None admin] PATH_INFO: `/v3/roles/member` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:11.459559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7354b56-8239-414f-8e74-8c76cf24a79a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:11.471549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-c7354b56-8239-414f-8e74-8c76cf24a79a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:11.471549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:11.471549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:11.471549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:11.471549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:11.471549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:11.471549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:11.471549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:11.471549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:11.471549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:11.471549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:11.471549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:11.471549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:11.471549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:11.471549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:11.471549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:11.472462 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:11.472462 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:11.472462 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:11.472462 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:11.472462 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:11.472462 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:11.472462 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:11.472462 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:11.472462 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:11.472462 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:11.472462 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:11.472462 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:11.472462 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:11.472462 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:11.472462 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:11.472462 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:11.474260 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:11.474260 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:11.474260 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:11.474260 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:11.474260 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:11.474260 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:11.474260 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:11.474260 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:11.474260 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:11.474260 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:11.474260 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:11.474260 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:11.474260 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 76/163] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:11.480417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7354b56-8239-414f-8e74-8c76cf24a79a None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:11.561627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:11.580259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:11.632387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225da314-dd30-4686-839c-ea7aa49524ef None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:11.694258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7354b56-8239-414f-8e74-8c76cf24a79a None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:11.705853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8502de1f-a6c1-4187-a8a0-11f75a05c68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:11.706757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502de1f-a6c1-4187-a8a0-11f75a05c68f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:11.707042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502de1f-a6c1-4187-a8a0-11f75a05c68f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:11.707317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8502de1f-a6c1-4187-a8a0-11f75a05c68f None admin] PATH_INFO: `/v3/users/3e371762e9974f2484669486a37339c7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:11.713693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502de1f-a6c1-4187-a8a0-11f75a05c68f None admin] RBAC: Authorizing `identity:get_user(user_id=3e371762e9974f2484669486a37339c7)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:11.713693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8502de1f-a6c1-4187-a8a0-11f75a05c68f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:11.714326 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 88/164] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:11 2022] GET /identity/v3/users/3e371762e9974f2484669486a37339c7 => generated 247 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:11.714969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa509c-c0e9-420c-942b-d63e8a303210 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:11.715243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa509c-c0e9-420c-942b-d63e8a303210 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:11.715521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa509c-c0e9-420c-942b-d63e8a303210 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:11.716984 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 89/165] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:11.723334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5242885-7e53-4362-931b-9c6af905880f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:11.723666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5242885-7e53-4362-931b-9c6af905880f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:11.723951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5242885-7e53-4362-931b-9c6af905880f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:11.754026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fc965-aae5-4102-bd2d-2619a9eac52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:11.755612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fc965-aae5-4102-bd2d-2619a9eac52f None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:11.756137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fc965-aae5-4102-bd2d-2619a9eac52f None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:11.756613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602fc965-aae5-4102-bd2d-2619a9eac52f None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:11.757571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fc965-aae5-4102-bd2d-2619a9eac52f None admin] RBAC: Adding query filter params (name=member) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:11.758240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fc965-aae5-4102-bd2d-2619a9eac52f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:11.759562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a5242885-7e53-4362-931b-9c6af905880f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:11.760130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602fc965-aae5-4102-bd2d-2619a9eac52f None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:11.760761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a5242885-7e53-4362-931b-9c6af905880f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:11.774413 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 77/166] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:11.775728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fc965-aae5-4102-bd2d-2619a9eac52f None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:11.915524 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 90/167] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:11.916449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5242885-7e53-4362-931b-9c6af905880f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:11.941436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:11.949783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602fc965-aae5-4102-bd2d-2619a9eac52f None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:12.010158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c997df6-789e-4f5d-a90b-09835207c1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:12.010828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c997df6-789e-4f5d-a90b-09835207c1c4 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:12.011117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c997df6-789e-4f5d-a90b-09835207c1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:12.011400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c997df6-789e-4f5d-a90b-09835207c1c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:12.057480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:12.113662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5242885-7e53-4362-931b-9c6af905880f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:12.194885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c997df6-789e-4f5d-a90b-09835207c1c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:12.196381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c997df6-789e-4f5d-a90b-09835207c1c4 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:12.214598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ffba21-d3ee-4ec4-a887-4bb6c2552d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:12.215428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffba21-d3ee-4ec4-a887-4bb6c2552d80 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:12.215719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffba21-d3ee-4ec4-a887-4bb6c2552d80 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:12.216004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ffba21-d3ee-4ec4-a887-4bb6c2552d80 None admin] PATH_INFO: `/v3/users/6edbb2f56e344bc2b039e08db2a25dbc` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:12.217254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ffba21-d3ee-4ec4-a887-4bb6c2552d80 None admin] RBAC: Authorizing `identity:get_user(user_id=6edbb2f56e344bc2b039e08db2a25dbc)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:12.218740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ffba21-d3ee-4ec4-a887-4bb6c2552d80 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:12.220393 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 91/168] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:11 2022] GET /identity/v3/users/6edbb2f56e344bc2b039e08db2a25dbc => generated 275 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:12.221157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ffba21-d3ee-4ec4-a887-4bb6c2552d80 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:12.434035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ffba21-d3ee-4ec4-a887-4bb6c2552d80 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:12.452667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:12.469198 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 78/169] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 694 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:12.470036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c997df6-789e-4f5d-a90b-09835207c1c4 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:12.514133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e91d7-8420-45cc-9ea8-8d2ac3bfb8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:12.515030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e91d7-8420-45cc-9ea8-8d2ac3bfb8be None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:12.515318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e91d7-8420-45cc-9ea8-8d2ac3bfb8be None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:12.515607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e91d7-8420-45cc-9ea8-8d2ac3bfb8be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:12.532010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:12.538056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e91d7-8420-45cc-9ea8-8d2ac3bfb8be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:12.546587 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-a90e91d7-8420-45cc-9ea8-8d2ac3bfb8be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:12.546587 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:12.546587 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:12.546587 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:12.546587 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:12.546587 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:12.546587 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:12.546587 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:12.546587 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:12.546587 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:12.546587 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:12.546587 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:12.546587 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:12.546587 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:12.546587 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:12.546587 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:12.550047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:12.550047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:12.550047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:12.550047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:12.550047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:12.550047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:12.550047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:12.550047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:12.550047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:12.550047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:12.550047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:12.550047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:12.550047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:12.550047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:12.550047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:12.550047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:12.550838 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:12.550838 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:12.550838 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:12.550838 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:12.550838 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:12.550838 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:12.550838 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:12.550838 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:12.550838 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:12.550838 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:12.550838 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:12.550838 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:12.553543 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 92/170] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:41:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:12.554468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e91d7-8420-45cc-9ea8-8d2ac3bfb8be None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:12.643570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c997df6-789e-4f5d-a90b-09835207c1c4 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:12.683242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9332954-a95a-49fb-b36d-aa36fed163a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:12.684273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9332954-a95a-49fb-b36d-aa36fed163a8 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:12.684770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9332954-a95a-49fb-b36d-aa36fed163a8 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:12.685238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9332954-a95a-49fb-b36d-aa36fed163a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:12.686101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e91d7-8420-45cc-9ea8-8d2ac3bfb8be None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:12.746684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e793cc7-ba35-4230-8d91-f7abc8714f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:12.747510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e793cc7-ba35-4230-8d91-f7abc8714f32 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:12.747909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e793cc7-ba35-4230-8d91-f7abc8714f32 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:12.748137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e793cc7-ba35-4230-8d91-f7abc8714f32 None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:12.749362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e793cc7-ba35-4230-8d91-f7abc8714f32 None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:12.750885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e793cc7-ba35-4230-8d91-f7abc8714f32 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:12.752680 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 93/171] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:12 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:12.754164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298b034-fc32-4be8-a6a7-604a66d58cd6 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:12.758061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298b034-fc32-4be8-a6a7-604a66d58cd6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:12.758061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298b034-fc32-4be8-a6a7-604a66d58cd6 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:12.758061 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 94/172] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:12.758061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298b034-fc32-4be8-a6a7-604a66d58cd6 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:12.769221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:12.776861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9332954-a95a-49fb-b36d-aa36fed163a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:12.778610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9332954-a95a-49fb-b36d-aa36fed163a8 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:12.840038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:12.889319 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 79/173] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:12.890658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9332954-a95a-49fb-b36d-aa36fed163a8 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:12.891307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298b034-fc32-4be8-a6a7-604a66d58cd6 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:12.928882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1a680-b988-47cf-9c5e-ec6c375c8c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:12.930267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1a680-b988-47cf-9c5e-ec6c375c8c75 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:12.930267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1a680-b988-47cf-9c5e-ec6c375c8c75 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:12.930267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1a680-b988-47cf-9c5e-ec6c375c8c75 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:12.931004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1a680-b988-47cf-9c5e-ec6c375c8c75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:12.931313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1a680-b988-47cf-9c5e-ec6c375c8c75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:12.932876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1a680-b988-47cf-9c5e-ec6c375c8c75 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:12.942005 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 95/174] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:41:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:12.943395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813113d6-8511-4160-bb1c-f544befa25bc None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:12.943685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813113d6-8511-4160-bb1c-f544befa25bc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:12.948171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813113d6-8511-4160-bb1c-f544befa25bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:12.968375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-813113d6-8511-4160-bb1c-f544befa25bc None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:12.969827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-813113d6-8511-4160-bb1c-f544befa25bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:13.006074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:13.011076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9332954-a95a-49fb-b36d-aa36fed163a8 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:13.050532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e4fd6-942e-41da-914d-162229bf5f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:13.051644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e4fd6-942e-41da-914d-162229bf5f17 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:13.052286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e4fd6-942e-41da-914d-162229bf5f17 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:13.052758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e4fd6-942e-41da-914d-162229bf5f17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:13.053878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e4fd6-942e-41da-914d-162229bf5f17 None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=e872f1acf4e544f6963d85f6afc13e1c, user.id=3e371762e9974f2484669486a37339c7) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:13.054402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e4fd6-942e-41da-914d-162229bf5f17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:13.056118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e4fd6-942e-41da-914d-162229bf5f17 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:13.059858 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 96/175] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:13.061079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813113d6-8511-4160-bb1c-f544befa25bc None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:13.087573 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 80/176] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:41:12 2022] GET /identity/v3/role_assignments?user.id=3e371762e9974f2484669486a37339c7&scope.project.id=e872f1acf4e544f6963d85f6afc13e1c&role.id=7805b93b32f1472f8365936d4d57d49a => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:13.088757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e4fd6-942e-41da-914d-162229bf5f17 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:13.173988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:13.236063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e4fd6-942e-41da-914d-162229bf5f17 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:13.242578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813113d6-8511-4160-bb1c-f544befa25bc None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:13.274949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4dba5f-be5c-4dd8-b14b-24e7209d8157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:13.276477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dba5f-be5c-4dd8-b14b-24e7209d8157 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:13.277033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dba5f-be5c-4dd8-b14b-24e7209d8157 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:13.277550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4dba5f-be5c-4dd8-b14b-24e7209d8157 None admin] PATH_INFO: `/v3/users/6edbb2f56e344bc2b039e08db2a25dbc` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:13.279498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4dba5f-be5c-4dd8-b14b-24e7209d8157 None admin] RBAC: Authorizing `identity:get_user(user_id=6edbb2f56e344bc2b039e08db2a25dbc)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:13.281165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4dba5f-be5c-4dd8-b14b-24e7209d8157 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:13.283172 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 81/177] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:13 2022] GET /identity/v3/users/6edbb2f56e344bc2b039e08db2a25dbc => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:13.284127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4dba5f-be5c-4dd8-b14b-24e7209d8157 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:13.314615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef3ede-5bc3-4181-b0c4-949dc846b05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:13.315493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef3ede-5bc3-4181-b0c4-949dc846b05b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:13.315777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef3ede-5bc3-4181-b0c4-949dc846b05b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:13.316057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef3ede-5bc3-4181-b0c4-949dc846b05b None admin] PATH_INFO: `/v3/projects/b8c5ffd284744fbf8e8eda63ea09cd3e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:13.317427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef3ede-5bc3-4181-b0c4-949dc846b05b None admin] RBAC: Authorizing `identity:get_project(project_id=b8c5ffd284744fbf8e8eda63ea09cd3e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:13.319004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef3ede-5bc3-4181-b0c4-949dc846b05b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:13.321268 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 97/178] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:13 2022] GET /identity/v3/projects/b8c5ffd284744fbf8e8eda63ea09cd3e => generated 312 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:13.322196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef3ede-5bc3-4181-b0c4-949dc846b05b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:13.326219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:13.409102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4dba5f-be5c-4dd8-b14b-24e7209d8157 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:13.490553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ea512-258e-4279-9d91-c13f54729475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:13.490553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ea512-258e-4279-9d91-c13f54729475 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:13.490553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ea512-258e-4279-9d91-c13f54729475 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:13.490553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5ea512-258e-4279-9d91-c13f54729475 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:13.490918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5ea512-258e-4279-9d91-c13f54729475 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:13.490985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef3ede-5bc3-4181-b0c4-949dc846b05b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:13.491065 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-2a5ea512-258e-4279-9d91-c13f54729475 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:41:13.491065 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:13.491065 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:13.491065 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:13.491065 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:13.491065 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:13.491065 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:13.491065 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:13.491065 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:13.491065 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:13.491065 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:13.491065 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:13.491065 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:13.491065 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:13.491065 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:13.491065 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:13.491854 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:13.491854 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:13.491854 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:13.491854 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:13.491854 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:13.491854 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:13.491854 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:13.491854 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:13.491854 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:13.491854 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:13.491854 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:13.491854 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:13.491854 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:13.491854 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:13.491854 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:13.491854 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:13.492770 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:13.492770 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:13.492770 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:13.492770 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:13.492770 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:13.492770 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:13.492770 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:13.492770 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:13.492770 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:13.492770 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:13.492770 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:41:13.492770 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:13.492770 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 82/179] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:41:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:13.492770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ea512-258e-4279-9d91-c13f54729475 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:13.525504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b0a11-9c0c-4f19-a907-6852409d4dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:13.526556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b0a11-9c0c-4f19-a907-6852409d4dcb None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:13.527043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b0a11-9c0c-4f19-a907-6852409d4dcb None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:13.527988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b0a11-9c0c-4f19-a907-6852409d4dcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:13.540354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:13.588802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:13.600788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5ea512-258e-4279-9d91-c13f54729475 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:13.621433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b0a11-9c0c-4f19-a907-6852409d4dcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:13.623011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b0a11-9c0c-4f19-a907-6852409d4dcb None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:13.643774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdcaca-4795-432d-b2f5-398ae053c705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:13.644469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdcaca-4795-432d-b2f5-398ae053c705 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:13.644737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdcaca-4795-432d-b2f5-398ae053c705 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:13.645018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdcaca-4795-432d-b2f5-398ae053c705 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:13.646845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdcaca-4795-432d-b2f5-398ae053c705 None admin] RBAC: Adding query filter params (role.id=c0c09ba97c694896ae14fceb6982b1a5, scope.project.id=b8c5ffd284744fbf8e8eda63ea09cd3e, user.id=6edbb2f56e344bc2b039e08db2a25dbc) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:13.647180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdcaca-4795-432d-b2f5-398ae053c705 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:13.648668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdcaca-4795-432d-b2f5-398ae053c705 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:13.655433 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 83/180] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:41:13 2022] GET /identity/v3/role_assignments?user.id=6edbb2f56e344bc2b039e08db2a25dbc&scope.project.id=b8c5ffd284744fbf8e8eda63ea09cd3e&role.id=c0c09ba97c694896ae14fceb6982b1a5 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:13.659736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b29c3-2ebf-406b-a32d-88b3234b9aeb None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:13.659736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b29c3-2ebf-406b-a32d-88b3234b9aeb None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:13.659736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b29c3-2ebf-406b-a32d-88b3234b9aeb None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:13.662486 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 84/181] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:13.663365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b29c3-2ebf-406b-a32d-88b3234b9aeb None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:13.766985 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 98/182] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:13.771047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7480b-7b1c-40d0-9bcc-58902283bc71 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:13.771345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7480b-7b1c-40d0-9bcc-58902283bc71 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:13.771627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7480b-7b1c-40d0-9bcc-58902283bc71 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:13.778033 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 99/183] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:13.780144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6f1af-feb4-4d4b-826b-873c97f0aa62 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:13.781121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6f1af-feb4-4d4b-826b-873c97f0aa62 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:13.781541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6f1af-feb4-4d4b-826b-873c97f0aa62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:13.803795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b29c3-2ebf-406b-a32d-88b3234b9aeb None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:13.804690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5ba6f1af-feb4-4d4b-826b-873c97f0aa62 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:13.806259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5ba6f1af-feb4-4d4b-826b-873c97f0aa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:13.841981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:13.844841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d24c4e-ae59-4f5e-ac7a-d85de7a931cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:13.845650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d24c4e-ae59-4f5e-ac7a-d85de7a931cc None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:13.846076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d24c4e-ae59-4f5e-ac7a-d85de7a931cc None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:13.846461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d24c4e-ae59-4f5e-ac7a-d85de7a931cc None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:13.847343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d24c4e-ae59-4f5e-ac7a-d85de7a931cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:13.847763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d24c4e-ae59-4f5e-ac7a-d85de7a931cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:13.849348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d24c4e-ae59-4f5e-ac7a-d85de7a931cc None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:13.859804 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 85/184] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:13.861085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d24c4e-ae59-4f5e-ac7a-d85de7a931cc None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:13.885226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:13.910358 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 100/185] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:13.913466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53aef1-10bc-457f-ba51-90fa47277636 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:13.913466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53aef1-10bc-457f-ba51-90fa47277636 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:13.913466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53aef1-10bc-457f-ba51-90fa47277636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:13.932103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-dd53aef1-10bc-457f-ba51-90fa47277636 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:13.933451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dd53aef1-10bc-457f-ba51-90fa47277636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:13.996794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d24c4e-ae59-4f5e-ac7a-d85de7a931cc None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:14.029026 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 101/186] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:14.030279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53aef1-10bc-457f-ba51-90fa47277636 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:14.051137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30400f52-7ea9-4924-8a78-b559f6507963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:14.052336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30400f52-7ea9-4924-8a78-b559f6507963 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:14.052621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30400f52-7ea9-4924-8a78-b559f6507963 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:14.052895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30400f52-7ea9-4924-8a78-b559f6507963 None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:14.054139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30400f52-7ea9-4924-8a78-b559f6507963 None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:14.055726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30400f52-7ea9-4924-8a78-b559f6507963 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:14.062053 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 86/187] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:13 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:14.062053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30400f52-7ea9-4924-8a78-b559f6507963 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:14.142111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:14.147779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:14.154751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53aef1-10bc-457f-ba51-90fa47277636 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:14.168972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30400f52-7ea9-4924-8a78-b559f6507963 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:14.193207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d052625-3e75-486b-a353-1d8c93233b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:14.194128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d052625-3e75-486b-a353-1d8c93233b45 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:14.194420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d052625-3e75-486b-a353-1d8c93233b45 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:14.194695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d052625-3e75-486b-a353-1d8c93233b45 None admin] PATH_INFO: `/v3/users/853965ff331c4b4b9b0e578fbe397541` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:14.204828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c729cf-1e59-4a7c-b96d-ea4e264e6b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:14.205872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c729cf-1e59-4a7c-b96d-ea4e264e6b9b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:14.206353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c729cf-1e59-4a7c-b96d-ea4e264e6b9b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:14.206662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c729cf-1e59-4a7c-b96d-ea4e264e6b9b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:14.208210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d052625-3e75-486b-a353-1d8c93233b45 None admin] RBAC: Authorizing `identity:get_user(user_id=853965ff331c4b4b9b0e578fbe397541)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:14.209694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d052625-3e75-486b-a353-1d8c93233b45 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:14.211567 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 102/188] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:14 2022] GET /identity/v3/users/853965ff331c4b4b9b0e578fbe397541 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:14.212201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d052625-3e75-486b-a353-1d8c93233b45 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:14.218091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c729cf-1e59-4a7c-b96d-ea4e264e6b9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:14.226239 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-17c729cf-1e59-4a7c-b96d-ea4e264e6b9b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:14.226239 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:14.226239 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:14.226239 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:14.226239 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:14.226239 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:14.226239 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:14.226239 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:14.226239 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:14.226239 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:14.226239 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:14.226239 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:14.226239 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:14.226239 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:14.226239 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:14.226239 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:14.227096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:14.227096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:14.227096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:14.227096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:14.227096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:14.227096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:14.227096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:14.227096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:14.227096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:14.227096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:14.227096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:14.227096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:14.227096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:14.227096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:14.227096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:14.227096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:14.227903 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:14.227903 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:14.227903 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:14.227903 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:14.227903 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:14.227903 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:14.227903 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:14.227903 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:14.227903 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:14.227903 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:14.227903 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:14.227903 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:14.227903 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 87/189] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:41:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:14.228574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c729cf-1e59-4a7c-b96d-ea4e264e6b9b None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:14.312353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d052625-3e75-486b-a353-1d8c93233b45 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:14.327252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c729cf-1e59-4a7c-b96d-ea4e264e6b9b None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:14.345917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3380a1-7df8-4b7b-a209-05507900ed7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:14.346717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3380a1-7df8-4b7b-a209-05507900ed7a None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:14.347004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3380a1-7df8-4b7b-a209-05507900ed7a None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:14.347287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3380a1-7df8-4b7b-a209-05507900ed7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:14.348505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3380a1-7df8-4b7b-a209-05507900ed7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:14.350504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3380a1-7df8-4b7b-a209-05507900ed7a None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:14.352171 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 103/190] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:14.353336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ac20c-91d5-4572-8736-456b9ece403c None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:14.353627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ac20c-91d5-4572-8736-456b9ece403c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:14.353937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7ac20c-91d5-4572-8736-456b9ece403c None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:14.358015 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 104/191] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:14.358015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ac20c-91d5-4572-8736-456b9ece403c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:14.363751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca53df-197c-491f-ad8b-ad8dbc80750a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:14.364444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca53df-197c-491f-ad8b-ad8dbc80750a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:14.364825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca53df-197c-491f-ad8b-ad8dbc80750a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:14.365106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca53df-197c-491f-ad8b-ad8dbc80750a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:14.365991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca53df-197c-491f-ad8b-ad8dbc80750a None admin] RBAC: Adding query filter params (role.id=92b887e453ad4fa9aa5c49f7594a7e53, scope.project.id=e872f1acf4e544f6963d85f6afc13e1c, user.id=6edbb2f56e344bc2b039e08db2a25dbc) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:14.366307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca53df-197c-491f-ad8b-ad8dbc80750a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:14.367764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca53df-197c-491f-ad8b-ad8dbc80750a None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:14.371512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:14.373904 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 88/192] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:41:14 2022] GET /identity/v3/role_assignments?user.id=6edbb2f56e344bc2b039e08db2a25dbc&scope.project.id=e872f1acf4e544f6963d85f6afc13e1c&role.id=92b887e453ad4fa9aa5c49f7594a7e53 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:14.374683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca53df-197c-491f-ad8b-ad8dbc80750a None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:14.387926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:14.488527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7ac20c-91d5-4572-8736-456b9ece403c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:14.501810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca53df-197c-491f-ad8b-ad8dbc80750a None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:14.527737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60753ba-3292-4c42-ac55-79b95952c303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:14.528420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60753ba-3292-4c42-ac55-79b95952c303 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:14.528707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60753ba-3292-4c42-ac55-79b95952c303 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:14.528985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60753ba-3292-4c42-ac55-79b95952c303 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:14.547647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a942ded-a9c7-462f-a851-0fa40740a6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:14.548957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a942ded-a9c7-462f-a851-0fa40740a6c7 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:14.549883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a942ded-a9c7-462f-a851-0fa40740a6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:14.550204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a942ded-a9c7-462f-a851-0fa40740a6c7 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:14.551112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a942ded-a9c7-462f-a851-0fa40740a6c7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:14.551422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a942ded-a9c7-462f-a851-0fa40740a6c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:14.553457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a942ded-a9c7-462f-a851-0fa40740a6c7 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:14.566016 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 89/193] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:41:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:14.567131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a942ded-a9c7-462f-a851-0fa40740a6c7 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:14.609308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60753ba-3292-4c42-ac55-79b95952c303 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:14.611142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60753ba-3292-4c42-ac55-79b95952c303 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:14.626484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:14.652007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:14.681745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a942ded-a9c7-462f-a851-0fa40740a6c7 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:14.712761 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 105/194] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:14.712761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a876e-c557-4d8a-9c08-10b868347365 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:14.712761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a876e-c557-4d8a-9c08-10b868347365 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:14.712761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0a876e-c557-4d8a-9c08-10b868347365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:14.717528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afc27e-0f17-4b75-9c78-2976032db8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:14.717528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afc27e-0f17-4b75-9c78-2976032db8e8 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:14.717528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afc27e-0f17-4b75-9c78-2976032db8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:14.717528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5afc27e-0f17-4b75-9c78-2976032db8e8 None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:14.717778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5afc27e-0f17-4b75-9c78-2976032db8e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:14.719081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5afc27e-0f17-4b75-9c78-2976032db8e8 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:14.730917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8f0a876e-c557-4d8a-9c08-10b868347365 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:14.732100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8f0a876e-c557-4d8a-9c08-10b868347365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:14.795941 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 106/195] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:14.796741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a876e-c557-4d8a-9c08-10b868347365 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:14.840917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:14.862084 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 90/196] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:41:14 2022] POST /identity/v3/users => generated 297 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:41:14.865438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afc27e-0f17-4b75-9c78-2976032db8e8 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:14.898558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0a876e-c557-4d8a-9c08-10b868347365 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:14.939934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26850ccc-2120-4a87-9789-2075224650f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:14.940841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26850ccc-2120-4a87-9789-2075224650f9 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:14.941133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26850ccc-2120-4a87-9789-2075224650f9 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:14.941418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26850ccc-2120-4a87-9789-2075224650f9 None admin] PATH_INFO: `/v3/users/853965ff331c4b4b9b0e578fbe397541` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:14.943490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26850ccc-2120-4a87-9789-2075224650f9 None admin] RBAC: Authorizing `identity:get_user(user_id=853965ff331c4b4b9b0e578fbe397541)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:14.944990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26850ccc-2120-4a87-9789-2075224650f9 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:14.946929 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 107/197] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:14 2022] GET /identity/v3/users/853965ff331c4b4b9b0e578fbe397541 => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:14.951554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26850ccc-2120-4a87-9789-2075224650f9 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:14.993083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5afc27e-0f17-4b75-9c78-2976032db8e8 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:15.023220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:15.032953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c83ccf-8d63-4e9b-9fd4-ced253530f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:15.033847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c83ccf-8d63-4e9b-9fd4-ced253530f5b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:15.035685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c83ccf-8d63-4e9b-9fd4-ced253530f5b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:15.035685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c83ccf-8d63-4e9b-9fd4-ced253530f5b None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:15.035685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c83ccf-8d63-4e9b-9fd4-ced253530f5b None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:15.037170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c83ccf-8d63-4e9b-9fd4-ced253530f5b None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:15.039219 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 91/198] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:14 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:15.044723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c83ccf-8d63-4e9b-9fd4-ced253530f5b None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:15.091479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26850ccc-2120-4a87-9789-2075224650f9 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:15.110089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:15.139708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72afe9-eb53-489a-b342-ae3152efb1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:15.140521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72afe9-eb53-489a-b342-ae3152efb1f5 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:15.141057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72afe9-eb53-489a-b342-ae3152efb1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:15.141345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72afe9-eb53-489a-b342-ae3152efb1f5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:15.154433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72afe9-eb53-489a-b342-ae3152efb1f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:15.162288 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-7c72afe9-eb53-489a-b342-ae3152efb1f5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:15.162288 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:15.162288 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:15.162288 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:15.162288 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:15.162288 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:15.162288 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:15.162288 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:15.162288 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:15.162288 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:15.162288 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:15.162288 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:15.162288 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:15.162288 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:15.162288 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:15.162288 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:15.163201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:15.163201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:15.163201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:15.163201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:15.163201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:15.163201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:15.163201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:15.163201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:15.163201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:15.163201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:15.163201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:15.163201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:15.163201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:15.163201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:15.163201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:15.163201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:15.163976 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:15.163976 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:15.163976 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:15.163976 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:15.163976 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:15.163976 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:15.163976 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:15.163976 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:15.163976 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:15.163976 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:15.163976 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:15.163976 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:15.163976 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 108/199] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:15.164613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72afe9-eb53-489a-b342-ae3152efb1f5 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:15.179536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c83ccf-8d63-4e9b-9fd4-ced253530f5b None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:15.221680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78cc44-63b7-47f2-9a35-9e39ed565f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:15.222391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78cc44-63b7-47f2-9a35-9e39ed565f28 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:15.222677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78cc44-63b7-47f2-9a35-9e39ed565f28 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:15.223169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78cc44-63b7-47f2-9a35-9e39ed565f28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:15.292402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72afe9-eb53-489a-b342-ae3152efb1f5 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:15.317816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78cc44-63b7-47f2-9a35-9e39ed565f28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:15.319326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78cc44-63b7-47f2-9a35-9e39ed565f28 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:15.328520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:15.332465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fb344-b749-4fae-83c6-e71e90b1b4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:15.333164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fb344-b749-4fae-83c6-e71e90b1b4ea None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:15.333455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fb344-b749-4fae-83c6-e71e90b1b4ea None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:15.333741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fb344-b749-4fae-83c6-e71e90b1b4ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:15.334818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9fb344-b749-4fae-83c6-e71e90b1b4ea None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=0381e5a4ddde4d9ab4394d1417743579, user.id=853965ff331c4b4b9b0e578fbe397541) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:15.335139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9fb344-b749-4fae-83c6-e71e90b1b4ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:15.336731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9fb344-b749-4fae-83c6-e71e90b1b4ea None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:15.344823 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 109/200] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:41:15 2022] GET /identity/v3/role_assignments?user.id=853965ff331c4b4b9b0e578fbe397541&scope.project.id=0381e5a4ddde4d9ab4394d1417743579&role.id=7805b93b32f1472f8365936d4d57d49a => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:15.344823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fb344-b749-4fae-83c6-e71e90b1b4ea None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:15.386302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:15.433288 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 92/201] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:15.438960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78cc44-63b7-47f2-9a35-9e39ed565f28 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:15.544401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:15.547125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fb344-b749-4fae-83c6-e71e90b1b4ea None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:15.552189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78cc44-63b7-47f2-9a35-9e39ed565f28 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:15.595640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02224635-1058-403a-9584-491e57659539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:15.596351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02224635-1058-403a-9584-491e57659539 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:15.597181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02224635-1058-403a-9584-491e57659539 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:15.597558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02224635-1058-403a-9584-491e57659539 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:15.599119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02224635-1058-403a-9584-491e57659539 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:15.599503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02224635-1058-403a-9584-491e57659539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:15.599807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3c756-efc6-4b09-bb81-8164ef3372dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:15.600862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3c756-efc6-4b09-bb81-8164ef3372dc None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:15.601137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02224635-1058-403a-9584-491e57659539 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:15.601491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3c756-efc6-4b09-bb81-8164ef3372dc None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:15.601960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3c756-efc6-4b09-bb81-8164ef3372dc None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:15.603368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3c756-efc6-4b09-bb81-8164ef3372dc None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:15.604998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3c756-efc6-4b09-bb81-8164ef3372dc None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:15.613863 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 93/202] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:15 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:15.614086 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 110/203] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:15.614086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02224635-1058-403a-9584-491e57659539 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:15.616912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3c756-efc6-4b09-bb81-8164ef3372dc None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:15.725253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:15.729548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02224635-1058-403a-9584-491e57659539 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:15.768722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c7c52-5518-4da7-9ea2-993ea0ddcc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:15.769936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c7c52-5518-4da7-9ea2-993ea0ddcc37 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:15.770966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c7c52-5518-4da7-9ea2-993ea0ddcc37 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:15.771802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c7c52-5518-4da7-9ea2-993ea0ddcc37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:15.773156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c7c52-5518-4da7-9ea2-993ea0ddcc37 None admin] RBAC: Adding query filter params (role.id=92b887e453ad4fa9aa5c49f7594a7e53, scope.project.id=0381e5a4ddde4d9ab4394d1417743579, user.id=853965ff331c4b4b9b0e578fbe397541) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:15.773676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c7c52-5518-4da7-9ea2-993ea0ddcc37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:15.775604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0c7c52-5518-4da7-9ea2-993ea0ddcc37 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:15.799877 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 111/204] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:41:15 2022] GET /identity/v3/role_assignments?user.id=853965ff331c4b4b9b0e578fbe397541&scope.project.id=0381e5a4ddde4d9ab4394d1417743579&role.id=92b887e453ad4fa9aa5c49f7594a7e53 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:15.858176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3c756-efc6-4b09-bb81-8164ef3372dc None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:15.923952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c997075-dfc8-442f-b1f1-89603839524d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:15.924848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c997075-dfc8-442f-b1f1-89603839524d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:15.925129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c997075-dfc8-442f-b1f1-89603839524d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:15.925404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c997075-dfc8-442f-b1f1-89603839524d None admin] PATH_INFO: `/v3/users/a1e90098a71445c086e97d571c6cf4bc` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:15.926701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c997075-dfc8-442f-b1f1-89603839524d None admin] RBAC: Authorizing `identity:get_user(user_id=a1e90098a71445c086e97d571c6cf4bc)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:15.928185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c997075-dfc8-442f-b1f1-89603839524d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:15.929850 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 94/205] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:15 2022] GET /identity/v3/users/a1e90098a71445c086e97d571c6cf4bc => generated 289 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:15.935667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c7c52-5518-4da7-9ea2-993ea0ddcc37 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:16.097118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c7c52-5518-4da7-9ea2-993ea0ddcc37 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:16.137420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039d39e-bfed-494d-9c97-0f3003983cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:16.138040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039d39e-bfed-494d-9c97-0f3003983cc7 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:16.138331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039d39e-bfed-494d-9c97-0f3003983cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:16.138612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039d39e-bfed-494d-9c97-0f3003983cc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:16.174890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:16.215507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039d39e-bfed-494d-9c97-0f3003983cc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:16.217003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039d39e-bfed-494d-9c97-0f3003983cc7 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:16.334322 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 112/206] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:16.339845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c997075-dfc8-442f-b1f1-89603839524d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:16.391621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:16.486428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c997075-dfc8-442f-b1f1-89603839524d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:16.611799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a44b69-72e3-4ce3-8c99-c33621fbcee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:16.613038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a44b69-72e3-4ce3-8c99-c33621fbcee2 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:16.613516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a44b69-72e3-4ce3-8c99-c33621fbcee2 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:16.614633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a44b69-72e3-4ce3-8c99-c33621fbcee2 None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:16.616043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a44b69-72e3-4ce3-8c99-c33621fbcee2 None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:16.617685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a44b69-72e3-4ce3-8c99-c33621fbcee2 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:16.619610 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 95/207] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:16 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:16.625545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039d39e-bfed-494d-9c97-0f3003983cc7 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:16.709517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:16.752013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039d39e-bfed-494d-9c97-0f3003983cc7 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:16.789218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87879e49-d116-4b55-94a8-dfa3bebf0118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:16.790254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87879e49-d116-4b55-94a8-dfa3bebf0118 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:16.790734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87879e49-d116-4b55-94a8-dfa3bebf0118 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:16.791202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87879e49-d116-4b55-94a8-dfa3bebf0118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:16.792249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87879e49-d116-4b55-94a8-dfa3bebf0118 None admin] RBAC: Adding query filter params (role.id=19a96330fa9044a28cedef74491c0d4c, scope.project.id=e872f1acf4e544f6963d85f6afc13e1c, user.id=a1e90098a71445c086e97d571c6cf4bc) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:16.792752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87879e49-d116-4b55-94a8-dfa3bebf0118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:16.794420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87879e49-d116-4b55-94a8-dfa3bebf0118 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:16.800710 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 113/208] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:41:16 2022] GET /identity/v3/role_assignments?user.id=a1e90098a71445c086e97d571c6cf4bc&scope.project.id=e872f1acf4e544f6963d85f6afc13e1c&role.id=19a96330fa9044a28cedef74491c0d4c => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:17.653688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fee39-5fc2-445a-aa14-f92a6f28872b None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:17.654719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fee39-5fc2-445a-aa14-f92a6f28872b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:17.655203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16fee39-5fc2-445a-aa14-f92a6f28872b None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:17.656785 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 96/209] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:17.663627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975458fe-bc4c-4315-ae9a-cc22b33865e0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:17.663627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975458fe-bc4c-4315-ae9a-cc22b33865e0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:17.663627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975458fe-bc4c-4315-ae9a-cc22b33865e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:17.682008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-975458fe-bc4c-4315-ae9a-cc22b33865e0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:17.687471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-975458fe-bc4c-4315-ae9a-cc22b33865e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:17.757960 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 114/210] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:41:17.768374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74d125-4b93-4bb0-87ca-7f12808d37c4 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:17.768919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74d125-4b93-4bb0-87ca-7f12808d37c4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:17.769398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab74d125-4b93-4bb0-87ca-7f12808d37c4 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:17.771700 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 97/211] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:17.777305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73fbbf-2812-4509-990a-16e228d13766 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:17.777912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73fbbf-2812-4509-990a-16e228d13766 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:17.778395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73fbbf-2812-4509-990a-16e228d13766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:17.780003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74d125-4b93-4bb0-87ca-7f12808d37c4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:17.798211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7f73fbbf-2812-4509-990a-16e228d13766 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:17.799400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7f73fbbf-2812-4509-990a-16e228d13766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:17.855477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:17.874372 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 115/212] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:17.903114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:17.905639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73fbbf-2812-4509-990a-16e228d13766 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:17.908893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab74d125-4b93-4bb0-87ca-7f12808d37c4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:17.959937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd3b35-0e87-4bf3-a1ca-90d419ef99bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:17.961714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd3b35-0e87-4bf3-a1ca-90d419ef99bb None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:17.962181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd3b35-0e87-4bf3-a1ca-90d419ef99bb None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:17.962600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd3b35-0e87-4bf3-a1ca-90d419ef99bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:17.976564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd3b35-0e87-4bf3-a1ca-90d419ef99bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:17.987798 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-bdfd3b35-0e87-4bf3-a1ca-90d419ef99bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:17.987798 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:17.987798 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:17.987798 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:17.987798 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:17.987798 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:17.987798 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:17.987798 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:17.987798 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:17.987798 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:17.987798 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:17.987798 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:17.987798 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:17.987798 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:17.987798 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:17.987798 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:17.988881 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:17.988881 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:17.988881 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:17.988881 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:17.988881 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:17.988881 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:17.988881 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:17.988881 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:17.988881 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:17.988881 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:17.988881 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:17.988881 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:17.988881 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:17.988881 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:17.988881 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:17.988881 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:17.990371 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:17.990371 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:17.990371 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:17.990371 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:17.990371 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:17.990371 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:17.990371 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:17.990371 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:17.990371 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:17.990371 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:17.990371 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:17.990371 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:17.992791 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 98/213] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:17.999091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd3b35-0e87-4bf3-a1ca-90d419ef99bb None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:18.024546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73fbbf-2812-4509-990a-16e228d13766 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:18.063555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6ca35-485b-40cf-868a-b8021f277bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:18.064197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6ca35-485b-40cf-868a-b8021f277bd5 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:18.064712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6ca35-485b-40cf-868a-b8021f277bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:18.065202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6ca35-485b-40cf-868a-b8021f277bd5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:18.066220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c6ca35-485b-40cf-868a-b8021f277bd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:18.069350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c6ca35-485b-40cf-868a-b8021f277bd5 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:18.073793 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 116/214] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:18.073793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180a958-22a8-40e7-81fb-e61687fbdc79 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:18.073793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180a958-22a8-40e7-81fb-e61687fbdc79 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:18.073793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f180a958-22a8-40e7-81fb-e61687fbdc79 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:18.073793 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 117/215] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:18.077362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180a958-22a8-40e7-81fb-e61687fbdc79 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:18.130671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd3b35-0e87-4bf3-a1ca-90d419ef99bb None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:18.136699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:18.168062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f1152-5e73-4cd6-9cb0-c8ba24ab8430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:18.168961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f1152-5e73-4cd6-9cb0-c8ba24ab8430 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:18.169420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f1152-5e73-4cd6-9cb0-c8ba24ab8430 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:18.169701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f1152-5e73-4cd6-9cb0-c8ba24ab8430 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:18.170491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f1152-5e73-4cd6-9cb0-c8ba24ab8430 None admin] RBAC: Adding query filter params (name=member) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:18.170799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f1152-5e73-4cd6-9cb0-c8ba24ab8430 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:18.172194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f1152-5e73-4cd6-9cb0-c8ba24ab8430 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:18.180507 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 99/216] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:18.183106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1446e7-38d3-4682-b72f-ad8de9b9255e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:18.183393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1446e7-38d3-4682-b72f-ad8de9b9255e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:18.183669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1446e7-38d3-4682-b72f-ad8de9b9255e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:18.187543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f180a958-22a8-40e7-81fb-e61687fbdc79 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:18.201281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4c1446e7-38d3-4682-b72f-ad8de9b9255e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:18.202517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4c1446e7-38d3-4682-b72f-ad8de9b9255e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:18.217078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:18.225522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3ff4d-2859-4497-88ba-6550ba65dff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:18.226228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3ff4d-2859-4497-88ba-6550ba65dff7 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:18.226505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3ff4d-2859-4497-88ba-6550ba65dff7 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:18.226781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3ff4d-2859-4497-88ba-6550ba65dff7 None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:18.227490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3ff4d-2859-4497-88ba-6550ba65dff7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:18.228956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3ff4d-2859-4497-88ba-6550ba65dff7 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:18.271939 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 100/217] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:18.278160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1446e7-38d3-4682-b72f-ad8de9b9255e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:18.321624 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 118/218] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:41:18 2022] POST /identity/v3/users => generated 297 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:41:18.322541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3ff4d-2859-4497-88ba-6550ba65dff7 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:18.421383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1446e7-38d3-4682-b72f-ad8de9b9255e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:18.445678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3ff4d-2859-4497-88ba-6550ba65dff7 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:18.457342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:18.463522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f44f3-226c-40bc-ba14-2656c06366eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:18.464799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f44f3-226c-40bc-ba14-2656c06366eb None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:18.465297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f44f3-226c-40bc-ba14-2656c06366eb None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:18.465808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f44f3-226c-40bc-ba14-2656c06366eb None admin] PATH_INFO: `/v3/users/fd9c4adcec884ec6accb328898e94e1e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:18.482078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f44f3-226c-40bc-ba14-2656c06366eb None admin] RBAC: Authorizing `identity:get_user(user_id=fd9c4adcec884ec6accb328898e94e1e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:18.484277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f44f3-226c-40bc-ba14-2656c06366eb None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:18.484541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca729d6-c975-438d-a2a7-25a2e85066a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:18.485269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca729d6-c975-438d-a2a7-25a2e85066a5 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:18.485559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca729d6-c975-438d-a2a7-25a2e85066a5 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:18.485859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca729d6-c975-438d-a2a7-25a2e85066a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:18.491833 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 101/219] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:18 2022] GET /identity/v3/users/fd9c4adcec884ec6accb328898e94e1e => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:18.494455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f44f3-226c-40bc-ba14-2656c06366eb None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:18.499313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca729d6-c975-438d-a2a7-25a2e85066a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:18.508298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-4ca729d6-c975-438d-a2a7-25a2e85066a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:41:18.508298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:18.508298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:18.508298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:18.508298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:18.508298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:18.508298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:18.508298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:18.508298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:18.508298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:18.508298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:18.508298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:18.508298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:18.508298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:18.508298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:18.508298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:18.512463 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:18.512463 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:18.512463 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:18.512463 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:18.512463 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:18.512463 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:18.512463 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:18.512463 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:18.512463 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:18.512463 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:18.512463 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:18.512463 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:18.512463 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:18.512463 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:18.512463 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:18.512463 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:18.513249 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:18.513249 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:18.513249 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:18.513249 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:18.513249 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:18.513249 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:18.513249 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:18.513249 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:18.513249 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:18.513249 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:18.513249 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:41:18.513249 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:18.513895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:18.516116 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 119/220] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:41:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:18.516895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca729d6-c975-438d-a2a7-25a2e85066a5 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:18.633696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f44f3-226c-40bc-ba14-2656c06366eb None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:18.643861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca729d6-c975-438d-a2a7-25a2e85066a5 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:18.671232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e636e377-58a9-426e-ba0a-5f54f7a1476c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:18.671936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636e377-58a9-426e-ba0a-5f54f7a1476c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:18.672177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636e377-58a9-426e-ba0a-5f54f7a1476c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:18.672446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e636e377-58a9-426e-ba0a-5f54f7a1476c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:18.687244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c84f5-4721-46bc-b9a9-b45426567707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:18.688275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c84f5-4721-46bc-b9a9-b45426567707 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:18.688692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c84f5-4721-46bc-b9a9-b45426567707 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:18.688981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c84f5-4721-46bc-b9a9-b45426567707 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:18.690151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c84f5-4721-46bc-b9a9-b45426567707 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:18.690467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c84f5-4721-46bc-b9a9-b45426567707 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:18.691858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95c84f5-4721-46bc-b9a9-b45426567707 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:18.704900 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 120/221] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:18.706303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd31f5-8f7f-4324-8e6f-5a90f40b9ba8 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:18.706303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd31f5-8f7f-4324-8e6f-5a90f40b9ba8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:18.706624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dd31f5-8f7f-4324-8e6f-5a90f40b9ba8 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:18.708191 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 121/222] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:18.712939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd31f5-8f7f-4324-8e6f-5a90f40b9ba8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:18.748106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:18.766386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636e377-58a9-426e-ba0a-5f54f7a1476c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:18.768203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e636e377-58a9-426e-ba0a-5f54f7a1476c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:18.781395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:18.835713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dd31f5-8f7f-4324-8e6f-5a90f40b9ba8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:18.880796 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 102/223] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 20:41:18.882255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f136515-5236-459e-99c8-069cee784c35 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:18.882639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f136515-5236-459e-99c8-069cee784c35 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:18.883013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f136515-5236-459e-99c8-069cee784c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:18.887932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e993b0b-d4f8-4d8c-8e6d-ff97057a6271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:18.889145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e993b0b-d4f8-4d8c-8e6d-ff97057a6271 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:18.890384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e993b0b-d4f8-4d8c-8e6d-ff97057a6271 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:18.891044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e993b0b-d4f8-4d8c-8e6d-ff97057a6271 None admin] PATH_INFO: `/v3/users/853965ff331c4b4b9b0e578fbe397541` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:18.893019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e993b0b-d4f8-4d8c-8e6d-ff97057a6271 None admin] RBAC: Authorizing `identity:get_user(user_id=853965ff331c4b4b9b0e578fbe397541)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:18.895085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e993b0b-d4f8-4d8c-8e6d-ff97057a6271 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:18.897175 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 122/224] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:18 2022] GET /identity/v3/users/853965ff331c4b4b9b0e578fbe397541 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:18.897985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e993b0b-d4f8-4d8c-8e6d-ff97057a6271 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:18.910004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0f136515-5236-459e-99c8-069cee784c35 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:18.911329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0f136515-5236-459e-99c8-069cee784c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:19.003866 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 103/225] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:19.005064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f136515-5236-459e-99c8-069cee784c35 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:19.022333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:19.025392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e993b0b-d4f8-4d8c-8e6d-ff97057a6271 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:19.059353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:19.078311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050d795-6c66-40be-aa5f-3548b4849f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:19.079448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050d795-6c66-40be-aa5f-3548b4849f52 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:19.079890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050d795-6c66-40be-aa5f-3548b4849f52 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:19.080170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0050d795-6c66-40be-aa5f-3548b4849f52 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:19.081976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050d795-6c66-40be-aa5f-3548b4849f52 None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:19.084304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0050d795-6c66-40be-aa5f-3548b4849f52 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:19.086069 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 123/226] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:18 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:19.086833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050d795-6c66-40be-aa5f-3548b4849f52 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:19.126254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f136515-5236-459e-99c8-069cee784c35 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:19.162541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c861a9d-8e2e-4509-ab1c-f9a586580c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:19.163494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c861a9d-8e2e-4509-ab1c-f9a586580c80 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:19.163818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c861a9d-8e2e-4509-ab1c-f9a586580c80 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:19.164104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c861a9d-8e2e-4509-ab1c-f9a586580c80 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:19.165487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c861a9d-8e2e-4509-ab1c-f9a586580c80 None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:19.166995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c861a9d-8e2e-4509-ab1c-f9a586580c80 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:19.168796 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 104/227] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:19 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:19.169639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c861a9d-8e2e-4509-ab1c-f9a586580c80 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:19.214526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0050d795-6c66-40be-aa5f-3548b4849f52 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:19.251241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be25ed6-2a3a-4b6c-8381-be9f84029f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:19.252351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be25ed6-2a3a-4b6c-8381-be9f84029f3a None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:19.252881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be25ed6-2a3a-4b6c-8381-be9f84029f3a None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:19.253171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be25ed6-2a3a-4b6c-8381-be9f84029f3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:19.264215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be25ed6-2a3a-4b6c-8381-be9f84029f3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:19.272736 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-0be25ed6-2a3a-4b6c-8381-be9f84029f3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:19.272736 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:19.272736 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:19.272736 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:19.272736 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:19.272736 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:19.272736 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:19.272736 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:19.272736 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:19.272736 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:19.272736 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:19.272736 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:19.272736 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:19.272736 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:19.272736 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:19.272736 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:19.273612 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:19.273612 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:19.273612 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:19.273612 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:19.273612 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:19.273612 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:19.273612 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:19.273612 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:19.273612 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:19.273612 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:19.273612 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:19.273612 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:19.273612 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:19.273612 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:19.273612 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:19.273612 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:19.274439 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:19.274439 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:19.274439 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:19.274439 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:19.274439 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:19.274439 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:19.274439 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:19.274439 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:19.274439 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:19.274439 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:19.274439 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:19.274439 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:19.274439 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 124/228] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:41:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:19.275560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be25ed6-2a3a-4b6c-8381-be9f84029f3a None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:19.284320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c861a9d-8e2e-4509-ab1c-f9a586580c80 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:19.315025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:19.318236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:19.322742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68cc55-e6fb-46ce-8667-be6106eb9553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:19.325953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68cc55-e6fb-46ce-8667-be6106eb9553 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:19.325953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68cc55-e6fb-46ce-8667-be6106eb9553 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:19.325953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca68cc55-e6fb-46ce-8667-be6106eb9553 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:19.325953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68cc55-e6fb-46ce-8667-be6106eb9553 None admin] RBAC: Adding query filter params (role.id=c0c09ba97c694896ae14fceb6982b1a5, scope.project.id=0381e5a4ddde4d9ab4394d1417743579, user.id=fd9c4adcec884ec6accb328898e94e1e) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:19.325953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68cc55-e6fb-46ce-8667-be6106eb9553 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:19.327220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca68cc55-e6fb-46ce-8667-be6106eb9553 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:19.333969 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 105/229] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:41:19 2022] GET /identity/v3/role_assignments?user.id=fd9c4adcec884ec6accb328898e94e1e&scope.project.id=0381e5a4ddde4d9ab4394d1417743579&role.id=c0c09ba97c694896ae14fceb6982b1a5 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:19.347062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68cc55-e6fb-46ce-8667-be6106eb9553 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:19.403587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be25ed6-2a3a-4b6c-8381-be9f84029f3a None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:19.452344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8cd21d-fe25-4fbf-9ab0-47aaaf9a18e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:19.453416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cd21d-fe25-4fbf-9ab0-47aaaf9a18e5 None admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:19.453698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cd21d-fe25-4fbf-9ab0-47aaaf9a18e5 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:19.453998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cd21d-fe25-4fbf-9ab0-47aaaf9a18e5 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579/users/853965ff331c4b4b9b0e578fbe397541/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:19.456247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8cd21d-fe25-4fbf-9ab0-47aaaf9a18e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=0381e5a4ddde4d9ab4394d1417743579, user_id=853965ff331c4b4b9b0e578fbe397541, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:19.459030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8cd21d-fe25-4fbf-9ab0-47aaaf9a18e5 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:19.521884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca68cc55-e6fb-46ce-8667-be6106eb9553 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:19.560114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5291e-8c1a-49f8-99f5-62756c4256bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:19.560794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5291e-8c1a-49f8-99f5-62756c4256bc None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:19.561085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5291e-8c1a-49f8-99f5-62756c4256bc None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:19.563140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5291e-8c1a-49f8-99f5-62756c4256bc None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:19.563140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5291e-8c1a-49f8-99f5-62756c4256bc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:19.563140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5291e-8c1a-49f8-99f5-62756c4256bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:19.564034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5291e-8c1a-49f8-99f5-62756c4256bc None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:19.573209 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 106/230] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:41:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:19.580052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5291e-8c1a-49f8-99f5-62756c4256bc None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:19.594936 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 125/231] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:41:19 2022] PUT /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579/users/853965ff331c4b4b9b0e578fbe397541/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:41:19.619687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:19.699243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5291e-8c1a-49f8-99f5-62756c4256bc None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:19.736834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68a00a-7374-405e-8e12-e0bba3c5f9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:19.737787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68a00a-7374-405e-8e12-e0bba3c5f9f5 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:19.738114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68a00a-7374-405e-8e12-e0bba3c5f9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:19.738393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db68a00a-7374-405e-8e12-e0bba3c5f9f5 None admin] PATH_INFO: `/v3/users/853965ff331c4b4b9b0e578fbe397541` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:19.740239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68a00a-7374-405e-8e12-e0bba3c5f9f5 None admin] RBAC: Authorizing `identity:get_user(user_id=853965ff331c4b4b9b0e578fbe397541)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:19.741726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db68a00a-7374-405e-8e12-e0bba3c5f9f5 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:19.743660 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 107/232] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:19 2022] GET /identity/v3/users/853965ff331c4b4b9b0e578fbe397541 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:19.749681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8cd21d-fe25-4fbf-9ab0-47aaaf9a18e5 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:19.866760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:19.879280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8cd21d-fe25-4fbf-9ab0-47aaaf9a18e5 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:19.916076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28331fa2-bdfc-4222-bf45-add80915e255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:19.917575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28331fa2-bdfc-4222-bf45-add80915e255 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:19.917886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28331fa2-bdfc-4222-bf45-add80915e255 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:19.918175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28331fa2-bdfc-4222-bf45-add80915e255 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:19.919386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28331fa2-bdfc-4222-bf45-add80915e255 None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:19.920855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28331fa2-bdfc-4222-bf45-add80915e255 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:19.922915 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 126/233] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:19 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:19.929165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68a00a-7374-405e-8e12-e0bba3c5f9f5 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:20.043616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:20.047391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db68a00a-7374-405e-8e12-e0bba3c5f9f5 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:20.113069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e45b3a-d2c6-4982-a131-b9a5ae32eaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:20.114868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e45b3a-d2c6-4982-a131-b9a5ae32eaab None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:20.115308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e45b3a-d2c6-4982-a131-b9a5ae32eaab None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:20.115738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e45b3a-d2c6-4982-a131-b9a5ae32eaab None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579/users/853965ff331c4b4b9b0e578fbe397541/roles/92b887e453ad4fa9aa5c49f7594a7e53` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:20.123713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e45b3a-d2c6-4982-a131-b9a5ae32eaab None admin] RBAC: Authorizing `identity:create_grant(project_id=0381e5a4ddde4d9ab4394d1417743579, user_id=853965ff331c4b4b9b0e578fbe397541, role_id=92b887e453ad4fa9aa5c49f7594a7e53)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:20.125350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e45b3a-d2c6-4982-a131-b9a5ae32eaab None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:20.248607 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 108/234] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:41:19 2022] PUT /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579/users/853965ff331c4b4b9b0e578fbe397541/roles/92b887e453ad4fa9aa5c49f7594a7e53 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:41:20.998120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b74272-692f-468a-8376-fef0343e5d94 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:20.999029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b74272-692f-468a-8376-fef0343e5d94 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:20.999511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b74272-692f-468a-8376-fef0343e5d94 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:21.001355 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 127/235] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:21.006971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259ab64-4aa7-4a5c-bc37-f439decf9bc8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:21.007804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259ab64-4aa7-4a5c-bc37-f439decf9bc8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:21.008266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259ab64-4aa7-4a5c-bc37-f439decf9bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:21.028938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1259ab64-4aa7-4a5c-bc37-f439decf9bc8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:21.030442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1259ab64-4aa7-4a5c-bc37-f439decf9bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:21.045336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d4ee9-1216-4ebe-941d-b5bcd87799cd None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:21.045981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d4ee9-1216-4ebe-941d-b5bcd87799cd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:21.046468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d4ee9-1216-4ebe-941d-b5bcd87799cd None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:21.048297 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 128/236] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:21.053560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328de5ff-4560-4cbe-831c-be12f7efa02f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:21.053864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328de5ff-4560-4cbe-831c-be12f7efa02f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:21.054167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328de5ff-4560-4cbe-831c-be12f7efa02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:21.077374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-328de5ff-4560-4cbe-831c-be12f7efa02f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:21.078627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-328de5ff-4560-4cbe-831c-be12f7efa02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:21.119350 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 109/237] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:21.171859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259ab64-4aa7-4a5c-bc37-f439decf9bc8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:21.205807 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 129/238] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:21.240467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328de5ff-4560-4cbe-831c-be12f7efa02f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:21.334434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:21.357563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259ab64-4aa7-4a5c-bc37-f439decf9bc8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:21.380551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:21.411663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328de5ff-4560-4cbe-831c-be12f7efa02f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:21.428026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c3d4b-8d89-40df-a214-0feb5573ee26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:21.428835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c3d4b-8d89-40df-a214-0feb5573ee26 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:21.429120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c3d4b-8d89-40df-a214-0feb5573ee26 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:21.429399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c3d4b-8d89-40df-a214-0feb5573ee26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:21.432626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c3d4b-8d89-40df-a214-0feb5573ee26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:21.435099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c3d4b-8d89-40df-a214-0feb5573ee26 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:21.436754 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 110/239] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:21.444755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c3d4b-8d89-40df-a214-0feb5573ee26 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:21.483646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11173a1-6011-466a-835f-7ccc0df89684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:21.488591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11173a1-6011-466a-835f-7ccc0df89684 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:21.489060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11173a1-6011-466a-835f-7ccc0df89684 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:21.489874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11173a1-6011-466a-835f-7ccc0df89684 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:21.511835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11173a1-6011-466a-835f-7ccc0df89684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:21.523481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-f11173a1-6011-466a-835f-7ccc0df89684 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:21.523481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:21.523481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:21.523481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:21.523481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:21.523481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:21.523481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:21.523481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:21.523481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:21.523481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:21.523481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:21.523481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:21.523481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:21.523481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:21.523481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:21.523481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:21.524437 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:21.524437 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:21.524437 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:21.524437 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:21.524437 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:21.524437 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:21.524437 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:21.524437 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:21.524437 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:21.524437 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:21.524437 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:21.524437 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:21.524437 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:21.524437 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:21.524437 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:21.524437 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:21.526261 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:21.526261 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:21.526261 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:21.526261 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:21.526261 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:21.526261 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:21.526261 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:21.526261 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:21.526261 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:21.526261 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:21.526261 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:21.526261 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:21.526261 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 130/240] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:41:21.530249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11173a1-6011-466a-835f-7ccc0df89684 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:21.629310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c3d4b-8d89-40df-a214-0feb5573ee26 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:21.669298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be104f4-9b7a-4111-916b-4369e69c430e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:21.670002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be104f4-9b7a-4111-916b-4369e69c430e None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:21.670294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be104f4-9b7a-4111-916b-4369e69c430e None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:21.670576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be104f4-9b7a-4111-916b-4369e69c430e None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:21.671314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be104f4-9b7a-4111-916b-4369e69c430e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:21.673061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be104f4-9b7a-4111-916b-4369e69c430e None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:21.715113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11173a1-6011-466a-835f-7ccc0df89684 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:21.756455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a50dde-6137-45a8-a7e7-3acbfc1efd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:21.757182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a50dde-6137-45a8-a7e7-3acbfc1efd77 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:21.757470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a50dde-6137-45a8-a7e7-3acbfc1efd77 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:21.757746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a50dde-6137-45a8-a7e7-3acbfc1efd77 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:21.758589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a50dde-6137-45a8-a7e7-3acbfc1efd77 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:21.758945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a50dde-6137-45a8-a7e7-3acbfc1efd77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:21.760346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a50dde-6137-45a8-a7e7-3acbfc1efd77 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:21.769814 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 131/241] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:21.775747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a50dde-6137-45a8-a7e7-3acbfc1efd77 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:21.790566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:21.793500 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 111/242] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:41:21 2022] POST /identity/v3/users => generated 293 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:41:21.885667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a50dde-6137-45a8-a7e7-3acbfc1efd77 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:21.922810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c75c5-1baa-41ed-9b76-da7fd6bc766c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:21.923694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c75c5-1baa-41ed-9b76-da7fd6bc766c None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:21.923979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c75c5-1baa-41ed-9b76-da7fd6bc766c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:21.924259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c75c5-1baa-41ed-9b76-da7fd6bc766c None admin] PATH_INFO: `/v3/users/231b9c8ff9fd4c87a30301ceaeae1a44` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:21.939456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c75c5-1baa-41ed-9b76-da7fd6bc766c None admin] RBAC: Authorizing `identity:get_user(user_id=231b9c8ff9fd4c87a30301ceaeae1a44)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:21.940949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c75c5-1baa-41ed-9b76-da7fd6bc766c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:21.942766 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 132/243] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:21 2022] GET /identity/v3/users/231b9c8ff9fd4c87a30301ceaeae1a44 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:21.948383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be104f4-9b7a-4111-916b-4369e69c430e None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:21.970557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:22.073807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be104f4-9b7a-4111-916b-4369e69c430e None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:22.121239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2358b-0448-414d-92f7-22fec4daed96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:22.122369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2358b-0448-414d-92f7-22fec4daed96 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:22.122865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2358b-0448-414d-92f7-22fec4daed96 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:22.123350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2358b-0448-414d-92f7-22fec4daed96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:22.201403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2358b-0448-414d-92f7-22fec4daed96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:22.203284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2358b-0448-414d-92f7-22fec4daed96 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:22.221433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:22.318808 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 112/244] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:22.324193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c75c5-1baa-41ed-9b76-da7fd6bc766c None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:22.368333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a656e-2a1a-42c5-aa8a-0a926abc4f11 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:22.368635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a656e-2a1a-42c5-aa8a-0a926abc4f11 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:22.369372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a656e-2a1a-42c5-aa8a-0a926abc4f11 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:22.370871 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 113/245] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:41:22.376991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e55b9-5db7-4300-b844-dcbd0710d548 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:22.377280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e55b9-5db7-4300-b844-dcbd0710d548 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:22.377566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e55b9-5db7-4300-b844-dcbd0710d548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:22.398789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-297e55b9-5db7-4300-b844-dcbd0710d548 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:22.399104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-297e55b9-5db7-4300-b844-dcbd0710d548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:22.425164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:22.469717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c75c5-1baa-41ed-9b76-da7fd6bc766c None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:22.478681 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 114/246] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:22.504108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e55b9-5db7-4300-b844-dcbd0710d548 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:22.514163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac2ef7-1b13-49d3-944d-da1b5d0d8f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:22.515134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac2ef7-1b13-49d3-944d-da1b5d0d8f26 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:22.515538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac2ef7-1b13-49d3-944d-da1b5d0d8f26 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:22.516081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac2ef7-1b13-49d3-944d-da1b5d0d8f26 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:22.518031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac2ef7-1b13-49d3-944d-da1b5d0d8f26 None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:22.519520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac2ef7-1b13-49d3-944d-da1b5d0d8f26 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:22.522007 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 133/247] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:22 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:22.528971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac2ef7-1b13-49d3-944d-da1b5d0d8f26 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:22.565640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:22.632876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e55b9-5db7-4300-b844-dcbd0710d548 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:22.662037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac2ef7-1b13-49d3-944d-da1b5d0d8f26 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:22.671730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50b6559-479b-4d3d-9798-e1623d486963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:22.672875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b6559-479b-4d3d-9798-e1623d486963 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:22.673361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b6559-479b-4d3d-9798-e1623d486963 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:22.673887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50b6559-479b-4d3d-9798-e1623d486963 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:22.685972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50b6559-479b-4d3d-9798-e1623d486963 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:22.695366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-b50b6559-479b-4d3d-9798-e1623d486963 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:22.695366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:22.695366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:22.695366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:22.695366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:22.695366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:22.695366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:22.695366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:22.695366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:22.695366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:22.695366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:22.695366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:22.695366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:22.695366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:22.695366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:22.695366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:22.696272 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:22.696272 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:22.696272 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:22.696272 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:22.696272 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:22.696272 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:22.696272 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:22.696272 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:22.696272 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:22.696272 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:22.696272 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:22.696272 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:22.696272 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:22.696272 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:22.696272 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:22.696272 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:22.697108 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:22.697108 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:22.697108 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:22.697108 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:22.697108 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:22.697108 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:22.697108 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:22.697108 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:22.697108 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:22.697108 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:22.697108 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:22.697108 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:22.699830 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 115/248] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:41:22.700062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b63b04-9119-4c50-b8be-4a3574f8d90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:22.700732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b63b04-9119-4c50-b8be-4a3574f8d90c None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:22.701205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b63b04-9119-4c50-b8be-4a3574f8d90c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:22.701592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b63b04-9119-4c50-b8be-4a3574f8d90c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:22.705861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b63b04-9119-4c50-b8be-4a3574f8d90c None admin] RBAC: Adding query filter params (role.id=19a96330fa9044a28cedef74491c0d4c, scope.project.id=0381e5a4ddde4d9ab4394d1417743579, user.id=231b9c8ff9fd4c87a30301ceaeae1a44) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:22.705861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b63b04-9119-4c50-b8be-4a3574f8d90c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:22.705861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b63b04-9119-4c50-b8be-4a3574f8d90c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:22.706427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2df7bd-57ac-47e1-b25a-ca5b4a0a6e19 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:22.706912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2df7bd-57ac-47e1-b25a-ca5b4a0a6e19 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:22.707372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2df7bd-57ac-47e1-b25a-ca5b4a0a6e19 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:22.708580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:22.709685 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 116/249] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:22.711317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2df7bd-57ac-47e1-b25a-ca5b4a0a6e19 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:22.716199 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 134/250] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:41:22 2022] GET /identity/v3/role_assignments?user.id=231b9c8ff9fd4c87a30301ceaeae1a44&scope.project.id=0381e5a4ddde4d9ab4394d1417743579&role.id=19a96330fa9044a28cedef74491c0d4c => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:22.720304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4119f-afd6-467f-be81-5a745a3134c8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:22.720304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4119f-afd6-467f-be81-5a745a3134c8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:22.720304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d4119f-afd6-467f-be81-5a745a3134c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:22.740478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-48d4119f-afd6-467f-be81-5a745a3134c8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:22.742307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-48d4119f-afd6-467f-be81-5a745a3134c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:22.822284 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 135/251] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:22.831163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2df7bd-57ac-47e1-b25a-ca5b4a0a6e19 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:22.835399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:22.855645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4119f-afd6-467f-be81-5a745a3134c8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:22.873154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe8cd3-8515-49df-9ba6-eef07dd29482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:22.874584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe8cd3-8515-49df-9ba6-eef07dd29482 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:22.875329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe8cd3-8515-49df-9ba6-eef07dd29482 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:22.876007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe8cd3-8515-49df-9ba6-eef07dd29482 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:22.877066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe8cd3-8515-49df-9ba6-eef07dd29482 None admin] RBAC: Adding query filter params (name=member) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:22.877562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe8cd3-8515-49df-9ba6-eef07dd29482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:22.879215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe8cd3-8515-49df-9ba6-eef07dd29482 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:22.889547 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 117/252] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:22.897616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe8cd3-8515-49df-9ba6-eef07dd29482 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:23.064376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d4119f-afd6-467f-be81-5a745a3134c8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:23.106241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:23.116275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c7300-528e-4d34-b278-f004970bcd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:23.117129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c7300-528e-4d34-b278-f004970bcd2d None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:23.117417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c7300-528e-4d34-b278-f004970bcd2d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:23.117699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c7300-528e-4d34-b278-f004970bcd2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:23.123952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe8cd3-8515-49df-9ba6-eef07dd29482 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:23.133294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c7300-528e-4d34-b278-f004970bcd2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:23.142135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-802c7300-528e-4d34-b278-f004970bcd2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:41:23.142135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:23.142135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:23.142135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:23.142135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:23.142135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:23.142135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:23.142135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:23.142135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:23.142135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:23.142135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:23.142135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:23.142135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:23.142135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:23.142135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:23.142135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:23.143297 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:23.143297 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:23.143297 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:23.143297 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:23.143297 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:23.143297 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:23.143297 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:23.143297 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:23.143297 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:23.143297 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:23.143297 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:23.143297 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:23.143297 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:23.143297 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:23.143297 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:23.143297 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:23.144135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:23.144135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:23.144135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:23.144135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:23.144135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:23.144135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:23.144135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:23.144135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:23.144135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:23.144135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:23.144135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:41:23.144135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:23.148609 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 136/253] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:41:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:23.157987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c7300-528e-4d34-b278-f004970bcd2d None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:23.192234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd5c40-a12b-4f5c-b21a-b44ac71529fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:23.194046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd5c40-a12b-4f5c-b21a-b44ac71529fb None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:23.194531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd5c40-a12b-4f5c-b21a-b44ac71529fb None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:23.195006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dd5c40-a12b-4f5c-b21a-b44ac71529fb None admin] PATH_INFO: `/v3/users/fd9c4adcec884ec6accb328898e94e1e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:23.196523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd5c40-a12b-4f5c-b21a-b44ac71529fb None admin] RBAC: Authorizing `identity:get_user(user_id=fd9c4adcec884ec6accb328898e94e1e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:23.198394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dd5c40-a12b-4f5c-b21a-b44ac71529fb None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:23.200539 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 118/254] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:22 2022] GET /identity/v3/users/fd9c4adcec884ec6accb328898e94e1e => generated 297 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:23.206113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd5c40-a12b-4f5c-b21a-b44ac71529fb None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:23.266054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:23.274899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c7300-528e-4d34-b278-f004970bcd2d None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:23.310216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964fa28-1ce8-49a7-bcec-0d86b76d90a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:23.311640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964fa28-1ce8-49a7-bcec-0d86b76d90a5 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:23.312085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964fa28-1ce8-49a7-bcec-0d86b76d90a5 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:23.312411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964fa28-1ce8-49a7-bcec-0d86b76d90a5 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:23.313290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964fa28-1ce8-49a7-bcec-0d86b76d90a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:23.313695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964fa28-1ce8-49a7-bcec-0d86b76d90a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:23.315528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964fa28-1ce8-49a7-bcec-0d86b76d90a5 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:23.330317 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 137/255] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:23.333671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e07b2f-dc26-4d79-9804-d152f3db0506 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:23.333671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e07b2f-dc26-4d79-9804-d152f3db0506 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:23.333671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e07b2f-dc26-4d79-9804-d152f3db0506 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:23.333671 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 138/256] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:23.335211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e07b2f-dc26-4d79-9804-d152f3db0506 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:23.360141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dd5c40-a12b-4f5c-b21a-b44ac71529fb None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:23.394482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:23.406382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a2482-f5a2-47cf-8c2a-a7a569ffa6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:23.407806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a2482-f5a2-47cf-8c2a-a7a569ffa6e5 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:23.408099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a2482-f5a2-47cf-8c2a-a7a569ffa6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:23.408378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a2482-f5a2-47cf-8c2a-a7a569ffa6e5 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:23.409579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a2482-f5a2-47cf-8c2a-a7a569ffa6e5 None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:23.411734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a2482-f5a2-47cf-8c2a-a7a569ffa6e5 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:23.414150 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 119/257] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:23 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:23.416118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b74fc-4937-4ac5-a11f-39614f3577f7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:23.416403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b74fc-4937-4ac5-a11f-39614f3577f7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:23.416682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b74fc-4937-4ac5-a11f-39614f3577f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:23.437095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f40b74fc-4937-4ac5-a11f-39614f3577f7 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:23.438314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f40b74fc-4937-4ac5-a11f-39614f3577f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:23.470302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e07b2f-dc26-4d79-9804-d152f3db0506 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:23.517417 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 120/258] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:23.518432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b74fc-4937-4ac5-a11f-39614f3577f7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:23.563204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a3667-72b8-4f8a-86ea-68fe04702609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:23.564446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a3667-72b8-4f8a-86ea-68fe04702609 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:23.564937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a3667-72b8-4f8a-86ea-68fe04702609 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:23.565441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a3667-72b8-4f8a-86ea-68fe04702609 None admin] PATH_INFO: `/v3/users/853965ff331c4b4b9b0e578fbe397541` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:23.566997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a3667-72b8-4f8a-86ea-68fe04702609 None admin] RBAC: Authorizing `identity:get_user(user_id=853965ff331c4b4b9b0e578fbe397541)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:23.568712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a3667-72b8-4f8a-86ea-68fe04702609 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:23.570622 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 139/259] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:23 2022] GET /identity/v3/users/853965ff331c4b4b9b0e578fbe397541 => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:23.574558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a3667-72b8-4f8a-86ea-68fe04702609 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:23.577265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:23.685650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a3667-72b8-4f8a-86ea-68fe04702609 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:23.708054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:23.724611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c877b-c8b7-461f-8f47-34bdcb7ced0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:23.725428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c877b-c8b7-461f-8f47-34bdcb7ced0e None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:23.725707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c877b-c8b7-461f-8f47-34bdcb7ced0e None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:23.726022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37c877b-c8b7-461f-8f47-34bdcb7ced0e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:23.738004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37c877b-c8b7-461f-8f47-34bdcb7ced0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:23.747117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-a37c877b-c8b7-461f-8f47-34bdcb7ced0e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:23.747117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:23.747117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:23.747117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:23.747117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:23.747117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:23.747117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:23.747117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:23.747117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:23.747117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:23.747117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:23.747117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:23.747117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:23.747117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:23.747117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:23.747117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:23.748000 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:23.748000 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:23.748000 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:23.748000 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:23.748000 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:23.748000 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:23.748000 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:23.748000 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:23.748000 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:23.748000 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:23.748000 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:23.748000 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:23.748000 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:23.748000 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:23.748000 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:23.748000 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:23.748779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:23.748779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:23.748779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:23.748779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:23.748779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:23.748779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:23.748779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:23.748779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:23.748779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:23.748779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:23.748779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:23.748779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:23.748779 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 140/260] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:41:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:41:23.750170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c877b-c8b7-461f-8f47-34bdcb7ced0e None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:23.775766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b74fc-4937-4ac5-a11f-39614f3577f7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:23.844707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e284fb6-50f6-490f-b4ce-696350c2cea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:23.845525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e284fb6-50f6-490f-b4ce-696350c2cea8 None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:23.845807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e284fb6-50f6-490f-b4ce-696350c2cea8 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:23.846115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e284fb6-50f6-490f-b4ce-696350c2cea8 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579/users/fd9c4adcec884ec6accb328898e94e1e/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:23.848098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e284fb6-50f6-490f-b4ce-696350c2cea8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0381e5a4ddde4d9ab4394d1417743579, user_id=fd9c4adcec884ec6accb328898e94e1e, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:23.849592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e284fb6-50f6-490f-b4ce-696350c2cea8 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:23.861027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37c877b-c8b7-461f-8f47-34bdcb7ced0e None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:23.896925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801a09e-f73e-4698-aa95-dd8876a66ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:23.897631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801a09e-f73e-4698-aa95-dd8876a66ee6 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:23.897954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801a09e-f73e-4698-aa95-dd8876a66ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:23.898233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801a09e-f73e-4698-aa95-dd8876a66ee6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:23.951056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:23.952503 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 121/261] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:41:23 2022] PUT /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579/users/fd9c4adcec884ec6accb328898e94e1e/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:41:23.954381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e284fb6-50f6-490f-b4ce-696350c2cea8 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:23.977510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d801a09e-f73e-4698-aa95-dd8876a66ee6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:23.979118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d801a09e-f73e-4698-aa95-dd8876a66ee6 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:24.090698 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 141/262] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:24.103259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801a09e-f73e-4698-aa95-dd8876a66ee6 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:24.166231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:24.183354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e284fb6-50f6-490f-b4ce-696350c2cea8 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:24.233298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:24.249422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55186e94-133a-4773-b1b7-5fea238a1541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:24.250551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55186e94-133a-4773-b1b7-5fea238a1541 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:24.251061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55186e94-133a-4773-b1b7-5fea238a1541 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:24.251534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55186e94-133a-4773-b1b7-5fea238a1541 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:24.252494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55186e94-133a-4773-b1b7-5fea238a1541 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:24.252992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55186e94-133a-4773-b1b7-5fea238a1541 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:24.254608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55186e94-133a-4773-b1b7-5fea238a1541 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:24.260831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801a09e-f73e-4698-aa95-dd8876a66ee6 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:24.266152 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 122/263] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:41:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:24.281492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55186e94-133a-4773-b1b7-5fea238a1541 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:24.324305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de21bd-2b8d-42ba-bbe1-24010f695103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:24.325164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de21bd-2b8d-42ba-bbe1-24010f695103 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:24.325441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de21bd-2b8d-42ba-bbe1-24010f695103 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:24.325722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de21bd-2b8d-42ba-bbe1-24010f695103 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:24.327061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de21bd-2b8d-42ba-bbe1-24010f695103 None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:24.330434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de21bd-2b8d-42ba-bbe1-24010f695103 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:24.332338 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 142/264] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:24 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:24.339117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de21bd-2b8d-42ba-bbe1-24010f695103 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:24.484992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55186e94-133a-4773-b1b7-5fea238a1541 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:24.509384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de21bd-2b8d-42ba-bbe1-24010f695103 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:24.545134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a844f-6c9d-4d39-a89c-d2842682bfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:24.545794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a844f-6c9d-4d39-a89c-d2842682bfec None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:24.546126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a844f-6c9d-4d39-a89c-d2842682bfec None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:24.546395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a844f-6c9d-4d39-a89c-d2842682bfec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:24.547226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a844f-6c9d-4d39-a89c-d2842682bfec None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=0381e5a4ddde4d9ab4394d1417743579, user.id=853965ff331c4b4b9b0e578fbe397541) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:24.547539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a844f-6c9d-4d39-a89c-d2842682bfec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:24.548985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a844f-6c9d-4d39-a89c-d2842682bfec None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:24.551480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2a40b-d2dd-48d9-9584-bfe5bc73a139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:24.552900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2a40b-d2dd-48d9-9584-bfe5bc73a139 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:24.553186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2a40b-d2dd-48d9-9584-bfe5bc73a139 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:24.553463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2a40b-d2dd-48d9-9584-bfe5bc73a139 None admin] PATH_INFO: `/v3/users/853965ff331c4b4b9b0e578fbe397541` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:24.554722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2a40b-d2dd-48d9-9584-bfe5bc73a139 None admin] RBAC: Authorizing `identity:get_user(user_id=853965ff331c4b4b9b0e578fbe397541)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:24.555962 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 143/265] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:41:24 2022] GET /identity/v3/role_assignments?user.id=853965ff331c4b4b9b0e578fbe397541&scope.project.id=0381e5a4ddde4d9ab4394d1417743579&role.id=7805b93b32f1472f8365936d4d57d49a => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:24.562303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a2a40b-d2dd-48d9-9584-bfe5bc73a139 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:24.564606 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 123/266] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:24 2022] GET /identity/v3/users/853965ff331c4b4b9b0e578fbe397541 => generated 283 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:24.569855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a844f-6c9d-4d39-a89c-d2842682bfec None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:24.598845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:24.719501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a844f-6c9d-4d39-a89c-d2842682bfec None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:24.760533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ba9a9-ef94-46a8-ab09-f70c8323585a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:24.761122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ba9a9-ef94-46a8-ab09-f70c8323585a None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:24.761407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ba9a9-ef94-46a8-ab09-f70c8323585a None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:24.761691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ba9a9-ef94-46a8-ab09-f70c8323585a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:24.850596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ba9a9-ef94-46a8-ab09-f70c8323585a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:24.852079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ba9a9-ef94-46a8-ab09-f70c8323585a None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:24.863767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:24.984611 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 144/267] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:24.990390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2a40b-d2dd-48d9-9584-bfe5bc73a139 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:25.023324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:25.110506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a2a40b-d2dd-48d9-9584-bfe5bc73a139 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:25.146663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ecbe11-d394-4e13-ae42-9de023a8a385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:25.147540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecbe11-d394-4e13-ae42-9de023a8a385 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:25.147825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecbe11-d394-4e13-ae42-9de023a8a385 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:25.148104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ecbe11-d394-4e13-ae42-9de023a8a385 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:25.149364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ecbe11-d394-4e13-ae42-9de023a8a385 None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:25.150859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ecbe11-d394-4e13-ae42-9de023a8a385 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:25.152930 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 124/268] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:24 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:25.158530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ba9a9-ef94-46a8-ab09-f70c8323585a None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:25.289007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ba9a9-ef94-46a8-ab09-f70c8323585a None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:25.305922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:25.335490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8059415-8a93-478c-b540-c7b6430561f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:25.336233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8059415-8a93-478c-b540-c7b6430561f4 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:25.336447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8059415-8a93-478c-b540-c7b6430561f4 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:25.336728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8059415-8a93-478c-b540-c7b6430561f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:25.337579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8059415-8a93-478c-b540-c7b6430561f4 None admin] RBAC: Adding query filter params (role.id=92b887e453ad4fa9aa5c49f7594a7e53, scope.project.id=0381e5a4ddde4d9ab4394d1417743579, user.id=853965ff331c4b4b9b0e578fbe397541) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:25.337922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8059415-8a93-478c-b540-c7b6430561f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:25.339563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8059415-8a93-478c-b540-c7b6430561f4 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:25.346326 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 145/269] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:41:25 2022] GET /identity/v3/role_assignments?user.id=853965ff331c4b4b9b0e578fbe397541&scope.project.id=0381e5a4ddde4d9ab4394d1417743579&role.id=92b887e453ad4fa9aa5c49f7594a7e53 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:25.634951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b184f-eb00-4fa0-8c26-baf4735a0f35 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:25.635471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b184f-eb00-4fa0-8c26-baf4735a0f35 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:25.635471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b184f-eb00-4fa0-8c26-baf4735a0f35 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:25.636848 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 125/270] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:25.653240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8762561-1ed7-40e1-9f55-ed6d51e7372e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:25.653803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8762561-1ed7-40e1-9f55-ed6d51e7372e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:25.654254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8762561-1ed7-40e1-9f55-ed6d51e7372e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:25.677935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b8762561-1ed7-40e1-9f55-ed6d51e7372e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:25.679373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b8762561-1ed7-40e1-9f55-ed6d51e7372e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:25.762300 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 146/271] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:25.792101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b184f-eb00-4fa0-8c26-baf4735a0f35 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:25.865492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc9d8d-d2fb-4bd2-9502-81b879a5d86c None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:25.865810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc9d8d-d2fb-4bd2-9502-81b879a5d86c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:25.866381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc9d8d-d2fb-4bd2-9502-81b879a5d86c None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:25.868814 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 147/272] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:41:25.874676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a970d-48d7-4577-80e3-d796c0834fef None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:25.874975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a970d-48d7-4577-80e3-d796c0834fef None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:25.875261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569a970d-48d7-4577-80e3-d796c0834fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:25.901171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-569a970d-48d7-4577-80e3-d796c0834fef None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:25.901828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-569a970d-48d7-4577-80e3-d796c0834fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:25.925698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:25.940154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b184f-eb00-4fa0-8c26-baf4735a0f35 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:25.980045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac979e4d-f0db-4402-87f3-e9b617604ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:25.980842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac979e4d-f0db-4402-87f3-e9b617604ebf None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:25.981125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac979e4d-f0db-4402-87f3-e9b617604ebf None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:25.981401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac979e4d-f0db-4402-87f3-e9b617604ebf None admin] PATH_INFO: `/v3/domains/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:25.982570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac979e4d-f0db-4402-87f3-e9b617604ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:25.984024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac979e4d-f0db-4402-87f3-e9b617604ebf None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:25.984604 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 148/273] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:41:25.989974 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 126/274] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:25.992176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d449ac-0a08-44d2-aa79-9d9262933b19 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:25.992176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d449ac-0a08-44d2-aa79-9d9262933b19 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:25.992176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d449ac-0a08-44d2-aa79-9d9262933b19 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:25.992176 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 149/275] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:25.992833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac979e4d-f0db-4402-87f3-e9b617604ebf None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:25.997541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094a6e5-ce53-4956-8415-df18ac93868a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:25.997820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094a6e5-ce53-4956-8415-df18ac93868a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:25.998127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094a6e5-ce53-4956-8415-df18ac93868a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:26.021078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4094a6e5-ce53-4956-8415-df18ac93868a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:26.022292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4094a6e5-ce53-4956-8415-df18ac93868a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:26.050659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:26.096527 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 150/276] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:26.097454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094a6e5-ce53-4956-8415-df18ac93868a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:26.115004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac979e4d-f0db-4402-87f3-e9b617604ebf None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:26.154370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fd167-3048-4a1e-974a-676f416ac152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:26.155073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fd167-3048-4a1e-974a-676f416ac152 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:26.155436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fd167-3048-4a1e-974a-676f416ac152 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:26.155792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fd167-3048-4a1e-974a-676f416ac152 None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:26.156583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fd167-3048-4a1e-974a-676f416ac152 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:26.158079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fd167-3048-4a1e-974a-676f416ac152 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:26.216702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094a6e5-ce53-4956-8415-df18ac93868a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:26.246949 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 127/277] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:41:25 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:41:26.246949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fd167-3048-4a1e-974a-676f416ac152 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:26.262291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4a301-dd90-43c2-8e85-9dd21b2144fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:26.263438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4a301-dd90-43c2-8e85-9dd21b2144fe None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:26.263721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4a301-dd90-43c2-8e85-9dd21b2144fe None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:26.264048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4a301-dd90-43c2-8e85-9dd21b2144fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:26.266048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:26.280487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4a301-dd90-43c2-8e85-9dd21b2144fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:26.290986 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-89b4a301-dd90-43c2-8e85-9dd21b2144fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:26.290986 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:26.290986 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:26.290986 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:26.290986 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:26.290986 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:26.290986 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:26.290986 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:26.290986 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:26.290986 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:26.290986 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:26.290986 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:26.290986 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:26.290986 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:26.290986 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:26.290986 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:26.292041 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:26.292041 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:26.292041 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:26.292041 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:26.292041 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:26.292041 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:26.292041 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:26.292041 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:26.292041 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:26.292041 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:26.292041 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:26.292041 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:26.292041 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:26.292041 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:26.292041 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:26.292041 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:26.292823 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:26.292823 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:26.292823 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:26.292823 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:26.292823 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:26.292823 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:26.292823 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:26.292823 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:26.292823 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:26.292823 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:26.292823 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:26.292823 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:26.293461 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 151/278] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:26.299578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4a301-dd90-43c2-8e85-9dd21b2144fe None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:26.346249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:26.378590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fd167-3048-4a1e-974a-676f416ac152 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:26.420078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f8c15-238c-4d2a-a687-70452fbc3338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:26.421362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f8c15-238c-4d2a-a687-70452fbc3338 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:26.421872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f8c15-238c-4d2a-a687-70452fbc3338 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:26.422348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f8c15-238c-4d2a-a687-70452fbc3338 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:26.436024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f8c15-238c-4d2a-a687-70452fbc3338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:26.446047 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-b46f8c15-238c-4d2a-a687-70452fbc3338 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:26.446047 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:26.446047 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:26.446047 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:26.446047 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:26.446047 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:26.446047 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:26.446047 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:26.446047 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:26.446047 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:26.446047 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:26.446047 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:26.446047 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:26.446047 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:26.446047 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:26.446047 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:26.452190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:26.452190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:26.452190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:26.452190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:26.452190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:26.452190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:26.452190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:26.452190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:26.452190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:26.452190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:26.452190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:26.452190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:26.452190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:26.452190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:26.452190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:26.452190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:26.452986 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:26.452986 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:26.452986 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:26.452986 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:26.452986 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:26.452986 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:26.452986 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:26.452986 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:26.452986 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:26.452986 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:26.452986 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:26.452986 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:26.452986 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 128/279] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:26.453644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4a301-dd90-43c2-8e85-9dd21b2144fe None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:26.455783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f8c15-238c-4d2a-a687-70452fbc3338 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:26.500036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbfb415-473b-460b-9720-28644b29fb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:26.500787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfb415-473b-460b-9720-28644b29fb1f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:26.501075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfb415-473b-460b-9720-28644b29fb1f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:26.501361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbfb415-473b-460b-9720-28644b29fb1f None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:26.504236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbfb415-473b-460b-9720-28644b29fb1f None admin] RBAC: Adding query filter params (name=reader) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:26.504574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbfb415-473b-460b-9720-28644b29fb1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:26.506251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbfb415-473b-460b-9720-28644b29fb1f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:26.525830 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 152/280] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:26.528128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbfb415-473b-460b-9720-28644b29fb1f None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:26.560857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:26.593971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f8c15-238c-4d2a-a687-70452fbc3338 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:26.636517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:26.639369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0ccb9-ff4d-41c2-b3e0-d8adb6602301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:26.641069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0ccb9-ff4d-41c2-b3e0-d8adb6602301 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:26.641566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0ccb9-ff4d-41c2-b3e0-d8adb6602301 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:26.642094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b0ccb9-ff4d-41c2-b3e0-d8adb6602301 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:26.643069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0ccb9-ff4d-41c2-b3e0-d8adb6602301 None admin] RBAC: Adding query filter params (name=member) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:26.643597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0ccb9-ff4d-41c2-b3e0-d8adb6602301 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:26.645221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b0ccb9-ff4d-41c2-b3e0-d8adb6602301 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:26.654836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbfb415-473b-460b-9720-28644b29fb1f None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:26.655344 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 129/281] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:26.662216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0ccb9-ff4d-41c2-b3e0-d8adb6602301 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:26.702779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145f2b3-2b74-4447-8ec7-2aca99a67c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:26.703586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145f2b3-2b74-4447-8ec7-2aca99a67c8a None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:26.703865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145f2b3-2b74-4447-8ec7-2aca99a67c8a None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:26.704138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145f2b3-2b74-4447-8ec7-2aca99a67c8a None admin] PATH_INFO: `/v3/users/231b9c8ff9fd4c87a30301ceaeae1a44` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:26.706165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145f2b3-2b74-4447-8ec7-2aca99a67c8a None admin] RBAC: Authorizing `identity:get_user(user_id=231b9c8ff9fd4c87a30301ceaeae1a44)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:26.707683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145f2b3-2b74-4447-8ec7-2aca99a67c8a None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:26.713867 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 153/282] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:26 2022] GET /identity/v3/users/231b9c8ff9fd4c87a30301ceaeae1a44 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:26.717515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145f2b3-2b74-4447-8ec7-2aca99a67c8a None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:26.807777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b0ccb9-ff4d-41c2-b3e0-d8adb6602301 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:26.842845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145f2b3-2b74-4447-8ec7-2aca99a67c8a None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:26.850092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ebeaa-8f65-41a3-b4fa-b9d34b61113c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:26.851007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ebeaa-8f65-41a3-b4fa-b9d34b61113c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:26.851269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ebeaa-8f65-41a3-b4fa-b9d34b61113c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:26.851550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ebeaa-8f65-41a3-b4fa-b9d34b61113c None admin] PATH_INFO: `/v3/users/ffae2718686a42a9b980b3a5b53e01d6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:26.854980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:26.870623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ebeaa-8f65-41a3-b4fa-b9d34b61113c None admin] RBAC: Authorizing `identity:get_user(user_id=ffae2718686a42a9b980b3a5b53e01d6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:26.872405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ebeaa-8f65-41a3-b4fa-b9d34b61113c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:26.874945 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 130/283] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:26 2022] GET /identity/v3/users/ffae2718686a42a9b980b3a5b53e01d6 => generated 293 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:26.880938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ebeaa-8f65-41a3-b4fa-b9d34b61113c None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:26.891134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d04c2-e871-4b26-b73c-6d57aa896724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:26.892367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d04c2-e871-4b26-b73c-6d57aa896724 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:26.892904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d04c2-e871-4b26-b73c-6d57aa896724 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:26.893382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d04c2-e871-4b26-b73c-6d57aa896724 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:26.895000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d04c2-e871-4b26-b73c-6d57aa896724 None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:26.896930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d04c2-e871-4b26-b73c-6d57aa896724 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:26.899357 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 154/284] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:26 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:26.905240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d04c2-e871-4b26-b73c-6d57aa896724 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:26.934999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:26.986479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ebeaa-8f65-41a3-b4fa-b9d34b61113c None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:27.022728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b964de-61b8-45bf-84dc-c223c7300eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:27.023392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b964de-61b8-45bf-84dc-c223c7300eb4 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:27.023679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b964de-61b8-45bf-84dc-c223c7300eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:27.023959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b964de-61b8-45bf-84dc-c223c7300eb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:27.024798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b964de-61b8-45bf-84dc-c223c7300eb4 None admin] RBAC: Adding query filter params (role.id=c0c09ba97c694896ae14fceb6982b1a5, user.id=ffae2718686a42a9b980b3a5b53e01d6, scope.system=all) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:27.025114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b964de-61b8-45bf-84dc-c223c7300eb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:27.026312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d04c2-e871-4b26-b73c-6d57aa896724 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:27.026613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b964de-61b8-45bf-84dc-c223c7300eb4 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:27.032322 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 131/285] 10.222.0.33 () {56 vars in 1308 bytes} [Sat May 28 20:41:26 2022] GET /identity/v3/role_assignments?user.id=ffae2718686a42a9b980b3a5b53e01d6&scope.system=all&role.id=c0c09ba97c694896ae14fceb6982b1a5 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:27.033604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90551bcc-1f49-487b-8c82-2b8f4029cb36 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:27.042035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90551bcc-1f49-487b-8c82-2b8f4029cb36 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:27.042455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90551bcc-1f49-487b-8c82-2b8f4029cb36 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:27.044149 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 132/286] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:27 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:27.049981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a8d35-73df-477b-a0ff-8e11c02c3658 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:27.050271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a8d35-73df-477b-a0ff-8e11c02c3658 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:27.050552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a8d35-73df-477b-a0ff-8e11c02c3658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:27.082639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8db2a-1259-4162-9990-666dfe58c180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:27.084199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8db2a-1259-4162-9990-666dfe58c180 None admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:27.084721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8db2a-1259-4162-9990-666dfe58c180 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:27.084950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-891a8d35-73df-477b-a0ff-8e11c02c3658 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:27.085416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8db2a-1259-4162-9990-666dfe58c180 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579/users/231b9c8ff9fd4c87a30301ceaeae1a44/roles/19a96330fa9044a28cedef74491c0d4c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:27.086209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-891a8d35-73df-477b-a0ff-8e11c02c3658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:27.089901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8db2a-1259-4162-9990-666dfe58c180 None admin] RBAC: Authorizing `identity:create_grant(project_id=0381e5a4ddde4d9ab4394d1417743579, user_id=231b9c8ff9fd4c87a30301ceaeae1a44, role_id=19a96330fa9044a28cedef74491c0d4c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:27.091789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b8db2a-1259-4162-9990-666dfe58c180 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:27.156632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:27.173540 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 133/287] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:27.186268 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 155/288] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:41:26 2022] PUT /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579/users/231b9c8ff9fd4c87a30301ceaeae1a44/roles/19a96330fa9044a28cedef74491c0d4c => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:41:27.202089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a8d35-73df-477b-a0ff-8e11c02c3658 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:27.325189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a8d35-73df-477b-a0ff-8e11c02c3658 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:27.362450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90947617-222e-41a5-843d-2372221ca9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:27.363652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90947617-222e-41a5-843d-2372221ca9ea None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:27.364191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90947617-222e-41a5-843d-2372221ca9ea None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:27.364663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90947617-222e-41a5-843d-2372221ca9ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:27.389099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90947617-222e-41a5-843d-2372221ca9ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:27.398269 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-90947617-222e-41a5-843d-2372221ca9ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:27.398269 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:27.398269 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:27.398269 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:27.398269 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:27.398269 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:27.398269 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:27.398269 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:27.398269 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:27.398269 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:27.398269 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:27.398269 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:27.398269 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:27.398269 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:27.398269 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:27.398269 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:27.399229 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:27.399229 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:27.399229 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:27.399229 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:27.399229 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:27.399229 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:27.399229 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:27.399229 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:27.399229 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:27.399229 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:27.399229 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:27.399229 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:27.399229 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:27.399229 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:27.399229 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:27.399229 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:27.400010 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:27.400010 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:27.400010 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:27.400010 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:27.400010 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:27.400010 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:27.400010 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:27.400010 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:27.400010 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:27.400010 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:27.400010 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:27.400010 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:27.402448 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 134/289] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:27.408525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8db2a-1259-4162-9990-666dfe58c180 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:27.513947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:27.533293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b8db2a-1259-4162-9990-666dfe58c180 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:27.568463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a935fc-1d2e-4ca6-b6bf-6275ed374f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:27.569148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a935fc-1d2e-4ca6-b6bf-6275ed374f1b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:27.569436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a935fc-1d2e-4ca6-b6bf-6275ed374f1b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:27.569718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a935fc-1d2e-4ca6-b6bf-6275ed374f1b None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:27.571114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a935fc-1d2e-4ca6-b6bf-6275ed374f1b None admin] RBAC: Adding query filter params (name=member) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:27.571703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a935fc-1d2e-4ca6-b6bf-6275ed374f1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:27.573097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a935fc-1d2e-4ca6-b6bf-6275ed374f1b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:27.585919 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 156/290] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:27.588907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90947617-222e-41a5-843d-2372221ca9ea None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:27.644531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:27.709357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90947617-222e-41a5-843d-2372221ca9ea None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:27.744985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd302fb8-ab09-4358-924c-35fe0010ab2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:27.746330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd302fb8-ab09-4358-924c-35fe0010ab2b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:27.746753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd302fb8-ab09-4358-924c-35fe0010ab2b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:27.747168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd302fb8-ab09-4358-924c-35fe0010ab2b None admin] PATH_INFO: `/v3/users/fd9c4adcec884ec6accb328898e94e1e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:27.748515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd302fb8-ab09-4358-924c-35fe0010ab2b None admin] RBAC: Authorizing `identity:get_user(user_id=fd9c4adcec884ec6accb328898e94e1e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:27.750132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd302fb8-ab09-4358-924c-35fe0010ab2b None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:27.752095 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 135/291] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:27 2022] GET /identity/v3/users/fd9c4adcec884ec6accb328898e94e1e => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:27.758367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a935fc-1d2e-4ca6-b6bf-6275ed374f1b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:27.864529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a935fc-1d2e-4ca6-b6bf-6275ed374f1b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:27.900631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68437b47-4857-4aee-aeb1-50bb87f72f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:27.901686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68437b47-4857-4aee-aeb1-50bb87f72f16 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:27.902191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68437b47-4857-4aee-aeb1-50bb87f72f16 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:27.902653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68437b47-4857-4aee-aeb1-50bb87f72f16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:27.918259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:27.973049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68437b47-4857-4aee-aeb1-50bb87f72f16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:27.975160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68437b47-4857-4aee-aeb1-50bb87f72f16 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:28.081876 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 157/292] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:28.088010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd302fb8-ab09-4358-924c-35fe0010ab2b None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:28.222216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd302fb8-ab09-4358-924c-35fe0010ab2b None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:28.243391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:28.271406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b252003b-e4cc-4f72-ae86-a14c7e8294ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:28.272280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252003b-e4cc-4f72-ae86-a14c7e8294ed None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:28.272564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252003b-e4cc-4f72-ae86-a14c7e8294ed None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:28.272845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252003b-e4cc-4f72-ae86-a14c7e8294ed None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:28.274080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252003b-e4cc-4f72-ae86-a14c7e8294ed None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:28.275541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b252003b-e4cc-4f72-ae86-a14c7e8294ed None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:28.277250 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 136/293] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:28 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:28.283274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68437b47-4857-4aee-aeb1-50bb87f72f16 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:28.355788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:28.417726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68437b47-4857-4aee-aeb1-50bb87f72f16 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:28.458539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ba713-2fd4-4f82-ba59-3ea418e35210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:28.459208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ba713-2fd4-4f82-ba59-3ea418e35210 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:28.459497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ba713-2fd4-4f82-ba59-3ea418e35210 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:28.459777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ba713-2fd4-4f82-ba59-3ea418e35210 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:28.460617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ba713-2fd4-4f82-ba59-3ea418e35210 None admin] RBAC: Adding query filter params (role.id=c0c09ba97c694896ae14fceb6982b1a5, scope.project.id=0381e5a4ddde4d9ab4394d1417743579, user.id=fd9c4adcec884ec6accb328898e94e1e) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:28.460936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ba713-2fd4-4f82-ba59-3ea418e35210 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:28.462520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ba713-2fd4-4f82-ba59-3ea418e35210 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:28.468751 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 158/294] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:41:28 2022] GET /identity/v3/role_assignments?user.id=fd9c4adcec884ec6accb328898e94e1e&scope.project.id=0381e5a4ddde4d9ab4394d1417743579&role.id=c0c09ba97c694896ae14fceb6982b1a5 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:29.357149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023c0d4-9e93-43c3-b37c-a817711055f4 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:29.357627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023c0d4-9e93-43c3-b37c-a817711055f4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:29.357627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f023c0d4-9e93-43c3-b37c-a817711055f4 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:29.359325 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 137/295] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:29.364745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6216231-dd63-4378-8113-c34de27f7c84 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:29.365040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6216231-dd63-4378-8113-c34de27f7c84 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:29.365530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6216231-dd63-4378-8113-c34de27f7c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:29.367218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589896d9-95d8-4ece-bc64-45b09e651af7 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:29.367510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589896d9-95d8-4ece-bc64-45b09e651af7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:29.367792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589896d9-95d8-4ece-bc64-45b09e651af7 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:29.369339 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 138/296] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:29.374845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707f67f-fe0e-4085-9a69-3aeea95de55f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:29.375380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707f67f-fe0e-4085-9a69-3aeea95de55f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:29.375720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707f67f-fe0e-4085-9a69-3aeea95de55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:29.392940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a6216231-dd63-4378-8113-c34de27f7c84 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:29.394305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b707f67f-fe0e-4085-9a69-3aeea95de55f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:29.394711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a6216231-dd63-4378-8113-c34de27f7c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:29.396009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b707f67f-fe0e-4085-9a69-3aeea95de55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:29.476181 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 159/297] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:29.485601 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 139/298] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:29.499288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6216231-dd63-4378-8113-c34de27f7c84 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:29.514371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707f67f-fe0e-4085-9a69-3aeea95de55f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:29.544405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:29.610820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6216231-dd63-4378-8113-c34de27f7c84 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:29.648819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707f67f-fe0e-4085-9a69-3aeea95de55f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:29.651003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0ffe8-7d31-4282-ae7a-a1759b6e2902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:29.652268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0ffe8-7d31-4282-ae7a-a1759b6e2902 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:29.652774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0ffe8-7d31-4282-ae7a-a1759b6e2902 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:29.653328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0ffe8-7d31-4282-ae7a-a1759b6e2902 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:29.655020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0ffe8-7d31-4282-ae7a-a1759b6e2902 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:29.657046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0ffe8-7d31-4282-ae7a-a1759b6e2902 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:29.659376 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 160/299] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:29.666094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0ffe8-7d31-4282-ae7a-a1759b6e2902 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:29.672687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:29.687820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c0fa7-a2dc-405a-96e1-17d741237956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:29.688863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c0fa7-a2dc-405a-96e1-17d741237956 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:29.689286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c0fa7-a2dc-405a-96e1-17d741237956 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:29.689698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c0fa7-a2dc-405a-96e1-17d741237956 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:29.700784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c0fa7-a2dc-405a-96e1-17d741237956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:29.709624 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-4c3c0fa7-a2dc-405a-96e1-17d741237956 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:29.709624 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:29.709624 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:29.709624 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:29.709624 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:29.709624 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:29.709624 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:29.709624 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:29.709624 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:29.709624 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:29.709624 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:29.709624 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:29.709624 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:29.709624 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:29.709624 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:29.709624 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:29.710523 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:29.710523 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:29.710523 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:29.710523 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:29.710523 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:29.710523 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:29.710523 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:29.710523 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:29.710523 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:29.710523 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:29.710523 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:29.710523 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:29.710523 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:29.710523 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:29.710523 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:29.710523 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:29.711301 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:29.711301 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:29.711301 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:29.711301 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:29.711301 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:29.711301 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:29.711301 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:29.711301 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:29.711301 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:29.711301 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:29.711301 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:29.711301 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:29.713632 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 140/300] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:41:29.719451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c0fa7-a2dc-405a-96e1-17d741237956 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:29.780146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0ffe8-7d31-4282-ae7a-a1759b6e2902 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:29.789204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:29.819856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c13c87-7400-4f97-a943-b1ada06b4bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:29.820877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c13c87-7400-4f97-a943-b1ada06b4bb4 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:29.822330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c13c87-7400-4f97-a943-b1ada06b4bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:29.822770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c13c87-7400-4f97-a943-b1ada06b4bb4 None admin] PATH_INFO: `/v3/groups` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:29.823668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c13c87-7400-4f97-a943-b1ada06b4bb4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:29.825527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c13c87-7400-4f97-a943-b1ada06b4bb4 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:29.827681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c0fa7-a2dc-405a-96e1-17d741237956 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:29.861644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482143ac-cabf-4b53-872b-fd4eef9464f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:29.862271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482143ac-cabf-4b53-872b-fd4eef9464f0 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:29.862558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482143ac-cabf-4b53-872b-fd4eef9464f0 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:29.862844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482143ac-cabf-4b53-872b-fd4eef9464f0 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:29.863588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482143ac-cabf-4b53-872b-fd4eef9464f0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:29.864427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482143ac-cabf-4b53-872b-fd4eef9464f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:29.865807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482143ac-cabf-4b53-872b-fd4eef9464f0 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:29.878310 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 141/301] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:29.878310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a7e76-d8b6-43cc-bf86-fdaaa0c8ea8e None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:29.878310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a7e76-d8b6-43cc-bf86-fdaaa0c8ea8e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:29.878310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a7e76-d8b6-43cc-bf86-fdaaa0c8ea8e None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:29.878310 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 142/302] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:29.880385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a7e76-d8b6-43cc-bf86-fdaaa0c8ea8e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:29.890298 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 161/303] 10.222.0.33 () {60 vars in 1134 bytes} [Sat May 28 20:41:29 2022] POST /identity/v3/groups => generated 229 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:41:29.892178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c593478-5a40-4819-b88a-4bb3532345f5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:29.892632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c593478-5a40-4819-b88a-4bb3532345f5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:29.893074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c593478-5a40-4819-b88a-4bb3532345f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:29.924567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:29.926363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4c593478-5a40-4819-b88a-4bb3532345f5 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:29.928454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4c593478-5a40-4819-b88a-4bb3532345f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:29.999408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a7e76-d8b6-43cc-bf86-fdaaa0c8ea8e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:30.017891 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 162/304] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:30.019552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a2910-640f-4f1c-b1fb-431ddb30bc16 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:30.019836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a2910-640f-4f1c-b1fb-431ddb30bc16 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:30.020827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2a2910-640f-4f1c-b1fb-431ddb30bc16 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:30.033867 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 163/305] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:30.033867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b204e-e3e9-4bfa-9250-066f10999cca None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:30.033867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b204e-e3e9-4bfa-9250-066f10999cca None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:30.033867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b204e-e3e9-4bfa-9250-066f10999cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:30.043365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3594c-e868-4cc0-8df8-d1449dd9e0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:30.044328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3594c-e868-4cc0-8df8-d1449dd9e0b3 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:30.044608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3594c-e868-4cc0-8df8-d1449dd9e0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:30.044889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3594c-e868-4cc0-8df8-d1449dd9e0b3 None admin] PATH_INFO: `/v3/users/14f77a0ac1094e008e5d3a5be3928fb8` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:30.055180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f41b204e-e3e9-4bfa-9250-066f10999cca None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:30.056442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f41b204e-e3e9-4bfa-9250-066f10999cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:30.064764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3594c-e868-4cc0-8df8-d1449dd9e0b3 None admin] RBAC: Authorizing `identity:get_user(user_id=14f77a0ac1094e008e5d3a5be3928fb8)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:30.067263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc3594c-e868-4cc0-8df8-d1449dd9e0b3 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:30.074529 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 143/306] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:29 2022] GET /identity/v3/users/14f77a0ac1094e008e5d3a5be3928fb8 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:30.074529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3594c-e868-4cc0-8df8-d1449dd9e0b3 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:30.136917 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 164/307] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:30.137604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b204e-e3e9-4bfa-9250-066f10999cca None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:30.167709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:30.187208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:30.194618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3594c-e868-4cc0-8df8-d1449dd9e0b3 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:30.229653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2423339-5e99-40bb-9af8-4cb2c40719d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:30.230490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2423339-5e99-40bb-9af8-4cb2c40719d2 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:30.230784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2423339-5e99-40bb-9af8-4cb2c40719d2 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:30.231062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2423339-5e99-40bb-9af8-4cb2c40719d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:30.242250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2423339-5e99-40bb-9af8-4cb2c40719d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:30.246373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b204e-e3e9-4bfa-9250-066f10999cca None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:30.250302 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-f2423339-5e99-40bb-9af8-4cb2c40719d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:30.250302 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:30.250302 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:30.250302 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:30.250302 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:30.250302 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:30.250302 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:30.250302 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:30.250302 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:30.250302 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:30.250302 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:30.250302 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:30.250302 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:30.250302 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:30.250302 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:30.250302 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:30.251182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:30.251182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:30.251182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:30.251182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:30.251182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:30.251182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:30.251182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:30.251182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:30.251182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:30.251182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:30.251182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:30.251182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:30.251182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:30.251182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:30.251182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:30.251182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:30.256989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:30.256989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:30.256989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:30.256989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:30.256989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:30.256989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:30.256989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:30.256989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:30.256989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:30.256989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:30.256989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:30.256989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:30.256989 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 144/308] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:30.256989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2423339-5e99-40bb-9af8-4cb2c40719d2 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:30.287828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b0010-ad19-4cf4-9e0a-843db8d50501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:30.288497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b0010-ad19-4cf4-9e0a-843db8d50501 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:30.288847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b0010-ad19-4cf4-9e0a-843db8d50501 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:30.289065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b0010-ad19-4cf4-9e0a-843db8d50501 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:30.289942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b0010-ad19-4cf4-9e0a-843db8d50501 None admin] RBAC: Adding query filter params (role.id=19a96330fa9044a28cedef74491c0d4c, user.id=14f77a0ac1094e008e5d3a5be3928fb8, scope.system=all) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:30.290264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b0010-ad19-4cf4-9e0a-843db8d50501 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:30.291734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b0010-ad19-4cf4-9e0a-843db8d50501 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:30.297012 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 165/309] 10.222.0.33 () {56 vars in 1308 bytes} [Sat May 28 20:41:30 2022] GET /identity/v3/role_assignments?user.id=14f77a0ac1094e008e5d3a5be3928fb8&scope.system=all&role.id=19a96330fa9044a28cedef74491c0d4c => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:30.311382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b0010-ad19-4cf4-9e0a-843db8d50501 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:30.381117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2423339-5e99-40bb-9af8-4cb2c40719d2 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:30.429161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065dc03-4f74-438f-8f75-5c4b56869710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:30.430796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065dc03-4f74-438f-8f75-5c4b56869710 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:30.431289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065dc03-4f74-438f-8f75-5c4b56869710 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:30.431769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065dc03-4f74-438f-8f75-5c4b56869710 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:30.436788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:30.446915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065dc03-4f74-438f-8f75-5c4b56869710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:30.449548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b0010-ad19-4cf4-9e0a-843db8d50501 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:30.455688 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-f065dc03-4f74-438f-8f75-5c4b56869710 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:30.455688 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:30.455688 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:30.455688 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:30.455688 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:30.455688 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:30.455688 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:30.455688 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:30.455688 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:30.455688 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:30.455688 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:30.455688 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:30.455688 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:30.455688 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:30.455688 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:30.455688 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:30.456632 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:30.456632 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:30.456632 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:30.456632 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:30.456632 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:30.456632 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:30.456632 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:30.456632 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:30.456632 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:30.456632 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:30.456632 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:30.456632 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:30.456632 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:30.456632 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:30.456632 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:30.456632 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:30.457734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:30.457734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:30.457734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:30.457734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:30.457734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:30.457734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:30.457734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:30.457734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:30.457734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:30.457734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:30.457734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:30.457734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:30.457734 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 145/310] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:30.463095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065dc03-4f74-438f-8f75-5c4b56869710 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:30.483266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:30.504103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ebf3a-0b6b-453d-863b-37f7d7f6da09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:30.505004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ebf3a-0b6b-453d-863b-37f7d7f6da09 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:30.505291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ebf3a-0b6b-453d-863b-37f7d7f6da09 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:30.505619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ebf3a-0b6b-453d-863b-37f7d7f6da09 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:30.506459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ebf3a-0b6b-453d-863b-37f7d7f6da09 None admin] RBAC: Adding query filter params (name=member) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:30.506768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ebf3a-0b6b-453d-863b-37f7d7f6da09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:30.509300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ebf3a-0b6b-453d-863b-37f7d7f6da09 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:30.520238 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 166/311] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:30.527095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ebf3a-0b6b-453d-863b-37f7d7f6da09 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:30.580545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065dc03-4f74-438f-8f75-5c4b56869710 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:30.616936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509a4a3-6865-41e2-8e54-025fee23ca52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:30.617931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509a4a3-6865-41e2-8e54-025fee23ca52 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:30.618452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509a4a3-6865-41e2-8e54-025fee23ca52 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:30.618929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509a4a3-6865-41e2-8e54-025fee23ca52 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:30.619874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509a4a3-6865-41e2-8e54-025fee23ca52 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:30.620378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509a4a3-6865-41e2-8e54-025fee23ca52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:30.622134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509a4a3-6865-41e2-8e54-025fee23ca52 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:30.629680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ebf3a-0b6b-453d-863b-37f7d7f6da09 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:30.631552 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 146/312] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:30.636938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509a4a3-6865-41e2-8e54-025fee23ca52 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:30.670883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e723bb-ab3e-4257-bd49-9e091dc6ca07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:30.671686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e723bb-ab3e-4257-bd49-9e091dc6ca07 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:30.671969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e723bb-ab3e-4257-bd49-9e091dc6ca07 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:30.672245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e723bb-ab3e-4257-bd49-9e091dc6ca07 None admin] PATH_INFO: `/v3/users/ffae2718686a42a9b980b3a5b53e01d6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:30.673491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e723bb-ab3e-4257-bd49-9e091dc6ca07 None admin] RBAC: Authorizing `identity:get_user(user_id=ffae2718686a42a9b980b3a5b53e01d6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:30.675082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e723bb-ab3e-4257-bd49-9e091dc6ca07 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:30.681620 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 167/313] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:30 2022] GET /identity/v3/users/ffae2718686a42a9b980b3a5b53e01d6 => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:30.682442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e723bb-ab3e-4257-bd49-9e091dc6ca07 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:30.714276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:30.723957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:30.748085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509a4a3-6865-41e2-8e54-025fee23ca52 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:30.784576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e22c0e-7017-4160-b76c-c912ef93ec1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:30.784909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e723bb-ab3e-4257-bd49-9e091dc6ca07 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:30.786695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e22c0e-7017-4160-b76c-c912ef93ec1d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:30.787171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e22c0e-7017-4160-b76c-c912ef93ec1d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:30.787610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e22c0e-7017-4160-b76c-c912ef93ec1d None admin] PATH_INFO: `/v3/users/231b9c8ff9fd4c87a30301ceaeae1a44` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:30.790786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e22c0e-7017-4160-b76c-c912ef93ec1d None admin] RBAC: Authorizing `identity:get_user(user_id=231b9c8ff9fd4c87a30301ceaeae1a44)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:30.790786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e22c0e-7017-4160-b76c-c912ef93ec1d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:30.793123 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 147/314] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:30 2022] GET /identity/v3/users/231b9c8ff9fd4c87a30301ceaeae1a44 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:30.798051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e22c0e-7017-4160-b76c-c912ef93ec1d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:30.836337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f269ea-4c02-43e5-bf32-c5928a6433ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:30.837203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f269ea-4c02-43e5-bf32-c5928a6433ac None admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:30.837486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f269ea-4c02-43e5-bf32-c5928a6433ac None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:30.837762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f269ea-4c02-43e5-bf32-c5928a6433ac None admin] PATH_INFO: `/v3/system/users/ffae2718686a42a9b980b3a5b53e01d6/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:30.839280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f269ea-4c02-43e5-bf32-c5928a6433ac None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ffae2718686a42a9b980b3a5b53e01d6, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:30.840742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f269ea-4c02-43e5-bf32-c5928a6433ac None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:30.912339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e22c0e-7017-4160-b76c-c912ef93ec1d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:30.936597 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 168/315] 10.222.0.33 () {58 vars in 1334 bytes} [Sat May 28 20:41:30 2022] PUT /identity/v3/system/users/ffae2718686a42a9b980b3a5b53e01d6/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:41:30.952855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe0898-a3f8-45b7-b373-9e9cca34518e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:30.953537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe0898-a3f8-45b7-b373-9e9cca34518e None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:30.953888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe0898-a3f8-45b7-b373-9e9cca34518e None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:30.954175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe0898-a3f8-45b7-b373-9e9cca34518e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:30.972495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:31.036138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe0898-a3f8-45b7-b373-9e9cca34518e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:31.037669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe0898-a3f8-45b7-b373-9e9cca34518e None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:31.151369 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 148/316] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 20:41:31.156834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f269ea-4c02-43e5-bf32-c5928a6433ac None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:31.252581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:31.270233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f269ea-4c02-43e5-bf32-c5928a6433ac None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:31.312578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfba82d-41bf-4a89-9cbf-9d3e92c0bfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:31.313470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfba82d-41bf-4a89-9cbf-9d3e92c0bfe7 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:31.313755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfba82d-41bf-4a89-9cbf-9d3e92c0bfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:31.314051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfba82d-41bf-4a89-9cbf-9d3e92c0bfe7 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:31.315242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfba82d-41bf-4a89-9cbf-9d3e92c0bfe7 None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:31.316688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfba82d-41bf-4a89-9cbf-9d3e92c0bfe7 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:31.319364 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 169/317] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:31 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:31.324883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe0898-a3f8-45b7-b373-9e9cca34518e None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:31.375759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:31.436433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe0898-a3f8-45b7-b373-9e9cca34518e None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:31.475409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d1457-6b40-4db2-95ea-09a4ede1ff07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:31.476450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d1457-6b40-4db2-95ea-09a4ede1ff07 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:31.476938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d1457-6b40-4db2-95ea-09a4ede1ff07 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:31.477454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d1457-6b40-4db2-95ea-09a4ede1ff07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:31.478651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d1457-6b40-4db2-95ea-09a4ede1ff07 None admin] RBAC: Adding query filter params (role.id=19a96330fa9044a28cedef74491c0d4c, scope.project.id=0381e5a4ddde4d9ab4394d1417743579, user.id=231b9c8ff9fd4c87a30301ceaeae1a44) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:31.479350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d1457-6b40-4db2-95ea-09a4ede1ff07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:31.481023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d1457-6b40-4db2-95ea-09a4ede1ff07 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:31.488516 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 149/318] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:41:31 2022] GET /identity/v3/role_assignments?user.id=231b9c8ff9fd4c87a30301ceaeae1a44&scope.project.id=0381e5a4ddde4d9ab4394d1417743579&role.id=19a96330fa9044a28cedef74491c0d4c => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:32.285646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e86412-588a-43c9-92b8-19083206a760 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:32.286723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e86412-588a-43c9-92b8-19083206a760 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:32.286820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e86412-588a-43c9-92b8-19083206a760 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:32.288321 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 170/319] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:32.293374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba9394-3248-4265-b2af-dac479a74bac None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:32.293665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba9394-3248-4265-b2af-dac479a74bac None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:32.293978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ba9394-3248-4265-b2af-dac479a74bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:32.310669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a1ba9394-3248-4265-b2af-dac479a74bac None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:32.311895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a1ba9394-3248-4265-b2af-dac479a74bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:32.380918 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 150/320] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:32.404602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e86412-588a-43c9-92b8-19083206a760 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:32.515308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e86412-588a-43c9-92b8-19083206a760 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:32.550348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e329a5e-7e0b-4cc5-a8dd-66ac4da5e536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:32.551152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e329a5e-7e0b-4cc5-a8dd-66ac4da5e536 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:32.551999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e329a5e-7e0b-4cc5-a8dd-66ac4da5e536 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:32.552451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e329a5e-7e0b-4cc5-a8dd-66ac4da5e536 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:32.553686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e329a5e-7e0b-4cc5-a8dd-66ac4da5e536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:32.555200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e329a5e-7e0b-4cc5-a8dd-66ac4da5e536 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:32.560403 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 171/321] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:32.564121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba9394-3248-4265-b2af-dac479a74bac None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:32.603432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:32.654933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a951b2-9de2-4ba9-9395-dc8d89d20832 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:32.655232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a951b2-9de2-4ba9-9395-dc8d89d20832 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:32.655517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a951b2-9de2-4ba9-9395-dc8d89d20832 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:32.656902 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 172/322] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:41:32.662396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c84cff-efbc-419d-ba92-a9db6f4a058b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:32.663012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c84cff-efbc-419d-ba92-a9db6f4a058b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:32.663407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c84cff-efbc-419d-ba92-a9db6f4a058b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:32.665828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ba9394-3248-4265-b2af-dac479a74bac None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:32.681040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-90c84cff-efbc-419d-ba92-a9db6f4a058b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:32.682398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-90c84cff-efbc-419d-ba92-a9db6f4a058b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:32.697029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:32.702782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e92da7-fe46-4256-bf4e-45191f8af8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:32.703514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e92da7-fe46-4256-bf4e-45191f8af8e1 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:32.703802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e92da7-fe46-4256-bf4e-45191f8af8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:32.704081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e92da7-fe46-4256-bf4e-45191f8af8e1 None admin] PATH_INFO: `/v3/groups` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:32.708220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e92da7-fe46-4256-bf4e-45191f8af8e1 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:32.708220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e92da7-fe46-4256-bf4e-45191f8af8e1 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:32.752574 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 173/323] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:32.777327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c84cff-efbc-419d-ba92-a9db6f4a058b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:32.828903 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 151/324] 10.222.0.33 () {60 vars in 1134 bytes} [Sat May 28 20:41:32 2022] POST /identity/v3/groups => generated 226 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:41:32.887314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c84cff-efbc-419d-ba92-a9db6f4a058b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:32.925571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc1a35-7736-4948-8ec0-27ccecc860ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:32.926947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc1a35-7736-4948-8ec0-27ccecc860ed None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:32.927236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc1a35-7736-4948-8ec0-27ccecc860ed None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:32.927511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc1a35-7736-4948-8ec0-27ccecc860ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:32.939186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc1a35-7736-4948-8ec0-27ccecc860ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:32.947755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-9bfc1a35-7736-4948-8ec0-27ccecc860ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:32.947755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:32.947755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:32.947755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:32.947755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:32.947755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:32.947755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:32.947755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:32.947755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:32.947755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:32.947755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:32.947755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:32.947755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:32.947755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:32.947755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:32.947755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:32.948765 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:32.948765 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:32.948765 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:32.948765 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:32.948765 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:32.948765 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:32.948765 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:32.948765 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:32.948765 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:32.948765 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:32.948765 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:32.948765 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:32.948765 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:32.948765 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:32.948765 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:32.948765 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:32.954788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:32.954788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:32.954788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:32.954788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:32.954788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:32.954788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:32.954788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:32.954788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:32.954788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:32.954788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:32.954788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:32.954788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:32.954788 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 174/325] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:32.955785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e92da7-fe46-4256-bf4e-45191f8af8e1 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:33.056106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:33.063044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e92da7-fe46-4256-bf4e-45191f8af8e1 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:33.103096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb31899-d00f-4bf8-8021-cde17ee0dfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:33.103096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb31899-d00f-4bf8-8021-cde17ee0dfa4 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:33.103096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb31899-d00f-4bf8-8021-cde17ee0dfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:33.103096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb31899-d00f-4bf8-8021-cde17ee0dfa4 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:33.103509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb31899-d00f-4bf8-8021-cde17ee0dfa4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:33.103509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb31899-d00f-4bf8-8021-cde17ee0dfa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:33.103509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb31899-d00f-4bf8-8021-cde17ee0dfa4 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:33.114561 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 152/326] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:33.121752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc1a35-7736-4948-8ec0-27ccecc860ed None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:33.137416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:33.234119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc1a35-7736-4948-8ec0-27ccecc860ed None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:33.271422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799adab-892f-485b-aefc-73a5db604123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:33.272311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799adab-892f-485b-aefc-73a5db604123 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:33.272597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799adab-892f-485b-aefc-73a5db604123 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:33.272871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799adab-892f-485b-aefc-73a5db604123 None admin] PATH_INFO: `/v3/users/14f77a0ac1094e008e5d3a5be3928fb8` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:33.277876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9799adab-892f-485b-aefc-73a5db604123 None admin] RBAC: Authorizing `identity:get_user(user_id=14f77a0ac1094e008e5d3a5be3928fb8)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:33.277876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9799adab-892f-485b-aefc-73a5db604123 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:33.277876 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 175/327] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:33 2022] GET /identity/v3/users/14f77a0ac1094e008e5d3a5be3928fb8 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:33.283745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb31899-d00f-4bf8-8021-cde17ee0dfa4 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:33.308096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d2e7c-97f3-4c56-9a3d-704537759661 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:33.308617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d2e7c-97f3-4c56-9a3d-704537759661 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:33.308713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d2e7c-97f3-4c56-9a3d-704537759661 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:33.310493 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 176/328] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:41:33.316370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96af25-345b-4684-a297-974e8c9ad87c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:33.316667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96af25-345b-4684-a297-974e8c9ad87c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:33.316934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96af25-345b-4684-a297-974e8c9ad87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:33.335171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fc96af25-345b-4684-a297-974e8c9ad87c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:33.336389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fc96af25-345b-4684-a297-974e8c9ad87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:33.424325 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 177/329] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:33.427378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb31899-d00f-4bf8-8021-cde17ee0dfa4 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:33.458076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96af25-345b-4684-a297-974e8c9ad87c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:33.465143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b928e-9a4e-433a-be7f-7b81cb796570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:33.466349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b928e-9a4e-433a-be7f-7b81cb796570 None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:33.466640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b928e-9a4e-433a-be7f-7b81cb796570 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:33.466927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b928e-9a4e-433a-be7f-7b81cb796570 None admin] PATH_INFO: `/v3/system/users/14f77a0ac1094e008e5d3a5be3928fb8/roles/19a96330fa9044a28cedef74491c0d4c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:33.468476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b928e-9a4e-433a-be7f-7b81cb796570 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=14f77a0ac1094e008e5d3a5be3928fb8, role_id=19a96330fa9044a28cedef74491c0d4c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:33.469966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b928e-9a4e-433a-be7f-7b81cb796570 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:33.471592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:33.549197 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 153/330] 10.222.0.33 () {58 vars in 1334 bytes} [Sat May 28 20:41:33 2022] PUT /identity/v3/system/users/14f77a0ac1094e008e5d3a5be3928fb8/roles/19a96330fa9044a28cedef74491c0d4c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:41:33.568899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc96af25-345b-4684-a297-974e8c9ad87c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:33.608621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39f3b4-3c91-4c3c-9da6-e09e151b3ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:33.609828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39f3b4-3c91-4c3c-9da6-e09e151b3ccf None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:33.610350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39f3b4-3c91-4c3c-9da6-e09e151b3ccf None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:33.610818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39f3b4-3c91-4c3c-9da6-e09e151b3ccf None admin] PATH_INFO: `/v3/roles/member` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:33.623925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39f3b4-3c91-4c3c-9da6-e09e151b3ccf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:33.632637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-eb39f3b4-3c91-4c3c-9da6-e09e151b3ccf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:33.632637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:33.632637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:33.632637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:33.632637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:33.632637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:33.632637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:33.632637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:33.632637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:33.632637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:33.632637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:33.632637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:33.632637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:33.632637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:33.632637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:33.632637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:33.633574 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:33.633574 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:33.633574 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:33.633574 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:33.633574 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:33.633574 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:33.633574 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:33.633574 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:33.633574 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:33.633574 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:33.633574 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:33.633574 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:33.633574 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:33.633574 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:33.633574 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:33.633574 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:33.634404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:33.634404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:33.634404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:33.634404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:33.634404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:33.634404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:33.634404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:33.634404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:33.634404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:33.634404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:33.634404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:33.634404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:33.637313 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 178/331] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:33.644005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b928e-9a4e-433a-be7f-7b81cb796570 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:33.675477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:33.765629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b928e-9a4e-433a-be7f-7b81cb796570 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:33.807675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c9ba1-e4af-4b33-b647-34c0740c93ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:33.808638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c9ba1-e4af-4b33-b647-34c0740c93ba None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:33.809071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c9ba1-e4af-4b33-b647-34c0740c93ba None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:33.809553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c9ba1-e4af-4b33-b647-34c0740c93ba None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:33.810498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c9ba1-e4af-4b33-b647-34c0740c93ba None admin] RBAC: Adding query filter params (name=member) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:33.810944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c9ba1-e4af-4b33-b647-34c0740c93ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:33.812579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c9ba1-e4af-4b33-b647-34c0740c93ba None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:33.822469 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 154/332] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:33.828383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39f3b4-3c91-4c3c-9da6-e09e151b3ccf None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:33.899719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:33.935394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39f3b4-3c91-4c3c-9da6-e09e151b3ccf None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:33.970144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa8380-570b-4b85-943b-8f8d82c88e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:33.971031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa8380-570b-4b85-943b-8f8d82c88e36 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:33.971311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa8380-570b-4b85-943b-8f8d82c88e36 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:33.971586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa8380-570b-4b85-943b-8f8d82c88e36 None admin] PATH_INFO: `/v3/users/ffae2718686a42a9b980b3a5b53e01d6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:33.972875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa8380-570b-4b85-943b-8f8d82c88e36 None admin] RBAC: Authorizing `identity:get_user(user_id=ffae2718686a42a9b980b3a5b53e01d6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:33.975023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa8380-570b-4b85-943b-8f8d82c88e36 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:33.976690 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 179/333] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:33 2022] GET /identity/v3/users/ffae2718686a42a9b980b3a5b53e01d6 => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:33.982422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c9ba1-e4af-4b33-b647-34c0740c93ba None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:34.085017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:34.102067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c9ba1-e4af-4b33-b647-34c0740c93ba None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:34.144900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351db5f-d391-491f-a9a2-0bc92dee1f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:34.146142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351db5f-d391-491f-a9a2-0bc92dee1f20 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:34.146431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351db5f-d391-491f-a9a2-0bc92dee1f20 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:34.146712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351db5f-d391-491f-a9a2-0bc92dee1f20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:34.147561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351db5f-d391-491f-a9a2-0bc92dee1f20 None admin] RBAC: Adding query filter params (role.id=c0c09ba97c694896ae14fceb6982b1a5, user.id=ffae2718686a42a9b980b3a5b53e01d6, scope.system=all) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:34.147876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351db5f-d391-491f-a9a2-0bc92dee1f20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:34.149353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351db5f-d391-491f-a9a2-0bc92dee1f20 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:34.162691 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 155/334] 10.222.0.33 () {56 vars in 1308 bytes} [Sat May 28 20:41:33 2022] GET /identity/v3/role_assignments?user.id=ffae2718686a42a9b980b3a5b53e01d6&scope.system=all&role.id=c0c09ba97c694896ae14fceb6982b1a5 => generated 510 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:35.774817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4c2ee-5e4e-48b5-b041-f5d0343333f6 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:35.775364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4c2ee-5e4e-48b5-b041-f5d0343333f6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:35.775364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4c2ee-5e4e-48b5-b041-f5d0343333f6 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:35.777089 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 180/335] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:35.782983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e0b5f-b688-4e77-a69e-59a6e50b4c40 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:35.783470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e0b5f-b688-4e77-a69e-59a6e50b4c40 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:35.783548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e0b5f-b688-4e77-a69e-59a6e50b4c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:35.804558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4b4e0b5f-b688-4e77-a69e-59a6e50b4c40 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:35.805774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4b4e0b5f-b688-4e77-a69e-59a6e50b4c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:35.884219 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 156/336] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:41:35.925894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4c2ee-5e4e-48b5-b041-f5d0343333f6 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:36.037459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4c2ee-5e4e-48b5-b041-f5d0343333f6 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:36.078009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a887a-fbe3-495f-9c61-643bbea412d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:36.078822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a887a-fbe3-495f-9c61-643bbea412d7 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:36.079108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a887a-fbe3-495f-9c61-643bbea412d7 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:36.079383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a887a-fbe3-495f-9c61-643bbea412d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:36.088533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:36.102081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a887a-fbe3-495f-9c61-643bbea412d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:36.117486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-088a887a-fbe3-495f-9c61-643bbea412d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:36.117486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:36.117486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:36.117486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:36.117486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:36.117486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:36.117486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:36.117486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:36.117486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:36.117486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:36.117486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:36.117486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:36.117486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:36.117486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:36.117486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:36.117486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:36.118400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:36.118400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:36.118400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:36.118400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:36.118400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:36.118400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:36.118400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:36.118400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:36.118400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:36.118400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:36.118400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:36.118400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:36.118400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:36.118400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:36.118400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:36.118400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:36.119201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:36.119201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:36.119201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:36.119201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:36.119201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:36.119201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:36.119201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:36.119201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:36.119201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:36.119201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:36.119201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:36.119201 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:36.119201 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 181/337] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:36.119853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e0b5f-b688-4e77-a69e-59a6e50b4c40 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:36.155065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc9eec-f92d-481d-8617-5b722dafef0b None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:36.155717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc9eec-f92d-481d-8617-5b722dafef0b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:36.156248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc9eec-f92d-481d-8617-5b722dafef0b None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:36.158117 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 182/338] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:36.164011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba439f-9bf9-4232-9b68-a860f9b55aea None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:36.164579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba439f-9bf9-4232-9b68-a860f9b55aea None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:36.165150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bba439f-9bf9-4232-9b68-a860f9b55aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:36.183773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5bba439f-9bf9-4232-9b68-a860f9b55aea None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:36.185312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5bba439f-9bf9-4232-9b68-a860f9b55aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:36.228506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:36.272648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e0b5f-b688-4e77-a69e-59a6e50b4c40 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:36.282291 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 183/339] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:41:36.314661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba439f-9bf9-4232-9b68-a860f9b55aea None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:36.346540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff368a6c-98a5-4e42-8eaf-4ad458c9ac58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:36.347228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff368a6c-98a5-4e42-8eaf-4ad458c9ac58 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:36.347517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff368a6c-98a5-4e42-8eaf-4ad458c9ac58 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:36.347801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff368a6c-98a5-4e42-8eaf-4ad458c9ac58 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:36.349118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff368a6c-98a5-4e42-8eaf-4ad458c9ac58 None admin] RBAC: Adding query filter params (name=member) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:36.349429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff368a6c-98a5-4e42-8eaf-4ad458c9ac58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:36.350865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff368a6c-98a5-4e42-8eaf-4ad458c9ac58 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:36.368133 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 157/340] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:36.373466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff368a6c-98a5-4e42-8eaf-4ad458c9ac58 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:36.491552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bba439f-9bf9-4232-9b68-a860f9b55aea None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:36.544859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff368a6c-98a5-4e42-8eaf-4ad458c9ac58 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:36.558196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:36.592495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41613f2-0388-4b44-9381-dc0afc781749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:36.593205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41613f2-0388-4b44-9381-dc0afc781749 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:36.593474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41613f2-0388-4b44-9381-dc0afc781749 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:36.593751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41613f2-0388-4b44-9381-dc0afc781749 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:36.594140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b059cf-eb81-45e8-9362-2fd1d4b01b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:36.594802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b059cf-eb81-45e8-9362-2fd1d4b01b29 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:36.595092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b059cf-eb81-45e8-9362-2fd1d4b01b29 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:36.595371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b059cf-eb81-45e8-9362-2fd1d4b01b29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:36.600442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:36.621019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41613f2-0388-4b44-9381-dc0afc781749 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:36.633451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-c41613f2-0388-4b44-9381-dc0afc781749 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:36.633451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:36.633451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:36.633451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:36.633451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:36.633451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:36.633451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:36.633451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:36.633451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:36.633451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:36.633451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:36.633451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:36.633451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:36.633451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:36.633451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:36.633451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:36.634444 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:36.634444 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:36.634444 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:36.634444 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:36.634444 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:36.634444 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:36.634444 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:36.634444 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:36.634444 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:36.634444 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:36.634444 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:36.634444 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:36.634444 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:36.634444 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:36.634444 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:36.634444 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:36.635246 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:36.635246 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:36.635246 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:36.635246 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:36.635246 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:36.635246 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:36.635246 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:36.635246 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:36.635246 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:36.635246 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:36.635246 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:36.635246 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:36.642037 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 184/341] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:41:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:36.643639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41613f2-0388-4b44-9381-dc0afc781749 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:36.698736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b059cf-eb81-45e8-9362-2fd1d4b01b29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:36.700391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b059cf-eb81-45e8-9362-2fd1d4b01b29 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:36.819187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41613f2-0388-4b44-9381-dc0afc781749 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:36.849343 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 158/342] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:36.855159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b059cf-eb81-45e8-9362-2fd1d4b01b29 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:36.864114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be996c-cb99-48ec-9bd4-4aa3597b2fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:36.865127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be996c-cb99-48ec-9bd4-4aa3597b2fa3 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:36.865611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be996c-cb99-48ec-9bd4-4aa3597b2fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:36.866159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be996c-cb99-48ec-9bd4-4aa3597b2fa3 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:36.867137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be996c-cb99-48ec-9bd4-4aa3597b2fa3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:36.867643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be996c-cb99-48ec-9bd4-4aa3597b2fa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:36.869213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be996c-cb99-48ec-9bd4-4aa3597b2fa3 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:36.878558 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 185/343] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:41:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:36.883690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be996c-cb99-48ec-9bd4-4aa3597b2fa3 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:36.914310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:36.940756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:36.996788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be996c-cb99-48ec-9bd4-4aa3597b2fa3 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:37.012172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b059cf-eb81-45e8-9362-2fd1d4b01b29 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:37.035862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33875aa2-0fd6-45d3-a826-0d8ff52b72e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:37.036669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33875aa2-0fd6-45d3-a826-0d8ff52b72e6 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:37.037030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33875aa2-0fd6-45d3-a826-0d8ff52b72e6 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:37.037374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33875aa2-0fd6-45d3-a826-0d8ff52b72e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:37.080499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e31141-ad48-42bc-9498-fd29317c203c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:37.081888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e31141-ad48-42bc-9498-fd29317c203c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:37.082390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e31141-ad48-42bc-9498-fd29317c203c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:37.083435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e31141-ad48-42bc-9498-fd29317c203c None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:37.088309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e31141-ad48-42bc-9498-fd29317c203c None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:37.090052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e31141-ad48-42bc-9498-fd29317c203c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:37.091998 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 159/344] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:36 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:37.097467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e31141-ad48-42bc-9498-fd29317c203c None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:37.114970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33875aa2-0fd6-45d3-a826-0d8ff52b72e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:37.116391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33875aa2-0fd6-45d3-a826-0d8ff52b72e6 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:37.168143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:37.225311 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 186/345] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 20:41:37.230881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33875aa2-0fd6-45d3-a826-0d8ff52b72e6 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:37.244035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e31141-ad48-42bc-9498-fd29317c203c None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:37.266356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:37.295585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21ae168-99ea-4426-87f2-bced73673e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:37.296806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ae168-99ea-4426-87f2-bced73673e25 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:37.297321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ae168-99ea-4426-87f2-bced73673e25 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:37.297912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21ae168-99ea-4426-87f2-bced73673e25 None admin] PATH_INFO: `/v3/groups/28500a8dd46844218ebe85f28b5287d2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:37.308766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ae168-99ea-4426-87f2-bced73673e25 None admin] RBAC: Authorizing `identity:get_group(group_id=28500a8dd46844218ebe85f28b5287d2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:37.310631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21ae168-99ea-4426-87f2-bced73673e25 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:37.312564 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 160/346] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 20:41:37 2022] GET /identity/v3/groups/28500a8dd46844218ebe85f28b5287d2 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:37.314029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0668cfb-88a5-47bf-bbdc-7d98cdc56d61 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:37.314541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0668cfb-88a5-47bf-bbdc-7d98cdc56d61 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:37.315022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0668cfb-88a5-47bf-bbdc-7d98cdc56d61 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:37.316509 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 161/347] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:37.322524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eecd5e-b295-498a-a78a-cebd4b5ea194 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:37.323087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eecd5e-b295-498a-a78a-cebd4b5ea194 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:37.323574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eecd5e-b295-498a-a78a-cebd4b5ea194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:37.341951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f8eecd5e-b295-498a-a78a-cebd4b5ea194 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:37.342542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33875aa2-0fd6-45d3-a826-0d8ff52b72e6 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:37.343689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f8eecd5e-b295-498a-a78a-cebd4b5ea194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:37.382558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797d68c-ddc3-48ba-96e8-8e6e95448361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:37.382558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797d68c-ddc3-48ba-96e8-8e6e95448361 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:37.382558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797d68c-ddc3-48ba-96e8-8e6e95448361 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:37.382558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797d68c-ddc3-48ba-96e8-8e6e95448361 None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:37.383094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797d68c-ddc3-48ba-96e8-8e6e95448361 None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:37.384435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797d68c-ddc3-48ba-96e8-8e6e95448361 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:37.389967 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 187/348] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:37 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:37.391302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797d68c-ddc3-48ba-96e8-8e6e95448361 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:37.423975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:37.425036 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 162/349] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:37.426157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eecd5e-b295-498a-a78a-cebd4b5ea194 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:37.547481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eecd5e-b295-498a-a78a-cebd4b5ea194 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:37.551041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:37.587396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b0394-d3b6-449a-8465-8390dd479c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:37.588802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b0394-d3b6-449a-8465-8390dd479c05 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:37.589681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b0394-d3b6-449a-8465-8390dd479c05 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:37.590310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b0394-d3b6-449a-8465-8390dd479c05 None admin] PATH_INFO: `/v3/groups/28500a8dd46844218ebe85f28b5287d2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:37.591815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b0394-d3b6-449a-8465-8390dd479c05 None admin] RBAC: Authorizing `identity:get_group(group_id=28500a8dd46844218ebe85f28b5287d2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:37.593480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b0394-d3b6-449a-8465-8390dd479c05 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:37.596657 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 163/350] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 20:41:37 2022] GET /identity/v3/groups/28500a8dd46844218ebe85f28b5287d2 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:37.597690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b0394-d3b6-449a-8465-8390dd479c05 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:37.626122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797d68c-ddc3-48ba-96e8-8e6e95448361 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:37.682094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cabd4f-5929-42bb-ad61-6ec6fd813f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:37.682682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cabd4f-5929-42bb-ad61-6ec6fd813f79 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:37.683018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cabd4f-5929-42bb-ad61-6ec6fd813f79 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:37.683294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cabd4f-5929-42bb-ad61-6ec6fd813f79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:37.684220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cabd4f-5929-42bb-ad61-6ec6fd813f79 None admin] RBAC: Adding query filter params (group.id=28500a8dd46844218ebe85f28b5287d2, role.id=c0c09ba97c694896ae14fceb6982b1a5, scope.project.id=e872f1acf4e544f6963d85f6afc13e1c) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:37.684446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cabd4f-5929-42bb-ad61-6ec6fd813f79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:37.685966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cabd4f-5929-42bb-ad61-6ec6fd813f79 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:37.697213 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 188/351] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 20:41:37 2022] GET /identity/v3/role_assignments?group.id=28500a8dd46844218ebe85f28b5287d2&scope.project.id=e872f1acf4e544f6963d85f6afc13e1c&role.id=c0c09ba97c694896ae14fceb6982b1a5 => generated 260 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:37.698261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cabd4f-5929-42bb-ad61-6ec6fd813f79 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:37.732489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b0394-d3b6-449a-8465-8390dd479c05 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:37.798325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:37.818881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5547c8-fe1e-49e3-8c72-8edec41cd23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:37.819702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5547c8-fe1e-49e3-8c72-8edec41cd23b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:37.819981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5547c8-fe1e-49e3-8c72-8edec41cd23b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:37.820262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5547c8-fe1e-49e3-8c72-8edec41cd23b None admin] PATH_INFO: `/v3/roles/member` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:37.822838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cabd4f-5929-42bb-ad61-6ec6fd813f79 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:37.848028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5547c8-fe1e-49e3-8c72-8edec41cd23b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:37.856319 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-dc5547c8-fe1e-49e3-8c72-8edec41cd23b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:37.856319 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:37.856319 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:37.856319 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:37.856319 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:37.856319 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:37.856319 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:37.856319 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:37.856319 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:37.856319 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:37.856319 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:37.856319 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:37.856319 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:37.856319 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:37.856319 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:37.856319 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:37.857717 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:37.857717 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:37.857717 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:37.857717 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:37.857717 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:37.857717 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:37.857717 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:37.857717 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:37.857717 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:37.857717 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:37.857717 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:37.857717 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:37.857717 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:37.857717 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:37.857717 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:37.857717 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:37.858588 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:37.858588 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:37.858588 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:37.858588 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:37.858588 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:37.858588 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:37.858588 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:37.858588 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:37.858588 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:37.858588 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:37.858588 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:37.858588 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:37.863189 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 164/352] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:37.864598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d0ca7-f0c4-4644-befd-c1f3bff437ed None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:37.864884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d0ca7-f0c4-4644-befd-c1f3bff437ed None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:37.865160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d0ca7-f0c4-4644-befd-c1f3bff437ed None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:37.865478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599feed-bc3d-447a-b302-77a63fdd4c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:37.866178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599feed-bc3d-447a-b302-77a63fdd4c6c None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:37.866621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599feed-bc3d-447a-b302-77a63fdd4c6c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:37.866736 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 165/353] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:37.866821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a599feed-bc3d-447a-b302-77a63fdd4c6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:37.867569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599feed-bc3d-447a-b302-77a63fdd4c6c None admin] RBAC: Adding query filter params (group.id=28500a8dd46844218ebe85f28b5287d2, role.id=92b887e453ad4fa9aa5c49f7594a7e53, scope.project.id=e872f1acf4e544f6963d85f6afc13e1c) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:37.867886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599feed-bc3d-447a-b302-77a63fdd4c6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:37.869379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a599feed-bc3d-447a-b302-77a63fdd4c6c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:37.875988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e17f4-b46b-4486-a621-c22d30fc22b1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:37.876327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e17f4-b46b-4486-a621-c22d30fc22b1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:37.876537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65e17f4-b46b-4486-a621-c22d30fc22b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:37.876785 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 189/354] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 20:41:37 2022] GET /identity/v3/role_assignments?group.id=28500a8dd46844218ebe85f28b5287d2&scope.project.id=e872f1acf4e544f6963d85f6afc13e1c&role.id=92b887e453ad4fa9aa5c49f7594a7e53 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:37.882398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599feed-bc3d-447a-b302-77a63fdd4c6c None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:37.918846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a65e17f4-b46b-4486-a621-c22d30fc22b1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:37.920120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a65e17f4-b46b-4486-a621-c22d30fc22b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:37.966419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:38.076058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a599feed-bc3d-447a-b302-77a63fdd4c6c None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:38.084159 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 166/355] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 28 20:41:38.111516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e17f4-b46b-4486-a621-c22d30fc22b1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:38.168954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df191e0-5035-4cc2-9650-c35dd7d7c587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:38.169646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df191e0-5035-4cc2-9650-c35dd7d7c587 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:38.170012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df191e0-5035-4cc2-9650-c35dd7d7c587 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:38.170297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df191e0-5035-4cc2-9650-c35dd7d7c587 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:38.171060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df191e0-5035-4cc2-9650-c35dd7d7c587 None admin] RBAC: Adding query filter params (name=member) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:38.171370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df191e0-5035-4cc2-9650-c35dd7d7c587 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:38.172774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df191e0-5035-4cc2-9650-c35dd7d7c587 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:38.194647 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 190/356] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:38.195816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70beb1f-dae5-4ccd-867b-aa9968e4a701 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:38.196106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70beb1f-dae5-4ccd-867b-aa9968e4a701 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:38.196384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70beb1f-dae5-4ccd-867b-aa9968e4a701 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:38.197784 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 191/357] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:38.208330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4587a5-6639-47cb-ad65-a01590bc1d85 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:38.208943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4587a5-6639-47cb-ad65-a01590bc1d85 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:38.209461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4587a5-6639-47cb-ad65-a01590bc1d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:38.222528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:38.227834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2d4587a5-6639-47cb-ad65-a01590bc1d85 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:38.229096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2d4587a5-6639-47cb-ad65-a01590bc1d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:38.307200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65e17f4-b46b-4486-a621-c22d30fc22b1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:38.327323 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 192/358] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:38.334333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4587a5-6639-47cb-ad65-a01590bc1d85 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:38.365121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50800e07-e926-4490-9142-7c8d5e4e0187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:38.365954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50800e07-e926-4490-9142-7c8d5e4e0187 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:38.366241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50800e07-e926-4490-9142-7c8d5e4e0187 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:38.366520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50800e07-e926-4490-9142-7c8d5e4e0187 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:38.381800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50800e07-e926-4490-9142-7c8d5e4e0187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:38.389602 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-50800e07-e926-4490-9142-7c8d5e4e0187 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:38.389602 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:38.389602 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:38.389602 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:38.389602 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:38.389602 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:38.389602 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:38.389602 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:38.389602 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:38.389602 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:38.389602 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:38.389602 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:38.389602 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:38.389602 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:38.389602 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:38.389602 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:38.390512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:38.390512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:38.390512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:38.390512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:38.390512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:38.390512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:38.390512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:38.390512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:38.390512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:38.390512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:38.390512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:38.390512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:38.390512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:38.390512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:38.390512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:38.390512 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:38.391695 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:38.391695 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:38.391695 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:38.391695 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:38.391695 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:38.391695 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:38.391695 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:38.391695 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:38.391695 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:38.391695 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:38.391695 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 28 20:41:38.391695 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:38.391695 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 167/359] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:41:38.392945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f701cd-afb3-4468-a781-e23d720bed04 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:38.393234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f701cd-afb3-4468-a781-e23d720bed04 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:38.393510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f701cd-afb3-4468-a781-e23d720bed04 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:38.395753 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 168/360] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:38.402640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f701cd-afb3-4468-a781-e23d720bed04 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:38.429420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:38.455361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4587a5-6639-47cb-ad65-a01590bc1d85 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:38.502271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b1164-e9a2-40f0-a096-49762ddbcc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:38.502852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b1164-e9a2-40f0-a096-49762ddbcc95 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:38.503134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b1164-e9a2-40f0-a096-49762ddbcc95 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:38.503408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b1164-e9a2-40f0-a096-49762ddbcc95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:38.509409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:38.532500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f701cd-afb3-4468-a781-e23d720bed04 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:38.581282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9836c-28a5-49f8-9aa0-4f99579bd095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:38.582120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9836c-28a5-49f8-9aa0-4f99579bd095 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:38.582409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9836c-28a5-49f8-9aa0-4f99579bd095 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:38.583277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9836c-28a5-49f8-9aa0-4f99579bd095 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:38.584053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9836c-28a5-49f8-9aa0-4f99579bd095 None admin] RBAC: Adding query filter params (name=reader) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:38.584483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9836c-28a5-49f8-9aa0-4f99579bd095 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:38.585914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9836c-28a5-49f8-9aa0-4f99579bd095 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:38.597565 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 169/361] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:38.599812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b1164-e9a2-40f0-a096-49762ddbcc95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:38.601726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b1164-e9a2-40f0-a096-49762ddbcc95 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:38.603903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719eaeb1-e7fd-4798-92b9-baf8c463925f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:38.604325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719eaeb1-e7fd-4798-92b9-baf8c463925f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:38.604616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719eaeb1-e7fd-4798-92b9-baf8c463925f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:38.624261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-719eaeb1-e7fd-4798-92b9-baf8c463925f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:38.625501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-719eaeb1-e7fd-4798-92b9-baf8c463925f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:38.711245 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 170/362] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:38.712113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719eaeb1-e7fd-4798-92b9-baf8c463925f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:38.731297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:38.764179 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 193/363] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:38.765003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b1164-e9a2-40f0-a096-49762ddbcc95 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:38.811463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:38.867820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719eaeb1-e7fd-4798-92b9-baf8c463925f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:38.893010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b1164-e9a2-40f0-a096-49762ddbcc95 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:38.916052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167318e-1769-40ec-b2cf-631c7d080f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:38.917799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167318e-1769-40ec-b2cf-631c7d080f23 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:38.918352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167318e-1769-40ec-b2cf-631c7d080f23 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:38.918777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167318e-1769-40ec-b2cf-631c7d080f23 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:38.931410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167318e-1769-40ec-b2cf-631c7d080f23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:38.936272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0960d2-0ac5-4afd-96b8-8624630dbf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:38.937158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0960d2-0ac5-4afd-96b8-8624630dbf1d None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:38.937438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0960d2-0ac5-4afd-96b8-8624630dbf1d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:38.937714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0960d2-0ac5-4afd-96b8-8624630dbf1d None admin] PATH_INFO: `/v3/users/14f77a0ac1094e008e5d3a5be3928fb8` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:38.939029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0960d2-0ac5-4afd-96b8-8624630dbf1d None admin] RBAC: Authorizing `identity:get_user(user_id=14f77a0ac1094e008e5d3a5be3928fb8)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:38.940472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0960d2-0ac5-4afd-96b8-8624630dbf1d None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:38.944055 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 194/364] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:41:38 2022] GET /identity/v3/users/14f77a0ac1094e008e5d3a5be3928fb8 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:38.944198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0960d2-0ac5-4afd-96b8-8624630dbf1d None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:38.951681 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-f167318e-1769-40ec-b2cf-631c7d080f23 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:38.951681 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:38.951681 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:38.951681 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:38.951681 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:38.951681 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:38.951681 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:38.951681 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:38.951681 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:38.951681 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:38.951681 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:38.951681 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:38.951681 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:38.951681 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:38.951681 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:38.951681 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:38.957487 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:38.957487 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:38.957487 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:38.957487 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:38.957487 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:38.957487 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:38.957487 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:38.957487 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:38.957487 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:38.957487 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:38.957487 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:38.957487 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:38.957487 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:38.957487 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:38.957487 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:38.957487 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:38.958366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:38.958366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:38.958366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:38.958366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:38.958366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:38.958366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:38.958366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:38.958366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:38.958366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:38.958366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:38.958366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:38.958366 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:38.958366 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 171/365] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:41:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:38.958366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167318e-1769-40ec-b2cf-631c7d080f23 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:39.050693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0960d2-0ac5-4afd-96b8-8624630dbf1d None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:39.054230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:39.083807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167318e-1769-40ec-b2cf-631c7d080f23 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:39.103342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e98161-c62e-4d1f-a89a-bd63547dcc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:39.104479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e98161-c62e-4d1f-a89a-bd63547dcc2f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:39.104956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e98161-c62e-4d1f-a89a-bd63547dcc2f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:39.105424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e98161-c62e-4d1f-a89a-bd63547dcc2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:39.107731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:39.116831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e98161-c62e-4d1f-a89a-bd63547dcc2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:39.124607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f762f02-bda9-4cc8-a363-8485b2c26307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:39.124607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f762f02-bda9-4cc8-a363-8485b2c26307 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:39.124607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f762f02-bda9-4cc8-a363-8485b2c26307 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:39.124607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f762f02-bda9-4cc8-a363-8485b2c26307 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:39.124913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f762f02-bda9-4cc8-a363-8485b2c26307 None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:39.125612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f762f02-bda9-4cc8-a363-8485b2c26307 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:39.127090 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-77e98161-c62e-4d1f-a89a-bd63547dcc2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:41:39.127090 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:39.127090 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:39.127090 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:39.127090 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:39.127090 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:39.127090 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:39.127090 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:39.127090 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:39.127090 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:39.127090 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:39.127090 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:39.127090 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:39.127090 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:39.127090 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:39.127090 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:39.127946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:39.127946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:39.127946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:39.127946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:39.127946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:39.127946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:39.127946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:39.127946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:39.127946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:39.127946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:39.127946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:39.127946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:39.127946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:39.127946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:39.127946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:39.127946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:39.128734 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:39.128734 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:39.128734 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:39.128734 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:39.128734 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:39.128734 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:39.128734 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:39.128734 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:39.128734 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:39.128734 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:39.128734 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:41:39.128734 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:39.129333 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 172/366] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:38 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:39.129333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f762f02-bda9-4cc8-a363-8485b2c26307 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:39.131480 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 195/367] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:41:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:39.132407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e98161-c62e-4d1f-a89a-bd63547dcc2f None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:39.265442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f762f02-bda9-4cc8-a363-8485b2c26307 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:39.266309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e98161-c62e-4d1f-a89a-bd63547dcc2f None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:39.314206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2cf09-bef6-44cb-8d0a-9e4d76590210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:39.314799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2cf09-bef6-44cb-8d0a-9e4d76590210 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:39.315086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2cf09-bef6-44cb-8d0a-9e4d76590210 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:39.315366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2cf09-bef6-44cb-8d0a-9e4d76590210 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:39.316220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2cf09-bef6-44cb-8d0a-9e4d76590210 None admin] RBAC: Adding query filter params (role.id=19a96330fa9044a28cedef74491c0d4c, user.id=14f77a0ac1094e008e5d3a5be3928fb8, scope.system=all) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:39.316536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2cf09-bef6-44cb-8d0a-9e4d76590210 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:39.318574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2cf09-bef6-44cb-8d0a-9e4d76590210 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:39.322574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:39.324242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592d2b0-62fe-45be-b1eb-5854940642ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:39.325393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592d2b0-62fe-45be-b1eb-5854940642ad None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:39.326802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592d2b0-62fe-45be-b1eb-5854940642ad None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:39.327404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592d2b0-62fe-45be-b1eb-5854940642ad None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:39.329966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592d2b0-62fe-45be-b1eb-5854940642ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:39.336320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592d2b0-62fe-45be-b1eb-5854940642ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:39.336320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592d2b0-62fe-45be-b1eb-5854940642ad None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:39.336522 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 173/368] 10.222.0.33 () {56 vars in 1308 bytes} [Sat May 28 20:41:39 2022] GET /identity/v3/role_assignments?user.id=14f77a0ac1094e008e5d3a5be3928fb8&scope.system=all&role.id=19a96330fa9044a28cedef74491c0d4c => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:39.341862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2cf09-bef6-44cb-8d0a-9e4d76590210 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:39.354698 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 196/369] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:41:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:39.356331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592d2b0-62fe-45be-b1eb-5854940642ad None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:39.411820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:39.503078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592d2b0-62fe-45be-b1eb-5854940642ad None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:39.522473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2cf09-bef6-44cb-8d0a-9e4d76590210 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:39.540286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e25cfc-6330-4427-8f96-7aad397331dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:39.540967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e25cfc-6330-4427-8f96-7aad397331dc None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:39.541255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e25cfc-6330-4427-8f96-7aad397331dc None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:39.541552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e25cfc-6330-4427-8f96-7aad397331dc None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:39.542433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e25cfc-6330-4427-8f96-7aad397331dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:39.542747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e25cfc-6330-4427-8f96-7aad397331dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:39.544203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e25cfc-6330-4427-8f96-7aad397331dc None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:39.555913 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 197/370] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:39.557018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e25cfc-6330-4427-8f96-7aad397331dc None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:39.581682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a009a3-7dfc-4246-b69c-9cd2e32076a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:39.583062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a009a3-7dfc-4246-b69c-9cd2e32076a2 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:39.583562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a009a3-7dfc-4246-b69c-9cd2e32076a2 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:39.584038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a009a3-7dfc-4246-b69c-9cd2e32076a2 None admin] PATH_INFO: `/v3/groups/28500a8dd46844218ebe85f28b5287d2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:39.585437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a009a3-7dfc-4246-b69c-9cd2e32076a2 None admin] RBAC: Authorizing `identity:get_group(group_id=28500a8dd46844218ebe85f28b5287d2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:39.587357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a009a3-7dfc-4246-b69c-9cd2e32076a2 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:39.589357 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 174/371] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 20:41:39 2022] GET /identity/v3/groups/28500a8dd46844218ebe85f28b5287d2 => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:39.590346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a009a3-7dfc-4246-b69c-9cd2e32076a2 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:39.662257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:39.674632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e25cfc-6330-4427-8f96-7aad397331dc None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:39.686880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:39.713044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314552f3-d775-4e80-a553-725ad79922bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:39.713724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314552f3-d775-4e80-a553-725ad79922bf None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:39.714588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314552f3-d775-4e80-a553-725ad79922bf None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:39.714875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314552f3-d775-4e80-a553-725ad79922bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:39.733518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a009a3-7dfc-4246-b69c-9cd2e32076a2 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:39.774903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8453cea-9a96-43d7-ae03-b00487cfa427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:39.775897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8453cea-9a96-43d7-ae03-b00487cfa427 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:39.776363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8453cea-9a96-43d7-ae03-b00487cfa427 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:39.776819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8453cea-9a96-43d7-ae03-b00487cfa427 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:39.791889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314552f3-d775-4e80-a553-725ad79922bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:39.793381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314552f3-d775-4e80-a553-725ad79922bf None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:39.855383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8453cea-9a96-43d7-ae03-b00487cfa427 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:39.857265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8453cea-9a96-43d7-ae03-b00487cfa427 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:39.885416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:39.900112 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 198/372] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:39.901044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314552f3-d775-4e80-a553-725ad79922bf None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:39.929296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:39.975102 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 175/373] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:39.976244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8453cea-9a96-43d7-ae03-b00487cfa427 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:40.014303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314552f3-d775-4e80-a553-725ad79922bf None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:40.059773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111764d6-7708-4b68-be1c-eed3f95789b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:40.059773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111764d6-7708-4b68-be1c-eed3f95789b7 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:40.059773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111764d6-7708-4b68-be1c-eed3f95789b7 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:40.059773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111764d6-7708-4b68-be1c-eed3f95789b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:40.060162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111764d6-7708-4b68-be1c-eed3f95789b7 None admin] RBAC: Adding query filter params (group.id=28500a8dd46844218ebe85f28b5287d2, role.id=c0c09ba97c694896ae14fceb6982b1a5, scope.project.id=0381e5a4ddde4d9ab4394d1417743579) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:40.060162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111764d6-7708-4b68-be1c-eed3f95789b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:40.061508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111764d6-7708-4b68-be1c-eed3f95789b7 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:40.069393 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 199/374] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 20:41:39 2022] GET /identity/v3/role_assignments?group.id=28500a8dd46844218ebe85f28b5287d2&scope.project.id=0381e5a4ddde4d9ab4394d1417743579&role.id=c0c09ba97c694896ae14fceb6982b1a5 => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:40.074803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111764d6-7708-4b68-be1c-eed3f95789b7 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:40.132433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8453cea-9a96-43d7-ae03-b00487cfa427 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:40.184078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce2d35-a040-48b9-87c9-b2781b63733d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:40.185205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce2d35-a040-48b9-87c9-b2781b63733d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:40.185494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce2d35-a040-48b9-87c9-b2781b63733d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:40.185774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ce2d35-a040-48b9-87c9-b2781b63733d None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:40.190120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce2d35-a040-48b9-87c9-b2781b63733d None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:40.191641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ce2d35-a040-48b9-87c9-b2781b63733d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:40.197004 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 176/375] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:39 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:40.201521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce2d35-a040-48b9-87c9-b2781b63733d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:40.208214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:40.251921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:40.309211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111764d6-7708-4b68-be1c-eed3f95789b7 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:40.370630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ce2d35-a040-48b9-87c9-b2781b63733d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:40.388935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c7634-2b1f-4d98-b550-70815790654c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:40.389739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c7634-2b1f-4d98-b550-70815790654c None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:40.390361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c7634-2b1f-4d98-b550-70815790654c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:40.390641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c7634-2b1f-4d98-b550-70815790654c None admin] PATH_INFO: `/v3/projects/a7d6b819de6044b98f2c78098814b8f2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:40.394319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c7634-2b1f-4d98-b550-70815790654c None admin] RBAC: Authorizing `identity:get_project(project_id=a7d6b819de6044b98f2c78098814b8f2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:40.396949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c7634-2b1f-4d98-b550-70815790654c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:40.399157 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 200/376] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:40 2022] GET /identity/v3/projects/a7d6b819de6044b98f2c78098814b8f2 => generated 344 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:40.404708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c7634-2b1f-4d98-b550-70815790654c None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:40.417303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52958afc-2edd-41de-b5a7-de95adc24a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:40.418650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52958afc-2edd-41de-b5a7-de95adc24a4b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:40.418941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52958afc-2edd-41de-b5a7-de95adc24a4b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:40.419219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52958afc-2edd-41de-b5a7-de95adc24a4b None admin] PATH_INFO: `/v3/groups/28500a8dd46844218ebe85f28b5287d2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:40.421548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52958afc-2edd-41de-b5a7-de95adc24a4b None admin] RBAC: Authorizing `identity:get_group(group_id=28500a8dd46844218ebe85f28b5287d2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:40.423112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52958afc-2edd-41de-b5a7-de95adc24a4b None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:40.425123 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 177/377] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 20:41:40 2022] GET /identity/v3/groups/28500a8dd46844218ebe85f28b5287d2 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:40.430784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52958afc-2edd-41de-b5a7-de95adc24a4b None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:40.541465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c7634-2b1f-4d98-b550-70815790654c None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:40.574861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:40.577544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:40.586041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52958afc-2edd-41de-b5a7-de95adc24a4b None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:40.597412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac58ecb-4c11-425f-8d0d-2f68b2811a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:40.597412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac58ecb-4c11-425f-8d0d-2f68b2811a59 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:40.597412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac58ecb-4c11-425f-8d0d-2f68b2811a59 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:40.597412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac58ecb-4c11-425f-8d0d-2f68b2811a59 None admin] PATH_INFO: `/v3/groups/ea8fe0f960394faa9fdd30ea59aac3f5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:40.602760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac58ecb-4c11-425f-8d0d-2f68b2811a59 None admin] RBAC: Authorizing `identity:get_group(group_id=ea8fe0f960394faa9fdd30ea59aac3f5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:40.606874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac58ecb-4c11-425f-8d0d-2f68b2811a59 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:40.609306 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 201/378] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 20:41:40 2022] GET /identity/v3/groups/ea8fe0f960394faa9fdd30ea59aac3f5 => generated 229 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:40.614786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac58ecb-4c11-425f-8d0d-2f68b2811a59 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:40.632120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4274dd-fa79-4c28-b219-8a7072b7d2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:40.633213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4274dd-fa79-4c28-b219-8a7072b7d2fb None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:40.633675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4274dd-fa79-4c28-b219-8a7072b7d2fb None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:40.634165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4274dd-fa79-4c28-b219-8a7072b7d2fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:40.635178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4274dd-fa79-4c28-b219-8a7072b7d2fb None admin] RBAC: Adding query filter params (group.id=28500a8dd46844218ebe85f28b5287d2, role.id=92b887e453ad4fa9aa5c49f7594a7e53, scope.project.id=0381e5a4ddde4d9ab4394d1417743579) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:40.643207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4274dd-fa79-4c28-b219-8a7072b7d2fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:40.643207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4274dd-fa79-4c28-b219-8a7072b7d2fb None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:40.644008 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 178/379] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 20:41:40 2022] GET /identity/v3/role_assignments?group.id=28500a8dd46844218ebe85f28b5287d2&scope.project.id=0381e5a4ddde4d9ab4394d1417743579&role.id=92b887e453ad4fa9aa5c49f7594a7e53 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:40.739792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac58ecb-4c11-425f-8d0d-2f68b2811a59 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:40.778063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bce1618-9244-47b4-b1c6-5a6e3fc5a036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:40.778750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce1618-9244-47b4-b1c6-5a6e3fc5a036 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:40.779037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce1618-9244-47b4-b1c6-5a6e3fc5a036 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:40.779323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bce1618-9244-47b4-b1c6-5a6e3fc5a036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:40.780177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce1618-9244-47b4-b1c6-5a6e3fc5a036 None admin] RBAC: Adding query filter params (group.id=ea8fe0f960394faa9fdd30ea59aac3f5, role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=a7d6b819de6044b98f2c78098814b8f2) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:40.780496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce1618-9244-47b4-b1c6-5a6e3fc5a036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:40.782003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bce1618-9244-47b4-b1c6-5a6e3fc5a036 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:40.788625 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 202/380] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 20:41:40 2022] GET /identity/v3/role_assignments?group.id=ea8fe0f960394faa9fdd30ea59aac3f5&scope.project.id=a7d6b819de6044b98f2c78098814b8f2&role.id=7805b93b32f1472f8365936d4d57d49a => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:40.829282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453b69b-1180-4d9f-91e2-7f35ef8fb496 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:40.829580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453b69b-1180-4d9f-91e2-7f35ef8fb496 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:40.829864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453b69b-1180-4d9f-91e2-7f35ef8fb496 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:40.831597 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 179/381] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:40.838164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3909a-a1c4-44fc-be30-c85ea51a82ee None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:40.838477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3909a-a1c4-44fc-be30-c85ea51a82ee None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:40.838739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3909a-a1c4-44fc-be30-c85ea51a82ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:40.862682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-65f3909a-a1c4-44fc-be30-c85ea51a82ee None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:40.864594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-65f3909a-a1c4-44fc-be30-c85ea51a82ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:40.867552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea9b33-cdaf-47e4-96cb-b75d4115d764 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:40.868118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea9b33-cdaf-47e4-96cb-b75d4115d764 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:40.868404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea9b33-cdaf-47e4-96cb-b75d4115d764 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:40.870041 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 180/382] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:40.875462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bf92f-97b8-4bf6-90dd-664b3a1d86ae None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:40.875783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bf92f-97b8-4bf6-90dd-664b3a1d86ae None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:40.876022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bf92f-97b8-4bf6-90dd-664b3a1d86ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:40.911747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b00bf92f-97b8-4bf6-90dd-664b3a1d86ae None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:40.912971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b00bf92f-97b8-4bf6-90dd-664b3a1d86ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:40.929405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:40.955749 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 203/383] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:40.984115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3909a-a1c4-44fc-be30-c85ea51a82ee None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:41.052660 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 181/384] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:41.092686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bf92f-97b8-4bf6-90dd-664b3a1d86ae None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:41.128579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3909a-a1c4-44fc-be30-c85ea51a82ee None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:41.181116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0df5f5-5249-4f54-bca8-3bf038efada2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:41.181886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0df5f5-5249-4f54-bca8-3bf038efada2 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:41.182177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0df5f5-5249-4f54-bca8-3bf038efada2 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:41.182454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0df5f5-5249-4f54-bca8-3bf038efada2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:41.195380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0df5f5-5249-4f54-bca8-3bf038efada2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:41.204406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-af0df5f5-5249-4f54-bca8-3bf038efada2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:41.204406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:41.204406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:41.204406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:41.204406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:41.204406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:41.204406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:41.204406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:41.204406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:41.204406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:41.204406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:41.204406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:41.204406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:41.204406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:41.204406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:41.204406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:41.205314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:41.205314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:41.205314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:41.205314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:41.205314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:41.205314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:41.205314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:41.205314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:41.205314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:41.205314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:41.205314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:41.205314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:41.205314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:41.205314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:41.205314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:41.205314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:41.206145 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:41.206145 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:41.206145 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:41.206145 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:41.206145 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:41.206145 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:41.206145 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:41.206145 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:41.206145 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:41.206145 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:41.206145 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:41.206145 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:41.206145 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 204/385] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:41:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:41:41.212604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0df5f5-5249-4f54-bca8-3bf038efada2 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:41.224832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:41.248815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bf92f-97b8-4bf6-90dd-664b3a1d86ae None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:41.271600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:41.338967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e78558-0419-48e7-979c-e0736c43a764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:41.339780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e78558-0419-48e7-979c-e0736c43a764 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:41.340063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e78558-0419-48e7-979c-e0736c43a764 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:41.340339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e78558-0419-48e7-979c-e0736c43a764 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:41.350090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0df5f5-5249-4f54-bca8-3bf038efada2 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:41.364316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e78558-0419-48e7-979c-e0736c43a764 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:41.377877 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-78e78558-0419-48e7-979c-e0736c43a764 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:41.377877 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:41.377877 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:41.377877 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:41.377877 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:41.377877 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:41.377877 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:41.377877 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:41.377877 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:41.377877 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:41.377877 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:41.377877 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:41.377877 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:41.377877 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:41.377877 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:41.377877 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:41.378800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:41.378800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:41.378800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:41.378800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:41.378800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:41.378800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:41.378800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:41.378800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:41.378800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:41.378800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:41.378800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:41.378800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:41.378800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:41.378800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:41.378800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:41.378800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:41.379657 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:41.379657 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:41.379657 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:41.379657 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:41.379657 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:41.379657 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:41.379657 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:41.379657 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:41.379657 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:41.379657 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:41.379657 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:41.379657 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:41.379657 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 182/386] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:41.382254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e78558-0419-48e7-979c-e0736c43a764 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:41.423003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285e4ae-da04-4a8c-8ecb-159ddaf22eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:41.424066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285e4ae-da04-4a8c-8ecb-159ddaf22eeb None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:41.424544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285e4ae-da04-4a8c-8ecb-159ddaf22eeb None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:41.425011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8285e4ae-da04-4a8c-8ecb-159ddaf22eeb None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:41.425985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285e4ae-da04-4a8c-8ecb-159ddaf22eeb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:41.426538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285e4ae-da04-4a8c-8ecb-159ddaf22eeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:41.431351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8285e4ae-da04-4a8c-8ecb-159ddaf22eeb None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:41.445582 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 205/387] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:41:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:41.453272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285e4ae-da04-4a8c-8ecb-159ddaf22eeb None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:41.507087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e78558-0419-48e7-979c-e0736c43a764 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:41.554507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d6e91-9ac5-404f-97b6-1dbda058b095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:41.555277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d6e91-9ac5-404f-97b6-1dbda058b095 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:41.555569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d6e91-9ac5-404f-97b6-1dbda058b095 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:41.555848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50d6e91-9ac5-404f-97b6-1dbda058b095 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:41.556619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d6e91-9ac5-404f-97b6-1dbda058b095 None admin] RBAC: Adding query filter params (name=member) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:41.556925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d6e91-9ac5-404f-97b6-1dbda058b095 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:41.558353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50d6e91-9ac5-404f-97b6-1dbda058b095 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:41.577051 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 183/388] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:41.585111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d6e91-9ac5-404f-97b6-1dbda058b095 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:41.619332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:41.666260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:41.670037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8285e4ae-da04-4a8c-8ecb-159ddaf22eeb None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:41.743062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1fe7a-d085-4d98-a0fc-6c7daa38846d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:41.743920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1fe7a-d085-4d98-a0fc-6c7daa38846d None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:41.744262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1fe7a-d085-4d98-a0fc-6c7daa38846d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:41.744597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1fe7a-d085-4d98-a0fc-6c7daa38846d None admin] PATH_INFO: `/v3/groups/28500a8dd46844218ebe85f28b5287d2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:41.746006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1fe7a-d085-4d98-a0fc-6c7daa38846d None admin] RBAC: Authorizing `identity:get_group(group_id=28500a8dd46844218ebe85f28b5287d2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:41.749173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1fe7a-d085-4d98-a0fc-6c7daa38846d None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:41.755596 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 206/389] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 20:41:41 2022] GET /identity/v3/groups/28500a8dd46844218ebe85f28b5287d2 => generated 226 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:41.756772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1fe7a-d085-4d98-a0fc-6c7daa38846d None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:41.788055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50d6e91-9ac5-404f-97b6-1dbda058b095 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:41.837706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6323fd-cb6e-4e5a-b9c7-bc5d86d0214f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:41.838610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6323fd-cb6e-4e5a-b9c7-bc5d86d0214f None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:41.838894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6323fd-cb6e-4e5a-b9c7-bc5d86d0214f None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:41.839172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6323fd-cb6e-4e5a-b9c7-bc5d86d0214f None admin] PATH_INFO: `/v3/groups/28500a8dd46844218ebe85f28b5287d2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:41.840449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6323fd-cb6e-4e5a-b9c7-bc5d86d0214f None admin] RBAC: Authorizing `identity:get_group(group_id=28500a8dd46844218ebe85f28b5287d2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:41.842001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6323fd-cb6e-4e5a-b9c7-bc5d86d0214f None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:41.846449 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 184/390] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 20:41:41 2022] GET /identity/v3/groups/28500a8dd46844218ebe85f28b5287d2 => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:41.852645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6323fd-cb6e-4e5a-b9c7-bc5d86d0214f None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:41.958451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:41.978585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6323fd-cb6e-4e5a-b9c7-bc5d86d0214f None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:42.016434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e947ec-2f46-4e2d-a62a-51fd7f1d1157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:42.017226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e947ec-2f46-4e2d-a62a-51fd7f1d1157 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:42.017517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e947ec-2f46-4e2d-a62a-51fd7f1d1157 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:42.017801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e947ec-2f46-4e2d-a62a-51fd7f1d1157 None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:42.019462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e947ec-2f46-4e2d-a62a-51fd7f1d1157 None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:42.021937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e947ec-2f46-4e2d-a62a-51fd7f1d1157 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:42.024577 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 185/391] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:41 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:42.031871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e947ec-2f46-4e2d-a62a-51fd7f1d1157 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:42.039810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1fe7a-d085-4d98-a0fc-6c7daa38846d None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:42.143323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e947ec-2f46-4e2d-a62a-51fd7f1d1157 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:42.150575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c10db-a5d3-446c-be3e-3f0e26d0dc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:42.151476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c10db-a5d3-446c-be3e-3f0e26d0dc21 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:42.151768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c10db-a5d3-446c-be3e-3f0e26d0dc21 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:42.152054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c10db-a5d3-446c-be3e-3f0e26d0dc21 None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:42.155294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c10db-a5d3-446c-be3e-3f0e26d0dc21 None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:42.156772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c10db-a5d3-446c-be3e-3f0e26d0dc21 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:42.158820 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 207/392] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:41 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:42.178902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c10db-a5d3-446c-be3e-3f0e26d0dc21 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:42.187480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d6ffa-764e-4d4d-abe1-dc85c1ef5ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:42.188381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d6ffa-764e-4d4d-abe1-dc85c1ef5ba0 None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:42.188664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d6ffa-764e-4d4d-abe1-dc85c1ef5ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:42.188939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d6ffa-764e-4d4d-abe1-dc85c1ef5ba0 None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c/groups/28500a8dd46844218ebe85f28b5287d2/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:42.191044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d6ffa-764e-4d4d-abe1-dc85c1ef5ba0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e872f1acf4e544f6963d85f6afc13e1c, group_id=28500a8dd46844218ebe85f28b5287d2, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:42.192748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d6ffa-764e-4d4d-abe1-dc85c1ef5ba0 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:42.228775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:42.275433 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 186/393] 10.222.0.33 () {58 vars in 1442 bytes} [Sat May 28 20:41:42 2022] PUT /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c/groups/28500a8dd46844218ebe85f28b5287d2/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 20:41:42.322025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c10db-a5d3-446c-be3e-3f0e26d0dc21 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:42.367524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca062ed-0ae1-460f-b52e-eebae7ec2b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:42.368427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca062ed-0ae1-460f-b52e-eebae7ec2b3c None admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:42.368713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca062ed-0ae1-460f-b52e-eebae7ec2b3c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:42.368991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca062ed-0ae1-460f-b52e-eebae7ec2b3c None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c/groups/28500a8dd46844218ebe85f28b5287d2/roles/92b887e453ad4fa9aa5c49f7594a7e53` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:42.371661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca062ed-0ae1-460f-b52e-eebae7ec2b3c None admin] RBAC: Authorizing `identity:create_grant(project_id=e872f1acf4e544f6963d85f6afc13e1c, group_id=28500a8dd46844218ebe85f28b5287d2, role_id=92b887e453ad4fa9aa5c49f7594a7e53)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:42.373126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca062ed-0ae1-460f-b52e-eebae7ec2b3c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:42.475591 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 208/394] 10.222.0.33 () {58 vars in 1442 bytes} [Sat May 28 20:41:42 2022] PUT /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c/groups/28500a8dd46844218ebe85f28b5287d2/roles/92b887e453ad4fa9aa5c49f7594a7e53 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:41:43.092250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a88883-79bf-4138-9b3b-667530d1ed68 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:43.092565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a88883-79bf-4138-9b3b-667530d1ed68 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:43.092821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a88883-79bf-4138-9b3b-667530d1ed68 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:43.094502 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 187/395] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:43.100228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef3ddd-ffb0-42ea-8ac7-cf8e36eb9626 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:43.100711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef3ddd-ffb0-42ea-8ac7-cf8e36eb9626 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:43.100787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ef3ddd-ffb0-42ea-8ac7-cf8e36eb9626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:43.123274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-41ef3ddd-ffb0-42ea-8ac7-cf8e36eb9626 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:43.124486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-41ef3ddd-ffb0-42ea-8ac7-cf8e36eb9626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:43.212567 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 209/396] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:43.259394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a88883-79bf-4138-9b3b-667530d1ed68 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:43.279880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:43.392773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a88883-79bf-4138-9b3b-667530d1ed68 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:43.435913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044255b-5ab2-4813-8e34-8a104b4168f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:43.437058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044255b-5ab2-4813-8e34-8a104b4168f2 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:43.437633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044255b-5ab2-4813-8e34-8a104b4168f2 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:43.438124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044255b-5ab2-4813-8e34-8a104b4168f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:43.455474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044255b-5ab2-4813-8e34-8a104b4168f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:43.467056 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-9044255b-5ab2-4813-8e34-8a104b4168f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:43.467056 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:43.467056 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:43.467056 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:43.467056 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:43.467056 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:43.467056 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:43.467056 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:43.467056 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:43.467056 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:43.467056 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:43.467056 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:43.467056 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:43.467056 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:43.467056 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:43.467056 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:43.467951 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:43.467951 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:43.467951 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:43.467951 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:43.467951 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:43.467951 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:43.467951 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:43.467951 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:43.467951 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:43.467951 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:43.467951 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:43.467951 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:43.467951 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:43.467951 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:43.467951 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:43.467951 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:43.468726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:43.468726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:43.468726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:43.468726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:43.468726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:43.468726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:43.468726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:43.468726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:43.468726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:43.468726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:43.468726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:43.468726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:43.471169 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 188/397] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:43.478129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef3ddd-ffb0-42ea-8ac7-cf8e36eb9626 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:43.527395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:43.555875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cafcdd-20bc-4e8b-86a8-aa891afb9ed6 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:43.556185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cafcdd-20bc-4e8b-86a8-aa891afb9ed6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:43.556452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cafcdd-20bc-4e8b-86a8-aa891afb9ed6 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:43.558369 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 189/398] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:41:43.566080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b04bea-07e6-4002-88f9-dce4c96fa851 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:43.566370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b04bea-07e6-4002-88f9-dce4c96fa851 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:43.566657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b04bea-07e6-4002-88f9-dce4c96fa851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:43.598831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-91b04bea-07e6-4002-88f9-dce4c96fa851 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:43.600103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-91b04bea-07e6-4002-88f9-dce4c96fa851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:43.620674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ef3ddd-ffb0-42ea-8ac7-cf8e36eb9626 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:43.662012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fddd38-f33f-46ba-b8ee-2a7cd5ec9f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:43.662619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fddd38-f33f-46ba-b8ee-2a7cd5ec9f8c None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:43.662907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fddd38-f33f-46ba-b8ee-2a7cd5ec9f8c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:43.663186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fddd38-f33f-46ba-b8ee-2a7cd5ec9f8c None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:43.663947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fddd38-f33f-46ba-b8ee-2a7cd5ec9f8c None admin] RBAC: Adding query filter params (name=member) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:43.664255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fddd38-f33f-46ba-b8ee-2a7cd5ec9f8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:43.665640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fddd38-f33f-46ba-b8ee-2a7cd5ec9f8c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:43.678148 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 210/399] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:43.686158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fddd38-f33f-46ba-b8ee-2a7cd5ec9f8c None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:43.725340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:43.735738 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 190/400] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:43.771617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b04bea-07e6-4002-88f9-dce4c96fa851 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:43.786210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:43.794419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fddd38-f33f-46ba-b8ee-2a7cd5ec9f8c None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:43.831817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33920b61-7974-4349-99ee-6c34f4727705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:43.832487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33920b61-7974-4349-99ee-6c34f4727705 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:43.833290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33920b61-7974-4349-99ee-6c34f4727705 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:43.833568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33920b61-7974-4349-99ee-6c34f4727705 None admin] PATH_INFO: `/v3/groups/28500a8dd46844218ebe85f28b5287d2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:43.834808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33920b61-7974-4349-99ee-6c34f4727705 None admin] RBAC: Authorizing `identity:get_group(group_id=28500a8dd46844218ebe85f28b5287d2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:43.836251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33920b61-7974-4349-99ee-6c34f4727705 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:43.838321 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 211/401] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 20:41:43 2022] GET /identity/v3/groups/28500a8dd46844218ebe85f28b5287d2 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:43.839664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96477b96-36ae-4ac3-8e8f-236aa0cda33b None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:43.840014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96477b96-36ae-4ac3-8e8f-236aa0cda33b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:43.840296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96477b96-36ae-4ac3-8e8f-236aa0cda33b None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:43.841933 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 212/402] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:43.848774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796522a-e420-4266-9c1b-57023eaaf84f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:43.849019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796522a-e420-4266-9c1b-57023eaaf84f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:43.849265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3796522a-e420-4266-9c1b-57023eaaf84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:43.869021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3796522a-e420-4266-9c1b-57023eaaf84f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:43.869916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3796522a-e420-4266-9c1b-57023eaaf84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:43.896264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b04bea-07e6-4002-88f9-dce4c96fa851 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:43.932447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62752a7d-b71b-4ee7-bf3f-4565f9ce77a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:43.933579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62752a7d-b71b-4ee7-bf3f-4565f9ce77a9 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:43.934088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62752a7d-b71b-4ee7-bf3f-4565f9ce77a9 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:43.934579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62752a7d-b71b-4ee7-bf3f-4565f9ce77a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:43.948345 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 213/403] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:41:43.948345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796522a-e420-4266-9c1b-57023eaaf84f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:43.955157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62752a7d-b71b-4ee7-bf3f-4565f9ce77a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:43.965126 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-62752a7d-b71b-4ee7-bf3f-4565f9ce77a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:43.965126 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:43.965126 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:43.965126 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:43.965126 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:43.965126 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:43.965126 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:43.965126 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:43.965126 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:43.965126 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:43.965126 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:43.965126 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:43.965126 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:43.965126 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:43.965126 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:43.965126 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:43.966337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:43.966337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:43.966337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:43.966337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:43.966337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:43.966337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:43.966337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:43.966337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:43.966337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:43.966337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:43.966337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:43.966337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:43.966337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:43.966337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:43.966337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:43.966337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:43.967159 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:43.967159 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:43.967159 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:43.967159 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:43.967159 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:43.967159 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:43.967159 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:43.967159 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:43.967159 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:43.967159 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:43.967159 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:43.967159 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:43.969960 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 191/404] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:41:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:41:43.971113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62752a7d-b71b-4ee7-bf3f-4565f9ce77a9 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:44.042468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:44.102455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62752a7d-b71b-4ee7-bf3f-4565f9ce77a9 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:44.104382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:44.133562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3796522a-e420-4266-9c1b-57023eaaf84f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:44.153598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4262e-8ce0-4f49-a875-7c83414e3ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:44.155282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4262e-8ce0-4f49-a875-7c83414e3ed2 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:44.155282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4262e-8ce0-4f49-a875-7c83414e3ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:44.155282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac4262e-8ce0-4f49-a875-7c83414e3ed2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:44.168831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac4262e-8ce0-4f49-a875-7c83414e3ed2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:44.179048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09c915-68b5-433a-b3b3-1e4004b1e4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:44.179622 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-0ac4262e-8ce0-4f49-a875-7c83414e3ed2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:41:44.179622 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:44.179622 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:44.179622 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:44.179622 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:44.179622 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:44.179622 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:44.179622 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:44.179622 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:44.179622 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:44.179622 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:44.179622 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:44.179622 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:44.179622 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:44.179622 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:44.179622 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:44.180497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:44.180497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:44.180497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:44.180497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:44.180497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:44.180497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:44.180497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:44.180497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:44.180497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:44.180497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:44.180497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:44.180497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:44.180497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:44.180497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:44.180497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:44.180497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:44.181280 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:44.181280 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:44.181280 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:44.181280 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:44.181280 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:44.181280 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:44.181280 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:44.181280 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:44.181280 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:44.181280 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:44.181280 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:41:44.181280 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:44.181280 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 192/405] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:41:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:44.182439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4262e-8ce0-4f49-a875-7c83414e3ed2 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:44.183364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09c915-68b5-433a-b3b3-1e4004b1e4b7 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:44.183856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09c915-68b5-433a-b3b3-1e4004b1e4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:44.184331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b09c915-68b5-433a-b3b3-1e4004b1e4b7 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:44.185929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09c915-68b5-433a-b3b3-1e4004b1e4b7 None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:44.187727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b09c915-68b5-433a-b3b3-1e4004b1e4b7 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:44.189648 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 214/406] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:43 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:44.190637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09c915-68b5-433a-b3b3-1e4004b1e4b7 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:44.321434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b09c915-68b5-433a-b3b3-1e4004b1e4b7 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:44.323664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac4262e-8ce0-4f49-a875-7c83414e3ed2 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:44.345201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:44.360519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862357cf-2d74-4d8d-9b45-5ea6f9a72ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:44.360693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6b4f3-875a-4d7e-93bc-3c3da7321d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:44.361199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862357cf-2d74-4d8d-9b45-5ea6f9a72ca1 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:44.361486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862357cf-2d74-4d8d-9b45-5ea6f9a72ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:44.369830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862357cf-2d74-4d8d-9b45-5ea6f9a72ca1 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:44.370051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6b4f3-875a-4d7e-93bc-3c3da7321d62 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:44.370051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6b4f3-875a-4d7e-93bc-3c3da7321d62 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:44.370051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a6b4f3-875a-4d7e-93bc-3c3da7321d62 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:44.370246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862357cf-2d74-4d8d-9b45-5ea6f9a72ca1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:44.370246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862357cf-2d74-4d8d-9b45-5ea6f9a72ca1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:44.370380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6b4f3-875a-4d7e-93bc-3c3da7321d62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:44.370380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6b4f3-875a-4d7e-93bc-3c3da7321d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:44.370495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862357cf-2d74-4d8d-9b45-5ea6f9a72ca1 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:44.370572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a6b4f3-875a-4d7e-93bc-3c3da7321d62 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:44.374820 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 193/407] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:41:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:44.375805 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 215/408] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:44.382211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862357cf-2d74-4d8d-9b45-5ea6f9a72ca1 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:44.383346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6b4f3-875a-4d7e-93bc-3c3da7321d62 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:44.404706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:44.490419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862357cf-2d74-4d8d-9b45-5ea6f9a72ca1 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:44.491370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a6b4f3-875a-4d7e-93bc-3c3da7321d62 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:44.527485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafbac7-abf2-49ef-91a4-afc5c62023d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:44.527686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d918b7-70fe-436e-8e5f-40cb1cad7ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:44.528577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafbac7-abf2-49ef-91a4-afc5c62023d6 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:44.528920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d918b7-70fe-436e-8e5f-40cb1cad7ffd None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:44.529057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafbac7-abf2-49ef-91a4-afc5c62023d6 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:44.529416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafbac7-abf2-49ef-91a4-afc5c62023d6 None admin] PATH_INFO: `/v3/groups/28500a8dd46844218ebe85f28b5287d2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:44.529662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d918b7-70fe-436e-8e5f-40cb1cad7ffd None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:44.530121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d918b7-70fe-436e-8e5f-40cb1cad7ffd None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579/groups/28500a8dd46844218ebe85f28b5287d2/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:44.530901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fafbac7-abf2-49ef-91a4-afc5c62023d6 None admin] RBAC: Authorizing `identity:get_group(group_id=28500a8dd46844218ebe85f28b5287d2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:44.532451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fafbac7-abf2-49ef-91a4-afc5c62023d6 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:44.532647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d918b7-70fe-436e-8e5f-40cb1cad7ffd None admin] RBAC: Authorizing `identity:create_grant(project_id=0381e5a4ddde4d9ab4394d1417743579, group_id=28500a8dd46844218ebe85f28b5287d2, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:44.534279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d918b7-70fe-436e-8e5f-40cb1cad7ffd None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:44.534482 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 216/409] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 20:41:44 2022] GET /identity/v3/groups/28500a8dd46844218ebe85f28b5287d2 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:44.535758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafbac7-abf2-49ef-91a4-afc5c62023d6 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:44.625832 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 194/410] 10.222.0.33 () {58 vars in 1442 bytes} [Sat May 28 20:41:44 2022] PUT /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579/groups/28500a8dd46844218ebe85f28b5287d2/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:41:44.627095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d918b7-70fe-436e-8e5f-40cb1cad7ffd None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:44.628993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:44.650354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafbac7-abf2-49ef-91a4-afc5c62023d6 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:44.692894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:44.716609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d1de1-92ca-45c9-84a4-24714a8d204f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:44.718860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d1de1-92ca-45c9-84a4-24714a8d204f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:44.719352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d1de1-92ca-45c9-84a4-24714a8d204f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:44.719865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593d1de1-92ca-45c9-84a4-24714a8d204f None admin] PATH_INFO: `/v3/groups/ea8fe0f960394faa9fdd30ea59aac3f5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:44.721380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d1de1-92ca-45c9-84a4-24714a8d204f None admin] RBAC: Authorizing `identity:get_group(group_id=ea8fe0f960394faa9fdd30ea59aac3f5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:44.726482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593d1de1-92ca-45c9-84a4-24714a8d204f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:44.737732 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 217/411] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 20:41:44 2022] GET /identity/v3/groups/ea8fe0f960394faa9fdd30ea59aac3f5 => generated 229 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:44.740757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d1de1-92ca-45c9-84a4-24714a8d204f None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:44.833669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d918b7-70fe-436e-8e5f-40cb1cad7ffd None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:44.878160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947807f6-89f9-4b2b-81a5-8d74d6c5ea17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:44.880023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947807f6-89f9-4b2b-81a5-8d74d6c5ea17 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:44.880972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947807f6-89f9-4b2b-81a5-8d74d6c5ea17 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:44.881589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947807f6-89f9-4b2b-81a5-8d74d6c5ea17 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:44.883226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947807f6-89f9-4b2b-81a5-8d74d6c5ea17 None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:44.885545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947807f6-89f9-4b2b-81a5-8d74d6c5ea17 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:44.888200 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 195/412] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:44 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:44.894594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947807f6-89f9-4b2b-81a5-8d74d6c5ea17 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:44.906662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593d1de1-92ca-45c9-84a4-24714a8d204f None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:44.965242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4d187-4764-4b8f-826c-b5251b1cc6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:44.966258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4d187-4764-4b8f-826c-b5251b1cc6f9 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:44.966540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4d187-4764-4b8f-826c-b5251b1cc6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:44.966815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4d187-4764-4b8f-826c-b5251b1cc6f9 None admin] PATH_INFO: `/v3/projects/a7d6b819de6044b98f2c78098814b8f2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:44.969794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4d187-4764-4b8f-826c-b5251b1cc6f9 None admin] RBAC: Authorizing `identity:get_project(project_id=a7d6b819de6044b98f2c78098814b8f2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:44.971270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb4d187-4764-4b8f-826c-b5251b1cc6f9 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:44.975037 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 218/413] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:44 2022] GET /identity/v3/projects/a7d6b819de6044b98f2c78098814b8f2 => generated 344 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:44.980708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4d187-4764-4b8f-826c-b5251b1cc6f9 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:45.019578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:45.091855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947807f6-89f9-4b2b-81a5-8d74d6c5ea17 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:45.118861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:45.138541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4d187-4764-4b8f-826c-b5251b1cc6f9 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:45.157925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19608ca8-98c0-4bb4-9141-35f6e3f0dee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:45.161870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19608ca8-98c0-4bb4-9141-35f6e3f0dee3 None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:45.161870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19608ca8-98c0-4bb4-9141-35f6e3f0dee3 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:45.161870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19608ca8-98c0-4bb4-9141-35f6e3f0dee3 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579/groups/28500a8dd46844218ebe85f28b5287d2/roles/92b887e453ad4fa9aa5c49f7594a7e53` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:45.163100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19608ca8-98c0-4bb4-9141-35f6e3f0dee3 None admin] RBAC: Authorizing `identity:create_grant(project_id=0381e5a4ddde4d9ab4394d1417743579, group_id=28500a8dd46844218ebe85f28b5287d2, role_id=92b887e453ad4fa9aa5c49f7594a7e53)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:45.165042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19608ca8-98c0-4bb4-9141-35f6e3f0dee3 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:45.188933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34daaf7b-7d37-43b0-8027-b80a6bad103b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:45.189910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34daaf7b-7d37-43b0-8027-b80a6bad103b None admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:45.190201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34daaf7b-7d37-43b0-8027-b80a6bad103b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:45.190509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34daaf7b-7d37-43b0-8027-b80a6bad103b None admin] PATH_INFO: `/v3/projects/a7d6b819de6044b98f2c78098814b8f2/groups/ea8fe0f960394faa9fdd30ea59aac3f5/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:45.193150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34daaf7b-7d37-43b0-8027-b80a6bad103b None admin] RBAC: Authorizing `identity:create_grant(project_id=a7d6b819de6044b98f2c78098814b8f2, group_id=ea8fe0f960394faa9fdd30ea59aac3f5, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:45.194755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34daaf7b-7d37-43b0-8027-b80a6bad103b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:45.263832 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 196/414] 10.222.0.33 () {58 vars in 1442 bytes} [Sat May 28 20:41:44 2022] PUT /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579/groups/28500a8dd46844218ebe85f28b5287d2/roles/92b887e453ad4fa9aa5c49f7594a7e53 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:41:45.292446 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 219/415] 10.222.0.33 () {58 vars in 1442 bytes} [Sat May 28 20:41:44 2022] PUT /identity/v3/projects/a7d6b819de6044b98f2c78098814b8f2/groups/ea8fe0f960394faa9fdd30ea59aac3f5/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 20:41:45.312603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167f7ee-8640-4635-899c-aea40577a0b8 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:45.312971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167f7ee-8640-4635-899c-aea40577a0b8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:45.313172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167f7ee-8640-4635-899c-aea40577a0b8 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:45.314992 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 197/416] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:45.321028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4a331-56e6-4ce8-800e-6513ce089fb9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:45.321354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4a331-56e6-4ce8-800e-6513ce089fb9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:45.321598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b4a331-56e6-4ce8-800e-6513ce089fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:45.340066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-81b4a331-56e6-4ce8-800e-6513ce089fb9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:45.341371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-81b4a331-56e6-4ce8-800e-6513ce089fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:45.439865 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 220/417] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:45.466221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167f7ee-8640-4635-899c-aea40577a0b8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:45.588590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167f7ee-8640-4635-899c-aea40577a0b8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:45.628451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb29c14-620f-4bcc-9078-969e84687ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:45.629259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb29c14-620f-4bcc-9078-969e84687ab4 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:45.629577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb29c14-620f-4bcc-9078-969e84687ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:45.629865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb29c14-620f-4bcc-9078-969e84687ab4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:45.641402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb29c14-620f-4bcc-9078-969e84687ab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:45.643116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e46f0-36ae-43f6-9539-33ccbf766eca None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:45.643607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e46f0-36ae-43f6-9539-33ccbf766eca None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:45.644098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e46f0-36ae-43f6-9539-33ccbf766eca None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:45.646291 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 221/418] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:45.651254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-0eb29c14-620f-4bcc-9078-969e84687ab4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:45.651254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:45.651254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:45.651254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:45.651254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:45.651254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:45.651254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:45.651254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:45.651254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:45.651254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:45.651254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:45.651254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:45.651254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:45.651254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:45.651254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:45.651254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:45.652130 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:45.652130 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:45.652130 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:45.652130 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:45.652130 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:45.652130 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:45.652130 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:45.652130 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:45.652130 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:45.652130 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:45.652130 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:45.652130 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:45.652130 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:45.652130 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:45.652130 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:45.652130 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:45.652927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:45.652927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:45.652927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:45.652927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:45.652927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:45.652927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:45.652927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:45.652927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:45.652927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:45.652927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:45.652927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:45.652927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:45.652927 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 198/419] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:45.655442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c2b81-b498-4aa5-96e1-17862bffa980 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:45.655933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c2b81-b498-4aa5-96e1-17862bffa980 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:45.656419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c2b81-b498-4aa5-96e1-17862bffa980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:45.658855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb29c14-620f-4bcc-9078-969e84687ab4 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:45.676528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c34c2b81-b498-4aa5-96e1-17862bffa980 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:45.677721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c34c2b81-b498-4aa5-96e1-17862bffa980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:45.680688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:45.738213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:45.755880 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 222/420] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:45.793626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb29c14-620f-4bcc-9078-969e84687ab4 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:45.839721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c2b81-b498-4aa5-96e1-17862bffa980 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:45.842741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d35c6d-9d40-4449-a43b-b1819bfa6f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:45.843639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d35c6d-9d40-4449-a43b-b1819bfa6f1f None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:45.844138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d35c6d-9d40-4449-a43b-b1819bfa6f1f None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:45.844609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d35c6d-9d40-4449-a43b-b1819bfa6f1f None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:45.845553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d35c6d-9d40-4449-a43b-b1819bfa6f1f None admin] RBAC: Adding query filter params (name=member) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:45.846117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d35c6d-9d40-4449-a43b-b1819bfa6f1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:45.849157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d35c6d-9d40-4449-a43b-b1819bfa6f1f None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:45.858587 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 199/421] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:45.869862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d35c6d-9d40-4449-a43b-b1819bfa6f1f None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:45.968785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c2b81-b498-4aa5-96e1-17862bffa980 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:46.025659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d698fa4-e7cf-41f1-b6e3-0deaa97cd267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:46.026509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d698fa4-e7cf-41f1-b6e3-0deaa97cd267 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:46.026796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d698fa4-e7cf-41f1-b6e3-0deaa97cd267 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:46.027071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d698fa4-e7cf-41f1-b6e3-0deaa97cd267 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:46.038020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:46.041139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d698fa4-e7cf-41f1-b6e3-0deaa97cd267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:46.051424 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-4d698fa4-e7cf-41f1-b6e3-0deaa97cd267 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:46.051424 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:46.051424 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:46.051424 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:46.051424 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:46.051424 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:46.051424 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:46.051424 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:46.051424 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:46.051424 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:46.051424 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:46.051424 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:46.051424 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:46.051424 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:46.051424 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:46.051424 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:46.052438 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:46.052438 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:46.052438 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:46.052438 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:46.052438 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:46.052438 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:46.052438 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:46.052438 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:46.052438 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:46.052438 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:46.052438 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:46.052438 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:46.052438 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:46.052438 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:46.052438 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:46.052438 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:46.053235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:46.053235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:46.053235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:46.053235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:46.053235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:46.053235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:46.053235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:46.053235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:46.053235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:46.053235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:46.053235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:46.053235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:46.056145 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 223/422] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:41:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:41:46.074850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d698fa4-e7cf-41f1-b6e3-0deaa97cd267 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:46.096578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:46.116480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d35c6d-9d40-4449-a43b-b1819bfa6f1f None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:46.188682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d698fa4-e7cf-41f1-b6e3-0deaa97cd267 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:46.192910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df301c7d-1064-488f-bfdf-49f056c03d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:46.193460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df301c7d-1064-488f-bfdf-49f056c03d67 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:46.193743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df301c7d-1064-488f-bfdf-49f056c03d67 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:46.194050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df301c7d-1064-488f-bfdf-49f056c03d67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:46.227916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7ff12-b7ee-4631-bb03-35f56aea6f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:46.229077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7ff12-b7ee-4631-bb03-35f56aea6f27 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:46.229558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7ff12-b7ee-4631-bb03-35f56aea6f27 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:46.230078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7ff12-b7ee-4631-bb03-35f56aea6f27 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:46.231211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7ff12-b7ee-4631-bb03-35f56aea6f27 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:46.231730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7ff12-b7ee-4631-bb03-35f56aea6f27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:46.233898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7ff12-b7ee-4631-bb03-35f56aea6f27 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:46.244122 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 224/423] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:41:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:46.250161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7ff12-b7ee-4631-bb03-35f56aea6f27 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:46.320571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df301c7d-1064-488f-bfdf-49f056c03d67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:46.322229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df301c7d-1064-488f-bfdf-49f056c03d67 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:46.399615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:46.432345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:46.444823 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 200/424] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:46.449942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df301c7d-1064-488f-bfdf-49f056c03d67 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:46.490468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7ff12-b7ee-4631-bb03-35f56aea6f27 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:46.561275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35327e6-7d50-47af-ac05-1ed1247890bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:46.561963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35327e6-7d50-47af-ac05-1ed1247890bc None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:46.563199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35327e6-7d50-47af-ac05-1ed1247890bc None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:46.563665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35327e6-7d50-47af-ac05-1ed1247890bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:46.645657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df301c7d-1064-488f-bfdf-49f056c03d67 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:46.704474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35327e6-7d50-47af-ac05-1ed1247890bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:46.705924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35327e6-7d50-47af-ac05-1ed1247890bc None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:46.712605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06680145-2beb-4a22-8f44-214c6cc3daaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:46.713501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06680145-2beb-4a22-8f44-214c6cc3daaa None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:46.713789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06680145-2beb-4a22-8f44-214c6cc3daaa None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:46.714095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06680145-2beb-4a22-8f44-214c6cc3daaa None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:46.716649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06680145-2beb-4a22-8f44-214c6cc3daaa None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:46.718177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06680145-2beb-4a22-8f44-214c6cc3daaa None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:46.726299 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 201/425] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:46 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:46.727256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06680145-2beb-4a22-8f44-214c6cc3daaa None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:46.823304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:46.836287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:46.854618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06680145-2beb-4a22-8f44-214c6cc3daaa None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:46.890296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac2d96-ee81-4121-966a-5720062fa10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:46.891434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac2d96-ee81-4121-966a-5720062fa10c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:46.891726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac2d96-ee81-4121-966a-5720062fa10c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:46.892010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac2d96-ee81-4121-966a-5720062fa10c None admin] PATH_INFO: `/v3/groups/28500a8dd46844218ebe85f28b5287d2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:46.893222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac2d96-ee81-4121-966a-5720062fa10c None admin] RBAC: Authorizing `identity:get_group(group_id=28500a8dd46844218ebe85f28b5287d2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:46.894799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac2d96-ee81-4121-966a-5720062fa10c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:46.896658 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 202/426] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 20:41:46 2022] GET /identity/v3/groups/28500a8dd46844218ebe85f28b5287d2 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:46.902952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac2d96-ee81-4121-966a-5720062fa10c None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:46.933113 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 225/427] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:46.940634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35327e6-7d50-47af-ac05-1ed1247890bc None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:47.084854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac2d96-ee81-4121-966a-5720062fa10c None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:47.113077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35327e6-7d50-47af-ac05-1ed1247890bc None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:47.131333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:47.146528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583080e-0015-4e5f-90c2-f267d97013cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:47.147189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583080e-0015-4e5f-90c2-f267d97013cc None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:47.147474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583080e-0015-4e5f-90c2-f267d97013cc None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:47.147750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583080e-0015-4e5f-90c2-f267d97013cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:47.148595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583080e-0015-4e5f-90c2-f267d97013cc None admin] RBAC: Adding query filter params (group.id=28500a8dd46844218ebe85f28b5287d2, role.id=c0c09ba97c694896ae14fceb6982b1a5, scope.project.id=e872f1acf4e544f6963d85f6afc13e1c) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:47.148904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583080e-0015-4e5f-90c2-f267d97013cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:47.150390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583080e-0015-4e5f-90c2-f267d97013cc None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:47.159831 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 203/428] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 20:41:46 2022] GET /identity/v3/role_assignments?group.id=28500a8dd46844218ebe85f28b5287d2&scope.project.id=e872f1acf4e544f6963d85f6afc13e1c&role.id=c0c09ba97c694896ae14fceb6982b1a5 => generated 611 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:47.184661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a63749-ab15-45d1-8c4c-b0bf8e209cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:47.185551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a63749-ab15-45d1-8c4c-b0bf8e209cc7 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:47.185858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a63749-ab15-45d1-8c4c-b0bf8e209cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:47.186140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a63749-ab15-45d1-8c4c-b0bf8e209cc7 None admin] PATH_INFO: `/v3/projects/e872f1acf4e544f6963d85f6afc13e1c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:47.190718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a63749-ab15-45d1-8c4c-b0bf8e209cc7 None admin] RBAC: Authorizing `identity:get_project(project_id=e872f1acf4e544f6963d85f6afc13e1c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:47.192244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a63749-ab15-45d1-8c4c-b0bf8e209cc7 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:47.194373 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 226/429] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:46 2022] GET /identity/v3/projects/e872f1acf4e544f6963d85f6afc13e1c => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:47.199840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583080e-0015-4e5f-90c2-f267d97013cc None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:47.343105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583080e-0015-4e5f-90c2-f267d97013cc None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:47.382248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6917c4-d4dd-4ec7-9e1f-a26b3693bfc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:47.383124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6917c4-d4dd-4ec7-9e1f-a26b3693bfc3 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:47.383410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6917c4-d4dd-4ec7-9e1f-a26b3693bfc3 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:47.383691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6917c4-d4dd-4ec7-9e1f-a26b3693bfc3 None admin] PATH_INFO: `/v3/groups/28500a8dd46844218ebe85f28b5287d2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:47.384908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6917c4-d4dd-4ec7-9e1f-a26b3693bfc3 None admin] RBAC: Authorizing `identity:get_group(group_id=28500a8dd46844218ebe85f28b5287d2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:47.386414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6917c4-d4dd-4ec7-9e1f-a26b3693bfc3 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:47.388593 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 204/430] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 20:41:47 2022] GET /identity/v3/groups/28500a8dd46844218ebe85f28b5287d2 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:47.394714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a63749-ab15-45d1-8c4c-b0bf8e209cc7 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:47.442698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:47.529025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a63749-ab15-45d1-8c4c-b0bf8e209cc7 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:47.565859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511e483-badb-44ca-a605-b83fecb3f909 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:47.566161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511e483-badb-44ca-a605-b83fecb3f909 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:47.566435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511e483-badb-44ca-a605-b83fecb3f909 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:47.568717 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 205/431] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:47.574485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce03643-1c5b-4c4c-8720-941c1ba7ec75 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:47.574651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05528ed1-bc60-4d01-9229-f6ebc15b9489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:47.574865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce03643-1c5b-4c4c-8720-941c1ba7ec75 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:47.575048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce03643-1c5b-4c4c-8720-941c1ba7ec75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:47.575280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05528ed1-bc60-4d01-9229-f6ebc15b9489 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:47.575571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05528ed1-bc60-4d01-9229-f6ebc15b9489 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:47.575855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05528ed1-bc60-4d01-9229-f6ebc15b9489 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:47.576791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05528ed1-bc60-4d01-9229-f6ebc15b9489 None admin] RBAC: Adding query filter params (group.id=28500a8dd46844218ebe85f28b5287d2, role.id=92b887e453ad4fa9aa5c49f7594a7e53, scope.project.id=e872f1acf4e544f6963d85f6afc13e1c) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:47.577108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05528ed1-bc60-4d01-9229-f6ebc15b9489 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:47.578621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05528ed1-bc60-4d01-9229-f6ebc15b9489 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:47.586794 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 227/432] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 20:41:47 2022] GET /identity/v3/role_assignments?group.id=28500a8dd46844218ebe85f28b5287d2&scope.project.id=e872f1acf4e544f6963d85f6afc13e1c&role.id=92b887e453ad4fa9aa5c49f7594a7e53 => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:47.598155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3ce03643-1c5b-4c4c-8720-941c1ba7ec75 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:47.599330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3ce03643-1c5b-4c4c-8720-941c1ba7ec75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:47.683368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:47.699723 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 206/433] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:41:47.727801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05528ed1-bc60-4d01-9229-f6ebc15b9489 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:47.844020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:47.852408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05528ed1-bc60-4d01-9229-f6ebc15b9489 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:47.894689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb845d37-10ce-4d94-b0ce-cdf973c51460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:47.896076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb845d37-10ce-4d94-b0ce-cdf973c51460 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:47.896362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb845d37-10ce-4d94-b0ce-cdf973c51460 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:47.896642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb845d37-10ce-4d94-b0ce-cdf973c51460 None admin] PATH_INFO: `/v3/roles/member` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:47.909434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb845d37-10ce-4d94-b0ce-cdf973c51460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:47.918289 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-cb845d37-10ce-4d94-b0ce-cdf973c51460 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:47.918289 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:47.918289 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:47.918289 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:47.918289 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:47.918289 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:47.918289 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:47.918289 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:47.918289 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:47.918289 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:47.918289 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:47.918289 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:47.918289 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:47.918289 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:47.918289 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:47.918289 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:47.919240 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:47.919240 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:47.919240 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:47.919240 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:47.919240 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:47.919240 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:47.919240 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:47.919240 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:47.919240 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:47.919240 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:47.919240 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:47.919240 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:47.919240 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:47.919240 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:47.919240 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:47.919240 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:47.920499 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:47.920499 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:47.920499 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:47.920499 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:47.920499 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:47.920499 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:47.920499 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:47.920499 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:47.920499 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:47.920499 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:47.920499 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 28 20:41:47.920499 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:47.920499 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 228/434] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:47.927337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce03643-1c5b-4c4c-8720-941c1ba7ec75 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:48.038144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce03643-1c5b-4c4c-8720-941c1ba7ec75 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:48.044477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca691d4-515e-41db-b04e-4917c64efcd6 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:48.044765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca691d4-515e-41db-b04e-4917c64efcd6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:48.045046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca691d4-515e-41db-b04e-4917c64efcd6 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:48.046812 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 229/435] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:48.051680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577437ef-0c62-4ea1-b7cc-1b3efd0b0480 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:48.052139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577437ef-0c62-4ea1-b7cc-1b3efd0b0480 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:48.052588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577437ef-0c62-4ea1-b7cc-1b3efd0b0480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:48.080953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-577437ef-0c62-4ea1-b7cc-1b3efd0b0480 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:48.082174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-577437ef-0c62-4ea1-b7cc-1b3efd0b0480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:48.090754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c5b806-ddeb-40d4-b8e0-df2f6824a6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:48.091893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5b806-ddeb-40d4-b8e0-df2f6824a6a2 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:48.092184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5b806-ddeb-40d4-b8e0-df2f6824a6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:48.092467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c5b806-ddeb-40d4-b8e0-df2f6824a6a2 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:48.093594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5b806-ddeb-40d4-b8e0-df2f6824a6a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:48.094014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5b806-ddeb-40d4-b8e0-df2f6824a6a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:48.096590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c5b806-ddeb-40d4-b8e0-df2f6824a6a2 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:48.107474 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 207/436] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:41:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:48.108702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42b124-248a-4049-bc31-0e7654ad1956 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:48.108997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42b124-248a-4049-bc31-0e7654ad1956 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:48.109282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f42b124-248a-4049-bc31-0e7654ad1956 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:48.110545 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 208/437] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:48.115275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42b124-248a-4049-bc31-0e7654ad1956 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:48.173100 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 230/438] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:41:48.174866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314192d4-ea8c-4be6-9393-915839e1b4ea None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:48.175357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314192d4-ea8c-4be6-9393-915839e1b4ea None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:48.175849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314192d4-ea8c-4be6-9393-915839e1b4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:48.177308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:48.193636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-314192d4-ea8c-4be6-9393-915839e1b4ea None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:48.195432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-314192d4-ea8c-4be6-9393-915839e1b4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:48.236878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f42b124-248a-4049-bc31-0e7654ad1956 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:48.269892 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 231/439] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:48.278070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1948b59-971f-4c74-8a02-60f4bd66e26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:48.278175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314192d4-ea8c-4be6-9393-915839e1b4ea None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:48.278248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1948b59-971f-4c74-8a02-60f4bd66e26c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:48.278248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1948b59-971f-4c74-8a02-60f4bd66e26c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:48.278248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1948b59-971f-4c74-8a02-60f4bd66e26c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:48.288436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:48.345813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1948b59-971f-4c74-8a02-60f4bd66e26c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:48.347320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1948b59-971f-4c74-8a02-60f4bd66e26c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:48.373150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314192d4-ea8c-4be6-9393-915839e1b4ea None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:48.396831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:48.406726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f4dd7-cb08-4eb4-b57e-99db443ff5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:48.409801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f4dd7-cb08-4eb4-b57e-99db443ff5f1 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:48.409801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f4dd7-cb08-4eb4-b57e-99db443ff5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:48.409801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f4dd7-cb08-4eb4-b57e-99db443ff5f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:48.418958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f4dd7-cb08-4eb4-b57e-99db443ff5f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:48.427460 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-9c9f4dd7-cb08-4eb4-b57e-99db443ff5f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:41:48.427460 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:48.427460 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:48.427460 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:48.427460 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:48.427460 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:48.427460 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:48.427460 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:48.427460 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:48.427460 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:48.427460 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:48.427460 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:48.427460 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:48.427460 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:48.427460 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:48.427460 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:48.428362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:48.428362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:48.428362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:48.428362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:48.428362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:48.428362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:48.428362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:48.428362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:48.428362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:48.428362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:48.428362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:48.428362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:48.428362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:48.428362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:48.428362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:48.428362 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:48.429339 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:48.429339 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:48.429339 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:48.429339 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:48.429339 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:48.429339 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:48.429339 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:48.429339 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:48.429339 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:48.429339 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:48.429339 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:41:48.429339 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:48.429339 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 232/440] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:41:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:48.430551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f4dd7-cb08-4eb4-b57e-99db443ff5f1 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:48.445928 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 209/441] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 20:41:48.446750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1948b59-971f-4c74-8a02-60f4bd66e26c None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:48.504944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:48.529356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f4dd7-cb08-4eb4-b57e-99db443ff5f1 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:48.546467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1948b59-971f-4c74-8a02-60f4bd66e26c None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:48.564147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3d923-2eb6-48fd-879c-735ee93117c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:48.564959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3d923-2eb6-48fd-879c-735ee93117c9 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:48.565248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3d923-2eb6-48fd-879c-735ee93117c9 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:48.565528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3d923-2eb6-48fd-879c-735ee93117c9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:48.575634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3d923-2eb6-48fd-879c-735ee93117c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:48.579738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a53f08-5514-4416-ac83-7d0ec17686f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:48.580395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a53f08-5514-4416-ac83-7d0ec17686f9 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:48.580683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a53f08-5514-4416-ac83-7d0ec17686f9 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:48.580959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a53f08-5514-4416-ac83-7d0ec17686f9 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:48.581734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a53f08-5514-4416-ac83-7d0ec17686f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:48.582080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a53f08-5514-4416-ac83-7d0ec17686f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:48.583532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a53f08-5514-4416-ac83-7d0ec17686f9 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:48.583788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-7cb3d923-2eb6-48fd-879c-735ee93117c9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:48.583788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:48.583788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:48.583788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:48.583788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:48.583788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:48.583788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:48.583788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:48.583788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:48.583788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:48.583788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:48.583788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:48.583788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:48.583788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:48.583788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:48.583788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:48.584788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:48.584788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:48.584788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:48.584788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:48.584788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:48.584788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:48.584788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:48.584788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:48.584788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:48.584788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:48.584788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:48.584788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:48.584788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:48.584788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:48.584788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:48.584788 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:48.585609 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:48.585609 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:48.585609 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:48.585609 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:48.585609 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:48.585609 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:48.585609 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:48.585609 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:48.585609 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:48.585609 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:48.585609 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:48.585609 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:48.585609 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 233/442] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:41:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:48.586468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3d923-2eb6-48fd-879c-735ee93117c9 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:48.596139 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 210/443] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:48.596938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a53f08-5514-4416-ac83-7d0ec17686f9 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:48.612305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:48.693973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3d923-2eb6-48fd-879c-735ee93117c9 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:48.696908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a53f08-5514-4416-ac83-7d0ec17686f9 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:48.727779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9377b4b-3557-443e-8ea7-353bd93c19e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:48.730060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9377b4b-3557-443e-8ea7-353bd93c19e7 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:48.730060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9377b4b-3557-443e-8ea7-353bd93c19e7 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:48.730060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9377b4b-3557-443e-8ea7-353bd93c19e7 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:48.730300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f405f-2db2-45d2-9f7c-6574eb843f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:48.730403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9377b4b-3557-443e-8ea7-353bd93c19e7 None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:48.730740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f405f-2db2-45d2-9f7c-6574eb843f43 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:48.731028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f405f-2db2-45d2-9f7c-6574eb843f43 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:48.733025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f405f-2db2-45d2-9f7c-6574eb843f43 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:48.733134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9377b4b-3557-443e-8ea7-353bd93c19e7 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:48.733218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f405f-2db2-45d2-9f7c-6574eb843f43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:48.733218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f405f-2db2-45d2-9f7c-6574eb843f43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:48.733569 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 234/444] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:48 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:48.733724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f405f-2db2-45d2-9f7c-6574eb843f43 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:48.734364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9377b4b-3557-443e-8ea7-353bd93c19e7 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:48.740065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:48.745145 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 211/445] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:41:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:48.745970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f405f-2db2-45d2-9f7c-6574eb843f43 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:48.822336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:48.837583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9377b4b-3557-443e-8ea7-353bd93c19e7 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:48.845222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f405f-2db2-45d2-9f7c-6574eb843f43 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:48.871973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2077c9-fd8f-4940-9fd4-6f27eea81748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:48.872626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2077c9-fd8f-4940-9fd4-6f27eea81748 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:48.872909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2077c9-fd8f-4940-9fd4-6f27eea81748 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:48.873191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2077c9-fd8f-4940-9fd4-6f27eea81748 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:48.879119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3036af7-861e-40c8-9164-40dfcd638137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:48.879996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3036af7-861e-40c8-9164-40dfcd638137 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:48.880282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3036af7-861e-40c8-9164-40dfcd638137 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:48.880561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3036af7-861e-40c8-9164-40dfcd638137 None admin] PATH_INFO: `/v3/groups/28500a8dd46844218ebe85f28b5287d2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:48.881745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3036af7-861e-40c8-9164-40dfcd638137 None admin] RBAC: Authorizing `identity:get_group(group_id=28500a8dd46844218ebe85f28b5287d2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:48.883291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3036af7-861e-40c8-9164-40dfcd638137 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:48.884920 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 212/446] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 20:41:48 2022] GET /identity/v3/groups/28500a8dd46844218ebe85f28b5287d2 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:48.885688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3036af7-861e-40c8-9164-40dfcd638137 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:48.945864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2077c9-fd8f-4940-9fd4-6f27eea81748 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:48.947264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2077c9-fd8f-4940-9fd4-6f27eea81748 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:48.950037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:48.986699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3036af7-861e-40c8-9164-40dfcd638137 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:49.022956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27f976-1a94-48e5-9c80-13f53bbb024f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:49.023608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27f976-1a94-48e5-9c80-13f53bbb024f None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:49.023895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27f976-1a94-48e5-9c80-13f53bbb024f None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:49.024175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27f976-1a94-48e5-9c80-13f53bbb024f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:49.038468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:49.047436 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 235/447] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:49.049954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2077c9-fd8f-4940-9fd4-6f27eea81748 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:49.093755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27f976-1a94-48e5-9c80-13f53bbb024f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:49.095242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27f976-1a94-48e5-9c80-13f53bbb024f None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:49.149371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2077c9-fd8f-4940-9fd4-6f27eea81748 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:49.150908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:49.182490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4aa0c-37f9-4afd-836d-5b0c9fdddffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:49.183127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4aa0c-37f9-4afd-836d-5b0c9fdddffd None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:49.183410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4aa0c-37f9-4afd-836d-5b0c9fdddffd None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:49.183684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4aa0c-37f9-4afd-836d-5b0c9fdddffd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:49.184517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4aa0c-37f9-4afd-836d-5b0c9fdddffd None admin] RBAC: Adding query filter params (group.id=28500a8dd46844218ebe85f28b5287d2, role.id=c0c09ba97c694896ae14fceb6982b1a5, scope.project.id=0381e5a4ddde4d9ab4394d1417743579) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:49.184830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4aa0c-37f9-4afd-836d-5b0c9fdddffd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:49.186310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4aa0c-37f9-4afd-836d-5b0c9fdddffd None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:49.191193 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 213/448] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:49.192529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27f976-1a94-48e5-9c80-13f53bbb024f None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:49.192632 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 236/449] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 20:41:49 2022] GET /identity/v3/role_assignments?group.id=28500a8dd46844218ebe85f28b5287d2&scope.project.id=0381e5a4ddde4d9ab4394d1417743579&role.id=c0c09ba97c694896ae14fceb6982b1a5 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:49.197423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4aa0c-37f9-4afd-836d-5b0c9fdddffd None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:49.247341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:49.311245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27f976-1a94-48e5-9c80-13f53bbb024f None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:49.312982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4aa0c-37f9-4afd-836d-5b0c9fdddffd None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:49.348447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbc34a-a8a1-4893-a872-ec19c66d5ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:49.349331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbc34a-a8a1-4893-a872-ec19c66d5ea4 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:49.349702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbc34a-a8a1-4893-a872-ec19c66d5ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:49.349981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbc34a-a8a1-4893-a872-ec19c66d5ea4 None admin] PATH_INFO: `/v3/projects/a7d6b819de6044b98f2c78098814b8f2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:49.351191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbc34a-a8a1-4893-a872-ec19c66d5ea4 None admin] RBAC: Authorizing `identity:get_project(project_id=a7d6b819de6044b98f2c78098814b8f2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:49.352671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbc34a-a8a1-4893-a872-ec19c66d5ea4 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:49.354149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6389de7-0ac6-4437-95d4-5b842cd859a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:49.358890 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 214/450] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:49 2022] GET /identity/v3/projects/a7d6b819de6044b98f2c78098814b8f2 => generated 344 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:49.359057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6389de7-0ac6-4437-95d4-5b842cd859a0 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:49.359057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6389de7-0ac6-4437-95d4-5b842cd859a0 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:49.359057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6389de7-0ac6-4437-95d4-5b842cd859a0 None admin] PATH_INFO: `/v3/projects/0381e5a4ddde4d9ab4394d1417743579` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:49.359057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6389de7-0ac6-4437-95d4-5b842cd859a0 None admin] RBAC: Authorizing `identity:get_project(project_id=0381e5a4ddde4d9ab4394d1417743579)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:49.359057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6389de7-0ac6-4437-95d4-5b842cd859a0 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:49.359972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbc34a-a8a1-4893-a872-ec19c66d5ea4 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:49.360383 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 237/451] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:41:49 2022] GET /identity/v3/projects/0381e5a4ddde4d9ab4394d1417743579 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:49.366457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6389de7-0ac6-4437-95d4-5b842cd859a0 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:49.407053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:49.482461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbc34a-a8a1-4893-a872-ec19c66d5ea4 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:49.494161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6389de7-0ac6-4437-95d4-5b842cd859a0 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:49.497752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:49.521519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde43779-0741-4cc3-84ba-95ce7aaf1320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:49.522428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde43779-0741-4cc3-84ba-95ce7aaf1320 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:49.522714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde43779-0741-4cc3-84ba-95ce7aaf1320 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:49.522998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde43779-0741-4cc3-84ba-95ce7aaf1320 None admin] PATH_INFO: `/v3/groups/ea8fe0f960394faa9fdd30ea59aac3f5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:49.524196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde43779-0741-4cc3-84ba-95ce7aaf1320 None admin] RBAC: Authorizing `identity:get_group(group_id=ea8fe0f960394faa9fdd30ea59aac3f5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:49.525660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde43779-0741-4cc3-84ba-95ce7aaf1320 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:49.528480 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 215/452] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 20:41:49 2022] GET /identity/v3/groups/ea8fe0f960394faa9fdd30ea59aac3f5 => generated 229 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:49.534399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800df2f2-22b8-488f-8da6-b9cbd40993f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:49.534559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde43779-0741-4cc3-84ba-95ce7aaf1320 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:49.534953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800df2f2-22b8-488f-8da6-b9cbd40993f1 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:49.536138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800df2f2-22b8-488f-8da6-b9cbd40993f1 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:49.536138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800df2f2-22b8-488f-8da6-b9cbd40993f1 None admin] PATH_INFO: `/v3/groups/28500a8dd46844218ebe85f28b5287d2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:49.537121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800df2f2-22b8-488f-8da6-b9cbd40993f1 None admin] RBAC: Authorizing `identity:get_group(group_id=28500a8dd46844218ebe85f28b5287d2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:49.538706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800df2f2-22b8-488f-8da6-b9cbd40993f1 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:49.540347 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 238/453] 10.222.0.33 () {56 vars in 1180 bytes} [Sat May 28 20:41:49 2022] GET /identity/v3/groups/28500a8dd46844218ebe85f28b5287d2 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:49.546401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800df2f2-22b8-488f-8da6-b9cbd40993f1 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:49.639540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde43779-0741-4cc3-84ba-95ce7aaf1320 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:49.657662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800df2f2-22b8-488f-8da6-b9cbd40993f1 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:49.659190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:49.673103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eebad1-7f5a-46cd-8213-e74e19fa5656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:49.673856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eebad1-7f5a-46cd-8213-e74e19fa5656 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:49.674162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eebad1-7f5a-46cd-8213-e74e19fa5656 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:49.674450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eebad1-7f5a-46cd-8213-e74e19fa5656 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:49.675292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eebad1-7f5a-46cd-8213-e74e19fa5656 None admin] RBAC: Adding query filter params (group.id=ea8fe0f960394faa9fdd30ea59aac3f5, role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=a7d6b819de6044b98f2c78098814b8f2) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:49.675609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eebad1-7f5a-46cd-8213-e74e19fa5656 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:49.677605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eebad1-7f5a-46cd-8213-e74e19fa5656 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:49.683740 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 216/454] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 20:41:49 2022] GET /identity/v3/role_assignments?group.id=ea8fe0f960394faa9fdd30ea59aac3f5&scope.project.id=a7d6b819de6044b98f2c78098814b8f2&role.id=7805b93b32f1472f8365936d4d57d49a => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:49.692624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a4876-8b32-4ac3-a710-1be6ce2eb3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:49.693302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a4876-8b32-4ac3-a710-1be6ce2eb3ed None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:49.693590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a4876-8b32-4ac3-a710-1be6ce2eb3ed None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:49.693976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a4876-8b32-4ac3-a710-1be6ce2eb3ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:49.694839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a4876-8b32-4ac3-a710-1be6ce2eb3ed None admin] RBAC: Adding query filter params (group.id=28500a8dd46844218ebe85f28b5287d2, role.id=92b887e453ad4fa9aa5c49f7594a7e53, scope.project.id=0381e5a4ddde4d9ab4394d1417743579) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:49.695157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a4876-8b32-4ac3-a710-1be6ce2eb3ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:49.696678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a4876-8b32-4ac3-a710-1be6ce2eb3ed None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:49.703665 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 239/455] 10.222.0.33 () {56 vars in 1376 bytes} [Sat May 28 20:41:49 2022] GET /identity/v3/role_assignments?group.id=28500a8dd46844218ebe85f28b5287d2&scope.project.id=0381e5a4ddde4d9ab4394d1417743579&role.id=92b887e453ad4fa9aa5c49f7594a7e53 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:52.725954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5d577-51d8-4d08-83b8-927f6dce5815 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:52.726497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5d577-51d8-4d08-83b8-927f6dce5815 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:52.726497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b5d577-51d8-4d08-83b8-927f6dce5815 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:52.728142 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 217/456] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:52.733418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9f290-2a35-432d-866c-dcff43e5f0df None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:52.733807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9f290-2a35-432d-866c-dcff43e5f0df None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:52.734026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9f290-2a35-432d-866c-dcff43e5f0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:52.758857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-86d9f290-2a35-432d-866c-dcff43e5f0df None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:52.760095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-86d9f290-2a35-432d-866c-dcff43e5f0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:52.803645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeb744-eb7e-4025-acb1-6f4aee93482d None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:52.803970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeb744-eb7e-4025-acb1-6f4aee93482d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:52.804200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aeb744-eb7e-4025-acb1-6f4aee93482d None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:52.806291 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 218/457] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:52.811939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7123b-cf1b-4f8d-b0b4-4fb865a6f357 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:52.812228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7123b-cf1b-4f8d-b0b4-4fb865a6f357 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:52.812715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7123b-cf1b-4f8d-b0b4-4fb865a6f357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:52.835168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2ea7123b-cf1b-4f8d-b0b4-4fb865a6f357 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:52.836662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2ea7123b-cf1b-4f8d-b0b4-4fb865a6f357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:52.872114 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 240/458] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:52.904497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:52.919091 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 219/459] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:52.920642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9f290-2a35-432d-866c-dcff43e5f0df None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:52.950449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7123b-cf1b-4f8d-b0b4-4fb865a6f357 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:53.042044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:53.048896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9f290-2a35-432d-866c-dcff43e5f0df None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:53.099002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd744bb-8284-420f-a7c3-ac9076f28f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:53.099831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd744bb-8284-420f-a7c3-ac9076f28f51 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:53.100114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd744bb-8284-420f-a7c3-ac9076f28f51 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:53.100391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd744bb-8284-420f-a7c3-ac9076f28f51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:53.105207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7123b-cf1b-4f8d-b0b4-4fb865a6f357 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:53.126522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd744bb-8284-420f-a7c3-ac9076f28f51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:53.128006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd744bb-8284-420f-a7c3-ac9076f28f51 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:53.150825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcbac0-5e21-499e-a019-618ba13acadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:53.151168 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-1fd744bb-8284-420f-a7c3-ac9076f28f51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:53.151168 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:53.151168 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:53.151168 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:53.151168 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:53.151168 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:53.151168 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:53.151168 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:53.151168 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:53.151168 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:53.151168 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:53.151168 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:53.151168 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:41:53.151168 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:41:53.151168 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:41:53.151991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:41:53.151991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:53.151991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:53.151991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:53.151991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:53.151991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:53.151991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:53.151991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:53.151991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:53.151991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:53.151991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:53.151991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:53.151991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:53.151991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:53.151991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:53.151991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:53.151991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:53.152838 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:41:53.152838 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:41:53.152838 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:53.152838 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:53.153063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcbac0-5e21-499e-a019-618ba13acadb None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:53.153063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcbac0-5e21-499e-a019-618ba13acadb None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:53.153063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcbac0-5e21-499e-a019-618ba13acadb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:53.153232 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 241/460] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:53.159856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd744bb-8284-420f-a7c3-ac9076f28f51 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:53.163218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcbac0-5e21-499e-a019-618ba13acadb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:53.164713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcbac0-5e21-499e-a019-618ba13acadb None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:53.184499 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-7adcbac0-5e21-499e-a019-618ba13acadb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:53.184499 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:53.184499 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:53.184499 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:53.184499 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:53.184499 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:53.184499 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:53.184499 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:53.184499 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:53.184499 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:53.184499 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:53.184499 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:53.184499 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:41:53.184499 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:41:53.184499 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:41:53.185387 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:41:53.185387 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:53.185387 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:53.185387 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:53.185387 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:53.185387 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:53.185387 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:53.185387 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:53.185387 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:53.185387 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:53.185387 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:53.185387 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:53.185387 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:53.185387 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:53.185387 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:53.185387 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:53.185387 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:53.186297 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:41:53.186297 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:41:53.186297 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:53.186297 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:53.186297 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 220/461] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:41:53.188936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cd197-5a6d-441c-b2a9-a2f6017f631f None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:53.188936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cd197-5a6d-441c-b2a9-a2f6017f631f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:53.188936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cd197-5a6d-441c-b2a9-a2f6017f631f None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:53.189395 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 221/462] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:53.194385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cd197-5a6d-441c-b2a9-a2f6017f631f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:53.222816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:53.275924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd744bb-8284-420f-a7c3-ac9076f28f51 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:53.304015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cd197-5a6d-441c-b2a9-a2f6017f631f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:53.311233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb06425a-1933-4448-9504-6dd747f1e2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:53.311903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06425a-1933-4448-9504-6dd747f1e2d4 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:53.312292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06425a-1933-4448-9504-6dd747f1e2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:53.312578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb06425a-1933-4448-9504-6dd747f1e2d4 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:53.313307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06425a-1933-4448-9504-6dd747f1e2d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:53.313616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06425a-1933-4448-9504-6dd747f1e2d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:53.315165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb06425a-1933-4448-9504-6dd747f1e2d4 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:53.332083 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 242/463] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:53.334058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1269e4b-74a9-464e-932c-7268b94bff11 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:53.334354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1269e4b-74a9-464e-932c-7268b94bff11 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:53.334637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1269e4b-74a9-464e-932c-7268b94bff11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:53.340397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2567f-8317-4464-8f02-47298ba717ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:53.341063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2567f-8317-4464-8f02-47298ba717ba None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:53.341348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2567f-8317-4464-8f02-47298ba717ba None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:53.341682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2567f-8317-4464-8f02-47298ba717ba None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:53.342449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2567f-8317-4464-8f02-47298ba717ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:53.342814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2567f-8317-4464-8f02-47298ba717ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:53.344213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec2567f-8317-4464-8f02-47298ba717ba None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:53.351682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a1269e4b-74a9-464e-932c-7268b94bff11 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:53.352870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a1269e4b-74a9-464e-932c-7268b94bff11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:53.354888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:53.364874 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 222/464] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:53.364874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2567f-8317-4464-8f02-47298ba717ba None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:53.429448 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 243/465] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:53.431078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd900d-efe3-49f0-aa0e-209916157718 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:53.431373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd900d-efe3-49f0-aa0e-209916157718 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:53.431657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd900d-efe3-49f0-aa0e-209916157718 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:53.433082 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 244/466] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:53.433605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd900d-efe3-49f0-aa0e-209916157718 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:53.467418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:53.483193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec2567f-8317-4464-8f02-47298ba717ba None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:53.518802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cc8ed-76bd-4af7-9a40-438321fdfa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:53.519480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cc8ed-76bd-4af7-9a40-438321fdfa50 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:53.519773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cc8ed-76bd-4af7-9a40-438321fdfa50 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:53.520085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cc8ed-76bd-4af7-9a40-438321fdfa50 None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:53.520797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cc8ed-76bd-4af7-9a40-438321fdfa50 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:53.522300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cc8ed-76bd-4af7-9a40-438321fdfa50 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:53.554862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd900d-efe3-49f0-aa0e-209916157718 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:53.589069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c59aa-8a47-4c8d-85fb-f8f28cad4d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:53.589729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c59aa-8a47-4c8d-85fb-f8f28cad4d90 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:53.590040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c59aa-8a47-4c8d-85fb-f8f28cad4d90 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:53.590313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c59aa-8a47-4c8d-85fb-f8f28cad4d90 None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:53.591010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c59aa-8a47-4c8d-85fb-f8f28cad4d90 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:53.592479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c59aa-8a47-4c8d-85fb-f8f28cad4d90 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:53.601508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:53.656689 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 223/467] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:41:53 2022] POST /identity/v3/users => generated 246 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:41:53.658162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993f42f-f9ef-4c33-b5f4-776bdc2dda43 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:53.658362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993f42f-f9ef-4c33-b5f4-776bdc2dda43 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:53.658647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993f42f-f9ef-4c33-b5f4-776bdc2dda43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:53.678038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4993f42f-f9ef-4c33-b5f4-776bdc2dda43 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:53.679511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4993f42f-f9ef-4c33-b5f4-776bdc2dda43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:53.690222 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 245/468] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:41:53 2022] POST /identity/v3/users => generated 248 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:41:53.691955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c59aa-8a47-4c8d-85fb-f8f28cad4d90 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:53.831284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c59aa-8a47-4c8d-85fb-f8f28cad4d90 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:53.839697 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 224/469] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:53.871597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468cf5d-d1bf-4acc-a538-06a179ea428e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:53.872783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468cf5d-d1bf-4acc-a538-06a179ea428e None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:53.873050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993f42f-f9ef-4c33-b5f4-776bdc2dda43 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:53.873487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468cf5d-d1bf-4acc-a538-06a179ea428e None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:53.874025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468cf5d-d1bf-4acc-a538-06a179ea428e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:53.893413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468cf5d-d1bf-4acc-a538-06a179ea428e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:53.895289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468cf5d-d1bf-4acc-a538-06a179ea428e None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:53.916581 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-0468cf5d-d1bf-4acc-a538-06a179ea428e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:53.916581 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:53.916581 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:53.916581 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:53.916581 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:53.916581 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:53.916581 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:53.916581 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:53.916581 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:53.916581 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:53.916581 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:53.916581 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:53.916581 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:41:53.916581 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:41:53.916581 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:41:53.917561 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:41:53.917561 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:53.917561 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:53.917561 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:53.917561 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:53.917561 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:53.917561 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:53.917561 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:53.917561 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:53.917561 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:53.917561 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:53.917561 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:53.917561 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:53.917561 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:53.917561 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:53.917561 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:53.917561 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:53.918830 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:41:53.918830 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:41:53.918830 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:53.918830 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:53.921857 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 246/470] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:53.924123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468cf5d-d1bf-4acc-a538-06a179ea428e None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:53.995955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:53.997985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:54.032601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993f42f-f9ef-4c33-b5f4-776bdc2dda43 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:54.076342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468cf5d-d1bf-4acc-a538-06a179ea428e None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:54.084255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac6dd3-78b6-40e4-9a91-19cb6233baa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:54.084998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac6dd3-78b6-40e4-9a91-19cb6233baa8 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:54.085284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac6dd3-78b6-40e4-9a91-19cb6233baa8 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:54.085558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac6dd3-78b6-40e4-9a91-19cb6233baa8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:54.096855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac6dd3-78b6-40e4-9a91-19cb6233baa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:54.099024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac6dd3-78b6-40e4-9a91-19cb6233baa8 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:54.121232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78db99a-2fa2-492a-9823-2b131e9ec329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:54.128237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78db99a-2fa2-492a-9823-2b131e9ec329 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:54.128237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78db99a-2fa2-492a-9823-2b131e9ec329 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:54.128237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78db99a-2fa2-492a-9823-2b131e9ec329 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:54.128237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78db99a-2fa2-492a-9823-2b131e9ec329 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:54.128508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-18ac6dd3-78b6-40e4-9a91-19cb6233baa8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:54.128508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:54.128508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:54.128508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:54.128508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:54.128508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:54.128508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:54.128508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:54.128508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:54.128508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:54.128508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:54.128508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:54.128508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:41:54.128508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:41:54.128508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:41:54.129345 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:41:54.129345 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:54.129345 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:54.129345 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:54.129345 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:54.129345 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:54.129345 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:54.129345 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:54.129345 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:54.129345 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:54.129345 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:54.129345 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:54.129345 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:54.129345 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:54.129345 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:54.129345 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:54.129345 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:54.130252 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:41:54.130252 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:41:54.130252 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:54.130252 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:54.130461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78db99a-2fa2-492a-9823-2b131e9ec329 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:54.130461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78db99a-2fa2-492a-9823-2b131e9ec329 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:54.130593 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 225/471] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:41:54.130593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac6dd3-78b6-40e4-9a91-19cb6233baa8 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:54.139997 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 247/472] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:54.146442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78db99a-2fa2-492a-9823-2b131e9ec329 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:54.257247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac6dd3-78b6-40e4-9a91-19cb6233baa8 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:54.257446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78db99a-2fa2-492a-9823-2b131e9ec329 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:54.266242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:54.296239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:54.299209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53530c6b-18f9-4e20-ac3d-6da1e73295e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:54.300505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53530c6b-18f9-4e20-ac3d-6da1e73295e5 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:54.301875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53530c6b-18f9-4e20-ac3d-6da1e73295e5 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:54.302621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53530c6b-18f9-4e20-ac3d-6da1e73295e5 None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:54.303730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53530c6b-18f9-4e20-ac3d-6da1e73295e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:54.305485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53530c6b-18f9-4e20-ac3d-6da1e73295e5 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:54.310537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562843ae-e6fb-4b6c-81ff-e3a76429dc89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:54.311462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562843ae-e6fb-4b6c-81ff-e3a76429dc89 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:54.311966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562843ae-e6fb-4b6c-81ff-e3a76429dc89 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:54.312441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562843ae-e6fb-4b6c-81ff-e3a76429dc89 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:54.313663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562843ae-e6fb-4b6c-81ff-e3a76429dc89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:54.314193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562843ae-e6fb-4b6c-81ff-e3a76429dc89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:54.317405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562843ae-e6fb-4b6c-81ff-e3a76429dc89 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:54.330559 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 226/473] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:54.332104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa71fb-bc55-401b-b0c5-95324cdf7868 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:54.332489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa71fb-bc55-401b-b0c5-95324cdf7868 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:54.332855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa71fb-bc55-401b-b0c5-95324cdf7868 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:54.336241 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 227/474] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:54.336820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa71fb-bc55-401b-b0c5-95324cdf7868 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:54.414774 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 248/475] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:41:54 2022] POST /identity/v3/users => generated 248 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:41:54.416549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96165adb-5d23-4ba1-9af3-f3ef813d6856 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:54.416839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96165adb-5d23-4ba1-9af3-f3ef813d6856 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:54.417330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96165adb-5d23-4ba1-9af3-f3ef813d6856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:54.443467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-96165adb-5d23-4ba1-9af3-f3ef813d6856 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:54.444822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-96165adb-5d23-4ba1-9af3-f3ef813d6856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:54.482424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa71fb-bc55-401b-b0c5-95324cdf7868 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:54.561265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee348791-fce0-4646-a5fb-660e2e8e846d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:54.562708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee348791-fce0-4646-a5fb-660e2e8e846d None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:54.563002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee348791-fce0-4646-a5fb-660e2e8e846d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:54.563280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee348791-fce0-4646-a5fb-660e2e8e846d None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:54.564016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee348791-fce0-4646-a5fb-660e2e8e846d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:54.565501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee348791-fce0-4646-a5fb-660e2e8e846d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:54.569202 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 249/476] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:54.597503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96165adb-5d23-4ba1-9af3-f3ef813d6856 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:54.656667 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 228/477] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:41:54 2022] POST /identity/v3/users => generated 249 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:41:54.726644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:54.736746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96165adb-5d23-4ba1-9af3-f3ef813d6856 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:54.780270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d109a-3c38-43e9-8976-d5a692a12738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:54.780975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d109a-3c38-43e9-8976-d5a692a12738 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:54.781260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d109a-3c38-43e9-8976-d5a692a12738 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:54.781539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3d109a-3c38-43e9-8976-d5a692a12738 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:54.782354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d109a-3c38-43e9-8976-d5a692a12738 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:54.783780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3d109a-3c38-43e9-8976-d5a692a12738 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:54.828159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.sql.core [None req-de3d109a-3c38-43e9-8976-d5a692a12738 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 28 20:41:54.828159 ncdv-master-1 devstack@keystone.service[103103]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 28 20:41:54.828159 ncdv-master-1 devstack@keystone.service[103103]: [parameters: {'id': 'd00c92b191f44818a94ad2d2ed71dea7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 28 20:41:54.828159 ncdv-master-1 devstack@keystone.service[103103]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=103103) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 28 20:41:54.846484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-de3d109a-3c38-43e9-8976-d5a692a12738 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 28 20:41:54.846484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:54.846484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 20:41:54.846484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 20:41:54.846484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 20:41:54.846484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 20:41:54.846484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 20:41:54.846484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application result = self._query(query) May 28 20:41:54.846484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 20:41:54.846484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application conn.query(q) May 28 20:41:54.846484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 20:41:54.846484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 20:41:54.846484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 20:41:54.846484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application result.read() May 28 20:41:54.846484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return ref.to_dict() May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 20:41:54.849585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application next(self.gen) May 28 20:41:54.851644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 28 20:41:54.851644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application yield resource May 28 20:41:54.851644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 20:41:54.851644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application next(self.gen) May 28 20:41:54.851644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 28 20:41:54.851644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.session.rollback() May 28 20:41:54.851644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 28 20:41:54.851644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.force_reraise() May 28 20:41:54.851644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 28 20:41:54.851644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise self.value May 28 20:41:54.851644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 28 20:41:54.851644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 28 20:41:54.851644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 28 20:41:54.851644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application session.commit() May 28 20:41:54.851644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 28 20:41:54.851644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 28 20:41:54.851644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 28 20:41:54.853015 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._prepare_impl() May 28 20:41:54.853015 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 28 20:41:54.853015 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.session.flush() May 28 20:41:54.853015 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 28 20:41:54.853015 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._flush(objects) May 28 20:41:54.853015 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 28 20:41:54.853015 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 28 20:41:54.853015 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 28 20:41:54.853015 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application compat.raise_( May 28 20:41:54.853015 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 20:41:54.853015 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception May 28 20:41:54.853015 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 28 20:41:54.853015 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application flush_context.execute() May 28 20:41:54.853015 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 28 20:41:54.853015 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rec.execute(self) May 28 20:41:54.853015 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 28 20:41:54.853015 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 28 20:41:54.863063 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 28 20:41:54.863063 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application _emit_insert_statements( May 28 20:41:54.863063 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 28 20:41:54.863063 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application c = connection._execute_20( May 28 20:41:54.863063 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 28 20:41:54.863063 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 28 20:41:54.863063 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 28 20:41:54.863063 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 28 20:41:54.863063 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 28 20:41:54.863063 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ret = self._execute_context( May 28 20:41:54.863063 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 28 20:41:54.863063 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 28 20:41:54.863063 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 28 20:41:54.863063 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 28 20:41:54.863063 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 20:41:54.863063 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception May 28 20:41:54.865123 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 20:41:54.865123 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 20:41:54.865123 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 20:41:54.865123 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 20:41:54.865123 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 20:41:54.865123 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application result = self._query(query) May 28 20:41:54.865123 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 20:41:54.865123 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application conn.query(q) May 28 20:41:54.865123 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 20:41:54.865123 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 20:41:54.865123 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 20:41:54.865123 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application result.read() May 28 20:41:54.865123 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 20:41:54.865123 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 20:41:54.865123 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 20:41:54.865123 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 20:41:54.867066 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 20:41:54.867066 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 20:41:54.867066 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 20:41:54.867066 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 20:41:54.867066 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 28 20:41:54.867066 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 28 20:41:54.867066 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [parameters: {'id': 'd00c92b191f44818a94ad2d2ed71dea7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 28 20:41:54.867066 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 28 20:41:54.867066 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:54.867066 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:41:54.867066 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:54.867066 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:54.867066 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:54.867066 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:54.867066 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:54.867066 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:54.869258 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:54.869258 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:54.869258 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:54.869258 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:54.869258 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:54.869258 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:54.869258 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 28 20:41:54.869258 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 28 20:41:54.869258 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:54.869258 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:54.869258 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 28 20:41:54.869258 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 28 20:41:54.869258 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 28 20:41:54.869258 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 28 20:41:54.869258 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 28 20:41:54.869258 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:54.871060 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 250/478] 10.222.0.33 () {60 vars in 1131 bytes} [Sat May 28 20:41:54 2022] POST /identity/v3/roles => generated 144 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 28 20:41:54.871194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee348791-fce0-4646-a5fb-660e2e8e846d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:54.914557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:54.975050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee348791-fce0-4646-a5fb-660e2e8e846d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:55.012340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f8093-6cb7-4c66-8c0c-6f675517ee32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:55.013445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f8093-6cb7-4c66-8c0c-6f675517ee32 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:55.013988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f8093-6cb7-4c66-8c0c-6f675517ee32 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:55.014451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f8093-6cb7-4c66-8c0c-6f675517ee32 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:55.025908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f8093-6cb7-4c66-8c0c-6f675517ee32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:55.034340 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-f10f8093-6cb7-4c66-8c0c-6f675517ee32 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:55.034340 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:55.034340 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:55.034340 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:55.034340 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:55.034340 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:55.034340 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:55.034340 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:55.034340 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:55.034340 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:55.034340 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:55.034340 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:55.034340 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:55.034340 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:55.034340 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:55.034340 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:55.035270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:55.035270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:55.035270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:55.035270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:55.035270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:55.035270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:55.035270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:55.035270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:55.035270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:55.035270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:55.035270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:55.035270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:55.035270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:55.035270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:55.035270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:55.035270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:55.036153 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:55.036153 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:55.036153 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:55.036153 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:55.036153 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:55.036153 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:55.036153 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:55.036153 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:55.036153 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:55.036153 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:55.036153 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 28 20:41:55.036153 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:55.038668 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 229/479] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:41:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:55.043942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d109a-3c38-43e9-8976-d5a692a12738 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:55.169103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3d109a-3c38-43e9-8976-d5a692a12738 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:55.212276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafcdbe5-bfc0-448a-9357-78cbd8f0e97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:55.213273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcdbe5-bfc0-448a-9357-78cbd8f0e97d None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:55.213711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcdbe5-bfc0-448a-9357-78cbd8f0e97d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:55.214169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcdbe5-bfc0-448a-9357-78cbd8f0e97d None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:55.215102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcdbe5-bfc0-448a-9357-78cbd8f0e97d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:55.215561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcdbe5-bfc0-448a-9357-78cbd8f0e97d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:55.217125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcdbe5-bfc0-448a-9357-78cbd8f0e97d None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:55.226761 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 251/480] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:41:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:56.664319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98691d46-7855-4eef-a801-00573f61af31 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:56.664849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98691d46-7855-4eef-a801-00573f61af31 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:56.664849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98691d46-7855-4eef-a801-00573f61af31 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:56.666533 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 230/481] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:56.672008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a485ebd-0d84-453a-a4fd-f33e040e5dda None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:56.672408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a485ebd-0d84-453a-a4fd-f33e040e5dda None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:56.672574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a485ebd-0d84-453a-a4fd-f33e040e5dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:56.699065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5a485ebd-0d84-453a-a4fd-f33e040e5dda None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:56.700249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54f38d-9e37-4ff0-9f8e-7bfdc77c6e7f None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:56.700370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5a485ebd-0d84-453a-a4fd-f33e040e5dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:56.700547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54f38d-9e37-4ff0-9f8e-7bfdc77c6e7f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:56.700832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae54f38d-9e37-4ff0-9f8e-7bfdc77c6e7f None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:56.702469 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 231/482] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:56.707567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9a84-f0d6-42a8-94f3-bd6d9a3892c8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:56.708352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9a84-f0d6-42a8-94f3-bd6d9a3892c8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:56.708713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9a84-f0d6-42a8-94f3-bd6d9a3892c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:56.715565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:56.737607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7fec9a84-f0d6-42a8-94f3-bd6d9a3892c8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:56.738875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7fec9a84-f0d6-42a8-94f3-bd6d9a3892c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:56.812289 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 252/483] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:41:56.836240 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 232/484] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:41:56.853462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a485ebd-0d84-453a-a4fd-f33e040e5dda None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:56.870596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec9a84-f0d6-42a8-94f3-bd6d9a3892c8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:56.921171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:57.047541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a485ebd-0d84-453a-a4fd-f33e040e5dda None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:57.064253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec9a84-f0d6-42a8-94f3-bd6d9a3892c8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:57.117791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d184a7e-d7e9-411f-9607-31ca967e45cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:57.118626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d184a7e-d7e9-411f-9607-31ca967e45cc None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:57.118913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d184a7e-d7e9-411f-9607-31ca967e45cc None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:57.119194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d184a7e-d7e9-411f-9607-31ca967e45cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:57.131259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74a975-6939-441f-8052-61c75e08fa9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:57.132075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74a975-6939-441f-8052-61c75e08fa9b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:57.132357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74a975-6939-441f-8052-61c75e08fa9b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:57.132630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74a975-6939-441f-8052-61c75e08fa9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:57.143455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d184a7e-d7e9-411f-9607-31ca967e45cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:57.153717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-8d184a7e-d7e9-411f-9607-31ca967e45cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:41:57.153717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:57.153717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:57.153717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:57.153717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:57.153717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:57.153717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:57.153717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:57.153717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:57.153717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:57.153717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:57.153717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:57.153717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:57.153717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:57.153717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:57.153717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:57.154647 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:57.154647 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:57.154647 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:57.154647 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:57.154647 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:57.154647 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:57.154647 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:57.154647 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:57.154647 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:57.154647 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:57.154647 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:57.154647 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:57.154647 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:57.154647 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:57.154647 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:57.154647 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:57.155455 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:57.155455 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:57.155455 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:57.155455 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:57.155455 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:57.155455 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:57.155455 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:57.155455 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:57.155455 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:57.155455 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:57.155455 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:41:57.155455 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:57.155455 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 253/485] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:41:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:57.161986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d184a7e-d7e9-411f-9607-31ca967e45cc None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:57.166554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74a975-6939-441f-8052-61c75e08fa9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:57.173785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:57.174493 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-0d74a975-6939-441f-8052-61c75e08fa9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:41:57.174493 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:57.174493 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:57.174493 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:57.174493 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:57.174493 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:57.174493 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:57.174493 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:57.174493 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:57.174493 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:57.174493 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:57.174493 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:57.174493 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:57.174493 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:57.174493 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:57.174493 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:57.175381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:57.175381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:57.175381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:57.175381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:57.175381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:57.175381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:57.175381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:57.175381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:57.175381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:57.175381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:57.175381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:57.175381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:57.175381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:57.175381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:57.175381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:57.175381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:57.176899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:57.176899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:57.176899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:57.176899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:57.176899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:57.176899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:57.176899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:57.176899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:57.176899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:57.176899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:57.176899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:41:57.176899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:57.176899 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 233/486] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:41:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:57.186838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74a975-6939-441f-8052-61c75e08fa9b None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:57.267710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d184a7e-d7e9-411f-9607-31ca967e45cc None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:57.306295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e1812-55a5-4f81-95e4-b76f9cfcc347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:57.306966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e1812-55a5-4f81-95e4-b76f9cfcc347 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:57.307258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e1812-55a5-4f81-95e4-b76f9cfcc347 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:57.307536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e1812-55a5-4f81-95e4-b76f9cfcc347 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:57.308289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e1812-55a5-4f81-95e4-b76f9cfcc347 None admin] RBAC: Adding query filter params (name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:57.308594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e1812-55a5-4f81-95e4-b76f9cfcc347 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:57.309997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e1812-55a5-4f81-95e4-b76f9cfcc347 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:57.322508 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 254/487] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:41:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:57.323853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1dccd-6507-4c6e-9f97-02a4d91ed94f None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:57.324139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1dccd-6507-4c6e-9f97-02a4d91ed94f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:57.324415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1dccd-6507-4c6e-9f97-02a4d91ed94f None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:57.325881 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 255/488] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:57.326401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1dccd-6507-4c6e-9f97-02a4d91ed94f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:57.342470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:57.353363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74a975-6939-441f-8052-61c75e08fa9b None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:57.390376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7bd8d3-a41c-47c5-b7ac-1b936e30753c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:57.391315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bd8d3-a41c-47c5-b7ac-1b936e30753c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:57.391597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bd8d3-a41c-47c5-b7ac-1b936e30753c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:57.391975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7bd8d3-a41c-47c5-b7ac-1b936e30753c None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:57.392847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bd8d3-a41c-47c5-b7ac-1b936e30753c None admin] RBAC: Adding query filter params (name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:57.393161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bd8d3-a41c-47c5-b7ac-1b936e30753c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:57.394600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7bd8d3-a41c-47c5-b7ac-1b936e30753c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:57.403188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:57.404772 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 234/489] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:41:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:57.406055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9f6c3-5b85-480e-9f63-11e872205c02 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:57.406344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9f6c3-5b85-480e-9f63-11e872205c02 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:57.406621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9f6c3-5b85-480e-9f63-11e872205c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:57.425309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c6b9f6c3-5b85-480e-9f63-11e872205c02 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:57.427023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c6b9f6c3-5b85-480e-9f63-11e872205c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:57.437375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1dccd-6507-4c6e-9f97-02a4d91ed94f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:57.476274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2610d23-a1a8-47ce-9863-b999301c9fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:57.477150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2610d23-a1a8-47ce-9863-b999301c9fd2 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:57.477431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2610d23-a1a8-47ce-9863-b999301c9fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:57.477763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2610d23-a1a8-47ce-9863-b999301c9fd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:57.491070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2610d23-a1a8-47ce-9863-b999301c9fd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:57.492640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2610d23-a1a8-47ce-9863-b999301c9fd2 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:57.506373 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 235/490] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:57.507235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9f6c3-5b85-480e-9f63-11e872205c02 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:57.518211 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-f2610d23-a1a8-47ce-9863-b999301c9fd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:57.518211 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:57.518211 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:57.518211 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:57.518211 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:57.518211 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:57.518211 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:57.518211 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:57.518211 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:57.518211 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:57.518211 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:57.518211 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:57.518211 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:41:57.518211 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:41:57.518211 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:41:57.519083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:41:57.519083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:57.519083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:57.519083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:57.519083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:57.519083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:57.519083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:57.519083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:57.519083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:57.519083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:57.519083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:57.519083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:57.519083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:57.519083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:57.519083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:57.519083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:57.519083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:57.520893 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:41:57.520893 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:41:57.520893 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:57.520893 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:57.520893 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 256/491] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:57.524864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2610d23-a1a8-47ce-9863-b999301c9fd2 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:57.599135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:57.620438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9f6c3-5b85-480e-9f63-11e872205c02 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:57.650498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2610d23-a1a8-47ce-9863-b999301c9fd2 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:57.658774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf25cd-2734-458b-b494-b617461570e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:57.659622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf25cd-2734-458b-b494-b617461570e0 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:57.660016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf25cd-2734-458b-b494-b617461570e0 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:57.660301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf25cd-2734-458b-b494-b617461570e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:57.671896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf25cd-2734-458b-b494-b617461570e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:57.673388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf25cd-2734-458b-b494-b617461570e0 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:57.674561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:57.686628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb16d6-e5d2-4b50-88c2-9a52012ebfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:57.687435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb16d6-e5d2-4b50-88c2-9a52012ebfc2 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:57.687608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb16d6-e5d2-4b50-88c2-9a52012ebfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:57.687878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb16d6-e5d2-4b50-88c2-9a52012ebfc2 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:57.688610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb16d6-e5d2-4b50-88c2-9a52012ebfc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:57.688915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb16d6-e5d2-4b50-88c2-9a52012ebfc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:57.690320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb16d6-e5d2-4b50-88c2-9a52012ebfc2 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:57.692028 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-0acf25cd-2734-458b-b494-b617461570e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:57.692028 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:57.692028 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:57.692028 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:57.692028 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:57.692028 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:57.692028 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:57.692028 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:57.692028 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:57.692028 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:57.692028 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:57.692028 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:57.692028 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:41:57.692028 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:41:57.692028 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:41:57.692855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:41:57.692855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:57.692855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:57.692855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:57.692855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:57.692855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:57.692855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:57.692855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:57.692855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:57.692855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:57.692855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:57.692855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:57.692855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:57.692855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:57.692855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:57.692855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:57.692855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:57.693716 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:41:57.693716 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:41:57.693716 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:57.693716 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:57.693716 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 236/492] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:57.695004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf25cd-2734-458b-b494-b617461570e0 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:57.708115 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 257/493] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:41:57.709471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a62a3-a488-425c-86c9-66de1d85ff83 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:57.709764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a62a3-a488-425c-86c9-66de1d85ff83 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:57.710066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a62a3-a488-425c-86c9-66de1d85ff83 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:57.713971 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 258/494] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:57.713971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a62a3-a488-425c-86c9-66de1d85ff83 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:57.810500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf25cd-2734-458b-b494-b617461570e0 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:57.821803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a62a3-a488-425c-86c9-66de1d85ff83 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:57.846766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fd84b-a105-44a0-b061-eaf3705adaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:57.847660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fd84b-a105-44a0-b061-eaf3705adaf6 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:57.847994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fd84b-a105-44a0-b061-eaf3705adaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:57.848911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fd84b-a105-44a0-b061-eaf3705adaf6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:57.850802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:57.857039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1f444-b139-4dc8-be25-9d4fb81a22df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:57.857768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1f444-b139-4dc8-be25-9d4fb81a22df None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:57.867854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1f444-b139-4dc8-be25-9d4fb81a22df None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:57.867854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1f444-b139-4dc8-be25-9d4fb81a22df None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:57.867854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1f444-b139-4dc8-be25-9d4fb81a22df None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:57.867854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1f444-b139-4dc8-be25-9d4fb81a22df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:57.868119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75fd84b-a105-44a0-b061-eaf3705adaf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:57.868206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1f444-b139-4dc8-be25-9d4fb81a22df None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:57.868431 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-d75fd84b-a105-44a0-b061-eaf3705adaf6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:41:57.868431 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:57.868431 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:57.868431 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:57.868431 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:57.868431 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:57.868431 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:57.868431 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:57.868431 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:57.868431 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:57.868431 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:57.868431 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:57.868431 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:57.868431 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:57.868431 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:57.868431 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:57.869258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:57.869258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:57.869258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:57.869258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:57.869258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:57.869258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:57.869258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:57.869258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:57.869258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:57.869258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:57.869258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:57.869258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:57.869258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:57.869258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:57.869258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:57.869258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:57.870441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:57.870441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:57.870441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:57.870441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:57.870441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:57.870441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:57.870441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:57.870441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:57.870441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:57.870441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:57.870441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:41:57.870441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:57.870441 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 237/495] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:41:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:57.870441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fd84b-a105-44a0-b061-eaf3705adaf6 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:57.881388 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 259/496] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:57.881388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de748808-6139-4b74-89ce-8f8d6af71ff1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:57.881388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de748808-6139-4b74-89ce-8f8d6af71ff1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:57.881388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de748808-6139-4b74-89ce-8f8d6af71ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:57.896026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-de748808-6139-4b74-89ce-8f8d6af71ff1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:57.897218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-de748808-6139-4b74-89ce-8f8d6af71ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:57.931518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:57.973471 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 260/497] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:57.973471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de748808-6139-4b74-89ce-8f8d6af71ff1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:57.994285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fd84b-a105-44a0-b061-eaf3705adaf6 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:58.032120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23d928-2017-447b-9188-e3da082857c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:58.033192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23d928-2017-447b-9188-e3da082857c7 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:58.033192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23d928-2017-447b-9188-e3da082857c7 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:58.033465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da23d928-2017-447b-9188-e3da082857c7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:58.041246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23d928-2017-447b-9188-e3da082857c7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:58.042741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da23d928-2017-447b-9188-e3da082857c7 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:58.059747 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-da23d928-2017-447b-9188-e3da082857c7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 20:41:58.059747 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:58.059747 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:58.059747 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:58.059747 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:58.059747 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:58.059747 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:58.059747 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:58.059747 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:58.059747 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:58.059747 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:58.059747 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:58.059747 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:41:58.059747 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:41:58.059747 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:41:58.059747 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:58.060655 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:58.061627 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:58.061627 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:58.061627 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:41:58.061627 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:41:58.061627 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:41:58.061627 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:41:58.061627 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:41:58.061627 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:41:58.061627 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 20:41:58.061627 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:58.061627 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 238/498] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 20:41:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:58.061627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23d928-2017-447b-9188-e3da082857c7 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:58.088251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de748808-6139-4b74-89ce-8f8d6af71ff1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:58.123272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be4eb3-ccfc-4bd6-b385-dd97ae08d2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:58.123943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be4eb3-ccfc-4bd6-b385-dd97ae08d2a6 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:58.124075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:58.124250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be4eb3-ccfc-4bd6-b385-dd97ae08d2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:58.124496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be4eb3-ccfc-4bd6-b385-dd97ae08d2a6 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:58.125258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be4eb3-ccfc-4bd6-b385-dd97ae08d2a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:58.125561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be4eb3-ccfc-4bd6-b385-dd97ae08d2a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:58.126968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be4eb3-ccfc-4bd6-b385-dd97ae08d2a6 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:58.135882 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 261/499] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:41:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:58.136730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be4eb3-ccfc-4bd6-b385-dd97ae08d2a6 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:58.171302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:58.173406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da23d928-2017-447b-9188-e3da082857c7 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:58.206166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680dd6b-cb00-4085-9f7f-2f1a034297eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:58.206955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680dd6b-cb00-4085-9f7f-2f1a034297eb None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:58.207270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680dd6b-cb00-4085-9f7f-2f1a034297eb None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:58.208073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680dd6b-cb00-4085-9f7f-2f1a034297eb None admin] PATH_INFO: `/v3/users/glance` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:58.215588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680dd6b-cb00-4085-9f7f-2f1a034297eb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:58.217051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680dd6b-cb00-4085-9f7f-2f1a034297eb None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:58.230873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-f680dd6b-cb00-4085-9f7f-2f1a034297eb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 28 20:41:58.230873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:58.230873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:58.230873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:58.230873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:58.230873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:58.230873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:58.230873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:58.230873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:58.230873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:58.230873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:58.230873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:58.230873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:41:58.230873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:41:58.230873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:41:58.230873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:58.231810 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:58.232718 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:58.232718 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:58.232718 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:41:58.232718 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:41:58.232718 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:41:58.232718 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:41:58.232718 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:41:58.232718 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:41:58.232718 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 28 20:41:58.232718 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:58.232718 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 239/500] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:58.232718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680dd6b-cb00-4085-9f7f-2f1a034297eb None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:58.238380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be4eb3-ccfc-4bd6-b385-dd97ae08d2a6 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:58.272773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0f76f-1e93-4572-a0dc-572532a28a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:58.273560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0f76f-1e93-4572-a0dc-572532a28a23 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:58.273878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0f76f-1e93-4572-a0dc-572532a28a23 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:58.274148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0f76f-1e93-4572-a0dc-572532a28a23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:58.285787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0f76f-1e93-4572-a0dc-572532a28a23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:41:58.294176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-1ec0f76f-1e93-4572-a0dc-572532a28a23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:41:58.294176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:58.294176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:58.294176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:58.294176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:58.294176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:58.294176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:58.294176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:58.294176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:58.294176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:58.294176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:58.294176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:58.294176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:41:58.294176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:41:58.294176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:41:58.294176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:41:58.295105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:41:58.295105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:41:58.295105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:41:58.295105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:41:58.295105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:58.295105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:58.295105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:58.295105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:58.295105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:58.295105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:58.295105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:58.295105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:58.295105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:58.295105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:58.295105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:58.295105 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:58.296067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:58.296067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:58.296067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:58.296067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:58.296067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:41:58.296067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:41:58.296067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:41:58.296067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:41:58.296067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:41:58.296067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:41:58.296067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:41:58.296067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:58.296067 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 262/501] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:41:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:58.297333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0f76f-1e93-4572-a0dc-572532a28a23 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:58.335771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680dd6b-cb00-4085-9f7f-2f1a034297eb None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:58.348888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:58.370151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501bea5-bcfe-4702-9429-2189050dc9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:58.370820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501bea5-bcfe-4702-9429-2189050dc9ca None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:58.371106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501bea5-bcfe-4702-9429-2189050dc9ca None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:58.371395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501bea5-bcfe-4702-9429-2189050dc9ca None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:58.372232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501bea5-bcfe-4702-9429-2189050dc9ca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:58.381519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501bea5-bcfe-4702-9429-2189050dc9ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:58.381519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d501bea5-bcfe-4702-9429-2189050dc9ca None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:58.396618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:58.403521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0f76f-1e93-4572-a0dc-572532a28a23 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:58.410151 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 240/502] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 20:41:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:58.411893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e4066-cdaf-45b5-b785-831791e1fc5b None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:58.412536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e4066-cdaf-45b5-b785-831791e1fc5b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:58.412830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9e4066-cdaf-45b5-b785-831791e1fc5b None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:58.414672 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 241/503] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:41:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:41:58.415364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e4066-cdaf-45b5-b785-831791e1fc5b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:58.439013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9e634-936e-4392-b482-628e1bd6081b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:58.439807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9e634-936e-4392-b482-628e1bd6081b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:58.440098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9e634-936e-4392-b482-628e1bd6081b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:58.440378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a9e634-936e-4392-b482-628e1bd6081b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:58.451405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a9e634-936e-4392-b482-628e1bd6081b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:58.452880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a9e634-936e-4392-b482-628e1bd6081b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:58.469585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-39a9e634-936e-4392-b482-628e1bd6081b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:58.469585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:58.469585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:58.469585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:58.469585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:58.469585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:58.469585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:58.469585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:58.469585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:58.469585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:58.469585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:58.469585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:58.469585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:41:58.469585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:41:58.469585 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:41:58.470473 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:41:58.470473 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:58.470473 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:58.470473 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:58.470473 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:58.470473 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:58.470473 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:58.470473 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:58.470473 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:58.470473 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:58.470473 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:58.470473 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:58.470473 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:58.470473 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:58.470473 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:58.470473 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:58.470473 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:58.471335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:41:58.471335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:41:58.471335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:58.471335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:58.471335 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 263/504] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:58.471969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9e634-936e-4392-b482-628e1bd6081b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:58.523634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9e4066-cdaf-45b5-b785-831791e1fc5b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:58.557987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913f81d-6438-4ae8-9fae-eead5a336ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:58.558658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913f81d-6438-4ae8-9fae-eead5a336ec9 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:58.558941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913f81d-6438-4ae8-9fae-eead5a336ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:58.559227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913f81d-6438-4ae8-9fae-eead5a336ec9 None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:58.560066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913f81d-6438-4ae8-9fae-eead5a336ec9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:58.560377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913f81d-6438-4ae8-9fae-eead5a336ec9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:58.561829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913f81d-6438-4ae8-9fae-eead5a336ec9 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:58.574378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a9e634-936e-4392-b482-628e1bd6081b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:58.579834 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 242/505] 10.222.0.33 () {56 vars in 1137 bytes} [Sat May 28 20:41:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:58.580639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913f81d-6438-4ae8-9fae-eead5a336ec9 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:58.606984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ad9ae-7b10-4f89-8aa6-9cf3402f7bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:58.607657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ad9ae-7b10-4f89-8aa6-9cf3402f7bdb None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:58.607947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ad9ae-7b10-4f89-8aa6-9cf3402f7bdb None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:58.608233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ad9ae-7b10-4f89-8aa6-9cf3402f7bdb None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:58.608992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ad9ae-7b10-4f89-8aa6-9cf3402f7bdb None admin] RBAC: Adding query filter params (name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:58.609304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ad9ae-7b10-4f89-8aa6-9cf3402f7bdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:58.610735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ad9ae-7b10-4f89-8aa6-9cf3402f7bdb None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:58.611212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:58.619391 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 264/506] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:41:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:58.622650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cc039-ce7b-44d6-938f-6fb248ffe7dd None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:58.622650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cc039-ce7b-44d6-938f-6fb248ffe7dd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:58.622650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cc039-ce7b-44d6-938f-6fb248ffe7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:58.637509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-023cc039-ce7b-44d6-938f-6fb248ffe7dd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:41:58.638797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-023cc039-ce7b-44d6-938f-6fb248ffe7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:41:58.640968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:58.681163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913f81d-6438-4ae8-9fae-eead5a336ec9 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:58.704633 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 265/507] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:41:58.705523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cc039-ce7b-44d6-938f-6fb248ffe7dd None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:58.715190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1fde9-4360-4076-b207-1beb9204fd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:58.715841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1fde9-4360-4076-b207-1beb9204fd5b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:58.716125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1fde9-4360-4076-b207-1beb9204fd5b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:58.716404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b1fde9-4360-4076-b207-1beb9204fd5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:58.786184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1fde9-4360-4076-b207-1beb9204fd5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:58.787586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b1fde9-4360-4076-b207-1beb9204fd5b None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:58.807347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cc039-ce7b-44d6-938f-6fb248ffe7dd None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:58.816548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:58.840545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e230d-81a5-410d-9f5f-9a68007d9945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:58.841212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e230d-81a5-410d-9f5f-9a68007d9945 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:58.841499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e230d-81a5-410d-9f5f-9a68007d9945 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:58.841779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e230d-81a5-410d-9f5f-9a68007d9945 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:58.842541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e230d-81a5-410d-9f5f-9a68007d9945 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:58.842848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e230d-81a5-410d-9f5f-9a68007d9945 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:58.844326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e230d-81a5-410d-9f5f-9a68007d9945 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:58.851624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:58.856469 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 266/508] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:58.856469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e230d-81a5-410d-9f5f-9a68007d9945 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:58.884808 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 243/509] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:58.885641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1fde9-4360-4076-b207-1beb9204fd5b None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:58.956095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e230d-81a5-410d-9f5f-9a68007d9945 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:58.982066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b1fde9-4360-4076-b207-1beb9204fd5b None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:58.989479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5711416-459e-4919-bbe5-22f0fa6496df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:58.990166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5711416-459e-4919-bbe5-22f0fa6496df None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:58.990457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5711416-459e-4919-bbe5-22f0fa6496df None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:58.990741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5711416-459e-4919-bbe5-22f0fa6496df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:59.009903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:59.015077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833fc45-b536-47bd-8c5a-1ff7c3ae2acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:59.016104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833fc45-b536-47bd-8c5a-1ff7c3ae2acd None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:59.016378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833fc45-b536-47bd-8c5a-1ff7c3ae2acd None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:59.016651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833fc45-b536-47bd-8c5a-1ff7c3ae2acd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:59.026416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833fc45-b536-47bd-8c5a-1ff7c3ae2acd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:59.027887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833fc45-b536-47bd-8c5a-1ff7c3ae2acd None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:59.045216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-4833fc45-b536-47bd-8c5a-1ff7c3ae2acd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:59.045216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:59.045216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:59.045216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:59.045216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:59.045216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:59.045216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:59.045216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:59.045216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:59.045216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:59.045216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:59.045216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:59.045216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:41:59.045216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:41:59.045216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:41:59.046230 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:41:59.046230 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:59.046230 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:59.046230 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:59.046230 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:59.046230 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:59.046230 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:59.046230 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:59.046230 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:59.046230 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:59.046230 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:59.046230 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:59.046230 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:59.046230 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:59.046230 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:59.046230 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:59.046230 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:59.047167 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:41:59.047167 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:41:59.047167 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:59.047167 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:59.047167 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 244/510] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:59.047167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833fc45-b536-47bd-8c5a-1ff7c3ae2acd None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:59.057296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:59.061421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5711416-459e-4919-bbe5-22f0fa6496df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:59.062951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5711416-459e-4919-bbe5-22f0fa6496df None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:59.149445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833fc45-b536-47bd-8c5a-1ff7c3ae2acd None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:59.163846 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 267/511] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:41:59.164688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5711416-459e-4919-bbe5-22f0fa6496df None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:59.184384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d70ab6-799e-41cd-b9f8-5cff3cf18a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:59.185174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d70ab6-799e-41cd-b9f8-5cff3cf18a2e None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:59.185456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d70ab6-799e-41cd-b9f8-5cff3cf18a2e None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:59.185730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d70ab6-799e-41cd-b9f8-5cff3cf18a2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:59.195710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d70ab6-799e-41cd-b9f8-5cff3cf18a2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:59.197176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d70ab6-799e-41cd-b9f8-5cff3cf18a2e None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:59.215037 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-12d70ab6-799e-41cd-b9f8-5cff3cf18a2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:59.215037 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:59.215037 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:59.215037 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:59.215037 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:59.215037 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:59.215037 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:59.215037 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:59.215037 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:59.215037 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:59.215037 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:59.215037 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:59.215037 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:41:59.215037 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:41:59.215037 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:41:59.215949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:41:59.215949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:59.215949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:59.215949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:59.215949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:59.215949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:59.215949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:59.215949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:59.215949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:59.215949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:59.215949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:59.215949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:59.215949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:59.215949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:59.215949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:59.215949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:59.215949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:59.216774 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:41:59.216774 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:41:59.216774 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:59.216774 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:59.216774 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 245/512] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:59.217042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d70ab6-799e-41cd-b9f8-5cff3cf18a2e None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:59.249695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:59.265108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:59.269188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5711416-459e-4919-bbe5-22f0fa6496df None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:59.303866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a181276-e31f-44d4-92b6-40c0014df166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:59.304581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a181276-e31f-44d4-92b6-40c0014df166 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:59.304870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a181276-e31f-44d4-92b6-40c0014df166 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:59.305142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a181276-e31f-44d4-92b6-40c0014df166 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:59.312030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a181276-e31f-44d4-92b6-40c0014df166 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:59.313469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a181276-e31f-44d4-92b6-40c0014df166 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:59.316106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d70ab6-799e-41cd-b9f8-5cff3cf18a2e None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:59.328546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-8a181276-e31f-44d4-92b6-40c0014df166 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 28 20:41:59.328546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:59.328546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:59.328546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:59.328546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:59.328546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:59.328546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:59.328546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:59.328546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:59.328546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:59.328546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:59.328546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:59.328546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:41:59.328546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:41:59.328546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:41:59.328546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:59.329642 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:59.330602 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:59.330602 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:59.330602 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:41:59.330602 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:41:59.330602 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:41:59.330602 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:41:59.330602 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:41:59.330602 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:41:59.330602 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 28 20:41:59.330602 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:41:59.330602 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 268/513] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:41:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:59.330602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a181276-e31f-44d4-92b6-40c0014df166 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:59.355132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba678364-996f-4187-899d-16998a585dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:59.355920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba678364-996f-4187-899d-16998a585dc6 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:59.356203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba678364-996f-4187-899d-16998a585dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:59.356483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba678364-996f-4187-899d-16998a585dc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:59.367481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba678364-996f-4187-899d-16998a585dc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:59.368941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba678364-996f-4187-899d-16998a585dc6 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:59.387318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-ba678364-996f-4187-899d-16998a585dc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:59.387318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:59.387318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:59.387318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:59.387318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:59.387318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:59.387318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:59.387318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:59.387318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:59.387318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:59.387318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:59.387318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:59.387318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:41:59.387318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:41:59.387318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:41:59.388265 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:41:59.388265 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:59.388265 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:59.388265 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:59.388265 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:59.388265 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:59.388265 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:59.388265 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:59.388265 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:59.388265 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:59.388265 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:59.388265 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:59.388265 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:59.388265 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:59.388265 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:59.388265 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:59.388265 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:59.389094 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:41:59.389094 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:41:59.389094 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:59.389094 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:59.389094 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 246/514] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:59.389664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba678364-996f-4187-899d-16998a585dc6 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:59.434404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a181276-e31f-44d4-92b6-40c0014df166 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:59.467627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ee39c-e2c8-444c-bb4f-b417496cae84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:59.468288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ee39c-e2c8-444c-bb4f-b417496cae84 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:59.468604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ee39c-e2c8-444c-bb4f-b417496cae84 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:59.468890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ee39c-e2c8-444c-bb4f-b417496cae84 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:59.469618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ee39c-e2c8-444c-bb4f-b417496cae84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:59.469970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ee39c-e2c8-444c-bb4f-b417496cae84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:59.471361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ee39c-e2c8-444c-bb4f-b417496cae84 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:59.477016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:59.484867 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 269/515] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:59.484867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ee39c-e2c8-444c-bb4f-b417496cae84 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:59.491370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba678364-996f-4187-899d-16998a585dc6 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:59.504721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:59.526584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adde0fc-8a00-4345-9fac-2ba15df909fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:59.527376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adde0fc-8a00-4345-9fac-2ba15df909fb None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:59.527666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adde0fc-8a00-4345-9fac-2ba15df909fb None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:59.527977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adde0fc-8a00-4345-9fac-2ba15df909fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:59.537882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adde0fc-8a00-4345-9fac-2ba15df909fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:59.539353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adde0fc-8a00-4345-9fac-2ba15df909fb None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:59.556457 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-9adde0fc-8a00-4345-9fac-2ba15df909fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:59.556457 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:59.556457 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:59.556457 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:59.556457 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:59.556457 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:59.556457 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:59.556457 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:59.556457 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:59.556457 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:59.556457 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:59.556457 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:59.556457 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:41:59.556457 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:41:59.556457 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:41:59.557441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:41:59.557441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:59.557441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:59.557441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:59.557441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:59.557441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:59.557441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:59.557441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:59.557441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:59.557441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:59.557441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:59.557441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:59.557441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:59.557441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:59.557441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:59.557441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:59.557441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:59.558501 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:41:59.558501 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:41:59.558501 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:41:59.558501 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:59.558501 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 247/516] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:59.558778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adde0fc-8a00-4345-9fac-2ba15df909fb None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:59.590158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ee39c-e2c8-444c-bb4f-b417496cae84 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:59.623224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2f8bc-ea39-4543-9e0b-9d755e1a23f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:59.623888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2f8bc-ea39-4543-9e0b-9d755e1a23f7 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:59.624168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2f8bc-ea39-4543-9e0b-9d755e1a23f7 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:59.624442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2f8bc-ea39-4543-9e0b-9d755e1a23f7 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:59.625161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2f8bc-ea39-4543-9e0b-9d755e1a23f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:59.625463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2f8bc-ea39-4543-9e0b-9d755e1a23f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:59.626852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2f8bc-ea39-4543-9e0b-9d755e1a23f7 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:59.638723 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 270/517] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:59.639548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2f8bc-ea39-4543-9e0b-9d755e1a23f7 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:59.661006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adde0fc-8a00-4345-9fac-2ba15df909fb None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:59.694625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af5627-ac4f-4a2b-b9c6-8520fc3d8f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:59.695296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af5627-ac4f-4a2b-b9c6-8520fc3d8f89 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:59.695582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af5627-ac4f-4a2b-b9c6-8520fc3d8f89 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:59.695863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af5627-ac4f-4a2b-b9c6-8520fc3d8f89 None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:59.696704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af5627-ac4f-4a2b-b9c6-8520fc3d8f89 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:59.697018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af5627-ac4f-4a2b-b9c6-8520fc3d8f89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:59.698501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af5627-ac4f-4a2b-b9c6-8520fc3d8f89 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:59.706489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:59.716532 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 248/518] 10.222.0.33 () {56 vars in 1137 bytes} [Sat May 28 20:41:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:59.718374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af5627-ac4f-4a2b-b9c6-8520fc3d8f89 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:59.718481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:59.738510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2f8bc-ea39-4543-9e0b-9d755e1a23f7 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:59.771857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf58a35-5d1c-49cc-a6d3-25172e3be047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:59.772535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf58a35-5d1c-49cc-a6d3-25172e3be047 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:59.772954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf58a35-5d1c-49cc-a6d3-25172e3be047 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:59.773623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf58a35-5d1c-49cc-a6d3-25172e3be047 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:59.774480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf58a35-5d1c-49cc-a6d3-25172e3be047 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:59.774819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf58a35-5d1c-49cc-a6d3-25172e3be047 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:59.776203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf58a35-5d1c-49cc-a6d3-25172e3be047 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:59.787630 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 271/519] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:59.788410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf58a35-5d1c-49cc-a6d3-25172e3be047 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:59.819055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af5627-ac4f-4a2b-b9c6-8520fc3d8f89 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:59.852093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3fed7-4f07-4acf-93e3-54934311dd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:59.852882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3fed7-4f07-4acf-93e3-54934311dd9b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:59.853165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3fed7-4f07-4acf-93e3-54934311dd9b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:59.853443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3fed7-4f07-4acf-93e3-54934311dd9b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:59.860349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3fed7-4f07-4acf-93e3-54934311dd9b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:59.861790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3fed7-4f07-4acf-93e3-54934311dd9b None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:59.876703 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-c0e3fed7-4f07-4acf-93e3-54934311dd9b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 28 20:41:59.876703 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:41:59.876703 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:41:59.876703 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:41:59.876703 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:41:59.876703 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:41:59.876703 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:41:59.876703 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:41:59.876703 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:41:59.876703 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:41:59.876703 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:41:59.876703 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:41:59.876703 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:41:59.876703 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:41:59.876703 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:41:59.876703 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:41:59.877985 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:41:59.879118 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:41:59.879118 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:41:59.879118 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:41:59.879118 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:41:59.879118 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:41:59.879118 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:41:59.879118 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:41:59.879118 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:41:59.879118 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 28 20:41:59.879118 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:41:59.879118 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 249/520] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:41:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:41:59.879118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3fed7-4f07-4acf-93e3-54934311dd9b None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:59.891450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf58a35-5d1c-49cc-a6d3-25172e3be047 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:41:59.925874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841f7f9-be31-4227-8c66-bb93280c9fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:41:59.927190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841f7f9-be31-4227-8c66-bb93280c9fed None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:41:59.927482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841f7f9-be31-4227-8c66-bb93280c9fed None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:41:59.927762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841f7f9-be31-4227-8c66-bb93280c9fed None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:41:59.928497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841f7f9-be31-4227-8c66-bb93280c9fed None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:41:59.928805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841f7f9-be31-4227-8c66-bb93280c9fed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:41:59.930238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841f7f9-be31-4227-8c66-bb93280c9fed None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:41:59.935842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:59.936985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:41:59.942404 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 272/521] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:41:59.943183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841f7f9-be31-4227-8c66-bb93280c9fed None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:41:59.978045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3fed7-4f07-4acf-93e3-54934311dd9b None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:00.011864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e4a4a-46b9-4e55-87d4-a2538393ec59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:00.012526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e4a4a-46b9-4e55-87d4-a2538393ec59 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:00.012812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e4a4a-46b9-4e55-87d4-a2538393ec59 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:00.013089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e4a4a-46b9-4e55-87d4-a2538393ec59 None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:00.013794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e4a4a-46b9-4e55-87d4-a2538393ec59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:00.015273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e4a4a-46b9-4e55-87d4-a2538393ec59 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:00.038717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841f7f9-be31-4227-8c66-bb93280c9fed None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:00.072177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb1490-92dd-47b6-956b-4b8e70c7c0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:00.072730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1490-92dd-47b6-956b-4b8e70c7c0f1 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:00.073015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1490-92dd-47b6-956b-4b8e70c7c0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:00.073293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb1490-92dd-47b6-956b-4b8e70c7c0f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:00.135263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:00.136576 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 250/522] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:41:59 2022] POST /identity/v3/users => generated 247 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:42:00.137397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e4a4a-46b9-4e55-87d4-a2538393ec59 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:00.138805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb1490-92dd-47b6-956b-4b8e70c7c0f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:00.140286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb1490-92dd-47b6-956b-4b8e70c7c0f1 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:00.254145 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 273/523] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:41:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:42:00.255225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb1490-92dd-47b6-956b-4b8e70c7c0f1 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:00.278368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e4a4a-46b9-4e55-87d4-a2538393ec59 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:00.293349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:00.319524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af1710-926b-45c8-9920-d0d7bd127b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:00.320327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af1710-926b-45c8-9920-d0d7bd127b0c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:00.320609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af1710-926b-45c8-9920-d0d7bd127b0c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:00.320883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af1710-926b-45c8-9920-d0d7bd127b0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:00.334609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af1710-926b-45c8-9920-d0d7bd127b0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:00.337956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af1710-926b-45c8-9920-d0d7bd127b0c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:00.355237 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-32af1710-926b-45c8-9920-d0d7bd127b0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:00.355237 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:00.355237 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:00.355237 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:00.355237 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:00.355237 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:00.355237 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:00.355237 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:00.355237 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:00.355237 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:00.355237 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:00.355237 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:00.355237 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:00.355237 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:00.355237 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:00.356091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:00.356091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:00.356091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:00.356091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:00.356091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:00.356091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:00.356091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:00.356091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:00.356091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:00.356091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:00.356091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:00.356091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:00.356091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:00.356091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:00.356091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:00.356091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:00.356091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:00.356939 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:00.356939 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:00.356939 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:00.356939 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:00.356939 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:00.356939 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:00.356939 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:00.356939 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:00.356939 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 251/524] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:00.357399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af1710-926b-45c8-9920-d0d7bd127b0c None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:00.375683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:00.379676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb1490-92dd-47b6-956b-4b8e70c7c0f1 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:00.419468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f5d3d-564e-408e-82a1-4aaff1041d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:00.420242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f5d3d-564e-408e-82a1-4aaff1041d3d None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:00.420503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f5d3d-564e-408e-82a1-4aaff1041d3d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:00.420698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3f5d3d-564e-408e-82a1-4aaff1041d3d None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:00.421550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f5d3d-564e-408e-82a1-4aaff1041d3d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:00.421877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f5d3d-564e-408e-82a1-4aaff1041d3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:00.423330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3f5d3d-564e-408e-82a1-4aaff1041d3d None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:00.465851 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 274/525] 10.222.0.33 () {56 vars in 1139 bytes} [Sat May 28 20:42:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:00.465851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f5d3d-564e-408e-82a1-4aaff1041d3d None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:00.475954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af1710-926b-45c8-9920-d0d7bd127b0c None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:00.514754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc8384-f7f0-4623-9c7d-2c6fde23afb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:00.515906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc8384-f7f0-4623-9c7d-2c6fde23afb8 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:00.516192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc8384-f7f0-4623-9c7d-2c6fde23afb8 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:00.516467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc8384-f7f0-4623-9c7d-2c6fde23afb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:00.523253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc8384-f7f0-4623-9c7d-2c6fde23afb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:00.524701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc8384-f7f0-4623-9c7d-2c6fde23afb8 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:00.539870 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-96dc8384-f7f0-4623-9c7d-2c6fde23afb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:00.539870 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:00.539870 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:00.539870 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:00.539870 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:00.539870 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:00.539870 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:00.539870 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:00.539870 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:00.539870 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:00.539870 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:00.539870 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:00.539870 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:00.539870 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:00.539870 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:00.540748 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:00.540748 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:00.540748 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:00.540748 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:00.540748 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:00.540748 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:00.540748 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:00.540748 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:00.540748 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:00.540748 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:00.540748 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:00.540748 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:00.540748 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:00.540748 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:00.540748 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:00.540748 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:00.540748 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:00.541637 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:00.541637 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:00.541637 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:00.541637 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:00.541637 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:00.541637 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:00.541637 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:00.541637 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:00.541637 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 252/526] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:00.542705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc8384-f7f0-4623-9c7d-2c6fde23afb8 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:00.573327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:00.577275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3f5d3d-564e-408e-82a1-4aaff1041d3d None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:00.613040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6ed6a-5a32-4217-ae9f-530ebe756c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:00.614337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6ed6a-5a32-4217-ae9f-530ebe756c0e None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:00.614624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6ed6a-5a32-4217-ae9f-530ebe756c0e None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:00.614899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6ed6a-5a32-4217-ae9f-530ebe756c0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:00.627446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6ed6a-5a32-4217-ae9f-530ebe756c0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:00.628917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6ed6a-5a32-4217-ae9f-530ebe756c0e None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:00.649377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc8384-f7f0-4623-9c7d-2c6fde23afb8 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:00.652657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-b0d6ed6a-5a32-4217-ae9f-530ebe756c0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:00.652657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:00.652657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:00.652657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:00.652657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:00.652657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:00.652657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:00.652657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:00.652657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:00.652657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:00.652657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:00.652657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:00.652657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:00.652657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:00.652657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:00.653536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:00.653536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:00.653536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:00.653536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:00.653536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:00.653536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:00.653536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:00.653536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:00.653536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:00.653536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:00.653536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:00.653536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:00.653536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:00.653536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:00.653536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:00.653536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:00.653536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:00.659499 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:00.659499 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:00.659499 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:00.659499 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:00.659499 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 275/527] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:00.659499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6ed6a-5a32-4217-ae9f-530ebe756c0e None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:00.666339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:00.687853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbed9d3-0f06-4f83-839d-2d7ede3ea745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:00.688560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbed9d3-0f06-4f83-839d-2d7ede3ea745 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:00.688844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbed9d3-0f06-4f83-839d-2d7ede3ea745 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:00.689120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbed9d3-0f06-4f83-839d-2d7ede3ea745 None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:00.689944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbed9d3-0f06-4f83-839d-2d7ede3ea745 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:00.690263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbed9d3-0f06-4f83-839d-2d7ede3ea745 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:00.691727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbed9d3-0f06-4f83-839d-2d7ede3ea745 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:00.710706 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 253/528] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:00.711714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbed9d3-0f06-4f83-839d-2d7ede3ea745 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:00.767730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6ed6a-5a32-4217-ae9f-530ebe756c0e None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:00.805439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:00.805776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90955f8-bc53-4a01-9bbc-1ee8660c9561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:00.806866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90955f8-bc53-4a01-9bbc-1ee8660c9561 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:00.806866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90955f8-bc53-4a01-9bbc-1ee8660c9561 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:00.807059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90955f8-bc53-4a01-9bbc-1ee8660c9561 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:00.818723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbed9d3-0f06-4f83-839d-2d7ede3ea745 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:00.854795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ab139-6077-4667-a7c9-67b4df366ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:00.855456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ab139-6077-4667-a7c9-67b4df366ac7 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:00.855751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ab139-6077-4667-a7c9-67b4df366ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:00.856012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ab139-6077-4667-a7c9-67b4df366ac7 None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:00.856802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ab139-6077-4667-a7c9-67b4df366ac7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:00.857109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ab139-6077-4667-a7c9-67b4df366ac7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:00.858606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ab139-6077-4667-a7c9-67b4df366ac7 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:00.870658 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 254/529] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:00.871392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ab139-6077-4667-a7c9-67b4df366ac7 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:00.882881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90955f8-bc53-4a01-9bbc-1ee8660c9561 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:00.884403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90955f8-bc53-4a01-9bbc-1ee8660c9561 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:00.897155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:00.977824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ab139-6077-4667-a7c9-67b4df366ac7 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:00.997880 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 276/530] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:42:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:42:00.997880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90955f8-bc53-4a01-9bbc-1ee8660c9561 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:01.020421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbf2fe-aa12-42ca-9797-3405001a04c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:01.021098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbf2fe-aa12-42ca-9797-3405001a04c5 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:01.021538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbf2fe-aa12-42ca-9797-3405001a04c5 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:01.021851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbf2fe-aa12-42ca-9797-3405001a04c5 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:01.022572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fbf2fe-aa12-42ca-9797-3405001a04c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:01.022879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fbf2fe-aa12-42ca-9797-3405001a04c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:01.024274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fbf2fe-aa12-42ca-9797-3405001a04c5 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:01.036263 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 255/531] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:01.041306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbf2fe-aa12-42ca-9797-3405001a04c5 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:01.055663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:01.107520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90955f8-bc53-4a01-9bbc-1ee8660c9561 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:01.117862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:01.145438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eef9bc-b557-44c0-8f81-36a11cb733fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:01.146080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eef9bc-b557-44c0-8f81-36a11cb733fb None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:01.146368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eef9bc-b557-44c0-8f81-36a11cb733fb None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:01.146648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eef9bc-b557-44c0-8f81-36a11cb733fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:01.147517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eef9bc-b557-44c0-8f81-36a11cb733fb None admin] RBAC: Adding query filter params (role.id=4db40e559d064cf6b4f061e785b42dbb, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=cfdebb5e6b8d4e2d86f0a03a30c21671) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:01.148005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eef9bc-b557-44c0-8f81-36a11cb733fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:01.150376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73eef9bc-b557-44c0-8f81-36a11cb733fb None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:01.152450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbf2fe-aa12-42ca-9797-3405001a04c5 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:01.165878 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 277/532] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:00 2022] GET /identity/v3/role_assignments?user.id=cfdebb5e6b8d4e2d86f0a03a30c21671&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=4db40e559d064cf6b4f061e785b42dbb => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:01.165878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eef9bc-b557-44c0-8f81-36a11cb733fb None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:01.200577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29385b-5d11-474e-9124-cc0790a61522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:01.201630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29385b-5d11-474e-9124-cc0790a61522 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:01.202015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29385b-5d11-474e-9124-cc0790a61522 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:01.202789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f29385b-5d11-474e-9124-cc0790a61522 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:01.204007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29385b-5d11-474e-9124-cc0790a61522 None admin] RBAC: Adding query filter params (role.id=4db40e559d064cf6b4f061e785b42dbb, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=224c1a0427c942e5bfef2bac75648d80) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:01.204384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29385b-5d11-474e-9124-cc0790a61522 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:01.206374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f29385b-5d11-474e-9124-cc0790a61522 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:01.215321 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 256/533] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:01 2022] GET /identity/v3/role_assignments?user.id=224c1a0427c942e5bfef2bac75648d80&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=4db40e559d064cf6b4f061e785b42dbb => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:01.216623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29385b-5d11-474e-9124-cc0790a61522 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:01.306977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eef9bc-b557-44c0-8f81-36a11cb733fb None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:01.374248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:01.391390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c407217-b810-4944-a366-6cac97ebba0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:01.392297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c407217-b810-4944-a366-6cac97ebba0f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:01.392644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c407217-b810-4944-a366-6cac97ebba0f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:01.392920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c407217-b810-4944-a366-6cac97ebba0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:01.413393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c407217-b810-4944-a366-6cac97ebba0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:01.414894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c407217-b810-4944-a366-6cac97ebba0f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:01.420843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f29385b-5d11-474e-9124-cc0790a61522 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:01.448062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-8c407217-b810-4944-a366-6cac97ebba0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:01.448062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:01.448062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:01.448062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:01.448062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:01.448062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:01.448062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:01.448062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:01.448062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:01.448062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:01.448062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:01.448062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:01.448062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:01.448062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:01.448062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:01.448999 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:01.448999 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:01.448999 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:01.448999 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:01.448999 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:01.448999 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:01.448999 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:01.448999 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:01.448999 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:01.448999 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:01.448999 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:01.448999 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:01.448999 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:01.448999 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:01.448999 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:01.448999 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:01.448999 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:01.449877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:01.449877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:01.449877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:01.449877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:01.453942 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 278/534] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:01.455405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c407217-b810-4944-a366-6cac97ebba0f None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:01.484880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:01.502650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed4dc2-44e2-4479-8737-3dbcfb054784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:01.503565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed4dc2-44e2-4479-8737-3dbcfb054784 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:01.503851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed4dc2-44e2-4479-8737-3dbcfb054784 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:01.504174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed4dc2-44e2-4479-8737-3dbcfb054784 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:01.526716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed4dc2-44e2-4479-8737-3dbcfb054784 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:01.528175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fed4dc2-44e2-4479-8737-3dbcfb054784 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:01.555193 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-7fed4dc2-44e2-4479-8737-3dbcfb054784 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:01.555193 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:01.555193 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:01.555193 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:01.555193 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:01.555193 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:01.555193 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:01.555193 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:01.555193 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:01.555193 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:01.555193 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:01.555193 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:01.555193 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:01.555193 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:01.555193 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:01.556114 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:01.556114 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:01.556114 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:01.556114 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:01.556114 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:01.556114 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:01.556114 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:01.556114 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:01.556114 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:01.556114 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:01.556114 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:01.556114 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:01.556114 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:01.556114 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:01.556114 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:01.556114 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:01.556114 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:01.556949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:01.556949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:01.556949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:01.556949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:01.556949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:01.556949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:01.556949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:01.556949 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:01.556949 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 257/535] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:01.562152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed4dc2-44e2-4479-8737-3dbcfb054784 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:01.570299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c407217-b810-4944-a366-6cac97ebba0f None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:01.619596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947df2b-12f0-4ec2-b97c-8f44bfc6584a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:01.620630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947df2b-12f0-4ec2-b97c-8f44bfc6584a None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:01.621106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947df2b-12f0-4ec2-b97c-8f44bfc6584a None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:01.621752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947df2b-12f0-4ec2-b97c-8f44bfc6584a None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:01.623921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947df2b-12f0-4ec2-b97c-8f44bfc6584a None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:01.625124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947df2b-12f0-4ec2-b97c-8f44bfc6584a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:01.627366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947df2b-12f0-4ec2-b97c-8f44bfc6584a None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:01.651684 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 279/536] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:01.657071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947df2b-12f0-4ec2-b97c-8f44bfc6584a None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:01.764106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fed4dc2-44e2-4479-8737-3dbcfb054784 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:01.814350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:01.833142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947df2b-12f0-4ec2-b97c-8f44bfc6584a None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:01.834693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9d10a-178e-48c1-b164-803ed5bd9483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:01.835537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9d10a-178e-48c1-b164-803ed5bd9483 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:01.835824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9d10a-178e-48c1-b164-803ed5bd9483 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:01.836104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a9d10a-178e-48c1-b164-803ed5bd9483 None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:01.837735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9d10a-178e-48c1-b164-803ed5bd9483 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:01.838100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9d10a-178e-48c1-b164-803ed5bd9483 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:01.839553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a9d10a-178e-48c1-b164-803ed5bd9483 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:01.855891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:01.856273 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 258/537] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:01.861946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9d10a-178e-48c1-b164-803ed5bd9483 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:01.888173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c2ad3-bbbd-4e49-8bb0-d9d29717354f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:01.889306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c2ad3-bbbd-4e49-8bb0-d9d29717354f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:01.889765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c2ad3-bbbd-4e49-8bb0-d9d29717354f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:01.890256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c2ad3-bbbd-4e49-8bb0-d9d29717354f None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:01.902278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c2ad3-bbbd-4e49-8bb0-d9d29717354f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:01.904738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c2ad3-bbbd-4e49-8bb0-d9d29717354f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:01.932419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-ae6c2ad3-bbbd-4e49-8bb0-d9d29717354f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:01.932419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:01.932419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:01.932419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:01.932419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:01.932419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:01.932419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:01.932419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:01.932419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:01.932419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:01.932419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:01.932419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:01.932419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:01.932419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:01.932419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:01.933314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:01.933314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:01.933314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:01.933314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:01.933314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:01.933314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:01.933314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:01.933314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:01.933314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:01.933314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:01.933314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:01.933314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:01.933314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:01.933314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:01.933314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:01.933314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:01.933314 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:01.934181 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:01.934181 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:01.934181 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:01.934181 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:01.934181 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:01.934181 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:01.934181 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:01.934181 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:01.938724 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 280/538] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:42:01.949645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c2ad3-bbbd-4e49-8bb0-d9d29717354f None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:02.055453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a9d10a-178e-48c1-b164-803ed5bd9483 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:02.074102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c2ad3-bbbd-4e49-8bb0-d9d29717354f None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:02.115001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa83c1-4d22-4471-b471-284ec7e58dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:02.115685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa83c1-4d22-4471-b471-284ec7e58dc0 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:02.115974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa83c1-4d22-4471-b471-284ec7e58dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:02.116277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa83c1-4d22-4471-b471-284ec7e58dc0 None admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:02.117061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa83c1-4d22-4471-b471-284ec7e58dc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:02.117493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa83c1-4d22-4471-b471-284ec7e58dc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:02.119304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa83c1-4d22-4471-b471-284ec7e58dc0 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:02.126618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341de874-a44d-450f-8a56-de567038e884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:02.127561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341de874-a44d-450f-8a56-de567038e884 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:02.128039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341de874-a44d-450f-8a56-de567038e884 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:02.128528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341de874-a44d-450f-8a56-de567038e884 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:02.129656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341de874-a44d-450f-8a56-de567038e884 None admin] RBAC: Adding query filter params (role.id=4db40e559d064cf6b4f061e785b42dbb, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=afc08d15f24e46e29c34d618d955bd28) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:02.130198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341de874-a44d-450f-8a56-de567038e884 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:02.131878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341de874-a44d-450f-8a56-de567038e884 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:02.147613 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 259/539] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:01 2022] GET /identity/v3/role_assignments?user.id=afc08d15f24e46e29c34d618d955bd28&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=4db40e559d064cf6b4f061e785b42dbb => generated 259 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:02.154168 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 281/540] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:02.165916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341de874-a44d-450f-8a56-de567038e884 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:02.274517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:02.336839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341de874-a44d-450f-8a56-de567038e884 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:02.385395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0195a614-dc0c-4de1-b7ad-6de11976f571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:02.386075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195a614-dc0c-4de1-b7ad-6de11976f571 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:02.386359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195a614-dc0c-4de1-b7ad-6de11976f571 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:02.386634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0195a614-dc0c-4de1-b7ad-6de11976f571 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:02.387472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195a614-dc0c-4de1-b7ad-6de11976f571 None admin] RBAC: Adding query filter params (role.id=4db40e559d064cf6b4f061e785b42dbb, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=1b2a9ebcd3f84b758322a80c6e590de0) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:02.387787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195a614-dc0c-4de1-b7ad-6de11976f571 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:02.389246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0195a614-dc0c-4de1-b7ad-6de11976f571 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:02.398623 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 260/541] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:02 2022] GET /identity/v3/role_assignments?user.id=1b2a9ebcd3f84b758322a80c6e590de0&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=4db40e559d064cf6b4f061e785b42dbb => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:02.675429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cccb09-259d-4933-8122-8e5be2557bf4 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:02.675768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cccb09-259d-4933-8122-8e5be2557bf4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:02.676537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cccb09-259d-4933-8122-8e5be2557bf4 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:02.678150 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 282/542] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:02.683519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd659ad3-6e35-4578-81af-35bacf43d166 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:02.691293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd659ad3-6e35-4578-81af-35bacf43d166 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:02.691293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd659ad3-6e35-4578-81af-35bacf43d166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:02.701546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-fd659ad3-6e35-4578-81af-35bacf43d166 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:02.702851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fd659ad3-6e35-4578-81af-35bacf43d166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:02.803505 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 261/543] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:42:02.836712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cccb09-259d-4933-8122-8e5be2557bf4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:02.841986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:02.999284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cccb09-259d-4933-8122-8e5be2557bf4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:03.078579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dda135-b78c-4fdb-b588-d397d494c686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:03.079732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dda135-b78c-4fdb-b588-d397d494c686 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:03.080210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dda135-b78c-4fdb-b588-d397d494c686 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:03.080679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dda135-b78c-4fdb-b588-d397d494c686 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:03.103170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dda135-b78c-4fdb-b588-d397d494c686 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:03.119540 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-e7dda135-b78c-4fdb-b588-d397d494c686 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:03.119540 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:03.119540 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:03.119540 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:03.119540 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:03.119540 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:03.119540 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:03.119540 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:03.119540 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:03.119540 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:03.119540 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:03.119540 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:03.119540 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:03.119540 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:03.119540 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:03.119540 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:03.120457 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:03.120457 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:03.120457 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:03.120457 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:03.120457 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:03.120457 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:03.120457 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:03.120457 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:03.120457 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:03.120457 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:03.120457 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:03.120457 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:03.120457 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:03.120457 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:03.120457 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:03.120457 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:03.121252 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:03.121252 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:03.121252 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:03.121252 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:03.121252 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:03.121252 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:03.121252 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:03.121252 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:03.121252 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:03.121252 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:03.121252 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:03.121252 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:03.126352 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 283/544] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:42:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:03.132375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd659ad3-6e35-4578-81af-35bacf43d166 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:03.280533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:03.389641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd659ad3-6e35-4578-81af-35bacf43d166 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:03.464041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b451fb-44e7-44dd-a81b-8429ad2c035d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:03.464710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b451fb-44e7-44dd-a81b-8429ad2c035d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:03.465031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b451fb-44e7-44dd-a81b-8429ad2c035d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:03.465308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b451fb-44e7-44dd-a81b-8429ad2c035d None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:03.466155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b451fb-44e7-44dd-a81b-8429ad2c035d None admin] RBAC: Adding query filter params (name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:03.466463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b451fb-44e7-44dd-a81b-8429ad2c035d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:03.467853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b451fb-44e7-44dd-a81b-8429ad2c035d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:03.485221 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 262/545] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:42:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:03.490589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dda135-b78c-4fdb-b588-d397d494c686 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:03.627455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:03.685152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dda135-b78c-4fdb-b588-d397d494c686 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:03.762720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e40923f-4085-413b-bdf2-f732e6d1d2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:03.763896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40923f-4085-413b-bdf2-f732e6d1d2b9 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:03.764368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40923f-4085-413b-bdf2-f732e6d1d2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:03.764829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e40923f-4085-413b-bdf2-f732e6d1d2b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:03.781011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40923f-4085-413b-bdf2-f732e6d1d2b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:03.782769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e40923f-4085-413b-bdf2-f732e6d1d2b9 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:03.807695 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-0e40923f-4085-413b-bdf2-f732e6d1d2b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:03.807695 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:03.807695 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:03.807695 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:03.807695 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:03.807695 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:03.807695 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:03.807695 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:03.807695 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:03.807695 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:03.807695 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:03.807695 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:03.807695 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:03.807695 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:03.807695 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:03.808626 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:03.808626 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:03.808626 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:03.808626 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:03.808626 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:03.808626 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:03.808626 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:03.808626 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:03.808626 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:03.808626 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:03.808626 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:03.808626 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:03.808626 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:03.808626 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:03.808626 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:03.808626 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:03.808626 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:03.809495 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:03.809495 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:03.809495 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:03.809495 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:03.811483 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 284/546] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:03.816867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b451fb-44e7-44dd-a81b-8429ad2c035d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:03.904198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eff66-3459-4321-ae95-ad50b4d5a399 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:03.904706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eff66-3459-4321-ae95-ad50b4d5a399 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:03.904706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032eff66-3459-4321-ae95-ad50b4d5a399 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:03.906325 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 285/547] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:03.919364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a1196-97a2-4769-b7ee-fcdf48ba2a83 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:03.919662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a1196-97a2-4769-b7ee-fcdf48ba2a83 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:03.919924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908a1196-97a2-4769-b7ee-fcdf48ba2a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:03.962725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-908a1196-97a2-4769-b7ee-fcdf48ba2a83 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:03.962725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-908a1196-97a2-4769-b7ee-fcdf48ba2a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:04.003279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b451fb-44e7-44dd-a81b-8429ad2c035d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:04.038254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:04.050525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410660e-be50-48f8-88c7-771ec5e2d678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:04.051177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410660e-be50-48f8-88c7-771ec5e2d678 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:04.051470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410660e-be50-48f8-88c7-771ec5e2d678 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:04.051756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f410660e-be50-48f8-88c7-771ec5e2d678 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:04.052497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410660e-be50-48f8-88c7-771ec5e2d678 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:04.052810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410660e-be50-48f8-88c7-771ec5e2d678 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:04.054244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f410660e-be50-48f8-88c7-771ec5e2d678 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:04.074279 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 263/548] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:04.081042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410660e-be50-48f8-88c7-771ec5e2d678 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:04.099160 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 286/549] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:42:04.134656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a1196-97a2-4769-b7ee-fcdf48ba2a83 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:04.210103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:04.222551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f410660e-be50-48f8-88c7-771ec5e2d678 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:04.269057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e2a26-78c0-48df-ba9b-98d1d4dcd768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:04.270674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e2a26-78c0-48df-ba9b-98d1d4dcd768 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:04.270961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e2a26-78c0-48df-ba9b-98d1d4dcd768 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:04.271237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e2a26-78c0-48df-ba9b-98d1d4dcd768 None admin] PATH_INFO: `/v3/users/swift` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:04.272247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908a1196-97a2-4769-b7ee-fcdf48ba2a83 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:04.280351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e2a26-78c0-48df-ba9b-98d1d4dcd768 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:04.281859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e2a26-78c0-48df-ba9b-98d1d4dcd768 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:04.303540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-471e2a26-78c0-48df-ba9b-98d1d4dcd768 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 20:42:04.303540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:04.303540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:04.303540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:04.303540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:04.303540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:04.303540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:04.303540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:04.303540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:04.303540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:04.303540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:04.303540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:04.303540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:04.303540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:04.303540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:04.303540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:04.304494 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:04.305417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:04.305417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:04.305417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:04.305417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:04.305417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:04.305417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:04.305417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:04.305417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:04.305417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 20:42:04.305417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:04.310133 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 264/550] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:42:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:04.311652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0a9a5-3ac2-4eb7-829d-64d14281a43a None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:04.311942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0a9a5-3ac2-4eb7-829d-64d14281a43a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:04.312222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb0a9a5-3ac2-4eb7-829d-64d14281a43a None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:04.313401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0540d-8db6-4179-90fb-d7ea6462a221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:04.313856 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 265/551] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:04.314470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0540d-8db6-4179-90fb-d7ea6462a221 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:04.314596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0a9a5-3ac2-4eb7-829d-64d14281a43a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:04.314923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0540d-8db6-4179-90fb-d7ea6462a221 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:04.315389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0540d-8db6-4179-90fb-d7ea6462a221 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:04.335386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0540d-8db6-4179-90fb-d7ea6462a221 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:04.345933 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-2af0540d-8db6-4179-90fb-d7ea6462a221 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:04.345933 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:04.345933 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:04.345933 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:04.345933 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:04.345933 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:04.345933 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:04.345933 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:04.345933 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:04.345933 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:04.345933 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:04.345933 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:04.345933 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:04.345933 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:04.345933 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:04.345933 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:04.347623 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:04.347623 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:04.347623 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:04.347623 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:04.347623 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:04.347623 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:04.347623 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:04.347623 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:04.347623 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:04.347623 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:04.347623 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:04.347623 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:04.347623 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:04.347623 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:04.347623 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:04.347623 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:04.349518 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:04.349518 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:04.349518 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:04.349518 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:04.349518 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:04.349518 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:04.349518 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:04.349518 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:04.349518 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:04.349518 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:04.349518 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:04.349518 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:04.357856 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 287/552] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:42:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:42:04.357856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89c9f3-dfc3-4a3e-867e-e7c35c711604 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:04.357856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89c9f3-dfc3-4a3e-867e-e7c35c711604 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:04.357856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b89c9f3-dfc3-4a3e-867e-e7c35c711604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:04.363289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:04.378050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8b89c9f3-dfc3-4a3e-867e-e7c35c711604 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:04.379732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8b89c9f3-dfc3-4a3e-867e-e7c35c711604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:04.452519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb0a9a5-3ac2-4eb7-829d-64d14281a43a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:04.459867 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 288/553] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:42:04.460873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89c9f3-dfc3-4a3e-867e-e7c35c711604 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:04.510782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0438b4-710a-4b6e-8fb9-f7927779edc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:04.511659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0438b4-710a-4b6e-8fb9-f7927779edc1 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:04.512097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0438b4-710a-4b6e-8fb9-f7927779edc1 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:04.512480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0438b4-710a-4b6e-8fb9-f7927779edc1 None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:04.513812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0438b4-710a-4b6e-8fb9-f7927779edc1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:04.514261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0438b4-710a-4b6e-8fb9-f7927779edc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:04.516235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0438b4-710a-4b6e-8fb9-f7927779edc1 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:04.525119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:04.537206 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 266/554] 10.222.0.33 () {56 vars in 1135 bytes} [Sat May 28 20:42:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:04.538529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0438b4-710a-4b6e-8fb9-f7927779edc1 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:04.616728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b89c9f3-dfc3-4a3e-867e-e7c35c711604 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:04.658574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:04.661904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e6a85-288f-45f7-b571-41a8ada9c95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:04.662710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e6a85-288f-45f7-b571-41a8ada9c95d None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:04.663096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e6a85-288f-45f7-b571-41a8ada9c95d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:04.663474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e6a85-288f-45f7-b571-41a8ada9c95d None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:04.664347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e6a85-288f-45f7-b571-41a8ada9c95d None admin] RBAC: Adding query filter params (name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:04.664758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e6a85-288f-45f7-b571-41a8ada9c95d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:04.666037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0438b4-710a-4b6e-8fb9-f7927779edc1 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:04.666513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036e6a85-288f-45f7-b571-41a8ada9c95d None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:04.675747 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 289/555] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:42:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:04.677243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e6a85-288f-45f7-b571-41a8ada9c95d None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:04.719405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9b2da-f10b-489d-b0fd-5220320c7243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:04.720135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9b2da-f10b-489d-b0fd-5220320c7243 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:04.720425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9b2da-f10b-489d-b0fd-5220320c7243 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:04.720703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9b2da-f10b-489d-b0fd-5220320c7243 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:04.735034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9b2da-f10b-489d-b0fd-5220320c7243 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:04.746479 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-06a9b2da-f10b-489d-b0fd-5220320c7243 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:04.746479 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:04.746479 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:04.746479 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:04.746479 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:04.746479 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:04.746479 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:04.746479 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:04.746479 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:04.746479 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:04.746479 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:04.746479 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:04.746479 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:04.746479 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:04.746479 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:04.746479 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:04.748209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:04.748209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:04.748209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:04.748209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:04.748209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:04.748209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:04.748209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:04.748209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:04.748209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:04.748209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:04.748209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:04.748209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:04.748209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:04.748209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:04.748209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:04.748209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:04.749722 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:04.749722 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:04.749722 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:04.749722 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:04.749722 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:04.749722 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:04.749722 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:04.749722 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:04.749722 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:04.749722 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:04.749722 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:04.749722 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:04.757878 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 267/556] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:42:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:04.757878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9b2da-f10b-489d-b0fd-5220320c7243 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:04.800183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:04.806290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e6a85-288f-45f7-b571-41a8ada9c95d None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:04.854598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0d10b-6167-47d9-bf4e-427149a086c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:04.855382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0d10b-6167-47d9-bf4e-427149a086c6 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:04.855674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0d10b-6167-47d9-bf4e-427149a086c6 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:04.855955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0d10b-6167-47d9-bf4e-427149a086c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:04.878405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9b2da-f10b-489d-b0fd-5220320c7243 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:04.919943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90d7c1-a3e3-484a-b9d1-16e279310e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:04.921032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90d7c1-a3e3-484a-b9d1-16e279310e95 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:04.921324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90d7c1-a3e3-484a-b9d1-16e279310e95 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:04.921604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90d7c1-a3e3-484a-b9d1-16e279310e95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:04.936157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90d7c1-a3e3-484a-b9d1-16e279310e95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:04.937657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90d7c1-a3e3-484a-b9d1-16e279310e95 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:04.941867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0d10b-6167-47d9-bf4e-427149a086c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:04.943602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0d10b-6167-47d9-bf4e-427149a086c6 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:04.959012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-bb90d7c1-a3e3-484a-b9d1-16e279310e95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:04.959012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:04.959012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:04.959012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:04.959012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:04.959012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:04.959012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:04.959012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:04.959012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:04.959012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:04.959012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:04.959012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:04.959012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:04.959012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:04.959012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:04.960587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:04.960587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:04.960587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:04.960587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:04.960587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:04.960587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:04.960587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:04.960587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:04.960587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:04.960587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:04.960587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:04.960587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:04.960587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:04.960587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:04.960587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:04.960587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:04.960587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:04.961931 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:04.961931 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:04.961931 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:04.961931 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:04.965367 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 268/557] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:04.966363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90d7c1-a3e3-484a-b9d1-16e279310e95 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:04.978372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:05.052483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:05.058067 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 290/558] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:42:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:42:05.059035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0d10b-6167-47d9-bf4e-427149a086c6 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:05.088357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90d7c1-a3e3-484a-b9d1-16e279310e95 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:05.135144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee47a6-e197-4115-98a9-8b94d7bb8703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:05.135807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee47a6-e197-4115-98a9-8b94d7bb8703 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:05.136088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee47a6-e197-4115-98a9-8b94d7bb8703 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:05.136359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee47a6-e197-4115-98a9-8b94d7bb8703 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:05.137111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee47a6-e197-4115-98a9-8b94d7bb8703 None admin] RBAC: Adding query filter params (name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:05.137416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee47a6-e197-4115-98a9-8b94d7bb8703 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:05.139153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee47a6-e197-4115-98a9-8b94d7bb8703 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:05.148610 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 269/559] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:42:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:05.149713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb900d-06eb-4d46-9651-e9506397eada None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:05.150034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb900d-06eb-4d46-9651-e9506397eada None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:05.150312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deb900d-06eb-4d46-9651-e9506397eada None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:05.151677 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 270/560] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:05.156132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb900d-06eb-4d46-9651-e9506397eada None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:05.173050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0d10b-6167-47d9-bf4e-427149a086c6 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:05.215749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36393c8-3a98-41de-a59d-676d4535d6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:05.217093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36393c8-3a98-41de-a59d-676d4535d6a6 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:05.217381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36393c8-3a98-41de-a59d-676d4535d6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:05.217656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36393c8-3a98-41de-a59d-676d4535d6a6 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:05.218674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36393c8-3a98-41de-a59d-676d4535d6a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:05.219379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36393c8-3a98-41de-a59d-676d4535d6a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:05.221263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36393c8-3a98-41de-a59d-676d4535d6a6 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:05.227505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:05.238734 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 291/561] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:05.238734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36393c8-3a98-41de-a59d-676d4535d6a6 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:05.266904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deb900d-06eb-4d46-9651-e9506397eada None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:05.296019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:05.300633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d643d8-088e-4818-8ef3-99aa099a8ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:05.301439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d643d8-088e-4818-8ef3-99aa099a8ed8 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:05.301724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d643d8-088e-4818-8ef3-99aa099a8ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:05.302034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d643d8-088e-4818-8ef3-99aa099a8ed8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:05.312339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d643d8-088e-4818-8ef3-99aa099a8ed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:05.313889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d643d8-088e-4818-8ef3-99aa099a8ed8 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:05.330943 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-31d643d8-088e-4818-8ef3-99aa099a8ed8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:05.330943 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:05.330943 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:05.330943 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:05.330943 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:05.330943 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:05.330943 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:05.330943 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:05.330943 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:05.330943 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:05.330943 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:05.330943 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:05.330943 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:05.330943 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:05.330943 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:05.331920 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:05.331920 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:05.331920 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:05.331920 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:05.331920 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:05.331920 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:05.331920 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:05.331920 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:05.331920 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:05.331920 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:05.331920 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:05.331920 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:05.331920 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:05.331920 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:05.331920 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:05.331920 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:05.331920 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:05.333373 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:05.333373 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:05.333373 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:05.333373 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:05.333373 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 271/562] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:05.333920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438e425-6067-4a62-9f59-46ec29f1ce5e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:05.334194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438e425-6067-4a62-9f59-46ec29f1ce5e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:05.334471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438e425-6067-4a62-9f59-46ec29f1ce5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:05.340635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36393c8-3a98-41de-a59d-676d4535d6a6 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:05.350882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a438e425-6067-4a62-9f59-46ec29f1ce5e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:05.352117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a438e425-6067-4a62-9f59-46ec29f1ce5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:05.374299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ceec10-8fa6-4372-ac93-68daa85885d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:05.375336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ceec10-8fa6-4372-ac93-68daa85885d6 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:05.375336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ceec10-8fa6-4372-ac93-68daa85885d6 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:05.375613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ceec10-8fa6-4372-ac93-68daa85885d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:05.386043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ceec10-8fa6-4372-ac93-68daa85885d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:05.387498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ceec10-8fa6-4372-ac93-68daa85885d6 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:05.404777 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-b0ceec10-8fa6-4372-ac93-68daa85885d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:05.404777 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:05.404777 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:05.404777 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:05.404777 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:05.404777 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:05.404777 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:05.404777 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:05.404777 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:05.404777 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:05.404777 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:05.404777 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:05.404777 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:05.404777 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:05.404777 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:05.405724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:05.405724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:05.405724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:05.405724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:05.405724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:05.405724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:05.405724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:05.405724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:05.405724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:05.405724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:05.405724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:05.405724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:05.405724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:05.405724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:05.405724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:05.405724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:05.405724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:05.406676 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:05.406676 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:05.406676 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:05.406676 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:05.406676 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 292/563] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:05.406986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ceec10-8fa6-4372-ac93-68daa85885d6 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:05.419026 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 272/564] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:42:05.420320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5348b-7df1-403c-a19d-a764a2ee5c3d None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:05.420608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5348b-7df1-403c-a19d-a764a2ee5c3d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:05.420884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5348b-7df1-403c-a19d-a764a2ee5c3d None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:05.422152 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 273/565] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:05.422828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5348b-7df1-403c-a19d-a764a2ee5c3d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:05.483534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:05.514545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ceec10-8fa6-4372-ac93-68daa85885d6 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:05.525327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:05.534344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c5348b-7df1-403c-a19d-a764a2ee5c3d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:05.548250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfd547-a05d-44ff-974f-57bc501a71f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:05.549074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfd547-a05d-44ff-974f-57bc501a71f5 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:05.549393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfd547-a05d-44ff-974f-57bc501a71f5 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:05.549675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfd547-a05d-44ff-974f-57bc501a71f5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:05.558599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfd547-a05d-44ff-974f-57bc501a71f5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:05.561598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbfd547-a05d-44ff-974f-57bc501a71f5 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:05.576207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-9cbfd547-a05d-44ff-974f-57bc501a71f5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 20:42:05.576207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:05.576207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:05.576207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:05.576207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:05.576207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:05.576207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:05.576207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:05.576207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:05.576207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:05.576207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:05.576207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:05.576207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:05.576207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:05.576207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:05.576207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:05.577357 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:05.578348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:05.578348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:05.578348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:05.578348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:05.578348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:05.578348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:05.578348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:05.578348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:05.578348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 20:42:05.578348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:05.578348 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 293/566] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 20:42:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:05.578955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfd547-a05d-44ff-974f-57bc501a71f5 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:05.686780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eff29a-34bd-4eb5-ae2f-1e12e38f851f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:05.687429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eff29a-34bd-4eb5-ae2f-1e12e38f851f None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:05.687712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eff29a-34bd-4eb5-ae2f-1e12e38f851f None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:05.687988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eff29a-34bd-4eb5-ae2f-1e12e38f851f None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:05.688743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eff29a-34bd-4eb5-ae2f-1e12e38f851f None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:05.689048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eff29a-34bd-4eb5-ae2f-1e12e38f851f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:05.689402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfd547-a05d-44ff-974f-57bc501a71f5 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:05.690458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eff29a-34bd-4eb5-ae2f-1e12e38f851f None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:05.702278 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 274/567] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:05.705509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4225f8d-8af7-4825-9536-5cbe37cd9b4f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:05.705509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4225f8d-8af7-4825-9536-5cbe37cd9b4f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:05.705509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4225f8d-8af7-4825-9536-5cbe37cd9b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:05.719969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f4225f8d-8af7-4825-9536-5cbe37cd9b4f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:05.721167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f4225f8d-8af7-4825-9536-5cbe37cd9b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:05.728477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce437084-15f8-4444-b34a-41f94405a75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:05.729141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce437084-15f8-4444-b34a-41f94405a75d None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:05.729420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce437084-15f8-4444-b34a-41f94405a75d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:05.729693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce437084-15f8-4444-b34a-41f94405a75d None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:05.730455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce437084-15f8-4444-b34a-41f94405a75d None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:05.730762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce437084-15f8-4444-b34a-41f94405a75d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:05.732237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce437084-15f8-4444-b34a-41f94405a75d None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:05.743661 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 294/568] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:05.744484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce437084-15f8-4444-b34a-41f94405a75d None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:05.778072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:05.787241 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 275/569] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:42:05.788075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4225f8d-8af7-4825-9536-5cbe37cd9b4f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:05.828618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:05.849232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce437084-15f8-4444-b34a-41f94405a75d None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:05.882556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bb3d9-b2c8-4047-9117-1fe1d61bd408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:05.883343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bb3d9-b2c8-4047-9117-1fe1d61bd408 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:05.883622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bb3d9-b2c8-4047-9117-1fe1d61bd408 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:05.883894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bb3d9-b2c8-4047-9117-1fe1d61bd408 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:05.885115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4225f8d-8af7-4825-9536-5cbe37cd9b4f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:05.894133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93bb3d9-b2c8-4047-9117-1fe1d61bd408 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:05.902395 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-a93bb3d9-b2c8-4047-9117-1fe1d61bd408 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:05.902395 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:05.902395 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:05.902395 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:05.902395 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:05.902395 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:05.902395 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:05.902395 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:05.902395 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:05.902395 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:05.902395 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:05.902395 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:05.902395 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:05.902395 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:05.902395 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:05.902395 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:05.903345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:05.903345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:05.903345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:05.903345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:05.903345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:05.903345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:05.903345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:05.903345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:05.903345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:05.903345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:05.903345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:05.903345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:05.903345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:05.903345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:05.903345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:05.903345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:05.904182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:05.904182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:05.904182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:05.904182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:05.904182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:05.904182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:05.904182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:05.904182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:05.904182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:05.904182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:05.904182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:05.904182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:05.904182 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 295/570] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:42:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:05.904863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bb3d9-b2c8-4047-9117-1fe1d61bd408 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:05.923721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43984a5-becd-4bba-a0b1-67466e2791cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:05.924395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43984a5-becd-4bba-a0b1-67466e2791cc None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:05.924684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43984a5-becd-4bba-a0b1-67466e2791cc None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:05.924962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43984a5-becd-4bba-a0b1-67466e2791cc None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:05.925803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43984a5-becd-4bba-a0b1-67466e2791cc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:05.926184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43984a5-becd-4bba-a0b1-67466e2791cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:05.927641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43984a5-becd-4bba-a0b1-67466e2791cc None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:05.948700 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 276/571] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 20:42:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:05.948700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43984a5-becd-4bba-a0b1-67466e2791cc None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:05.993309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:06.002957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93bb3d9-b2c8-4047-9117-1fe1d61bd408 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:06.036760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c502a-1e66-4caa-a50e-cacd2d3d0e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:06.037573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c502a-1e66-4caa-a50e-cacd2d3d0e2c None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:06.037883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c502a-1e66-4caa-a50e-cacd2d3d0e2c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:06.038170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c502a-1e66-4caa-a50e-cacd2d3d0e2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:06.038369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:06.041876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43984a5-becd-4bba-a0b1-67466e2791cc None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:06.044804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c502a-1e66-4caa-a50e-cacd2d3d0e2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:06.046312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c502a-1e66-4caa-a50e-cacd2d3d0e2c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:06.060378 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-708c502a-1e66-4caa-a50e-cacd2d3d0e2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:06.060378 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:06.060378 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:06.060378 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:06.060378 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:06.060378 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:06.060378 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:06.060378 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:06.060378 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:06.060378 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:06.060378 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:06.060378 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:06.060378 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:06.060378 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:06.060378 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:06.061480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:06.061480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:06.061480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:06.061480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:06.061480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:06.061480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:06.061480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:06.061480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:06.061480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:06.061480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:06.061480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:06.061480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:06.061480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:06.061480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:06.061480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:06.061480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:06.061480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:06.062657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:06.062657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:06.062657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:06.062657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:06.062657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:06.062657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:06.062657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:06.062657 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:06.066363 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 296/572] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:06.067509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c502a-1e66-4caa-a50e-cacd2d3d0e2c None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:06.078512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4ee2e-74e6-4a42-89f6-e7dd7a936e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:06.079303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4ee2e-74e6-4a42-89f6-e7dd7a936e08 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:06.079585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4ee2e-74e6-4a42-89f6-e7dd7a936e08 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:06.079854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4ee2e-74e6-4a42-89f6-e7dd7a936e08 None admin] PATH_INFO: `/v3/users/glance` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:06.087433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4ee2e-74e6-4a42-89f6-e7dd7a936e08 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:06.088878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4ee2e-74e6-4a42-89f6-e7dd7a936e08 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:06.103142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-0ec4ee2e-74e6-4a42-89f6-e7dd7a936e08 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 28 20:42:06.103142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:06.103142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:06.103142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:06.103142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:06.103142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:06.103142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:06.103142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:06.103142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:06.103142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:06.103142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:06.103142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:06.103142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:06.103142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:06.103142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:06.103142 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:06.104220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:06.109853 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:06.109853 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:06.109853 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:06.109853 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:06.109853 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:06.109853 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:06.109853 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:06.109853 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:06.109853 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 28 20:42:06.109853 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:06.109853 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 277/573] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:42:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:06.109853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4ee2e-74e6-4a42-89f6-e7dd7a936e08 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:06.168576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c502a-1e66-4caa-a50e-cacd2d3d0e2c None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:06.200960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7959f0-9d5a-4256-ae5a-5a4147c39a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:06.201751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7959f0-9d5a-4256-ae5a-5a4147c39a34 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:06.201897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4ee2e-74e6-4a42-89f6-e7dd7a936e08 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:06.202069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7959f0-9d5a-4256-ae5a-5a4147c39a34 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:06.202349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7959f0-9d5a-4256-ae5a-5a4147c39a34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:06.212902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7959f0-9d5a-4256-ae5a-5a4147c39a34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:06.220672 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-0e7959f0-9d5a-4256-ae5a-5a4147c39a34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:06.220672 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:06.220672 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:06.220672 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:06.220672 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:06.220672 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:06.220672 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:06.220672 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:06.220672 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:06.220672 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:06.220672 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:06.220672 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:06.220672 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:06.220672 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:06.220672 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:06.220672 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:06.221735 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:06.221735 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:06.221735 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:06.221735 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:06.221735 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:06.221735 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:06.221735 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:06.221735 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:06.221735 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:06.221735 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:06.221735 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:06.221735 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:06.221735 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:06.221735 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:06.221735 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:06.221735 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:06.222752 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:06.222752 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:06.222752 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:06.222752 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:06.222752 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:06.222752 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:06.222752 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:06.222752 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:06.222752 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:06.222752 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:06.222752 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:06.222752 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:06.222752 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 297/574] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:42:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:06.226651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7959f0-9d5a-4256-ae5a-5a4147c39a34 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:06.235731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d684c-3821-44a8-b286-72d87ab6ad64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:06.236386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d684c-3821-44a8-b286-72d87ab6ad64 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:06.236666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d684c-3821-44a8-b286-72d87ab6ad64 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:06.236937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d684c-3821-44a8-b286-72d87ab6ad64 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:06.237683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d684c-3821-44a8-b286-72d87ab6ad64 None admin] RBAC: Adding query filter params (name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:06.238022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d684c-3821-44a8-b286-72d87ab6ad64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:06.238690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:06.239431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d684c-3821-44a8-b286-72d87ab6ad64 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:06.251598 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 278/575] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:42:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:06.252407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d684c-3821-44a8-b286-72d87ab6ad64 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:06.276332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:06.326904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7959f0-9d5a-4256-ae5a-5a4147c39a34 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:06.351671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d684c-3821-44a8-b286-72d87ab6ad64 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:06.361248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5745be-c5e2-4b66-899a-d5d468b35815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:06.362067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5745be-c5e2-4b66-899a-d5d468b35815 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:06.362349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5745be-c5e2-4b66-899a-d5d468b35815 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:06.362626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5745be-c5e2-4b66-899a-d5d468b35815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:06.373018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5745be-c5e2-4b66-899a-d5d468b35815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:06.374494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5745be-c5e2-4b66-899a-d5d468b35815 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:06.386083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40905848-ce85-4169-b167-2a804aaf18a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:06.386737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40905848-ce85-4169-b167-2a804aaf18a8 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:06.387021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40905848-ce85-4169-b167-2a804aaf18a8 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:06.387296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40905848-ce85-4169-b167-2a804aaf18a8 None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:06.388083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40905848-ce85-4169-b167-2a804aaf18a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:06.388392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40905848-ce85-4169-b167-2a804aaf18a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:06.389806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40905848-ce85-4169-b167-2a804aaf18a8 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:06.391807 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-7d5745be-c5e2-4b66-899a-d5d468b35815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:06.391807 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:06.391807 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:06.391807 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:06.391807 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:06.391807 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:06.391807 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:06.391807 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:06.391807 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:06.391807 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:06.391807 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:06.391807 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:06.391807 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:06.391807 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:06.391807 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:06.392802 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:06.392802 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:06.392802 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:06.392802 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:06.392802 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:06.392802 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:06.392802 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:06.392802 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:06.392802 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:06.392802 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:06.392802 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:06.392802 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:06.392802 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:06.392802 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:06.392802 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:06.392802 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:06.392802 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:06.393748 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:06.393748 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:06.393748 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:06.393748 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:06.393748 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 298/576] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:06.393748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5745be-c5e2-4b66-899a-d5d468b35815 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:06.405326 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 279/577] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:06.405326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40905848-ce85-4169-b167-2a804aaf18a8 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:06.458530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:06.487417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:06.497414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5745be-c5e2-4b66-899a-d5d468b35815 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:06.506917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40905848-ce85-4169-b167-2a804aaf18a8 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:06.531382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c525b65-30c4-4474-b429-0c79b091b51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:06.532055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525b65-30c4-4474-b429-0c79b091b51e None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:06.532345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525b65-30c4-4474-b429-0c79b091b51e None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:06.532629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c525b65-30c4-4474-b429-0c79b091b51e None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:06.533473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c525b65-30c4-4474-b429-0c79b091b51e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:06.533787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c525b65-30c4-4474-b429-0c79b091b51e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:06.535407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c525b65-30c4-4474-b429-0c79b091b51e None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:06.539321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cfc56-3d35-4ed4-bf2f-c3887914bef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:06.539979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cfc56-3d35-4ed4-bf2f-c3887914bef1 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:06.540264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cfc56-3d35-4ed4-bf2f-c3887914bef1 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:06.540539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cfc56-3d35-4ed4-bf2f-c3887914bef1 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:06.541298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cfc56-3d35-4ed4-bf2f-c3887914bef1 None admin] RBAC: Adding query filter params (name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:06.541881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cfc56-3d35-4ed4-bf2f-c3887914bef1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:06.543884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cfc56-3d35-4ed4-bf2f-c3887914bef1 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:06.551724 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 280/578] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:42:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:06.552516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cfc56-3d35-4ed4-bf2f-c3887914bef1 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:06.557225 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 299/579] 10.222.0.33 () {56 vars in 1137 bytes} [Sat May 28 20:42:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:06.557225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c525b65-30c4-4474-b429-0c79b091b51e None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:06.659004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cfc56-3d35-4ed4-bf2f-c3887914bef1 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:06.694785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e4598-b7ac-4fbb-9bca-40dca43cf513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:06.695586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e4598-b7ac-4fbb-9bca-40dca43cf513 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:06.695867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e4598-b7ac-4fbb-9bca-40dca43cf513 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:06.696140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e4598-b7ac-4fbb-9bca-40dca43cf513 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:06.707320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e4598-b7ac-4fbb-9bca-40dca43cf513 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:06.708782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e4598-b7ac-4fbb-9bca-40dca43cf513 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:06.710711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:06.726800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-237e4598-b7ac-4fbb-9bca-40dca43cf513 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:06.726800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:06.726800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:06.726800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:06.726800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:06.726800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:06.726800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:06.726800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:06.726800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:06.726800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:06.726800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:06.726800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:06.726800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:06.726800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:06.726800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:06.727725 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:06.727725 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:06.727725 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:06.727725 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:06.727725 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:06.727725 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:06.727725 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:06.727725 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:06.727725 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:06.727725 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:06.727725 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:06.727725 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:06.727725 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:06.727725 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:06.727725 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:06.727725 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:06.727725 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:06.728613 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:06.728613 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:06.728613 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:06.728613 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:06.728613 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 281/580] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:06.728967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e4598-b7ac-4fbb-9bca-40dca43cf513 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:06.777909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c525b65-30c4-4474-b429-0c79b091b51e None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:06.797165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:06.812083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3a96e-a1e9-49eb-8ea3-f4843f34af52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:06.814561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3a96e-a1e9-49eb-8ea3-f4843f34af52 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:06.814561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3a96e-a1e9-49eb-8ea3-f4843f34af52 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:06.814561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed3a96e-a1e9-49eb-8ea3-f4843f34af52 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:06.814561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3a96e-a1e9-49eb-8ea3-f4843f34af52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:06.814561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3a96e-a1e9-49eb-8ea3-f4843f34af52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:06.815671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed3a96e-a1e9-49eb-8ea3-f4843f34af52 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:06.827821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e4598-b7ac-4fbb-9bca-40dca43cf513 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:06.828315 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 300/581] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:06.829209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3a96e-a1e9-49eb-8ea3-f4843f34af52 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:06.861611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90a00a-58cc-40de-856d-bf4cadfc7613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:06.862287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90a00a-58cc-40de-856d-bf4cadfc7613 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:06.862573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90a00a-58cc-40de-856d-bf4cadfc7613 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:06.862849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f90a00a-58cc-40de-856d-bf4cadfc7613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:06.864194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90a00a-58cc-40de-856d-bf4cadfc7613 None admin] RBAC: Adding query filter params (role.id=4db40e559d064cf6b4f061e785b42dbb, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=a01bba888a644eb18fc457aff578aa4b) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:06.864545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90a00a-58cc-40de-856d-bf4cadfc7613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:06.866021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f90a00a-58cc-40de-856d-bf4cadfc7613 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:06.871490 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 282/582] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:06 2022] GET /identity/v3/role_assignments?user.id=a01bba888a644eb18fc457aff578aa4b&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=4db40e559d064cf6b4f061e785b42dbb => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:06.872264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90a00a-58cc-40de-856d-bf4cadfc7613 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:06.934955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed3a96e-a1e9-49eb-8ea3-f4843f34af52 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:06.940302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:06.975395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aef126-76e3-4d5b-acd7-a9ccd18d44ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:06.976370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aef126-76e3-4d5b-acd7-a9ccd18d44ba None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:06.976749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aef126-76e3-4d5b-acd7-a9ccd18d44ba None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:06.977134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aef126-76e3-4d5b-acd7-a9ccd18d44ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:06.986711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f90a00a-58cc-40de-856d-bf4cadfc7613 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:06.989381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aef126-76e3-4d5b-acd7-a9ccd18d44ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:06.991055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aef126-76e3-4d5b-acd7-a9ccd18d44ba None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:07.010882 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-f9aef126-76e3-4d5b-acd7-a9ccd18d44ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:07.010882 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:07.010882 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:07.010882 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:07.010882 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:07.010882 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:07.010882 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:07.010882 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:07.010882 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:07.010882 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:07.010882 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:07.010882 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:07.010882 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:07.010882 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:07.010882 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:07.014757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:07.014757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:07.014757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:07.014757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:07.014757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:07.014757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:07.014757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:07.014757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:07.014757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:07.014757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:07.014757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:07.014757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:07.014757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:07.014757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:07.014757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:07.014757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:07.014757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:07.016184 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:07.016184 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:07.016184 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:07.016184 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:07.016184 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 301/583] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:07.016184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aef126-76e3-4d5b-acd7-a9ccd18d44ba None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:07.029298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397292ea-df22-4029-acd0-22b39b5389ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:07.030150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397292ea-df22-4029-acd0-22b39b5389ea None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:07.030480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397292ea-df22-4029-acd0-22b39b5389ea None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:07.030767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397292ea-df22-4029-acd0-22b39b5389ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:07.047320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397292ea-df22-4029-acd0-22b39b5389ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:07.048864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397292ea-df22-4029-acd0-22b39b5389ea None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:07.053243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:07.072222 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-397292ea-df22-4029-acd0-22b39b5389ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:07.072222 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:07.072222 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:07.072222 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:07.072222 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:07.072222 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:07.072222 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:07.072222 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:07.072222 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:07.072222 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:07.072222 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:07.072222 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:07.072222 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:07.072222 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:07.072222 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:07.073347 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:07.073347 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:07.073347 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:07.073347 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:07.073347 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:07.073347 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:07.073347 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:07.073347 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:07.073347 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:07.073347 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:07.073347 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:07.073347 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:07.073347 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:07.073347 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:07.073347 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:07.073347 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:07.073347 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:07.074458 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:07.074458 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:07.074458 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:07.074458 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:07.074458 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 283/584] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:07.074759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397292ea-df22-4029-acd0-22b39b5389ea None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:07.138468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aef126-76e3-4d5b-acd7-a9ccd18d44ba None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:07.180950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271906c6-3248-40d2-98ce-53169758edb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:07.181620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271906c6-3248-40d2-98ce-53169758edb3 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:07.181944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271906c6-3248-40d2-98ce-53169758edb3 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:07.182227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271906c6-3248-40d2-98ce-53169758edb3 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:07.182958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271906c6-3248-40d2-98ce-53169758edb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:07.183267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271906c6-3248-40d2-98ce-53169758edb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:07.183624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397292ea-df22-4029-acd0-22b39b5389ea None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:07.184649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271906c6-3248-40d2-98ce-53169758edb3 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:07.198299 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 302/585] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:07.198468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:07.198562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271906c6-3248-40d2-98ce-53169758edb3 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:07.225237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb787aca-bd6e-48c6-87d5-c08becd73524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:07.226063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb787aca-bd6e-48c6-87d5-c08becd73524 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:07.226351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb787aca-bd6e-48c6-87d5-c08becd73524 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:07.226890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb787aca-bd6e-48c6-87d5-c08becd73524 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:07.234417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb787aca-bd6e-48c6-87d5-c08becd73524 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:07.235956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb787aca-bd6e-48c6-87d5-c08becd73524 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:07.250786 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-cb787aca-bd6e-48c6-87d5-c08becd73524 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:07.250786 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:07.250786 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:07.250786 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:07.250786 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:07.250786 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:07.250786 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:07.250786 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:07.250786 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:07.250786 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:07.250786 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:07.250786 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:07.250786 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:07.250786 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:07.250786 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:07.251851 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:07.251851 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:07.251851 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:07.251851 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:07.251851 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:07.251851 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:07.251851 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:07.251851 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:07.251851 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:07.251851 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:07.251851 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:07.251851 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:07.251851 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:07.251851 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:07.251851 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:07.251851 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:07.251851 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:07.252800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:07.252800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:07.252800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:07.252800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:07.252800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:07.252800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:07.252800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:07.252800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:07.252800 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 284/586] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:07.253287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb787aca-bd6e-48c6-87d5-c08becd73524 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:07.298289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:07.309454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271906c6-3248-40d2-98ce-53169758edb3 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:07.345012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4726518-dc0b-49d4-be07-832ea1d0859f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:07.345732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4726518-dc0b-49d4-be07-832ea1d0859f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:07.346046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4726518-dc0b-49d4-be07-832ea1d0859f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:07.346324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4726518-dc0b-49d4-be07-832ea1d0859f None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:07.347048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4726518-dc0b-49d4-be07-832ea1d0859f None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:07.347469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4726518-dc0b-49d4-be07-832ea1d0859f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:07.349590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4726518-dc0b-49d4-be07-832ea1d0859f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:07.358250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb787aca-bd6e-48c6-87d5-c08becd73524 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:07.366947 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 303/587] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:07.366947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4726518-dc0b-49d4-be07-832ea1d0859f None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:07.400261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebc617-24a2-47e2-bb0e-fbce217000e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:07.400925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebc617-24a2-47e2-bb0e-fbce217000e2 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:07.401208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebc617-24a2-47e2-bb0e-fbce217000e2 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:07.401488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebc617-24a2-47e2-bb0e-fbce217000e2 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:07.402265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebc617-24a2-47e2-bb0e-fbce217000e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:07.402578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebc617-24a2-47e2-bb0e-fbce217000e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:07.403961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebc617-24a2-47e2-bb0e-fbce217000e2 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:07.420036 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 285/588] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:07.420036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebc617-24a2-47e2-bb0e-fbce217000e2 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:07.448780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:07.475534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4726518-dc0b-49d4-be07-832ea1d0859f None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:07.512002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada57b7-bb3c-41df-905b-645e7355b860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:07.512907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada57b7-bb3c-41df-905b-645e7355b860 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:07.513185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada57b7-bb3c-41df-905b-645e7355b860 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:07.513465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada57b7-bb3c-41df-905b-645e7355b860 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:07.521931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebc617-24a2-47e2-bb0e-fbce217000e2 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:07.522330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada57b7-bb3c-41df-905b-645e7355b860 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:07.523905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ada57b7-bb3c-41df-905b-645e7355b860 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:07.539323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-9ada57b7-bb3c-41df-905b-645e7355b860 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 28 20:42:07.539323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:07.539323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:07.539323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:07.539323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:07.539323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:07.539323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:07.539323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:07.539323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:07.539323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:07.539323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:07.539323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:07.539323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:07.539323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:07.539323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:07.539323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:07.540286 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:07.541292 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:07.541292 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:07.541292 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:07.541292 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:07.541292 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:07.541292 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:07.541292 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:07.541292 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:07.541292 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 28 20:42:07.541292 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:07.545062 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 304/589] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:42:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:07.545928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada57b7-bb3c-41df-905b-645e7355b860 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:07.556652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c2f29-7110-49b6-93d6-57acfb6a749c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:07.557316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c2f29-7110-49b6-93d6-57acfb6a749c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:07.557600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c2f29-7110-49b6-93d6-57acfb6a749c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:07.557912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c2f29-7110-49b6-93d6-57acfb6a749c None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:07.558714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c2f29-7110-49b6-93d6-57acfb6a749c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:07.559022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c2f29-7110-49b6-93d6-57acfb6a749c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:07.560447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47c2f29-7110-49b6-93d6-57acfb6a749c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:07.562506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:07.578503 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 286/590] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:07.578503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c2f29-7110-49b6-93d6-57acfb6a749c None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:07.667928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:07.668456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ada57b7-bb3c-41df-905b-645e7355b860 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:07.679048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c2f29-7110-49b6-93d6-57acfb6a749c None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:07.704328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d4a98-3392-4b4e-b419-554cb45e49d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:07.705050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d4a98-3392-4b4e-b419-554cb45e49d7 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:07.705333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d4a98-3392-4b4e-b419-554cb45e49d7 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:07.705610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d4a98-3392-4b4e-b419-554cb45e49d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:07.713039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d4a98-3392-4b4e-b419-554cb45e49d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:07.714522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8d4a98-3392-4b4e-b419-554cb45e49d7 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:07.717746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e156acf-e525-48c4-a2af-5d273fad14f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:07.718741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e156acf-e525-48c4-a2af-5d273fad14f6 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:07.719176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e156acf-e525-48c4-a2af-5d273fad14f6 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:07.719600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e156acf-e525-48c4-a2af-5d273fad14f6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:07.727549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e156acf-e525-48c4-a2af-5d273fad14f6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:07.729027 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-6a8d4a98-3392-4b4e-b419-554cb45e49d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:07.729027 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:07.729027 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:07.729027 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:07.729027 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:07.729027 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:07.729027 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:07.729027 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:07.729027 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:07.729027 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:07.729027 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:07.729027 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:07.729027 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:07.729027 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:07.729027 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:07.730095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:07.730095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:07.730095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:07.730095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:07.730095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:07.730095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:07.730095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:07.730095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:07.730095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:07.730095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:07.730095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:07.730095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:07.730095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:07.730095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:07.730095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:07.730095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:07.730095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:07.731030 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:07.731030 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:07.731030 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:07.731030 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:07.731030 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:07.731030 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:07.731030 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:07.731030 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:07.731030 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 305/591] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:07.731516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d4a98-3392-4b4e-b419-554cb45e49d7 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:07.731837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e156acf-e525-48c4-a2af-5d273fad14f6 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:07.750869 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-8e156acf-e525-48c4-a2af-5d273fad14f6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 28 20:42:07.750869 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:07.750869 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:07.750869 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:07.750869 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:07.750869 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:07.750869 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:07.750869 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:07.750869 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:07.750869 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:07.750869 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:07.750869 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:07.750869 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:07.750869 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:07.750869 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:07.750869 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:07.751862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:07.752805 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:07.752805 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:07.752805 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:07.752805 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:07.752805 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:07.752805 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:07.752805 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:07.752805 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:07.752805 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 28 20:42:07.752805 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:07.752805 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 287/592] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:42:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:07.753405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e156acf-e525-48c4-a2af-5d273fad14f6 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:07.807130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:07.837181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d4a98-3392-4b4e-b419-554cb45e49d7 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:07.862859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e156acf-e525-48c4-a2af-5d273fad14f6 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:07.883805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b55a0-7698-43b8-9db4-59a5134100b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:07.885355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b55a0-7698-43b8-9db4-59a5134100b7 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:07.886135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b55a0-7698-43b8-9db4-59a5134100b7 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:07.886639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b55a0-7698-43b8-9db4-59a5134100b7 None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:07.888324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b55a0-7698-43b8-9db4-59a5134100b7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:07.889130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b55a0-7698-43b8-9db4-59a5134100b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:07.891216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b55a0-7698-43b8-9db4-59a5134100b7 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:07.899418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263c286-d886-4daa-b4ec-a2381a99a5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:07.900309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263c286-d886-4daa-b4ec-a2381a99a5cd None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:07.900592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263c286-d886-4daa-b4ec-a2381a99a5cd None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:07.901604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263c286-d886-4daa-b4ec-a2381a99a5cd None admin] PATH_INFO: `/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/cfdebb5e6b8d4e2d86f0a03a30c21671/roles/4db40e559d064cf6b4f061e785b42dbb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:07.909619 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 306/593] 10.222.0.33 () {56 vars in 1137 bytes} [Sat May 28 20:42:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:07.914580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b55a0-7698-43b8-9db4-59a5134100b7 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:07.916999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263c286-d886-4daa-b4ec-a2381a99a5cd None admin] RBAC: Authorizing `identity:create_grant(project_id=3186f9d3c2b341afa8b098ebcaaf974b, user_id=cfdebb5e6b8d4e2d86f0a03a30c21671, role_id=4db40e559d064cf6b4f061e785b42dbb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:07.918885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263c286-d886-4daa-b4ec-a2381a99a5cd None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:07.995609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:08.009868 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 288/594] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:42:07 2022] PUT /identity/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/cfdebb5e6b8d4e2d86f0a03a30c21671/roles/4db40e559d064cf6b4f061e785b42dbb => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:42:08.009868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263c286-d886-4daa-b4ec-a2381a99a5cd None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:08.017427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b55a0-7698-43b8-9db4-59a5134100b7 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:08.063657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:08.076957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f841083-f15d-405c-8223-a13a6b605c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:08.078807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f841083-f15d-405c-8223-a13a6b605c9d None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:08.079321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f841083-f15d-405c-8223-a13a6b605c9d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:08.079783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f841083-f15d-405c-8223-a13a6b605c9d None admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:08.081341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f841083-f15d-405c-8223-a13a6b605c9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:08.082762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f841083-f15d-405c-8223-a13a6b605c9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:08.084729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f841083-f15d-405c-8223-a13a6b605c9d None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:08.101785 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 307/595] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:08.102991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f841083-f15d-405c-8223-a13a6b605c9d None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:08.158828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263c286-d886-4daa-b4ec-a2381a99a5cd None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:08.200641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467b932-931d-4a22-b304-cea6e1629213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:08.201332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467b932-931d-4a22-b304-cea6e1629213 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:08.201673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467b932-931d-4a22-b304-cea6e1629213 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:08.201920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8467b932-931d-4a22-b304-cea6e1629213 None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:08.202838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8467b932-931d-4a22-b304-cea6e1629213 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:08.203158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8467b932-931d-4a22-b304-cea6e1629213 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:08.204610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8467b932-931d-4a22-b304-cea6e1629213 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:08.225529 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 289/596] 10.222.0.33 () {56 vars in 1139 bytes} [Sat May 28 20:42:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:08.226620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467b932-931d-4a22-b304-cea6e1629213 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:08.240852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f841083-f15d-405c-8223-a13a6b605c9d None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:08.281608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb8db8-48c0-44b4-8711-ce14e538fdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:08.282601 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb8db8-48c0-44b4-8711-ce14e538fdac None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:08.283264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb8db8-48c0-44b4-8711-ce14e538fdac None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:08.283550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bb8db8-48c0-44b4-8711-ce14e538fdac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:08.294668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:08.297409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb8db8-48c0-44b4-8711-ce14e538fdac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:08.299379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bb8db8-48c0-44b4-8711-ce14e538fdac None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:08.325798 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-73bb8db8-48c0-44b4-8711-ce14e538fdac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:08.325798 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:08.325798 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:08.325798 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:08.325798 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:08.325798 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:08.325798 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:08.325798 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:08.325798 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:08.325798 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:08.325798 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:08.325798 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:08.325798 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:08.325798 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:08.325798 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:08.326935 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:08.326935 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:08.326935 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:08.326935 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:08.326935 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:08.326935 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:08.326935 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:08.326935 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:08.326935 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:08.326935 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:08.326935 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:08.326935 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:08.326935 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:08.326935 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:08.326935 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:08.326935 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:08.326935 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:08.328055 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:08.328055 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:08.328055 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:08.328055 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:08.328055 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 308/597] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:08.328368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb8db8-48c0-44b4-8711-ce14e538fdac None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:08.349598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8467b932-931d-4a22-b304-cea6e1629213 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:08.385827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:08.388295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581163d-e8dc-4393-88a9-6fb3f19334a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:08.389184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581163d-e8dc-4393-88a9-6fb3f19334a4 None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:08.389465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581163d-e8dc-4393-88a9-6fb3f19334a4 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:08.389823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581163d-e8dc-4393-88a9-6fb3f19334a4 None admin] PATH_INFO: `/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/224c1a0427c942e5bfef2bac75648d80/roles/4db40e559d064cf6b4f061e785b42dbb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:08.407091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581163d-e8dc-4393-88a9-6fb3f19334a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=3186f9d3c2b341afa8b098ebcaaf974b, user_id=224c1a0427c942e5bfef2bac75648d80, role_id=4db40e559d064cf6b4f061e785b42dbb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:08.408530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581163d-e8dc-4393-88a9-6fb3f19334a4 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:08.448667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bb8db8-48c0-44b4-8711-ce14e538fdac None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:08.484283 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 290/598] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:42:08 2022] PUT /identity/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/224c1a0427c942e5bfef2bac75648d80/roles/4db40e559d064cf6b4f061e785b42dbb => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:42:08.485643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581163d-e8dc-4393-88a9-6fb3f19334a4 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:08.493526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329c7db-7e29-4d5a-9e6c-ba2000823fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:08.495319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329c7db-7e29-4d5a-9e6c-ba2000823fdf None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:08.495319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329c7db-7e29-4d5a-9e6c-ba2000823fdf None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:08.495319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329c7db-7e29-4d5a-9e6c-ba2000823fdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:08.516428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329c7db-7e29-4d5a-9e6c-ba2000823fdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:08.518920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329c7db-7e29-4d5a-9e6c-ba2000823fdf None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:08.555950 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-4329c7db-7e29-4d5a-9e6c-ba2000823fdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:08.555950 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:08.555950 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:08.555950 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:08.555950 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:08.555950 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:08.555950 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:08.555950 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:08.555950 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:08.555950 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:08.555950 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:08.555950 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:08.555950 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:08.555950 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:08.555950 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:08.556866 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:08.556866 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:08.556866 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:08.556866 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:08.556866 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:08.556866 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:08.556866 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:08.556866 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:08.556866 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:08.556866 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:08.556866 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:08.556866 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:08.556866 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:08.556866 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:08.556866 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:08.556866 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:08.556866 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:08.557772 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:08.557772 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:08.557772 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:08.557772 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:08.557772 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 309/599] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:08.564516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329c7db-7e29-4d5a-9e6c-ba2000823fdf None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:08.706964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581163d-e8dc-4393-88a9-6fb3f19334a4 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:08.725415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:08.751197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:08.758630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0248c-8892-4229-8515-0477d321512f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:08.759261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0248c-8892-4229-8515-0477d321512f None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:08.761730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0248c-8892-4229-8515-0477d321512f None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:08.762089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0248c-8892-4229-8515-0477d321512f None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:08.762863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0248c-8892-4229-8515-0477d321512f None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:08.763225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0248c-8892-4229-8515-0477d321512f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:08.764579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0248c-8892-4229-8515-0477d321512f None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:08.768743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329c7db-7e29-4d5a-9e6c-ba2000823fdf None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:08.786742 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 291/600] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:08.787565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0248c-8892-4229-8515-0477d321512f None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:08.812804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52662771-fb4a-4022-9bd8-4c925e6da1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:08.813813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52662771-fb4a-4022-9bd8-4c925e6da1ca None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:08.814328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52662771-fb4a-4022-9bd8-4c925e6da1ca None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:08.815267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52662771-fb4a-4022-9bd8-4c925e6da1ca None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:08.817055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52662771-fb4a-4022-9bd8-4c925e6da1ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:08.817602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52662771-fb4a-4022-9bd8-4c925e6da1ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:08.819782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52662771-fb4a-4022-9bd8-4c925e6da1ca None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:08.839100 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 310/601] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:08.845133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52662771-fb4a-4022-9bd8-4c925e6da1ca None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:08.953457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0248c-8892-4229-8515-0477d321512f None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:09.000481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0828144-fd1a-4f0b-9f31-cdc47d39b5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:09.001295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0828144-fd1a-4f0b-9f31-cdc47d39b5b1 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:09.001787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0828144-fd1a-4f0b-9f31-cdc47d39b5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:09.001952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0828144-fd1a-4f0b-9f31-cdc47d39b5b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:09.009744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0828144-fd1a-4f0b-9f31-cdc47d39b5b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:09.011508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0828144-fd1a-4f0b-9f31-cdc47d39b5b1 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:09.026785 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-b0828144-fd1a-4f0b-9f31-cdc47d39b5b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:09.026785 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:09.026785 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:09.026785 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:09.026785 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:09.026785 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:09.026785 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:09.026785 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:09.026785 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:09.026785 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:09.026785 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:09.026785 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:09.026785 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:09.026785 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:09.026785 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:09.027741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:09.027741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:09.027741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:09.027741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:09.027741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:09.027741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:09.027741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:09.027741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:09.027741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:09.027741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:09.027741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:09.027741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:09.027741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:09.027741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:09.027741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:09.027741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:09.027741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:09.028640 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:09.028640 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:09.028640 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:09.028640 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:09.028640 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:09.028640 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:09.028640 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:09.028640 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:09.028640 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 292/602] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:09.033815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0828144-fd1a-4f0b-9f31-cdc47d39b5b1 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:09.039370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52662771-fb4a-4022-9bd8-4c925e6da1ca None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:09.065480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:09.078667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3689518-f925-4ce8-b47e-fdefcef990b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:09.079592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3689518-f925-4ce8-b47e-fdefcef990b3 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:09.079936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3689518-f925-4ce8-b47e-fdefcef990b3 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:09.089654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3689518-f925-4ce8-b47e-fdefcef990b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:09.089654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3689518-f925-4ce8-b47e-fdefcef990b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:09.089654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3689518-f925-4ce8-b47e-fdefcef990b3 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:09.108737 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-d3689518-f925-4ce8-b47e-fdefcef990b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:09.108737 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:09.108737 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:09.108737 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:09.108737 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:09.108737 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:09.108737 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:09.108737 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:09.108737 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:09.108737 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:09.108737 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:09.108737 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:09.108737 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:09.108737 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:09.108737 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:09.109885 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:09.109885 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:09.109885 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:09.109885 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:09.109885 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:09.109885 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:09.109885 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:09.109885 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:09.109885 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:09.109885 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:09.109885 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:09.109885 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:09.109885 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:09.109885 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:09.109885 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:09.109885 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:09.109885 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:09.111005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:09.111005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:09.111005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:09.111005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:09.111005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:09.111005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:09.111005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:09.111005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:09.112954 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 311/603] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:09.126925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3689518-f925-4ce8-b47e-fdefcef990b3 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:09.156455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0828144-fd1a-4f0b-9f31-cdc47d39b5b1 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:09.157468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:09.194731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fa49c-02f0-4177-82bc-2a7d1492860d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:09.195792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fa49c-02f0-4177-82bc-2a7d1492860d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:09.196349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fa49c-02f0-4177-82bc-2a7d1492860d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:09.196848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7fa49c-02f0-4177-82bc-2a7d1492860d None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:09.197872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7fa49c-02f0-4177-82bc-2a7d1492860d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:09.198683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7fa49c-02f0-4177-82bc-2a7d1492860d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:09.200342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7fa49c-02f0-4177-82bc-2a7d1492860d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:09.218076 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 293/604] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:09.220674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fa49c-02f0-4177-82bc-2a7d1492860d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:09.299640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3689518-f925-4ce8-b47e-fdefcef990b3 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:09.319300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:09.333432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7fa49c-02f0-4177-82bc-2a7d1492860d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:09.335277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aeba4f-0587-414c-8e5f-f6f5beb8dd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:09.336019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aeba4f-0587-414c-8e5f-f6f5beb8dd24 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:09.336372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aeba4f-0587-414c-8e5f-f6f5beb8dd24 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:09.336721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aeba4f-0587-414c-8e5f-f6f5beb8dd24 None admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:09.337582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aeba4f-0587-414c-8e5f-f6f5beb8dd24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:09.337991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aeba4f-0587-414c-8e5f-f6f5beb8dd24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:09.339500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aeba4f-0587-414c-8e5f-f6f5beb8dd24 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:09.357552 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 312/605] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:09.357552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86ac4f-099a-415a-86c1-211bb852cdd7 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:09.357552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86ac4f-099a-415a-86c1-211bb852cdd7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:09.357552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86ac4f-099a-415a-86c1-211bb852cdd7 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:09.360301 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 313/606] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:09.361079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86ac4f-099a-415a-86c1-211bb852cdd7 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:09.378865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c88117d-9aa1-40e6-ad5f-47a668f70fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:09.380768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88117d-9aa1-40e6-ad5f-47a668f70fe2 None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:09.381252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88117d-9aa1-40e6-ad5f-47a668f70fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:09.381756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88117d-9aa1-40e6-ad5f-47a668f70fe2 None admin] PATH_INFO: `/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/afc08d15f24e46e29c34d618d955bd28/roles/4db40e559d064cf6b4f061e785b42dbb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:09.402579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c88117d-9aa1-40e6-ad5f-47a668f70fe2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3186f9d3c2b341afa8b098ebcaaf974b, user_id=afc08d15f24e46e29c34d618d955bd28, role_id=4db40e559d064cf6b4f061e785b42dbb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:09.404167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c88117d-9aa1-40e6-ad5f-47a668f70fe2 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:09.443042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:09.469483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86ac4f-099a-415a-86c1-211bb852cdd7 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:09.502244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d0b97-cc55-407a-abd4-5aa181587465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:09.503127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d0b97-cc55-407a-abd4-5aa181587465 None admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:09.503405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d0b97-cc55-407a-abd4-5aa181587465 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:09.503677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d0b97-cc55-407a-abd4-5aa181587465 None admin] PATH_INFO: `/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/1b2a9ebcd3f84b758322a80c6e590de0/roles/4db40e559d064cf6b4f061e785b42dbb` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:09.522492 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 294/607] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:42:09 2022] PUT /identity/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/afc08d15f24e46e29c34d618d955bd28/roles/4db40e559d064cf6b4f061e785b42dbb => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:42:09.524518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bbb7d-dead-4a66-b16a-b09830bae38e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:09.525073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bbb7d-dead-4a66-b16a-b09830bae38e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:09.525613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bbb7d-dead-4a66-b16a-b09830bae38e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:09.533274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d0b97-cc55-407a-abd4-5aa181587465 None admin] RBAC: Authorizing `identity:create_grant(project_id=3186f9d3c2b341afa8b098ebcaaf974b, user_id=1b2a9ebcd3f84b758322a80c6e590de0, role_id=4db40e559d064cf6b4f061e785b42dbb)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:09.534140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d0b97-cc55-407a-abd4-5aa181587465 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:09.546258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-123bbb7d-dead-4a66-b16a-b09830bae38e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:09.549866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-123bbb7d-dead-4a66-b16a-b09830bae38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:09.626598 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 314/608] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:42:09 2022] PUT /identity/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/1b2a9ebcd3f84b758322a80c6e590de0/roles/4db40e559d064cf6b4f061e785b42dbb => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:42:09.633829 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 295/609] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:42:09.674541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d0b97-cc55-407a-abd4-5aa181587465 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:09.810907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d0b97-cc55-407a-abd4-5aa181587465 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:09.851082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:09.863147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30450f12-20dc-4618-91bb-b5d631e7da2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:09.864912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30450f12-20dc-4618-91bb-b5d631e7da2b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:09.865392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30450f12-20dc-4618-91bb-b5d631e7da2b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:09.865830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30450f12-20dc-4618-91bb-b5d631e7da2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:09.881127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30450f12-20dc-4618-91bb-b5d631e7da2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:09.889079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-30450f12-20dc-4618-91bb-b5d631e7da2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:09.889079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:09.889079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:09.889079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:09.889079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:09.889079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:09.889079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:09.889079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:09.889079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:09.889079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:09.889079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:09.889079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:09.889079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:09.889079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:09.889079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:09.889079 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:09.890235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:09.890235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:09.890235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:09.890235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:09.890235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:09.890235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:09.890235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:09.890235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:09.890235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:09.890235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:09.890235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:09.890235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:09.890235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:09.890235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:09.890235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:09.890235 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:09.891515 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:09.891515 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:09.891515 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:09.891515 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:09.891515 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:09.891515 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:09.891515 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:09.891515 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:09.891515 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:09.891515 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:09.891515 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:09.891515 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:09.894071 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 315/610] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:42:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:42:09.900008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bbb7d-dead-4a66-b16a-b09830bae38e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:09.981966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:10.019589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bbb7d-dead-4a66-b16a-b09830bae38e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:10.079933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b246206-accf-4e4b-909b-661698511144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:10.080608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b246206-accf-4e4b-909b-661698511144 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:10.080886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b246206-accf-4e4b-909b-661698511144 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:10.081162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b246206-accf-4e4b-909b-661698511144 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:10.081942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b246206-accf-4e4b-909b-661698511144 None admin] RBAC: Adding query filter params (name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:10.082285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b246206-accf-4e4b-909b-661698511144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:10.083664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b246206-accf-4e4b-909b-661698511144 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:10.100366 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 296/611] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:42:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:10.107081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30450f12-20dc-4618-91bb-b5d631e7da2b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:10.224798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30450f12-20dc-4618-91bb-b5d631e7da2b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:10.259888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db12872-6bad-46a7-b94b-3a7f01b342c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:10.260676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db12872-6bad-46a7-b94b-3a7f01b342c4 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:10.260960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db12872-6bad-46a7-b94b-3a7f01b342c4 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:10.261235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db12872-6bad-46a7-b94b-3a7f01b342c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:10.272305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db12872-6bad-46a7-b94b-3a7f01b342c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:10.273886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db12872-6bad-46a7-b94b-3a7f01b342c4 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:10.291814 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-3db12872-6bad-46a7-b94b-3a7f01b342c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:10.291814 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:10.291814 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:10.291814 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:10.291814 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:10.291814 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:10.291814 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:10.291814 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:10.291814 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:10.291814 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:10.291814 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:10.291814 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:10.291814 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:10.291814 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:10.291814 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:10.292726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:10.292726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:10.292726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:10.292726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:10.292726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:10.292726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:10.292726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:10.292726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:10.292726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:10.292726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:10.292726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:10.292726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:10.292726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:10.292726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:10.292726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:10.292726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:10.292726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:10.293610 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:10.293610 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:10.293610 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:10.293610 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:10.293610 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 316/612] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:10.299826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b246206-accf-4e4b-909b-661698511144 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:10.421895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b246206-accf-4e4b-909b-661698511144 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:10.466280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:10.472226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036231c-100d-474c-a889-40727b4826b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:10.473452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036231c-100d-474c-a889-40727b4826b6 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:10.473736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036231c-100d-474c-a889-40727b4826b6 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:10.474046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036231c-100d-474c-a889-40727b4826b6 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:10.485047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036231c-100d-474c-a889-40727b4826b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:10.485047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036231c-100d-474c-a889-40727b4826b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:10.485047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036231c-100d-474c-a889-40727b4826b6 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:10.496464 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 297/613] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:10.502138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db12872-6bad-46a7-b94b-3a7f01b342c4 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:10.546715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:10.617574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db12872-6bad-46a7-b94b-3a7f01b342c4 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:10.657183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f72d686-568d-42a2-ba54-b708997a398e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:10.658132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72d686-568d-42a2-ba54-b708997a398e None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:10.658347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72d686-568d-42a2-ba54-b708997a398e None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:10.658626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f72d686-568d-42a2-ba54-b708997a398e None admin] PATH_INFO: `/v3/users/swift` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:10.668424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72d686-568d-42a2-ba54-b708997a398e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:10.669941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f72d686-568d-42a2-ba54-b708997a398e None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:10.684312 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-5f72d686-568d-42a2-ba54-b708997a398e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 20:42:10.684312 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:10.684312 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:10.684312 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:10.684312 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:10.684312 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:10.684312 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:10.684312 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:10.684312 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:10.684312 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:10.684312 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:10.684312 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:10.684312 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:10.684312 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:10.684312 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:10.684312 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:10.685279 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:10.686224 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:10.686224 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:10.686224 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:10.686224 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:10.686224 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:10.686224 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:10.686224 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:10.686224 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:10.686224 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 20:42:10.686224 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:10.686224 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 317/614] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:42:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:10.694929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036231c-100d-474c-a889-40727b4826b6 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:10.759980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce749d59-0e66-45c1-855d-0319b109a1d7 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:10.759980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce749d59-0e66-45c1-855d-0319b109a1d7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:10.759980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce749d59-0e66-45c1-855d-0319b109a1d7 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:10.759980 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 318/615] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:10.760350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa024e1-9211-4184-9c36-da53e686557c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:10.760616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa024e1-9211-4184-9c36-da53e686557c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:10.760792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa024e1-9211-4184-9c36-da53e686557c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:10.794429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0fa024e1-9211-4184-9c36-da53e686557c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:10.795654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0fa024e1-9211-4184-9c36-da53e686557c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:10.822039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036231c-100d-474c-a889-40727b4826b6 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:10.857810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742a8d0-eb7d-44c3-a891-164b6582ef79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:10.858922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742a8d0-eb7d-44c3-a891-164b6582ef79 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:10.859408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742a8d0-eb7d-44c3-a891-164b6582ef79 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:10.859888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d742a8d0-eb7d-44c3-a891-164b6582ef79 None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:10.860969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742a8d0-eb7d-44c3-a891-164b6582ef79 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:10.861487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742a8d0-eb7d-44c3-a891-164b6582ef79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:10.863215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d742a8d0-eb7d-44c3-a891-164b6582ef79 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:10.885939 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 298/616] 10.222.0.33 () {56 vars in 1135 bytes} [Sat May 28 20:42:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:10.888094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742a8d0-eb7d-44c3-a891-164b6582ef79 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:10.929409 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 319/617] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:42:10.954625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa024e1-9211-4184-9c36-da53e686557c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:10.990201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:11.002213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:11.098184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d742a8d0-eb7d-44c3-a891-164b6582ef79 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:11.138830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa024e1-9211-4184-9c36-da53e686557c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:11.141338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e180c58-40b0-411f-857d-c036cd91c2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:11.142200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e180c58-40b0-411f-857d-c036cd91c2d3 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:11.142485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e180c58-40b0-411f-857d-c036cd91c2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:11.142761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e180c58-40b0-411f-857d-c036cd91c2d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:11.165520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e180c58-40b0-411f-857d-c036cd91c2d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:11.167083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e180c58-40b0-411f-857d-c036cd91c2d3 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:11.194797 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-7e180c58-40b0-411f-857d-c036cd91c2d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:11.194797 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:11.194797 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:11.194797 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:11.194797 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:11.194797 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:11.194797 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:11.194797 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:11.194797 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:11.194797 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:11.194797 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:11.194797 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:11.194797 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:11.194797 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:11.194797 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:11.195769 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:11.195769 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:11.195769 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:11.195769 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:11.195769 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:11.195769 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:11.195769 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:11.195769 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:11.195769 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:11.195769 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:11.195769 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:11.195769 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:11.195769 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:11.195769 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:11.195769 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:11.195769 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:11.195769 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:11.196663 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:11.196663 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:11.196663 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:11.196663 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:11.196663 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 299/618] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:11.203338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e180c58-40b0-411f-857d-c036cd91c2d3 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:11.221861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11582b-1d10-45f1-8c2c-06eef417db4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:11.221861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11582b-1d10-45f1-8c2c-06eef417db4d None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:11.221861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11582b-1d10-45f1-8c2c-06eef417db4d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:11.221861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11582b-1d10-45f1-8c2c-06eef417db4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:11.230640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11582b-1d10-45f1-8c2c-06eef417db4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:11.248239 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-8e11582b-1d10-45f1-8c2c-06eef417db4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:11.248239 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:11.248239 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:11.248239 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:11.248239 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:11.248239 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:11.248239 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:11.248239 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:11.248239 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:11.248239 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:11.248239 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:11.248239 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:11.248239 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:11.248239 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:11.248239 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:11.248239 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:11.249230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:11.249230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:11.249230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:11.249230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:11.249230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:11.249230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:11.249230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:11.249230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:11.249230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:11.249230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:11.249230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:11.249230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:11.249230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:11.249230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:11.249230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:11.249230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:11.250107 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:11.250107 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:11.250107 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:11.250107 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:11.250107 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:11.250107 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:11.250107 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:11.250107 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:11.250107 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:11.250107 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:11.250107 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:11.250107 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:11.253177 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 320/619] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:42:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:11.256255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11582b-1d10-45f1-8c2c-06eef417db4d None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:11.394673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e180c58-40b0-411f-857d-c036cd91c2d3 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:11.398160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:11.417694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:11.436837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11582b-1d10-45f1-8c2c-06eef417db4d None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:11.437543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de95fc-7f5f-4a06-8399-7dceb2700b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:11.438237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de95fc-7f5f-4a06-8399-7dceb2700b32 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:11.438521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de95fc-7f5f-4a06-8399-7dceb2700b32 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:11.438963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de95fc-7f5f-4a06-8399-7dceb2700b32 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:11.440513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de95fc-7f5f-4a06-8399-7dceb2700b32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:11.440829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de95fc-7f5f-4a06-8399-7dceb2700b32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:11.442816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de95fc-7f5f-4a06-8399-7dceb2700b32 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:11.457365 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 300/620] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:11.459927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6780f7-b1df-402d-af0b-7c656141161f None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:11.460221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6780f7-b1df-402d-af0b-7c656141161f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:11.460506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6780f7-b1df-402d-af0b-7c656141161f None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:11.462883 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 301/621] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:11.467643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6780f7-b1df-402d-af0b-7c656141161f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:11.486647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d164faa-be98-4387-9e75-a043bc661aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:11.487380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d164faa-be98-4387-9e75-a043bc661aa6 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:11.487665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d164faa-be98-4387-9e75-a043bc661aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:11.487994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d164faa-be98-4387-9e75-a043bc661aa6 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:11.489558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d164faa-be98-4387-9e75-a043bc661aa6 None admin] RBAC: Adding query filter params (name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:11.489900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d164faa-be98-4387-9e75-a043bc661aa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:11.492173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d164faa-be98-4387-9e75-a043bc661aa6 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:11.503494 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 321/622] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:42:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:11.505099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c7c66-d00c-4aff-936f-dd0ac23393cc None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:11.505395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c7c66-d00c-4aff-936f-dd0ac23393cc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:11.505680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001c7c66-d00c-4aff-936f-dd0ac23393cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:11.527908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-001c7c66-d00c-4aff-936f-dd0ac23393cc None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:11.529666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-001c7c66-d00c-4aff-936f-dd0ac23393cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:11.600911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6780f7-b1df-402d-af0b-7c656141161f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:11.625858 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 322/623] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:42:11.625858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c7c66-d00c-4aff-936f-dd0ac23393cc None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:11.660300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a272e23-8472-4385-afb1-397a1b81e5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:11.661021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a272e23-8472-4385-afb1-397a1b81e5c0 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:11.661307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a272e23-8472-4385-afb1-397a1b81e5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:11.661582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a272e23-8472-4385-afb1-397a1b81e5c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:11.669012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a272e23-8472-4385-afb1-397a1b81e5c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:11.670583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a272e23-8472-4385-afb1-397a1b81e5c0 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:11.687045 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-6a272e23-8472-4385-afb1-397a1b81e5c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:11.687045 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:11.687045 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:11.687045 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:11.687045 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:11.687045 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:11.687045 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:11.687045 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:11.687045 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:11.687045 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:11.687045 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:11.687045 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:11.687045 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:11.687045 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:11.687045 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:11.687988 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:11.687988 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:11.687988 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:11.687988 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:11.687988 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:11.687988 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:11.687988 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:11.687988 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:11.687988 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:11.687988 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:11.687988 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:11.687988 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:11.687988 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:11.687988 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:11.687988 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:11.687988 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:11.687988 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:11.688885 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:11.688885 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:11.688885 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:11.688885 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:11.688885 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:11.688885 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:11.688885 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:11.688885 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:11.688885 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 302/624] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:11.688885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a272e23-8472-4385-afb1-397a1b81e5c0 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:11.717829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:11.726423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:11.755775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001c7c66-d00c-4aff-936f-dd0ac23393cc None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:11.795079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee28e1c-4d10-49da-ac18-d37f7c494d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:11.795831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee28e1c-4d10-49da-ac18-d37f7c494d08 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:11.796226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee28e1c-4d10-49da-ac18-d37f7c494d08 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:11.796537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee28e1c-4d10-49da-ac18-d37f7c494d08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:11.808470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee28e1c-4d10-49da-ac18-d37f7c494d08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:11.810062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee28e1c-4d10-49da-ac18-d37f7c494d08 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:11.822707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a272e23-8472-4385-afb1-397a1b81e5c0 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:11.829861 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-cee28e1c-4d10-49da-ac18-d37f7c494d08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:11.829861 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:11.829861 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:11.829861 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:11.829861 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:11.829861 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:11.829861 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:11.829861 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:11.829861 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:11.829861 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:11.829861 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:11.829861 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:11.829861 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:11.829861 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:11.829861 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:11.830971 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:11.830971 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:11.830971 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:11.830971 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:11.830971 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:11.830971 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:11.830971 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:11.830971 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:11.830971 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:11.830971 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:11.830971 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:11.830971 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:11.830971 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:11.830971 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:11.830971 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:11.830971 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:11.830971 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:11.832653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:11.832653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:11.832653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:11.832653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:11.838179 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 323/625] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:11.839088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee28e1c-4d10-49da-ac18-d37f7c494d08 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:11.869564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ba4c7-19bf-488c-838c-5c3ec371dd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:11.871178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ba4c7-19bf-488c-838c-5c3ec371dd57 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:11.871853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ba4c7-19bf-488c-838c-5c3ec371dd57 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:11.872276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ba4c7-19bf-488c-838c-5c3ec371dd57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:11.891748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1ba4c7-19bf-488c-838c-5c3ec371dd57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:11.902672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-0f1ba4c7-19bf-488c-838c-5c3ec371dd57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:11.902672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:11.902672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:11.902672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:11.902672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:11.902672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:11.902672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:11.902672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:11.902672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:11.902672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:11.902672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:11.902672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:11.902672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:11.902672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:11.902672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:11.902672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:11.903846 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:11.903846 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:11.903846 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:11.903846 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:11.903846 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:11.903846 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:11.903846 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:11.903846 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:11.903846 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:11.903846 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:11.903846 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:11.903846 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:11.903846 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:11.903846 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:11.903846 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:11.903846 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:11.904872 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:11.904872 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:11.904872 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:11.904872 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:11.904872 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:11.904872 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:11.904872 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:11.904872 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:11.904872 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:11.904872 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:11.904872 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:11.904872 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:11.908151 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 303/626] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:42:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:11.909157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ba4c7-19bf-488c-838c-5c3ec371dd57 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:11.966305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee28e1c-4d10-49da-ac18-d37f7c494d08 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:12.002155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:12.005538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:12.012705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e365054-b632-41ab-bfb5-78cce1cec5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:12.013476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e365054-b632-41ab-bfb5-78cce1cec5c9 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:12.013771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e365054-b632-41ab-bfb5-78cce1cec5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:12.014163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e365054-b632-41ab-bfb5-78cce1cec5c9 None admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:12.015250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e365054-b632-41ab-bfb5-78cce1cec5c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:12.015519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e365054-b632-41ab-bfb5-78cce1cec5c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:12.017719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e365054-b632-41ab-bfb5-78cce1cec5c9 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:12.026329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1ba4c7-19bf-488c-838c-5c3ec371dd57 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:12.033927 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 324/627] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:12.035031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e365054-b632-41ab-bfb5-78cce1cec5c9 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:12.076546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d47d70-a03d-482b-aa00-885beced6a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:12.077329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d47d70-a03d-482b-aa00-885beced6a83 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:12.077565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d47d70-a03d-482b-aa00-885beced6a83 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:12.077945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d47d70-a03d-482b-aa00-885beced6a83 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:12.078701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d47d70-a03d-482b-aa00-885beced6a83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:12.079321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d47d70-a03d-482b-aa00-885beced6a83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:12.080514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d47d70-a03d-482b-aa00-885beced6a83 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:12.099451 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 304/628] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:12.100462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d47d70-a03d-482b-aa00-885beced6a83 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:12.181022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e365054-b632-41ab-bfb5-78cce1cec5c9 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:12.218741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6483907-a942-477a-99de-0c58997618dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:12.219335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6483907-a942-477a-99de-0c58997618dc None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:12.219695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6483907-a942-477a-99de-0c58997618dc None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:12.219955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6483907-a942-477a-99de-0c58997618dc None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:12.220721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6483907-a942-477a-99de-0c58997618dc None admin] RBAC: Adding query filter params (name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:12.221031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6483907-a942-477a-99de-0c58997618dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:12.223366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6483907-a942-477a-99de-0c58997618dc None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:12.230601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d47d70-a03d-482b-aa00-885beced6a83 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:12.236635 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 325/629] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:42:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:12.237605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6483907-a942-477a-99de-0c58997618dc None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:12.275646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f964c10-c108-4948-8fc5-93bfa2af6218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:12.278035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f964c10-c108-4948-8fc5-93bfa2af6218 None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:12.278636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f964c10-c108-4948-8fc5-93bfa2af6218 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:12.279151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f964c10-c108-4948-8fc5-93bfa2af6218 None admin] PATH_INFO: `/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/a01bba888a644eb18fc457aff578aa4b/roles/4db40e559d064cf6b4f061e785b42dbb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:12.294746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:12.299038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:12.299722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f964c10-c108-4948-8fc5-93bfa2af6218 None admin] RBAC: Authorizing `identity:create_grant(project_id=3186f9d3c2b341afa8b098ebcaaf974b, user_id=a01bba888a644eb18fc457aff578aa4b, role_id=4db40e559d064cf6b4f061e785b42dbb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:12.301844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f964c10-c108-4948-8fc5-93bfa2af6218 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:12.345159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6483907-a942-477a-99de-0c58997618dc None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:12.378541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4214e2c-e0c5-4d0a-86a2-3b3a742c7eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:12.379365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4214e2c-e0c5-4d0a-86a2-3b3a742c7eef None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:12.379647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4214e2c-e0c5-4d0a-86a2-3b3a742c7eef None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:12.379925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4214e2c-e0c5-4d0a-86a2-3b3a742c7eef None admin] PATH_INFO: `/v3/users/nova` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:12.387426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4214e2c-e0c5-4d0a-86a2-3b3a742c7eef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:12.388898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4214e2c-e0c5-4d0a-86a2-3b3a742c7eef None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:12.394988 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 305/630] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:42:12 2022] PUT /identity/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/a01bba888a644eb18fc457aff578aa4b/roles/4db40e559d064cf6b4f061e785b42dbb => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:42:12.395824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f964c10-c108-4948-8fc5-93bfa2af6218 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:12.404821 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-f4214e2c-e0c5-4d0a-86a2-3b3a742c7eef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 20:42:12.404821 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:12.404821 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:12.404821 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:12.404821 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:12.404821 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:12.404821 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:12.404821 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:12.404821 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:12.404821 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:12.404821 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:12.404821 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:12.404821 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:12.404821 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:12.404821 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:12.404821 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:12.406127 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:12.407091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:12.407091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:12.407091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:12.407091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:12.407091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:12.407091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:12.407091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:12.407091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:12.407091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 20:42:12.407091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:12.407091 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 326/631] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 20:42:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:12.408481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a9049-2cd6-4fa6-92ee-04189bbdefe9 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:12.408781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a9049-2cd6-4fa6-92ee-04189bbdefe9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:12.409058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a9049-2cd6-4fa6-92ee-04189bbdefe9 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:12.411231 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 327/632] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:12.412513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9734b92-286e-4b1c-aeae-b660c5ac15e4 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:12.412640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9734b92-286e-4b1c-aeae-b660c5ac15e4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:12.412831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9734b92-286e-4b1c-aeae-b660c5ac15e4 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:12.414233 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 328/633] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:12.419208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9734b92-286e-4b1c-aeae-b660c5ac15e4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:12.525232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f964c10-c108-4948-8fc5-93bfa2af6218 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:12.544574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9734b92-286e-4b1c-aeae-b660c5ac15e4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:12.559352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:12.565120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07e1d23-4c1b-43d8-87b1-b3db35860033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:12.566023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e1d23-4c1b-43d8-87b1-b3db35860033 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:12.566258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e1d23-4c1b-43d8-87b1-b3db35860033 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:12.571943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e1d23-4c1b-43d8-87b1-b3db35860033 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:12.579398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07e1d23-4c1b-43d8-87b1-b3db35860033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:12.580894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07e1d23-4c1b-43d8-87b1-b3db35860033 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:12.583373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebd2f0e-a853-4dde-9634-d448958e5dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:12.584426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd2f0e-a853-4dde-9634-d448958e5dfe None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:12.584718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd2f0e-a853-4dde-9634-d448958e5dfe None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:12.585003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebd2f0e-a853-4dde-9634-d448958e5dfe None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:12.585888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd2f0e-a853-4dde-9634-d448958e5dfe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:12.586212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd2f0e-a853-4dde-9634-d448958e5dfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:12.587663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebd2f0e-a853-4dde-9634-d448958e5dfe None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:12.602964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-f07e1d23-4c1b-43d8-87b1-b3db35860033 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:12.602964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:12.602964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:12.602964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:12.602964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:12.602964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:12.602964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:12.602964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:12.602964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:12.602964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:12.602964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:12.602964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:12.602964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:12.602964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:12.602964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:12.604119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:12.604119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:12.604119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:12.604119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:12.604119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:12.604119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:12.604119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:12.604119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:12.604119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:12.604119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:12.604119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:12.604119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:12.604119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:12.604119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:12.604119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:12.604119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:12.604119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:12.605161 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:12.605161 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:12.605161 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:12.605161 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:12.605161 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 306/634] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:12.605569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a9b94-fafc-4994-8500-af842a51f54d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:12.605883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a9b94-fafc-4994-8500-af842a51f54d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:12.606150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a9b94-fafc-4994-8500-af842a51f54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:12.608575 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 329/635] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 20:42:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:12.609922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3874111-3351-4af2-b3e2-ba8fdc53b040 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:12.610188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3874111-3351-4af2-b3e2-ba8fdc53b040 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:12.610479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3874111-3351-4af2-b3e2-ba8fdc53b040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:12.628422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-370a9b94-fafc-4994-8500-af842a51f54d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:12.629662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-370a9b94-fafc-4994-8500-af842a51f54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:12.632133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e3874111-3351-4af2-b3e2-ba8fdc53b040 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:12.634192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e3874111-3351-4af2-b3e2-ba8fdc53b040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:12.666850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:12.701999 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 307/636] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:42:12.703494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a9b94-fafc-4994-8500-af842a51f54d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:12.712325 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 330/637] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:42:12.713470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3874111-3351-4af2-b3e2-ba8fdc53b040 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:12.822858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a9b94-fafc-4994-8500-af842a51f54d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:12.827815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:12.830919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3874111-3351-4af2-b3e2-ba8fdc53b040 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:12.864450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4c7d8-df95-46cc-9c12-0819ff39214f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:12.865124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4c7d8-df95-46cc-9c12-0819ff39214f None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:12.865407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4c7d8-df95-46cc-9c12-0819ff39214f None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:12.865680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d4c7d8-df95-46cc-9c12-0819ff39214f None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:12.866506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d4c7d8-df95-46cc-9c12-0819ff39214f None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:12.866812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d4c7d8-df95-46cc-9c12-0819ff39214f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:12.868184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d4c7d8-df95-46cc-9c12-0819ff39214f None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:12.872758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee51f8a-ac5b-4c22-9628-1c459acd1123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:12.872758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee51f8a-ac5b-4c22-9628-1c459acd1123 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:12.872758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee51f8a-ac5b-4c22-9628-1c459acd1123 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:12.872758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee51f8a-ac5b-4c22-9628-1c459acd1123 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:12.884813 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 308/638] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:12.884813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4c7d8-df95-46cc-9c12-0819ff39214f None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:12.920334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:12.954659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee51f8a-ac5b-4c22-9628-1c459acd1123 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:12.956135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee51f8a-ac5b-4c22-9628-1c459acd1123 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:12.991130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d4c7d8-df95-46cc-9c12-0819ff39214f None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:13.026512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40aa7d4-fa80-4a91-8c57-33f6040d10c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:13.027308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40aa7d4-fa80-4a91-8c57-33f6040d10c4 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:13.027596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40aa7d4-fa80-4a91-8c57-33f6040d10c4 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:13.027874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40aa7d4-fa80-4a91-8c57-33f6040d10c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:13.039188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40aa7d4-fa80-4a91-8c57-33f6040d10c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:13.046892 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-d40aa7d4-fa80-4a91-8c57-33f6040d10c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:13.046892 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:13.046892 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:13.046892 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:13.046892 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:13.046892 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:13.046892 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:13.046892 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:13.046892 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:13.046892 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:13.046892 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:13.046892 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:13.046892 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:13.046892 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:13.046892 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:13.046892 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:13.047837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:13.047837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:13.047837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:13.047837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:13.047837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:13.047837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:13.047837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:13.047837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:13.047837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:13.047837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:13.047837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:13.047837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:13.047837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:13.047837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:13.047837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:13.047837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:13.048761 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:13.048761 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:13.048761 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:13.048761 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:13.048761 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:13.048761 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:13.048761 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:13.048761 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:13.048761 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:13.048761 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:13.048761 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:13.048761 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:13.048761 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 309/639] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:42:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:13.049447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40aa7d4-fa80-4a91-8c57-33f6040d10c4 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:13.075058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:13.079536 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 331/640] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:42:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:42:13.080413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee51f8a-ac5b-4c22-9628-1c459acd1123 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:13.155264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:13.164948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40aa7d4-fa80-4a91-8c57-33f6040d10c4 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:13.189168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee51f8a-ac5b-4c22-9628-1c459acd1123 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:13.201044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19751c68-dbe3-4418-a2e1-45141fcf34c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:13.201873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19751c68-dbe3-4418-a2e1-45141fcf34c9 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:13.202158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19751c68-dbe3-4418-a2e1-45141fcf34c9 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:13.202447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19751c68-dbe3-4418-a2e1-45141fcf34c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:13.213831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19751c68-dbe3-4418-a2e1-45141fcf34c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:13.221634 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-19751c68-dbe3-4418-a2e1-45141fcf34c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:13.221634 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:13.221634 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:13.221634 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:13.221634 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:13.221634 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:13.221634 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:13.221634 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:13.221634 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:13.221634 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:13.221634 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:13.221634 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:13.221634 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:13.221634 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:13.221634 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:13.221634 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:13.222572 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:13.222572 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:13.222572 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:13.222572 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:13.222572 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:13.222572 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:13.222572 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:13.222572 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:13.222572 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:13.222572 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:13.222572 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:13.222572 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:13.222572 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:13.222572 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:13.222572 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:13.222572 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:13.223449 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:13.223449 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:13.223449 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:13.223449 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:13.223449 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:13.223449 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:13.223449 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:13.223449 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:13.223449 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:13.223449 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:13.223449 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:13.223449 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:13.223449 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 310/641] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:42:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:13.224184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19751c68-dbe3-4418-a2e1-45141fcf34c9 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:13.231565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff4f6f-c48d-4c7e-ae65-b502541b2ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:13.232384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff4f6f-c48d-4c7e-ae65-b502541b2ffb None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:13.232669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff4f6f-c48d-4c7e-ae65-b502541b2ffb None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:13.232948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff4f6f-c48d-4c7e-ae65-b502541b2ffb None admin] PATH_INFO: `/v3/users/glance` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:13.241354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff4f6f-c48d-4c7e-ae65-b502541b2ffb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:13.242835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ff4f6f-c48d-4c7e-ae65-b502541b2ffb None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:13.257481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-b2ff4f6f-c48d-4c7e-ae65-b502541b2ffb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 28 20:42:13.257481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:13.257481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:13.257481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:13.257481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:13.257481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:13.257481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:13.257481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:13.257481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:13.257481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:13.257481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:13.257481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:13.257481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:13.257481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:13.257481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:13.257481 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:13.258537 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:13.259514 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:13.259514 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:13.259514 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:13.259514 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:13.259514 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:13.259514 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:13.259514 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:13.259514 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:13.259514 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 28 20:42:13.259514 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:13.259514 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 332/642] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:42:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:13.259514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff4f6f-c48d-4c7e-ae65-b502541b2ffb None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:13.321387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:13.336096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19751c68-dbe3-4418-a2e1-45141fcf34c9 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:13.362746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff4f6f-c48d-4c7e-ae65-b502541b2ffb None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:13.370553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db17f6f-bc56-49cd-8bff-c9f043027b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:13.371213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db17f6f-bc56-49cd-8bff-c9f043027b66 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:13.371498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db17f6f-bc56-49cd-8bff-c9f043027b66 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:13.371781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db17f6f-bc56-49cd-8bff-c9f043027b66 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:13.372536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db17f6f-bc56-49cd-8bff-c9f043027b66 None admin] RBAC: Adding query filter params (name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:13.372844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db17f6f-bc56-49cd-8bff-c9f043027b66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:13.374263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db17f6f-bc56-49cd-8bff-c9f043027b66 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:13.387187 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 311/643] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:42:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:13.387187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db17f6f-bc56-49cd-8bff-c9f043027b66 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:13.402531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de8616-ac88-4a90-a80c-df937b99c6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:13.403269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:13.403388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de8616-ac88-4a90-a80c-df937b99c6aa None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:13.403604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de8616-ac88-4a90-a80c-df937b99c6aa None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:13.404149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de8616-ac88-4a90-a80c-df937b99c6aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:13.414667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de8616-ac88-4a90-a80c-df937b99c6aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:13.416263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de8616-ac88-4a90-a80c-df937b99c6aa None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:13.434565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-34de8616-ac88-4a90-a80c-df937b99c6aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:13.434565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:13.434565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:13.434565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:13.434565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:13.434565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:13.434565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:13.434565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:13.434565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:13.434565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:13.434565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:13.434565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:13.434565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:13.434565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:13.434565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:13.435505 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:13.435505 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:13.435505 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:13.435505 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:13.435505 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:13.435505 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:13.435505 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:13.435505 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:13.435505 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:13.435505 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:13.435505 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:13.435505 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:13.435505 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:13.435505 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:13.435505 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:13.435505 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:13.435505 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:13.436553 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:13.436553 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:13.436553 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:13.436553 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:13.436553 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 333/644] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:13.436858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de8616-ac88-4a90-a80c-df937b99c6aa None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:13.494722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db17f6f-bc56-49cd-8bff-c9f043027b66 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:13.532485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4da4fc-94a7-4d0f-9085-bb50fffc7c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:13.533177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4da4fc-94a7-4d0f-9085-bb50fffc7c2d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:13.533465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4da4fc-94a7-4d0f-9085-bb50fffc7c2d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:13.533746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4da4fc-94a7-4d0f-9085-bb50fffc7c2d None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:13.534579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4da4fc-94a7-4d0f-9085-bb50fffc7c2d None admin] RBAC: Adding query filter params (name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:13.534896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4da4fc-94a7-4d0f-9085-bb50fffc7c2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:13.536883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4da4fc-94a7-4d0f-9085-bb50fffc7c2d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:13.546163 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 312/645] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:42:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:13.546968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4da4fc-94a7-4d0f-9085-bb50fffc7c2d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:13.552954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de8616-ac88-4a90-a80c-df937b99c6aa None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:13.566110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:13.591166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6448fa-3435-427c-adbc-bad55b50f426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:13.591789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6448fa-3435-427c-adbc-bad55b50f426 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:13.592064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6448fa-3435-427c-adbc-bad55b50f426 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:13.592345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6448fa-3435-427c-adbc-bad55b50f426 None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:13.593246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6448fa-3435-427c-adbc-bad55b50f426 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:13.594261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6448fa-3435-427c-adbc-bad55b50f426 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:13.596065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6448fa-3435-427c-adbc-bad55b50f426 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:13.621048 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 334/646] 10.222.0.33 () {56 vars in 1137 bytes} [Sat May 28 20:42:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:13.621048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6448fa-3435-427c-adbc-bad55b50f426 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:13.634566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:13.658879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4da4fc-94a7-4d0f-9085-bb50fffc7c2d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:13.695350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc5ceb-967d-4eb8-8483-f4c20ba7a0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:13.696147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc5ceb-967d-4eb8-8483-f4c20ba7a0ba None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:13.696844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc5ceb-967d-4eb8-8483-f4c20ba7a0ba None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:13.697130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fc5ceb-967d-4eb8-8483-f4c20ba7a0ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:13.711488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc5ceb-967d-4eb8-8483-f4c20ba7a0ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:13.713708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fc5ceb-967d-4eb8-8483-f4c20ba7a0ba None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:13.720911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6448fa-3435-427c-adbc-bad55b50f426 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:13.736268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-33fc5ceb-967d-4eb8-8483-f4c20ba7a0ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:13.736268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:13.736268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:13.736268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:13.736268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:13.736268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:13.736268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:13.736268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:13.736268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:13.736268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:13.736268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:13.736268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:13.736268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:13.736268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:13.736268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:13.737292 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:13.737292 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:13.737292 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:13.737292 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:13.737292 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:13.737292 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:13.737292 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:13.737292 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:13.737292 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:13.737292 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:13.737292 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:13.737292 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:13.737292 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:13.737292 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:13.737292 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:13.737292 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:13.737292 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:13.738740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:13.738740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:13.738740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:13.738740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:13.739384 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 313/647] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:13.740260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc5ceb-967d-4eb8-8483-f4c20ba7a0ba None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:13.761241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5978e5-4f97-440f-a94c-6de4c1e11d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:13.761831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5978e5-4f97-440f-a94c-6de4c1e11d82 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:13.762151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5978e5-4f97-440f-a94c-6de4c1e11d82 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:13.762429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5978e5-4f97-440f-a94c-6de4c1e11d82 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:13.763156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5978e5-4f97-440f-a94c-6de4c1e11d82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:13.763466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5978e5-4f97-440f-a94c-6de4c1e11d82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:13.764838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5978e5-4f97-440f-a94c-6de4c1e11d82 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:13.778488 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 335/648] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:13.778846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5978e5-4f97-440f-a94c-6de4c1e11d82 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:13.812610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:13.848278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fc5ceb-967d-4eb8-8483-f4c20ba7a0ba None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:13.874443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:13.883195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041de63-a93f-4414-90cd-6becf09863cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:13.883988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041de63-a93f-4414-90cd-6becf09863cd None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:13.884268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041de63-a93f-4414-90cd-6becf09863cd None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:13.884540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041de63-a93f-4414-90cd-6becf09863cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:13.892769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5978e5-4f97-440f-a94c-6de4c1e11d82 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:13.896877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041de63-a93f-4414-90cd-6becf09863cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:13.898360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041de63-a93f-4414-90cd-6becf09863cd None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:13.916927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-e041de63-a93f-4414-90cd-6becf09863cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:13.916927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:13.916927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:13.916927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:13.916927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:13.916927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:13.916927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:13.916927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:13.916927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:13.916927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:13.916927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:13.916927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:13.916927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:13.916927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:13.916927 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:13.917880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:13.917880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:13.917880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:13.917880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:13.917880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:13.917880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:13.917880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:13.917880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:13.917880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:13.917880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:13.917880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:13.917880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:13.917880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:13.917880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:13.917880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:13.917880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:13.917880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:13.918774 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:13.918774 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:13.918774 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:13.918774 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:13.918774 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 314/649] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:13.919071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041de63-a93f-4414-90cd-6becf09863cd None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:13.933902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5748acc-2e71-46b2-9300-e6fe85171788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:13.934926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5748acc-2e71-46b2-9300-e6fe85171788 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:13.935416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5748acc-2e71-46b2-9300-e6fe85171788 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:13.935783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5748acc-2e71-46b2-9300-e6fe85171788 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:14.007616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5748acc-2e71-46b2-9300-e6fe85171788 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:14.009081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5748acc-2e71-46b2-9300-e6fe85171788 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:14.032560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041de63-a93f-4414-90cd-6becf09863cd None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:14.040350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:14.072593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93965a7-fb08-4ca2-9e89-58c6b7b7ec83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:14.074023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93965a7-fb08-4ca2-9e89-58c6b7b7ec83 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:14.074309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93965a7-fb08-4ca2-9e89-58c6b7b7ec83 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:14.074611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93965a7-fb08-4ca2-9e89-58c6b7b7ec83 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:14.075307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93965a7-fb08-4ca2-9e89-58c6b7b7ec83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:14.075612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93965a7-fb08-4ca2-9e89-58c6b7b7ec83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:14.077002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93965a7-fb08-4ca2-9e89-58c6b7b7ec83 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:14.093670 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 315/650] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:14.093670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93965a7-fb08-4ca2-9e89-58c6b7b7ec83 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:14.125066 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 336/651] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:42:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:42:14.126220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5748acc-2e71-46b2-9300-e6fe85171788 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:14.138342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:14.199621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93965a7-fb08-4ca2-9e89-58c6b7b7ec83 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:14.235276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd6ac9-96b7-48e0-a68e-c2eb459822da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:14.235653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5748acc-2e71-46b2-9300-e6fe85171788 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:14.236077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd6ac9-96b7-48e0-a68e-c2eb459822da None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:14.236346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd6ac9-96b7-48e0-a68e-c2eb459822da None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:14.236622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd6ac9-96b7-48e0-a68e-c2eb459822da None admin] PATH_INFO: `/v3/projects/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:14.243740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd6ac9-96b7-48e0-a68e-c2eb459822da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:14.245201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd6ac9-96b7-48e0-a68e-c2eb459822da None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:14.259437 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-4afd6ac9-96b7-48e0-a68e-c2eb459822da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:14.259437 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:14.259437 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:14.259437 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:14.259437 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:14.259437 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:14.259437 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:14.259437 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:14.259437 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:14.259437 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:14.259437 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:14.259437 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:14.259437 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:14.259437 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:14.259437 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:14.260354 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:14.260354 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:14.260354 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:14.260354 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:14.260354 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:14.260354 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:14.260354 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:14.260354 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:14.260354 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:14.260354 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:14.260354 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:14.260354 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:14.260354 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:14.260354 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:14.260354 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:14.260354 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:14.260354 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:14.261238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:14.261238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:14.261238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:14.261238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:14.261238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:14.261238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:14.261238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:14.261238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:14.261238 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 316/652] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:14.262328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd6ac9-96b7-48e0-a68e-c2eb459822da None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:14.270437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:14.276174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b37064-2aca-4b56-8765-7d4743d90abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:14.276845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b37064-2aca-4b56-8765-7d4743d90abf None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:14.277130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b37064-2aca-4b56-8765-7d4743d90abf None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:14.277478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b37064-2aca-4b56-8765-7d4743d90abf None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:14.278257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b37064-2aca-4b56-8765-7d4743d90abf None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:14.278567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b37064-2aca-4b56-8765-7d4743d90abf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:14.279970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b37064-2aca-4b56-8765-7d4743d90abf None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:14.292221 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 337/653] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:14.293492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b37064-2aca-4b56-8765-7d4743d90abf None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:14.371830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:14.375357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd6ac9-96b7-48e0-a68e-c2eb459822da None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:14.401631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b37064-2aca-4b56-8765-7d4743d90abf None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:14.423417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19196631-9e02-4f1a-ada0-66940b2170d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:14.424582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19196631-9e02-4f1a-ada0-66940b2170d4 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:14.425414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19196631-9e02-4f1a-ada0-66940b2170d4 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:14.425998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19196631-9e02-4f1a-ada0-66940b2170d4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:14.434852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19196631-9e02-4f1a-ada0-66940b2170d4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:14.436386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19196631-9e02-4f1a-ada0-66940b2170d4 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:14.443209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be5b7a-c384-4ebd-81ab-22d96308415f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:14.444008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be5b7a-c384-4ebd-81ab-22d96308415f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:14.444296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be5b7a-c384-4ebd-81ab-22d96308415f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:14.444638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be5b7a-c384-4ebd-81ab-22d96308415f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:14.451440 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-19196631-9e02-4f1a-ada0-66940b2170d4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 28 20:42:14.451440 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:14.451440 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:14.451440 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:14.451440 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:14.451440 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:14.451440 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:14.451440 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:14.451440 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:14.451440 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:14.451440 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:14.451440 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:14.451440 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:14.451440 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:14.451440 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:14.451440 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:14.452417 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:14.458424 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:14.458424 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:14.458424 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:14.458424 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:14.458424 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:14.458424 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:14.458424 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:14.458424 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:14.458424 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 28 20:42:14.458424 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:14.458424 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 317/654] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:42:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:14.458424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19196631-9e02-4f1a-ada0-66940b2170d4 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:14.459087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be5b7a-c384-4ebd-81ab-22d96308415f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:14.459742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be5b7a-c384-4ebd-81ab-22d96308415f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:14.477546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-69be5b7a-c384-4ebd-81ab-22d96308415f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:14.477546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:14.477546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:14.477546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:14.477546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:14.477546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:14.477546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:14.477546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:14.477546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:14.477546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:14.477546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:14.477546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:14.477546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:14.477546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:14.477546 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:14.478863 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:14.478863 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:14.478863 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:14.478863 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:14.478863 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:14.478863 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:14.478863 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:14.478863 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:14.478863 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:14.478863 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:14.478863 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:14.478863 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:14.478863 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:14.478863 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:14.478863 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:14.478863 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:14.478863 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:14.480047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:14.480047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:14.480047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:14.480047 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:14.483660 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 338/655] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:14.489570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be5b7a-c384-4ebd-81ab-22d96308415f None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:14.535181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:14.575290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19196631-9e02-4f1a-ada0-66940b2170d4 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:14.597526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be5b7a-c384-4ebd-81ab-22d96308415f None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:14.615903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3cbb01-d295-45aa-805a-71c656268f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:14.616564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3cbb01-d295-45aa-805a-71c656268f9c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:14.616854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3cbb01-d295-45aa-805a-71c656268f9c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:14.617135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3cbb01-d295-45aa-805a-71c656268f9c None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:14.617955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3cbb01-d295-45aa-805a-71c656268f9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:14.618267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3cbb01-d295-45aa-805a-71c656268f9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:14.619713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3cbb01-d295-45aa-805a-71c656268f9c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:14.636910 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 318/656] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:14.636910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3cbb01-d295-45aa-805a-71c656268f9c None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:14.641960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5f3ee-a5fb-48f1-a130-4bec9d1b1ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:14.642817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5f3ee-a5fb-48f1-a130-4bec9d1b1ba8 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:14.643418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5f3ee-a5fb-48f1-a130-4bec9d1b1ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:14.643527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb5f3ee-a5fb-48f1-a130-4bec9d1b1ba8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:14.652466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5f3ee-a5fb-48f1-a130-4bec9d1b1ba8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:14.654183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb5f3ee-a5fb-48f1-a130-4bec9d1b1ba8 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:14.657486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:14.670334 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-9eb5f3ee-a5fb-48f1-a130-4bec9d1b1ba8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 28 20:42:14.670334 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:14.670334 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:14.670334 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:14.670334 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:14.670334 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:14.670334 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:14.670334 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:14.670334 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:14.670334 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:14.670334 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:14.670334 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:14.670334 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:14.670334 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:14.670334 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:14.670334 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:14.671480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:14.672452 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:14.672452 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:14.672452 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:14.672452 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:14.672452 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:14.672452 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:14.672452 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:14.672452 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:14.672452 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 28 20:42:14.672452 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:14.672452 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 339/657] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:42:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:14.673050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5f3ee-a5fb-48f1-a130-4bec9d1b1ba8 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:14.741618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3cbb01-d295-45aa-805a-71c656268f9c None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:14.774831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb5f3ee-a5fb-48f1-a130-4bec9d1b1ba8 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:14.776244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05055d04-0ca6-401f-806c-62c20d16af89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:14.776418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:14.776923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05055d04-0ca6-401f-806c-62c20d16af89 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:14.777198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05055d04-0ca6-401f-806c-62c20d16af89 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:14.777477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05055d04-0ca6-401f-806c-62c20d16af89 None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:14.778354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05055d04-0ca6-401f-806c-62c20d16af89 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:14.778674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05055d04-0ca6-401f-806c-62c20d16af89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:14.780116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05055d04-0ca6-401f-806c-62c20d16af89 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:14.797123 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 319/658] 10.222.0.33 () {56 vars in 1139 bytes} [Sat May 28 20:42:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:14.797945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05055d04-0ca6-401f-806c-62c20d16af89 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:14.813739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17b3e0-db57-4bba-8291-773afae1272c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:14.814434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17b3e0-db57-4bba-8291-773afae1272c None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:14.814724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17b3e0-db57-4bba-8291-773afae1272c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:14.815005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17b3e0-db57-4bba-8291-773afae1272c None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:14.815733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17b3e0-db57-4bba-8291-773afae1272c None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:14.816043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17b3e0-db57-4bba-8291-773afae1272c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:14.817981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17b3e0-db57-4bba-8291-773afae1272c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:14.829488 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 340/659] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:14.830297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17b3e0-db57-4bba-8291-773afae1272c None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:14.877103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:14.899430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05055d04-0ca6-401f-806c-62c20d16af89 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:14.930621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17b3e0-db57-4bba-8291-773afae1272c None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:14.934616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44abc110-4605-4e85-bc4c-bb6c37b8e9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:14.935266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abc110-4605-4e85-bc4c-bb6c37b8e9b3 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:14.935549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abc110-4605-4e85-bc4c-bb6c37b8e9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:14.935825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abc110-4605-4e85-bc4c-bb6c37b8e9b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:14.936654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44abc110-4605-4e85-bc4c-bb6c37b8e9b3 None admin] RBAC: Adding query filter params (role.id=4db40e559d064cf6b4f061e785b42dbb, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=cfdebb5e6b8d4e2d86f0a03a30c21671) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:14.936967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44abc110-4605-4e85-bc4c-bb6c37b8e9b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:14.938453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44abc110-4605-4e85-bc4c-bb6c37b8e9b3 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:14.944883 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 320/660] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:14 2022] GET /identity/v3/role_assignments?user.id=cfdebb5e6b8d4e2d86f0a03a30c21671&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=4db40e559d064cf6b4f061e785b42dbb => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:14.945915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f38cc-a8eb-4f31-ad15-94be5183f2c5 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:14.946206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f38cc-a8eb-4f31-ad15-94be5183f2c5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:14.946485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f38cc-a8eb-4f31-ad15-94be5183f2c5 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:14.947852 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 321/661] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:14.948393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f38cc-a8eb-4f31-ad15-94be5183f2c5 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:14.981537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bfc92-e29d-4ed4-8d4d-c258adeebc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:14.982442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bfc92-e29d-4ed4-8d4d-c258adeebc96 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:14.982732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bfc92-e29d-4ed4-8d4d-c258adeebc96 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:14.983011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6bfc92-e29d-4ed4-8d4d-c258adeebc96 None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:14.983882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bfc92-e29d-4ed4-8d4d-c258adeebc96 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:14.984235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bfc92-e29d-4ed4-8d4d-c258adeebc96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:14.985737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6bfc92-e29d-4ed4-8d4d-c258adeebc96 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:15.011299 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 341/662] 10.222.0.33 () {56 vars in 1137 bytes} [Sat May 28 20:42:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:15.011299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bfc92-e29d-4ed4-8d4d-c258adeebc96 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:15.036969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:15.087664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f38cc-a8eb-4f31-ad15-94be5183f2c5 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:15.122137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6bfc92-e29d-4ed4-8d4d-c258adeebc96 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:15.126587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333f954-82d5-41bd-8ab3-7099e3b5f9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:15.127261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333f954-82d5-41bd-8ab3-7099e3b5f9fd None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:15.127548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333f954-82d5-41bd-8ab3-7099e3b5f9fd None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:15.128287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5333f954-82d5-41bd-8ab3-7099e3b5f9fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:15.147356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:15.161686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6654f556-890f-4f8a-9a21-c73bb9f6dd43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:15.162774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654f556-890f-4f8a-9a21-c73bb9f6dd43 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:15.163061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654f556-890f-4f8a-9a21-c73bb9f6dd43 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:15.163768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6654f556-890f-4f8a-9a21-c73bb9f6dd43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:15.171059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654f556-890f-4f8a-9a21-c73bb9f6dd43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:15.172719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6654f556-890f-4f8a-9a21-c73bb9f6dd43 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:15.187591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-6654f556-890f-4f8a-9a21-c73bb9f6dd43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:15.187591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:15.187591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:15.187591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:15.187591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:15.187591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:15.187591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:15.187591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:15.187591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:15.187591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:15.187591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:15.187591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:15.187591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:15.187591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:15.187591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:15.188559 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:15.188559 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:15.188559 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:15.188559 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:15.188559 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:15.188559 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:15.188559 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:15.188559 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:15.188559 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:15.188559 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:15.188559 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:15.188559 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:15.188559 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:15.188559 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:15.188559 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:15.188559 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:15.188559 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:15.189482 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:15.189482 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:15.189482 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:15.189482 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:15.189482 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:15.189482 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:15.189482 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:15.189482 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:15.189482 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 342/663] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:15.190053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed75e0-5db0-4ce9-97b2-0e18b0ae16f5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:15.190286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed75e0-5db0-4ce9-97b2-0e18b0ae16f5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:15.190566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ed75e0-5db0-4ce9-97b2-0e18b0ae16f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:15.211942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-18ed75e0-5db0-4ce9-97b2-0e18b0ae16f5 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:15.213316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-18ed75e0-5db0-4ce9-97b2-0e18b0ae16f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:15.218512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333f954-82d5-41bd-8ab3-7099e3b5f9fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:15.219988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5333f954-82d5-41bd-8ab3-7099e3b5f9fd None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:15.294026 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 343/664] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:42:15.294026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed75e0-5db0-4ce9-97b2-0e18b0ae16f5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:15.307948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:15.337527 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 322/665] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:42:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:42:15.339347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333f954-82d5-41bd-8ab3-7099e3b5f9fd None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:15.386116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:15.401748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ed75e0-5db0-4ce9-97b2-0e18b0ae16f5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:15.438008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c686d9-9410-4df3-b242-48ec35353492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:15.438675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c686d9-9410-4df3-b242-48ec35353492 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:15.438980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c686d9-9410-4df3-b242-48ec35353492 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:15.439446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c686d9-9410-4df3-b242-48ec35353492 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:15.450891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5333f954-82d5-41bd-8ab3-7099e3b5f9fd None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:15.486272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac213c8-ff4f-448c-9104-fd7126fd564d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:15.486928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac213c8-ff4f-448c-9104-fd7126fd564d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:15.487208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac213c8-ff4f-448c-9104-fd7126fd564d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:15.487482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac213c8-ff4f-448c-9104-fd7126fd564d None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:15.488881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac213c8-ff4f-448c-9104-fd7126fd564d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:15.489206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac213c8-ff4f-448c-9104-fd7126fd564d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:15.490669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac213c8-ff4f-448c-9104-fd7126fd564d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:15.502598 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 323/666] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:15.503404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac213c8-ff4f-448c-9104-fd7126fd564d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:15.526931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c686d9-9410-4df3-b242-48ec35353492 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:15.528824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c686d9-9410-4df3-b242-48ec35353492 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:15.543264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:15.612087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac213c8-ff4f-448c-9104-fd7126fd564d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:15.614739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:15.638351 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 344/667] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:42:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:42:15.638351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c686d9-9410-4df3-b242-48ec35353492 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:15.656247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71e627-cdeb-4606-9059-97af1f3fc02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:15.657039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71e627-cdeb-4606-9059-97af1f3fc02a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:15.657320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71e627-cdeb-4606-9059-97af1f3fc02a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:15.657592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71e627-cdeb-4606-9059-97af1f3fc02a None admin] PATH_INFO: `/v3/roles/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:15.669704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71e627-cdeb-4606-9059-97af1f3fc02a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:15.677776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-bc71e627-cdeb-4606-9059-97af1f3fc02a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:15.677776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:15.677776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:15.677776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:15.677776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:15.677776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:15.677776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:15.677776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:15.677776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:15.677776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:15.677776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:15.677776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:15.677776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:15.677776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:15.677776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:15.677776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:15.685068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:15.685068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:15.685068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:15.685068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:15.685068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:15.685068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:15.685068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:15.685068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:15.685068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:15.685068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:15.685068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:15.685068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:15.685068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:15.685068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:15.685068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:15.685068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:15.686181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:15.686181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:15.686181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:15.686181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:15.686181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:15.686181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:15.686181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:15.686181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:15.686181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:15.686181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:15.686181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:15.686181 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:15.686181 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 324/668] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:42:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:15.686181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71e627-cdeb-4606-9059-97af1f3fc02a None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:15.751038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c686d9-9410-4df3-b242-48ec35353492 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:15.762312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:15.790369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a24ed4-792b-4478-a88a-09e01cc54441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:15.791166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a24ed4-792b-4478-a88a-09e01cc54441 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:15.791449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a24ed4-792b-4478-a88a-09e01cc54441 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:15.791722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a24ed4-792b-4478-a88a-09e01cc54441 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:15.791831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71e627-cdeb-4606-9059-97af1f3fc02a None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:15.804443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a24ed4-792b-4478-a88a-09e01cc54441 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:15.805999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a24ed4-792b-4478-a88a-09e01cc54441 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:15.827326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-e8a24ed4-792b-4478-a88a-09e01cc54441 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:15.827326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:15.827326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:15.827326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:15.827326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:15.827326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:15.827326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:15.827326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:15.827326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:15.827326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:15.827326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:15.827326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:15.827326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:15.827326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:15.827326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:15.828277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:15.828277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:15.828277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:15.828277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:15.828277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:15.828277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:15.828277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:15.828277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:15.828277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:15.828277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:15.828277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:15.828277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:15.828277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:15.828277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:15.828277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:15.828277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:15.828277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:15.829164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:15.829164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:15.829164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:15.829164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:15.829164 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 345/669] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:15.829446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a24ed4-792b-4478-a88a-09e01cc54441 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:15.836801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f226e5-848f-417a-b2d7-2a7c4f9d2dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:15.837470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f226e5-848f-417a-b2d7-2a7c4f9d2dd7 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:15.837754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f226e5-848f-417a-b2d7-2a7c4f9d2dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:15.838086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f226e5-848f-417a-b2d7-2a7c4f9d2dd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:15.838934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f226e5-848f-417a-b2d7-2a7c4f9d2dd7 None admin] RBAC: Adding query filter params (role.id=4db40e559d064cf6b4f061e785b42dbb, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=224c1a0427c942e5bfef2bac75648d80) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:15.839254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f226e5-848f-417a-b2d7-2a7c4f9d2dd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:15.840709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f226e5-848f-417a-b2d7-2a7c4f9d2dd7 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:15.847171 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 325/670] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:15 2022] GET /identity/v3/role_assignments?user.id=224c1a0427c942e5bfef2bac75648d80&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=4db40e559d064cf6b4f061e785b42dbb => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:15.848513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f226e5-848f-417a-b2d7-2a7c4f9d2dd7 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:15.886749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:15.971369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a24ed4-792b-4478-a88a-09e01cc54441 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:15.987114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f226e5-848f-417a-b2d7-2a7c4f9d2dd7 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:16.013890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc3c5d-7818-4c78-ae1f-88209aeb8c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:16.014695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc3c5d-7818-4c78-ae1f-88209aeb8c34 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:16.014979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc3c5d-7818-4c78-ae1f-88209aeb8c34 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:16.015256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dc3c5d-7818-4c78-ae1f-88209aeb8c34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:16.031554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc3c5d-7818-4c78-ae1f-88209aeb8c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:16.033720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dc3c5d-7818-4c78-ae1f-88209aeb8c34 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:16.033973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf8212-69c6-4493-ab4f-9fe12d498e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:16.034732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf8212-69c6-4493-ab4f-9fe12d498e96 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:16.035085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf8212-69c6-4493-ab4f-9fe12d498e96 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:16.035472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf8212-69c6-4493-ab4f-9fe12d498e96 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:16.036313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf8212-69c6-4493-ab4f-9fe12d498e96 None admin] RBAC: Adding query filter params (name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:16.036692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf8212-69c6-4493-ab4f-9fe12d498e96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:16.038335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf8212-69c6-4493-ab4f-9fe12d498e96 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:16.051811 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 326/671] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:42:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:16.051811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf8212-69c6-4493-ab4f-9fe12d498e96 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:16.060812 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-d5dc3c5d-7818-4c78-ae1f-88209aeb8c34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:16.060812 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:16.060812 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:16.060812 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:16.060812 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:16.060812 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:16.060812 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:16.060812 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:16.060812 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:16.060812 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:16.060812 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:16.060812 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:16.060812 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:16.060812 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:16.060812 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:16.062042 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:16.062042 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:16.062042 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:16.062042 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:16.062042 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:16.062042 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:16.062042 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:16.062042 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:16.062042 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:16.062042 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:16.062042 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:16.062042 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:16.062042 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:16.062042 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:16.062042 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:16.062042 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:16.062042 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:16.063213 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:16.063213 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:16.063213 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:16.063213 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:16.063979 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 346/672] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:16.064870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc3c5d-7818-4c78-ae1f-88209aeb8c34 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:16.084167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:16.167841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:16.184546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf8212-69c6-4493-ab4f-9fe12d498e96 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:16.201576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dc3c5d-7818-4c78-ae1f-88209aeb8c34 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:16.231213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17917bdb-a810-405d-b884-fcf2fbb85c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:16.232159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17917bdb-a810-405d-b884-fcf2fbb85c0d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:16.232448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17917bdb-a810-405d-b884-fcf2fbb85c0d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:16.232729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17917bdb-a810-405d-b884-fcf2fbb85c0d None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:16.233746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17917bdb-a810-405d-b884-fcf2fbb85c0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:16.234237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17917bdb-a810-405d-b884-fcf2fbb85c0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:16.235623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17917bdb-a810-405d-b884-fcf2fbb85c0d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:16.246379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e86c35-98b3-48a5-a014-fb7a13cf2145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:16.247730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e86c35-98b3-48a5-a014-fb7a13cf2145 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:16.248218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e86c35-98b3-48a5-a014-fb7a13cf2145 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:16.248695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e86c35-98b3-48a5-a014-fb7a13cf2145 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:16.249126 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 327/673] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:16.254612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17917bdb-a810-405d-b884-fcf2fbb85c0d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:16.262877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e86c35-98b3-48a5-a014-fb7a13cf2145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:16.265271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e86c35-98b3-48a5-a014-fb7a13cf2145 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:16.285416 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-28e86c35-98b3-48a5-a014-fb7a13cf2145 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:16.285416 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:16.285416 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:16.285416 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:16.285416 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:16.285416 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:16.285416 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:16.285416 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:16.285416 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:16.285416 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:16.285416 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:16.285416 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:16.285416 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:16.285416 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:16.285416 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:16.286656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:16.286656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:16.286656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:16.286656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:16.286656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:16.286656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:16.286656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:16.286656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:16.286656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:16.286656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:16.286656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:16.286656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:16.286656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:16.286656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:16.286656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:16.286656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:16.286656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:16.287549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:16.287549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:16.287549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:16.287549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:16.287549 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 347/674] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:16.288167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e86c35-98b3-48a5-a014-fb7a13cf2145 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:16.358114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:16.392443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17917bdb-a810-405d-b884-fcf2fbb85c0d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:16.395754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e86c35-98b3-48a5-a014-fb7a13cf2145 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:16.434982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb062e-bb57-4cf4-b4af-addffc12b99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:16.435764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb062e-bb57-4cf4-b4af-addffc12b99a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:16.436075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb062e-bb57-4cf4-b4af-addffc12b99a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:16.436421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb062e-bb57-4cf4-b4af-addffc12b99a None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:16.437238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb062e-bb57-4cf4-b4af-addffc12b99a None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:16.437614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb062e-bb57-4cf4-b4af-addffc12b99a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:16.438676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debaae6a-daf8-4883-a310-ad1d658f6f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:16.439152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb062e-bb57-4cf4-b4af-addffc12b99a None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:16.440051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debaae6a-daf8-4883-a310-ad1d658f6f8c None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:16.440782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debaae6a-daf8-4883-a310-ad1d658f6f8c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:16.441243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debaae6a-daf8-4883-a310-ad1d658f6f8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:16.451205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debaae6a-daf8-4883-a310-ad1d658f6f8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:16.455064 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 328/675] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:16.455175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debaae6a-daf8-4883-a310-ad1d658f6f8c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:16.455272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb062e-bb57-4cf4-b4af-addffc12b99a None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:16.465296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:16.469939 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-debaae6a-daf8-4883-a310-ad1d658f6f8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:16.469939 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:16.469939 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:16.469939 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:16.469939 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:16.469939 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:16.469939 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:16.469939 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:16.469939 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:16.469939 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:16.469939 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:16.469939 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:16.469939 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:16.469939 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:16.469939 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:16.470827 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:16.470827 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:16.470827 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:16.470827 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:16.470827 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:16.470827 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:16.470827 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:16.470827 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:16.470827 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:16.470827 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:16.470827 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:16.470827 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:16.470827 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:16.470827 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:16.470827 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:16.470827 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:16.470827 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:16.471719 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:16.471719 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:16.471719 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:16.471719 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:16.471719 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:16.471719 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:16.471719 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:16.471719 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:16.476246 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 348/676] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:16.480205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debaae6a-daf8-4883-a310-ad1d658f6f8c None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:16.587334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debaae6a-daf8-4883-a310-ad1d658f6f8c None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:16.589439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb062e-bb57-4cf4-b4af-addffc12b99a None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:16.622460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:16.625499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832b737-6d78-49b2-9b00-69710699d489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:16.626309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832b737-6d78-49b2-9b00-69710699d489 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:16.626714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832b737-6d78-49b2-9b00-69710699d489 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:16.627095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a832b737-6d78-49b2-9b00-69710699d489 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:16.627926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832b737-6d78-49b2-9b00-69710699d489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:16.628347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832b737-6d78-49b2-9b00-69710699d489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:16.630407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a832b737-6d78-49b2-9b00-69710699d489 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:16.634837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91870f46-2430-44e0-aa16-da00293591a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:16.640163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91870f46-2430-44e0-aa16-da00293591a2 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:16.640163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91870f46-2430-44e0-aa16-da00293591a2 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:16.640163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91870f46-2430-44e0-aa16-da00293591a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:16.643771 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 329/677] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:16.644946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832b737-6d78-49b2-9b00-69710699d489 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:16.645824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91870f46-2430-44e0-aa16-da00293591a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:16.647481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91870f46-2430-44e0-aa16-da00293591a2 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:16.668133 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-91870f46-2430-44e0-aa16-da00293591a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:16.668133 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:16.668133 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:16.668133 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:16.668133 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:16.668133 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:16.668133 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:16.668133 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:16.668133 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:16.668133 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:16.668133 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:16.668133 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:16.668133 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:16.668133 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:16.668133 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:16.669285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:16.669285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:16.669285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:16.669285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:16.669285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:16.669285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:16.669285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:16.669285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:16.669285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:16.669285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:16.669285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:16.669285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:16.669285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:16.669285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:16.669285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:16.669285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:16.669285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:16.670406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:16.670406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:16.670406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:16.670406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:16.670406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:16.670406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:16.670406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:16.670406 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:16.673386 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 349/678] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:16.674656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91870f46-2430-44e0-aa16-da00293591a2 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:16.738164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:16.780990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a832b737-6d78-49b2-9b00-69710699d489 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:16.803951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91870f46-2430-44e0-aa16-da00293591a2 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:16.825531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5b1eb-8aa5-47c6-8662-1e77abde0cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:16.826239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5b1eb-8aa5-47c6-8662-1e77abde0cfb None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:16.826525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5b1eb-8aa5-47c6-8662-1e77abde0cfb None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:16.826804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5b1eb-8aa5-47c6-8662-1e77abde0cfb None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:16.827616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5b1eb-8aa5-47c6-8662-1e77abde0cfb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:16.827925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5b1eb-8aa5-47c6-8662-1e77abde0cfb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:16.830128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5b1eb-8aa5-47c6-8662-1e77abde0cfb None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:16.844573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dda082-0694-406d-85b3-f6b6e2ef50ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:16.845442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dda082-0694-406d-85b3-f6b6e2ef50ac None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:16.845818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dda082-0694-406d-85b3-f6b6e2ef50ac None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:16.846243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dda082-0694-406d-85b3-f6b6e2ef50ac None admin] PATH_INFO: `/v3/users/swift` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:16.846737 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 330/679] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:16.847587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5b1eb-8aa5-47c6-8662-1e77abde0cfb None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:16.860107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dda082-0694-406d-85b3-f6b6e2ef50ac None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:16.861912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dda082-0694-406d-85b3-f6b6e2ef50ac None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:16.879512 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-53dda082-0694-406d-85b3-f6b6e2ef50ac None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 20:42:16.879512 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:16.879512 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:16.879512 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:16.879512 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:16.879512 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:16.879512 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:16.879512 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:16.879512 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:16.879512 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:16.879512 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:16.879512 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:16.879512 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:16.879512 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:16.879512 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:16.879512 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:16.881250 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:16.883265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:16.883265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:16.883265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:16.883265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:16.883265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:16.883265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:16.883265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:16.883265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:16.883265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 20:42:16.883265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:16.883265 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 350/680] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:42:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:16.884511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dda082-0694-406d-85b3-f6b6e2ef50ac None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:16.949991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:16.967503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5b1eb-8aa5-47c6-8662-1e77abde0cfb None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:16.996300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:17.005743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76768e32-a1f6-4b39-aaab-81d7c02d2053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:17.006635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76768e32-a1f6-4b39-aaab-81d7c02d2053 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:17.006983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76768e32-a1f6-4b39-aaab-81d7c02d2053 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:17.007327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76768e32-a1f6-4b39-aaab-81d7c02d2053 None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:17.008240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76768e32-a1f6-4b39-aaab-81d7c02d2053 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:17.008623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76768e32-a1f6-4b39-aaab-81d7c02d2053 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:17.010228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76768e32-a1f6-4b39-aaab-81d7c02d2053 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:17.025067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dda082-0694-406d-85b3-f6b6e2ef50ac None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:17.029750 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 331/681] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:17.031629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76768e32-a1f6-4b39-aaab-81d7c02d2053 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:17.073291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea89e9-ce61-4391-b77a-ef7ccf912d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:17.074087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea89e9-ce61-4391-b77a-ef7ccf912d16 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:17.074439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea89e9-ce61-4391-b77a-ef7ccf912d16 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:17.074791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea89e9-ce61-4391-b77a-ef7ccf912d16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:17.075755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea89e9-ce61-4391-b77a-ef7ccf912d16 None admin] RBAC: Adding query filter params (role.id=4db40e559d064cf6b4f061e785b42dbb, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=1b2a9ebcd3f84b758322a80c6e590de0) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:17.076157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea89e9-ce61-4391-b77a-ef7ccf912d16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:17.077785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea89e9-ce61-4391-b77a-ef7ccf912d16 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:17.094076 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 351/682] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:16 2022] GET /identity/v3/role_assignments?user.id=1b2a9ebcd3f84b758322a80c6e590de0&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=4db40e559d064cf6b4f061e785b42dbb => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:17.095259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea89e9-ce61-4391-b77a-ef7ccf912d16 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:17.172379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76768e32-a1f6-4b39-aaab-81d7c02d2053 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:17.217679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2e259-4b06-42b5-b786-d83cfa4eb4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:17.218960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2e259-4b06-42b5-b786-d83cfa4eb4fd None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:17.219403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2e259-4b06-42b5-b786-d83cfa4eb4fd None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:17.219859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2e259-4b06-42b5-b786-d83cfa4eb4fd None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:17.221391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2e259-4b06-42b5-b786-d83cfa4eb4fd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:17.221910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2e259-4b06-42b5-b786-d83cfa4eb4fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:17.223629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2e259-4b06-42b5-b786-d83cfa4eb4fd None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:17.248126 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 332/683] 10.222.0.33 () {56 vars in 1135 bytes} [Sat May 28 20:42:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:17.253436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2e259-4b06-42b5-b786-d83cfa4eb4fd None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:17.312524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:17.327305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea89e9-ce61-4391-b77a-ef7ccf912d16 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:17.350392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:17.384245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4222b55-8921-4a45-9ada-0dd2afccbc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:17.385102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4222b55-8921-4a45-9ada-0dd2afccbc9e None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:17.385514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4222b55-8921-4a45-9ada-0dd2afccbc9e None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:17.385940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4222b55-8921-4a45-9ada-0dd2afccbc9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:17.386194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2e259-4b06-42b5-b786-d83cfa4eb4fd None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:17.387000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4222b55-8921-4a45-9ada-0dd2afccbc9e None admin] RBAC: Adding query filter params (role.id=4db40e559d064cf6b4f061e785b42dbb, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=afc08d15f24e46e29c34d618d955bd28) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:17.387419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4222b55-8921-4a45-9ada-0dd2afccbc9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:17.389002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4222b55-8921-4a45-9ada-0dd2afccbc9e None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:17.397172 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 352/684] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:17 2022] GET /identity/v3/role_assignments?user.id=afc08d15f24e46e29c34d618d955bd28&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=4db40e559d064cf6b4f061e785b42dbb => generated 608 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:17.405897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15cfef-9bc8-4a55-b80d-3d23e6859230 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:17.405897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15cfef-9bc8-4a55-b80d-3d23e6859230 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:17.405897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15cfef-9bc8-4a55-b80d-3d23e6859230 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:17.405897 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 353/685] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:17.407129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e24860-2058-44d9-a277-0c6886f81eaa None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:17.407498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e24860-2058-44d9-a277-0c6886f81eaa None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:17.407784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e24860-2058-44d9-a277-0c6886f81eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:17.426552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d4e24860-2058-44d9-a277-0c6886f81eaa None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:17.427986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d4e24860-2058-44d9-a277-0c6886f81eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:17.441089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982bbb3-550a-44d5-a8de-ffaadf2f73a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:17.441898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982bbb3-550a-44d5-a8de-ffaadf2f73a3 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:17.442279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982bbb3-550a-44d5-a8de-ffaadf2f73a3 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:17.442628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982bbb3-550a-44d5-a8de-ffaadf2f73a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:17.523270 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 354/686] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:42:17.537432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982bbb3-550a-44d5-a8de-ffaadf2f73a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:17.539821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982bbb3-550a-44d5-a8de-ffaadf2f73a3 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:17.554426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e24860-2058-44d9-a277-0c6886f81eaa None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:17.593691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:17.661296 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 333/687] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:42:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:42:17.666824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982bbb3-550a-44d5-a8de-ffaadf2f73a3 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:17.695326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e24860-2058-44d9-a277-0c6886f81eaa None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:17.699693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:17.746792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718241c6-2a9e-4d2a-bcc6-08383fa03dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:17.747632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718241c6-2a9e-4d2a-bcc6-08383fa03dcd None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:17.747924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718241c6-2a9e-4d2a-bcc6-08383fa03dcd None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:17.748202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718241c6-2a9e-4d2a-bcc6-08383fa03dcd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:17.759596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718241c6-2a9e-4d2a-bcc6-08383fa03dcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:17.767618 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-718241c6-2a9e-4d2a-bcc6-08383fa03dcd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:42:17.767618 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:17.767618 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:17.767618 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:17.767618 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:17.767618 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:17.767618 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:17.767618 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:17.767618 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:17.767618 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:17.767618 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:17.767618 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:17.767618 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:17.767618 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:17.767618 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:17.767618 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:17.768605 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:17.768605 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:17.768605 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:17.768605 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:17.768605 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:17.768605 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:17.768605 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:17.768605 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:17.768605 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:17.768605 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:17.768605 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:17.768605 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:17.768605 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:17.768605 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:17.768605 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:17.768605 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:17.769489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:17.769489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:17.769489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:17.769489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:17.769489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:17.769489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:17.769489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:17.769489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:17.769489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:17.769489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:17.769489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:42:17.769489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:17.769489 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 355/688] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:42:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:17.777414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718241c6-2a9e-4d2a-bcc6-08383fa03dcd None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:17.813474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982bbb3-550a-44d5-a8de-ffaadf2f73a3 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:17.853290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e734aca-41bd-498c-8ec6-d5f24e1d9e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:17.854688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e734aca-41bd-498c-8ec6-d5f24e1d9e7d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:17.855210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e734aca-41bd-498c-8ec6-d5f24e1d9e7d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:17.855779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e734aca-41bd-498c-8ec6-d5f24e1d9e7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:17.870028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e734aca-41bd-498c-8ec6-d5f24e1d9e7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:17.871909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e734aca-41bd-498c-8ec6-d5f24e1d9e7d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:17.881507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:17.895765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-1e734aca-41bd-498c-8ec6-d5f24e1d9e7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:17.895765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:17.895765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:17.895765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:17.895765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:17.895765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:17.895765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:17.895765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:17.895765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:17.895765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:17.895765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:17.895765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:17.895765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:17.895765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:17.895765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:17.899172 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:17.899172 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:17.899172 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:17.899172 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:17.899172 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:17.899172 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:17.899172 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:17.899172 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:17.899172 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:17.899172 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:17.899172 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:17.899172 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:17.899172 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:17.899172 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:17.899172 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:17.899172 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:17.899172 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:17.900083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:17.900083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:17.900083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:17.900083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:17.900083 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 334/689] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:42:17.904996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e734aca-41bd-498c-8ec6-d5f24e1d9e7d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:17.908505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718241c6-2a9e-4d2a-bcc6-08383fa03dcd None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:17.956361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3707c1-6379-4eb9-8def-44801fd92d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:17.957572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3707c1-6379-4eb9-8def-44801fd92d44 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:17.958130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3707c1-6379-4eb9-8def-44801fd92d44 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:17.958694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3707c1-6379-4eb9-8def-44801fd92d44 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:17.959769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3707c1-6379-4eb9-8def-44801fd92d44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:17.960296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3707c1-6379-4eb9-8def-44801fd92d44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:17.967586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3707c1-6379-4eb9-8def-44801fd92d44 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:17.982323 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 356/690] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:42:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:17.987779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3707c1-6379-4eb9-8def-44801fd92d44 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:18.030199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e734aca-41bd-498c-8ec6-d5f24e1d9e7d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:18.057194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:18.074585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd841e81-39a3-45fc-9820-6eabecabb866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:18.075651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd841e81-39a3-45fc-9820-6eabecabb866 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:18.075944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd841e81-39a3-45fc-9820-6eabecabb866 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:18.076225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd841e81-39a3-45fc-9820-6eabecabb866 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:18.077002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd841e81-39a3-45fc-9820-6eabecabb866 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:18.077313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd841e81-39a3-45fc-9820-6eabecabb866 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:18.078928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd841e81-39a3-45fc-9820-6eabecabb866 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:18.091683 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 335/691] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:18.097491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd841e81-39a3-45fc-9820-6eabecabb866 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:18.174314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:18.214317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd841e81-39a3-45fc-9820-6eabecabb866 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:18.252211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95189b5-4c83-4e8b-9a84-e396e8f85ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:18.253013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95189b5-4c83-4e8b-9a84-e396e8f85ff9 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:18.253297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95189b5-4c83-4e8b-9a84-e396e8f85ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:18.253571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95189b5-4c83-4e8b-9a84-e396e8f85ff9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:18.262337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95189b5-4c83-4e8b-9a84-e396e8f85ff9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:18.263880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95189b5-4c83-4e8b-9a84-e396e8f85ff9 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:18.280257 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-a95189b5-4c83-4e8b-9a84-e396e8f85ff9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:18.280257 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:18.280257 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:18.280257 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:18.280257 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:18.280257 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:18.280257 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:18.280257 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:18.280257 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:18.280257 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:18.280257 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:18.280257 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:18.280257 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:18.280257 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:18.280257 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:18.281286 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:18.281286 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:18.281286 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:18.281286 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:18.281286 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:18.281286 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:18.281286 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:18.281286 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:18.281286 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:18.281286 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:18.281286 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:18.281286 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:18.281286 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:18.281286 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:18.281286 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:18.281286 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:18.281286 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:18.282209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:18.282209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:18.282209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:18.282209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:18.282209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:18.282209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:18.282209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:18.282209 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:18.285923 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 336/692] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:18.288291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95189b5-4c83-4e8b-9a84-e396e8f85ff9 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:18.339060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3707c1-6379-4eb9-8def-44801fd92d44 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:18.406044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4e725-eb82-4c66-a906-6c022e863f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:18.407792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4e725-eb82-4c66-a906-6c022e863f74 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:18.408088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4e725-eb82-4c66-a906-6c022e863f74 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:18.409525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4e725-eb82-4c66-a906-6c022e863f74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:18.429602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f4e725-eb82-4c66-a906-6c022e863f74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:18.431110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f4e725-eb82-4c66-a906-6c022e863f74 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:18.446927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95189b5-4c83-4e8b-9a84-e396e8f85ff9 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:18.456854 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-74f4e725-eb82-4c66-a906-6c022e863f74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:18.456854 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:18.456854 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:18.456854 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:18.456854 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:18.456854 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:18.456854 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:18.456854 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:18.456854 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:18.456854 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:18.456854 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:18.456854 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:18.456854 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:18.456854 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:18.456854 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:18.457779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:18.457779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:18.457779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:18.457779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:18.457779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:18.457779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:18.457779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:18.457779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:18.457779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:18.457779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:18.457779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:18.457779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:18.457779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:18.457779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:18.457779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:18.457779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:18.457779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:18.458707 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:18.458707 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:18.458707 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:18.458707 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:18.458707 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 357/693] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 471 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:42:18.460307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1bea0-2b15-45af-8b67-dbfaf7001d79 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:18.460596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1bea0-2b15-45af-8b67-dbfaf7001d79 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:18.460877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1bea0-2b15-45af-8b67-dbfaf7001d79 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:18.465967 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 358/694] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:18.466998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1bea0-2b15-45af-8b67-dbfaf7001d79 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:18.474690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:18.495100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf81ad6-201d-4fa2-a181-fae8cacf5690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:18.495871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf81ad6-201d-4fa2-a181-fae8cacf5690 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:18.496160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf81ad6-201d-4fa2-a181-fae8cacf5690 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:18.496436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf81ad6-201d-4fa2-a181-fae8cacf5690 None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:18.497366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf81ad6-201d-4fa2-a181-fae8cacf5690 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:18.497687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf81ad6-201d-4fa2-a181-fae8cacf5690 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:18.499754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf81ad6-201d-4fa2-a181-fae8cacf5690 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:18.521496 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 337/695] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:18.521496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc9e73-fa92-4593-9d8a-e8a944fe32ad None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:18.521496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc9e73-fa92-4593-9d8a-e8a944fe32ad None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:18.521496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc9e73-fa92-4593-9d8a-e8a944fe32ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:18.538937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bccc9e73-fa92-4593-9d8a-e8a944fe32ad None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:18.540245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bccc9e73-fa92-4593-9d8a-e8a944fe32ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:18.570089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:18.595142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1bea0-2b15-45af-8b67-dbfaf7001d79 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:18.627192 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 338/696] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:42:18.628377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc9e73-fa92-4593-9d8a-e8a944fe32ad None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:18.657222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276a249-3358-44b5-a943-0df013a32de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:18.658385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276a249-3358-44b5-a943-0df013a32de2 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:18.658877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276a249-3358-44b5-a943-0df013a32de2 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:18.659360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276a249-3358-44b5-a943-0df013a32de2 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:18.660948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276a249-3358-44b5-a943-0df013a32de2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:18.662320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276a249-3358-44b5-a943-0df013a32de2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:18.664525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276a249-3358-44b5-a943-0df013a32de2 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:18.681464 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 359/697] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:18.682668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276a249-3358-44b5-a943-0df013a32de2 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:18.757913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc9e73-fa92-4593-9d8a-e8a944fe32ad None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:18.768715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:18.795827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a8c7f-ad60-4f74-99f6-d2cda0888523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:18.796871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a8c7f-ad60-4f74-99f6-d2cda0888523 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:18.797358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a8c7f-ad60-4f74-99f6-d2cda0888523 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:18.798099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a8c7f-ad60-4f74-99f6-d2cda0888523 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:18.799479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a8c7f-ad60-4f74-99f6-d2cda0888523 None admin] RBAC: Adding query filter params (role.id=4db40e559d064cf6b4f061e785b42dbb, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=a01bba888a644eb18fc457aff578aa4b) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:18.800044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a8c7f-ad60-4f74-99f6-d2cda0888523 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:18.801795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a8c7f-ad60-4f74-99f6-d2cda0888523 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:18.813976 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 339/698] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:18 2022] GET /identity/v3/role_assignments?user.id=a01bba888a644eb18fc457aff578aa4b&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=4db40e559d064cf6b4f061e785b42dbb => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:18.815361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a8c7f-ad60-4f74-99f6-d2cda0888523 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:18.819097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276a249-3358-44b5-a943-0df013a32de2 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:18.861011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf13fc-9623-41ae-9820-6ad371bcc4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:18.861011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf13fc-9623-41ae-9820-6ad371bcc4e9 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:18.861011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf13fc-9623-41ae-9820-6ad371bcc4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:18.861011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf13fc-9623-41ae-9820-6ad371bcc4e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:18.870831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:18.878987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf13fc-9623-41ae-9820-6ad371bcc4e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:18.880508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf13fc-9623-41ae-9820-6ad371bcc4e9 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:18.916834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-2fdf13fc-9623-41ae-9820-6ad371bcc4e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:18.916834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:18.916834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:18.916834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:18.916834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:18.916834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:18.916834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:18.916834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:18.916834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:18.916834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:18.916834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:18.916834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:18.916834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:18.916834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:18.916834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:18.917828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:18.917828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:18.917828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:18.917828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:18.917828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:18.917828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:18.917828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:18.917828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:18.917828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:18.917828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:18.917828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:18.917828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:18.917828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:18.917828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:18.917828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:18.917828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:18.917828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:18.918787 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:18.918787 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:18.918787 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:18.918787 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:18.926663 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 360/699] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:18.933890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf13fc-9623-41ae-9820-6ad371bcc4e9 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:19.049861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a8c7f-ad60-4f74-99f6-d2cda0888523 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:19.054405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf13fc-9623-41ae-9820-6ad371bcc4e9 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:19.094700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c35ec-191e-404d-8f4b-5d6c03443809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:19.096126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c35ec-191e-404d-8f4b-5d6c03443809 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:19.097020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c35ec-191e-404d-8f4b-5d6c03443809 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:19.097805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901c35ec-191e-404d-8f4b-5d6c03443809 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:19.107139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5087562-5d5d-441f-b9ae-085cb6711301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:19.107446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5087562-5d5d-441f-b9ae-085cb6711301 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:19.107710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5087562-5d5d-441f-b9ae-085cb6711301 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:19.107992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5087562-5d5d-441f-b9ae-085cb6711301 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:19.108730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5087562-5d5d-441f-b9ae-085cb6711301 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:19.109036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5087562-5d5d-441f-b9ae-085cb6711301 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:19.111371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c35ec-191e-404d-8f4b-5d6c03443809 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:19.111729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5087562-5d5d-441f-b9ae-085cb6711301 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:19.115309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901c35ec-191e-404d-8f4b-5d6c03443809 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:19.133358 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 361/700] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:19.135213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5087562-5d5d-441f-b9ae-085cb6711301 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:19.140287 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-901c35ec-191e-404d-8f4b-5d6c03443809 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 20:42:19.140287 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:19.140287 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:19.140287 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:19.140287 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:19.140287 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:19.140287 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:19.140287 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:19.140287 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:19.140287 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:19.140287 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:19.140287 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:19.140287 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:19.140287 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:19.140287 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:19.140287 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:19.141472 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:19.142802 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:19.142802 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:19.142802 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:19.142802 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:19.142802 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:19.142802 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:19.142802 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:19.142802 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:19.142802 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 20:42:19.142802 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:19.146271 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 340/701] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 20:42:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:19.152032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c35ec-191e-404d-8f4b-5d6c03443809 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:19.193688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:19.194981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:19.249443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5087562-5d5d-441f-b9ae-085cb6711301 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:19.285444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901c35ec-191e-404d-8f4b-5d6c03443809 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:19.297502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec1372-e729-40db-b0e5-fe39d279fd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:19.298549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec1372-e729-40db-b0e5-fe39d279fd37 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:19.299329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec1372-e729-40db-b0e5-fe39d279fd37 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:19.300134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec1372-e729-40db-b0e5-fe39d279fd37 None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:19.301097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec1372-e729-40db-b0e5-fe39d279fd37 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:19.303635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec1372-e729-40db-b0e5-fe39d279fd37 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:19.327663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3268b1-4744-4968-b7ee-6e653a103f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:19.328343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3268b1-4744-4968-b7ee-6e653a103f9e None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:19.330760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3268b1-4744-4968-b7ee-6e653a103f9e None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:19.330760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3268b1-4744-4968-b7ee-6e653a103f9e None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:19.330760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3268b1-4744-4968-b7ee-6e653a103f9e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:19.330760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3268b1-4744-4968-b7ee-6e653a103f9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:19.331452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3268b1-4744-4968-b7ee-6e653a103f9e None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:19.352745 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 341/702] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 20:42:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:19.357734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3268b1-4744-4968-b7ee-6e653a103f9e None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:19.393455 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 362/703] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:42:19 2022] POST /identity/v3/users => generated 254 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 20:42:19.457770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:19.468819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3268b1-4744-4968-b7ee-6e653a103f9e None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:19.507520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dbeb3-6d3a-4abf-9665-533c88c749db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:19.508187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dbeb3-6d3a-4abf-9665-533c88c749db None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:19.508479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dbeb3-6d3a-4abf-9665-533c88c749db None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:19.508761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74dbeb3-6d3a-4abf-9665-533c88c749db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:19.597954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74dbeb3-6d3a-4abf-9665-533c88c749db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:19.599410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74dbeb3-6d3a-4abf-9665-533c88c749db None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:19.716265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:19.728035 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 342/704] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:42:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 20:42:19.733896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec1372-e729-40db-b0e5-fe39d279fd37 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:19.875996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec1372-e729-40db-b0e5-fe39d279fd37 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:19.883353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:19.914150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a60a32-3b07-43cd-90cd-27f339b2a3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:19.914958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a60a32-3b07-43cd-90cd-27f339b2a3ed None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:19.915245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a60a32-3b07-43cd-90cd-27f339b2a3ed None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:19.915525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a60a32-3b07-43cd-90cd-27f339b2a3ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:19.928476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a60a32-3b07-43cd-90cd-27f339b2a3ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:19.930035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a60a32-3b07-43cd-90cd-27f339b2a3ed None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:19.949549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-f1a60a32-3b07-43cd-90cd-27f339b2a3ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:19.949549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:19.949549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:19.949549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:19.949549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:19.949549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:19.949549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:19.949549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:19.949549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:19.949549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:19.949549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:19.949549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:19.949549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:19.949549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:19.949549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:19.950530 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:19.950530 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:19.950530 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:19.950530 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:19.950530 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:19.950530 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:19.950530 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:19.950530 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:19.950530 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:19.950530 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:19.950530 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:19.950530 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:19.950530 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:19.950530 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:19.950530 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:19.950530 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:19.950530 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:19.951417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:19.951417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:19.951417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:19.951417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:19.951417 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 363/705] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:19.960214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dbeb3-6d3a-4abf-9665-533c88c749db None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:20.018350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac625eee-dc89-4e69-9cb0-481a4470d857 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:20.018677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac625eee-dc89-4e69-9cb0-481a4470d857 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:20.018966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac625eee-dc89-4e69-9cb0-481a4470d857 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:20.020806 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 364/706] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:20.025826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066993c4-2336-461b-9ce3-11da90430e5a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:20.026146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066993c4-2336-461b-9ce3-11da90430e5a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:20.026431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066993c4-2336-461b-9ce3-11da90430e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:20.055163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-066993c4-2336-461b-9ce3-11da90430e5a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:20.056541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-066993c4-2336-461b-9ce3-11da90430e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:20.089473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74dbeb3-6d3a-4abf-9665-533c88c749db None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:20.127991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a10b30-191a-4405-8f71-e1d357ddf318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:20.128584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a10b30-191a-4405-8f71-e1d357ddf318 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:20.128925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a10b30-191a-4405-8f71-e1d357ddf318 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:20.129444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a10b30-191a-4405-8f71-e1d357ddf318 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:20.130051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a10b30-191a-4405-8f71-e1d357ddf318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:20.130362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a10b30-191a-4405-8f71-e1d357ddf318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:20.131750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a10b30-191a-4405-8f71-e1d357ddf318 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:20.138013 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 365/707] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:42:20.149655 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 343/708] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:20.150910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066993c4-2336-461b-9ce3-11da90430e5a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:20.176015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a10b30-191a-4405-8f71-e1d357ddf318 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:20.271312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:20.282976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:20.306336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066993c4-2336-461b-9ce3-11da90430e5a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:20.329953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a10b30-191a-4405-8f71-e1d357ddf318 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:20.349069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cab9242-f642-467a-a672-3d3ded927f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:20.350268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab9242-f642-467a-a672-3d3ded927f77 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:20.350555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab9242-f642-467a-a672-3d3ded927f77 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:20.350831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cab9242-f642-467a-a672-3d3ded927f77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:20.358900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab9242-f642-467a-a672-3d3ded927f77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:20.360820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cab9242-f642-467a-a672-3d3ded927f77 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:20.372069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3768fe-acdf-4678-a412-fc65b409e2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:20.373111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3768fe-acdf-4678-a412-fc65b409e2b0 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:20.373593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3768fe-acdf-4678-a412-fc65b409e2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:20.374119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3768fe-acdf-4678-a412-fc65b409e2b0 None admin] PATH_INFO: `/v3/services/network` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:20.374971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3768fe-acdf-4678-a412-fc65b409e2b0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:20.376245 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-7cab9242-f642-467a-a672-3d3ded927f77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:20.376245 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:20.376245 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:20.376245 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:20.376245 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:20.376245 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:20.376245 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:20.376245 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:20.376245 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:20.376245 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:20.376245 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:20.376245 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:20.376245 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:20.376245 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:20.376245 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:20.377118 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:20.377118 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:20.377118 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:20.377118 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:20.377118 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:20.377118 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:20.377118 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:20.377118 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:20.377118 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:20.377118 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:20.377118 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:20.377118 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:20.377118 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:20.377118 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:20.377118 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:20.377118 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:20.377118 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:20.378112 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:20.378112 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:20.378112 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:20.378112 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:20.378112 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:20.378112 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:20.378112 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:20.378112 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:20.378112 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 366/709] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:20.379161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d64f4-b747-4548-aafa-b21c5ae597e7 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:20.379448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d64f4-b747-4548-aafa-b21c5ae597e7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:20.379726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d64f4-b747-4548-aafa-b21c5ae597e7 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:20.380073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3768fe-acdf-4678-a412-fc65b409e2b0 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:20.385251 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 367/710] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:20.386012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d64f4-b747-4548-aafa-b21c5ae597e7 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:20.403548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-8c3768fe-acdf-4678-a412-fc65b409e2b0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 28 20:42:20.403548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:20.403548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:20.403548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:20.403548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:20.403548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:20.403548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:20.403548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:20.403548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 20:42:20.403548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:20.403548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:20.403548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:20.403548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:20.403548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:20.403548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:20.403548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:20.403548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:20.404795 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:20.404795 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:20.404795 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:20.404795 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:20.404795 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:20.404795 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:20.404795 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:20.404795 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:20.404795 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:20.404795 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:20.404795 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:20.404795 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:20.404795 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:20.404795 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:20.404795 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:20.404795 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:20.405907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:20.405907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:20.405907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:20.405907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:20.405907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:20.405907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:20.405907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:20.405907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:20.405907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:20.405907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:20.405907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:20.405907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:20.405907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 20:42:20.405907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:20.409210 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 344/711] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:20 2022] GET /identity/v3/services/network => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:42:20.411462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7e039-0675-43c1-9221-e29fc5c2b0c1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:20.411910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7e039-0675-43c1-9221-e29fc5c2b0c1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:20.412741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7e039-0675-43c1-9221-e29fc5c2b0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:20.434764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5ff7e039-0675-43c1-9221-e29fc5c2b0c1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:20.436876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5ff7e039-0675-43c1-9221-e29fc5c2b0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:20.553650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d64f4-b747-4548-aafa-b21c5ae597e7 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:20.556693 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 345/712] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:42:20.557924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7e039-0675-43c1-9221-e29fc5c2b0c1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:20.591807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729df2bc-66ee-4ddd-a490-331b8a8d405d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:20.592564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729df2bc-66ee-4ddd-a490-331b8a8d405d None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:20.592851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729df2bc-66ee-4ddd-a490-331b8a8d405d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:20.593177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729df2bc-66ee-4ddd-a490-331b8a8d405d None admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:20.594014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729df2bc-66ee-4ddd-a490-331b8a8d405d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:20.594327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729df2bc-66ee-4ddd-a490-331b8a8d405d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:20.595772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729df2bc-66ee-4ddd-a490-331b8a8d405d None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:20.607826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:20.608723 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 368/713] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:20.609748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729df2bc-66ee-4ddd-a490-331b8a8d405d None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:20.620458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:20.666685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7e039-0675-43c1-9221-e29fc5c2b0c1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:20.703597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a38fe-4e31-469e-9892-06448b9384e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:20.705131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a38fe-4e31-469e-9892-06448b9384e6 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:20.705131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a38fe-4e31-469e-9892-06448b9384e6 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:20.705131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a38fe-4e31-469e-9892-06448b9384e6 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:20.706228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a38fe-4e31-469e-9892-06448b9384e6 None admin] RBAC: Adding query filter params (name=network) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:20.706692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a38fe-4e31-469e-9892-06448b9384e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:20.708753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a38fe-4e31-469e-9892-06448b9384e6 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:20.717930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729df2bc-66ee-4ddd-a490-331b8a8d405d None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:20.718141 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 346/714] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 20:42:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:20.719153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a38fe-4e31-469e-9892-06448b9384e6 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:20.754930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a6f3c-4b3a-4a92-baa3-a1fce363a326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:20.755953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a6f3c-4b3a-4a92-baa3-a1fce363a326 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:20.756230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a6f3c-4b3a-4a92-baa3-a1fce363a326 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:20.756519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a6f3c-4b3a-4a92-baa3-a1fce363a326 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:20.757861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a6f3c-4b3a-4a92-baa3-a1fce363a326 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:20.758800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a6f3c-4b3a-4a92-baa3-a1fce363a326 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:20.774991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-e23a6f3c-4b3a-4a92-baa3-a1fce363a326 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 20:42:20.774991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:20.774991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:20.774991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:20.774991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:20.774991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:20.774991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:20.774991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:20.774991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 20:42:20.774991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:20.774991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:20.774991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:20.774991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:20.774991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:20.774991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:20.774991 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:20.778722 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:20.778722 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:20.778722 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:20.778722 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:20.778722 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:20.778722 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:20.778722 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:20.778722 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:20.778722 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:20.778722 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:20.778722 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:20.778722 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:20.778722 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:20.778722 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:20.778722 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:20.778722 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:20.779565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:20.779565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:20.779565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:20.779565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:20.779565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:20.779565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:20.779565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:20.779565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:20.779565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:20.779565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:20.779565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:20.779565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:20.779565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:20.779565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 20:42:20.779565 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:20.779565 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 369/715] 10.222.0.33 () {56 vars in 1129 bytes} [Sat May 28 20:42:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:20.780410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a6f3c-4b3a-4a92-baa3-a1fce363a326 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:20.836311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a38fe-4e31-469e-9892-06448b9384e6 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:20.846370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:20.856654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:20.873814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e79501-0c08-4317-87da-f7d5fab00a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:20.874584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e79501-0c08-4317-87da-f7d5fab00a1c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:20.874924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e79501-0c08-4317-87da-f7d5fab00a1c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:20.875267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e79501-0c08-4317-87da-f7d5fab00a1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:20.876171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e79501-0c08-4317-87da-f7d5fab00a1c None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=cfdebb5e6b8d4e2d86f0a03a30c21671) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:20.876554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e79501-0c08-4317-87da-f7d5fab00a1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:20.878092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e79501-0c08-4317-87da-f7d5fab00a1c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:20.884683 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 347/716] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:20 2022] GET /identity/v3/role_assignments?user.id=cfdebb5e6b8d4e2d86f0a03a30c21671&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=7805b93b32f1472f8365936d4d57d49a => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:20.885620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e79501-0c08-4317-87da-f7d5fab00a1c None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:20.892115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a6f3c-4b3a-4a92-baa3-a1fce363a326 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:20.942411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b70ef-1358-4887-be7f-af832f065beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:20.943889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b70ef-1358-4887-be7f-af832f065beb None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:20.944623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b70ef-1358-4887-be7f-af832f065beb None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:20.945055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b70ef-1358-4887-be7f-af832f065beb None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:20.945987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b70ef-1358-4887-be7f-af832f065beb None admin] RBAC: Adding query filter params (type=network) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:20.946699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b70ef-1358-4887-be7f-af832f065beb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:20.949181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744b70ef-1358-4887-be7f-af832f065beb None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:20.958118 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 370/717] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 20:42:20 2022] GET /identity/v3/services?type=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:21.016151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e79501-0c08-4317-87da-f7d5fab00a1c None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:21.072725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d87a8a-8a0e-429a-9a27-aa3c0841270e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:21.073681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d87a8a-8a0e-429a-9a27-aa3c0841270e None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:21.074169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d87a8a-8a0e-429a-9a27-aa3c0841270e None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:21.074591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d87a8a-8a0e-429a-9a27-aa3c0841270e None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:21.075474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d87a8a-8a0e-429a-9a27-aa3c0841270e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:21.075959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d87a8a-8a0e-429a-9a27-aa3c0841270e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:21.077537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d87a8a-8a0e-429a-9a27-aa3c0841270e None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:21.083527 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 348/718] 10.222.0.33 () {56 vars in 1124 bytes} [Sat May 28 20:42:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:21.088928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b70ef-1358-4887-be7f-af832f065beb None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:21.232931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744b70ef-1358-4887-be7f-af832f065beb None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:21.279160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a06cf2b-e874-47c5-8cd4-9cfa7dae5e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:21.279848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06cf2b-e874-47c5-8cd4-9cfa7dae5e2c None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:21.280135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06cf2b-e874-47c5-8cd4-9cfa7dae5e2c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:21.280414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a06cf2b-e874-47c5-8cd4-9cfa7dae5e2c None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:21.281143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06cf2b-e874-47c5-8cd4-9cfa7dae5e2c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:21.281451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06cf2b-e874-47c5-8cd4-9cfa7dae5e2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:21.282864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a06cf2b-e874-47c5-8cd4-9cfa7dae5e2c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:21.285920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:21.289367 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 371/719] 10.222.0.33 () {56 vars in 1124 bytes} [Sat May 28 20:42:21 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:21.992851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e76c48-ea73-4919-a6d4-8ea9152c4915 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:21.993370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e76c48-ea73-4919-a6d4-8ea9152c4915 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:21.993370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e76c48-ea73-4919-a6d4-8ea9152c4915 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:21.995050 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 349/720] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:22.000319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8884a50-0d04-4e6a-824d-1a210ca13ca6 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:22.000720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8884a50-0d04-4e6a-824d-1a210ca13ca6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:22.000906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8884a50-0d04-4e6a-824d-1a210ca13ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:22.019365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c8884a50-0d04-4e6a-824d-1a210ca13ca6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:22.021125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c8884a50-0d04-4e6a-824d-1a210ca13ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:22.096236 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 372/721] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:42:22.121610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e76c48-ea73-4919-a6d4-8ea9152c4915 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:22.177850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:22.266619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e76c48-ea73-4919-a6d4-8ea9152c4915 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:22.309935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9eb19-c0fb-4f4f-b9e6-93fb1c2dc2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:22.310737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9eb19-c0fb-4f4f-b9e6-93fb1c2dc2a8 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:22.311018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9eb19-c0fb-4f4f-b9e6-93fb1c2dc2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:22.311291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d9eb19-c0fb-4f4f-b9e6-93fb1c2dc2a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:22.328271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d9eb19-c0fb-4f4f-b9e6-93fb1c2dc2a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:22.337149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-08d9eb19-c0fb-4f4f-b9e6-93fb1c2dc2a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:42:22.337149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:22.337149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:22.337149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:22.337149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:22.337149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:22.337149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:22.337149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:22.337149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:22.337149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:22.337149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:22.337149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:22.337149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:22.337149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:22.337149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:22.337149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:22.338216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:22.338216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:22.338216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:22.338216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:22.338216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:22.338216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:22.338216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:22.338216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:22.338216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:22.338216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:22.338216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:22.338216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:22.338216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:22.338216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:22.338216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:22.338216 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:22.339068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:22.339068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:22.339068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:22.339068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:22.339068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:22.339068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:22.339068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:22.339068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:22.339068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:22.339068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:22.339068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:42:22.339068 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:22.339068 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 350/722] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:42:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:22.344936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8884a50-0d04-4e6a-824d-1a210ca13ca6 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:22.461624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8884a50-0d04-4e6a-824d-1a210ca13ca6 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:22.498446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:22.505336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151ae549-3c7a-42df-b483-0ecfcf27310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:22.506182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ae549-3c7a-42df-b483-0ecfcf27310c None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:22.506510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ae549-3c7a-42df-b483-0ecfcf27310c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:22.506791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151ae549-3c7a-42df-b483-0ecfcf27310c None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:22.507555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ae549-3c7a-42df-b483-0ecfcf27310c None admin] RBAC: Adding query filter params (name=admin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:22.507863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ae549-3c7a-42df-b483-0ecfcf27310c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:22.509242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151ae549-3c7a-42df-b483-0ecfcf27310c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:22.518509 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 373/723] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:42:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:22.524145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9eb19-c0fb-4f4f-b9e6-93fb1c2dc2a8 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:22.654985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:22.661591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d9eb19-c0fb-4f4f-b9e6-93fb1c2dc2a8 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:22.708412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e12fde-9aec-4e15-8a08-c845e823d15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:22.708412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e12fde-9aec-4e15-8a08-c845e823d15d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:22.708412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e12fde-9aec-4e15-8a08-c845e823d15d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:22.708412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e12fde-9aec-4e15-8a08-c845e823d15d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:22.714090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a2425-9304-48f8-ac6d-9f90ede6e6e8 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:22.714386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a2425-9304-48f8-ac6d-9f90ede6e6e8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:22.714670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a2425-9304-48f8-ac6d-9f90ede6e6e8 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:22.716274 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 374/724] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:22.721398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad690d-8648-400e-ae06-4ca27024a7b9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:22.721692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad690d-8648-400e-ae06-4ca27024a7b9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:22.722008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad690d-8648-400e-ae06-4ca27024a7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:22.724479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e12fde-9aec-4e15-8a08-c845e823d15d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:22.726519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e12fde-9aec-4e15-8a08-c845e823d15d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:22.740792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a9ad690d-8648-400e-ae06-4ca27024a7b9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:22.742112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a9ad690d-8648-400e-ae06-4ca27024a7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:22.750137 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-34e12fde-9aec-4e15-8a08-c845e823d15d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:22.750137 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:22.750137 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:22.750137 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:22.750137 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:22.750137 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:22.750137 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:22.750137 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:22.750137 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:22.750137 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:22.750137 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:22.750137 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:22.750137 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:22.750137 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:22.750137 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:22.751022 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:22.751022 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:22.751022 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:22.751022 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:22.751022 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:22.751022 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:22.751022 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:22.751022 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:22.751022 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:22.751022 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:22.751022 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:22.751022 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:22.751022 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:22.751022 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:22.751022 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:22.751022 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:22.751022 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:22.751898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:22.751898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:22.751898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:22.751898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:22.754119 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 351/725] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:22.760977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e12fde-9aec-4e15-8a08-c845e823d15d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:22.852475 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 375/726] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 20:42:22.891488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad690d-8648-400e-ae06-4ca27024a7b9 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:22.939578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e12fde-9aec-4e15-8a08-c845e823d15d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:23.010541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07135331-2985-4f62-a54e-f5582c58d344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:23.012027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07135331-2985-4f62-a54e-f5582c58d344 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:23.012572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07135331-2985-4f62-a54e-f5582c58d344 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:23.013066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07135331-2985-4f62-a54e-f5582c58d344 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:23.014390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07135331-2985-4f62-a54e-f5582c58d344 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:23.016193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07135331-2985-4f62-a54e-f5582c58d344 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:23.018073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07135331-2985-4f62-a54e-f5582c58d344 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:23.041654 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 352/727] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:23.050881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07135331-2985-4f62-a54e-f5582c58d344 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:23.067989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:23.091575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad690d-8648-400e-ae06-4ca27024a7b9 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:23.133791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:23.163119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc1dd5-2cd0-4dae-a537-278f548d88cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:23.164578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc1dd5-2cd0-4dae-a537-278f548d88cd None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:23.165110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc1dd5-2cd0-4dae-a537-278f548d88cd None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:23.165617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc1dd5-2cd0-4dae-a537-278f548d88cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:23.190112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc1dd5-2cd0-4dae-a537-278f548d88cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:23.197946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-acbc1dd5-2cd0-4dae-a537-278f548d88cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:23.197946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:23.197946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:23.197946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:23.197946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:23.197946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:23.197946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:23.197946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:23.197946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:23.197946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:23.197946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:23.197946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:23.197946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:23.197946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:23.197946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:23.197946 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:23.198896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:23.198896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:23.198896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:23.198896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:23.198896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:23.198896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:23.198896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:23.198896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:23.198896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:23.198896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:23.198896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:23.198896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:23.198896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:23.198896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:23.198896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:23.198896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:23.199733 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:23.199733 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:23.199733 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:23.199733 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:23.199733 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:23.199733 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:23.199733 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:23.199733 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:23.199733 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:23.199733 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:23.199733 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:23.199733 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:23.208365 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 376/728] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:42:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:23.209962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc1dd5-2cd0-4dae-a537-278f548d88cd None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:23.212392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07135331-2985-4f62-a54e-f5582c58d344 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:23.282160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c359052-afae-46b4-b384-d2a465d2231a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:23.282964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c359052-afae-46b4-b384-d2a465d2231a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:23.283254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c359052-afae-46b4-b384-d2a465d2231a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:23.283532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c359052-afae-46b4-b384-d2a465d2231a None admin] PATH_INFO: `/v3/users/swift` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:23.294971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c359052-afae-46b4-b384-d2a465d2231a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:23.296942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c359052-afae-46b4-b384-d2a465d2231a None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:23.327611 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-3c359052-afae-46b4-b384-d2a465d2231a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 20:42:23.327611 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:23.327611 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:23.327611 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:23.327611 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:23.327611 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:23.327611 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:23.327611 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:23.327611 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:23.327611 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:23.327611 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:23.327611 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:23.327611 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:23.327611 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:23.327611 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:23.327611 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:23.328665 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:23.329646 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:23.329646 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:23.329646 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:23.329646 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:23.329646 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:23.329646 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:23.329646 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:23.329646 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:23.329646 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 20:42:23.329646 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:23.343181 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 353/729] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:42:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:23.355590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c359052-afae-46b4-b384-d2a465d2231a None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:23.413372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc1dd5-2cd0-4dae-a537-278f548d88cd None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:23.449592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f1498-f2d7-4991-b198-f140a9679b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:23.457300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f1498-f2d7-4991-b198-f140a9679b08 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:23.457300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f1498-f2d7-4991-b198-f140a9679b08 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:23.457300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f1498-f2d7-4991-b198-f140a9679b08 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:23.457300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f1498-f2d7-4991-b198-f140a9679b08 None admin] RBAC: Adding query filter params (name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:23.457300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f1498-f2d7-4991-b198-f140a9679b08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:23.457300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f1498-f2d7-4991-b198-f140a9679b08 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:23.463481 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 377/730] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:42:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:23.469716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f1498-f2d7-4991-b198-f140a9679b08 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:23.518209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:23.557744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c359052-afae-46b4-b384-d2a465d2231a None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:23.584754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:23.602693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf4cc7-dd95-480b-880a-3d2130b03838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:23.604809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf4cc7-dd95-480b-880a-3d2130b03838 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:23.605281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf4cc7-dd95-480b-880a-3d2130b03838 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:23.605751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf4cc7-dd95-480b-880a-3d2130b03838 None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:23.606796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf4cc7-dd95-480b-880a-3d2130b03838 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:23.607701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf4cc7-dd95-480b-880a-3d2130b03838 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:23.610083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf4cc7-dd95-480b-880a-3d2130b03838 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:23.635303 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 354/731] 10.222.0.33 () {56 vars in 1135 bytes} [Sat May 28 20:42:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:23.640388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7eb23-bb46-442b-b2da-506dcad9ba15 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:23.640388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7eb23-bb46-442b-b2da-506dcad9ba15 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:23.640388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7eb23-bb46-442b-b2da-506dcad9ba15 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:23.642229 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 355/732] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:23.643249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7eb23-bb46-442b-b2da-506dcad9ba15 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:23.654745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f1498-f2d7-4991-b198-f140a9679b08 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:23.692650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a66ec-6db2-49a8-921a-35e77c61b5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:23.692958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a66ec-6db2-49a8-921a-35e77c61b5d0 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:23.693204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a66ec-6db2-49a8-921a-35e77c61b5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:23.693480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a66ec-6db2-49a8-921a-35e77c61b5d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:23.705964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a66ec-6db2-49a8-921a-35e77c61b5d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:23.707701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a66ec-6db2-49a8-921a-35e77c61b5d0 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:23.733616 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-2f2a66ec-6db2-49a8-921a-35e77c61b5d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:23.733616 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:23.733616 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:23.733616 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:23.733616 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:23.733616 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:23.733616 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:23.733616 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:23.733616 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:23.733616 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:23.733616 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:23.733616 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:23.733616 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:23.733616 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:23.733616 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:23.734655 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:23.734655 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:23.734655 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:23.734655 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:23.734655 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:23.734655 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:23.734655 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:23.734655 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:23.734655 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:23.734655 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:23.734655 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:23.734655 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:23.734655 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:23.734655 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:23.734655 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:23.734655 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:23.734655 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:23.735596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:23.735596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:23.735596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:23.735596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:23.739072 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 378/733] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:23.740628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79307e51-062b-4c67-8ae5-10b68c2ecd13 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:23.741049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79307e51-062b-4c67-8ae5-10b68c2ecd13 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:23.741494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79307e51-062b-4c67-8ae5-10b68c2ecd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:23.760884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-79307e51-062b-4c67-8ae5-10b68c2ecd13 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:23.762329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-79307e51-062b-4c67-8ae5-10b68c2ecd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:23.795615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7eb23-bb46-442b-b2da-506dcad9ba15 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:23.825677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:23.835566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4899a6-497d-4cad-ad6e-510f691e4ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:23.836349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4899a6-497d-4cad-ad6e-510f691e4ffa None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:23.836672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4899a6-497d-4cad-ad6e-510f691e4ffa None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:23.837070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4899a6-497d-4cad-ad6e-510f691e4ffa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:23.844617 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 379/734] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:42:23.845826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79307e51-062b-4c67-8ae5-10b68c2ecd13 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:23.878398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:23.938479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4899a6-497d-4cad-ad6e-510f691e4ffa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:23.940041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4899a6-497d-4cad-ad6e-510f691e4ffa None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:23.969234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79307e51-062b-4c67-8ae5-10b68c2ecd13 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:24.008818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee9631-6eb0-49c1-9909-c75ee242628c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:24.009514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee9631-6eb0-49c1-9909-c75ee242628c None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:24.009796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee9631-6eb0-49c1-9909-c75ee242628c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:24.010104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee9631-6eb0-49c1-9909-c75ee242628c None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:24.010830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee9631-6eb0-49c1-9909-c75ee242628c None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:24.011134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee9631-6eb0-49c1-9909-c75ee242628c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:24.012504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee9631-6eb0-49c1-9909-c75ee242628c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:24.023963 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 380/735] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:24.024763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee9631-6eb0-49c1-9909-c75ee242628c None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:24.058128 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 356/736] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:42:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:42:24.059466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73df14d-b2c6-4944-9474-7550c554a27f None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:24.059760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73df14d-b2c6-4944-9474-7550c554a27f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:24.060042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73df14d-b2c6-4944-9474-7550c554a27f None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:24.061487 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 357/737] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:24.065821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73df14d-b2c6-4944-9474-7550c554a27f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:24.095663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:24.118649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:24.133261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee9631-6eb0-49c1-9909-c75ee242628c None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:24.165467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94898596-b218-45ea-87aa-9493c681eee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:24.166285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94898596-b218-45ea-87aa-9493c681eee6 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:24.166570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94898596-b218-45ea-87aa-9493c681eee6 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:24.166844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94898596-b218-45ea-87aa-9493c681eee6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:24.167748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73df14d-b2c6-4944-9474-7550c554a27f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:24.178028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94898596-b218-45ea-87aa-9493c681eee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:24.185579 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-94898596-b218-45ea-87aa-9493c681eee6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:42:24.185579 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:24.185579 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:24.185579 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:24.185579 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:24.185579 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:24.185579 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:24.185579 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:24.185579 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:24.185579 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:24.185579 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:24.185579 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:24.185579 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:24.185579 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:24.185579 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:24.185579 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:24.186800 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:24.186800 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:24.186800 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:24.186800 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:24.186800 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:24.186800 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:24.186800 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:24.186800 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:24.186800 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:24.186800 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:24.186800 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:24.186800 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:24.186800 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:24.186800 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:24.186800 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:24.186800 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:24.187663 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:24.187663 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:24.187663 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:24.187663 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:24.187663 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:24.187663 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:24.187663 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:24.187663 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:24.187663 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:24.187663 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:24.187663 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:42:24.187663 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:24.192293 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 381/738] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:42:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:24.193905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94898596-b218-45ea-87aa-9493c681eee6 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:24.202867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1aae73-4af7-4376-b153-21ceb3339ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:24.203672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1aae73-4af7-4376-b153-21ceb3339ef1 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:24.203953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1aae73-4af7-4376-b153-21ceb3339ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:24.204224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1aae73-4af7-4376-b153-21ceb3339ef1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:24.211261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1aae73-4af7-4376-b153-21ceb3339ef1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:24.212712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1aae73-4af7-4376-b153-21ceb3339ef1 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:24.228102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-8c1aae73-4af7-4376-b153-21ceb3339ef1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 20:42:24.228102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:24.228102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:24.228102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:24.228102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:24.228102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:24.228102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:24.228102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:24.228102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:24.228102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:24.228102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:24.228102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:24.228102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:24.228102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:24.228102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:24.228102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:24.229055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:24.230309 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:24.230309 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:24.230309 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:24.230309 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:24.230309 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:24.230309 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:24.230309 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:24.230309 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:24.230309 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 20:42:24.230309 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:24.230309 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 358/739] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 20:42:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:24.230924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d37eb-72e0-4e51-8417-07dea55e1f9c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:24.230924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d37eb-72e0-4e51-8417-07dea55e1f9c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:24.230924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20d37eb-72e0-4e51-8417-07dea55e1f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:24.247046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e20d37eb-72e0-4e51-8417-07dea55e1f9c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:24.248227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e20d37eb-72e0-4e51-8417-07dea55e1f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:24.290961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94898596-b218-45ea-87aa-9493c681eee6 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:24.312617 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 359/740] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:42:24.313931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd602c6a-34f1-491d-9696-ef999c93fd20 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:24.314224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd602c6a-34f1-491d-9696-ef999c93fd20 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:24.314505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd602c6a-34f1-491d-9696-ef999c93fd20 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:24.315751 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 360/741] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:24.316443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd602c6a-34f1-491d-9696-ef999c93fd20 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:24.335876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c54a8c-9251-4952-8a25-8b824313d125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:24.336669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c54a8c-9251-4952-8a25-8b824313d125 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:24.336952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c54a8c-9251-4952-8a25-8b824313d125 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:24.337227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c54a8c-9251-4952-8a25-8b824313d125 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:24.338211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:24.347427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c54a8c-9251-4952-8a25-8b824313d125 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:24.349418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c54a8c-9251-4952-8a25-8b824313d125 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:24.357118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:24.368486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-61c54a8c-9251-4952-8a25-8b824313d125 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:24.368486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:24.368486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:24.368486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:24.368486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:24.368486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:24.368486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:24.368486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:24.368486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:24.368486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:24.368486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:24.368486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:24.368486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:24.368486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:24.368486 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:24.369568 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:24.369568 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:24.369568 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:24.369568 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:24.369568 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:24.369568 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:24.369568 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:24.369568 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:24.369568 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:24.369568 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:24.369568 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:24.369568 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:24.369568 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:24.369568 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:24.369568 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:24.369568 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:24.369568 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:24.370621 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:24.370621 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:24.370621 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:24.370621 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:24.370621 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 382/742] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:24.370900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c54a8c-9251-4952-8a25-8b824313d125 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:24.424885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd602c6a-34f1-491d-9696-ef999c93fd20 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:24.459602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a05d9c-1dc3-44e3-97d4-feb0450c77cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:24.460258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a05d9c-1dc3-44e3-97d4-feb0450c77cd None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:24.460542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a05d9c-1dc3-44e3-97d4-feb0450c77cd None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:24.460819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a05d9c-1dc3-44e3-97d4-feb0450c77cd None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:24.461561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a05d9c-1dc3-44e3-97d4-feb0450c77cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:24.461895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a05d9c-1dc3-44e3-97d4-feb0450c77cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:24.463273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a05d9c-1dc3-44e3-97d4-feb0450c77cd None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:24.468447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c54a8c-9251-4952-8a25-8b824313d125 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:24.471590 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 361/743] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:42:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:24.472891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52411ab-23ed-4968-9146-1ca2929c1ebe None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:24.473183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52411ab-23ed-4968-9146-1ca2929c1ebe None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:24.473467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52411ab-23ed-4968-9146-1ca2929c1ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:24.489674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f52411ab-23ed-4968-9146-1ca2929c1ebe None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:24.490822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f52411ab-23ed-4968-9146-1ca2929c1ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:24.502941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c66307-67fb-4244-809e-887f60b3a6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:24.503613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c66307-67fb-4244-809e-887f60b3a6bd None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:24.504410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c66307-67fb-4244-809e-887f60b3a6bd None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:24.504690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c66307-67fb-4244-809e-887f60b3a6bd None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:24.505545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c66307-67fb-4244-809e-887f60b3a6bd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:24.505856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c66307-67fb-4244-809e-887f60b3a6bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:24.507306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c66307-67fb-4244-809e-887f60b3a6bd None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:24.525425 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 383/744] 10.222.0.33 () {56 vars in 1149 bytes} [Sat May 28 20:42:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:24.526277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c66307-67fb-4244-809e-887f60b3a6bd None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:24.562336 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 362/745] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 20:42:24.563178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52411ab-23ed-4968-9146-1ca2929c1ebe None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:24.577857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:24.583387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:24.625424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c66307-67fb-4244-809e-887f60b3a6bd None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:24.659549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c98e5-da39-4f06-be0f-42bc8a325e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:24.660236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c98e5-da39-4f06-be0f-42bc8a325e0a None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:24.660519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c98e5-da39-4f06-be0f-42bc8a325e0a None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:24.660805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c98e5-da39-4f06-be0f-42bc8a325e0a None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:24.661525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c98e5-da39-4f06-be0f-42bc8a325e0a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:24.662940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c98e5-da39-4f06-be0f-42bc8a325e0a None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:24.666303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52411ab-23ed-4968-9146-1ca2929c1ebe None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:24.700449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40cce9a-db8f-4651-8518-a22cb44ef7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:24.701101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cce9a-db8f-4651-8518-a22cb44ef7e5 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:24.701381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cce9a-db8f-4651-8518-a22cb44ef7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:24.701651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cce9a-db8f-4651-8518-a22cb44ef7e5 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:24.702399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40cce9a-db8f-4651-8518-a22cb44ef7e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:24.702703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40cce9a-db8f-4651-8518-a22cb44ef7e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:24.704091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40cce9a-db8f-4651-8518-a22cb44ef7e5 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:24.714868 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 363/746] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:24.715630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40cce9a-db8f-4651-8518-a22cb44ef7e5 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:24.760735 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 384/747] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 20:42:24 2022] POST /identity/v3/services => generated 240 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:42:24.761405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c98e5-da39-4f06-be0f-42bc8a325e0a None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:24.797482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:24.836932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40cce9a-db8f-4651-8518-a22cb44ef7e5 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:24.882517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba451a6-5cf0-481f-a405-77f8ae163505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:24.883332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba451a6-5cf0-481f-a405-77f8ae163505 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:24.883614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba451a6-5cf0-481f-a405-77f8ae163505 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:24.883916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba451a6-5cf0-481f-a405-77f8ae163505 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:24.888676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c98e5-da39-4f06-be0f-42bc8a325e0a None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:24.896049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba451a6-5cf0-481f-a405-77f8ae163505 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:24.897547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba451a6-5cf0-481f-a405-77f8ae163505 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:24.909200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:24.916293 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-dba451a6-5cf0-481f-a405-77f8ae163505 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:24.916293 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:24.916293 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:24.916293 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:24.916293 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:24.916293 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:24.916293 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:24.916293 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:24.916293 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:24.916293 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:24.916293 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:24.916293 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:24.916293 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:24.916293 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:24.916293 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:24.917246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:24.917246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:24.917246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:24.917246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:24.917246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:24.917246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:24.917246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:24.917246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:24.917246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:24.917246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:24.917246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:24.917246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:24.917246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:24.917246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:24.917246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:24.917246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:24.917246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:24.918650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:24.918650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:24.918650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:24.918650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:24.918650 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 364/748] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:24.918650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba451a6-5cf0-481f-a405-77f8ae163505 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:24.935790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb842d9-291e-4211-a3b8-51144ba12b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:24.936456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb842d9-291e-4211-a3b8-51144ba12b11 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:24.936753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb842d9-291e-4211-a3b8-51144ba12b11 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:24.937029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb842d9-291e-4211-a3b8-51144ba12b11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:25.014197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb842d9-291e-4211-a3b8-51144ba12b11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:25.015875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb842d9-291e-4211-a3b8-51144ba12b11 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:25.039894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba451a6-5cf0-481f-a405-77f8ae163505 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:25.064000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:25.081786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d1f54-4e0d-476b-bcf9-7a7617adfe34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:25.082667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d1f54-4e0d-476b-bcf9-7a7617adfe34 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:25.083126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d1f54-4e0d-476b-bcf9-7a7617adfe34 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:25.083237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d1f54-4e0d-476b-bcf9-7a7617adfe34 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:25.083946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d1f54-4e0d-476b-bcf9-7a7617adfe34 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:25.085383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d1f54-4e0d-476b-bcf9-7a7617adfe34 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:25.125156 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 385/749] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:42:24 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 20:42:25.126032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb842d9-291e-4211-a3b8-51144ba12b11 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:25.145013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:25.173429 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 365/750] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 20:42:24 2022] POST /identity/v3/services => generated 251 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:42:25.174457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d1f54-4e0d-476b-bcf9-7a7617adfe34 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:25.249780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb842d9-291e-4211-a3b8-51144ba12b11 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:25.303287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b656-e4ea-473b-849b-9456694b6de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:25.304105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b656-e4ea-473b-849b-9456694b6de2 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:25.304392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b656-e4ea-473b-849b-9456694b6de2 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:25.304673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9b656-e4ea-473b-849b-9456694b6de2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:25.312930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9b656-e4ea-473b-849b-9456694b6de2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:25.314464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d9b656-e4ea-473b-849b-9456694b6de2 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:25.318709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d1f54-4e0d-476b-bcf9-7a7617adfe34 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:25.329496 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-85d9b656-e4ea-473b-849b-9456694b6de2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:25.329496 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:25.329496 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:25.329496 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:25.329496 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:25.329496 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:25.329496 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:25.329496 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:25.329496 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:25.329496 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:25.329496 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:25.329496 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:25.329496 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:25.329496 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:25.329496 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:25.330489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:25.330489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:25.330489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:25.330489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:25.330489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:25.330489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:25.330489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:25.330489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:25.330489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:25.330489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:25.330489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:25.330489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:25.330489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:25.330489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:25.330489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:25.330489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:25.330489 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:25.331391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:25.331391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:25.331391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:25.331391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:25.331391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:25.331391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:25.331391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:25.331391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:25.331391 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 386/751] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:25.332498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b656-e4ea-473b-849b-9456694b6de2 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:25.364036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6536339-56e3-4c92-8bc7-ac14496768c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:25.364700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6536339-56e3-4c92-8bc7-ac14496768c9 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:25.365042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6536339-56e3-4c92-8bc7-ac14496768c9 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:25.365329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6536339-56e3-4c92-8bc7-ac14496768c9 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:25.366147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6536339-56e3-4c92-8bc7-ac14496768c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:25.366458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6536339-56e3-4c92-8bc7-ac14496768c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:25.367851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6536339-56e3-4c92-8bc7-ac14496768c9 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:25.382884 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 366/752] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:25.383830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6536339-56e3-4c92-8bc7-ac14496768c9 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:25.411880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:25.428613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:25.461176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9b656-e4ea-473b-849b-9456694b6de2 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:25.505185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894cdbb3-b246-467f-a243-5b8f32824466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:25.505397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6536339-56e3-4c92-8bc7-ac14496768c9 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:25.506031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cdbb3-b246-467f-a243-5b8f32824466 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:25.506327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cdbb3-b246-467f-a243-5b8f32824466 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:25.506612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894cdbb3-b246-467f-a243-5b8f32824466 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:25.520831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894cdbb3-b246-467f-a243-5b8f32824466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:25.523003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894cdbb3-b246-467f-a243-5b8f32824466 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:25.542706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4b0b3-ce03-4a18-be4b-1722ffd62687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:25.543372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4b0b3-ce03-4a18-be4b-1722ffd62687 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:25.543660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4b0b3-ce03-4a18-be4b-1722ffd62687 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:25.544004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae4b0b3-ce03-4a18-be4b-1722ffd62687 None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:25.544802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4b0b3-ce03-4a18-be4b-1722ffd62687 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:25.545164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4b0b3-ce03-4a18-be4b-1722ffd62687 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:25.545453 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-894cdbb3-b246-467f-a243-5b8f32824466 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:25.545453 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:25.545453 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:25.545453 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:25.545453 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:25.545453 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:25.545453 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:25.545453 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:25.545453 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:25.545453 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:25.545453 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:25.545453 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:25.545453 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:25.545453 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:25.545453 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:25.546391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:25.546391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:25.546391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:25.546391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:25.546391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:25.546391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:25.546391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:25.546391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:25.546391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:25.546391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:25.546391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:25.546391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:25.546391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:25.546391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:25.546391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:25.546391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:25.546391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:25.547313 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:25.547313 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:25.547313 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:25.547313 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:25.547534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae4b0b3-ce03-4a18-be4b-1722ffd62687 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:25.547613 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 387/753] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:25.548756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894cdbb3-b246-467f-a243-5b8f32824466 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:25.560036 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 367/754] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:25.560859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4b0b3-ce03-4a18-be4b-1722ffd62687 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:25.663128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:25.670104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae4b0b3-ce03-4a18-be4b-1722ffd62687 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:25.680714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894cdbb3-b246-467f-a243-5b8f32824466 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:25.710259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde47d5-9740-412a-afb0-ef23653000be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:25.710946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde47d5-9740-412a-afb0-ef23653000be None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:25.711233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde47d5-9740-412a-afb0-ef23653000be None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:25.711514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde47d5-9740-412a-afb0-ef23653000be None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:25.712245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde47d5-9740-412a-afb0-ef23653000be None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:25.712699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde47d5-9740-412a-afb0-ef23653000be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:25.714749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde47d5-9740-412a-afb0-ef23653000be None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:25.718260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:25.722437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd3eea-77c0-4bf3-8608-c1b3859625ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:25.724035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd3eea-77c0-4bf3-8608-c1b3859625ba None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:25.724526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd3eea-77c0-4bf3-8608-c1b3859625ba None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:25.725045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd3eea-77c0-4bf3-8608-c1b3859625ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:25.730857 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 368/755] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:25.731893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde47d5-9740-412a-afb0-ef23653000be None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:25.734197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd3eea-77c0-4bf3-8608-c1b3859625ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:25.736212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd3eea-77c0-4bf3-8608-c1b3859625ba None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:25.752966 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-96bd3eea-77c0-4bf3-8608-c1b3859625ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 20:42:25.752966 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:25.752966 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:25.752966 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:25.752966 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:25.752966 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:25.752966 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:25.752966 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:25.752966 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:25.752966 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:25.752966 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:25.752966 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:25.752966 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:25.752966 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:25.752966 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:25.752966 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:25.754566 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:25.755608 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:25.755608 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:25.755608 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:25.755608 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:25.755608 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:25.755608 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:25.755608 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:25.755608 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:25.755608 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 20:42:25.755608 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:25.758408 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 388/756] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 20:42:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:25.759532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd3eea-77c0-4bf3-8608-c1b3859625ba None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:25.858336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde47d5-9740-412a-afb0-ef23653000be None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:25.872350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd3eea-77c0-4bf3-8608-c1b3859625ba None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:25.901782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d9f09-1190-4218-be61-ed2e14003c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:25.902503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d9f09-1190-4218-be61-ed2e14003c7b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:25.902789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d9f09-1190-4218-be61-ed2e14003c7b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:25.903067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d9f09-1190-4218-be61-ed2e14003c7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:25.904190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d9f09-1190-4218-be61-ed2e14003c7b None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=a01bba888a644eb18fc457aff578aa4b) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:25.904503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d9f09-1190-4218-be61-ed2e14003c7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:25.906461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d9f09-1190-4218-be61-ed2e14003c7b None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:25.921886 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 369/757] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:25 2022] GET /identity/v3/role_assignments?user.id=a01bba888a644eb18fc457aff578aa4b&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=7805b93b32f1472f8365936d4d57d49a => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:25.921886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d9f09-1190-4218-be61-ed2e14003c7b None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:25.924367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d5995-b67f-4c1b-ab41-b2a08e4edf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:25.925206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d5995-b67f-4c1b-ab41-b2a08e4edf3b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:25.925493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d5995-b67f-4c1b-ab41-b2a08e4edf3b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:25.925814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d5995-b67f-4c1b-ab41-b2a08e4edf3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:25.944617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d5995-b67f-4c1b-ab41-b2a08e4edf3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:25.946180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d5995-b67f-4c1b-ab41-b2a08e4edf3b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:25.965215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:25.978484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-d08d5995-b67f-4c1b-ab41-b2a08e4edf3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:25.978484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:25.978484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:25.978484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:25.978484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:25.978484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:25.978484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:25.978484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:25.978484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:25.978484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:25.978484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:25.978484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:25.978484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:25.978484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:25.978484 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:25.979485 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:25.979485 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:25.979485 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:25.979485 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:25.979485 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:25.979485 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:25.979485 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:25.979485 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:25.979485 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:25.979485 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:25.979485 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:25.979485 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:25.979485 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:25.979485 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:25.979485 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:25.979485 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:25.979485 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:25.980394 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:25.980394 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:25.980394 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:25.980394 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:25.980394 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:25.980394 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:25.980394 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:25.980394 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:25.980394 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 389/758] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:25.987369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d5995-b67f-4c1b-ab41-b2a08e4edf3b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:26.084887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d9f09-1190-4218-be61-ed2e14003c7b None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:26.089173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:26.149952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c5356-844b-498d-a25e-499d81af51cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:26.149952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c5356-844b-498d-a25e-499d81af51cb None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:26.149952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c5356-844b-498d-a25e-499d81af51cb None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:26.149952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96c5356-844b-498d-a25e-499d81af51cb None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:26.150353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c5356-844b-498d-a25e-499d81af51cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:26.150353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c5356-844b-498d-a25e-499d81af51cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:26.150353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96c5356-844b-498d-a25e-499d81af51cb None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:26.170463 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 370/759] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 20:42:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:26.176882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c5356-844b-498d-a25e-499d81af51cb None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:26.188499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d5995-b67f-4c1b-ab41-b2a08e4edf3b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:26.239075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcfa8c-9b4a-4af9-b5cf-dbd6a1f891c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:26.240109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcfa8c-9b4a-4af9-b5cf-dbd6a1f891c5 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:26.240600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcfa8c-9b4a-4af9-b5cf-dbd6a1f891c5 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:26.241078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcfa8c-9b4a-4af9-b5cf-dbd6a1f891c5 None admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:26.242128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcfa8c-9b4a-4af9-b5cf-dbd6a1f891c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:26.242650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcfa8c-9b4a-4af9-b5cf-dbd6a1f891c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:26.244313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dcfa8c-9b4a-4af9-b5cf-dbd6a1f891c5 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:26.264123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:26.272494 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 390/760] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:26.278042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcfa8c-9b4a-4af9-b5cf-dbd6a1f891c5 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:26.287858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96c5356-844b-498d-a25e-499d81af51cb None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:26.329982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a61df-a0de-43ff-9dd1-01aeb9ccc1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:26.329982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a61df-a0de-43ff-9dd1-01aeb9ccc1b3 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:26.329982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a61df-a0de-43ff-9dd1-01aeb9ccc1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:26.329982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a61df-a0de-43ff-9dd1-01aeb9ccc1b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:26.337571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a61df-a0de-43ff-9dd1-01aeb9ccc1b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:26.345953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a61df-a0de-43ff-9dd1-01aeb9ccc1b3 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:26.365528 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-5b3a61df-a0de-43ff-9dd1-01aeb9ccc1b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:26.365528 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:26.365528 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:26.365528 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:26.365528 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:26.365528 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:26.365528 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:26.365528 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:26.365528 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:26.365528 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:26.365528 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:26.365528 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:26.365528 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:26.365528 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:26.365528 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:26.366560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:26.366560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:26.366560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:26.366560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:26.366560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:26.366560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:26.366560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:26.366560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:26.366560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:26.366560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:26.366560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:26.366560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:26.366560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:26.366560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:26.366560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:26.366560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:26.366560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:26.367451 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:26.367451 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:26.367451 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:26.367451 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:26.374504 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 371/761] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:26.380072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a61df-a0de-43ff-9dd1-01aeb9ccc1b3 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:26.421437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcfa8c-9b4a-4af9-b5cf-dbd6a1f891c5 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:26.424756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:26.458157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14033b-6460-48fb-87b5-0796cfba4482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:26.459202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14033b-6460-48fb-87b5-0796cfba4482 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:26.459691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14033b-6460-48fb-87b5-0796cfba4482 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:26.460172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14033b-6460-48fb-87b5-0796cfba4482 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:26.461310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14033b-6460-48fb-87b5-0796cfba4482 None admin] RBAC: Adding query filter params (role.id=4db40e559d064cf6b4f061e785b42dbb, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=c1c1ea9ba4c740b3b7ed67c8335bfad2) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:26.462035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14033b-6460-48fb-87b5-0796cfba4482 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:26.463667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14033b-6460-48fb-87b5-0796cfba4482 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:26.470229 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 391/762] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:26 2022] GET /identity/v3/role_assignments?user.id=c1c1ea9ba4c740b3b7ed67c8335bfad2&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=4db40e559d064cf6b4f061e785b42dbb => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:26.545300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a61df-a0de-43ff-9dd1-01aeb9ccc1b3 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:26.578714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:26.589567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ffb56-411e-4bb5-abd3-d73b5fb724f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:26.592480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ffb56-411e-4bb5-abd3-d73b5fb724f4 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:26.592942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ffb56-411e-4bb5-abd3-d73b5fb724f4 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:26.593387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ffb56-411e-4bb5-abd3-d73b5fb724f4 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:26.594336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ffb56-411e-4bb5-abd3-d73b5fb724f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:26.594813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ffb56-411e-4bb5-abd3-d73b5fb724f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:26.596395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ffb56-411e-4bb5-abd3-d73b5fb724f4 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:26.615873 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 372/763] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:26.621657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14033b-6460-48fb-87b5-0796cfba4482 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:26.786643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14033b-6460-48fb-87b5-0796cfba4482 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:26.828003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be9109-91e4-40b4-8209-cb1196bdd79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:26.829193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be9109-91e4-40b4-8209-cb1196bdd79a None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:26.829688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be9109-91e4-40b4-8209-cb1196bdd79a None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:26.830234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be9109-91e4-40b4-8209-cb1196bdd79a None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:26.844645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be9109-91e4-40b4-8209-cb1196bdd79a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:26.846470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be9109-91e4-40b4-8209-cb1196bdd79a None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:26.862034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-d7be9109-91e4-40b4-8209-cb1196bdd79a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:26.862034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:26.862034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:26.862034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:26.862034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:26.862034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:26.862034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:26.862034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:26.862034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:26.862034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:26.862034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:26.862034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:26.862034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:26.862034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:26.862034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:26.873725 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:26.873725 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:26.873725 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:26.873725 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:26.873725 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:26.873725 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:26.873725 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:26.873725 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:26.873725 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:26.873725 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:26.873725 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:26.873725 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:26.873725 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:26.873725 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:26.873725 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:26.873725 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:26.873725 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:26.874891 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:26.874891 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:26.874891 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:26.874891 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:26.874891 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:26.874891 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:26.874891 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:26.874891 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:26.874891 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 392/764] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:26.875396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ffb56-411e-4bb5-abd3-d73b5fb724f4 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:27.019738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ffb56-411e-4bb5-abd3-d73b5fb724f4 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:27.076469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218af2e0-4178-4a02-8e10-b3125a2b3954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:27.077146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218af2e0-4178-4a02-8e10-b3125a2b3954 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:27.077485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218af2e0-4178-4a02-8e10-b3125a2b3954 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:27.077756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218af2e0-4178-4a02-8e10-b3125a2b3954 None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:27.078850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218af2e0-4178-4a02-8e10-b3125a2b3954 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:27.079167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218af2e0-4178-4a02-8e10-b3125a2b3954 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:27.080707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218af2e0-4178-4a02-8e10-b3125a2b3954 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:27.099552 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 373/765] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:27.105599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be9109-91e4-40b4-8209-cb1196bdd79a None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:27.160716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:27.257048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be9109-91e4-40b4-8209-cb1196bdd79a None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:27.302249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760861ba-a419-400a-9c2c-37d6d5b056c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:27.303145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760861ba-a419-400a-9c2c-37d6d5b056c9 None admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:27.303429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760861ba-a419-400a-9c2c-37d6d5b056c9 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:27.303704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760861ba-a419-400a-9c2c-37d6d5b056c9 None admin] PATH_INFO: `/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/cfdebb5e6b8d4e2d86f0a03a30c21671/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:27.305667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760861ba-a419-400a-9c2c-37d6d5b056c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3186f9d3c2b341afa8b098ebcaaf974b, user_id=cfdebb5e6b8d4e2d86f0a03a30c21671, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:27.307151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760861ba-a419-400a-9c2c-37d6d5b056c9 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:27.371600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1834e-7cc8-4075-8203-6e1adb01a7aa None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:27.371894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1834e-7cc8-4075-8203-6e1adb01a7aa None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:27.372082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1834e-7cc8-4075-8203-6e1adb01a7aa None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:27.373713 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 374/766] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:27.378596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af9c6f-17ff-46a9-8290-cc0b68db5350 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:27.378914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af9c6f-17ff-46a9-8290-cc0b68db5350 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:27.379193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af9c6f-17ff-46a9-8290-cc0b68db5350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:27.395666 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 393/767] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:42:27 2022] PUT /identity/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/cfdebb5e6b8d4e2d86f0a03a30c21671/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:42:27.405334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b9af9c6f-17ff-46a9-8290-cc0b68db5350 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:27.413014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b9af9c6f-17ff-46a9-8290-cc0b68db5350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:27.486824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:27.516129 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 375/768] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:42:27.552613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760861ba-a419-400a-9c2c-37d6d5b056c9 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:27.675032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:27.684309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760861ba-a419-400a-9c2c-37d6d5b056c9 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:27.721444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f3a1d-de5d-4be7-9c07-fe033ebfd197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:27.722287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f3a1d-de5d-4be7-9c07-fe033ebfd197 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:27.722634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f3a1d-de5d-4be7-9c07-fe033ebfd197 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:27.723411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f3a1d-de5d-4be7-9c07-fe033ebfd197 None admin] PATH_INFO: `/v3/services/network` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:27.724087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f3a1d-de5d-4be7-9c07-fe033ebfd197 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:27.725534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f3a1d-de5d-4be7-9c07-fe033ebfd197 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:27.739762 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-643f3a1d-de5d-4be7-9c07-fe033ebfd197 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 28 20:42:27.739762 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:27.739762 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:27.739762 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:27.739762 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:27.739762 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:27.739762 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:27.739762 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:27.739762 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 20:42:27.739762 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:27.739762 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:27.739762 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:27.739762 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:27.739762 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:27.739762 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:27.739762 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:27.739762 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:27.740769 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:27.740769 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:27.740769 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:27.740769 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:27.740769 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:27.740769 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:27.740769 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:27.740769 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:27.740769 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:27.740769 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:27.740769 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:27.740769 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:27.740769 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:27.740769 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:27.740769 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:27.740769 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:27.741591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:27.741591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:27.741591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:27.741591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:27.741591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:27.741591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:27.741591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:27.741591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:27.741591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:27.741591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:27.741591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:27.741591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:27.741591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 20:42:27.741591 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:27.741591 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 394/769] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:27 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:27.748169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af9c6f-17ff-46a9-8290-cc0b68db5350 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:27.833602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0017c4-d96b-46e8-89ea-03e8af6d48a9 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:27.833957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0017c4-d96b-46e8-89ea-03e8af6d48a9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:27.834217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0017c4-d96b-46e8-89ea-03e8af6d48a9 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:27.836913 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 395/770] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:27.841027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6baaece-f292-4221-b795-24f11d101ffa None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:27.841314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6baaece-f292-4221-b795-24f11d101ffa None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:27.841592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6baaece-f292-4221-b795-24f11d101ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:27.871829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b6baaece-f292-4221-b795-24f11d101ffa None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:27.873027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b6baaece-f292-4221-b795-24f11d101ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:27.887360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af9c6f-17ff-46a9-8290-cc0b68db5350 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:27.951215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2427f8-6f0e-4404-9b20-2a4bdd34b005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:27.951802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2427f8-6f0e-4404-9b20-2a4bdd34b005 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:27.952088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2427f8-6f0e-4404-9b20-2a4bdd34b005 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:27.952364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2427f8-6f0e-4404-9b20-2a4bdd34b005 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:27.953078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2427f8-6f0e-4404-9b20-2a4bdd34b005 None admin] RBAC: Adding query filter params (name=network) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:27.953385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2427f8-6f0e-4404-9b20-2a4bdd34b005 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:27.954793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2427f8-6f0e-4404-9b20-2a4bdd34b005 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:27.967236 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 376/771] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 20:42:27 2022] GET /identity/v3/services?name=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:27.973423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2427f8-6f0e-4404-9b20-2a4bdd34b005 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:27.990105 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 396/772] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:42:28.034222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:28.037502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6baaece-f292-4221-b795-24f11d101ffa None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:28.148519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2427f8-6f0e-4404-9b20-2a4bdd34b005 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:28.187488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:28.212496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a83b2c-f896-40fc-9cba-da922b4f33d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:28.213138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a83b2c-f896-40fc-9cba-da922b4f33d9 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:28.213426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a83b2c-f896-40fc-9cba-da922b4f33d9 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:28.213760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a83b2c-f896-40fc-9cba-da922b4f33d9 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:28.214514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a83b2c-f896-40fc-9cba-da922b4f33d9 None admin] RBAC: Adding query filter params (type=network) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:28.214826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a83b2c-f896-40fc-9cba-da922b4f33d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:28.216218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a83b2c-f896-40fc-9cba-da922b4f33d9 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:28.223099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6baaece-f292-4221-b795-24f11d101ffa None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:28.231940 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 377/773] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 20:42:27 2022] GET /identity/v3/services?type=network => generated 350 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:28.232790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a83b2c-f896-40fc-9cba-da922b4f33d9 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:28.279707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0aa56b-2259-4d84-b743-0facefb556f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:28.281259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0aa56b-2259-4d84-b743-0facefb556f6 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:28.281765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0aa56b-2259-4d84-b743-0facefb556f6 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:28.282270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0aa56b-2259-4d84-b743-0facefb556f6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:28.283133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0aa56b-2259-4d84-b743-0facefb556f6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:28.284725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0aa56b-2259-4d84-b743-0facefb556f6 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:28.297507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-fe0aa56b-2259-4d84-b743-0facefb556f6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 20:42:28.297507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:28.297507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:28.297507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:28.297507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:28.297507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:28.297507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:28.297507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:28.297507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 20:42:28.297507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:28.297507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:28.297507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:28.297507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:28.297507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:28.297507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:28.297507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:28.298487 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:28.298487 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:28.298487 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:28.298487 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:28.298487 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:28.298487 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:28.298487 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:28.298487 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:28.298487 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:28.298487 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:28.298487 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:28.298487 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:28.298487 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:28.298487 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:28.298487 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:28.298487 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:28.299321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:28.299321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:28.299321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:28.299321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:28.299321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:28.299321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:28.299321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:28.299321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:28.299321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:28.299321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:28.299321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:28.299321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:28.299321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:28.299321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 20:42:28.299321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:28.302387 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 397/774] 10.222.0.33 () {56 vars in 1129 bytes} [Sat May 28 20:42:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:42:28.309627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0aa56b-2259-4d84-b743-0facefb556f6 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:28.410373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a83b2c-f896-40fc-9cba-da922b4f33d9 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:28.427662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:28.473071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a3090-b915-4885-807a-17f515cc3df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:28.473658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a3090-b915-4885-807a-17f515cc3df2 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:28.477802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a3090-b915-4885-807a-17f515cc3df2 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:28.478258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a3090-b915-4885-807a-17f515cc3df2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:28.479168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a3090-b915-4885-807a-17f515cc3df2 None admin] RBAC: Adding query filter params (interface=public, service_id=c18fc27da02a48a590319e1b0c4992fb, region_id=RegionOne) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:28.479352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a3090-b915-4885-807a-17f515cc3df2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:28.480778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a3090-b915-4885-807a-17f515cc3df2 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:28.500352 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 378/775] 10.222.0.33 () {56 vars in 1251 bytes} [Sat May 28 20:42:28 2022] GET /identity/v3/endpoints?service_id=c18fc27da02a48a590319e1b0c4992fb&interface=public®ion_id=RegionOne => generated 194 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:28.502492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0aa56b-2259-4d84-b743-0facefb556f6 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:28.507244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a3090-b915-4885-807a-17f515cc3df2 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:28.557490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf71bb-5b94-4cc9-ada6-fbc60f622435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:28.558225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf71bb-5b94-4cc9-ada6-fbc60f622435 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:28.558483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf71bb-5b94-4cc9-ada6-fbc60f622435 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:28.558759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbf71bb-5b94-4cc9-ada6-fbc60f622435 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:28.559566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf71bb-5b94-4cc9-ada6-fbc60f622435 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:28.559877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf71bb-5b94-4cc9-ada6-fbc60f622435 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:28.561316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbf71bb-5b94-4cc9-ada6-fbc60f622435 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:28.572775 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 398/776] 10.222.0.33 () {56 vars in 1124 bytes} [Sat May 28 20:42:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:28.577130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf71bb-5b94-4cc9-ada6-fbc60f622435 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:28.603759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:28.700628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a3090-b915-4885-807a-17f515cc3df2 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:28.703188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbf71bb-5b94-4cc9-ada6-fbc60f622435 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:28.738912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca1cd9-e00a-4f1e-915e-2f82cfb23efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:28.739506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca1cd9-e00a-4f1e-915e-2f82cfb23efc None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:28.739797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca1cd9-e00a-4f1e-915e-2f82cfb23efc None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:28.740083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca1cd9-e00a-4f1e-915e-2f82cfb23efc None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:28.740805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca1cd9-e00a-4f1e-915e-2f82cfb23efc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:28.741120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca1cd9-e00a-4f1e-915e-2f82cfb23efc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:28.742708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca1cd9-e00a-4f1e-915e-2f82cfb23efc None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:28.748015 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 399/777] 10.222.0.33 () {56 vars in 1124 bytes} [Sat May 28 20:42:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:28.753762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca1cd9-e00a-4f1e-915e-2f82cfb23efc None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:28.786277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64eb5c0-f7d6-4de9-ba22-dedfc4e6a217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:28.786964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64eb5c0-f7d6-4de9-ba22-dedfc4e6a217 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:28.787257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64eb5c0-f7d6-4de9-ba22-dedfc4e6a217 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:28.787542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64eb5c0-f7d6-4de9-ba22-dedfc4e6a217 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:28.788247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64eb5c0-f7d6-4de9-ba22-dedfc4e6a217 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:28.789635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64eb5c0-f7d6-4de9-ba22-dedfc4e6a217 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:28.795717 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 379/778] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 20:42:28 2022] GET /identity/v3/services => generated 772 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:28.854239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:28.870231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca1cd9-e00a-4f1e-915e-2f82cfb23efc None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:28.909276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95b29f-f1c2-46a9-bf7d-22db9f425ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:28.910521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95b29f-f1c2-46a9-bf7d-22db9f425ab8 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:28.910812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95b29f-f1c2-46a9-bf7d-22db9f425ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:28.911093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba95b29f-f1c2-46a9-bf7d-22db9f425ab8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:28.911902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba95b29f-f1c2-46a9-bf7d-22db9f425ab8 None admin] RBAC: Adding query filter params (interface=public, service_id=34ceb424e044440fb38dfaf5fb78de88, region_id=RegionOne) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:28.912221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba95b29f-f1c2-46a9-bf7d-22db9f425ab8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:28.913608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba95b29f-f1c2-46a9-bf7d-22db9f425ab8 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:28.919480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be801e4-6212-4e4c-aedb-c5c221ba292c None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:28.919776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be801e4-6212-4e4c-aedb-c5c221ba292c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:28.920063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be801e4-6212-4e4c-aedb-c5c221ba292c None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:28.921762 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 380/779] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:28.922637 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 400/780] 10.222.0.33 () {56 vars in 1251 bytes} [Sat May 28 20:42:28 2022] GET /identity/v3/endpoints?service_id=34ceb424e044440fb38dfaf5fb78de88&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:28.928141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c52df5-27c2-4a12-9e46-b3439b61286e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:28.928482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95b29f-f1c2-46a9-bf7d-22db9f425ab8 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:28.928588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c52df5-27c2-4a12-9e46-b3439b61286e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:28.928865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c52df5-27c2-4a12-9e46-b3439b61286e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:28.960228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-38c52df5-27c2-4a12-9e46-b3439b61286e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:28.961459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-38c52df5-27c2-4a12-9e46-b3439b61286e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:29.041558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba95b29f-f1c2-46a9-bf7d-22db9f425ab8 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:29.066268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:29.091416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce8767-e554-4222-bdd2-6ee4a36c3dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:29.092246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce8767-e554-4222-bdd2-6ee4a36c3dc5 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:29.092541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce8767-e554-4222-bdd2-6ee4a36c3dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:29.092826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce8767-e554-4222-bdd2-6ee4a36c3dc5 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:29.094110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce8767-e554-4222-bdd2-6ee4a36c3dc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:29.095578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce8767-e554-4222-bdd2-6ee4a36c3dc5 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:29.099906 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 381/781] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:29.101775 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 401/782] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 20:42:28 2022] GET /identity/v3/services => generated 772 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:29.127276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c52df5-27c2-4a12-9e46-b3439b61286e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:29.245445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c52df5-27c2-4a12-9e46-b3439b61286e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:29.280971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c272e6-9c66-4ce5-8032-1f325874f65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:29.281770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c272e6-9c66-4ce5-8032-1f325874f65a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:29.282080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c272e6-9c66-4ce5-8032-1f325874f65a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:29.282356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c272e6-9c66-4ce5-8032-1f325874f65a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:29.293778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c272e6-9c66-4ce5-8032-1f325874f65a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:29.301372 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-88c272e6-9c66-4ce5-8032-1f325874f65a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:42:29.301372 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:29.301372 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:29.301372 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:29.301372 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:29.301372 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:29.301372 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:29.301372 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:29.301372 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:29.301372 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:29.301372 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:29.301372 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:29.301372 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:29.301372 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:29.301372 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:29.301372 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:29.302357 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:29.302357 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:29.302357 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:29.302357 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:29.302357 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:29.302357 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:29.302357 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:29.302357 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:29.302357 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:29.302357 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:29.302357 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:29.302357 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:29.302357 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:29.302357 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:29.302357 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:29.302357 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:29.303228 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:29.303228 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:29.303228 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:29.303228 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:29.303228 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:29.303228 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:29.303228 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:29.303228 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:29.303228 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:29.303228 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:29.303228 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:42:29.303228 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:29.303228 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 382/783] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:42:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:29.308878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce8767-e554-4222-bdd2-6ee4a36c3dc5 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:29.311882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:29.416031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7afbf-a335-4658-8996-e97b3dc0598a None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:29.416665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7afbf-a335-4658-8996-e97b3dc0598a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:29.417163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7afbf-a335-4658-8996-e97b3dc0598a None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:29.419078 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 383/784] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:29.424293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324c3dd-a8da-45a2-b798-6c34fcae204e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:29.424833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324c3dd-a8da-45a2-b798-6c34fcae204e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:29.425319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324c3dd-a8da-45a2-b798-6c34fcae204e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:29.430605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce8767-e554-4222-bdd2-6ee4a36c3dc5 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:29.446184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7324c3dd-a8da-45a2-b798-6c34fcae204e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:29.448089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7324c3dd-a8da-45a2-b798-6c34fcae204e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:29.478633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3779d0-125b-42b0-9724-50ee83085043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:29.479638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3779d0-125b-42b0-9724-50ee83085043 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:29.480108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3779d0-125b-42b0-9724-50ee83085043 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:29.480741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3779d0-125b-42b0-9724-50ee83085043 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:29.481687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:29.483741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3779d0-125b-42b0-9724-50ee83085043 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:29.484252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3779d0-125b-42b0-9724-50ee83085043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:29.486406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3779d0-125b-42b0-9724-50ee83085043 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:29.498950 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 402/785] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:42:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:29.506129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3779d0-125b-42b0-9724-50ee83085043 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:29.541526 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 384/786] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:42:29.569574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7324c3dd-a8da-45a2-b798-6c34fcae204e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:29.648164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:29.698401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3779d0-125b-42b0-9724-50ee83085043 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:29.707462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7324c3dd-a8da-45a2-b798-6c34fcae204e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:29.747645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0882836-3c93-4d66-8014-21a2885415ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:29.748960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0882836-3c93-4d66-8014-21a2885415ca None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:29.749445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0882836-3c93-4d66-8014-21a2885415ca None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:29.750338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0882836-3c93-4d66-8014-21a2885415ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:29.763423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0882836-3c93-4d66-8014-21a2885415ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:29.773322 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-e0882836-3c93-4d66-8014-21a2885415ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:29.773322 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:29.773322 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:29.773322 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:29.773322 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:29.773322 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:29.773322 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:29.773322 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:29.773322 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:29.773322 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:29.773322 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:29.773322 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:29.773322 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:29.773322 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:29.773322 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:29.773322 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:29.774548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:29.774548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:29.774548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:29.774548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:29.774548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:29.774548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:29.774548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:29.774548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:29.774548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:29.774548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:29.774548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:29.774548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:29.774548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:29.774548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:29.774548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:29.774548 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:29.775455 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:29.775455 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:29.775455 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:29.775455 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:29.775455 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:29.775455 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:29.775455 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:29.775455 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:29.775455 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:29.775455 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:29.775455 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:29.775455 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:29.778088 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 385/787] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:42:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:29.783497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0882836-3c93-4d66-8014-21a2885415ca None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:29.785308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d78f5f-ecb4-4ddf-8b28-edbda088de8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:29.787086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d78f5f-ecb4-4ddf-8b28-edbda088de8e None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:29.787382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d78f5f-ecb4-4ddf-8b28-edbda088de8e None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:29.787666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d78f5f-ecb4-4ddf-8b28-edbda088de8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:29.806445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:29.819018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d78f5f-ecb4-4ddf-8b28-edbda088de8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:29.823750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d78f5f-ecb4-4ddf-8b28-edbda088de8e None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:29.862226 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-11d78f5f-ecb4-4ddf-8b28-edbda088de8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:29.862226 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:29.862226 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:29.862226 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:29.862226 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:29.862226 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:29.862226 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:29.862226 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:29.862226 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:29.862226 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:29.862226 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:29.862226 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:29.862226 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:29.862226 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:29.862226 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:29.863767 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:29.863767 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:29.863767 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:29.863767 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:29.863767 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:29.863767 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:29.863767 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:29.863767 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:29.863767 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:29.863767 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:29.863767 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:29.863767 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:29.863767 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:29.863767 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:29.863767 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:29.863767 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:29.863767 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:29.865009 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:29.865009 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:29.865009 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:29.865009 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:29.865009 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 403/788] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:29.869743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d78f5f-ecb4-4ddf-8b28-edbda088de8e None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:29.902099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0882836-3c93-4d66-8014-21a2885415ca None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:29.940105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb53b91-e9b0-4677-8899-aff34d317d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:29.941033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb53b91-e9b0-4677-8899-aff34d317d9d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:29.942086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb53b91-e9b0-4677-8899-aff34d317d9d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:29.942510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb53b91-e9b0-4677-8899-aff34d317d9d None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:29.943568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb53b91-e9b0-4677-8899-aff34d317d9d None admin] RBAC: Adding query filter params (name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:29.944024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb53b91-e9b0-4677-8899-aff34d317d9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:29.945547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb53b91-e9b0-4677-8899-aff34d317d9d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:29.955244 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 386/789] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:42:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:29.961504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb53b91-e9b0-4677-8899-aff34d317d9d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:30.054278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:30.078765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb53b91-e9b0-4677-8899-aff34d317d9d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:30.118463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390ac7d-6ad5-45cb-86ba-755254fd5741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:30.119664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390ac7d-6ad5-45cb-86ba-755254fd5741 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:30.120228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390ac7d-6ad5-45cb-86ba-755254fd5741 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:30.120652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390ac7d-6ad5-45cb-86ba-755254fd5741 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:30.132762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390ac7d-6ad5-45cb-86ba-755254fd5741 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:30.134696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390ac7d-6ad5-45cb-86ba-755254fd5741 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:30.154631 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-2390ac7d-6ad5-45cb-86ba-755254fd5741 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:30.154631 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:30.154631 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:30.154631 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:30.154631 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:30.154631 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:30.154631 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:30.154631 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:30.154631 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:30.154631 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:30.154631 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:30.154631 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:30.154631 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:30.154631 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:30.154631 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:30.155578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:30.155578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:30.155578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:30.155578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:30.155578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:30.155578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:30.155578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:30.155578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:30.155578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:30.155578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:30.155578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:30.155578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:30.155578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:30.155578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:30.155578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:30.155578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:30.155578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:30.156488 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:30.156488 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:30.156488 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:30.156488 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:30.158925 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 387/790] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:30.165948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390ac7d-6ad5-45cb-86ba-755254fd5741 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:30.179997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d78f5f-ecb4-4ddf-8b28-edbda088de8e None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:30.210879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:30.249628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73b780-d076-4f6c-8e3a-4fc36f9c5726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:30.251437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73b780-d076-4f6c-8e3a-4fc36f9c5726 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:30.251868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73b780-d076-4f6c-8e3a-4fc36f9c5726 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:30.252316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73b780-d076-4f6c-8e3a-4fc36f9c5726 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:30.253192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73b780-d076-4f6c-8e3a-4fc36f9c5726 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:30.254172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73b780-d076-4f6c-8e3a-4fc36f9c5726 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:30.256261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73b780-d076-4f6c-8e3a-4fc36f9c5726 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:30.273253 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 404/791] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:30.274930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb199-b291-46af-9c9a-6c03d015eea3 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:30.275363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb199-b291-46af-9c9a-6c03d015eea3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:30.275796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb199-b291-46af-9c9a-6c03d015eea3 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:30.277641 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 405/792] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:30.278492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66beb199-b291-46af-9c9a-6c03d015eea3 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:30.328722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390ac7d-6ad5-45cb-86ba-755254fd5741 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:30.362299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:30.369046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746022c-a712-4434-9397-baffd2f4ca3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:30.372795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746022c-a712-4434-9397-baffd2f4ca3a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:30.372795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746022c-a712-4434-9397-baffd2f4ca3a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:30.372795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746022c-a712-4434-9397-baffd2f4ca3a None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:30.372795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746022c-a712-4434-9397-baffd2f4ca3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:30.372795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746022c-a712-4434-9397-baffd2f4ca3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:30.373197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746022c-a712-4434-9397-baffd2f4ca3a None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:30.390230 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 388/793] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:30.390230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98563f99-0c22-4cc0-96f2-e8d171b768bd None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:30.390230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98563f99-0c22-4cc0-96f2-e8d171b768bd None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:30.390230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98563f99-0c22-4cc0-96f2-e8d171b768bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:30.408480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-98563f99-0c22-4cc0-96f2-e8d171b768bd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:30.411033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-98563f99-0c22-4cc0-96f2-e8d171b768bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:30.421207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66beb199-b291-46af-9c9a-6c03d015eea3 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:30.465110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e67714-beef-4973-87fd-61e2c247a929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:30.465941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e67714-beef-4973-87fd-61e2c247a929 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:30.466228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e67714-beef-4973-87fd-61e2c247a929 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:30.466507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e67714-beef-4973-87fd-61e2c247a929 None admin] PATH_INFO: `/v3/users/swift` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:30.476468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e67714-beef-4973-87fd-61e2c247a929 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:30.478424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e67714-beef-4973-87fd-61e2c247a929 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:30.497148 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-62e67714-beef-4973-87fd-61e2c247a929 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 20:42:30.497148 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:30.497148 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:30.497148 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:30.497148 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:30.497148 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:30.497148 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:30.497148 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:30.497148 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:30.497148 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:30.497148 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:30.497148 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:30.497148 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:30.497148 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:30.497148 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:30.497148 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:30.498135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:30.499528 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:30.499528 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:30.499528 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:30.499528 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:30.499528 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:30.499528 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:30.499528 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:30.499528 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:30.499528 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 20:42:30.499528 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:30.500059 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 389/794] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:30.500059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98563f99-0c22-4cc0-96f2-e8d171b768bd None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:30.503238 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 406/795] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:42:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:30.509430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e67714-beef-4973-87fd-61e2c247a929 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:30.553819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:30.624390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98563f99-0c22-4cc0-96f2-e8d171b768bd None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:30.632544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e67714-beef-4973-87fd-61e2c247a929 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:30.645198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:30.660874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb9b3c-7d12-478a-a68e-555d8c80e1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:30.661942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb9b3c-7d12-478a-a68e-555d8c80e1bb None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:30.662379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb9b3c-7d12-478a-a68e-555d8c80e1bb None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:30.662806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb9b3c-7d12-478a-a68e-555d8c80e1bb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:30.669772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f093288-06a0-4622-bd29-ea0abdaed984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:30.669772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f093288-06a0-4622-bd29-ea0abdaed984 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:30.669772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f093288-06a0-4622-bd29-ea0abdaed984 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:30.670086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f093288-06a0-4622-bd29-ea0abdaed984 None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:30.670872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f093288-06a0-4622-bd29-ea0abdaed984 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:30.671183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f093288-06a0-4622-bd29-ea0abdaed984 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:30.671843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb9b3c-7d12-478a-a68e-555d8c80e1bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:30.672617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f093288-06a0-4622-bd29-ea0abdaed984 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:30.673644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb9b3c-7d12-478a-a68e-555d8c80e1bb None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:30.688855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-b5cb9b3c-7d12-478a-a68e-555d8c80e1bb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 20:42:30.688855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:30.688855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:30.688855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:30.688855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:30.688855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:30.688855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:30.688855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:30.688855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:30.688855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:30.688855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:30.688855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:30.688855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:30.688855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:30.688855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:30.688855 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:30.690058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:30.691123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:30.691123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:30.691123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:30.691123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:30.691123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:30.691123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:30.691123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:30.691123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:30.691123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 20:42:30.691123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:30.694123 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 390/796] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 20:42:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:30.695012 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 407/797] 10.222.0.33 () {56 vars in 1135 bytes} [Sat May 28 20:42:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:30.695502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb9b3c-7d12-478a-a68e-555d8c80e1bb None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:30.695887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f093288-06a0-4622-bd29-ea0abdaed984 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:30.795137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:30.802343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f093288-06a0-4622-bd29-ea0abdaed984 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:30.829475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb9b3c-7d12-478a-a68e-555d8c80e1bb None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:30.846445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760e703-0efa-4134-80e7-aff41a10e457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:30.847260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760e703-0efa-4134-80e7-aff41a10e457 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:30.847604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760e703-0efa-4134-80e7-aff41a10e457 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:30.848395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760e703-0efa-4134-80e7-aff41a10e457 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:30.862503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760e703-0efa-4134-80e7-aff41a10e457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:30.864328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760e703-0efa-4134-80e7-aff41a10e457 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:30.874095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ca8fc-3d4f-4c28-a554-017313f65776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:30.875327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ca8fc-3d4f-4c28-a554-017313f65776 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:30.875823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ca8fc-3d4f-4c28-a554-017313f65776 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:30.876309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ca8fc-3d4f-4c28-a554-017313f65776 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:30.885987 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-5760e703-0efa-4134-80e7-aff41a10e457 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:30.885987 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:30.885987 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:30.885987 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:30.885987 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:30.885987 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:30.885987 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:30.885987 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:30.885987 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:30.885987 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:30.885987 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:30.885987 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:30.885987 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:30.885987 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:30.885987 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:30.886881 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:30.886881 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:30.886881 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:30.886881 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:30.886881 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:30.886881 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:30.886881 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:30.886881 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:30.886881 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:30.886881 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:30.886881 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:30.886881 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:30.886881 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:30.886881 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:30.886881 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:30.886881 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:30.886881 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:30.887755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:30.887755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:30.887755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:30.887755 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:30.887755 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 408/798] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:30.893125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760e703-0efa-4134-80e7-aff41a10e457 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:30.895866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ca8fc-3d4f-4c28-a554-017313f65776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:30.904192 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-b76ca8fc-3d4f-4c28-a554-017313f65776 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:42:30.904192 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:30.904192 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:30.904192 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:30.904192 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:30.904192 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:30.904192 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:30.904192 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:30.904192 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:30.904192 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:30.904192 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:30.904192 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:30.904192 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:30.904192 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:30.904192 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:30.904192 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:30.905168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:30.905168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:30.905168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:30.905168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:30.905168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:30.905168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:30.905168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:30.905168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:30.905168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:30.905168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:30.905168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:30.905168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:30.905168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:30.905168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:30.905168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:30.905168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:30.906384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:30.906384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:30.906384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:30.906384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:30.906384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:30.906384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:30.906384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:30.906384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:30.906384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:30.906384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:30.906384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:42:30.906384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:30.917646 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 391/799] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:42:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:30.917646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ca8fc-3d4f-4c28-a554-017313f65776 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:30.965086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:31.029599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760e703-0efa-4134-80e7-aff41a10e457 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:31.031775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ca8fc-3d4f-4c28-a554-017313f65776 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:31.077053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192ee01-aa08-4875-8f02-cf47479a30a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:31.078111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192ee01-aa08-4875-8f02-cf47479a30a8 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:31.078220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf350df-11db-4f11-860d-dbfb599b1f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:31.078706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192ee01-aa08-4875-8f02-cf47479a30a8 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:31.078833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf350df-11db-4f11-860d-dbfb599b1f73 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:31.079086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf350df-11db-4f11-860d-dbfb599b1f73 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:31.079363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf350df-11db-4f11-860d-dbfb599b1f73 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:31.079684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192ee01-aa08-4875-8f02-cf47479a30a8 None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:31.080096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf350df-11db-4f11-860d-dbfb599b1f73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:31.080402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf350df-11db-4f11-860d-dbfb599b1f73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:31.081149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192ee01-aa08-4875-8f02-cf47479a30a8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:31.081652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192ee01-aa08-4875-8f02-cf47479a30a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:31.081921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf350df-11db-4f11-860d-dbfb599b1f73 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:31.084160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192ee01-aa08-4875-8f02-cf47479a30a8 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:31.101391 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 392/800] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:31.101391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf350df-11db-4f11-860d-dbfb599b1f73 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:31.109487 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 409/801] 10.222.0.33 () {56 vars in 1149 bytes} [Sat May 28 20:42:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:31.110427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192ee01-aa08-4875-8f02-cf47479a30a8 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:31.115811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:31.213608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:31.217424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf350df-11db-4f11-860d-dbfb599b1f73 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:31.250727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192ee01-aa08-4875-8f02-cf47479a30a8 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:31.257210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee5e53-ef08-4a52-883f-bc07176b19b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:31.258185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee5e53-ef08-4a52-883f-bc07176b19b3 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:31.258675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee5e53-ef08-4a52-883f-bc07176b19b3 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:31.259175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ee5e53-ef08-4a52-883f-bc07176b19b3 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:31.260661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee5e53-ef08-4a52-883f-bc07176b19b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:31.261177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee5e53-ef08-4a52-883f-bc07176b19b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:31.263641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ee5e53-ef08-4a52-883f-bc07176b19b3 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:31.277319 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 393/802] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:42:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:31.278487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee5e53-ef08-4a52-883f-bc07176b19b3 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:31.292638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6be30d-7442-4bbe-a612-3be1b880bb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:31.293443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6be30d-7442-4bbe-a612-3be1b880bb68 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:31.293724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6be30d-7442-4bbe-a612-3be1b880bb68 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:31.294048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6be30d-7442-4bbe-a612-3be1b880bb68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:31.301162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6be30d-7442-4bbe-a612-3be1b880bb68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:31.302659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6be30d-7442-4bbe-a612-3be1b880bb68 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:31.317548 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-3a6be30d-7442-4bbe-a612-3be1b880bb68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:31.317548 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:31.317548 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:31.317548 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:31.317548 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:31.317548 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:31.317548 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:31.317548 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:31.317548 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:31.317548 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:31.317548 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:31.317548 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:31.317548 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:31.317548 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:31.317548 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:31.318493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:31.318493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:31.318493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:31.318493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:31.318493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:31.318493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:31.318493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:31.318493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:31.318493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:31.318493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:31.318493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:31.318493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:31.318493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:31.318493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:31.318493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:31.318493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:31.318493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:31.319404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:31.319404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:31.319404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:31.319404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:31.319404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:31.319404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:31.319404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:31.319404 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:31.323841 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 410/803] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:31.324679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6be30d-7442-4bbe-a612-3be1b880bb68 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:31.395222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:31.422588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ee5e53-ef08-4a52-883f-bc07176b19b3 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:31.437740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6be30d-7442-4bbe-a612-3be1b880bb68 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:31.462830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e43cc28-2ad4-48cf-87d6-865f3f920d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:31.463736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43cc28-2ad4-48cf-87d6-865f3f920d9b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:31.463958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43cc28-2ad4-48cf-87d6-865f3f920d9b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:31.464234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43cc28-2ad4-48cf-87d6-865f3f920d9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:31.476950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e43cc28-2ad4-48cf-87d6-865f3f920d9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:31.478653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e43cc28-2ad4-48cf-87d6-865f3f920d9b None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:31.483666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf659bc7-78ce-41db-83cb-81416248d784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:31.484479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf659bc7-78ce-41db-83cb-81416248d784 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:31.485099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf659bc7-78ce-41db-83cb-81416248d784 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:31.485379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf659bc7-78ce-41db-83cb-81416248d784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:31.498278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf659bc7-78ce-41db-83cb-81416248d784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:31.498564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-2e43cc28-2ad4-48cf-87d6-865f3f920d9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:31.498564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:31.498564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:31.498564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:31.498564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:31.498564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:31.498564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:31.498564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:31.498564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:31.498564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:31.498564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:31.498564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:31.498564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:31.498564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:31.498564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:31.499448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:31.499448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:31.499448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:31.499448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:31.499448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:31.499448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:31.499448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:31.499448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:31.499448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:31.499448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:31.499448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:31.499448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:31.499448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:31.499448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:31.499448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:31.499448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:31.499448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:31.500362 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:31.500362 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:31.500362 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:31.500362 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:31.500362 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 394/804] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:31.500653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf659bc7-78ce-41db-83cb-81416248d784 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:31.500731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e43cc28-2ad4-48cf-87d6-865f3f920d9b None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:31.521413 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-cf659bc7-78ce-41db-83cb-81416248d784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:31.521413 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:31.521413 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:31.521413 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:31.521413 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:31.521413 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:31.521413 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:31.521413 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:31.521413 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:31.521413 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:31.521413 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:31.521413 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:31.521413 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:31.521413 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:31.521413 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:31.522443 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:31.522443 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:31.522443 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:31.522443 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:31.522443 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:31.522443 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:31.522443 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:31.522443 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:31.522443 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:31.522443 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:31.522443 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:31.522443 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:31.522443 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:31.522443 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:31.522443 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:31.522443 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:31.522443 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:31.523356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:31.523356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:31.523356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:31.523356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:31.525918 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 411/805] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:31.526959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf659bc7-78ce-41db-83cb-81416248d784 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:31.529062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:31.623101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e43cc28-2ad4-48cf-87d6-865f3f920d9b None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:31.636884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf659bc7-78ce-41db-83cb-81416248d784 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:31.654660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:31.664308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497839b-74a4-444b-93d0-f6423c102d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:31.664973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497839b-74a4-444b-93d0-f6423c102d71 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:31.665369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497839b-74a4-444b-93d0-f6423c102d71 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:31.665668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497839b-74a4-444b-93d0-f6423c102d71 None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:31.666462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497839b-74a4-444b-93d0-f6423c102d71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:31.666775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497839b-74a4-444b-93d0-f6423c102d71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:31.668219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2497839b-74a4-444b-93d0-f6423c102d71 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:31.672537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffc7c34-17cf-46a1-8426-bb00f7bbb964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:31.673196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc7c34-17cf-46a1-8426-bb00f7bbb964 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:31.673481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc7c34-17cf-46a1-8426-bb00f7bbb964 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:31.673761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffc7c34-17cf-46a1-8426-bb00f7bbb964 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:31.674523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc7c34-17cf-46a1-8426-bb00f7bbb964 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:31.674831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc7c34-17cf-46a1-8426-bb00f7bbb964 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:31.676739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffc7c34-17cf-46a1-8426-bb00f7bbb964 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:31.686294 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 395/806] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:31.686294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497839b-74a4-444b-93d0-f6423c102d71 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:31.694497 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 412/807] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:31.695855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fa21c-416e-4eee-8fce-8a573e5b0116 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:31.696145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fa21c-416e-4eee-8fce-8a573e5b0116 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:31.696422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0fa21c-416e-4eee-8fce-8a573e5b0116 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:31.699660 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 413/808] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:31.699660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fa21c-416e-4eee-8fce-8a573e5b0116 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:31.773319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:31.798423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2497839b-74a4-444b-93d0-f6423c102d71 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:31.807195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0fa21c-416e-4eee-8fce-8a573e5b0116 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:31.831965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de3836-8369-4912-a63a-e1a2f33709c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:31.832623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de3836-8369-4912-a63a-e1a2f33709c7 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:31.832906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de3836-8369-4912-a63a-e1a2f33709c7 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:31.833186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de3836-8369-4912-a63a-e1a2f33709c7 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:31.833939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de3836-8369-4912-a63a-e1a2f33709c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:31.834244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de3836-8369-4912-a63a-e1a2f33709c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:31.835611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6de3836-8369-4912-a63a-e1a2f33709c7 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:31.840701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3624bdf-ec74-429a-a1a5-a9c1b6147a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:31.841592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3624bdf-ec74-429a-a1a5-a9c1b6147a50 None admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:31.841911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3624bdf-ec74-429a-a1a5-a9c1b6147a50 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:31.842197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3624bdf-ec74-429a-a1a5-a9c1b6147a50 None admin] PATH_INFO: `/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/a01bba888a644eb18fc457aff578aa4b/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:31.844188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3624bdf-ec74-429a-a1a5-a9c1b6147a50 None admin] RBAC: Authorizing `identity:create_grant(project_id=3186f9d3c2b341afa8b098ebcaaf974b, user_id=a01bba888a644eb18fc457aff578aa4b, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:31.845622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3624bdf-ec74-429a-a1a5-a9c1b6147a50 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:31.846724 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 396/809] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:31.848027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de3836-8369-4912-a63a-e1a2f33709c7 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:31.896293 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 414/810] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:42:31 2022] PUT /identity/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/a01bba888a644eb18fc457aff578aa4b/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:42:31.897554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdede85-e02d-4cfe-ab0e-64c6cb34b28a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:31.897862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdede85-e02d-4cfe-ab0e-64c6cb34b28a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:31.898144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdede85-e02d-4cfe-ab0e-64c6cb34b28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:31.916482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-afdede85-e02d-4cfe-ab0e-64c6cb34b28a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:31.917762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-afdede85-e02d-4cfe-ab0e-64c6cb34b28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:31.954715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:31.958750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6de3836-8369-4912-a63a-e1a2f33709c7 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:31.998971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f32603-6efa-4dae-9034-ae4328339ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:31.999850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f32603-6efa-4dae-9034-ae4328339ec7 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:32.000138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f32603-6efa-4dae-9034-ae4328339ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:32.000415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f32603-6efa-4dae-9034-ae4328339ec7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:32.003528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:32.017063 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 415/811] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:32.017614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f32603-6efa-4dae-9034-ae4328339ec7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:32.017759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c4bd9-a612-493e-8a14-588ffa60c8c4 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:32.017759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c4bd9-a612-493e-8a14-588ffa60c8c4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:32.017759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c4bd9-a612-493e-8a14-588ffa60c8c4 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:32.017993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f32603-6efa-4dae-9034-ae4328339ec7 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:32.018080 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 416/812] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:32.018080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c4bd9-a612-493e-8a14-588ffa60c8c4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:32.031615 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-26f32603-6efa-4dae-9034-ae4328339ec7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:32.031615 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:32.031615 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:32.031615 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:32.031615 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:32.031615 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:32.031615 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:32.031615 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:32.031615 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:32.031615 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:32.031615 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:32.031615 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:32.031615 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:32.031615 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:32.031615 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:32.032563 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:32.032563 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:32.032563 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:32.032563 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:32.032563 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:32.032563 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:32.032563 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:32.032563 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:32.032563 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:32.032563 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:32.032563 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:32.032563 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:32.032563 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:32.032563 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:32.032563 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:32.032563 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:32.032563 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:32.033488 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:32.033488 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:32.033488 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:32.033488 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:32.033488 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:32.033488 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:32.033488 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:32.033488 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:32.033488 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 397/813] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:32.034696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56011731-8bb1-4c59-8046-449f6413277d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:32.034696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56011731-8bb1-4c59-8046-449f6413277d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:32.035503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56011731-8bb1-4c59-8046-449f6413277d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:32.055330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-56011731-8bb1-4c59-8046-449f6413277d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:32.056568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-56011731-8bb1-4c59-8046-449f6413277d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:32.140828 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 398/814] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:32.141772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56011731-8bb1-4c59-8046-449f6413277d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:32.162173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c4bd9-a612-493e-8a14-588ffa60c8c4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:32.199486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17366b57-5bb5-44fb-9cf4-cbea38d2af0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:32.200280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17366b57-5bb5-44fb-9cf4-cbea38d2af0f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:32.200562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17366b57-5bb5-44fb-9cf4-cbea38d2af0f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:32.200836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17366b57-5bb5-44fb-9cf4-cbea38d2af0f None admin] PATH_INFO: `/v3/users/nova` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:32.208574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17366b57-5bb5-44fb-9cf4-cbea38d2af0f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:32.210059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17366b57-5bb5-44fb-9cf4-cbea38d2af0f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:32.225187 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-17366b57-5bb5-44fb-9cf4-cbea38d2af0f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 20:42:32.225187 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:32.225187 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:32.225187 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:32.225187 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:32.225187 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:32.225187 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:32.225187 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:32.225187 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:32.225187 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:32.225187 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:32.225187 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:32.225187 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:32.225187 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:32.225187 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:32.225187 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:32.226182 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:32.227100 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:32.227100 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:32.227100 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:32.227100 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:32.227100 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:32.227100 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:32.227100 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:32.227100 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:32.227100 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 20:42:32.227100 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:32.227100 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 417/815] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 20:42:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:32.227685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17366b57-5bb5-44fb-9cf4-cbea38d2af0f None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:32.256022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:32.258838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56011731-8bb1-4c59-8046-449f6413277d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:32.291713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:32.305089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c230a-fd9f-4b38-b815-6ba7fe09c96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:32.306524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c230a-fd9f-4b38-b815-6ba7fe09c96c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:32.307010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c230a-fd9f-4b38-b815-6ba7fe09c96c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:32.307987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c230a-fd9f-4b38-b815-6ba7fe09c96c None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:32.309379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c230a-fd9f-4b38-b815-6ba7fe09c96c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:32.310050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c230a-fd9f-4b38-b815-6ba7fe09c96c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:32.312033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c230a-fd9f-4b38-b815-6ba7fe09c96c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:32.330819 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 399/816] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:32.330819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c230a-fd9f-4b38-b815-6ba7fe09c96c None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:32.338128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17366b57-5bb5-44fb-9cf4-cbea38d2af0f None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:32.374939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132ef33-8290-4fe7-97bf-893c9f54b89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:32.375737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132ef33-8290-4fe7-97bf-893c9f54b89b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:32.376022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132ef33-8290-4fe7-97bf-893c9f54b89b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:32.376301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132ef33-8290-4fe7-97bf-893c9f54b89b None admin] PATH_INFO: `/v3/services/network` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:32.376954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132ef33-8290-4fe7-97bf-893c9f54b89b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:32.378362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132ef33-8290-4fe7-97bf-893c9f54b89b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:32.392091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-c132ef33-8290-4fe7-97bf-893c9f54b89b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 28 20:42:32.392091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:32.392091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:32.392091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:32.392091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:32.392091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:32.392091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:32.392091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:32.392091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 20:42:32.392091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:32.392091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:32.392091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:32.392091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:32.392091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:32.392091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:32.392091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:32.392091 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:32.393109 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:32.393109 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:32.393109 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:32.393109 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:32.393109 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:32.393109 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:32.393109 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:32.393109 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:32.393109 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:32.393109 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:32.393109 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:32.393109 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:32.393109 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:32.393109 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:32.393109 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:32.393109 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:32.393977 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:32.393977 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:32.393977 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:32.393977 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:32.393977 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:32.393977 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:32.393977 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:32.393977 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:32.393977 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:32.393977 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:32.393977 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:32.393977 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:32.393977 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 28 20:42:32.393977 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:32.393977 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 418/817] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:32 2022] GET /identity/v3/services/network => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:32.395315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132ef33-8290-4fe7-97bf-893c9f54b89b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:32.441265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c230a-fd9f-4b38-b815-6ba7fe09c96c None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:32.479421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206fa70-7f2d-47a0-bed5-8e4148a928df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:32.480392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206fa70-7f2d-47a0-bed5-8e4148a928df None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:32.480595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206fa70-7f2d-47a0-bed5-8e4148a928df None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:32.480872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206fa70-7f2d-47a0-bed5-8e4148a928df None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:32.481915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206fa70-7f2d-47a0-bed5-8e4148a928df None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:32.482957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206fa70-7f2d-47a0-bed5-8e4148a928df None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:32.495381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-4206fa70-7f2d-47a0-bed5-8e4148a928df None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 20:42:32.495381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:32.495381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:32.495381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:32.495381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:32.495381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:32.495381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:32.495381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:32.495381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 20:42:32.495381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:32.495381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:32.495381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:32.495381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:32.495381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:32.495381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:32.495381 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:32.496426 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:32.496426 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:32.496426 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:32.496426 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:32.496426 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:32.496426 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:32.496426 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:32.496426 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:32.496426 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:32.496426 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:32.496426 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:32.496426 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:32.496426 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:32.496426 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:32.496426 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:32.496426 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:32.497258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:32.497258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:32.497258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:32.497258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:32.497258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:32.497258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:32.497258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:32.497258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:32.497258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:32.497258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:32.497258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:32.497258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:32.497258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:32.497258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 28 20:42:32.497258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:32.497258 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 400/818] 10.222.0.33 () {56 vars in 1129 bytes} [Sat May 28 20:42:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:32.498131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206fa70-7f2d-47a0-bed5-8e4148a928df None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:32.506621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:32.516788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132ef33-8290-4fe7-97bf-893c9f54b89b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:32.549285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:32.550362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ce648-dec6-47a4-84e4-f60d5bdb5618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:32.551060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ce648-dec6-47a4-84e4-f60d5bdb5618 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:32.551342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ce648-dec6-47a4-84e4-f60d5bdb5618 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:32.551617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213ce648-dec6-47a4-84e4-f60d5bdb5618 None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:32.552454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ce648-dec6-47a4-84e4-f60d5bdb5618 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:32.552764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ce648-dec6-47a4-84e4-f60d5bdb5618 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:32.554221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213ce648-dec6-47a4-84e4-f60d5bdb5618 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:32.571886 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 419/819] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 20:42:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:32.573583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ce648-dec6-47a4-84e4-f60d5bdb5618 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:32.603867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206fa70-7f2d-47a0-bed5-8e4148a928df None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:32.644141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937cb8e-99ad-46bf-bc9e-b7a8b05438fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:32.645048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937cb8e-99ad-46bf-bc9e-b7a8b05438fe None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:32.645330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937cb8e-99ad-46bf-bc9e-b7a8b05438fe None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:32.645597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937cb8e-99ad-46bf-bc9e-b7a8b05438fe None admin] PATH_INFO: `/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/c1c1ea9ba4c740b3b7ed67c8335bfad2/roles/4db40e559d064cf6b4f061e785b42dbb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:32.661773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937cb8e-99ad-46bf-bc9e-b7a8b05438fe None admin] RBAC: Authorizing `identity:create_grant(project_id=3186f9d3c2b341afa8b098ebcaaf974b, user_id=c1c1ea9ba4c740b3b7ed67c8335bfad2, role_id=4db40e559d064cf6b4f061e785b42dbb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:32.663253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7937cb8e-99ad-46bf-bc9e-b7a8b05438fe None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:32.686495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213ce648-dec6-47a4-84e4-f60d5bdb5618 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:32.713681 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 401/820] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:42:32 2022] PUT /identity/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/c1c1ea9ba4c740b3b7ed67c8335bfad2/roles/4db40e559d064cf6b4f061e785b42dbb => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:42:32.722257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937cb8e-99ad-46bf-bc9e-b7a8b05438fe None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:32.729868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19445ccc-5d38-4497-bb1d-c82a87cb18be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:32.730714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19445ccc-5d38-4497-bb1d-c82a87cb18be None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:32.731002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19445ccc-5d38-4497-bb1d-c82a87cb18be None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:32.731282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19445ccc-5d38-4497-bb1d-c82a87cb18be None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:32.732079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19445ccc-5d38-4497-bb1d-c82a87cb18be None admin] RBAC: Adding query filter params (name=network) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:32.732398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19445ccc-5d38-4497-bb1d-c82a87cb18be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:32.733815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19445ccc-5d38-4497-bb1d-c82a87cb18be None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:32.740435 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 420/821] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 20:42:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:32.741336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19445ccc-5d38-4497-bb1d-c82a87cb18be None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:32.757673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:32.842731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:32.846784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7937cb8e-99ad-46bf-bc9e-b7a8b05438fe None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:32.873420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19445ccc-5d38-4497-bb1d-c82a87cb18be None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:32.888095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28fab9-fe9b-4269-909d-c5dce803134a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:32.888947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28fab9-fe9b-4269-909d-c5dce803134a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:32.889327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28fab9-fe9b-4269-909d-c5dce803134a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:32.889697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28fab9-fe9b-4269-909d-c5dce803134a None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:32.890834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28fab9-fe9b-4269-909d-c5dce803134a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:32.891239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28fab9-fe9b-4269-909d-c5dce803134a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:32.892843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28fab9-fe9b-4269-909d-c5dce803134a None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:32.904109 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 402/822] 10.222.0.33 () {56 vars in 1124 bytes} [Sat May 28 20:42:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:32.905246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28fab9-fe9b-4269-909d-c5dce803134a None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:32.924372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabeffda-7a25-49ff-92c1-a9528fa96115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:32.925040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeffda-7a25-49ff-92c1-a9528fa96115 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:32.925330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeffda-7a25-49ff-92c1-a9528fa96115 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:32.925952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabeffda-7a25-49ff-92c1-a9528fa96115 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:33.006615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabeffda-7a25-49ff-92c1-a9528fa96115 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:33.008427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabeffda-7a25-49ff-92c1-a9528fa96115 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:33.027195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:33.038598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28fab9-fe9b-4269-909d-c5dce803134a None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:33.081111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed52ca-3027-42c2-ba29-ff0a71bf4f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:33.082630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed52ca-3027-42c2-ba29-ff0a71bf4f03 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:33.083066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed52ca-3027-42c2-ba29-ff0a71bf4f03 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:33.083489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed52ca-3027-42c2-ba29-ff0a71bf4f03 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:33.084375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed52ca-3027-42c2-ba29-ff0a71bf4f03 None admin] RBAC: Adding query filter params (type=network) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:33.084832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed52ca-3027-42c2-ba29-ff0a71bf4f03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:33.086398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed52ca-3027-42c2-ba29-ff0a71bf4f03 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:33.092068 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 403/823] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 20:42:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:33.098072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed52ca-3027-42c2-ba29-ff0a71bf4f03 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:33.129658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:33.152472 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 421/824] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:42:32 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 20:42:33.153723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabeffda-7a25-49ff-92c1-a9528fa96115 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:33.212942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed52ca-3027-42c2-ba29-ff0a71bf4f03 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:33.250637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0594998-e976-40d4-b658-e7ff731f1403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:33.251615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0594998-e976-40d4-b658-e7ff731f1403 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:33.252311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0594998-e976-40d4-b658-e7ff731f1403 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:33.253421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0594998-e976-40d4-b658-e7ff731f1403 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:33.254943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0594998-e976-40d4-b658-e7ff731f1403 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:33.255802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0594998-e976-40d4-b658-e7ff731f1403 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:33.257997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0594998-e976-40d4-b658-e7ff731f1403 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:33.264731 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 404/825] 10.222.0.33 () {56 vars in 1124 bytes} [Sat May 28 20:42:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:33.266114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0594998-e976-40d4-b658-e7ff731f1403 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:33.288994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabeffda-7a25-49ff-92c1-a9528fa96115 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:33.307204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:33.337814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f667a-4bd9-4755-bacf-a8fd85ac0559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:33.338808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f667a-4bd9-4755-bacf-a8fd85ac0559 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:33.339162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f667a-4bd9-4755-bacf-a8fd85ac0559 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:33.339531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f667a-4bd9-4755-bacf-a8fd85ac0559 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:33.340605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f667a-4bd9-4755-bacf-a8fd85ac0559 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:33.342780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f667a-4bd9-4755-bacf-a8fd85ac0559 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:33.375069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:33.385677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0594998-e976-40d4-b658-e7ff731f1403 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:33.422925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2c685-6fda-4018-97c1-231e5cd29cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:33.424152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2c685-6fda-4018-97c1-231e5cd29cc0 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:33.424615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2c685-6fda-4018-97c1-231e5cd29cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:33.425060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2c685-6fda-4018-97c1-231e5cd29cc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:33.427987 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 422/826] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 20:42:33 2022] POST /identity/v3/endpoints => generated 327 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:42:33.429154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f667a-4bd9-4755-bacf-a8fd85ac0559 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:33.440960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e2c685-6fda-4018-97c1-231e5cd29cc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:33.442994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e2c685-6fda-4018-97c1-231e5cd29cc0 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:33.471684 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-21e2c685-6fda-4018-97c1-231e5cd29cc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:33.471684 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:33.471684 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:33.471684 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:33.471684 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:33.471684 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:33.471684 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:33.471684 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:33.471684 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:33.471684 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:33.471684 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:33.471684 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:33.471684 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:33.471684 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:33.471684 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:33.473254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:33.473254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:33.473254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:33.473254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:33.473254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:33.473254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:33.473254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:33.473254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:33.473254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:33.473254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:33.473254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:33.473254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:33.473254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:33.473254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:33.473254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:33.473254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:33.473254 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:33.474220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:33.474220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:33.474220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:33.474220 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:33.476617 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 405/827] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:33.482419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2c685-6fda-4018-97c1-231e5cd29cc0 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:33.659575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2c685-6fda-4018-97c1-231e5cd29cc0 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:33.685566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f667a-4bd9-4755-bacf-a8fd85ac0559 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:33.711121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2932c-42b3-4d54-a0ad-448e104c1887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:33.712130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2932c-42b3-4d54-a0ad-448e104c1887 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:33.712414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2932c-42b3-4d54-a0ad-448e104c1887 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:33.712695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f2932c-42b3-4d54-a0ad-448e104c1887 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:33.713485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f2932c-42b3-4d54-a0ad-448e104c1887 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:33.713863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f2932c-42b3-4d54-a0ad-448e104c1887 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:33.716067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f2932c-42b3-4d54-a0ad-448e104c1887 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:33.732506 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 406/828] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:33.732755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ceb5d1-d2c2-4f03-b35c-4e18f4b1da0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:33.733494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ceb5d1-d2c2-4f03-b35c-4e18f4b1da0f None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:33.737994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ceb5d1-d2c2-4f03-b35c-4e18f4b1da0f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:33.737994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ceb5d1-d2c2-4f03-b35c-4e18f4b1da0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:33.737994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ceb5d1-d2c2-4f03-b35c-4e18f4b1da0f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:33.737994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ceb5d1-d2c2-4f03-b35c-4e18f4b1da0f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:33.739264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2932c-42b3-4d54-a0ad-448e104c1887 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:33.743171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:33.815117 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 423/829] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 20:42:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:42:33.865862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f2932c-42b3-4d54-a0ad-448e104c1887 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:33.911283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b90020-a7f9-4d16-a320-33f65233509b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:33.912004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b90020-a7f9-4d16-a320-33f65233509b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:33.912287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b90020-a7f9-4d16-a320-33f65233509b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:33.912651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b90020-a7f9-4d16-a320-33f65233509b None admin] PATH_INFO: `/v3/projects/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:33.920378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b90020-a7f9-4d16-a320-33f65233509b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:33.921974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b90020-a7f9-4d16-a320-33f65233509b None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:33.938245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-24b90020-a7f9-4d16-a320-33f65233509b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:33.938245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:33.938245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:33.938245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:33.938245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:33.938245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:33.938245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:33.938245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:33.938245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:33.938245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:33.938245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:33.938245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:33.938245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:33.938245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:33.938245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:33.939511 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:33.939511 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:33.939511 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:33.939511 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:33.939511 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:33.939511 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:33.939511 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:33.939511 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:33.939511 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:33.939511 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:33.939511 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:33.939511 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:33.939511 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:33.939511 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:33.939511 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:33.939511 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:33.939511 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:33.940625 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:33.940625 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:33.940625 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:33.940625 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:33.940625 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:33.940625 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:33.940625 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:33.940625 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:33.940625 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 407/830] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:33.947067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ceb5d1-d2c2-4f03-b35c-4e18f4b1da0f None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:33.972101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:34.064370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ceb5d1-d2c2-4f03-b35c-4e18f4b1da0f None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:34.103610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b671c-c2eb-44dd-9012-81e0c34c652a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:34.104384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b671c-c2eb-44dd-9012-81e0c34c652a None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:34.104745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b671c-c2eb-44dd-9012-81e0c34c652a None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:34.105099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b671c-c2eb-44dd-9012-81e0c34c652a None admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:34.106011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b671c-c2eb-44dd-9012-81e0c34c652a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:34.106399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b671c-c2eb-44dd-9012-81e0c34c652a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:34.107926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1b671c-c2eb-44dd-9012-81e0c34c652a None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:34.121238 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 424/831] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:34.127015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b90020-a7f9-4d16-a320-33f65233509b None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:34.189207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:34.234586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b90020-a7f9-4d16-a320-33f65233509b None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:34.270198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42258377-e2b5-467f-a0eb-6d304286b5b2 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:34.270559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42258377-e2b5-467f-a0eb-6d304286b5b2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:34.270752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42258377-e2b5-467f-a0eb-6d304286b5b2 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:34.272384 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 425/832] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:42:34.272616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788030a-cfe9-4f79-8f6e-d741410c07d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:34.273282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788030a-cfe9-4f79-8f6e-d741410c07d1 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:34.273673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788030a-cfe9-4f79-8f6e-d741410c07d1 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:34.273995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3788030a-cfe9-4f79-8f6e-d741410c07d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:34.274838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788030a-cfe9-4f79-8f6e-d741410c07d1 None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=cfdebb5e6b8d4e2d86f0a03a30c21671) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:34.275213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788030a-cfe9-4f79-8f6e-d741410c07d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:34.277057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3788030a-cfe9-4f79-8f6e-d741410c07d1 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:34.277434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d55fb-5ba3-4fa2-84f5-7d9d52e1c8c1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:34.277723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d55fb-5ba3-4fa2-84f5-7d9d52e1c8c1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:34.278044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d55fb-5ba3-4fa2-84f5-7d9d52e1c8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:34.284471 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 408/833] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:34 2022] GET /identity/v3/role_assignments?user.id=cfdebb5e6b8d4e2d86f0a03a30c21671&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=7805b93b32f1472f8365936d4d57d49a => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:34.296576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ce3d55fb-5ba3-4fa2-84f5-7d9d52e1c8c1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:34.297803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ce3d55fb-5ba3-4fa2-84f5-7d9d52e1c8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:34.374946 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 426/834] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:42:34.401816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788030a-cfe9-4f79-8f6e-d741410c07d1 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:34.514817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3788030a-cfe9-4f79-8f6e-d741410c07d1 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:34.539086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:34.552132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee42ea-5260-454f-a960-602df0892533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:34.553148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee42ea-5260-454f-a960-602df0892533 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:34.553436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee42ea-5260-454f-a960-602df0892533 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:34.553706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee42ea-5260-454f-a960-602df0892533 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:34.564464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcee42ea-5260-454f-a960-602df0892533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:34.573964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-fcee42ea-5260-454f-a960-602df0892533 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:42:34.573964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:34.573964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:34.573964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:34.573964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:34.573964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:34.573964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:34.573964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:34.573964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:34.573964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:34.573964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:34.573964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:34.573964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:34.573964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:34.573964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:34.573964 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:34.574992 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:34.574992 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:34.574992 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:34.574992 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:34.574992 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:34.574992 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:34.574992 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:34.574992 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:34.574992 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:34.574992 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:34.574992 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:34.574992 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:34.574992 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:34.574992 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:34.574992 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:34.574992 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:34.575917 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:34.575917 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:34.575917 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:34.575917 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:34.575917 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:34.575917 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:34.575917 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:34.575917 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:34.575917 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:34.575917 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:34.575917 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:42:34.575917 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:34.575917 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 409/835] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:42:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:34.581808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d55fb-5ba3-4fa2-84f5-7d9d52e1c8c1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:34.593711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:34.686992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d55fb-5ba3-4fa2-84f5-7d9d52e1c8c1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:34.724237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35924944-a858-46fb-8a3a-595d5e5a4a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:34.724912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35924944-a858-46fb-8a3a-595d5e5a4a1d None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:34.725193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35924944-a858-46fb-8a3a-595d5e5a4a1d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:34.725471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35924944-a858-46fb-8a3a-595d5e5a4a1d None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:34.726259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35924944-a858-46fb-8a3a-595d5e5a4a1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:34.726567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35924944-a858-46fb-8a3a-595d5e5a4a1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:34.727941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35924944-a858-46fb-8a3a-595d5e5a4a1d None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:34.736810 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 427/836] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:42:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:34.742825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee42ea-5260-454f-a960-602df0892533 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:34.849705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee42ea-5260-454f-a960-602df0892533 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:34.886884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5243ed-5d30-4094-abf2-cb04abaae08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:34.887796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5243ed-5d30-4094-abf2-cb04abaae08e None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:34.888084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5243ed-5d30-4094-abf2-cb04abaae08e None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:34.888363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5243ed-5d30-4094-abf2-cb04abaae08e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:34.899891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5243ed-5d30-4094-abf2-cb04abaae08e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:34.905223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5243ed-5d30-4094-abf2-cb04abaae08e None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:34.919603 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-cc5243ed-5d30-4094-abf2-cb04abaae08e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:34.919603 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:34.919603 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:34.919603 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:34.919603 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:34.919603 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:34.919603 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:34.919603 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:34.919603 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:34.919603 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:34.919603 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:34.919603 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:34.919603 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:34.919603 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:34.919603 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:34.920554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:34.920554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:34.920554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:34.920554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:34.920554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:34.920554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:34.920554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:34.920554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:34.920554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:34.920554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:34.920554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:34.920554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:34.920554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:34.920554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:34.920554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:34.920554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:34.920554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:34.922096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:34.922096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:34.922096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:34.922096 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:34.922096 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 410/837] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:34.926944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35924944-a858-46fb-8a3a-595d5e5a4a1d None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:35.006215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:35.038763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35924944-a858-46fb-8a3a-595d5e5a4a1d None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:35.076210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57e83a-a8a7-4a06-ae91-19c65764784d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:35.076886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57e83a-a8a7-4a06-ae91-19c65764784d None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:35.077172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57e83a-a8a7-4a06-ae91-19c65764784d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:35.077450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57e83a-a8a7-4a06-ae91-19c65764784d None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:35.078215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57e83a-a8a7-4a06-ae91-19c65764784d None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:35.078526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57e83a-a8a7-4a06-ae91-19c65764784d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:35.079942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57e83a-a8a7-4a06-ae91-19c65764784d None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:35.092590 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 428/838] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:35.098556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5243ed-5d30-4094-abf2-cb04abaae08e None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:35.137609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:35.168069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142006d-7b08-4def-9459-bddd97c4030c None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:35.168741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142006d-7b08-4def-9459-bddd97c4030c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:35.169238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142006d-7b08-4def-9459-bddd97c4030c None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:35.174841 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 429/839] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:35.177158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b311184-7644-42f1-84a1-e0d008f35bfd None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:35.177681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b311184-7644-42f1-84a1-e0d008f35bfd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:35.178257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b311184-7644-42f1-84a1-e0d008f35bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:35.195806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8b311184-7644-42f1-84a1-e0d008f35bfd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:35.197362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8b311184-7644-42f1-84a1-e0d008f35bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:35.215751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5243ed-5d30-4094-abf2-cb04abaae08e None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:35.264553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf08fa-9a2c-4251-8550-fdad4cd67996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:35.265932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf08fa-9a2c-4251-8550-fdad4cd67996 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:35.266210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf08fa-9a2c-4251-8550-fdad4cd67996 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:35.266491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf08fa-9a2c-4251-8550-fdad4cd67996 None admin] PATH_INFO: `/v3/users/swift` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:35.276399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf08fa-9a2c-4251-8550-fdad4cd67996 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:35.276399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf08fa-9a2c-4251-8550-fdad4cd67996 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:35.276836 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 430/840] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:42:35.290328 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-4abf08fa-9a2c-4251-8550-fdad4cd67996 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 28 20:42:35.290328 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:35.290328 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:35.290328 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:35.290328 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:35.290328 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:35.290328 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:35.290328 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:35.290328 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:35.290328 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:35.290328 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:35.290328 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:35.290328 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:35.290328 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:35.290328 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:35.290328 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:35.291268 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:35.292749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:35.292749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:35.292749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:35.292749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:35.292749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:35.292749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:35.292749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:35.292749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:35.292749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 28 20:42:35.292749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:35.292749 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 411/841] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:42:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:35.297228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b311184-7644-42f1-84a1-e0d008f35bfd None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:35.302291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf08fa-9a2c-4251-8550-fdad4cd67996 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:35.388152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:35.406513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:35.413451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf08fa-9a2c-4251-8550-fdad4cd67996 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:35.428694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b311184-7644-42f1-84a1-e0d008f35bfd None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:35.449701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0ab56-f608-4c85-b230-b099272b6dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:35.450554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0ab56-f608-4c85-b230-b099272b6dca None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:35.450842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0ab56-f608-4c85-b230-b099272b6dca None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:35.451121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0ab56-f608-4c85-b230-b099272b6dca None admin] PATH_INFO: `/v3/roles/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:35.463940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0ab56-f608-4c85-b230-b099272b6dca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:35.472157 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-d6d0ab56-f608-4c85-b230-b099272b6dca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:35.472157 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:35.472157 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:35.472157 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:35.472157 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:35.472157 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:35.472157 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:35.472157 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:35.472157 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:35.472157 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:35.472157 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:35.472157 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:35.472157 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:35.472157 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:35.472157 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:35.472157 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:35.473633 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:35.473633 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:35.473633 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:35.473633 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:35.473633 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:35.473633 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:35.473633 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:35.473633 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:35.473633 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:35.473633 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:35.473633 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:35.473633 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:35.473633 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:35.473633 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:35.473633 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:35.473633 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:35.474830 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:35.474830 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:35.474830 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:35.474830 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:35.474830 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:35.474830 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:35.474830 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:35.474830 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:35.474830 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:35.474830 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:35.474830 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:42:35.474830 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:35.474830 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 412/842] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:42:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:42:35.476023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f3982-2c01-4129-a3a5-95d43dc4e6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:35.476780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f3982-2c01-4129-a3a5-95d43dc4e6f3 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:35.477137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f3982-2c01-4129-a3a5-95d43dc4e6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:35.477484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f3982-2c01-4129-a3a5-95d43dc4e6f3 None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:35.485886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f3982-2c01-4129-a3a5-95d43dc4e6f3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:35.485886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f3982-2c01-4129-a3a5-95d43dc4e6f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:35.486080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0ab56-f608-4c85-b230-b099272b6dca None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:35.486176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f3982-2c01-4129-a3a5-95d43dc4e6f3 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:35.506017 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 431/843] 10.222.0.33 () {56 vars in 1135 bytes} [Sat May 28 20:42:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:35.509134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f3982-2c01-4129-a3a5-95d43dc4e6f3 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:35.598864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0ab56-f608-4c85-b230-b099272b6dca None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:35.636835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943dfa0-e44d-455b-8284-7c918ee5b182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:35.637667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943dfa0-e44d-455b-8284-7c918ee5b182 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:35.638031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943dfa0-e44d-455b-8284-7c918ee5b182 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:35.638366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943dfa0-e44d-455b-8284-7c918ee5b182 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:35.639672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943dfa0-e44d-455b-8284-7c918ee5b182 None admin] RBAC: Adding query filter params (name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:35.639986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943dfa0-e44d-455b-8284-7c918ee5b182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:35.642128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943dfa0-e44d-455b-8284-7c918ee5b182 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:35.646066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f3982-2c01-4129-a3a5-95d43dc4e6f3 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:35.657866 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 413/844] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:42:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:35.659131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943dfa0-e44d-455b-8284-7c918ee5b182 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:35.673348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:35.682461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:35.694153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24881131-1d3c-4873-84e7-acef7aefd547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:35.695989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24881131-1d3c-4873-84e7-acef7aefd547 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:35.696423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24881131-1d3c-4873-84e7-acef7aefd547 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:35.696855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24881131-1d3c-4873-84e7-acef7aefd547 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:35.777669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24881131-1d3c-4873-84e7-acef7aefd547 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:35.779403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24881131-1d3c-4873-84e7-acef7aefd547 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:35.782947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943dfa0-e44d-455b-8284-7c918ee5b182 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:35.818890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c206c9-4490-45b7-8a2d-4c38ec975e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:35.819861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c206c9-4490-45b7-8a2d-4c38ec975e2f None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:35.820270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c206c9-4490-45b7-8a2d-4c38ec975e2f None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:35.820642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c206c9-4490-45b7-8a2d-4c38ec975e2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:35.832759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c206c9-4490-45b7-8a2d-4c38ec975e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:35.834388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c206c9-4490-45b7-8a2d-4c38ec975e2f None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:35.852826 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-74c206c9-4490-45b7-8a2d-4c38ec975e2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:35.852826 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:35.852826 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:35.852826 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:35.852826 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:35.852826 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:35.852826 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:35.852826 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:35.852826 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:35.852826 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:35.852826 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:35.852826 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:35.852826 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:35.852826 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:35.852826 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:35.854026 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:35.854026 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:35.854026 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:35.854026 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:35.854026 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:35.854026 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:35.854026 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:35.854026 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:35.854026 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:35.854026 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:35.854026 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:35.854026 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:35.854026 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:35.854026 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:35.854026 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:35.854026 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:35.854026 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:35.854919 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:35.854919 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:35.854919 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:35.854919 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:35.859561 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 414/845] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:42:35.866272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c206c9-4490-45b7-8a2d-4c38ec975e2f None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:35.895919 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 432/846] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:42:35 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 28 20:42:35.897058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24881131-1d3c-4873-84e7-acef7aefd547 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:35.919051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:35.933246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:35.968535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c206c9-4490-45b7-8a2d-4c38ec975e2f None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:36.003496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1bb47-98e7-4b55-9670-3845f8b64b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:36.004765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1bb47-98e7-4b55-9670-3845f8b64b8c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:36.005021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24881131-1d3c-4873-84e7-acef7aefd547 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:36.005138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1bb47-98e7-4b55-9670-3845f8b64b8c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:36.005311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1bb47-98e7-4b55-9670-3845f8b64b8c None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:36.006073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1bb47-98e7-4b55-9670-3845f8b64b8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:36.006386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1bb47-98e7-4b55-9670-3845f8b64b8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:36.007896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1bb47-98e7-4b55-9670-3845f8b64b8c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:36.027133 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 415/847] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:36.028480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1bb47-98e7-4b55-9670-3845f8b64b8c None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:36.046695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be7080-3fb2-4861-971a-f9a8a5e284d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:36.047859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be7080-3fb2-4861-971a-f9a8a5e284d5 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:36.048354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be7080-3fb2-4861-971a-f9a8a5e284d5 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:36.048825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be7080-3fb2-4861-971a-f9a8a5e284d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:36.060299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be7080-3fb2-4861-971a-f9a8a5e284d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:36.062146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be7080-3fb2-4861-971a-f9a8a5e284d5 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:36.090696 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-f5be7080-3fb2-4861-971a-f9a8a5e284d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:36.090696 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:36.090696 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:36.090696 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:36.090696 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:36.090696 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:36.090696 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:36.090696 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:36.090696 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:36.090696 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:36.090696 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:36.090696 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:36.090696 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:36.090696 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:36.090696 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:36.091656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:36.091656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:36.091656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:36.091656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:36.091656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:36.091656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:36.091656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:36.091656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:36.091656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:36.091656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:36.091656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:36.091656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:36.091656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:36.091656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:36.091656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:36.091656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:36.091656 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:36.092536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:36.092536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:36.092536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:36.092536 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:36.100961 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 433/848] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:36.101512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be7080-3fb2-4861-971a-f9a8a5e284d5 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:36.159916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1bb47-98e7-4b55-9670-3845f8b64b8c None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:36.180697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:36.197540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:36.208738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a061b-13bb-4b82-ba5f-aa3ccd3b13e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:36.209988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a061b-13bb-4b82-ba5f-aa3ccd3b13e0 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:36.210467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a061b-13bb-4b82-ba5f-aa3ccd3b13e0 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:36.210959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a061b-13bb-4b82-ba5f-aa3ccd3b13e0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:36.214066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be7080-3fb2-4861-971a-f9a8a5e284d5 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:36.220499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a061b-13bb-4b82-ba5f-aa3ccd3b13e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:36.223422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a061b-13bb-4b82-ba5f-aa3ccd3b13e0 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:36.243540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-071a061b-13bb-4b82-ba5f-aa3ccd3b13e0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 20:42:36.243540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:36.243540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:36.243540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:36.243540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:36.243540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:36.243540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:36.243540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:36.243540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:36.243540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:36.243540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:36.243540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:36.243540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:36.243540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:36.243540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:36.243540 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:36.244549 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:36.245598 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:36.245598 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:36.245598 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:36.245598 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:36.245598 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:36.245598 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:36.245598 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:36.245598 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:36.245598 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 20:42:36.245598 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:36.251835 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 416/849] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 20:42:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:42:36.252927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a061b-13bb-4b82-ba5f-aa3ccd3b13e0 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:36.261690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9374af-7f7b-4095-a463-09549cba4d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:36.262417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9374af-7f7b-4095-a463-09549cba4d58 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:36.262703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9374af-7f7b-4095-a463-09549cba4d58 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:36.262983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9374af-7f7b-4095-a463-09549cba4d58 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:36.263730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9374af-7f7b-4095-a463-09549cba4d58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:36.264043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9374af-7f7b-4095-a463-09549cba4d58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:36.265693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9374af-7f7b-4095-a463-09549cba4d58 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:36.283051 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 434/850] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:36.284740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9374af-7f7b-4095-a463-09549cba4d58 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:36.371009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a061b-13bb-4b82-ba5f-aa3ccd3b13e0 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:36.400689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9374af-7f7b-4095-a463-09549cba4d58 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:36.406669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080784d-8be5-4e43-8621-5e5b9771ba5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:36.407343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080784d-8be5-4e43-8621-5e5b9771ba5c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:36.407629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080784d-8be5-4e43-8621-5e5b9771ba5c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:36.408222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080784d-8be5-4e43-8621-5e5b9771ba5c None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:36.408726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080784d-8be5-4e43-8621-5e5b9771ba5c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:36.409031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080784d-8be5-4e43-8621-5e5b9771ba5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:36.410584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080784d-8be5-4e43-8621-5e5b9771ba5c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:36.433341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:36.433999 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 417/851] 10.222.0.33 () {56 vars in 1149 bytes} [Sat May 28 20:42:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:36.435508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dafcf7-841d-42ab-bbeb-c4e03d787eb0 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:36.435907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dafcf7-841d-42ab-bbeb-c4e03d787eb0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:36.436448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dafcf7-841d-42ab-bbeb-c4e03d787eb0 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:36.437215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b01414-375d-47bd-bc0d-65a31869acb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:36.441910 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 418/852] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:36.442043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01414-375d-47bd-bc0d-65a31869acb1 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:36.442043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01414-375d-47bd-bc0d-65a31869acb1 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:36.442043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01414-375d-47bd-bc0d-65a31869acb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:36.447032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dafcf7-841d-42ab-bbeb-c4e03d787eb0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:36.448778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b01414-375d-47bd-bc0d-65a31869acb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:36.450291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b01414-375d-47bd-bc0d-65a31869acb1 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:36.465021 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-26b01414-375d-47bd-bc0d-65a31869acb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:36.465021 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:36.465021 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:36.465021 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:36.465021 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:36.465021 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:36.465021 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:36.465021 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:36.465021 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:36.465021 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:36.465021 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:36.465021 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:36.465021 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:36.465021 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:36.465021 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:36.466154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:36.466154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:36.466154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:36.466154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:36.466154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:36.466154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:36.466154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:36.466154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:36.466154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:36.466154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:36.466154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:36.466154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:36.466154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:36.466154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:36.466154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:36.466154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:36.466154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:36.467400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:36.467400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:36.467400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:36.467400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:36.467400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:36.467400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:36.467400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:36.467400 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:36.470932 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 435/853] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:36.472443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857dd6f-78da-4225-903b-c086fdb89f65 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:36.472739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857dd6f-78da-4225-903b-c086fdb89f65 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:36.473025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7857dd6f-78da-4225-903b-c086fdb89f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:36.489419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7857dd6f-78da-4225-903b-c086fdb89f65 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:36.490652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7857dd6f-78da-4225-903b-c086fdb89f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:36.496272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:36.560015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dafcf7-841d-42ab-bbeb-c4e03d787eb0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:36.565886 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 436/854] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:36.565886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857dd6f-78da-4225-903b-c086fdb89f65 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:36.618383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061415d5-3b41-4d3c-ad47-7a62b766688d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:36.619051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061415d5-3b41-4d3c-ad47-7a62b766688d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:36.619306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061415d5-3b41-4d3c-ad47-7a62b766688d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:36.619584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061415d5-3b41-4d3c-ad47-7a62b766688d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:36.678236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7857dd6f-78da-4225-903b-c086fdb89f65 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:36.688127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061415d5-3b41-4d3c-ad47-7a62b766688d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:36.689987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061415d5-3b41-4d3c-ad47-7a62b766688d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:36.703589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:36.716436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb6e4d-32b1-4f44-b70d-2e929ae8bd0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:36.718850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb6e4d-32b1-4f44-b70d-2e929ae8bd0b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:36.718850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb6e4d-32b1-4f44-b70d-2e929ae8bd0b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:36.718850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb6e4d-32b1-4f44-b70d-2e929ae8bd0b None admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:36.718850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb6e4d-32b1-4f44-b70d-2e929ae8bd0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:36.719161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb6e4d-32b1-4f44-b70d-2e929ae8bd0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:36.720774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fb6e4d-32b1-4f44-b70d-2e929ae8bd0b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:36.738765 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 437/855] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:36.738765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb6e4d-32b1-4f44-b70d-2e929ae8bd0b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:36.740245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:36.795366 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 419/856] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:42:36 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 20:42:36.796768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b51ff-f24d-4e5a-9998-20b5c1ea4f47 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:36.797066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b51ff-f24d-4e5a-9998-20b5c1ea4f47 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:36.797350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b51ff-f24d-4e5a-9998-20b5c1ea4f47 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:36.798634 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 420/857] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:36.799327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b51ff-f24d-4e5a-9998-20b5c1ea4f47 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:36.839180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb6e4d-32b1-4f44-b70d-2e929ae8bd0b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:36.872630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5cd1b-fa2d-472c-9127-fe8dd67b3f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:36.873446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5cd1b-fa2d-472c-9127-fe8dd67b3f09 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:36.873715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5cd1b-fa2d-472c-9127-fe8dd67b3f09 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:36.874025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5cd1b-fa2d-472c-9127-fe8dd67b3f09 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:36.874708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5cd1b-fa2d-472c-9127-fe8dd67b3f09 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:36.876061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5cd1b-fa2d-472c-9127-fe8dd67b3f09 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:36.887729 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-7cb5cd1b-fa2d-472c-9127-fe8dd67b3f09 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 20:42:36.887729 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:36.887729 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:36.887729 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:36.887729 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:36.887729 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:36.887729 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:36.887729 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:36.887729 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 20:42:36.887729 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:36.887729 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:36.887729 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:36.887729 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:36.887729 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:36.887729 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:36.887729 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:36.887729 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:36.888754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:36.888754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:36.888754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:36.888754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:36.888754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:36.888754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:36.888754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:36.888754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:36.888754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:36.888754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:36.888754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:36.888754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:36.888754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:36.888754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:36.888754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:36.888754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:36.889603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:36.889603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:36.889603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:36.889603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:36.889603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:36.889603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:36.889603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:36.889603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:36.889603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:36.889603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:36.889603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:36.889603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:36.889603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 20:42:36.889603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:36.889603 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 438/858] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:42:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:36.890751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5cd1b-fa2d-472c-9127-fe8dd67b3f09 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:36.901812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b51ff-f24d-4e5a-9998-20b5c1ea4f47 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:36.935927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24685679-6580-4f24-8b26-7d98c4ce7c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:36.936609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24685679-6580-4f24-8b26-7d98c4ce7c31 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:36.936737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:36.936820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24685679-6580-4f24-8b26-7d98c4ce7c31 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:36.937039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24685679-6580-4f24-8b26-7d98c4ce7c31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:36.937898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24685679-6580-4f24-8b26-7d98c4ce7c31 None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=a01bba888a644eb18fc457aff578aa4b) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:36.938216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24685679-6580-4f24-8b26-7d98c4ce7c31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:36.939656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24685679-6580-4f24-8b26-7d98c4ce7c31 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:36.945455 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 421/859] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:36 2022] GET /identity/v3/role_assignments?user.id=a01bba888a644eb18fc457aff578aa4b&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=7805b93b32f1472f8365936d4d57d49a => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:36.946722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057deba6-70c4-422b-ad6c-e279e24fd19b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:36.947016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057deba6-70c4-422b-ad6c-e279e24fd19b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:36.947296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057deba6-70c4-422b-ad6c-e279e24fd19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:36.966942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-057deba6-70c4-422b-ad6c-e279e24fd19b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:36.968206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-057deba6-70c4-422b-ad6c-e279e24fd19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:36.973086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:36.997380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5cd1b-fa2d-472c-9127-fe8dd67b3f09 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:37.037750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ba6ed-df35-4075-bf77-ce0189c7d573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:37.038852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ba6ed-df35-4075-bf77-ce0189c7d573 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:37.039011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ba6ed-df35-4075-bf77-ce0189c7d573 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:37.039287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ba6ed-df35-4075-bf77-ce0189c7d573 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:37.048617 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 422/860] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:37.049941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057deba6-70c4-422b-ad6c-e279e24fd19b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:37.051450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ba6ed-df35-4075-bf77-ce0189c7d573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:37.052946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ba6ed-df35-4075-bf77-ce0189c7d573 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:37.081750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-571ba6ed-df35-4075-bf77-ce0189c7d573 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:37.081750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:37.081750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:37.081750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:37.081750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:37.081750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:37.081750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:37.081750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:37.081750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:37.081750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:37.081750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:37.081750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:37.081750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:37.081750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:37.081750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:37.083387 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:37.083387 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:37.083387 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:37.083387 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:37.083387 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:37.083387 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:37.083387 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:37.083387 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:37.083387 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:37.083387 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:37.083387 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:37.083387 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:37.083387 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:37.083387 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:37.083387 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:37.083387 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:37.083387 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:37.084290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:37.084290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:37.084290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:37.084290 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:37.086001 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 439/861] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:37.087266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ba6ed-df35-4075-bf77-ce0189c7d573 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:37.174878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057deba6-70c4-422b-ad6c-e279e24fd19b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:37.202011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:37.205808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ba6ed-df35-4075-bf77-ce0189c7d573 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:37.214298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9fc50-0628-42cb-8ae1-792fc2163eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:37.215349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9fc50-0628-42cb-8ae1-792fc2163eb8 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:37.215681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9fc50-0628-42cb-8ae1-792fc2163eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:37.215967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a9fc50-0628-42cb-8ae1-792fc2163eb8 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:37.217011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9fc50-0628-42cb-8ae1-792fc2163eb8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:37.217327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9fc50-0628-42cb-8ae1-792fc2163eb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:37.219177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a9fc50-0628-42cb-8ae1-792fc2163eb8 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:37.229581 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 423/862] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:42:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:37.229581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9fc50-0628-42cb-8ae1-792fc2163eb8 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:37.234302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:37.240993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e24e4b-8371-4547-a074-4778e89da335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:37.243289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e24e4b-8371-4547-a074-4778e89da335 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:37.243289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e24e4b-8371-4547-a074-4778e89da335 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:37.243289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e24e4b-8371-4547-a074-4778e89da335 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:37.243289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e24e4b-8371-4547-a074-4778e89da335 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:37.244433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e24e4b-8371-4547-a074-4778e89da335 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:37.257200 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-87e24e4b-8371-4547-a074-4778e89da335 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 20:42:37.257200 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:37.257200 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:37.257200 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:37.257200 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:37.257200 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:37.257200 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:37.257200 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:37.257200 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 20:42:37.257200 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:37.257200 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:37.257200 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:37.257200 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:37.257200 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:37.257200 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:37.257200 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:37.258188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:37.258188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:37.258188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:37.258188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:37.258188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:37.258188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:37.258188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:37.258188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:37.258188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:37.258188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:37.258188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:37.258188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:37.258188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:37.258188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:37.258188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:37.258188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:37.259577 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:37.259577 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:37.259577 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:37.259577 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:37.259577 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:37.259577 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:37.259577 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:37.259577 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:37.259577 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:37.259577 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:37.259577 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:37.259577 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:37.259577 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:37.259577 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 20:42:37.259577 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:37.259577 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 440/863] 10.222.0.33 () {56 vars in 1132 bytes} [Sat May 28 20:42:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:37.261158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e24e4b-8371-4547-a074-4778e89da335 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:37.353101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a9fc50-0628-42cb-8ae1-792fc2163eb8 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:37.371254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e24e4b-8371-4547-a074-4778e89da335 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:37.389202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53179ee9-9fab-424a-9ea9-75a292242d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:37.389896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53179ee9-9fab-424a-9ea9-75a292242d72 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:37.390181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53179ee9-9fab-424a-9ea9-75a292242d72 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:37.390723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53179ee9-9fab-424a-9ea9-75a292242d72 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:37.391607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53179ee9-9fab-424a-9ea9-75a292242d72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:37.391921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53179ee9-9fab-424a-9ea9-75a292242d72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:37.393295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53179ee9-9fab-424a-9ea9-75a292242d72 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:37.404954 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 424/864] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:37.405722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53179ee9-9fab-424a-9ea9-75a292242d72 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:37.412415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5ca2c-eca5-4b2f-9749-b42be6a2076a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:37.413079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5ca2c-eca5-4b2f-9749-b42be6a2076a None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:37.413361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5ca2c-eca5-4b2f-9749-b42be6a2076a None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:37.413639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b5ca2c-eca5-4b2f-9749-b42be6a2076a None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:37.414430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5ca2c-eca5-4b2f-9749-b42be6a2076a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:37.414737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5ca2c-eca5-4b2f-9749-b42be6a2076a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:37.416136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b5ca2c-eca5-4b2f-9749-b42be6a2076a None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:37.420981 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 441/865] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:42:37 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:37.421753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5ca2c-eca5-4b2f-9749-b42be6a2076a None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:37.469698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:37.481746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:37.540032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53179ee9-9fab-424a-9ea9-75a292242d72 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:37.560705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b5ca2c-eca5-4b2f-9749-b42be6a2076a None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:37.588472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edac63-77b5-4ce4-9e55-f3109892f463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:37.589716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edac63-77b5-4ce4-9e55-f3109892f463 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:37.590190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edac63-77b5-4ce4-9e55-f3109892f463 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:37.590757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edac63-77b5-4ce4-9e55-f3109892f463 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:37.592069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edac63-77b5-4ce4-9e55-f3109892f463 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:37.592795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edac63-77b5-4ce4-9e55-f3109892f463 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:37.594901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edac63-77b5-4ce4-9e55-f3109892f463 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:37.599825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b27625-7364-423b-a036-ad52fd388ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:37.600769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b27625-7364-423b-a036-ad52fd388ff9 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:37.605649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b27625-7364-423b-a036-ad52fd388ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:37.605649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b27625-7364-423b-a036-ad52fd388ff9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:37.606525 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 425/866] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 20:42:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:37.608041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edac63-77b5-4ce4-9e55-f3109892f463 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:37.612275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b27625-7364-423b-a036-ad52fd388ff9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:37.614455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b27625-7364-423b-a036-ad52fd388ff9 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:37.631154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-59b27625-7364-423b-a036-ad52fd388ff9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:37.631154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:37.631154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:37.631154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:37.631154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:37.631154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:37.631154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:37.631154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:37.631154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:37.631154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:37.631154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:37.631154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:37.631154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:37.631154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:37.631154 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:37.632176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:37.632176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:37.632176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:37.632176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:37.632176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:37.632176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:37.632176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:37.632176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:37.632176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:37.632176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:37.632176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:37.632176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:37.632176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:37.632176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:37.632176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:37.632176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:37.632176 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:37.633078 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:37.633078 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:37.633078 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:37.633078 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:37.633078 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:37.633078 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:37.633078 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:37.633078 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:37.635245 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 442/867] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:37.641757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b27625-7364-423b-a036-ad52fd388ff9 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:37.720416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edac63-77b5-4ce4-9e55-f3109892f463 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:37.741095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:37.748732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:37.751929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b27625-7364-423b-a036-ad52fd388ff9 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:37.756826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef7ced-7e25-46b8-a69c-1918775a7be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:37.770972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef7ced-7e25-46b8-a69c-1918775a7be3 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:37.770972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef7ced-7e25-46b8-a69c-1918775a7be3 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:37.770972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef7ced-7e25-46b8-a69c-1918775a7be3 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:37.770972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef7ced-7e25-46b8-a69c-1918775a7be3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:37.770972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef7ced-7e25-46b8-a69c-1918775a7be3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:37.770972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef7ced-7e25-46b8-a69c-1918775a7be3 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:37.771440 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 426/868] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 20:42:37 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:37.804279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e70210-dee7-4c74-9acc-7a1d0c5adff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:37.804963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e70210-dee7-4c74-9acc-7a1d0c5adff6 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:37.805351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e70210-dee7-4c74-9acc-7a1d0c5adff6 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:37.805633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e70210-dee7-4c74-9acc-7a1d0c5adff6 None admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:37.806484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e70210-dee7-4c74-9acc-7a1d0c5adff6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:37.806796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e70210-dee7-4c74-9acc-7a1d0c5adff6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:37.808271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e70210-dee7-4c74-9acc-7a1d0c5adff6 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:37.823159 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 443/869] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:37.831098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef7ced-7e25-46b8-a69c-1918775a7be3 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:37.944775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef7ced-7e25-46b8-a69c-1918775a7be3 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:37.980734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ca0319-ddb5-4351-92db-4cd44825400a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:37.981330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca0319-ddb5-4351-92db-4cd44825400a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:37.981705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca0319-ddb5-4351-92db-4cd44825400a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:37.982018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ca0319-ddb5-4351-92db-4cd44825400a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:37.982844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca0319-ddb5-4351-92db-4cd44825400a None admin] RBAC: Adding query filter params (role.id=4db40e559d064cf6b4f061e785b42dbb, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=c1c1ea9ba4c740b3b7ed67c8335bfad2) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:37.983154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca0319-ddb5-4351-92db-4cd44825400a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:37.984588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ca0319-ddb5-4351-92db-4cd44825400a None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:37.990855 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 427/870] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:37 2022] GET /identity/v3/role_assignments?user.id=c1c1ea9ba4c740b3b7ed67c8335bfad2&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=4db40e559d064cf6b4f061e785b42dbb => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:39.702463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daa53b-2158-4929-9c52-ef8c26294a32 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:39.702989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daa53b-2158-4929-9c52-ef8c26294a32 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:39.703358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26daa53b-2158-4929-9c52-ef8c26294a32 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:39.706109 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 444/871] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:39.715061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a8e8a-1069-4cc4-a662-7a77f1dd6c36 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:39.717293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a8e8a-1069-4cc4-a662-7a77f1dd6c36 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:39.717293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811a8e8a-1069-4cc4-a662-7a77f1dd6c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:39.745773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-811a8e8a-1069-4cc4-a662-7a77f1dd6c36 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:39.751766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-811a8e8a-1069-4cc4-a662-7a77f1dd6c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:39.835835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:39.864023 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 428/872] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:39.906031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26daa53b-2158-4929-9c52-ef8c26294a32 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:40.027527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26daa53b-2158-4929-9c52-ef8c26294a32 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:40.074373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50610a-1f41-4176-9657-a5e7c2d6dd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:40.075203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50610a-1f41-4176-9657-a5e7c2d6dd2a None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:40.075485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50610a-1f41-4176-9657-a5e7c2d6dd2a None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:40.076124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50610a-1f41-4176-9657-a5e7c2d6dd2a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:40.076789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50610a-1f41-4176-9657-a5e7c2d6dd2a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:40.078257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50610a-1f41-4176-9657-a5e7c2d6dd2a None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:40.090789 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-3b50610a-1f41-4176-9657-a5e7c2d6dd2a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 20:42:40.090789 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:40.090789 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:40.090789 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:40.090789 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:40.090789 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:40.090789 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:40.090789 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:40.090789 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 20:42:40.090789 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:40.090789 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:40.090789 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:40.090789 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:40.090789 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:40.090789 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:40.090789 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:40.090789 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:40.092590 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:40.092590 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:40.092590 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:40.092590 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:40.092590 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:40.092590 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:40.092590 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:40.092590 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:40.092590 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:40.092590 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:40.092590 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:40.092590 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:40.092590 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:40.092590 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:40.092590 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:40.092590 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:40.094231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:40.094231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:40.094231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:40.094231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:40.094231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:40.094231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:40.094231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:40.094231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:40.094231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:40.094231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:40.094231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:40.094231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:40.094231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 20:42:40.094231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:40.094231 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 445/873] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 20:42:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:40.103572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a8e8a-1069-4cc4-a662-7a77f1dd6c36 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:40.141150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01583b1-bed5-4219-9eb3-900f1f797d81 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:40.141559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01583b1-bed5-4219-9eb3-900f1f797d81 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:40.141799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01583b1-bed5-4219-9eb3-900f1f797d81 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:40.143662 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 446/874] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:42:40.149011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0d3ff-47f1-4ee0-b2fb-58bf0839cb9e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:40.149298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0d3ff-47f1-4ee0-b2fb-58bf0839cb9e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:40.149578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0d3ff-47f1-4ee0-b2fb-58bf0839cb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:40.167864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ebf0d3ff-47f1-4ee0-b2fb-58bf0839cb9e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:40.169051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ebf0d3ff-47f1-4ee0-b2fb-58bf0839cb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:40.205185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:40.248502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811a8e8a-1069-4cc4-a662-7a77f1dd6c36 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:40.262551 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 447/875] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:42:40.285202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0d3ff-47f1-4ee0-b2fb-58bf0839cb9e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:40.296810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5ce72-c16e-40dd-949b-012281d10228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:40.297877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5ce72-c16e-40dd-949b-012281d10228 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:40.298344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5ce72-c16e-40dd-949b-012281d10228 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:40.298762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a5ce72-c16e-40dd-949b-012281d10228 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:40.299513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5ce72-c16e-40dd-949b-012281d10228 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:40.299827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5ce72-c16e-40dd-949b-012281d10228 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:40.302120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a5ce72-c16e-40dd-949b-012281d10228 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:40.309308 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 429/876] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 20:42:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:40.314710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5ce72-c16e-40dd-949b-012281d10228 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:40.383373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:40.392561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0d3ff-47f1-4ee0-b2fb-58bf0839cb9e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:40.431016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cc18d1-1c3d-4519-99ac-713335023207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:40.431612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc18d1-1c3d-4519-99ac-713335023207 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:40.431901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc18d1-1c3d-4519-99ac-713335023207 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:40.432179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cc18d1-1c3d-4519-99ac-713335023207 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:40.432901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc18d1-1c3d-4519-99ac-713335023207 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:40.434301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cc18d1-1c3d-4519-99ac-713335023207 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:40.442491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a5ce72-c16e-40dd-949b-012281d10228 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:40.477736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e74d92-4eec-487c-9a1e-77473d02b351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:40.478443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e74d92-4eec-487c-9a1e-77473d02b351 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:40.478727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e74d92-4eec-487c-9a1e-77473d02b351 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:40.479007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e74d92-4eec-487c-9a1e-77473d02b351 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:40.479725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e74d92-4eec-487c-9a1e-77473d02b351 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:40.480037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e74d92-4eec-487c-9a1e-77473d02b351 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:40.481415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e74d92-4eec-487c-9a1e-77473d02b351 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:40.485886 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 430/877] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 20:42:40 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:40.487182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8344d6-b4e1-4979-965c-401fc5d7ed96 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:40.487461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8344d6-b4e1-4979-965c-401fc5d7ed96 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:40.487743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8344d6-b4e1-4979-965c-401fc5d7ed96 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:40.489196 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 431/878] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:40.495125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb7c77-bd5c-4a04-81ba-578ce4217b2d None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:40.495414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb7c77-bd5c-4a04-81ba-578ce4217b2d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:40.495700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb7c77-bd5c-4a04-81ba-578ce4217b2d None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:40.497073 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 432/879] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:42:40.498193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808c36f-27a1-4fcf-8697-99dba9f5f01c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:40.498551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808c36f-27a1-4fcf-8697-99dba9f5f01c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:40.498841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6808c36f-27a1-4fcf-8697-99dba9f5f01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:40.503181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:40.514052 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 448/880] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 20:42:40 2022] POST /identity/v3/services => generated 269 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:42:40.516285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5374d-0caa-464c-a284-0810c9e4801d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:40.517005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5374d-0caa-464c-a284-0810c9e4801d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:40.517483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d5374d-0caa-464c-a284-0810c9e4801d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:40.518108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6808c36f-27a1-4fcf-8697-99dba9f5f01c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:40.520044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6808c36f-27a1-4fcf-8697-99dba9f5f01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:40.542829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-07d5374d-0caa-464c-a284-0810c9e4801d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:40.544726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-07d5374d-0caa-464c-a284-0810c9e4801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:40.626345 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 433/881] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:40 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:40.650091 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 449/882] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:40 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:40.651175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808c36f-27a1-4fcf-8697-99dba9f5f01c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:40.679258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5374d-0caa-464c-a284-0810c9e4801d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:40.700878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:40.773207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6808c36f-27a1-4fcf-8697-99dba9f5f01c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:40.808466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d5374d-0caa-464c-a284-0810c9e4801d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:40.813308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa34aee-0584-4243-8be6-e78f10d03c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:40.814673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa34aee-0584-4243-8be6-e78f10d03c7a None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:40.814673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa34aee-0584-4243-8be6-e78f10d03c7a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:40.814673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa34aee-0584-4243-8be6-e78f10d03c7a None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:40.817266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa34aee-0584-4243-8be6-e78f10d03c7a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:40.817266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa34aee-0584-4243-8be6-e78f10d03c7a None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:40.841499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:40.847632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80381c1a-f586-4a66-af0f-8fc3241262ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:40.848508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80381c1a-f586-4a66-af0f-8fc3241262ae None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:40.848795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80381c1a-f586-4a66-af0f-8fc3241262ae None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:40.849194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80381c1a-f586-4a66-af0f-8fc3241262ae None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:40.859832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80381c1a-f586-4a66-af0f-8fc3241262ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:40.868331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-80381c1a-f586-4a66-af0f-8fc3241262ae None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 20:42:40.868331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:40.868331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:40.868331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:40.868331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:40.868331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:40.868331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:40.868331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:40.868331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:40.868331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:40.868331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:40.868331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:40.868331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:40.868331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:40.868331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:40.868331 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:40.869285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:40.869285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:40.869285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:40.869285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:40.869285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:40.869285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:40.869285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:40.869285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:40.869285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:40.869285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:40.869285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:40.869285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:40.869285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:40.869285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:40.869285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:40.869285 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:40.870564 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:40.870564 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:40.870564 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:40.870564 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:40.870564 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:40.870564 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:40.870564 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:40.870564 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:40.870564 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:40.870564 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:40.870564 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 20:42:40.870564 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:40.870564 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 450/883] 10.222.0.33 () {56 vars in 1120 bytes} [Sat May 28 20:42:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:42:40.875639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80381c1a-f586-4a66-af0f-8fc3241262ae None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:40.927163 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 434/884] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 20:42:40 2022] POST /identity/v3/services => generated 251 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:42:41.007181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80381c1a-f586-4a66-af0f-8fc3241262ae None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:41.052513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d0af9-b4f2-4251-82d3-d1032f7222b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:41.053609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d0af9-b4f2-4251-82d3-d1032f7222b5 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:41.054085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d0af9-b4f2-4251-82d3-d1032f7222b5 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:41.054670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11d0af9-b4f2-4251-82d3-d1032f7222b5 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:41.055595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d0af9-b4f2-4251-82d3-d1032f7222b5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:41.056052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d0af9-b4f2-4251-82d3-d1032f7222b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:41.064013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11d0af9-b4f2-4251-82d3-d1032f7222b5 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:41.085870 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 451/885] 10.222.0.33 () {56 vars in 1115 bytes} [Sat May 28 20:42:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:41.091683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa34aee-0584-4243-8be6-e78f10d03c7a None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:41.203141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:41.206379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa34aee-0584-4243-8be6-e78f10d03c7a None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:41.243136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b989a-60f9-43df-a1f2-f2480902c29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:41.244226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b989a-60f9-43df-a1f2-f2480902c29d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:41.244709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b989a-60f9-43df-a1f2-f2480902c29d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:41.245137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b989a-60f9-43df-a1f2-f2480902c29d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:41.257054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b989a-60f9-43df-a1f2-f2480902c29d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:41.258770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092b989a-60f9-43df-a1f2-f2480902c29d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:41.277332 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-092b989a-60f9-43df-a1f2-f2480902c29d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:41.277332 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:41.277332 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:41.277332 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:41.277332 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:41.277332 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:41.277332 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:41.277332 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:41.277332 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:41.277332 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:41.277332 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:41.277332 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:41.277332 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:41.277332 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:41.277332 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:41.278497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:41.278497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:41.278497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:41.278497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:41.278497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:41.278497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:41.278497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:41.278497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:41.278497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:41.278497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:41.278497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:41.278497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:41.278497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:41.278497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:41.278497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:41.278497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:41.278497 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:41.279710 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:41.279710 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:41.279710 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:41.279710 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:41.281655 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 435/886] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:41.287600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d0af9-b4f2-4251-82d3-d1032f7222b5 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:41.341005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:41.398573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11d0af9-b4f2-4251-82d3-d1032f7222b5 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:41.433523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abbb4a-0f6b-4245-b357-d11b692d182f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:41.434221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abbb4a-0f6b-4245-b357-d11b692d182f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:41.434503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abbb4a-0f6b-4245-b357-d11b692d182f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:41.434779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17abbb4a-0f6b-4245-b357-d11b692d182f None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:41.435502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17abbb4a-0f6b-4245-b357-d11b692d182f None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:41.435807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17abbb4a-0f6b-4245-b357-d11b692d182f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:41.437182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17abbb4a-0f6b-4245-b357-d11b692d182f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:41.450619 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 452/887] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:41.456204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b989a-60f9-43df-a1f2-f2480902c29d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:41.561426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b989a-60f9-43df-a1f2-f2480902c29d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:41.596294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5151d0d-a955-4739-8f79-91740acd5f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:41.597032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5151d0d-a955-4739-8f79-91740acd5f97 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:41.597862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5151d0d-a955-4739-8f79-91740acd5f97 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:41.598143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5151d0d-a955-4739-8f79-91740acd5f97 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:41.605567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5151d0d-a955-4739-8f79-91740acd5f97 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:41.607048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5151d0d-a955-4739-8f79-91740acd5f97 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:41.622489 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-b5151d0d-a955-4739-8f79-91740acd5f97 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 20:42:41.622489 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:41.622489 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:41.622489 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:41.622489 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:41.622489 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:41.622489 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:41.622489 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:41.622489 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:41.622489 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:41.622489 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:41.622489 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:41.622489 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:41.622489 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:41.622489 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:41.622489 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:41.623448 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:41.624383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:41.624383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:41.624383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:41.624383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:41.624383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:41.624383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:41.624383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:41.624383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:41.624383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 20:42:41.624383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:41.624383 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 436/888] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 20:42:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:41.630992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abbb4a-0f6b-4245-b357-d11b692d182f None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:41.740255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17abbb4a-0f6b-4245-b357-d11b692d182f None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:41.750030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:41.779524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ae633-b61c-4c75-afd9-f99ca3dca167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:41.780219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ae633-b61c-4c75-afd9-f99ca3dca167 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:41.780510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ae633-b61c-4c75-afd9-f99ca3dca167 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:41.780859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ae633-b61c-4c75-afd9-f99ca3dca167 None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:41.781732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ae633-b61c-4c75-afd9-f99ca3dca167 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:41.782085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ae633-b61c-4c75-afd9-f99ca3dca167 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:41.783573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05ae633-b61c-4c75-afd9-f99ca3dca167 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:41.804794 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 453/889] 10.222.0.33 () {56 vars in 1149 bytes} [Sat May 28 20:42:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:41.810425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5151d0d-a955-4739-8f79-91740acd5f97 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:41.826542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:41.917403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5151d0d-a955-4739-8f79-91740acd5f97 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:41.952368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5741c-0d7a-4ffe-884d-cd73e0feb2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:41.953361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5741c-0d7a-4ffe-884d-cd73e0feb2c2 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:41.953866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5741c-0d7a-4ffe-884d-cd73e0feb2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:41.954344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5741c-0d7a-4ffe-884d-cd73e0feb2c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:42.023535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5741c-0d7a-4ffe-884d-cd73e0feb2c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:42.025292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5741c-0d7a-4ffe-884d-cd73e0feb2c2 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:42.042181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:42.139310 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 437/890] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:42:41 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 20:42:42.145256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ae633-b61c-4c75-afd9-f99ca3dca167 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:42.252421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05ae633-b61c-4c75-afd9-f99ca3dca167 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:42.288031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f115043-032f-4ac6-bf7a-3c3bdafe349f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:42.288827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f115043-032f-4ac6-bf7a-3c3bdafe349f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:42.289107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f115043-032f-4ac6-bf7a-3c3bdafe349f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:42.289383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f115043-032f-4ac6-bf7a-3c3bdafe349f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:42.305645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f115043-032f-4ac6-bf7a-3c3bdafe349f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:42.305645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f115043-032f-4ac6-bf7a-3c3bdafe349f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:42.323653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-0f115043-032f-4ac6-bf7a-3c3bdafe349f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:42.323653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:42.323653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:42.323653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:42.323653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:42.323653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:42.323653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:42.323653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:42.323653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:42.323653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:42.323653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:42.323653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:42.323653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:42.323653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:42.323653 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:42.324607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:42.324607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:42.324607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:42.324607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:42.324607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:42.324607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:42.324607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:42.324607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:42.324607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:42.324607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:42.324607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:42.324607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:42.324607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:42.324607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:42.324607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:42.324607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:42.324607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:42.325494 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:42.325494 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:42.325494 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:42.325494 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:42.327821 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 454/891] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:42.333240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5741c-0d7a-4ffe-884d-cd73e0feb2c2 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:42.439297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5741c-0d7a-4ffe-884d-cd73e0feb2c2 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:42.454162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:42.476405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b3173-1ca0-4e2c-9ef4-a20fa151ac5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:42.477085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b3173-1ca0-4e2c-9ef4-a20fa151ac5c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:42.477372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b3173-1ca0-4e2c-9ef4-a20fa151ac5c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:42.477658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b3173-1ca0-4e2c-9ef4-a20fa151ac5c None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:42.478415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b3173-1ca0-4e2c-9ef4-a20fa151ac5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:42.478726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b3173-1ca0-4e2c-9ef4-a20fa151ac5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:42.480224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b3173-1ca0-4e2c-9ef4-a20fa151ac5c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:42.492908 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 438/892] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:42.498394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f115043-032f-4ac6-bf7a-3c3bdafe349f None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:42.518013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:42.601989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f115043-032f-4ac6-bf7a-3c3bdafe349f None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:42.636773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb766d-7d67-4510-8221-b4202615da49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:42.637574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb766d-7d67-4510-8221-b4202615da49 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:42.637869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb766d-7d67-4510-8221-b4202615da49 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:42.638155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb766d-7d67-4510-8221-b4202615da49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:42.645671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb766d-7d67-4510-8221-b4202615da49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:42.647165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb766d-7d67-4510-8221-b4202615da49 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:42.661852 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-6bbb766d-7d67-4510-8221-b4202615da49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:42.661852 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:42.661852 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:42.661852 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:42.661852 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:42.661852 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:42.661852 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:42.661852 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:42.661852 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:42.661852 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:42.661852 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:42.661852 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:42.661852 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:42.661852 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:42.661852 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:42.662756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:42.662756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:42.662756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:42.662756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:42.662756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:42.662756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:42.662756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:42.662756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:42.662756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:42.662756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:42.662756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:42.662756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:42.662756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:42.662756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:42.662756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:42.662756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:42.662756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:42.663637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:42.663637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:42.663637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:42.663637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:42.663637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:42.663637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:42.663637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:42.663637 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:42.663637 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 455/893] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:42.669964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b3173-1ca0-4e2c-9ef4-a20fa151ac5c None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:42.782440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b3173-1ca0-4e2c-9ef4-a20fa151ac5c None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:42.821873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3503f376-112d-47b7-bc68-b58909483e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:42.822657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503f376-112d-47b7-bc68-b58909483e9f None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:42.822822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503f376-112d-47b7-bc68-b58909483e9f None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:42.823093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3503f376-112d-47b7-bc68-b58909483e9f None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:42.823885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503f376-112d-47b7-bc68-b58909483e9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:42.824197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503f376-112d-47b7-bc68-b58909483e9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:42.825640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3503f376-112d-47b7-bc68-b58909483e9f None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:42.839221 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 439/894] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:42.845077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb766d-7d67-4510-8221-b4202615da49 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:42.927616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:42.929320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500acb47-fbb8-4d0d-b1d9-27a639f0168b None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:42.929606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500acb47-fbb8-4d0d-b1d9-27a639f0168b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:42.929924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500acb47-fbb8-4d0d-b1d9-27a639f0168b None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:42.932427 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 440/895] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:42:42.940961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8b285-ca7f-4033-b8be-1be6af1794f4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:42.941444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8b285-ca7f-4033-b8be-1be6af1794f4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:42.941823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8b285-ca7f-4033-b8be-1be6af1794f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:42.951555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb766d-7d67-4510-8221-b4202615da49 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:42.963977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c3c8b285-ca7f-4033-b8be-1be6af1794f4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:42.965954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c3c8b285-ca7f-4033-b8be-1be6af1794f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:42.991930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d86b08-5c68-4d84-8c56-9f6359b06184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:42.992601 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d86b08-5c68-4d84-8c56-9f6359b06184 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:42.992890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d86b08-5c68-4d84-8c56-9f6359b06184 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:42.993172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d86b08-5c68-4d84-8c56-9f6359b06184 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:42.994053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d86b08-5c68-4d84-8c56-9f6359b06184 None admin] RBAC: Adding query filter params (role.id=50445f06754843f49827b854ec728850, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=c1c1ea9ba4c740b3b7ed67c8335bfad2) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:42.994376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d86b08-5c68-4d84-8c56-9f6359b06184 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:42.995827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d86b08-5c68-4d84-8c56-9f6359b06184 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:43.000820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:43.002379 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 456/896] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:42 2022] GET /identity/v3/role_assignments?user.id=c1c1ea9ba4c740b3b7ed67c8335bfad2&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=50445f06754843f49827b854ec728850 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:43.004042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd06bf-4b2e-419e-b2bf-d3e85ff41e0a None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:43.004337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd06bf-4b2e-419e-b2bf-d3e85ff41e0a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:43.004622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd06bf-4b2e-419e-b2bf-d3e85ff41e0a None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:43.006099 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 457/897] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:43.011779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5873a05-cbb9-42ac-9a50-5edc74f5407e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:43.024597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5873a05-cbb9-42ac-9a50-5edc74f5407e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:43.024597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5873a05-cbb9-42ac-9a50-5edc74f5407e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:43.033885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d5873a05-cbb9-42ac-9a50-5edc74f5407e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:43.035103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d5873a05-cbb9-42ac-9a50-5edc74f5407e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:43.074082 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 441/898] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:42 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:42:43.100466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8b285-ca7f-4033-b8be-1be6af1794f4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:43.128984 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 458/899] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:42:43.150259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5873a05-cbb9-42ac-9a50-5edc74f5407e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:43.226168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8b285-ca7f-4033-b8be-1be6af1794f4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:43.239808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:43.274098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290f612-acc2-4ff9-a78f-2f3643a82ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:43.274831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290f612-acc2-4ff9-a78f-2f3643a82ca8 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:43.275709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290f612-acc2-4ff9-a78f-2f3643a82ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:43.275996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290f612-acc2-4ff9-a78f-2f3643a82ca8 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:43.277104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290f612-acc2-4ff9-a78f-2f3643a82ca8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:43.279434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290f612-acc2-4ff9-a78f-2f3643a82ca8 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:43.281805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5873a05-cbb9-42ac-9a50-5edc74f5407e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:43.320788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c741fd-af32-4d17-8730-ed3c61ee9ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:43.321523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c741fd-af32-4d17-8730-ed3c61ee9ae1 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:43.321801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c741fd-af32-4d17-8730-ed3c61ee9ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:43.322122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c741fd-af32-4d17-8730-ed3c61ee9ae1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:43.322852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c741fd-af32-4d17-8730-ed3c61ee9ae1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:43.324224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c741fd-af32-4d17-8730-ed3c61ee9ae1 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:43.337311 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-00c741fd-af32-4d17-8730-ed3c61ee9ae1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 20:42:43.337311 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:43.337311 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:43.337311 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:43.337311 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:43.337311 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:43.337311 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:43.337311 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:43.337311 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 20:42:43.337311 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:43.337311 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:43.337311 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:43.337311 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:43.337311 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:43.337311 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:43.337311 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:43.338454 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:43.338454 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:43.338454 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:43.338454 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:43.338454 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:43.338454 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:43.338454 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:43.338454 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:43.338454 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:43.338454 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:43.338454 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:43.338454 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:43.338454 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:43.338454 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:43.338454 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:43.338454 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:43.339302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:43.339302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:43.339302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:43.339302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:43.339302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:43.339302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:43.339302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:43.339302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:43.339302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:43.339302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:43.339302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:43.339302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:43.339302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:43.339302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 20:42:43.339302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:43.339302 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 459/900] 10.222.0.33 () {56 vars in 1132 bytes} [Sat May 28 20:42:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:43.346202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c741fd-af32-4d17-8730-ed3c61ee9ae1 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:43.357183 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 442/901] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 20:42:43 2022] POST /identity/v3/services => generated 241 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 20:42:43.479561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c741fd-af32-4d17-8730-ed3c61ee9ae1 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:43.485133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa98f4-891c-49a4-bc35-19ce5e60ed2e None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:43.485764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa98f4-891c-49a4-bc35-19ce5e60ed2e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:43.486432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa98f4-891c-49a4-bc35-19ce5e60ed2e None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:43.488907 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 443/902] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:42:43.494770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13d3e6-3798-4eac-adeb-057e4e08f6aa None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:43.495404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13d3e6-3798-4eac-adeb-057e4e08f6aa None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:43.495902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13d3e6-3798-4eac-adeb-057e4e08f6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:43.515122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6f13d3e6-3798-4eac-adeb-057e4e08f6aa None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:43.516968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6f13d3e6-3798-4eac-adeb-057e4e08f6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:43.523281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:43.528348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f42c3-232a-4a8c-8a7e-e022fd11f2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:43.529033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f42c3-232a-4a8c-8a7e-e022fd11f2ec None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:43.529318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f42c3-232a-4a8c-8a7e-e022fd11f2ec None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:43.529595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f42c3-232a-4a8c-8a7e-e022fd11f2ec None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:43.530728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f42c3-232a-4a8c-8a7e-e022fd11f2ec None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:43.530995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f42c3-232a-4a8c-8a7e-e022fd11f2ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:43.532538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f42c3-232a-4a8c-8a7e-e022fd11f2ec None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:43.538672 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 460/903] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 20:42:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:43.540770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:43.544045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f42c3-232a-4a8c-8a7e-e022fd11f2ec None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:43.614266 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 444/904] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:42:43.637038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13d3e6-3798-4eac-adeb-057e4e08f6aa None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:43.666705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f42c3-232a-4a8c-8a7e-e022fd11f2ec None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:43.721214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15459794-047c-4e40-a7b3-859424a4ff59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:43.723001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15459794-047c-4e40-a7b3-859424a4ff59 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:43.723520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15459794-047c-4e40-a7b3-859424a4ff59 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:43.724428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15459794-047c-4e40-a7b3-859424a4ff59 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:43.725361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15459794-047c-4e40-a7b3-859424a4ff59 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:43.726271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15459794-047c-4e40-a7b3-859424a4ff59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:43.728553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15459794-047c-4e40-a7b3-859424a4ff59 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:43.735896 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 461/905] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 20:42:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:43.743643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15459794-047c-4e40-a7b3-859424a4ff59 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:43.759743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13d3e6-3798-4eac-adeb-057e4e08f6aa None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:43.809127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d58725b-6bab-4429-b066-bac6fc2635c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:43.810898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:43.812147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58725b-6bab-4429-b066-bac6fc2635c9 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:43.812759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58725b-6bab-4429-b066-bac6fc2635c9 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:43.813229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d58725b-6bab-4429-b066-bac6fc2635c9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:43.814649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d58725b-6bab-4429-b066-bac6fc2635c9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:43.816596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d58725b-6bab-4429-b066-bac6fc2635c9 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:43.832390 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-3d58725b-6bab-4429-b066-bac6fc2635c9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 20:42:43.832390 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:43.832390 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:43.832390 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:43.832390 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:43.832390 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:43.832390 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:43.832390 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:43.832390 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 20:42:43.832390 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:43.832390 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:43.832390 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:43.832390 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:43.832390 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:43.832390 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:43.832390 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:43.832390 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:43.833867 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:43.833867 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:43.833867 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:43.833867 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:43.833867 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:43.833867 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:43.833867 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:43.833867 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:43.833867 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:43.833867 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:43.833867 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:43.833867 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:43.833867 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:43.833867 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:43.833867 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:43.833867 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:43.834742 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:43.834742 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:43.834742 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:43.834742 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:43.834742 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:43.834742 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:43.834742 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:43.834742 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:43.834742 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:43.834742 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:43.834742 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:43.834742 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:43.834742 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 20:42:43.834742 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:43.842041 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 445/906] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:42:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:43.842641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d58725b-6bab-4429-b066-bac6fc2635c9 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:43.866912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:43.870594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15459794-047c-4e40-a7b3-859424a4ff59 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:43.915965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f9519-60e1-4d8a-9db7-d9249a5204f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:43.917244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f9519-60e1-4d8a-9db7-d9249a5204f3 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:43.918244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f9519-60e1-4d8a-9db7-d9249a5204f3 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:43.919001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f9519-60e1-4d8a-9db7-d9249a5204f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:43.920336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f9519-60e1-4d8a-9db7-d9249a5204f3 None admin] RBAC: Adding query filter params (interface=public, service_id=4d67879742964f16b4d94456043be1a4, region_id=RegionOne) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:43.921252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f9519-60e1-4d8a-9db7-d9249a5204f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:43.923160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f9519-60e1-4d8a-9db7-d9249a5204f3 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:43.936650 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 462/907] 10.222.0.33 () {56 vars in 1251 bytes} [Sat May 28 20:42:43 2022] GET /identity/v3/endpoints?service_id=4d67879742964f16b4d94456043be1a4&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:43.937780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f9519-60e1-4d8a-9db7-d9249a5204f3 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:43.966929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d58725b-6bab-4429-b066-bac6fc2635c9 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:44.008237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534d2de-face-4a17-aaec-383e6f579b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:44.009313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534d2de-face-4a17-aaec-383e6f579b00 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:44.009799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534d2de-face-4a17-aaec-383e6f579b00 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:44.010330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534d2de-face-4a17-aaec-383e6f579b00 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:44.011411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534d2de-face-4a17-aaec-383e6f579b00 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:44.011922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534d2de-face-4a17-aaec-383e6f579b00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:44.013658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534d2de-face-4a17-aaec-383e6f579b00 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:44.024351 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 446/908] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:42:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:44.025586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534d2de-face-4a17-aaec-383e6f579b00 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:44.055590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f9519-60e1-4d8a-9db7-d9249a5204f3 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:44.079555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:44.090722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552087c-64f7-4479-9bac-526c87d6d745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:44.091404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552087c-64f7-4479-9bac-526c87d6d745 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:44.097872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552087c-64f7-4479-9bac-526c87d6d745 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:44.097872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3552087c-64f7-4479-9bac-526c87d6d745 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:44.097872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3552087c-64f7-4479-9bac-526c87d6d745 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:44.097872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3552087c-64f7-4479-9bac-526c87d6d745 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:44.109880 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 463/909] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 20:42:43 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 20:42:44.142894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:44.165154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534d2de-face-4a17-aaec-383e6f579b00 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:44.203399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36e961-aa51-43a0-897c-644a42e65065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:44.204083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36e961-aa51-43a0-897c-644a42e65065 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:44.204369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36e961-aa51-43a0-897c-644a42e65065 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:44.204650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd36e961-aa51-43a0-897c-644a42e65065 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:44.205377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd36e961-aa51-43a0-897c-644a42e65065 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:44.205687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd36e961-aa51-43a0-897c-644a42e65065 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:44.207707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd36e961-aa51-43a0-897c-644a42e65065 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:44.218076 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 447/910] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:42:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:44.220109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552087c-64f7-4479-9bac-526c87d6d745 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:44.349785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3552087c-64f7-4479-9bac-526c87d6d745 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:44.384100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93bae25-98d5-4c95-89a2-daf2f083e87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:44.384761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bae25-98d5-4c95-89a2-daf2f083e87d None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:44.385047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bae25-98d5-4c95-89a2-daf2f083e87d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:44.385326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bae25-98d5-4c95-89a2-daf2f083e87d None admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:44.386161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93bae25-98d5-4c95-89a2-daf2f083e87d None admin] RBAC: Adding query filter params (interface=public, service_id=c36ef756954d4699bf3ea111d801d851, region_id=RegionOne) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:44.386479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93bae25-98d5-4c95-89a2-daf2f083e87d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:44.387853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93bae25-98d5-4c95-89a2-daf2f083e87d None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:44.397984 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 464/911] 10.222.0.33 () {56 vars in 1251 bytes} [Sat May 28 20:42:44 2022] GET /identity/v3/endpoints?service_id=c36ef756954d4699bf3ea111d801d851&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:44.400460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36e961-aa51-43a0-897c-644a42e65065 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:44.502487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd36e961-aa51-43a0-897c-644a42e65065 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:44.543365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a9b6ef-dfa3-4252-8ad3-e3864c156312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:44.544429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9b6ef-dfa3-4252-8ad3-e3864c156312 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:44.544931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9b6ef-dfa3-4252-8ad3-e3864c156312 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:44.545423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a9b6ef-dfa3-4252-8ad3-e3864c156312 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:44.546364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9b6ef-dfa3-4252-8ad3-e3864c156312 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:44.547966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a9b6ef-dfa3-4252-8ad3-e3864c156312 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:44.554931 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 448/912] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 20:42:44 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 20:42:45.342779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b87d0-6145-4b3c-b546-153b1951cf8f None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:45.343285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b87d0-6145-4b3c-b546-153b1951cf8f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:45.343285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b87d0-6145-4b3c-b546-153b1951cf8f None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:45.344838 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 465/913] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:45.350741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa22285-0571-49a3-83a2-b0d7f89b6300 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:45.351549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa22285-0571-49a3-83a2-b0d7f89b6300 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:45.352049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa22285-0571-49a3-83a2-b0d7f89b6300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:45.370268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9aa22285-0571-49a3-83a2-b0d7f89b6300 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:45.371794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9aa22285-0571-49a3-83a2-b0d7f89b6300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:45.374073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:45.444127 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 449/914] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:45.466804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b87d0-6145-4b3c-b546-153b1951cf8f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:45.524136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:45.573560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b87d0-6145-4b3c-b546-153b1951cf8f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:45.611768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd05dc0-6390-4195-9cf3-e52d5d564132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:45.612519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd05dc0-6390-4195-9cf3-e52d5d564132 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:45.613488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd05dc0-6390-4195-9cf3-e52d5d564132 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:45.613772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd05dc0-6390-4195-9cf3-e52d5d564132 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:45.626496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd05dc0-6390-4195-9cf3-e52d5d564132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:45.635006 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-9dd05dc0-6390-4195-9cf3-e52d5d564132 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 20:42:45.635006 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:45.635006 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:45.635006 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:45.635006 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:45.635006 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:45.635006 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:45.635006 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:45.635006 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:45.635006 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:45.635006 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:45.635006 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:45.635006 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:45.635006 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:45.635006 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:45.635006 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:45.635943 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:45.635943 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:45.635943 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:45.635943 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:45.635943 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:45.635943 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:45.635943 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:45.635943 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:45.635943 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:45.635943 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:45.635943 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:45.635943 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:45.635943 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:45.635943 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:45.635943 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:45.635943 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:45.636811 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:45.636811 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:45.636811 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:45.636811 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:45.636811 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:45.636811 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:45.636811 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:45.636811 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:45.636811 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:45.636811 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:45.636811 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 20:42:45.636811 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:45.636811 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 466/915] 10.222.0.33 () {56 vars in 1120 bytes} [Sat May 28 20:42:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:45.642750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa22285-0571-49a3-83a2-b0d7f89b6300 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:45.750042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa22285-0571-49a3-83a2-b0d7f89b6300 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:45.786336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24f27d-cf2f-4d7f-bbf6-e688b3647c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:45.787369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24f27d-cf2f-4d7f-bbf6-e688b3647c66 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:45.787881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24f27d-cf2f-4d7f-bbf6-e688b3647c66 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:45.788382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24f27d-cf2f-4d7f-bbf6-e688b3647c66 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:45.789332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24f27d-cf2f-4d7f-bbf6-e688b3647c66 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:45.789924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24f27d-cf2f-4d7f-bbf6-e688b3647c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:45.791503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24f27d-cf2f-4d7f-bbf6-e688b3647c66 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:45.800550 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 450/916] 10.222.0.33 () {56 vars in 1115 bytes} [Sat May 28 20:42:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:45.806072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:45.807619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd05dc0-6390-4195-9cf3-e52d5d564132 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:45.808219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e3117-0e62-422e-83b1-67b1c4e1db8a None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:45.808513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e3117-0e62-422e-83b1-67b1c4e1db8a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:45.808794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e3117-0e62-422e-83b1-67b1c4e1db8a None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:45.810428 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 451/917] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:42:45.815594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e7cba-0a81-4a68-ba6c-75d96e1202d4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:45.815922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e7cba-0a81-4a68-ba6c-75d96e1202d4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:45.816225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e7cba-0a81-4a68-ba6c-75d96e1202d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:45.833492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1c9e7cba-0a81-4a68-ba6c-75d96e1202d4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:45.834791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1c9e7cba-0a81-4a68-ba6c-75d96e1202d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:45.914011 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 452/918] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:42:45.937281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd05dc0-6390-4195-9cf3-e52d5d564132 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:45.939548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e7cba-0a81-4a68-ba6c-75d96e1202d4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:45.953323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:45.982201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c8970-0ea8-438b-8caf-562ef8a56409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:45.983265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c8970-0ea8-438b-8caf-562ef8a56409 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:45.983585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c8970-0ea8-438b-8caf-562ef8a56409 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:45.984739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c8970-0ea8-438b-8caf-562ef8a56409 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:45.998366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c8970-0ea8-438b-8caf-562ef8a56409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:46.000609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0c8970-0ea8-438b-8caf-562ef8a56409 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:46.025945 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-ba0c8970-0ea8-438b-8caf-562ef8a56409 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:46.025945 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:46.025945 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:46.025945 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:46.025945 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:46.025945 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:46.025945 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:46.025945 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:46.025945 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:46.025945 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:46.025945 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:46.025945 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:46.025945 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:46.025945 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:46.025945 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:46.026870 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:46.026870 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:46.026870 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:46.026870 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:46.026870 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:46.026870 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:46.026870 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:46.026870 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:46.026870 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:46.026870 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:46.026870 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:46.026870 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:46.026870 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:46.026870 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:46.026870 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:46.026870 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:46.026870 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:46.027779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:46.027779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:46.027779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:46.027779 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:46.029353 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 467/919] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:46.034142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c8970-0ea8-438b-8caf-562ef8a56409 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:46.064388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e7cba-0a81-4a68-ba6c-75d96e1202d4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:46.104241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:46.112001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8eab6-947a-416d-8512-34d9930b126e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:46.113112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8eab6-947a-416d-8512-34d9930b126e None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:46.113400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8eab6-947a-416d-8512-34d9930b126e None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:46.113678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8eab6-947a-416d-8512-34d9930b126e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:46.114638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8eab6-947a-416d-8512-34d9930b126e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:46.116752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e8eab6-947a-416d-8512-34d9930b126e None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:46.133298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-73e8eab6-947a-416d-8512-34d9930b126e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 20:42:46.133298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:46.133298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:46.133298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:46.133298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:46.133298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:46.133298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:46.133298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:46.133298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 20:42:46.133298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:46.133298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:46.133298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:46.133298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:46.133298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:46.133298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:46.133298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:46.133298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:46.134560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:46.134560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:46.134560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:46.134560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:46.134560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:46.134560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:46.134560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:46.134560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:46.134560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:46.134560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:46.134560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:46.134560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:46.134560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:46.134560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:46.134560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:46.134560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:46.135573 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:46.135573 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:46.135573 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:46.135573 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:46.135573 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:46.135573 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:46.135573 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:46.135573 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:46.135573 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:46.135573 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:46.135573 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:46.135573 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:46.135573 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 20:42:46.135573 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:46.137920 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 453/920] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 20:42:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:42:46.142080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8eab6-947a-416d-8512-34d9930b126e None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:46.169761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c8970-0ea8-438b-8caf-562ef8a56409 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:46.214946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dca786-c1a8-441f-b0bd-ff89dfc8a8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:46.215619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dca786-c1a8-441f-b0bd-ff89dfc8a8d9 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:46.215920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dca786-c1a8-441f-b0bd-ff89dfc8a8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:46.216336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dca786-c1a8-441f-b0bd-ff89dfc8a8d9 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:46.217891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dca786-c1a8-441f-b0bd-ff89dfc8a8d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:46.218223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dca786-c1a8-441f-b0bd-ff89dfc8a8d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:46.219619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dca786-c1a8-441f-b0bd-ff89dfc8a8d9 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:46.237892 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 468/921] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:46.240399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dca786-c1a8-441f-b0bd-ff89dfc8a8d9 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:46.253983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8eab6-947a-416d-8512-34d9930b126e None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:46.276237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:46.294990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42e7b2-2ff6-45bd-bde1-00492fa57b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:46.295806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42e7b2-2ff6-45bd-bde1-00492fa57b85 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:46.296194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42e7b2-2ff6-45bd-bde1-00492fa57b85 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:46.296572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42e7b2-2ff6-45bd-bde1-00492fa57b85 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:46.297769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42e7b2-2ff6-45bd-bde1-00492fa57b85 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:46.298203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42e7b2-2ff6-45bd-bde1-00492fa57b85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:46.300343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42e7b2-2ff6-45bd-bde1-00492fa57b85 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:46.311116 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 454/922] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 20:42:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:46.313997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42e7b2-2ff6-45bd-bde1-00492fa57b85 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:46.378284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dca786-c1a8-441f-b0bd-ff89dfc8a8d9 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:46.394910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:46.419399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893f8b3-2e2b-41d2-bc35-81e8fe8d6be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:46.420349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893f8b3-2e2b-41d2-bc35-81e8fe8d6be1 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:46.420559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893f8b3-2e2b-41d2-bc35-81e8fe8d6be1 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:46.420839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893f8b3-2e2b-41d2-bc35-81e8fe8d6be1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:46.429122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893f8b3-2e2b-41d2-bc35-81e8fe8d6be1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:46.430763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893f8b3-2e2b-41d2-bc35-81e8fe8d6be1 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:46.444559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42e7b2-2ff6-45bd-bde1-00492fa57b85 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:46.448135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-a893f8b3-2e2b-41d2-bc35-81e8fe8d6be1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 20:42:46.448135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:46.448135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:46.448135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:46.448135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:46.448135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:46.448135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:46.448135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:46.448135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:46.448135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:46.448135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:46.448135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:46.448135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:46.448135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:46.448135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:46.448135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:46.449132 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:46.450117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:46.450117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:46.450117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:46.450117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:46.450117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:46.450117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:46.450117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:46.450117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:46.450117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 20:42:46.450117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:46.454136 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 469/923] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 20:42:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:46.455062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893f8b3-2e2b-41d2-bc35-81e8fe8d6be1 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:46.487520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d329f1-82a8-4a2f-a417-fb662e8f33c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:46.489079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d329f1-82a8-4a2f-a417-fb662e8f33c9 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:46.489569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d329f1-82a8-4a2f-a417-fb662e8f33c9 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:46.490076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d329f1-82a8-4a2f-a417-fb662e8f33c9 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:46.491068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d329f1-82a8-4a2f-a417-fb662e8f33c9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:46.491597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d329f1-82a8-4a2f-a417-fb662e8f33c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:46.493203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d329f1-82a8-4a2f-a417-fb662e8f33c9 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:46.499522 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 455/924] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 20:42:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:46.506103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d329f1-82a8-4a2f-a417-fb662e8f33c9 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:46.543345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:46.559113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893f8b3-2e2b-41d2-bc35-81e8fe8d6be1 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:46.597749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0349e-6942-43c8-a24c-e09106f2e9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:46.598630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0349e-6942-43c8-a24c-e09106f2e9eb None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:46.598922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0349e-6942-43c8-a24c-e09106f2e9eb None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:46.599207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0349e-6942-43c8-a24c-e09106f2e9eb None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:46.600061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0349e-6942-43c8-a24c-e09106f2e9eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:46.600375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0349e-6942-43c8-a24c-e09106f2e9eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:46.601819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0349e-6942-43c8-a24c-e09106f2e9eb None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:46.617488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d329f1-82a8-4a2f-a417-fb662e8f33c9 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:46.620332 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 470/925] 10.222.0.33 () {56 vars in 1149 bytes} [Sat May 28 20:42:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:46.621279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b2a97-9266-4863-8c8e-2d6e7199a22a None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:46.621570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b2a97-9266-4863-8c8e-2d6e7199a22a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:46.621863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b2a97-9266-4863-8c8e-2d6e7199a22a None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:46.623272 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 471/926] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:46.628044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b2a97-9266-4863-8c8e-2d6e7199a22a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:46.645763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:46.657736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36474017-0adc-4a58-af2e-f98ba492b9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:46.658433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36474017-0adc-4a58-af2e-f98ba492b9fc None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:46.658715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36474017-0adc-4a58-af2e-f98ba492b9fc None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:46.659250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36474017-0adc-4a58-af2e-f98ba492b9fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:46.659843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36474017-0adc-4a58-af2e-f98ba492b9fc None admin] RBAC: Adding query filter params (interface=public, service_id=1520b8cc69444fdd936e9afdc5a93025, region_id=RegionOne) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:46.660150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36474017-0adc-4a58-af2e-f98ba492b9fc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:46.661521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36474017-0adc-4a58-af2e-f98ba492b9fc None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:46.667087 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 456/927] 10.222.0.33 () {56 vars in 1251 bytes} [Sat May 28 20:42:46 2022] GET /identity/v3/endpoints?service_id=1520b8cc69444fdd936e9afdc5a93025&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:46.668423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01a771-9808-4d3a-8cc1-ebb98cbb2c13 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:46.668423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01a771-9808-4d3a-8cc1-ebb98cbb2c13 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:46.668604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01a771-9808-4d3a-8cc1-ebb98cbb2c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:46.687828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ea01a771-9808-4d3a-8cc1-ebb98cbb2c13 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:46.688969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ea01a771-9808-4d3a-8cc1-ebb98cbb2c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:46.745972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b2a97-9266-4863-8c8e-2d6e7199a22a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:46.768056 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 457/928] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:46.769201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01a771-9808-4d3a-8cc1-ebb98cbb2c13 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:46.789672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb4598-514a-4393-b4c4-732f47496da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:46.790820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb4598-514a-4393-b4c4-732f47496da7 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:46.791329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb4598-514a-4393-b4c4-732f47496da7 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:46.791758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cb4598-514a-4393-b4c4-732f47496da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:46.804048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb4598-514a-4393-b4c4-732f47496da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:46.805516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cb4598-514a-4393-b4c4-732f47496da7 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:46.807325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:46.824053 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-69cb4598-514a-4393-b4c4-732f47496da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:46.824053 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:46.824053 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:46.824053 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:46.824053 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:46.824053 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:46.824053 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:46.824053 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:46.824053 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:46.824053 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:46.824053 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:46.824053 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:46.824053 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:46.824053 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:46.824053 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:46.825010 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:46.825010 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:46.825010 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:46.825010 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:46.825010 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:46.825010 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:46.825010 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:46.825010 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:46.825010 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:46.825010 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:46.825010 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:46.825010 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:46.825010 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:46.825010 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:46.825010 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:46.825010 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:46.825010 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:46.830714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:46.830714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:46.830714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:46.830714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:46.830714 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 472/929] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:46.830714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb4598-514a-4393-b4c4-732f47496da7 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:46.890983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01a771-9808-4d3a-8cc1-ebb98cbb2c13 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:46.907374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:46.926959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b287036-9431-4605-81a5-2012fa21bd7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:46.927715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b287036-9431-4605-81a5-2012fa21bd7d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:46.928002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b287036-9431-4605-81a5-2012fa21bd7d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:46.928324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b287036-9431-4605-81a5-2012fa21bd7d None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:46.929015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b287036-9431-4605-81a5-2012fa21bd7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:46.929439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cb4598-514a-4393-b4c4-732f47496da7 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:46.930474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b287036-9431-4605-81a5-2012fa21bd7d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:46.936898 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 458/930] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 20:42:46 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 20:42:46.937967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b287036-9431-4605-81a5-2012fa21bd7d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:46.968762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d9ff2f-94e5-4bec-b76a-5241acf051f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:46.969717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9ff2f-94e5-4bec-b76a-5241acf051f8 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:46.970089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9ff2f-94e5-4bec-b76a-5241acf051f8 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:46.970432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d9ff2f-94e5-4bec-b76a-5241acf051f8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:46.971179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9ff2f-94e5-4bec-b76a-5241acf051f8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:46.973188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d9ff2f-94e5-4bec-b76a-5241acf051f8 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:46.986844 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-b3d9ff2f-94e5-4bec-b76a-5241acf051f8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 20:42:46.986844 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:46.986844 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:46.986844 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:46.986844 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:46.986844 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:46.986844 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:46.986844 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:46.986844 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 20:42:46.986844 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:46.986844 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:46.986844 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:46.986844 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:46.986844 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:46.986844 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:46.986844 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:46.988659 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:46.988659 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:46.988659 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:46.988659 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:46.988659 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:46.988659 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:46.988659 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:46.988659 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:46.988659 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:46.988659 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:46.988659 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:46.988659 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:46.988659 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:46.988659 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:46.988659 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:46.988659 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:46.989693 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:46.989693 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:46.989693 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:46.989693 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:46.989693 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:46.989693 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:46.989693 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:46.989693 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:46.989693 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:46.989693 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:46.989693 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:46.989693 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:46.989693 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:46.989693 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 28 20:42:46.989693 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:46.990594 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 473/931] 10.222.0.33 () {56 vars in 1132 bytes} [Sat May 28 20:42:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:46.991754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ce355-e802-40fd-b5ce-f1b68d58011e None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:46.992042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ce355-e802-40fd-b5ce-f1b68d58011e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:46.992316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ce355-e802-40fd-b5ce-f1b68d58011e None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:46.993936 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 474/932] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:46.999429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fec53d-f16c-4116-a3d6-3ebefe2b1da6 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:46.999724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fec53d-f16c-4116-a3d6-3ebefe2b1da6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:47.000006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fec53d-f16c-4116-a3d6-3ebefe2b1da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:47.017567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-44fec53d-f16c-4116-a3d6-3ebefe2b1da6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:47.018836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-44fec53d-f16c-4116-a3d6-3ebefe2b1da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:47.063848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b287036-9431-4605-81a5-2012fa21bd7d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:47.089772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:47.104425 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 475/933] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:42:47.104686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ee7fc-6202-4cea-9ec6-1fa3427e250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:47.105437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ee7fc-6202-4cea-9ec6-1fa3427e250f None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:47.105729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ee7fc-6202-4cea-9ec6-1fa3427e250f None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:47.106040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ee7fc-6202-4cea-9ec6-1fa3427e250f None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:47.106906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ee7fc-6202-4cea-9ec6-1fa3427e250f None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:47.107213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ee7fc-6202-4cea-9ec6-1fa3427e250f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:47.108632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ee7fc-6202-4cea-9ec6-1fa3427e250f None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:47.121328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fec53d-f16c-4116-a3d6-3ebefe2b1da6 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:47.125871 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 459/934] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:47.130080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ee7fc-6202-4cea-9ec6-1fa3427e250f None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:47.178366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:47.232215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fec53d-f16c-4116-a3d6-3ebefe2b1da6 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:47.242876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ee7fc-6202-4cea-9ec6-1fa3427e250f None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:47.272797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6131278-d83d-4156-a441-b77e56b1626f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:47.273465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6131278-d83d-4156-a441-b77e56b1626f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:47.273748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6131278-d83d-4156-a441-b77e56b1626f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:47.274062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6131278-d83d-4156-a441-b77e56b1626f None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:47.274777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6131278-d83d-4156-a441-b77e56b1626f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:47.275086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6131278-d83d-4156-a441-b77e56b1626f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:47.276453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6131278-d83d-4156-a441-b77e56b1626f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:47.279250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e007527-eee6-4d70-b6a6-4ce14488ac48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:47.280045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e007527-eee6-4d70-b6a6-4ce14488ac48 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:47.280329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e007527-eee6-4d70-b6a6-4ce14488ac48 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:47.280605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e007527-eee6-4d70-b6a6-4ce14488ac48 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:47.281504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e007527-eee6-4d70-b6a6-4ce14488ac48 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:47.281610 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 476/935] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 20:42:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:47.282146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6131278-d83d-4156-a441-b77e56b1626f None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:47.287886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e007527-eee6-4d70-b6a6-4ce14488ac48 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:47.301311 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-7e007527-eee6-4d70-b6a6-4ce14488ac48 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 20:42:47.301311 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:47.301311 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:47.301311 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:47.301311 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:47.301311 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:47.301311 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:47.301311 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:47.301311 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 20:42:47.301311 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:47.301311 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:47.301311 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:47.301311 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:47.301311 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:47.301311 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:47.301311 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:47.301311 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:47.302383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:47.302383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:47.302383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:47.302383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:47.302383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:47.302383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:47.302383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:47.302383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:47.302383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:47.302383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:47.302383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:47.302383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:47.302383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:47.302383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:47.302383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:47.302383 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:47.303240 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:47.303240 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:47.303240 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:47.303240 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:47.303240 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:47.303240 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:47.303240 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:47.303240 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:47.303240 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:47.303240 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:47.303240 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:47.303240 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:47.303240 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 28 20:42:47.303240 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:47.303240 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 460/936] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:42:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:42:47.304051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e007527-eee6-4d70-b6a6-4ce14488ac48 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:47.343566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:47.398338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6131278-d83d-4156-a441-b77e56b1626f None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:47.414445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e007527-eee6-4d70-b6a6-4ce14488ac48 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:47.417896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:47.433058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff37900-1238-4d4d-a776-8bd8e7045c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:47.433879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff37900-1238-4d4d-a776-8bd8e7045c0d None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:47.434172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff37900-1238-4d4d-a776-8bd8e7045c0d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:47.434451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff37900-1238-4d4d-a776-8bd8e7045c0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:47.441212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff37900-1238-4d4d-a776-8bd8e7045c0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:47.442711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff37900-1238-4d4d-a776-8bd8e7045c0d None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:47.450418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737aa893-608f-41c4-98b3-ee7f6ce2c16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:47.455944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737aa893-608f-41c4-98b3-ee7f6ce2c16a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:47.455944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737aa893-608f-41c4-98b3-ee7f6ce2c16a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:47.455944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737aa893-608f-41c4-98b3-ee7f6ce2c16a None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:47.455944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737aa893-608f-41c4-98b3-ee7f6ce2c16a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:47.455944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737aa893-608f-41c4-98b3-ee7f6ce2c16a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:47.455944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737aa893-608f-41c4-98b3-ee7f6ce2c16a None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:47.458347 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-0ff37900-1238-4d4d-a776-8bd8e7045c0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:47.458347 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:47.458347 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:47.458347 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:47.458347 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:47.458347 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:47.458347 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:47.458347 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:47.458347 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:47.458347 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:47.458347 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:47.458347 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:47.458347 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:47.458347 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:47.458347 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:47.459397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:47.459397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:47.459397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:47.459397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:47.459397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:47.459397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:47.459397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:47.459397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:47.459397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:47.459397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:47.459397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:47.459397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:47.459397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:47.459397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:47.459397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:47.459397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:47.459397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:47.460494 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:47.460494 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:47.460494 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:47.460494 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:47.460494 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:47.460494 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:47.460494 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:47.460494 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:47.460494 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 477/937] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:47.466346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff37900-1238-4d4d-a776-8bd8e7045c0d None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:47.468205 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 461/938] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 20:42:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:47.473186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737aa893-608f-41c4-98b3-ee7f6ce2c16a None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:47.571735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff37900-1238-4d4d-a776-8bd8e7045c0d None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:47.576797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737aa893-608f-41c4-98b3-ee7f6ce2c16a None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:47.586290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:47.608399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beae876-dea6-41ae-b958-f70f02049075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:47.609162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beae876-dea6-41ae-b958-f70f02049075 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:47.609438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beae876-dea6-41ae-b958-f70f02049075 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:47.609716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beae876-dea6-41ae-b958-f70f02049075 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:47.610460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beae876-dea6-41ae-b958-f70f02049075 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:47.610772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beae876-dea6-41ae-b958-f70f02049075 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:47.610883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919eca44-46c2-45d7-ba8f-00f1a80db9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:47.611543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919eca44-46c2-45d7-ba8f-00f1a80db9d0 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:47.611827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919eca44-46c2-45d7-ba8f-00f1a80db9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:47.612101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919eca44-46c2-45d7-ba8f-00f1a80db9d0 None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:47.612194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beae876-dea6-41ae-b958-f70f02049075 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:47.613070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919eca44-46c2-45d7-ba8f-00f1a80db9d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:47.613375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919eca44-46c2-45d7-ba8f-00f1a80db9d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:47.614844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919eca44-46c2-45d7-ba8f-00f1a80db9d0 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:47.622409 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 478/939] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:42:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:47.622409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beae876-dea6-41ae-b958-f70f02049075 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:47.628132 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 462/940] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:47.629332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919eca44-46c2-45d7-ba8f-00f1a80db9d0 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:47.658301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:47.733538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919eca44-46c2-45d7-ba8f-00f1a80db9d0 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:47.738459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beae876-dea6-41ae-b958-f70f02049075 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:47.770867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06abb6d-21b3-4585-a2b9-8f442291dc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:47.771531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06abb6d-21b3-4585-a2b9-8f442291dc05 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:47.771812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06abb6d-21b3-4585-a2b9-8f442291dc05 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:47.773921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06abb6d-21b3-4585-a2b9-8f442291dc05 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:47.773921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06abb6d-21b3-4585-a2b9-8f442291dc05 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:47.773921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06abb6d-21b3-4585-a2b9-8f442291dc05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:47.774505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06abb6d-21b3-4585-a2b9-8f442291dc05 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:47.779861 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 463/941] 10.222.0.33 () {56 vars in 1114 bytes} [Sat May 28 20:42:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:47.780590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06abb6d-21b3-4585-a2b9-8f442291dc05 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:47.789196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2242f03-4d13-4f24-8b92-c805f771cf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:47.789892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2242f03-4d13-4f24-8b92-c805f771cf89 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:47.790182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2242f03-4d13-4f24-8b92-c805f771cf89 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:47.790460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2242f03-4d13-4f24-8b92-c805f771cf89 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:47.791210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2242f03-4d13-4f24-8b92-c805f771cf89 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:47.792585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2242f03-4d13-4f24-8b92-c805f771cf89 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:47.871730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:47.880464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06abb6d-21b3-4585-a2b9-8f442291dc05 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:47.885899 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 479/942] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 20:42:47 2022] POST /identity/v3/endpoints => generated 347 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:42:47.892211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2242f03-4d13-4f24-8b92-c805f771cf89 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:47.934408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089cea58-ef64-4854-a06c-4651a6c5a6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:47.935704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cea58-ef64-4854-a06c-4651a6c5a6b3 None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:47.935827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cea58-ef64-4854-a06c-4651a6c5a6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:47.936180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089cea58-ef64-4854-a06c-4651a6c5a6b3 None admin] PATH_INFO: `/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/c1c1ea9ba4c740b3b7ed67c8335bfad2/roles/50445f06754843f49827b854ec728850` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:47.938677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cea58-ef64-4854-a06c-4651a6c5a6b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=3186f9d3c2b341afa8b098ebcaaf974b, user_id=c1c1ea9ba4c740b3b7ed67c8335bfad2, role_id=50445f06754843f49827b854ec728850)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:47.939764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:47.940523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089cea58-ef64-4854-a06c-4651a6c5a6b3 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:48.011590 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 464/943] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:42:47 2022] PUT /identity/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/c1c1ea9ba4c740b3b7ed67c8335bfad2/roles/50445f06754843f49827b854ec728850 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:42:48.029148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2242f03-4d13-4f24-8b92-c805f771cf89 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:48.079771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467ef6c-f1c9-446d-b10c-7f25ae6f4bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:48.080773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467ef6c-f1c9-446d-b10c-7f25ae6f4bc4 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:48.081208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467ef6c-f1c9-446d-b10c-7f25ae6f4bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:48.082252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467ef6c-f1c9-446d-b10c-7f25ae6f4bc4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:48.083034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467ef6c-f1c9-446d-b10c-7f25ae6f4bc4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:48.084454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467ef6c-f1c9-446d-b10c-7f25ae6f4bc4 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:48.189075 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 480/944] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 20:42:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:42:49.268551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711a5b5-a1a6-4b7d-8ab8-7d9daa022e26 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:49.270287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711a5b5-a1a6-4b7d-8ab8-7d9daa022e26 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:49.270775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f711a5b5-a1a6-4b7d-8ab8-7d9daa022e26 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:49.272623 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 465/945] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:49.278029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae11e69-a61a-4695-9bb4-e065a8dce964 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:49.278835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae11e69-a61a-4695-9bb4-e065a8dce964 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:49.279318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae11e69-a61a-4695-9bb4-e065a8dce964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:49.296963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9ae11e69-a61a-4695-9bb4-e065a8dce964 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:49.298496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9ae11e69-a61a-4695-9bb4-e065a8dce964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:49.376782 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 481/946] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:49.401507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711a5b5-a1a6-4b7d-8ab8-7d9daa022e26 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:49.511178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f711a5b5-a1a6-4b7d-8ab8-7d9daa022e26 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:49.546617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27574f-2a09-4f45-bdd8-025c8770c9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:49.547428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27574f-2a09-4f45-bdd8-025c8770c9cf None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:49.547718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27574f-2a09-4f45-bdd8-025c8770c9cf None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:49.547998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27574f-2a09-4f45-bdd8-025c8770c9cf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:49.548841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27574f-2a09-4f45-bdd8-025c8770c9cf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:49.550845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27574f-2a09-4f45-bdd8-025c8770c9cf None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:49.563587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-6d27574f-2a09-4f45-bdd8-025c8770c9cf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 20:42:49.563587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:49.563587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:49.563587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:49.563587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:49.563587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:49.563587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:49.563587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:49.563587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 20:42:49.563587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:49.563587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:49.563587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:49.563587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:49.563587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:49.563587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:49.563587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:49.563587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:49.564822 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:49.564822 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:49.564822 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:49.564822 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:49.564822 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:49.564822 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:49.564822 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:49.564822 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:49.564822 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:49.564822 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:49.564822 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:49.564822 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:49.564822 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:49.564822 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:49.564822 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:49.564822 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:49.565669 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:49.565669 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:49.565669 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:49.565669 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:49.565669 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:49.565669 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:49.565669 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:49.565669 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:49.565669 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:49.565669 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:49.565669 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:49.565669 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:49.565669 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 20:42:49.565669 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:49.565669 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 466/947] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 20:42:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:49.575068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae11e69-a61a-4695-9bb4-e065a8dce964 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:49.591635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:49.680820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae11e69-a61a-4695-9bb4-e065a8dce964 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:49.719546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39736337-08ad-49f9-8c02-48fe3a4cc3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:49.720599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39736337-08ad-49f9-8c02-48fe3a4cc3cf None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:49.721081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39736337-08ad-49f9-8c02-48fe3a4cc3cf None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:49.721552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39736337-08ad-49f9-8c02-48fe3a4cc3cf None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:49.723082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39736337-08ad-49f9-8c02-48fe3a4cc3cf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:49.723630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39736337-08ad-49f9-8c02-48fe3a4cc3cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:49.725314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39736337-08ad-49f9-8c02-48fe3a4cc3cf None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:49.730959 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 482/948] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 20:42:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:49.736909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27574f-2a09-4f45-bdd8-025c8770c9cf None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:49.753457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:49.848174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27574f-2a09-4f45-bdd8-025c8770c9cf None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:49.883963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108658d-52f4-4441-8fa7-a1bdc435249b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:49.884634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108658d-52f4-4441-8fa7-a1bdc435249b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:49.884919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108658d-52f4-4441-8fa7-a1bdc435249b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:49.885192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108658d-52f4-4441-8fa7-a1bdc435249b None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:49.885919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108658d-52f4-4441-8fa7-a1bdc435249b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:49.886233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108658d-52f4-4441-8fa7-a1bdc435249b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:49.887606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0108658d-52f4-4441-8fa7-a1bdc435249b None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:49.893372 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 467/949] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 20:42:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:49.898882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39736337-08ad-49f9-8c02-48fe3a4cc3cf None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:49.984067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:50.007462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39736337-08ad-49f9-8c02-48fe3a4cc3cf None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:50.044726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07f892-a192-4e01-9e34-cd3d1e4ba104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:50.045671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07f892-a192-4e01-9e34-cd3d1e4ba104 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:50.046135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07f892-a192-4e01-9e34-cd3d1e4ba104 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:50.046553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07f892-a192-4e01-9e34-cd3d1e4ba104 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:50.047443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07f892-a192-4e01-9e34-cd3d1e4ba104 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:50.048975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07f892-a192-4e01-9e34-cd3d1e4ba104 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:50.149685 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 483/950] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 20:42:49 2022] POST /identity/v3/endpoints => generated 349 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:42:50.272684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e6e36-230a-40b4-99a9-c3773df84cf1 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:50.273202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e6e36-230a-40b4-99a9-c3773df84cf1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:50.273281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e6e36-230a-40b4-99a9-c3773df84cf1 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:50.274922 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 468/951] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:50.280734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9997b-c76d-4556-95c8-7230c4789371 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:50.281211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9997b-c76d-4556-95c8-7230c4789371 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:50.281288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9997b-c76d-4556-95c8-7230c4789371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:50.300047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-91c9997b-c76d-4556-95c8-7230c4789371 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:50.301517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-91c9997b-c76d-4556-95c8-7230c4789371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:50.384662 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 484/952] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:50.409268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e6e36-230a-40b4-99a9-c3773df84cf1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:50.424170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03febb3-7d86-4d20-b79b-06b3c7932b83 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:50.424455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03febb3-7d86-4d20-b79b-06b3c7932b83 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:50.424733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03febb3-7d86-4d20-b79b-06b3c7932b83 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:50.429896 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 485/953] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:50.437155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591659f-fcd6-43af-9d43-9906b117a433 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:50.437440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591659f-fcd6-43af-9d43-9906b117a433 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:50.437719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591659f-fcd6-43af-9d43-9906b117a433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:50.460836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5591659f-fcd6-43af-9d43-9906b117a433 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:50.462060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5591659f-fcd6-43af-9d43-9906b117a433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:50.504524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:50.533769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e6e36-230a-40b4-99a9-c3773df84cf1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:50.541154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:50.553817 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 486/954] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:50.555494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef5b64-3b58-4305-b423-968fcb865198 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:50.555960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef5b64-3b58-4305-b423-968fcb865198 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:50.556411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef5b64-3b58-4305-b423-968fcb865198 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:50.557863 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 487/955] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:50.563349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f46992-20c9-4b23-9bc7-422147cc11d5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:50.563950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f46992-20c9-4b23-9bc7-422147cc11d5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:50.564456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f46992-20c9-4b23-9bc7-422147cc11d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:50.585547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828bfa9-5521-4c5b-9ea4-cd906e6b291a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:50.585919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-42f46992-20c9-4b23-9bc7-422147cc11d5 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:50.586393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828bfa9-5521-4c5b-9ea4-cd906e6b291a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:50.586678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828bfa9-5521-4c5b-9ea4-cd906e6b291a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:50.586953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828bfa9-5521-4c5b-9ea4-cd906e6b291a None admin] PATH_INFO: `/v3/services/compute` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:50.587118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-42f46992-20c9-4b23-9bc7-422147cc11d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:50.587609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828bfa9-5521-4c5b-9ea4-cd906e6b291a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:50.589071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828bfa9-5521-4c5b-9ea4-cd906e6b291a None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:50.600915 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-0828bfa9-5521-4c5b-9ea4-cd906e6b291a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 28 20:42:50.600915 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:50.600915 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:50.600915 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:50.600915 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:50.600915 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:50.600915 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:50.600915 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:50.600915 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 20:42:50.600915 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:50.600915 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:50.600915 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:50.600915 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:50.600915 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:50.600915 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:50.600915 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:50.600915 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:50.601933 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:50.601933 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:50.601933 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:50.601933 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:50.601933 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:50.601933 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:50.601933 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:50.601933 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:50.601933 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:50.601933 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:50.601933 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:50.601933 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:50.601933 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:50.601933 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:50.601933 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:50.601933 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:50.602972 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:50.602972 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:50.602972 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:50.602972 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:50.602972 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:50.602972 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:50.602972 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:50.602972 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:50.602972 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:50.602972 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:50.602972 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:50.602972 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:50.602972 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 20:42:50.602972 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:50.602972 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 469/956] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:50.604594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828bfa9-5521-4c5b-9ea4-cd906e6b291a None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:50.664381 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 488/957] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:50.665016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f46992-20c9-4b23-9bc7-422147cc11d5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:50.719242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828bfa9-5521-4c5b-9ea4-cd906e6b291a None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:50.757402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b28f8-f4cd-40e4-84b3-0a4f1a0bbf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:50.758246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b28f8-f4cd-40e4-84b3-0a4f1a0bbf86 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:50.758616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b28f8-f4cd-40e4-84b3-0a4f1a0bbf86 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:50.758842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b28f8-f4cd-40e4-84b3-0a4f1a0bbf86 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:50.767558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:50.772155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b28f8-f4cd-40e4-84b3-0a4f1a0bbf86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:42:50.773559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f46992-20c9-4b23-9bc7-422147cc11d5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:50.782864 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-107b28f8-f4cd-40e4-84b3-0a4f1a0bbf86 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 20:42:50.782864 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:50.782864 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:50.782864 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:50.782864 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:50.782864 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:50.782864 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:50.782864 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:50.782864 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:50.782864 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:50.782864 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:50.782864 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:50.782864 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:42:50.782864 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:42:50.782864 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:42:50.782864 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:42:50.783888 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:42:50.783888 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:42:50.783888 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:42:50.783888 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:42:50.783888 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:50.783888 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:50.783888 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:50.783888 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:50.783888 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:50.783888 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:50.783888 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:50.783888 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:50.783888 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:50.783888 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:50.783888 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:50.783888 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:50.785298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:50.785298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:50.785298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:50.785298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:50.785298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:42:50.785298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:42:50.785298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:42:50.785298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:42:50.785298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:42:50.785298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:42:50.785298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 20:42:50.785298 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:50.785298 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 470/958] 10.222.0.33 () {56 vars in 1120 bytes} [Sat May 28 20:42:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:50.792080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b28f8-f4cd-40e4-84b3-0a4f1a0bbf86 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:50.796767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:50.813095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e543b46-6b87-4f76-9fef-92bb4d7c2bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:50.813762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e543b46-6b87-4f76-9fef-92bb4d7c2bd5 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:50.814080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e543b46-6b87-4f76-9fef-92bb4d7c2bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:50.814361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e543b46-6b87-4f76-9fef-92bb4d7c2bd5 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:50.815140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e543b46-6b87-4f76-9fef-92bb4d7c2bd5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:50.815446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e543b46-6b87-4f76-9fef-92bb4d7c2bd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:50.816864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e543b46-6b87-4f76-9fef-92bb4d7c2bd5 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:50.821553 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 489/959] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 20:42:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:50.822368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e543b46-6b87-4f76-9fef-92bb4d7c2bd5 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:50.910211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b28f8-f4cd-40e4-84b3-0a4f1a0bbf86 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:50.932691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e543b46-6b87-4f76-9fef-92bb4d7c2bd5 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:50.946664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0d155-8b6a-4544-b9d9-40d0feafa2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:50.947564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0d155-8b6a-4544-b9d9-40d0feafa2eb None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:50.947990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0d155-8b6a-4544-b9d9-40d0feafa2eb None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:50.948420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0d155-8b6a-4544-b9d9-40d0feafa2eb None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:50.949169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0d155-8b6a-4544-b9d9-40d0feafa2eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:50.950643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0d155-8b6a-4544-b9d9-40d0feafa2eb None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:50.952628 ncdv-master-1 devstack@keystone.service[103104]: WARNING keystone.server.flask.common [None req-c0c0d155-8b6a-4544-b9d9-40d0feafa2eb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 28 20:42:50.970230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474e03e-ba99-4306-b477-581ec1f15611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:50.970905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474e03e-ba99-4306-b477-581ec1f15611 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:50.971190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474e03e-ba99-4306-b477-581ec1f15611 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:50.971469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474e03e-ba99-4306-b477-581ec1f15611 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:50.972829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474e03e-ba99-4306-b477-581ec1f15611 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:50.973169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474e03e-ba99-4306-b477-581ec1f15611 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:50.974608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474e03e-ba99-4306-b477-581ec1f15611 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:50.982842 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 490/960] 10.222.0.33 () {56 vars in 1115 bytes} [Sat May 28 20:42:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:50.983686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474e03e-ba99-4306-b477-581ec1f15611 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:50.990669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.sql.core [None req-c0c0d155-8b6a-4544-b9d9-40d0feafa2eb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 28 20:42:50.990669 ncdv-master-1 devstack@keystone.service[103104]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 28 20:42:50.990669 ncdv-master-1 devstack@keystone.service[103104]: [parameters: {'id': 'eb49edf624a242368c08f92b68c9a4b9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 28 20:42:50.990669 ncdv-master-1 devstack@keystone.service[103104]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=103104) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 28 20:42:51.007478 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-c0c0d155-8b6a-4544-b9d9-40d0feafa2eb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eb49edf624a242368c08f92b68c9a4b9.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eb49edf624a242368c08f92b68c9a4b9. May 28 20:42:51.007478 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:51.007478 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 20:42:51.007478 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 20:42:51.007478 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 20:42:51.007478 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 20:42:51.007478 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 20:42:51.007478 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application result = self._query(query) May 28 20:42:51.007478 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 20:42:51.007478 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application conn.query(q) May 28 20:42:51.007478 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 20:42:51.007478 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 20:42:51.007478 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application result.read() May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 28 20:42:51.008453 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return project_ref.to_dict() May 28 20:42:51.009560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 20:42:51.009560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application next(self.gen) May 28 20:42:51.009560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 28 20:42:51.009560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application yield resource May 28 20:42:51.009560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 20:42:51.009560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application next(self.gen) May 28 20:42:51.009560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 28 20:42:51.009560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.session.rollback() May 28 20:42:51.009560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 28 20:42:51.009560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.force_reraise() May 28 20:42:51.009560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 28 20:42:51.009560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise self.value May 28 20:42:51.009560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 28 20:42:51.009560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 28 20:42:51.009560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 28 20:42:51.009560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application session.commit() May 28 20:42:51.009560 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 28 20:42:51.011003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 28 20:42:51.011003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 28 20:42:51.011003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._prepare_impl() May 28 20:42:51.011003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 28 20:42:51.011003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.session.flush() May 28 20:42:51.011003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 28 20:42:51.011003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._flush(objects) May 28 20:42:51.011003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 28 20:42:51.011003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 28 20:42:51.011003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 28 20:42:51.011003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application compat.raise_( May 28 20:42:51.011003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 20:42:51.011003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception May 28 20:42:51.011003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 28 20:42:51.011003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application flush_context.execute() May 28 20:42:51.011003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 28 20:42:51.011003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rec.execute(self) May 28 20:42:51.011940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 28 20:42:51.011940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 28 20:42:51.011940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 28 20:42:51.011940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application _emit_insert_statements( May 28 20:42:51.011940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 28 20:42:51.011940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application c = connection._execute_20( May 28 20:42:51.011940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 28 20:42:51.011940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 28 20:42:51.011940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 28 20:42:51.011940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 28 20:42:51.011940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 28 20:42:51.011940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ret = self._execute_context( May 28 20:42:51.011940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 28 20:42:51.011940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 28 20:42:51.011940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 28 20:42:51.012739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 28 20:42:51.012739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 20:42:51.012739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception May 28 20:42:51.012739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 20:42:51.012739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 20:42:51.012739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 20:42:51.012739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 20:42:51.012739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 20:42:51.012739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application result = self._query(query) May 28 20:42:51.012739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 20:42:51.012739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application conn.query(q) May 28 20:42:51.012739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 20:42:51.012739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 20:42:51.012739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 20:42:51.012739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application result.read() May 28 20:42:51.012739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 20:42:51.012739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 20:42:51.015090 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 20:42:51.015090 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 20:42:51.015090 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 20:42:51.015090 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 20:42:51.015090 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 20:42:51.015090 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 20:42:51.015090 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 28 20:42:51.015090 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 28 20:42:51.015090 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [parameters: {'id': 'eb49edf624a242368c08f92b68c9a4b9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 28 20:42:51.015090 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 28 20:42:51.015090 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:51.015090 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:51.015090 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:51.015090 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:51.015090 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 28 20:42:51.017558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 28 20:42:51.017558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 28 20:42:51.017558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 28 20:42:51.017558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 28 20:42:51.017558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:51.017558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:51.017558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:51.017558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:51.017558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:51.017558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:51.017558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:51.017558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:51.017558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:51.017558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:51.017558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:51.017558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:51.017558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:51.020441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:51.020441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 28 20:42:51.020441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 28 20:42:51.020441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:51.020441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:51.020441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 28 20:42:51.020441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.Conflict( May 28 20:42:51.020441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id eb49edf624a242368c08f92b68c9a4b9. May 28 20:42:51.020441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:51.020441 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 471/961] 10.222.0.33 () {60 vars in 1140 bytes} [Sat May 28 20:42:50 2022] POST /identity/v3/projects => generated 262 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:51.020441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0d155-8b6a-4544-b9d9-40d0feafa2eb None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:51.056366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:51.077426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:51.115794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474e03e-ba99-4306-b477-581ec1f15611 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:51.128812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0d155-8b6a-4544-b9d9-40d0feafa2eb None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:51.155108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a3427-7a6c-45df-8419-51f671699621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:51.156021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a3427-7a6c-45df-8419-51f671699621 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:51.156310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a3427-7a6c-45df-8419-51f671699621 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:51.156997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684a3427-7a6c-45df-8419-51f671699621 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:51.158009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a3427-7a6c-45df-8419-51f671699621 None admin] RBAC: Adding query filter params (type=compute) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:51.158326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a3427-7a6c-45df-8419-51f671699621 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:51.160933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684a3427-7a6c-45df-8419-51f671699621 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:51.164899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd0177-0493-40e1-808c-3e8cf59be9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:51.165684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd0177-0493-40e1-808c-3e8cf59be9a7 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:51.165969 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 491/962] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 20:42:50 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:51.166072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd0177-0493-40e1-808c-3e8cf59be9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:51.166303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd0177-0493-40e1-808c-3e8cf59be9a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:51.170827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a3427-7a6c-45df-8419-51f671699621 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:51.188808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd0177-0493-40e1-808c-3e8cf59be9a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:51.191031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cd0177-0493-40e1-808c-3e8cf59be9a7 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:51.211762 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-d6cd0177-0493-40e1-808c-3e8cf59be9a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:51.211762 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:51.211762 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:51.211762 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:51.211762 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:51.211762 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:51.211762 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:51.211762 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:51.211762 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:51.211762 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:51.211762 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:51.211762 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:51.211762 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:51.211762 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:51.211762 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:51.212768 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:51.212768 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:51.212768 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:51.212768 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:51.212768 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:51.212768 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:51.212768 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:51.212768 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:51.212768 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:51.212768 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:51.212768 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:51.212768 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:51.212768 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:51.212768 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:51.212768 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:51.212768 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:51.212768 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:51.213663 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:51.213663 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:51.213663 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:51.213663 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:51.216198 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 472/963] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:51.221376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd0177-0493-40e1-808c-3e8cf59be9a7 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:51.290789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684a3427-7a6c-45df-8419-51f671699621 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:51.305557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:51.336967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b19f64-e243-42ba-8fda-57769de8e5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:51.337706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b19f64-e243-42ba-8fda-57769de8e5d0 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:51.338026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b19f64-e243-42ba-8fda-57769de8e5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:51.338309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b19f64-e243-42ba-8fda-57769de8e5d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:51.346696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b19f64-e243-42ba-8fda-57769de8e5d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:51.347926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b19f64-e243-42ba-8fda-57769de8e5d0 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:51.364441 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-b4b19f64-e243-42ba-8fda-57769de8e5d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:51.364441 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:51.364441 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:51.364441 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:51.364441 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:51.364441 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:51.364441 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:51.364441 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:51.364441 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:51.364441 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:51.364441 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:51.364441 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:51.364441 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:51.364441 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:51.364441 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:51.365417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:51.365417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:51.365417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:51.365417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:51.365417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:51.365417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:51.365417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:51.365417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:51.365417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:51.365417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:51.365417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:51.365417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:51.365417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:51.365417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:51.365417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:51.365417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:51.365417 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:51.366345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:51.366345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:51.366345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:51.366345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:51.366345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:51.366345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:51.366345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:51.366345 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:51.366345 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 492/964] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:51.371527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b19f64-e243-42ba-8fda-57769de8e5d0 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:51.378931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd0177-0493-40e1-808c-3e8cf59be9a7 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:51.392190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:51.424453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac368512-08ac-42f6-8a52-128286f2d0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:51.425326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac368512-08ac-42f6-8a52-128286f2d0ff None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:51.425713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac368512-08ac-42f6-8a52-128286f2d0ff None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:51.426136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac368512-08ac-42f6-8a52-128286f2d0ff None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:51.426991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac368512-08ac-42f6-8a52-128286f2d0ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:51.427402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac368512-08ac-42f6-8a52-128286f2d0ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:51.428879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac368512-08ac-42f6-8a52-128286f2d0ff None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:51.449635 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 473/965] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:51.450813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac368512-08ac-42f6-8a52-128286f2d0ff None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:51.494256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b19f64-e243-42ba-8fda-57769de8e5d0 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:51.531166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8234d-ecd2-4671-adc8-47da301c7b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:51.532169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8234d-ecd2-4671-adc8-47da301c7b38 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:51.532593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8234d-ecd2-4671-adc8-47da301c7b38 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:51.533020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8234d-ecd2-4671-adc8-47da301c7b38 None admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:51.534036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8234d-ecd2-4671-adc8-47da301c7b38 None admin] RBAC: Adding query filter params (name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:51.534524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8234d-ecd2-4671-adc8-47da301c7b38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:51.537866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8234d-ecd2-4671-adc8-47da301c7b38 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:51.556759 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 493/966] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 20:42:51 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:51.567999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac368512-08ac-42f6-8a52-128286f2d0ff None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:51.607013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e783a5-07c8-4e49-9341-083c87c3b5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:51.608220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e783a5-07c8-4e49-9341-083c87c3b5ff None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:51.608744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e783a5-07c8-4e49-9341-083c87c3b5ff None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:51.609231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e783a5-07c8-4e49-9341-083c87c3b5ff None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:51.618539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e783a5-07c8-4e49-9341-083c87c3b5ff None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:51.620464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e783a5-07c8-4e49-9341-083c87c3b5ff None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:51.637218 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-99e783a5-07c8-4e49-9341-083c87c3b5ff None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 20:42:51.637218 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:51.637218 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:51.637218 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:51.637218 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:51.637218 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:51.637218 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:51.637218 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:51.637218 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:51.637218 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:51.637218 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:51.637218 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:51.637218 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:51.637218 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:51.637218 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:51.637218 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:51.638377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:51.639348 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:51.639348 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:51.639348 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:51.639348 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:51.639348 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:51.639348 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:51.639348 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:51.639348 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:51.639348 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 28 20:42:51.639348 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:51.642438 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 474/967] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 20:42:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:51.651014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8234d-ecd2-4671-adc8-47da301c7b38 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:51.684481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:51.762782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8234d-ecd2-4671-adc8-47da301c7b38 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:51.799372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f478123-8117-4a82-bcc1-6e80ee0b9cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:51.800863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f478123-8117-4a82-bcc1-6e80ee0b9cac None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:51.801295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f478123-8117-4a82-bcc1-6e80ee0b9cac None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:51.801716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f478123-8117-4a82-bcc1-6e80ee0b9cac None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:51.802725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f478123-8117-4a82-bcc1-6e80ee0b9cac None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:51.803200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f478123-8117-4a82-bcc1-6e80ee0b9cac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:51.804780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f478123-8117-4a82-bcc1-6e80ee0b9cac None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:51.824169 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 494/968] 10.222.0.33 () {56 vars in 1149 bytes} [Sat May 28 20:42:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:51.830186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e783a5-07c8-4e49-9341-083c87c3b5ff None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:51.859909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:51.936437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e783a5-07c8-4e49-9341-083c87c3b5ff None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:51.972743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc2fa2-a840-4ea9-922e-4fb29dc10726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:51.973396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc2fa2-a840-4ea9-922e-4fb29dc10726 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:51.973677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc2fa2-a840-4ea9-922e-4fb29dc10726 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:51.973981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc2fa2-a840-4ea9-922e-4fb29dc10726 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:52.046049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc2fa2-a840-4ea9-922e-4fb29dc10726 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:52.047933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc2fa2-a840-4ea9-922e-4fb29dc10726 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:52.082611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:52.153022 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 475/969] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:42:51 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 28 20:42:52.158697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f478123-8117-4a82-bcc1-6e80ee0b9cac None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:52.251388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:52.264403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f478123-8117-4a82-bcc1-6e80ee0b9cac None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:52.304075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b667d2-4d84-47e6-ae2b-47c3d7ba58d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:52.304893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b667d2-4d84-47e6-ae2b-47c3d7ba58d7 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:52.305179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b667d2-4d84-47e6-ae2b-47c3d7ba58d7 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:52.305456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b667d2-4d84-47e6-ae2b-47c3d7ba58d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:52.317299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b667d2-4d84-47e6-ae2b-47c3d7ba58d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:52.318795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b667d2-4d84-47e6-ae2b-47c3d7ba58d7 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:52.337135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-91b667d2-4d84-47e6-ae2b-47c3d7ba58d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:52.337135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:52.337135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:52.337135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:52.337135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:52.337135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:52.337135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:52.337135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:52.337135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:52.337135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:52.337135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:52.337135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:52.337135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:42:52.337135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:42:52.337135 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:42:52.338129 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:42:52.338129 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:52.338129 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:52.338129 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:52.338129 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:52.338129 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:52.338129 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:52.338129 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:52.338129 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:52.338129 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:52.338129 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:52.338129 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:52.338129 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:52.338129 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:52.338129 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:52.338129 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:52.338129 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:52.339023 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:42:52.339023 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:42:52.339023 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:42:52.339023 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:52.339023 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 495/970] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:52.344497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc2fa2-a840-4ea9-922e-4fb29dc10726 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:52.448232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc2fa2-a840-4ea9-922e-4fb29dc10726 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:52.475699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:52.479895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8bb8c-a5ae-479d-9270-24fd7b39ab40 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:52.480182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8bb8c-a5ae-479d-9270-24fd7b39ab40 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:52.480538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8bb8c-a5ae-479d-9270-24fd7b39ab40 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:52.482074 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 496/971] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:52.483555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f935d-7358-4694-be33-6459abcbc2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:52.484890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f935d-7358-4694-be33-6459abcbc2c7 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:52.485175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f935d-7358-4694-be33-6459abcbc2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:52.485452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69f935d-7358-4694-be33-6459abcbc2c7 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:52.486226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f935d-7358-4694-be33-6459abcbc2c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:52.486532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f935d-7358-4694-be33-6459abcbc2c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:52.487411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746da78-9aa5-4a10-a6ad-6139e2e6d464 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:52.488259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746da78-9aa5-4a10-a6ad-6139e2e6d464 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:52.488443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69f935d-7358-4694-be33-6459abcbc2c7 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:52.488538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746da78-9aa5-4a10-a6ad-6139e2e6d464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:52.501990 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 476/972] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:52.508897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f935d-7358-4694-be33-6459abcbc2c7 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:52.513534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c746da78-9aa5-4a10-a6ad-6139e2e6d464 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:52.515035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c746da78-9aa5-4a10-a6ad-6139e2e6d464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:52.606312 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 497/973] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:52.629938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746da78-9aa5-4a10-a6ad-6139e2e6d464 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:52.631575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69f935d-7358-4694-be33-6459abcbc2c7 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:52.667104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb105c-913c-4306-b423-1564a78a9b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:52.668223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb105c-913c-4306-b423-1564a78a9b7a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:52.668703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb105c-913c-4306-b423-1564a78a9b7a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:52.669366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb105c-913c-4306-b423-1564a78a9b7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:52.677280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb105c-913c-4306-b423-1564a78a9b7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:52.679143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bb105c-913c-4306-b423-1564a78a9b7a None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:52.694909 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-68bb105c-913c-4306-b423-1564a78a9b7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:52.694909 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:52.694909 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:52.694909 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:52.694909 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:52.694909 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:52.694909 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:52.694909 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:52.694909 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:52.694909 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:52.694909 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:52.694909 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:52.694909 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:42:52.694909 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:42:52.694909 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:42:52.695778 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:42:52.695778 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:52.695778 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:52.695778 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:52.695778 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:52.695778 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:52.695778 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:52.695778 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:52.695778 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:52.695778 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:52.695778 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:52.695778 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:52.695778 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:52.695778 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:52.695778 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:52.695778 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:52.695778 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:52.696666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:42:52.696666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:42:52.696666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:42:52.696666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:42:52.696666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:42:52.696666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:42:52.696666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:42:52.696666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:52.699318 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 477/974] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:42:52.705405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb105c-913c-4306-b423-1564a78a9b7a None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:52.713718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:52.745448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746da78-9aa5-4a10-a6ad-6139e2e6d464 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:52.777555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:52.782063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfb62a-98d4-49ac-ad82-4821d6b07b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:52.782990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfb62a-98d4-49ac-ad82-4821d6b07b77 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:52.783282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfb62a-98d4-49ac-ad82-4821d6b07b77 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:52.783625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dfb62a-98d4-49ac-ad82-4821d6b07b77 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:52.784354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dfb62a-98d4-49ac-ad82-4821d6b07b77 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:52.786714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dfb62a-98d4-49ac-ad82-4821d6b07b77 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:52.801717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-83dfb62a-98d4-49ac-ad82-4821d6b07b77 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 20:42:52.801717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:52.801717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:52.801717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:52.801717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:52.801717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:52.801717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:52.801717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:52.801717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 20:42:52.801717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:52.801717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:52.801717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:52.801717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:52.801717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:52.801717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:52.801717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:52.801717 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:52.802831 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:52.802831 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:52.802831 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:52.802831 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:52.802831 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:52.802831 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:52.802831 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:52.802831 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:52.802831 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:52.802831 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:52.802831 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:52.802831 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:52.802831 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:52.802831 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:52.802831 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:52.802831 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:52.803706 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:52.803706 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:52.803706 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:52.803706 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:52.803706 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:52.803706 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:52.803706 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:52.803706 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:52.803706 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:52.803706 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:52.803706 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:52.803706 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:52.803706 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 20:42:52.803706 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:52.809273 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 498/975] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 20:42:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:42:52.810942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfb62a-98d4-49ac-ad82-4821d6b07b77 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:52.840165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb105c-913c-4306-b423-1564a78a9b7a None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:52.880025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476035d-2a95-4697-9ba2-bf57f5fdaedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:52.881038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476035d-2a95-4697-9ba2-bf57f5fdaedc None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:52.881473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476035d-2a95-4697-9ba2-bf57f5fdaedc None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:52.882124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476035d-2a95-4697-9ba2-bf57f5fdaedc None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:52.883410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476035d-2a95-4697-9ba2-bf57f5fdaedc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:52.883870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476035d-2a95-4697-9ba2-bf57f5fdaedc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:52.885470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476035d-2a95-4697-9ba2-bf57f5fdaedc None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:52.907323 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 478/976] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:42:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:52.907796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476035d-2a95-4697-9ba2-bf57f5fdaedc None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:52.944736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dfb62a-98d4-49ac-ad82-4821d6b07b77 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:52.980812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7883b-e76c-46c0-9eda-6f246f7fb5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:52.981825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7883b-e76c-46c0-9eda-6f246f7fb5a8 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:52.982356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7883b-e76c-46c0-9eda-6f246f7fb5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:52.982866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7883b-e76c-46c0-9eda-6f246f7fb5a8 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:52.983830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7883b-e76c-46c0-9eda-6f246f7fb5a8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:52.984340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7883b-e76c-46c0-9eda-6f246f7fb5a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:52.986009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad7883b-e76c-46c0-9eda-6f246f7fb5a8 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:52.987388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:52.992823 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 499/977] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 20:42:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:53.000744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7883b-e76c-46c0-9eda-6f246f7fb5a8 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:53.017703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476035d-2a95-4697-9ba2-bf57f5fdaedc None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:53.028849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:53.057187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b3667-6080-427c-bf53-6c2b227e6fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:53.057997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b3667-6080-427c-bf53-6c2b227e6fc6 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:53.058282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b3667-6080-427c-bf53-6c2b227e6fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:53.058555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b3667-6080-427c-bf53-6c2b227e6fc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:53.059389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b3667-6080-427c-bf53-6c2b227e6fc6 None admin] RBAC: Adding query filter params (role.id=50445f06754843f49827b854ec728850, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=c1c1ea9ba4c740b3b7ed67c8335bfad2) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:53.059702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b3667-6080-427c-bf53-6c2b227e6fc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:53.064845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b3667-6080-427c-bf53-6c2b227e6fc6 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:53.072844 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 479/978] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:42:52 2022] GET /identity/v3/role_assignments?user.id=c1c1ea9ba4c740b3b7ed67c8335bfad2&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=50445f06754843f49827b854ec728850 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:53.107704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad7883b-e76c-46c0-9eda-6f246f7fb5a8 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:53.147461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e77fb8-3dce-419a-af58-1451343288ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:53.148146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e77fb8-3dce-419a-af58-1451343288ac None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:53.148430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e77fb8-3dce-419a-af58-1451343288ac None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:53.148708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e77fb8-3dce-419a-af58-1451343288ac None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:53.149433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e77fb8-3dce-419a-af58-1451343288ac None admin] RBAC: Adding query filter params (type=object-store) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:53.149742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e77fb8-3dce-419a-af58-1451343288ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:53.151182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e77fb8-3dce-419a-af58-1451343288ac None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:53.157461 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 500/979] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 20:42:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:53.163583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b3667-6080-427c-bf53-6c2b227e6fc6 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:53.278947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b3667-6080-427c-bf53-6c2b227e6fc6 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:53.314799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7034589-cb11-4b9a-b668-65692ff94e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:53.315452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7034589-cb11-4b9a-b668-65692ff94e00 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:53.315753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7034589-cb11-4b9a-b668-65692ff94e00 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:53.316625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7034589-cb11-4b9a-b668-65692ff94e00 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:53.317511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7034589-cb11-4b9a-b668-65692ff94e00 None admin] RBAC: Adding query filter params (interface=admin, service_id=1520b8cc69444fdd936e9afdc5a93025, region_id=RegionOne) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:53.317788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7034589-cb11-4b9a-b668-65692ff94e00 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:53.319240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7034589-cb11-4b9a-b668-65692ff94e00 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:53.324945 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 480/980] 10.222.0.33 () {56 vars in 1249 bytes} [Sat May 28 20:42:53 2022] GET /identity/v3/endpoints?service_id=1520b8cc69444fdd936e9afdc5a93025&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:53.330162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e77fb8-3dce-419a-af58-1451343288ac None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:53.400238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:53.440312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e77fb8-3dce-419a-af58-1451343288ac None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:53.476155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7685581-5662-4651-b584-84e25090b190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:53.476827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7685581-5662-4651-b584-84e25090b190 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:53.477115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7685581-5662-4651-b584-84e25090b190 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:53.477389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7685581-5662-4651-b584-84e25090b190 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:53.478108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7685581-5662-4651-b584-84e25090b190 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:53.479490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7685581-5662-4651-b584-84e25090b190 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:53.486017 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 501/981] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 20:42:53 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 20:42:53.565139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42205b77-7ef3-4be7-8f9e-0e8286f73db7 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:53.565646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42205b77-7ef3-4be7-8f9e-0e8286f73db7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:53.565646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42205b77-7ef3-4be7-8f9e-0e8286f73db7 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:53.567470 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 481/982] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:53.573441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf74bd6-43be-4dd6-aee1-91f4fa5ed0d9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:53.573770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf74bd6-43be-4dd6-aee1-91f4fa5ed0d9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:53.574049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf74bd6-43be-4dd6-aee1-91f4fa5ed0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:53.591940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-baf74bd6-43be-4dd6-aee1-91f4fa5ed0d9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:53.593168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-baf74bd6-43be-4dd6-aee1-91f4fa5ed0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:53.668603 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 502/983] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:53.694517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42205b77-7ef3-4be7-8f9e-0e8286f73db7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:53.736479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:53.815709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42205b77-7ef3-4be7-8f9e-0e8286f73db7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:53.851238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940c602e-c07c-4994-b97f-ce5fed32d323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:53.852359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c602e-c07c-4994-b97f-ce5fed32d323 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:53.852856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c602e-c07c-4994-b97f-ce5fed32d323 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:53.853346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940c602e-c07c-4994-b97f-ce5fed32d323 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:53.854375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c602e-c07c-4994-b97f-ce5fed32d323 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:53.855969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940c602e-c07c-4994-b97f-ce5fed32d323 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:53.910484 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 482/984] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 20:42:53 2022] POST /identity/v3/services => generated 242 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:42:54.096815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb39e2-ba5a-4307-ab2d-5e3689073af2 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:54.097666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb39e2-ba5a-4307-ab2d-5e3689073af2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:54.097902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb39e2-ba5a-4307-ab2d-5e3689073af2 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:54.099559 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 503/985] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:54.104887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffbd20-4230-4665-8e79-fe0227491bf1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:54.105179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffbd20-4230-4665-8e79-fe0227491bf1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:54.105453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffbd20-4230-4665-8e79-fe0227491bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:54.123541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-02ffbd20-4230-4665-8e79-fe0227491bf1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:54.125065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-02ffbd20-4230-4665-8e79-fe0227491bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:54.203893 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 483/986] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:54.231380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb39e2-ba5a-4307-ab2d-5e3689073af2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:54.290319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:54.338322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb39e2-ba5a-4307-ab2d-5e3689073af2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:54.374281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc99f55-d0e1-4552-bc58-f379d88b79d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:54.374882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc99f55-d0e1-4552-bc58-f379d88b79d4 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:54.375167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc99f55-d0e1-4552-bc58-f379d88b79d4 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:54.375449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc99f55-d0e1-4552-bc58-f379d88b79d4 None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:54.376156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc99f55-d0e1-4552-bc58-f379d88b79d4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:54.377599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc99f55-d0e1-4552-bc58-f379d88b79d4 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:54.378977 ncdv-master-1 devstack@keystone.service[103103]: WARNING keystone.server.flask.common [None req-cbc99f55-d0e1-4552-bc58-f379d88b79d4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 28 20:42:54.420695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.sql.core [None req-cbc99f55-d0e1-4552-bc58-f379d88b79d4 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 28 20:42:54.420695 ncdv-master-1 devstack@keystone.service[103103]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 28 20:42:54.420695 ncdv-master-1 devstack@keystone.service[103103]: [parameters: {'user_id': 'd59ee5ce82754b46aa89d0d9aaecdccb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 28 20:42:54.420695 ncdv-master-1 devstack@keystone.service[103103]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=103103) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 28 20:42:54.429082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-cbc99f55-d0e1-4552-bc58-f379d88b79d4 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 28 20:42:54.429082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:54.429082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 20:42:54.429082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 20:42:54.429082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 20:42:54.429082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 20:42:54.429082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 20:42:54.429082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application result = self._query(query) May 28 20:42:54.429082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 20:42:54.429082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application conn.query(q) May 28 20:42:54.429082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 20:42:54.429082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 20:42:54.429082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 20:42:54.429082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application result.read() May 28 20:42:54.429082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 20:42:54.430046 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application next(self.gen) May 28 20:42:54.431039 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 28 20:42:54.431039 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application yield resource May 28 20:42:54.431039 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 20:42:54.431039 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application next(self.gen) May 28 20:42:54.431039 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 28 20:42:54.431039 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.session.rollback() May 28 20:42:54.431039 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 28 20:42:54.431039 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.force_reraise() May 28 20:42:54.431039 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 28 20:42:54.431039 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise self.value May 28 20:42:54.431039 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 28 20:42:54.431039 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 28 20:42:54.431039 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 28 20:42:54.431039 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application session.commit() May 28 20:42:54.431039 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 28 20:42:54.431039 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 28 20:42:54.431039 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 28 20:42:54.431922 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._prepare_impl() May 28 20:42:54.431922 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 28 20:42:54.431922 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.session.flush() May 28 20:42:54.431922 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 28 20:42:54.431922 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._flush(objects) May 28 20:42:54.431922 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 28 20:42:54.431922 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 28 20:42:54.431922 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 28 20:42:54.431922 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application compat.raise_( May 28 20:42:54.431922 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 20:42:54.431922 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception May 28 20:42:54.431922 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 28 20:42:54.431922 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application flush_context.execute() May 28 20:42:54.431922 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 28 20:42:54.431922 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rec.execute(self) May 28 20:42:54.431922 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 28 20:42:54.431922 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 28 20:42:54.432828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 28 20:42:54.432828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application _emit_insert_statements( May 28 20:42:54.432828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 28 20:42:54.432828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application result = connection._execute_20( May 28 20:42:54.432828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 28 20:42:54.432828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 28 20:42:54.432828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 28 20:42:54.432828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 28 20:42:54.432828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 28 20:42:54.432828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ret = self._execute_context( May 28 20:42:54.432828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 28 20:42:54.432828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 28 20:42:54.432828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 28 20:42:54.432828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 28 20:42:54.432828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 20:42:54.432828 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception May 28 20:42:54.435244 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 20:42:54.435244 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 20:42:54.435244 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 20:42:54.435244 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 20:42:54.435244 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 20:42:54.435244 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application result = self._query(query) May 28 20:42:54.435244 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 20:42:54.435244 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application conn.query(q) May 28 20:42:54.435244 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 20:42:54.435244 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 20:42:54.435244 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 20:42:54.435244 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application result.read() May 28 20:42:54.435244 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 20:42:54.435244 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 20:42:54.435244 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 20:42:54.435244 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 20:42:54.437196 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 20:42:54.437196 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 20:42:54.437196 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 20:42:54.437196 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 20:42:54.437196 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 28 20:42:54.437196 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 28 20:42:54.437196 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd59ee5ce82754b46aa89d0d9aaecdccb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 28 20:42:54.437196 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 28 20:42:54.437196 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:54.437196 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:54.437196 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:54.437196 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:54.437196 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:54.437196 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:54.437196 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:54.439356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:54.439356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:54.439356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:54.439356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:54.439356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:54.439356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:54.439356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:54.439356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 28 20:42:54.439356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 28 20:42:54.439356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:54.439356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:54.439356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:54.439356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:54.439356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:54.439356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:54.439356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 28 20:42:54.439356 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 28 20:42:54.442774 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 28 20:42:54.442774 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 28 20:42:54.442774 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 28 20:42:54.442774 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 28 20:42:54.442774 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 28 20:42:54.442774 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:54.442774 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 504/987] 10.222.0.33 () {60 vars in 1131 bytes} [Sat May 28 20:42:54 2022] POST /identity/v3/users => generated 160 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:54.443731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffbd20-4230-4665-8e79-fe0227491bf1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:54.462397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:54.541393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffbd20-4230-4665-8e79-fe0227491bf1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:54.577711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd82d93-e183-4cea-9e10-934bffce6d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:54.578529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd82d93-e183-4cea-9e10-934bffce6d3a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:54.578815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd82d93-e183-4cea-9e10-934bffce6d3a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:54.579094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd82d93-e183-4cea-9e10-934bffce6d3a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:54.586631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd82d93-e183-4cea-9e10-934bffce6d3a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:54.588079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd82d93-e183-4cea-9e10-934bffce6d3a None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:54.605097 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-9bd82d93-e183-4cea-9e10-934bffce6d3a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 28 20:42:54.605097 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:54.605097 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:54.605097 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:54.605097 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:54.605097 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:54.605097 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:54.605097 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:54.605097 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:54.605097 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:54.605097 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:54.605097 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:54.605097 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:42:54.605097 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:42:54.605097 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:42:54.605097 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:54.606083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:54.607012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:54.607012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:54.607012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:42:54.607012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:42:54.607012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:42:54.607012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:42:54.607012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:42:54.607012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:42:54.607012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 28 20:42:54.607012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:54.609521 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 484/988] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:42:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:54.616322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc99f55-d0e1-4552-bc58-f379d88b79d4 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:54.720930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc99f55-d0e1-4552-bc58-f379d88b79d4 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:54.742584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:54.759467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2137e200-973c-4f2d-be45-9cc43ecc0807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:54.760246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137e200-973c-4f2d-be45-9cc43ecc0807 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:54.760534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137e200-973c-4f2d-be45-9cc43ecc0807 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:54.760810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2137e200-973c-4f2d-be45-9cc43ecc0807 None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:54.761682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137e200-973c-4f2d-be45-9cc43ecc0807 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:54.762028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137e200-973c-4f2d-be45-9cc43ecc0807 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:54.763761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2137e200-973c-4f2d-be45-9cc43ecc0807 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:54.804511 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 505/989] 10.222.0.33 () {56 vars in 1101 bytes} [Sat May 28 20:42:54 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:55.506795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc09c9-65de-4482-a2e9-0d4195234542 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:55.507096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc09c9-65de-4482-a2e9-0d4195234542 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:55.507286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdc09c9-65de-4482-a2e9-0d4195234542 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:55.508803 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 485/990] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:55.514079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd658f-d0f0-4fff-9f12-67c577b452a4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:55.514523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd658f-d0f0-4fff-9f12-67c577b452a4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:55.514687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd658f-d0f0-4fff-9f12-67c577b452a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:55.531723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f8bd658f-d0f0-4fff-9f12-67c577b452a4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:55.533234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f8bd658f-d0f0-4fff-9f12-67c577b452a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:55.598625 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 506/991] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:55.622480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc09c9-65de-4482-a2e9-0d4195234542 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:55.708270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:55.730631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdc09c9-65de-4482-a2e9-0d4195234542 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:55.765996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7060969-b389-4caa-a2e8-3b6b251176ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:55.766707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7060969-b389-4caa-a2e8-3b6b251176ba None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:55.766990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7060969-b389-4caa-a2e8-3b6b251176ba None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:55.767267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7060969-b389-4caa-a2e8-3b6b251176ba None admin] PATH_INFO: `/v3/services/image` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:55.767924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7060969-b389-4caa-a2e8-3b6b251176ba None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:55.769339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7060969-b389-4caa-a2e8-3b6b251176ba None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:55.780842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-e7060969-b389-4caa-a2e8-3b6b251176ba None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 28 20:42:55.780842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:55.780842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:55.780842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:55.780842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:55.780842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:55.780842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:55.780842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:55.780842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 20:42:55.780842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:55.780842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:55.780842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:55.780842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:55.780842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:55.780842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:55.780842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:55.780842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:55.781828 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:55.781828 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:55.781828 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:55.781828 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:55.781828 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:55.781828 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:55.781828 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:55.781828 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:55.781828 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:55.781828 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:55.781828 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:55.781828 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:55.781828 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:55.781828 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:55.781828 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:55.781828 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:55.782734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:55.782734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:55.782734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:55.782734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:55.782734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:55.782734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:55.782734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:55.782734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:55.782734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:55.782734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:55.782734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:55.782734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:55.782734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 20:42:55.782734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:55.785946 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 486/992] 10.222.0.33 () {56 vars in 1105 bytes} [Sat May 28 20:42:55 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:55.790058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd658f-d0f0-4fff-9f12-67c577b452a4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:55.889219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd658f-d0f0-4fff-9f12-67c577b452a4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:55.898283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:55.899142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edcaf1-c7b3-4948-8524-63a69b97ed56 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:55.899425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edcaf1-c7b3-4948-8524-63a69b97ed56 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:55.899704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93edcaf1-c7b3-4948-8524-63a69b97ed56 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:55.901214 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 487/993] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:42:55.906051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724f381-2737-404a-93cc-8aaff512109b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:55.912974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724f381-2737-404a-93cc-8aaff512109b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:55.912974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724f381-2737-404a-93cc-8aaff512109b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:55.924081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e724f381-2737-404a-93cc-8aaff512109b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:55.925929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e724f381-2737-404a-93cc-8aaff512109b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:55.929060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220c6b0-b6e8-4bc1-bf88-03e86a8a4dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:55.929740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220c6b0-b6e8-4bc1-bf88-03e86a8a4dde None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:55.930055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220c6b0-b6e8-4bc1-bf88-03e86a8a4dde None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:55.930333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2220c6b0-b6e8-4bc1-bf88-03e86a8a4dde None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:55.931050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220c6b0-b6e8-4bc1-bf88-03e86a8a4dde None admin] RBAC: Adding query filter params (name=image) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:55.931360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220c6b0-b6e8-4bc1-bf88-03e86a8a4dde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:55.932752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2220c6b0-b6e8-4bc1-bf88-03e86a8a4dde None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:55.937817 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 507/994] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:55.943258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220c6b0-b6e8-4bc1-bf88-03e86a8a4dde None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:56.003783 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 488/995] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:42:56.027571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724f381-2737-404a-93cc-8aaff512109b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:56.052248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2220c6b0-b6e8-4bc1-bf88-03e86a8a4dde None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:56.086617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef30ef8-ba0d-4785-b9c1-c2f5945c5a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:56.087319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef30ef8-ba0d-4785-b9c1-c2f5945c5a42 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:56.087602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef30ef8-ba0d-4785-b9c1-c2f5945c5a42 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:56.087878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef30ef8-ba0d-4785-b9c1-c2f5945c5a42 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:56.088591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef30ef8-ba0d-4785-b9c1-c2f5945c5a42 None admin] RBAC: Adding query filter params (type=image) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:56.088920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef30ef8-ba0d-4785-b9c1-c2f5945c5a42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:56.090355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef30ef8-ba0d-4785-b9c1-c2f5945c5a42 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:56.094897 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 508/996] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:55 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:56.138487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:56.165092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724f381-2737-404a-93cc-8aaff512109b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:56.203842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4700ae6-1708-4bf2-a051-80be37b0856b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:56.204666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4700ae6-1708-4bf2-a051-80be37b0856b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:56.204951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4700ae6-1708-4bf2-a051-80be37b0856b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:56.205230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4700ae6-1708-4bf2-a051-80be37b0856b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:56.205937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4700ae6-1708-4bf2-a051-80be37b0856b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:56.207900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4700ae6-1708-4bf2-a051-80be37b0856b None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:56.224971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-a4700ae6-1708-4bf2-a051-80be37b0856b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 20:42:56.224971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:56.224971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:56.224971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:56.224971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:56.224971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:56.224971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:56.224971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:56.224971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 20:42:56.224971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:56.224971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:56.224971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:56.224971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:56.224971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:56.224971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:56.224971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:56.224971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:56.226091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:56.226091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:56.226091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:56.226091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:56.226091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:56.226091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:56.226091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:56.226091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:56.226091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:56.226091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:56.226091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:56.226091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:56.226091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:56.226091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:56.226091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:56.226091 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:56.235377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:56.235377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:56.235377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:56.235377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:56.235377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:56.235377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:56.235377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:56.235377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:56.235377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:56.235377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:56.235377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:56.235377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:56.235377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 28 20:42:56.235377 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:56.235377 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 489/997] 10.222.0.33 () {56 vars in 1126 bytes} [Sat May 28 20:42:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:42:56.236360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef30ef8-ba0d-4785-b9c1-c2f5945c5a42 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:56.273430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:56.315672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d00db3-0604-4b61-aefd-1656d8bde44e None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:56.315950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d00db3-0604-4b61-aefd-1656d8bde44e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:56.316184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d00db3-0604-4b61-aefd-1656d8bde44e None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:56.317569 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 490/998] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:56.322718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb5eb7-c060-47ed-9866-60979b23ed6f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:56.323006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb5eb7-c060-47ed-9866-60979b23ed6f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:56.323311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cb5eb7-c060-47ed-9866-60979b23ed6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:56.341568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b6cb5eb7-c060-47ed-9866-60979b23ed6f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:56.342803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b6cb5eb7-c060-47ed-9866-60979b23ed6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:56.344067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef30ef8-ba0d-4785-b9c1-c2f5945c5a42 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:56.381106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19062d7b-7750-4c2d-8059-39ed3bd00774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:56.381774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19062d7b-7750-4c2d-8059-39ed3bd00774 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:56.382095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19062d7b-7750-4c2d-8059-39ed3bd00774 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:56.382378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19062d7b-7750-4c2d-8059-39ed3bd00774 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:56.383091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19062d7b-7750-4c2d-8059-39ed3bd00774 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:56.383401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19062d7b-7750-4c2d-8059-39ed3bd00774 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:56.384777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19062d7b-7750-4c2d-8059-39ed3bd00774 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:56.389816 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 509/999] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 20:42:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:56.396251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19062d7b-7750-4c2d-8059-39ed3bd00774 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:56.423099 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 491/1000] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:42:56.446811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb5eb7-c060-47ed-9866-60979b23ed6f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:56.497768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:56.501617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19062d7b-7750-4c2d-8059-39ed3bd00774 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:56.523315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:56.539191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cee67a-5803-444e-bfb3-df687fea49d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:56.540033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cee67a-5803-444e-bfb3-df687fea49d4 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:56.540209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cee67a-5803-444e-bfb3-df687fea49d4 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:56.540526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cee67a-5803-444e-bfb3-df687fea49d4 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:56.541154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cee67a-5803-444e-bfb3-df687fea49d4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:56.541463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cee67a-5803-444e-bfb3-df687fea49d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:56.543366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cee67a-5803-444e-bfb3-df687fea49d4 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:56.553402 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 510/1001] 10.222.0.33 () {56 vars in 1122 bytes} [Sat May 28 20:42:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:42:56.554597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cee67a-5803-444e-bfb3-df687fea49d4 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:56.563042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cb5eb7-c060-47ed-9866-60979b23ed6f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:56.600560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11914bba-5557-4136-a64a-44a8fb48b520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:56.601350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11914bba-5557-4136-a64a-44a8fb48b520 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:56.601562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11914bba-5557-4136-a64a-44a8fb48b520 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:56.601862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11914bba-5557-4136-a64a-44a8fb48b520 None admin] PATH_INFO: `/v3/services/compute` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:56.602708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11914bba-5557-4136-a64a-44a8fb48b520 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:56.603898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11914bba-5557-4136-a64a-44a8fb48b520 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:56.615955 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-11914bba-5557-4136-a64a-44a8fb48b520 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 28 20:42:56.615955 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:56.615955 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:56.615955 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:56.615955 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:56.615955 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:56.615955 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:56.615955 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:56.615955 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 20:42:56.615955 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:56.615955 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:56.615955 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:56.615955 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:56.615955 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:56.615955 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:56.615955 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:56.615955 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:56.617012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:56.617012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:56.617012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:56.617012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:56.617012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:56.617012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:56.617012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:56.617012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:56.617012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:56.617012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:56.617012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:56.617012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:56.617012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:56.617012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:56.617012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:56.617012 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:56.617905 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:56.617905 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:42:56.617905 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:56.617905 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:56.617905 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:56.617905 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:56.617905 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:56.617905 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:56.617905 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:56.617905 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:56.617905 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:56.617905 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:56.617905 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 20:42:56.617905 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:42:56.625991 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 492/1002] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:56.627535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11914bba-5557-4136-a64a-44a8fb48b520 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:56.663229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cee67a-5803-444e-bfb3-df687fea49d4 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:56.699381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fa1e8-66f3-4c50-a98b-2829932c280b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:56.700044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fa1e8-66f3-4c50-a98b-2829932c280b None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:56.700327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fa1e8-66f3-4c50-a98b-2829932c280b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:56.700606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fa1e8-66f3-4c50-a98b-2829932c280b None admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:56.701352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415fa1e8-66f3-4c50-a98b-2829932c280b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:56.702746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415fa1e8-66f3-4c50-a98b-2829932c280b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:56.738522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11914bba-5557-4136-a64a-44a8fb48b520 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:56.772097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c626b6-3e20-4072-9464-eb06c1a269a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:56.772761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c626b6-3e20-4072-9464-eb06c1a269a2 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:56.773047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c626b6-3e20-4072-9464-eb06c1a269a2 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:56.773327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c626b6-3e20-4072-9464-eb06c1a269a2 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:56.774068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c626b6-3e20-4072-9464-eb06c1a269a2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:56.774565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c626b6-3e20-4072-9464-eb06c1a269a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:56.775748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c626b6-3e20-4072-9464-eb06c1a269a2 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:56.776817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:56.786151 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 493/1003] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 20:42:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:56.786327 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 511/1004] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 20:42:56 2022] POST /identity/v3/endpoints => generated 325 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 20:42:56.787495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c626b6-3e20-4072-9464-eb06c1a269a2 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:56.900628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c626b6-3e20-4072-9464-eb06c1a269a2 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:56.943938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6565808-a81b-4ce8-88e5-3b5f9934b1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:56.944618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6565808-a81b-4ce8-88e5-3b5f9934b1e8 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:56.944902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6565808-a81b-4ce8-88e5-3b5f9934b1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:56.945178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6565808-a81b-4ce8-88e5-3b5f9934b1e8 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:56.945919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6565808-a81b-4ce8-88e5-3b5f9934b1e8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:56.946229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6565808-a81b-4ce8-88e5-3b5f9934b1e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:56.947635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6565808-a81b-4ce8-88e5-3b5f9934b1e8 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:56.958083 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 494/1005] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 20:42:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:56.959143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fa1e8-66f3-4c50-a98b-2829932c280b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:56.985269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:57.076840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fa1e8-66f3-4c50-a98b-2829932c280b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:57.111215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bc0782-8fef-4fec-bcab-5c25ca2bd33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:57.111880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc0782-8fef-4fec-bcab-5c25ca2bd33f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:57.112167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc0782-8fef-4fec-bcab-5c25ca2bd33f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:57.112450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bc0782-8fef-4fec-bcab-5c25ca2bd33f None admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:57.113813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc0782-8fef-4fec-bcab-5c25ca2bd33f None admin] RBAC: Adding query filter params (interface=public, service_id=bcaf1e5969904bfb834f78f240945a0f, region_id=RegionOne) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:42:57.114160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc0782-8fef-4fec-bcab-5c25ca2bd33f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:57.115546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bc0782-8fef-4fec-bcab-5c25ca2bd33f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:57.120917 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 512/1006] 10.222.0.33 () {56 vars in 1251 bytes} [Sat May 28 20:42:56 2022] GET /identity/v3/endpoints?service_id=bcaf1e5969904bfb834f78f240945a0f&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:57.126298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6565808-a81b-4ce8-88e5-3b5f9934b1e8 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:57.188208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:57.234481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6565808-a81b-4ce8-88e5-3b5f9934b1e8 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:57.271382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcff849-49ff-41f0-845e-88b5b19feb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:57.271718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcff849-49ff-41f0-845e-88b5b19feb32 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:57.271965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcff849-49ff-41f0-845e-88b5b19feb32 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:57.272261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcff849-49ff-41f0-845e-88b5b19feb32 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:57.272898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcff849-49ff-41f0-845e-88b5b19feb32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:57.278061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcff849-49ff-41f0-845e-88b5b19feb32 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:57.289559 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 495/1007] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 20:42:57 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 20:42:58.389945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81686db6-e488-443c-a1ea-08cbef895908 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:58.390742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81686db6-e488-443c-a1ea-08cbef895908 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:58.390742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81686db6-e488-443c-a1ea-08cbef895908 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:58.392132 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 513/1008] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:58.398423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ead4a-0c65-479e-832b-c6677ad7792c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:58.398916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ead4a-0c65-479e-832b-c6677ad7792c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:58.399265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ead4a-0c65-479e-832b-c6677ad7792c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:58.417141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5d7ead4a-0c65-479e-832b-c6677ad7792c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:58.418361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5d7ead4a-0c65-479e-832b-c6677ad7792c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:58.490709 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 496/1009] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:58 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:58.516148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81686db6-e488-443c-a1ea-08cbef895908 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:58.609033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:58.625540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81686db6-e488-443c-a1ea-08cbef895908 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:58.660379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108fc27e-55be-4393-8be6-e01e8b84e6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:58.660972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fc27e-55be-4393-8be6-e01e8b84e6d8 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:58.661256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fc27e-55be-4393-8be6-e01e8b84e6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:58.661534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108fc27e-55be-4393-8be6-e01e8b84e6d8 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:58.662295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fc27e-55be-4393-8be6-e01e8b84e6d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:58.663703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108fc27e-55be-4393-8be6-e01e8b84e6d8 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:58.770727 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 514/1010] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 20:42:58 2022] POST /identity/v3/services => generated 242 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:42:59.107593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e827d-0909-48f5-91dd-d0145e432593 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:59.107929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e827d-0909-48f5-91dd-d0145e432593 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:59.108154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e827d-0909-48f5-91dd-d0145e432593 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:59.109664 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 497/1011] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:42:59.115208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71730183-3c11-4adf-9546-449ad43b0d1f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:59.115501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71730183-3c11-4adf-9546-449ad43b0d1f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:59.115781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71730183-3c11-4adf-9546-449ad43b0d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:59.132496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-71730183-3c11-4adf-9546-449ad43b0d1f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:59.133685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-71730183-3c11-4adf-9546-449ad43b0d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:59.207664 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 515/1012] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:59 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:42:59.233314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e827d-0909-48f5-91dd-d0145e432593 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:59.289737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:59.337275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e827d-0909-48f5-91dd-d0145e432593 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:59.370922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae18e2-f995-4360-8878-253b70783086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:59.371727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae18e2-f995-4360-8878-253b70783086 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:59.372006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae18e2-f995-4360-8878-253b70783086 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:59.372276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae18e2-f995-4360-8878-253b70783086 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:59.373477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae18e2-f995-4360-8878-253b70783086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:59.375005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae18e2-f995-4360-8878-253b70783086 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:59.376714 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 498/1013] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:42:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:42:59.383270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71730183-3c11-4adf-9546-449ad43b0d1f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:59.459052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:59.490259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71730183-3c11-4adf-9546-449ad43b0d1f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:59.525162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b3912-7cc9-4bb2-b589-6d4da0245670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:59.525831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b3912-7cc9-4bb2-b589-6d4da0245670 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:59.526139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b3912-7cc9-4bb2-b589-6d4da0245670 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:59.526418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b3912-7cc9-4bb2-b589-6d4da0245670 None admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:59.527127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b3912-7cc9-4bb2-b589-6d4da0245670 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:59.528562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b3912-7cc9-4bb2-b589-6d4da0245670 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:59.587421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cff6b-b1c4-4ca1-93fe-910f43971fd5 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:59.587916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cff6b-b1c4-4ca1-93fe-910f43971fd5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:59.587916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26cff6b-b1c4-4ca1-93fe-910f43971fd5 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:59.589249 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 499/1014] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:42:59.594174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36a84a-ab41-467e-8800-c0f0c49a9b37 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:59.594457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36a84a-ab41-467e-8800-c0f0c49a9b37 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:59.594734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36a84a-ab41-467e-8800-c0f0c49a9b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:59.611480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9d36a84a-ab41-467e-8800-c0f0c49a9b37 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:42:59.612495 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 516/1015] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 20:42:59 2022] POST /identity/v3/projects => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:42:59.612713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9d36a84a-ab41-467e-8800-c0f0c49a9b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:42:59.691245 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 500/1016] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:42:59 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:42:59.715649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b3912-7cc9-4bb2-b589-6d4da0245670 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:59.820654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b3912-7cc9-4bb2-b589-6d4da0245670 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:42:59.846820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:59.860537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6295b0-f156-46bb-badb-f896158e84ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:42:59.861336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6295b0-f156-46bb-badb-f896158e84ec None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:42:59.862087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6295b0-f156-46bb-badb-f896158e84ec None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:42:59.862087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6295b0-f156-46bb-badb-f896158e84ec None admin] PATH_INFO: `/v3/services/compute` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:42:59.863292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6295b0-f156-46bb-badb-f896158e84ec None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:42:59.864457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6295b0-f156-46bb-badb-f896158e84ec None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:42:59.876783 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-2f6295b0-f156-46bb-badb-f896158e84ec None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 28 20:42:59.876783 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:59.876783 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:42:59.876783 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:42:59.876783 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:42:59.876783 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:42:59.876783 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:42:59.876783 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:59.876783 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 20:42:59.876783 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:59.876783 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:42:59.876783 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:59.876783 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:42:59.876783 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:42:59.876783 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:42:59.876783 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:42:59.876783 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:42:59.878075 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:42:59.878075 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:42:59.878075 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:42:59.878075 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:42:59.878075 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:42:59.878075 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:42:59.878075 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:42:59.878075 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:42:59.878075 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:42:59.878075 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:42:59.878075 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:42:59.878075 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:42:59.878075 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:42:59.878075 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:42:59.878075 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:42:59.878075 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:42:59.879654 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:42:59.879654 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:42:59.879654 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:42:59.879654 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:42:59.879654 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:42:59.879654 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:42:59.879654 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:42:59.879654 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:42:59.879654 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:42:59.879654 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:42:59.879654 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:42:59.879654 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:42:59.879654 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 28 20:42:59.879654 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:42:59.885880 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 517/1017] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:42:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:42:59.889638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36a84a-ab41-467e-8800-c0f0c49a9b37 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:42:59.944083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:42:59.992940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36a84a-ab41-467e-8800-c0f0c49a9b37 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:00.027667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a8fc4-6d86-4f9e-8787-1d8285ab64e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:00.028331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a8fc4-6d86-4f9e-8787-1d8285ab64e3 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:00.028612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a8fc4-6d86-4f9e-8787-1d8285ab64e3 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:00.028886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a8fc4-6d86-4f9e-8787-1d8285ab64e3 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:00.029592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a8fc4-6d86-4f9e-8787-1d8285ab64e3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:00.029934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a8fc4-6d86-4f9e-8787-1d8285ab64e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:00.031293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a8fc4-6d86-4f9e-8787-1d8285ab64e3 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:00.041482 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 501/1018] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 20:42:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:00.042644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6295b0-f156-46bb-badb-f896158e84ec None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:00.149221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6295b0-f156-46bb-badb-f896158e84ec None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:00.183330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5db61-15cc-4520-a2e1-c57b225e1227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:00.183995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5db61-15cc-4520-a2e1-c57b225e1227 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:00.184278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5db61-15cc-4520-a2e1-c57b225e1227 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:00.184555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5db61-15cc-4520-a2e1-c57b225e1227 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:00.185263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5db61-15cc-4520-a2e1-c57b225e1227 None admin] RBAC: Adding query filter params (type=compute) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:00.185570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5db61-15cc-4520-a2e1-c57b225e1227 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:00.186966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5db61-15cc-4520-a2e1-c57b225e1227 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:00.191954 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 518/1019] 10.222.0.33 () {56 vars in 1112 bytes} [Sat May 28 20:43:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:00.197430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a8fc4-6d86-4f9e-8787-1d8285ab64e3 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:00.295538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a8fc4-6d86-4f9e-8787-1d8285ab64e3 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:00.309068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:00.328987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b6d4a-6dce-4554-b1e0-61aa3d029c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:00.329641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b6d4a-6dce-4554-b1e0-61aa3d029c19 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:00.329950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b6d4a-6dce-4554-b1e0-61aa3d029c19 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:00.330228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b6d4a-6dce-4554-b1e0-61aa3d029c19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:00.330975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b6d4a-6dce-4554-b1e0-61aa3d029c19 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:00.332954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114b6d4a-6dce-4554-b1e0-61aa3d029c19 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:00.441941 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 502/1020] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 20:43:00 2022] POST /identity/v3/endpoints => generated 334 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:43:01.090277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f9d16-6818-4877-b66f-0fb4564f4352 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:01.090766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f9d16-6818-4877-b66f-0fb4564f4352 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:01.090873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f9d16-6818-4877-b66f-0fb4564f4352 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:01.092327 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 519/1021] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:01.097341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc9a34-9e8a-4001-9283-5e2d45b8a893 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:01.097751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc9a34-9e8a-4001-9283-5e2d45b8a893 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:01.097946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc9a34-9e8a-4001-9283-5e2d45b8a893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:01.114398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5cbc9a34-9e8a-4001-9283-5e2d45b8a893 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:01.115856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5cbc9a34-9e8a-4001-9283-5e2d45b8a893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:01.191457 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 503/1022] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:43:01.215227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f9d16-6818-4877-b66f-0fb4564f4352 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:01.274377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:01.323084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f9d16-6818-4877-b66f-0fb4564f4352 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:01.356437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f47b53-47d1-43d9-962a-65b2a3bdfecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:01.357230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f47b53-47d1-43d9-962a-65b2a3bdfecf None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:01.357511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f47b53-47d1-43d9-962a-65b2a3bdfecf None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:01.357782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f47b53-47d1-43d9-962a-65b2a3bdfecf None admin] PATH_INFO: `/v3/services/image` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:01.358477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f47b53-47d1-43d9-962a-65b2a3bdfecf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:01.359866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f47b53-47d1-43d9-962a-65b2a3bdfecf None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:01.371996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-10f47b53-47d1-43d9-962a-65b2a3bdfecf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 28 20:43:01.371996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:01.371996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:43:01.371996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:43:01.371996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:43:01.371996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:43:01.371996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:43:01.371996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:43:01.371996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 20:43:01.371996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:01.371996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:43:01.371996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:01.371996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:01.371996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:01.371996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:01.371996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:01.371996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:01.372984 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:01.372984 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:01.372984 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:01.372984 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:01.372984 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:01.372984 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:01.372984 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:43:01.372984 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:43:01.372984 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:43:01.372984 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:43:01.372984 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:01.372984 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:01.372984 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:01.372984 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:01.372984 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:01.372984 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:01.374277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:01.374277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:43:01.374277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:01.374277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:01.374277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:01.374277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:01.374277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:01.374277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:01.374277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:01.374277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:01.374277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:43:01.374277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:43:01.374277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 20:43:01.374277 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:01.378726 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 520/1023] 10.222.0.33 () {56 vars in 1105 bytes} [Sat May 28 20:43:01 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:43:01.380950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc9a34-9e8a-4001-9283-5e2d45b8a893 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:01.475884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:01.482306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc9a34-9e8a-4001-9283-5e2d45b8a893 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:01.567628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673abb2f-e1f0-4b55-833a-725300332f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:01.568291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673abb2f-e1f0-4b55-833a-725300332f90 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:01.568570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673abb2f-e1f0-4b55-833a-725300332f90 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:01.568840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673abb2f-e1f0-4b55-833a-725300332f90 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:01.569542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673abb2f-e1f0-4b55-833a-725300332f90 None admin] RBAC: Adding query filter params (name=image) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:01.569866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673abb2f-e1f0-4b55-833a-725300332f90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:01.571237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673abb2f-e1f0-4b55-833a-725300332f90 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:01.582363 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 504/1024] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:43:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:01.582513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f47b53-47d1-43d9-962a-65b2a3bdfecf None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:01.686998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f47b53-47d1-43d9-962a-65b2a3bdfecf None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:01.704237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:01.722736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00104a-8133-43f0-873c-7e762c9e0efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:01.723387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00104a-8133-43f0-873c-7e762c9e0efb None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:01.723702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00104a-8133-43f0-873c-7e762c9e0efb None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:01.723975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00104a-8133-43f0-873c-7e762c9e0efb None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:01.724690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00104a-8133-43f0-873c-7e762c9e0efb None admin] RBAC: Adding query filter params (type=image) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:01.724999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00104a-8133-43f0-873c-7e762c9e0efb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:01.726400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00104a-8133-43f0-873c-7e762c9e0efb None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:01.736281 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 521/1025] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:43:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:01.738372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673abb2f-e1f0-4b55-833a-725300332f90 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:01.843389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673abb2f-e1f0-4b55-833a-725300332f90 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:01.877470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc241343-e95e-4dba-97bf-2d641f11f018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:01.878183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc241343-e95e-4dba-97bf-2d641f11f018 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:01.878468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc241343-e95e-4dba-97bf-2d641f11f018 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:01.878745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc241343-e95e-4dba-97bf-2d641f11f018 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:01.879526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc241343-e95e-4dba-97bf-2d641f11f018 None admin] RBAC: Adding query filter params (interface=public, service_id=eb6b5777327444dca8eadf8ab87e51d3, region_id=RegionOne) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:01.879838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc241343-e95e-4dba-97bf-2d641f11f018 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:01.881198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc241343-e95e-4dba-97bf-2d641f11f018 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:01.890412 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 505/1026] 10.222.0.33 () {56 vars in 1251 bytes} [Sat May 28 20:43:01 2022] GET /identity/v3/endpoints?service_id=eb6b5777327444dca8eadf8ab87e51d3&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:01.891988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00104a-8133-43f0-873c-7e762c9e0efb None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:01.958592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d973c8-8601-48bd-bba1-1b7dc3bfaa17 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:01.958903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d973c8-8601-48bd-bba1-1b7dc3bfaa17 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:01.959152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d973c8-8601-48bd-bba1-1b7dc3bfaa17 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:01.960655 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 506/1027] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:43:01.965488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e95c2-c829-4d74-9bb0-ee8239ce4714 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:01.965781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e95c2-c829-4d74-9bb0-ee8239ce4714 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:01.966108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e95c2-c829-4d74-9bb0-ee8239ce4714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:01.982844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5f7e95c2-c829-4d74-9bb0-ee8239ce4714 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:01.984037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5f7e95c2-c829-4d74-9bb0-ee8239ce4714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:01.997023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00104a-8133-43f0-873c-7e762c9e0efb None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:02.003291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:02.032746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa11d15-7b71-4c3f-b0d3-7d6f2ad89196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:02.033420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa11d15-7b71-4c3f-b0d3-7d6f2ad89196 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:02.033728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa11d15-7b71-4c3f-b0d3-7d6f2ad89196 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:02.034067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa11d15-7b71-4c3f-b0d3-7d6f2ad89196 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:02.035261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa11d15-7b71-4c3f-b0d3-7d6f2ad89196 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:02.036725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa11d15-7b71-4c3f-b0d3-7d6f2ad89196 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:02.045940 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 522/1028] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 20:43:01 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 20:43:02.056892 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 507/1029] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:43:02.085491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa11d15-7b71-4c3f-b0d3-7d6f2ad89196 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:02.136946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:02.198351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa11d15-7b71-4c3f-b0d3-7d6f2ad89196 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:02.233681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c314b45-1ffe-4746-87e6-3ed8370cfbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:02.234524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c314b45-1ffe-4746-87e6-3ed8370cfbf4 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:02.234813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c314b45-1ffe-4746-87e6-3ed8370cfbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:02.235096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c314b45-1ffe-4746-87e6-3ed8370cfbf4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:02.236298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c314b45-1ffe-4746-87e6-3ed8370cfbf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:02.238319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c314b45-1ffe-4746-87e6-3ed8370cfbf4 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:02.240129 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 523/1030] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:43:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:02.246682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e95c2-c829-4d74-9bb0-ee8239ce4714 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:02.353467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e95c2-c829-4d74-9bb0-ee8239ce4714 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:02.389318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd579072-82aa-4b85-af90-10e0bfee61e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:02.390521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd579072-82aa-4b85-af90-10e0bfee61e4 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:02.390955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd579072-82aa-4b85-af90-10e0bfee61e4 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:02.391377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd579072-82aa-4b85-af90-10e0bfee61e4 None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:02.392230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd579072-82aa-4b85-af90-10e0bfee61e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:02.393832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd579072-82aa-4b85-af90-10e0bfee61e4 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:02.496837 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 508/1031] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:43:02 2022] POST /identity/v3/users => generated 285 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:43:02.816487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50926a-917e-462a-a1ad-c8ee95f2ad10 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:02.816866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50926a-917e-462a-a1ad-c8ee95f2ad10 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:02.817065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50926a-917e-462a-a1ad-c8ee95f2ad10 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:02.818477 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 524/1032] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:02.823588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77407d4-c736-4c89-8ed7-c41c20ab447d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:02.823883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77407d4-c736-4c89-8ed7-c41c20ab447d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:02.824163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77407d4-c736-4c89-8ed7-c41c20ab447d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:02.841327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a77407d4-c736-4c89-8ed7-c41c20ab447d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:02.842537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a77407d4-c736-4c89-8ed7-c41c20ab447d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:02.845441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:02.913699 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 509/1033] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:43:02.938564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50926a-917e-462a-a1ad-c8ee95f2ad10 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:03.040404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50926a-917e-462a-a1ad-c8ee95f2ad10 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:03.053768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:03.079340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c738a-d7e5-4fed-abba-3f73ea38ba1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:03.080098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c738a-d7e5-4fed-abba-3f73ea38ba1b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:03.080382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c738a-d7e5-4fed-abba-3f73ea38ba1b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:03.080660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c738a-d7e5-4fed-abba-3f73ea38ba1b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:03.091985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c738a-d7e5-4fed-abba-3f73ea38ba1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:43:03.100088 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-818c738a-d7e5-4fed-abba-3f73ea38ba1b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 20:43:03.100088 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:03.100088 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:03.100088 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:03.100088 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:03.100088 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:03.100088 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:03.100088 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:03.100088 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:03.100088 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:03.100088 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:03.100088 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:03.100088 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:43:03.100088 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:43:03.100088 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:43:03.100088 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:43:03.101029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:43:03.101029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:43:03.101029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:43:03.101029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:43:03.101029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:03.101029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:03.101029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:03.101029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:03.101029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:03.101029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:03.101029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:03.101029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:43:03.101029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:03.101029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:03.101029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:03.101029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:03.101907 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:03.101907 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:03.101907 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:03.101907 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:03.101907 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:43:03.101907 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:43:03.101907 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:43:03.101907 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:43:03.101907 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:43:03.101907 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:43:03.101907 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 20:43:03.101907 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:03.101907 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 525/1034] 10.222.0.33 () {56 vars in 1120 bytes} [Sat May 28 20:43:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:43:03.107235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77407d4-c736-4c89-8ed7-c41c20ab447d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:03.211141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77407d4-c736-4c89-8ed7-c41c20ab447d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:03.245639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65422d-5154-4b84-a947-1b9628bee067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:03.246342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65422d-5154-4b84-a947-1b9628bee067 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:03.246629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65422d-5154-4b84-a947-1b9628bee067 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:03.246904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65422d-5154-4b84-a947-1b9628bee067 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:03.247658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65422d-5154-4b84-a947-1b9628bee067 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:03.247961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65422d-5154-4b84-a947-1b9628bee067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:03.249500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65422d-5154-4b84-a947-1b9628bee067 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:03.257906 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 510/1035] 10.222.0.33 () {56 vars in 1115 bytes} [Sat May 28 20:43:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:03.263714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c738a-d7e5-4fed-abba-3f73ea38ba1b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:03.368011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c738a-d7e5-4fed-abba-3f73ea38ba1b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:03.402759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5265d-54a5-4138-a7a7-fd0892369dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:03.403554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5265d-54a5-4138-a7a7-fd0892369dfa None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:03.403838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5265d-54a5-4138-a7a7-fd0892369dfa None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:03.404122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f5265d-54a5-4138-a7a7-fd0892369dfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:03.414830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5265d-54a5-4138-a7a7-fd0892369dfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:03.416294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f5265d-54a5-4138-a7a7-fd0892369dfa None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:03.434152 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-b5f5265d-54a5-4138-a7a7-fd0892369dfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:43:03.434152 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:03.434152 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:03.434152 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:03.434152 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:03.434152 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:03.434152 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:03.434152 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:03.434152 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:03.434152 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:03.434152 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:03.434152 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:03.434152 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:43:03.434152 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:43:03.434152 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:43:03.435022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:43:03.435022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:03.435022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:03.435022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:03.435022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:03.435022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:03.435022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:03.435022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:03.435022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:43:03.435022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:03.435022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:03.435022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:03.435022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:03.435022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:03.435022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:03.435022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:03.435022 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:03.440834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:43:03.440834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:43:03.440834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:43:03.440834 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:03.440834 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 526/1036] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:43:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:43:03.441125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65422d-5154-4b84-a947-1b9628bee067 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:03.444116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:03.540840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65422d-5154-4b84-a947-1b9628bee067 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:03.574339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fe284-c13d-4cc7-b94d-c28bca01db17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:03.574999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fe284-c13d-4cc7-b94d-c28bca01db17 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:03.575282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fe284-c13d-4cc7-b94d-c28bca01db17 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:03.575556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0fe284-c13d-4cc7-b94d-c28bca01db17 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:03.576276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fe284-c13d-4cc7-b94d-c28bca01db17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:03.576583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fe284-c13d-4cc7-b94d-c28bca01db17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:03.577984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0fe284-c13d-4cc7-b94d-c28bca01db17 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:03.593731 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 511/1037] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:03.595173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5265d-54a5-4138-a7a7-fd0892369dfa None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:03.622929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:03.700133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f5265d-54a5-4138-a7a7-fd0892369dfa None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:03.734673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae548de-5b39-4c39-ab90-278125a2ebfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:03.735467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae548de-5b39-4c39-ab90-278125a2ebfc None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:03.735751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae548de-5b39-4c39-ab90-278125a2ebfc None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:03.736028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae548de-5b39-4c39-ab90-278125a2ebfc None admin] PATH_INFO: `/v3/users/nova` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:03.743805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae548de-5b39-4c39-ab90-278125a2ebfc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:03.745248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae548de-5b39-4c39-ab90-278125a2ebfc None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:03.767754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-eae548de-5b39-4c39-ab90-278125a2ebfc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 20:43:03.767754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:03.767754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:03.767754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:03.767754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:03.767754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:03.767754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:03.767754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:03.767754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:03.767754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:03.767754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:03.767754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:03.767754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:43:03.767754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:43:03.767754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:43:03.767754 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:03.769276 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:03.770265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:03.770265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:03.770265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:43:03.770265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:43:03.770265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:43:03.770265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:43:03.770265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:43:03.770265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:43:03.770265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 20:43:03.770265 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:03.777507 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 527/1038] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 20:43:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:43:03.778844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fe284-c13d-4cc7-b94d-c28bca01db17 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:03.844291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:03.884456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0fe284-c13d-4cc7-b94d-c28bca01db17 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:03.932940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591294db-c233-486c-b24d-34c4296c3e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:03.934053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591294db-c233-486c-b24d-34c4296c3e0c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:03.934550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591294db-c233-486c-b24d-34c4296c3e0c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:03.935034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591294db-c233-486c-b24d-34c4296c3e0c None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:03.936077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591294db-c233-486c-b24d-34c4296c3e0c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:03.936631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591294db-c233-486c-b24d-34c4296c3e0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:03.938431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591294db-c233-486c-b24d-34c4296c3e0c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:03.962970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae548de-5b39-4c39-ab90-278125a2ebfc None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:03.973169 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 512/1039] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 20:43:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:04.051267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:04.075492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae548de-5b39-4c39-ab90-278125a2ebfc None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:04.111515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393f9ee-b0de-48c5-bd5a-82bb94c03e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:04.112174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393f9ee-b0de-48c5-bd5a-82bb94c03e0c None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:04.112483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393f9ee-b0de-48c5-bd5a-82bb94c03e0c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:04.112766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393f9ee-b0de-48c5-bd5a-82bb94c03e0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:04.195861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393f9ee-b0de-48c5-bd5a-82bb94c03e0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:04.197818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393f9ee-b0de-48c5-bd5a-82bb94c03e0c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:04.281441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:04.298575 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 528/1040] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:43:03 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 20:43:04.304144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591294db-c233-486c-b24d-34c4296c3e0c None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:04.375897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19743acb-5cb0-4bf2-a56a-b3c2f971467b None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:04.376294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19743acb-5cb0-4bf2-a56a-b3c2f971467b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:04.376639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19743acb-5cb0-4bf2-a56a-b3c2f971467b None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:04.378839 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 529/1041] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:43:04.383947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6b17f-4eb0-4bb6-8392-73dfd91d3f01 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:04.384234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6b17f-4eb0-4bb6-8392-73dfd91d3f01 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:04.384516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6b17f-4eb0-4bb6-8392-73dfd91d3f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:04.402007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-52b6b17f-4eb0-4bb6-8392-73dfd91d3f01 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:04.403209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-52b6b17f-4eb0-4bb6-8392-73dfd91d3f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:04.417652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591294db-c233-486c-b24d-34c4296c3e0c None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:04.443858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:04.460806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45f924-61bb-493a-a7cf-b0e7e5a52bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:04.462218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45f924-61bb-493a-a7cf-b0e7e5a52bd0 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:04.462504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45f924-61bb-493a-a7cf-b0e7e5a52bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:04.462783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f45f924-61bb-493a-a7cf-b0e7e5a52bd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:04.474247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45f924-61bb-493a-a7cf-b0e7e5a52bd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:04.475709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f45f924-61bb-493a-a7cf-b0e7e5a52bd0 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:04.481646 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 530/1042] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:43:04.498765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-4f45f924-61bb-493a-a7cf-b0e7e5a52bd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:43:04.498765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:04.498765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:04.498765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:04.498765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:04.498765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:04.498765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:04.498765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:04.498765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:04.498765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:04.498765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:04.498765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:04.498765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:43:04.498765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:43:04.498765 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:43:04.499691 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:43:04.499691 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:04.499691 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:04.499691 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:04.499691 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:04.499691 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:04.499691 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:04.499691 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:04.499691 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:43:04.499691 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:04.499691 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:04.499691 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:04.499691 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:04.499691 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:04.499691 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:04.499691 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:04.499691 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:04.500613 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:43:04.500613 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:43:04.500613 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:43:04.500613 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:43:04.500613 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 513/1043] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:43:04.501055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6b17f-4eb0-4bb6-8392-73dfd91d3f01 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:04.507204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45f924-61bb-493a-a7cf-b0e7e5a52bd0 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:04.609021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6b17f-4eb0-4bb6-8392-73dfd91d3f01 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:04.613622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f45f924-61bb-493a-a7cf-b0e7e5a52bd0 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:04.624679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:04.649203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8dccda-6c1c-41ca-9f70-253a9f39f7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:04.649400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf770a-9f87-4b3a-9276-fe456b146c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:04.649943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf770a-9f87-4b3a-9276-fe456b146c82 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:04.650047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dccda-6c1c-41ca-9f70-253a9f39f7b6 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:04.650223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf770a-9f87-4b3a-9276-fe456b146c82 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:04.650345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dccda-6c1c-41ca-9f70-253a9f39f7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:04.650504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bf770a-9f87-4b3a-9276-fe456b146c82 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:04.650595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8dccda-6c1c-41ca-9f70-253a9f39f7b6 None admin] PATH_INFO: `/v3/services/image` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:04.651214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf770a-9f87-4b3a-9276-fe456b146c82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:04.651312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8dccda-6c1c-41ca-9f70-253a9f39f7b6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:04.651523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf770a-9f87-4b3a-9276-fe456b146c82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:04.652884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8dccda-6c1c-41ca-9f70-253a9f39f7b6 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:04.652985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bf770a-9f87-4b3a-9276-fe456b146c82 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:04.665616 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 514/1044] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:04.668298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-3b8dccda-6c1c-41ca-9f70-253a9f39f7b6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 28 20:43:04.668298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:04.668298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:43:04.668298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:43:04.668298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:43:04.668298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:43:04.668298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:43:04.668298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:43:04.668298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 20:43:04.668298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:04.668298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:43:04.668298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:04.668298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:04.668298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:04.668298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:04.668298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:04.668298 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:04.669315 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:04.669315 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:04.669315 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:04.669315 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:04.669315 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:04.669315 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:04.669315 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:43:04.669315 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:43:04.669315 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:43:04.669315 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:43:04.669315 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:04.669315 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:04.669315 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:04.669315 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:04.669315 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:04.669315 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:04.677192 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:04.677192 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:43:04.677192 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:04.677192 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:04.677192 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:04.677192 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:04.677192 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:04.677192 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:04.677192 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:04.677192 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:04.677192 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:43:04.677192 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:43:04.677192 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 28 20:43:04.677192 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:04.677192 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 531/1045] 10.222.0.33 () {56 vars in 1105 bytes} [Sat May 28 20:43:04 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:43:04.678082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf770a-9f87-4b3a-9276-fe456b146c82 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:04.678175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8dccda-6c1c-41ca-9f70-253a9f39f7b6 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:04.709875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:04.785738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bf770a-9f87-4b3a-9276-fe456b146c82 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:04.788652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8dccda-6c1c-41ca-9f70-253a9f39f7b6 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:04.820967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1974b9a-1fe9-428d-9ef4-502f0e1cddc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:04.821717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1974b9a-1fe9-428d-9ef4-502f0e1cddc5 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:04.822031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1974b9a-1fe9-428d-9ef4-502f0e1cddc5 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:04.822303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1974b9a-1fe9-428d-9ef4-502f0e1cddc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:04.824491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7058445-8ff4-4580-9336-f5fa569e87b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:04.825178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7058445-8ff4-4580-9336-f5fa569e87b6 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:04.825446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7058445-8ff4-4580-9336-f5fa569e87b6 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:04.825731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7058445-8ff4-4580-9336-f5fa569e87b6 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:04.826472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7058445-8ff4-4580-9336-f5fa569e87b6 None admin] RBAC: Adding query filter params (name=image) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:04.826784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7058445-8ff4-4580-9336-f5fa569e87b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:04.828170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7058445-8ff4-4580-9336-f5fa569e87b6 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:04.830398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1974b9a-1fe9-428d-9ef4-502f0e1cddc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:04.832711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1974b9a-1fe9-428d-9ef4-502f0e1cddc5 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:04.833950 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 532/1046] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:43:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:04.839453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7058445-8ff4-4580-9336-f5fa569e87b6 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:04.852030 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-f1974b9a-1fe9-428d-9ef4-502f0e1cddc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:43:04.852030 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:04.852030 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:04.852030 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:04.852030 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:04.852030 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:04.852030 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:04.852030 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:04.852030 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:04.852030 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:04.852030 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:04.852030 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:04.852030 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:43:04.852030 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:43:04.852030 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:43:04.853100 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:43:04.853100 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:04.853100 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:04.853100 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:04.853100 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:04.853100 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:04.853100 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:04.853100 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:04.853100 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:43:04.853100 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:04.853100 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:04.853100 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:04.853100 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:04.853100 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:04.853100 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:04.853100 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:04.853100 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:04.854106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:43:04.854106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:43:04.854106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:43:04.854106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:43:04.854106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:43:04.854106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:43:04.854106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:43:04.854106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:43:04.854106 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 515/1047] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:43:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:43:04.859240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1974b9a-1fe9-428d-9ef4-502f0e1cddc5 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:04.886186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:04.948792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7058445-8ff4-4580-9336-f5fa569e87b6 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:04.961261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:04.970789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1974b9a-1fe9-428d-9ef4-502f0e1cddc5 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:04.984240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f8c4f-73ac-4e7a-965d-cef97056c971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:04.984926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f8c4f-73ac-4e7a-965d-cef97056c971 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:04.985256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f8c4f-73ac-4e7a-965d-cef97056c971 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:04.985541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f8c4f-73ac-4e7a-965d-cef97056c971 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:04.986316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f8c4f-73ac-4e7a-965d-cef97056c971 None admin] RBAC: Adding query filter params (type=image) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:04.986630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f8c4f-73ac-4e7a-965d-cef97056c971 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:04.988034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f8c4f-73ac-4e7a-965d-cef97056c971 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:04.992916 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 533/1048] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:43:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:04.994493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b051768-0647-439f-9f86-1a6e9adbbec0 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:04.994790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b051768-0647-439f-9f86-1a6e9adbbec0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:04.995077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b051768-0647-439f-9f86-1a6e9adbbec0 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:04.996450 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 534/1049] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:04.998864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b051768-0647-439f-9f86-1a6e9adbbec0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:05.004160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c772901-b95c-41a8-8176-791551b418e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:05.004816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c772901-b95c-41a8-8176-791551b418e4 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:05.005099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c772901-b95c-41a8-8176-791551b418e4 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:05.005375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c772901-b95c-41a8-8176-791551b418e4 None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:05.006220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c772901-b95c-41a8-8176-791551b418e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:05.006531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c772901-b95c-41a8-8176-791551b418e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:05.007978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c772901-b95c-41a8-8176-791551b418e4 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:05.020169 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 516/1050] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:43:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:05.021417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fad64-4fd8-4462-a99a-38be3af7e708 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:05.021712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fad64-4fd8-4462-a99a-38be3af7e708 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:05.022010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5fad64-4fd8-4462-a99a-38be3af7e708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:05.037867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6c5fad64-4fd8-4462-a99a-38be3af7e708 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:05.038970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6c5fad64-4fd8-4462-a99a-38be3af7e708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:05.102838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b051768-0647-439f-9f86-1a6e9adbbec0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:05.109999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:05.112249 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 517/1051] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:43:05.113605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fad64-4fd8-4462-a99a-38be3af7e708 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:05.149189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e42485-e4cf-457b-aa74-25c973fc23c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:05.149928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e42485-e4cf-457b-aa74-25c973fc23c0 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:05.150167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e42485-e4cf-457b-aa74-25c973fc23c0 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:05.150446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e42485-e4cf-457b-aa74-25c973fc23c0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:05.151188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e42485-e4cf-457b-aa74-25c973fc23c0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:05.152553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e42485-e4cf-457b-aa74-25c973fc23c0 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:05.195309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:05.212546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5fad64-4fd8-4462-a99a-38be3af7e708 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:05.221381 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 535/1052] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 20:43:04 2022] POST /identity/v3/endpoints => generated 327 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 20:43:05.225918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e42485-e4cf-457b-aa74-25c973fc23c0 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:05.263540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9af13-6c2b-48b4-bfae-f0bff6a19253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:05.264678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9af13-6c2b-48b4-bfae-f0bff6a19253 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:05.265558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9af13-6c2b-48b4-bfae-f0bff6a19253 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:05.266118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9af13-6c2b-48b4-bfae-f0bff6a19253 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:05.267176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9af13-6c2b-48b4-bfae-f0bff6a19253 None admin] RBAC: Adding query filter params (role.id=50445f06754843f49827b854ec728850, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=cfdebb5e6b8d4e2d86f0a03a30c21671) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:05.268329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9af13-6c2b-48b4-bfae-f0bff6a19253 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:05.271016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9af13-6c2b-48b4-bfae-f0bff6a19253 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:05.278460 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 518/1053] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:43:05 2022] GET /identity/v3/role_assignments?user.id=cfdebb5e6b8d4e2d86f0a03a30c21671&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=50445f06754843f49827b854ec728850 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:05.351760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e42485-e4cf-457b-aa74-25c973fc23c0 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:05.391623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878c9ce-521d-4028-be0b-a5b138125aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:05.392606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878c9ce-521d-4028-be0b-a5b138125aa7 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:05.392892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878c9ce-521d-4028-be0b-a5b138125aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:05.393167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2878c9ce-521d-4028-be0b-a5b138125aa7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:05.405937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2878c9ce-521d-4028-be0b-a5b138125aa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:43:05.414440 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-2878c9ce-521d-4028-be0b-a5b138125aa7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:43:05.414440 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:05.414440 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:05.414440 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:05.414440 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:05.414440 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:05.414440 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:05.414440 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:05.414440 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:05.414440 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:05.414440 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:05.414440 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:05.414440 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:43:05.414440 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:43:05.414440 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:43:05.414440 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:43:05.415549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:43:05.415549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:43:05.415549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:43:05.415549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:43:05.415549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:05.415549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:05.415549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:05.415549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:05.415549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:05.415549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:05.415549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:05.415549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:43:05.415549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:05.415549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:05.415549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:05.415549 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:05.421953 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:05.421953 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:05.421953 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:05.421953 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:05.421953 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:43:05.421953 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:43:05.421953 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:43:05.421953 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:43:05.421953 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:43:05.421953 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:43:05.421953 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:43:05.421953 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:05.421953 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 536/1054] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:43:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:43:05.422764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9af13-6c2b-48b4-bfae-f0bff6a19253 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:05.533063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9af13-6c2b-48b4-bfae-f0bff6a19253 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:05.566707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7256c-4469-4653-9f8d-00f34a707a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:05.567521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7256c-4469-4653-9f8d-00f34a707a6d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:05.567917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7256c-4469-4653-9f8d-00f34a707a6d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:05.568293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7256c-4469-4653-9f8d-00f34a707a6d None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:05.569164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7256c-4469-4653-9f8d-00f34a707a6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:05.569572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7256c-4469-4653-9f8d-00f34a707a6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:05.571081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7256c-4469-4653-9f8d-00f34a707a6d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:05.584728 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 519/1055] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:43:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:05.586377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878c9ce-521d-4028-be0b-a5b138125aa7 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:05.612220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:05.691277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2878c9ce-521d-4028-be0b-a5b138125aa7 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:05.725066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b8316-24f1-48e2-9cc7-f3c8b796d620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:05.725995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b8316-24f1-48e2-9cc7-f3c8b796d620 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:05.726285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b8316-24f1-48e2-9cc7-f3c8b796d620 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:05.726564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b8316-24f1-48e2-9cc7-f3c8b796d620 None admin] PATH_INFO: `/v3/users/3c18f3053359450195835269391cdd3a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:05.740312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b8316-24f1-48e2-9cc7-f3c8b796d620 None admin] RBAC: Authorizing `identity:get_user(user_id=3c18f3053359450195835269391cdd3a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:05.741770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0b8316-24f1-48e2-9cc7-f3c8b796d620 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:05.748175 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 537/1056] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:43:05 2022] GET /identity/v3/users/3c18f3053359450195835269391cdd3a => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:05.749580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7256c-4469-4653-9f8d-00f34a707a6d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:05.754490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:05.855249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7256c-4469-4653-9f8d-00f34a707a6d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:05.890375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb928d9-9f68-41c3-86ed-04f9e7a22f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:05.890969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb928d9-9f68-41c3-86ed-04f9e7a22f21 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:05.891257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb928d9-9f68-41c3-86ed-04f9e7a22f21 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:05.891537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb928d9-9f68-41c3-86ed-04f9e7a22f21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:05.960134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb928d9-9f68-41c3-86ed-04f9e7a22f21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:05.961612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb928d9-9f68-41c3-86ed-04f9e7a22f21 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:05.964323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:06.063889 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 520/1057] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:43:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 20:43:06.072200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b8316-24f1-48e2-9cc7-f3c8b796d620 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:06.156241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:06.177060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0b8316-24f1-48e2-9cc7-f3c8b796d620 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:06.212044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20b3e6-f142-4e0f-b410-ed8d237a0c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:06.212898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b3e6-f142-4e0f-b410-ed8d237a0c10 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:06.213180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b3e6-f142-4e0f-b410-ed8d237a0c10 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:06.213457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b3e6-f142-4e0f-b410-ed8d237a0c10 None admin] PATH_INFO: `/v3/projects/7dad805375304257ab32589b32f31511` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:06.214713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20b3e6-f142-4e0f-b410-ed8d237a0c10 None admin] RBAC: Authorizing `identity:get_project(project_id=7dad805375304257ab32589b32f31511)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:06.216144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20b3e6-f142-4e0f-b410-ed8d237a0c10 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:06.218066 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 538/1058] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:43:06 2022] GET /identity/v3/projects/7dad805375304257ab32589b32f31511 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:06.224376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb928d9-9f68-41c3-86ed-04f9e7a22f21 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:06.326735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb928d9-9f68-41c3-86ed-04f9e7a22f21 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:06.329086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:06.361496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd3927-c2f8-4c44-9c56-b33230f9e213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:06.362180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd3927-c2f8-4c44-9c56-b33230f9e213 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:06.363079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd3927-c2f8-4c44-9c56-b33230f9e213 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:06.363360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd3927-c2f8-4c44-9c56-b33230f9e213 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:06.364178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd3927-c2f8-4c44-9c56-b33230f9e213 None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=7dad805375304257ab32589b32f31511, user.id=3c18f3053359450195835269391cdd3a) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:06.364490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd3927-c2f8-4c44-9c56-b33230f9e213 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:06.365951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd3927-c2f8-4c44-9c56-b33230f9e213 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:06.371611 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 521/1059] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:43:06 2022] GET /identity/v3/role_assignments?user.id=3c18f3053359450195835269391cdd3a&scope.project.id=7dad805375304257ab32589b32f31511&role.id=7805b93b32f1472f8365936d4d57d49a => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:07.565457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405e284-a44a-4813-bfe3-87a11ab268ee None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:07.566573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405e284-a44a-4813-bfe3-87a11ab268ee None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:07.566680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405e284-a44a-4813-bfe3-87a11ab268ee None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:07.568257 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 539/1060] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:07.573587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4d08e-a819-4089-bfde-20a9935c0451 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:07.574043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4d08e-a819-4089-bfde-20a9935c0451 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:07.574223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4d08e-a819-4089-bfde-20a9935c0451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:07.581600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8c798-cd0a-49bd-b83c-bcbac3c5e227 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:07.581922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8c798-cd0a-49bd-b83c-bcbac3c5e227 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:07.582182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8c798-cd0a-49bd-b83c-bcbac3c5e227 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:07.583771 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 540/1061] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:43:07.588727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbe12d-d4e1-4e2f-b825-a0ab65d0db11 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:07.589014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbe12d-d4e1-4e2f-b825-a0ab65d0db11 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:07.589293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbe12d-d4e1-4e2f-b825-a0ab65d0db11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:07.591382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-29c4d08e-a819-4089-bfde-20a9935c0451 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:07.592590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-29c4d08e-a819-4089-bfde-20a9935c0451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:07.606048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6bbbe12d-d4e1-4e2f-b825-a0ab65d0db11 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:07.607230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6bbbe12d-d4e1-4e2f-b825-a0ab65d0db11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:07.666084 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 522/1062] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:43:07.676719 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 541/1063] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:43:07.689928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4d08e-a819-4089-bfde-20a9935c0451 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:07.703830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbe12d-d4e1-4e2f-b825-a0ab65d0db11 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:07.790671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:07.792981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:07.796791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4d08e-a819-4089-bfde-20a9935c0451 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:07.814568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbe12d-d4e1-4e2f-b825-a0ab65d0db11 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:07.832091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87d02b-89fb-41cf-9747-970cf5b05eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:07.832677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87d02b-89fb-41cf-9747-970cf5b05eae None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:07.832961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87d02b-89fb-41cf-9747-970cf5b05eae None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:07.833236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87d02b-89fb-41cf-9747-970cf5b05eae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:07.849447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b89db-79c7-459d-a59c-1d91e2bf872a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:07.850771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b89db-79c7-459d-a59c-1d91e2bf872a None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:07.851118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b89db-79c7-459d-a59c-1d91e2bf872a None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:07.851399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b89db-79c7-459d-a59c-1d91e2bf872a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:07.861413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89b89db-79c7-459d-a59c-1d91e2bf872a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:43:07.870095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-b89b89db-79c7-459d-a59c-1d91e2bf872a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 20:43:07.870095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:07.870095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:07.870095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:07.870095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:07.870095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:07.870095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:07.870095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:07.870095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:07.870095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:07.870095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:07.870095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:07.870095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:43:07.870095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:43:07.870095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:43:07.870095 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:43:07.871024 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:43:07.871024 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:43:07.871024 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:43:07.871024 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:43:07.871024 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:07.871024 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:07.871024 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:07.871024 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:07.871024 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:07.871024 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:07.871024 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:07.871024 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:43:07.871024 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:07.871024 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:07.871024 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:07.871024 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:07.871901 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:07.871901 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:07.871901 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:07.871901 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:07.871901 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:43:07.871901 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:43:07.871901 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:43:07.871901 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:43:07.871901 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:43:07.871901 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:43:07.871901 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 20:43:07.871901 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:07.871901 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 542/1064] 10.222.0.33 () {56 vars in 1120 bytes} [Sat May 28 20:43:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:43:07.877136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b89db-79c7-459d-a59c-1d91e2bf872a None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:07.913356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87d02b-89fb-41cf-9747-970cf5b05eae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:07.914856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87d02b-89fb-41cf-9747-970cf5b05eae None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:07.979572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b89db-79c7-459d-a59c-1d91e2bf872a None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:08.015876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b16db-ecf7-4931-b799-bc7dd9b28c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:08.016548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b16db-ecf7-4931-b799-bc7dd9b28c0b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:08.016838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b16db-ecf7-4931-b799-bc7dd9b28c0b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:08.017122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b16db-ecf7-4931-b799-bc7dd9b28c0b None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:08.017902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b16db-ecf7-4931-b799-bc7dd9b28c0b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:08.018221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b16db-ecf7-4931-b799-bc7dd9b28c0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:08.019764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b16db-ecf7-4931-b799-bc7dd9b28c0b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:08.021393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:08.025780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:08.026601 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 523/1065] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:43:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 20:43:08.033695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87d02b-89fb-41cf-9747-970cf5b05eae None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:08.033888 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 543/1066] 10.222.0.33 () {56 vars in 1115 bytes} [Sat May 28 20:43:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:08.041374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b16db-ecf7-4931-b799-bc7dd9b28c0b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:08.146443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87d02b-89fb-41cf-9747-970cf5b05eae None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:08.152153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b16db-ecf7-4931-b799-bc7dd9b28c0b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:08.184301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a5bff-7671-412f-98a6-0254e9509290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:08.185207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a5bff-7671-412f-98a6-0254e9509290 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:08.185489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a5bff-7671-412f-98a6-0254e9509290 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:08.185764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a5bff-7671-412f-98a6-0254e9509290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:08.188501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65238c2-86fb-4124-b6aa-d7fb64bb324b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:08.189302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65238c2-86fb-4124-b6aa-d7fb64bb324b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:08.189588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65238c2-86fb-4124-b6aa-d7fb64bb324b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:08.189885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65238c2-86fb-4124-b6aa-d7fb64bb324b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:08.197487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a5bff-7671-412f-98a6-0254e9509290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:08.199303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a5bff-7671-412f-98a6-0254e9509290 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:08.199994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65238c2-86fb-4124-b6aa-d7fb64bb324b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:08.201449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65238c2-86fb-4124-b6aa-d7fb64bb324b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:08.218652 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-380a5bff-7671-412f-98a6-0254e9509290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:43:08.218652 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:08.218652 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:08.218652 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:08.218652 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:08.218652 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:08.218652 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:08.218652 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:08.218652 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:08.218652 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:08.218652 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:08.218652 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:08.218652 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:43:08.218652 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:43:08.218652 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:43:08.219666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:43:08.219666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:08.219666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:08.219666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:08.219666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:08.219666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:08.219666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:08.219666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:08.219666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:43:08.219666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:08.219666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:08.219666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:08.219666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:08.219666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:08.219666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:08.219666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:08.219666 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:08.220734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:43:08.220734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:43:08.220734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:43:08.220734 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:43:08.220980 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-c65238c2-86fb-4124-b6aa-d7fb64bb324b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:43:08.220980 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:08.220980 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:08.220980 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:08.220980 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:08.220980 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:08.220980 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:08.220980 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:08.220980 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:08.220980 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:08.220980 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:08.220980 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:08.220980 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:43:08.220980 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:43:08.220980 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:43:08.221865 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:43:08.221865 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:08.221865 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:08.221865 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:08.221865 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:08.221865 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:08.221865 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:08.221865 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:08.221865 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:43:08.221865 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:08.221865 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:08.221865 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:08.221865 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:08.221865 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:08.221865 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:08.221865 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:08.221865 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:08.223310 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:43:08.223310 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:43:08.223310 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:43:08.223310 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:08.223310 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 544/1067] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:43:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:43:08.224881 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 524/1068] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:43:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:43:08.230401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65238c2-86fb-4124-b6aa-d7fb64bb324b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:08.230547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a5bff-7671-412f-98a6-0254e9509290 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:08.283951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:08.289882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:08.336100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a5bff-7671-412f-98a6-0254e9509290 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:08.336328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65238c2-86fb-4124-b6aa-d7fb64bb324b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:08.372008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04abd5-74ae-4676-948d-31137a03af07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:08.372669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04abd5-74ae-4676-948d-31137a03af07 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:08.372957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04abd5-74ae-4676-948d-31137a03af07 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:08.373238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04abd5-74ae-4676-948d-31137a03af07 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:08.374016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af04abd5-74ae-4676-948d-31137a03af07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:08.374332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af04abd5-74ae-4676-948d-31137a03af07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:08.374734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdf0f2-d589-4000-b62a-a539e719a253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:08.375721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af04abd5-74ae-4676-948d-31137a03af07 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:08.376051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdf0f2-d589-4000-b62a-a539e719a253 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:08.376535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdf0f2-d589-4000-b62a-a539e719a253 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:08.377034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdf0f2-d589-4000-b62a-a539e719a253 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:08.378016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdf0f2-d589-4000-b62a-a539e719a253 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:08.378545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdf0f2-d589-4000-b62a-a539e719a253 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:08.380102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdf0f2-d589-4000-b62a-a539e719a253 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:08.387417 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 545/1069] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:08.394188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04abd5-74ae-4676-948d-31137a03af07 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:08.398371 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 525/1070] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:08.513296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04abd5-74ae-4676-948d-31137a03af07 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:08.528291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:08.551168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189e2d3-9972-4b00-b2fc-d8fb8a210b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:08.552433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189e2d3-9972-4b00-b2fc-d8fb8a210b2e None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:08.552864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189e2d3-9972-4b00-b2fc-d8fb8a210b2e None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:08.553306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189e2d3-9972-4b00-b2fc-d8fb8a210b2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:08.563233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189e2d3-9972-4b00-b2fc-d8fb8a210b2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:08.566342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189e2d3-9972-4b00-b2fc-d8fb8a210b2e None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:08.583389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-2189e2d3-9972-4b00-b2fc-d8fb8a210b2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 20:43:08.583389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:08.583389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:08.583389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:08.583389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:08.583389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:08.583389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:08.583389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:08.583389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:08.583389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:08.583389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:08.583389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:08.583389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:43:08.583389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:43:08.583389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:43:08.583389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:08.584524 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:08.585671 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:08.585671 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:08.585671 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:43:08.585671 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:43:08.585671 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:43:08.585671 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:43:08.585671 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:43:08.585671 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:43:08.585671 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 20:43:08.585671 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:08.587477 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 546/1071] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 20:43:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:43:08.593617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdf0f2-d589-4000-b62a-a539e719a253 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:08.652726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930836f2-8d56-424b-891a-a2062fdd6792 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:08.653008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930836f2-8d56-424b-891a-a2062fdd6792 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:08.653222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930836f2-8d56-424b-891a-a2062fdd6792 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:08.658522 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 547/1072] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:08.659825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaee0f-c382-49c2-9252-c59cfb50fb9b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:08.660114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaee0f-c382-49c2-9252-c59cfb50fb9b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:08.660398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaee0f-c382-49c2-9252-c59cfb50fb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:08.677798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7bdaee0f-c382-49c2-9252-c59cfb50fb9b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:08.679369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7bdaee0f-c382-49c2-9252-c59cfb50fb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:08.709997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdf0f2-d589-4000-b62a-a539e719a253 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:08.721227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:08.747108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05fd61-6d63-42b8-a91b-de44c50b35ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:08.747775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05fd61-6d63-42b8-a91b-de44c50b35ca None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:08.748055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05fd61-6d63-42b8-a91b-de44c50b35ca None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:08.748326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05fd61-6d63-42b8-a91b-de44c50b35ca None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:08.749154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05fd61-6d63-42b8-a91b-de44c50b35ca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:08.749460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05fd61-6d63-42b8-a91b-de44c50b35ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:08.750984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05fd61-6d63-42b8-a91b-de44c50b35ca None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:08.755643 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 548/1073] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:43:08.779720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdaee0f-c382-49c2-9252-c59cfb50fb9b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:08.863291 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 526/1074] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 20:43:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:08.868780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05fd61-6d63-42b8-a91b-de44c50b35ca None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:08.953938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdaee0f-c382-49c2-9252-c59cfb50fb9b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:08.955391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:08.968571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05fd61-6d63-42b8-a91b-de44c50b35ca None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:08.990543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729eec19-ffb6-4710-910c-706e366a6170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:08.991324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729eec19-ffb6-4710-910c-706e366a6170 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:08.991610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729eec19-ffb6-4710-910c-706e366a6170 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:08.991876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729eec19-ffb6-4710-910c-706e366a6170 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:09.001955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729eec19-ffb6-4710-910c-706e366a6170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:43:09.003922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff4ae2-afc2-4fa2-b990-a142c0275563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:09.004707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff4ae2-afc2-4fa2-b990-a142c0275563 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:09.004991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff4ae2-afc2-4fa2-b990-a142c0275563 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:09.005293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff4ae2-afc2-4fa2-b990-a142c0275563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:09.010164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-729eec19-ffb6-4710-910c-706e366a6170 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:43:09.010164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:09.010164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:09.010164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:09.010164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:09.010164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:09.010164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:09.010164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:09.010164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:09.010164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:09.010164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:09.010164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:09.010164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:43:09.010164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:43:09.010164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:43:09.010164 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:43:09.011067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:43:09.011067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:43:09.011067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:43:09.011067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:43:09.011067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:09.011067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:09.011067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:09.011067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:09.011067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:09.011067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:09.011067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:09.011067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:43:09.011067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:09.011067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:09.011067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:09.011067 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:09.011927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:09.011927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:09.011927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:09.011927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:09.011927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:43:09.011927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:43:09.011927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:43:09.011927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:43:09.011927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:43:09.011927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:43:09.011927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:43:09.011927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:09.011927 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 549/1075] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:43:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:43:09.015805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff4ae2-afc2-4fa2-b990-a142c0275563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:09.017257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adff4ae2-afc2-4fa2-b990-a142c0275563 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:09.017608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729eec19-ffb6-4710-910c-706e366a6170 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:09.035048 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-adff4ae2-afc2-4fa2-b990-a142c0275563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:43:09.035048 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:09.035048 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:09.035048 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:09.035048 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:09.035048 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:09.035048 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:09.035048 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:09.035048 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:09.035048 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:09.035048 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:09.035048 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:09.035048 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:43:09.035048 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:43:09.035048 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:43:09.036145 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:43:09.036145 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:09.036145 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:09.036145 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:09.036145 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:09.036145 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:09.036145 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:09.036145 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:09.036145 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:43:09.036145 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:09.036145 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:09.036145 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:09.036145 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:09.036145 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:09.036145 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:09.036145 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:09.036145 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:09.037058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:43:09.037058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:43:09.037058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:43:09.037058 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:43:09.037058 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 527/1076] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:43:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:43:09.041740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff4ae2-afc2-4fa2-b990-a142c0275563 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:09.064935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:09.121165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729eec19-ffb6-4710-910c-706e366a6170 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:09.147553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff4ae2-afc2-4fa2-b990-a142c0275563 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:09.155362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa466238-72c8-4f5e-8aa2-91a6749ef4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:09.156035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa466238-72c8-4f5e-8aa2-91a6749ef4a8 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:09.156320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa466238-72c8-4f5e-8aa2-91a6749ef4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:09.156598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa466238-72c8-4f5e-8aa2-91a6749ef4a8 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:09.157358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa466238-72c8-4f5e-8aa2-91a6749ef4a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:09.157674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa466238-72c8-4f5e-8aa2-91a6749ef4a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:09.159097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa466238-72c8-4f5e-8aa2-91a6749ef4a8 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:09.167138 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 550/1077] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:43:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:09.174291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa466238-72c8-4f5e-8aa2-91a6749ef4a8 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:09.182517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b45015-9da4-40bc-975c-a01a9e1caa0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:09.183185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b45015-9da4-40bc-975c-a01a9e1caa0a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:09.183475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b45015-9da4-40bc-975c-a01a9e1caa0a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:09.183750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b45015-9da4-40bc-975c-a01a9e1caa0a None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:09.184473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b45015-9da4-40bc-975c-a01a9e1caa0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:09.184780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b45015-9da4-40bc-975c-a01a9e1caa0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:09.186192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b45015-9da4-40bc-975c-a01a9e1caa0a None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:09.197852 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 528/1078] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:09.199735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:09.203361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b45015-9da4-40bc-975c-a01a9e1caa0a None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:09.275768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa466238-72c8-4f5e-8aa2-91a6749ef4a8 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:09.282993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:09.302627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b45015-9da4-40bc-975c-a01a9e1caa0a None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:09.309266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8195b15-15f1-4d35-8e1e-ee0e49857ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:09.310166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8195b15-15f1-4d35-8e1e-ee0e49857ded None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:09.310450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8195b15-15f1-4d35-8e1e-ee0e49857ded None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:09.310727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8195b15-15f1-4d35-8e1e-ee0e49857ded None admin] PATH_INFO: `/v3/users/3c18f3053359450195835269391cdd3a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:09.312005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8195b15-15f1-4d35-8e1e-ee0e49857ded None admin] RBAC: Authorizing `identity:get_user(user_id=3c18f3053359450195835269391cdd3a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:09.313466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8195b15-15f1-4d35-8e1e-ee0e49857ded None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:09.315146 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 551/1079] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:43:09 2022] GET /identity/v3/users/3c18f3053359450195835269391cdd3a => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:09.320746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8195b15-15f1-4d35-8e1e-ee0e49857ded None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:09.336521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda4f37-9f55-4bc9-8a42-08194207d1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:09.337306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda4f37-9f55-4bc9-8a42-08194207d1c7 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:09.337590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda4f37-9f55-4bc9-8a42-08194207d1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:09.337900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda4f37-9f55-4bc9-8a42-08194207d1c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:09.344731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda4f37-9f55-4bc9-8a42-08194207d1c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:09.346209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffda4f37-9f55-4bc9-8a42-08194207d1c7 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:09.360989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-ffda4f37-9f55-4bc9-8a42-08194207d1c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:43:09.360989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:09.360989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:09.360989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:09.360989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:09.360989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:09.360989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:09.360989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:09.360989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:09.360989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:09.360989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:09.360989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:09.360989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:43:09.360989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:43:09.360989 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:43:09.361947 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:43:09.361947 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:09.361947 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:09.361947 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:09.361947 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:09.361947 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:09.361947 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:09.361947 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:09.361947 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:43:09.361947 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:09.361947 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:09.361947 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:09.361947 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:09.361947 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:09.361947 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:09.361947 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:09.361947 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:09.363508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:43:09.363508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:43:09.363508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:43:09.363508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:43:09.363508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:43:09.363508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:43:09.363508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:43:09.363508 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:43:09.363508 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 529/1080] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:43:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:43:09.368409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda4f37-9f55-4bc9-8a42-08194207d1c7 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:09.411170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:09.419695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8195b15-15f1-4d35-8e1e-ee0e49857ded None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:09.453556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a00f5-51f2-48ac-9c79-beb4491596f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:09.454461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a00f5-51f2-48ac-9c79-beb4491596f2 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:09.454748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a00f5-51f2-48ac-9c79-beb4491596f2 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:09.455026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a00f5-51f2-48ac-9c79-beb4491596f2 None admin] PATH_INFO: `/v3/projects/7dad805375304257ab32589b32f31511` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:09.456267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a00f5-51f2-48ac-9c79-beb4491596f2 None admin] RBAC: Authorizing `identity:get_project(project_id=7dad805375304257ab32589b32f31511)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:09.458225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a00f5-51f2-48ac-9c79-beb4491596f2 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:09.459970 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 552/1081] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:43:09 2022] GET /identity/v3/projects/7dad805375304257ab32589b32f31511 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:09.465889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a00f5-51f2-48ac-9c79-beb4491596f2 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:09.466533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda4f37-9f55-4bc9-8a42-08194207d1c7 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:09.500852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c550384-fb2e-4c7d-88fe-f47a0c86ce30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:09.501515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c550384-fb2e-4c7d-88fe-f47a0c86ce30 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:09.501816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c550384-fb2e-4c7d-88fe-f47a0c86ce30 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:09.502207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c550384-fb2e-4c7d-88fe-f47a0c86ce30 None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:09.503549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c550384-fb2e-4c7d-88fe-f47a0c86ce30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:09.503858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c550384-fb2e-4c7d-88fe-f47a0c86ce30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:09.505295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c550384-fb2e-4c7d-88fe-f47a0c86ce30 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:09.513641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:09.517293 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 530/1082] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:43:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:09.523563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c550384-fb2e-4c7d-88fe-f47a0c86ce30 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:09.563398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a00f5-51f2-48ac-9c79-beb4491596f2 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:09.597940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1358f-d62d-48df-a8c1-45fab8e2ad7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:09.598838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1358f-d62d-48df-a8c1-45fab8e2ad7d None admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:09.599125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1358f-d62d-48df-a8c1-45fab8e2ad7d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:09.599410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1358f-d62d-48df-a8c1-45fab8e2ad7d None admin] PATH_INFO: `/v3/projects/7dad805375304257ab32589b32f31511/users/3c18f3053359450195835269391cdd3a/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:09.601422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1358f-d62d-48df-a8c1-45fab8e2ad7d None admin] RBAC: Authorizing `identity:create_grant(project_id=7dad805375304257ab32589b32f31511, user_id=3c18f3053359450195835269391cdd3a, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:09.602967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1358f-d62d-48df-a8c1-45fab8e2ad7d None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:09.619700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c550384-fb2e-4c7d-88fe-f47a0c86ce30 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:09.652679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c01a4-c5de-4dd5-960e-19a755d1b070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:09.653555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c01a4-c5de-4dd5-960e-19a755d1b070 None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:09.653861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c01a4-c5de-4dd5-960e-19a755d1b070 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:09.654174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c01a4-c5de-4dd5-960e-19a755d1b070 None admin] PATH_INFO: `/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/cfdebb5e6b8d4e2d86f0a03a30c21671/roles/50445f06754843f49827b854ec728850` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:09.656387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c01a4-c5de-4dd5-960e-19a755d1b070 None admin] RBAC: Authorizing `identity:create_grant(project_id=3186f9d3c2b341afa8b098ebcaaf974b, user_id=cfdebb5e6b8d4e2d86f0a03a30c21671, role_id=50445f06754843f49827b854ec728850)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:09.657819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c01a4-c5de-4dd5-960e-19a755d1b070 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:09.672746 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 553/1083] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:43:09 2022] PUT /identity/v3/projects/7dad805375304257ab32589b32f31511/users/3c18f3053359450195835269391cdd3a/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 20:43:09.725327 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 531/1084] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:43:09 2022] PUT /identity/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/cfdebb5e6b8d4e2d86f0a03a30c21671/roles/50445f06754843f49827b854ec728850 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 20:43:11.954660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e053a-556b-43ac-aea3-89e807fc2ef6 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:11.955215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e053a-556b-43ac-aea3-89e807fc2ef6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:11.955215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e053a-556b-43ac-aea3-89e807fc2ef6 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:11.956620 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 554/1085] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:11.961931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621308d-7386-47f9-8f9d-640e7f6b490e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:11.962369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621308d-7386-47f9-8f9d-640e7f6b490e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:11.962482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621308d-7386-47f9-8f9d-640e7f6b490e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:11.971831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b5c7c-cbae-4405-a55c-1a4c700d8f13 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:11.972122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b5c7c-cbae-4405-a55c-1a4c700d8f13 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:11.972405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b5c7c-cbae-4405-a55c-1a4c700d8f13 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:11.973955 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 555/1086] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:43:11.978765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a3b99-de24-46fc-b049-8e67c6586ca4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:11.979054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a3b99-de24-46fc-b049-8e67c6586ca4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:11.979154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5621308d-7386-47f9-8f9d-640e7f6b490e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:11.979336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1a3b99-de24-46fc-b049-8e67c6586ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:11.980265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5621308d-7386-47f9-8f9d-640e7f6b490e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:11.995934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0b1a3b99-de24-46fc-b049-8e67c6586ca4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:11.997123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0b1a3b99-de24-46fc-b049-8e67c6586ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:12.011791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:12.037055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:12.048006 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 532/1087] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:43:12.065541 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 556/1088] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:43:12.073547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621308d-7386-47f9-8f9d-640e7f6b490e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:12.090464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a3b99-de24-46fc-b049-8e67c6586ca4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:12.172331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621308d-7386-47f9-8f9d-640e7f6b490e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:12.188152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1a3b99-de24-46fc-b049-8e67c6586ca4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:12.204886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817b784-49b5-44fd-a9ee-f06f51958b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:12.205618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817b784-49b5-44fd-a9ee-f06f51958b55 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:12.205930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817b784-49b5-44fd-a9ee-f06f51958b55 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:12.206755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817b784-49b5-44fd-a9ee-f06f51958b55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:12.216638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817b784-49b5-44fd-a9ee-f06f51958b55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:43:12.221646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255f605-ca02-4f23-ad79-8548dd03059c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:12.223039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255f605-ca02-4f23-ad79-8548dd03059c None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:12.223325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255f605-ca02-4f23-ad79-8548dd03059c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:12.223603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255f605-ca02-4f23-ad79-8548dd03059c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:12.224749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-9817b784-49b5-44fd-a9ee-f06f51958b55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:43:12.224749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:12.224749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:12.224749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:12.224749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:12.224749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:12.224749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:12.224749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:12.224749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:12.224749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:12.224749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:12.224749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:12.224749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:43:12.224749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:43:12.224749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:43:12.224749 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:43:12.225744 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:43:12.225744 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:43:12.225744 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:43:12.225744 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:43:12.225744 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:12.225744 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:12.225744 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:12.225744 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:12.225744 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:12.225744 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:12.225744 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:12.225744 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:43:12.225744 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:12.225744 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:12.225744 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:12.225744 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:12.226630 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:12.226630 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:12.226630 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:12.226630 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:12.226630 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:43:12.226630 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:43:12.226630 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:43:12.226630 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:43:12.226630 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:43:12.226630 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:43:12.226630 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:43:12.226630 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:43:12.226630 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 533/1089] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:43:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:43:12.231826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817b784-49b5-44fd-a9ee-f06f51958b55 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:12.233741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255f605-ca02-4f23-ad79-8548dd03059c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:43:12.242005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-1255f605-ca02-4f23-ad79-8548dd03059c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 20:43:12.242005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:12.242005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:12.242005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:12.242005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:12.242005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:12.242005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:12.242005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:12.242005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:12.242005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:12.242005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:12.242005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:12.242005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:43:12.242005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:43:12.242005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:43:12.242005 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:43:12.243032 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:43:12.243032 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:43:12.243032 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:43:12.243032 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:43:12.243032 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:12.243032 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:12.243032 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:12.243032 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:12.243032 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:12.243032 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:12.243032 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:12.243032 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:43:12.243032 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:12.243032 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:12.243032 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:12.243032 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:12.243867 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:12.243867 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:12.243867 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:12.243867 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:12.243867 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:43:12.243867 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:43:12.243867 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:43:12.243867 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:43:12.243867 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:43:12.243867 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:43:12.243867 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 28 20:43:12.243867 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:12.245976 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 557/1090] 10.222.0.33 () {56 vars in 1120 bytes} [Sat May 28 20:43:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:43:12.249222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255f605-ca02-4f23-ad79-8548dd03059c None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:12.261724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:12.282557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:12.334058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817b784-49b5-44fd-a9ee-f06f51958b55 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:12.349464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255f605-ca02-4f23-ad79-8548dd03059c None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:12.366452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905df2c-f36e-4906-b65e-a420e1217945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:12.367114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905df2c-f36e-4906-b65e-a420e1217945 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:12.367397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905df2c-f36e-4906-b65e-a420e1217945 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:12.367672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3905df2c-f36e-4906-b65e-a420e1217945 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:12.368422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905df2c-f36e-4906-b65e-a420e1217945 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:12.368732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905df2c-f36e-4906-b65e-a420e1217945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:12.370136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3905df2c-f36e-4906-b65e-a420e1217945 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:12.378542 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 534/1091] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:43:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:12.382410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b534c9-87bc-41bb-bec0-98c5051816eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:12.383079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b534c9-87bc-41bb-bec0-98c5051816eb None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:12.383452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b534c9-87bc-41bb-bec0-98c5051816eb None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:12.383633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b534c9-87bc-41bb-bec0-98c5051816eb None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:12.384215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905df2c-f36e-4906-b65e-a420e1217945 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:12.384381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b534c9-87bc-41bb-bec0-98c5051816eb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:12.384692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b534c9-87bc-41bb-bec0-98c5051816eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:12.386081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b534c9-87bc-41bb-bec0-98c5051816eb None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:12.395168 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 558/1092] 10.222.0.33 () {56 vars in 1115 bytes} [Sat May 28 20:43:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:12.400737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b534c9-87bc-41bb-bec0-98c5051816eb None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:12.483102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:12.486154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3905df2c-f36e-4906-b65e-a420e1217945 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:12.499031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:12.513808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b534c9-87bc-41bb-bec0-98c5051816eb None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:12.520937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690fcd6-dd30-4650-b6c3-cbdd61e1e0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:12.521815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690fcd6-dd30-4650-b6c3-cbdd61e1e0b6 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:12.522128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690fcd6-dd30-4650-b6c3-cbdd61e1e0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:12.522411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c690fcd6-dd30-4650-b6c3-cbdd61e1e0b6 None admin] PATH_INFO: `/v3/users/3c18f3053359450195835269391cdd3a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:12.523639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690fcd6-dd30-4650-b6c3-cbdd61e1e0b6 None admin] RBAC: Authorizing `identity:get_user(user_id=3c18f3053359450195835269391cdd3a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:12.525080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c690fcd6-dd30-4650-b6c3-cbdd61e1e0b6 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:12.526917 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 535/1093] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:43:12 2022] GET /identity/v3/users/3c18f3053359450195835269391cdd3a => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:12.532544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690fcd6-dd30-4650-b6c3-cbdd61e1e0b6 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:12.549720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50c1dd-fdc2-47af-b3a1-4d8fa306df65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:12.550582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50c1dd-fdc2-47af-b3a1-4d8fa306df65 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:12.550869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50c1dd-fdc2-47af-b3a1-4d8fa306df65 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:12.551144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50c1dd-fdc2-47af-b3a1-4d8fa306df65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:12.562635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50c1dd-fdc2-47af-b3a1-4d8fa306df65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:12.564172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50c1dd-fdc2-47af-b3a1-4d8fa306df65 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:12.582272 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-8f50c1dd-fdc2-47af-b3a1-4d8fa306df65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:43:12.582272 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:12.582272 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:12.582272 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:12.582272 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:12.582272 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:12.582272 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:12.582272 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:12.582272 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:12.582272 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:12.582272 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:12.582272 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:12.582272 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:43:12.582272 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:43:12.582272 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:43:12.583241 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:43:12.583241 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:12.583241 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:12.583241 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:12.583241 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:12.583241 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:12.583241 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:12.583241 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:12.583241 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:43:12.583241 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:12.583241 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:12.583241 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:12.583241 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:12.583241 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:12.583241 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:12.583241 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:12.583241 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:12.584171 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:43:12.584171 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:43:12.584171 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:43:12.584171 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:12.584404 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 559/1094] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:43:12.589874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50c1dd-fdc2-47af-b3a1-4d8fa306df65 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:12.633009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c690fcd6-dd30-4650-b6c3-cbdd61e1e0b6 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:12.666519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cd079-eb57-44c2-9367-7f8a919471ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:12.667170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cd079-eb57-44c2-9367-7f8a919471ef None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:12.667457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cd079-eb57-44c2-9367-7f8a919471ef None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:12.667735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213cd079-eb57-44c2-9367-7f8a919471ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:12.692326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50c1dd-fdc2-47af-b3a1-4d8fa306df65 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:12.715103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:12.718852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:12.725816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccdfa3-3c7f-4a21-b3ef-58e2822f7bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:12.726531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccdfa3-3c7f-4a21-b3ef-58e2822f7bff None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:12.726817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccdfa3-3c7f-4a21-b3ef-58e2822f7bff None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:12.727152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccdfa3-3c7f-4a21-b3ef-58e2822f7bff None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:12.727886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ccdfa3-3c7f-4a21-b3ef-58e2822f7bff None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:12.728195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ccdfa3-3c7f-4a21-b3ef-58e2822f7bff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:12.729577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ccdfa3-3c7f-4a21-b3ef-58e2822f7bff None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:12.740205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213cd079-eb57-44c2-9367-7f8a919471ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:12.741167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213cd079-eb57-44c2-9367-7f8a919471ef None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:12.742315 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 560/1095] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:12.747964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccdfa3-3c7f-4a21-b3ef-58e2822f7bff None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:12.841943 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 536/1096] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:43:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 28 20:43:12.847701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cd079-eb57-44c2-9367-7f8a919471ef None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:12.851804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccdfa3-3c7f-4a21-b3ef-58e2822f7bff None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:12.887807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a27a18-9237-474c-85e3-37fb90e70431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:12.888630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a27a18-9237-474c-85e3-37fb90e70431 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:12.888911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a27a18-9237-474c-85e3-37fb90e70431 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:12.889183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a27a18-9237-474c-85e3-37fb90e70431 None admin] PATH_INFO: `/v3/users/nova` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:12.896900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a27a18-9237-474c-85e3-37fb90e70431 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:12.900220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a27a18-9237-474c-85e3-37fb90e70431 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:12.913511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-d8a27a18-9237-474c-85e3-37fb90e70431 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 28 20:43:12.913511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:12.913511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:12.913511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:12.913511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:12.913511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:12.913511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:12.913511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:12.913511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:12.913511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:12.913511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:12.913511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:12.913511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:43:12.913511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:43:12.913511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:43:12.913511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:12.914819 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:12.915995 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:12.915995 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:12.915995 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:43:12.915995 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:43:12.915995 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:43:12.915995 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:43:12.915995 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:43:12.915995 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:43:12.915995 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 28 20:43:12.915995 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:12.915995 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 561/1097] 10.222.0.33 () {56 vars in 1093 bytes} [Sat May 28 20:43:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:43:12.920456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a27a18-9237-474c-85e3-37fb90e70431 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:12.934027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:12.953896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213cd079-eb57-44c2-9367-7f8a919471ef None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:12.973365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:13.000424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523d81c-c574-46ee-be44-f7f4b7b2967f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:13.001206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523d81c-c574-46ee-be44-f7f4b7b2967f None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:13.001492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523d81c-c574-46ee-be44-f7f4b7b2967f None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:13.001769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b523d81c-c574-46ee-be44-f7f4b7b2967f None admin] PATH_INFO: `/v3/projects/7dad805375304257ab32589b32f31511` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:13.003077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523d81c-c574-46ee-be44-f7f4b7b2967f None admin] RBAC: Authorizing `identity:get_project(project_id=7dad805375304257ab32589b32f31511)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:13.004502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b523d81c-c574-46ee-be44-f7f4b7b2967f None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:13.006234 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 537/1098] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:43:12 2022] GET /identity/v3/projects/7dad805375304257ab32589b32f31511 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:13.012186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523d81c-c574-46ee-be44-f7f4b7b2967f None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:13.028549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a27a18-9237-474c-85e3-37fb90e70431 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:13.064225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e64f48-5fa2-4070-883a-09f498094550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:13.064915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e64f48-5fa2-4070-883a-09f498094550 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:13.065201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e64f48-5fa2-4070-883a-09f498094550 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:13.065523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e64f48-5fa2-4070-883a-09f498094550 None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:13.066416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e64f48-5fa2-4070-883a-09f498094550 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:13.066733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e64f48-5fa2-4070-883a-09f498094550 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:13.068182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e64f48-5fa2-4070-883a-09f498094550 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:13.085887 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 562/1099] 10.222.0.33 () {56 vars in 1133 bytes} [Sat May 28 20:43:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:13.091448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e64f48-5fa2-4070-883a-09f498094550 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:13.120275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b523d81c-c574-46ee-be44-f7f4b7b2967f None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:13.155580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430cdbe1-362d-452c-8faf-caaf6e8110ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:13.156230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cdbe1-362d-452c-8faf-caaf6e8110ca None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:13.156513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cdbe1-362d-452c-8faf-caaf6e8110ca None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:13.156790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430cdbe1-362d-452c-8faf-caaf6e8110ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:13.157626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430cdbe1-362d-452c-8faf-caaf6e8110ca None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=7dad805375304257ab32589b32f31511, user.id=3c18f3053359450195835269391cdd3a) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:13.157970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430cdbe1-362d-452c-8faf-caaf6e8110ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:13.159914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430cdbe1-362d-452c-8faf-caaf6e8110ca None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:13.165812 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 538/1100] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:43:13 2022] GET /identity/v3/role_assignments?user.id=3c18f3053359450195835269391cdd3a&scope.project.id=7dad805375304257ab32589b32f31511&role.id=7805b93b32f1472f8365936d4d57d49a => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:13.193634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e64f48-5fa2-4070-883a-09f498094550 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:13.195414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:13.232228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90fd9d-1ab9-4d45-b4cd-8ec670ca9b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:13.232905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90fd9d-1ab9-4d45-b4cd-8ec670ca9b4f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:13.233192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90fd9d-1ab9-4d45-b4cd-8ec670ca9b4f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:13.233471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90fd9d-1ab9-4d45-b4cd-8ec670ca9b4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:13.305791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb90fd9d-1ab9-4d45-b4cd-8ec670ca9b4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:13.307314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb90fd9d-1ab9-4d45-b4cd-8ec670ca9b4f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:13.407172 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 563/1101] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:43:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 20:43:13.413724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430cdbe1-362d-452c-8faf-caaf6e8110ca None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:13.428559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:13.511770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430cdbe1-362d-452c-8faf-caaf6e8110ca None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:13.546277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399afb9-53bb-490f-b42b-02b0aa0c7d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:13.547071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399afb9-53bb-490f-b42b-02b0aa0c7d7f None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:13.547360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399afb9-53bb-490f-b42b-02b0aa0c7d7f None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:13.547640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399afb9-53bb-490f-b42b-02b0aa0c7d7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:13.557590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399afb9-53bb-490f-b42b-02b0aa0c7d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:13.559044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399afb9-53bb-490f-b42b-02b0aa0c7d7f None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:13.576556 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-3399afb9-53bb-490f-b42b-02b0aa0c7d7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:43:13.576556 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:13.576556 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:13.576556 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:13.576556 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:13.576556 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:13.576556 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:13.576556 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:13.576556 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:13.576556 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:13.576556 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:13.576556 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:13.576556 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:43:13.576556 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:43:13.576556 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:43:13.577590 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:43:13.577590 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:13.577590 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:13.577590 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:13.577590 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:13.577590 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:13.577590 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:13.577590 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:13.577590 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:43:13.577590 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:13.577590 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:13.577590 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:13.577590 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:13.577590 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:13.577590 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:13.577590 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:13.577590 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:13.578554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:43:13.578554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:43:13.578554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:43:13.578554 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:43:13.578554 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 539/1102] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:43:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:43:13.584030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90fd9d-1ab9-4d45-b4cd-8ec670ca9b4f None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:13.586344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:13.681213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90fd9d-1ab9-4d45-b4cd-8ec670ca9b4f None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:13.714832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429be382-d89a-4b7c-82fb-0da9a94a6838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:13.715501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429be382-d89a-4b7c-82fb-0da9a94a6838 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:13.715790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429be382-d89a-4b7c-82fb-0da9a94a6838 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:13.716073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429be382-d89a-4b7c-82fb-0da9a94a6838 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:13.716804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429be382-d89a-4b7c-82fb-0da9a94a6838 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:13.717114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429be382-d89a-4b7c-82fb-0da9a94a6838 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:13.718535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429be382-d89a-4b7c-82fb-0da9a94a6838 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:13.730754 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 564/1103] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:13.736323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399afb9-53bb-490f-b42b-02b0aa0c7d7f None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:13.798296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:13.836118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399afb9-53bb-490f-b42b-02b0aa0c7d7f None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:13.869206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3efc07-49dd-464f-b578-20dc1c9b4edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:13.870030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3efc07-49dd-464f-b578-20dc1c9b4edf None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:13.870316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3efc07-49dd-464f-b578-20dc1c9b4edf None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:13.870593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3efc07-49dd-464f-b578-20dc1c9b4edf None admin] PATH_INFO: `/v3/projects/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:13.877154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3efc07-49dd-464f-b578-20dc1c9b4edf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:13.879150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3efc07-49dd-464f-b578-20dc1c9b4edf None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:13.892898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-cf3efc07-49dd-464f-b578-20dc1c9b4edf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:43:13.892898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:13.892898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:13.892898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:13.892898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:13.892898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:13.892898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:13.892898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:13.892898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:13.892898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:13.892898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:13.892898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:13.892898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:43:13.892898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:43:13.892898 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:43:13.894719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:43:13.894719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:13.894719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:13.894719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:13.894719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:13.894719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:13.894719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:13.894719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:13.894719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:43:13.894719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:13.894719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:13.894719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:13.894719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:13.894719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:13.894719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:13.894719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:13.894719 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:13.895616 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:43:13.895616 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:43:13.895616 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:43:13.895616 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:43:13.895616 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:43:13.895616 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:43:13.895616 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:43:13.895616 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:43:13.895616 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 540/1104] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:43:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:43:13.900399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429be382-d89a-4b7c-82fb-0da9a94a6838 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:13.960669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:13.997594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429be382-d89a-4b7c-82fb-0da9a94a6838 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:14.032301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e780e7-23c5-4293-bf0a-b6e06f5bc6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:14.032997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e780e7-23c5-4293-bf0a-b6e06f5bc6bb None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:14.033286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e780e7-23c5-4293-bf0a-b6e06f5bc6bb None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:14.033567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e780e7-23c5-4293-bf0a-b6e06f5bc6bb None admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:14.034442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e780e7-23c5-4293-bf0a-b6e06f5bc6bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:14.034755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e780e7-23c5-4293-bf0a-b6e06f5bc6bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:14.036179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e780e7-23c5-4293-bf0a-b6e06f5bc6bb None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:14.047394 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 565/1105] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:43:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:14.053349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3efc07-49dd-464f-b578-20dc1c9b4edf None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:14.150852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3efc07-49dd-464f-b578-20dc1c9b4edf None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:14.172855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:14.183683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52494dc2-873d-49c8-84ee-d06b38988a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:14.184755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52494dc2-873d-49c8-84ee-d06b38988a09 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:14.184755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52494dc2-873d-49c8-84ee-d06b38988a09 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:14.185004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52494dc2-873d-49c8-84ee-d06b38988a09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:14.185698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52494dc2-873d-49c8-84ee-d06b38988a09 None admin] RBAC: Adding query filter params (role.id=50445f06754843f49827b854ec728850, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=cfdebb5e6b8d4e2d86f0a03a30c21671) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:14.186050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52494dc2-873d-49c8-84ee-d06b38988a09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:14.187500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52494dc2-873d-49c8-84ee-d06b38988a09 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:14.193811 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 541/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:43:14 2022] GET /identity/v3/role_assignments?user.id=cfdebb5e6b8d4e2d86f0a03a30c21671&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=50445f06754843f49827b854ec728850 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:15.463132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e6080-c603-48be-a476-9ce2f5eee8ef None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:15.463638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e6080-c603-48be-a476-9ce2f5eee8ef None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:15.463638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e6080-c603-48be-a476-9ce2f5eee8ef None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:15.465001 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 566/1107] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:15.470293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d08ef-f70c-4b0c-a8bc-f4375a9a34ec None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:15.470656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d08ef-f70c-4b0c-a8bc-f4375a9a34ec None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:15.470854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d08ef-f70c-4b0c-a8bc-f4375a9a34ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:15.487398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-238d08ef-f70c-4b0c-a8bc-f4375a9a34ec None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:15.488587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-238d08ef-f70c-4b0c-a8bc-f4375a9a34ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:15.552778 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 542/1108] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:43:15.580051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e6080-c603-48be-a476-9ce2f5eee8ef None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:15.679963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e6080-c603-48be-a476-9ce2f5eee8ef None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:15.701986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:15.713286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2ba672-31b5-40bf-8763-4b95ad5646e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:15.714060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ba672-31b5-40bf-8763-4b95ad5646e2 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:15.714350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ba672-31b5-40bf-8763-4b95ad5646e2 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:15.714631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ba672-31b5-40bf-8763-4b95ad5646e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:15.715810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ba672-31b5-40bf-8763-4b95ad5646e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:15.717252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2ba672-31b5-40bf-8763-4b95ad5646e2 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:15.718899 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 567/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:43:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:15.728188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d08ef-f70c-4b0c-a8bc-f4375a9a34ec None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:15.820255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d08ef-f70c-4b0c-a8bc-f4375a9a34ec None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:15.836530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:15.852551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2444e961-be28-4675-a44e-5010a1a68bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:15.853209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444e961-be28-4675-a44e-5010a1a68bd1 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:15.853493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444e961-be28-4675-a44e-5010a1a68bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:15.853768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2444e961-be28-4675-a44e-5010a1a68bd1 None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:15.854508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2444e961-be28-4675-a44e-5010a1a68bd1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:15.855945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2444e961-be28-4675-a44e-5010a1a68bd1 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:15.947299 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 543/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:43:15 2022] POST /identity/v3/users => generated 286 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:43:18.238131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfe922-7d8d-4c3c-a7c1-75449dbf8757 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:18.238688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfe922-7d8d-4c3c-a7c1-75449dbf8757 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:18.238688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfe922-7d8d-4c3c-a7c1-75449dbf8757 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:18.240054 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 568/1111] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:18.245195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f354e-0e6f-4ceb-b22d-2a6db04a4075 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:18.245561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f354e-0e6f-4ceb-b22d-2a6db04a4075 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:18.245762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f354e-0e6f-4ceb-b22d-2a6db04a4075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:18.263479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-886f354e-0e6f-4ceb-b22d-2a6db04a4075 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:18.264665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-886f354e-0e6f-4ceb-b22d-2a6db04a4075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:18.329416 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 544/1112] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:43:18.355383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfe922-7d8d-4c3c-a7c1-75449dbf8757 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:18.454645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfe922-7d8d-4c3c-a7c1-75449dbf8757 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:18.486781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd8f8b-5c4e-406e-b260-1b65ab979c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:18.487649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd8f8b-5c4e-406e-b260-1b65ab979c8f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:18.487928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd8f8b-5c4e-406e-b260-1b65ab979c8f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:18.488242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfd8f8b-5c4e-406e-b260-1b65ab979c8f None admin] PATH_INFO: `/v3/roles/92b887e453ad4fa9aa5c49f7594a7e53` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:18.489739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd8f8b-5c4e-406e-b260-1b65ab979c8f None admin] RBAC: Authorizing `identity:get_role(role_id=92b887e453ad4fa9aa5c49f7594a7e53)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:18.491135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfd8f8b-5c4e-406e-b260-1b65ab979c8f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:18.492527 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 569/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:43:18 2022] GET /identity/v3/roles/92b887e453ad4fa9aa5c49f7594a7e53 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:18.498232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f354e-0e6f-4ceb-b22d-2a6db04a4075 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:18.594081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f354e-0e6f-4ceb-b22d-2a6db04a4075 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:18.596248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:18.626224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69b51e-e739-4613-8eb5-e4b5e838361c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:18.627078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69b51e-e739-4613-8eb5-e4b5e838361c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:18.627360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69b51e-e739-4613-8eb5-e4b5e838361c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:18.627633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69b51e-e739-4613-8eb5-e4b5e838361c None admin] PATH_INFO: `/v3/users/cdabf898dddb4888afe86d7ac6c597c1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:18.641143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69b51e-e739-4613-8eb5-e4b5e838361c None admin] RBAC: Authorizing `identity:get_user(user_id=cdabf898dddb4888afe86d7ac6c597c1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:18.642622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69b51e-e739-4613-8eb5-e4b5e838361c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:18.644551 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 545/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:43:18 2022] GET /identity/v3/users/cdabf898dddb4888afe86d7ac6c597c1 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:18.650363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd8f8b-5c4e-406e-b260-1b65ab979c8f None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:18.708847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:18.750035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfd8f8b-5c4e-406e-b260-1b65ab979c8f None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:18.782937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302c5ba-ce86-498e-b3ba-462484e038d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:18.783573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302c5ba-ce86-498e-b3ba-462484e038d2 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:18.783857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302c5ba-ce86-498e-b3ba-462484e038d2 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:18.784176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e302c5ba-ce86-498e-b3ba-462484e038d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:18.848702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302c5ba-ce86-498e-b3ba-462484e038d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:18.850125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e302c5ba-ce86-498e-b3ba-462484e038d2 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:18.903884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:18.944510 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 570/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:43:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 20:43:18.950435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69b51e-e739-4613-8eb5-e4b5e838361c None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:19.048990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69b51e-e739-4613-8eb5-e4b5e838361c None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:19.082495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccefec0-e384-432e-a4d7-37ca24afb10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:19.083350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccefec0-e384-432e-a4d7-37ca24afb10b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:19.083634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccefec0-e384-432e-a4d7-37ca24afb10b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:19.083910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccefec0-e384-432e-a4d7-37ca24afb10b None admin] PATH_INFO: `/v3/projects/7dad805375304257ab32589b32f31511` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:19.085072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccefec0-e384-432e-a4d7-37ca24afb10b None admin] RBAC: Authorizing `identity:get_project(project_id=7dad805375304257ab32589b32f31511)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:19.086556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccefec0-e384-432e-a4d7-37ca24afb10b None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:19.088245 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 546/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:43:18 2022] GET /identity/v3/projects/7dad805375304257ab32589b32f31511 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:19.094175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302c5ba-ce86-498e-b3ba-462484e038d2 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:19.190100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e302c5ba-ce86-498e-b3ba-462484e038d2 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:19.223610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aff262e-93a8-4c3e-b74c-68b1d68eeecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:19.224263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff262e-93a8-4c3e-b74c-68b1d68eeecd None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:19.224551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff262e-93a8-4c3e-b74c-68b1d68eeecd None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:19.224834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff262e-93a8-4c3e-b74c-68b1d68eeecd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:19.225665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff262e-93a8-4c3e-b74c-68b1d68eeecd None admin] RBAC: Adding query filter params (role.id=92b887e453ad4fa9aa5c49f7594a7e53, scope.project.id=7dad805375304257ab32589b32f31511, user.id=cdabf898dddb4888afe86d7ac6c597c1) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:19.226015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff262e-93a8-4c3e-b74c-68b1d68eeecd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:19.227451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aff262e-93a8-4c3e-b74c-68b1d68eeecd None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:19.232827 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 571/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:43:19 2022] GET /identity/v3/role_assignments?user.id=cdabf898dddb4888afe86d7ac6c597c1&scope.project.id=7dad805375304257ab32589b32f31511&role.id=92b887e453ad4fa9aa5c49f7594a7e53 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:21.458071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce67f2-2d75-4bcd-a022-47a600ccb978 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:21.458590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce67f2-2d75-4bcd-a022-47a600ccb978 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:21.458590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce67f2-2d75-4bcd-a022-47a600ccb978 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:21.459934 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 547/1118] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:21.465168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca4720-4d3d-410d-abae-9c06e9fb1e7e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:21.465542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca4720-4d3d-410d-abae-9c06e9fb1e7e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:21.465744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca4720-4d3d-410d-abae-9c06e9fb1e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:21.482400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fdca4720-4d3d-410d-abae-9c06e9fb1e7e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:21.483598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fdca4720-4d3d-410d-abae-9c06e9fb1e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:21.499880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:21.549499 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 572/1119] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:43:21.573596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce67f2-2d75-4bcd-a022-47a600ccb978 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:21.603748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:21.673024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce67f2-2d75-4bcd-a022-47a600ccb978 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:21.706539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f5c12-bebd-46ab-b276-ae5234a99607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:21.707382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f5c12-bebd-46ab-b276-ae5234a99607 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:21.707666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f5c12-bebd-46ab-b276-ae5234a99607 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:21.707943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f5c12-bebd-46ab-b276-ae5234a99607 None admin] PATH_INFO: `/v3/roles/92b887e453ad4fa9aa5c49f7594a7e53` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:21.709470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f5c12-bebd-46ab-b276-ae5234a99607 None admin] RBAC: Authorizing `identity:get_role(role_id=92b887e453ad4fa9aa5c49f7594a7e53)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:21.710873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f5c12-bebd-46ab-b276-ae5234a99607 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:21.712283 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 548/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:43:21 2022] GET /identity/v3/roles/92b887e453ad4fa9aa5c49f7594a7e53 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:21.718108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca4720-4d3d-410d-abae-9c06e9fb1e7e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:21.814911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca4720-4d3d-410d-abae-9c06e9fb1e7e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:21.847695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d17e54-f010-4e1a-9429-ce5fbdef3e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:21.848563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d17e54-f010-4e1a-9429-ce5fbdef3e12 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:21.848849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d17e54-f010-4e1a-9429-ce5fbdef3e12 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:21.849128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d17e54-f010-4e1a-9429-ce5fbdef3e12 None admin] PATH_INFO: `/v3/users/cdabf898dddb4888afe86d7ac6c597c1` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:21.850340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d17e54-f010-4e1a-9429-ce5fbdef3e12 None admin] RBAC: Authorizing `identity:get_user(user_id=cdabf898dddb4888afe86d7ac6c597c1)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:21.851773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d17e54-f010-4e1a-9429-ce5fbdef3e12 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:21.853379 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 573/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:43:21 2022] GET /identity/v3/users/cdabf898dddb4888afe86d7ac6c597c1 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:21.859228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f5c12-bebd-46ab-b276-ae5234a99607 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:21.953687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:21.957448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f5c12-bebd-46ab-b276-ae5234a99607 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:21.990446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765efc1-4a3a-4d48-9199-32c7fce327eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:21.991262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765efc1-4a3a-4d48-9199-32c7fce327eb None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:21.991743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765efc1-4a3a-4d48-9199-32c7fce327eb None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:21.992179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765efc1-4a3a-4d48-9199-32c7fce327eb None admin] PATH_INFO: `/v3/projects/7dad805375304257ab32589b32f31511` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:21.993666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765efc1-4a3a-4d48-9199-32c7fce327eb None admin] RBAC: Authorizing `identity:get_project(project_id=7dad805375304257ab32589b32f31511)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:21.995139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765efc1-4a3a-4d48-9199-32c7fce327eb None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:21.996809 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 549/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:43:21 2022] GET /identity/v3/projects/7dad805375304257ab32589b32f31511 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:22.002669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d17e54-f010-4e1a-9429-ce5fbdef3e12 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:22.023831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:22.099383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d17e54-f010-4e1a-9429-ce5fbdef3e12 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:22.133223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16a4e8-3d6e-49a1-9b3a-780da129f3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:22.134126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16a4e8-3d6e-49a1-9b3a-780da129f3f5 None admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:22.134409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16a4e8-3d6e-49a1-9b3a-780da129f3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:22.134685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16a4e8-3d6e-49a1-9b3a-780da129f3f5 None admin] PATH_INFO: `/v3/projects/7dad805375304257ab32589b32f31511/users/cdabf898dddb4888afe86d7ac6c597c1/roles/92b887e453ad4fa9aa5c49f7594a7e53` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:22.136855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb16a4e8-3d6e-49a1-9b3a-780da129f3f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=7dad805375304257ab32589b32f31511, user_id=cdabf898dddb4888afe86d7ac6c597c1, role_id=92b887e453ad4fa9aa5c49f7594a7e53)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:22.138312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb16a4e8-3d6e-49a1-9b3a-780da129f3f5 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:22.192416 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 574/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:43:22 2022] PUT /identity/v3/projects/7dad805375304257ab32589b32f31511/users/cdabf898dddb4888afe86d7ac6c597c1/roles/92b887e453ad4fa9aa5c49f7594a7e53 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:43:24.421014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5021bfe-d590-4d46-bc13-7e0fd1ba8fac None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:24.421553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5021bfe-d590-4d46-bc13-7e0fd1ba8fac None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:24.421553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5021bfe-d590-4d46-bc13-7e0fd1ba8fac None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:24.422878 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 550/1124] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:24.428788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea3c88-81d6-4eeb-a6ad-210d561d264e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:24.429223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea3c88-81d6-4eeb-a6ad-210d561d264e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:24.429355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea3c88-81d6-4eeb-a6ad-210d561d264e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:24.445403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-00ea3c88-81d6-4eeb-a6ad-210d561d264e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:24.446639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-00ea3c88-81d6-4eeb-a6ad-210d561d264e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:24.511247 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 575/1125] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:43:24.537273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5021bfe-d590-4d46-bc13-7e0fd1ba8fac None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:24.635582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5021bfe-d590-4d46-bc13-7e0fd1ba8fac None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:24.669490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53416af6-6693-45a3-89af-f0480a2652b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:24.670348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53416af6-6693-45a3-89af-f0480a2652b9 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:24.670630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53416af6-6693-45a3-89af-f0480a2652b9 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:24.670902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53416af6-6693-45a3-89af-f0480a2652b9 None admin] PATH_INFO: `/v3/roles/92b887e453ad4fa9aa5c49f7594a7e53` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:24.672390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53416af6-6693-45a3-89af-f0480a2652b9 None admin] RBAC: Authorizing `identity:get_role(role_id=92b887e453ad4fa9aa5c49f7594a7e53)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:24.673730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53416af6-6693-45a3-89af-f0480a2652b9 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:24.675131 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 551/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:43:24 2022] GET /identity/v3/roles/92b887e453ad4fa9aa5c49f7594a7e53 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:24.680879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea3c88-81d6-4eeb-a6ad-210d561d264e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:24.692660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:24.777305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea3c88-81d6-4eeb-a6ad-210d561d264e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:24.810434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4011f-8246-4789-9052-feb831f908c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:24.811297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4011f-8246-4789-9052-feb831f908c7 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:24.811583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4011f-8246-4789-9052-feb831f908c7 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:24.811864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4011f-8246-4789-9052-feb831f908c7 None admin] PATH_INFO: `/v3/users/cdabf898dddb4888afe86d7ac6c597c1` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:24.813046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4011f-8246-4789-9052-feb831f908c7 None admin] RBAC: Authorizing `identity:get_user(user_id=cdabf898dddb4888afe86d7ac6c597c1)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:24.814520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4011f-8246-4789-9052-feb831f908c7 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:24.816756 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 576/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:43:24 2022] GET /identity/v3/users/cdabf898dddb4888afe86d7ac6c597c1 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:24.822388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53416af6-6693-45a3-89af-f0480a2652b9 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:24.854625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:24.918788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53416af6-6693-45a3-89af-f0480a2652b9 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:24.952347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729e36c-4651-4ee5-a6fe-7c34f01ba06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:24.952924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729e36c-4651-4ee5-a6fe-7c34f01ba06d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:24.953206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729e36c-4651-4ee5-a6fe-7c34f01ba06d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:24.953483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729e36c-4651-4ee5-a6fe-7c34f01ba06d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:25.019752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729e36c-4651-4ee5-a6fe-7c34f01ba06d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:25.021203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729e36c-4651-4ee5-a6fe-7c34f01ba06d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:25.053250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:25.119226 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 552/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:43:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 20:43:25.125104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4011f-8246-4789-9052-feb831f908c7 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:25.200857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:25.221865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4011f-8246-4789-9052-feb831f908c7 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:25.254143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9243ce5f-c97e-47fc-8ecb-97378edae769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:25.254974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243ce5f-c97e-47fc-8ecb-97378edae769 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:25.255256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243ce5f-c97e-47fc-8ecb-97378edae769 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:25.255526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9243ce5f-c97e-47fc-8ecb-97378edae769 None admin] PATH_INFO: `/v3/projects/7dad805375304257ab32589b32f31511` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:25.256685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243ce5f-c97e-47fc-8ecb-97378edae769 None admin] RBAC: Authorizing `identity:get_project(project_id=7dad805375304257ab32589b32f31511)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:25.258128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9243ce5f-c97e-47fc-8ecb-97378edae769 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:25.259774 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 577/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:43:25 2022] GET /identity/v3/projects/7dad805375304257ab32589b32f31511 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:25.265990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729e36c-4651-4ee5-a6fe-7c34f01ba06d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:25.359835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729e36c-4651-4ee5-a6fe-7c34f01ba06d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:25.392958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6a9d6-d85a-47d5-b0e9-e18d54477c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:25.393612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6a9d6-d85a-47d5-b0e9-e18d54477c2a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:25.393922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6a9d6-d85a-47d5-b0e9-e18d54477c2a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:25.394199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6a9d6-d85a-47d5-b0e9-e18d54477c2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:25.395015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6a9d6-d85a-47d5-b0e9-e18d54477c2a None admin] RBAC: Adding query filter params (role.id=92b887e453ad4fa9aa5c49f7594a7e53, scope.project.id=7dad805375304257ab32589b32f31511, user.id=cdabf898dddb4888afe86d7ac6c597c1) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:25.395321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6a9d6-d85a-47d5-b0e9-e18d54477c2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:25.396735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6a9d6-d85a-47d5-b0e9-e18d54477c2a None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:25.398526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:25.402871 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 553/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:43:25 2022] GET /identity/v3/role_assignments?user.id=cdabf898dddb4888afe86d7ac6c597c1&scope.project.id=7dad805375304257ab32589b32f31511&role.id=92b887e453ad4fa9aa5c49f7594a7e53 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:27.657718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eb910-7473-4e0b-be06-79ad59f8b290 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:27.658278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eb910-7473-4e0b-be06-79ad59f8b290 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:27.658278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eb910-7473-4e0b-be06-79ad59f8b290 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:27.660260 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 578/1131] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:27.665526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cdf5c-c890-49db-90b4-3217f280d7d9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:27.665964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cdf5c-c890-49db-90b4-3217f280d7d9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:27.666106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cdf5c-c890-49db-90b4-3217f280d7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:27.682533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-404cdf5c-c890-49db-90b4-3217f280d7d9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:27.683706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-404cdf5c-c890-49db-90b4-3217f280d7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:27.748851 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 554/1132] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:43:27.775396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7eb910-7473-4e0b-be06-79ad59f8b290 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:27.873388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7eb910-7473-4e0b-be06-79ad59f8b290 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:27.905749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e9f5a-352d-473c-b4f8-e23193f1cb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:27.906540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e9f5a-352d-473c-b4f8-e23193f1cb12 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:27.906824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e9f5a-352d-473c-b4f8-e23193f1cb12 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:27.907102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e9f5a-352d-473c-b4f8-e23193f1cb12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:27.908310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e9f5a-352d-473c-b4f8-e23193f1cb12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:27.910357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110e9f5a-352d-473c-b4f8-e23193f1cb12 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:27.911924 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 579/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:43:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:27.917565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cdf5c-c890-49db-90b4-3217f280d7d9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:28.013058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cdf5c-c890-49db-90b4-3217f280d7d9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:28.038964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:28.046471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dc062-7ab7-45af-8cc0-6cfe578592a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:28.047126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dc062-7ab7-45af-8cc0-6cfe578592a1 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:28.047411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dc062-7ab7-45af-8cc0-6cfe578592a1 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:28.047689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dc062-7ab7-45af-8cc0-6cfe578592a1 None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:28.048391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898dc062-7ab7-45af-8cc0-6cfe578592a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:28.049823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898dc062-7ab7-45af-8cc0-6cfe578592a1 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:28.159511 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 555/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 20:43:27 2022] POST /identity/v3/projects => generated 311 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:43:30.422718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd754366-d6fa-4460-85ac-b416fc212d8b None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:30.423217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd754366-d6fa-4460-85ac-b416fc212d8b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:30.423217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd754366-d6fa-4460-85ac-b416fc212d8b None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:30.425250 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 580/1135] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:30.430377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49c86b-cedd-46b3-93ff-68c076eb8155 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:30.430776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49c86b-cedd-46b3-93ff-68c076eb8155 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:30.430928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49c86b-cedd-46b3-93ff-68c076eb8155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:30.448169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9c49c86b-cedd-46b3-93ff-68c076eb8155 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:30.449357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9c49c86b-cedd-46b3-93ff-68c076eb8155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:30.513354 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 556/1136] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:43:30.540811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd754366-d6fa-4460-85ac-b416fc212d8b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:30.557459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:30.640660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd754366-d6fa-4460-85ac-b416fc212d8b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:30.673384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71142cd-2834-4d44-b550-9d8a223db725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:30.674151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71142cd-2834-4d44-b550-9d8a223db725 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:30.674461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71142cd-2834-4d44-b550-9d8a223db725 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:30.674735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71142cd-2834-4d44-b550-9d8a223db725 None admin] PATH_INFO: `/v3/domains/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:30.675911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71142cd-2834-4d44-b550-9d8a223db725 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:30.677346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71142cd-2834-4d44-b550-9d8a223db725 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:30.678987 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 581/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:43:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:30.684705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49c86b-cedd-46b3-93ff-68c076eb8155 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:30.781165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49c86b-cedd-46b3-93ff-68c076eb8155 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:30.791898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:30.814225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf61b98-f9f3-4e65-b8dc-001b4183d446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:30.814885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf61b98-f9f3-4e65-b8dc-001b4183d446 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:30.815194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf61b98-f9f3-4e65-b8dc-001b4183d446 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:30.815472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf61b98-f9f3-4e65-b8dc-001b4183d446 None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:30.816187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf61b98-f9f3-4e65-b8dc-001b4183d446 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:30.817625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf61b98-f9f3-4e65-b8dc-001b4183d446 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:30.906359 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 557/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:43:30 2022] POST /identity/v3/users => generated 286 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:43:33.204982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114384a1-93b2-4e37-aef7-8944f95ff730 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:33.205521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114384a1-93b2-4e37-aef7-8944f95ff730 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:33.205521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114384a1-93b2-4e37-aef7-8944f95ff730 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:33.206956 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 582/1139] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:33.212083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9595dd3-f427-43b0-96d3-b6393f834895 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:33.212489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9595dd3-f427-43b0-96d3-b6393f834895 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:33.212639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9595dd3-f427-43b0-96d3-b6393f834895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:33.229667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a9595dd3-f427-43b0-96d3-b6393f834895 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:33.230880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a9595dd3-f427-43b0-96d3-b6393f834895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:33.294846 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 558/1140] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:43:33.320591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114384a1-93b2-4e37-aef7-8944f95ff730 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:33.401251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:33.420030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114384a1-93b2-4e37-aef7-8944f95ff730 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:33.453633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e8c5b-45de-4215-aafc-d9fbf99dfdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:33.454446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e8c5b-45de-4215-aafc-d9fbf99dfdec None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:33.454733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e8c5b-45de-4215-aafc-d9fbf99dfdec None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:33.455014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e8c5b-45de-4215-aafc-d9fbf99dfdec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:33.464839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162e8c5b-45de-4215-aafc-d9fbf99dfdec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:43:33.473029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-162e8c5b-45de-4215-aafc-d9fbf99dfdec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:43:33.473029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:33.473029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:33.473029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:33.473029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:33.473029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:33.473029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:33.473029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:33.473029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:33.473029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:33.473029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:33.473029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:33.473029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:43:33.473029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:43:33.473029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:43:33.473029 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:43:33.474013 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:43:33.474013 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:43:33.474013 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:43:33.474013 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:43:33.474013 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:33.474013 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:33.474013 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:33.474013 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:33.474013 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:33.474013 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:33.474013 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:33.474013 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:43:33.474013 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:33.474013 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:33.474013 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:33.474013 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:33.474931 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:33.474931 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:33.474931 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:33.474931 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:33.474931 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:43:33.474931 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:43:33.474931 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:43:33.474931 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:43:33.474931 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:43:33.474931 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:43:33.474931 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:43:33.474931 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:33.474931 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 583/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:43:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:43:33.480162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9595dd3-f427-43b0-96d3-b6393f834895 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:33.571150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:33.576818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9595dd3-f427-43b0-96d3-b6393f834895 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:33.609399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7707e-9730-4dcd-927e-b08cb68abe3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:33.610136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7707e-9730-4dcd-927e-b08cb68abe3d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:33.610629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7707e-9730-4dcd-927e-b08cb68abe3d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:33.611281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7707e-9730-4dcd-927e-b08cb68abe3d None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:33.612686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7707e-9730-4dcd-927e-b08cb68abe3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:33.612998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7707e-9730-4dcd-927e-b08cb68abe3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:33.614424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac7707e-9730-4dcd-927e-b08cb68abe3d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:33.622223 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 559/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:43:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:33.628002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e8c5b-45de-4215-aafc-d9fbf99dfdec None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:33.725537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e8c5b-45de-4215-aafc-d9fbf99dfdec None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:33.756932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827d53d-d7e8-4de6-8ee3-5671c2e3a176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:33.757780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827d53d-d7e8-4de6-8ee3-5671c2e3a176 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:33.758089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827d53d-d7e8-4de6-8ee3-5671c2e3a176 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:33.758366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827d53d-d7e8-4de6-8ee3-5671c2e3a176 None admin] PATH_INFO: `/v3/users/2ac5785a0b1449b297122b0c27afadc7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:33.769673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:33.772030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827d53d-d7e8-4de6-8ee3-5671c2e3a176 None admin] RBAC: Authorizing `identity:get_user(user_id=2ac5785a0b1449b297122b0c27afadc7)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:33.773455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827d53d-d7e8-4de6-8ee3-5671c2e3a176 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:33.775090 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 584/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:43:33 2022] GET /identity/v3/users/2ac5785a0b1449b297122b0c27afadc7 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:33.780849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7707e-9730-4dcd-927e-b08cb68abe3d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:33.877198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7707e-9730-4dcd-927e-b08cb68abe3d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:33.910142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5e09e-a472-46af-972e-48f381e7962b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:33.910774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e09e-a472-46af-972e-48f381e7962b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:33.911059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e09e-a472-46af-972e-48f381e7962b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:33.911336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5e09e-a472-46af-972e-48f381e7962b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:33.936587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:33.975923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5e09e-a472-46af-972e-48f381e7962b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:33.977368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5e09e-a472-46af-972e-48f381e7962b None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:34.073500 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 560/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:43:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 20:43:34.079253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827d53d-d7e8-4de6-8ee3-5671c2e3a176 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:34.173994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827d53d-d7e8-4de6-8ee3-5671c2e3a176 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:34.206683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746cd2d-c0ea-4d68-8d78-e9b2d985453d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:34.207536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746cd2d-c0ea-4d68-8d78-e9b2d985453d None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:34.207813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746cd2d-c0ea-4d68-8d78-e9b2d985453d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:34.208085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746cd2d-c0ea-4d68-8d78-e9b2d985453d None admin] PATH_INFO: `/v3/projects/e39b7ce2b17c49a5b2e627337e1d09ad` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:34.209244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746cd2d-c0ea-4d68-8d78-e9b2d985453d None admin] RBAC: Authorizing `identity:get_project(project_id=e39b7ce2b17c49a5b2e627337e1d09ad)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:34.210686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746cd2d-c0ea-4d68-8d78-e9b2d985453d None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:34.212336 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 585/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:43:34 2022] GET /identity/v3/projects/e39b7ce2b17c49a5b2e627337e1d09ad => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:34.218348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5e09e-a472-46af-972e-48f381e7962b None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:34.274893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:34.313612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5e09e-a472-46af-972e-48f381e7962b None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:34.346618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa129a1-ca58-4463-adf0-32e49a3a623f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:34.347464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa129a1-ca58-4463-adf0-32e49a3a623f None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:34.348206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa129a1-ca58-4463-adf0-32e49a3a623f None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:34.348488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa129a1-ca58-4463-adf0-32e49a3a623f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:34.349317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa129a1-ca58-4463-adf0-32e49a3a623f None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=e39b7ce2b17c49a5b2e627337e1d09ad, user.id=2ac5785a0b1449b297122b0c27afadc7) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:34.349634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa129a1-ca58-4463-adf0-32e49a3a623f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:34.351113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa129a1-ca58-4463-adf0-32e49a3a623f None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:34.356446 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 561/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:43:34 2022] GET /identity/v3/role_assignments?user.id=2ac5785a0b1449b297122b0c27afadc7&scope.project.id=e39b7ce2b17c49a5b2e627337e1d09ad&role.id=7805b93b32f1472f8365936d4d57d49a => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:36.592066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d98c41-9dfa-4f66-b1fd-8b7aec53af43 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:36.592644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d98c41-9dfa-4f66-b1fd-8b7aec53af43 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:36.592644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d98c41-9dfa-4f66-b1fd-8b7aec53af43 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:36.594133 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 586/1147] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:36.599322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6977d22-2133-459e-b246-55b9ec3dd805 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:36.599727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6977d22-2133-459e-b246-55b9ec3dd805 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:36.599882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6977d22-2133-459e-b246-55b9ec3dd805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:36.616886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d6977d22-2133-459e-b246-55b9ec3dd805 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:36.618115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d6977d22-2133-459e-b246-55b9ec3dd805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:36.683508 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 562/1148] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:43:36.707382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d98c41-9dfa-4f66-b1fd-8b7aec53af43 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:36.773662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:36.806917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d98c41-9dfa-4f66-b1fd-8b7aec53af43 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:36.839473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a9ddb-86dd-4a06-ba2b-e4a31b16481b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:36.840252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a9ddb-86dd-4a06-ba2b-e4a31b16481b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:36.840538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a9ddb-86dd-4a06-ba2b-e4a31b16481b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:36.840815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8a9ddb-86dd-4a06-ba2b-e4a31b16481b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:36.850945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8a9ddb-86dd-4a06-ba2b-e4a31b16481b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:43:36.858644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-5e8a9ddb-86dd-4a06-ba2b-e4a31b16481b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:43:36.858644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:36.858644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:36.858644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:36.858644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:36.858644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:36.858644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:36.858644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:36.858644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:36.858644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:36.858644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:36.858644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:36.858644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:43:36.858644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:43:36.858644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:43:36.858644 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:43:36.859829 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:43:36.859829 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:43:36.859829 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:43:36.859829 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:43:36.859829 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:36.859829 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:36.859829 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:36.859829 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:36.859829 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:36.859829 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:36.859829 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:36.859829 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:43:36.859829 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:36.859829 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:36.859829 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:36.859829 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:36.860951 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:36.860951 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:36.860951 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:36.860951 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:36.860951 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:43:36.860951 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:43:36.860951 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:43:36.860951 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:43:36.860951 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:43:36.860951 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:43:36.860951 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:43:36.860951 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:36.860951 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 587/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:43:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:43:36.866545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6977d22-2133-459e-b246-55b9ec3dd805 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:36.916742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:36.962273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6977d22-2133-459e-b246-55b9ec3dd805 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:36.995513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf64cc-571b-4ca4-92ed-80d7504bcb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:36.996178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf64cc-571b-4ca4-92ed-80d7504bcb95 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:36.996465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf64cc-571b-4ca4-92ed-80d7504bcb95 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:36.996746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf64cc-571b-4ca4-92ed-80d7504bcb95 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:36.997497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf64cc-571b-4ca4-92ed-80d7504bcb95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:36.997807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf64cc-571b-4ca4-92ed-80d7504bcb95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:36.999215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acf64cc-571b-4ca4-92ed-80d7504bcb95 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:37.007066 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 563/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:43:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:37.014024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a9ddb-86dd-4a06-ba2b-e4a31b16481b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:37.120498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8a9ddb-86dd-4a06-ba2b-e4a31b16481b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:37.145304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:37.154001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16a73f-240e-452c-9f34-44f4d6282146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:37.155999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16a73f-240e-452c-9f34-44f4d6282146 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:37.155999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16a73f-240e-452c-9f34-44f4d6282146 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:37.155999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd16a73f-240e-452c-9f34-44f4d6282146 None admin] PATH_INFO: `/v3/users/2ac5785a0b1449b297122b0c27afadc7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:37.156619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16a73f-240e-452c-9f34-44f4d6282146 None admin] RBAC: Authorizing `identity:get_user(user_id=2ac5785a0b1449b297122b0c27afadc7)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:37.158068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd16a73f-240e-452c-9f34-44f4d6282146 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:37.160208 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 588/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:43:37 2022] GET /identity/v3/users/2ac5785a0b1449b297122b0c27afadc7 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:37.166025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf64cc-571b-4ca4-92ed-80d7504bcb95 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:37.262278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acf64cc-571b-4ca4-92ed-80d7504bcb95 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:37.275879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:37.294512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7684c-06d6-4a02-901a-e18b875c23b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:37.295354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7684c-06d6-4a02-901a-e18b875c23b9 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:37.295632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7684c-06d6-4a02-901a-e18b875c23b9 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:37.295905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7684c-06d6-4a02-901a-e18b875c23b9 None admin] PATH_INFO: `/v3/projects/e39b7ce2b17c49a5b2e627337e1d09ad` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:37.297060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7684c-06d6-4a02-901a-e18b875c23b9 None admin] RBAC: Authorizing `identity:get_project(project_id=e39b7ce2b17c49a5b2e627337e1d09ad)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:37.298511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7684c-06d6-4a02-901a-e18b875c23b9 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:37.300231 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 564/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:43:37 2022] GET /identity/v3/projects/e39b7ce2b17c49a5b2e627337e1d09ad => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:37.306423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16a73f-240e-452c-9f34-44f4d6282146 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:37.403788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd16a73f-240e-452c-9f34-44f4d6282146 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:37.435890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ec49b-6779-4ef8-b4ee-2b506894d37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:37.436759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ec49b-6779-4ef8-b4ee-2b506894d37c None admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:37.437042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ec49b-6779-4ef8-b4ee-2b506894d37c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:37.437321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ec49b-6779-4ef8-b4ee-2b506894d37c None admin] PATH_INFO: `/v3/projects/e39b7ce2b17c49a5b2e627337e1d09ad/users/2ac5785a0b1449b297122b0c27afadc7/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:37.439303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ec49b-6779-4ef8-b4ee-2b506894d37c None admin] RBAC: Authorizing `identity:create_grant(project_id=e39b7ce2b17c49a5b2e627337e1d09ad, user_id=2ac5785a0b1449b297122b0c27afadc7, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:37.441268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ec49b-6779-4ef8-b4ee-2b506894d37c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:37.506290 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 589/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:43:37 2022] PUT /identity/v3/projects/e39b7ce2b17c49a5b2e627337e1d09ad/users/2ac5785a0b1449b297122b0c27afadc7/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:43:39.733015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18a812-bbc8-4504-8ec3-652b616555ec None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:39.733574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18a812-bbc8-4504-8ec3-652b616555ec None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:39.733574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18a812-bbc8-4504-8ec3-652b616555ec None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:39.734983 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 565/1154] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:39.740224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4eed4-c9c6-4c2a-baec-06b011c3a563 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:39.740636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4eed4-c9c6-4c2a-baec-06b011c3a563 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:39.740774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4eed4-c9c6-4c2a-baec-06b011c3a563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:39.757813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ada4eed4-c9c6-4c2a-baec-06b011c3a563 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:39.759029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ada4eed4-c9c6-4c2a-baec-06b011c3a563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:39.807815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:39.823391 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 590/1155] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:43:39.850458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18a812-bbc8-4504-8ec3-652b616555ec None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:39.948714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18a812-bbc8-4504-8ec3-652b616555ec None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:39.981170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fbb38-40dc-4c80-83e4-93103fc08003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:39.981977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fbb38-40dc-4c80-83e4-93103fc08003 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:39.982261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fbb38-40dc-4c80-83e4-93103fc08003 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:39.982535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fbb38-40dc-4c80-83e4-93103fc08003 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:39.992697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fbb38-40dc-4c80-83e4-93103fc08003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:43:40.000098 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-174fbb38-40dc-4c80-83e4-93103fc08003 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:43:40.000098 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:40.000098 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:40.000098 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:40.000098 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:40.000098 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:40.000098 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:40.000098 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:40.000098 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:40.000098 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:40.000098 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:40.000098 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:40.000098 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:43:40.000098 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:43:40.000098 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:43:40.000098 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:43:40.001182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:43:40.001182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:43:40.001182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:43:40.001182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:43:40.001182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:40.001182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:40.001182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:40.001182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:40.001182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:40.001182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:40.001182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:40.001182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:43:40.001182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:40.001182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:40.001182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:40.001182 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:40.002101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:40.002101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:40.002101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:40.002101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:40.002101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:43:40.002101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:43:40.002101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:43:40.002101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:43:40.002101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:43:40.002101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:43:40.002101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:43:40.002101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:43:40.002101 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 566/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:43:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:43:40.007782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4eed4-c9c6-4c2a-baec-06b011c3a563 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:40.108527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4eed4-c9c6-4c2a-baec-06b011c3a563 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:40.141746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5ccf3-08df-4ccc-a7f2-8d04d88d738d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:40.142433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5ccf3-08df-4ccc-a7f2-8d04d88d738d None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:40.142717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5ccf3-08df-4ccc-a7f2-8d04d88d738d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:40.142995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5ccf3-08df-4ccc-a7f2-8d04d88d738d None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:40.144347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5ccf3-08df-4ccc-a7f2-8d04d88d738d None admin] RBAC: Adding query filter params (name=admin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:40.144650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5ccf3-08df-4ccc-a7f2-8d04d88d738d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:40.146053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5ccf3-08df-4ccc-a7f2-8d04d88d738d None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:40.153978 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 591/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:43:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:40.159786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fbb38-40dc-4c80-83e4-93103fc08003 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:40.201995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:40.256111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fbb38-40dc-4c80-83e4-93103fc08003 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:40.288553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05405494-acc6-48ae-a2cf-3945ff068a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:40.289401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05405494-acc6-48ae-a2cf-3945ff068a65 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:40.289681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05405494-acc6-48ae-a2cf-3945ff068a65 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:40.289985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05405494-acc6-48ae-a2cf-3945ff068a65 None admin] PATH_INFO: `/v3/users/2ac5785a0b1449b297122b0c27afadc7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:40.291152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05405494-acc6-48ae-a2cf-3945ff068a65 None admin] RBAC: Authorizing `identity:get_user(user_id=2ac5785a0b1449b297122b0c27afadc7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:40.292594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05405494-acc6-48ae-a2cf-3945ff068a65 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:40.294211 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 567/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:43:40 2022] GET /identity/v3/users/2ac5785a0b1449b297122b0c27afadc7 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:43:40.299936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5ccf3-08df-4ccc-a7f2-8d04d88d738d None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:40.348160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:40.397605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5ccf3-08df-4ccc-a7f2-8d04d88d738d None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:40.429622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40a1ec-9296-4d30-b7c5-3201c049ed85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:40.430283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40a1ec-9296-4d30-b7c5-3201c049ed85 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:40.430569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40a1ec-9296-4d30-b7c5-3201c049ed85 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:40.430847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40a1ec-9296-4d30-b7c5-3201c049ed85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:40.495221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40a1ec-9296-4d30-b7c5-3201c049ed85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:40.496690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40a1ec-9296-4d30-b7c5-3201c049ed85 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:40.542047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:40.591860 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 592/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:43:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 20:43:40.597827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05405494-acc6-48ae-a2cf-3945ff068a65 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:40.694792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05405494-acc6-48ae-a2cf-3945ff068a65 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:40.704936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:40.727642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db3582-fdab-4c56-9fa2-74c0fabd4aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:40.728490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db3582-fdab-4c56-9fa2-74c0fabd4aad None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:40.728769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db3582-fdab-4c56-9fa2-74c0fabd4aad None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:40.729040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db3582-fdab-4c56-9fa2-74c0fabd4aad None admin] PATH_INFO: `/v3/projects/e39b7ce2b17c49a5b2e627337e1d09ad` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:40.730238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db3582-fdab-4c56-9fa2-74c0fabd4aad None admin] RBAC: Authorizing `identity:get_project(project_id=e39b7ce2b17c49a5b2e627337e1d09ad)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:40.731671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db3582-fdab-4c56-9fa2-74c0fabd4aad None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:40.733397 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 568/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:43:40 2022] GET /identity/v3/projects/e39b7ce2b17c49a5b2e627337e1d09ad => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:40.740141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40a1ec-9296-4d30-b7c5-3201c049ed85 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:40.834934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40a1ec-9296-4d30-b7c5-3201c049ed85 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:40.866633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877c96c-1428-4607-b436-4c93c2b46436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:40.867262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877c96c-1428-4607-b436-4c93c2b46436 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:40.867543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877c96c-1428-4607-b436-4c93c2b46436 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:40.867819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877c96c-1428-4607-b436-4c93c2b46436 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:40.868642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877c96c-1428-4607-b436-4c93c2b46436 None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=e39b7ce2b17c49a5b2e627337e1d09ad, user.id=2ac5785a0b1449b297122b0c27afadc7) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:40.868953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877c96c-1428-4607-b436-4c93c2b46436 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:40.870437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877c96c-1428-4607-b436-4c93c2b46436 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:40.876037 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 593/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:43:40 2022] GET /identity/v3/role_assignments?user.id=2ac5785a0b1449b297122b0c27afadc7&scope.project.id=e39b7ce2b17c49a5b2e627337e1d09ad&role.id=7805b93b32f1472f8365936d4d57d49a => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:43.140510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e337793-4769-4f8a-969c-7447ea3b30b2 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:43.141044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e337793-4769-4f8a-969c-7447ea3b30b2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:43.141044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e337793-4769-4f8a-969c-7447ea3b30b2 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:43.142486 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 569/1162] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:43.147726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec845073-75cd-4519-9097-7200ae2788c4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:43.148136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec845073-75cd-4519-9097-7200ae2788c4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:43.148277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec845073-75cd-4519-9097-7200ae2788c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:43.165058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ec845073-75cd-4519-9097-7200ae2788c4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:43.166272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ec845073-75cd-4519-9097-7200ae2788c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:43.173247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:43.230931 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 594/1163] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:43:43.255175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e337793-4769-4f8a-969c-7447ea3b30b2 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:43.352687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e337793-4769-4f8a-969c-7447ea3b30b2 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:43.385552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cc996-2609-4689-b960-26fd5145623a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:43.386206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cc996-2609-4689-b960-26fd5145623a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:43.386487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cc996-2609-4689-b960-26fd5145623a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:43.386760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0cc996-2609-4689-b960-26fd5145623a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:43.430172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:43.451422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0cc996-2609-4689-b960-26fd5145623a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:43.452919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0cc996-2609-4689-b960-26fd5145623a None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:43.546101 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 570/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:43:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 20:43:43.554069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec845073-75cd-4519-9097-7200ae2788c4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:43.648602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec845073-75cd-4519-9097-7200ae2788c4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:43.680571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3b1ae-40f4-453d-98f8-5606ff2bc267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:43.681353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3b1ae-40f4-453d-98f8-5606ff2bc267 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:43.681633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3b1ae-40f4-453d-98f8-5606ff2bc267 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:43.681933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3b1ae-40f4-453d-98f8-5606ff2bc267 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:43.688396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3b1ae-40f4-453d-98f8-5606ff2bc267 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:43.689859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3b1ae-40f4-453d-98f8-5606ff2bc267 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:43.704153 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-1cd3b1ae-40f4-453d-98f8-5606ff2bc267 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 28 20:43:43.704153 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:43.704153 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 20:43:43.704153 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 20:43:43.704153 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:43:43.704153 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:43:43.704153 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:43:43.704153 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:43:43.704153 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 28 20:43:43.704153 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:43.704153 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:43:43.704153 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:43.704153 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:43.704153 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:43.704153 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:43.704153 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:43.704153 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:43.705355 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:43.705355 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:43.705355 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:43.705355 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:43.705355 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:43.705355 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:43.705355 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:43:43.705355 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:43:43.705355 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:43:43.705355 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:43:43.705355 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:43.705355 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:43.705355 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:43.705355 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:43.705355 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:43.705355 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:43.706606 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:43.706606 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:43:43.706606 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:43.706606 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:43.706606 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:43.706606 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:43.706606 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:43.706606 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:43.706606 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:43.706606 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:43.706606 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 20:43:43.706606 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:43:43.706606 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 28 20:43:43.706606 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:43:43.706606 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 595/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 20:43:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:43:43.711182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cc996-2609-4689-b960-26fd5145623a None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:43.786111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:43.806636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0cc996-2609-4689-b960-26fd5145623a None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:43.838900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d56ae7-4c2f-4949-b8ce-ed38b555b3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:43.839499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d56ae7-4c2f-4949-b8ce-ed38b555b3fe None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:43.839785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d56ae7-4c2f-4949-b8ce-ed38b555b3fe None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:43.840058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d56ae7-4c2f-4949-b8ce-ed38b555b3fe None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:43.840766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d56ae7-4c2f-4949-b8ce-ed38b555b3fe None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:43.841068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d56ae7-4c2f-4949-b8ce-ed38b555b3fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:43.842464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d56ae7-4c2f-4949-b8ce-ed38b555b3fe None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:43.849801 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 571/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Sat May 28 20:43:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:46.085257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc0295-c8c8-4517-9867-cc0283c10116 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:46.085701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc0295-c8c8-4517-9867-cc0283c10116 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:46.085817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc0295-c8c8-4517-9867-cc0283c10116 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:46.087244 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 596/1167] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:46.092503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b4487-b2bb-4959-b0e2-44c3a3e2de51 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:46.092903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b4487-b2bb-4959-b0e2-44c3a3e2de51 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:46.093057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b4487-b2bb-4959-b0e2-44c3a3e2de51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:46.109277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-942b4487-b2bb-4959-b0e2-44c3a3e2de51 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:46.110478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-942b4487-b2bb-4959-b0e2-44c3a3e2de51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:46.174830 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 572/1168] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:43:46.198974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc0295-c8c8-4517-9867-cc0283c10116 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:46.202518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:46.299932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbc0295-c8c8-4517-9867-cc0283c10116 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:46.332973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859e9cc-346d-487b-b3cf-2851fcda84c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:46.333564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859e9cc-346d-487b-b3cf-2851fcda84c0 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:46.333867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859e9cc-346d-487b-b3cf-2851fcda84c0 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:46.334164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859e9cc-346d-487b-b3cf-2851fcda84c0 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:46.334943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859e9cc-346d-487b-b3cf-2851fcda84c0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:46.336302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859e9cc-346d-487b-b3cf-2851fcda84c0 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:46.427824 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 597/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Sat May 28 20:43:46 2022] POST /identity/v3/domains => generated 267 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:43:48.707800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ded64-975f-4a43-b7d7-27e5509162f0 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:48.708366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ded64-975f-4a43-b7d7-27e5509162f0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:48.708366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ded64-975f-4a43-b7d7-27e5509162f0 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:48.709707 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 573/1170] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:48.715034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0c566-6949-4a62-b39b-9a57fc97e252 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:48.715448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0c566-6949-4a62-b39b-9a57fc97e252 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:48.715599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0c566-6949-4a62-b39b-9a57fc97e252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:48.732643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fdd0c566-6949-4a62-b39b-9a57fc97e252 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:48.733864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fdd0c566-6949-4a62-b39b-9a57fc97e252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:48.788768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:48.798929 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 598/1171] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:43:48.824287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ded64-975f-4a43-b7d7-27e5509162f0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:48.892504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:48.922197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ded64-975f-4a43-b7d7-27e5509162f0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:48.954982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ed5ea-b93e-457e-b8f7-f5cd3ac0ec22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:48.955834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ed5ea-b93e-457e-b8f7-f5cd3ac0ec22 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:48.956115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ed5ea-b93e-457e-b8f7-f5cd3ac0ec22 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:48.956963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ed5ea-b93e-457e-b8f7-f5cd3ac0ec22 None admin] PATH_INFO: `/v3/domains/b30c6626b90a4d63a0773c5e361500bd` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:48.967094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ed5ea-b93e-457e-b8f7-f5cd3ac0ec22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b30c6626b90a4d63a0773c5e361500bd)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:48.968485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ed5ea-b93e-457e-b8f7-f5cd3ac0ec22 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:48.970148 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 574/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Sat May 28 20:43:48 2022] GET /identity/v3/domains/b30c6626b90a4d63a0773c5e361500bd => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:48.975952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0c566-6949-4a62-b39b-9a57fc97e252 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:49.071499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0c566-6949-4a62-b39b-9a57fc97e252 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:49.104619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0da35-37d4-4bc9-b859-046443b72a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:49.105273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0da35-37d4-4bc9-b859-046443b72a8b None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:49.105557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0da35-37d4-4bc9-b859-046443b72a8b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:49.105863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0da35-37d4-4bc9-b859-046443b72a8b None admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:49.106562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0da35-37d4-4bc9-b859-046443b72a8b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:49.107992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0da35-37d4-4bc9-b859-046443b72a8b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:49.181561 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 599/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 20:43:48 2022] POST /identity/v3/projects => generated 361 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:43:51.457092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7856-428b-44b5-8821-4b9581260773 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:51.457601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7856-428b-44b5-8821-4b9581260773 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:51.457601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e7856-428b-44b5-8821-4b9581260773 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:51.458970 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 575/1174] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:51.464247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72fcdd-40ed-40e1-9f8a-b1d6b4e9f4c1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:51.464668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72fcdd-40ed-40e1-9f8a-b1d6b4e9f4c1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:51.464808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72fcdd-40ed-40e1-9f8a-b1d6b4e9f4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:51.481724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fc72fcdd-40ed-40e1-9f8a-b1d6b4e9f4c1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:51.482952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fc72fcdd-40ed-40e1-9f8a-b1d6b4e9f4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:51.521541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:51.547763 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 600/1175] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:43:51.574596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7856-428b-44b5-8821-4b9581260773 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:51.674536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148e7856-428b-44b5-8821-4b9581260773 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:51.702081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:51.707367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702f238-6f88-4cc0-b184-2a66e733520b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:51.708163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702f238-6f88-4cc0-b184-2a66e733520b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:51.708447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702f238-6f88-4cc0-b184-2a66e733520b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:51.708727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702f238-6f88-4cc0-b184-2a66e733520b None admin] PATH_INFO: `/v3/domains/b30c6626b90a4d63a0773c5e361500bd` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:51.709932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702f238-6f88-4cc0-b184-2a66e733520b None admin] RBAC: Authorizing `identity:get_domain(domain_id=b30c6626b90a4d63a0773c5e361500bd)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:51.711368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702f238-6f88-4cc0-b184-2a66e733520b None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:51.713110 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 576/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Sat May 28 20:43:51 2022] GET /identity/v3/domains/b30c6626b90a4d63a0773c5e361500bd => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:51.718776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72fcdd-40ed-40e1-9f8a-b1d6b4e9f4c1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:51.814588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72fcdd-40ed-40e1-9f8a-b1d6b4e9f4c1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:51.848625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e26e2-1d7e-41e9-8f57-53b02405c5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:51.849285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e26e2-1d7e-41e9-8f57-53b02405c5cd None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:51.849568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e26e2-1d7e-41e9-8f57-53b02405c5cd None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:51.849863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e26e2-1d7e-41e9-8f57-53b02405c5cd None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:51.850581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e26e2-1d7e-41e9-8f57-53b02405c5cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:51.852010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e26e2-1d7e-41e9-8f57-53b02405c5cd None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:51.923286 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 601/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:43:51 2022] POST /identity/v3/users => generated 311 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:43:54.214588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdce77b-865f-41ff-a26b-aacd0b3e9e31 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:54.215128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdce77b-865f-41ff-a26b-aacd0b3e9e31 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:54.215128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdce77b-865f-41ff-a26b-aacd0b3e9e31 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:54.216519 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 577/1178] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:54.221649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274d3d0-6ea4-4c57-88bc-739f7d488fae None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:54.222128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274d3d0-6ea4-4c57-88bc-739f7d488fae None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:54.222282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274d3d0-6ea4-4c57-88bc-739f7d488fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:54.238878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a274d3d0-6ea4-4c57-88bc-739f7d488fae None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:54.240054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a274d3d0-6ea4-4c57-88bc-739f7d488fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:54.262917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:54.304909 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 602/1179] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 28 20:43:54.331102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdce77b-865f-41ff-a26b-aacd0b3e9e31 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:54.431513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdce77b-865f-41ff-a26b-aacd0b3e9e31 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:54.463866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab5a24-73ec-47f8-ba0a-ab3cfd8c08f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:54.464653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab5a24-73ec-47f8-ba0a-ab3cfd8c08f3 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:54.464937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab5a24-73ec-47f8-ba0a-ab3cfd8c08f3 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:54.465239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ab5a24-73ec-47f8-ba0a-ab3cfd8c08f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:54.476066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ab5a24-73ec-47f8-ba0a-ab3cfd8c08f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:43:54.483650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-73ab5a24-73ec-47f8-ba0a-ab3cfd8c08f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:43:54.483650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:54.483650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:54.483650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:54.483650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:54.483650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:54.483650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:54.483650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:54.483650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:54.483650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:54.483650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:54.483650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:54.483650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:43:54.483650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:43:54.483650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:43:54.483650 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:43:54.484739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:43:54.484739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:43:54.484739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:43:54.484739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:43:54.484739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:54.484739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:54.484739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:54.484739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:54.484739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:54.484739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:54.484739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:54.484739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:43:54.484739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:54.484739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:54.484739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:54.484739 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:54.485791 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:54.485791 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:54.485791 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:54.485791 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:54.485791 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:43:54.485791 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:43:54.485791 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:43:54.485791 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:43:54.485791 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:43:54.485791 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:43:54.485791 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:43:54.485791 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:43:54.485791 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 578/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:43:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:43:54.491643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274d3d0-6ea4-4c57-88bc-739f7d488fae None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:54.587654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274d3d0-6ea4-4c57-88bc-739f7d488fae None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:54.621010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee23be-8239-4ac7-b816-bcea5207ed36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:54.621671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee23be-8239-4ac7-b816-bcea5207ed36 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:54.621991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee23be-8239-4ac7-b816-bcea5207ed36 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:54.622273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee23be-8239-4ac7-b816-bcea5207ed36 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:54.623019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee23be-8239-4ac7-b816-bcea5207ed36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:54.623326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee23be-8239-4ac7-b816-bcea5207ed36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:54.624693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee23be-8239-4ac7-b816-bcea5207ed36 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:54.633010 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 603/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:43:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:54.638752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab5a24-73ec-47f8-ba0a-ab3cfd8c08f3 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:54.697431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:54.736758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ab5a24-73ec-47f8-ba0a-ab3cfd8c08f3 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:54.770815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b8233-317a-4550-84b2-251f419269ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:54.771679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b8233-317a-4550-84b2-251f419269ba None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:54.771964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b8233-317a-4550-84b2-251f419269ba None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:54.772240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b8233-317a-4550-84b2-251f419269ba None admin] PATH_INFO: `/v3/users/6c6fd1f1d93a4bb5bf9478ded7d95202` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:54.785005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b8233-317a-4550-84b2-251f419269ba None admin] RBAC: Authorizing `identity:get_user(user_id=6c6fd1f1d93a4bb5bf9478ded7d95202)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:54.786503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b8233-317a-4550-84b2-251f419269ba None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:54.788137 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 579/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:43:54 2022] GET /identity/v3/users/6c6fd1f1d93a4bb5bf9478ded7d95202 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:54.793922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee23be-8239-4ac7-b816-bcea5207ed36 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:54.818841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:54.891523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee23be-8239-4ac7-b816-bcea5207ed36 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:54.923336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95db4b-da52-4988-8567-94f04dcfe081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:54.923980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95db4b-da52-4988-8567-94f04dcfe081 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:54.924297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95db4b-da52-4988-8567-94f04dcfe081 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:54.924581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d95db4b-da52-4988-8567-94f04dcfe081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:54.990469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d95db4b-da52-4988-8567-94f04dcfe081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:54.991913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d95db4b-da52-4988-8567-94f04dcfe081 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:55.013631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:55.086887 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 604/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:43:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 20:43:55.092771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b8233-317a-4550-84b2-251f419269ba None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:55.189545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b8233-317a-4550-84b2-251f419269ba None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:55.208070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:55.224217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c8f57-0fd1-45be-a101-27ba66c56710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:55.225098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c8f57-0fd1-45be-a101-27ba66c56710 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:55.225381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c8f57-0fd1-45be-a101-27ba66c56710 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:55.225656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c8f57-0fd1-45be-a101-27ba66c56710 None admin] PATH_INFO: `/v3/projects/603527c5e7ba440eb2110f420de922fc` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:55.226871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c8f57-0fd1-45be-a101-27ba66c56710 None admin] RBAC: Authorizing `identity:get_project(project_id=603527c5e7ba440eb2110f420de922fc)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:55.228312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c8f57-0fd1-45be-a101-27ba66c56710 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:55.230182 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 580/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:43:55 2022] GET /identity/v3/projects/603527c5e7ba440eb2110f420de922fc => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:55.236400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95db4b-da52-4988-8567-94f04dcfe081 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:55.335359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d95db4b-da52-4988-8567-94f04dcfe081 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:55.357034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:55.368633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718d6d1-0fdc-442f-ae8e-78b72e122908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:55.369284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718d6d1-0fdc-442f-ae8e-78b72e122908 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:55.369569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718d6d1-0fdc-442f-ae8e-78b72e122908 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:55.369863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718d6d1-0fdc-442f-ae8e-78b72e122908 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:55.370712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718d6d1-0fdc-442f-ae8e-78b72e122908 None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=603527c5e7ba440eb2110f420de922fc, user.id=6c6fd1f1d93a4bb5bf9478ded7d95202) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:55.371027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718d6d1-0fdc-442f-ae8e-78b72e122908 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:55.372454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718d6d1-0fdc-442f-ae8e-78b72e122908 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:55.377780 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 605/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:43:55 2022] GET /identity/v3/role_assignments?user.id=6c6fd1f1d93a4bb5bf9478ded7d95202&scope.project.id=603527c5e7ba440eb2110f420de922fc&role.id=7805b93b32f1472f8365936d4d57d49a => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:57.592909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115d4b2-dafa-4f69-a842-3ddf7d6c79fa None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:57.593440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115d4b2-dafa-4f69-a842-3ddf7d6c79fa None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:57.593543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2115d4b2-dafa-4f69-a842-3ddf7d6c79fa None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:57.594867 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 581/1186] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:43:57.600840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5dfc8-6f31-48bc-8510-bfe8d86f91db None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:57.601282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5dfc8-6f31-48bc-8510-bfe8d86f91db None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:57.601413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5dfc8-6f31-48bc-8510-bfe8d86f91db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:57.619278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5da5dfc8-6f31-48bc-8510-bfe8d86f91db None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:43:57.620454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5da5dfc8-6f31-48bc-8510-bfe8d86f91db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:43:57.684349 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 606/1187] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:43:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:43:57.707881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115d4b2-dafa-4f69-a842-3ddf7d6c79fa None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:57.806072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2115d4b2-dafa-4f69-a842-3ddf7d6c79fa None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:57.838900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f5f55-fe3a-4b04-ab5f-b3b921ed41d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:57.839673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f5f55-fe3a-4b04-ab5f-b3b921ed41d1 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:57.839959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f5f55-fe3a-4b04-ab5f-b3b921ed41d1 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:57.840238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f5f55-fe3a-4b04-ab5f-b3b921ed41d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:57.850153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f5f55-fe3a-4b04-ab5f-b3b921ed41d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:43:57.857594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-941f5f55-fe3a-4b04-ab5f-b3b921ed41d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:43:57.857594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:43:57.857594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:43:57.857594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:43:57.857594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:43:57.857594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:43:57.857594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:43:57.857594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:43:57.857594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:43:57.857594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:43:57.857594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:43:57.857594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:43:57.857594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:43:57.857594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:43:57.857594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:43:57.857594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:43:57.858741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:43:57.858741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:43:57.858741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:43:57.858741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:43:57.858741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:43:57.858741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:43:57.858741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:43:57.858741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:43:57.858741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:43:57.858741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:43:57.858741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:43:57.858741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:43:57.858741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:43:57.858741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:43:57.858741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:43:57.858741 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:43:57.859919 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:43:57.859919 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:43:57.859919 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:43:57.859919 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:43:57.859919 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:43:57.859919 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:43:57.859919 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:43:57.859919 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:43:57.859919 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:43:57.859919 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:43:57.859919 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:43:57.859919 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:43:57.859919 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 582/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:43:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:43:57.865010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5dfc8-6f31-48bc-8510-bfe8d86f91db None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:57.960064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5dfc8-6f31-48bc-8510-bfe8d86f91db None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:57.976149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:57.993395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f12d0f-8c66-4653-8f45-bedabf821be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:57.994076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f12d0f-8c66-4653-8f45-bedabf821be3 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:57.994365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f12d0f-8c66-4653-8f45-bedabf821be3 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:57.994636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f12d0f-8c66-4653-8f45-bedabf821be3 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:57.995379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f12d0f-8c66-4653-8f45-bedabf821be3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:43:57.995685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f12d0f-8c66-4653-8f45-bedabf821be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:57.997043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f12d0f-8c66-4653-8f45-bedabf821be3 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:58.004679 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 607/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:43:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:58.011618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f5f55-fe3a-4b04-ab5f-b3b921ed41d1 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:58.057236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:58.108855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f5f55-fe3a-4b04-ab5f-b3b921ed41d1 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:58.141036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9af172-d8b8-45b2-a1c7-a37f4c2fd9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:58.141924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9af172-d8b8-45b2-a1c7-a37f4c2fd9d3 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:58.142207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9af172-d8b8-45b2-a1c7-a37f4c2fd9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:58.142485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9af172-d8b8-45b2-a1c7-a37f4c2fd9d3 None admin] PATH_INFO: `/v3/users/6c6fd1f1d93a4bb5bf9478ded7d95202` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:58.143656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9af172-d8b8-45b2-a1c7-a37f4c2fd9d3 None admin] RBAC: Authorizing `identity:get_user(user_id=6c6fd1f1d93a4bb5bf9478ded7d95202)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:58.145123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9af172-d8b8-45b2-a1c7-a37f4c2fd9d3 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:58.146757 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 583/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:43:58 2022] GET /identity/v3/users/6c6fd1f1d93a4bb5bf9478ded7d95202 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:58.152428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f12d0f-8c66-4653-8f45-bedabf821be3 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:58.249038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f12d0f-8c66-4653-8f45-bedabf821be3 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:58.280915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13f00f-5319-4909-bef8-31e570ec9a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:58.281763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13f00f-5319-4909-bef8-31e570ec9a88 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:58.282080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13f00f-5319-4909-bef8-31e570ec9a88 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:58.282361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13f00f-5319-4909-bef8-31e570ec9a88 None admin] PATH_INFO: `/v3/projects/603527c5e7ba440eb2110f420de922fc` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:58.283539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13f00f-5319-4909-bef8-31e570ec9a88 None admin] RBAC: Authorizing `identity:get_project(project_id=603527c5e7ba440eb2110f420de922fc)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:58.285494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13f00f-5319-4909-bef8-31e570ec9a88 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:58.287184 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 608/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:43:58 2022] GET /identity/v3/projects/603527c5e7ba440eb2110f420de922fc => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:43:58.292716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9af172-d8b8-45b2-a1c7-a37f4c2fd9d3 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:43:58.388421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9af172-d8b8-45b2-a1c7-a37f4c2fd9d3 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:43:58.402044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:43:58.421411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7827838-e006-467d-b7ed-48e479c1f835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:43:58.422315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7827838-e006-467d-b7ed-48e479c1f835 None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:43:58.422598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7827838-e006-467d-b7ed-48e479c1f835 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:43:58.422873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7827838-e006-467d-b7ed-48e479c1f835 None admin] PATH_INFO: `/v3/projects/603527c5e7ba440eb2110f420de922fc/users/6c6fd1f1d93a4bb5bf9478ded7d95202/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:43:58.424810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7827838-e006-467d-b7ed-48e479c1f835 None admin] RBAC: Authorizing `identity:create_grant(project_id=603527c5e7ba440eb2110f420de922fc, user_id=6c6fd1f1d93a4bb5bf9478ded7d95202, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:43:58.426297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7827838-e006-467d-b7ed-48e479c1f835 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:43:58.493514 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 584/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:43:58 2022] PUT /identity/v3/projects/603527c5e7ba440eb2110f420de922fc/users/6c6fd1f1d93a4bb5bf9478ded7d95202/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:44:00.708585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e7843-df57-49f7-8865-48f529910816 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:44:00.709108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e7843-df57-49f7-8865-48f529910816 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:44:00.709202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51e7843-df57-49f7-8865-48f529910816 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:44:00.710550 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 609/1193] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:44:00.715764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0049bd1-1b02-4a28-970a-b0430e8997f3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:44:00.716154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0049bd1-1b02-4a28-970a-b0430e8997f3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:44:00.716317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0049bd1-1b02-4a28-970a-b0430e8997f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:44:00.732622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b0049bd1-1b02-4a28-970a-b0430e8997f3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:44:00.733812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b0049bd1-1b02-4a28-970a-b0430e8997f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:44:00.799623 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 585/1194] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:44:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:44:00.826249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e7843-df57-49f7-8865-48f529910816 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:44:00.873142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:44:00.925974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51e7843-df57-49f7-8865-48f529910816 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:44:00.958646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce189c-6d7d-4a97-8cad-0306288be232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:44:00.959364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce189c-6d7d-4a97-8cad-0306288be232 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:44:00.959644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce189c-6d7d-4a97-8cad-0306288be232 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:44:00.959919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce189c-6d7d-4a97-8cad-0306288be232 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:44:00.970110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce189c-6d7d-4a97-8cad-0306288be232 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:44:00.978209 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-83ce189c-6d7d-4a97-8cad-0306288be232 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:44:00.978209 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:44:00.978209 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:44:00.978209 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:44:00.978209 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:44:00.978209 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:44:00.978209 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:44:00.978209 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:44:00.978209 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:44:00.978209 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:44:00.978209 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:44:00.978209 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:44:00.978209 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:44:00.978209 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:44:00.978209 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:44:00.978209 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:44:00.979143 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:44:00.979143 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:44:00.979143 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:44:00.979143 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:44:00.979143 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:44:00.979143 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:44:00.979143 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:44:00.979143 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:44:00.979143 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:44:00.979143 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:44:00.979143 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:44:00.979143 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:44:00.979143 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:44:00.979143 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:44:00.979143 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:44:00.979143 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:44:00.979998 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:44:00.979998 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:44:00.979998 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:44:00.979998 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:44:00.979998 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:44:00.979998 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:44:00.979998 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:44:00.979998 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:44:00.979998 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:44:00.979998 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:44:00.979998 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:44:00.979998 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:44:00.979998 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 610/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:44:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:44:00.985467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0049bd1-1b02-4a28-970a-b0430e8997f3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:44:01.081895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0049bd1-1b02-4a28-970a-b0430e8997f3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:44:01.084163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:44:01.115231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53469e5d-cb53-4c3e-b04c-5bc8282adffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:44:01.115885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53469e5d-cb53-4c3e-b04c-5bc8282adffa None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:44:01.116164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53469e5d-cb53-4c3e-b04c-5bc8282adffa None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:44:01.116437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53469e5d-cb53-4c3e-b04c-5bc8282adffa None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:44:01.117177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53469e5d-cb53-4c3e-b04c-5bc8282adffa None admin] RBAC: Adding query filter params (name=admin) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:44:01.117482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53469e5d-cb53-4c3e-b04c-5bc8282adffa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:44:01.118878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53469e5d-cb53-4c3e-b04c-5bc8282adffa None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:44:01.126618 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 586/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:44:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:44:01.132390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce189c-6d7d-4a97-8cad-0306288be232 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:44:01.229589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce189c-6d7d-4a97-8cad-0306288be232 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:44:01.236593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:44:01.263183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758b970-23a6-4efd-9663-87c58816caf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:44:01.264053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758b970-23a6-4efd-9663-87c58816caf0 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:44:01.264337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758b970-23a6-4efd-9663-87c58816caf0 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:44:01.264873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758b970-23a6-4efd-9663-87c58816caf0 None admin] PATH_INFO: `/v3/users/6c6fd1f1d93a4bb5bf9478ded7d95202` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:44:01.266277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758b970-23a6-4efd-9663-87c58816caf0 None admin] RBAC: Authorizing `identity:get_user(user_id=6c6fd1f1d93a4bb5bf9478ded7d95202)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:44:01.268146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0758b970-23a6-4efd-9663-87c58816caf0 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:44:01.269792 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 611/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Sat May 28 20:44:01 2022] GET /identity/v3/users/6c6fd1f1d93a4bb5bf9478ded7d95202 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:44:01.275626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53469e5d-cb53-4c3e-b04c-5bc8282adffa None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:44:01.374516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53469e5d-cb53-4c3e-b04c-5bc8282adffa None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:44:01.406980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd28dd4-35d5-444d-8bba-9f8bdb9af0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:44:01.407612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd28dd4-35d5-444d-8bba-9f8bdb9af0b4 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:44:01.407896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd28dd4-35d5-444d-8bba-9f8bdb9af0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:44:01.408173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd28dd4-35d5-444d-8bba-9f8bdb9af0b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:44:01.440500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:44:01.473905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd28dd4-35d5-444d-8bba-9f8bdb9af0b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:44:01.475349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd28dd4-35d5-444d-8bba-9f8bdb9af0b4 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:44:01.569062 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 587/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:44:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 20:44:01.574961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758b970-23a6-4efd-9663-87c58816caf0 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:44:01.686121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0758b970-23a6-4efd-9663-87c58816caf0 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:44:01.720979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe94d97-7edb-4497-b835-ece06cf1ebbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:44:01.721830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe94d97-7edb-4497-b835-ece06cf1ebbd None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:44:01.722166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe94d97-7edb-4497-b835-ece06cf1ebbd None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:44:01.722541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe94d97-7edb-4497-b835-ece06cf1ebbd None admin] PATH_INFO: `/v3/projects/603527c5e7ba440eb2110f420de922fc` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:44:01.723667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe94d97-7edb-4497-b835-ece06cf1ebbd None admin] RBAC: Authorizing `identity:get_project(project_id=603527c5e7ba440eb2110f420de922fc)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:44:01.725108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe94d97-7edb-4497-b835-ece06cf1ebbd None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:44:01.726830 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 612/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Sat May 28 20:44:01 2022] GET /identity/v3/projects/603527c5e7ba440eb2110f420de922fc => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:44:01.733006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd28dd4-35d5-444d-8bba-9f8bdb9af0b4 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:44:01.809103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:44:01.848741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd28dd4-35d5-444d-8bba-9f8bdb9af0b4 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:44:01.885774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c52f90-869e-4de3-93ab-f325bc7ab395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:44:01.886465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c52f90-869e-4de3-93ab-f325bc7ab395 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:44:01.886748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c52f90-869e-4de3-93ab-f325bc7ab395 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:44:01.887022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c52f90-869e-4de3-93ab-f325bc7ab395 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:44:01.887920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c52f90-869e-4de3-93ab-f325bc7ab395 None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=603527c5e7ba440eb2110f420de922fc, user.id=6c6fd1f1d93a4bb5bf9478ded7d95202) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:44:01.888230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c52f90-869e-4de3-93ab-f325bc7ab395 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:44:01.890000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c52f90-869e-4de3-93ab-f325bc7ab395 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:44:01.896819 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 588/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:44:01 2022] GET /identity/v3/role_assignments?user.id=6c6fd1f1d93a4bb5bf9478ded7d95202&scope.project.id=603527c5e7ba440eb2110f420de922fc&role.id=7805b93b32f1472f8365936d4d57d49a => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:44:11.716799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=79 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:44:11.893182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=35 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:17.857152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab37f7-fba5-410e-a02f-9f40f9c2de98 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:17.857749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab37f7-fba5-410e-a02f-9f40f9c2de98 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:17.857882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab37f7-fba5-410e-a02f-9f40f9c2de98 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:17.859233 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 613/1201] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:45:17.864678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff6f34-0189-4325-bcb7-d46b87cb23dd None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:17.873912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff6f34-0189-4325-bcb7-d46b87cb23dd None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:17.873912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff6f34-0189-4325-bcb7-d46b87cb23dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:17.881752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f4ff6f34-0189-4325-bcb7-d46b87cb23dd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:45:17.882949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f4ff6f34-0189-4325-bcb7-d46b87cb23dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:45:17.948163 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 589/1202] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:45:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:45:17.976510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab37f7-fba5-410e-a02f-9f40f9c2de98 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:18.078766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab37f7-fba5-410e-a02f-9f40f9c2de98 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:18.112232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7d791-cdb5-4d6f-a2d8-e005b9d6fd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:18.113042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7d791-cdb5-4d6f-a2d8-e005b9d6fd53 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:18.113318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7d791-cdb5-4d6f-a2d8-e005b9d6fd53 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:18.113591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7d791-cdb5-4d6f-a2d8-e005b9d6fd53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:18.123487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7d791-cdb5-4d6f-a2d8-e005b9d6fd53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:18.126951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7d791-cdb5-4d6f-a2d8-e005b9d6fd53 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:18.142472 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-c1c7d791-cdb5-4d6f-a2d8-e005b9d6fd53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:18.142472 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:18.142472 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:18.142472 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:18.142472 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:18.142472 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:18.142472 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:18.142472 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:18.142472 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:18.142472 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:18.142472 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:18.142472 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:18.142472 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:45:18.142472 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:45:18.142472 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:45:18.143596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:45:18.143596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:18.143596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:18.143596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:18.143596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:18.143596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:18.143596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:18.143596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:18.143596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:45:18.143596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:18.143596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:18.143596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:18.143596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:18.143596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:18.143596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:18.143596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:18.143596 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:18.144704 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:45:18.144704 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:45:18.144704 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:18.144704 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:45:18.144704 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 614/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:18.149586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff6f34-0189-4325-bcb7-d46b87cb23dd None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:18.246391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff6f34-0189-4325-bcb7-d46b87cb23dd None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:18.274413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:18.280819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937e633-121a-4fc6-a572-ba1b78675402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:18.281507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937e633-121a-4fc6-a572-ba1b78675402 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:18.281791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937e633-121a-4fc6-a572-ba1b78675402 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:18.282100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937e633-121a-4fc6-a572-ba1b78675402 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:18.282824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937e633-121a-4fc6-a572-ba1b78675402 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:18.283132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937e633-121a-4fc6-a572-ba1b78675402 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:18.284576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937e633-121a-4fc6-a572-ba1b78675402 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:18.295598 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 590/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:18.301192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7d791-cdb5-4d6f-a2d8-e005b9d6fd53 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:18.354980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:18.399710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7d791-cdb5-4d6f-a2d8-e005b9d6fd53 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:18.433010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179d242-9c7d-41b5-9ec8-56c9ffd5f35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:18.433663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179d242-9c7d-41b5-9ec8-56c9ffd5f35d None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:18.433981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179d242-9c7d-41b5-9ec8-56c9ffd5f35d None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:18.434253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179d242-9c7d-41b5-9ec8-56c9ffd5f35d None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:18.435053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179d242-9c7d-41b5-9ec8-56c9ffd5f35d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:18.436472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179d242-9c7d-41b5-9ec8-56c9ffd5f35d None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:18.521437 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 615/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Sat May 28 20:45:18 2022] POST /identity/v3/users => generated 251 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:45:20.803646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae60bc-2626-4cbf-b67a-4796de647bab None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:20.804197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae60bc-2626-4cbf-b67a-4796de647bab None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:20.804290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae60bc-2626-4cbf-b67a-4796de647bab None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:20.805608 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 591/1206] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:45:20.810925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9df50-21cc-4d8d-9b99-2a24d20f172d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:20.811304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9df50-21cc-4d8d-9b99-2a24d20f172d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:20.811479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f9df50-21cc-4d8d-9b99-2a24d20f172d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:20.828102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-50f9df50-21cc-4d8d-9b99-2a24d20f172d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:45:20.829289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-50f9df50-21cc-4d8d-9b99-2a24d20f172d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:45:20.895084 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 616/1207] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:45:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:45:20.921301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae60bc-2626-4cbf-b67a-4796de647bab None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:21.020628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae60bc-2626-4cbf-b67a-4796de647bab None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:21.053415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d4fb8-6b14-4ba5-9523-dd3221b4af09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:21.054292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d4fb8-6b14-4ba5-9523-dd3221b4af09 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:21.054523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d4fb8-6b14-4ba5-9523-dd3221b4af09 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:21.054793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d4fb8-6b14-4ba5-9523-dd3221b4af09 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:21.066032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d4fb8-6b14-4ba5-9523-dd3221b4af09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:45:21.074258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-461d4fb8-6b14-4ba5-9523-dd3221b4af09 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:45:21.074258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:21.074258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:21.074258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:21.074258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:21.074258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:21.074258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:21.074258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:21.074258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:21.074258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:21.074258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:21.074258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:21.074258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:45:21.074258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:45:21.074258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:45:21.074258 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:45:21.075376 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:45:21.075376 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:45:21.075376 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:45:21.075376 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:45:21.075376 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:21.075376 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:21.075376 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:21.075376 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:21.075376 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:21.075376 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:21.075376 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:21.075376 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:21.075376 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:21.075376 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:21.075376 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:21.075376 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:21.076651 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:21.076651 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:21.076651 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:21.076651 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:21.076651 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:45:21.076651 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:45:21.076651 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:45:21.076651 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:45:21.076651 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:45:21.076651 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:45:21.076651 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:45:21.076651 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:21.076651 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 592/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:45:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:21.081383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9df50-21cc-4d8d-9b99-2a24d20f172d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:21.120605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:21.178384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f9df50-21cc-4d8d-9b99-2a24d20f172d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:21.212421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091b481-5552-40e5-87c5-01644103004f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:21.214045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091b481-5552-40e5-87c5-01644103004f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:21.218637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091b481-5552-40e5-87c5-01644103004f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:21.218637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091b481-5552-40e5-87c5-01644103004f None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:21.218637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091b481-5552-40e5-87c5-01644103004f None admin] RBAC: Adding query filter params (name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:21.218637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091b481-5552-40e5-87c5-01644103004f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:21.218637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091b481-5552-40e5-87c5-01644103004f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:21.225104 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 617/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:45:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:45:21.230721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d4fb8-6b14-4ba5-9523-dd3221b4af09 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:21.280880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:21.327800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d4fb8-6b14-4ba5-9523-dd3221b4af09 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:21.360464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4f127-7659-45c3-bfb4-b20a8ba1899d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:21.361256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4f127-7659-45c3-bfb4-b20a8ba1899d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:21.361542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4f127-7659-45c3-bfb4-b20a8ba1899d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:21.361819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4f127-7659-45c3-bfb4-b20a8ba1899d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:21.372292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4f127-7659-45c3-bfb4-b20a8ba1899d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:21.380761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4f127-7659-45c3-bfb4-b20a8ba1899d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:21.391539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-01a4f127-7659-45c3-bfb4-b20a8ba1899d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:21.391539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:21.391539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:21.391539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:21.391539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:21.391539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:21.391539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:21.391539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:21.391539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:21.391539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:21.391539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:21.391539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:21.391539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:45:21.391539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:45:21.391539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:45:21.393325 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:45:21.393325 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:21.393325 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:21.393325 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:21.393325 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:21.393325 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:21.393325 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:21.393325 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:21.393325 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:21.393325 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:21.393325 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:21.393325 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:21.393325 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:21.393325 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:21.393325 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:21.393325 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:21.393325 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:21.394538 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:45:21.394538 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:45:21.394538 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:21.394538 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:21.394538 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 593/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:21.398550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091b481-5552-40e5-87c5-01644103004f None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:21.495893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:21.498847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091b481-5552-40e5-87c5-01644103004f None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:21.532206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05f96f-46d6-4440-838d-c5616fd05078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:21.532870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05f96f-46d6-4440-838d-c5616fd05078 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:21.533158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05f96f-46d6-4440-838d-c5616fd05078 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:21.533436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05f96f-46d6-4440-838d-c5616fd05078 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:21.534198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05f96f-46d6-4440-838d-c5616fd05078 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:21.534511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05f96f-46d6-4440-838d-c5616fd05078 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:21.535973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05f96f-46d6-4440-838d-c5616fd05078 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:21.548477 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 618/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:21.554134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4f127-7659-45c3-bfb4-b20a8ba1899d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:21.651228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4f127-7659-45c3-bfb4-b20a8ba1899d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:21.652660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:21.683824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db06d9a-62e8-456b-bcdb-ff8088fc16bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:21.684607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db06d9a-62e8-456b-bcdb-ff8088fc16bb None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:21.684885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db06d9a-62e8-456b-bcdb-ff8088fc16bb None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:21.685163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db06d9a-62e8-456b-bcdb-ff8088fc16bb None admin] PATH_INFO: `/v3/users/placement` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:21.692355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db06d9a-62e8-456b-bcdb-ff8088fc16bb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:21.693863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db06d9a-62e8-456b-bcdb-ff8088fc16bb None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:21.710696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-7db06d9a-62e8-456b-bcdb-ff8088fc16bb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 20:45:21.710696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:21.710696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:21.710696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:21.710696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:21.710696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:21.710696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:21.710696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:21.710696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:21.710696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:21.710696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:21.710696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:21.710696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:45:21.710696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:45:21.710696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:45:21.710696 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:21.711837 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:21.713039 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:21.713039 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:21.713039 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:45:21.713039 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:45:21.713039 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:45:21.713039 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:45:21.713039 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:45:21.713039 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:45:21.713039 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 20:45:21.713039 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:21.713039 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 594/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:21.717943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05f96f-46d6-4440-838d-c5616fd05078 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:21.816411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05f96f-46d6-4440-838d-c5616fd05078 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:21.848874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c88995-7403-4db0-b0a6-d2cba70d43df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:21.849537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c88995-7403-4db0-b0a6-d2cba70d43df None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:21.849820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c88995-7403-4db0-b0a6-d2cba70d43df None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:21.850129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c88995-7403-4db0-b0a6-d2cba70d43df None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:21.850968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c88995-7403-4db0-b0a6-d2cba70d43df None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:21.851274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c88995-7403-4db0-b0a6-d2cba70d43df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:21.852703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c88995-7403-4db0-b0a6-d2cba70d43df None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:21.869865 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 619/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Sat May 28 20:45:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:21.876074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db06d9a-62e8-456b-bcdb-ff8088fc16bb None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:21.973626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db06d9a-62e8-456b-bcdb-ff8088fc16bb None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:22.008463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7a2c0-d75f-447d-a1d9-3d4df82a94f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:22.009090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7a2c0-d75f-447d-a1d9-3d4df82a94f8 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:22.009374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7a2c0-d75f-447d-a1d9-3d4df82a94f8 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:22.009649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7a2c0-d75f-447d-a1d9-3d4df82a94f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:22.038517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:22.075819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7a2c0-d75f-447d-a1d9-3d4df82a94f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:22.077277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b7a2c0-d75f-447d-a1d9-3d4df82a94f8 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:22.173281 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 595/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:45:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 20:45:22.178764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c88995-7403-4db0-b0a6-d2cba70d43df None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:22.197600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:22.277485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c88995-7403-4db0-b0a6-d2cba70d43df None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:22.311030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52381ed-c956-4e36-a973-a35f697c397b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:22.311813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52381ed-c956-4e36-a973-a35f697c397b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:22.312094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52381ed-c956-4e36-a973-a35f697c397b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:22.312369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52381ed-c956-4e36-a973-a35f697c397b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:22.322353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52381ed-c956-4e36-a973-a35f697c397b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:22.324050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52381ed-c956-4e36-a973-a35f697c397b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:22.341254 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-f52381ed-c956-4e36-a973-a35f697c397b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:22.341254 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:22.341254 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:22.341254 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:22.341254 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:22.341254 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:22.341254 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:22.341254 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:22.341254 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:22.341254 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:22.341254 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:22.341254 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:22.341254 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:45:22.341254 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:45:22.341254 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:45:22.342389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:45:22.342389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:22.342389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:22.342389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:22.342389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:22.342389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:22.342389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:22.342389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:22.342389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:45:22.342389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:22.342389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:22.342389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:22.342389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:22.342389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:22.342389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:22.342389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:22.342389 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:22.347968 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:45:22.347968 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:45:22.347968 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:22.347968 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:45:22.347968 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 620/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:22.348476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7a2c0-d75f-447d-a1d9-3d4df82a94f8 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:22.408841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:22.445565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b7a2c0-d75f-447d-a1d9-3d4df82a94f8 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:22.478970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ef7ed-b0c4-4594-891f-23c94d4e9a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:22.480170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ef7ed-b0c4-4594-891f-23c94d4e9a72 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:22.480529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ef7ed-b0c4-4594-891f-23c94d4e9a72 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:22.480826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ef7ed-b0c4-4594-891f-23c94d4e9a72 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:22.481541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ef7ed-b0c4-4594-891f-23c94d4e9a72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:22.481871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ef7ed-b0c4-4594-891f-23c94d4e9a72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:22.483230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ef7ed-b0c4-4594-891f-23c94d4e9a72 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:22.494419 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 596/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:22.500179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52381ed-c956-4e36-a973-a35f697c397b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:22.567554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:22.597923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52381ed-c956-4e36-a973-a35f697c397b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:22.631149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51773630-84e0-4196-bd2f-16eb7e13168b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:22.631966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51773630-84e0-4196-bd2f-16eb7e13168b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:22.632247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51773630-84e0-4196-bd2f-16eb7e13168b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:22.632519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51773630-84e0-4196-bd2f-16eb7e13168b None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:22.639770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51773630-84e0-4196-bd2f-16eb7e13168b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:22.641241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51773630-84e0-4196-bd2f-16eb7e13168b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:22.655390 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-51773630-84e0-4196-bd2f-16eb7e13168b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:45:22.655390 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:22.655390 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:22.655390 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:22.655390 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:22.655390 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:22.655390 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:22.655390 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:22.655390 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:22.655390 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:22.655390 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:22.655390 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:22.655390 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:45:22.655390 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:45:22.655390 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:45:22.656451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:45:22.656451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:22.656451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:22.656451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:22.656451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:22.656451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:22.656451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:22.656451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:22.656451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:45:22.656451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:22.656451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:22.656451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:22.656451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:22.656451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:22.656451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:22.656451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:22.656451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:22.657563 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:45:22.657563 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:45:22.657563 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:45:22.657563 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:45:22.657563 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:45:22.657563 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:45:22.657563 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:45:22.657563 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:45:22.657563 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 621/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:45:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:22.662373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ef7ed-b0c4-4594-891f-23c94d4e9a72 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:22.761374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ef7ed-b0c4-4594-891f-23c94d4e9a72 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:22.783936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:22.797447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7eec4-b6a7-42fd-97a7-e85929de95fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:22.798144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7eec4-b6a7-42fd-97a7-e85929de95fb None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:22.798430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7eec4-b6a7-42fd-97a7-e85929de95fb None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:22.798709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7eec4-b6a7-42fd-97a7-e85929de95fb None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:22.799495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7eec4-b6a7-42fd-97a7-e85929de95fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:22.799809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7eec4-b6a7-42fd-97a7-e85929de95fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:22.801292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7eec4-b6a7-42fd-97a7-e85929de95fb None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:22.813172 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 597/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:45:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:22.819304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51773630-84e0-4196-bd2f-16eb7e13168b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:22.918345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51773630-84e0-4196-bd2f-16eb7e13168b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:22.945888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:22.952160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e78919-189e-47ca-bff9-b622672b0ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:22.952914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e78919-189e-47ca-bff9-b622672b0ef6 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:22.953201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e78919-189e-47ca-bff9-b622672b0ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:22.953476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e78919-189e-47ca-bff9-b622672b0ef6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:22.954380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e78919-189e-47ca-bff9-b622672b0ef6 None admin] RBAC: Adding query filter params (role.id=4db40e559d064cf6b4f061e785b42dbb, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=b02348c479904bac893fa8d71d077550) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:22.954685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e78919-189e-47ca-bff9-b622672b0ef6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:22.956110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e78919-189e-47ca-bff9-b622672b0ef6 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:22.961439 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 622/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:45:22 2022] GET /identity/v3/role_assignments?user.id=b02348c479904bac893fa8d71d077550&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=4db40e559d064cf6b4f061e785b42dbb => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:25.201961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3878b2-8423-4864-bf23-b90ae4d194a0 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:25.202511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3878b2-8423-4864-bf23-b90ae4d194a0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:25.202511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3878b2-8423-4864-bf23-b90ae4d194a0 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:25.203888 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 598/1220] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:45:25.209145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8173dfb-bcdd-442e-b963-fe3d517776db None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:25.211963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8173dfb-bcdd-442e-b963-fe3d517776db None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:25.211963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8173dfb-bcdd-442e-b963-fe3d517776db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:25.226648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a8173dfb-bcdd-442e-b963-fe3d517776db None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:45:25.227825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a8173dfb-bcdd-442e-b963-fe3d517776db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:45:25.295685 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 623/1221] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:45:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:45:25.319649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3878b2-8423-4864-bf23-b90ae4d194a0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:25.418629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3878b2-8423-4864-bf23-b90ae4d194a0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:25.452205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce862d-026b-4ab9-9d9b-1e60cf2ba434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:25.452987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce862d-026b-4ab9-9d9b-1e60cf2ba434 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:25.453271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce862d-026b-4ab9-9d9b-1e60cf2ba434 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:25.453546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fce862d-026b-4ab9-9d9b-1e60cf2ba434 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:25.463447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fce862d-026b-4ab9-9d9b-1e60cf2ba434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:45:25.471101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-4fce862d-026b-4ab9-9d9b-1e60cf2ba434 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:45:25.471101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:25.471101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:25.471101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:25.471101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:25.471101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:25.471101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:25.471101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:25.471101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:25.471101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:25.471101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:25.471101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:25.471101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:45:25.471101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:45:25.471101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:45:25.471101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:45:25.472211 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:45:25.472211 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:45:25.472211 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:45:25.472211 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:45:25.472211 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:25.472211 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:25.472211 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:25.472211 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:25.472211 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:25.472211 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:25.472211 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:25.472211 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:25.472211 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:25.472211 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:25.472211 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:25.472211 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:25.473123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:25.473123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:25.473123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:25.473123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:25.473123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:45:25.473123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:45:25.473123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:45:25.473123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:45:25.473123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:45:25.473123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:45:25.473123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:45:25.473123 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:25.473123 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 599/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:45:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:25.478309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8173dfb-bcdd-442e-b963-fe3d517776db None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:25.574957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8173dfb-bcdd-442e-b963-fe3d517776db None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:25.597429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:25.608382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff737d-e121-4b05-b6f9-ebe06091b33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:25.609041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff737d-e121-4b05-b6f9-ebe06091b33b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:25.609321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff737d-e121-4b05-b6f9-ebe06091b33b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:25.609593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff737d-e121-4b05-b6f9-ebe06091b33b None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:25.610374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff737d-e121-4b05-b6f9-ebe06091b33b None admin] RBAC: Adding query filter params (name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:25.610679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff737d-e121-4b05-b6f9-ebe06091b33b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:25.612156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff737d-e121-4b05-b6f9-ebe06091b33b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:25.620426 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 624/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:45:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:25.627631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce862d-026b-4ab9-9d9b-1e60cf2ba434 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:25.671337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:25.725370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fce862d-026b-4ab9-9d9b-1e60cf2ba434 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:25.758099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac849de-ea70-4f8b-9d9f-ff33e8255c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:25.758876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac849de-ea70-4f8b-9d9f-ff33e8255c10 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:25.759155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac849de-ea70-4f8b-9d9f-ff33e8255c10 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:25.759428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac849de-ea70-4f8b-9d9f-ff33e8255c10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:25.770230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac849de-ea70-4f8b-9d9f-ff33e8255c10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:25.771678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac849de-ea70-4f8b-9d9f-ff33e8255c10 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:25.788547 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-eac849de-ea70-4f8b-9d9f-ff33e8255c10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:25.788547 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:25.788547 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:25.788547 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:25.788547 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:25.788547 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:25.788547 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:25.788547 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:25.788547 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:25.788547 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:25.788547 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:25.788547 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:25.788547 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:45:25.788547 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:45:25.788547 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:45:25.789777 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:45:25.789777 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:25.789777 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:25.789777 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:25.789777 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:25.789777 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:25.789777 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:25.789777 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:25.789777 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:25.789777 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:25.789777 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:25.789777 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:25.789777 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:25.789777 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:25.789777 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:25.789777 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:25.789777 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:25.791021 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:45:25.791021 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:45:25.791021 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:25.791021 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:25.791021 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 600/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:25.796215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff737d-e121-4b05-b6f9-ebe06091b33b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:25.895377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff737d-e121-4b05-b6f9-ebe06091b33b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:25.928602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9630af-e795-459a-b3f3-8878b0ca7ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:25.929365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9630af-e795-459a-b3f3-8878b0ca7ebf None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:25.929648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9630af-e795-459a-b3f3-8878b0ca7ebf None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:25.929958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9630af-e795-459a-b3f3-8878b0ca7ebf None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:25.930694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9630af-e795-459a-b3f3-8878b0ca7ebf None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:25.931002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9630af-e795-459a-b3f3-8878b0ca7ebf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:25.932436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9630af-e795-459a-b3f3-8878b0ca7ebf None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:25.943806 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 625/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:25.949416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac849de-ea70-4f8b-9d9f-ff33e8255c10 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:26.043593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:26.050250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac849de-ea70-4f8b-9d9f-ff33e8255c10 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:26.084201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3942dfd-bd2d-4c6b-a693-4b02815d9233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:26.084998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3942dfd-bd2d-4c6b-a693-4b02815d9233 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:26.085282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3942dfd-bd2d-4c6b-a693-4b02815d9233 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:26.085560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3942dfd-bd2d-4c6b-a693-4b02815d9233 None admin] PATH_INFO: `/v3/users/placement` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:26.093129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3942dfd-bd2d-4c6b-a693-4b02815d9233 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:26.094640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3942dfd-bd2d-4c6b-a693-4b02815d9233 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:26.109594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-f3942dfd-bd2d-4c6b-a693-4b02815d9233 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 20:45:26.109594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:26.109594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:26.109594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:26.109594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:26.109594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:26.109594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:26.109594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:26.109594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:26.109594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:26.109594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:26.109594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:26.109594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:45:26.109594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:45:26.109594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:45:26.109594 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:26.110806 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:26.112105 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:26.112105 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:26.112105 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:45:26.112105 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:45:26.112105 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:45:26.112105 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:45:26.112105 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:45:26.112105 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:45:26.112105 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 20:45:26.112105 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:26.112105 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 601/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:26.117099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9630af-e795-459a-b3f3-8878b0ca7ebf None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:26.164267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:26.216374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9630af-e795-459a-b3f3-8878b0ca7ebf None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:26.249787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259149f-7aa6-4b42-93c2-caeabfbdcc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:26.250489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259149f-7aa6-4b42-93c2-caeabfbdcc6c None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:26.250867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259149f-7aa6-4b42-93c2-caeabfbdcc6c None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:26.251160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259149f-7aa6-4b42-93c2-caeabfbdcc6c None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:26.251998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259149f-7aa6-4b42-93c2-caeabfbdcc6c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:26.252308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259149f-7aa6-4b42-93c2-caeabfbdcc6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:26.253739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259149f-7aa6-4b42-93c2-caeabfbdcc6c None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:26.271297 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 626/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Sat May 28 20:45:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:26.277850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3942dfd-bd2d-4c6b-a693-4b02815d9233 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:26.374210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3942dfd-bd2d-4c6b-a693-4b02815d9233 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:26.408115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447899b3-a2e6-4dbc-bf28-e9eaa048513c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:26.408887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447899b3-a2e6-4dbc-bf28-e9eaa048513c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:26.409169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447899b3-a2e6-4dbc-bf28-e9eaa048513c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:26.409440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447899b3-a2e6-4dbc-bf28-e9eaa048513c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:26.419754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447899b3-a2e6-4dbc-bf28-e9eaa048513c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:26.421137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447899b3-a2e6-4dbc-bf28-e9eaa048513c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:26.438897 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-447899b3-a2e6-4dbc-bf28-e9eaa048513c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:26.438897 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:26.438897 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:26.438897 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:26.438897 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:26.438897 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:26.438897 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:26.438897 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:26.438897 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:26.438897 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:26.438897 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:26.438897 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:26.438897 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:45:26.438897 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:45:26.438897 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:45:26.439948 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:45:26.439948 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:26.439948 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:26.439948 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:26.439948 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:26.439948 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:26.439948 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:26.439948 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:26.439948 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:26.439948 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:26.439948 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:26.439948 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:26.439948 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:26.439948 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:26.439948 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:26.439948 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:26.439948 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:26.441150 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:45:26.441150 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:45:26.441150 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:26.441150 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:26.441150 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 602/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:26.445858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259149f-7aa6-4b42-93c2-caeabfbdcc6c None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:26.544468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259149f-7aa6-4b42-93c2-caeabfbdcc6c None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:26.546787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:26.577799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cfed4-a075-40e9-8db5-acc0575182b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:26.578493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cfed4-a075-40e9-8db5-acc0575182b3 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:26.578783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cfed4-a075-40e9-8db5-acc0575182b3 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:26.579067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cfed4-a075-40e9-8db5-acc0575182b3 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:26.579798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0cfed4-a075-40e9-8db5-acc0575182b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:26.580209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0cfed4-a075-40e9-8db5-acc0575182b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:26.581575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0cfed4-a075-40e9-8db5-acc0575182b3 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:26.592966 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 627/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:26.598795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447899b3-a2e6-4dbc-bf28-e9eaa048513c None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:26.608362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:26.695696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447899b3-a2e6-4dbc-bf28-e9eaa048513c None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:26.728108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06c718-a5af-4eda-9989-cec88e4bb5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:26.728888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06c718-a5af-4eda-9989-cec88e4bb5bc None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:26.729175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06c718-a5af-4eda-9989-cec88e4bb5bc None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:26.729453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c06c718-a5af-4eda-9989-cec88e4bb5bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:26.737477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06c718-a5af-4eda-9989-cec88e4bb5bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:26.738954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c06c718-a5af-4eda-9989-cec88e4bb5bc None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:26.753144 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-6c06c718-a5af-4eda-9989-cec88e4bb5bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:45:26.753144 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:26.753144 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:26.753144 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:26.753144 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:26.753144 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:26.753144 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:26.753144 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:26.753144 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:26.753144 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:26.753144 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:26.753144 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:26.753144 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:45:26.753144 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:45:26.753144 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:45:26.754264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:45:26.754264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:26.754264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:26.754264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:26.754264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:26.754264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:26.754264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:26.754264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:26.754264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:26.754264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:26.754264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:26.754264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:26.754264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:26.754264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:26.754264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:26.754264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:26.754264 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:26.755396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:45:26.755396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:45:26.755396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:45:26.755396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:45:26.755396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:45:26.755396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:45:26.755396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:45:26.755396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:26.755396 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 603/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:45:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:26.760160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cfed4-a075-40e9-8db5-acc0575182b3 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:26.859932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cfed4-a075-40e9-8db5-acc0575182b3 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:26.893306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772aeb4-6303-4308-a4c4-7a3a2e242fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:26.894010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772aeb4-6303-4308-a4c4-7a3a2e242fb2 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:26.894291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772aeb4-6303-4308-a4c4-7a3a2e242fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:26.894574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772aeb4-6303-4308-a4c4-7a3a2e242fb2 None admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:26.895370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772aeb4-6303-4308-a4c4-7a3a2e242fb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:26.895667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772aeb4-6303-4308-a4c4-7a3a2e242fb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:26.897106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f772aeb4-6303-4308-a4c4-7a3a2e242fb2 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:26.908978 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 628/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:45:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:26.915151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06c718-a5af-4eda-9989-cec88e4bb5bc None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:26.980534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:27.013809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c06c718-a5af-4eda-9989-cec88e4bb5bc None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:27.046851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30427752-c96a-43c8-8427-914f20dac3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:27.047730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30427752-c96a-43c8-8427-914f20dac3d9 None admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:27.048013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30427752-c96a-43c8-8427-914f20dac3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:27.048286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30427752-c96a-43c8-8427-914f20dac3d9 None admin] PATH_INFO: `/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/b02348c479904bac893fa8d71d077550/roles/4db40e559d064cf6b4f061e785b42dbb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:27.062388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30427752-c96a-43c8-8427-914f20dac3d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=3186f9d3c2b341afa8b098ebcaaf974b, user_id=b02348c479904bac893fa8d71d077550, role_id=4db40e559d064cf6b4f061e785b42dbb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:27.063847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30427752-c96a-43c8-8427-914f20dac3d9 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:27.156608 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 604/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:45:26 2022] PUT /identity/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/b02348c479904bac893fa8d71d077550/roles/4db40e559d064cf6b4f061e785b42dbb => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:45:29.414869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182f5a2-052f-4132-8956-7d81199186fc None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:29.415370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182f5a2-052f-4132-8956-7d81199186fc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:29.415370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2182f5a2-052f-4132-8956-7d81199186fc None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:29.416705 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 629/1233] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:45:29.422063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc9820-62bb-4fa4-b89d-87acbfab649e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:29.422507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc9820-62bb-4fa4-b89d-87acbfab649e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:29.422610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc9820-62bb-4fa4-b89d-87acbfab649e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:29.438869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6afc9820-62bb-4fa4-b89d-87acbfab649e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:45:29.440059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6afc9820-62bb-4fa4-b89d-87acbfab649e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:45:29.506482 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 605/1234] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:45:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:45:29.531867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182f5a2-052f-4132-8956-7d81199186fc None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:29.557645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:29.634335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2182f5a2-052f-4132-8956-7d81199186fc None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:29.667843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356d494-9a97-4f86-9202-b7763fccd421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:29.669136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356d494-9a97-4f86-9202-b7763fccd421 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:29.669420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356d494-9a97-4f86-9202-b7763fccd421 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:29.669697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356d494-9a97-4f86-9202-b7763fccd421 None admin] PATH_INFO: `/v3/roles/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:29.679637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356d494-9a97-4f86-9202-b7763fccd421 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:45:29.687979 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-c356d494-9a97-4f86-9202-b7763fccd421 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 28 20:45:29.687979 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:29.687979 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:29.687979 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:29.687979 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:29.687979 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:29.687979 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:29.687979 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:29.687979 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:29.687979 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:29.687979 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:29.687979 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:29.687979 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:45:29.687979 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:45:29.687979 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:45:29.687979 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:45:29.689172 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:45:29.689172 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:45:29.689172 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:45:29.689172 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:45:29.689172 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:29.689172 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:29.689172 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:29.689172 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:29.689172 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:29.689172 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:29.689172 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:29.689172 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:45:29.689172 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:29.689172 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:29.689172 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:29.689172 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:29.690302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:29.690302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:29.690302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:29.690302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:29.690302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:45:29.690302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:45:29.690302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:45:29.690302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:45:29.690302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:45:29.690302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:45:29.690302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 28 20:45:29.690302 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:45:29.690302 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 630/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Sat May 28 20:45:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:29.695398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc9820-62bb-4fa4-b89d-87acbfab649e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:29.742331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:29.796437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc9820-62bb-4fa4-b89d-87acbfab649e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:29.831173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e602c-c2a4-49ed-88a5-88957ac46d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:29.831850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e602c-c2a4-49ed-88a5-88957ac46d1c None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:29.832135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e602c-c2a4-49ed-88a5-88957ac46d1c None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:29.832413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3e602c-c2a4-49ed-88a5-88957ac46d1c None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:29.833166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e602c-c2a4-49ed-88a5-88957ac46d1c None admin] RBAC: Adding query filter params (name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:29.833477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e602c-c2a4-49ed-88a5-88957ac46d1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:29.834908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3e602c-c2a4-49ed-88a5-88957ac46d1c None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:29.843007 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 606/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Sat May 28 20:45:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:29.848726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356d494-9a97-4f86-9202-b7763fccd421 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:29.932057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:29.948075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356d494-9a97-4f86-9202-b7763fccd421 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:29.981219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267ad99-f242-4f13-9928-7bae02c6bdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:29.982146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267ad99-f242-4f13-9928-7bae02c6bdb3 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:29.982436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267ad99-f242-4f13-9928-7bae02c6bdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:29.982718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267ad99-f242-4f13-9928-7bae02c6bdb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:29.992896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267ad99-f242-4f13-9928-7bae02c6bdb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:29.994423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267ad99-f242-4f13-9928-7bae02c6bdb3 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:30.011509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-b267ad99-f242-4f13-9928-7bae02c6bdb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:30.011509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:30.011509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:30.011509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:30.011509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:30.011509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:30.011509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:30.011509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:30.011509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:30.011509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:30.011509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:30.011509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:30.011509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:45:30.011509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:45:30.011509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:45:30.012785 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:45:30.012785 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:30.012785 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:30.012785 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:30.012785 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:30.012785 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:30.012785 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:30.012785 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:30.012785 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:45:30.012785 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:30.012785 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:30.012785 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:30.012785 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:30.012785 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:30.012785 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:30.012785 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:30.012785 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:30.014335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:45:30.014335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:45:30.014335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:30.014335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:45:30.014335 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 631/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:30.018736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e602c-c2a4-49ed-88a5-88957ac46d1c None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:30.119105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3e602c-c2a4-49ed-88a5-88957ac46d1c None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:30.128321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:30.153386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b62105-c718-4a5a-b36e-ec51162ad43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:30.154086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b62105-c718-4a5a-b36e-ec51162ad43d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:30.154369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b62105-c718-4a5a-b36e-ec51162ad43d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:30.154663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b62105-c718-4a5a-b36e-ec51162ad43d None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:30.155377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b62105-c718-4a5a-b36e-ec51162ad43d None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:30.155683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b62105-c718-4a5a-b36e-ec51162ad43d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:30.157079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b62105-c718-4a5a-b36e-ec51162ad43d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:30.168533 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 607/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:30.174358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267ad99-f242-4f13-9928-7bae02c6bdb3 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:30.276759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267ad99-f242-4f13-9928-7bae02c6bdb3 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:30.310712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef233f-f659-4a79-8045-ee7c66648aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:30.311509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef233f-f659-4a79-8045-ee7c66648aa5 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:30.311790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef233f-f659-4a79-8045-ee7c66648aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:30.312066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fef233f-f659-4a79-8045-ee7c66648aa5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:30.314738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:30.319667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef233f-f659-4a79-8045-ee7c66648aa5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:30.321116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fef233f-f659-4a79-8045-ee7c66648aa5 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:30.335750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-6fef233f-f659-4a79-8045-ee7c66648aa5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 20:45:30.335750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:30.335750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:30.335750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:30.335750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:30.335750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:30.335750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:30.335750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:30.335750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:30.335750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:30.335750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:30.335750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:30.335750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:45:30.335750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:45:30.335750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:45:30.335750 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:30.336691 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:30.337658 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:30.337658 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:30.337658 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:45:30.337658 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:45:30.337658 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:45:30.337658 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:45:30.337658 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:45:30.337658 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:45:30.337658 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 20:45:30.337658 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:45:30.337658 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 632/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:30.342747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b62105-c718-4a5a-b36e-ec51162ad43d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:30.440008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b62105-c718-4a5a-b36e-ec51162ad43d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:30.473955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22f3c9-9083-41a0-bb03-e3729ec7e5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:30.474622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22f3c9-9083-41a0-bb03-e3729ec7e5f3 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:30.474905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22f3c9-9083-41a0-bb03-e3729ec7e5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:30.475181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22f3c9-9083-41a0-bb03-e3729ec7e5f3 None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:30.476042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22f3c9-9083-41a0-bb03-e3729ec7e5f3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:30.476351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22f3c9-9083-41a0-bb03-e3729ec7e5f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:30.477780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22f3c9-9083-41a0-bb03-e3729ec7e5f3 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:30.495239 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 608/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Sat May 28 20:45:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:30.500908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef233f-f659-4a79-8045-ee7c66648aa5 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:30.599990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fef233f-f659-4a79-8045-ee7c66648aa5 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:30.633336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bc100-4a9c-4cca-812e-7e572cc733a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:30.634019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bc100-4a9c-4cca-812e-7e572cc733a8 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:30.634319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bc100-4a9c-4cca-812e-7e572cc733a8 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:30.634588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412bc100-4a9c-4cca-812e-7e572cc733a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:30.697374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:30.704313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bc100-4a9c-4cca-812e-7e572cc733a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:30.705766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412bc100-4a9c-4cca-812e-7e572cc733a8 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:30.800276 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 633/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:45:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 20:45:30.806232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22f3c9-9083-41a0-bb03-e3729ec7e5f3 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:30.835401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:30.907505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22f3c9-9083-41a0-bb03-e3729ec7e5f3 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:30.941669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a9ebc-50c0-4058-9c82-7eb71a19a01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:30.942492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a9ebc-50c0-4058-9c82-7eb71a19a01e None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:30.942840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a9ebc-50c0-4058-9c82-7eb71a19a01e None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:30.943096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8a9ebc-50c0-4058-9c82-7eb71a19a01e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:30.953017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a9ebc-50c0-4058-9c82-7eb71a19a01e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:30.954499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8a9ebc-50c0-4058-9c82-7eb71a19a01e None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:30.971582 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-5f8a9ebc-50c0-4058-9c82-7eb71a19a01e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:30.971582 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:30.971582 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:30.971582 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:30.971582 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:30.971582 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:30.971582 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:30.971582 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:30.971582 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:30.971582 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:30.971582 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:30.971582 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:30.971582 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:45:30.971582 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:45:30.971582 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:45:30.974792 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:45:30.974792 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:30.974792 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:30.974792 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:30.974792 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:30.974792 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:30.974792 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:30.974792 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:30.974792 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:30.974792 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:30.974792 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:30.974792 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:30.974792 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:30.974792 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:30.974792 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:30.974792 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:30.974792 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:30.975763 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:45:30.975763 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:45:30.975763 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:30.975763 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:30.975763 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 609/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:45:30.978533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bc100-4a9c-4cca-812e-7e572cc733a8 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:31.076238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:31.082057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412bc100-4a9c-4cca-812e-7e572cc733a8 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:31.116696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d3064-ee91-4347-890d-701837f23206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:31.117453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d3064-ee91-4347-890d-701837f23206 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:31.117771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d3064-ee91-4347-890d-701837f23206 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:31.118080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d3064-ee91-4347-890d-701837f23206 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:31.118813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d3064-ee91-4347-890d-701837f23206 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:31.119124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d3064-ee91-4347-890d-701837f23206 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:31.120547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d3064-ee91-4347-890d-701837f23206 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:31.132974 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 634/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:31.139468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a9ebc-50c0-4058-9c82-7eb71a19a01e None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:31.216553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:31.237052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8a9ebc-50c0-4058-9c82-7eb71a19a01e None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:31.272108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3e152-d86b-4fca-8bf5-e1d63ac698b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:31.272896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3e152-d86b-4fca-8bf5-e1d63ac698b0 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:31.273180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3e152-d86b-4fca-8bf5-e1d63ac698b0 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:31.273453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e3e152-d86b-4fca-8bf5-e1d63ac698b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:31.280335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3e152-d86b-4fca-8bf5-e1d63ac698b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:31.281793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e3e152-d86b-4fca-8bf5-e1d63ac698b0 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:31.295261 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-b9e3e152-d86b-4fca-8bf5-e1d63ac698b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:45:31.295261 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:31.295261 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:31.295261 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:31.295261 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:31.295261 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:31.295261 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:31.295261 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:31.295261 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:31.295261 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:31.295261 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:31.295261 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:31.295261 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:45:31.295261 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:45:31.295261 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:45:31.296323 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:45:31.296323 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:31.296323 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:31.296323 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:31.296323 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:31.296323 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:31.296323 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:31.296323 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:31.296323 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:31.296323 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:31.296323 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:31.296323 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:31.296323 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:31.296323 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:31.296323 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:31.296323 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:31.296323 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:31.297784 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:45:31.297784 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:45:31.297784 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:45:31.297784 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:45:31.297784 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:45:31.297784 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:45:31.297784 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:45:31.297784 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:31.297784 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 610/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:45:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:31.302275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d3064-ee91-4347-890d-701837f23206 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:31.399421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d3064-ee91-4347-890d-701837f23206 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:31.432208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18f255-2938-4b2e-9fac-7588263edf4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:31.432869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18f255-2938-4b2e-9fac-7588263edf4b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:31.433148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18f255-2938-4b2e-9fac-7588263edf4b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:31.433448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18f255-2938-4b2e-9fac-7588263edf4b None admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:31.434265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18f255-2938-4b2e-9fac-7588263edf4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:31.434575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18f255-2938-4b2e-9fac-7588263edf4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:31.435993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18f255-2938-4b2e-9fac-7588263edf4b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:31.447688 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 635/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:45:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:31.454351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3e152-d86b-4fca-8bf5-e1d63ac698b0 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:31.553829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e3e152-d86b-4fca-8bf5-e1d63ac698b0 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:31.586436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94382ee-6705-4a54-a547-f10b86984bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:31.587085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94382ee-6705-4a54-a547-f10b86984bac None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:31.587370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94382ee-6705-4a54-a547-f10b86984bac None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:31.587647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94382ee-6705-4a54-a547-f10b86984bac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:31.588474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94382ee-6705-4a54-a547-f10b86984bac None admin] RBAC: Adding query filter params (role.id=4db40e559d064cf6b4f061e785b42dbb, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=b02348c479904bac893fa8d71d077550) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:31.588785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94382ee-6705-4a54-a547-f10b86984bac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:31.590310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94382ee-6705-4a54-a547-f10b86984bac None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:31.596174 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 611/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:45:31 2022] GET /identity/v3/role_assignments?user.id=b02348c479904bac893fa8d71d077550&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=4db40e559d064cf6b4f061e785b42dbb => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:33.939412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25992b-be1d-490b-aa95-02fe5f90e45d None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:33.939948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25992b-be1d-490b-aa95-02fe5f90e45d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:33.939948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d25992b-be1d-490b-aa95-02fe5f90e45d None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:33.941349 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 636/1247] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:45:33.946550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5514976-09f6-4981-99f3-bef4ba589ac3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:33.946922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5514976-09f6-4981-99f3-bef4ba589ac3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:33.947106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5514976-09f6-4981-99f3-bef4ba589ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:33.952257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:33.964124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f5514976-09f6-4981-99f3-bef4ba589ac3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:45:33.965587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f5514976-09f6-4981-99f3-bef4ba589ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:45:34.031849 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 612/1248] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:45:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:45:34.058123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25992b-be1d-490b-aa95-02fe5f90e45d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:34.077355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:34.161686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d25992b-be1d-490b-aa95-02fe5f90e45d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:34.195702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c204b-e0bf-4449-920f-65d6188e3f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:34.196467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c204b-e0bf-4449-920f-65d6188e3f66 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:34.196752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c204b-e0bf-4449-920f-65d6188e3f66 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:34.197029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c204b-e0bf-4449-920f-65d6188e3f66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:34.207552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c204b-e0bf-4449-920f-65d6188e3f66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:45:34.215177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-464c204b-e0bf-4449-920f-65d6188e3f66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:45:34.215177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:34.215177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:34.215177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:34.215177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:34.215177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:34.215177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:34.215177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:34.215177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:34.215177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:34.215177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:34.215177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:34.215177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:45:34.215177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:45:34.215177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:45:34.215177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:45:34.216233 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:45:34.216233 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:45:34.216233 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:45:34.216233 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:45:34.216233 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:34.216233 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:34.216233 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:34.216233 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:34.216233 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:34.216233 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:34.216233 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:34.216233 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:45:34.216233 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:34.216233 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:34.216233 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:34.216233 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:34.217488 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:34.217488 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:34.217488 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:34.217488 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:34.217488 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:45:34.217488 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:45:34.217488 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:45:34.217488 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:45:34.217488 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:45:34.217488 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:45:34.217488 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:45:34.217488 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:45:34.217488 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 637/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:45:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:45:34.222655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5514976-09f6-4981-99f3-bef4ba589ac3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:34.322661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5514976-09f6-4981-99f3-bef4ba589ac3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:34.352046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:34.357066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e4910-bc23-4874-a2a7-787f0f7723e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:34.357724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e4910-bc23-4874-a2a7-787f0f7723e3 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:34.358031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e4910-bc23-4874-a2a7-787f0f7723e3 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:34.358304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e4910-bc23-4874-a2a7-787f0f7723e3 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:34.359049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e4910-bc23-4874-a2a7-787f0f7723e3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:34.359428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e4910-bc23-4874-a2a7-787f0f7723e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:34.360848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e4910-bc23-4874-a2a7-787f0f7723e3 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:34.368704 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 613/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:45:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:34.374932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c204b-e0bf-4449-920f-65d6188e3f66 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:34.452995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:34.475668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c204b-e0bf-4449-920f-65d6188e3f66 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:34.508345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d907e0b-e595-419c-b867-c5eb11250078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:34.509134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d907e0b-e595-419c-b867-c5eb11250078 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:34.509423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d907e0b-e595-419c-b867-c5eb11250078 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:34.509706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d907e0b-e595-419c-b867-c5eb11250078 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:34.521516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d907e0b-e595-419c-b867-c5eb11250078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:34.523001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d907e0b-e595-419c-b867-c5eb11250078 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:34.540335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-5d907e0b-e595-419c-b867-c5eb11250078 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:34.540335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:34.540335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:34.540335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:34.540335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:34.540335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:34.540335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:34.540335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:34.540335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:34.540335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:34.540335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:34.540335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:34.540335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:45:34.540335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:45:34.540335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:45:34.541335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:45:34.541335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:34.541335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:34.541335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:34.541335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:34.541335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:34.541335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:34.541335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:34.541335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:45:34.541335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:34.541335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:34.541335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:34.541335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:34.541335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:34.541335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:34.541335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:34.541335 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:34.542264 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:45:34.542264 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:45:34.542264 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:34.542264 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:45:34.542264 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 638/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:34.547433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e4910-bc23-4874-a2a7-787f0f7723e3 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:34.645092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e4910-bc23-4874-a2a7-787f0f7723e3 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:34.677788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6debd10-3856-4817-b447-40a9a2cd385f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:34.678460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6debd10-3856-4817-b447-40a9a2cd385f None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:34.678740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6debd10-3856-4817-b447-40a9a2cd385f None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:34.679014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6debd10-3856-4817-b447-40a9a2cd385f None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:34.679752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6debd10-3856-4817-b447-40a9a2cd385f None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:34.680069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6debd10-3856-4817-b447-40a9a2cd385f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:34.681433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6debd10-3856-4817-b447-40a9a2cd385f None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:34.692745 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 614/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:34.698560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d907e0b-e595-419c-b867-c5eb11250078 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:34.801217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d907e0b-e595-419c-b867-c5eb11250078 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:34.826704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:34.834389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eac104-5fbc-416e-a5ef-80a5573d03f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:34.835179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eac104-5fbc-416e-a5ef-80a5573d03f1 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:34.835460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eac104-5fbc-416e-a5ef-80a5573d03f1 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:34.835734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eac104-5fbc-416e-a5ef-80a5573d03f1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:34.843732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eac104-5fbc-416e-a5ef-80a5573d03f1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:34.845168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eac104-5fbc-416e-a5ef-80a5573d03f1 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:34.860076 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-77eac104-5fbc-416e-a5ef-80a5573d03f1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 20:45:34.860076 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:34.860076 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:34.860076 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:34.860076 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:34.860076 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:34.860076 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:34.860076 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:34.860076 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:34.860076 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:34.860076 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:34.860076 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:34.860076 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:45:34.860076 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:45:34.860076 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:45:34.860076 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:34.861282 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:34.862757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:34.862757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:34.862757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:45:34.862757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:45:34.862757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:45:34.862757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:45:34.862757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:45:34.862757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:45:34.862757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 20:45:34.862757 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:45:34.862757 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 639/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:34.867392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6debd10-3856-4817-b447-40a9a2cd385f None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:34.922522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:34.964025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6debd10-3856-4817-b447-40a9a2cd385f None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:34.997439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549a2c4-1351-46ae-95e8-024db50a9de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:34.998130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549a2c4-1351-46ae-95e8-024db50a9de0 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:34.998413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549a2c4-1351-46ae-95e8-024db50a9de0 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:34.998687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549a2c4-1351-46ae-95e8-024db50a9de0 None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:34.999510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549a2c4-1351-46ae-95e8-024db50a9de0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:34.999818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549a2c4-1351-46ae-95e8-024db50a9de0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:35.001234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549a2c4-1351-46ae-95e8-024db50a9de0 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:35.018289 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 615/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Sat May 28 20:45:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:35.024213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eac104-5fbc-416e-a5ef-80a5573d03f1 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:35.124643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eac104-5fbc-416e-a5ef-80a5573d03f1 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:35.158852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e752e-2e76-4e84-8f8e-4363360310d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:35.159539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e752e-2e76-4e84-8f8e-4363360310d4 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:35.159819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e752e-2e76-4e84-8f8e-4363360310d4 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:35.160094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e752e-2e76-4e84-8f8e-4363360310d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:35.214054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:35.226057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e752e-2e76-4e84-8f8e-4363360310d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:35.227544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0e752e-2e76-4e84-8f8e-4363360310d4 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:35.324421 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 640/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:45:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 28 20:45:35.330359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549a2c4-1351-46ae-95e8-024db50a9de0 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:35.430110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549a2c4-1351-46ae-95e8-024db50a9de0 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:35.445622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:35.466965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc8054-db82-4a8f-944d-5f1c9360a128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:35.467746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc8054-db82-4a8f-944d-5f1c9360a128 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:35.468026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc8054-db82-4a8f-944d-5f1c9360a128 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:35.468298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc8054-db82-4a8f-944d-5f1c9360a128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:35.477917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc8054-db82-4a8f-944d-5f1c9360a128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:35.479406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc8054-db82-4a8f-944d-5f1c9360a128 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:35.497899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-47bc8054-db82-4a8f-944d-5f1c9360a128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:35.497899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:35.497899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:35.497899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:35.497899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:35.497899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:35.497899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:35.497899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:35.497899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:35.497899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:35.497899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:35.497899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:35.497899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:45:35.497899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:45:35.497899 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:45:35.499007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:45:35.499007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:35.499007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:35.499007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:35.499007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:35.499007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:35.499007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:35.499007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:35.499007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:35.499007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:35.499007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:35.499007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:35.499007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:35.499007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:35.499007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:35.499007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:35.499007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:35.500110 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:45:35.500110 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:45:35.500110 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:35.500110 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:35.500110 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 616/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:35.504863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e752e-2e76-4e84-8f8e-4363360310d4 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:35.590462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:35.602105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e752e-2e76-4e84-8f8e-4363360310d4 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:35.635454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038fab2-ed00-401a-8be3-427325014122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:35.636110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038fab2-ed00-401a-8be3-427325014122 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:35.636469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038fab2-ed00-401a-8be3-427325014122 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:35.637301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038fab2-ed00-401a-8be3-427325014122 None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:35.638075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038fab2-ed00-401a-8be3-427325014122 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:35.638639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038fab2-ed00-401a-8be3-427325014122 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:35.640010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038fab2-ed00-401a-8be3-427325014122 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:35.650898 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 641/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:35.656737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc8054-db82-4a8f-944d-5f1c9360a128 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:35.758245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc8054-db82-4a8f-944d-5f1c9360a128 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:35.791676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b6c8a-f3f2-42ae-8c77-13fc1f64d4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:35.792464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b6c8a-f3f2-42ae-8c77-13fc1f64d4f9 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:35.793360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b6c8a-f3f2-42ae-8c77-13fc1f64d4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:35.793647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b6c8a-f3f2-42ae-8c77-13fc1f64d4f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:35.800312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b6c8a-f3f2-42ae-8c77-13fc1f64d4f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:35.801770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b6c8a-f3f2-42ae-8c77-13fc1f64d4f9 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:35.816083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-537b6c8a-f3f2-42ae-8c77-13fc1f64d4f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:45:35.816083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:35.816083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:35.816083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:35.816083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:35.816083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:35.816083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:35.816083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:35.816083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:35.816083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:35.816083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:35.816083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:35.816083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:45:35.816083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:45:35.816083 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:45:35.817154 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:45:35.817154 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:35.817154 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:35.817154 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:35.817154 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:35.817154 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:35.817154 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:35.817154 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:35.817154 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:35.817154 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:35.817154 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:35.817154 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:35.817154 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:35.817154 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:35.817154 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:35.817154 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:35.817154 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:35.823119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:45:35.823119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:45:35.823119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:45:35.823119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:45:35.823119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:45:35.823119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:45:35.823119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:45:35.823119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:35.823119 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 617/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:45:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:35.823727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038fab2-ed00-401a-8be3-427325014122 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:35.925232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038fab2-ed00-401a-8be3-427325014122 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:35.958751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8778a31-753a-415c-85c6-c79fd9ca9373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:35.959440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8778a31-753a-415c-85c6-c79fd9ca9373 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:35.959752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8778a31-753a-415c-85c6-c79fd9ca9373 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:35.960029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8778a31-753a-415c-85c6-c79fd9ca9373 None admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:35.960817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8778a31-753a-415c-85c6-c79fd9ca9373 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:35.961124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8778a31-753a-415c-85c6-c79fd9ca9373 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:35.962600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8778a31-753a-415c-85c6-c79fd9ca9373 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:35.974548 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 642/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:45:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:35.981281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b6c8a-f3f2-42ae-8c77-13fc1f64d4f9 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:36.007403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:36.079995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b6c8a-f3f2-42ae-8c77-13fc1f64d4f9 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:36.115508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173a4fa4-e466-4da4-a59e-f2771a70c205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:36.116159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a4fa4-e466-4da4-a59e-f2771a70c205 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:36.116443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a4fa4-e466-4da4-a59e-f2771a70c205 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:36.116722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a4fa4-e466-4da4-a59e-f2771a70c205 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:36.117542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a4fa4-e466-4da4-a59e-f2771a70c205 None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=b02348c479904bac893fa8d71d077550) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:36.117875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a4fa4-e466-4da4-a59e-f2771a70c205 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:36.119398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173a4fa4-e466-4da4-a59e-f2771a70c205 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:36.125409 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 618/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:45:35 2022] GET /identity/v3/role_assignments?user.id=b02348c479904bac893fa8d71d077550&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=7805b93b32f1472f8365936d4d57d49a => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:38.378414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce312d3d-9c95-4a29-ae1c-b1733146774e None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:38.378941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce312d3d-9c95-4a29-ae1c-b1733146774e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:38.378941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce312d3d-9c95-4a29-ae1c-b1733146774e None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:38.380264 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 643/1261] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:45:38.385632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2418ee-bb71-4b7a-9f8c-c402123f9f45 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:38.386032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2418ee-bb71-4b7a-9f8c-c402123f9f45 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:38.386228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2418ee-bb71-4b7a-9f8c-c402123f9f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:38.403251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8d2418ee-bb71-4b7a-9f8c-c402123f9f45 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:45:38.404429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8d2418ee-bb71-4b7a-9f8c-c402123f9f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:45:38.470684 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 619/1262] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:45:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:45:38.494498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce312d3d-9c95-4a29-ae1c-b1733146774e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:38.505410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:38.601137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce312d3d-9c95-4a29-ae1c-b1733146774e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:38.635481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a8895-84f3-4b22-b697-b63644e1d8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:38.636205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a8895-84f3-4b22-b697-b63644e1d8dc None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:38.637109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a8895-84f3-4b22-b697-b63644e1d8dc None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:38.637385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a8895-84f3-4b22-b697-b63644e1d8dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:38.647546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697a8895-84f3-4b22-b697-b63644e1d8dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:45:38.655509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-697a8895-84f3-4b22-b697-b63644e1d8dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:45:38.655509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:38.655509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:38.655509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:38.655509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:38.655509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:38.655509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:38.655509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:38.655509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:38.655509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:38.655509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:38.655509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:38.655509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:45:38.655509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:45:38.655509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:45:38.655509 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:45:38.656694 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:45:38.656694 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:45:38.656694 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:45:38.656694 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:45:38.656694 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:38.656694 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:38.656694 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:38.656694 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:38.656694 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:38.656694 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:38.656694 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:38.656694 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:45:38.656694 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:38.656694 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:38.656694 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:38.656694 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:38.657887 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:38.657887 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:38.657887 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:38.657887 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:38.657887 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:45:38.657887 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:45:38.657887 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:45:38.657887 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:45:38.657887 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:45:38.657887 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:45:38.657887 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:45:38.657887 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:45:38.657887 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 644/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:45:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:38.662877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2418ee-bb71-4b7a-9f8c-c402123f9f45 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:38.681063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:38.760402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2418ee-bb71-4b7a-9f8c-c402123f9f45 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:38.795766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac67bd5-62fd-485c-9c39-b939feb6afd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:38.796425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac67bd5-62fd-485c-9c39-b939feb6afd7 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:38.796707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac67bd5-62fd-485c-9c39-b939feb6afd7 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:38.796981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac67bd5-62fd-485c-9c39-b939feb6afd7 None admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:38.797740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac67bd5-62fd-485c-9c39-b939feb6afd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:38.798061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac67bd5-62fd-485c-9c39-b939feb6afd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:38.799482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac67bd5-62fd-485c-9c39-b939feb6afd7 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:38.807343 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 620/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:45:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:38.814655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a8895-84f3-4b22-b697-b63644e1d8dc None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:38.880519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:38.913186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a8895-84f3-4b22-b697-b63644e1d8dc None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:38.945579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e307a-6447-4a74-b9ca-e8ec9c2009b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:38.946429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e307a-6447-4a74-b9ca-e8ec9c2009b4 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:38.951365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e307a-6447-4a74-b9ca-e8ec9c2009b4 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:38.951365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e307a-6447-4a74-b9ca-e8ec9c2009b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:38.957679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e307a-6447-4a74-b9ca-e8ec9c2009b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:38.959143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e307a-6447-4a74-b9ca-e8ec9c2009b4 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:38.976062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-c47e307a-6447-4a74-b9ca-e8ec9c2009b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:38.976062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:38.976062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:38.976062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:38.976062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:38.976062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:38.976062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:38.976062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:38.976062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:38.976062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:38.976062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:38.976062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:38.976062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:45:38.976062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:45:38.976062 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:45:38.977117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:45:38.977117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:38.977117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:38.977117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:38.977117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:38.977117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:38.977117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:38.977117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:38.977117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:45:38.977117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:38.977117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:38.977117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:38.977117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:38.977117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:38.977117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:38.977117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:38.977117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:38.978107 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:45:38.978107 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:45:38.978107 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:38.978107 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:45:38.978107 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 645/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:38.983196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac67bd5-62fd-485c-9c39-b939feb6afd7 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:39.066228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:39.086182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac67bd5-62fd-485c-9c39-b939feb6afd7 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:39.126308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e22e92-e289-44ad-a9e2-636bd5d55ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:39.126979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e22e92-e289-44ad-a9e2-636bd5d55ce3 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:39.127261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e22e92-e289-44ad-a9e2-636bd5d55ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:39.127545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e22e92-e289-44ad-a9e2-636bd5d55ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:39.128260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e22e92-e289-44ad-a9e2-636bd5d55ce3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:39.128568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e22e92-e289-44ad-a9e2-636bd5d55ce3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:39.129968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e22e92-e289-44ad-a9e2-636bd5d55ce3 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:39.142425 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 621/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:39.148379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e307a-6447-4a74-b9ca-e8ec9c2009b4 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:39.250263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e307a-6447-4a74-b9ca-e8ec9c2009b4 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:39.270290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:39.283393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637a792-048a-4c5c-9e2a-131b63188fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:39.284191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637a792-048a-4c5c-9e2a-131b63188fc9 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:39.284472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637a792-048a-4c5c-9e2a-131b63188fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:39.284749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637a792-048a-4c5c-9e2a-131b63188fc9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:39.292159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637a792-048a-4c5c-9e2a-131b63188fc9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:39.294120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6637a792-048a-4c5c-9e2a-131b63188fc9 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:39.308840 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-6637a792-048a-4c5c-9e2a-131b63188fc9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 20:45:39.308840 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:39.308840 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:39.308840 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:39.308840 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:39.308840 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:39.308840 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:39.308840 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:39.308840 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:39.308840 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:39.308840 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:39.308840 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:39.308840 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:45:39.308840 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:45:39.308840 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:45:39.308840 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:39.310020 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:39.311391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:39.311391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:39.311391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:45:39.311391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:45:39.311391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:45:39.311391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:45:39.311391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:45:39.311391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:45:39.311391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 20:45:39.311391 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:45:39.311391 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 646/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:39.316016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e22e92-e289-44ad-a9e2-636bd5d55ce3 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:39.413155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e22e92-e289-44ad-a9e2-636bd5d55ce3 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:39.446891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad2f05-3ea5-4f6d-8a96-960cc65d0a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:39.447546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad2f05-3ea5-4f6d-8a96-960cc65d0a1d None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:39.447832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad2f05-3ea5-4f6d-8a96-960cc65d0a1d None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:39.448108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad2f05-3ea5-4f6d-8a96-960cc65d0a1d None admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:39.449490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad2f05-3ea5-4f6d-8a96-960cc65d0a1d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:39.449808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad2f05-3ea5-4f6d-8a96-960cc65d0a1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:39.451279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fad2f05-3ea5-4f6d-8a96-960cc65d0a1d None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:39.459688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:39.470648 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 622/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Sat May 28 20:45:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:39.476209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637a792-048a-4c5c-9e2a-131b63188fc9 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:39.576020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637a792-048a-4c5c-9e2a-131b63188fc9 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:39.608914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a652bc-75a4-4460-8cce-7a536f7b8e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:39.609698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a652bc-75a4-4460-8cce-7a536f7b8e6f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:39.610021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a652bc-75a4-4460-8cce-7a536f7b8e6f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:39.610388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a652bc-75a4-4460-8cce-7a536f7b8e6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:39.621984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a652bc-75a4-4460-8cce-7a536f7b8e6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:39.623567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a652bc-75a4-4460-8cce-7a536f7b8e6f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:39.641207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-31a652bc-75a4-4460-8cce-7a536f7b8e6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:39.641207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:39.641207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:39.641207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:39.641207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:39.641207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:39.641207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:39.641207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:39.641207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:39.641207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:39.641207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:39.641207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:39.641207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:45:39.641207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:45:39.641207 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:45:39.642308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:45:39.642308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:39.642308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:39.642308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:39.642308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:39.642308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:39.642308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:39.642308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:39.642308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:45:39.642308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:39.642308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:39.642308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:39.642308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:39.642308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:39.642308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:39.642308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:39.642308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:39.643421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:45:39.643421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:45:39.643421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:39.643421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:45:39.643421 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 647/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:39.648168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad2f05-3ea5-4f6d-8a96-960cc65d0a1d None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:39.747528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fad2f05-3ea5-4f6d-8a96-960cc65d0a1d None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:39.782132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef13a4-1369-4098-a857-ea77f96e4508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:39.782796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef13a4-1369-4098-a857-ea77f96e4508 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:39.783080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef13a4-1369-4098-a857-ea77f96e4508 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:39.783356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef13a4-1369-4098-a857-ea77f96e4508 None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:39.784071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef13a4-1369-4098-a857-ea77f96e4508 None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:39.784375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef13a4-1369-4098-a857-ea77f96e4508 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:39.785750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef13a4-1369-4098-a857-ea77f96e4508 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:39.796986 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 623/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:39.802931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a652bc-75a4-4460-8cce-7a536f7b8e6f None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:39.834941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:39.900896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a652bc-75a4-4460-8cce-7a536f7b8e6f None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:39.935844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83604b-ac17-4a10-947d-b15bc09408be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:39.936638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83604b-ac17-4a10-947d-b15bc09408be None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:39.936922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83604b-ac17-4a10-947d-b15bc09408be None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:39.937199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83604b-ac17-4a10-947d-b15bc09408be None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:39.943723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83604b-ac17-4a10-947d-b15bc09408be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:39.945164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83604b-ac17-4a10-947d-b15bc09408be None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:39.959728 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-af83604b-ac17-4a10-947d-b15bc09408be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:45:39.959728 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:39.959728 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:39.959728 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:39.959728 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:39.959728 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:39.959728 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:39.959728 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:39.959728 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:39.959728 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:39.959728 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:39.959728 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:39.959728 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:45:39.959728 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:45:39.959728 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:45:39.961502 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:45:39.961502 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:39.961502 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:39.961502 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:39.961502 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:39.961502 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:39.961502 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:39.961502 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:39.961502 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:45:39.961502 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:39.961502 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:39.961502 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:39.961502 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:39.961502 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:39.961502 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:39.961502 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:39.961502 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:39.962759 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:45:39.962759 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:45:39.962759 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:45:39.962759 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:45:39.962759 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:45:39.962759 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:45:39.962759 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:45:39.962759 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:45:39.962759 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 648/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:45:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:39.967324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef13a4-1369-4098-a857-ea77f96e4508 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:40.026757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:40.069337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef13a4-1369-4098-a857-ea77f96e4508 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:40.103056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c11b262-846b-4e0a-afcb-606b683597e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:40.103658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11b262-846b-4e0a-afcb-606b683597e4 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:40.103943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11b262-846b-4e0a-afcb-606b683597e4 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:40.104219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c11b262-846b-4e0a-afcb-606b683597e4 None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:40.105004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c11b262-846b-4e0a-afcb-606b683597e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:40.105891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c11b262-846b-4e0a-afcb-606b683597e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:40.107336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c11b262-846b-4e0a-afcb-606b683597e4 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:40.119108 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 624/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:45:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:40.125285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83604b-ac17-4a10-947d-b15bc09408be None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:40.224223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:40.233577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83604b-ac17-4a10-947d-b15bc09408be None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:40.266616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51728f7d-5161-4091-8065-ef9e240a54a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:40.267496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51728f7d-5161-4091-8065-ef9e240a54a3 None admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:40.267778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51728f7d-5161-4091-8065-ef9e240a54a3 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:40.268051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51728f7d-5161-4091-8065-ef9e240a54a3 None admin] PATH_INFO: `/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/b02348c479904bac893fa8d71d077550/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:40.270070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51728f7d-5161-4091-8065-ef9e240a54a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=3186f9d3c2b341afa8b098ebcaaf974b, user_id=b02348c479904bac893fa8d71d077550, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:40.271496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51728f7d-5161-4091-8065-ef9e240a54a3 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:40.426204 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 649/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Sat May 28 20:45:40 2022] PUT /identity/v3/projects/3186f9d3c2b341afa8b098ebcaaf974b/users/b02348c479904bac893fa8d71d077550/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:45:42.696416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df442bd4-96f2-4478-bc29-f710ee52a9a6 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:42.696973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df442bd4-96f2-4478-bc29-f710ee52a9a6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:42.696973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df442bd4-96f2-4478-bc29-f710ee52a9a6 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:42.698356 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 625/1274] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:45:42.704265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc909a8-d733-418f-ac4b-a0f5b1d04046 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:42.704699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc909a8-d733-418f-ac4b-a0f5b1d04046 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:42.704829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc909a8-d733-418f-ac4b-a0f5b1d04046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:42.720997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8dc909a8-d733-418f-ac4b-a0f5b1d04046 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:45:42.722235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8dc909a8-d733-418f-ac4b-a0f5b1d04046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:45:42.787988 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 650/1275] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:45:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:45:42.814581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df442bd4-96f2-4478-bc29-f710ee52a9a6 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:42.915488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df442bd4-96f2-4478-bc29-f710ee52a9a6 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:42.933058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:42.949622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de25469-89ff-4321-b989-f33df5dbe7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:42.950386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de25469-89ff-4321-b989-f33df5dbe7d9 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:42.950670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de25469-89ff-4321-b989-f33df5dbe7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:42.950945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de25469-89ff-4321-b989-f33df5dbe7d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:42.961040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de25469-89ff-4321-b989-f33df5dbe7d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 20:45:42.969389 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-6de25469-89ff-4321-b989-f33df5dbe7d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:45:42.969389 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:42.969389 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:42.969389 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:42.969389 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:42.969389 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:42.969389 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:42.969389 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:42.969389 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:42.969389 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:42.969389 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:42.969389 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:42.969389 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 28 20:45:42.969389 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(role_id) May 28 20:45:42.969389 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 28 20:45:42.969389 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 28 20:45:42.979102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 20:45:42.979102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 20:45:42.979102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 20:45:42.979102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 20:45:42.979102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:42.979102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:42.979102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:42.979102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:42.979102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:42.979102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:42.979102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:42.979102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:42.979102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:42.979102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:42.979102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:42.979102 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:42.979971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:42.979971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:42.979971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:42.979971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:42.979971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 20:45:42.979971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 20:45:42.979971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 20:45:42.979971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 20:45:42.979971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 20:45:42.979971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 20:45:42.979971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 28 20:45:42.979971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:42.979971 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 626/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Sat May 28 20:45:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:42.980709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc909a8-d733-418f-ac4b-a0f5b1d04046 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:43.059159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:43.077264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc909a8-d733-418f-ac4b-a0f5b1d04046 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:43.111294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad518b-d056-4b04-bed1-ac05cdac4ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:43.111969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad518b-d056-4b04-bed1-ac05cdac4ab0 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:43.112266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad518b-d056-4b04-bed1-ac05cdac4ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:43.112773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad518b-d056-4b04-bed1-ac05cdac4ab0 None admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:43.113753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad518b-d056-4b04-bed1-ac05cdac4ab0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:43.114143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad518b-d056-4b04-bed1-ac05cdac4ab0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:43.116062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad518b-d056-4b04-bed1-ac05cdac4ab0 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:43.124676 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 651/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Sat May 28 20:45:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:43.130612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de25469-89ff-4321-b989-f33df5dbe7d9 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:43.229580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de25469-89ff-4321-b989-f33df5dbe7d9 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:43.262721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd569e-e574-468b-bfec-e3d5be501921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:43.263757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd569e-e574-468b-bfec-e3d5be501921 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:43.264039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd569e-e574-468b-bfec-e3d5be501921 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:43.264312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedd569e-e574-468b-bfec-e3d5be501921 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:43.275115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd569e-e574-468b-bfec-e3d5be501921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:43.276557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedd569e-e574-468b-bfec-e3d5be501921 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:43.294396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-fedd569e-e574-468b-bfec-e3d5be501921 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:43.294396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:43.294396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:43.294396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:43.294396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:43.294396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:43.294396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:43.294396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:43.294396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:43.294396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:43.294396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:43.294396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:43.294396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:45:43.294396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:45:43.294396 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:45:43.295492 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:45:43.295492 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:43.295492 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:43.295492 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:43.295492 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:43.295492 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:43.295492 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:43.295492 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:43.295492 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:43.295492 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:43.295492 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:43.295492 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:43.295492 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:43.295492 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:43.295492 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:43.295492 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:43.295492 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:43.296627 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:45:43.296627 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:45:43.296627 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:43.296627 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:43.296627 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 627/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:43.301528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad518b-d056-4b04-bed1-ac05cdac4ab0 None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:43.399575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad518b-d056-4b04-bed1-ac05cdac4ab0 None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:43.432716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910544c5-69b7-4c3b-8491-447019fcee5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:43.433928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910544c5-69b7-4c3b-8491-447019fcee5b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:43.434217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910544c5-69b7-4c3b-8491-447019fcee5b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:43.434497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910544c5-69b7-4c3b-8491-447019fcee5b None admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:43.435242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910544c5-69b7-4c3b-8491-447019fcee5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:43.435552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910544c5-69b7-4c3b-8491-447019fcee5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:43.436930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910544c5-69b7-4c3b-8491-447019fcee5b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:43.449680 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 652/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:43.455253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd569e-e574-468b-bfec-e3d5be501921 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:43.485597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:43.552550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedd569e-e574-468b-bfec-e3d5be501921 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:43.585792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0537938-84d6-4d4d-8a75-6760810191e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:43.586602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0537938-84d6-4d4d-8a75-6760810191e8 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:43.586889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0537938-84d6-4d4d-8a75-6760810191e8 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:43.587167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0537938-84d6-4d4d-8a75-6760810191e8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:43.594499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0537938-84d6-4d4d-8a75-6760810191e8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:43.595951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0537938-84d6-4d4d-8a75-6760810191e8 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:43.610579 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-d0537938-84d6-4d4d-8a75-6760810191e8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 28 20:45:43.610579 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:43.610579 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:43.610579 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:43.610579 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:43.610579 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:43.610579 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:43.610579 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:43.610579 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:43.610579 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:43.610579 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:43.610579 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:43.610579 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 20:45:43.610579 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 20:45:43.610579 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 20:45:43.610579 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:43.612149 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:43.613318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:43.613318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:43.613318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 20:45:43.613318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 20:45:43.613318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 20:45:43.613318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 20:45:43.613318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 20:45:43.613318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 20:45:43.613318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 28 20:45:43.613318 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:43.613318 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 628/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:43.617693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910544c5-69b7-4c3b-8491-447019fcee5b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:43.622266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:43.717426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910544c5-69b7-4c3b-8491-447019fcee5b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:43.751213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9e79d-ca71-4338-a0f0-6e9281d8d50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:43.751827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9e79d-ca71-4338-a0f0-6e9281d8d50b None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:43.752116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9e79d-ca71-4338-a0f0-6e9281d8d50b None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:43.752396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc9e79d-ca71-4338-a0f0-6e9281d8d50b None admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:43.753232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9e79d-ca71-4338-a0f0-6e9281d8d50b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:43.753544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9e79d-ca71-4338-a0f0-6e9281d8d50b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:43.755075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc9e79d-ca71-4338-a0f0-6e9281d8d50b None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:43.772668 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 653/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Sat May 28 20:45:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 20:45:43.779054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0537938-84d6-4d4d-8a75-6760810191e8 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:43.868880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:43.878050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0537938-84d6-4d4d-8a75-6760810191e8 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:43.910710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849d749-3b72-4506-95ec-394be9ae4f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:43.911348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849d749-3b72-4506-95ec-394be9ae4f74 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:43.911633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849d749-3b72-4506-95ec-394be9ae4f74 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:43.911909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0849d749-3b72-4506-95ec-394be9ae4f74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:43.982453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849d749-3b72-4506-95ec-394be9ae4f74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:43.983894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0849d749-3b72-4506-95ec-394be9ae4f74 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:44.071000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:44.081168 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 629/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Sat May 28 20:45:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 28 20:45:44.086989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9e79d-ca71-4338-a0f0-6e9281d8d50b None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:44.149238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:44.188774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc9e79d-ca71-4338-a0f0-6e9281d8d50b None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:44.224192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ea05e-ee67-49a9-8105-3475d998003e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:44.224985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ea05e-ee67-49a9-8105-3475d998003e None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:44.225267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ea05e-ee67-49a9-8105-3475d998003e None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:44.225541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ea05e-ee67-49a9-8105-3475d998003e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:44.235375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ea05e-ee67-49a9-8105-3475d998003e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:44.236805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0ea05e-ee67-49a9-8105-3475d998003e None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:44.254467 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-3c0ea05e-ee67-49a9-8105-3475d998003e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:44.254467 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:44.254467 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:44.254467 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:44.254467 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:44.254467 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:44.254467 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:44.254467 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:44.254467 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:44.254467 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:44.254467 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:44.254467 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:44.254467 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 20:45:44.254467 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 20:45:44.254467 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 20:45:44.255544 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 20:45:44.255544 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:44.255544 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:44.255544 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:44.255544 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:44.255544 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:44.255544 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:44.255544 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:44.255544 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:45:44.255544 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:44.255544 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:44.255544 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:44.255544 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:44.255544 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:44.255544 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:44.255544 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:44.255544 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:44.256534 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 28 20:45:44.256534 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 20:45:44.256534 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 28 20:45:44.256534 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:45:44.256534 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 654/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:45:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:44.261943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849d749-3b72-4506-95ec-394be9ae4f74 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:44.361082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0849d749-3b72-4506-95ec-394be9ae4f74 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:44.396292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c180db-1c99-458c-8fbd-9e0b91729f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:44.396951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c180db-1c99-458c-8fbd-9e0b91729f4a None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:44.397241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c180db-1c99-458c-8fbd-9e0b91729f4a None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:44.397521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c180db-1c99-458c-8fbd-9e0b91729f4a None admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:44.398277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c180db-1c99-458c-8fbd-9e0b91729f4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:44.398589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c180db-1c99-458c-8fbd-9e0b91729f4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:44.399978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c180db-1c99-458c-8fbd-9e0b91729f4a None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:44.410880 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 630/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Sat May 28 20:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:44.416807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ea05e-ee67-49a9-8105-3475d998003e None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:44.526276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ea05e-ee67-49a9-8105-3475d998003e None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:44.541677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:44.564073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c215ea-038c-44d2-a778-e6cbe2b23f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:44.564883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c215ea-038c-44d2-a778-e6cbe2b23f5e None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:44.565165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c215ea-038c-44d2-a778-e6cbe2b23f5e None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:44.565439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c215ea-038c-44d2-a778-e6cbe2b23f5e None admin] PATH_INFO: `/v3/projects/service` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:44.572960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c215ea-038c-44d2-a778-e6cbe2b23f5e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:44.574512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c215ea-038c-44d2-a778-e6cbe2b23f5e None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:44.590491 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-c7c215ea-038c-44d2-a778-e6cbe2b23f5e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:45:44.590491 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:44.590491 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:44.590491 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:44.590491 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:44.590491 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:44.590491 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:44.590491 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:44.590491 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:44.590491 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:44.590491 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:44.590491 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:44.590491 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 28 20:45:44.590491 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(project_id) May 28 20:45:44.590491 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 28 20:45:44.591665 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 28 20:45:44.591665 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:44.591665 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:44.591665 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:44.591665 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:44.591665 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:44.591665 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:44.591665 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:44.591665 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 20:45:44.591665 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:44.591665 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:44.591665 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:44.591665 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:44.591665 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:44.591665 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:44.591665 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:44.591665 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:44.592808 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 28 20:45:44.592808 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 28 20:45:44.592808 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 20:45:44.592808 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 20:45:44.592808 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 20:45:44.592808 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 20:45:44.592808 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 28 20:45:44.592808 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 20:45:44.592808 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 655/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Sat May 28 20:45:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:44.600032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c180db-1c99-458c-8fbd-9e0b91729f4a None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:44.654385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:44.711905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c180db-1c99-458c-8fbd-9e0b91729f4a None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:44.748524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423217c-4f5c-46fa-950e-3e109bb22640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:44.749200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423217c-4f5c-46fa-950e-3e109bb22640 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:44.749487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423217c-4f5c-46fa-950e-3e109bb22640 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:44.749770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423217c-4f5c-46fa-950e-3e109bb22640 None admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:44.753000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423217c-4f5c-46fa-950e-3e109bb22640 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:44.753000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423217c-4f5c-46fa-950e-3e109bb22640 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:44.753000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423217c-4f5c-46fa-950e-3e109bb22640 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:44.766281 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 631/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Sat May 28 20:45:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:44.772634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c215ea-038c-44d2-a778-e6cbe2b23f5e None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:44.880022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c215ea-038c-44d2-a778-e6cbe2b23f5e None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:44.915152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716eef3b-1ac9-4bda-a458-ef6e9ef350b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:44.915740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716eef3b-1ac9-4bda-a458-ef6e9ef350b5 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:44.916027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716eef3b-1ac9-4bda-a458-ef6e9ef350b5 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:44.916303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716eef3b-1ac9-4bda-a458-ef6e9ef350b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:44.917142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716eef3b-1ac9-4bda-a458-ef6e9ef350b5 None admin] RBAC: Adding query filter params (role.id=7805b93b32f1472f8365936d4d57d49a, scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b, user.id=b02348c479904bac893fa8d71d077550) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:44.917522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716eef3b-1ac9-4bda-a458-ef6e9ef350b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:44.919498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716eef3b-1ac9-4bda-a458-ef6e9ef350b5 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:44.925973 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 656/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Sat May 28 20:45:44 2022] GET /identity/v3/role_assignments?user.id=b02348c479904bac893fa8d71d077550&scope.project.id=3186f9d3c2b341afa8b098ebcaaf974b&role.id=7805b93b32f1472f8365936d4d57d49a => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:47.222850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04680005-47ce-4db0-839b-2959a6794d71 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:47.223407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04680005-47ce-4db0-839b-2959a6794d71 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:47.223407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04680005-47ce-4db0-839b-2959a6794d71 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:47.224764 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 632/1288] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 20:45:47.230064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a29d5-279c-4358-8c7a-8a530a91ce97 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:47.230446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a29d5-279c-4358-8c7a-8a530a91ce97 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:47.230624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524a29d5-279c-4358-8c7a-8a530a91ce97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:47.248020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-524a29d5-279c-4358-8c7a-8a530a91ce97 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:45:47.249219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-524a29d5-279c-4358-8c7a-8a530a91ce97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:45:47.279275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:47.315178 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 657/1289] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:45:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:45:47.338851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04680005-47ce-4db0-839b-2959a6794d71 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:47.438256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04680005-47ce-4db0-839b-2959a6794d71 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:47.448693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:47.470704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eafca5-bcd6-4883-8fe0-08bac157b951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:47.471480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eafca5-bcd6-4883-8fe0-08bac157b951 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:47.471763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eafca5-bcd6-4883-8fe0-08bac157b951 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:47.472037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eafca5-bcd6-4883-8fe0-08bac157b951 None admin] PATH_INFO: `/v3/services/placement` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:47.472738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eafca5-bcd6-4883-8fe0-08bac157b951 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:47.474174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eafca5-bcd6-4883-8fe0-08bac157b951 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:47.486361 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-71eafca5-bcd6-4883-8fe0-08bac157b951 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 28 20:45:47.486361 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:47.486361 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:45:47.486361 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:45:47.486361 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:45:47.486361 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:45:47.486361 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:45:47.486361 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:45:47.486361 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 20:45:47.486361 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:47.486361 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:45:47.486361 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:47.486361 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:47.486361 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:47.486361 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:47.486361 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:47.486361 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:47.487433 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:47.487433 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:47.487433 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:47.487433 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:47.487433 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:47.487433 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:47.487433 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:45:47.487433 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:45:47.487433 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:45:47.487433 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:45:47.487433 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:47.487433 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:47.487433 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:47.487433 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:47.487433 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:47.487433 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:47.488291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:47.488291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:47.488291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:47.488291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:47.488291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:47.488291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:47.488291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:47.488291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:47.488291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:47.488291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:47.488291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:45:47.488291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:45:47.488291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 20:45:47.488291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:47.488291 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 633/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 20:45:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 28 20:45:47.494888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a29d5-279c-4358-8c7a-8a530a91ce97 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:47.592213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524a29d5-279c-4358-8c7a-8a530a91ce97 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:47.625583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c5ac9-8ea8-45e7-9ef2-dbc1adeb34d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:47.626279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c5ac9-8ea8-45e7-9ef2-dbc1adeb34d0 None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:47.626566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c5ac9-8ea8-45e7-9ef2-dbc1adeb34d0 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:47.626847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c5ac9-8ea8-45e7-9ef2-dbc1adeb34d0 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:47.627563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c5ac9-8ea8-45e7-9ef2-dbc1adeb34d0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:47.627873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c5ac9-8ea8-45e7-9ef2-dbc1adeb34d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:47.629467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c5ac9-8ea8-45e7-9ef2-dbc1adeb34d0 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:47.634068 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 658/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Sat May 28 20:45:47 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:47.640245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eafca5-bcd6-4883-8fe0-08bac157b951 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:47.738620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eafca5-bcd6-4883-8fe0-08bac157b951 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:47.772394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc8124-99c5-4227-a4f9-756928d811c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:47.772966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc8124-99c5-4227-a4f9-756928d811c7 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:47.773252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc8124-99c5-4227-a4f9-756928d811c7 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:47.773533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc8124-99c5-4227-a4f9-756928d811c7 None admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:47.774273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc8124-99c5-4227-a4f9-756928d811c7 None admin] RBAC: Adding query filter params (type=placement) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:47.774584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc8124-99c5-4227-a4f9-756928d811c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:47.775956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc8124-99c5-4227-a4f9-756928d811c7 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:47.780413 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 634/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Sat May 28 20:45:47 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:50.050300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36bdd-4063-487d-9928-ab58e8d2e832 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:50.050855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36bdd-4063-487d-9928-ab58e8d2e832 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:50.050855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d36bdd-4063-487d-9928-ab58e8d2e832 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:50.052235 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 659/1293] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:45:50.057325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210609e-9136-4149-af31-ffdb6eabafcb None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:50.057730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210609e-9136-4149-af31-ffdb6eabafcb None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:50.058051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210609e-9136-4149-af31-ffdb6eabafcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:50.074952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9210609e-9136-4149-af31-ffdb6eabafcb None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:45:50.076139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9210609e-9136-4149-af31-ffdb6eabafcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:45:50.106022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:50.142967 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 635/1294] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:45:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:45:50.167127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d36bdd-4063-487d-9928-ab58e8d2e832 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:50.199448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:50.269261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d36bdd-4063-487d-9928-ab58e8d2e832 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:50.302840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432bfd3-1989-4c48-88b8-bc924b137016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:50.303437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432bfd3-1989-4c48-88b8-bc924b137016 None admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:50.303724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432bfd3-1989-4c48-88b8-bc924b137016 None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:50.304002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432bfd3-1989-4c48-88b8-bc924b137016 None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:50.304728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432bfd3-1989-4c48-88b8-bc924b137016 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:50.306122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432bfd3-1989-4c48-88b8-bc924b137016 None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:50.423817 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 660/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Sat May 28 20:45:50 2022] POST /identity/v3/services => generated 246 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:45:52.680054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07531f-ce52-4407-a8ec-228da1deb851 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:52.680586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07531f-ce52-4407-a8ec-228da1deb851 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:52.680586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07531f-ce52-4407-a8ec-228da1deb851 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:52.681988 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 636/1296] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:45:52.687727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46e3ee-e3b5-4d92-a1ef-c4745b5ac5b0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:52.688163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46e3ee-e3b5-4d92-a1ef-c4745b5ac5b0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:52.688313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46e3ee-e3b5-4d92-a1ef-c4745b5ac5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:52.704893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fa46e3ee-e3b5-4d92-a1ef-c4745b5ac5b0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:45:52.706123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fa46e3ee-e3b5-4d92-a1ef-c4745b5ac5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:45:52.776944 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 661/1297] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:45:52 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:45:52.801820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07531f-ce52-4407-a8ec-228da1deb851 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:52.907178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07531f-ce52-4407-a8ec-228da1deb851 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:52.940584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8428b-938b-4075-b464-0c6fbe668559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:52.941308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8428b-938b-4075-b464-0c6fbe668559 None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:52.941592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8428b-938b-4075-b464-0c6fbe668559 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:52.941940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8428b-938b-4075-b464-0c6fbe668559 None admin] PATH_INFO: `/v3/services/placement` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:52.942590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8428b-938b-4075-b464-0c6fbe668559 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:52.943955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8428b-938b-4075-b464-0c6fbe668559 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:52.956337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-4fd8428b-938b-4075-b464-0c6fbe668559 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 28 20:45:52.956337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:52.956337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:45:52.956337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:45:52.956337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:45:52.956337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:45:52.956337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:45:52.956337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:45:52.956337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 20:45:52.956337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:52.956337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:45:52.956337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:52.956337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:52.956337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:52.956337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:52.956337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:52.956337 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:52.957578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:52.957578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:52.957578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:52.957578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:52.957578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:52.957578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:52.957578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:45:52.957578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:45:52.957578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:45:52.957578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:45:52.957578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:52.957578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:52.957578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:52.957578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:52.957578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:52.957578 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:52.958641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:52.958641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:52.958641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:52.958641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:52.958641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:52.958641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:52.958641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:52.958641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:52.958641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:52.958641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:52.958641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:45:52.958641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:45:52.958641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 20:45:52.958641 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:52.958641 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 637/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 20:45:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:52.965132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46e3ee-e3b5-4d92-a1ef-c4745b5ac5b0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:52.983532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:53.067533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46e3ee-e3b5-4d92-a1ef-c4745b5ac5b0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:53.102755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e224a0-444b-483e-a1a1-12c303aeb58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:53.103420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e224a0-444b-483e-a1a1-12c303aeb58a None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:53.103704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e224a0-444b-483e-a1a1-12c303aeb58a None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:53.103982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e224a0-444b-483e-a1a1-12c303aeb58a None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:53.104973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e224a0-444b-483e-a1a1-12c303aeb58a None admin] RBAC: Adding query filter params (name=placement) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:53.105359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e224a0-444b-483e-a1a1-12c303aeb58a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:53.106769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e224a0-444b-483e-a1a1-12c303aeb58a None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:53.111896 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 662/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Sat May 28 20:45:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:53.117796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8428b-938b-4075-b464-0c6fbe668559 None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:53.152999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:53.221189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8428b-938b-4075-b464-0c6fbe668559 None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:53.254010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75a6e3-2ade-47f0-8453-be9c1d734c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:53.254655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75a6e3-2ade-47f0-8453-be9c1d734c7b None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:53.254937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75a6e3-2ade-47f0-8453-be9c1d734c7b None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:53.255210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75a6e3-2ade-47f0-8453-be9c1d734c7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:53.255989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75a6e3-2ade-47f0-8453-be9c1d734c7b None admin] RBAC: Adding query filter params (interface=public, service_id=8cdf74d2d3c54ac3a836224388a659c0, region_id=RegionOne) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:53.256293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75a6e3-2ade-47f0-8453-be9c1d734c7b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:53.257657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75a6e3-2ade-47f0-8453-be9c1d734c7b None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:53.262597 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 638/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Sat May 28 20:45:53 2022] GET /identity/v3/endpoints?service_id=8cdf74d2d3c54ac3a836224388a659c0&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:53.268485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e224a0-444b-483e-a1a1-12c303aeb58a None admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:53.348857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:53.366380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e224a0-444b-483e-a1a1-12c303aeb58a None admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:53.399350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e415442-c968-41bd-a322-742713d4fc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:53.400004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e415442-c968-41bd-a322-742713d4fc4f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:53.400285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e415442-c968-41bd-a322-742713d4fc4f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:53.400558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e415442-c968-41bd-a322-742713d4fc4f None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:53.401237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e415442-c968-41bd-a322-742713d4fc4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:53.403231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e415442-c968-41bd-a322-742713d4fc4f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:53.410234 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 663/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Sat May 28 20:45:53 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 20:45:55.656782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d151fe-e38d-403f-8cc4-47345968b680 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:55.657425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d151fe-e38d-403f-8cc4-47345968b680 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:55.657425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d151fe-e38d-403f-8cc4-47345968b680 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:55.658884 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 639/1302] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:45:55.664003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e4589-c859-4711-8295-af614917a4d0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:55.664443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e4589-c859-4711-8295-af614917a4d0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:55.664537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e4589-c859-4711-8295-af614917a4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:55.680699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7b6e4589-c859-4711-8295-af614917a4d0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:45:55.681905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7b6e4589-c859-4711-8295-af614917a4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:45:55.750249 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 664/1303] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:45:55 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 20:45:55.774395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d151fe-e38d-403f-8cc4-47345968b680 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:55.870887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:55.877485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d151fe-e38d-403f-8cc4-47345968b680 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:55.911486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af392c29-4c55-4859-b644-c15b6340600e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:55.912217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af392c29-4c55-4859-b644-c15b6340600e None admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:55.912502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af392c29-4c55-4859-b644-c15b6340600e None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:55.912778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af392c29-4c55-4859-b644-c15b6340600e None admin] PATH_INFO: `/v3/services/placement` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:55.913442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af392c29-4c55-4859-b644-c15b6340600e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:55.914914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af392c29-4c55-4859-b644-c15b6340600e None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:55.927168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-af392c29-4c55-4859-b644-c15b6340600e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 28 20:45:55.927168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:55.927168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 20:45:55.927168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 20:45:55.927168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 20:45:55.927168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 20:45:55.927168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 20:45:55.927168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:45:55.927168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 20:45:55.927168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:55.927168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 20:45:55.927168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:55.927168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 20:45:55.927168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 20:45:55.927168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 20:45:55.927168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 20:45:55.927168 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 20:45:55.928441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 20:45:55.928441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 20:45:55.928441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 20:45:55.928441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 20:45:55.928441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 20:45:55.928441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 20:45:55.928441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 20:45:55.928441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 20:45:55.928441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 20:45:55.928441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 20:45:55.928441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 20:45:55.928441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 20:45:55.928441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 20:45:55.928441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 20:45:55.928441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 20:45:55.928441 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 20:45:55.929343 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 20:45:55.929343 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 20:45:55.929343 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 20:45:55.929343 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 20:45:55.929343 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 20:45:55.929343 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 20:45:55.929343 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 20:45:55.929343 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 20:45:55.929343 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 20:45:55.929343 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 20:45:55.929343 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 20:45:55.929343 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 20:45:55.929343 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 28 20:45:55.929343 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 20:45:55.929343 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 640/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Sat May 28 20:45:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 20:45:55.935874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e4589-c859-4711-8295-af614917a4d0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:56.011981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:45:56.034430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e4589-c859-4711-8295-af614917a4d0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:56.068035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9219d83-146f-4a9f-a12f-838afa60487f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:56.068701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9219d83-146f-4a9f-a12f-838afa60487f None admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:56.068987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9219d83-146f-4a9f-a12f-838afa60487f None admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:56.069272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9219d83-146f-4a9f-a12f-838afa60487f None admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:56.070008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9219d83-146f-4a9f-a12f-838afa60487f None admin] RBAC: Adding query filter params (name=placement) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:45:56.070311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9219d83-146f-4a9f-a12f-838afa60487f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:56.071697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9219d83-146f-4a9f-a12f-838afa60487f None admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:56.077458 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 665/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Sat May 28 20:45:55 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:45:56.083379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af392c29-4c55-4859-b644-c15b6340600e None admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:45:56.181772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af392c29-4c55-4859-b644-c15b6340600e None admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:45:56.215170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff907eb4-ba59-4151-a06e-6e75f499f430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:45:56.215823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff907eb4-ba59-4151-a06e-6e75f499f430 None admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:45:56.216113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff907eb4-ba59-4151-a06e-6e75f499f430 None admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:45:56.216396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff907eb4-ba59-4151-a06e-6e75f499f430 None admin] PATH_INFO: `/v3/endpoints` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:45:56.217217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff907eb4-ba59-4151-a06e-6e75f499f430 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:45:56.218627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff907eb4-ba59-4151-a06e-6e75f499f430 None admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:45:56.286993 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 641/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Sat May 28 20:45:56 2022] POST /identity/v3/endpoints => generated 331 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:46:06.073739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=30 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:46:06.282408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:51:46.822255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650c8d5-9be1-4bf3-bcf9-8641e5ef01f2 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:51:46.822732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650c8d5-9be1-4bf3-bcf9-8641e5ef01f2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:51:46.822971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650c8d5-9be1-4bf3-bcf9-8641e5ef01f2 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:51:46.828136 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 666/1307] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:51:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:51:46.829452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01de008-1235-44ba-9a60-e39055ebb503 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:51:46.829902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01de008-1235-44ba-9a60-e39055ebb503 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:51:46.830212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01de008-1235-44ba-9a60-e39055ebb503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:51:46.986869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a01de008-1235-44ba-9a60-e39055ebb503 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:51:46.990021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a01de008-1235-44ba-9a60-e39055ebb503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:51:47.133539 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 642/1308] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 20:51:47.192183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b02a2c-fbc0-47b5-89cb-4596c345ca34 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:51:47.192920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b02a2c-fbc0-47b5-89cb-4596c345ca34 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:51:47.193530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b02a2c-fbc0-47b5-89cb-4596c345ca34 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:51:47.196203 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 667/1309] 10.222.0.33 () {54 vars in 964 bytes} [Sat May 28 20:51:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:51:47.203623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dedcb-be4d-4dfa-a667-4ca1c02d369a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:51:47.203944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dedcb-be4d-4dfa-a667-4ca1c02d369a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:51:47.204233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86dedcb-be4d-4dfa-a667-4ca1c02d369a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:51:47.302498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c86dedcb-be4d-4dfa-a667-4ca1c02d369a None None] MFA Rules not processed for user `1b2a9ebcd3f84b758322a80c6e590de0`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:51:47.303397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c86dedcb-be4d-4dfa-a667-4ca1c02d369a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:51:47.328039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:51:47.406490 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 643/1310] 10.222.0.33 () {58 vars in 1061 bytes} [Sat May 28 20:51:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:51:47.594333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b02a2c-fbc0-47b5-89cb-4596c345ca34 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:51:47.594333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c6b02a2c-fbc0-47b5-89cb-4596c345ca34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:51:47.594333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b02a2c-fbc0-47b5-89cb-4596c345ca34 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:51:47.619622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c231f2-c243-4f9c-bbac-0f576273fa36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:51:47.620636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c231f2-c243-4f9c-bbac-0f576273fa36 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:51:47.621110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c231f2-c243-4f9c-bbac-0f576273fa36 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:51:47.621720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c231f2-c243-4f9c-bbac-0f576273fa36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:51:47.623608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c0c231f2-c243-4f9c-bbac-0f576273fa36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:51:47.681456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:51:47.739268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c231f2-c243-4f9c-bbac-0f576273fa36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:51:47.741086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c231f2-c243-4f9c-bbac-0f576273fa36 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:51:47.762381 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 668/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:51:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 20:51:50.369530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d22a33-569f-4bfc-bc23-8f1ef0eeb909 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:51:50.370081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d22a33-569f-4bfc-bc23-8f1ef0eeb909 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:51:50.370175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d22a33-569f-4bfc-bc23-8f1ef0eeb909 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:51:50.371494 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 644/1312] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:51:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:51:50.377722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138b637-9383-4894-b71e-8421f4f0e533 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:51:50.382150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138b637-9383-4894-b71e-8421f4f0e533 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:51:50.382150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138b637-9383-4894-b71e-8421f4f0e533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:51:50.473380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c138b637-9383-4894-b71e-8421f4f0e533 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:51:50.474642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c138b637-9383-4894-b71e-8421f4f0e533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:51:50.578601 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 669/1313] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:51:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:51:50.619142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d22a33-569f-4bfc-bc23-8f1ef0eeb909 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:51:50.620801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c0d22a33-569f-4bfc-bc23-8f1ef0eeb909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:51:50.728484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d22a33-569f-4bfc-bc23-8f1ef0eeb909 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:51:50.799472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:51:50.802164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a936902-e690-47a3-98a2-2999f4576313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:51:50.802857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a936902-e690-47a3-98a2-2999f4576313 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:51:50.803138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a936902-e690-47a3-98a2-2999f4576313 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:51:50.803407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a936902-e690-47a3-98a2-2999f4576313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:51:50.805057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7a936902-e690-47a3-98a2-2999f4576313 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:51:50.917881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a936902-e690-47a3-98a2-2999f4576313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:51:50.919247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a936902-e690-47a3-98a2-2999f4576313 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:51:50.937641 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 645/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:51:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:51:53.732169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176912d-b78c-4b68-b801-52d2f52b18c7 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:51:53.732716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176912d-b78c-4b68-b801-52d2f52b18c7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:51:53.732716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d176912d-b78c-4b68-b801-52d2f52b18c7 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:51:53.734155 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 670/1315] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:51:53.739851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407427-8199-4d49-9f96-a4085de356b6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:51:53.740284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407427-8199-4d49-9f96-a4085de356b6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:51:53.740456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407427-8199-4d49-9f96-a4085de356b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:51:53.805940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b5407427-8199-4d49-9f96-a4085de356b6 None None] MFA Rules not processed for user `6edbb2f56e344bc2b039e08db2a25dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:51:53.807156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b5407427-8199-4d49-9f96-a4085de356b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:51:53.857369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:51:53.901261 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 646/1316] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:51:53 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:51:53.943816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176912d-b78c-4b68-b801-52d2f52b18c7 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:51:53.945341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d176912d-b78c-4b68-b801-52d2f52b18c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:51:53.955163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:51:54.046999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d176912d-b78c-4b68-b801-52d2f52b18c7 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:51:54.119364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf0761-66af-4332-8fc7-a27446dde4ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:51:54.120003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf0761-66af-4332-8fc7-a27446dde4ab service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:51:54.120282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf0761-66af-4332-8fc7-a27446dde4ab service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:51:54.120554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf0761-66af-4332-8fc7-a27446dde4ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:51:54.122222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8cbf0761-66af-4332-8fc7-a27446dde4ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:51:54.182817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:51:54.228973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf0761-66af-4332-8fc7-a27446dde4ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:51:54.230461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf0761-66af-4332-8fc7-a27446dde4ab service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:51:54.248526 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 671/1317] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:51:53 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:51:57.375215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0c1cd-2a44-4a73-ac0c-d1bf16a53356 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:51:57.375735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0c1cd-2a44-4a73-ac0c-d1bf16a53356 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:51:57.375735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe0c1cd-2a44-4a73-ac0c-d1bf16a53356 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:51:57.377057 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 647/1318] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:51:57.383167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa381f98-57a3-465d-b398-9ac2fab78b0e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:51:57.383550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa381f98-57a3-465d-b398-9ac2fab78b0e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:51:57.383723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa381f98-57a3-465d-b398-9ac2fab78b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:51:57.447640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-aa381f98-57a3-465d-b398-9ac2fab78b0e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:51:57.448858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-aa381f98-57a3-465d-b398-9ac2fab78b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:51:57.542803 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 672/1319] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:51:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:51:57.595861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43eb73-9b67-462d-94a0-e1e9f2353f6b None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:51:57.596663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43eb73-9b67-462d-94a0-e1e9f2353f6b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:51:57.596942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43eb73-9b67-462d-94a0-e1e9f2353f6b None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:51:57.598364 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 648/1320] 10.222.0.33 () {54 vars in 964 bytes} [Sat May 28 20:51:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:51:57.603873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109ae5c-7537-4590-b628-369eee9e7424 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:51:57.604174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109ae5c-7537-4590-b628-369eee9e7424 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:51:57.604457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9109ae5c-7537-4590-b628-369eee9e7424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:51:57.624994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:51:57.677682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9109ae5c-7537-4590-b628-369eee9e7424 None None] MFA Rules not processed for user `1b2a9ebcd3f84b758322a80c6e590de0`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:51:57.679572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9109ae5c-7537-4590-b628-369eee9e7424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:51:57.754063 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 673/1321] 10.222.0.33 () {58 vars in 1061 bytes} [Sat May 28 20:51:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:51:57.763342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43eb73-9b67-462d-94a0-e1e9f2353f6b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:51:57.764855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0a43eb73-9b67-462d-94a0-e1e9f2353f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:51:57.866513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a43eb73-9b67-462d-94a0-e1e9f2353f6b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:51:57.938369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7165450e-0762-41f2-86aa-c05bdba058e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:51:57.938946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165450e-0762-41f2-86aa-c05bdba058e7 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:51:57.939223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165450e-0762-41f2-86aa-c05bdba058e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:51:57.939494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7165450e-0762-41f2-86aa-c05bdba058e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:51:57.941153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7165450e-0762-41f2-86aa-c05bdba058e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:51:57.948448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:51:58.048149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165450e-0762-41f2-86aa-c05bdba058e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:51:58.049687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7165450e-0762-41f2-86aa-c05bdba058e7 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:51:58.067352 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 649/1322] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:51:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:52:01.747757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7ffa5-cc3d-471e-91f9-c93da04f69d4 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:01.748318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7ffa5-cc3d-471e-91f9-c93da04f69d4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:01.748318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7ffa5-cc3d-471e-91f9-c93da04f69d4 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:01.749698 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 674/1323] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:01.755637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d2956-bcc1-4a76-97df-f0deb17bf5da None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:01.756304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d2956-bcc1-4a76-97df-f0deb17bf5da None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:01.756449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d2956-bcc1-4a76-97df-f0deb17bf5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:01.820532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d96d2956-bcc1-4a76-97df-f0deb17bf5da None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:01.821736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d96d2956-bcc1-4a76-97df-f0deb17bf5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:01.879115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:52:01.918592 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 650/1324] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:52:01.956785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7ffa5-cc3d-471e-91f9-c93da04f69d4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:52:01.958383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ddb7ffa5-cc3d-471e-91f9-c93da04f69d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:02.062260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7ffa5-cc3d-471e-91f9-c93da04f69d4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:52:02.074916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:52:02.136580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e2a62-6a29-4832-86ba-cde6aca0b278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:52:02.137232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e2a62-6a29-4832-86ba-cde6aca0b278 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:02.137518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e2a62-6a29-4832-86ba-cde6aca0b278 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:02.137796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e2a62-6a29-4832-86ba-cde6aca0b278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:02.139489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-305e2a62-6a29-4832-86ba-cde6aca0b278 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:02.251183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e2a62-6a29-4832-86ba-cde6aca0b278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:52:02.252782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e2a62-6a29-4832-86ba-cde6aca0b278 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:52:02.270845 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 675/1325] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:52:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:52:04.603173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa5356-1000-4df7-8501-ec922d1b07fd None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:04.603708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa5356-1000-4df7-8501-ec922d1b07fd None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:04.603708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa5356-1000-4df7-8501-ec922d1b07fd None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:04.605111 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 651/1326] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:52:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:04.610811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af714f99-0c46-448d-8704-ea55214ed5ed None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:04.611240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af714f99-0c46-448d-8704-ea55214ed5ed None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:04.612436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af714f99-0c46-448d-8704-ea55214ed5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:04.659589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:52:04.678170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-af714f99-0c46-448d-8704-ea55214ed5ed None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:04.679387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-af714f99-0c46-448d-8704-ea55214ed5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:04.779182 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 676/1327] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:52:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:52:04.817495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa5356-1000-4df7-8501-ec922d1b07fd None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:52:04.819096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c3fa5356-1000-4df7-8501-ec922d1b07fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:04.923386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa5356-1000-4df7-8501-ec922d1b07fd None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:52:04.998713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd733c27-f413-4fda-8815-5e6c006157c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:52:04.999360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd733c27-f413-4fda-8815-5e6c006157c7 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:04.999645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd733c27-f413-4fda-8815-5e6c006157c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:04.999923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd733c27-f413-4fda-8815-5e6c006157c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:05.001576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fd733c27-f413-4fda-8815-5e6c006157c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:05.019891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:52:05.118590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd733c27-f413-4fda-8815-5e6c006157c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:52:05.120028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd733c27-f413-4fda-8815-5e6c006157c7 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:52:05.139128 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 652/1328] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:52:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:52:07.445479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ef090-61e4-463f-9c1e-4dd5fef7501e None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:07.446041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ef090-61e4-463f-9c1e-4dd5fef7501e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:07.446041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ef090-61e4-463f-9c1e-4dd5fef7501e None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:07.447369 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 677/1329] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:07.452539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c127836-47b1-4c78-9520-2afed087ebce None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:07.452983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c127836-47b1-4c78-9520-2afed087ebce None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:07.453139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c127836-47b1-4c78-9520-2afed087ebce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:07.516726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4c127836-47b1-4c78-9520-2afed087ebce None None] MFA Rules not processed for user `6edbb2f56e344bc2b039e08db2a25dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:07.518567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4c127836-47b1-4c78-9520-2afed087ebce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:07.582749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:52:07.613462 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 653/1330] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:52:07 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 20:52:07.653187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ef090-61e4-463f-9c1e-4dd5fef7501e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:52:07.654963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f95ef090-61e4-463f-9c1e-4dd5fef7501e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:07.759286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ef090-61e4-463f-9c1e-4dd5fef7501e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:52:07.779015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:52:07.833076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc038c94-314d-4e6b-b5a0-a2b3bf067235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:52:07.833754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc038c94-314d-4e6b-b5a0-a2b3bf067235 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:07.834070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc038c94-314d-4e6b-b5a0-a2b3bf067235 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:07.834347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc038c94-314d-4e6b-b5a0-a2b3bf067235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:07.836030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dc038c94-314d-4e6b-b5a0-a2b3bf067235 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:07.950069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc038c94-314d-4e6b-b5a0-a2b3bf067235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:52:07.951547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc038c94-314d-4e6b-b5a0-a2b3bf067235 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:52:07.970073 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 678/1331] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:52:07 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:52:10.696383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f3268-b039-4839-b9e3-c230a0e607a9 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:10.696901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f3268-b039-4839-b9e3-c230a0e607a9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:10.697012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f3268-b039-4839-b9e3-c230a0e607a9 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:10.698354 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 654/1332] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:10.703958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f217f76-14c6-4c6f-90b2-4dd29dd80a08 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:10.704420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f217f76-14c6-4c6f-90b2-4dd29dd80a08 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:10.704554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f217f76-14c6-4c6f-90b2-4dd29dd80a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:10.774760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:52:10.777516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9f217f76-14c6-4c6f-90b2-4dd29dd80a08 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:10.778774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9f217f76-14c6-4c6f-90b2-4dd29dd80a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:10.875466 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 679/1333] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:52:10.912869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f3268-b039-4839-b9e3-c230a0e607a9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:52:10.914444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-599f3268-b039-4839-b9e3-c230a0e607a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:11.021126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599f3268-b039-4839-b9e3-c230a0e607a9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:52:11.098899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57126c77-668d-489b-ab2c-3c68939972e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:52:11.099560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57126c77-668d-489b-ab2c-3c68939972e5 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:11.099845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57126c77-668d-489b-ab2c-3c68939972e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:11.100123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57126c77-668d-489b-ab2c-3c68939972e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:11.101758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-57126c77-668d-489b-ab2c-3c68939972e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:11.130240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:52:11.214979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57126c77-668d-489b-ab2c-3c68939972e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:52:11.216454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57126c77-668d-489b-ab2c-3c68939972e5 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:52:11.239284 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 655/1334] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:52:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:52:13.587880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38484b28-3563-4974-afb0-a05b5e000631 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:13.588456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38484b28-3563-4974-afb0-a05b5e000631 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:13.588456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38484b28-3563-4974-afb0-a05b5e000631 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:13.589804 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 680/1335] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:13.595062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28ae17-af70-4271-aed8-6908a36a468e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:13.595453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28ae17-af70-4271-aed8-6908a36a468e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:13.595660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28ae17-af70-4271-aed8-6908a36a468e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:13.660183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8b28ae17-af70-4271-aed8-6908a36a468e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:13.661402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8b28ae17-af70-4271-aed8-6908a36a468e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:13.735517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:52:13.758761 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 656/1336] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:52:13.800978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38484b28-3563-4974-afb0-a05b5e000631 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:52:13.802533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-38484b28-3563-4974-afb0-a05b5e000631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:13.903570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38484b28-3563-4974-afb0-a05b5e000631 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:52:13.931144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:52:13.975066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565428ae-9263-44f5-8b1d-fede8dabba39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:52:13.975705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565428ae-9263-44f5-8b1d-fede8dabba39 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:13.975987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565428ae-9263-44f5-8b1d-fede8dabba39 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:13.976263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565428ae-9263-44f5-8b1d-fede8dabba39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:13.977941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-565428ae-9263-44f5-8b1d-fede8dabba39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:14.084313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565428ae-9263-44f5-8b1d-fede8dabba39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:52:14.085754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565428ae-9263-44f5-8b1d-fede8dabba39 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:52:14.103761 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 681/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:52:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:52:16.940676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98854b9-301f-4fe3-892f-b86cd2a85cc6 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:16.941238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98854b9-301f-4fe3-892f-b86cd2a85cc6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:16.941238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98854b9-301f-4fe3-892f-b86cd2a85cc6 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:16.942712 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 657/1338] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:52:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:16.947922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7875b-952b-46bf-a742-6a1ffd288d13 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:16.948370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7875b-952b-46bf-a742-6a1ffd288d13 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:16.948482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb7875b-952b-46bf-a742-6a1ffd288d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:17.014445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3eb7875b-952b-46bf-a742-6a1ffd288d13 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:17.015645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3eb7875b-952b-46bf-a742-6a1ffd288d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:17.024064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:52:17.116942 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 682/1339] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:52:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:52:17.153503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98854b9-301f-4fe3-892f-b86cd2a85cc6 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:52:17.156877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c98854b9-301f-4fe3-892f-b86cd2a85cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:17.267645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98854b9-301f-4fe3-892f-b86cd2a85cc6 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:52:17.271288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a0953-6527-4961-be0a-3b1d62f61794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:52:17.271960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a0953-6527-4961-be0a-3b1d62f61794 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:17.272245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a0953-6527-4961-be0a-3b1d62f61794 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:17.272528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a0953-6527-4961-be0a-3b1d62f61794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:17.274378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a92a0953-6527-4961-be0a-3b1d62f61794 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:17.378706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a0953-6527-4961-be0a-3b1d62f61794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:52:17.380151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a0953-6527-4961-be0a-3b1d62f61794 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:52:17.390432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:52:17.399510 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 658/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:52:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:52:22.006139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c44d5-3a83-48bb-8d76-ce82f37c18a5 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:22.006706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c44d5-3a83-48bb-8d76-ce82f37c18a5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:22.006706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c44d5-3a83-48bb-8d76-ce82f37c18a5 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:22.008015 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 683/1341] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:22.013320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a3697-8c2e-43f9-a13c-65c219c0c510 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:22.013787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a3697-8c2e-43f9-a13c-65c219c0c510 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:22.014030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94a3697-8c2e-43f9-a13c-65c219c0c510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:22.061389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c94a3697-8c2e-43f9-a13c-65c219c0c510 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:22.062539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c94a3697-8c2e-43f9-a13c-65c219c0c510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:22.138460 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 659/1342] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:52:22.179716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c44d5-3a83-48bb-8d76-ce82f37c18a5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:52:22.191410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c44d5-3a83-48bb-8d76-ce82f37c18a5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:52:22.194912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d150cc-be74-4097-b912-7aa62e07e1f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:52:22.195546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d150cc-be74-4097-b912-7aa62e07e1f1 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:22.195830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d150cc-be74-4097-b912-7aa62e07e1f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:22.196106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d150cc-be74-4097-b912-7aa62e07e1f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:22.204812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d150cc-be74-4097-b912-7aa62e07e1f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:52:22.206790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d150cc-be74-4097-b912-7aa62e07e1f1 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:52:22.216231 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 684/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:52:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:52:25.477041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78612504-d225-4c30-b4c9-695d80e54b55 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:25.477634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78612504-d225-4c30-b4c9-695d80e54b55 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:25.477634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78612504-d225-4c30-b4c9-695d80e54b55 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:25.479151 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 660/1344] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:25.484502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c082773-3bd4-44e5-acfb-e9da637495a5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:25.484933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c082773-3bd4-44e5-acfb-e9da637495a5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:25.485139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c082773-3bd4-44e5-acfb-e9da637495a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:25.513643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9c082773-3bd4-44e5-acfb-e9da637495a5 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:25.514887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9c082773-3bd4-44e5-acfb-e9da637495a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:25.523439 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 685/1345] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:52:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 20:52:25.560735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78612504-d225-4c30-b4c9-695d80e54b55 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:52:25.572116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78612504-d225-4c30-b4c9-695d80e54b55 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:52:25.576036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c54b7-ca2e-4254-a6ac-cb55429098e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:52:25.576679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c54b7-ca2e-4254-a6ac-cb55429098e6 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:25.576967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c54b7-ca2e-4254-a6ac-cb55429098e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:25.577244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c54b7-ca2e-4254-a6ac-cb55429098e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:25.587235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c54b7-ca2e-4254-a6ac-cb55429098e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:52:25.588677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c54b7-ca2e-4254-a6ac-cb55429098e6 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:52:25.598293 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 661/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:52:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:52:30.083769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195273f9-b401-467e-bd39-2d7748696b84 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:30.084315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195273f9-b401-467e-bd39-2d7748696b84 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:30.084315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195273f9-b401-467e-bd39-2d7748696b84 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:30.085716 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 686/1347] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:30.091027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb181fe-076f-454d-b0c2-cf22dc8070e6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:30.091461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb181fe-076f-454d-b0c2-cf22dc8070e6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:30.091565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb181fe-076f-454d-b0c2-cf22dc8070e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:30.109296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-aeb181fe-076f-454d-b0c2-cf22dc8070e6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:30.111184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-aeb181fe-076f-454d-b0c2-cf22dc8070e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:30.119524 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 662/1348] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:52:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:52:30.159777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195273f9-b401-467e-bd39-2d7748696b84 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:52:30.170711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195273f9-b401-467e-bd39-2d7748696b84 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:52:30.174335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3589b2-c281-437d-af39-1bc11373f863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:52:30.175458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3589b2-c281-437d-af39-1bc11373f863 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:30.175743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3589b2-c281-437d-af39-1bc11373f863 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:30.176023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3589b2-c281-437d-af39-1bc11373f863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:30.184931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3589b2-c281-437d-af39-1bc11373f863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:52:30.186410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3589b2-c281-437d-af39-1bc11373f863 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:52:30.196317 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 687/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:52:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:52:35.698430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938f7f1-4414-4ec0-8950-23eca446d729 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:35.699305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938f7f1-4414-4ec0-8950-23eca446d729 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:35.699755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938f7f1-4414-4ec0-8950-23eca446d729 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:35.701616 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 663/1350] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 20:52:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:35.707862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdd059-d366-4916-a8b3-045f8b3f9e54 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:35.708279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdd059-d366-4916-a8b3-045f8b3f9e54 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:35.708424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdd059-d366-4916-a8b3-045f8b3f9e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:35.785424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fffdd059-d366-4916-a8b3-045f8b3f9e54 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:35.786670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fffdd059-d366-4916-a8b3-045f8b3f9e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:35.840934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:52:35.879507 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 688/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 20:52:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:52:40.103931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=56 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:52:40.189855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980deda9-d41a-4f2c-a2c4-9e5ff4eed904 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:40.190227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980deda9-d41a-4f2c-a2c4-9e5ff4eed904 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:40.190484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980deda9-d41a-4f2c-a2c4-9e5ff4eed904 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:40.192120 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 664/1352] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 20:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:40.287096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27c801-c245-4c39-bb8d-7dcbef97b571 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:40.287621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27c801-c245-4c39-bb8d-7dcbef97b571 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:40.287735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d27c801-c245-4c39-bb8d-7dcbef97b571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:40.305799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9d27c801-c245-4c39-bb8d-7dcbef97b571 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:40.309067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9d27c801-c245-4c39-bb8d-7dcbef97b571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:40.318061 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 689/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 20:52:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:52:41.908337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f76a-e5c6-4a24-bf80-e056cd417b40 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:41.908869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f76a-e5c6-4a24-bf80-e056cd417b40 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:41.908978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205f76a-e5c6-4a24-bf80-e056cd417b40 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:41.910328 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 665/1354] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 20:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:41.915186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88685bf8-99e2-492e-8c6f-b6878f76dd99 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:41.915599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88685bf8-99e2-492e-8c6f-b6878f76dd99 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:41.915742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88685bf8-99e2-492e-8c6f-b6878f76dd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:41.933864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-88685bf8-99e2-492e-8c6f-b6878f76dd99 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:41.935089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-88685bf8-99e2-492e-8c6f-b6878f76dd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:41.944165 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 690/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 20:52:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 20:52:45.969957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd357b0-7462-4944-80cd-251397ab25b0 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:45.970889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd357b0-7462-4944-80cd-251397ab25b0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:45.971209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd357b0-7462-4944-80cd-251397ab25b0 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:45.972580 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 666/1356] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:52:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:45.977705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe2c20-0312-4418-83ea-68e17a01a0b7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:45.978165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe2c20-0312-4418-83ea-68e17a01a0b7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:45.979930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe2c20-0312-4418-83ea-68e17a01a0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:45.997069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2bfe2c20-0312-4418-83ea-68e17a01a0b7 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:45.998320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2bfe2c20-0312-4418-83ea-68e17a01a0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:46.007110 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 691/1357] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:52:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:52:46.072500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c4564-8605-4c22-96ea-3140a674442d None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:46.072790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c4564-8605-4c22-96ea-3140a674442d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:46.073073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c4564-8605-4c22-96ea-3140a674442d None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:46.074602 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 667/1358] 10.222.0.33 () {54 vars in 961 bytes} [Sat May 28 20:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:46.081666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c363fe3-314a-450a-aaad-3b030165f457 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:46.082227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c363fe3-314a-450a-aaad-3b030165f457 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:46.082745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c363fe3-314a-450a-aaad-3b030165f457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:46.142068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0c363fe3-314a-450a-aaad-3b030165f457 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:46.143569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0c363fe3-314a-450a-aaad-3b030165f457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:46.229422 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 692/1359] 10.222.0.33 () {58 vars in 1058 bytes} [Sat May 28 20:52:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:52:46.238778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c4564-8605-4c22-96ea-3140a674442d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:52:46.254373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c4564-8605-4c22-96ea-3140a674442d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:52:46.260399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e865ca6-e2a5-4dd5-84b9-42658f867523 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:52:46.261160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e865ca6-e2a5-4dd5-84b9-42658f867523 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:46.261542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e865ca6-e2a5-4dd5-84b9-42658f867523 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:46.261946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e865ca6-e2a5-4dd5-84b9-42658f867523 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:46.271178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e865ca6-e2a5-4dd5-84b9-42658f867523 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:52:46.272888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e865ca6-e2a5-4dd5-84b9-42658f867523 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:52:46.284720 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 668/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:52:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:52:48.785157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8349c-593d-48da-9031-83bb0b830d40 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:48.785681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8349c-593d-48da-9031-83bb0b830d40 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:48.785801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db8349c-593d-48da-9031-83bb0b830d40 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:48.789954 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 693/1361] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:48.792645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368aeec-99f5-4772-a44d-0bbdf37edc7b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:48.793054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368aeec-99f5-4772-a44d-0bbdf37edc7b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:48.793239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d368aeec-99f5-4772-a44d-0bbdf37edc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:48.811689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d368aeec-99f5-4772-a44d-0bbdf37edc7b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:48.813194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d368aeec-99f5-4772-a44d-0bbdf37edc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:48.822300 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 669/1362] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:52:48.869118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8349c-593d-48da-9031-83bb0b830d40 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:52:48.885693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db8349c-593d-48da-9031-83bb0b830d40 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:52:48.891557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f44d0f-4093-41c6-9a52-8a41afa907de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:52:48.892251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f44d0f-4093-41c6-9a52-8a41afa907de service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:48.892523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f44d0f-4093-41c6-9a52-8a41afa907de service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:48.892801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f44d0f-4093-41c6-9a52-8a41afa907de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:48.901555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f44d0f-4093-41c6-9a52-8a41afa907de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:52:48.903495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f44d0f-4093-41c6-9a52-8a41afa907de service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:52:48.913884 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 694/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:52:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:52:51.727045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26f908-4ed5-49e6-8bf0-bb8056eda917 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:51.727045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26f908-4ed5-49e6-8bf0-bb8056eda917 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:51.727045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26f908-4ed5-49e6-8bf0-bb8056eda917 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:51.728221 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 670/1364] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:51.737859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f18120-896c-4e4a-b94a-315d180d1f33 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:51.737859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f18120-896c-4e4a-b94a-315d180d1f33 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:51.737859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f18120-896c-4e4a-b94a-315d180d1f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:51.761661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-91f18120-896c-4e4a-b94a-315d180d1f33 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:51.762930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-91f18120-896c-4e4a-b94a-315d180d1f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:51.771884 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 695/1365] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:52:51.820724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f16d1-6776-40e4-82d2-4af77d2ece46 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:51.821066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f16d1-6776-40e4-82d2-4af77d2ece46 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:51.821356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f16d1-6776-40e4-82d2-4af77d2ece46 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:51.823034 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 671/1366] 10.222.0.33 () {54 vars in 961 bytes} [Sat May 28 20:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:51.828265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40739d-06a1-437b-853d-5433e0cdd5d1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:51.831087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40739d-06a1-437b-853d-5433e0cdd5d1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:51.831087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40739d-06a1-437b-853d-5433e0cdd5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:51.850834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cc40739d-06a1-437b-853d-5433e0cdd5d1 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:51.852071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cc40739d-06a1-437b-853d-5433e0cdd5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:51.861345 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 696/1367] 10.222.0.33 () {58 vars in 1058 bytes} [Sat May 28 20:52:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:52:51.869893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f16d1-6776-40e4-82d2-4af77d2ece46 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:52:51.884797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f16d1-6776-40e4-82d2-4af77d2ece46 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:52:51.890305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8074305f-9a07-4cb5-9a58-ccfe814c3f9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:52:51.891003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074305f-9a07-4cb5-9a58-ccfe814c3f9a service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:51.891351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074305f-9a07-4cb5-9a58-ccfe814c3f9a service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:51.891706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8074305f-9a07-4cb5-9a58-ccfe814c3f9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:51.899816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074305f-9a07-4cb5-9a58-ccfe814c3f9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:52:51.901464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8074305f-9a07-4cb5-9a58-ccfe814c3f9a service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:52:51.910867 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 672/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:52:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 20:52:54.448882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76e6a5-97b5-4b8a-bbd5-de48be5c4dd6 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:54.449420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76e6a5-97b5-4b8a-bbd5-de48be5c4dd6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:54.449420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76e6a5-97b5-4b8a-bbd5-de48be5c4dd6 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:54.451168 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 697/1369] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:54.457214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7781660-4534-40ab-a82e-1a05471777fa None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:54.458178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7781660-4534-40ab-a82e-1a05471777fa None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:54.458681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7781660-4534-40ab-a82e-1a05471777fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:54.487330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f7781660-4534-40ab-a82e-1a05471777fa None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:54.488600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f7781660-4534-40ab-a82e-1a05471777fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:54.501405 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 673/1370] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:52:54.557726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76e6a5-97b5-4b8a-bbd5-de48be5c4dd6 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:52:54.573496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76e6a5-97b5-4b8a-bbd5-de48be5c4dd6 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:52:54.579377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c396dc1-29e2-428b-ba14-617a041a6497 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:52:54.580287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c396dc1-29e2-428b-ba14-617a041a6497 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:54.580826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c396dc1-29e2-428b-ba14-617a041a6497 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:54.581260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c396dc1-29e2-428b-ba14-617a041a6497 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:54.591079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c396dc1-29e2-428b-ba14-617a041a6497 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:52:54.593052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c396dc1-29e2-428b-ba14-617a041a6497 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:52:54.616205 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 698/1371] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:52:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:52:57.353398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c6d1d-b5a9-4769-b543-e8f0dc3e03ac None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:57.368530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c6d1d-b5a9-4769-b543-e8f0dc3e03ac None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:57.369096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c6d1d-b5a9-4769-b543-e8f0dc3e03ac None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:57.377101 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 674/1372] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:52:57 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:57.383059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804ca66-2c2d-4f2e-92d9-53b3329e6dd2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:57.383059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804ca66-2c2d-4f2e-92d9-53b3329e6dd2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:57.383059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804ca66-2c2d-4f2e-92d9-53b3329e6dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:57.406010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5804ca66-2c2d-4f2e-92d9-53b3329e6dd2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:57.407175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5804ca66-2c2d-4f2e-92d9-53b3329e6dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:57.416307 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 699/1373] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 20:52:57.478377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c6d1d-b5a9-4769-b543-e8f0dc3e03ac None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:52:57.508587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c6d1d-b5a9-4769-b543-e8f0dc3e03ac None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:52:57.520737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d2f03-6e5d-4a92-a1eb-6b28fe69c333 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:52:57.521670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d2f03-6e5d-4a92-a1eb-6b28fe69c333 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:57.521979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d2f03-6e5d-4a92-a1eb-6b28fe69c333 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:57.522314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d2f03-6e5d-4a92-a1eb-6b28fe69c333 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:57.531846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d2f03-6e5d-4a92-a1eb-6b28fe69c333 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:52:57.534396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d2f03-6e5d-4a92-a1eb-6b28fe69c333 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:52:57.546829 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 675/1374] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:52:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:52:58.370159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ced8a-5554-4004-b0f4-bc3afc6a0d20 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:58.371044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ced8a-5554-4004-b0f4-bc3afc6a0d20 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:58.371044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18ced8a-5554-4004-b0f4-bc3afc6a0d20 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:58.372028 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 700/1375] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:58.378165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4210e82-cac7-49d2-ab2e-046b2ba24c11 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:58.380671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4210e82-cac7-49d2-ab2e-046b2ba24c11 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:58.380671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4210e82-cac7-49d2-ab2e-046b2ba24c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:58.397457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e4210e82-cac7-49d2-ab2e-046b2ba24c11 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:58.398690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e4210e82-cac7-49d2-ab2e-046b2ba24c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:58.406784 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 676/1376] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:52:58.748711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232cc4f-cc31-4e5d-ac21-bc98be4414aa None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:58.748711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232cc4f-cc31-4e5d-ac21-bc98be4414aa None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:58.748711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3232cc4f-cc31-4e5d-ac21-bc98be4414aa None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:58.750792 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 701/1377] 10.222.0.33 () {54 vars in 963 bytes} [Sat May 28 20:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:52:58.759253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda103a-34a2-48bf-858e-806ab32591d3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:58.759253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda103a-34a2-48bf-858e-806ab32591d3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:58.759253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda103a-34a2-48bf-858e-806ab32591d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:58.810019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7fda103a-34a2-48bf-858e-806ab32591d3 None None] MFA Rules not processed for user `afc08d15f24e46e29c34d618d955bd28`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:52:58.811244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7fda103a-34a2-48bf-858e-806ab32591d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:52:58.873136 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 677/1378] 10.222.0.33 () {58 vars in 1060 bytes} [Sat May 28 20:52:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:52:58.881559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3232cc4f-cc31-4e5d-ac21-bc98be4414aa None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:52:58.892291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3232cc4f-cc31-4e5d-ac21-bc98be4414aa None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:52:58.896449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149005d2-8fc0-40d0-96f8-da35dede49f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:52:58.897088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149005d2-8fc0-40d0-96f8-da35dede49f2 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:52:58.897371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149005d2-8fc0-40d0-96f8-da35dede49f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:52:58.897648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149005d2-8fc0-40d0-96f8-da35dede49f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:52:58.907048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149005d2-8fc0-40d0-96f8-da35dede49f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:52:58.909110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149005d2-8fc0-40d0-96f8-da35dede49f2 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:52:58.921115 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 702/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:00.473662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3186fe-e105-46f7-acbf-9558ab979de9 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:00.474315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3186fe-e105-46f7-acbf-9558ab979de9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:00.474315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3186fe-e105-46f7-acbf-9558ab979de9 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:00.475968 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 678/1380] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:00.480969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3f1ce-fc77-4838-b010-d68bf7905b52 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:00.481461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3f1ce-fc77-4838-b010-d68bf7905b52 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:00.481551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3f1ce-fc77-4838-b010-d68bf7905b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:00.500270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4cc3f1ce-fc77-4838-b010-d68bf7905b52 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:00.501473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4cc3f1ce-fc77-4838-b010-d68bf7905b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:00.510465 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 703/1381] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:53:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:00.558298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3186fe-e105-46f7-acbf-9558ab979de9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:00.573813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3186fe-e105-46f7-acbf-9558ab979de9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:00.580141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535d9a1-1963-4c01-923c-1e3c357d5ce0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:00.581813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535d9a1-1963-4c01-923c-1e3c357d5ce0 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:00.581813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535d9a1-1963-4c01-923c-1e3c357d5ce0 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:00.581813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535d9a1-1963-4c01-923c-1e3c357d5ce0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:00.591505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535d9a1-1963-4c01-923c-1e3c357d5ce0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:00.593214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535d9a1-1963-4c01-923c-1e3c357d5ce0 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:00.605002 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 679/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:03.196712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c92bdd-5a3c-4f80-923e-5262ccd7c181 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:03.197260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c92bdd-5a3c-4f80-923e-5262ccd7c181 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:03.197260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c92bdd-5a3c-4f80-923e-5262ccd7c181 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:03.199311 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 704/1383] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:53:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:03.205665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907aa3b7-a9a1-4cb7-8822-956909cea6e9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:03.206700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907aa3b7-a9a1-4cb7-8822-956909cea6e9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:03.210703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907aa3b7-a9a1-4cb7-8822-956909cea6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:03.228307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-907aa3b7-a9a1-4cb7-8822-956909cea6e9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:03.230098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-907aa3b7-a9a1-4cb7-8822-956909cea6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:03.241949 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 680/1384] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 20:53:03.287071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c92bdd-5a3c-4f80-923e-5262ccd7c181 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:03.304961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:53:03.311223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c92bdd-5a3c-4f80-923e-5262ccd7c181 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:03.317104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bda9f2e-8df0-4ec6-906b-85052b995066 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:03.317766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda9f2e-8df0-4ec6-906b-85052b995066 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:03.318149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda9f2e-8df0-4ec6-906b-85052b995066 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:03.318484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bda9f2e-8df0-4ec6-906b-85052b995066 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:03.329337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda9f2e-8df0-4ec6-906b-85052b995066 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:03.331751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bda9f2e-8df0-4ec6-906b-85052b995066 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:03.344851 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 705/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:05.661707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c29fb-5da6-4f53-aa70-cf2586259fe4 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:05.662340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c29fb-5da6-4f53-aa70-cf2586259fe4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:05.662340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c29fb-5da6-4f53-aa70-cf2586259fe4 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:05.663587 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 681/1386] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:05.669121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111d26b-f3f2-4541-a93e-5621bc25e9b7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:05.669582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111d26b-f3f2-4541-a93e-5621bc25e9b7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:05.669682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111d26b-f3f2-4541-a93e-5621bc25e9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:05.691179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7111d26b-f3f2-4541-a93e-5621bc25e9b7 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:05.692884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7111d26b-f3f2-4541-a93e-5621bc25e9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:05.701922 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 706/1387] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:53:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:05.759054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c29fb-5da6-4f53-aa70-cf2586259fe4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:05.774365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c29fb-5da6-4f53-aa70-cf2586259fe4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:05.780137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bec13e-5c61-46a2-a551-2e433a28c16e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:05.784124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bec13e-5c61-46a2-a551-2e433a28c16e service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:05.784124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bec13e-5c61-46a2-a551-2e433a28c16e service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:05.784124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bec13e-5c61-46a2-a551-2e433a28c16e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:05.790432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bec13e-5c61-46a2-a551-2e433a28c16e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:05.792010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bec13e-5c61-46a2-a551-2e433a28c16e service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:05.811868 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 682/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:08.453535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b64d1-d4da-4519-b6d6-5d5d764bac5f None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:08.454188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b64d1-d4da-4519-b6d6-5d5d764bac5f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:08.454188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19b64d1-d4da-4519-b6d6-5d5d764bac5f None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:08.455964 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 707/1389] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:53:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:08.462653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930da21-7dfa-4b7a-ad46-7b90ee4e3b62 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:08.463146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930da21-7dfa-4b7a-ad46-7b90ee4e3b62 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:08.463639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930da21-7dfa-4b7a-ad46-7b90ee4e3b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:08.486412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2930da21-7dfa-4b7a-ad46-7b90ee4e3b62 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:08.487538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2930da21-7dfa-4b7a-ad46-7b90ee4e3b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:08.495584 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 683/1390] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:53:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:08.542627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b64d1-d4da-4519-b6d6-5d5d764bac5f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:08.557107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19b64d1-d4da-4519-b6d6-5d5d764bac5f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:08.562946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3994b226-eaae-4643-b871-c07c9d160be7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:08.563537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994b226-eaae-4643-b871-c07c9d160be7 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:08.563820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994b226-eaae-4643-b871-c07c9d160be7 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:08.564101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3994b226-eaae-4643-b871-c07c9d160be7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:08.572886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994b226-eaae-4643-b871-c07c9d160be7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:08.574629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3994b226-eaae-4643-b871-c07c9d160be7 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:08.588923 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 708/1391] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:11.100701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae9b8d-120d-41f8-a48d-82eec6c71032 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:11.101296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae9b8d-120d-41f8-a48d-82eec6c71032 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:11.101296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae9b8d-120d-41f8-a48d-82eec6c71032 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:11.102670 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 684/1392] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:11.108352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fb024-a12f-47f4-9c34-ff580b72d407 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:11.108813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fb024-a12f-47f4-9c34-ff580b72d407 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:11.109104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fb024-a12f-47f4-9c34-ff580b72d407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:11.131441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fc2fb024-a12f-47f4-9c34-ff580b72d407 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:11.132708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fc2fb024-a12f-47f4-9c34-ff580b72d407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:11.141911 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 709/1393] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:53:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:11.188163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae9b8d-120d-41f8-a48d-82eec6c71032 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:11.197027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=84 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:53:11.203822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae9b8d-120d-41f8-a48d-82eec6c71032 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:11.210224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233486df-1ea8-4cc2-9a87-6af4fc571701 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:11.210825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233486df-1ea8-4cc2-9a87-6af4fc571701 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:11.211144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233486df-1ea8-4cc2-9a87-6af4fc571701 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:11.211423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233486df-1ea8-4cc2-9a87-6af4fc571701 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:11.220952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233486df-1ea8-4cc2-9a87-6af4fc571701 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:11.222653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233486df-1ea8-4cc2-9a87-6af4fc571701 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:11.232916 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 685/1394] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:13.852736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452dc4f-9969-44bb-9cf1-7ec3c549f219 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:13.853382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452dc4f-9969-44bb-9cf1-7ec3c549f219 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:13.853382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452dc4f-9969-44bb-9cf1-7ec3c549f219 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:13.854712 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 710/1395] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:13.860037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070b419-87ba-4479-a4e5-1ad1efdc299d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:13.860566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070b419-87ba-4479-a4e5-1ad1efdc299d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:13.860676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070b419-87ba-4479-a4e5-1ad1efdc299d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:13.879832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5070b419-87ba-4479-a4e5-1ad1efdc299d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:13.881056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5070b419-87ba-4479-a4e5-1ad1efdc299d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:13.889475 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 686/1396] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:53:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:13.935972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452dc4f-9969-44bb-9cf1-7ec3c549f219 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:13.951598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452dc4f-9969-44bb-9cf1-7ec3c549f219 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:13.957148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96cf73-ee86-41fd-863b-b3f7879f6acd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:13.957796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96cf73-ee86-41fd-863b-b3f7879f6acd service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:13.958119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96cf73-ee86-41fd-863b-b3f7879f6acd service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:13.958404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96cf73-ee86-41fd-863b-b3f7879f6acd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:13.966805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96cf73-ee86-41fd-863b-b3f7879f6acd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:13.968423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96cf73-ee86-41fd-863b-b3f7879f6acd service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:13.978488 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 711/1397] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:15.697387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fbced-a37d-46ae-ae93-f55a1115a49e None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:15.698047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fbced-a37d-46ae-ae93-f55a1115a49e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:15.698047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fbced-a37d-46ae-ae93-f55a1115a49e None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:15.699326 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 687/1398] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:15.704625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9194272-e3c1-4ae1-ab9f-5f427090db98 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:15.705075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9194272-e3c1-4ae1-ab9f-5f427090db98 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:15.705173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9194272-e3c1-4ae1-ab9f-5f427090db98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:15.723199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b9194272-e3c1-4ae1-ab9f-5f427090db98 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:15.724415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b9194272-e3c1-4ae1-ab9f-5f427090db98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:15.732402 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 712/1399] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 20:53:15.789591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0e518-f8bb-4436-9a8b-19b05c6a2b7e None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:15.789991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0e518-f8bb-4436-9a8b-19b05c6a2b7e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:15.790341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea0e518-f8bb-4436-9a8b-19b05c6a2b7e None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:15.791784 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 688/1400] 10.222.0.33 () {54 vars in 962 bytes} [Sat May 28 20:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:15.797080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6c3b8-39ec-4951-9f58-fea72e90c675 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:15.797372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6c3b8-39ec-4951-9f58-fea72e90c675 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:15.797655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6c3b8-39ec-4951-9f58-fea72e90c675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:15.846485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2ba6c3b8-39ec-4951-9f58-fea72e90c675 None None] MFA Rules not processed for user `224c1a0427c942e5bfef2bac75648d80`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:15.847682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2ba6c3b8-39ec-4951-9f58-fea72e90c675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:15.912337 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 713/1401] 10.222.0.33 () {58 vars in 1059 bytes} [Sat May 28 20:53:15 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:15.919999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0e518-f8bb-4436-9a8b-19b05c6a2b7e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:15.934504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea0e518-f8bb-4436-9a8b-19b05c6a2b7e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:15.938059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48afe0d-1e92-4341-ba5d-44fdc229347c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:15.938685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48afe0d-1e92-4341-ba5d-44fdc229347c service glance] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:15.939004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48afe0d-1e92-4341-ba5d-44fdc229347c service glance] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:15.939286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48afe0d-1e92-4341-ba5d-44fdc229347c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:15.948519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48afe0d-1e92-4341-ba5d-44fdc229347c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:15.949960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48afe0d-1e92-4341-ba5d-44fdc229347c service glance] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:15.960202 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 689/1402] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:16.403084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694d3d5-6905-40b6-ba24-acc31b8e4dfa None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:16.403382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694d3d5-6905-40b6-ba24-acc31b8e4dfa None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:16.403672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694d3d5-6905-40b6-ba24-acc31b8e4dfa None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:16.405094 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 714/1403] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:16.410561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7a02e-910c-4520-bf02-d9d64d5d4901 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:16.410855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7a02e-910c-4520-bf02-d9d64d5d4901 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:16.411139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d7a02e-910c-4520-bf02-d9d64d5d4901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:16.429276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-50d7a02e-910c-4520-bf02-d9d64d5d4901 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:16.430504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-50d7a02e-910c-4520-bf02-d9d64d5d4901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:16.438606 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 690/1404] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:53:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:16.485604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694d3d5-6905-40b6-ba24-acc31b8e4dfa None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:16.502316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694d3d5-6905-40b6-ba24-acc31b8e4dfa None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:16.508201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5f884-e54e-4202-a46b-d8d4963be5e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:16.508845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5f884-e54e-4202-a46b-d8d4963be5e0 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:16.509131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5f884-e54e-4202-a46b-d8d4963be5e0 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:16.509413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5f884-e54e-4202-a46b-d8d4963be5e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:16.518508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5f884-e54e-4202-a46b-d8d4963be5e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:16.520106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5f884-e54e-4202-a46b-d8d4963be5e0 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:16.529979 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 715/1405] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:16.653103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce1590-3ef0-4d6e-8285-874bb49b7ea7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:16.653392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce1590-3ef0-4d6e-8285-874bb49b7ea7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:16.653674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce1590-3ef0-4d6e-8285-874bb49b7ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:16.701751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-21ce1590-3ef0-4d6e-8285-874bb49b7ea7 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:16.703142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-21ce1590-3ef0-4d6e-8285-874bb49b7ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:16.766626 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 691/1406] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 20:53:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:16.815853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1a23b-d716-4ff7-9961-a39cf55ecdf8 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:16.816487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1a23b-d716-4ff7-9961-a39cf55ecdf8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:16.816487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1a23b-d716-4ff7-9961-a39cf55ecdf8 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:16.818041 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 716/1407] 10.222.0.33 () {54 vars in 942 bytes} [Sat May 28 20:53:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:16.825520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc3d6f-73f4-4e5e-b573-26a69a117072 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:16.827996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc3d6f-73f4-4e5e-b573-26a69a117072 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:16.827996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc3d6f-73f4-4e5e-b573-26a69a117072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:16.884027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-10fc3d6f-73f4-4e5e-b573-26a69a117072 None None] MFA Rules not processed for user `a01bba888a644eb18fc457aff578aa4b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:16.885180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-10fc3d6f-73f4-4e5e-b573-26a69a117072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:16.963974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:53:16.969753 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 692/1408] 10.222.0.33 () {58 vars in 1039 bytes} [Sat May 28 20:53:16 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:16.980107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1a23b-d716-4ff7-9961-a39cf55ecdf8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:16.994898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1a23b-d716-4ff7-9961-a39cf55ecdf8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:16.999914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b7afa5-667d-4c8f-8759-be0a35648868 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:17.000566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7afa5-667d-4c8f-8759-be0a35648868 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:17.000851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7afa5-667d-4c8f-8759-be0a35648868 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:17.001130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b7afa5-667d-4c8f-8759-be0a35648868 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:17.010353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7afa5-667d-4c8f-8759-be0a35648868 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:17.011978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b7afa5-667d-4c8f-8759-be0a35648868 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:17.021083 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 717/1409] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 20:53:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 20:53:26.984134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:53:27.018395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=75 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:53:41.820970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf8631-5c3e-4c9f-a096-29a5ed93bbc0 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:41.821576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf8631-5c3e-4c9f-a096-29a5ed93bbc0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:41.821576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf8631-5c3e-4c9f-a096-29a5ed93bbc0 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:41.822959 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 693/1410] 10.222.0.33 () {54 vars in 918 bytes} [Sat May 28 20:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:41.828101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a86b7-e344-41d6-ad2c-4c3ef6de6c75 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:41.828541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a86b7-e344-41d6-ad2c-4c3ef6de6c75 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:41.828664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702a86b7-e344-41d6-ad2c-4c3ef6de6c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:41.847603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-702a86b7-e344-41d6-ad2c-4c3ef6de6c75 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:41.848821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-702a86b7-e344-41d6-ad2c-4c3ef6de6c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:41.857747 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 718/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Sat May 28 20:53:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:41.871345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2391c1-2b1e-4461-8686-a47fd2ca0cd0 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:41.871633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2391c1-2b1e-4461-8686-a47fd2ca0cd0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:41.871914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2391c1-2b1e-4461-8686-a47fd2ca0cd0 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:41.873278 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 694/1412] 10.222.0.33 () {54 vars in 918 bytes} [Sat May 28 20:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:41.877949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9acd9-ca40-41a2-a292-acb8bae93dc2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:41.878244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9acd9-ca40-41a2-a292-acb8bae93dc2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:41.879266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9acd9-ca40-41a2-a292-acb8bae93dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:41.897562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4dd9acd9-ca40-41a2-a292-acb8bae93dc2 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:41.898799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4dd9acd9-ca40-41a2-a292-acb8bae93dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:41.907881 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 719/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Sat May 28 20:53:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:44.646057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c0a9b-0a19-41cd-801c-1693dff7d54a None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:44.646600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c0a9b-0a19-41cd-801c-1693dff7d54a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:44.646722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c0a9b-0a19-41cd-801c-1693dff7d54a None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:44.648016 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 695/1414] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 20:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:44.653327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85173272-8d8f-4c11-a3aa-6ab61b965841 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:44.653751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85173272-8d8f-4c11-a3aa-6ab61b965841 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:44.653959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85173272-8d8f-4c11-a3aa-6ab61b965841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:44.673076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-85173272-8d8f-4c11-a3aa-6ab61b965841 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:44.674333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-85173272-8d8f-4c11-a3aa-6ab61b965841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:44.682911 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 720/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 20:53:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:44.727176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c0a9b-0a19-41cd-801c-1693dff7d54a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:44.742414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c0a9b-0a19-41cd-801c-1693dff7d54a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:44.748012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36763116-1ac6-4cc5-93c3-faa755fe5986 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:44.748650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36763116-1ac6-4cc5-93c3-faa755fe5986 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:44.748937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36763116-1ac6-4cc5-93c3-faa755fe5986 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:44.749220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36763116-1ac6-4cc5-93c3-faa755fe5986 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:44.758116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36763116-1ac6-4cc5-93c3-faa755fe5986 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:44.759803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36763116-1ac6-4cc5-93c3-faa755fe5986 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:44.770128 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 696/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:45.788424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6768d-eebf-42f5-b3ab-7c4eacd15822 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:45.788973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6768d-eebf-42f5-b3ab-7c4eacd15822 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:45.789088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6768d-eebf-42f5-b3ab-7c4eacd15822 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:45.790429 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 721/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 20:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:45.795242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f4acc-f100-4077-8d1f-f2882aca62d8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:45.795669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f4acc-f100-4077-8d1f-f2882aca62d8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:45.795782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f4acc-f100-4077-8d1f-f2882aca62d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:45.814268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3d0f4acc-f100-4077-8d1f-f2882aca62d8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:45.815472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3d0f4acc-f100-4077-8d1f-f2882aca62d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:45.823830 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 697/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 20:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:46.040924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6768d-eebf-42f5-b3ab-7c4eacd15822 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:46.055586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6768d-eebf-42f5-b3ab-7c4eacd15822 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:46.061189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071dc4e-2f97-4e02-a4ca-f8f6e8bf3a84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:46.061928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071dc4e-2f97-4e02-a4ca-f8f6e8bf3a84 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:46.064448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071dc4e-2f97-4e02-a4ca-f8f6e8bf3a84 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:46.064448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071dc4e-2f97-4e02-a4ca-f8f6e8bf3a84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:46.071785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071dc4e-2f97-4e02-a4ca-f8f6e8bf3a84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:46.073441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071dc4e-2f97-4e02-a4ca-f8f6e8bf3a84 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:46.083728 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 722/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:47.026584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a35cc-897e-4038-a253-3650cf4b33a3 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:47.027184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a35cc-897e-4038-a253-3650cf4b33a3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:47.027184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0a35cc-897e-4038-a253-3650cf4b33a3 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:47.028523 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 698/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 20:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:47.033552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe534c-7b8c-45d4-b8a2-97ef35d75813 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:47.033998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe534c-7b8c-45d4-b8a2-97ef35d75813 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:47.034136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe534c-7b8c-45d4-b8a2-97ef35d75813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:47.177908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6ffe534c-7b8c-45d4-b8a2-97ef35d75813 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:47.179145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6ffe534c-7b8c-45d4-b8a2-97ef35d75813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:47.187394 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 723/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 20:53:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:47.343042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a35cc-897e-4038-a253-3650cf4b33a3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:47.357869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0a35cc-897e-4038-a253-3650cf4b33a3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:47.363349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837abdec-8545-4552-883c-07f2d4411bf3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:47.363984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837abdec-8545-4552-883c-07f2d4411bf3 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:47.364269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837abdec-8545-4552-883c-07f2d4411bf3 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:47.364544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837abdec-8545-4552-883c-07f2d4411bf3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:47.372805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837abdec-8545-4552-883c-07f2d4411bf3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:47.374443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837abdec-8545-4552-883c-07f2d4411bf3 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:47.384139 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 699/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:48.298558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836cfdb-d678-4d86-948a-bbcc18a1005a None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:48.299088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836cfdb-d678-4d86-948a-bbcc18a1005a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:48.299186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836cfdb-d678-4d86-948a-bbcc18a1005a None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:48.301033 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 724/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 20:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:48.306211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081ef1a-84ab-4bd2-935d-eafee68c4154 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:48.306633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081ef1a-84ab-4bd2-935d-eafee68c4154 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:48.306767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081ef1a-84ab-4bd2-935d-eafee68c4154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:48.325210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7081ef1a-84ab-4bd2-935d-eafee68c4154 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:48.326378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7081ef1a-84ab-4bd2-935d-eafee68c4154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:48.334343 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 700/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 20:53:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 20:53:48.488594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836cfdb-d678-4d86-948a-bbcc18a1005a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:48.502811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836cfdb-d678-4d86-948a-bbcc18a1005a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:48.508445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61eece5-ac4b-4980-b6eb-735d3070f54f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:48.509079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61eece5-ac4b-4980-b6eb-735d3070f54f service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:48.509366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61eece5-ac4b-4980-b6eb-735d3070f54f service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:48.509645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61eece5-ac4b-4980-b6eb-735d3070f54f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:48.517672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61eece5-ac4b-4980-b6eb-735d3070f54f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:48.519336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61eece5-ac4b-4980-b6eb-735d3070f54f service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:48.528694 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 725/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:49.438708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a961fd-2610-43aa-8cf6-c5954401a321 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:49.439230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a961fd-2610-43aa-8cf6-c5954401a321 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:49.439230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a961fd-2610-43aa-8cf6-c5954401a321 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:49.440668 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 701/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 20:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:49.445510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbc8d5-c1d3-4c73-ba74-b7c9436307ac None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:49.446065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbc8d5-c1d3-4c73-ba74-b7c9436307ac None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:49.446173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbc8d5-c1d3-4c73-ba74-b7c9436307ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:49.463966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a8fbc8d5-c1d3-4c73-ba74-b7c9436307ac None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:49.465313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a8fbc8d5-c1d3-4c73-ba74-b7c9436307ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:49.473645 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 726/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 20:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:49.632941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a961fd-2610-43aa-8cf6-c5954401a321 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:49.647192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a961fd-2610-43aa-8cf6-c5954401a321 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:49.652561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269ee40-7976-4c00-98da-8aaf9f905b69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:49.653193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269ee40-7976-4c00-98da-8aaf9f905b69 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:49.653476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269ee40-7976-4c00-98da-8aaf9f905b69 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:49.653752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269ee40-7976-4c00-98da-8aaf9f905b69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:49.661915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9269ee40-7976-4c00-98da-8aaf9f905b69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:49.663535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9269ee40-7976-4c00-98da-8aaf9f905b69 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:49.672794 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 702/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:50.574359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9a30b-30b2-4aa7-8917-a3f1396a5aae None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:50.574921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9a30b-30b2-4aa7-8917-a3f1396a5aae None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:50.574921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9a30b-30b2-4aa7-8917-a3f1396a5aae None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:50.576318 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 727/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 20:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:50.581323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e47008-2a19-4b04-b93e-f9ffdab28531 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:50.582294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e47008-2a19-4b04-b93e-f9ffdab28531 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:50.582405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e47008-2a19-4b04-b93e-f9ffdab28531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:50.600473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-24e47008-2a19-4b04-b93e-f9ffdab28531 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:50.601658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-24e47008-2a19-4b04-b93e-f9ffdab28531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:50.609447 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 703/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 20:53:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:50.766210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9a30b-30b2-4aa7-8917-a3f1396a5aae None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:50.780042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9a30b-30b2-4aa7-8917-a3f1396a5aae None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:50.785936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78daafc0-64e5-4f05-8e01-258c92d0b811 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:50.786564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78daafc0-64e5-4f05-8e01-258c92d0b811 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:50.786842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78daafc0-64e5-4f05-8e01-258c92d0b811 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:50.787114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78daafc0-64e5-4f05-8e01-258c92d0b811 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:50.794968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78daafc0-64e5-4f05-8e01-258c92d0b811 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:50.796545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78daafc0-64e5-4f05-8e01-258c92d0b811 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:50.806263 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 728/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:51.703769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc3279-3cfe-43d6-ab0a-5cc3fa191853 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:51.704302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc3279-3cfe-43d6-ab0a-5cc3fa191853 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:51.704302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc3279-3cfe-43d6-ab0a-5cc3fa191853 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:51.705646 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 704/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 20:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:51.710646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47a619-7d24-4d34-9ac9-2557e2a10f3e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:51.711092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47a619-7d24-4d34-9ac9-2557e2a10f3e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:51.711199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f47a619-7d24-4d34-9ac9-2557e2a10f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:51.730040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6f47a619-7d24-4d34-9ac9-2557e2a10f3e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:51.731235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6f47a619-7d24-4d34-9ac9-2557e2a10f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:51.739859 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 729/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 20:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:51.895067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc3279-3cfe-43d6-ab0a-5cc3fa191853 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:51.910278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc3279-3cfe-43d6-ab0a-5cc3fa191853 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:51.915764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94599cd-98dd-4e2a-8fa3-7e0744f3a5ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:51.916408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94599cd-98dd-4e2a-8fa3-7e0744f3a5ea service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:51.916698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94599cd-98dd-4e2a-8fa3-7e0744f3a5ea service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:51.916975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94599cd-98dd-4e2a-8fa3-7e0744f3a5ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:51.925649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94599cd-98dd-4e2a-8fa3-7e0744f3a5ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:51.927285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94599cd-98dd-4e2a-8fa3-7e0744f3a5ea service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:51.936617 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 705/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:52.837442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c35c2-370e-439e-bd67-074e1c363653 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:52.838013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c35c2-370e-439e-bd67-074e1c363653 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:52.838013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c35c2-370e-439e-bd67-074e1c363653 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:52.839397 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 730/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 20:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:52.844178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09a229-d5e1-4d68-b9da-8ebe3451e3c8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:52.844585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09a229-d5e1-4d68-b9da-8ebe3451e3c8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:52.844735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09a229-d5e1-4d68-b9da-8ebe3451e3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:52.863796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-fb09a229-d5e1-4d68-b9da-8ebe3451e3c8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:52.864977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fb09a229-d5e1-4d68-b9da-8ebe3451e3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:52.872638 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 706/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 20:53:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:53.026647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c35c2-370e-439e-bd67-074e1c363653 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:53.041544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c35c2-370e-439e-bd67-074e1c363653 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:53.047022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecfa300-f3f3-4b84-b36f-fb2ca1885f7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:53.047660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfa300-f3f3-4b84-b36f-fb2ca1885f7f service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:53.047948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfa300-f3f3-4b84-b36f-fb2ca1885f7f service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:53.048224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecfa300-f3f3-4b84-b36f-fb2ca1885f7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:53.057035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfa300-f3f3-4b84-b36f-fb2ca1885f7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:53.058673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecfa300-f3f3-4b84-b36f-fb2ca1885f7f service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:53.069363 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 731/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:53.972659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ebfa0-c9a5-4fa5-8e13-274ab9a94c16 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:53.973235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ebfa0-c9a5-4fa5-8e13-274ab9a94c16 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:53.973235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ebfa0-c9a5-4fa5-8e13-274ab9a94c16 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:53.974658 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 707/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 20:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:53.979715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b45b8-a7d0-421b-8f12-43a5a67963fe None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:53.980096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b45b8-a7d0-421b-8f12-43a5a67963fe None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:53.980291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b45b8-a7d0-421b-8f12-43a5a67963fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:53.997775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-267b45b8-a7d0-421b-8f12-43a5a67963fe None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:53.999018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-267b45b8-a7d0-421b-8f12-43a5a67963fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:54.007479 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 732/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 20:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:54.162391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ebfa0-c9a5-4fa5-8e13-274ab9a94c16 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:54.177094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ebfa0-c9a5-4fa5-8e13-274ab9a94c16 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:54.182463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff18f00-c32b-4d95-98ff-22fd6b1b470c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:54.183101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff18f00-c32b-4d95-98ff-22fd6b1b470c service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:54.183383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff18f00-c32b-4d95-98ff-22fd6b1b470c service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:54.183661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff18f00-c32b-4d95-98ff-22fd6b1b470c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:54.192364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff18f00-c32b-4d95-98ff-22fd6b1b470c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:54.193995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff18f00-c32b-4d95-98ff-22fd6b1b470c service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:54.203330 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 708/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:56.353618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34e063-2d01-477a-87cc-c2eff2f9b924 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:56.354173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34e063-2d01-477a-87cc-c2eff2f9b924 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:56.354173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34e063-2d01-477a-87cc-c2eff2f9b924 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:56.355519 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 733/1441] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 20:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:56.360955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affd235-2a5f-44c5-bc35-a4d26f620619 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:56.361430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affd235-2a5f-44c5-bc35-a4d26f620619 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:56.361592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5affd235-2a5f-44c5-bc35-a4d26f620619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:56.380213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5affd235-2a5f-44c5-bc35-a4d26f620619 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:56.381399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5affd235-2a5f-44c5-bc35-a4d26f620619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:56.389092 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 709/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 20:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:56.429103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34e063-2d01-477a-87cc-c2eff2f9b924 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:56.439752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34e063-2d01-477a-87cc-c2eff2f9b924 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:56.444058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0cafb6-a5d2-40fb-b0d1-668551fc821c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:56.444698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cafb6-a5d2-40fb-b0d1-668551fc821c service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:56.444979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cafb6-a5d2-40fb-b0d1-668551fc821c service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:56.445253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0cafb6-a5d2-40fb-b0d1-668551fc821c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:56.454291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0cafb6-a5d2-40fb-b0d1-668551fc821c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:56.455722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0cafb6-a5d2-40fb-b0d1-668551fc821c service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:56.465058 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 734/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:53:58.879921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1621f9b-c85b-4236-acd7-783a629877d7 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:58.880487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1621f9b-c85b-4236-acd7-783a629877d7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:58.880487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1621f9b-c85b-4236-acd7-783a629877d7 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:58.881923 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 710/1444] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 20:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:53:58.887244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd68a9-5540-4132-b74d-2e4d4729a16e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:58.887687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd68a9-5540-4132-b74d-2e4d4729a16e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:58.887797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd68a9-5540-4132-b74d-2e4d4729a16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:58.906070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-afcd68a9-5540-4132-b74d-2e4d4729a16e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:53:58.910138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-afcd68a9-5540-4132-b74d-2e4d4729a16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:53:58.915836 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 735/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 20:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:53:58.955893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1621f9b-c85b-4236-acd7-783a629877d7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:53:58.967347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1621f9b-c85b-4236-acd7-783a629877d7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:53:58.971032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583f551-dd70-430d-ba0e-2aaa2c96945d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:53:58.971702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583f551-dd70-430d-ba0e-2aaa2c96945d service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:53:58.972018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583f551-dd70-430d-ba0e-2aaa2c96945d service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:53:58.972288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583f551-dd70-430d-ba0e-2aaa2c96945d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:53:58.981160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583f551-dd70-430d-ba0e-2aaa2c96945d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:53:58.982664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583f551-dd70-430d-ba0e-2aaa2c96945d service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:53:58.991919 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 711/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:53:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:54:01.637864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e73b4d-a308-431e-adaa-e2325b78a624 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:01.638426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e73b4d-a308-431e-adaa-e2325b78a624 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:01.638426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e73b4d-a308-431e-adaa-e2325b78a624 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:01.639793 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 736/1447] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 20:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:54:01.645265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db13298-d095-4d34-8ef4-2f8fde9b1117 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:01.645700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db13298-d095-4d34-8ef4-2f8fde9b1117 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:01.645915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db13298-d095-4d34-8ef4-2f8fde9b1117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:01.663391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3db13298-d095-4d34-8ef4-2f8fde9b1117 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:54:01.664589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3db13298-d095-4d34-8ef4-2f8fde9b1117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:54:01.672426 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 712/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 20:54:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:54:01.713863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e73b4d-a308-431e-adaa-e2325b78a624 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:54:01.724655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e73b4d-a308-431e-adaa-e2325b78a624 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:54:01.728107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03ba50-7b49-4d94-8fc0-859b1473c9dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:54:01.728734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03ba50-7b49-4d94-8fc0-859b1473c9dd service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:01.729014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03ba50-7b49-4d94-8fc0-859b1473c9dd service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:01.729284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03ba50-7b49-4d94-8fc0-859b1473c9dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:01.738560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03ba50-7b49-4d94-8fc0-859b1473c9dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:54:01.740005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03ba50-7b49-4d94-8fc0-859b1473c9dd service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:54:01.749459 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 737/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:54:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:54:04.151545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21f43c-916b-4356-80a5-3c5589b5af47 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:04.152155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21f43c-916b-4356-80a5-3c5589b5af47 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:04.152155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d21f43c-916b-4356-80a5-3c5589b5af47 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:04.153457 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 713/1450] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 20:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:54:04.158963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0a71f-5c5c-454e-bba2-0c8172e4a678 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:04.159998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0a71f-5c5c-454e-bba2-0c8172e4a678 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:04.160107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0a71f-5c5c-454e-bba2-0c8172e4a678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:04.177389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-bca0a71f-5c5c-454e-bba2-0c8172e4a678 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:54:04.178614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bca0a71f-5c5c-454e-bba2-0c8172e4a678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:54:04.186394 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 738/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 20:54:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:54:04.227131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21f43c-916b-4356-80a5-3c5589b5af47 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:54:04.237687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d21f43c-916b-4356-80a5-3c5589b5af47 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:54:04.241233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f13b3bf-0182-4732-9b0d-6a753af11928 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:54:04.241897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13b3bf-0182-4732-9b0d-6a753af11928 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:04.242180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13b3bf-0182-4732-9b0d-6a753af11928 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:04.242454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f13b3bf-0182-4732-9b0d-6a753af11928 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:04.251332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f13b3bf-0182-4732-9b0d-6a753af11928 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:54:04.252762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f13b3bf-0182-4732-9b0d-6a753af11928 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:54:04.262836 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 714/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:54:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:54:06.587921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6704b1-7286-4046-9cdf-6e5d0b7c49c5 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:06.588457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6704b1-7286-4046-9cdf-6e5d0b7c49c5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:06.588457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6704b1-7286-4046-9cdf-6e5d0b7c49c5 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:06.589866 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 739/1453] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 20:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:54:06.595156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052100a-b29d-4e38-9952-5d217f35c868 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:06.595706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052100a-b29d-4e38-9952-5d217f35c868 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:06.595821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3052100a-b29d-4e38-9952-5d217f35c868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:06.614399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3052100a-b29d-4e38-9952-5d217f35c868 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:54:06.615642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3052100a-b29d-4e38-9952-5d217f35c868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:54:06.623523 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 715/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 20:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:54:06.663567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6704b1-7286-4046-9cdf-6e5d0b7c49c5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:54:06.674147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6704b1-7286-4046-9cdf-6e5d0b7c49c5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:54:06.678293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434cfa33-9cca-4a08-ac8f-1597b52c44e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:54:06.682200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cfa33-9cca-4a08-ac8f-1597b52c44e9 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:06.682200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cfa33-9cca-4a08-ac8f-1597b52c44e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:06.682200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cfa33-9cca-4a08-ac8f-1597b52c44e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:06.688693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434cfa33-9cca-4a08-ac8f-1597b52c44e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:54:06.690156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434cfa33-9cca-4a08-ac8f-1597b52c44e9 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:54:06.700038 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 740/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:54:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:54:09.051717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0d72a-431f-4c8d-9ee2-bf885ae7f84f None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:09.052285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0d72a-431f-4c8d-9ee2-bf885ae7f84f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:09.052285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0d72a-431f-4c8d-9ee2-bf885ae7f84f None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:09.053628 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 716/1456] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 20:54:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:54:09.059162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f51754-b447-4650-91c7-57e8baa54f2f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:09.059565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f51754-b447-4650-91c7-57e8baa54f2f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:09.059726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f51754-b447-4650-91c7-57e8baa54f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:09.066594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=70 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:54:09.078834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f9f51754-b447-4650-91c7-57e8baa54f2f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:54:09.080029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f9f51754-b447-4650-91c7-57e8baa54f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:54:09.087965 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 741/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 20:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:54:09.129710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0d72a-431f-4c8d-9ee2-bf885ae7f84f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:54:09.131520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a7a0d72a-431f-4c8d-9ee2-bf885ae7f84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:54:09.139240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=64 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:54:09.145003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0d72a-431f-4c8d-9ee2-bf885ae7f84f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:54:09.148642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddd150-0eda-462d-b28a-d96b326effd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:54:09.149282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddd150-0eda-462d-b28a-d96b326effd8 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:09.149570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddd150-0eda-462d-b28a-d96b326effd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:09.149930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddd150-0eda-462d-b28a-d96b326effd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:09.161156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddd150-0eda-462d-b28a-d96b326effd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:54:09.162639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddd150-0eda-462d-b28a-d96b326effd8 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:54:09.172324 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 717/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:54:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:54:11.522675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619326d-cedf-4fef-b210-983983e12fef None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:11.523193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619326d-cedf-4fef-b210-983983e12fef None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:11.523310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619326d-cedf-4fef-b210-983983e12fef None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:11.524595 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 742/1459] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 20:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:54:11.529824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd20e8-24da-4fc3-96d6-11e8468dd992 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:11.530257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd20e8-24da-4fc3-96d6-11e8468dd992 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:11.530431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd20e8-24da-4fc3-96d6-11e8468dd992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:11.548753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-fcfd20e8-24da-4fc3-96d6-11e8468dd992 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:54:11.550020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fcfd20e8-24da-4fc3-96d6-11e8468dd992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:54:11.559072 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 718/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 20:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 20:54:11.603042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619326d-cedf-4fef-b210-983983e12fef None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:54:11.614277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f619326d-cedf-4fef-b210-983983e12fef None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:54:11.618036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab15a36-5510-46f8-b29c-b60b8e3787dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:54:11.618670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab15a36-5510-46f8-b29c-b60b8e3787dd service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:11.618955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab15a36-5510-46f8-b29c-b60b8e3787dd service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:11.619231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab15a36-5510-46f8-b29c-b60b8e3787dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:11.629232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab15a36-5510-46f8-b29c-b60b8e3787dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:54:11.630684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab15a36-5510-46f8-b29c-b60b8e3787dd service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:54:11.640130 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 743/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:54:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:54:13.999274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160e79a-46ff-4521-8419-ba6a93bda53c None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:13.999781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160e79a-46ff-4521-8419-ba6a93bda53c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:13.999781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160e79a-46ff-4521-8419-ba6a93bda53c None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:14.001134 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 719/1462] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 20:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:54:14.006877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef85f6-550e-4b0d-bbcf-f29c8d87b421 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:14.007310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef85f6-550e-4b0d-bbcf-f29c8d87b421 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:14.007409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef85f6-550e-4b0d-bbcf-f29c8d87b421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:14.025349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-38ef85f6-550e-4b0d-bbcf-f29c8d87b421 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:54:14.026677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-38ef85f6-550e-4b0d-bbcf-f29c8d87b421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:54:14.035025 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 744/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 20:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:54:14.074564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160e79a-46ff-4521-8419-ba6a93bda53c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:54:14.085269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160e79a-46ff-4521-8419-ba6a93bda53c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:54:14.089162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39759c4c-895b-47c4-9a95-09444038b261 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:54:14.089888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39759c4c-895b-47c4-9a95-09444038b261 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:14.090180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39759c4c-895b-47c4-9a95-09444038b261 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:14.090460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39759c4c-895b-47c4-9a95-09444038b261 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:14.100102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39759c4c-895b-47c4-9a95-09444038b261 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:54:14.101574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39759c4c-895b-47c4-9a95-09444038b261 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:54:14.111202 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 720/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:54:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:54:16.395968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f900c-5493-4111-83d4-d7af165a91e9 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:16.396545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f900c-5493-4111-83d4-d7af165a91e9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:16.396545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f900c-5493-4111-83d4-d7af165a91e9 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:16.397926 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 745/1465] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 20:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:54:16.403162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c120d-3dfe-449e-a22b-1bf352f57816 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:16.403571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c120d-3dfe-449e-a22b-1bf352f57816 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:16.403718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58c120d-3dfe-449e-a22b-1bf352f57816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:16.421236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e58c120d-3dfe-449e-a22b-1bf352f57816 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:54:16.422454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e58c120d-3dfe-449e-a22b-1bf352f57816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:54:16.430198 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 721/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 20:54:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:54:16.468873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f900c-5493-4111-83d4-d7af165a91e9 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:54:16.479679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f900c-5493-4111-83d4-d7af165a91e9 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:54:16.483246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7544b-f79f-47ad-b729-460f0222b2b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:54:16.483884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7544b-f79f-47ad-b729-460f0222b2b3 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:16.484163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7544b-f79f-47ad-b729-460f0222b2b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:16.484438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7544b-f79f-47ad-b729-460f0222b2b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:16.493340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7544b-f79f-47ad-b729-460f0222b2b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:54:16.494815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7544b-f79f-47ad-b729-460f0222b2b3 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:54:16.504873 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 746/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:54:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:54:18.795172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2348d-8e22-43f3-b145-dfee16163095 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:18.795795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2348d-8e22-43f3-b145-dfee16163095 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:18.795795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2348d-8e22-43f3-b145-dfee16163095 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:18.797099 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 722/1468] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 20:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:54:18.803056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cdfc8-6d2f-4b15-abf3-30583ea1be79 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:18.803458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cdfc8-6d2f-4b15-abf3-30583ea1be79 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:18.803631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5cdfc8-6d2f-4b15-abf3-30583ea1be79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:18.820970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ef5cdfc8-6d2f-4b15-abf3-30583ea1be79 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:54:18.822211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ef5cdfc8-6d2f-4b15-abf3-30583ea1be79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:54:18.829982 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 747/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 20:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:54:18.869235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2348d-8e22-43f3-b145-dfee16163095 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:54:18.880738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2348d-8e22-43f3-b145-dfee16163095 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:54:18.884187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81fcd6-f497-4d25-9c4a-189f4f11778a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:54:18.884815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81fcd6-f497-4d25-9c4a-189f4f11778a service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:18.885094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81fcd6-f497-4d25-9c4a-189f4f11778a service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:18.885367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81fcd6-f497-4d25-9c4a-189f4f11778a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:18.894093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81fcd6-f497-4d25-9c4a-189f4f11778a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:54:18.895524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81fcd6-f497-4d25-9c4a-189f4f11778a service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:54:18.904727 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 723/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:54:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:54:28.815953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=20 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:54:28.900607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=23 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:54:53.459032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de58e91-e6df-45cd-81e0-0cd0dbfd79cd None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:53.459588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de58e91-e6df-45cd-81e0-0cd0dbfd79cd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:53.459588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de58e91-e6df-45cd-81e0-0cd0dbfd79cd None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:53.460888 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 748/1471] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:54:53.466229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b52e2d-b1fd-4dbc-af2a-b4d96a86a242 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:53.466707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b52e2d-b1fd-4dbc-af2a-b4d96a86a242 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:53.466817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b52e2d-b1fd-4dbc-af2a-b4d96a86a242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:53.484699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b8b52e2d-b1fd-4dbc-af2a-b4d96a86a242 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:54:53.485925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b8b52e2d-b1fd-4dbc-af2a-b4d96a86a242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:54:53.493658 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 724/1472] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 20:54:53.542124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5d64c-85ad-43fc-9b91-de2f2293f7e8 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:53.542426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5d64c-85ad-43fc-9b91-de2f2293f7e8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:53.542709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec5d64c-85ad-43fc-9b91-de2f2293f7e8 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:53.544066 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 749/1473] 10.222.0.33 () {54 vars in 962 bytes} [Sat May 28 20:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:54:53.549017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484256b-9a94-4fa2-8a53-5b3cd48ff84a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:53.549305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484256b-9a94-4fa2-8a53-5b3cd48ff84a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:53.549587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484256b-9a94-4fa2-8a53-5b3cd48ff84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:53.567455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9484256b-9a94-4fa2-8a53-5b3cd48ff84a None None] MFA Rules not processed for user `224c1a0427c942e5bfef2bac75648d80`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:54:53.568792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9484256b-9a94-4fa2-8a53-5b3cd48ff84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:54:53.575091 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 725/1474] 10.222.0.33 () {58 vars in 1059 bytes} [Sat May 28 20:54:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:54:53.582763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5d64c-85ad-43fc-9b91-de2f2293f7e8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:54:53.593353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec5d64c-85ad-43fc-9b91-de2f2293f7e8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:54:53.597488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d0728-e8aa-480a-82b7-5bfc6a65d449 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:54:53.600393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d0728-e8aa-480a-82b7-5bfc6a65d449 service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:53.600393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d0728-e8aa-480a-82b7-5bfc6a65d449 service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:53.600393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d0728-e8aa-480a-82b7-5bfc6a65d449 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:53.607709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d0728-e8aa-480a-82b7-5bfc6a65d449 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:54:53.609264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d0728-e8aa-480a-82b7-5bfc6a65d449 service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:54:53.618999 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 750/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:54:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:54:56.012642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0984a8-5d20-4051-859a-ac1e27d5a0b5 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:56.013186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0984a8-5d20-4051-859a-ac1e27d5a0b5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:56.013293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0984a8-5d20-4051-859a-ac1e27d5a0b5 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:56.014637 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 726/1476] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:54:56.020686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f77580-8dbc-48c4-8af1-9a9fb028799a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:56.021083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f77580-8dbc-48c4-8af1-9a9fb028799a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:56.021245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f77580-8dbc-48c4-8af1-9a9fb028799a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:56.038612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a1f77580-8dbc-48c4-8af1-9a9fb028799a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:54:56.039804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a1f77580-8dbc-48c4-8af1-9a9fb028799a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:54:56.047691 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 751/1477] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:54:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:54:56.074025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0984a8-5d20-4051-859a-ac1e27d5a0b5 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:54:56.087511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0984a8-5d20-4051-859a-ac1e27d5a0b5 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:54:56.092039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8bf8b5-b1a4-4753-9ae9-5878d31bf525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:54:56.092703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bf8b5-b1a4-4753-9ae9-5878d31bf525 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:56.093031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bf8b5-b1a4-4753-9ae9-5878d31bf525 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:56.093308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8bf8b5-b1a4-4753-9ae9-5878d31bf525 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:56.094140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bf8b5-b1a4-4753-9ae9-5878d31bf525 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:54:56.095934 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-bb8bf8b5-b1a4-4753-9ae9-5878d31bf525 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 20:54:56.095934 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 20:54:56.095934 ncdv-master-1 devstack@keystone.service[103104]: May 28 20:54:56.097089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8bf8b5-b1a4-4753-9ae9-5878d31bf525 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:54:56.116131 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 727/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Sat May 28 20:54:56 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 20:54:58.412300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fd71a-b8b4-4639-9f29-51d263af501d None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:58.412842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fd71a-b8b4-4639-9f29-51d263af501d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:58.412842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1fd71a-b8b4-4639-9f29-51d263af501d None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:58.414209 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 752/1479] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:54:58.419462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263db3b-ffb3-44d3-bc27-af978938ce44 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:58.419882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263db3b-ffb3-44d3-bc27-af978938ce44 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:58.420024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e263db3b-ffb3-44d3-bc27-af978938ce44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:58.438322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e263db3b-ffb3-44d3-bc27-af978938ce44 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:54:58.439576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e263db3b-ffb3-44d3-bc27-af978938ce44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:54:58.447480 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 728/1480] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:54:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:54:58.493638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fd71a-b8b4-4639-9f29-51d263af501d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:54:58.508166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1fd71a-b8b4-4639-9f29-51d263af501d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:54:58.513572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ecdda-464e-4f10-ab77-115e9b1d0655 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:54:58.514239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ecdda-464e-4f10-ab77-115e9b1d0655 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:54:58.514541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ecdda-464e-4f10-ab77-115e9b1d0655 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:54:58.514817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ecdda-464e-4f10-ab77-115e9b1d0655 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:54:58.523465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ecdda-464e-4f10-ab77-115e9b1d0655 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:54:58.525084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ecdda-464e-4f10-ab77-115e9b1d0655 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:54:58.535298 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 753/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:54:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:55:00.985281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca22c9-d1de-481e-a977-9b2278659b98 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:55:00.986432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca22c9-d1de-481e-a977-9b2278659b98 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:55:00.986584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca22c9-d1de-481e-a977-9b2278659b98 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:55:00.987962 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 729/1482] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:55:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:55:00.993342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc349a-11dd-43a9-b57e-71749ccfa27d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:55:00.993787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc349a-11dd-43a9-b57e-71749ccfa27d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:55:00.993942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc349a-11dd-43a9-b57e-71749ccfa27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:55:01.012278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6bdc349a-11dd-43a9-b57e-71749ccfa27d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:55:01.013481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6bdc349a-11dd-43a9-b57e-71749ccfa27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:55:01.021247 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 754/1483] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:55:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:55:01.060629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca22c9-d1de-481e-a977-9b2278659b98 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:55:01.072132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca22c9-d1de-481e-a977-9b2278659b98 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:55:01.075700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ed58c-1221-4761-9748-34ab6ed67c66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:55:01.076330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ed58c-1221-4761-9748-34ab6ed67c66 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:55:01.076611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ed58c-1221-4761-9748-34ab6ed67c66 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:55:01.076884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ed58c-1221-4761-9748-34ab6ed67c66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:55:01.086820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ed58c-1221-4761-9748-34ab6ed67c66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:55:01.088262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ed58c-1221-4761-9748-34ab6ed67c66 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:55:01.098401 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 730/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:55:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:55:03.415645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a0c23-8d73-4403-9485-485b903234d4 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:55:03.416200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a0c23-8d73-4403-9485-485b903234d4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:55:03.416200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a0c23-8d73-4403-9485-485b903234d4 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:55:03.417489 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 755/1485] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:55:03.422913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc97e24-da5d-40ba-889a-56439fa345d1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:55:03.423308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc97e24-da5d-40ba-889a-56439fa345d1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:55:03.423481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc97e24-da5d-40ba-889a-56439fa345d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:55:03.441555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-fbc97e24-da5d-40ba-889a-56439fa345d1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:55:03.442796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fbc97e24-da5d-40ba-889a-56439fa345d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:55:03.450791 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 731/1486] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:55:03.491127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a0c23-8d73-4403-9485-485b903234d4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:55:03.502101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a0c23-8d73-4403-9485-485b903234d4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:55:03.505931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5980bd-636b-4c59-8c0f-988e7354de9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:55:03.506651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5980bd-636b-4c59-8c0f-988e7354de9a service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:55:03.506941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5980bd-636b-4c59-8c0f-988e7354de9a service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:55:03.507219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5980bd-636b-4c59-8c0f-988e7354de9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:55:03.515885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5980bd-636b-4c59-8c0f-988e7354de9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:55:03.518027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5980bd-636b-4c59-8c0f-988e7354de9a service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:55:03.527167 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 756/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:55:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:55:05.894641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7dc37-dd49-4d9f-b3c6-b3bf45867bfa None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:55:05.895209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7dc37-dd49-4d9f-b3c6-b3bf45867bfa None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:55:05.895209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7dc37-dd49-4d9f-b3c6-b3bf45867bfa None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:55:05.896576 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 732/1488] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:55:05.901917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404b9f1-4f77-4251-85e1-f2caf9260545 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:55:05.902330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404b9f1-4f77-4251-85e1-f2caf9260545 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:55:05.902479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b404b9f1-4f77-4251-85e1-f2caf9260545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:55:05.920797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b404b9f1-4f77-4251-85e1-f2caf9260545 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:55:05.922046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b404b9f1-4f77-4251-85e1-f2caf9260545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:55:05.930603 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 757/1489] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:55:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:55:05.973287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7dc37-dd49-4d9f-b3c6-b3bf45867bfa None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:55:05.984138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7dc37-dd49-4d9f-b3c6-b3bf45867bfa None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:55:05.988284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb84ee39-8072-4d35-9402-9af82c152bd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:55:05.988957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84ee39-8072-4d35-9402-9af82c152bd9 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:55:05.989238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84ee39-8072-4d35-9402-9af82c152bd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:55:05.989511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb84ee39-8072-4d35-9402-9af82c152bd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:55:05.998649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84ee39-8072-4d35-9402-9af82c152bd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:55:06.006788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb84ee39-8072-4d35-9402-9af82c152bd9 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:55:06.009643 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 733/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:55:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:55:08.604790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250bfd7-d0e7-4a1c-b9c7-6a53912cd3a8 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:55:08.605363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250bfd7-d0e7-4a1c-b9c7-6a53912cd3a8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:55:08.605363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250bfd7-d0e7-4a1c-b9c7-6a53912cd3a8 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:55:08.606762 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 758/1491] 10.222.0.33 () {54 vars in 926 bytes} [Sat May 28 20:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:55:08.611892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf822ec7-4f2c-4090-9785-006d621781b3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:55:08.612294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf822ec7-4f2c-4090-9785-006d621781b3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:55:08.612448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf822ec7-4f2c-4090-9785-006d621781b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:55:08.630257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bf822ec7-4f2c-4090-9785-006d621781b3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:55:08.631454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bf822ec7-4f2c-4090-9785-006d621781b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:55:08.640940 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 734/1492] 10.222.0.33 () {58 vars in 1023 bytes} [Sat May 28 20:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:55:08.682344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250bfd7-d0e7-4a1c-b9c7-6a53912cd3a8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:55:08.694064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250bfd7-d0e7-4a1c-b9c7-6a53912cd3a8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:55:08.697624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9b8f23-ae32-4807-8436-407c04948c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:55:08.698301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b8f23-ae32-4807-8436-407c04948c9c service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:55:08.698586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b8f23-ae32-4807-8436-407c04948c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:55:08.701085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9b8f23-ae32-4807-8436-407c04948c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:55:08.708650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b8f23-ae32-4807-8436-407c04948c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:55:08.710112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9b8f23-ae32-4807-8436-407c04948c9c service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:55:08.719334 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 759/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:55:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:55:18.624825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=33 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:55:18.715239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=33 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:57:19.844871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a18b5-92f3-4f2a-a50d-f8d4b398336b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:57:19.845495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a18b5-92f3-4f2a-a50d-f8d4b398336b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:57:19.845495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a18b5-92f3-4f2a-a50d-f8d4b398336b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:57:19.864243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9a6a18b5-92f3-4f2a-a50d-f8d4b398336b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:57:19.865437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9a6a18b5-92f3-4f2a-a50d-f8d4b398336b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:57:19.873217 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 735/1494] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 20:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:57:19.883195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05971265-6f78-401f-a347-150b105b960a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:57:19.883679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05971265-6f78-401f-a347-150b105b960a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:57:19.883791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05971265-6f78-401f-a347-150b105b960a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:57:19.902826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-05971265-6f78-401f-a347-150b105b960a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:57:19.904021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-05971265-6f78-401f-a347-150b105b960a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:57:19.911838 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 760/1495] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 20:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:57:19.916249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a18b5-92f3-4f2a-a50d-f8d4b398336b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:57:19.925775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a18b5-92f3-4f2a-a50d-f8d4b398336b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:57:19.930206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933dfdf-7b66-4cad-b116-cb4b511924b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:57:19.930850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933dfdf-7b66-4cad-b116-cb4b511924b4 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:57:19.931128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933dfdf-7b66-4cad-b116-cb4b511924b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:57:19.931399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c933dfdf-7b66-4cad-b116-cb4b511924b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:57:19.932117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c933dfdf-7b66-4cad-b116-cb4b511924b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 20:57:19.932422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c933dfdf-7b66-4cad-b116-cb4b511924b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:57:19.933433 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-c933dfdf-7b66-4cad-b116-cb4b511924b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 20:57:19.933433 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 20:57:19.933433 ncdv-master-1 devstack@keystone.service[103104]: May 28 20:57:19.934414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c933dfdf-7b66-4cad-b116-cb4b511924b4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:57:19.946095 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 736/1496] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 20:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 20:57:19.954117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05971265-6f78-401f-a347-150b105b960a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:57:19.963794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05971265-6f78-401f-a347-150b105b960a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:57:19.968323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5698529-de4c-43c6-9f10-7ab3a30810e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:57:19.968955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5698529-de4c-43c6-9f10-7ab3a30810e7 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:57:19.969234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5698529-de4c-43c6-9f10-7ab3a30810e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:57:19.969511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5698529-de4c-43c6-9f10-7ab3a30810e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:57:19.970243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5698529-de4c-43c6-9f10-7ab3a30810e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:57:19.971984 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-f5698529-de4c-43c6-9f10-7ab3a30810e7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 20:57:19.971984 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 20:57:19.971984 ncdv-master-1 devstack@keystone.service[103103]: May 28 20:57:19.973166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5698529-de4c-43c6-9f10-7ab3a30810e7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:57:20.065022 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 761/1497] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 20:57:19 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:57:20.069408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933dfdf-7b66-4cad-b116-cb4b511924b4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:57:20.079784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c933dfdf-7b66-4cad-b116-cb4b511924b4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:57:20.135237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb67fe-6538-45e5-9107-0afe8b789f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:57:20.135833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb67fe-6538-45e5-9107-0afe8b789f35 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:57:20.136116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb67fe-6538-45e5-9107-0afe8b789f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:57:20.136392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfb67fe-6538-45e5-9107-0afe8b789f35 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:57:20.137088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb67fe-6538-45e5-9107-0afe8b789f35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:57:20.138145 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-3cfb67fe-6538-45e5-9107-0afe8b789f35 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 20:57:20.138145 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 20:57:20.138145 ncdv-master-1 devstack@keystone.service[103104]: May 28 20:57:20.139361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfb67fe-6538-45e5-9107-0afe8b789f35 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:57:20.195006 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 737/1498] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 20:57:20 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 20:57:20.199312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5698529-de4c-43c6-9f10-7ab3a30810e7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:57:20.209634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5698529-de4c-43c6-9f10-7ab3a30810e7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:57:20.214266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15727ed-66ea-430c-88cd-11e3132691ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:57:20.215030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15727ed-66ea-430c-88cd-11e3132691ba admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:57:20.215317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15727ed-66ea-430c-88cd-11e3132691ba admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:57:20.215674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15727ed-66ea-430c-88cd-11e3132691ba admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:57:20.216409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15727ed-66ea-430c-88cd-11e3132691ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:57:20.217410 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-f15727ed-66ea-430c-88cd-11e3132691ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 20:57:20.217410 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 20:57:20.217410 ncdv-master-1 devstack@keystone.service[103103]: May 28 20:57:20.218333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15727ed-66ea-430c-88cd-11e3132691ba admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:57:20.231381 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 762/1499] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 20:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 20:57:20.235355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb67fe-6538-45e5-9107-0afe8b789f35 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:57:20.245814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfb67fe-6538-45e5-9107-0afe8b789f35 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:57:20.250615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18362bb0-c0c4-4fdc-8b70-a2146a42dcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:57:20.251267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18362bb0-c0c4-4fdc-8b70-a2146a42dcb4 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:57:20.251552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18362bb0-c0c4-4fdc-8b70-a2146a42dcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:57:20.252459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18362bb0-c0c4-4fdc-8b70-a2146a42dcb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:57:20.253189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18362bb0-c0c4-4fdc-8b70-a2146a42dcb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:57:20.254257 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-18362bb0-c0c4-4fdc-8b70-a2146a42dcb4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 20:57:20.254257 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 20:57:20.254257 ncdv-master-1 devstack@keystone.service[103104]: May 28 20:57:20.255145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18362bb0-c0c4-4fdc-8b70-a2146a42dcb4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:57:20.268501 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 738/1500] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 20:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 20:57:20.272457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15727ed-66ea-430c-88cd-11e3132691ba admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:57:20.282217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15727ed-66ea-430c-88cd-11e3132691ba admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:57:20.286741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8ab12-0962-42a5-9829-87af26d1d354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:57:20.287603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8ab12-0962-42a5-9829-87af26d1d354 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:57:20.287884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8ab12-0962-42a5-9829-87af26d1d354 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:57:20.288157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8ab12-0962-42a5-9829-87af26d1d354 admin admin] PATH_INFO: `/v3/projects/40c47c1d80f24ffbae0066b7306f969f/users/46c50ef1735f4b8f903b54127a31030f/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:57:20.302923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc8ab12-0962-42a5-9829-87af26d1d354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40c47c1d80f24ffbae0066b7306f969f, user_id=46c50ef1735f4b8f903b54127a31030f, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:57:20.303974 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-bbc8ab12-0962-42a5-9829-87af26d1d354 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 20:57:20.303974 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 20:57:20.303974 ncdv-master-1 devstack@keystone.service[103103]: May 28 20:57:20.305941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc8ab12-0962-42a5-9829-87af26d1d354 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:57:20.360674 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 763/1501] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 20:57:20 2022] PUT /identity/v3/projects/40c47c1d80f24ffbae0066b7306f969f/users/46c50ef1735f4b8f903b54127a31030f/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:57:20.372543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab55460-195c-4cfe-beb3-e8bcddba659b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:57:20.372835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab55460-195c-4cfe-beb3-e8bcddba659b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:57:20.373116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab55460-195c-4cfe-beb3-e8bcddba659b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:57:20.400345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bab55460-195c-4cfe-beb3-e8bcddba659b None None] MFA Rules not processed for user `46c50ef1735f4b8f903b54127a31030f`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:57:20.401485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bab55460-195c-4cfe-beb3-e8bcddba659b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:57:20.465078 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 739/1502] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 20:57:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:57:20.478599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614999c-0a65-4eb3-9736-0ac849685311 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:57:20.479107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614999c-0a65-4eb3-9736-0ac849685311 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:57:20.479411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c614999c-0a65-4eb3-9736-0ac849685311 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:57:20.480774 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 764/1503] 10.222.0.33 () {54 vars in 963 bytes} [Sat May 28 20:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:57:20.485726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d7dad-8de7-45b7-bfc7-11d22fb8f503 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:57:20.486086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d7dad-8de7-45b7-bfc7-11d22fb8f503 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:57:20.486900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d7dad-8de7-45b7-bfc7-11d22fb8f503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:57:20.505400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-531d7dad-8de7-45b7-bfc7-11d22fb8f503 None None] MFA Rules not processed for user `afc08d15f24e46e29c34d618d955bd28`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:57:20.506679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-531d7dad-8de7-45b7-bfc7-11d22fb8f503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:57:20.549918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:57:20.557435 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 740/1504] 10.222.0.33 () {58 vars in 1060 bytes} [Sat May 28 20:57:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 20:57:20.565349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614999c-0a65-4eb3-9736-0ac849685311 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:57:20.575889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c614999c-0a65-4eb3-9736-0ac849685311 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:57:20.579444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a0745-4af2-486e-9903-f93a4e31415d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:57:20.580078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a0745-4af2-486e-9903-f93a4e31415d service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:57:20.580365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a0745-4af2-486e-9903-f93a4e31415d service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:57:20.580642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a0745-4af2-486e-9903-f93a4e31415d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:57:20.588974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a0745-4af2-486e-9903-f93a4e31415d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:57:20.591045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a0745-4af2-486e-9903-f93a4e31415d service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:57:20.599953 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 765/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:57:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 20:57:20.643500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ef173-2fae-4a8c-811d-6aa6dbaf98ab None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:57:20.643789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ef173-2fae-4a8c-811d-6aa6dbaf98ab None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:57:20.644069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ef173-2fae-4a8c-811d-6aa6dbaf98ab None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:57:20.645354 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 741/1506] 10.222.0.33 () {52 vars in 829 bytes} [Sat May 28 20:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 20:57:20.649051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a0745-4af2-486e-9903-f93a4e31415d service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:57:20.715253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a0745-4af2-486e-9903-f93a4e31415d service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:57:20.719660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec54b6-5510-429b-9e42-3b7363112abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:57:20.721033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec54b6-5510-429b-9e42-3b7363112abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:57:20.721314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec54b6-5510-429b-9e42-3b7363112abd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:57:20.721585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec54b6-5510-429b-9e42-3b7363112abd admin admin] PATH_INFO: `/v3/users/46c50ef1735f4b8f903b54127a31030f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:57:20.722801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec54b6-5510-429b-9e42-3b7363112abd admin admin] RBAC: Authorizing `identity:delete_user(user_id=46c50ef1735f4b8f903b54127a31030f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:57:20.723806 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-f3ec54b6-5510-429b-9e42-3b7363112abd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 20:57:20.723806 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 20:57:20.723806 ncdv-master-1 devstack@keystone.service[103103]: May 28 20:57:20.725000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec54b6-5510-429b-9e42-3b7363112abd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:57:20.943342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3ec54b6-5510-429b-9e42-3b7363112abd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46c50ef1735f4b8f903b54127a31030f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 20:57:20.955489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3ec54b6-5510-429b-9e42-3b7363112abd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46c50ef1735f4b8f903b54127a31030f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 20:57:20.957816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3ec54b6-5510-429b-9e42-3b7363112abd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46c50ef1735f4b8f903b54127a31030f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 20:57:20.957816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3ec54b6-5510-429b-9e42-3b7363112abd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46c50ef1735f4b8f903b54127a31030f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 20:57:21.028335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3ec54b6-5510-429b-9e42-3b7363112abd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46c50ef1735f4b8f903b54127a31030f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 20:57:21.045324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:57:21.053498 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 766/1507] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 20:57:20 2022] DELETE /identity/v3/users/46c50ef1735f4b8f903b54127a31030f => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:57:21.063480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7dd80-1e5a-4f25-996e-788497208036 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:57:21.063910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7dd80-1e5a-4f25-996e-788497208036 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:57:21.064053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7dd80-1e5a-4f25-996e-788497208036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:57:21.082809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-89c7dd80-1e5a-4f25-996e-788497208036 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 20:57:21.084027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-89c7dd80-1e5a-4f25-996e-788497208036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:57:21.143172 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 742/1508] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 20:57:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 20:57:21.153420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec54b6-5510-429b-9e42-3b7363112abd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:57:21.154841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f3ec54b6-5510-429b-9e42-3b7363112abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:57:21.212837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec54b6-5510-429b-9e42-3b7363112abd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:57:21.216452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683a4f1-f0f2-48f9-ac0f-06ece1053b89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:57:21.217093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683a4f1-f0f2-48f9-ac0f-06ece1053b89 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:57:21.217376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683a4f1-f0f2-48f9-ac0f-06ece1053b89 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:57:21.217653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683a4f1-f0f2-48f9-ac0f-06ece1053b89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:57:21.227334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7683a4f1-f0f2-48f9-ac0f-06ece1053b89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:57:21.228789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7683a4f1-f0f2-48f9-ac0f-06ece1053b89 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:57:21.238108 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 767/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 20:57:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 20:57:21.704799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7dd80-1e5a-4f25-996e-788497208036 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 20:57:21.706375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-89c7dd80-1e5a-4f25-996e-788497208036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 20:57:21.718745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7dd80-1e5a-4f25-996e-788497208036 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 20:57:21.723453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016be18-7650-4b80-a397-d38f40e83e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 20:57:21.724309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016be18-7650-4b80-a397-d38f40e83e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 20:57:21.724586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016be18-7650-4b80-a397-d38f40e83e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 20:57:21.724863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016be18-7650-4b80-a397-d38f40e83e06 admin admin] PATH_INFO: `/v3/projects/40c47c1d80f24ffbae0066b7306f969f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 20:57:21.726079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016be18-7650-4b80-a397-d38f40e83e06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40c47c1d80f24ffbae0066b7306f969f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 20:57:21.730447 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-6016be18-7650-4b80-a397-d38f40e83e06 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 20:57:21.730447 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 20:57:21.730447 ncdv-master-1 devstack@keystone.service[103104]: May 28 20:57:21.730447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016be18-7650-4b80-a397-d38f40e83e06 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 20:57:22.250670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6016be18-7650-4b80-a397-d38f40e83e06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40c47c1d80f24ffbae0066b7306f969f', 'request_id': 'req-6016be18-7650-4b80-a397-d38f40e83e06'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 20:57:22.256589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6016be18-7650-4b80-a397-d38f40e83e06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40c47c1d80f24ffbae0066b7306f969f', 'request_id': 'req-6016be18-7650-4b80-a397-d38f40e83e06'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 20:57:22.313866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6016be18-7650-4b80-a397-d38f40e83e06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40c47c1d80f24ffbae0066b7306f969f', 'request_id': 'req-6016be18-7650-4b80-a397-d38f40e83e06'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 20:57:22.332104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6016be18-7650-4b80-a397-d38f40e83e06 admin admin] The token cache is being invalidate because project 40c47c1d80f24ffbae0066b7306f969f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 20:57:22.332393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6016be18-7650-4b80-a397-d38f40e83e06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 20:57:22.333120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6016be18-7650-4b80-a397-d38f40e83e06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 20:57:22.334739 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 743/1510] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 20:57:21 2022] DELETE /identity/v3/projects/40c47c1d80f24ffbae0066b7306f969f => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 20:57:31.234063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:57:32.330062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:00:25.149148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548394a-f109-4e4c-a827-456da3b0e9a1 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:00:25.149740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548394a-f109-4e4c-a827-456da3b0e9a1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:00:25.149740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548394a-f109-4e4c-a827-456da3b0e9a1 None None] PATH_INFO: `/v3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:00:25.151154 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 768/1511] 10.222.0.41 () {54 vars in 925 bytes} [Sat May 28 21:00:25 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:00:25.156987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43180d2-4f37-4088-a23e-4a0c6e66b29c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:00:25.157397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43180d2-4f37-4088-a23e-4a0c6e66b29c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:00:25.157552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43180d2-4f37-4088-a23e-4a0c6e66b29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:00:25.177778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a43180d2-4f37-4088-a23e-4a0c6e66b29c None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:00:25.179031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a43180d2-4f37-4088-a23e-4a0c6e66b29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:00:25.367381 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 744/1512] 10.222.0.41 () {58 vars in 1015 bytes} [Sat May 28 21:00:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:00:27.511888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328adabf-4c52-4c3e-b2b3-49cdf4e26ecf None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:00:27.512488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328adabf-4c52-4c3e-b2b3-49cdf4e26ecf None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:00:27.512488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328adabf-4c52-4c3e-b2b3-49cdf4e26ecf None None] PATH_INFO: `/v3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:00:27.513789 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 769/1513] 10.222.0.15 () {54 vars in 925 bytes} [Sat May 28 21:00:27 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:00:27.520034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a71a64-667a-4079-a22a-f7c2b6932486 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:00:27.520478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a71a64-667a-4079-a22a-f7c2b6932486 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:00:27.520626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a71a64-667a-4079-a22a-f7c2b6932486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:00:27.539574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-14a71a64-667a-4079-a22a-f7c2b6932486 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:00:27.540780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-14a71a64-667a-4079-a22a-f7c2b6932486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:00:27.549417 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 745/1514] 10.222.0.15 () {58 vars in 1015 bytes} [Sat May 28 21:00:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:00:29.494687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3004ac6-db24-4a0f-a669-4e6753328b74 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:00:29.495229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3004ac6-db24-4a0f-a669-4e6753328b74 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:00:29.495334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3004ac6-db24-4a0f-a669-4e6753328b74 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:00:29.496696 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 770/1515] 10.222.0.33 () {54 vars in 960 bytes} [Sat May 28 21:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:00:29.501750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec309c-17f1-4d1b-9126-6f6a8ea37944 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:00:29.502183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec309c-17f1-4d1b-9126-6f6a8ea37944 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:00:29.502343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec309c-17f1-4d1b-9126-6f6a8ea37944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:00:29.522132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-84ec309c-17f1-4d1b-9126-6f6a8ea37944 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:00:29.523838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-84ec309c-17f1-4d1b-9126-6f6a8ea37944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:00:29.533100 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 746/1516] 10.222.0.33 () {58 vars in 1057 bytes} [Sat May 28 21:00:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:00:29.541569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3004ac6-db24-4a0f-a669-4e6753328b74 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:00:29.555438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3004ac6-db24-4a0f-a669-4e6753328b74 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:00:29.560725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e0229-5591-4dce-b581-34df14503c46 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:00:29.561445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e0229-5591-4dce-b581-34df14503c46 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:00:29.561801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e0229-5591-4dce-b581-34df14503c46 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:00:29.562186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e0229-5591-4dce-b581-34df14503c46 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:00:29.571120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e0229-5591-4dce-b581-34df14503c46 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:00:29.572733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e0229-5591-4dce-b581-34df14503c46 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:00:29.583350 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 771/1517] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:00:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:00:31.235796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948b3f8-e74f-4e0c-b3ba-c42aa0974b62 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:00:31.236217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948b3f8-e74f-4e0c-b3ba-c42aa0974b62 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:00:31.236353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948b3f8-e74f-4e0c-b3ba-c42aa0974b62 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:00:31.237784 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 747/1518] 10.222.0.33 () {54 vars in 960 bytes} [Sat May 28 21:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:00:31.242969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed599362-468c-4bc6-9384-70cdc683a574 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:00:31.243398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed599362-468c-4bc6-9384-70cdc683a574 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:00:31.243553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed599362-468c-4bc6-9384-70cdc683a574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:00:31.261681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ed599362-468c-4bc6-9384-70cdc683a574 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:00:31.265312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ed599362-468c-4bc6-9384-70cdc683a574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:00:31.271952 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 772/1519] 10.222.0.33 () {58 vars in 1057 bytes} [Sat May 28 21:00:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:00:31.342463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948b3f8-e74f-4e0c-b3ba-c42aa0974b62 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:00:31.355900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948b3f8-e74f-4e0c-b3ba-c42aa0974b62 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:00:31.361237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584e8e59-54a9-4842-9638-2f4a958aa710 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:00:31.361933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e8e59-54a9-4842-9638-2f4a958aa710 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:00:31.362221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e8e59-54a9-4842-9638-2f4a958aa710 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:00:31.362494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584e8e59-54a9-4842-9638-2f4a958aa710 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:00:31.370645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e8e59-54a9-4842-9638-2f4a958aa710 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:00:31.372278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584e8e59-54a9-4842-9638-2f4a958aa710 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:00:31.382660 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 748/1520] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:00:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:00:41.256601 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=9 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:00:41.378658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=41 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:01:07.484469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a17c6-92c9-4e8c-bf99-15b55c36615d None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:01:07.485035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a17c6-92c9-4e8c-bf99-15b55c36615d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:01:07.485035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19a17c6-92c9-4e8c-bf99-15b55c36615d None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:01:07.486502 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 773/1521] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 21:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:01:07.491377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6b328-cf0b-4812-9572-a08832bf2149 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:01:07.491810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6b328-cf0b-4812-9572-a08832bf2149 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:01:07.491921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b6b328-cf0b-4812-9572-a08832bf2149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:01:07.510471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a1b6b328-cf0b-4812-9572-a08832bf2149 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:01:07.511679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a1b6b328-cf0b-4812-9572-a08832bf2149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:01:07.570689 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 749/1522] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 21:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:01:07.744041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a17c6-92c9-4e8c-bf99-15b55c36615d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:01:07.745905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a19a17c6-92c9-4e8c-bf99-15b55c36615d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:01:07.819946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19a17c6-92c9-4e8c-bf99-15b55c36615d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:01:07.825276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ec0d8-3f0d-4973-8d41-b7e1feabcc72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:01:07.825946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ec0d8-3f0d-4973-8d41-b7e1feabcc72 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:01:07.826231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ec0d8-3f0d-4973-8d41-b7e1feabcc72 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:01:07.826509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ec0d8-3f0d-4973-8d41-b7e1feabcc72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:01:07.834565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ec0d8-3f0d-4973-8d41-b7e1feabcc72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:01:07.836142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ec0d8-3f0d-4973-8d41-b7e1feabcc72 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:01:07.845439 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 774/1523] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:01:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:01:08.722746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589282d5-d74b-418e-b955-b1da608b9d89 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:01:08.723803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589282d5-d74b-418e-b955-b1da608b9d89 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:01:08.723803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589282d5-d74b-418e-b955-b1da608b9d89 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:01:08.725293 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 750/1524] 127.0.0.1 () {54 vars in 901 bytes} [Sat May 28 21:01:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:01:08.730374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7a6fa-89c8-4e2a-b7ae-858bd2a249bc None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:01:08.731186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7a6fa-89c8-4e2a-b7ae-858bd2a249bc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:01:08.731471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7a6fa-89c8-4e2a-b7ae-858bd2a249bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:01:08.751924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9de7a6fa-89c8-4e2a-b7ae-858bd2a249bc None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:01:08.753162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9de7a6fa-89c8-4e2a-b7ae-858bd2a249bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:01:08.762268 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 775/1525] 127.0.0.1 () {58 vars in 998 bytes} [Sat May 28 21:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:01:08.920456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589282d5-d74b-418e-b955-b1da608b9d89 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:01:08.935587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589282d5-d74b-418e-b955-b1da608b9d89 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:01:08.941126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f11da-54ee-41d5-a783-46e6ecf3c699 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:01:08.941763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f11da-54ee-41d5-a783-46e6ecf3c699 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:01:08.942731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f11da-54ee-41d5-a783-46e6ecf3c699 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:01:08.943014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f11da-54ee-41d5-a783-46e6ecf3c699 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:01:08.951453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f11da-54ee-41d5-a783-46e6ecf3c699 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:01:08.953052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7f11da-54ee-41d5-a783-46e6ecf3c699 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:01:08.962736 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 751/1526] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:01:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:01:12.928152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530e909-aefd-46fa-bfd5-283ae61f4549 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:01:12.928757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530e909-aefd-46fa-bfd5-283ae61f4549 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:01:12.928757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530e909-aefd-46fa-bfd5-283ae61f4549 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:01:12.930127 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 776/1527] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 21:01:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:01:12.935448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d02fba-b87c-435a-b4bc-76cc3b67a32c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:01:12.935912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d02fba-b87c-435a-b4bc-76cc3b67a32c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:01:12.936020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d02fba-b87c-435a-b4bc-76cc3b67a32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:01:12.954407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-90d02fba-b87c-435a-b4bc-76cc3b67a32c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:01:12.955597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-90d02fba-b87c-435a-b4bc-76cc3b67a32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:01:12.963426 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 752/1528] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 21:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:01:13.001067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530e909-aefd-46fa-bfd5-283ae61f4549 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:01:13.002785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2530e909-aefd-46fa-bfd5-283ae61f4549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:01:13.059001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530e909-aefd-46fa-bfd5-283ae61f4549 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:01:13.062630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43785ce-e3db-4a78-9916-62573b600483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:01:13.063282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43785ce-e3db-4a78-9916-62573b600483 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:01:13.063573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43785ce-e3db-4a78-9916-62573b600483 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:01:13.063872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43785ce-e3db-4a78-9916-62573b600483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:01:13.073141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43785ce-e3db-4a78-9916-62573b600483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:01:13.074620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43785ce-e3db-4a78-9916-62573b600483 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:01:13.083937 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 777/1529] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:01:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:01:15.357687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e6e07-706c-4152-a87b-ed4bc23583e1 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:01:15.358245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e6e07-706c-4152-a87b-ed4bc23583e1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:01:15.358245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e6e07-706c-4152-a87b-ed4bc23583e1 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:01:15.359571 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 753/1530] 127.0.0.1 () {54 vars in 916 bytes} [Sat May 28 21:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:01:15.364911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e69eb2-02d6-4edf-aba9-7c7767039c85 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:01:15.365330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e69eb2-02d6-4edf-aba9-7c7767039c85 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:01:15.365491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e69eb2-02d6-4edf-aba9-7c7767039c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:01:15.384915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c7e69eb2-02d6-4edf-aba9-7c7767039c85 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:01:15.386154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c7e69eb2-02d6-4edf-aba9-7c7767039c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:01:15.394087 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 778/1531] 127.0.0.1 () {58 vars in 1013 bytes} [Sat May 28 21:01:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:01:15.432991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e6e07-706c-4152-a87b-ed4bc23583e1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:01:15.434747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-414e6e07-706c-4152-a87b-ed4bc23583e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:01:15.448272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e6e07-706c-4152-a87b-ed4bc23583e1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:01:15.451732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aad5b3-e7bd-4345-bb63-b63ded3164a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:01:15.452410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aad5b3-e7bd-4345-bb63-b63ded3164a6 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:01:15.452690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aad5b3-e7bd-4345-bb63-b63ded3164a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:01:15.452960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aad5b3-e7bd-4345-bb63-b63ded3164a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:01:15.462474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aad5b3-e7bd-4345-bb63-b63ded3164a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:01:15.463898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aad5b3-e7bd-4345-bb63-b63ded3164a6 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:01:15.473085 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 754/1532] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:01:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:01:25.378932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=66 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:01:25.469010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=42 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:08.200230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b87667-44d0-42a5-aaf9-99d29bfd056d None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:08.200780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b87667-44d0-42a5-aaf9-99d29bfd056d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:08.200780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b87667-44d0-42a5-aaf9-99d29bfd056d None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:08.202213 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 779/1533] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:02:08.207262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff2291-7d7d-4799-ad5b-3a658b686a38 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:08.207700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff2291-7d7d-4799-ad5b-3a658b686a38 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:08.207967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff2291-7d7d-4799-ad5b-3a658b686a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:08.226761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a8ff2291-7d7d-4799-ad5b-3a658b686a38 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:08.228049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a8ff2291-7d7d-4799-ad5b-3a658b686a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:08.237069 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 755/1534] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:02:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:02:18.221378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:46.429082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fc5ab-9a80-42ac-9d2b-e8964cda9f78 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:46.429642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fc5ab-9a80-42ac-9d2b-e8964cda9f78 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:46.429642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740fc5ab-9a80-42ac-9d2b-e8964cda9f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:46.538535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-740fc5ab-9a80-42ac-9d2b-e8964cda9f78 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:46.539773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-740fc5ab-9a80-42ac-9d2b-e8964cda9f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:46.624254 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 780/1535] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:46.635375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6721d0-ef57-483c-ba95-9b48cdbb9cae None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:46.635915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6721d0-ef57-483c-ba95-9b48cdbb9cae None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:46.635915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6721d0-ef57-483c-ba95-9b48cdbb9cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:46.664909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e2c66-4fce-4ea4-84be-54db9a56b08e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:46.666462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e2c66-4fce-4ea4-84be-54db9a56b08e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:46.666462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e2c66-4fce-4ea4-84be-54db9a56b08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:46.671188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2f6721d0-ef57-483c-ba95-9b48cdbb9cae None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:46.672445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2f6721d0-ef57-483c-ba95-9b48cdbb9cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:46.687755 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 756/1536] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:02:46.692395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6721d0-ef57-483c-ba95-9b48cdbb9cae None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:46.692835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-477e2c66-4fce-4ea4-84be-54db9a56b08e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:46.694147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-477e2c66-4fce-4ea4-84be-54db9a56b08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:46.706849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6721d0-ef57-483c-ba95-9b48cdbb9cae None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:46.707314 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 781/1537] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:02:46.719064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9849b-e7d4-4fb8-bb1b-5d303e67fdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:46.719736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9849b-e7d4-4fb8-bb1b-5d303e67fdd5 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:46.719846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df45d70-97d8-4121-995f-641d19498d16 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:46.720008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9849b-e7d4-4fb8-bb1b-5d303e67fdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:46.720103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df45d70-97d8-4121-995f-641d19498d16 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:46.720281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9849b-e7d4-4fb8-bb1b-5d303e67fdd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:46.720379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df45d70-97d8-4121-995f-641d19498d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:46.721015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9849b-e7d4-4fb8-bb1b-5d303e67fdd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:02:46.721322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9849b-e7d4-4fb8-bb1b-5d303e67fdd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:46.722986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9849b-e7d4-4fb8-bb1b-5d303e67fdd5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:46.747175 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 757/1538] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:02:46.752852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5df45d70-97d8-4121-995f-641d19498d16 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:46.754796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9849b-e7d4-4fb8-bb1b-5d303e67fdd5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:46.757624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5df45d70-97d8-4121-995f-641d19498d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:46.767332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9849b-e7d4-4fb8-bb1b-5d303e67fdd5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:46.774341 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 782/1539] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 28 21:02:46.779051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df45d70-97d8-4121-995f-641d19498d16 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:46.779414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e721f-15f2-466e-8691-f621d94bb959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:46.780089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e721f-15f2-466e-8691-f621d94bb959 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:46.780369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e721f-15f2-466e-8691-f621d94bb959 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:46.780646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e721f-15f2-466e-8691-f621d94bb959 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:46.781341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e721f-15f2-466e-8691-f621d94bb959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:46.782518 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-ff6e721f-15f2-466e-8691-f621d94bb959 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:02:46.782518 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:02:46.782518 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:02:46.783743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e721f-15f2-466e-8691-f621d94bb959 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:46.795605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df45d70-97d8-4121-995f-641d19498d16 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:46.804509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2767b1-a3dd-435a-a592-4eba581c2e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:46.805208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2767b1-a3dd-435a-a592-4eba581c2e05 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:46.805560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2767b1-a3dd-435a-a592-4eba581c2e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:46.805861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2767b1-a3dd-435a-a592-4eba581c2e05 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:46.806623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2767b1-a3dd-435a-a592-4eba581c2e05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:02:46.806936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2767b1-a3dd-435a-a592-4eba581c2e05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:46.808178 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-aa2767b1-a3dd-435a-a592-4eba581c2e05 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:02:46.808178 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:02:46.808178 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:02:46.809113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2767b1-a3dd-435a-a592-4eba581c2e05 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:46.832746 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 783/1540] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:02:46.842516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2767b1-a3dd-435a-a592-4eba581c2e05 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:46.857625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2767b1-a3dd-435a-a592-4eba581c2e05 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:46.866119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9c4c2-7c3d-4588-93ba-8ef06c6bd90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:46.866824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9c4c2-7c3d-4588-93ba-8ef06c6bd90a admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:46.867116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9c4c2-7c3d-4588-93ba-8ef06c6bd90a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:46.867439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d9c4c2-7c3d-4588-93ba-8ef06c6bd90a admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:46.868160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9c4c2-7c3d-4588-93ba-8ef06c6bd90a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:46.870276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d9c4c2-7c3d-4588-93ba-8ef06c6bd90a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:46.903641 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 758/1541] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:02:46 2022] POST /identity/v3/projects => generated 397 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:02:46.908150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fba62-b5e2-41f8-ade6-8fa4bf73297e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:46.909906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fba62-b5e2-41f8-ade6-8fa4bf73297e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:46.909906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70fba62-b5e2-41f8-ade6-8fa4bf73297e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:46.951150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e70fba62-b5e2-41f8-ade6-8fa4bf73297e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:46.952519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e70fba62-b5e2-41f8-ade6-8fa4bf73297e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:46.975058 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 784/1542] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:02:46 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:02:46.977759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9c4c2-7c3d-4588-93ba-8ef06c6bd90a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:47.082069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d9c4c2-7c3d-4588-93ba-8ef06c6bd90a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:47.092702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edeb5cc-9f41-4635-8390-b19258e0069c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:47.093464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edeb5cc-9f41-4635-8390-b19258e0069c admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.093755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edeb5cc-9f41-4635-8390-b19258e0069c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.094061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edeb5cc-9f41-4635-8390-b19258e0069c admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.094776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edeb5cc-9f41-4635-8390-b19258e0069c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:47.095726 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 759/1543] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:02:47.095899 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-4edeb5cc-9f41-4635-8390-b19258e0069c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:02:47.095899 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:02:47.095899 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:02:47.096602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70fba62-b5e2-41f8-ade6-8fa4bf73297e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:47.097167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edeb5cc-9f41-4635-8390-b19258e0069c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:47.119937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70fba62-b5e2-41f8-ade6-8fa4bf73297e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:47.137858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d11d9-b973-482c-a377-1deb66a9c88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:47.138646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d11d9-b973-482c-a377-1deb66a9c88c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.138943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d11d9-b973-482c-a377-1deb66a9c88c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.139229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d11d9-b973-482c-a377-1deb66a9c88c admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.140080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d11d9-b973-482c-a377-1deb66a9c88c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:47.143087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d11d9-b973-482c-a377-1deb66a9c88c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:47.196567 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 785/1544] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:02:46 2022] POST /identity/v3/users => generated 299 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:47.198203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae5242-2671-49e0-b213-d698102f2117 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.198611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae5242-2671-49e0-b213-d698102f2117 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.198897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae5242-2671-49e0-b213-d698102f2117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.220929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:47.221729 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 760/1545] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:02:47 2022] POST /identity/v3/users => generated 292 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:47.223862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907aafd-ea69-4f2f-90a9-785891afd359 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.224367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907aafd-ea69-4f2f-90a9-785891afd359 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.224928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d907aafd-ea69-4f2f-90a9-785891afd359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.242882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-dcae5242-2671-49e0-b213-d698102f2117 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:47.244126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dcae5242-2671-49e0-b213-d698102f2117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:47.257031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d907aafd-ea69-4f2f-90a9-785891afd359 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:47.259130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d907aafd-ea69-4f2f-90a9-785891afd359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:47.267049 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 786/1546] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:47.268223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd92d3-ebc0-4d08-b5e9-c0b21f137b3c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.268514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd92d3-ebc0-4d08-b5e9-c0b21f137b3c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.268791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bd92d3-ebc0-4d08-b5e9-c0b21f137b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.276161 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 761/1547] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:47.276870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907aafd-ea69-4f2f-90a9-785891afd359 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:47.289566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d907aafd-ea69-4f2f-90a9-785891afd359 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:47.294689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03ab99-6111-42ae-abc6-febead353931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:47.295491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03ab99-6111-42ae-abc6-febead353931 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.295932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03ab99-6111-42ae-abc6-febead353931 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.296094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03ab99-6111-42ae-abc6-febead353931 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.296841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03ab99-6111-42ae-abc6-febead353931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:47.298933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03ab99-6111-42ae-abc6-febead353931 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:47.300403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-92bd92d3-ebc0-4d08-b5e9-c0b21f137b3c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:47.301724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-92bd92d3-ebc0-4d08-b5e9-c0b21f137b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:47.309820 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 762/1548] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:47.310252 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 787/1549] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:47.310684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03ab99-6111-42ae-abc6-febead353931 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:47.311649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfefc8-0a10-4c58-94ab-9c6c204e24b0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.314073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfefc8-0a10-4c58-94ab-9c6c204e24b0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.314387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfefc8-0a10-4c58-94ab-9c6c204e24b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.321523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03ab99-6111-42ae-abc6-febead353931 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:47.334114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c9dfefc8-0a10-4c58-94ab-9c6c204e24b0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:47.335648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e315cb-976e-4629-b7f9-9612acc0a0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:47.335899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c9dfefc8-0a10-4c58-94ab-9c6c204e24b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:47.336314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e315cb-976e-4629-b7f9-9612acc0a0dd admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.336597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e315cb-976e-4629-b7f9-9612acc0a0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.336874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e315cb-976e-4629-b7f9-9612acc0a0dd admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.337669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e315cb-976e-4629-b7f9-9612acc0a0dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:47.339904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e315cb-976e-4629-b7f9-9612acc0a0dd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:47.344564 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 788/1550] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:47.346084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb5e42-b172-4dd3-8449-8d699b4fd871 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.346535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb5e42-b172-4dd3-8449-8d699b4fd871 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.346972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb5e42-b172-4dd3-8449-8d699b4fd871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.350696 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 763/1551] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:47.352182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee6ea5-05ff-4df2-b098-24c7621a16fd None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.352481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee6ea5-05ff-4df2-b098-24c7621a16fd None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.352765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee6ea5-05ff-4df2-b098-24c7621a16fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.367256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-efeb5e42-b172-4dd3-8449-8d699b4fd871 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:47.369092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-efeb5e42-b172-4dd3-8449-8d699b4fd871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:47.373488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-68ee6ea5-05ff-4df2-b098-24c7621a16fd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:47.374752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-68ee6ea5-05ff-4df2-b098-24c7621a16fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:47.378391 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 789/1552] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:47.384297 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 764/1553] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:47.385660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09053f-c525-462e-8881-3f712a7ffa91 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.385983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09053f-c525-462e-8881-3f712a7ffa91 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.386262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09053f-c525-462e-8881-3f712a7ffa91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.387144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc00ff-2c5c-41b1-ac5b-bf2bc13c4fca None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.387433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc00ff-2c5c-41b1-ac5b-bf2bc13c4fca None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.387712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc00ff-2c5c-41b1-ac5b-bf2bc13c4fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.407368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-da09053f-c525-462e-8881-3f712a7ffa91 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:47.408580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-da09053f-c525-462e-8881-3f712a7ffa91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:47.410311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-20dc00ff-2c5c-41b1-ac5b-bf2bc13c4fca None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:47.411463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-20dc00ff-2c5c-41b1-ac5b-bf2bc13c4fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:47.416741 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 765/1554] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:47.425999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09053f-c525-462e-8881-3f712a7ffa91 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:47.426996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09053f-c525-462e-8881-3f712a7ffa91 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:47.427440 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 790/1555] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:47.428274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc00ff-2c5c-41b1-ac5b-bf2bc13c4fca None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:47.432697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06075312-0a11-434e-a81f-96751e38a334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:47.433352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06075312-0a11-434e-a81f-96751e38a334 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.433631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06075312-0a11-434e-a81f-96751e38a334 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.433961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06075312-0a11-434e-a81f-96751e38a334 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.434656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06075312-0a11-434e-a81f-96751e38a334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:47.436286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06075312-0a11-434e-a81f-96751e38a334 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:47.439726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc00ff-2c5c-41b1-ac5b-bf2bc13c4fca None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:47.446079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a464f-1499-4704-b7c1-473fdee88695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:47.446764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a464f-1499-4704-b7c1-473fdee88695 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.447075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a464f-1499-4704-b7c1-473fdee88695 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.447352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a464f-1499-4704-b7c1-473fdee88695 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.447791 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 766/1556] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:47.448626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06075312-0a11-434e-a81f-96751e38a334 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:47.451362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a464f-1499-4704-b7c1-473fdee88695 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:02:47.451678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a464f-1499-4704-b7c1-473fdee88695 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:47.453326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a464f-1499-4704-b7c1-473fdee88695 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:47.458432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06075312-0a11-434e-a81f-96751e38a334 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:47.463149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466e403-e251-4af5-8444-42a4dbb93fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:47.463796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466e403-e251-4af5-8444-42a4dbb93fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.464075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466e403-e251-4af5-8444-42a4dbb93fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.464347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6466e403-e251-4af5-8444-42a4dbb93fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.465065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6466e403-e251-4af5-8444-42a4dbb93fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:47.471915 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 791/1557] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:02:47.472040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6466e403-e251-4af5-8444-42a4dbb93fd7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:47.472119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec6264-06c0-4880-b4c9-eeff1b392ffa None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.472119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec6264-06c0-4880-b4c9-eeff1b392ffa None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.472119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec6264-06c0-4880-b4c9-eeff1b392ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.476993 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 767/1558] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:47.477785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466e403-e251-4af5-8444-42a4dbb93fd7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:47.487467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6466e403-e251-4af5-8444-42a4dbb93fd7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:47.491407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-44ec6264-06c0-4880-b4c9-eeff1b392ffa None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:47.492630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-44ec6264-06c0-4880-b4c9-eeff1b392ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:47.492755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022e297-2e62-43a5-8940-923935a49e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:47.493280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022e297-2e62-43a5-8940-923935a49e21 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.493719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022e297-2e62-43a5-8940-923935a49e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.494030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022e297-2e62-43a5-8940-923935a49e21 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.494751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022e297-2e62-43a5-8940-923935a49e21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:02:47.495062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022e297-2e62-43a5-8940-923935a49e21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:47.496914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022e297-2e62-43a5-8940-923935a49e21 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:47.500781 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 792/1559] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:47.502089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba1267-1e3c-4025-8b9c-ca00e63e103e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.502387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba1267-1e3c-4025-8b9c-ca00e63e103e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.502768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba1267-1e3c-4025-8b9c-ca00e63e103e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.509738 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 768/1560] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:02:47.510999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c41e16-1cf5-4805-83f9-3da94a11e11e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.511283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c41e16-1cf5-4805-83f9-3da94a11e11e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.511559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c41e16-1cf5-4805-83f9-3da94a11e11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.527793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-89ba1267-1e3c-4025-8b9c-ca00e63e103e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:47.528988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-89ba1267-1e3c-4025-8b9c-ca00e63e103e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:47.529182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-62c41e16-1cf5-4805-83f9-3da94a11e11e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:47.530396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-62c41e16-1cf5-4805-83f9-3da94a11e11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:47.536863 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 793/1561] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:47.537620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba1267-1e3c-4025-8b9c-ca00e63e103e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:47.542206 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 769/1562] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:47.542206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c41e16-1cf5-4805-83f9-3da94a11e11e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:47.549985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba1267-1e3c-4025-8b9c-ca00e63e103e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:47.552303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c41e16-1cf5-4805-83f9-3da94a11e11e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:47.555149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d9ad0-44bd-411f-8e07-448cb3c8c9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:47.555800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d9ad0-44bd-411f-8e07-448cb3c8c9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.556087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d9ad0-44bd-411f-8e07-448cb3c8c9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.556367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d9ad0-44bd-411f-8e07-448cb3c8c9e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.557127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d9ad0-44bd-411f-8e07-448cb3c8c9e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:02:47.557439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d9ad0-44bd-411f-8e07-448cb3c8c9e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:47.557536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19526bce-8c76-4e17-b33b-6c254c5173c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:47.558399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19526bce-8c76-4e17-b33b-6c254c5173c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.558678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19526bce-8c76-4e17-b33b-6c254c5173c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.558950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19526bce-8c76-4e17-b33b-6c254c5173c1 admin admin] PATH_INFO: `/v3/projects/7133f6301e034a2faf8215358bf3e691/users/d870c609a91f4540bfbd7471106fc398/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.559124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d9ad0-44bd-411f-8e07-448cb3c8c9e7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:47.571660 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 794/1563] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:02:47.573808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19526bce-8c76-4e17-b33b-6c254c5173c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7133f6301e034a2faf8215358bf3e691, user_id=d870c609a91f4540bfbd7471106fc398, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:47.574887 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-19526bce-8c76-4e17-b33b-6c254c5173c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:02:47.574887 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:02:47.574887 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:02:47.575129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d9ad0-44bd-411f-8e07-448cb3c8c9e7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:47.576840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19526bce-8c76-4e17-b33b-6c254c5173c1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:47.579292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:47.585645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d9ad0-44bd-411f-8e07-448cb3c8c9e7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:47.591310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9990e63-0734-4e67-9fcd-aa233e596f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:47.591310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9990e63-0734-4e67-9fcd-aa233e596f43 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.591310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9990e63-0734-4e67-9fcd-aa233e596f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.591562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9990e63-0734-4e67-9fcd-aa233e596f43 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.592058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9990e63-0734-4e67-9fcd-aa233e596f43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:47.593977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9990e63-0734-4e67-9fcd-aa233e596f43 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:47.652023 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 770/1564] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:47 2022] PUT /identity/v3/projects/7133f6301e034a2faf8215358bf3e691/users/d870c609a91f4540bfbd7471106fc398/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:47.652890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19526bce-8c76-4e17-b33b-6c254c5173c1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:47.667817 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 795/1565] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:02:47 2022] POST /identity/v3/projects => generated 383 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:47.667817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9990e63-0734-4e67-9fcd-aa233e596f43 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:47.720096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19526bce-8c76-4e17-b33b-6c254c5173c1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:47.724720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d54854-8a97-4fcd-8384-b8768a9c81fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:47.725537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d54854-8a97-4fcd-8384-b8768a9c81fd admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.725818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d54854-8a97-4fcd-8384-b8768a9c81fd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.726154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d54854-8a97-4fcd-8384-b8768a9c81fd admin admin] PATH_INFO: `/v3/projects/083c66f515ee434ea52034fa5348b303/users/c501469061044294a4edd6689b5f0aed/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.726579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9990e63-0734-4e67-9fcd-aa233e596f43 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:47.731150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7314670-4a1e-4260-9231-833cbd65c4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:47.731806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7314670-4a1e-4260-9231-833cbd65c4ee admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.732091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7314670-4a1e-4260-9231-833cbd65c4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.732377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7314670-4a1e-4260-9231-833cbd65c4ee admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.733223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7314670-4a1e-4260-9231-833cbd65c4ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:02:47.733918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7314670-4a1e-4260-9231-833cbd65c4ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:47.735223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7314670-4a1e-4260-9231-833cbd65c4ee admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:47.740527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d54854-8a97-4fcd-8384-b8768a9c81fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=083c66f515ee434ea52034fa5348b303, user_id=c501469061044294a4edd6689b5f0aed, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:47.743764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d54854-8a97-4fcd-8384-b8768a9c81fd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:47.747371 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 796/1566] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:02:47.748171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7314670-4a1e-4260-9231-833cbd65c4ee admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:47.760138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7314670-4a1e-4260-9231-833cbd65c4ee admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:47.764944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991084d1-61d3-43c5-a699-9e0d23a4b3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:47.765577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991084d1-61d3-43c5-a699-9e0d23a4b3b0 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.765915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991084d1-61d3-43c5-a699-9e0d23a4b3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.766203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991084d1-61d3-43c5-a699-9e0d23a4b3b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.766883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991084d1-61d3-43c5-a699-9e0d23a4b3b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:47.768808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991084d1-61d3-43c5-a699-9e0d23a4b3b0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:47.789366 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 771/1567] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:47 2022] PUT /identity/v3/projects/083c66f515ee434ea52034fa5348b303/users/c501469061044294a4edd6689b5f0aed/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:47.790376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d54854-8a97-4fcd-8384-b8768a9c81fd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:47.836281 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 797/1568] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:02:47 2022] POST /identity/v3/projects => generated 377 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:47.837115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991084d1-61d3-43c5-a699-9e0d23a4b3b0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:47.851547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d54854-8a97-4fcd-8384-b8768a9c81fd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:47.883459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:47.908199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991084d1-61d3-43c5-a699-9e0d23a4b3b0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:47.908397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a9377-e9ab-4f4b-b5d9-189a40701977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:47.909072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a9377-e9ab-4f4b-b5d9-189a40701977 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.909355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a9377-e9ab-4f4b-b5d9-189a40701977 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.909632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a9377-e9ab-4f4b-b5d9-189a40701977 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.910485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a9377-e9ab-4f4b-b5d9-189a40701977 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:02:47.910782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a9377-e9ab-4f4b-b5d9-189a40701977 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:47.912414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a9377-e9ab-4f4b-b5d9-189a40701977 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:47.914487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39c042-b441-4b8f-a8df-107a8c2ab20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:47.915149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39c042-b441-4b8f-a8df-107a8c2ab20a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.915433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39c042-b441-4b8f-a8df-107a8c2ab20a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.915739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39c042-b441-4b8f-a8df-107a8c2ab20a admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.916468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39c042-b441-4b8f-a8df-107a8c2ab20a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:02:47.916778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39c042-b441-4b8f-a8df-107a8c2ab20a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:47.918396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a39c042-b441-4b8f-a8df-107a8c2ab20a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:47.924890 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 772/1569] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:02:47.925523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a9377-e9ab-4f4b-b5d9-189a40701977 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:47.931236 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 798/1570] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:02:47.932446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8822be-e39f-428f-adc4-2cf39449dca7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.932992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8822be-e39f-428f-adc4-2cf39449dca7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.933281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8822be-e39f-428f-adc4-2cf39449dca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.943011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a9377-e9ab-4f4b-b5d9-189a40701977 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:47.948022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bc595-2a75-4540-9e57-38d3d5c12205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:47.948679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bc595-2a75-4540-9e57-38d3d5c12205 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:47.948986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bc595-2a75-4540-9e57-38d3d5c12205 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:47.949246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54bc595-2a75-4540-9e57-38d3d5c12205 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:47.949985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54bc595-2a75-4540-9e57-38d3d5c12205 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:47.951915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54bc595-2a75-4540-9e57-38d3d5c12205 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:47.960749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-db8822be-e39f-428f-adc4-2cf39449dca7 None None] MFA Rules not processed for user `d870c609a91f4540bfbd7471106fc398`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:47.961997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-db8822be-e39f-428f-adc4-2cf39449dca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:47.966511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:48.024132 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 773/1571] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:02:47 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:48.025100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bc595-2a75-4540-9e57-38d3d5c12205 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:48.028929 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 799/1572] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:48.029924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8822be-e39f-428f-adc4-2cf39449dca7 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:48.042666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8822be-e39f-428f-adc4-2cf39449dca7 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:48.092835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54bc595-2a75-4540-9e57-38d3d5c12205 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:48.098215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9ddb1-8f33-4dc0-ad5c-4f08b5973fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:48.098878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9ddb1-8f33-4dc0-ad5c-4f08b5973fd6 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.099160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9ddb1-8f33-4dc0-ad5c-4f08b5973fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.099438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9ddb1-8f33-4dc0-ad5c-4f08b5973fd6 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.099845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a888397-1587-4db9-9fc1-0098cba947be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:48.100194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d9ddb1-8f33-4dc0-ad5c-4f08b5973fd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:48.101032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a888397-1587-4db9-9fc1-0098cba947be admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.101294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a888397-1587-4db9-9fc1-0098cba947be admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.101684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a888397-1587-4db9-9fc1-0098cba947be admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.102449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a888397-1587-4db9-9fc1-0098cba947be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:48.103862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d9ddb1-8f33-4dc0-ad5c-4f08b5973fd6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:48.104548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a888397-1587-4db9-9fc1-0098cba947be admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:48.154829 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 774/1573] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:02:48 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:48.156100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a58ef-d23e-42e1-bad0-1b001166b35b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.156388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a58ef-d23e-42e1-bad0-1b001166b35b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.156663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a58ef-d23e-42e1-bad0-1b001166b35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.181898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-088a58ef-d23e-42e1-bad0-1b001166b35b None None] MFA Rules not processed for user `c501469061044294a4edd6689b5f0aed`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:48.183095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-088a58ef-d23e-42e1-bad0-1b001166b35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:48.191673 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 800/1574] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:02:48 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:48.191673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a888397-1587-4db9-9fc1-0098cba947be admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:48.241423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:48.253607 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 775/1575] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:48.255610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a888397-1587-4db9-9fc1-0098cba947be admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:48.258568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a58ef-d23e-42e1-bad0-1b001166b35b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:48.261006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e0862-6d07-46a1-9872-4665235fbaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:48.261868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e0862-6d07-46a1-9872-4665235fbaee admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.262171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e0862-6d07-46a1-9872-4665235fbaee admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.262453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e0862-6d07-46a1-9872-4665235fbaee admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.263291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e0862-6d07-46a1-9872-4665235fbaee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:48.266488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41e0862-6d07-46a1-9872-4665235fbaee admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:48.271927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a58ef-d23e-42e1-bad0-1b001166b35b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:48.274253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609b86b-b090-4277-bbc8-dc7cb23f99c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:48.274952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609b86b-b090-4277-bbc8-dc7cb23f99c4 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.275239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609b86b-b090-4277-bbc8-dc7cb23f99c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.275539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609b86b-b090-4277-bbc8-dc7cb23f99c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.276231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609b86b-b090-4277-bbc8-dc7cb23f99c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:48.278165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609b86b-b090-4277-bbc8-dc7cb23f99c4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:48.355724 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 801/1576] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:02:48 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:48.356680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e0862-6d07-46a1-9872-4665235fbaee admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:48.366050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e0862-6d07-46a1-9872-4665235fbaee admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:48.370663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4413d5-4215-480f-893d-aae6c28ffe5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:48.371552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4413d5-4215-480f-893d-aae6c28ffe5f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.371839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4413d5-4215-480f-893d-aae6c28ffe5f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.372120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4413d5-4215-480f-893d-aae6c28ffe5f admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.372797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4413d5-4215-480f-893d-aae6c28ffe5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:48.374808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4413d5-4215-480f-893d-aae6c28ffe5f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:48.407934 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 776/1577] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:02:48 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:48.408116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609b86b-b090-4277-bbc8-dc7cb23f99c4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:48.457366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:48.467783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609b86b-b090-4277-bbc8-dc7cb23f99c4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:48.472817 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 802/1578] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:02:48 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:48.472817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4413d5-4215-480f-893d-aae6c28ffe5f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:48.473345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b83cc7-d9a9-42f4-8e1e-2279193f3066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:48.473345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b83cc7-d9a9-42f4-8e1e-2279193f3066 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.473508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b83cc7-d9a9-42f4-8e1e-2279193f3066 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.473692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b83cc7-d9a9-42f4-8e1e-2279193f3066 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.474493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b83cc7-d9a9-42f4-8e1e-2279193f3066 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:48.477089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b83cc7-d9a9-42f4-8e1e-2279193f3066 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:48.537028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4413d5-4215-480f-893d-aae6c28ffe5f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:48.541704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b238f97-5b76-48cc-abae-f12c8e2bf64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:48.542392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b238f97-5b76-48cc-abae-f12c8e2bf64a admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.542691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b238f97-5b76-48cc-abae-f12c8e2bf64a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.545959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b238f97-5b76-48cc-abae-f12c8e2bf64a admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.545959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b238f97-5b76-48cc-abae-f12c8e2bf64a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:48.545959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b238f97-5b76-48cc-abae-f12c8e2bf64a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:48.560310 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 777/1579] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:02:48 2022] POST /identity/v3/users => generated 291 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:48.561295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b83cc7-d9a9-42f4-8e1e-2279193f3066 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:48.571650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b83cc7-d9a9-42f4-8e1e-2279193f3066 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:48.576260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ffd4a-a43e-4a13-b9c8-f37e38a3d0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:48.580516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ffd4a-a43e-4a13-b9c8-f37e38a3d0b9 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.580516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ffd4a-a43e-4a13-b9c8-f37e38a3d0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.580516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ffd4a-a43e-4a13-b9c8-f37e38a3d0b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.580516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ffd4a-a43e-4a13-b9c8-f37e38a3d0b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:48.580815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ffd4a-a43e-4a13-b9c8-f37e38a3d0b9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:48.590897 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 778/1580] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:48.591773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ffd4a-a43e-4a13-b9c8-f37e38a3d0b9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:48.600965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ffd4a-a43e-4a13-b9c8-f37e38a3d0b9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:48.605518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7c1e9-ce31-449e-8873-0f6bf7a66d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:48.606155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7c1e9-ce31-449e-8873-0f6bf7a66d44 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.606439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7c1e9-ce31-449e-8873-0f6bf7a66d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.606714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7c1e9-ce31-449e-8873-0f6bf7a66d44 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.607371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7c1e9-ce31-449e-8873-0f6bf7a66d44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:48.609216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7c1e9-ce31-449e-8873-0f6bf7a66d44 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:48.634209 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 803/1581] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:02:48 2022] POST /identity/v3/projects => generated 399 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:48.635048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b238f97-5b76-48cc-abae-f12c8e2bf64a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:48.692879 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 779/1582] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:02:48 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:48.694821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7c1e9-ce31-449e-8873-0f6bf7a66d44 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:48.705163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7c1e9-ce31-449e-8873-0f6bf7a66d44 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:48.716069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b238f97-5b76-48cc-abae-f12c8e2bf64a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:48.719839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb1410-811e-4f3c-8d08-fb15a5a5c940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:48.720497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb1410-811e-4f3c-8d08-fb15a5a5c940 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.720778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb1410-811e-4f3c-8d08-fb15a5a5c940 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.721055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb1410-811e-4f3c-8d08-fb15a5a5c940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.751540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:48.761165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fc741-401f-4a55-8c55-0eb759d689cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:48.761859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fc741-401f-4a55-8c55-0eb759d689cf admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.762197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fc741-401f-4a55-8c55-0eb759d689cf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.764213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082fc741-401f-4a55-8c55-0eb759d689cf admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.764213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082fc741-401f-4a55-8c55-0eb759d689cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:48.764918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082fc741-401f-4a55-8c55-0eb759d689cf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:48.775246 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 780/1583] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:48.776104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fc741-401f-4a55-8c55-0eb759d689cf admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:48.779613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb1410-811e-4f3c-8d08-fb15a5a5c940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:48.781218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb1410-811e-4f3c-8d08-fb15a5a5c940 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:48.785861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082fc741-401f-4a55-8c55-0eb759d689cf admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:48.790423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff0f62-5c8d-4a0f-8ac9-45dfbc7c5fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:48.790928 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 804/1584] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:48.791045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff0f62-5c8d-4a0f-8ac9-45dfbc7c5fc5 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.791379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff0f62-5c8d-4a0f-8ac9-45dfbc7c5fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.791658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb1410-811e-4f3c-8d08-fb15a5a5c940 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:48.791795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff0f62-5c8d-4a0f-8ac9-45dfbc7c5fc5 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.792448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff0f62-5c8d-4a0f-8ac9-45dfbc7c5fc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:48.794387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff0f62-5c8d-4a0f-8ac9-45dfbc7c5fc5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:48.801830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb1410-811e-4f3c-8d08-fb15a5a5c940 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:48.806888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7ce8c-9d9c-4b1b-bfbf-e906e1615b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:48.807561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7ce8c-9d9c-4b1b-bfbf-e906e1615b3f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.807843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7ce8c-9d9c-4b1b-bfbf-e906e1615b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.808121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f7ce8c-9d9c-4b1b-bfbf-e906e1615b3f admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.808841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7ce8c-9d9c-4b1b-bfbf-e906e1615b3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:48.811266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f7ce8c-9d9c-4b1b-bfbf-e906e1615b3f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:48.887624 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 781/1585] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:02:48 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:48.888571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff0f62-5c8d-4a0f-8ac9-45dfbc7c5fc5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:48.899056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff0f62-5c8d-4a0f-8ac9-45dfbc7c5fc5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:48.904001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0dfc03-a54b-45b0-bc50-04232a065236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:48.904667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dfc03-a54b-45b0-bc50-04232a065236 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.904966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dfc03-a54b-45b0-bc50-04232a065236 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.905250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0dfc03-a54b-45b0-bc50-04232a065236 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.906043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0dfc03-a54b-45b0-bc50-04232a065236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:48.907664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0dfc03-a54b-45b0-bc50-04232a065236 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:48.918947 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 782/1586] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:48.919867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0dfc03-a54b-45b0-bc50-04232a065236 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:48.931143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0dfc03-a54b-45b0-bc50-04232a065236 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:48.935812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c1403-5c35-4ac8-b315-4ce7cefb1dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:48.936500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c1403-5c35-4ac8-b315-4ce7cefb1dde admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.936788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c1403-5c35-4ac8-b315-4ce7cefb1dde admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.937071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c1403-5c35-4ac8-b315-4ce7cefb1dde admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.938099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c1403-5c35-4ac8-b315-4ce7cefb1dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:48.939536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720c1403-5c35-4ac8-b315-4ce7cefb1dde admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:48.953537 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 783/1587] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:48.953537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c1403-5c35-4ac8-b315-4ce7cefb1dde admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:48.958930 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 805/1588] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:02:48 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:48.960252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7ce8c-9d9c-4b1b-bfbf-e906e1615b3f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:48.964682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c1403-5c35-4ac8-b315-4ce7cefb1dde admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:48.970042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb931b-07f4-4d58-a440-e1736897ad8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:48.972586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb931b-07f4-4d58-a440-e1736897ad8b admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.972586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb931b-07f4-4d58-a440-e1736897ad8b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.972586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeb931b-07f4-4d58-a440-e1736897ad8b admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.972586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb931b-07f4-4d58-a440-e1736897ad8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:48.972884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f7ce8c-9d9c-4b1b-bfbf-e906e1615b3f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:48.973962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeb931b-07f4-4d58-a440-e1736897ad8b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:48.977610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f86529-321d-44ea-b013-28749f33e660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:48.978953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f86529-321d-44ea-b013-28749f33e660 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:48.979327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f86529-321d-44ea-b013-28749f33e660 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:48.979675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f86529-321d-44ea-b013-28749f33e660 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:48.980734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f86529-321d-44ea-b013-28749f33e660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:48.983781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f86529-321d-44ea-b013-28749f33e660 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:48.996120 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 806/1589] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:48.997118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f86529-321d-44ea-b013-28749f33e660 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:49.007648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f86529-321d-44ea-b013-28749f33e660 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:49.012461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bc324-3685-4ed9-b4ca-7dd04de65b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:49.013121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bc324-3685-4ed9-b4ca-7dd04de65b58 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.013411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bc324-3685-4ed9-b4ca-7dd04de65b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.013696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1bc324-3685-4ed9-b4ca-7dd04de65b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.014489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bc324-3685-4ed9-b4ca-7dd04de65b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:49.016183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1bc324-3685-4ed9-b4ca-7dd04de65b58 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:49.028897 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 807/1590] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:49.028897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bc324-3685-4ed9-b4ca-7dd04de65b58 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:49.031925 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 784/1591] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:02:48 2022] POST /identity/v3/users => generated 306 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:49.031925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb931b-07f4-4d58-a440-e1736897ad8b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:49.042880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeb931b-07f4-4d58-a440-e1736897ad8b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:49.043029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1bc324-3685-4ed9-b4ca-7dd04de65b58 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:49.048050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0f521-0223-4302-b04e-4e5fad7960ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:49.048431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6aba34-9ba5-4060-8828-57deda2eb120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:49.048902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0f521-0223-4302-b04e-4e5fad7960ad admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.049233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6aba34-9ba5-4060-8828-57deda2eb120 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.049319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0f521-0223-4302-b04e-4e5fad7960ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.049650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6aba34-9ba5-4060-8828-57deda2eb120 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.049948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0f521-0223-4302-b04e-4e5fad7960ad admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.050155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6aba34-9ba5-4060-8828-57deda2eb120 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.058731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0f521-0223-4302-b04e-4e5fad7960ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:49.058850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6aba34-9ba5-4060-8828-57deda2eb120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:49.058945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b0f521-0223-4302-b04e-4e5fad7960ad admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:49.059060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6aba34-9ba5-4060-8828-57deda2eb120 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:49.069810 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 785/1592] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:49.069810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6aba34-9ba5-4060-8828-57deda2eb120 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:49.074295 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 808/1593] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:49.080626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0f521-0223-4302-b04e-4e5fad7960ad admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:49.082168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6aba34-9ba5-4060-8828-57deda2eb120 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:49.087546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0405c-414a-4caf-a19b-47c57ea52e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:49.088431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0405c-414a-4caf-a19b-47c57ea52e68 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.088788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0405c-414a-4caf-a19b-47c57ea52e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.089139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a0405c-414a-4caf-a19b-47c57ea52e68 admin admin] PATH_INFO: `/v3/projects/abb5b358addb4e769f10ad12da48b59b/users/8d1a844b3b0e4936af8a7ed0179c4786/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.090620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0f521-0223-4302-b04e-4e5fad7960ad admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:49.100257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0840d4a-e00a-4ef6-b5ee-6c6f95327ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:49.101347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0840d4a-e00a-4ef6-b5ee-6c6f95327ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.111012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0840d4a-e00a-4ef6-b5ee-6c6f95327ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.111012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0840d4a-e00a-4ef6-b5ee-6c6f95327ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.111012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0840d4a-e00a-4ef6-b5ee-6c6f95327ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:49.111012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0840d4a-e00a-4ef6-b5ee-6c6f95327ea2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:49.111563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0405c-414a-4caf-a19b-47c57ea52e68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abb5b358addb4e769f10ad12da48b59b, user_id=8d1a844b3b0e4936af8a7ed0179c4786, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:49.111563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a0405c-414a-4caf-a19b-47c57ea52e68 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:49.122535 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 809/1594] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:49.123637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0840d4a-e00a-4ef6-b5ee-6c6f95327ea2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:49.135016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0840d4a-e00a-4ef6-b5ee-6c6f95327ea2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:49.142988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597788d-35a7-4120-b41d-ed2e39c0da67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:49.143649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597788d-35a7-4120-b41d-ed2e39c0da67 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.143960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597788d-35a7-4120-b41d-ed2e39c0da67 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.144237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597788d-35a7-4120-b41d-ed2e39c0da67 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.144994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597788d-35a7-4120-b41d-ed2e39c0da67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:49.146667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597788d-35a7-4120-b41d-ed2e39c0da67 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:49.156833 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 810/1595] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:49.157597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597788d-35a7-4120-b41d-ed2e39c0da67 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:49.168668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597788d-35a7-4120-b41d-ed2e39c0da67 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:49.173290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447db91f-979e-470b-bfe4-c30f00a96043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:49.174167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447db91f-979e-470b-bfe4-c30f00a96043 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.174453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447db91f-979e-470b-bfe4-c30f00a96043 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.174729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447db91f-979e-470b-bfe4-c30f00a96043 admin admin] PATH_INFO: `/v3/projects/d257d738741646539f55b9dab71b33bf/users/f0598f70d72e4fc092db4dfa495eef0d/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.189414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447db91f-979e-470b-bfe4-c30f00a96043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d257d738741646539f55b9dab71b33bf, user_id=f0598f70d72e4fc092db4dfa495eef0d, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:49.192105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447db91f-979e-470b-bfe4-c30f00a96043 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:49.201182 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 786/1596] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:49 2022] PUT /identity/v3/projects/abb5b358addb4e769f10ad12da48b59b/users/8d1a844b3b0e4936af8a7ed0179c4786/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:49.202037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0405c-414a-4caf-a19b-47c57ea52e68 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:49.223255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:49.266234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a0405c-414a-4caf-a19b-47c57ea52e68 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:49.270823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a466ba3-20aa-4e3c-b162-80d20b02815e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:49.271480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a466ba3-20aa-4e3c-b162-80d20b02815e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.271760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a466ba3-20aa-4e3c-b162-80d20b02815e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.272036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a466ba3-20aa-4e3c-b162-80d20b02815e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.272815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a466ba3-20aa-4e3c-b162-80d20b02815e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:49.274828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a466ba3-20aa-4e3c-b162-80d20b02815e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:49.280788 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 811/1597] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:49 2022] PUT /identity/v3/projects/d257d738741646539f55b9dab71b33bf/users/f0598f70d72e4fc092db4dfa495eef0d/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:49.288970 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 787/1598] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:49.288970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a466ba3-20aa-4e3c-b162-80d20b02815e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:49.292038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447db91f-979e-470b-bfe4-c30f00a96043 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:49.296477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a466ba3-20aa-4e3c-b162-80d20b02815e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:49.354937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d548476-5dc9-44f1-beb9-13c37b153784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:49.355606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d548476-5dc9-44f1-beb9-13c37b153784 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.355896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d548476-5dc9-44f1-beb9-13c37b153784 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.356293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d548476-5dc9-44f1-beb9-13c37b153784 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.356473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447db91f-979e-470b-bfe4-c30f00a96043 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:49.357064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d548476-5dc9-44f1-beb9-13c37b153784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:49.358746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d548476-5dc9-44f1-beb9-13c37b153784 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:49.361011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8505ec-121c-4879-8a9f-a9c560b2b9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:49.361919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8505ec-121c-4879-8a9f-a9c560b2b9f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.362205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8505ec-121c-4879-8a9f-a9c560b2b9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.362485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8505ec-121c-4879-8a9f-a9c560b2b9f4 admin admin] PATH_INFO: `/v3/projects/9ec9619348174863907663b9b935773b/users/f35355f9f95643ecb2ffc62e648f431a/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.372791 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 788/1599] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:49.372791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d548476-5dc9-44f1-beb9-13c37b153784 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:49.380123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d548476-5dc9-44f1-beb9-13c37b153784 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:49.383867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8505ec-121c-4879-8a9f-a9c560b2b9f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ec9619348174863907663b9b935773b, user_id=f35355f9f95643ecb2ffc62e648f431a, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:49.384178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8505ec-121c-4879-8a9f-a9c560b2b9f4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:49.384818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d7c7f-4f90-4e6d-b4d9-aa199f616479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:49.385471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d7c7f-4f90-4e6d-b4d9-aa199f616479 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.385761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d7c7f-4f90-4e6d-b4d9-aa199f616479 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.386074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d7c7f-4f90-4e6d-b4d9-aa199f616479 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.386925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d7c7f-4f90-4e6d-b4d9-aa199f616479 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:49.388513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d7c7f-4f90-4e6d-b4d9-aa199f616479 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:49.398555 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 789/1600] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:49.399500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d7c7f-4f90-4e6d-b4d9-aa199f616479 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:49.409898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d7c7f-4f90-4e6d-b4d9-aa199f616479 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:49.415008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee7556-6ad5-4fe7-b9fa-2164f42734fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:49.415856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee7556-6ad5-4fe7-b9fa-2164f42734fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.416153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee7556-6ad5-4fe7-b9fa-2164f42734fa admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.416431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ee7556-6ad5-4fe7-b9fa-2164f42734fa admin admin] PATH_INFO: `/v3/users/c501469061044294a4edd6689b5f0aed` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.417610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee7556-6ad5-4fe7-b9fa-2164f42734fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c501469061044294a4edd6689b5f0aed)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:49.418661 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-a1ee7556-6ad5-4fe7-b9fa-2164f42734fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:02:49.418661 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:02:49.418661 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:02:49.419867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ee7556-6ad5-4fe7-b9fa-2164f42734fa admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:49.469063 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 812/1601] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:49 2022] PUT /identity/v3/projects/9ec9619348174863907663b9b935773b/users/f35355f9f95643ecb2ffc62e648f431a/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:49.470051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8505ec-121c-4879-8a9f-a9c560b2b9f4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:49.489198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:49.530610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8505ec-121c-4879-8a9f-a9c560b2b9f4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:49.535249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bd3045-6c0a-48b0-a32d-35332f754f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:49.536128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd3045-6c0a-48b0-a32d-35332f754f57 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.536448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd3045-6c0a-48b0-a32d-35332f754f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.536684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bd3045-6c0a-48b0-a32d-35332f754f57 admin admin] PATH_INFO: `/v3/projects/2588d52f501b4da8bd7200289912191d/users/cf5f88829a634821a49cffe623125155/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.551661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd3045-6c0a-48b0-a32d-35332f754f57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2588d52f501b4da8bd7200289912191d, user_id=cf5f88829a634821a49cffe623125155, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:49.555094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bd3045-6c0a-48b0-a32d-35332f754f57 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:49.598664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a1ee7556-6ad5-4fe7-b9fa-2164f42734fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c501469061044294a4edd6689b5f0aed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:49.599319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a1ee7556-6ad5-4fe7-b9fa-2164f42734fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c501469061044294a4edd6689b5f0aed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:49.609850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a1ee7556-6ad5-4fe7-b9fa-2164f42734fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c501469061044294a4edd6689b5f0aed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:49.657372 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 813/1602] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:49 2022] PUT /identity/v3/projects/2588d52f501b4da8bd7200289912191d/users/cf5f88829a634821a49cffe623125155/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:49.658741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bae7f-ca11-4ddb-a818-dd69dd244058 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.659025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bae7f-ca11-4ddb-a818-dd69dd244058 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.659299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bae7f-ca11-4ddb-a818-dd69dd244058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.674384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a1ee7556-6ad5-4fe7-b9fa-2164f42734fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c501469061044294a4edd6689b5f0aed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:49.675013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a1ee7556-6ad5-4fe7-b9fa-2164f42734fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c501469061044294a4edd6689b5f0aed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:49.685018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-180bae7f-ca11-4ddb-a818-dd69dd244058 None None] MFA Rules not processed for user `8d1a844b3b0e4936af8a7ed0179c4786`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:49.686260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-180bae7f-ca11-4ddb-a818-dd69dd244058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:49.701913 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 790/1603] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:02:49 2022] DELETE /identity/v3/users/c501469061044294a4edd6689b5f0aed => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:49.702781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee7556-6ad5-4fe7-b9fa-2164f42734fa admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:49.704146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a1ee7556-6ad5-4fe7-b9fa-2164f42734fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:49.717711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:49.751574 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 814/1604] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:49.753001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274f334-d82b-4e13-b342-f717be1ccee2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.753367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274f334-d82b-4e13-b342-f717be1ccee2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.753722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274f334-d82b-4e13-b342-f717be1ccee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.769123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ee7556-6ad5-4fe7-b9fa-2164f42734fa admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:49.775290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9867e0-5682-45ab-a963-35b592c44beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:49.776180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9867e0-5682-45ab-a963-35b592c44beb admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.776463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9867e0-5682-45ab-a963-35b592c44beb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.776761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9867e0-5682-45ab-a963-35b592c44beb admin admin] PATH_INFO: `/v3/projects/0eeea9efb9364971b8335deff54816d3/users/2fb485e80a0c42e6bae2142de7c23338/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.780723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3274f334-d82b-4e13-b342-f717be1ccee2 None None] MFA Rules not processed for user `f0598f70d72e4fc092db4dfa495eef0d`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:49.782173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3274f334-d82b-4e13-b342-f717be1ccee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:49.793340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9867e0-5682-45ab-a963-35b592c44beb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eeea9efb9364971b8335deff54816d3, user_id=2fb485e80a0c42e6bae2142de7c23338, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:49.796281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9867e0-5682-45ab-a963-35b592c44beb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:49.846675 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 815/1605] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:49.847553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274f334-d82b-4e13-b342-f717be1ccee2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:49.848817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3274f334-d82b-4e13-b342-f717be1ccee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:49.862700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274f334-d82b-4e13-b342-f717be1ccee2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:49.866243 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 791/1606] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:49 2022] PUT /identity/v3/projects/0eeea9efb9364971b8335deff54816d3/users/2fb485e80a0c42e6bae2142de7c23338/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:49.866243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9867e0-5682-45ab-a963-35b592c44beb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:49.866243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0f9867e0-5682-45ab-a963-35b592c44beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:49.880444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:49.920067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6e77e-96b6-41a2-ba13-eb6579a92903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:49.920961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6e77e-96b6-41a2-ba13-eb6579a92903 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.921243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6e77e-96b6-41a2-ba13-eb6579a92903 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.921516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6e77e-96b6-41a2-ba13-eb6579a92903 admin admin] PATH_INFO: `/v3/projects/8c0200afe2ad4f6db3a7a21cf37f65c8/users/89ed91b9d56241d99b6df99ede3fb446/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.931970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9867e0-5682-45ab-a963-35b592c44beb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:49.936600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa987cf-d333-42d8-86dd-ecd68d3e6026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:49.936960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6e77e-96b6-41a2-ba13-eb6579a92903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c0200afe2ad4f6db3a7a21cf37f65c8, user_id=89ed91b9d56241d99b6df99ede3fb446, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:49.937480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa987cf-d333-42d8-86dd-ecd68d3e6026 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:49.937859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa987cf-d333-42d8-86dd-ecd68d3e6026 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:49.948341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa987cf-d333-42d8-86dd-ecd68d3e6026 admin admin] PATH_INFO: `/v3/projects/281d71da66424c949d37bda848d84fc2/users/39143b8a6ad4488ca98a8e5749c8eeb9/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:49.948471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6e77e-96b6-41a2-ba13-eb6579a92903 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:49.952333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa987cf-d333-42d8-86dd-ecd68d3e6026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=281d71da66424c949d37bda848d84fc2, user_id=39143b8a6ad4488ca98a8e5749c8eeb9, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:49.955065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa987cf-d333-42d8-86dd-ecd68d3e6026 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:50.057711 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 816/1607] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:49 2022] PUT /identity/v3/projects/8c0200afe2ad4f6db3a7a21cf37f65c8/users/89ed91b9d56241d99b6df99ede3fb446/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:50.059386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c9cc0-ff16-4a43-bf92-0921d7ed487e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:50.059853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c9cc0-ff16-4a43-bf92-0921d7ed487e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:50.060402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50c9cc0-ff16-4a43-bf92-0921d7ed487e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:50.072020 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 792/1608] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:49 2022] PUT /identity/v3/projects/281d71da66424c949d37bda848d84fc2/users/39143b8a6ad4488ca98a8e5749c8eeb9/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:50.073330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10431403-3df8-4862-9afe-e0210ca20643 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:50.073623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10431403-3df8-4862-9afe-e0210ca20643 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:50.073943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10431403-3df8-4862-9afe-e0210ca20643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:50.094526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e50c9cc0-ff16-4a43-bf92-0921d7ed487e None None] MFA Rules not processed for user `f35355f9f95643ecb2ffc62e648f431a`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:50.095824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e50c9cc0-ff16-4a43-bf92-0921d7ed487e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:50.101397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-10431403-3df8-4862-9afe-e0210ca20643 None None] MFA Rules not processed for user `cf5f88829a634821a49cffe623125155`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:50.102624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-10431403-3df8-4862-9afe-e0210ca20643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:50.159432 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 817/1609] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:50.160727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d4663-5e89-4a32-8270-4921acd52925 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:50.161020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d4663-5e89-4a32-8270-4921acd52925 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:50.161309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d4663-5e89-4a32-8270-4921acd52925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:50.173138 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 793/1610] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:50.173977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10431403-3df8-4862-9afe-e0210ca20643 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:50.175247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-10431403-3df8-4862-9afe-e0210ca20643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:50.178173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a68d4663-5e89-4a32-8270-4921acd52925 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:50.179350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a68d4663-5e89-4a32-8270-4921acd52925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:50.184193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:50.239014 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 818/1611] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:50.240050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10431403-3df8-4862-9afe-e0210ca20643 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:50.240196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d4663-5e89-4a32-8270-4921acd52925 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:50.241566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a68d4663-5e89-4a32-8270-4921acd52925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:50.245063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213147d-86cc-4f70-a85a-80edf976a7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:50.245730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213147d-86cc-4f70-a85a-80edf976a7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:50.246304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213147d-86cc-4f70-a85a-80edf976a7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:50.246584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213147d-86cc-4f70-a85a-80edf976a7f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:50.247429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8213147d-86cc-4f70-a85a-80edf976a7f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:50.249580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8213147d-86cc-4f70-a85a-80edf976a7f8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:50.254580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d4663-5e89-4a32-8270-4921acd52925 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:50.259724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af710100-c241-4784-9f91-4b5cfbeb5944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:50.260380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af710100-c241-4784-9f91-4b5cfbeb5944 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:50.260762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af710100-c241-4784-9f91-4b5cfbeb5944 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:50.261039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af710100-c241-4784-9f91-4b5cfbeb5944 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:50.261864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af710100-c241-4784-9f91-4b5cfbeb5944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:50.263767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af710100-c241-4784-9f91-4b5cfbeb5944 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:50.360542 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 794/1612] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:02:50 2022] POST /identity/v3/projects => generated 379 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:50.360542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213147d-86cc-4f70-a85a-80edf976a7f8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:50.368139 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 819/1613] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:02:50 2022] POST /identity/v3/projects => generated 377 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:50.369566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca38a8-2efa-4c97-8c87-ea36d33fa880 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:50.369876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca38a8-2efa-4c97-8c87-ea36d33fa880 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:50.370169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca38a8-2efa-4c97-8c87-ea36d33fa880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:50.389980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:50.398506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-00ca38a8-2efa-4c97-8c87-ea36d33fa880 None None] MFA Rules not processed for user `89ed91b9d56241d99b6df99ede3fb446`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:50.399826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-00ca38a8-2efa-4c97-8c87-ea36d33fa880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:50.424813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213147d-86cc-4f70-a85a-80edf976a7f8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:50.464043 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 820/1614] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:50.465437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e8f8d-5bc0-47fe-8bfe-3a7d421dbcc7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:50.465729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e8f8d-5bc0-47fe-8bfe-3a7d421dbcc7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:50.466040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e8f8d-5bc0-47fe-8bfe-3a7d421dbcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:50.487340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d8f0df-6b6f-4a14-8240-665c9362e66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:50.488199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8f0df-6b6f-4a14-8240-665c9362e66b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:50.488486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8f0df-6b6f-4a14-8240-665c9362e66b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:50.488767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d8f0df-6b6f-4a14-8240-665c9362e66b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:50.489510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d8f0df-6b6f-4a14-8240-665c9362e66b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:50.491829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d8f0df-6b6f-4a14-8240-665c9362e66b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:50.492802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-484e8f8d-5bc0-47fe-8bfe-3a7d421dbcc7 None None] MFA Rules not processed for user `39143b8a6ad4488ca98a8e5749c8eeb9`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:50.494042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-484e8f8d-5bc0-47fe-8bfe-3a7d421dbcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:50.502258 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 795/1615] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:50.506337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d8f0df-6b6f-4a14-8240-665c9362e66b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:50.508523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-09d8f0df-6b6f-4a14-8240-665c9362e66b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:50.520357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d8f0df-6b6f-4a14-8240-665c9362e66b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:50.524874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2aee0-351d-43cb-bdc4-091156c669f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:50.525503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2aee0-351d-43cb-bdc4-091156c669f3 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:50.525793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2aee0-351d-43cb-bdc4-091156c669f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:50.526107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2aee0-351d-43cb-bdc4-091156c669f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:50.526782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2aee0-351d-43cb-bdc4-091156c669f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:50.528671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2aee0-351d-43cb-bdc4-091156c669f3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:50.560657 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 821/1616] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:50.561626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e8f8d-5bc0-47fe-8bfe-3a7d421dbcc7 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:50.562943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-484e8f8d-5bc0-47fe-8bfe-3a7d421dbcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:50.576554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e8f8d-5bc0-47fe-8bfe-3a7d421dbcc7 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:50.584901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b6b7d-c3b8-465f-b052-a04475c18f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:50.584901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b6b7d-c3b8-465f-b052-a04475c18f0d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:50.584901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b6b7d-c3b8-465f-b052-a04475c18f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:50.584901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95b6b7d-c3b8-465f-b052-a04475c18f0d admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:50.585246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b6b7d-c3b8-465f-b052-a04475c18f0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:50.585246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95b6b7d-c3b8-465f-b052-a04475c18f0d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:50.607501 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 796/1617] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:02:50 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:50.608488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2aee0-351d-43cb-bdc4-091156c669f3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:50.613435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5cf2aee0-351d-43cb-bdc4-091156c669f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:50.627502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:50.669398 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 822/1618] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:02:50 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:50.669398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b6b7d-c3b8-465f-b052-a04475c18f0d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:50.684211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2aee0-351d-43cb-bdc4-091156c669f3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:50.730117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95b6b7d-c3b8-465f-b052-a04475c18f0d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:50.732604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e1931-7d27-4bea-88b4-6026d4643db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:50.733257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e1931-7d27-4bea-88b4-6026d4643db7 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:50.733534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e1931-7d27-4bea-88b4-6026d4643db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:50.733810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e1931-7d27-4bea-88b4-6026d4643db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:50.735013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9cbc6-73dc-4378-925a-a0253fd53f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:50.735782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9cbc6-73dc-4378-925a-a0253fd53f22 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:50.736071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9cbc6-73dc-4378-925a-a0253fd53f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:50.736348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9cbc6-73dc-4378-925a-a0253fd53f22 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:50.737075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9cbc6-73dc-4378-925a-a0253fd53f22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:50.739094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c9cbc6-73dc-4378-925a-a0253fd53f22 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:50.743415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e1931-7d27-4bea-88b4-6026d4643db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:50.744841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e1931-7d27-4bea-88b4-6026d4643db7 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:50.754739 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 797/1619] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:50.755478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e1931-7d27-4bea-88b4-6026d4643db7 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:50.765290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e1931-7d27-4bea-88b4-6026d4643db7 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:50.770201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c69ae-12d6-4e2e-b36d-e7378b6ad944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:50.770885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c69ae-12d6-4e2e-b36d-e7378b6ad944 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:50.771173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c69ae-12d6-4e2e-b36d-e7378b6ad944 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:50.771451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9c69ae-12d6-4e2e-b36d-e7378b6ad944 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:50.772246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c69ae-12d6-4e2e-b36d-e7378b6ad944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:50.774271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9c69ae-12d6-4e2e-b36d-e7378b6ad944 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:50.806476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:50.813077 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 823/1620] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:02:50 2022] POST /identity/v3/users => generated 296 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:50.813913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9cbc6-73dc-4378-925a-a0253fd53f22 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:50.823810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c9cbc6-73dc-4378-925a-a0253fd53f22 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:50.828972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba911a74-b38b-4ce3-93f6-49ef065de044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:50.829611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba911a74-b38b-4ce3-93f6-49ef065de044 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:50.829936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba911a74-b38b-4ce3-93f6-49ef065de044 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:50.830218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba911a74-b38b-4ce3-93f6-49ef065de044 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:50.830910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba911a74-b38b-4ce3-93f6-49ef065de044 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:50.832841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba911a74-b38b-4ce3-93f6-49ef065de044 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:50.837366 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 798/1621] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:02:50 2022] POST /identity/v3/users => generated 295 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:50.838206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c69ae-12d6-4e2e-b36d-e7378b6ad944 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:50.849040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9c69ae-12d6-4e2e-b36d-e7378b6ad944 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:50.854377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301bf817-efc3-4183-8aaf-7d45b93e427c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:50.855254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bf817-efc3-4183-8aaf-7d45b93e427c admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:50.855563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bf817-efc3-4183-8aaf-7d45b93e427c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:50.855813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301bf817-efc3-4183-8aaf-7d45b93e427c admin admin] PATH_INFO: `/v3/projects/0eeea9efb9364971b8335deff54816d3/users/2fb485e80a0c42e6bae2142de7c23338/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:50.857779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301bf817-efc3-4183-8aaf-7d45b93e427c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eeea9efb9364971b8335deff54816d3, user_id=2fb485e80a0c42e6bae2142de7c23338, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:50.860461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301bf817-efc3-4183-8aaf-7d45b93e427c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:50.873878 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 799/1622] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:50 2022] PUT /identity/v3/projects/0eeea9efb9364971b8335deff54816d3/users/2fb485e80a0c42e6bae2142de7c23338/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:50.873878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301bf817-efc3-4183-8aaf-7d45b93e427c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:50.908180 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 824/1623] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:02:50 2022] POST /identity/v3/projects => generated 383 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:50.908180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba911a74-b38b-4ce3-93f6-49ef065de044 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:50.938054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301bf817-efc3-4183-8aaf-7d45b93e427c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:50.976900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:51.003439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b70e0-9795-4b96-9779-f30c7d9edbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:51.004763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b70e0-9795-4b96-9779-f30c7d9edbc9 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.005158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b70e0-9795-4b96-9779-f30c7d9edbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.005487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b70e0-9795-4b96-9779-f30c7d9edbc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.006591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b70e0-9795-4b96-9779-f30c7d9edbc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:51.008799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b70e0-9795-4b96-9779-f30c7d9edbc9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:51.015834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba911a74-b38b-4ce3-93f6-49ef065de044 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:51.021027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd73594-02f9-4914-a5ef-45deabdad8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:51.021344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd73594-02f9-4914-a5ef-45deabdad8aa admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.021633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd73594-02f9-4914-a5ef-45deabdad8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.021941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd73594-02f9-4914-a5ef-45deabdad8aa admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.022815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd73594-02f9-4914-a5ef-45deabdad8aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:51.024792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd73594-02f9-4914-a5ef-45deabdad8aa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:51.105373 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 800/1624] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:02:50 2022] POST /identity/v3/projects => generated 367 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:51.106227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b70e0-9795-4b96-9779-f30c7d9edbc9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:51.155699 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 825/1625] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:02:50 2022] POST /identity/v3/users => generated 297 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:51.156651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd73594-02f9-4914-a5ef-45deabdad8aa admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:51.167548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd73594-02f9-4914-a5ef-45deabdad8aa admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:51.170554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b70e0-9795-4b96-9779-f30c7d9edbc9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:51.172339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f1f12-997a-48dc-8519-4a684b31b2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:51.172995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f1f12-997a-48dc-8519-4a684b31b2c2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.173282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f1f12-997a-48dc-8519-4a684b31b2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.173628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f1f12-997a-48dc-8519-4a684b31b2c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.174402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f1f12-997a-48dc-8519-4a684b31b2c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:51.175268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5faf5-5f31-4c5b-8c4f-943c19b2513d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:51.176078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270f1f12-997a-48dc-8519-4a684b31b2c2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:51.178091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5faf5-5f31-4c5b-8c4f-943c19b2513d admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.178584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5faf5-5f31-4c5b-8c4f-943c19b2513d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.178953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5faf5-5f31-4c5b-8c4f-943c19b2513d admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.179917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5faf5-5f31-4c5b-8c4f-943c19b2513d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:51.183648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5faf5-5f31-4c5b-8c4f-943c19b2513d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:51.194575 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 826/1626] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:51.194575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f1f12-997a-48dc-8519-4a684b31b2c2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:51.200498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f1f12-997a-48dc-8519-4a684b31b2c2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:51.207804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be20d9f-e712-46b5-93c8-5132323a777f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:51.208702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be20d9f-e712-46b5-93c8-5132323a777f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.209107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be20d9f-e712-46b5-93c8-5132323a777f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.209550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be20d9f-e712-46b5-93c8-5132323a777f admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.210975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be20d9f-e712-46b5-93c8-5132323a777f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:51.212696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be20d9f-e712-46b5-93c8-5132323a777f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:51.224346 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 827/1627] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:51.226330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e61f6-0870-4f1b-a15b-70a4a58a910d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.226620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e61f6-0870-4f1b-a15b-70a4a58a910d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.226899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e61f6-0870-4f1b-a15b-70a4a58a910d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.253264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-228e61f6-0870-4f1b-a15b-70a4a58a910d None None] MFA Rules not processed for user `2fb485e80a0c42e6bae2142de7c23338`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:51.254572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-228e61f6-0870-4f1b-a15b-70a4a58a910d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:51.280607 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 801/1628] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:02:51 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:51.281508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5faf5-5f31-4c5b-8c4f-943c19b2513d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:51.292292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5faf5-5f31-4c5b-8c4f-943c19b2513d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:51.297142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2703d-3cc1-4388-95d2-9e22bebc555e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:51.297821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2703d-3cc1-4388-95d2-9e22bebc555e admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.301340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2703d-3cc1-4388-95d2-9e22bebc555e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.301340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2703d-3cc1-4388-95d2-9e22bebc555e admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.301340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e2703d-3cc1-4388-95d2-9e22bebc555e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:51.301340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e2703d-3cc1-4388-95d2-9e22bebc555e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:51.323868 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 828/1629] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:51 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:51.324825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e61f6-0870-4f1b-a15b-70a4a58a910d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:51.334399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e61f6-0870-4f1b-a15b-70a4a58a910d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:51.339649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780e93a-878d-4f64-b8f9-b1829e877eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:51.340294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780e93a-878d-4f64-b8f9-b1829e877eea admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.340991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780e93a-878d-4f64-b8f9-b1829e877eea admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.340991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780e93a-878d-4f64-b8f9-b1829e877eea admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.341639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780e93a-878d-4f64-b8f9-b1829e877eea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:51.343802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780e93a-878d-4f64-b8f9-b1829e877eea admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:51.383761 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 802/1630] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:02:51 2022] POST /identity/v3/users => generated 298 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:51.384716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2703d-3cc1-4388-95d2-9e22bebc555e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:51.394645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2703d-3cc1-4388-95d2-9e22bebc555e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:51.399985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f1d6b-2edb-4d20-884e-2cc67d4dd5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:51.400631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f1d6b-2edb-4d20-884e-2cc67d4dd5eb admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.400913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f1d6b-2edb-4d20-884e-2cc67d4dd5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.401190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f1d6b-2edb-4d20-884e-2cc67d4dd5eb admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.401959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f1d6b-2edb-4d20-884e-2cc67d4dd5eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:51.403611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f1d6b-2edb-4d20-884e-2cc67d4dd5eb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:51.414826 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 803/1631] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:51.415578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f1d6b-2edb-4d20-884e-2cc67d4dd5eb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:51.417281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bd0f1d6b-2edb-4d20-884e-2cc67d4dd5eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:51.430251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f1d6b-2edb-4d20-884e-2cc67d4dd5eb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:51.434899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848496c5-f603-4d7d-99e7-cd6442bd1f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:51.435740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848496c5-f603-4d7d-99e7-cd6442bd1f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.436023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848496c5-f603-4d7d-99e7-cd6442bd1f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.436298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848496c5-f603-4d7d-99e7-cd6442bd1f4f admin admin] PATH_INFO: `/v3/projects/083c66f515ee434ea52034fa5348b303` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.437465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848496c5-f603-4d7d-99e7-cd6442bd1f4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=083c66f515ee434ea52034fa5348b303)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:51.439462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848496c5-f603-4d7d-99e7-cd6442bd1f4f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:51.458119 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 829/1632] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:02:51 2022] POST /identity/v3/users => generated 290 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:51.462054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780e93a-878d-4f64-b8f9-b1829e877eea admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:51.469811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780e93a-878d-4f64-b8f9-b1829e877eea admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:51.474512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592a90a-65fc-4fc1-ac55-f6b13afce4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:51.475433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592a90a-65fc-4fc1-ac55-f6b13afce4c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.476224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592a90a-65fc-4fc1-ac55-f6b13afce4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.476500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1592a90a-65fc-4fc1-ac55-f6b13afce4c8 admin admin] PATH_INFO: `/v3/projects/8d62542caece43d0a138849e9f70e55c/users/d2cee368d9a14c8bb3e665eed1fdeb7e/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.485340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:51.491961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592a90a-65fc-4fc1-ac55-f6b13afce4c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d62542caece43d0a138849e9f70e55c, user_id=d2cee368d9a14c8bb3e665eed1fdeb7e, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:51.494784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1592a90a-65fc-4fc1-ac55-f6b13afce4c8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:51.566148 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 830/1633] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:51 2022] PUT /identity/v3/projects/8d62542caece43d0a138849e9f70e55c/users/d2cee368d9a14c8bb3e665eed1fdeb7e/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:51.570536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592a90a-65fc-4fc1-ac55-f6b13afce4c8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:51.574882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-848496c5-f603-4d7d-99e7-cd6442bd1f4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '083c66f515ee434ea52034fa5348b303', 'request_id': 'req-848496c5-f603-4d7d-99e7-cd6442bd1f4f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:51.580634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-848496c5-f603-4d7d-99e7-cd6442bd1f4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '083c66f515ee434ea52034fa5348b303', 'request_id': 'req-848496c5-f603-4d7d-99e7-cd6442bd1f4f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:51.622954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-848496c5-f603-4d7d-99e7-cd6442bd1f4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '083c66f515ee434ea52034fa5348b303', 'request_id': 'req-848496c5-f603-4d7d-99e7-cd6442bd1f4f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:51.633931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1592a90a-65fc-4fc1-ac55-f6b13afce4c8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:51.634690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-848496c5-f603-4d7d-99e7-cd6442bd1f4f admin admin] The token cache is being invalidate because project 083c66f515ee434ea52034fa5348b303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:02:51.634984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-848496c5-f603-4d7d-99e7-cd6442bd1f4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:51.635639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-848496c5-f603-4d7d-99e7-cd6442bd1f4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:51.637213 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 804/1634] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:02:51 2022] DELETE /identity/v3/projects/083c66f515ee434ea52034fa5348b303 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:51.638013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848496c5-f603-4d7d-99e7-cd6442bd1f4f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:51.639358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-848496c5-f603-4d7d-99e7-cd6442bd1f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:51.652129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:INSERT=8,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:51.697742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06378b-ec7b-45d6-b9dd-20b0200a8c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:51.698614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06378b-ec7b-45d6-b9dd-20b0200a8c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.698891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06378b-ec7b-45d6-b9dd-20b0200a8c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.699162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06378b-ec7b-45d6-b9dd-20b0200a8c1c admin admin] PATH_INFO: `/v3/projects/003ac163682d47fbb4ec29e4afeb1963/users/4037ef977e9c4d8692cb5d6c56499328/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.706744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848496c5-f603-4d7d-99e7-cd6442bd1f4f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:51.711687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258102c-81d7-46d3-9dca-2b6ed5240c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:51.712357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258102c-81d7-46d3-9dca-2b6ed5240c78 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.712647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258102c-81d7-46d3-9dca-2b6ed5240c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.712992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258102c-81d7-46d3-9dca-2b6ed5240c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.713754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258102c-81d7-46d3-9dca-2b6ed5240c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:51.715509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06378b-ec7b-45d6-b9dd-20b0200a8c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=003ac163682d47fbb4ec29e4afeb1963, user_id=4037ef977e9c4d8692cb5d6c56499328, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:51.716438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258102c-81d7-46d3-9dca-2b6ed5240c78 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:51.718543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06378b-ec7b-45d6-b9dd-20b0200a8c1c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:51.727365 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 805/1635] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:51.728719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834e7ec-ff6c-422c-86b9-a32816511ed9 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.729019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834e7ec-ff6c-422c-86b9-a32816511ed9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.729305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b834e7ec-ff6c-422c-86b9-a32816511ed9 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.730913 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 806/1636] 10.222.0.33 () {54 vars in 961 bytes} [Sat May 28 21:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:02:51.731637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834e7ec-ff6c-422c-86b9-a32816511ed9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:51.733403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b834e7ec-ff6c-422c-86b9-a32816511ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:51.749680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b834e7ec-ff6c-422c-86b9-a32816511ed9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:51.754374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17014a-e0c7-4d78-81f9-fbd3ce73c3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:51.755049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17014a-e0c7-4d78-81f9-fbd3ce73c3ee admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.755336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17014a-e0c7-4d78-81f9-fbd3ce73c3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.755620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce17014a-e0c7-4d78-81f9-fbd3ce73c3ee admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.756348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17014a-e0c7-4d78-81f9-fbd3ce73c3ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:51.758016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce17014a-e0c7-4d78-81f9-fbd3ce73c3ee admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:51.767873 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 807/1637] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:51.768625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17014a-e0c7-4d78-81f9-fbd3ce73c3ee admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:51.770271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ce17014a-e0c7-4d78-81f9-fbd3ce73c3ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:51.782876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce17014a-e0c7-4d78-81f9-fbd3ce73c3ee admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:51.784042 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 831/1638] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:51 2022] PUT /identity/v3/projects/003ac163682d47fbb4ec29e4afeb1963/users/4037ef977e9c4d8692cb5d6c56499328/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:51.784871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06378b-ec7b-45d6-b9dd-20b0200a8c1c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:51.786555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0b06378b-ec7b-45d6-b9dd-20b0200a8c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:51.842449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3318b3b-c36f-4c16-82df-4a166351bbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:51.843348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3318b3b-c36f-4c16-82df-4a166351bbda admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.843636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3318b3b-c36f-4c16-82df-4a166351bbda admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.843944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3318b3b-c36f-4c16-82df-4a166351bbda admin admin] PATH_INFO: `/v3/projects/70426a0425f54267b0964ce4505fbffb/users/27755a59f8064b99b322481803a0206c/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.850778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06378b-ec7b-45d6-b9dd-20b0200a8c1c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:51.855478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98a393-46be-415a-96a0-05b2dc9c7b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:51.856145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98a393-46be-415a-96a0-05b2dc9c7b5f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.856425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98a393-46be-415a-96a0-05b2dc9c7b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.856809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd98a393-46be-415a-96a0-05b2dc9c7b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.857544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98a393-46be-415a-96a0-05b2dc9c7b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:51.858806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3318b3b-c36f-4c16-82df-4a166351bbda admin admin] RBAC: Authorizing `identity:create_grant(project_id=70426a0425f54267b0964ce4505fbffb, user_id=27755a59f8064b99b322481803a0206c, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:51.859138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd98a393-46be-415a-96a0-05b2dc9c7b5f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:51.861533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3318b3b-c36f-4c16-82df-4a166351bbda admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:51.869502 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 832/1639] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:51.870332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98a393-46be-415a-96a0-05b2dc9c7b5f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:51.872187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fd98a393-46be-415a-96a0-05b2dc9c7b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:51.884527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd98a393-46be-415a-96a0-05b2dc9c7b5f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:51.889054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581eaa1e-fdf1-4051-ad27-e18c91ec7092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:51.889678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eaa1e-fdf1-4051-ad27-e18c91ec7092 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.890016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eaa1e-fdf1-4051-ad27-e18c91ec7092 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.891523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eaa1e-fdf1-4051-ad27-e18c91ec7092 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.891523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581eaa1e-fdf1-4051-ad27-e18c91ec7092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:51.892520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581eaa1e-fdf1-4051-ad27-e18c91ec7092 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:51.902608 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 833/1640] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:51.903797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ffb5d-ab39-4d23-8c85-1d8f552dc0eb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.904085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ffb5d-ab39-4d23-8c85-1d8f552dc0eb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.904381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ffb5d-ab39-4d23-8c85-1d8f552dc0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.910569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:51.923165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-884ffb5d-ab39-4d23-8c85-1d8f552dc0eb None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:51.924349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-884ffb5d-ab39-4d23-8c85-1d8f552dc0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:51.932381 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 808/1641] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:51 2022] PUT /identity/v3/projects/70426a0425f54267b0964ce4505fbffb/users/27755a59f8064b99b322481803a0206c/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:51.933406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3318b3b-c36f-4c16-82df-4a166351bbda admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:51.988924 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 834/1642] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:51.990272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1854016-a817-486d-9a74-91ed9140b037 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:51.990598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1854016-a817-486d-9a74-91ed9140b037 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:51.990886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1854016-a817-486d-9a74-91ed9140b037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:51.999965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3318b3b-c36f-4c16-82df-4a166351bbda admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:52.004820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55bdb1d-2b14-46b9-b9b0-928b2d4cf9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:52.005704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bdb1d-2b14-46b9-b9b0-928b2d4cf9b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.006165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bdb1d-2b14-46b9-b9b0-928b2d4cf9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.006458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bdb1d-2b14-46b9-b9b0-928b2d4cf9b3 admin admin] PATH_INFO: `/v3/projects/4c27b736741a4c6f9c7a8fc2bd133a95/users/b9c8dbd5526b46a2ba24b0db5ca3571e/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.021683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55bdb1d-2b14-46b9-b9b0-928b2d4cf9b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c27b736741a4c6f9c7a8fc2bd133a95, user_id=b9c8dbd5526b46a2ba24b0db5ca3571e, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:52.024955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55bdb1d-2b14-46b9-b9b0-928b2d4cf9b3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:52.049101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f1854016-a817-486d-9a74-91ed9140b037 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:52.050453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f1854016-a817-486d-9a74-91ed9140b037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:52.072620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:52.078475 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 809/1643] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:51 2022] PUT /identity/v3/projects/4c27b736741a4c6f9c7a8fc2bd133a95/users/b9c8dbd5526b46a2ba24b0db5ca3571e/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:52.079345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55bdb1d-2b14-46b9-b9b0-928b2d4cf9b3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:52.135888 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 835/1644] 10.222.0.33 () {58 vars in 1058 bytes} [Sat May 28 21:02:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:52.136808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1854016-a817-486d-9a74-91ed9140b037 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:52.138234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f1854016-a817-486d-9a74-91ed9140b037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:52.142495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55bdb1d-2b14-46b9-b9b0-928b2d4cf9b3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:52.147834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d99b06-8fe2-4bee-ba3f-6f61be852782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:52.148771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d99b06-8fe2-4bee-ba3f-6f61be852782 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.149057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d99b06-8fe2-4bee-ba3f-6f61be852782 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.149334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d99b06-8fe2-4bee-ba3f-6f61be852782 admin admin] PATH_INFO: `/v3/projects/6277287bf774466582bc214b11186b70/users/d1b067ff75914828a49edd61f6a9318b/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.151759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1854016-a817-486d-9a74-91ed9140b037 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:52.156315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cf91f-c058-44c9-b849-67ec49b082f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:52.156974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cf91f-c058-44c9-b849-67ec49b082f1 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.157328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cf91f-c058-44c9-b849-67ec49b082f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.157619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cf91f-c058-44c9-b849-67ec49b082f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.158398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8cf91f-c058-44c9-b849-67ec49b082f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:52.160013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8cf91f-c058-44c9-b849-67ec49b082f1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:52.164870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d99b06-8fe2-4bee-ba3f-6f61be852782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6277287bf774466582bc214b11186b70, user_id=d1b067ff75914828a49edd61f6a9318b, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:52.167709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d99b06-8fe2-4bee-ba3f-6f61be852782 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:52.170656 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 836/1645] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:52.171508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cf91f-c058-44c9-b849-67ec49b082f1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:52.181165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cf91f-c058-44c9-b849-67ec49b082f1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:52.185747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bded2-db95-4ded-b5ca-82a01d76e51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:52.186638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bded2-db95-4ded-b5ca-82a01d76e51d admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.186931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bded2-db95-4ded-b5ca-82a01d76e51d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.187213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bded2-db95-4ded-b5ca-82a01d76e51d admin admin] PATH_INFO: `/v3/projects/0d723d6e08fa41c89a20c1fb568c5741/users/e63ff1a4d0f349e09f23613f4d94c280/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.201544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7bded2-db95-4ded-b5ca-82a01d76e51d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d723d6e08fa41c89a20c1fb568c5741, user_id=e63ff1a4d0f349e09f23613f4d94c280, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:52.204241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7bded2-db95-4ded-b5ca-82a01d76e51d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:52.223403 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 810/1646] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:52 2022] PUT /identity/v3/projects/6277287bf774466582bc214b11186b70/users/d1b067ff75914828a49edd61f6a9318b/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:52.224209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d99b06-8fe2-4bee-ba3f-6f61be852782 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:52.280677 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 837/1647] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:52 2022] PUT /identity/v3/projects/0d723d6e08fa41c89a20c1fb568c5741/users/e63ff1a4d0f349e09f23613f4d94c280/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:52.281518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bded2-db95-4ded-b5ca-82a01d76e51d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:52.288821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d99b06-8fe2-4bee-ba3f-6f61be852782 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:52.295599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:52.344662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bded2-db95-4ded-b5ca-82a01d76e51d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:52.345715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00f633-f362-4277-8009-207d228ca054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:52.346639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00f633-f362-4277-8009-207d228ca054 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.346923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00f633-f362-4277-8009-207d228ca054 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.347200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00f633-f362-4277-8009-207d228ca054 admin admin] PATH_INFO: `/v3/projects/8d62542caece43d0a138849e9f70e55c/users/d2cee368d9a14c8bb3e665eed1fdeb7e/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.349216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00f633-f362-4277-8009-207d228ca054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d62542caece43d0a138849e9f70e55c, user_id=d2cee368d9a14c8bb3e665eed1fdeb7e, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:52.349880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45abab5-5d20-4743-89d9-8cd3729d03c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:52.350620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45abab5-5d20-4743-89d9-8cd3729d03c8 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.350969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45abab5-5d20-4743-89d9-8cd3729d03c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.351313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45abab5-5d20-4743-89d9-8cd3729d03c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.352020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00f633-f362-4277-8009-207d228ca054 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:52.352143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45abab5-5d20-4743-89d9-8cd3729d03c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:52.353700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45abab5-5d20-4743-89d9-8cd3729d03c8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:52.362580 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 811/1648] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:52 2022] PUT /identity/v3/projects/8d62542caece43d0a138849e9f70e55c/users/d2cee368d9a14c8bb3e665eed1fdeb7e/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:52.366590 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 838/1649] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:52.366734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475671a-dcd4-4ba1-8715-dced0c523ab9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.366734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475671a-dcd4-4ba1-8715-dced0c523ab9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.366882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45abab5-5d20-4743-89d9-8cd3729d03c8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:52.366959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475671a-dcd4-4ba1-8715-dced0c523ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.377694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45abab5-5d20-4743-89d9-8cd3729d03c8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:52.386068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e475671a-dcd4-4ba1-8715-dced0c523ab9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:52.387318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e475671a-dcd4-4ba1-8715-dced0c523ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:52.404774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:52.435988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d5511-471a-47e5-9f1e-9099a09c3a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:52.436682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d5511-471a-47e5-9f1e-9099a09c3a5e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.436971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d5511-471a-47e5-9f1e-9099a09c3a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.437277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d5511-471a-47e5-9f1e-9099a09c3a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.438047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d5511-471a-47e5-9f1e-9099a09c3a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:52.439650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d5511-471a-47e5-9f1e-9099a09c3a5e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:52.448658 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 812/1650] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:52.452564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475671a-dcd4-4ba1-8715-dced0c523ab9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:52.452687 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 839/1651] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:52.452687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d5511-471a-47e5-9f1e-9099a09c3a5e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:52.462112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d5511-471a-47e5-9f1e-9099a09c3a5e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:52.466716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ae48d-c940-4ee7-bea6-9c4d50b6e17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:52.467585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ae48d-c940-4ee7-bea6-9c4d50b6e17f admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.467866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ae48d-c940-4ee7-bea6-9c4d50b6e17f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.468142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ae48d-c940-4ee7-bea6-9c4d50b6e17f admin admin] PATH_INFO: `/v3/projects/003ac163682d47fbb4ec29e4afeb1963/users/4037ef977e9c4d8692cb5d6c56499328/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.470129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ae48d-c940-4ee7-bea6-9c4d50b6e17f admin admin] RBAC: Authorizing `identity:create_grant(project_id=003ac163682d47fbb4ec29e4afeb1963, user_id=4037ef977e9c4d8692cb5d6c56499328, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:52.472757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ae48d-c940-4ee7-bea6-9c4d50b6e17f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:52.483528 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 840/1652] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:52 2022] PUT /identity/v3/projects/003ac163682d47fbb4ec29e4afeb1963/users/4037ef977e9c4d8692cb5d6c56499328/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:52.484306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ae48d-c940-4ee7-bea6-9c4d50b6e17f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:52.524802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475671a-dcd4-4ba1-8715-dced0c523ab9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:52.556500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ae48d-c940-4ee7-bea6-9c4d50b6e17f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:52.561496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c2d0d-9e58-4091-bb89-98072410ea23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:52.562194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c2d0d-9e58-4091-bb89-98072410ea23 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.562483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c2d0d-9e58-4091-bb89-98072410ea23 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.562761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c2d0d-9e58-4091-bb89-98072410ea23 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.563496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c2d0d-9e58-4091-bb89-98072410ea23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:52.565152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c2d0d-9e58-4091-bb89-98072410ea23 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:52.575246 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 841/1653] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:52.576246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c2d0d-9e58-4091-bb89-98072410ea23 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:52.585619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c2d0d-9e58-4091-bb89-98072410ea23 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:52.586759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc5de2-0faf-4f85-af6d-81cbe8c1be18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:52.587460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc5de2-0faf-4f85-af6d-81cbe8c1be18 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.587750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc5de2-0faf-4f85-af6d-81cbe8c1be18 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.588025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc5de2-0faf-4f85-af6d-81cbe8c1be18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.589754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-16bc5de2-0faf-4f85-af6d-81cbe8c1be18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:52.590419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f722a-f042-48a9-bcfb-fb26d780981b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:52.591092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f722a-f042-48a9-bcfb-fb26d780981b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.591382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f722a-f042-48a9-bcfb-fb26d780981b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.591663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f722a-f042-48a9-bcfb-fb26d780981b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.592385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f722a-f042-48a9-bcfb-fb26d780981b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:52.594013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f722a-f042-48a9-bcfb-fb26d780981b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:52.604599 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 842/1654] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:52.605440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f722a-f042-48a9-bcfb-fb26d780981b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:52.617565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f722a-f042-48a9-bcfb-fb26d780981b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:52.622499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae75c8c9-a03f-4ab8-9e25-8e3dcf25fa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:52.623358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75c8c9-a03f-4ab8-9e25-8e3dcf25fa45 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.623642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75c8c9-a03f-4ab8-9e25-8e3dcf25fa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.623923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae75c8c9-a03f-4ab8-9e25-8e3dcf25fa45 admin admin] PATH_INFO: `/v3/projects/70426a0425f54267b0964ce4505fbffb/users/27755a59f8064b99b322481803a0206c/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.625950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75c8c9-a03f-4ab8-9e25-8e3dcf25fa45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70426a0425f54267b0964ce4505fbffb, user_id=27755a59f8064b99b322481803a0206c, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:52.628598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae75c8c9-a03f-4ab8-9e25-8e3dcf25fa45 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:52.638898 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 843/1655] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:52 2022] PUT /identity/v3/projects/70426a0425f54267b0964ce4505fbffb/users/27755a59f8064b99b322481803a0206c/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:52.640086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56333c52-6bf0-4872-9d50-9216e8b92bba None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.640491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56333c52-6bf0-4872-9d50-9216e8b92bba None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.640780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56333c52-6bf0-4872-9d50-9216e8b92bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.646168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:52.660725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:52.664966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc5de2-0faf-4f85-af6d-81cbe8c1be18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:52.666718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc5de2-0faf-4f85-af6d-81cbe8c1be18 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:52.668572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-56333c52-6bf0-4872-9d50-9216e8b92bba None None] MFA Rules not processed for user `d2cee368d9a14c8bb3e665eed1fdeb7e`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:52.669791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-56333c52-6bf0-4872-9d50-9216e8b92bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:52.676449 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 813/1656] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:52 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:52.677331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc5de2-0faf-4f85-af6d-81cbe8c1be18 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:52.687661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc5de2-0faf-4f85-af6d-81cbe8c1be18 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:52.744209 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 844/1657] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:52.744209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56333c52-6bf0-4872-9d50-9216e8b92bba None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:52.749357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488cd218-e6c6-4def-912b-1406c0550c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:52.750077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cd218-e6c6-4def-912b-1406c0550c5d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.750369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cd218-e6c6-4def-912b-1406c0550c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.750652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488cd218-e6c6-4def-912b-1406c0550c5d admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.751562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cd218-e6c6-4def-912b-1406c0550c5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:02:52.751790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cd218-e6c6-4def-912b-1406c0550c5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:52.753417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488cd218-e6c6-4def-912b-1406c0550c5d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:52.770246 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 814/1658] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:02:52.771637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f46d74-57a4-4727-abaa-341518ccd724 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.771935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f46d74-57a4-4727-abaa-341518ccd724 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.781090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f46d74-57a4-4727-abaa-341518ccd724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.800842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-59f46d74-57a4-4727-abaa-341518ccd724 None None] MFA Rules not processed for user `4037ef977e9c4d8692cb5d6c56499328`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:52.802296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-59f46d74-57a4-4727-abaa-341518ccd724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:52.814135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56333c52-6bf0-4872-9d50-9216e8b92bba None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:52.819148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092a3dc-6a6f-4906-8fe0-825c7dac0d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:52.820311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092a3dc-6a6f-4906-8fe0-825c7dac0d12 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.820696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092a3dc-6a6f-4906-8fe0-825c7dac0d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.821085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092a3dc-6a6f-4906-8fe0-825c7dac0d12 admin admin] PATH_INFO: `/v3/projects/4c27b736741a4c6f9c7a8fc2bd133a95/users/b9c8dbd5526b46a2ba24b0db5ca3571e/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.823417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092a3dc-6a6f-4906-8fe0-825c7dac0d12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c27b736741a4c6f9c7a8fc2bd133a95, user_id=b9c8dbd5526b46a2ba24b0db5ca3571e, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:52.827044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092a3dc-6a6f-4906-8fe0-825c7dac0d12 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:52.848475 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 845/1659] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:52 2022] PUT /identity/v3/projects/4c27b736741a4c6f9c7a8fc2bd133a95/users/b9c8dbd5526b46a2ba24b0db5ca3571e/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:52.848475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092a3dc-6a6f-4906-8fe0-825c7dac0d12 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:52.880653 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 815/1660] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:52 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:52.880653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f46d74-57a4-4727-abaa-341518ccd724 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:52.891777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f46d74-57a4-4727-abaa-341518ccd724 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:52.930481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092a3dc-6a6f-4906-8fe0-825c7dac0d12 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:52.936134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1e553-ddb8-4a4c-95b5-0b174db77fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:52.940832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1e553-ddb8-4a4c-95b5-0b174db77fc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.940832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1e553-ddb8-4a4c-95b5-0b174db77fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.940832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1e553-ddb8-4a4c-95b5-0b174db77fc6 admin admin] PATH_INFO: `/v3/projects/6277287bf774466582bc214b11186b70/users/d1b067ff75914828a49edd61f6a9318b/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.940832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1e553-ddb8-4a4c-95b5-0b174db77fc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6277287bf774466582bc214b11186b70, user_id=d1b067ff75914828a49edd61f6a9318b, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:52.943440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1e553-ddb8-4a4c-95b5-0b174db77fc6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:52.954615 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 846/1661] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:52 2022] PUT /identity/v3/projects/6277287bf774466582bc214b11186b70/users/d1b067ff75914828a49edd61f6a9318b/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:52.958410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb82712-4724-467a-b1a7-3040f01ebeae None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.958410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb82712-4724-467a-b1a7-3040f01ebeae None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.958410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb82712-4724-467a-b1a7-3040f01ebeae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.971686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad71a5b7-e702-4463-96ac-0943bbc53c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:52.972692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71a5b7-e702-4463-96ac-0943bbc53c1a admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.972975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71a5b7-e702-4463-96ac-0943bbc53c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.976318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad71a5b7-e702-4463-96ac-0943bbc53c1a admin admin] PATH_INFO: `/v3/projects/0d723d6e08fa41c89a20c1fb568c5741/users/e63ff1a4d0f349e09f23613f4d94c280/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.976588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71a5b7-e702-4463-96ac-0943bbc53c1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d723d6e08fa41c89a20c1fb568c5741, user_id=e63ff1a4d0f349e09f23613f4d94c280, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:52.979317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad71a5b7-e702-4463-96ac-0943bbc53c1a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:52.989984 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 816/1662] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:52 2022] PUT /identity/v3/projects/0d723d6e08fa41c89a20c1fb568c5741/users/e63ff1a4d0f349e09f23613f4d94c280/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:52.990156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-aeb82712-4724-467a-b1a7-3040f01ebeae None None] MFA Rules not processed for user `27755a59f8064b99b322481803a0206c`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:52.991302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2707e-0b81-41f4-9f8b-86e00f5d47bb None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:52.991428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-aeb82712-4724-467a-b1a7-3040f01ebeae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:52.991619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2707e-0b81-41f4-9f8b-86e00f5d47bb None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:52.991892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2707e-0b81-41f4-9f8b-86e00f5d47bb None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:52.993617 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 817/1663] 10.222.0.33 () {54 vars in 961 bytes} [Sat May 28 21:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:02:52.994253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2707e-0b81-41f4-9f8b-86e00f5d47bb None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:53.001720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:53.009455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:53.067575 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 847/1664] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:53.068618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d5430-ccba-4c54-bba5-6e28043c2317 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:53.068917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d5430-ccba-4c54-bba5-6e28043c2317 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:53.069737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d5430-ccba-4c54-bba5-6e28043c2317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:53.073741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2707e-0b81-41f4-9f8b-86e00f5d47bb None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:53.078468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e1af1-f18b-4710-8bd0-cfab7527dfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:53.079241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e1af1-f18b-4710-8bd0-cfab7527dfcf admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:53.079595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e1af1-f18b-4710-8bd0-cfab7527dfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:53.079929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39e1af1-f18b-4710-8bd0-cfab7527dfcf admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:53.080679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e1af1-f18b-4710-8bd0-cfab7527dfcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:53.082719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39e1af1-f18b-4710-8bd0-cfab7527dfcf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:53.096318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f45d5430-ccba-4c54-bba5-6e28043c2317 None None] MFA Rules not processed for user `b9c8dbd5526b46a2ba24b0db5ca3571e`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:53.097536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f45d5430-ccba-4c54-bba5-6e28043c2317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:53.165354 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 848/1665] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:53.166652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489aa33-1851-4773-9828-65a32885653d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:53.166945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489aa33-1851-4773-9828-65a32885653d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:53.167227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489aa33-1851-4773-9828-65a32885653d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:53.179609 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 818/1666] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:02:52 2022] POST /identity/v3/projects => generated 403 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:53.180446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e1af1-f18b-4710-8bd0-cfab7527dfcf admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:53.196289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b489aa33-1851-4773-9828-65a32885653d None None] MFA Rules not processed for user `d1b067ff75914828a49edd61f6a9318b`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:53.197496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b489aa33-1851-4773-9828-65a32885653d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:53.255104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39e1af1-f18b-4710-8bd0-cfab7527dfcf admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:53.260774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af6269-79ca-4639-95f5-f80cdf2dd2d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:53.261419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af6269-79ca-4639-95f5-f80cdf2dd2d9 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:53.261702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af6269-79ca-4639-95f5-f80cdf2dd2d9 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:53.262013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af6269-79ca-4639-95f5-f80cdf2dd2d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:53.268133 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 849/1667] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:53.269409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8d222-5e2f-4f25-82b7-a5a85c2fb48a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:53.269704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8d222-5e2f-4f25-82b7-a5a85c2fb48a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:53.270021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c8d222-5e2f-4f25-82b7-a5a85c2fb48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:53.287195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f0c8d222-5e2f-4f25-82b7-a5a85c2fb48a None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:53.288907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f0c8d222-5e2f-4f25-82b7-a5a85c2fb48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:53.298324 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 850/1668] 10.222.0.33 () {58 vars in 1058 bytes} [Sat May 28 21:02:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:53.299575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5309a-5777-46f3-88f8-71e0e5823659 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:53.299815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5309a-5777-46f3-88f8-71e0e5823659 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:53.300095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5309a-5777-46f3-88f8-71e0e5823659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:53.308081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:53.323082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af6269-79ca-4639-95f5-f80cdf2dd2d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:53.324652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af6269-79ca-4639-95f5-f80cdf2dd2d9 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:53.327300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-83e5309a-5777-46f3-88f8-71e0e5823659 None None] MFA Rules not processed for user `e63ff1a4d0f349e09f23613f4d94c280`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:53.328544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-83e5309a-5777-46f3-88f8-71e0e5823659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:53.334532 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 819/1669] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:53.335390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af6269-79ca-4639-95f5-f80cdf2dd2d9 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:53.364615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:53.400480 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 851/1670] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:53 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:53.401422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5309a-5777-46f3-88f8-71e0e5823659 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:53.403984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af6269-79ca-4639-95f5-f80cdf2dd2d9 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:53.408911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05962d-92b8-4177-bce1-b3c114331e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:53.409587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05962d-92b8-4177-bce1-b3c114331e06 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:53.409977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05962d-92b8-4177-bce1-b3c114331e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:53.410359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d05962d-92b8-4177-bce1-b3c114331e06 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:53.411059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05962d-92b8-4177-bce1-b3c114331e06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:53.413576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d05962d-92b8-4177-bce1-b3c114331e06 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:53.416196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5309a-5777-46f3-88f8-71e0e5823659 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:53.421827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701d98b-f6ce-4521-9118-4b1cfc877769 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:53.422561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701d98b-f6ce-4521-9118-4b1cfc877769 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:53.422850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701d98b-f6ce-4521-9118-4b1cfc877769 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:53.423128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701d98b-f6ce-4521-9118-4b1cfc877769 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:53.493374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701d98b-f6ce-4521-9118-4b1cfc877769 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:53.495087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701d98b-f6ce-4521-9118-4b1cfc877769 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:53.504916 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 852/1671] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:53.521649 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 820/1672] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:02:53 2022] POST /identity/v3/users => generated 302 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:53.526500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701d98b-f6ce-4521-9118-4b1cfc877769 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:53.536563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701d98b-f6ce-4521-9118-4b1cfc877769 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:53.541741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77f79f-8530-4f9b-bc04-3f4b4f4b8bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:53.542422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77f79f-8530-4f9b-bc04-3f4b4f4b8bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:53.542704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77f79f-8530-4f9b-bc04-3f4b4f4b8bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:53.542982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf77f79f-8530-4f9b-bc04-3f4b4f4b8bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:53.543705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77f79f-8530-4f9b-bc04-3f4b4f4b8bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:53.545313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf77f79f-8530-4f9b-bc04-3f4b4f4b8bd8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:53.555316 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 853/1673] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:02:53.559149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05962d-92b8-4177-bce1-b3c114331e06 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:53.569400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d05962d-92b8-4177-bce1-b3c114331e06 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:53.579924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b20b23-5fa8-4289-aeac-da072f6fda83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:53.579924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b20b23-5fa8-4289-aeac-da072f6fda83 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:53.579924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b20b23-5fa8-4289-aeac-da072f6fda83 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:53.579924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b20b23-5fa8-4289-aeac-da072f6fda83 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:53.580301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b20b23-5fa8-4289-aeac-da072f6fda83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:53.580762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b20b23-5fa8-4289-aeac-da072f6fda83 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:53.596822 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 821/1674] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:02:53.601295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77f79f-8530-4f9b-bc04-3f4b4f4b8bd8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:53.606912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b20b23-5fa8-4289-aeac-da072f6fda83 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:53.611938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf77f79f-8530-4f9b-bc04-3f4b4f4b8bd8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:53.616665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003b687-dab3-4d56-b583-0aadeaccfa46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:53.617560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003b687-dab3-4d56-b583-0aadeaccfa46 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:53.617891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003b687-dab3-4d56-b583-0aadeaccfa46 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:53.618164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003b687-dab3-4d56-b583-0aadeaccfa46 admin admin] PATH_INFO: `/v3/projects/0e60b122ef584254b06fd8258c987f8d/users/cdeaa660f8ec465bba68096d0ed4b1ae/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:53.621450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b20b23-5fa8-4289-aeac-da072f6fda83 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:53.627355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732c780-c0db-477a-896c-4f71a3394f3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:53.628107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732c780-c0db-477a-896c-4f71a3394f3f service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:53.628529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732c780-c0db-477a-896c-4f71a3394f3f service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:53.628929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732c780-c0db-477a-896c-4f71a3394f3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:53.631056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2732c780-c0db-477a-896c-4f71a3394f3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:53.634247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003b687-dab3-4d56-b583-0aadeaccfa46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e60b122ef584254b06fd8258c987f8d, user_id=cdeaa660f8ec465bba68096d0ed4b1ae, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:53.637042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003b687-dab3-4d56-b583-0aadeaccfa46 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:53.697316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732c780-c0db-477a-896c-4f71a3394f3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:53.698959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2732c780-c0db-477a-896c-4f71a3394f3f service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:53.708211 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 822/1675] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:53.716989 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 854/1676] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:02:53 2022] PUT /identity/v3/projects/0e60b122ef584254b06fd8258c987f8d/users/cdeaa660f8ec465bba68096d0ed4b1ae/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:53.723615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46b855-a096-42c5-a930-6b14dea73013 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:53.723615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46b855-a096-42c5-a930-6b14dea73013 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:53.723615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46b855-a096-42c5-a930-6b14dea73013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:53.727810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003b687-dab3-4d56-b583-0aadeaccfa46 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:53.745707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9d46b855-a096-42c5-a930-6b14dea73013 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:53.747313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9d46b855-a096-42c5-a930-6b14dea73013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:53.793680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:53.807918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003b687-dab3-4d56-b583-0aadeaccfa46 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:53.814323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911cb59-65c2-48f1-86bd-1e64399b6324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:53.814323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911cb59-65c2-48f1-86bd-1e64399b6324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:53.814323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911cb59-65c2-48f1-86bd-1e64399b6324 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:53.814323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3911cb59-65c2-48f1-86bd-1e64399b6324 admin admin] PATH_INFO: `/v3/users/f35355f9f95643ecb2ffc62e648f431a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:53.814690 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 823/1677] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:53.815509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46b855-a096-42c5-a930-6b14dea73013 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:53.818563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911cb59-65c2-48f1-86bd-1e64399b6324 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f35355f9f95643ecb2ffc62e648f431a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:53.821008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3911cb59-65c2-48f1-86bd-1e64399b6324 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:53.860731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:53.886276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46b855-a096-42c5-a930-6b14dea73013 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:53.891913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2a2a4-f3af-4d89-a5be-22f342bbc928 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:53.892563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2a2a4-f3af-4d89-a5be-22f342bbc928 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:53.892852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2a2a4-f3af-4d89-a5be-22f342bbc928 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:53.893135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f2a2a4-f3af-4d89-a5be-22f342bbc928 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:53.956739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2a2a4-f3af-4d89-a5be-22f342bbc928 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:53.958379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f2a2a4-f3af-4d89-a5be-22f342bbc928 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:53.967861 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 824/1678] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:53.968648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2a2a4-f3af-4d89-a5be-22f342bbc928 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:53.979823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f2a2a4-f3af-4d89-a5be-22f342bbc928 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:53.986802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5315c6b-7bf4-490e-9243-7a25507614c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:53.987447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5315c6b-7bf4-490e-9243-7a25507614c2 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:53.987728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5315c6b-7bf4-490e-9243-7a25507614c2 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:53.988008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5315c6b-7bf4-490e-9243-7a25507614c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:53.993515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3911cb59-65c2-48f1-86bd-1e64399b6324 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f35355f9f95643ecb2ffc62e648f431a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:54.000765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3911cb59-65c2-48f1-86bd-1e64399b6324 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f35355f9f95643ecb2ffc62e648f431a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:54.002643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3911cb59-65c2-48f1-86bd-1e64399b6324 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f35355f9f95643ecb2ffc62e648f431a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:54.002643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3911cb59-65c2-48f1-86bd-1e64399b6324 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f35355f9f95643ecb2ffc62e648f431a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:54.046934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3911cb59-65c2-48f1-86bd-1e64399b6324 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f35355f9f95643ecb2ffc62e648f431a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:54.048757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5315c6b-7bf4-490e-9243-7a25507614c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:54.050290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5315c6b-7bf4-490e-9243-7a25507614c2 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:54.060690 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 825/1679] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:54.061504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5315c6b-7bf4-490e-9243-7a25507614c2 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:54.063041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e5315c6b-7bf4-490e-9243-7a25507614c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:54.079267 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 855/1680] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:02:53 2022] DELETE /identity/v3/users/f35355f9f95643ecb2ffc62e648f431a => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:02:54.080269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911cb59-65c2-48f1-86bd-1e64399b6324 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:54.084402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3911cb59-65c2-48f1-86bd-1e64399b6324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:54.130473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5315c6b-7bf4-490e-9243-7a25507614c2 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:54.150549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:54.170618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3911cb59-65c2-48f1-86bd-1e64399b6324 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:54.176279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85cfb3-6c09-497c-bbe9-9c85b40f6290 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:54.176931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85cfb3-6c09-497c-bbe9-9c85b40f6290 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:54.177222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85cfb3-6c09-497c-bbe9-9c85b40f6290 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:54.177507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85cfb3-6c09-497c-bbe9-9c85b40f6290 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:54.179463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3a85cfb3-6c09-497c-bbe9-9c85b40f6290 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:54.180806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e20e66-dd5f-478b-87cc-9710dfeecad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:54.181480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e20e66-dd5f-478b-87cc-9710dfeecad2 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:54.181812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e20e66-dd5f-478b-87cc-9710dfeecad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:54.182347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e20e66-dd5f-478b-87cc-9710dfeecad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:54.184662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-40e20e66-dd5f-478b-87cc-9710dfeecad2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:54.241858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85cfb3-6c09-497c-bbe9-9c85b40f6290 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:54.243467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85cfb3-6c09-497c-bbe9-9c85b40f6290 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:54.248739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:54.249907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e20e66-dd5f-478b-87cc-9710dfeecad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:54.251376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e20e66-dd5f-478b-87cc-9710dfeecad2 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:54.253722 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 856/1681] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:54 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:54.259561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee715b-b413-40a6-a995-2bcc89e4ce82 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:54.259850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee715b-b413-40a6-a995-2bcc89e4ce82 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:54.260131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee715b-b413-40a6-a995-2bcc89e4ce82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:54.262656 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 826/1682] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:54.264180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a93297-8b5e-4c6f-8496-1329b122d341 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:54.264568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a93297-8b5e-4c6f-8496-1329b122d341 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:54.264946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a93297-8b5e-4c6f-8496-1329b122d341 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:54.267234 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 827/1683] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:02:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:02:54.273202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a93297-8b5e-4c6f-8496-1329b122d341 None None] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:02:54.275036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-82a93297-8b5e-4c6f-8496-1329b122d341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:54.278231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8bee715b-b413-40a6-a995-2bcc89e4ce82 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:54.279875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8bee715b-b413-40a6-a995-2bcc89e4ce82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:54.290306 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 857/1684] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:54.296238 ncdv-master-1 devstack@keystone.service[103104]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-82a93297-8b5e-4c6f-8496-1329b122d341 None None] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. May 28 21:02:54.296616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a93297-8b5e-4c6f-8496-1329b122d341 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:54.298060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-82a93297-8b5e-4c6f-8496-1329b122d341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:54.301263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee715b-b413-40a6-a995-2bcc89e4ce82 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:54.309711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8bee715b-b413-40a6-a995-2bcc89e4ce82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:54.322178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee715b-b413-40a6-a995-2bcc89e4ce82 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:54.326008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f99fa-c79b-4718-8cc8-8b10a46c2fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:54.326720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f99fa-c79b-4718-8cc8-8b10a46c2fea service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:54.326997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f99fa-c79b-4718-8cc8-8b10a46c2fea service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:54.327300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f99fa-c79b-4718-8cc8-8b10a46c2fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:54.338176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f99fa-c79b-4718-8cc8-8b10a46c2fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:54.339726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f99fa-c79b-4718-8cc8-8b10a46c2fea service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:54.350921 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 858/1685] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:02:54.377235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a93297-8b5e-4c6f-8496-1329b122d341 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:54.383414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647cfe8-699c-421a-9cad-e9d2ba2ca492 tempest-FlavorsAccessTestJSON-525584540 tempest-FlavorsAccessTestJSON-525584540-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4037ef977e9c4d8692cb5d6c56499328', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '003ac163682d47fbb4ec29e4afeb1963', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:54.384525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647cfe8-699c-421a-9cad-e9d2ba2ca492 tempest-FlavorsAccessTestJSON-525584540 tempest-FlavorsAccessTestJSON-525584540-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:54.384809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647cfe8-699c-421a-9cad-e9d2ba2ca492 tempest-FlavorsAccessTestJSON-525584540 tempest-FlavorsAccessTestJSON-525584540-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:54.385490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647cfe8-699c-421a-9cad-e9d2ba2ca492 tempest-FlavorsAccessTestJSON-525584540 tempest-FlavorsAccessTestJSON-525584540-project-admin] PATH_INFO: `/v3/projects/d257d738741646539f55b9dab71b33bf` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:54.386746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647cfe8-699c-421a-9cad-e9d2ba2ca492 tempest-FlavorsAccessTestJSON-525584540 tempest-FlavorsAccessTestJSON-525584540-project-admin] RBAC: Authorizing `identity:get_project(project_id=d257d738741646539f55b9dab71b33bf)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:54.388976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a647cfe8-699c-421a-9cad-e9d2ba2ca492 tempest-FlavorsAccessTestJSON-525584540 tempest-FlavorsAccessTestJSON-525584540-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:54.390810 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 828/1686] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:02:54 2022] GET /identity/v3/projects/d257d738741646539f55b9dab71b33bf => generated 377 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:02:54.453543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c024d9-4a5a-4660-912e-2d5049159b6b None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:54.453863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c024d9-4a5a-4660-912e-2d5049159b6b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:54.454137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c024d9-4a5a-4660-912e-2d5049159b6b None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:54.455506 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 859/1687] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:02:54.461389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0f80f-0bd7-4930-894e-a343b414b6ff None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:54.461803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0f80f-0bd7-4930-894e-a343b414b6ff None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:54.462131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0f80f-0bd7-4930-894e-a343b414b6ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:54.502352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c024d9-4a5a-4660-912e-2d5049159b6b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:54.514762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ffb0f80f-0bd7-4930-894e-a343b414b6ff None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:54.516209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ffb0f80f-0bd7-4930-894e-a343b414b6ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:54.519677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c024d9-4a5a-4660-912e-2d5049159b6b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:54.526135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864945a-2565-4ea0-84a6-234e699ba03b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:54.526902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864945a-2565-4ea0-84a6-234e699ba03b service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:54.527306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864945a-2565-4ea0-84a6-234e699ba03b service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:54.527692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0864945a-2565-4ea0-84a6-234e699ba03b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:54.530982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0864945a-2565-4ea0-84a6-234e699ba03b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:54.597106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:54.613479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0864945a-2565-4ea0-84a6-234e699ba03b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:54.615715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0864945a-2565-4ea0-84a6-234e699ba03b service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:54.628426 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 860/1688] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:02:54.634983 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 829/1689] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:02:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:54.668575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864945a-2565-4ea0-84a6-234e699ba03b service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:54.670915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0864945a-2565-4ea0-84a6-234e699ba03b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:54.692239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0864945a-2565-4ea0-84a6-234e699ba03b service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:54.700329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00328f-d0f1-4319-807f-4e0327e12c81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:54.700987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00328f-d0f1-4319-807f-4e0327e12c81 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:54.701306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00328f-d0f1-4319-807f-4e0327e12c81 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:54.701585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00328f-d0f1-4319-807f-4e0327e12c81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:54.713195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00328f-d0f1-4319-807f-4e0327e12c81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:54.715681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00328f-d0f1-4319-807f-4e0327e12c81 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:54.730081 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 861/1690] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:54.731400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00328f-d0f1-4319-807f-4e0327e12c81 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:54.731559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0f80f-0bd7-4930-894e-a343b414b6ff None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:54.733387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9d00328f-d0f1-4319-807f-4e0327e12c81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:54.748177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0f80f-0bd7-4930-894e-a343b414b6ff None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:54.751578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00328f-d0f1-4319-807f-4e0327e12c81 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:54.757292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5a399-ce80-4b4f-b1bc-d5d687e1b740 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:54.757431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20507897-741f-4e52-83e3-c04e6eacf8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:54.758294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20507897-741f-4e52-83e3-c04e6eacf8ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:54.758583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20507897-741f-4e52-83e3-c04e6eacf8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:54.758863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20507897-741f-4e52-83e3-c04e6eacf8ad admin admin] PATH_INFO: `/v3/users/39143b8a6ad4488ca98a8e5749c8eeb9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:54.759092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5a399-ce80-4b4f-b1bc-d5d687e1b740 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:54.759656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5a399-ce80-4b4f-b1bc-d5d687e1b740 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:54.760440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca5a399-ce80-4b4f-b1bc-d5d687e1b740 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:54.762806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3ca5a399-ce80-4b4f-b1bc-d5d687e1b740 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:54.763834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20507897-741f-4e52-83e3-c04e6eacf8ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=39143b8a6ad4488ca98a8e5749c8eeb9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:54.766320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20507897-741f-4e52-83e3-c04e6eacf8ad admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:54.844860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca5a399-ce80-4b4f-b1bc-d5d687e1b740 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:54.846698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca5a399-ce80-4b4f-b1bc-d5d687e1b740 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:54.857217 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 830/1691] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:54.858136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5a399-ce80-4b4f-b1bc-d5d687e1b740 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:54.859508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3ca5a399-ce80-4b4f-b1bc-d5d687e1b740 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:54.874443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca5a399-ce80-4b4f-b1bc-d5d687e1b740 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:54.880018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2b1c5f-bdd1-4ed2-a10c-a6801143f930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:54.880880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b1c5f-bdd1-4ed2-a10c-a6801143f930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:54.881163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b1c5f-bdd1-4ed2-a10c-a6801143f930 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:54.883698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2b1c5f-bdd1-4ed2-a10c-a6801143f930 admin admin] PATH_INFO: `/v3/projects/9ec9619348174863907663b9b935773b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:54.883698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b1c5f-bdd1-4ed2-a10c-a6801143f930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ec9619348174863907663b9b935773b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:54.885234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2b1c5f-bdd1-4ed2-a10c-a6801143f930 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:55.092766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20507897-741f-4e52-83e3-c04e6eacf8ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39143b8a6ad4488ca98a8e5749c8eeb9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:55.093067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7f2b1c5f-bdd1-4ed2-a10c-a6801143f930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ec9619348174863907663b9b935773b', 'request_id': 'req-7f2b1c5f-bdd1-4ed2-a10c-a6801143f930'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:55.099403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7f2b1c5f-bdd1-4ed2-a10c-a6801143f930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ec9619348174863907663b9b935773b', 'request_id': 'req-7f2b1c5f-bdd1-4ed2-a10c-a6801143f930'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:55.101997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20507897-741f-4e52-83e3-c04e6eacf8ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39143b8a6ad4488ca98a8e5749c8eeb9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:55.102491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20507897-741f-4e52-83e3-c04e6eacf8ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39143b8a6ad4488ca98a8e5749c8eeb9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:55.106204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20507897-741f-4e52-83e3-c04e6eacf8ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39143b8a6ad4488ca98a8e5749c8eeb9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:55.194584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7f2b1c5f-bdd1-4ed2-a10c-a6801143f930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ec9619348174863907663b9b935773b', 'request_id': 'req-7f2b1c5f-bdd1-4ed2-a10c-a6801143f930'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:55.220754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7f2b1c5f-bdd1-4ed2-a10c-a6801143f930 admin admin] The token cache is being invalidate because project 9ec9619348174863907663b9b935773b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:02:55.220754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7f2b1c5f-bdd1-4ed2-a10c-a6801143f930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:55.220754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7f2b1c5f-bdd1-4ed2-a10c-a6801143f930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:55.224204 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 831/1692] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:02:54 2022] DELETE /identity/v3/projects/9ec9619348174863907663b9b935773b => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:55.224204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec566e-e21a-485b-810c-5100b4f23efc None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:55.224204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec566e-e21a-485b-810c-5100b4f23efc None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:55.229494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ec566e-e21a-485b-810c-5100b4f23efc None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:55.230634 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 832/1693] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:02:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:02:55.231354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec566e-e21a-485b-810c-5100b4f23efc None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:55.233395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a6ec566e-e21a-485b-810c-5100b4f23efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:55.274672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20507897-741f-4e52-83e3-c04e6eacf8ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39143b8a6ad4488ca98a8e5749c8eeb9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:55.297037 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 862/1694] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:02:54 2022] DELETE /identity/v3/users/39143b8a6ad4488ca98a8e5749c8eeb9 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:55.297998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20507897-741f-4e52-83e3-c04e6eacf8ad admin admin] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:02:55.301318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-20507897-741f-4e52-83e3-c04e6eacf8ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:55.323049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:55.328917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ec566e-e21a-485b-810c-5100b4f23efc None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:55.347615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:55.385392 ncdv-master-1 devstack@keystone.service[103103]: WARNING keystone.server.flask.request_processing.middleware.auth_context [None req-20507897-741f-4e52-83e3-c04e6eacf8ad admin admin] A valid token was submitted as a service token, but it was not a valid service token. This is incorrect but backwards compatible behaviour. This will be removed in future releases. May 28 21:02:55.385793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20507897-741f-4e52-83e3-c04e6eacf8ad admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:55.387183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-20507897-741f-4e52-83e3-c04e6eacf8ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:55.389956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587f78a-4734-4aa0-a265-e8674c8be42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:55.390825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587f78a-4734-4aa0-a265-e8674c8be42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:55.391112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587f78a-4734-4aa0-a265-e8674c8be42e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:55.391393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587f78a-4734-4aa0-a265-e8674c8be42e admin admin] PATH_INFO: `/v3/users/27755a59f8064b99b322481803a0206c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:55.393039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587f78a-4734-4aa0-a265-e8674c8be42e admin admin] RBAC: Authorizing `identity:delete_user(user_id=27755a59f8064b99b322481803a0206c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:55.395117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587f78a-4734-4aa0-a265-e8674c8be42e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:55.454643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20507897-741f-4e52-83e3-c04e6eacf8ad admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:55.459245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a850fa-052b-4ffd-9f08-0a1127f80507 tempest-FlavorsAccessTestJSON-525584540 tempest-FlavorsAccessTestJSON-525584540-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4037ef977e9c4d8692cb5d6c56499328', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '003ac163682d47fbb4ec29e4afeb1963', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:55.460140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a850fa-052b-4ffd-9f08-0a1127f80507 tempest-FlavorsAccessTestJSON-525584540 tempest-FlavorsAccessTestJSON-525584540-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:55.460421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a850fa-052b-4ffd-9f08-0a1127f80507 tempest-FlavorsAccessTestJSON-525584540 tempest-FlavorsAccessTestJSON-525584540-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:55.460696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a850fa-052b-4ffd-9f08-0a1127f80507 tempest-FlavorsAccessTestJSON-525584540 tempest-FlavorsAccessTestJSON-525584540-project-admin] PATH_INFO: `/v3/projects/d257d738741646539f55b9dab71b33bf` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:55.461971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a850fa-052b-4ffd-9f08-0a1127f80507 tempest-FlavorsAccessTestJSON-525584540 tempest-FlavorsAccessTestJSON-525584540-project-admin] RBAC: Authorizing `identity:get_project(project_id=d257d738741646539f55b9dab71b33bf)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:55.464025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a850fa-052b-4ffd-9f08-0a1127f80507 tempest-FlavorsAccessTestJSON-525584540 tempest-FlavorsAccessTestJSON-525584540-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:55.465734 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 863/1695] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:02:55 2022] GET /identity/v3/projects/d257d738741646539f55b9dab71b33bf => generated 377 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:02:55.466901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a097a83-d346-4db3-9dd6-d771262cf1d7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:55.467186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a097a83-d346-4db3-9dd6-d771262cf1d7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:55.467464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a097a83-d346-4db3-9dd6-d771262cf1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:55.484583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4a097a83-d346-4db3-9dd6-d771262cf1d7 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:55.485788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4a097a83-d346-4db3-9dd6-d771262cf1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:55.494018 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 864/1696] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:55.495275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62840661-9b4a-4941-8a3c-d4917c2ac904 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:55.495563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62840661-9b4a-4941-8a3c-d4917c2ac904 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:55.496202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62840661-9b4a-4941-8a3c-d4917c2ac904 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:55.499452 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 865/1697] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:02:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:02:55.508826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda6004-3036-4bb8-b5ae-6a004f8b7c63 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:55.509150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda6004-3036-4bb8-b5ae-6a004f8b7c63 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:55.509438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda6004-3036-4bb8-b5ae-6a004f8b7c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:55.527463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ceda6004-3036-4bb8-b5ae-6a004f8b7c63 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:55.528714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ceda6004-3036-4bb8-b5ae-6a004f8b7c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:55.592641 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 866/1698] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:02:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:02:55.593478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda6004-3036-4bb8-b5ae-6a004f8b7c63 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:55.594785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ceda6004-3036-4bb8-b5ae-6a004f8b7c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:55.621879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b587f78a-4734-4aa0-a265-e8674c8be42e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27755a59f8064b99b322481803a0206c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:55.622690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b587f78a-4734-4aa0-a265-e8674c8be42e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27755a59f8064b99b322481803a0206c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:55.645164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b587f78a-4734-4aa0-a265-e8674c8be42e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27755a59f8064b99b322481803a0206c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:55.682128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda6004-3036-4bb8-b5ae-6a004f8b7c63 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:55.685827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc0c51-9a19-476a-ad51-e0f349985eae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:55.687703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc0c51-9a19-476a-ad51-e0f349985eae service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:55.687703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc0c51-9a19-476a-ad51-e0f349985eae service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:55.687703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc0c51-9a19-476a-ad51-e0f349985eae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:55.697422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc0c51-9a19-476a-ad51-e0f349985eae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:55.698923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc0c51-9a19-476a-ad51-e0f349985eae service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:55.703352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b587f78a-4734-4aa0-a265-e8674c8be42e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27755a59f8064b99b322481803a0206c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:55.705202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b587f78a-4734-4aa0-a265-e8674c8be42e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27755a59f8064b99b322481803a0206c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:55.708950 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 867/1699] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:55.710421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc0c51-9a19-476a-ad51-e0f349985eae service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:55.711980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cdfc0c51-9a19-476a-ad51-e0f349985eae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:55.727503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc0c51-9a19-476a-ad51-e0f349985eae service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:55.731452 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 833/1700] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:02:55 2022] DELETE /identity/v3/users/27755a59f8064b99b322481803a0206c => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:55.735406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9a15f-838e-4cfb-b2e4-264c98d21d4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:55.735623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9a15f-838e-4cfb-b2e4-264c98d21d4f service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:55.735837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9a15f-838e-4cfb-b2e4-264c98d21d4f service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:55.736271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9a15f-838e-4cfb-b2e4-264c98d21d4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:55.738526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-45e9a15f-838e-4cfb-b2e4-264c98d21d4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:55.752793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9a15f-838e-4cfb-b2e4-264c98d21d4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:55.762565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e9a15f-838e-4cfb-b2e4-264c98d21d4f service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:55.775281 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 868/1701] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:55.973706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:56.000164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587f78a-4734-4aa0-a265-e8674c8be42e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:56.001781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b587f78a-4734-4aa0-a265-e8674c8be42e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:56.113296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587f78a-4734-4aa0-a265-e8674c8be42e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:56.121824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5ba09-15a6-4dd7-b83a-e779e7c62057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:56.123760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5ba09-15a6-4dd7-b83a-e779e7c62057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:56.126460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5ba09-15a6-4dd7-b83a-e779e7c62057 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:56.126882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5ba09-15a6-4dd7-b83a-e779e7c62057 admin admin] PATH_INFO: `/v3/users/cf5f88829a634821a49cffe623125155` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:56.129164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5ba09-15a6-4dd7-b83a-e779e7c62057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf5f88829a634821a49cffe623125155)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:56.132693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5ba09-15a6-4dd7-b83a-e779e7c62057 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:56.302073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:56.349127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9a15f-838e-4cfb-b2e4-264c98d21d4f service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:56.350869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-45e9a15f-838e-4cfb-b2e4-264c98d21d4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:56.374696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9a15f-838e-4cfb-b2e4-264c98d21d4f service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:56.379892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5cca1-51de-4f2d-a217-a52045b4b593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:56.382532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5cca1-51de-4f2d-a217-a52045b4b593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:56.382887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5cca1-51de-4f2d-a217-a52045b4b593 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:56.383232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5cca1-51de-4f2d-a217-a52045b4b593 admin admin] PATH_INFO: `/v3/users/f0598f70d72e4fc092db4dfa495eef0d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:56.384606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5cca1-51de-4f2d-a217-a52045b4b593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0598f70d72e4fc092db4dfa495eef0d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:56.386699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5cca1-51de-4f2d-a217-a52045b4b593 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:56.434209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3fc5ba09-15a6-4dd7-b83a-e779e7c62057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf5f88829a634821a49cffe623125155'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:56.434890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3fc5ba09-15a6-4dd7-b83a-e779e7c62057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf5f88829a634821a49cffe623125155'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:56.445675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3fc5ba09-15a6-4dd7-b83a-e779e7c62057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf5f88829a634821a49cffe623125155'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:56.526708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3fc5ba09-15a6-4dd7-b83a-e779e7c62057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf5f88829a634821a49cffe623125155'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:56.527518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3fc5ba09-15a6-4dd7-b83a-e779e7c62057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf5f88829a634821a49cffe623125155'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:56.559145 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 834/1702] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:02:55 2022] DELETE /identity/v3/users/cf5f88829a634821a49cffe623125155 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:56.559145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5ba09-15a6-4dd7-b83a-e779e7c62057 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:56.559430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3fc5ba09-15a6-4dd7-b83a-e779e7c62057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:56.630561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dbd5cca1-51de-4f2d-a217-a52045b4b593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0598f70d72e4fc092db4dfa495eef0d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:56.639893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dbd5cca1-51de-4f2d-a217-a52045b4b593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0598f70d72e4fc092db4dfa495eef0d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:56.639893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dbd5cca1-51de-4f2d-a217-a52045b4b593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0598f70d72e4fc092db4dfa495eef0d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:56.640461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dbd5cca1-51de-4f2d-a217-a52045b4b593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0598f70d72e4fc092db4dfa495eef0d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:56.684488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5ba09-15a6-4dd7-b83a-e779e7c62057 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:56.690922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13320d-1ef3-4b6d-bf70-3a549a971394 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:56.694791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13320d-1ef3-4b6d-bf70-3a549a971394 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:56.694791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13320d-1ef3-4b6d-bf70-3a549a971394 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:56.694791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da13320d-1ef3-4b6d-bf70-3a549a971394 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:56.695576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-da13320d-1ef3-4b6d-bf70-3a549a971394 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:56.746480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dbd5cca1-51de-4f2d-a217-a52045b4b593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0598f70d72e4fc092db4dfa495eef0d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:56.781711 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 869/1703] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:02:56 2022] DELETE /identity/v3/users/f0598f70d72e4fc092db4dfa495eef0d => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:56.784437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc2929-6122-4f2b-99a7-0869ccdc2365 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:56.784789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc2929-6122-4f2b-99a7-0869ccdc2365 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:56.785082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc2929-6122-4f2b-99a7-0869ccdc2365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:56.808854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f9dc2929-6122-4f2b-99a7-0869ccdc2365 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:56.810169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f9dc2929-6122-4f2b-99a7-0869ccdc2365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:56.810321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13320d-1ef3-4b6d-bf70-3a549a971394 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:56.811792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da13320d-1ef3-4b6d-bf70-3a549a971394 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:56.826942 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 835/1704] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:56 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:56.828358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13320d-1ef3-4b6d-bf70-3a549a971394 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:56.830691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-da13320d-1ef3-4b6d-bf70-3a549a971394 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:56.899231 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 870/1705] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:56.900971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc2929-6122-4f2b-99a7-0869ccdc2365 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:56.908875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f9dc2929-6122-4f2b-99a7-0869ccdc2365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:56.926065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:56.926838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc2929-6122-4f2b-99a7-0869ccdc2365 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:56.933926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8056dc14-b6a5-4de3-ae0e-9bdc01812723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:56.937979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056dc14-b6a5-4de3-ae0e-9bdc01812723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:56.937979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056dc14-b6a5-4de3-ae0e-9bdc01812723 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:56.937979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8056dc14-b6a5-4de3-ae0e-9bdc01812723 admin admin] PATH_INFO: `/v3/projects/281d71da66424c949d37bda848d84fc2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:56.938308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056dc14-b6a5-4de3-ae0e-9bdc01812723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=281d71da66424c949d37bda848d84fc2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:56.939390 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-8056dc14-b6a5-4de3-ae0e-9bdc01812723 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:02:56.939390 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:02:56.939390 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:02:56.940607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8056dc14-b6a5-4de3-ae0e-9bdc01812723 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:56.979613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da13320d-1ef3-4b6d-bf70-3a549a971394 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:56.988714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5ff84-1f2a-47ee-a43e-c105526c2683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:56.989604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5ff84-1f2a-47ee-a43e-c105526c2683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:56.990184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5ff84-1f2a-47ee-a43e-c105526c2683 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:56.990323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb5ff84-1f2a-47ee-a43e-c105526c2683 admin admin] PATH_INFO: `/v3/projects/70426a0425f54267b0964ce4505fbffb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:56.994753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5ff84-1f2a-47ee-a43e-c105526c2683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70426a0425f54267b0964ce4505fbffb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:56.998707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb5ff84-1f2a-47ee-a43e-c105526c2683 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:57.097299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8056dc14-b6a5-4de3-ae0e-9bdc01812723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '281d71da66424c949d37bda848d84fc2', 'request_id': 'req-8056dc14-b6a5-4de3-ae0e-9bdc01812723'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:57.118862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8056dc14-b6a5-4de3-ae0e-9bdc01812723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '281d71da66424c949d37bda848d84fc2', 'request_id': 'req-8056dc14-b6a5-4de3-ae0e-9bdc01812723'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:57.122766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:57.153532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cbb5ff84-1f2a-47ee-a43e-c105526c2683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70426a0425f54267b0964ce4505fbffb', 'request_id': 'req-cbb5ff84-1f2a-47ee-a43e-c105526c2683'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:57.166386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cbb5ff84-1f2a-47ee-a43e-c105526c2683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70426a0425f54267b0964ce4505fbffb', 'request_id': 'req-cbb5ff84-1f2a-47ee-a43e-c105526c2683'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:57.194807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8056dc14-b6a5-4de3-ae0e-9bdc01812723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '281d71da66424c949d37bda848d84fc2', 'request_id': 'req-8056dc14-b6a5-4de3-ae0e-9bdc01812723'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:57.217253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8056dc14-b6a5-4de3-ae0e-9bdc01812723 admin admin] The token cache is being invalidate because project 281d71da66424c949d37bda848d84fc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:02:57.218316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8056dc14-b6a5-4de3-ae0e-9bdc01812723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:57.218913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8056dc14-b6a5-4de3-ae0e-9bdc01812723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:57.220834 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 871/1706] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:02:56 2022] DELETE /identity/v3/projects/281d71da66424c949d37bda848d84fc2 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:57.223204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e4a55-b535-4b49-832e-ec721820ab04 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:57.225338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e4a55-b535-4b49-832e-ec721820ab04 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:57.226096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853e4a55-b535-4b49-832e-ec721820ab04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:57.249476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-853e4a55-b535-4b49-832e-ec721820ab04 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:57.250762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-853e4a55-b535-4b49-832e-ec721820ab04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:57.258145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cbb5ff84-1f2a-47ee-a43e-c105526c2683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70426a0425f54267b0964ce4505fbffb', 'request_id': 'req-cbb5ff84-1f2a-47ee-a43e-c105526c2683'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:57.292095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cbb5ff84-1f2a-47ee-a43e-c105526c2683 admin admin] The token cache is being invalidate because project 70426a0425f54267b0964ce4505fbffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:02:57.293700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cbb5ff84-1f2a-47ee-a43e-c105526c2683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:57.293700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cbb5ff84-1f2a-47ee-a43e-c105526c2683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:57.294851 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 836/1707] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:02:56 2022] DELETE /identity/v3/projects/70426a0425f54267b0964ce4505fbffb => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:57.295674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5ff84-1f2a-47ee-a43e-c105526c2683 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:57.297793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cbb5ff84-1f2a-47ee-a43e-c105526c2683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:57.340036 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 872/1708] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:57.340036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e4a55-b535-4b49-832e-ec721820ab04 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:57.340036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-853e4a55-b535-4b49-832e-ec721820ab04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:57.363566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853e4a55-b535-4b49-832e-ec721820ab04 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:57.378809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47251fd5-858e-45e8-97b9-9e80cdb39d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:57.381518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47251fd5-858e-45e8-97b9-9e80cdb39d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:57.382282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47251fd5-858e-45e8-97b9-9e80cdb39d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:57.382582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47251fd5-858e-45e8-97b9-9e80cdb39d78 admin admin] PATH_INFO: `/v3/users/e63ff1a4d0f349e09f23613f4d94c280` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:57.383852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47251fd5-858e-45e8-97b9-9e80cdb39d78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e63ff1a4d0f349e09f23613f4d94c280)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:57.385799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47251fd5-858e-45e8-97b9-9e80cdb39d78 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:57.410391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb5ff84-1f2a-47ee-a43e-c105526c2683 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:57.414788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72b9ba-53b6-44a9-a237-3eec8c330b97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:57.415451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72b9ba-53b6-44a9-a237-3eec8c330b97 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:57.415733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72b9ba-53b6-44a9-a237-3eec8c330b97 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:57.419027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72b9ba-53b6-44a9-a237-3eec8c330b97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:57.419027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6d72b9ba-53b6-44a9-a237-3eec8c330b97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:57.431563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72b9ba-53b6-44a9-a237-3eec8c330b97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:57.432740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72b9ba-53b6-44a9-a237-3eec8c330b97 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:57.446205 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 837/1709] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:57.448243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e9e3e-98ce-4927-ac67-4185e9a5f3fb None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:57.448548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e9e3e-98ce-4927-ac67-4185e9a5f3fb None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:57.448830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e9e3e-98ce-4927-ac67-4185e9a5f3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:57.519920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e86e9e3e-98ce-4927-ac67-4185e9a5f3fb None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:57.521042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e86e9e3e-98ce-4927-ac67-4185e9a5f3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:57.569335 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 838/1710] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:57.569335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e9e3e-98ce-4927-ac67-4185e9a5f3fb None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:57.581628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e9e3e-98ce-4927-ac67-4185e9a5f3fb None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:57.585808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5a2fa-0cf1-40b5-b8ad-dafb0ca382ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:57.586704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5a2fa-0cf1-40b5-b8ad-dafb0ca382ad service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:57.587137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5a2fa-0cf1-40b5-b8ad-dafb0ca382ad service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:57.587571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5a2fa-0cf1-40b5-b8ad-dafb0ca382ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:57.597939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba5a2fa-0cf1-40b5-b8ad-dafb0ca382ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:57.600707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba5a2fa-0cf1-40b5-b8ad-dafb0ca382ad service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:57.613942 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 839/1711] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:57.615043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5a2fa-0cf1-40b5-b8ad-dafb0ca382ad service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:57.617156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8ba5a2fa-0cf1-40b5-b8ad-dafb0ca382ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:57.619390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47251fd5-858e-45e8-97b9-9e80cdb39d78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e63ff1a4d0f349e09f23613f4d94c280'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:57.637793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47251fd5-858e-45e8-97b9-9e80cdb39d78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e63ff1a4d0f349e09f23613f4d94c280'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:57.641273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47251fd5-858e-45e8-97b9-9e80cdb39d78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e63ff1a4d0f349e09f23613f4d94c280'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:57.642222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47251fd5-858e-45e8-97b9-9e80cdb39d78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e63ff1a4d0f349e09f23613f4d94c280'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:57.667854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5a2fa-0cf1-40b5-b8ad-dafb0ca382ad service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:57.690595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13412b62-8a71-4a0b-953f-76786f30a64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:57.693823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13412b62-8a71-4a0b-953f-76786f30a64b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:57.693823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13412b62-8a71-4a0b-953f-76786f30a64b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:57.696497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13412b62-8a71-4a0b-953f-76786f30a64b admin admin] PATH_INFO: `/v3/users/8d1a844b3b0e4936af8a7ed0179c4786` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:57.701072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13412b62-8a71-4a0b-953f-76786f30a64b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d1a844b3b0e4936af8a7ed0179c4786)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:57.707348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13412b62-8a71-4a0b-953f-76786f30a64b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:57.721710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47251fd5-858e-45e8-97b9-9e80cdb39d78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e63ff1a4d0f349e09f23613f4d94c280'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:57.762625 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 873/1712] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:02:57 2022] DELETE /identity/v3/users/e63ff1a4d0f349e09f23613f4d94c280 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:57.767686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bccf0-44b9-4bac-81fc-dd55dcdbf818 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:57.769406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bccf0-44b9-4bac-81fc-dd55dcdbf818 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:57.769911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0bccf0-44b9-4bac-81fc-dd55dcdbf818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:57.824809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4f0bccf0-44b9-4bac-81fc-dd55dcdbf818 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:57.831535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4f0bccf0-44b9-4bac-81fc-dd55dcdbf818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:57.981285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-13412b62-8a71-4a0b-953f-76786f30a64b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d1a844b3b0e4936af8a7ed0179c4786'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:57.982788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-13412b62-8a71-4a0b-953f-76786f30a64b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d1a844b3b0e4936af8a7ed0179c4786'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:57.987081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:58.007770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-13412b62-8a71-4a0b-953f-76786f30a64b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d1a844b3b0e4936af8a7ed0179c4786'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:58.008241 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 874/1713] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:58.010469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bccf0-44b9-4bac-81fc-dd55dcdbf818 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:58.024318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0bccf0-44b9-4bac-81fc-dd55dcdbf818 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:58.029597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f22dd7-7229-4bd2-a99f-e54187bca0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:58.030379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f22dd7-7229-4bd2-a99f-e54187bca0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:58.030732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f22dd7-7229-4bd2-a99f-e54187bca0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:58.031038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f22dd7-7229-4bd2-a99f-e54187bca0a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:58.031771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f22dd7-7229-4bd2-a99f-e54187bca0a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:02:58.032233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f22dd7-7229-4bd2-a99f-e54187bca0a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:58.034048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f22dd7-7229-4bd2-a99f-e54187bca0a3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:58.053981 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 875/1714] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:02:58.053981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f22dd7-7229-4bd2-a99f-e54187bca0a3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:58.064805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f22dd7-7229-4bd2-a99f-e54187bca0a3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:58.072963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bdd53-3c17-4624-85ac-3d3ef0f9b3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:58.073791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bdd53-3c17-4624-85ac-3d3ef0f9b3d0 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:58.074194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bdd53-3c17-4624-85ac-3d3ef0f9b3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:58.074553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bdd53-3c17-4624-85ac-3d3ef0f9b3d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:58.075393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872bdd53-3c17-4624-85ac-3d3ef0f9b3d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:58.081589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872bdd53-3c17-4624-85ac-3d3ef0f9b3d0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:58.143491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-13412b62-8a71-4a0b-953f-76786f30a64b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d1a844b3b0e4936af8a7ed0179c4786'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:58.147462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-13412b62-8a71-4a0b-953f-76786f30a64b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d1a844b3b0e4936af8a7ed0179c4786'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:58.171527 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 840/1715] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:02:57 2022] DELETE /identity/v3/users/8d1a844b3b0e4936af8a7ed0179c4786 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:58.176905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9368e9b-d3ce-4e52-b39f-035e9a696224 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:58.177426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9368e9b-d3ce-4e52-b39f-035e9a696224 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:58.178033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9368e9b-d3ce-4e52-b39f-035e9a696224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:58.239407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b9368e9b-d3ce-4e52-b39f-035e9a696224 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:58.242680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b9368e9b-d3ce-4e52-b39f-035e9a696224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:58.256363 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 876/1716] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:02:58 2022] POST /identity/v3/projects => generated 395 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:58.256363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb6826-38e5-493f-8725-bead8db21bc4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:58.256363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb6826-38e5-493f-8725-bead8db21bc4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:58.256363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afb6826-38e5-493f-8725-bead8db21bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:58.285783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:58.285783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3afb6826-38e5-493f-8725-bead8db21bc4 None None] MFA Rules not processed for user `89ed91b9d56241d99b6df99ede3fb446`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:58.287548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3afb6826-38e5-493f-8725-bead8db21bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:58.361869 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 841/1717] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:58.367556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9368e9b-d3ce-4e52-b39f-035e9a696224 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:58.370978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b9368e9b-d3ce-4e52-b39f-035e9a696224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:58.390970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9368e9b-d3ce-4e52-b39f-035e9a696224 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:58.394554 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 877/1718] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:58.396060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb6826-38e5-493f-8725-bead8db21bc4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:58.419182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3afb6826-38e5-493f-8725-bead8db21bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:58.489525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514306e1-e5a4-4951-a4ba-af58cd64668a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:58.490455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514306e1-e5a4-4951-a4ba-af58cd64668a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:58.490805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514306e1-e5a4-4951-a4ba-af58cd64668a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:58.499001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514306e1-e5a4-4951-a4ba-af58cd64668a admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:58.499001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514306e1-e5a4-4951-a4ba-af58cd64668a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:58.499001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514306e1-e5a4-4951-a4ba-af58cd64668a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:58.550177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afb6826-38e5-493f-8725-bead8db21bc4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:58.559402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83310937-161b-4cae-9938-c5d769685074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:58.559402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83310937-161b-4cae-9938-c5d769685074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:58.559402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83310937-161b-4cae-9938-c5d769685074 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:58.559402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83310937-161b-4cae-9938-c5d769685074 admin admin] PATH_INFO: `/v3/projects/2588d52f501b4da8bd7200289912191d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:58.561186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83310937-161b-4cae-9938-c5d769685074 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2588d52f501b4da8bd7200289912191d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:58.563705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83310937-161b-4cae-9938-c5d769685074 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:58.652028 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 842/1719] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:02:58 2022] POST /identity/v3/users => generated 298 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:02:58.653087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514306e1-e5a4-4951-a4ba-af58cd64668a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:58.654853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-514306e1-e5a4-4951-a4ba-af58cd64668a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:58.691213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-83310937-161b-4cae-9938-c5d769685074 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2588d52f501b4da8bd7200289912191d', 'request_id': 'req-83310937-161b-4cae-9938-c5d769685074'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:58.711402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-83310937-161b-4cae-9938-c5d769685074 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2588d52f501b4da8bd7200289912191d', 'request_id': 'req-83310937-161b-4cae-9938-c5d769685074'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:58.743958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514306e1-e5a4-4951-a4ba-af58cd64668a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:58.773303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-83310937-161b-4cae-9938-c5d769685074 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2588d52f501b4da8bd7200289912191d', 'request_id': 'req-83310937-161b-4cae-9938-c5d769685074'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:58.807226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-83310937-161b-4cae-9938-c5d769685074 admin admin] The token cache is being invalidate because project 2588d52f501b4da8bd7200289912191d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:02:58.807703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-83310937-161b-4cae-9938-c5d769685074 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:58.808539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-83310937-161b-4cae-9938-c5d769685074 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:58.811148 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 878/1720] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:02:58 2022] DELETE /identity/v3/projects/2588d52f501b4da8bd7200289912191d => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:58.812199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83310937-161b-4cae-9938-c5d769685074 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:58.814058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-83310937-161b-4cae-9938-c5d769685074 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:58.828708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a4917-4f9a-4e6a-adab-4c05726d9852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:58.829559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a4917-4f9a-4e6a-adab-4c05726d9852 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:58.829981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a4917-4f9a-4e6a-adab-4c05726d9852 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:58.830469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a4917-4f9a-4e6a-adab-4c05726d9852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:58.832457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0c4a4917-4f9a-4e6a-adab-4c05726d9852 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:58.915958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a4917-4f9a-4e6a-adab-4c05726d9852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:58.917728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a4917-4f9a-4e6a-adab-4c05726d9852 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:58.921348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83310937-161b-4cae-9938-c5d769685074 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:58.928781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac80ac2-2579-428f-bc87-a8685bd7d734 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:58.929597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac80ac2-2579-428f-bc87-a8685bd7d734 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:58.930201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac80ac2-2579-428f-bc87-a8685bd7d734 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:58.930376 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 843/1721] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:58.930508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac80ac2-2579-428f-bc87-a8685bd7d734 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:58.931464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a4917-4f9a-4e6a-adab-4c05726d9852 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:58.932695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2ac80ac2-2579-428f-bc87-a8685bd7d734 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:58.932968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0c4a4917-4f9a-4e6a-adab-4c05726d9852 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:58.948609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a4917-4f9a-4e6a-adab-4c05726d9852 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:58.954150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d281f2-a7e5-4c6d-806c-26b65b7bad4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:58.955068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d281f2-a7e5-4c6d-806c-26b65b7bad4c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:58.956218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d281f2-a7e5-4c6d-806c-26b65b7bad4c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:58.956987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d281f2-a7e5-4c6d-806c-26b65b7bad4c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:58.958070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d281f2-a7e5-4c6d-806c-26b65b7bad4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:58.965710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d281f2-a7e5-4c6d-806c-26b65b7bad4c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:58.977101 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 844/1722] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:58.977880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d281f2-a7e5-4c6d-806c-26b65b7bad4c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:58.986416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-57d281f2-a7e5-4c6d-806c-26b65b7bad4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:59.005972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d281f2-a7e5-4c6d-806c-26b65b7bad4c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:59.012259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200cd67-f654-4a64-9351-653c94d52aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:59.013385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200cd67-f654-4a64-9351-653c94d52aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:59.013875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200cd67-f654-4a64-9351-653c94d52aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:59.014347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0200cd67-f654-4a64-9351-653c94d52aaa admin admin] PATH_INFO: `/v3/projects/0d723d6e08fa41c89a20c1fb568c5741` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:59.015832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0200cd67-f654-4a64-9351-653c94d52aaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d723d6e08fa41c89a20c1fb568c5741)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:59.018098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0200cd67-f654-4a64-9351-653c94d52aaa admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:59.036729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac80ac2-2579-428f-bc87-a8685bd7d734 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:59.039448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac80ac2-2579-428f-bc87-a8685bd7d734 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:59.051065 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 879/1723] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:02:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:02:59.052380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac80ac2-2579-428f-bc87-a8685bd7d734 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:59.054525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2ac80ac2-2579-428f-bc87-a8685bd7d734 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:59.079575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac80ac2-2579-428f-bc87-a8685bd7d734 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:59.090271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a94f3-41c5-4f9b-8c97-2558fa87e92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:59.091514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a94f3-41c5-4f9b-8c97-2558fa87e92b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:59.092525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a94f3-41c5-4f9b-8c97-2558fa87e92b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:59.100080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a94f3-41c5-4f9b-8c97-2558fa87e92b admin admin] PATH_INFO: `/v3/users/b9c8dbd5526b46a2ba24b0db5ca3571e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:59.100080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a94f3-41c5-4f9b-8c97-2558fa87e92b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9c8dbd5526b46a2ba24b0db5ca3571e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:59.100080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a94f3-41c5-4f9b-8c97-2558fa87e92b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:59.166443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0200cd67-f654-4a64-9351-653c94d52aaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d723d6e08fa41c89a20c1fb568c5741', 'request_id': 'req-0200cd67-f654-4a64-9351-653c94d52aaa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:59.172343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0200cd67-f654-4a64-9351-653c94d52aaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d723d6e08fa41c89a20c1fb568c5741', 'request_id': 'req-0200cd67-f654-4a64-9351-653c94d52aaa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:59.309103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0200cd67-f654-4a64-9351-653c94d52aaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d723d6e08fa41c89a20c1fb568c5741', 'request_id': 'req-0200cd67-f654-4a64-9351-653c94d52aaa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:59.337256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7d1a94f3-41c5-4f9b-8c97-2558fa87e92b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9c8dbd5526b46a2ba24b0db5ca3571e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:59.342084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0200cd67-f654-4a64-9351-653c94d52aaa admin admin] The token cache is being invalidate because project 0d723d6e08fa41c89a20c1fb568c5741 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:02:59.342375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0200cd67-f654-4a64-9351-653c94d52aaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:59.343154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0200cd67-f654-4a64-9351-653c94d52aaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:59.344886 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 845/1724] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:02:58 2022] DELETE /identity/v3/projects/0d723d6e08fa41c89a20c1fb568c5741 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:59.345692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200cd67-f654-4a64-9351-653c94d52aaa admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:59.347189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0200cd67-f654-4a64-9351-653c94d52aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:59.354275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7d1a94f3-41c5-4f9b-8c97-2558fa87e92b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9c8dbd5526b46a2ba24b0db5ca3571e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:59.355016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7d1a94f3-41c5-4f9b-8c97-2558fa87e92b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9c8dbd5526b46a2ba24b0db5ca3571e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:59.355753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7d1a94f3-41c5-4f9b-8c97-2558fa87e92b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9c8dbd5526b46a2ba24b0db5ca3571e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:59.423221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7d1a94f3-41c5-4f9b-8c97-2558fa87e92b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9c8dbd5526b46a2ba24b0db5ca3571e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:59.430862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0200cd67-f654-4a64-9351-653c94d52aaa admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:59.441445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b8e53-5fc2-4be0-985a-d5c178d02fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:59.443353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b8e53-5fc2-4be0-985a-d5c178d02fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:59.444091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b8e53-5fc2-4be0-985a-d5c178d02fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:59.444091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b8e53-5fc2-4be0-985a-d5c178d02fdb admin admin] PATH_INFO: `/v3/projects/d257d738741646539f55b9dab71b33bf` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:59.448131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b8e53-5fc2-4be0-985a-d5c178d02fdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d257d738741646539f55b9dab71b33bf)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:59.453285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b8e53-5fc2-4be0-985a-d5c178d02fdb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:59.463221 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 880/1725] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:02:59 2022] DELETE /identity/v3/users/b9c8dbd5526b46a2ba24b0db5ca3571e => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:59.463221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a94f3-41c5-4f9b-8c97-2558fa87e92b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:59.470521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7d1a94f3-41c5-4f9b-8c97-2558fa87e92b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:59.591647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-299b8e53-5fc2-4be0-985a-d5c178d02fdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd257d738741646539f55b9dab71b33bf', 'request_id': 'req-299b8e53-5fc2-4be0-985a-d5c178d02fdb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:59.598579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a94f3-41c5-4f9b-8c97-2558fa87e92b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:59.607460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-299b8e53-5fc2-4be0-985a-d5c178d02fdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd257d738741646539f55b9dab71b33bf', 'request_id': 'req-299b8e53-5fc2-4be0-985a-d5c178d02fdb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:59.617645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:59.669247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-299b8e53-5fc2-4be0-985a-d5c178d02fdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd257d738741646539f55b9dab71b33bf', 'request_id': 'req-299b8e53-5fc2-4be0-985a-d5c178d02fdb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:59.687139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-299b8e53-5fc2-4be0-985a-d5c178d02fdb admin admin] The token cache is being invalidate because project d257d738741646539f55b9dab71b33bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:02:59.687440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-299b8e53-5fc2-4be0-985a-d5c178d02fdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:59.689336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-299b8e53-5fc2-4be0-985a-d5c178d02fdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:59.691469 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 846/1726] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:02:59 2022] DELETE /identity/v3/projects/d257d738741646539f55b9dab71b33bf => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:02:59.692785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a13b1-f673-47c2-bd3e-183ac4287e3e None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:59.693075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a13b1-f673-47c2-bd3e-183ac4287e3e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:59.694101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228a13b1-f673-47c2-bd3e-183ac4287e3e None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:59.696845 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 847/1727] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:02:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:02:59.699562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fddfff-a9e4-4584-bb2b-f72eebfb951a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:59.700248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fddfff-a9e4-4584-bb2b-f72eebfb951a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:59.700529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fddfff-a9e4-4584-bb2b-f72eebfb951a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:59.700814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fddfff-a9e4-4584-bb2b-f72eebfb951a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:59.708053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fddfff-a9e4-4584-bb2b-f72eebfb951a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:59.708053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fddfff-a9e4-4584-bb2b-f72eebfb951a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:59.708238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7b072-25ec-4c9c-98bf-2d4f262684ea None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:59.708238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7b072-25ec-4c9c-98bf-2d4f262684ea None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:59.708238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7b072-25ec-4c9c-98bf-2d4f262684ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:59.732108 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 881/1728] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:02:59 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:02:59.733197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fddfff-a9e4-4584-bb2b-f72eebfb951a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:59.735168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-82fddfff-a9e4-4584-bb2b-f72eebfb951a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:59.739510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-08f7b072-25ec-4c9c-98bf-2d4f262684ea None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:02:59.741053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-08f7b072-25ec-4c9c-98bf-2d4f262684ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:59.751258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fddfff-a9e4-4584-bb2b-f72eebfb951a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:59.754468 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 848/1729] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:02:59.756292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7b072-25ec-4c9c-98bf-2d4f262684ea None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:02:59.759794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-08f7b072-25ec-4c9c-98bf-2d4f262684ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:02:59.761977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:59.765026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3cb53b-4ff7-43b6-9936-437851f57725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:59.766234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cb53b-4ff7-43b6-9936-437851f57725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:59.767156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cb53b-4ff7-43b6-9936-437851f57725 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:59.767156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3cb53b-4ff7-43b6-9936-437851f57725 admin admin] PATH_INFO: `/v3/projects/abb5b358addb4e769f10ad12da48b59b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:59.768568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3cb53b-4ff7-43b6-9936-437851f57725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abb5b358addb4e769f10ad12da48b59b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:59.770790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3cb53b-4ff7-43b6-9936-437851f57725 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:59.779446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7b072-25ec-4c9c-98bf-2d4f262684ea None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:02:59.784798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f639f-f0a5-4806-90e1-8758bb5255c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:02:59.788418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f639f-f0a5-4806-90e1-8758bb5255c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:02:59.788418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f639f-f0a5-4806-90e1-8758bb5255c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:02:59.788418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f639f-f0a5-4806-90e1-8758bb5255c9 admin admin] PATH_INFO: `/v3/users/4037ef977e9c4d8692cb5d6c56499328` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:02:59.788418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f639f-f0a5-4806-90e1-8758bb5255c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4037ef977e9c4d8692cb5d6c56499328)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:02:59.790601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f639f-f0a5-4806-90e1-8758bb5255c9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:02:59.920096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8e3cb53b-4ff7-43b6-9936-437851f57725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abb5b358addb4e769f10ad12da48b59b', 'request_id': 'req-8e3cb53b-4ff7-43b6-9936-437851f57725'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:59.932676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8e3cb53b-4ff7-43b6-9936-437851f57725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abb5b358addb4e769f10ad12da48b59b', 'request_id': 'req-8e3cb53b-4ff7-43b6-9936-437851f57725'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:02:59.999688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-df2f639f-f0a5-4806-90e1-8758bb5255c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4037ef977e9c4d8692cb5d6c56499328'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:00.000442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-df2f639f-f0a5-4806-90e1-8758bb5255c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4037ef977e9c4d8692cb5d6c56499328'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:00.005786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8e3cb53b-4ff7-43b6-9936-437851f57725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abb5b358addb4e769f10ad12da48b59b', 'request_id': 'req-8e3cb53b-4ff7-43b6-9936-437851f57725'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:00.019239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-df2f639f-f0a5-4806-90e1-8758bb5255c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4037ef977e9c4d8692cb5d6c56499328'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:00.022145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8e3cb53b-4ff7-43b6-9936-437851f57725 admin admin] The token cache is being invalidate because project abb5b358addb4e769f10ad12da48b59b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:00.022419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8e3cb53b-4ff7-43b6-9936-437851f57725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:00.023111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8e3cb53b-4ff7-43b6-9936-437851f57725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:00.024989 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 882/1730] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:02:59 2022] DELETE /identity/v3/projects/abb5b358addb4e769f10ad12da48b59b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:00.026357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33eb3b-fa08-490d-b446-0cdd435fd606 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:00.026649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33eb3b-fa08-490d-b446-0cdd435fd606 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:00.026929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf33eb3b-fa08-490d-b446-0cdd435fd606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:00.051606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cf33eb3b-fa08-490d-b446-0cdd435fd606 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:00.052853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cf33eb3b-fa08-490d-b446-0cdd435fd606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:00.085948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-df2f639f-f0a5-4806-90e1-8758bb5255c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4037ef977e9c4d8692cb5d6c56499328'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:00.086878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-df2f639f-f0a5-4806-90e1-8758bb5255c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4037ef977e9c4d8692cb5d6c56499328'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:00.109686 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 849/1731] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:02:59 2022] DELETE /identity/v3/users/4037ef977e9c4d8692cb5d6c56499328 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:00.110970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f639f-f0a5-4806-90e1-8758bb5255c9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:00.112757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-df2f639f-f0a5-4806-90e1-8758bb5255c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:00.173893 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 883/1732] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:03:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:00.173893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9842a07-c226-4172-b8ba-9f33e325ff1a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:00.174185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9842a07-c226-4172-b8ba-9f33e325ff1a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:00.174572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9842a07-c226-4172-b8ba-9f33e325ff1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:00.200517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a9842a07-c226-4172-b8ba-9f33e325ff1a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:00.202673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a9842a07-c226-4172-b8ba-9f33e325ff1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:00.218337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f639f-f0a5-4806-90e1-8758bb5255c9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:00.226801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4660b7-5dd5-47f1-9afc-ebc887a28800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:00.228140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4660b7-5dd5-47f1-9afc-ebc887a28800 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:00.228667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4660b7-5dd5-47f1-9afc-ebc887a28800 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:00.229140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4660b7-5dd5-47f1-9afc-ebc887a28800 admin admin] PATH_INFO: `/v3/projects/0d902454e7bd44339240a17aef7d1335/users/13e536776b8f4bd99fbf6a7b5f02fc79/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:00.266854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4660b7-5dd5-47f1-9afc-ebc887a28800 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d902454e7bd44339240a17aef7d1335, user_id=13e536776b8f4bd99fbf6a7b5f02fc79, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:00.269600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4660b7-5dd5-47f1-9afc-ebc887a28800 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:00.289014 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 884/1733] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:00.290008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9842a07-c226-4172-b8ba-9f33e325ff1a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:00.291585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a9842a07-c226-4172-b8ba-9f33e325ff1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:00.304894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9842a07-c226-4172-b8ba-9f33e325ff1a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:00.309783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef32ca-278e-4407-a9ca-3d70c9e267b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:00.310719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef32ca-278e-4407-a9ca-3d70c9e267b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:00.311000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef32ca-278e-4407-a9ca-3d70c9e267b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:00.311276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef32ca-278e-4407-a9ca-3d70c9e267b0 admin admin] PATH_INFO: `/v3/users/d2cee368d9a14c8bb3e665eed1fdeb7e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:00.312495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef32ca-278e-4407-a9ca-3d70c9e267b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2cee368d9a14c8bb3e665eed1fdeb7e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:00.314519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef32ca-278e-4407-a9ca-3d70c9e267b0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:00.362136 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 850/1734] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:00 2022] PUT /identity/v3/projects/0d902454e7bd44339240a17aef7d1335/users/13e536776b8f4bd99fbf6a7b5f02fc79/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:00.363145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4660b7-5dd5-47f1-9afc-ebc887a28800 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:00.372770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3a4660b7-5dd5-47f1-9afc-ebc887a28800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:00.492320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4660b7-5dd5-47f1-9afc-ebc887a28800 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:00.498134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8f61a-9b45-44c6-a4c0-decf4982228e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:00.498810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8f61a-9b45-44c6-a4c0-decf4982228e service glance] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:00.499097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8f61a-9b45-44c6-a4c0-decf4982228e service glance] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:00.499376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f8f61a-9b45-44c6-a4c0-decf4982228e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:00.526566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-31ef32ca-278e-4407-a9ca-3d70c9e267b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2cee368d9a14c8bb3e665eed1fdeb7e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:00.543165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-31ef32ca-278e-4407-a9ca-3d70c9e267b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2cee368d9a14c8bb3e665eed1fdeb7e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:00.555798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-31ef32ca-278e-4407-a9ca-3d70c9e267b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2cee368d9a14c8bb3e665eed1fdeb7e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:00.558800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-31ef32ca-278e-4407-a9ca-3d70c9e267b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2cee368d9a14c8bb3e665eed1fdeb7e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:00.561312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:00.628357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8f61a-9b45-44c6-a4c0-decf4982228e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:00.630239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f8f61a-9b45-44c6-a4c0-decf4982228e service glance] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:00.651324 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 851/1735] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:00.651324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8f61a-9b45-44c6-a4c0-decf4982228e service glance] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:00.651740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f7f8f61a-9b45-44c6-a4c0-decf4982228e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:00.680865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-31ef32ca-278e-4407-a9ca-3d70c9e267b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2cee368d9a14c8bb3e665eed1fdeb7e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:00.692844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:00.744644 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 885/1736] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:00 2022] DELETE /identity/v3/users/d2cee368d9a14c8bb3e665eed1fdeb7e => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:00.747127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef32ca-278e-4407-a9ca-3d70c9e267b0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:00.749931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-31ef32ca-278e-4407-a9ca-3d70c9e267b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:00.816230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f8f61a-9b45-44c6-a4c0-decf4982228e service glance] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:00.887888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef32ca-278e-4407-a9ca-3d70c9e267b0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:00.903215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e6c8f7-0a52-4820-bf48-918ad6749bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:00.903600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6c8f7-0a52-4820-bf48-918ad6749bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:00.903890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6c8f7-0a52-4820-bf48-918ad6749bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:00.904175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e6c8f7-0a52-4820-bf48-918ad6749bc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:00.904937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6c8f7-0a52-4820-bf48-918ad6749bc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:03:00.905252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6c8f7-0a52-4820-bf48-918ad6749bc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:00.907353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e6c8f7-0a52-4820-bf48-918ad6749bc3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:00.942714 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 886/1737] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:00.944214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b3169-be79-4e39-9310-fe765b8d05a1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:00.944518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b3169-be79-4e39-9310-fe765b8d05a1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:00.944810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b3169-be79-4e39-9310-fe765b8d05a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:00.946445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca502e-8b05-4fb0-a697-18a5933b011b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:00.947308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca502e-8b05-4fb0-a697-18a5933b011b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:00.954076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca502e-8b05-4fb0-a697-18a5933b011b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:00.954076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca502e-8b05-4fb0-a697-18a5933b011b admin admin] PATH_INFO: `/v3/projects/4c27b736741a4c6f9c7a8fc2bd133a95` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:00.954076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca502e-8b05-4fb0-a697-18a5933b011b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c27b736741a4c6f9c7a8fc2bd133a95)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:00.954076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca502e-8b05-4fb0-a697-18a5933b011b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:01.011790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-936b3169-be79-4e39-9310-fe765b8d05a1 None None] MFA Rules not processed for user `13e536776b8f4bd99fbf6a7b5f02fc79`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:01.011790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-936b3169-be79-4e39-9310-fe765b8d05a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:01.129676 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 887/1738] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:01.129676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b3169-be79-4e39-9310-fe765b8d05a1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:01.129676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-936b3169-be79-4e39-9310-fe765b8d05a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:01.137882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6bca502e-8b05-4fb0-a697-18a5933b011b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c27b736741a4c6f9c7a8fc2bd133a95', 'request_id': 'req-6bca502e-8b05-4fb0-a697-18a5933b011b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:01.146372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6bca502e-8b05-4fb0-a697-18a5933b011b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c27b736741a4c6f9c7a8fc2bd133a95', 'request_id': 'req-6bca502e-8b05-4fb0-a697-18a5933b011b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:01.157189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b3169-be79-4e39-9310-fe765b8d05a1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:01.232977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e4e6aa-a6fd-4c2f-a379-045bf4f4bff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:01.234106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4e6aa-a6fd-4c2f-a379-045bf4f4bff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:01.234398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4e6aa-a6fd-4c2f-a379-045bf4f4bff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:01.234834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e4e6aa-a6fd-4c2f-a379-045bf4f4bff4 admin admin] PATH_INFO: `/v3/projects/003ac163682d47fbb4ec29e4afeb1963` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:01.234980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6bca502e-8b05-4fb0-a697-18a5933b011b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c27b736741a4c6f9c7a8fc2bd133a95', 'request_id': 'req-6bca502e-8b05-4fb0-a697-18a5933b011b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:01.236293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4e6aa-a6fd-4c2f-a379-045bf4f4bff4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=003ac163682d47fbb4ec29e4afeb1963)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:01.238402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e4e6aa-a6fd-4c2f-a379-045bf4f4bff4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:01.254377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6bca502e-8b05-4fb0-a697-18a5933b011b admin admin] The token cache is being invalidate because project 4c27b736741a4c6f9c7a8fc2bd133a95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:01.254736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6bca502e-8b05-4fb0-a697-18a5933b011b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:01.255818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6bca502e-8b05-4fb0-a697-18a5933b011b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:01.259559 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 852/1739] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:00 2022] DELETE /identity/v3/projects/4c27b736741a4c6f9c7a8fc2bd133a95 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:01.260333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca502e-8b05-4fb0-a697-18a5933b011b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:01.262884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6bca502e-8b05-4fb0-a697-18a5933b011b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:01.278031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca502e-8b05-4fb0-a697-18a5933b011b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:01.284117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733460f-e7b9-4c82-81e0-2890ce06f87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:01.284892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733460f-e7b9-4c82-81e0-2890ce06f87b admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:01.285894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733460f-e7b9-4c82-81e0-2890ce06f87b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:01.285894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733460f-e7b9-4c82-81e0-2890ce06f87b admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:01.286262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733460f-e7b9-4c82-81e0-2890ce06f87b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:01.288240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733460f-e7b9-4c82-81e0-2890ce06f87b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:01.402284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-16e4e6aa-a6fd-4c2f-a379-045bf4f4bff4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '003ac163682d47fbb4ec29e4afeb1963', 'request_id': 'req-16e4e6aa-a6fd-4c2f-a379-045bf4f4bff4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:01.423329 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 853/1740] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:01 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:01.424338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733460f-e7b9-4c82-81e0-2890ce06f87b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:01.427051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5733460f-e7b9-4c82-81e0-2890ce06f87b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:01.434967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-16e4e6aa-a6fd-4c2f-a379-045bf4f4bff4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '003ac163682d47fbb4ec29e4afeb1963', 'request_id': 'req-16e4e6aa-a6fd-4c2f-a379-045bf4f4bff4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:01.515264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-16e4e6aa-a6fd-4c2f-a379-045bf4f4bff4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '003ac163682d47fbb4ec29e4afeb1963', 'request_id': 'req-16e4e6aa-a6fd-4c2f-a379-045bf4f4bff4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:01.534517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-16e4e6aa-a6fd-4c2f-a379-045bf4f4bff4 admin admin] The token cache is being invalidate because project 003ac163682d47fbb4ec29e4afeb1963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:01.535124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-16e4e6aa-a6fd-4c2f-a379-045bf4f4bff4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:01.536411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-16e4e6aa-a6fd-4c2f-a379-045bf4f4bff4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:01.540593 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 888/1741] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:01 2022] DELETE /identity/v3/projects/003ac163682d47fbb4ec29e4afeb1963 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:01.543010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0035a-6eea-4ac8-8183-28cc4e2c9b4c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:01.543978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0035a-6eea-4ac8-8183-28cc4e2c9b4c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:01.544925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0035a-6eea-4ac8-8183-28cc4e2c9b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:01.590688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733460f-e7b9-4c82-81e0-2890ce06f87b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:01.597192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c755f20-cb4d-448c-a5b1-e80af93e7036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:01.597963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-57f0035a-6eea-4ac8-8183-28cc4e2c9b4c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:01.598079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c755f20-cb4d-448c-a5b1-e80af93e7036 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:01.598251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c755f20-cb4d-448c-a5b1-e80af93e7036 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:01.598629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c755f20-cb4d-448c-a5b1-e80af93e7036 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:01.599534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c755f20-cb4d-448c-a5b1-e80af93e7036 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:01.600092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-57f0035a-6eea-4ac8-8183-28cc4e2c9b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:01.606381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c755f20-cb4d-448c-a5b1-e80af93e7036 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:01.613565 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 889/1742] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:01.614586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0035a-6eea-4ac8-8183-28cc4e2c9b4c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:01.624664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-57f0035a-6eea-4ac8-8183-28cc4e2c9b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:01.637225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f0035a-6eea-4ac8-8183-28cc4e2c9b4c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:01.642033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ac017-5c5b-4cd2-8624-a61721db2175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:01.643086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ac017-5c5b-4cd2-8624-a61721db2175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:01.643445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ac017-5c5b-4cd2-8624-a61721db2175 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:01.643810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ac017-5c5b-4cd2-8624-a61721db2175 admin admin] PATH_INFO: `/v3/projects/8d62542caece43d0a138849e9f70e55c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:01.645093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ac017-5c5b-4cd2-8624-a61721db2175 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d62542caece43d0a138849e9f70e55c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:01.647698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3ac017-5c5b-4cd2-8624-a61721db2175 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:01.700865 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 854/1743] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:01 2022] POST /identity/v3/projects => generated 395 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:01.701671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c755f20-cb4d-448c-a5b1-e80af93e7036 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:01.706037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0c755f20-cb4d-448c-a5b1-e80af93e7036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:01.766423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ac3ac017-5c5b-4cd2-8624-a61721db2175 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d62542caece43d0a138849e9f70e55c', 'request_id': 'req-ac3ac017-5c5b-4cd2-8624-a61721db2175'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:01.777739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ac3ac017-5c5b-4cd2-8624-a61721db2175 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d62542caece43d0a138849e9f70e55c', 'request_id': 'req-ac3ac017-5c5b-4cd2-8624-a61721db2175'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:01.778965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:01.824483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c755f20-cb4d-448c-a5b1-e80af93e7036 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:01.840674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ac3ac017-5c5b-4cd2-8624-a61721db2175 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d62542caece43d0a138849e9f70e55c', 'request_id': 'req-ac3ac017-5c5b-4cd2-8624-a61721db2175'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:01.861720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ac3ac017-5c5b-4cd2-8624-a61721db2175 admin admin] The token cache is being invalidate because project 8d62542caece43d0a138849e9f70e55c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:01.862041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ac3ac017-5c5b-4cd2-8624-a61721db2175 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:01.868125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ac3ac017-5c5b-4cd2-8624-a61721db2175 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:01.873800 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 890/1744] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:01 2022] DELETE /identity/v3/projects/8d62542caece43d0a138849e9f70e55c => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:01.873800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99aa16-d110-4d2d-a1bd-1e90ebcee11b None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:01.873800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99aa16-d110-4d2d-a1bd-1e90ebcee11b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:01.873800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99aa16-d110-4d2d-a1bd-1e90ebcee11b None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:01.879814 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 891/1745] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:03:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:01.881751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7e41b-9c2c-422f-b93d-e8c198968c51 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:01.881931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7e41b-9c2c-422f-b93d-e8c198968c51 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:01.882179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7e41b-9c2c-422f-b93d-e8c198968c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:01.916773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7bd7e41b-9c2c-422f-b93d-e8c198968c51 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:01.918422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7bd7e41b-9c2c-422f-b93d-e8c198968c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:01.949128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc8c75-7973-4ce6-b3ee-faf89096d966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:01.950856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc8c75-7973-4ce6-b3ee-faf89096d966 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:01.952250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc8c75-7973-4ce6-b3ee-faf89096d966 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:01.952812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc8c75-7973-4ce6-b3ee-faf89096d966 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:01.953701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc8c75-7973-4ce6-b3ee-faf89096d966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:01.956387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc8c75-7973-4ce6-b3ee-faf89096d966 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:01.972272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:02.055689 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 855/1746] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:01 2022] POST /identity/v3/users => generated 294 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:02.055689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc8c75-7973-4ce6-b3ee-faf89096d966 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:02.063045 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 892/1747] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:02.064712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979c881-d873-4d1a-90c9-563cbf43f90e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.065004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979c881-d873-4d1a-90c9-563cbf43f90e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.065282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979c881-d873-4d1a-90c9-563cbf43f90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.067724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-86bc8c75-7973-4ce6-b3ee-faf89096d966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:02.109551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b979c881-d873-4d1a-90c9-563cbf43f90e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:02.111200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b979c881-d873-4d1a-90c9-563cbf43f90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:02.135222 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 893/1748] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:02.136092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979c881-d873-4d1a-90c9-563cbf43f90e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:02.138159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b979c881-d873-4d1a-90c9-563cbf43f90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:02.154043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979c881-d873-4d1a-90c9-563cbf43f90e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:02.160108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15d0a1-6adc-4c02-8c27-674a40ba3e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:02.160347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15d0a1-6adc-4c02-8c27-674a40ba3e6a admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.160638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15d0a1-6adc-4c02-8c27-674a40ba3e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.160916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15d0a1-6adc-4c02-8c27-674a40ba3e6a admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.161659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15d0a1-6adc-4c02-8c27-674a40ba3e6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:02.164169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15d0a1-6adc-4c02-8c27-674a40ba3e6a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:02.169284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc8c75-7973-4ce6-b3ee-faf89096d966 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:02.175438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eabef82-81b4-4216-839f-6b6f87fdb1c3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:02.176240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabef82-81b4-4216-839f-6b6f87fdb1c3 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.178211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabef82-81b4-4216-839f-6b6f87fdb1c3 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.178211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eabef82-81b4-4216-839f-6b6f87fdb1c3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.179685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4eabef82-81b4-4216-839f-6b6f87fdb1c3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:02.195155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabef82-81b4-4216-839f-6b6f87fdb1c3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:02.197146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eabef82-81b4-4216-839f-6b6f87fdb1c3 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:02.208429 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 856/1749] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:02.210703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c05a9-5b6d-47fd-bcda-fc4dc215d4f1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.211114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c05a9-5b6d-47fd-bcda-fc4dc215d4f1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.211484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9c05a9-5b6d-47fd-bcda-fc4dc215d4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.234116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4a9c05a9-5b6d-47fd-bcda-fc4dc215d4f1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:02.235485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4a9c05a9-5b6d-47fd-bcda-fc4dc215d4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:02.246156 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 857/1750] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:02.247658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fc9c1-e8e4-4f5f-ba19-0e810264ef20 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.248022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fc9c1-e8e4-4f5f-ba19-0e810264ef20 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.248369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fc9c1-e8e4-4f5f-ba19-0e810264ef20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.270178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c64fc9c1-e8e4-4f5f-ba19-0e810264ef20 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:02.271453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c64fc9c1-e8e4-4f5f-ba19-0e810264ef20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:02.300033 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 894/1751] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:02 2022] POST /identity/v3/users => generated 304 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:02.301028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15d0a1-6adc-4c02-8c27-674a40ba3e6a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:02.307294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4a15d0a1-6adc-4c02-8c27-674a40ba3e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:02.321358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15d0a1-6adc-4c02-8c27-674a40ba3e6a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:02.327625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cfadb-18df-47fe-95e4-23ed21a1fca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:02.328406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cfadb-18df-47fe-95e4-23ed21a1fca4 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.328753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cfadb-18df-47fe-95e4-23ed21a1fca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.347412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0cfadb-18df-47fe-95e4-23ed21a1fca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.347412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cfadb-18df-47fe-95e4-23ed21a1fca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:02.347412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0cfadb-18df-47fe-95e4-23ed21a1fca4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:02.350765 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 858/1752] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:03:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:02.352128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea4e72-0404-4bee-a850-fdef235781ed None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.352412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea4e72-0404-4bee-a850-fdef235781ed None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.352684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bea4e72-0404-4bee-a850-fdef235781ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.355796 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 895/1753] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:02.356587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cfadb-18df-47fe-95e4-23ed21a1fca4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:02.381363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0cfadb-18df-47fe-95e4-23ed21a1fca4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:02.384006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3bea4e72-0404-4bee-a850-fdef235781ed None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:02.385160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3bea4e72-0404-4bee-a850-fdef235781ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:02.386425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282df81-8c90-4995-8ab7-743d03b27930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:02.387095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282df81-8c90-4995-8ab7-743d03b27930 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.387381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282df81-8c90-4995-8ab7-743d03b27930 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.387664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282df81-8c90-4995-8ab7-743d03b27930 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.388401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282df81-8c90-4995-8ab7-743d03b27930 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:03:02.388716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282df81-8c90-4995-8ab7-743d03b27930 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:02.390456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282df81-8c90-4995-8ab7-743d03b27930 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:02.393987 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 859/1754] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:02.395344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc83921-4e8b-4f14-9e18-84048ccb5f3c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.395631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc83921-4e8b-4f14-9e18-84048ccb5f3c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.395903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc83921-4e8b-4f14-9e18-84048ccb5f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.405033 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 896/1755] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:02.405989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282df81-8c90-4995-8ab7-743d03b27930 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:02.416558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282df81-8c90-4995-8ab7-743d03b27930 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:02.421501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bbb82-b549-445d-a13d-434a7ad83f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:02.422215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bbb82-b549-445d-a13d-434a7ad83f00 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.422503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bbb82-b549-445d-a13d-434a7ad83f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.422785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bbb82-b549-445d-a13d-434a7ad83f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.423529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4bbb82-b549-445d-a13d-434a7ad83f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:02.423939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ebc83921-4e8b-4f14-9e18-84048ccb5f3c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:02.425146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4bbb82-b549-445d-a13d-434a7ad83f00 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:02.425287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ebc83921-4e8b-4f14-9e18-84048ccb5f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:02.436619 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 860/1756] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:02.437856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc83921-4e8b-4f14-9e18-84048ccb5f3c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:02.439593 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 897/1757] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:02.440714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bbb82-b549-445d-a13d-434a7ad83f00 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:02.442663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-df4bbb82-b549-445d-a13d-434a7ad83f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:02.449584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc83921-4e8b-4f14-9e18-84048ccb5f3c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:02.454588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086aabc-6180-4720-8b52-db28071fe33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:02.455236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086aabc-6180-4720-8b52-db28071fe33b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.455518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086aabc-6180-4720-8b52-db28071fe33b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.455793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086aabc-6180-4720-8b52-db28071fe33b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.456527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086aabc-6180-4720-8b52-db28071fe33b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:02.458192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086aabc-6180-4720-8b52-db28071fe33b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:02.470279 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 861/1758] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:02.471580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41780a-dfc5-4dd4-aa0e-63abd30df8a2 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.472016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41780a-dfc5-4dd4-aa0e-63abd30df8a2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.472300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41780a-dfc5-4dd4-aa0e-63abd30df8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.505034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0c41780a-dfc5-4dd4-aa0e-63abd30df8a2 None None] MFA Rules not processed for user `cdeaa660f8ec465bba68096d0ed4b1ae`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:02.508020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0c41780a-dfc5-4dd4-aa0e-63abd30df8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:02.514215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bbb82-b549-445d-a13d-434a7ad83f00 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:02.518880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95fd76-2ce7-4c90-883d-1fdc23d79600 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:02.519540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95fd76-2ce7-4c90-883d-1fdc23d79600 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.519823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95fd76-2ce7-4c90-883d-1fdc23d79600 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.520099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95fd76-2ce7-4c90-883d-1fdc23d79600 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.532680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95fd76-2ce7-4c90-883d-1fdc23d79600 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:02.534284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae95fd76-2ce7-4c90-883d-1fdc23d79600 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:02.545985 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 898/1759] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:02.546813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95fd76-2ce7-4c90-883d-1fdc23d79600 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:02.556905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95fd76-2ce7-4c90-883d-1fdc23d79600 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:02.562862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfc3f7-6c01-4c4f-adff-df320d985066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:02.563527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfc3f7-6c01-4c4f-adff-df320d985066 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.563814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfc3f7-6c01-4c4f-adff-df320d985066 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.564094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfc3f7-6c01-4c4f-adff-df320d985066 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.564842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cfc3f7-6c01-4c4f-adff-df320d985066 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:03:02.565140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cfc3f7-6c01-4c4f-adff-df320d985066 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:02.567196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cfc3f7-6c01-4c4f-adff-df320d985066 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:02.575358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:02.582066 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 899/1760] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:02.582259 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 862/1761] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:02.582259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41780a-dfc5-4dd4-aa0e-63abd30df8a2 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:02.583880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfc3f7-6c01-4c4f-adff-df320d985066 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:02.594301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41780a-dfc5-4dd4-aa0e-63abd30df8a2 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:02.595455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfc3f7-6c01-4c4f-adff-df320d985066 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:02.599447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d31b0-c518-4c0c-a1a6-8fa95ad7a7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:02.600766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d31b0-c518-4c0c-a1a6-8fa95ad7a7ec admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.600961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c56ce5-626e-47b9-9d6b-5374105e109c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:02.601065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d31b0-c518-4c0c-a1a6-8fa95ad7a7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.605707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d31b0-c518-4c0c-a1a6-8fa95ad7a7ec admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.605823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c56ce5-626e-47b9-9d6b-5374105e109c admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.605823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c56ce5-626e-47b9-9d6b-5374105e109c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.606021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d31b0-c518-4c0c-a1a6-8fa95ad7a7ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:03:02.606101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c56ce5-626e-47b9-9d6b-5374105e109c admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.606197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d31b0-c518-4c0c-a1a6-8fa95ad7a7ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:02.606279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c56ce5-626e-47b9-9d6b-5374105e109c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:02.606356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d31b0-c518-4c0c-a1a6-8fa95ad7a7ec admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:02.606441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c56ce5-626e-47b9-9d6b-5374105e109c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:02.618325 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 863/1762] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:03:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:02.619139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d31b0-c518-4c0c-a1a6-8fa95ad7a7ec admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:02.633595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d31b0-c518-4c0c-a1a6-8fa95ad7a7ec admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:02.638317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1c284-36ee-444a-8d33-6380218d223e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:02.639196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1c284-36ee-444a-8d33-6380218d223e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.639481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1c284-36ee-444a-8d33-6380218d223e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.639757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b1c284-36ee-444a-8d33-6380218d223e admin admin] PATH_INFO: `/v3/projects/38bea6ec7e1c4836868191839fd7b889/users/4d52e97d8e154a7986918ed3918cb4e2/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.655571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1c284-36ee-444a-8d33-6380218d223e admin admin] RBAC: Authorizing `identity:create_grant(project_id=38bea6ec7e1c4836868191839fd7b889, user_id=4d52e97d8e154a7986918ed3918cb4e2, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:02.658273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b1c284-36ee-444a-8d33-6380218d223e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:02.688318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:02.694654 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 900/1763] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:02 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:02.695452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c56ce5-626e-47b9-9d6b-5374105e109c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:02.740705 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 864/1764] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:02 2022] PUT /identity/v3/projects/38bea6ec7e1c4836868191839fd7b889/users/4d52e97d8e154a7986918ed3918cb4e2/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:02.740705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1c284-36ee-444a-8d33-6380218d223e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:02.764243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c56ce5-626e-47b9-9d6b-5374105e109c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:02.805752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b1c284-36ee-444a-8d33-6380218d223e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:02.810846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c51e10-c1d1-47f8-87b7-93cda60216ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:02.811519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c51e10-c1d1-47f8-87b7-93cda60216ed admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.811810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c51e10-c1d1-47f8-87b7-93cda60216ed admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.812091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c51e10-c1d1-47f8-87b7-93cda60216ed admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.812800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c51e10-c1d1-47f8-87b7-93cda60216ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:02.814834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c51e10-c1d1-47f8-87b7-93cda60216ed admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:02.831750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ccf94c-cd61-4303-aece-9ccd3bfeb80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:02.832722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccf94c-cd61-4303-aece-9ccd3bfeb80d admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:02.833006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccf94c-cd61-4303-aece-9ccd3bfeb80d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:02.833287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ccf94c-cd61-4303-aece-9ccd3bfeb80d admin admin] PATH_INFO: `/v3/projects/0397fe0f76e24a83aa24ecec35dfb61e/users/ad6f51b7b6834fefa06c58829399de85/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:02.848599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ccf94c-cd61-4303-aece-9ccd3bfeb80d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0397fe0f76e24a83aa24ecec35dfb61e, user_id=ad6f51b7b6834fefa06c58829399de85, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:02.851893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ccf94c-cd61-4303-aece-9ccd3bfeb80d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:02.883402 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 865/1765] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:02 2022] POST /identity/v3/projects => generated 403 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:02.886393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c51e10-c1d1-47f8-87b7-93cda60216ed admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:02.886393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-78c51e10-c1d1-47f8-87b7-93cda60216ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:02.926893 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 901/1766] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:02 2022] PUT /identity/v3/projects/0397fe0f76e24a83aa24ecec35dfb61e/users/ad6f51b7b6834fefa06c58829399de85/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:02.934825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ccf94c-cd61-4303-aece-9ccd3bfeb80d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:02.971382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c51e10-c1d1-47f8-87b7-93cda60216ed admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:03.003027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ccf94c-cd61-4303-aece-9ccd3bfeb80d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:03.008468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cdd06-2b26-44f7-b16d-c5b4b410d2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:03.009152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cdd06-2b26-44f7-b16d-c5b4b410d2e9 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:03.009443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cdd06-2b26-44f7-b16d-c5b4b410d2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:03.009726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cdd06-2b26-44f7-b16d-c5b4b410d2e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:03.010627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991cdd06-2b26-44f7-b16d-c5b4b410d2e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:03.012676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991cdd06-2b26-44f7-b16d-c5b4b410d2e9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:03.041960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def89e09-0106-47e6-b56a-569054168b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:03.042712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def89e09-0106-47e6-b56a-569054168b55 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:03.043072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def89e09-0106-47e6-b56a-569054168b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:03.043429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def89e09-0106-47e6-b56a-569054168b55 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:03.044346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def89e09-0106-47e6-b56a-569054168b55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:03.046421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def89e09-0106-47e6-b56a-569054168b55 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:03.086309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:03.129928 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 902/1767] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:02 2022] POST /identity/v3/projects => generated 391 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:03.129928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cdd06-2b26-44f7-b16d-c5b4b410d2e9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:03.182816 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 866/1768] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:02 2022] POST /identity/v3/projects => generated 403 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:03.182816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def89e09-0106-47e6-b56a-569054168b55 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:03.193898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cdd06-2b26-44f7-b16d-c5b4b410d2e9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:03.248719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def89e09-0106-47e6-b56a-569054168b55 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:03.254427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ab8b6-e59b-43da-ac7e-50517d1715b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:03.255173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ab8b6-e59b-43da-ac7e-50517d1715b3 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:03.255525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ab8b6-e59b-43da-ac7e-50517d1715b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:03.255810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01ab8b6-e59b-43da-ac7e-50517d1715b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:03.261363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ab8b6-e59b-43da-ac7e-50517d1715b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:03.261363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01ab8b6-e59b-43da-ac7e-50517d1715b3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:03.268444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2eabc5-982e-4de8-a6dc-586aae726fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:03.269416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2eabc5-982e-4de8-a6dc-586aae726fcc admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:03.269934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2eabc5-982e-4de8-a6dc-586aae726fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:03.270433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2eabc5-982e-4de8-a6dc-586aae726fcc admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:03.271305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2eabc5-982e-4de8-a6dc-586aae726fcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:03.274414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2eabc5-982e-4de8-a6dc-586aae726fcc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:03.277864 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 867/1769] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:03.279028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ab8b6-e59b-43da-ac7e-50517d1715b3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:03.292121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01ab8b6-e59b-43da-ac7e-50517d1715b3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:03.296860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b74a43-1982-45d7-ae32-558d39cb8588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:03.297535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b74a43-1982-45d7-ae32-558d39cb8588 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:03.297820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b74a43-1982-45d7-ae32-558d39cb8588 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:03.298157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b74a43-1982-45d7-ae32-558d39cb8588 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:03.298858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b74a43-1982-45d7-ae32-558d39cb8588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:03.300794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b74a43-1982-45d7-ae32-558d39cb8588 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:03.303294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:03.363976 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 903/1770] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:03 2022] POST /identity/v3/users => generated 292 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:03.363976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6489bb-5a98-46ec-b0b3-fcd770833a1e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:03.363976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6489bb-5a98-46ec-b0b3-fcd770833a1e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:03.363976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6489bb-5a98-46ec-b0b3-fcd770833a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:03.384948 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 868/1771] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:03 2022] POST /identity/v3/users => generated 302 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:03.384948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b74a43-1982-45d7-ae32-558d39cb8588 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:03.392486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-df6489bb-5a98-46ec-b0b3-fcd770833a1e None None] MFA Rules not processed for user `ad6f51b7b6834fefa06c58829399de85`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:03.393968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-df6489bb-5a98-46ec-b0b3-fcd770833a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:03.395474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b74a43-1982-45d7-ae32-558d39cb8588 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:03.400901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be28b36-6606-4d91-b894-dbff18669787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:03.401566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be28b36-6606-4d91-b894-dbff18669787 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:03.401870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be28b36-6606-4d91-b894-dbff18669787 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:03.402160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be28b36-6606-4d91-b894-dbff18669787 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:03.402849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be28b36-6606-4d91-b894-dbff18669787 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:03.405079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be28b36-6606-4d91-b894-dbff18669787 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:03.467279 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 904/1772] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:03.467279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6489bb-5a98-46ec-b0b3-fcd770833a1e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:03.477302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6489bb-5a98-46ec-b0b3-fcd770833a1e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:03.482507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0bf59-6178-4e7f-9f39-592ad5caa1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:03.483167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0bf59-6178-4e7f-9f39-592ad5caa1f7 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:03.483455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0bf59-6178-4e7f-9f39-592ad5caa1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:03.483794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0bf59-6178-4e7f-9f39-592ad5caa1f7 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:03.484415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b0bf59-6178-4e7f-9f39-592ad5caa1f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:03.486377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b0bf59-6178-4e7f-9f39-592ad5caa1f7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:03.519077 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 869/1773] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:03 2022] POST /identity/v3/users => generated 296 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:03.519077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be28b36-6606-4d91-b894-dbff18669787 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:03.533374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be28b36-6606-4d91-b894-dbff18669787 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:03.539815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e90c0b-f5a2-434c-8da2-909f61c3c20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:03.541038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e90c0b-f5a2-434c-8da2-909f61c3c20c admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:03.541683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e90c0b-f5a2-434c-8da2-909f61c3c20c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:03.543310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e90c0b-f5a2-434c-8da2-909f61c3c20c admin admin] PATH_INFO: `/v3/projects/38bea6ec7e1c4836868191839fd7b889/users/4d52e97d8e154a7986918ed3918cb4e2/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:03.545686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e90c0b-f5a2-434c-8da2-909f61c3c20c admin admin] RBAC: Authorizing `identity:create_grant(project_id=38bea6ec7e1c4836868191839fd7b889, user_id=4d52e97d8e154a7986918ed3918cb4e2, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:03.549184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:03.549797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e90c0b-f5a2-434c-8da2-909f61c3c20c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:03.564936 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 870/1774] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:03 2022] PUT /identity/v3/projects/38bea6ec7e1c4836868191839fd7b889/users/4d52e97d8e154a7986918ed3918cb4e2/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:03.565927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e90c0b-f5a2-434c-8da2-909f61c3c20c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:03.595480 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 905/1775] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:03 2022] POST /identity/v3/users => generated 308 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:03.597639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0bf59-6178-4e7f-9f39-592ad5caa1f7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:03.609707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0bf59-6178-4e7f-9f39-592ad5caa1f7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:03.682159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e90c0b-f5a2-434c-8da2-909f61c3c20c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:03.682696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acb54a-fd82-4750-bd40-b534e7412461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:03.683570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acb54a-fd82-4750-bd40-b534e7412461 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:03.684017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acb54a-fd82-4750-bd40-b534e7412461 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:03.684393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acb54a-fd82-4750-bd40-b534e7412461 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:03.685276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60acb54a-fd82-4750-bd40-b534e7412461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:03.687470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60acb54a-fd82-4750-bd40-b534e7412461 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:03.689966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d1f70-36da-444f-8571-dc71d64b017c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:03.691673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d1f70-36da-444f-8571-dc71d64b017c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:03.692104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d1f70-36da-444f-8571-dc71d64b017c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:03.692404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d1f70-36da-444f-8571-dc71d64b017c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:03.694603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d1f70-36da-444f-8571-dc71d64b017c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:03.696746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d1f70-36da-444f-8571-dc71d64b017c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:03.703517 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 906/1776] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:03.704488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acb54a-fd82-4750-bd40-b534e7412461 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:03.709185 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 871/1777] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:03.710657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d1f70-36da-444f-8571-dc71d64b017c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:03.719598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acb54a-fd82-4750-bd40-b534e7412461 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:03.723094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d1f70-36da-444f-8571-dc71d64b017c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:03.725020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb4d6e-b242-48bd-95b9-b5e6457baade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:03.725730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb4d6e-b242-48bd-95b9-b5e6457baade admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:03.726054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb4d6e-b242-48bd-95b9-b5e6457baade admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:03.726343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb4d6e-b242-48bd-95b9-b5e6457baade admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:03.727106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb4d6e-b242-48bd-95b9-b5e6457baade admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:03.729038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb4d6e-b242-48bd-95b9-b5e6457baade admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:03.729212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb61dc-4d8b-42c7-8018-ac84c788958a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:03.743835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb61dc-4d8b-42c7-8018-ac84c788958a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:03.744252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb61dc-4d8b-42c7-8018-ac84c788958a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:03.744603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb61dc-4d8b-42c7-8018-ac84c788958a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:03.745307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb61dc-4d8b-42c7-8018-ac84c788958a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:03.748152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb61dc-4d8b-42c7-8018-ac84c788958a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:03.750175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:03.760391 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 872/1778] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:03.761811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967cd586-ae05-43b0-bdf8-260f0a99e9c5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:03.762136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967cd586-ae05-43b0-bdf8-260f0a99e9c5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:03.762424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967cd586-ae05-43b0-bdf8-260f0a99e9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:03.791607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-967cd586-ae05-43b0-bdf8-260f0a99e9c5 None None] MFA Rules not processed for user `4d52e97d8e154a7986918ed3918cb4e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:03.792862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-967cd586-ae05-43b0-bdf8-260f0a99e9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:03.820065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:03.851140 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 907/1779] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:03 2022] POST /identity/v3/projects => generated 389 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:03.852239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb4d6e-b242-48bd-95b9-b5e6457baade admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:03.887877 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 873/1780] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:03.888825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967cd586-ae05-43b0-bdf8-260f0a99e9c5 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:03.900848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967cd586-ae05-43b0-bdf8-260f0a99e9c5 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:03.955438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb4d6e-b242-48bd-95b9-b5e6457baade admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:03.961320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f5525-36eb-4185-a517-a1a4dc06ddeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:03.962173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f5525-36eb-4185-a517-a1a4dc06ddeb admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:03.962522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f5525-36eb-4185-a517-a1a4dc06ddeb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:03.962620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:03.962937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f5525-36eb-4185-a517-a1a4dc06ddeb admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:03.963747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f5525-36eb-4185-a517-a1a4dc06ddeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:03.965520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f5525-36eb-4185-a517-a1a4dc06ddeb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:03.976767 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 908/1781] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:03.977582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f5525-36eb-4185-a517-a1a4dc06ddeb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:03.981972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59ec13-45ef-4fbd-8b04-1f8dd15e7f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:03.982842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59ec13-45ef-4fbd-8b04-1f8dd15e7f23 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:03.983330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59ec13-45ef-4fbd-8b04-1f8dd15e7f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:03.983609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59ec13-45ef-4fbd-8b04-1f8dd15e7f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:03.984398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59ec13-45ef-4fbd-8b04-1f8dd15e7f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:03.986026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59ec13-45ef-4fbd-8b04-1f8dd15e7f23 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:03.988616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f5525-36eb-4185-a517-a1a4dc06ddeb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:03.993941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70d872-758b-4b32-bdcb-b162ef5d9014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:03.994613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70d872-758b-4b32-bdcb-b162ef5d9014 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:03.994912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70d872-758b-4b32-bdcb-b162ef5d9014 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:03.995184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70d872-758b-4b32-bdcb-b162ef5d9014 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:03.996204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70d872-758b-4b32-bdcb-b162ef5d9014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:03.998016 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 874/1782] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:03.998121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70d872-758b-4b32-bdcb-b162ef5d9014 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:03.999255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59ec13-45ef-4fbd-8b04-1f8dd15e7f23 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:04.009815 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 909/1783] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:04.010477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59ec13-45ef-4fbd-8b04-1f8dd15e7f23 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:04.010673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70d872-758b-4b32-bdcb-b162ef5d9014 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:04.015681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36b72f-96cf-4acc-b1cd-61abf3773586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:04.016578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36b72f-96cf-4acc-b1cd-61abf3773586 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:04.016866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36b72f-96cf-4acc-b1cd-61abf3773586 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:04.017818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea36b72f-96cf-4acc-b1cd-61abf3773586 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:04.019465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36b72f-96cf-4acc-b1cd-61abf3773586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:04.021111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea36b72f-96cf-4acc-b1cd-61abf3773586 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:04.021226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70d872-758b-4b32-bdcb-b162ef5d9014 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:04.024826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:04.028517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6400ca-0068-4635-9261-f26d72d45e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:04.032967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6400ca-0068-4635-9261-f26d72d45e3f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:04.032967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6400ca-0068-4635-9261-f26d72d45e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:04.032967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6400ca-0068-4635-9261-f26d72d45e3f admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:04.032967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6400ca-0068-4635-9261-f26d72d45e3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:04.032967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6400ca-0068-4635-9261-f26d72d45e3f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:04.035090 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 875/1784] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:04.036003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36b72f-96cf-4acc-b1cd-61abf3773586 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:04.048747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea36b72f-96cf-4acc-b1cd-61abf3773586 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:04.054671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e788424-94ce-4b91-906e-2f43a2c06e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:04.055359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e788424-94ce-4b91-906e-2f43a2c06e09 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:04.055647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e788424-94ce-4b91-906e-2f43a2c06e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:04.055934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e788424-94ce-4b91-906e-2f43a2c06e09 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:04.056655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e788424-94ce-4b91-906e-2f43a2c06e09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:04.058798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e788424-94ce-4b91-906e-2f43a2c06e09 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:04.117137 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 910/1785] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:04 2022] POST /identity/v3/users => generated 301 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:04.118297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6400ca-0068-4635-9261-f26d72d45e3f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:04.128385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6400ca-0068-4635-9261-f26d72d45e3f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:04.135614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd671f-84b4-4033-bd0d-9ef7ec1c80ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:04.136549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd671f-84b4-4033-bd0d-9ef7ec1c80ec admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:04.136832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd671f-84b4-4033-bd0d-9ef7ec1c80ec admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:04.137111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd671f-84b4-4033-bd0d-9ef7ec1c80ec admin admin] PATH_INFO: `/v3/projects/db5eec07b6534c968b59a4b21cb75975/users/16fedb3759d94951a633d58d8f91abd3/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:04.140990 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 876/1786] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:04 2022] POST /identity/v3/projects => generated 395 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:04.141751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e788424-94ce-4b91-906e-2f43a2c06e09 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:04.155935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd671f-84b4-4033-bd0d-9ef7ec1c80ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=db5eec07b6534c968b59a4b21cb75975, user_id=16fedb3759d94951a633d58d8f91abd3, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:04.159303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd671f-84b4-4033-bd0d-9ef7ec1c80ec admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:04.205808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e788424-94ce-4b91-906e-2f43a2c06e09 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:04.210663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafada7c-ab5a-4c42-9951-6b07fd2d72dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:04.211577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafada7c-ab5a-4c42-9951-6b07fd2d72dd admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:04.211865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafada7c-ab5a-4c42-9951-6b07fd2d72dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:04.212159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafada7c-ab5a-4c42-9951-6b07fd2d72dd admin admin] PATH_INFO: `/v3/projects/dc1a016f5efa407387d7cccddf3cf0f0/users/1acd163a2a65407db8846a8878b0c6be/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:04.227696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafada7c-ab5a-4c42-9951-6b07fd2d72dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc1a016f5efa407387d7cccddf3cf0f0, user_id=1acd163a2a65407db8846a8878b0c6be, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:04.230444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafada7c-ab5a-4c42-9951-6b07fd2d72dd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:04.252195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:04.279185 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 911/1787] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:04 2022] PUT /identity/v3/projects/db5eec07b6534c968b59a4b21cb75975/users/16fedb3759d94951a633d58d8f91abd3/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:04.279185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd671f-84b4-4033-bd0d-9ef7ec1c80ec admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:04.310052 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 877/1788] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:04 2022] PUT /identity/v3/projects/dc1a016f5efa407387d7cccddf3cf0f0/users/1acd163a2a65407db8846a8878b0c6be/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:04.311764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafada7c-ab5a-4c42-9951-6b07fd2d72dd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:04.326951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:04.366615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd671f-84b4-4033-bd0d-9ef7ec1c80ec admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:04.382204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafada7c-ab5a-4c42-9951-6b07fd2d72dd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:04.388463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbb6b6-5fc5-4ff4-9722-1ad0fd86f569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:04.389398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbb6b6-5fc5-4ff4-9722-1ad0fd86f569 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:04.389683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbb6b6-5fc5-4ff4-9722-1ad0fd86f569 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:04.389995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbb6b6-5fc5-4ff4-9722-1ad0fd86f569 admin admin] PATH_INFO: `/v3/projects/657641a7c7a84eba962742035e6c2640/users/979c95b05b59471a954ca8a9097c8660/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:04.412198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bbb6b6-5fc5-4ff4-9722-1ad0fd86f569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=657641a7c7a84eba962742035e6c2640, user_id=979c95b05b59471a954ca8a9097c8660, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:04.417829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bbb6b6-5fc5-4ff4-9722-1ad0fd86f569 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:04.426809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6f247-6efc-483f-916c-e505e18a8a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:04.427709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6f247-6efc-483f-916c-e505e18a8a52 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:04.427993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6f247-6efc-483f-916c-e505e18a8a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:04.428914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6f247-6efc-483f-916c-e505e18a8a52 admin admin] PATH_INFO: `/v3/projects/de28695bc5a94ae8b467fb876daa5f47/users/aacbd21e477d4531915ae01e4d5b84bc/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:04.444974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6f247-6efc-483f-916c-e505e18a8a52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de28695bc5a94ae8b467fb876daa5f47, user_id=aacbd21e477d4531915ae01e4d5b84bc, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:04.447756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e6f247-6efc-483f-916c-e505e18a8a52 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:04.486880 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 878/1789] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:04 2022] PUT /identity/v3/projects/657641a7c7a84eba962742035e6c2640/users/979c95b05b59471a954ca8a9097c8660/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:04.496864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbb6b6-5fc5-4ff4-9722-1ad0fd86f569 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:04.550630 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 912/1790] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:04 2022] PUT /identity/v3/projects/de28695bc5a94ae8b467fb876daa5f47/users/aacbd21e477d4531915ae01e4d5b84bc/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:04.551607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6f247-6efc-483f-916c-e505e18a8a52 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:04.563363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbb6b6-5fc5-4ff4-9722-1ad0fd86f569 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:04.566231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:04.623635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6f247-6efc-483f-916c-e505e18a8a52 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:04.628514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145deaef-3ee2-4f6a-9702-f83e2e0a372b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:04.629213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145deaef-3ee2-4f6a-9702-f83e2e0a372b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:04.629502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145deaef-3ee2-4f6a-9702-f83e2e0a372b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:04.629821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145deaef-3ee2-4f6a-9702-f83e2e0a372b admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:04.630555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145deaef-3ee2-4f6a-9702-f83e2e0a372b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:04.632647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145deaef-3ee2-4f6a-9702-f83e2e0a372b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:04.632768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f524c-22dd-4312-83d0-fae108d896ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:04.633386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f524c-22dd-4312-83d0-fae108d896ca admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:04.638324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f524c-22dd-4312-83d0-fae108d896ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:04.638324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6f524c-22dd-4312-83d0-fae108d896ca admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:04.638324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f524c-22dd-4312-83d0-fae108d896ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:04.638324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6f524c-22dd-4312-83d0-fae108d896ca admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:04.650631 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 879/1791] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:04.650631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f524c-22dd-4312-83d0-fae108d896ca admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:04.658146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6f524c-22dd-4312-83d0-fae108d896ca admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:04.658489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:04.663857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d2fb90-747c-4d4c-a75e-72dfeb10764d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:04.664511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2fb90-747c-4d4c-a75e-72dfeb10764d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:04.664795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2fb90-747c-4d4c-a75e-72dfeb10764d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:04.665078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d2fb90-747c-4d4c-a75e-72dfeb10764d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:04.665802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d2fb90-747c-4d4c-a75e-72dfeb10764d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:04.667514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d2fb90-747c-4d4c-a75e-72dfeb10764d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:04.677430 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 880/1792] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:04.678736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f45d5-5d5a-46c5-8641-09548a7dd4c3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:04.679022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f45d5-5d5a-46c5-8641-09548a7dd4c3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:04.679298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f45d5-5d5a-46c5-8641-09548a7dd4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:04.697636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e71f45d5-5d5a-46c5-8641-09548a7dd4c3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:04.698885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e71f45d5-5d5a-46c5-8641-09548a7dd4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:04.706891 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 881/1793] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:04.708127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d610a-6fba-401a-a62d-df3490af1cb7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:04.708417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d610a-6fba-401a-a62d-df3490af1cb7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:04.708698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d610a-6fba-401a-a62d-df3490af1cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:04.723589 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 913/1794] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:04 2022] POST /identity/v3/users => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:04.725634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a6c14-069f-4104-b461-18fe9e24db0c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:04.726074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a6c14-069f-4104-b461-18fe9e24db0c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:04.726449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a6c14-069f-4104-b461-18fe9e24db0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:04.738223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-964d610a-6fba-401a-a62d-df3490af1cb7 None None] MFA Rules not processed for user `979c95b05b59471a954ca8a9097c8660`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:04.739439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-964d610a-6fba-401a-a62d-df3490af1cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:04.756307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e28a6c14-069f-4104-b461-18fe9e24db0c None None] MFA Rules not processed for user `aacbd21e477d4531915ae01e4d5b84bc`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:04.757745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e28a6c14-069f-4104-b461-18fe9e24db0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:04.828765 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 882/1795] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:04.829663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d610a-6fba-401a-a62d-df3490af1cb7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:04.835236 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 914/1796] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:04.836160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a6c14-069f-4104-b461-18fe9e24db0c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:04.842025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:04.847604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a6c14-069f-4104-b461-18fe9e24db0c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:04.848554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d610a-6fba-401a-a62d-df3490af1cb7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:04.853791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589e35f-06e6-4025-8c74-9e144330bc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:04.856629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45129c27-ddc2-407b-bb19-2a77e76e4f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:04.856791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589e35f-06e6-4025-8c74-9e144330bc2d admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:04.856791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589e35f-06e6-4025-8c74-9e144330bc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:04.856791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589e35f-06e6-4025-8c74-9e144330bc2d admin admin] PATH_INFO: `/v3/projects/db5eec07b6534c968b59a4b21cb75975/users/16fedb3759d94951a633d58d8f91abd3/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:04.857056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45129c27-ddc2-407b-bb19-2a77e76e4f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:04.857056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45129c27-ddc2-407b-bb19-2a77e76e4f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:04.857056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45129c27-ddc2-407b-bb19-2a77e76e4f0a admin admin] PATH_INFO: `/v3/projects/ce82bcaad9354b0d86718aeec0d5bf6f/users/34c91c3ae5ce4ffe9ee08500d1aff221/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:04.858132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d589e35f-06e6-4025-8c74-9e144330bc2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=db5eec07b6534c968b59a4b21cb75975, user_id=16fedb3759d94951a633d58d8f91abd3, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:04.860898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d589e35f-06e6-4025-8c74-9e144330bc2d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:04.871945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45129c27-ddc2-407b-bb19-2a77e76e4f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce82bcaad9354b0d86718aeec0d5bf6f, user_id=34c91c3ae5ce4ffe9ee08500d1aff221, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:04.875959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45129c27-ddc2-407b-bb19-2a77e76e4f0a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:04.878406 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 915/1797] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:04 2022] PUT /identity/v3/projects/db5eec07b6534c968b59a4b21cb75975/users/16fedb3759d94951a633d58d8f91abd3/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:04.879339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589e35f-06e6-4025-8c74-9e144330bc2d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:04.898805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d589e35f-06e6-4025-8c74-9e144330bc2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:04.920926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:04.965001 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 883/1798] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:04 2022] PUT /identity/v3/projects/ce82bcaad9354b0d86718aeec0d5bf6f/users/34c91c3ae5ce4ffe9ee08500d1aff221/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:04.966034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45129c27-ddc2-407b-bb19-2a77e76e4f0a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:04.972979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589e35f-06e6-4025-8c74-9e144330bc2d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:05.038931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f73ac-79c9-4945-976b-106729e5fb60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:05.040187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f73ac-79c9-4945-976b-106729e5fb60 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:05.040580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f73ac-79c9-4945-976b-106729e5fb60 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:05.041043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f73ac-79c9-4945-976b-106729e5fb60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:05.103132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45129c27-ddc2-407b-bb19-2a77e76e4f0a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:05.128453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68f877-6888-416d-90f5-3d9445eaf16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:05.128453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68f877-6888-416d-90f5-3d9445eaf16f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:05.128747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68f877-6888-416d-90f5-3d9445eaf16f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:05.128980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f68f877-6888-416d-90f5-3d9445eaf16f admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:05.132665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f68f877-6888-416d-90f5-3d9445eaf16f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:05.136048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f68f877-6888-416d-90f5-3d9445eaf16f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:05.157461 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 884/1799] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:05.159110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68f877-6888-416d-90f5-3d9445eaf16f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:05.171515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f73ac-79c9-4945-976b-106729e5fb60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:05.172979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f73ac-79c9-4945-976b-106729e5fb60 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:05.182088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f68f877-6888-416d-90f5-3d9445eaf16f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:05.188669 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 916/1800] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:05.189615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f73ac-79c9-4945-976b-106729e5fb60 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:05.194942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae2501-5c5c-464e-b359-087ad799fb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:05.198257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae2501-5c5c-464e-b359-087ad799fb1e admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:05.200103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae2501-5c5c-464e-b359-087ad799fb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:05.204359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae2501-5c5c-464e-b359-087ad799fb1e admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:05.204772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f73ac-79c9-4945-976b-106729e5fb60 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:05.208446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae2501-5c5c-464e-b359-087ad799fb1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:05.210713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57a086-1573-476a-a50d-e53444b12fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:05.211570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57a086-1573-476a-a50d-e53444b12fb9 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:05.212070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57a086-1573-476a-a50d-e53444b12fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:05.212366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a57a086-1573-476a-a50d-e53444b12fb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:05.213083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57a086-1573-476a-a50d-e53444b12fb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:05.214770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae2501-5c5c-464e-b359-087ad799fb1e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:05.215216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a57a086-1573-476a-a50d-e53444b12fb9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:05.258521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:05.338259 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 917/1801] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:05 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:05.342641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57a086-1573-476a-a50d-e53444b12fb9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:05.391133 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 885/1802] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:05 2022] POST /identity/v3/projects => generated 391 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:05.392524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae2501-5c5c-464e-b359-087ad799fb1e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:05.456119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a57a086-1573-476a-a50d-e53444b12fb9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:05.516210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae2501-5c5c-464e-b359-087ad799fb1e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:05.521579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01c3b7-b939-4423-af26-e82fc58c90ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:05.522615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01c3b7-b939-4423-af26-e82fc58c90ed admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:05.522945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01c3b7-b939-4423-af26-e82fc58c90ed admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:05.523273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01c3b7-b939-4423-af26-e82fc58c90ed admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:05.524303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01c3b7-b939-4423-af26-e82fc58c90ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:05.527319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01c3b7-b939-4423-af26-e82fc58c90ed admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:05.532329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263a683-7058-4035-b3a9-981975471f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:05.535228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263a683-7058-4035-b3a9-981975471f9b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:05.535228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263a683-7058-4035-b3a9-981975471f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:05.535228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1263a683-7058-4035-b3a9-981975471f9b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:05.535228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263a683-7058-4035-b3a9-981975471f9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:05.537233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1263a683-7058-4035-b3a9-981975471f9b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:05.545355 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 886/1803] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:05.546467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01c3b7-b939-4423-af26-e82fc58c90ed admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:05.564857 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 918/1804] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:05.564857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263a683-7058-4035-b3a9-981975471f9b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:05.572309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01c3b7-b939-4423-af26-e82fc58c90ed admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:05.583298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc640b-b1e4-4833-baa5-c17b1c431cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:05.584227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc640b-b1e4-4833-baa5-c17b1c431cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:05.584809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc640b-b1e4-4833-baa5-c17b1c431cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:05.585377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc640b-b1e4-4833-baa5-c17b1c431cd4 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:05.586369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc640b-b1e4-4833-baa5-c17b1c431cd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:05.588665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc640b-b1e4-4833-baa5-c17b1c431cd4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:05.590772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1263a683-7058-4035-b3a9-981975471f9b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:05.597097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0230a-a1e2-4402-a46f-39998fd6e888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:05.597948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0230a-a1e2-4402-a46f-39998fd6e888 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:05.599387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0230a-a1e2-4402-a46f-39998fd6e888 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:05.599728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0230a-a1e2-4402-a46f-39998fd6e888 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:05.600864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0230a-a1e2-4402-a46f-39998fd6e888 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:05.603151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0230a-a1e2-4402-a46f-39998fd6e888 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:05.690358 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 887/1805] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:05 2022] POST /identity/v3/users => generated 298 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:05.692354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc640b-b1e4-4833-baa5-c17b1c431cd4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:05.714130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc640b-b1e4-4833-baa5-c17b1c431cd4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:05.721638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6754e023-9155-48aa-85ec-5eddba83790f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:05.723308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754e023-9155-48aa-85ec-5eddba83790f admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:05.723833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754e023-9155-48aa-85ec-5eddba83790f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:05.724321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6754e023-9155-48aa-85ec-5eddba83790f admin admin] PATH_INFO: `/v3/projects/751501c7dfad4ad0a00524c6baf4471d/users/835432d20a804515856b22e35d7b2769/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:05.727162 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 919/1806] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:05 2022] POST /identity/v3/users => generated 302 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:05.727984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0230a-a1e2-4402-a46f-39998fd6e888 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:05.744011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754e023-9155-48aa-85ec-5eddba83790f admin admin] RBAC: Authorizing `identity:create_grant(project_id=751501c7dfad4ad0a00524c6baf4471d, user_id=835432d20a804515856b22e35d7b2769, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:05.747599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6754e023-9155-48aa-85ec-5eddba83790f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:05.747721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0230a-a1e2-4402-a46f-39998fd6e888 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:05.759553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc03e71-afe5-4b7a-b4b7-c283615ab451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:05.760471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc03e71-afe5-4b7a-b4b7-c283615ab451 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:05.760760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc03e71-afe5-4b7a-b4b7-c283615ab451 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:05.762555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc03e71-afe5-4b7a-b4b7-c283615ab451 admin admin] PATH_INFO: `/v3/projects/ce82bcaad9354b0d86718aeec0d5bf6f/users/34c91c3ae5ce4ffe9ee08500d1aff221/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:05.763192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc03e71-afe5-4b7a-b4b7-c283615ab451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce82bcaad9354b0d86718aeec0d5bf6f, user_id=34c91c3ae5ce4ffe9ee08500d1aff221, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:05.766266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc03e71-afe5-4b7a-b4b7-c283615ab451 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:05.786074 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 920/1807] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:05 2022] PUT /identity/v3/projects/ce82bcaad9354b0d86718aeec0d5bf6f/users/34c91c3ae5ce4ffe9ee08500d1aff221/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:05.786074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc03e71-afe5-4b7a-b4b7-c283615ab451 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:05.795767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:05.805976 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 888/1808] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:05 2022] PUT /identity/v3/projects/751501c7dfad4ad0a00524c6baf4471d/users/835432d20a804515856b22e35d7b2769/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:05.807694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffb7de-13c0-4425-890d-f9b36a57f134 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:05.808192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffb7de-13c0-4425-890d-f9b36a57f134 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:05.808699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffb7de-13c0-4425-890d-f9b36a57f134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:05.817400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:05.832555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-43ffb7de-13c0-4425-890d-f9b36a57f134 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:05.836202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-43ffb7de-13c0-4425-890d-f9b36a57f134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:05.895537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc03e71-afe5-4b7a-b4b7-c283615ab451 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:05.922215 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 889/1809] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:03:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:05.924496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffb7de-13c0-4425-890d-f9b36a57f134 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:05.982077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9745442-35f1-4186-9313-3ac0f47b53d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:05.983924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9745442-35f1-4186-9313-3ac0f47b53d4 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:05.984361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9745442-35f1-4186-9313-3ac0f47b53d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:05.984745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9745442-35f1-4186-9313-3ac0f47b53d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:05.986201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9745442-35f1-4186-9313-3ac0f47b53d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:05.988639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9745442-35f1-4186-9313-3ac0f47b53d4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:06.009029 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 921/1810] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:06.011289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69c205-3887-4dc4-9def-c864da1fc29d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:06.012314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69c205-3887-4dc4-9def-c864da1fc29d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:06.012718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69c205-3887-4dc4-9def-c864da1fc29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:06.020993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffb7de-13c0-4425-890d-f9b36a57f134 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:06.032285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10507c59-c1f0-40b6-bea4-dbb679249608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:06.033243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10507c59-c1f0-40b6-bea4-dbb679249608 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:06.033716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10507c59-c1f0-40b6-bea4-dbb679249608 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:06.034427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10507c59-c1f0-40b6-bea4-dbb679249608 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:06.035569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10507c59-c1f0-40b6-bea4-dbb679249608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:06.038374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10507c59-c1f0-40b6-bea4-dbb679249608 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:06.055449 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 890/1811] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:06.055715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ff69c205-3887-4dc4-9def-c864da1fc29d None None] MFA Rules not processed for user `34c91c3ae5ce4ffe9ee08500d1aff221`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:06.057405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ff69c205-3887-4dc4-9def-c864da1fc29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:06.058290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3c379-9288-4946-92ac-1c5ab6186c3e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:06.058846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3c379-9288-4946-92ac-1c5ab6186c3e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:06.059311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c3c379-9288-4946-92ac-1c5ab6186c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:06.112518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a0c3c379-9288-4946-92ac-1c5ab6186c3e None None] MFA Rules not processed for user `835432d20a804515856b22e35d7b2769`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:06.136459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a0c3c379-9288-4946-92ac-1c5ab6186c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:06.204291 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 922/1812] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:06.217076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69c205-3887-4dc4-9def-c864da1fc29d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:06.238865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69c205-3887-4dc4-9def-c864da1fc29d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:06.265032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eeeec7-3bb7-4ee8-836c-889e7edd90fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:06.266969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eeeec7-3bb7-4ee8-836c-889e7edd90fc admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:06.267260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eeeec7-3bb7-4ee8-836c-889e7edd90fc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:06.268263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eeeec7-3bb7-4ee8-836c-889e7edd90fc admin admin] PATH_INFO: `/v3/projects/9077c3b2a44a44d1b8c81f3bb054c451/users/e87af1dda77942fcb376c392c8242b70/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:06.297620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eeeec7-3bb7-4ee8-836c-889e7edd90fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9077c3b2a44a44d1b8c81f3bb054c451, user_id=e87af1dda77942fcb376c392c8242b70, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:06.300382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eeeec7-3bb7-4ee8-836c-889e7edd90fc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:06.304603 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 891/1813] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:06.306417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3c379-9288-4946-92ac-1c5ab6186c3e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:06.323554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c3c379-9288-4946-92ac-1c5ab6186c3e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:06.331841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3395e-29e4-420a-bcce-bb15ad489cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:06.331841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3395e-29e4-420a-bcce-bb15ad489cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:06.332042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3395e-29e4-420a-bcce-bb15ad489cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:06.332426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3395e-29e4-420a-bcce-bb15ad489cb7 admin admin] PATH_INFO: `/v3/projects/d2fdef2647da40969db6927ef81349a6/users/e7af0c086a7140c3a8a1f4ad142815ea/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:06.361552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3395e-29e4-420a-bcce-bb15ad489cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2fdef2647da40969db6927ef81349a6, user_id=e7af0c086a7140c3a8a1f4ad142815ea, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:06.364364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3395e-29e4-420a-bcce-bb15ad489cb7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:06.404086 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 923/1814] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:06 2022] PUT /identity/v3/projects/9077c3b2a44a44d1b8c81f3bb054c451/users/e87af1dda77942fcb376c392c8242b70/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:06.407324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935222a-96f7-4115-91ee-087602fdf963 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:06.407324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935222a-96f7-4115-91ee-087602fdf963 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:06.407324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935222a-96f7-4115-91ee-087602fdf963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:06.426505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d935222a-96f7-4115-91ee-087602fdf963 None None] MFA Rules not processed for user `ad6f51b7b6834fefa06c58829399de85`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:06.427954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d935222a-96f7-4115-91ee-087602fdf963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:06.453676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:06.471379 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 892/1815] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:06 2022] PUT /identity/v3/projects/d2fdef2647da40969db6927ef81349a6/users/e7af0c086a7140c3a8a1f4ad142815ea/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:06.472386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3395e-29e4-420a-bcce-bb15ad489cb7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:06.474470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9cb3395e-29e4-420a-bcce-bb15ad489cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:06.482214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:06.517707 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 924/1816] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:06.517707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935222a-96f7-4115-91ee-087602fdf963 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:06.612028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3395e-29e4-420a-bcce-bb15ad489cb7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:06.624643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fdeab-9127-4c98-b3cc-7333a008350d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:06.628903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fdeab-9127-4c98-b3cc-7333a008350d service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:06.629224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fdeab-9127-4c98-b3cc-7333a008350d service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:06.629568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fdeab-9127-4c98-b3cc-7333a008350d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:06.649015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935222a-96f7-4115-91ee-087602fdf963 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:06.761186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a3f00-20eb-4970-a761-df60efbe6b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:06.761650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a3f00-20eb-4970-a761-df60efbe6b96 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:06.761973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a3f00-20eb-4970-a761-df60efbe6b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:06.762255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848a3f00-20eb-4970-a761-df60efbe6b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:06.763003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a3f00-20eb-4970-a761-df60efbe6b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:06.765984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fdeab-9127-4c98-b3cc-7333a008350d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:06.767683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fdeab-9127-4c98-b3cc-7333a008350d service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:06.769482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848a3f00-20eb-4970-a761-df60efbe6b96 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:06.799705 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 925/1817] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:06.800659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a3f00-20eb-4970-a761-df60efbe6b96 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:06.805000 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 893/1818] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:06.806041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fdeab-9127-4c98-b3cc-7333a008350d service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:06.810535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-848a3f00-20eb-4970-a761-df60efbe6b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:06.834020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fdeab-9127-4c98-b3cc-7333a008350d service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:06.849336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd5e40-0f65-4c93-9430-b9573dd40055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:06.854003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd5e40-0f65-4c93-9430-b9573dd40055 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:06.854003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd5e40-0f65-4c93-9430-b9573dd40055 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:06.854003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd5e40-0f65-4c93-9430-b9573dd40055 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:06.854003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd5e40-0f65-4c93-9430-b9573dd40055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:06.854813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd5e40-0f65-4c93-9430-b9573dd40055 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:06.874459 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 894/1819] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:06 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:06.875154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd5e40-0f65-4c93-9430-b9573dd40055 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:06.882699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-96dd5e40-0f65-4c93-9430-b9573dd40055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:06.910323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd5e40-0f65-4c93-9430-b9573dd40055 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:06.928290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc9897-b61e-49b3-ab51-733378e22c54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:06.930373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc9897-b61e-49b3-ab51-733378e22c54 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:06.931536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc9897-b61e-49b3-ab51-733378e22c54 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:06.932392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc9897-b61e-49b3-ab51-733378e22c54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:06.972779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848a3f00-20eb-4970-a761-df60efbe6b96 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:06.989123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ae859-cda4-4a3b-9e31-aade87a56a0d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:06.989541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ae859-cda4-4a3b-9e31-aade87a56a0d service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:06.989879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ae859-cda4-4a3b-9e31-aade87a56a0d service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:06.990135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6ae859-cda4-4a3b-9e31-aade87a56a0d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:07.037219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc9897-b61e-49b3-ab51-733378e22c54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:07.042999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc9897-b61e-49b3-ab51-733378e22c54 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:07.076849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ae859-cda4-4a3b-9e31-aade87a56a0d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:07.078671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6ae859-cda4-4a3b-9e31-aade87a56a0d service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:07.080273 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 895/1820] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:07.081764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc9897-b61e-49b3-ab51-733378e22c54 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:07.100819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc9897-b61e-49b3-ab51-733378e22c54 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:07.113506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d549edc-6b08-4a60-93cf-41f7b7d6e4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:07.113805 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 926/1821] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:03:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:03:07.114657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d549edc-6b08-4a60-93cf-41f7b7d6e4b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:07.114951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d549edc-6b08-4a60-93cf-41f7b7d6e4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:07.115230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d549edc-6b08-4a60-93cf-41f7b7d6e4b1 admin admin] PATH_INFO: `/v3/projects/9077c3b2a44a44d1b8c81f3bb054c451/users/e87af1dda77942fcb376c392c8242b70/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:07.119217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d549edc-6b08-4a60-93cf-41f7b7d6e4b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9077c3b2a44a44d1b8c81f3bb054c451, user_id=e87af1dda77942fcb376c392c8242b70, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:07.123214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d549edc-6b08-4a60-93cf-41f7b7d6e4b1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:07.123790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ae859-cda4-4a3b-9e31-aade87a56a0d service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:07.148970 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 896/1822] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:07 2022] PUT /identity/v3/projects/9077c3b2a44a44d1b8c81f3bb054c451/users/e87af1dda77942fcb376c392c8242b70/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:07.150511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2befa-8ca8-48b3-a95f-050cbba8ea46 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:07.150801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2befa-8ca8-48b3-a95f-050cbba8ea46 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:07.151082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2befa-8ca8-48b3-a95f-050cbba8ea46 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:07.153123 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 897/1823] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:03:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:07.154349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0fd77-da2f-4190-ad06-1fa870dcf141 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:07.154639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0fd77-da2f-4190-ad06-1fa870dcf141 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:07.154920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0fd77-da2f-4190-ad06-1fa870dcf141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:07.160148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6ae859-cda4-4a3b-9e31-aade87a56a0d service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:07.208155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b1c0fd77-da2f-4190-ad06-1fa870dcf141 None None] MFA Rules not processed for user `e87af1dda77942fcb376c392c8242b70`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:07.210891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b1c0fd77-da2f-4190-ad06-1fa870dcf141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:07.221540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:07.286263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:07.332045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d03e9-f439-4291-a80a-62212e4d51c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:07.332992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d03e9-f439-4291-a80a-62212e4d51c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:07.333425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d03e9-f439-4291-a80a-62212e4d51c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:07.334036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d03e9-f439-4291-a80a-62212e4d51c0 admin admin] PATH_INFO: `/v3/projects/d2fdef2647da40969db6927ef81349a6/users/e7af0c086a7140c3a8a1f4ad142815ea/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:07.340513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d03e9-f439-4291-a80a-62212e4d51c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2fdef2647da40969db6927ef81349a6, user_id=e7af0c086a7140c3a8a1f4ad142815ea, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:07.347526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d03e9-f439-4291-a80a-62212e4d51c0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:07.377493 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 927/1824] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:07 2022] PUT /identity/v3/projects/d2fdef2647da40969db6927ef81349a6/users/e7af0c086a7140c3a8a1f4ad142815ea/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:07.379222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d03e9-f439-4291-a80a-62212e4d51c0 admin admin] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:03:07.448952 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 898/1825] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:07.459058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5a3d6-011d-4b88-8d06-6e4dbeed8d2e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:07.459409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5a3d6-011d-4b88-8d06-6e4dbeed8d2e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:07.460770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b5a3d6-011d-4b88-8d06-6e4dbeed8d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:07.546368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-99b5a3d6-011d-4b88-8d06-6e4dbeed8d2e None None] MFA Rules not processed for user `e7af0c086a7140c3a8a1f4ad142815ea`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:07.554011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-99b5a3d6-011d-4b88-8d06-6e4dbeed8d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:07.659573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d03e9-f439-4291-a80a-62212e4d51c0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:07.820564 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 899/1826] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:07.823731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4597b-9344-4126-adfd-8dda8f52f47d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:07.824034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4597b-9344-4126-adfd-8dda8f52f47d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:07.824317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4597b-9344-4126-adfd-8dda8f52f47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:07.840172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:07.892906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d03e9-f439-4291-a80a-62212e4d51c0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:07.900240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3c5d5-9af4-40a4-b181-6f4cb4908fa7 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d52e97d8e154a7986918ed3918cb4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38bea6ec7e1c4836868191839fd7b889', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:07.903277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3c5d5-9af4-40a4-b181-6f4cb4908fa7 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:07.904407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3c5d5-9af4-40a4-b181-6f4cb4908fa7 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:07.904917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3c5d5-9af4-40a4-b181-6f4cb4908fa7 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] PATH_INFO: `/v3/projects/0d902454e7bd44339240a17aef7d1335` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:07.912026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3c5d5-9af4-40a4-b181-6f4cb4908fa7 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: Authorizing `identity:get_project(project_id=0d902454e7bd44339240a17aef7d1335)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:07.915455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d3c5d5-9af4-40a4-b181-6f4cb4908fa7 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:07.921119 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 928/1827] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:03:07 2022] GET /identity/v3/projects/0d902454e7bd44339240a17aef7d1335 => generated 395 bytes in 543 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:07.927751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-edb4597b-9344-4126-adfd-8dda8f52f47d None None] MFA Rules not processed for user `1b2a9ebcd3f84b758322a80c6e590de0`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:07.929041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-edb4597b-9344-4126-adfd-8dda8f52f47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:08.095103 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 900/1828] 10.222.0.41 () {58 vars in 1015 bytes} [Sat May 28 21:03:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:08.132369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3c5d5-9af4-40a4-b181-6f4cb4908fa7 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:08.155801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d3c5d5-9af4-40a4-b181-6f4cb4908fa7 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:08.167174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a54e6-910c-4112-8b8b-75da3285f2ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:08.168037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a54e6-910c-4112-8b8b-75da3285f2ba service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:08.168426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a54e6-910c-4112-8b8b-75da3285f2ba service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:08.168809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a54e6-910c-4112-8b8b-75da3285f2ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:08.191277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a54e6-910c-4112-8b8b-75da3285f2ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:08.195694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a54e6-910c-4112-8b8b-75da3285f2ba service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:08.213701 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 929/1829] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:08.627528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4597b-9344-4126-adfd-8dda8f52f47d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:08.637408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a54e6-910c-4112-8b8b-75da3285f2ba service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:08.643839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4597b-9344-4126-adfd-8dda8f52f47d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:08.655410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a54e6-910c-4112-8b8b-75da3285f2ba service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:08.656179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162da667-8efe-4431-bbfb-637a25714964 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:08.656839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162da667-8efe-4431-bbfb-637a25714964 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:08.657131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162da667-8efe-4431-bbfb-637a25714964 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:08.657411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162da667-8efe-4431-bbfb-637a25714964 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:08.668788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e5b30-8392-448a-bc3a-164ccd10122d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:08.670757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e5b30-8392-448a-bc3a-164ccd10122d service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:08.671286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e5b30-8392-448a-bc3a-164ccd10122d service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:08.672008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e5b30-8392-448a-bc3a-164ccd10122d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:08.688909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:08.699379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e5b30-8392-448a-bc3a-164ccd10122d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:08.703623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e5b30-8392-448a-bc3a-164ccd10122d service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:08.733275 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 930/1830] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:08.776693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162da667-8efe-4431-bbfb-637a25714964 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:08.781495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162da667-8efe-4431-bbfb-637a25714964 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:08.792570 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 901/1831] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:08.890088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8808a-fa83-4662-b0f1-87978886474f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:08.890736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8808a-fa83-4662-b0f1-87978886474f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:08.891275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8808a-fa83-4662-b0f1-87978886474f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:08.962180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3ac8808a-fa83-4662-b0f1-87978886474f None None] MFA Rules not processed for user `1acd163a2a65407db8846a8878b0c6be`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:08.963445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3ac8808a-fa83-4662-b0f1-87978886474f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:09.031628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ac908-3835-4829-9d5e-8c7600fd31d9 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:09.044965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ac908-3835-4829-9d5e-8c7600fd31d9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:09.044965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ac908-3835-4829-9d5e-8c7600fd31d9 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:09.044965 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 902/1832] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:09.056595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ac908-3835-4829-9d5e-8c7600fd31d9 None None] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:03:09.089673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ac908-3835-4829-9d5e-8c7600fd31d9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:09.145194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ac908-3835-4829-9d5e-8c7600fd31d9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:09.174758 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 931/1833] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:09.179919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8808a-fa83-4662-b0f1-87978886474f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:09.183002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cf6e0-e2c5-4002-ac58-5ae67fde1f29 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d52e97d8e154a7986918ed3918cb4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38bea6ec7e1c4836868191839fd7b889', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:09.190350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cf6e0-e2c5-4002-ac58-5ae67fde1f29 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:09.192170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cf6e0-e2c5-4002-ac58-5ae67fde1f29 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:09.193906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cf6e0-e2c5-4002-ac58-5ae67fde1f29 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] PATH_INFO: `/v3/projects/0d902454e7bd44339240a17aef7d1335` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:09.203041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83cf6e0-e2c5-4002-ac58-5ae67fde1f29 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: Authorizing `identity:get_project(project_id=0d902454e7bd44339240a17aef7d1335)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:09.234483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83cf6e0-e2c5-4002-ac58-5ae67fde1f29 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:09.278051 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 903/1834] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:03:09 2022] GET /identity/v3/projects/0d902454e7bd44339240a17aef7d1335 => generated 395 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:09.425629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac8808a-fa83-4662-b0f1-87978886474f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:09.444000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c16bf-d77f-4ad7-aee7-f72518f30da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:09.444000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c16bf-d77f-4ad7-aee7-f72518f30da5 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:09.444000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c16bf-d77f-4ad7-aee7-f72518f30da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:09.444000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c16bf-d77f-4ad7-aee7-f72518f30da5 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:09.451187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c16bf-d77f-4ad7-aee7-f72518f30da5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:09.459017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c16bf-d77f-4ad7-aee7-f72518f30da5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:09.475707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cf6e0-e2c5-4002-ac58-5ae67fde1f29 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:09.492047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83cf6e0-e2c5-4002-ac58-5ae67fde1f29 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:09.513490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4604c15-a2d5-4196-b576-d866d1430dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:09.546182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4604c15-a2d5-4196-b576-d866d1430dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:09.546395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4604c15-a2d5-4196-b576-d866d1430dac admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:09.546682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4604c15-a2d5-4196-b576-d866d1430dac admin admin] PATH_INFO: `/v3/users/aacbd21e477d4531915ae01e4d5b84bc` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:09.553435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4604c15-a2d5-4196-b576-d866d1430dac admin admin] RBAC: Authorizing `identity:delete_user(user_id=aacbd21e477d4531915ae01e4d5b84bc)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:09.560791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4604c15-a2d5-4196-b576-d866d1430dac admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:09.704016 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 932/1835] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:09 2022] POST /identity/v3/projects => generated 405 bytes in 525 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:09.713165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346b7b5-ef1a-4fc2-9b58-7ee99ba699c9 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:09.726185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346b7b5-ef1a-4fc2-9b58-7ee99ba699c9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:09.729000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346b7b5-ef1a-4fc2-9b58-7ee99ba699c9 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:09.729000 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 933/1836] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:03:09 2022] GET /identity => generated 270 bytes in 23 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:09.729000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346b7b5-ef1a-4fc2-9b58-7ee99ba699c9 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:09.899678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:09.941745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c4604c15-a2d5-4196-b576-d866d1430dac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aacbd21e477d4531915ae01e4d5b84bc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:09.942433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c4604c15-a2d5-4196-b576-d866d1430dac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aacbd21e477d4531915ae01e4d5b84bc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:09.973405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c4604c15-a2d5-4196-b576-d866d1430dac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aacbd21e477d4531915ae01e4d5b84bc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:10.022950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346b7b5-ef1a-4fc2-9b58-7ee99ba699c9 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:10.028487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c4604c15-a2d5-4196-b576-d866d1430dac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aacbd21e477d4531915ae01e4d5b84bc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:10.029406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c4604c15-a2d5-4196-b576-d866d1430dac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aacbd21e477d4531915ae01e4d5b84bc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:10.047121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35be01f-86d5-4bd3-8d4c-bce995150d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:10.048076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35be01f-86d5-4bd3-8d4c-bce995150d87 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:10.054029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35be01f-86d5-4bd3-8d4c-bce995150d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:10.054379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35be01f-86d5-4bd3-8d4c-bce995150d87 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:10.055428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35be01f-86d5-4bd3-8d4c-bce995150d87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:10.059852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35be01f-86d5-4bd3-8d4c-bce995150d87 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:10.088171 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 904/1837] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:09 2022] DELETE /identity/v3/users/aacbd21e477d4531915ae01e4d5b84bc => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:10.089295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4604c15-a2d5-4196-b576-d866d1430dac admin admin] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:03:10.094935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c4604c15-a2d5-4196-b576-d866d1430dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:10.192937 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 934/1838] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:09 2022] POST /identity/v3/users => generated 309 bytes in 464 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:10.195228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6da533-4872-41fe-9d79-e92901293db6 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:10.195770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6da533-4872-41fe-9d79-e92901293db6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:10.196125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6da533-4872-41fe-9d79-e92901293db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:10.265719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8d6da533-4872-41fe-9d79-e92901293db6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:10.267206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8d6da533-4872-41fe-9d79-e92901293db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:10.377742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4604c15-a2d5-4196-b576-d866d1430dac admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:10.382207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c4604c15-a2d5-4196-b576-d866d1430dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:10.429997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:10.448405 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 935/1839] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:10.449530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6da533-4872-41fe-9d79-e92901293db6 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:10.453543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8d6da533-4872-41fe-9d79-e92901293db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:10.500246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6da533-4872-41fe-9d79-e92901293db6 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:10.534624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01456a31-eb8d-4a3e-b17e-938ddb2ebce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:10.535482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01456a31-eb8d-4a3e-b17e-938ddb2ebce0 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:10.535871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01456a31-eb8d-4a3e-b17e-938ddb2ebce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:10.536253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01456a31-eb8d-4a3e-b17e-938ddb2ebce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:10.537117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01456a31-eb8d-4a3e-b17e-938ddb2ebce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:10.540342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01456a31-eb8d-4a3e-b17e-938ddb2ebce0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:10.568574 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 936/1840] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:10.570813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0884b1-246f-4915-b71b-10413f31ebd6 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:10.571389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0884b1-246f-4915-b71b-10413f31ebd6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:10.572279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0884b1-246f-4915-b71b-10413f31ebd6 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:10.576698 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 937/1841] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:03:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:10.578910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0884b1-246f-4915-b71b-10413f31ebd6 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:10.583760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4a0884b1-246f-4915-b71b-10413f31ebd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:10.624046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4604c15-a2d5-4196-b576-d866d1430dac admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:10.641558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222decc-850f-4f95-911c-56d1d5ff5305 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d52e97d8e154a7986918ed3918cb4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38bea6ec7e1c4836868191839fd7b889', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:10.654217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222decc-850f-4f95-911c-56d1d5ff5305 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:10.654663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222decc-850f-4f95-911c-56d1d5ff5305 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:10.655212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222decc-850f-4f95-911c-56d1d5ff5305 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] PATH_INFO: `/v3/projects/0d902454e7bd44339240a17aef7d1335` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:10.659363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222decc-850f-4f95-911c-56d1d5ff5305 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: Authorizing `identity:get_project(project_id=0d902454e7bd44339240a17aef7d1335)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:10.668407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a222decc-850f-4f95-911c-56d1d5ff5305 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:10.682776 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 905/1842] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:03:10 2022] GET /identity/v3/projects/0d902454e7bd44339240a17aef7d1335 => generated 395 bytes in 593 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:10.686315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222decc-850f-4f95-911c-56d1d5ff5305 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:10.734918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222decc-850f-4f95-911c-56d1d5ff5305 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:10.752893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec2276-9380-413a-ad19-361ae3b5181d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:10.756428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec2276-9380-413a-ad19-361ae3b5181d admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:10.770997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec2276-9380-413a-ad19-361ae3b5181d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:10.772340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec2276-9380-413a-ad19-361ae3b5181d admin admin] PATH_INFO: `/v3/projects/44391556bd3c44828209cd1aa4fb3fa2/users/5555f81ef14241f99c3592fb5632ec15/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:10.810382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec2276-9380-413a-ad19-361ae3b5181d admin admin] RBAC: Authorizing `identity:create_grant(project_id=44391556bd3c44828209cd1aa4fb3fa2, user_id=5555f81ef14241f99c3592fb5632ec15, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:10.814359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec2276-9380-413a-ad19-361ae3b5181d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:10.814599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0884b1-246f-4915-b71b-10413f31ebd6 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:10.838851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d9f6c7-8dbe-483a-855d-2142e0a6abd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:10.841700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9f6c7-8dbe-483a-855d-2142e0a6abd3 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:10.841700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9f6c7-8dbe-483a-855d-2142e0a6abd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:10.841700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d9f6c7-8dbe-483a-855d-2142e0a6abd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:10.867405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9f6c7-8dbe-483a-855d-2142e0a6abd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:10.869211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d9f6c7-8dbe-483a-855d-2142e0a6abd3 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:10.903532 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 938/1843] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:10.905282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549159d-5e01-4b08-aa34-4e73937bea32 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:10.907868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549159d-5e01-4b08-aa34-4e73937bea32 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:10.908401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549159d-5e01-4b08-aa34-4e73937bea32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:10.931199 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 906/1844] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:10 2022] PUT /identity/v3/projects/44391556bd3c44828209cd1aa4fb3fa2/users/5555f81ef14241f99c3592fb5632ec15/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:10.939516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec2276-9380-413a-ad19-361ae3b5181d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:10.965154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6549159d-5e01-4b08-aa34-4e73937bea32 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:10.968061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6549159d-5e01-4b08-aa34-4e73937bea32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:11.091733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:11.094925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec2276-9380-413a-ad19-361ae3b5181d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:11.110462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eae09a-8fcf-4621-84d6-c4d9222df0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:11.111197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eae09a-8fcf-4621-84d6-c4d9222df0a9 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:11.111533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eae09a-8fcf-4621-84d6-c4d9222df0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:11.111814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eae09a-8fcf-4621-84d6-c4d9222df0a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:11.112567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eae09a-8fcf-4621-84d6-c4d9222df0a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:11.114267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eae09a-8fcf-4621-84d6-c4d9222df0a9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:11.137568 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 907/1845] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:03:11.141377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eae09a-8fcf-4621-84d6-c4d9222df0a9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:11.144490 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 939/1846] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:03:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:11.145336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549159d-5e01-4b08-aa34-4e73937bea32 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:11.147763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f5eae09a-8fcf-4621-84d6-c4d9222df0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:11.171599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549159d-5e01-4b08-aa34-4e73937bea32 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:11.177854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eae09a-8fcf-4621-84d6-c4d9222df0a9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:11.183916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2822d-e0f4-40d4-b02e-74f54fc3a6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:11.196500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2822d-e0f4-40d4-b02e-74f54fc3a6d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:11.197132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2822d-e0f4-40d4-b02e-74f54fc3a6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:11.197505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc2822d-e0f4-40d4-b02e-74f54fc3a6d1 admin admin] PATH_INFO: `/v3/projects/44391556bd3c44828209cd1aa4fb3fa2/users/5555f81ef14241f99c3592fb5632ec15/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:11.202246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc2822d-e0f4-40d4-b02e-74f54fc3a6d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44391556bd3c44828209cd1aa4fb3fa2, user_id=5555f81ef14241f99c3592fb5632ec15, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:11.204280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc47a0a-0a4a-49aa-96f4-a753f2a13738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:11.205831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc47a0a-0a4a-49aa-96f4-a753f2a13738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:11.207234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc47a0a-0a4a-49aa-96f4-a753f2a13738 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:11.207234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc47a0a-0a4a-49aa-96f4-a753f2a13738 admin admin] PATH_INFO: `/v3/users/979c95b05b59471a954ca8a9097c8660` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:11.208879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc2822d-e0f4-40d4-b02e-74f54fc3a6d1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:11.210538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc47a0a-0a4a-49aa-96f4-a753f2a13738 admin admin] RBAC: Authorizing `identity:delete_user(user_id=979c95b05b59471a954ca8a9097c8660)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:11.214951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc47a0a-0a4a-49aa-96f4-a753f2a13738 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:11.250433 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 940/1847] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:11 2022] PUT /identity/v3/projects/44391556bd3c44828209cd1aa4fb3fa2/users/5555f81ef14241f99c3592fb5632ec15/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:11.302872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2822d-e0f4-40d4-b02e-74f54fc3a6d1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:11.306728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bbc2822d-e0f4-40d4-b02e-74f54fc3a6d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:11.463414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc2822d-e0f4-40d4-b02e-74f54fc3a6d1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:11.471859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e11cf7-a026-479f-8f22-0a45a0e498a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:11.472726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e11cf7-a026-479f-8f22-0a45a0e498a8 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:11.473020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e11cf7-a026-479f-8f22-0a45a0e498a8 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:11.473307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e11cf7-a026-479f-8f22-0a45a0e498a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:11.473415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3fc47a0a-0a4a-49aa-96f4-a753f2a13738 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '979c95b05b59471a954ca8a9097c8660'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:11.476645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3fc47a0a-0a4a-49aa-96f4-a753f2a13738 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '979c95b05b59471a954ca8a9097c8660'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:11.494701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3fc47a0a-0a4a-49aa-96f4-a753f2a13738 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '979c95b05b59471a954ca8a9097c8660'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:11.497239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e11cf7-a026-479f-8f22-0a45a0e498a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:11.502818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e11cf7-a026-479f-8f22-0a45a0e498a8 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:11.522239 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 941/1848] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:11.601041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3fc47a0a-0a4a-49aa-96f4-a753f2a13738 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '979c95b05b59471a954ca8a9097c8660'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:11.601870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3fc47a0a-0a4a-49aa-96f4-a753f2a13738 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '979c95b05b59471a954ca8a9097c8660'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:11.644757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:11.692497 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 908/1849] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:11 2022] DELETE /identity/v3/users/979c95b05b59471a954ca8a9097c8660 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:11.697499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62fddf-2e70-4633-a2cf-6fc2d173a078 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:11.700010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62fddf-2e70-4633-a2cf-6fc2d173a078 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:11.700010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e62fddf-2e70-4633-a2cf-6fc2d173a078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:11.759837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5e62fddf-2e70-4633-a2cf-6fc2d173a078 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:11.761510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5e62fddf-2e70-4633-a2cf-6fc2d173a078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:11.775233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc47a0a-0a4a-49aa-96f4-a753f2a13738 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:11.780754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3fc47a0a-0a4a-49aa-96f4-a753f2a13738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:11.920619 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 942/1850] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:12.010823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc47a0a-0a4a-49aa-96f4-a753f2a13738 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:12.019697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6e751-f2fa-491f-9c72-fbff4da19df7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:12.020629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6e751-f2fa-491f-9c72-fbff4da19df7 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:12.021126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6e751-f2fa-491f-9c72-fbff4da19df7 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:12.022570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6e751-f2fa-491f-9c72-fbff4da19df7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:12.031588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0dc6e751-f2fa-491f-9c72-fbff4da19df7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:12.087739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62fddf-2e70-4633-a2cf-6fc2d173a078 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:12.092156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5e62fddf-2e70-4633-a2cf-6fc2d173a078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:12.218215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6e751-f2fa-491f-9c72-fbff4da19df7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:12.220849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6e751-f2fa-491f-9c72-fbff4da19df7 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:12.235365 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 909/1851] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:12.261320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e62fddf-2e70-4633-a2cf-6fc2d173a078 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:12.266880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23607d3-ebda-4016-a3ca-648dba6a138e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:12.273433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23607d3-ebda-4016-a3ca-648dba6a138e service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:12.273873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23607d3-ebda-4016-a3ca-648dba6a138e service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:12.274563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23607d3-ebda-4016-a3ca-648dba6a138e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:12.289214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23607d3-ebda-4016-a3ca-648dba6a138e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:12.290309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23607d3-ebda-4016-a3ca-648dba6a138e service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:12.299145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:12.318170 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 943/1852] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:12.969234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c85804-0f50-4043-984d-8fdb434da7c8 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:12.969631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c85804-0f50-4043-984d-8fdb434da7c8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:12.969999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c85804-0f50-4043-984d-8fdb434da7c8 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:12.971525 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 910/1853] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:03:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:12.983769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23607d3-ebda-4016-a3ca-648dba6a138e service neutron] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:03:12.990335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d23607d3-ebda-4016-a3ca-648dba6a138e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:13.035904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23607d3-ebda-4016-a3ca-648dba6a138e service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:13.037248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d23607d3-ebda-4016-a3ca-648dba6a138e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:13.261188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23607d3-ebda-4016-a3ca-648dba6a138e service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:13.270565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2845ba-450b-4743-85e1-9d1f5eda2b74 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d52e97d8e154a7986918ed3918cb4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38bea6ec7e1c4836868191839fd7b889', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:13.275121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2845ba-450b-4743-85e1-9d1f5eda2b74 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:13.279015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2845ba-450b-4743-85e1-9d1f5eda2b74 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:13.279337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2845ba-450b-4743-85e1-9d1f5eda2b74 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] PATH_INFO: `/v3/projects/0d902454e7bd44339240a17aef7d1335` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:13.281071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2845ba-450b-4743-85e1-9d1f5eda2b74 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: Authorizing `identity:get_project(project_id=0d902454e7bd44339240a17aef7d1335)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:13.284124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c117129-b2e7-4e18-8885-13ad719c1ce2 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:13.285020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c117129-b2e7-4e18-8885-13ad719c1ce2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:13.285495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c117129-b2e7-4e18-8885-13ad719c1ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:13.288866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2845ba-450b-4743-85e1-9d1f5eda2b74 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:13.320083 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 944/1854] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:03:12 2022] GET /identity/v3/projects/0d902454e7bd44339240a17aef7d1335 => generated 395 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:13.326914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0c117129-b2e7-4e18-8885-13ad719c1ce2 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:13.331436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0c117129-b2e7-4e18-8885-13ad719c1ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:13.332733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2845ba-450b-4743-85e1-9d1f5eda2b74 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:13.341537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ff2845ba-450b-4743-85e1-9d1f5eda2b74 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:13.394427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2845ba-450b-4743-85e1-9d1f5eda2b74 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:13.401250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648fbe1f-a781-46ba-9551-f74b1d3cd54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:13.402676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fbe1f-a781-46ba-9551-f74b1d3cd54b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:13.402954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fbe1f-a781-46ba-9551-f74b1d3cd54b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:13.403265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648fbe1f-a781-46ba-9551-f74b1d3cd54b admin admin] PATH_INFO: `/v3/projects/de28695bc5a94ae8b467fb876daa5f47` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:13.415156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648fbe1f-a781-46ba-9551-f74b1d3cd54b admin admin] RBAC: Authorizing `identity:delete_project(project_id=de28695bc5a94ae8b467fb876daa5f47)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:13.417177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648fbe1f-a781-46ba-9551-f74b1d3cd54b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:13.535818 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 911/1855] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:03:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 28 21:03:13.537130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cc950-0146-47f3-9812-4e5709eb3f82 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:13.538005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cc950-0146-47f3-9812-4e5709eb3f82 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:13.538317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9cc950-0146-47f3-9812-4e5709eb3f82 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:13.547139 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 912/1856] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:03:13 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:13.558959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9cc950-0146-47f3-9812-4e5709eb3f82 None None] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:03:13.626248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-648fbe1f-a781-46ba-9551-f74b1d3cd54b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de28695bc5a94ae8b467fb876daa5f47', 'request_id': 'req-648fbe1f-a781-46ba-9551-f74b1d3cd54b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:13.628479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9cc950-0146-47f3-9812-4e5709eb3f82 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:13.652064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-648fbe1f-a781-46ba-9551-f74b1d3cd54b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de28695bc5a94ae8b467fb876daa5f47', 'request_id': 'req-648fbe1f-a781-46ba-9551-f74b1d3cd54b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:13.670689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9cc950-0146-47f3-9812-4e5709eb3f82 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:13.725713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-648fbe1f-a781-46ba-9551-f74b1d3cd54b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de28695bc5a94ae8b467fb876daa5f47', 'request_id': 'req-648fbe1f-a781-46ba-9551-f74b1d3cd54b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:13.747480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-648fbe1f-a781-46ba-9551-f74b1d3cd54b admin admin] The token cache is being invalidate because project de28695bc5a94ae8b467fb876daa5f47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:13.747653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-648fbe1f-a781-46ba-9551-f74b1d3cd54b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:13.749248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-648fbe1f-a781-46ba-9551-f74b1d3cd54b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:13.759092 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 945/1857] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:13 2022] DELETE /identity/v3/projects/de28695bc5a94ae8b467fb876daa5f47 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:13.759784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648fbe1f-a781-46ba-9551-f74b1d3cd54b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:13.776861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-648fbe1f-a781-46ba-9551-f74b1d3cd54b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:13.809250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:13.905732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade62f1a-87f5-4627-a001-ad65fa3c387e tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d52e97d8e154a7986918ed3918cb4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38bea6ec7e1c4836868191839fd7b889', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:13.906928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade62f1a-87f5-4627-a001-ad65fa3c387e tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:13.907411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade62f1a-87f5-4627-a001-ad65fa3c387e tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:13.907834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade62f1a-87f5-4627-a001-ad65fa3c387e tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] PATH_INFO: `/v3/projects/0d902454e7bd44339240a17aef7d1335` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:13.914849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade62f1a-87f5-4627-a001-ad65fa3c387e tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: Authorizing `identity:get_project(project_id=0d902454e7bd44339240a17aef7d1335)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:13.917308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade62f1a-87f5-4627-a001-ad65fa3c387e tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:13.923670 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 913/1858] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:03:13 2022] GET /identity/v3/projects/0d902454e7bd44339240a17aef7d1335 => generated 395 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:03:13.924798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade62f1a-87f5-4627-a001-ad65fa3c387e tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:13.930720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ade62f1a-87f5-4627-a001-ad65fa3c387e tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:14.023118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648fbe1f-a781-46ba-9551-f74b1d3cd54b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:14.036846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d35242-defd-4201-819b-f6f574c7a14e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:14.038177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d35242-defd-4201-819b-f6f574c7a14e service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:14.038454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d35242-defd-4201-819b-f6f574c7a14e service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:14.038689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d35242-defd-4201-819b-f6f574c7a14e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:14.046158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-39d35242-defd-4201-819b-f6f574c7a14e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:14.097397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:14.122693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade62f1a-87f5-4627-a001-ad65fa3c387e tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:14.139066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b48dc-8030-4945-911c-eb5a453e5d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:14.140111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b48dc-8030-4945-911c-eb5a453e5d34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:14.140406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b48dc-8030-4945-911c-eb5a453e5d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:14.140684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4b48dc-8030-4945-911c-eb5a453e5d34 admin admin] PATH_INFO: `/v3/users/e87af1dda77942fcb376c392c8242b70` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:14.142601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b48dc-8030-4945-911c-eb5a453e5d34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e87af1dda77942fcb376c392c8242b70)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:14.147343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4b48dc-8030-4945-911c-eb5a453e5d34 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:14.184516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d35242-defd-4201-819b-f6f574c7a14e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:14.189737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d35242-defd-4201-819b-f6f574c7a14e service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:14.210199 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 946/1859] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:03:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:03:14.465717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec4b48dc-8030-4945-911c-eb5a453e5d34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e87af1dda77942fcb376c392c8242b70'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:14.465717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec4b48dc-8030-4945-911c-eb5a453e5d34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e87af1dda77942fcb376c392c8242b70'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:14.472819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec4b48dc-8030-4945-911c-eb5a453e5d34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e87af1dda77942fcb376c392c8242b70'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:14.582786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec4b48dc-8030-4945-911c-eb5a453e5d34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e87af1dda77942fcb376c392c8242b70'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:14.583566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec4b48dc-8030-4945-911c-eb5a453e5d34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e87af1dda77942fcb376c392c8242b70'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:14.629816 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 914/1860] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:13 2022] DELETE /identity/v3/users/e87af1dda77942fcb376c392c8242b70 => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:14.672206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea21de-32ac-4afb-938b-a2b8032d51af None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:14.672783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea21de-32ac-4afb-938b-a2b8032d51af None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:14.673881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea21de-32ac-4afb-938b-a2b8032d51af None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:14.675260 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 947/1861] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:03:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:14.684933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b48dc-8030-4945-911c-eb5a453e5d34 admin admin] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:03:14.686658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ec4b48dc-8030-4945-911c-eb5a453e5d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:14.697722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea21de-32ac-4afb-938b-a2b8032d51af None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:14.699984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b9ea21de-32ac-4afb-938b-a2b8032d51af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:14.899139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:15.006070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea21de-32ac-4afb-938b-a2b8032d51af None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:15.051978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de7bf9-c91e-40b8-abeb-47415d67e38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:15.057991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b48dc-8030-4945-911c-eb5a453e5d34 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:15.059652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de7bf9-c91e-40b8-abeb-47415d67e38b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:15.065044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de7bf9-c91e-40b8-abeb-47415d67e38b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:15.069760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de7bf9-c91e-40b8-abeb-47415d67e38b admin admin] PATH_INFO: `/v3/projects/657641a7c7a84eba962742035e6c2640` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:15.089963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de7bf9-c91e-40b8-abeb-47415d67e38b admin admin] RBAC: Authorizing `identity:delete_project(project_id=657641a7c7a84eba962742035e6c2640)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:15.093481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ec4b48dc-8030-4945-911c-eb5a453e5d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:15.099163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de7bf9-c91e-40b8-abeb-47415d67e38b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:15.271363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4b48dc-8030-4945-911c-eb5a453e5d34 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:15.285881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec3f26-b4fc-4170-a55d-63578393a6c1 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4d52e97d8e154a7986918ed3918cb4e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38bea6ec7e1c4836868191839fd7b889', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:15.287318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec3f26-b4fc-4170-a55d-63578393a6c1 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:15.287862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec3f26-b4fc-4170-a55d-63578393a6c1 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:15.288353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec3f26-b4fc-4170-a55d-63578393a6c1 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] PATH_INFO: `/v3/projects/751501c7dfad4ad0a00524c6baf4471d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:15.290096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec3f26-b4fc-4170-a55d-63578393a6c1 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: Authorizing `identity:get_project(project_id=751501c7dfad4ad0a00524c6baf4471d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:15.294973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec3f26-b4fc-4170-a55d-63578393a6c1 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:15.306512 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 915/1862] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:03:14 2022] GET /identity/v3/projects/751501c7dfad4ad0a00524c6baf4471d => generated 395 bytes in 616 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:15.370824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-18de7bf9-c91e-40b8-abeb-47415d67e38b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '657641a7c7a84eba962742035e6c2640', 'request_id': 'req-18de7bf9-c91e-40b8-abeb-47415d67e38b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:15.390359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-18de7bf9-c91e-40b8-abeb-47415d67e38b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '657641a7c7a84eba962742035e6c2640', 'request_id': 'req-18de7bf9-c91e-40b8-abeb-47415d67e38b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:15.474949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-18de7bf9-c91e-40b8-abeb-47415d67e38b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '657641a7c7a84eba962742035e6c2640', 'request_id': 'req-18de7bf9-c91e-40b8-abeb-47415d67e38b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:15.509532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-18de7bf9-c91e-40b8-abeb-47415d67e38b admin admin] The token cache is being invalidate because project 657641a7c7a84eba962742035e6c2640 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:15.512470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-18de7bf9-c91e-40b8-abeb-47415d67e38b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:15.513016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-18de7bf9-c91e-40b8-abeb-47415d67e38b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:15.521367 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 948/1863] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:14 2022] DELETE /identity/v3/projects/657641a7c7a84eba962742035e6c2640 => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:03:15.526102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec3f26-b4fc-4170-a55d-63578393a6c1 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:15.528825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-33ec3f26-b4fc-4170-a55d-63578393a6c1 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:15.539764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de7bf9-c91e-40b8-abeb-47415d67e38b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:15.547297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-18de7bf9-c91e-40b8-abeb-47415d67e38b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:15.571547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec3f26-b4fc-4170-a55d-63578393a6c1 tempest-FlavorsAccessNegativeTestJSON-1342488242 tempest-FlavorsAccessNegativeTestJSON-1342488242-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:15.672031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:15.705427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de7bf9-c91e-40b8-abeb-47415d67e38b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:15.719744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c56be3-17c9-4773-acd2-708104ae864d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:15.720926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c56be3-17c9-4773-acd2-708104ae864d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:15.721410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c56be3-17c9-4773-acd2-708104ae864d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:15.721895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c56be3-17c9-4773-acd2-708104ae864d admin admin] PATH_INFO: `/v3/users/13e536776b8f4bd99fbf6a7b5f02fc79` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:15.723345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c56be3-17c9-4773-acd2-708104ae864d admin admin] RBAC: Authorizing `identity:delete_user(user_id=13e536776b8f4bd99fbf6a7b5f02fc79)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:15.726494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c56be3-17c9-4773-acd2-708104ae864d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:15.828076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0846be7-e867-4aca-a5a4-ac21119607c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:15.830865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0846be7-e867-4aca-a5a4-ac21119607c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:15.831344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0846be7-e867-4aca-a5a4-ac21119607c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:15.832414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0846be7-e867-4aca-a5a4-ac21119607c9 admin admin] PATH_INFO: `/v3/users/e7af0c086a7140c3a8a1f4ad142815ea` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:15.839254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0846be7-e867-4aca-a5a4-ac21119607c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7af0c086a7140c3a8a1f4ad142815ea)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:15.845412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0846be7-e867-4aca-a5a4-ac21119607c9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:16.045413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e5c56be3-17c9-4773-acd2-708104ae864d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13e536776b8f4bd99fbf6a7b5f02fc79'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:16.077616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e5c56be3-17c9-4773-acd2-708104ae864d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13e536776b8f4bd99fbf6a7b5f02fc79'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:16.090504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e5c56be3-17c9-4773-acd2-708104ae864d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13e536776b8f4bd99fbf6a7b5f02fc79'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:16.093816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e5c56be3-17c9-4773-acd2-708104ae864d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13e536776b8f4bd99fbf6a7b5f02fc79'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:16.161365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d0846be7-e867-4aca-a5a4-ac21119607c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7af0c086a7140c3a8a1f4ad142815ea'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:16.164197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d0846be7-e867-4aca-a5a4-ac21119607c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7af0c086a7140c3a8a1f4ad142815ea'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:16.190743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d0846be7-e867-4aca-a5a4-ac21119607c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7af0c086a7140c3a8a1f4ad142815ea'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:16.217459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e5c56be3-17c9-4773-acd2-708104ae864d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13e536776b8f4bd99fbf6a7b5f02fc79'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:16.284275 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 949/1864] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:15 2022] DELETE /identity/v3/users/13e536776b8f4bd99fbf6a7b5f02fc79 => generated 0 bytes in 757 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:16.289585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d0846be7-e867-4aca-a5a4-ac21119607c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7af0c086a7140c3a8a1f4ad142815ea'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:16.290303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d0846be7-e867-4aca-a5a4-ac21119607c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7af0c086a7140c3a8a1f4ad142815ea'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:16.291311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e11d3-8351-4d64-8e2d-6d78205b19d8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:16.292152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e11d3-8351-4d64-8e2d-6d78205b19d8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:16.292627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e11d3-8351-4d64-8e2d-6d78205b19d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:16.349095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-326e11d3-8351-4d64-8e2d-6d78205b19d8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:16.351225 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 916/1865] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:15 2022] DELETE /identity/v3/users/e7af0c086a7140c3a8a1f4ad142815ea => generated 0 bytes in 826 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:16.352871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-326e11d3-8351-4d64-8e2d-6d78205b19d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:16.453446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7492425-8b86-48dd-8bce-18b1033d4ece None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:16.453446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7492425-8b86-48dd-8bce-18b1033d4ece None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:16.453446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7492425-8b86-48dd-8bce-18b1033d4ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:16.501590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:16.510754 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 950/1866] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:16.532050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c7492425-8b86-48dd-8bce-18b1033d4ece None None] MFA Rules not processed for user `5555f81ef14241f99c3592fb5632ec15`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:16.533543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c7492425-8b86-48dd-8bce-18b1033d4ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:16.642869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e11d3-8351-4d64-8e2d-6d78205b19d8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:16.644908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-326e11d3-8351-4d64-8e2d-6d78205b19d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:16.734079 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 917/1867] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:16 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:03:16.829733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e11d3-8351-4d64-8e2d-6d78205b19d8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:16.838752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888dec8f-cae7-4ce1-8d7d-2ddb60ea87aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:16.839430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dec8f-cae7-4ce1-8d7d-2ddb60ea87aa service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:16.840819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dec8f-cae7-4ce1-8d7d-2ddb60ea87aa service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:16.840819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dec8f-cae7-4ce1-8d7d-2ddb60ea87aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:16.853060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888dec8f-cae7-4ce1-8d7d-2ddb60ea87aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:16.855298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888dec8f-cae7-4ce1-8d7d-2ddb60ea87aa service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:16.879865 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 951/1868] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:03:17.016353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b1322-2cbb-446b-b052-90894b436f44 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:17.020180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b1322-2cbb-446b-b052-90894b436f44 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:17.022291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6b1322-2cbb-446b-b052-90894b436f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:17.096213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2e6b1322-2cbb-446b-b052-90894b436f44 None None] MFA Rules not processed for user `1b2a9ebcd3f84b758322a80c6e590de0`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:17.101100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2e6b1322-2cbb-446b-b052-90894b436f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:17.116966 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 918/1869] 10.222.0.15 () {58 vars in 1015 bytes} [Sat May 28 21:03:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:17.161914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888dec8f-cae7-4ce1-8d7d-2ddb60ea87aa service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:17.175018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888dec8f-cae7-4ce1-8d7d-2ddb60ea87aa service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:17.188362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b1284-9b02-4cc6-82a1-196117a9e7f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:17.189804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b1284-9b02-4cc6-82a1-196117a9e7f2 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:17.189804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b1284-9b02-4cc6-82a1-196117a9e7f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:17.189804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b1284-9b02-4cc6-82a1-196117a9e7f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:17.206832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b1284-9b02-4cc6-82a1-196117a9e7f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:17.208409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4b1284-9b02-4cc6-82a1-196117a9e7f2 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:17.215793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:17.223045 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 952/1870] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:17.293804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b1322-2cbb-446b-b052-90894b436f44 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:17.296206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2e6b1322-2cbb-446b-b052-90894b436f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:17.318908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6b1322-2cbb-446b-b052-90894b436f44 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:17.326493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac853857-7146-4c2c-a04d-83a439417220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:17.327772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac853857-7146-4c2c-a04d-83a439417220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:17.328175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac853857-7146-4c2c-a04d-83a439417220 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:17.328922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac853857-7146-4c2c-a04d-83a439417220 admin admin] PATH_INFO: `/v3/projects/9077c3b2a44a44d1b8c81f3bb054c451` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:17.331225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac853857-7146-4c2c-a04d-83a439417220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9077c3b2a44a44d1b8c81f3bb054c451)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:17.334821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac853857-7146-4c2c-a04d-83a439417220 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:17.562834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ac853857-7146-4c2c-a04d-83a439417220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9077c3b2a44a44d1b8c81f3bb054c451', 'request_id': 'req-ac853857-7146-4c2c-a04d-83a439417220'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:17.590570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ac853857-7146-4c2c-a04d-83a439417220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9077c3b2a44a44d1b8c81f3bb054c451', 'request_id': 'req-ac853857-7146-4c2c-a04d-83a439417220'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:17.714674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ac853857-7146-4c2c-a04d-83a439417220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9077c3b2a44a44d1b8c81f3bb054c451', 'request_id': 'req-ac853857-7146-4c2c-a04d-83a439417220'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:17.736006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ac853857-7146-4c2c-a04d-83a439417220 admin admin] The token cache is being invalidate because project 9077c3b2a44a44d1b8c81f3bb054c451 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:17.736389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ac853857-7146-4c2c-a04d-83a439417220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:17.737795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ac853857-7146-4c2c-a04d-83a439417220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:17.740738 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 919/1871] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:17 2022] DELETE /identity/v3/projects/9077c3b2a44a44d1b8c81f3bb054c451 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:17.745915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b50185-203f-4628-8dd9-07f422caf409 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:17.746508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b50185-203f-4628-8dd9-07f422caf409 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:17.746946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b50185-203f-4628-8dd9-07f422caf409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:17.778538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e4b50185-203f-4628-8dd9-07f422caf409 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:17.779799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e4b50185-203f-4628-8dd9-07f422caf409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:17.956704 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 953/1872] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:17.964348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d524b2-a5b6-423c-a268-6a6c28764d13 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:17.964672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d524b2-a5b6-423c-a268-6a6c28764d13 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:17.964926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d524b2-a5b6-423c-a268-6a6c28764d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:18.000612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e8d524b2-a5b6-423c-a268-6a6c28764d13 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:18.001877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e8d524b2-a5b6-423c-a268-6a6c28764d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:18.021207 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 920/1873] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:18.026193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b50185-203f-4628-8dd9-07f422caf409 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:18.042911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b50185-203f-4628-8dd9-07f422caf409 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:18.056909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202ab2b-ec43-4963-aedd-3e5cff387214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:18.057868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202ab2b-ec43-4963-aedd-3e5cff387214 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:18.058150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202ab2b-ec43-4963-aedd-3e5cff387214 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:18.058437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202ab2b-ec43-4963-aedd-3e5cff387214 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:18.059205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202ab2b-ec43-4963-aedd-3e5cff387214 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:03:18.059521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202ab2b-ec43-4963-aedd-3e5cff387214 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:18.061189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202ab2b-ec43-4963-aedd-3e5cff387214 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:18.091349 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 954/1874] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:18.096970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d524b2-a5b6-423c-a268-6a6c28764d13 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:18.115131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d524b2-a5b6-423c-a268-6a6c28764d13 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:18.124684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529d835-2f01-43e9-87e1-1c847ec1eccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:18.125378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529d835-2f01-43e9-87e1-1c847ec1eccf admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:18.125667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529d835-2f01-43e9-87e1-1c847ec1eccf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:18.125971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529d835-2f01-43e9-87e1-1c847ec1eccf admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:18.126971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529d835-2f01-43e9-87e1-1c847ec1eccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:18.128670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529d835-2f01-43e9-87e1-1c847ec1eccf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:18.300831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202ab2b-ec43-4963-aedd-3e5cff387214 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:18.303726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8202ab2b-ec43-4963-aedd-3e5cff387214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:18.331527 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 921/1875] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:18 2022] POST /identity/v3/projects => generated 393 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:18.335597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202ab2b-ec43-4963-aedd-3e5cff387214 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:18.337277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529d835-2f01-43e9-87e1-1c847ec1eccf admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:18.512886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:18.553386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529d835-2f01-43e9-87e1-1c847ec1eccf admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:18.575638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6f6b6-edba-4581-8a81-d39204cb4ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:18.576271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6f6b6-edba-4581-8a81-d39204cb4ffb admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:18.576599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6f6b6-edba-4581-8a81-d39204cb4ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:18.576971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b6f6b6-edba-4581-8a81-d39204cb4ffb admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:18.577582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b6f6b6-edba-4581-8a81-d39204cb4ffb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:18.579620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b6f6b6-edba-4581-8a81-d39204cb4ffb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:18.595528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015ae53-4e89-4222-b8af-313105508875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:18.596680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015ae53-4e89-4222-b8af-313105508875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:18.596951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015ae53-4e89-4222-b8af-313105508875 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:18.597216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015ae53-4e89-4222-b8af-313105508875 admin admin] PATH_INFO: `/v3/projects/d2fdef2647da40969db6927ef81349a6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:18.599595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015ae53-4e89-4222-b8af-313105508875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2fdef2647da40969db6927ef81349a6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:18.605528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015ae53-4e89-4222-b8af-313105508875 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:18.694765 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 922/1876] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:18 2022] POST /identity/v3/users => generated 297 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:18.695746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6f6b6-edba-4581-8a81-d39204cb4ffb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:18.702868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-66b6f6b6-edba-4581-8a81-d39204cb4ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:18.739686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b6f6b6-edba-4581-8a81-d39204cb4ffb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:18.759977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba46701f-9ff0-4f77-a27c-5e21db77c4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:18.760900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46701f-9ff0-4f77-a27c-5e21db77c4a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:18.761255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46701f-9ff0-4f77-a27c-5e21db77c4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:18.761415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba46701f-9ff0-4f77-a27c-5e21db77c4a8 admin admin] PATH_INFO: `/v3/projects/0d902454e7bd44339240a17aef7d1335` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:18.762945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46701f-9ff0-4f77-a27c-5e21db77c4a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d902454e7bd44339240a17aef7d1335)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:18.765073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba46701f-9ff0-4f77-a27c-5e21db77c4a8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:18.822972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6015ae53-4e89-4222-b8af-313105508875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2fdef2647da40969db6927ef81349a6', 'request_id': 'req-6015ae53-4e89-4222-b8af-313105508875'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:18.839145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6015ae53-4e89-4222-b8af-313105508875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2fdef2647da40969db6927ef81349a6', 'request_id': 'req-6015ae53-4e89-4222-b8af-313105508875'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:18.920060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6015ae53-4e89-4222-b8af-313105508875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2fdef2647da40969db6927ef81349a6', 'request_id': 'req-6015ae53-4e89-4222-b8af-313105508875'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:18.948360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ba46701f-9ff0-4f77-a27c-5e21db77c4a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d902454e7bd44339240a17aef7d1335', 'request_id': 'req-ba46701f-9ff0-4f77-a27c-5e21db77c4a8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:18.953959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6015ae53-4e89-4222-b8af-313105508875 admin admin] The token cache is being invalidate because project d2fdef2647da40969db6927ef81349a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:18.954271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6015ae53-4e89-4222-b8af-313105508875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:18.957271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6015ae53-4e89-4222-b8af-313105508875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:18.965506 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 955/1877] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:18 2022] DELETE /identity/v3/projects/d2fdef2647da40969db6927ef81349a6 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:18.966286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015ae53-4e89-4222-b8af-313105508875 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:18.968874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6015ae53-4e89-4222-b8af-313105508875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:18.969607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ba46701f-9ff0-4f77-a27c-5e21db77c4a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d902454e7bd44339240a17aef7d1335', 'request_id': 'req-ba46701f-9ff0-4f77-a27c-5e21db77c4a8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:19.015286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:19.057413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ba46701f-9ff0-4f77-a27c-5e21db77c4a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d902454e7bd44339240a17aef7d1335', 'request_id': 'req-ba46701f-9ff0-4f77-a27c-5e21db77c4a8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:19.086724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ba46701f-9ff0-4f77-a27c-5e21db77c4a8 admin admin] The token cache is being invalidate because project 0d902454e7bd44339240a17aef7d1335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:19.086984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ba46701f-9ff0-4f77-a27c-5e21db77c4a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:19.087667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ba46701f-9ff0-4f77-a27c-5e21db77c4a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:19.089716 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 923/1878] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:18 2022] DELETE /identity/v3/projects/0d902454e7bd44339240a17aef7d1335 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:19.091075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3bf6c-d5a7-43de-b63a-e39d267aaaea None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:19.091376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3bf6c-d5a7-43de-b63a-e39d267aaaea None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:19.091652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3bf6c-d5a7-43de-b63a-e39d267aaaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:19.128824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-dfa3bf6c-d5a7-43de-b63a-e39d267aaaea None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:19.132916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-dfa3bf6c-d5a7-43de-b63a-e39d267aaaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:19.159851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015ae53-4e89-4222-b8af-313105508875 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:19.273295 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 924/1879] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:19.274310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3bf6c-d5a7-43de-b63a-e39d267aaaea None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:19.280942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-dfa3bf6c-d5a7-43de-b63a-e39d267aaaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:19.305731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3bf6c-d5a7-43de-b63a-e39d267aaaea None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:19.316712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bee89-24eb-46b0-973f-b040e05ffbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:19.318590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bee89-24eb-46b0-973f-b040e05ffbc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:19.321366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bee89-24eb-46b0-973f-b040e05ffbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:19.322007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bee89-24eb-46b0-973f-b040e05ffbc3 admin admin] PATH_INFO: `/v3/users/4d52e97d8e154a7986918ed3918cb4e2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:19.328112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780bee89-24eb-46b0-973f-b040e05ffbc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d52e97d8e154a7986918ed3918cb4e2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:19.333251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780bee89-24eb-46b0-973f-b040e05ffbc3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:19.349810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12880d7d-874f-4325-8abd-663724156cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:19.351342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12880d7d-874f-4325-8abd-663724156cce admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:19.353077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12880d7d-874f-4325-8abd-663724156cce admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:19.354747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12880d7d-874f-4325-8abd-663724156cce admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:19.360711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12880d7d-874f-4325-8abd-663724156cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:19.362816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12880d7d-874f-4325-8abd-663724156cce admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:19.401043 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 956/1880] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:19.402888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc818ea0-b105-4218-8db2-608f3054680f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:19.403080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc818ea0-b105-4218-8db2-608f3054680f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:19.403417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc818ea0-b105-4218-8db2-608f3054680f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:19.453890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fc818ea0-b105-4218-8db2-608f3054680f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:19.475502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fc818ea0-b105-4218-8db2-608f3054680f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:19.493436 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 957/1881] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:19.494828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc818ea0-b105-4218-8db2-608f3054680f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:19.498598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fc818ea0-b105-4218-8db2-608f3054680f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:19.520926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc818ea0-b105-4218-8db2-608f3054680f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:19.527916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13568353-efde-4373-8dbe-4b050d9e10a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:19.529384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13568353-efde-4373-8dbe-4b050d9e10a6 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:19.530011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13568353-efde-4373-8dbe-4b050d9e10a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:19.530474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13568353-efde-4373-8dbe-4b050d9e10a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:19.544655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13568353-efde-4373-8dbe-4b050d9e10a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:19.547097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13568353-efde-4373-8dbe-4b050d9e10a6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:19.570033 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 958/1882] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:19.573347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13568353-efde-4373-8dbe-4b050d9e10a6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:19.600494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13568353-efde-4373-8dbe-4b050d9e10a6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:19.612421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-780bee89-24eb-46b0-973f-b040e05ffbc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d52e97d8e154a7986918ed3918cb4e2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:19.621585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-780bee89-24eb-46b0-973f-b040e05ffbc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d52e97d8e154a7986918ed3918cb4e2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:19.626003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8f28a-3317-44dc-92c7-b7ca8991f53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:19.633081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8f28a-3317-44dc-92c7-b7ca8991f53c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:19.633081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8f28a-3317-44dc-92c7-b7ca8991f53c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:19.633081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8f28a-3317-44dc-92c7-b7ca8991f53c admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:19.633081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8f28a-3317-44dc-92c7-b7ca8991f53c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:03:19.633081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8f28a-3317-44dc-92c7-b7ca8991f53c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:19.634940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=82,keystone:INSERT=4,keystone:DELETE=14 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:19.637073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8f28a-3317-44dc-92c7-b7ca8991f53c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:19.641486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-780bee89-24eb-46b0-973f-b040e05ffbc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d52e97d8e154a7986918ed3918cb4e2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:19.667682 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 959/1883] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:03:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:19.669418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8f28a-3317-44dc-92c7-b7ca8991f53c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:19.692117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8f28a-3317-44dc-92c7-b7ca8991f53c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:19.708664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555624e-f194-4036-b1d3-adeb6a39a7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:19.712901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555624e-f194-4036-b1d3-adeb6a39a7cb admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:19.713588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555624e-f194-4036-b1d3-adeb6a39a7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:19.714064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555624e-f194-4036-b1d3-adeb6a39a7cb admin admin] PATH_INFO: `/v3/projects/2adfdd7542cf42259fc76dfa9387100d/users/27b10410ce634cdeb78f753e5fcfa35b/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:19.765415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-780bee89-24eb-46b0-973f-b040e05ffbc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d52e97d8e154a7986918ed3918cb4e2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:19.766350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-780bee89-24eb-46b0-973f-b040e05ffbc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d52e97d8e154a7986918ed3918cb4e2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:19.768825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555624e-f194-4036-b1d3-adeb6a39a7cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2adfdd7542cf42259fc76dfa9387100d, user_id=27b10410ce634cdeb78f753e5fcfa35b, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:19.777728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555624e-f194-4036-b1d3-adeb6a39a7cb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:19.826983 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 925/1884] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:19 2022] DELETE /identity/v3/users/4d52e97d8e154a7986918ed3918cb4e2 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:19.826983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bee89-24eb-46b0-973f-b040e05ffbc3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:19.828178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-780bee89-24eb-46b0-973f-b040e05ffbc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:19.907076 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 960/1885] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:19 2022] PUT /identity/v3/projects/2adfdd7542cf42259fc76dfa9387100d/users/27b10410ce634cdeb78f753e5fcfa35b/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:19.908696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431aff3-2bcc-42fa-ba7d-3081406b565e None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:19.908990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431aff3-2bcc-42fa-ba7d-3081406b565e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:19.909271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431aff3-2bcc-42fa-ba7d-3081406b565e None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:19.914945 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 961/1886] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:03:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:19.922969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1a84f-91bb-4133-a877-f6b61658375f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:19.922969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1a84f-91bb-4133-a877-f6b61658375f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:19.922969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1a84f-91bb-4133-a877-f6b61658375f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:20.061292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bee89-24eb-46b0-973f-b040e05ffbc3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:20.113984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-11a1a84f-91bb-4133-a877-f6b61658375f None None] MFA Rules not processed for user `27b10410ce634cdeb78f753e5fcfa35b`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:20.119862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-11a1a84f-91bb-4133-a877-f6b61658375f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:20.176947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0fff00-d0a9-4d12-902d-29a1d6c0ba02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:20.178992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fff00-d0a9-4d12-902d-29a1d6c0ba02 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:20.179554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fff00-d0a9-4d12-902d-29a1d6c0ba02 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:20.179983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0fff00-d0a9-4d12-902d-29a1d6c0ba02 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:20.180969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fff00-d0a9-4d12-902d-29a1d6c0ba02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:20.183538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0fff00-d0a9-4d12-902d-29a1d6c0ba02 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:20.322290 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 926/1887] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:19 2022] POST /identity/v3/projects => generated 399 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:20.322290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd7614-0394-4812-be2d-47ad0ac262f4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:20.322290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd7614-0394-4812-be2d-47ad0ac262f4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:20.322290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdd7614-0394-4812-be2d-47ad0ac262f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:20.377780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:20.396017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1cdd7614-0394-4812-be2d-47ad0ac262f4 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:20.398332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1cdd7614-0394-4812-be2d-47ad0ac262f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:20.405748 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 962/1888] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 490 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:20.405748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1a84f-91bb-4133-a877-f6b61658375f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:20.454091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1a84f-91bb-4133-a877-f6b61658375f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:20.565687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:20.599137 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 927/1889] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:03:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:20.602396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd7614-0394-4812-be2d-47ad0ac262f4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:20.609669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1cdd7614-0394-4812-be2d-47ad0ac262f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:20.637986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586215a-68e8-43db-899e-9e8a700ff513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:20.642262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586215a-68e8-43db-899e-9e8a700ff513 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:20.642598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586215a-68e8-43db-899e-9e8a700ff513 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:20.642941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586215a-68e8-43db-899e-9e8a700ff513 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:20.646056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586215a-68e8-43db-899e-9e8a700ff513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:20.652333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586215a-68e8-43db-899e-9e8a700ff513 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:20.779315 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 963/1890] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:20 2022] POST /identity/v3/users => generated 300 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:20.780269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586215a-68e8-43db-899e-9e8a700ff513 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:20.784445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6586215a-68e8-43db-899e-9e8a700ff513 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:20.865060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586215a-68e8-43db-899e-9e8a700ff513 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:20.890504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdd7614-0394-4812-be2d-47ad0ac262f4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:20.891179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3522b1-c38f-46ab-88be-9ab6173bbe34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:20.893259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3522b1-c38f-46ab-88be-9ab6173bbe34 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:20.893674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3522b1-c38f-46ab-88be-9ab6173bbe34 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:20.894750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3522b1-c38f-46ab-88be-9ab6173bbe34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:20.906638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa76ff-79cb-4061-aaac-a9ddec95b875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:20.907424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa76ff-79cb-4061-aaac-a9ddec95b875 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:20.907646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa76ff-79cb-4061-aaac-a9ddec95b875 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:20.907928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa76ff-79cb-4061-aaac-a9ddec95b875 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:20.908674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa76ff-79cb-4061-aaac-a9ddec95b875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:20.910709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa76ff-79cb-4061-aaac-a9ddec95b875 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:20.933617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3522b1-c38f-46ab-88be-9ab6173bbe34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:20.936955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3522b1-c38f-46ab-88be-9ab6173bbe34 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:20.984483 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 964/1891] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:20.997959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3522b1-c38f-46ab-88be-9ab6173bbe34 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:21.030997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3522b1-c38f-46ab-88be-9ab6173bbe34 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:21.045070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb786f-3809-41bc-931d-dd73501fc074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:21.048489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb786f-3809-41bc-931d-dd73501fc074 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:21.049024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb786f-3809-41bc-931d-dd73501fc074 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:21.049597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb786f-3809-41bc-931d-dd73501fc074 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:21.051129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb786f-3809-41bc-931d-dd73501fc074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:21.051348 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 928/1892] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:20 2022] POST /identity/v3/projects => generated 395 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:21.052482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa76ff-79cb-4061-aaac-a9ddec95b875 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:21.056920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb786f-3809-41bc-931d-dd73501fc074 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:21.090980 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 965/1893] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:21.099755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb786f-3809-41bc-931d-dd73501fc074 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:21.131861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb786f-3809-41bc-931d-dd73501fc074 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:21.285081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aadfd4-7737-42c4-b6e7-065b8d1a06fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:21.285792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aadfd4-7737-42c4-b6e7-065b8d1a06fb admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:21.286109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aadfd4-7737-42c4-b6e7-065b8d1a06fb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:21.287495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aadfd4-7737-42c4-b6e7-065b8d1a06fb admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:21.289116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aadfd4-7737-42c4-b6e7-065b8d1a06fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:21.295185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aadfd4-7737-42c4-b6e7-065b8d1a06fb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:21.314691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa76ff-79cb-4061-aaac-a9ddec95b875 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:21.321688 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 966/1894] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:21.328438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aadfd4-7737-42c4-b6e7-065b8d1a06fb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:21.328582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c5132-5659-42a8-a857-4ec245fbed7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:21.330288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c5132-5659-42a8-a857-4ec245fbed7a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:21.332132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c5132-5659-42a8-a857-4ec245fbed7a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:21.332523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792c5132-5659-42a8-a857-4ec245fbed7a admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:21.335095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c5132-5659-42a8-a857-4ec245fbed7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:21.342756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792c5132-5659-42a8-a857-4ec245fbed7a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:21.348674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aadfd4-7737-42c4-b6e7-065b8d1a06fb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:21.355373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24503a-b4da-4241-966a-44b4388bee7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:21.356875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24503a-b4da-4241-966a-44b4388bee7b admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:21.359863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24503a-b4da-4241-966a-44b4388bee7b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:21.359863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24503a-b4da-4241-966a-44b4388bee7b admin admin] PATH_INFO: `/v3/projects/812177ca96534d55a6eedb917b146b66/users/753243d6136d43e9904bdeed3bddfd99/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:21.382233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24503a-b4da-4241-966a-44b4388bee7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=812177ca96534d55a6eedb917b146b66, user_id=753243d6136d43e9904bdeed3bddfd99, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:21.385748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d24503a-b4da-4241-966a-44b4388bee7b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:21.476727 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 929/1895] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:21 2022] POST /identity/v3/users => generated 304 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:21.480206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c5132-5659-42a8-a857-4ec245fbed7a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:21.494733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792c5132-5659-42a8-a857-4ec245fbed7a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:21.502432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65218d73-ef0b-4e42-b575-8e0656b0f619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:21.503329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65218d73-ef0b-4e42-b575-8e0656b0f619 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:21.503717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65218d73-ef0b-4e42-b575-8e0656b0f619 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:21.504105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65218d73-ef0b-4e42-b575-8e0656b0f619 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:21.505294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65218d73-ef0b-4e42-b575-8e0656b0f619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:21.508098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65218d73-ef0b-4e42-b575-8e0656b0f619 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:21.523785 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 930/1896] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:03:21.528358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65218d73-ef0b-4e42-b575-8e0656b0f619 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:21.529824 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 967/1897] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:21 2022] PUT /identity/v3/projects/812177ca96534d55a6eedb917b146b66/users/753243d6136d43e9904bdeed3bddfd99/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:21.541620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283ed51-3e95-4a67-8963-896c3803c57d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:21.546591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283ed51-3e95-4a67-8963-896c3803c57d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:21.546591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b283ed51-3e95-4a67-8963-896c3803c57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:21.554683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65218d73-ef0b-4e42-b575-8e0656b0f619 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:21.603225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b283ed51-3e95-4a67-8963-896c3803c57d None None] MFA Rules not processed for user `753243d6136d43e9904bdeed3bddfd99`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:21.605940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b283ed51-3e95-4a67-8963-896c3803c57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:21.649113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:21.656298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd104ac-9bad-48ed-b52e-6628cf884ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:21.657812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd104ac-9bad-48ed-b52e-6628cf884ff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:21.658812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd104ac-9bad-48ed-b52e-6628cf884ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:21.659482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd104ac-9bad-48ed-b52e-6628cf884ff1 admin admin] PATH_INFO: `/v3/projects/03dc4ac6b75d454786872add5a5c1cab/users/26b3116232ef4be9804330348cf6c7bd/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:21.667922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:21.686261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd104ac-9bad-48ed-b52e-6628cf884ff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03dc4ac6b75d454786872add5a5c1cab, user_id=26b3116232ef4be9804330348cf6c7bd, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:21.694203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd104ac-9bad-48ed-b52e-6628cf884ff1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:21.745172 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 968/1898] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:21.750059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283ed51-3e95-4a67-8963-896c3803c57d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:21.773944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b283ed51-3e95-4a67-8963-896c3803c57d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:21.792568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c63b2a-df16-40fb-810d-859ac9f95d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:21.793462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c63b2a-df16-40fb-810d-859ac9f95d98 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:21.793997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c63b2a-df16-40fb-810d-859ac9f95d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:21.794497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c63b2a-df16-40fb-810d-859ac9f95d98 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:21.795595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c63b2a-df16-40fb-810d-859ac9f95d98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:21.798034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c63b2a-df16-40fb-810d-859ac9f95d98 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:21.825135 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 931/1899] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:21 2022] PUT /identity/v3/projects/03dc4ac6b75d454786872add5a5c1cab/users/26b3116232ef4be9804330348cf6c7bd/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:21.825135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9b84c-ad76-40ea-9302-d6fe44f1abc2 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:21.826541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9b84c-ad76-40ea-9302-d6fe44f1abc2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:21.827415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9b84c-ad76-40ea-9302-d6fe44f1abc2 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:21.838114 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 932/1900] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:03:21 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:21.839010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9b84c-ad76-40ea-9302-d6fe44f1abc2 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:21.922158 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 969/1901] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:21 2022] POST /identity/v3/projects => generated 397 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:21.923159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c63b2a-df16-40fb-810d-859ac9f95d98 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:22.048575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9b84c-ad76-40ea-9302-d6fe44f1abc2 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:22.090005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c63b2a-df16-40fb-810d-859ac9f95d98 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:22.117303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17960c3a-1ec2-4c31-b8b0-801c82fdde47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:22.123067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17960c3a-1ec2-4c31-b8b0-801c82fdde47 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:22.127064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17960c3a-1ec2-4c31-b8b0-801c82fdde47 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:22.127382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17960c3a-1ec2-4c31-b8b0-801c82fdde47 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:22.131826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17960c3a-1ec2-4c31-b8b0-801c82fdde47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:22.142392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17960c3a-1ec2-4c31-b8b0-801c82fdde47 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:22.268496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47a8e3-bcd9-4227-bb12-67961091410d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:22.271049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47a8e3-bcd9-4227-bb12-67961091410d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:22.274286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47a8e3-bcd9-4227-bb12-67961091410d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:22.274838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e47a8e3-bcd9-4227-bb12-67961091410d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:22.275943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47a8e3-bcd9-4227-bb12-67961091410d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:22.279261 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 970/1902] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:21 2022] POST /identity/v3/users => generated 305 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:22.282902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e47a8e3-bcd9-4227-bb12-67961091410d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:22.285917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3439bf3-3be7-4843-a7ab-21c34d977c62 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:22.286320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3439bf3-3be7-4843-a7ab-21c34d977c62 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:22.286617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3439bf3-3be7-4843-a7ab-21c34d977c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:22.315001 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 933/1903] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 476 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:22.324646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47a8e3-bcd9-4227-bb12-67961091410d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:22.325762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9e47a8e3-bcd9-4227-bb12-67961091410d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:22.361505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c3439bf3-3be7-4843-a7ab-21c34d977c62 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:22.367138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c3439bf3-3be7-4843-a7ab-21c34d977c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:22.370164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e47a8e3-bcd9-4227-bb12-67961091410d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:22.382982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15579836-9a7f-4851-b04f-635df5a37cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:22.383898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15579836-9a7f-4851-b04f-635df5a37cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:22.384167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15579836-9a7f-4851-b04f-635df5a37cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:22.384461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15579836-9a7f-4851-b04f-635df5a37cf2 admin admin] PATH_INFO: `/v3/users/1acd163a2a65407db8846a8878b0c6be` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:22.387985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15579836-9a7f-4851-b04f-635df5a37cf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1acd163a2a65407db8846a8878b0c6be)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:22.392276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15579836-9a7f-4851-b04f-635df5a37cf2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:22.557731 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 971/1904] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:03:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:22.559425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3439bf3-3be7-4843-a7ab-21c34d977c62 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:22.588132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3439bf3-3be7-4843-a7ab-21c34d977c62 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:22.603547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359c9b7-c8ab-4522-9f0f-03dbc4e4240b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:22.604729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359c9b7-c8ab-4522-9f0f-03dbc4e4240b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:22.605313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359c9b7-c8ab-4522-9f0f-03dbc4e4240b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:22.605767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359c9b7-c8ab-4522-9f0f-03dbc4e4240b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:22.609923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359c9b7-c8ab-4522-9f0f-03dbc4e4240b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:22.609923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359c9b7-c8ab-4522-9f0f-03dbc4e4240b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:22.630385 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 972/1905] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:22.632078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359c9b7-c8ab-4522-9f0f-03dbc4e4240b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:22.653252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359c9b7-c8ab-4522-9f0f-03dbc4e4240b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:22.663895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048c37d-ae54-4b6b-a388-2b86952ad98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:22.665210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048c37d-ae54-4b6b-a388-2b86952ad98e admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:22.665913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048c37d-ae54-4b6b-a388-2b86952ad98e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:22.666429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048c37d-ae54-4b6b-a388-2b86952ad98e admin admin] PATH_INFO: `/v3/projects/03dc4ac6b75d454786872add5a5c1cab/users/26b3116232ef4be9804330348cf6c7bd/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:22.669931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048c37d-ae54-4b6b-a388-2b86952ad98e admin admin] RBAC: Authorizing `identity:create_grant(project_id=03dc4ac6b75d454786872add5a5c1cab, user_id=26b3116232ef4be9804330348cf6c7bd, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:22.674290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e048c37d-ae54-4b6b-a388-2b86952ad98e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:22.675146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-15579836-9a7f-4851-b04f-635df5a37cf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1acd163a2a65407db8846a8878b0c6be'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:22.680099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-15579836-9a7f-4851-b04f-635df5a37cf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1acd163a2a65407db8846a8878b0c6be'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:22.707399 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 973/1906] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:22 2022] PUT /identity/v3/projects/03dc4ac6b75d454786872add5a5c1cab/users/26b3116232ef4be9804330348cf6c7bd/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:22.710095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048c37d-ae54-4b6b-a388-2b86952ad98e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:22.712696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e048c37d-ae54-4b6b-a388-2b86952ad98e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:22.719522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:22.725097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-15579836-9a7f-4851-b04f-635df5a37cf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1acd163a2a65407db8846a8878b0c6be'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:22.824488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-15579836-9a7f-4851-b04f-635df5a37cf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1acd163a2a65407db8846a8878b0c6be'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:22.825356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-15579836-9a7f-4851-b04f-635df5a37cf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1acd163a2a65407db8846a8878b0c6be'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:22.836897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e048c37d-ae54-4b6b-a388-2b86952ad98e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:22.847950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708b481-ea5a-49b4-b67a-a59d025fad67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:22.850351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708b481-ea5a-49b4-b67a-a59d025fad67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:22.851945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:22.853442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708b481-ea5a-49b4-b67a-a59d025fad67 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:22.854173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b708b481-ea5a-49b4-b67a-a59d025fad67 admin admin] PATH_INFO: `/v3/projects/38bea6ec7e1c4836868191839fd7b889` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:22.856523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708b481-ea5a-49b4-b67a-a59d025fad67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38bea6ec7e1c4836868191839fd7b889)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:22.859577 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 934/1907] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:22 2022] DELETE /identity/v3/users/1acd163a2a65407db8846a8878b0c6be => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:22.864581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b708b481-ea5a-49b4-b67a-a59d025fad67 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:22.883768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15579836-9a7f-4851-b04f-635df5a37cf2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:22.888120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-15579836-9a7f-4851-b04f-635df5a37cf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:23.087829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b708b481-ea5a-49b4-b67a-a59d025fad67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38bea6ec7e1c4836868191839fd7b889', 'request_id': 'req-b708b481-ea5a-49b4-b67a-a59d025fad67'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:23.099912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b708b481-ea5a-49b4-b67a-a59d025fad67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38bea6ec7e1c4836868191839fd7b889', 'request_id': 'req-b708b481-ea5a-49b4-b67a-a59d025fad67'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:23.120404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15579836-9a7f-4851-b04f-635df5a37cf2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:23.232442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee95169b-d787-4cb5-8c1a-32348ca02c4c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:23.235343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95169b-d787-4cb5-8c1a-32348ca02c4c service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:23.235814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95169b-d787-4cb5-8c1a-32348ca02c4c service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:23.236151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee95169b-d787-4cb5-8c1a-32348ca02c4c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:23.238796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ee95169b-d787-4cb5-8c1a-32348ca02c4c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:23.251690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b708b481-ea5a-49b4-b67a-a59d025fad67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38bea6ec7e1c4836868191839fd7b889', 'request_id': 'req-b708b481-ea5a-49b4-b67a-a59d025fad67'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:23.266986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95169b-d787-4cb5-8c1a-32348ca02c4c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:23.269746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b708b481-ea5a-49b4-b67a-a59d025fad67 admin admin] The token cache is being invalidate because project 38bea6ec7e1c4836868191839fd7b889 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:23.270070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b708b481-ea5a-49b4-b67a-a59d025fad67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:23.271397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee95169b-d787-4cb5-8c1a-32348ca02c4c service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:23.273016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b708b481-ea5a-49b4-b67a-a59d025fad67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:23.275840 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 974/1908] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:22 2022] DELETE /identity/v3/projects/38bea6ec7e1c4836868191839fd7b889 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:23.278804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708b481-ea5a-49b4-b67a-a59d025fad67 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:23.282181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b708b481-ea5a-49b4-b67a-a59d025fad67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:23.295851 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 935/1909] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:23.298443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fec171-58a7-4518-a011-81ac9ae008c3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:23.298956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fec171-58a7-4518-a011-81ac9ae008c3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:23.299432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fec171-58a7-4518-a011-81ac9ae008c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:23.368962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d1fec171-58a7-4518-a011-81ac9ae008c3 None None] MFA Rules not processed for user `26b3116232ef4be9804330348cf6c7bd`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:23.372095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d1fec171-58a7-4518-a011-81ac9ae008c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:23.469228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b708b481-ea5a-49b4-b67a-a59d025fad67 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:23.488369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f05e67-3b86-4f7a-a43f-fa075435c4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:23.490101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f05e67-3b86-4f7a-a43f-fa075435c4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:23.491079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f05e67-3b86-4f7a-a43f-fa075435c4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:23.491292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f05e67-3b86-4f7a-a43f-fa075435c4d4 admin admin] PATH_INFO: `/v3/projects/9397ab700e3b472597d1462ad128d87f/users/5c05235c31c84ed99ed6e18a9f9d9c9c/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:23.539900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f05e67-3b86-4f7a-a43f-fa075435c4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9397ab700e3b472597d1462ad128d87f, user_id=5c05235c31c84ed99ed6e18a9f9d9c9c, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:23.544654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f05e67-3b86-4f7a-a43f-fa075435c4d4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:23.574051 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 936/1910] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:23 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:23.574051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdad40-b6aa-44b7-9286-107c4298846d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:23.574051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdad40-b6aa-44b7-9286-107c4298846d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:23.574494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfdad40-b6aa-44b7-9286-107c4298846d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:23.623318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4dfdad40-b6aa-44b7-9286-107c4298846d None None] MFA Rules not processed for user `16fedb3759d94951a633d58d8f91abd3`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:23.624680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4dfdad40-b6aa-44b7-9286-107c4298846d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:23.630382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:23.655273 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 975/1911] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:23 2022] PUT /identity/v3/projects/9397ab700e3b472597d1462ad128d87f/users/5c05235c31c84ed99ed6e18a9f9d9c9c/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:23.657011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f05e67-3b86-4f7a-a43f-fa075435c4d4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:23.660342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a7f05e67-3b86-4f7a-a43f-fa075435c4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:23.715551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:23.750791 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 937/1912] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:23 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:23.750791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdad40-b6aa-44b7-9286-107c4298846d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:23.750791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4dfdad40-b6aa-44b7-9286-107c4298846d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:23.775551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfdad40-b6aa-44b7-9286-107c4298846d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:23.805024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f05e67-3b86-4f7a-a43f-fa075435c4d4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:23.811147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb86068-2d27-4739-98c4-7aea720a0d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:23.812260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb86068-2d27-4739-98c4-7aea720a0d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:23.812643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb86068-2d27-4739-98c4-7aea720a0d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:23.813124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb86068-2d27-4739-98c4-7aea720a0d5a admin admin] PATH_INFO: `/v3/users/835432d20a804515856b22e35d7b2769` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:23.815762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb86068-2d27-4739-98c4-7aea720a0d5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=835432d20a804515856b22e35d7b2769)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:23.818000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb86068-2d27-4739-98c4-7aea720a0d5a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:23.908383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b06517-3b59-47e1-ae62-4e644ebec06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:23.910181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b06517-3b59-47e1-ae62-4e644ebec06b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:23.910475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b06517-3b59-47e1-ae62-4e644ebec06b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:23.910767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b06517-3b59-47e1-ae62-4e644ebec06b admin admin] PATH_INFO: `/v3/projects/dc1a016f5efa407387d7cccddf3cf0f0` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:23.913322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b06517-3b59-47e1-ae62-4e644ebec06b admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc1a016f5efa407387d7cccddf3cf0f0)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:23.921888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b06517-3b59-47e1-ae62-4e644ebec06b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:23.998828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abb86068-2d27-4739-98c4-7aea720a0d5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '835432d20a804515856b22e35d7b2769'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:24.022136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abb86068-2d27-4739-98c4-7aea720a0d5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '835432d20a804515856b22e35d7b2769'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:24.022136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abb86068-2d27-4739-98c4-7aea720a0d5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '835432d20a804515856b22e35d7b2769'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:24.022136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abb86068-2d27-4739-98c4-7aea720a0d5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '835432d20a804515856b22e35d7b2769'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:24.097733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-80b06517-3b59-47e1-ae62-4e644ebec06b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc1a016f5efa407387d7cccddf3cf0f0', 'request_id': 'req-80b06517-3b59-47e1-ae62-4e644ebec06b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:24.111342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-80b06517-3b59-47e1-ae62-4e644ebec06b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc1a016f5efa407387d7cccddf3cf0f0', 'request_id': 'req-80b06517-3b59-47e1-ae62-4e644ebec06b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:24.137356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abb86068-2d27-4739-98c4-7aea720a0d5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '835432d20a804515856b22e35d7b2769'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:24.193880 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 976/1913] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:23 2022] DELETE /identity/v3/users/835432d20a804515856b22e35d7b2769 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:24.200308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb86068-2d27-4739-98c4-7aea720a0d5a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:24.208814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-abb86068-2d27-4739-98c4-7aea720a0d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:24.222820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-80b06517-3b59-47e1-ae62-4e644ebec06b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc1a016f5efa407387d7cccddf3cf0f0', 'request_id': 'req-80b06517-3b59-47e1-ae62-4e644ebec06b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:24.245795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-80b06517-3b59-47e1-ae62-4e644ebec06b admin admin] The token cache is being invalidate because project dc1a016f5efa407387d7cccddf3cf0f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:24.246404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-80b06517-3b59-47e1-ae62-4e644ebec06b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:24.249062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-80b06517-3b59-47e1-ae62-4e644ebec06b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:24.253362 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 938/1914] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:23 2022] DELETE /identity/v3/projects/dc1a016f5efa407387d7cccddf3cf0f0 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:24.254794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b06517-3b59-47e1-ae62-4e644ebec06b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:24.258894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-80b06517-3b59-47e1-ae62-4e644ebec06b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:24.339287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:24.424363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb86068-2d27-4739-98c4-7aea720a0d5a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:24.621005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b06517-3b59-47e1-ae62-4e644ebec06b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:24.623110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01e0d7-c164-4ddd-8802-dbe32264e1a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:24.623110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01e0d7-c164-4ddd-8802-dbe32264e1a6 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:24.623110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01e0d7-c164-4ddd-8802-dbe32264e1a6 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:24.623110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01e0d7-c164-4ddd-8802-dbe32264e1a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:24.623406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2c01e0d7-c164-4ddd-8802-dbe32264e1a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:24.623406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:24.623552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f39d7-858e-42c0-b915-0759f05ed81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:24.623552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f39d7-858e-42c0-b915-0759f05ed81c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:24.623552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f39d7-858e-42c0-b915-0759f05ed81c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:24.623552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f39d7-858e-42c0-b915-0759f05ed81c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:24.623831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f39d7-858e-42c0-b915-0759f05ed81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:24.623831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f39d7-858e-42c0-b915-0759f05ed81c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:24.623831 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 939/1915] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:24.623831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f39d7-858e-42c0-b915-0759f05ed81c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:24.623831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1e9f39d7-858e-42c0-b915-0759f05ed81c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:24.623831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f39d7-858e-42c0-b915-0759f05ed81c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:24.624216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7947a19-2f6c-4482-b7b5-e0d4bcb59b44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:24.624216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7947a19-2f6c-4482-b7b5-e0d4bcb59b44 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:24.624216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7947a19-2f6c-4482-b7b5-e0d4bcb59b44 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:24.624216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7947a19-2f6c-4482-b7b5-e0d4bcb59b44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:24.624491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b7947a19-2f6c-4482-b7b5-e0d4bcb59b44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:24.624572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01e0d7-c164-4ddd-8802-dbe32264e1a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:24.624572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01e0d7-c164-4ddd-8802-dbe32264e1a6 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:24.624572 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 977/1916] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:24.624572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01e0d7-c164-4ddd-8802-dbe32264e1a6 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:24.624572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2c01e0d7-c164-4ddd-8802-dbe32264e1a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:24.648640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01e0d7-c164-4ddd-8802-dbe32264e1a6 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:24.720998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1da20-0d83-42be-9227-84f24f5d3a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:24.721916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7947a19-2f6c-4482-b7b5-e0d4bcb59b44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:24.722109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1da20-0d83-42be-9227-84f24f5d3a2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:24.722109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1da20-0d83-42be-9227-84f24f5d3a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:24.722109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1da20-0d83-42be-9227-84f24f5d3a2f admin admin] PATH_INFO: `/v3/users/5555f81ef14241f99c3592fb5632ec15` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:24.722526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7947a19-2f6c-4482-b7b5-e0d4bcb59b44 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:24.722692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1da20-0d83-42be-9227-84f24f5d3a2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5555f81ef14241f99c3592fb5632ec15)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:24.722692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d1da20-0d83-42be-9227-84f24f5d3a2f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:24.723014 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 940/1917] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:24 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:24.723014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7947a19-2f6c-4482-b7b5-e0d4bcb59b44 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:24.723014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7947a19-2f6c-4482-b7b5-e0d4bcb59b44 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:24.723014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b4c99-93e9-4eae-8bdc-15b588a0b4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:24.723565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b4c99-93e9-4eae-8bdc-15b588a0b4a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:24.723565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b4c99-93e9-4eae-8bdc-15b588a0b4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:24.723565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b4c99-93e9-4eae-8bdc-15b588a0b4a8 admin admin] PATH_INFO: `/v3/projects/9397ab700e3b472597d1462ad128d87f/users/5c05235c31c84ed99ed6e18a9f9d9c9c/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:24.723565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b4c99-93e9-4eae-8bdc-15b588a0b4a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9397ab700e3b472597d1462ad128d87f, user_id=5c05235c31c84ed99ed6e18a9f9d9c9c, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:24.723565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3b4c99-93e9-4eae-8bdc-15b588a0b4a8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:24.729750 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 941/1918] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:24 2022] PUT /identity/v3/projects/9397ab700e3b472597d1462ad128d87f/users/5c05235c31c84ed99ed6e18a9f9d9c9c/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:24.736680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc049b7-1900-4faa-b361-0536d8c0eda9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:24.737034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc049b7-1900-4faa-b361-0536d8c0eda9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:24.737330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc049b7-1900-4faa-b361-0536d8c0eda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:24.798420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-fcc049b7-1900-4faa-b361-0536d8c0eda9 None None] MFA Rules not processed for user `5c05235c31c84ed99ed6e18a9f9d9c9c`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:24.798420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fcc049b7-1900-4faa-b361-0536d8c0eda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:24.957974 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 942/1919] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:25.223731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc049b7-1900-4faa-b361-0536d8c0eda9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:25.319405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc049b7-1900-4faa-b361-0536d8c0eda9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:25.325310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207b19e-f686-4831-8a68-934827e14be0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:25.326010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207b19e-f686-4831-8a68-934827e14be0 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:25.326294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207b19e-f686-4831-8a68-934827e14be0 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:25.326566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207b19e-f686-4831-8a68-934827e14be0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:25.328417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c207b19e-f686-4831-8a68-934827e14be0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:25.398957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207b19e-f686-4831-8a68-934827e14be0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:25.404599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207b19e-f686-4831-8a68-934827e14be0 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:25.412685 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 943/1920] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:25 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:25.413510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207b19e-f686-4831-8a68-934827e14be0 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:25.414920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c207b19e-f686-4831-8a68-934827e14be0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:25.481956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207b19e-f686-4831-8a68-934827e14be0 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:25.486664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa54dc-b543-423a-9521-438fe24da46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:25.487574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa54dc-b543-423a-9521-438fe24da46d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:25.487857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa54dc-b543-423a-9521-438fe24da46d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:25.488131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa54dc-b543-423a-9521-438fe24da46d admin admin] PATH_INFO: `/v3/users/27b10410ce634cdeb78f753e5fcfa35b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:25.489337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa54dc-b543-423a-9521-438fe24da46d admin admin] RBAC: Authorizing `identity:delete_user(user_id=27b10410ce634cdeb78f753e5fcfa35b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:25.491914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa54dc-b543-423a-9521-438fe24da46d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:25.684221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-39d1da20-0d83-42be-9227-84f24f5d3a2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5555f81ef14241f99c3592fb5632ec15'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:25.715087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-39d1da20-0d83-42be-9227-84f24f5d3a2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5555f81ef14241f99c3592fb5632ec15'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:25.716065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-39d1da20-0d83-42be-9227-84f24f5d3a2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5555f81ef14241f99c3592fb5632ec15'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:25.717587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-39d1da20-0d83-42be-9227-84f24f5d3a2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5555f81ef14241f99c3592fb5632ec15'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:25.774889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1eaa54dc-b543-423a-9521-438fe24da46d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27b10410ce634cdeb78f753e5fcfa35b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:25.776054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1eaa54dc-b543-423a-9521-438fe24da46d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27b10410ce634cdeb78f753e5fcfa35b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:25.793582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1eaa54dc-b543-423a-9521-438fe24da46d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27b10410ce634cdeb78f753e5fcfa35b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:25.853295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-39d1da20-0d83-42be-9227-84f24f5d3a2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5555f81ef14241f99c3592fb5632ec15'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:25.887629 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 978/1921] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:24 2022] DELETE /identity/v3/users/5555f81ef14241f99c3592fb5632ec15 => generated 0 bytes in 1277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:25.888819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1da20-0d83-42be-9227-84f24f5d3a2f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:25.892011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-39d1da20-0d83-42be-9227-84f24f5d3a2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:25.943375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1eaa54dc-b543-423a-9521-438fe24da46d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27b10410ce634cdeb78f753e5fcfa35b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:25.944126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1eaa54dc-b543-423a-9521-438fe24da46d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27b10410ce634cdeb78f753e5fcfa35b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:26.004373 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 944/1922] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:25 2022] DELETE /identity/v3/users/27b10410ce634cdeb78f753e5fcfa35b => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:26.009745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa54dc-b543-423a-9521-438fe24da46d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:26.011318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1eaa54dc-b543-423a-9521-438fe24da46d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:26.060620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:26.195797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa54dc-b543-423a-9521-438fe24da46d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:26.205628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d1da20-0d83-42be-9227-84f24f5d3a2f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:26.210124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5fe7e-4c33-44c1-9d6b-9ad0d419399e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:26.211308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5fe7e-4c33-44c1-9d6b-9ad0d419399e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:26.227581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5fe7e-4c33-44c1-9d6b-9ad0d419399e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:26.227581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5fe7e-4c33-44c1-9d6b-9ad0d419399e admin admin] PATH_INFO: `/v3/projects/751501c7dfad4ad0a00524c6baf4471d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:26.227581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5fe7e-4c33-44c1-9d6b-9ad0d419399e admin admin] RBAC: Authorizing `identity:delete_project(project_id=751501c7dfad4ad0a00524c6baf4471d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:26.227581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5fe7e-4c33-44c1-9d6b-9ad0d419399e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:26.302701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:26.379879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18392d43-838f-4a71-aae2-fb0992e33fbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:26.382628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18392d43-838f-4a71-aae2-fb0992e33fbb service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:26.385219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18392d43-838f-4a71-aae2-fb0992e33fbb service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:26.385898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18392d43-838f-4a71-aae2-fb0992e33fbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:26.390534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-18392d43-838f-4a71-aae2-fb0992e33fbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:26.395975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bbc5fe7e-4c33-44c1-9d6b-9ad0d419399e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '751501c7dfad4ad0a00524c6baf4471d', 'request_id': 'req-bbc5fe7e-4c33-44c1-9d6b-9ad0d419399e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:26.406257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bbc5fe7e-4c33-44c1-9d6b-9ad0d419399e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '751501c7dfad4ad0a00524c6baf4471d', 'request_id': 'req-bbc5fe7e-4c33-44c1-9d6b-9ad0d419399e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:26.516601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bbc5fe7e-4c33-44c1-9d6b-9ad0d419399e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '751501c7dfad4ad0a00524c6baf4471d', 'request_id': 'req-bbc5fe7e-4c33-44c1-9d6b-9ad0d419399e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:26.539618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bbc5fe7e-4c33-44c1-9d6b-9ad0d419399e admin admin] The token cache is being invalidate because project 751501c7dfad4ad0a00524c6baf4471d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:26.540694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bbc5fe7e-4c33-44c1-9d6b-9ad0d419399e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:26.553742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bbc5fe7e-4c33-44c1-9d6b-9ad0d419399e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:26.553742 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 945/1923] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:26 2022] DELETE /identity/v3/projects/751501c7dfad4ad0a00524c6baf4471d => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:26.553742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440db56-e8c9-4fbd-ba01-ae0cfb90dbc1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:26.553742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440db56-e8c9-4fbd-ba01-ae0cfb90dbc1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:26.554387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440db56-e8c9-4fbd-ba01-ae0cfb90dbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:26.558125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18392d43-838f-4a71-aae2-fb0992e33fbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:26.560522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18392d43-838f-4a71-aae2-fb0992e33fbb service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:26.611701 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 979/1924] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:26.620345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18392d43-838f-4a71-aae2-fb0992e33fbb service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:26.632512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-18392d43-838f-4a71-aae2-fb0992e33fbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:26.687693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1440db56-e8c9-4fbd-ba01-ae0cfb90dbc1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:26.688839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1440db56-e8c9-4fbd-ba01-ae0cfb90dbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:26.701460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18392d43-838f-4a71-aae2-fb0992e33fbb service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:26.868501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400d7b9-61ab-4129-bd47-dbe80762aa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:26.870207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400d7b9-61ab-4129-bd47-dbe80762aa5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:26.871809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400d7b9-61ab-4129-bd47-dbe80762aa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:26.872566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400d7b9-61ab-4129-bd47-dbe80762aa5c admin admin] PATH_INFO: `/v3/projects/44391556bd3c44828209cd1aa4fb3fa2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:26.875667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400d7b9-61ab-4129-bd47-dbe80762aa5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=44391556bd3c44828209cd1aa4fb3fa2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:26.879106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0400d7b9-61ab-4129-bd47-dbe80762aa5c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:26.882737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:26.943670 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 946/1925] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:26.952481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448186d-3ef6-4abf-a4dd-754aa9db27c1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:26.952976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448186d-3ef6-4abf-a4dd-754aa9db27c1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:26.953310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b448186d-3ef6-4abf-a4dd-754aa9db27c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:27.021555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b448186d-3ef6-4abf-a4dd-754aa9db27c1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:27.028184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b448186d-3ef6-4abf-a4dd-754aa9db27c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:27.040521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0400d7b9-61ab-4129-bd47-dbe80762aa5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44391556bd3c44828209cd1aa4fb3fa2', 'request_id': 'req-0400d7b9-61ab-4129-bd47-dbe80762aa5c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:27.051972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0400d7b9-61ab-4129-bd47-dbe80762aa5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44391556bd3c44828209cd1aa4fb3fa2', 'request_id': 'req-0400d7b9-61ab-4129-bd47-dbe80762aa5c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:27.152573 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 947/1926] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:27.165266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0400d7b9-61ab-4129-bd47-dbe80762aa5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44391556bd3c44828209cd1aa4fb3fa2', 'request_id': 'req-0400d7b9-61ab-4129-bd47-dbe80762aa5c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:27.165387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448186d-3ef6-4abf-a4dd-754aa9db27c1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:27.165387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b448186d-3ef6-4abf-a4dd-754aa9db27c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:27.179298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0400d7b9-61ab-4129-bd47-dbe80762aa5c admin admin] The token cache is being invalidate because project 44391556bd3c44828209cd1aa4fb3fa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:27.179732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0400d7b9-61ab-4129-bd47-dbe80762aa5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:27.181481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0400d7b9-61ab-4129-bd47-dbe80762aa5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:27.185741 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 980/1927] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:26 2022] DELETE /identity/v3/projects/44391556bd3c44828209cd1aa4fb3fa2 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:27.187813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400d7b9-61ab-4129-bd47-dbe80762aa5c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:27.192396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0400d7b9-61ab-4129-bd47-dbe80762aa5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:27.308393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b448186d-3ef6-4abf-a4dd-754aa9db27c1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:27.315429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27eac9-63dd-43dd-8bdc-b97426edeee4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:27.316364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27eac9-63dd-43dd-8bdc-b97426edeee4 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:27.316707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27eac9-63dd-43dd-8bdc-b97426edeee4 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:27.317044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27eac9-63dd-43dd-8bdc-b97426edeee4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:27.319009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ae27eac9-63dd-43dd-8bdc-b97426edeee4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:27.325109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400d7b9-61ab-4129-bd47-dbe80762aa5c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:27.335625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54f5d00-8677-4ab6-a40f-d7d05c2712bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:27.338306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f5d00-8677-4ab6-a40f-d7d05c2712bf service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:27.338306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f5d00-8677-4ab6-a40f-d7d05c2712bf service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:27.338306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54f5d00-8677-4ab6-a40f-d7d05c2712bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:27.342889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f54f5d00-8677-4ab6-a40f-d7d05c2712bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:27.362145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:27.382766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f5d00-8677-4ab6-a40f-d7d05c2712bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:27.386313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54f5d00-8677-4ab6-a40f-d7d05c2712bf service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:27.417058 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 981/1928] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:27.420118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964f0e9-d939-4016-acc8-e0d891d53e32 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:27.420411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964f0e9-d939-4016-acc8-e0d891d53e32 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:27.420711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964f0e9-d939-4016-acc8-e0d891d53e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:27.462394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27eac9-63dd-43dd-8bdc-b97426edeee4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:27.462394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27eac9-63dd-43dd-8bdc-b97426edeee4 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:27.470237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f964f0e9-d939-4016-acc8-e0d891d53e32 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:27.472549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f964f0e9-d939-4016-acc8-e0d891d53e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:27.494255 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 948/1929] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:27 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:27.496356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4554788-72ea-493c-9e09-47373a477eb2 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:27.496909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4554788-72ea-493c-9e09-47373a477eb2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:27.504023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4554788-72ea-493c-9e09-47373a477eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:27.508451 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 982/1930] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:27.509639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964f0e9-d939-4016-acc8-e0d891d53e32 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:27.529791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964f0e9-d939-4016-acc8-e0d891d53e32 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:27.547595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcb243-67ee-4a64-94ad-e198f338a5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:27.548609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcb243-67ee-4a64-94ad-e198f338a5ab admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:27.548895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcb243-67ee-4a64-94ad-e198f338a5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:27.549169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcb243-67ee-4a64-94ad-e198f338a5ab admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:27.550469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fcb243-67ee-4a64-94ad-e198f338a5ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:03:27.550667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fcb243-67ee-4a64-94ad-e198f338a5ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:27.552836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fcb243-67ee-4a64-94ad-e198f338a5ab admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:27.573382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b4554788-72ea-493c-9e09-47373a477eb2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:27.581864 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 983/1931] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:27.582573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b4554788-72ea-493c-9e09-47373a477eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:27.583298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcb243-67ee-4a64-94ad-e198f338a5ab admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:27.601243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcb243-67ee-4a64-94ad-e198f338a5ab admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:27.607348 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 949/1932] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:27.608365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb6458-0d28-45b4-a40d-cb716ee6c555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:27.609049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb6458-0d28-45b4-a40d-cb716ee6c555 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:27.609608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb6458-0d28-45b4-a40d-cb716ee6c555 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:27.609947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb6458-0d28-45b4-a40d-cb716ee6c555 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:27.610847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb6458-0d28-45b4-a40d-cb716ee6c555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:27.619247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb6458-0d28-45b4-a40d-cb716ee6c555 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:27.624140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1ed17-ed6f-49d5-a000-bb2bd25cc36b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:27.624836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1ed17-ed6f-49d5-a000-bb2bd25cc36b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:27.625286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1ed17-ed6f-49d5-a000-bb2bd25cc36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:27.681581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-aea1ed17-ed6f-49d5-a000-bb2bd25cc36b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:27.684189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-aea1ed17-ed6f-49d5-a000-bb2bd25cc36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:27.692906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:27.701781 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 950/1933] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:03:27.702863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1ed17-ed6f-49d5-a000-bb2bd25cc36b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:27.722321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1ed17-ed6f-49d5-a000-bb2bd25cc36b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:27.727501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c94a5-7360-42ce-b094-ff3b813d2d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:27.728443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c94a5-7360-42ce-b094-ff3b813d2d8b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:27.728885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c94a5-7360-42ce-b094-ff3b813d2d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:27.729383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161c94a5-7360-42ce-b094-ff3b813d2d8b admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:27.730878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161c94a5-7360-42ce-b094-ff3b813d2d8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:03:27.731418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161c94a5-7360-42ce-b094-ff3b813d2d8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:27.731591 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 984/1934] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:27 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:27.733465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161c94a5-7360-42ce-b094-ff3b813d2d8b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:27.736482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb6458-0d28-45b4-a40d-cb716ee6c555 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:27.778784 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 951/1935] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:27.783222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c94a5-7360-42ce-b094-ff3b813d2d8b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:27.814333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161c94a5-7360-42ce-b094-ff3b813d2d8b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:27.942537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e116e7-f8d2-496e-887c-ca90d22c716c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:27.943182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e116e7-f8d2-496e-887c-ca90d22c716c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:27.943472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e116e7-f8d2-496e-887c-ca90d22c716c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:27.943750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e116e7-f8d2-496e-887c-ca90d22c716c admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:27.944519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e116e7-f8d2-496e-887c-ca90d22c716c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:27.946783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e116e7-f8d2-496e-887c-ca90d22c716c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:27.965828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb6458-0d28-45b4-a40d-cb716ee6c555 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:27.971999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20309c96-0a76-41df-87bf-a7c2f4b3da0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:27.973251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20309c96-0a76-41df-87bf-a7c2f4b3da0c admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:27.973458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20309c96-0a76-41df-87bf-a7c2f4b3da0c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:27.973686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20309c96-0a76-41df-87bf-a7c2f4b3da0c admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:27.974547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20309c96-0a76-41df-87bf-a7c2f4b3da0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:27.977807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20309c96-0a76-41df-87bf-a7c2f4b3da0c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:28.021141 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 952/1936] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:27 2022] POST /identity/v3/projects => generated 375 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:28.025166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e116e7-f8d2-496e-887c-ca90d22c716c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:28.078984 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 985/1937] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:27 2022] POST /identity/v3/users => generated 293 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:28.080327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20309c96-0a76-41df-87bf-a7c2f4b3da0c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:28.102943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20309c96-0a76-41df-87bf-a7c2f4b3da0c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:28.180822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:28.190758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547a215-f7fb-459c-990d-ae1d96ce025c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:28.192602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547a215-f7fb-459c-990d-ae1d96ce025c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:28.193318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547a215-f7fb-459c-990d-ae1d96ce025c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:28.194580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547a215-f7fb-459c-990d-ae1d96ce025c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:28.195851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547a215-f7fb-459c-990d-ae1d96ce025c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:28.198079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547a215-f7fb-459c-990d-ae1d96ce025c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:28.219508 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 986/1938] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:28.224867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547a215-f7fb-459c-990d-ae1d96ce025c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:28.259020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e116e7-f8d2-496e-887c-ca90d22c716c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:28.259359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547a215-f7fb-459c-990d-ae1d96ce025c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:28.262296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:28.271432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335704bc-9743-4144-a5d3-02058066f823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:28.271432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335704bc-9743-4144-a5d3-02058066f823 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:28.271432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335704bc-9743-4144-a5d3-02058066f823 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:28.271432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335704bc-9743-4144-a5d3-02058066f823 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:28.271770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335704bc-9743-4144-a5d3-02058066f823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:28.273686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335704bc-9743-4144-a5d3-02058066f823 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:28.280729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0da497-95d7-4293-8706-1f92f93c843b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:28.285011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0da497-95d7-4293-8706-1f92f93c843b admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:28.285317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0da497-95d7-4293-8706-1f92f93c843b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:28.285685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0da497-95d7-4293-8706-1f92f93c843b admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:28.287036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0da497-95d7-4293-8706-1f92f93c843b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:28.289064 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 987/1939] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:28.293603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0da497-95d7-4293-8706-1f92f93c843b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:28.293774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335704bc-9743-4144-a5d3-02058066f823 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:28.314079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335704bc-9743-4144-a5d3-02058066f823 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:28.321073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c697412-4b58-4310-b271-0b4940254fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:28.322371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c697412-4b58-4310-b271-0b4940254fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:28.322553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c697412-4b58-4310-b271-0b4940254fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:28.323357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c697412-4b58-4310-b271-0b4940254fe5 admin admin] PATH_INFO: `/v3/projects/6ce3b7e72b974127a0b9f4c0db2d0bf6/users/10993092e2244c15b0ee389bdf39c92d/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:28.364656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c697412-4b58-4310-b271-0b4940254fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ce3b7e72b974127a0b9f4c0db2d0bf6, user_id=10993092e2244c15b0ee389bdf39c92d, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:28.372338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c697412-4b58-4310-b271-0b4940254fe5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:28.412018 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 953/1940] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:28 2022] POST /identity/v3/users => generated 288 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:28.413790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0da497-95d7-4293-8706-1f92f93c843b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:28.421156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2e0da497-95d7-4293-8706-1f92f93c843b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:28.436819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0da497-95d7-4293-8706-1f92f93c843b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:28.443807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2032028-d94b-4b66-b98d-8c0bf1a60a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:28.444921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2032028-d94b-4b66-b98d-8c0bf1a60a52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:28.445306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2032028-d94b-4b66-b98d-8c0bf1a60a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:28.445694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2032028-d94b-4b66-b98d-8c0bf1a60a52 admin admin] PATH_INFO: `/v3/users/753243d6136d43e9904bdeed3bddfd99` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:28.447442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2032028-d94b-4b66-b98d-8c0bf1a60a52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=753243d6136d43e9904bdeed3bddfd99)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:28.449717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2032028-d94b-4b66-b98d-8c0bf1a60a52 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:28.471042 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 988/1941] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:28 2022] PUT /identity/v3/projects/6ce3b7e72b974127a0b9f4c0db2d0bf6/users/10993092e2244c15b0ee389bdf39c92d/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:03:28.471042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c697412-4b58-4310-b271-0b4940254fe5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:28.588122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c697412-4b58-4310-b271-0b4940254fe5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:28.594386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47e3d2-7c42-4094-b679-f23d714950e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:28.595102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47e3d2-7c42-4094-b679-f23d714950e7 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:28.595419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47e3d2-7c42-4094-b679-f23d714950e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:28.596105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47e3d2-7c42-4094-b679-f23d714950e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:28.596845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47e3d2-7c42-4094-b679-f23d714950e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:28.598510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47e3d2-7c42-4094-b679-f23d714950e7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:28.611651 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 989/1942] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:28.614177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6b643-27b3-4a5f-be11-117774a0b281 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:28.614470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6b643-27b3-4a5f-be11-117774a0b281 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:28.614941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6b643-27b3-4a5f-be11-117774a0b281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:28.655684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5ab6b643-27b3-4a5f-be11-117774a0b281 None None] MFA Rules not processed for user `10993092e2244c15b0ee389bdf39c92d`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:28.657252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5ab6b643-27b3-4a5f-be11-117774a0b281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:28.674402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:28.676185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a2032028-d94b-4b66-b98d-8c0bf1a60a52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '753243d6136d43e9904bdeed3bddfd99'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:28.676845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a2032028-d94b-4b66-b98d-8c0bf1a60a52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '753243d6136d43e9904bdeed3bddfd99'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:28.684669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a2032028-d94b-4b66-b98d-8c0bf1a60a52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '753243d6136d43e9904bdeed3bddfd99'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:28.727768 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 990/1943] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:28.728614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6b643-27b3-4a5f-be11-117774a0b281 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:28.738647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6b643-27b3-4a5f-be11-117774a0b281 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:28.742718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a2032028-d94b-4b66-b98d-8c0bf1a60a52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '753243d6136d43e9904bdeed3bddfd99'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:28.743973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a2032028-d94b-4b66-b98d-8c0bf1a60a52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '753243d6136d43e9904bdeed3bddfd99'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:28.746875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166f9fa-3a39-4e93-b618-e0dde37948e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:28.747436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166f9fa-3a39-4e93-b618-e0dde37948e2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:28.747716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166f9fa-3a39-4e93-b618-e0dde37948e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:28.747988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7166f9fa-3a39-4e93-b618-e0dde37948e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:28.748731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166f9fa-3a39-4e93-b618-e0dde37948e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:28.750403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7166f9fa-3a39-4e93-b618-e0dde37948e2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:28.764982 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 991/1944] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:28.765805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166f9fa-3a39-4e93-b618-e0dde37948e2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:28.767835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7166f9fa-3a39-4e93-b618-e0dde37948e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:28.775987 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 954/1945] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:28 2022] DELETE /identity/v3/users/753243d6136d43e9904bdeed3bddfd99 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:28.779158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2032028-d94b-4b66-b98d-8c0bf1a60a52 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:28.782384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a2032028-d94b-4b66-b98d-8c0bf1a60a52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:28.784182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7166f9fa-3a39-4e93-b618-e0dde37948e2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:28.815570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:28.846515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e1383-29cf-4563-8ab7-ba75c696960b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:28.847222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e1383-29cf-4563-8ab7-ba75c696960b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:28.847511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e1383-29cf-4563-8ab7-ba75c696960b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:28.848253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e1383-29cf-4563-8ab7-ba75c696960b admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:28.848493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e1383-29cf-4563-8ab7-ba75c696960b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:28.851173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e1383-29cf-4563-8ab7-ba75c696960b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:28.870637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2032028-d94b-4b66-b98d-8c0bf1a60a52 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:28.875819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce60de-985d-49d0-a402-c53d76ff7e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:28.876685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce60de-985d-49d0-a402-c53d76ff7e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:28.876998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce60de-985d-49d0-a402-c53d76ff7e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:28.877272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce60de-985d-49d0-a402-c53d76ff7e64 admin admin] PATH_INFO: `/v3/projects/2adfdd7542cf42259fc76dfa9387100d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:28.878503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce60de-985d-49d0-a402-c53d76ff7e64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2adfdd7542cf42259fc76dfa9387100d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:28.880420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce60de-985d-49d0-a402-c53d76ff7e64 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:28.936447 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 992/1946] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:28 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:28.936447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e1383-29cf-4563-8ab7-ba75c696960b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:28.936447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-796e1383-29cf-4563-8ab7-ba75c696960b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:28.978268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f1ce60de-985d-49d0-a402-c53d76ff7e64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2adfdd7542cf42259fc76dfa9387100d', 'request_id': 'req-f1ce60de-985d-49d0-a402-c53d76ff7e64'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:28.985144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f1ce60de-985d-49d0-a402-c53d76ff7e64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2adfdd7542cf42259fc76dfa9387100d', 'request_id': 'req-f1ce60de-985d-49d0-a402-c53d76ff7e64'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:29.006172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e1383-29cf-4563-8ab7-ba75c696960b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:29.039613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f1ce60de-985d-49d0-a402-c53d76ff7e64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2adfdd7542cf42259fc76dfa9387100d', 'request_id': 'req-f1ce60de-985d-49d0-a402-c53d76ff7e64'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:29.051588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f1ce60de-985d-49d0-a402-c53d76ff7e64 admin admin] The token cache is being invalidate because project 2adfdd7542cf42259fc76dfa9387100d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:29.051885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f1ce60de-985d-49d0-a402-c53d76ff7e64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:29.052578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f1ce60de-985d-49d0-a402-c53d76ff7e64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:29.055049 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 955/1947] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:28 2022] DELETE /identity/v3/projects/2adfdd7542cf42259fc76dfa9387100d => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:29.056109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e5a35-8a04-4556-8f9e-6f44f654d1a2 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:29.056401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e5a35-8a04-4556-8f9e-6f44f654d1a2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:29.056677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e5a35-8a04-4556-8f9e-6f44f654d1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:29.076246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8b39e-abaf-4133-8095-a5117e01ee08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:29.077012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8b39e-abaf-4133-8095-a5117e01ee08 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:29.077292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8b39e-abaf-4133-8095-a5117e01ee08 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:29.077740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a8b39e-abaf-4133-8095-a5117e01ee08 admin admin] PATH_INFO: `/v3/projects/d6c38a96a96b4fba9e41ac58f5054803/users/c14b965da64c40e382c69050ba01b183/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:29.080551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-069e5a35-8a04-4556-8f9e-6f44f654d1a2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:29.081766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-069e5a35-8a04-4556-8f9e-6f44f654d1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:29.089740 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 956/1948] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:29.091815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e5a35-8a04-4556-8f9e-6f44f654d1a2 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:29.093101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-069e5a35-8a04-4556-8f9e-6f44f654d1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:29.096299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8b39e-abaf-4133-8095-a5117e01ee08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6c38a96a96b4fba9e41ac58f5054803, user_id=c14b965da64c40e382c69050ba01b183, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:29.099764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a8b39e-abaf-4133-8095-a5117e01ee08 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:29.112665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e5a35-8a04-4556-8f9e-6f44f654d1a2 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:29.117929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63684c4-52cf-45d5-bb30-f5645898135e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:29.118640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63684c4-52cf-45d5-bb30-f5645898135e admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:29.118933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63684c4-52cf-45d5-bb30-f5645898135e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:29.119204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63684c4-52cf-45d5-bb30-f5645898135e admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:29.119898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63684c4-52cf-45d5-bb30-f5645898135e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:29.122209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63684c4-52cf-45d5-bb30-f5645898135e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:29.143657 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 993/1949] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:28 2022] PUT /identity/v3/projects/d6c38a96a96b4fba9e41ac58f5054803/users/c14b965da64c40e382c69050ba01b183/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:29.144915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8b39e-abaf-4133-8095-a5117e01ee08 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:29.158287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-01a8b39e-abaf-4133-8095-a5117e01ee08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:29.204415 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 957/1950] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:29 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:29.204415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63684c4-52cf-45d5-bb30-f5645898135e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:29.204801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b63684c4-52cf-45d5-bb30-f5645898135e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:29.214885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:29.233165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a8b39e-abaf-4133-8095-a5117e01ee08 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:29.239034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65faa06-c3bf-4275-96b0-435b92087a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:29.239932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65faa06-c3bf-4275-96b0-435b92087a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:29.240258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65faa06-c3bf-4275-96b0-435b92087a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:29.240566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65faa06-c3bf-4275-96b0-435b92087a04 admin admin] PATH_INFO: `/v3/users/26b3116232ef4be9804330348cf6c7bd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:29.241484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:29.242277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65faa06-c3bf-4275-96b0-435b92087a04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26b3116232ef4be9804330348cf6c7bd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:29.244846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65faa06-c3bf-4275-96b0-435b92087a04 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:29.268414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63684c4-52cf-45d5-bb30-f5645898135e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:29.317679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057291e1-5e3b-43b9-a6ef-5f159f0c0bb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:29.318367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057291e1-5e3b-43b9-a6ef-5f159f0c0bb0 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:29.318656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057291e1-5e3b-43b9-a6ef-5f159f0c0bb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:29.318936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057291e1-5e3b-43b9-a6ef-5f159f0c0bb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:29.328208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057291e1-5e3b-43b9-a6ef-5f159f0c0bb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:29.330059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057291e1-5e3b-43b9-a6ef-5f159f0c0bb0 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:29.340514 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 958/1951] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:29.341694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf0985-133f-4088-9061-6ed4e1d07e2d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:29.347479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf0985-133f-4088-9061-6ed4e1d07e2d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:29.347479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf0985-133f-4088-9061-6ed4e1d07e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:29.369683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1baf0985-133f-4088-9061-6ed4e1d07e2d None None] MFA Rules not processed for user `c14b965da64c40e382c69050ba01b183`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:29.376164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1baf0985-133f-4088-9061-6ed4e1d07e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:29.470348 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 959/1952] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:29.471230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf0985-133f-4088-9061-6ed4e1d07e2d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:29.483740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baf0985-133f-4088-9061-6ed4e1d07e2d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:29.488679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110ed11-dee0-404d-a182-be6fe0538907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:29.489365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110ed11-dee0-404d-a182-be6fe0538907 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:29.489653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110ed11-dee0-404d-a182-be6fe0538907 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:29.489976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4110ed11-dee0-404d-a182-be6fe0538907 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:29.490719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110ed11-dee0-404d-a182-be6fe0538907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:29.492395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4110ed11-dee0-404d-a182-be6fe0538907 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:29.503601 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 960/1953] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:29.507330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110ed11-dee0-404d-a182-be6fe0538907 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:29.509447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4110ed11-dee0-404d-a182-be6fe0538907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:29.532498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4110ed11-dee0-404d-a182-be6fe0538907 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:29.537075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b65faa06-c3bf-4275-96b0-435b92087a04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26b3116232ef4be9804330348cf6c7bd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:29.539652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a0afa-a369-4d78-b714-eab91c922e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:29.540501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a0afa-a369-4d78-b714-eab91c922e51 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:29.541065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a0afa-a369-4d78-b714-eab91c922e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:29.541511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553a0afa-a369-4d78-b714-eab91c922e51 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:29.543083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a0afa-a369-4d78-b714-eab91c922e51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:29.547120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553a0afa-a369-4d78-b714-eab91c922e51 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:29.553498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b65faa06-c3bf-4275-96b0-435b92087a04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26b3116232ef4be9804330348cf6c7bd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:29.555434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b65faa06-c3bf-4275-96b0-435b92087a04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26b3116232ef4be9804330348cf6c7bd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:29.556600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b65faa06-c3bf-4275-96b0-435b92087a04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26b3116232ef4be9804330348cf6c7bd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:29.651413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b65faa06-c3bf-4275-96b0-435b92087a04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26b3116232ef4be9804330348cf6c7bd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:29.667366 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 961/1954] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:29 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:29.668475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a0afa-a369-4d78-b714-eab91c922e51 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:29.671772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-553a0afa-a369-4d78-b714-eab91c922e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:29.693348 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 994/1955] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:29 2022] DELETE /identity/v3/users/26b3116232ef4be9804330348cf6c7bd => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:29.701928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65faa06-c3bf-4275-96b0-435b92087a04 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:29.702406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b65faa06-c3bf-4275-96b0-435b92087a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:29.773677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:29.826317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553a0afa-a369-4d78-b714-eab91c922e51 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:29.835320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65faa06-c3bf-4275-96b0-435b92087a04 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:29.839390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d600a3f-698a-4676-85ce-cd680b45bca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:29.840499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d600a3f-698a-4676-85ce-cd680b45bca4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:29.841443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d600a3f-698a-4676-85ce-cd680b45bca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:29.841952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d600a3f-698a-4676-85ce-cd680b45bca4 admin admin] PATH_INFO: `/v3/projects/324ff1955efe413bbf9f57f7f63dd1d2/users/6b0490e37a9244b69afbac9aa94bad05/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:29.849052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f59de-a4a2-4f60-a6c1-4a0df6bac36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:29.854338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f59de-a4a2-4f60-a6c1-4a0df6bac36a admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:29.854640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f59de-a4a2-4f60-a6c1-4a0df6bac36a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:29.854893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f59de-a4a2-4f60-a6c1-4a0df6bac36a admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:29.855632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f59de-a4a2-4f60-a6c1-4a0df6bac36a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:29.857703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0f59de-a4a2-4f60-a6c1-4a0df6bac36a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:29.874980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d600a3f-698a-4676-85ce-cd680b45bca4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=324ff1955efe413bbf9f57f7f63dd1d2, user_id=6b0490e37a9244b69afbac9aa94bad05, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:29.881165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d600a3f-698a-4676-85ce-cd680b45bca4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:29.987281 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 962/1956] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:29 2022] PUT /identity/v3/projects/324ff1955efe413bbf9f57f7f63dd1d2/users/6b0490e37a9244b69afbac9aa94bad05/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:29.988627 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 995/1957] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:29 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:29.988941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce8698-bbb7-4349-a88f-3504377110f0 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:29.989244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce8698-bbb7-4349-a88f-3504377110f0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:29.989821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ce8698-bbb7-4349-a88f-3504377110f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:29.991942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f59de-a4a2-4f60-a6c1-4a0df6bac36a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:29.994265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1a0f59de-a4a2-4f60-a6c1-4a0df6bac36a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:30.012277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e8ce8698-bbb7-4349-a88f-3504377110f0 None None] MFA Rules not processed for user `cdeaa660f8ec465bba68096d0ed4b1ae`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:30.012975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f59de-a4a2-4f60-a6c1-4a0df6bac36a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:30.013651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e8ce8698-bbb7-4349-a88f-3504377110f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:30.084339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:30.092833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091f723-c9bc-4148-bc14-17f2ed2f0998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:30.093806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091f723-c9bc-4148-bc14-17f2ed2f0998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:30.094762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091f723-c9bc-4148-bc14-17f2ed2f0998 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:30.094762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091f723-c9bc-4148-bc14-17f2ed2f0998 admin admin] PATH_INFO: `/v3/projects/812177ca96534d55a6eedb917b146b66` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:30.095718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091f723-c9bc-4148-bc14-17f2ed2f0998 admin admin] RBAC: Authorizing `identity:delete_project(project_id=812177ca96534d55a6eedb917b146b66)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:30.098207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091f723-c9bc-4148-bc14-17f2ed2f0998 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:30.115113 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 963/1958] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:30.115113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce8698-bbb7-4349-a88f-3504377110f0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:30.122540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ce8698-bbb7-4349-a88f-3504377110f0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:30.127458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb207231-d75a-48b0-930c-194ab6dd429b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:30.128129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb207231-d75a-48b0-930c-194ab6dd429b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:30.128577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb207231-d75a-48b0-930c-194ab6dd429b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:30.128862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb207231-d75a-48b0-930c-194ab6dd429b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:30.131154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb207231-d75a-48b0-930c-194ab6dd429b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:30.133630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb207231-d75a-48b0-930c-194ab6dd429b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:30.144672 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 964/1959] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:30.145614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb207231-d75a-48b0-930c-194ab6dd429b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:30.157154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb207231-d75a-48b0-930c-194ab6dd429b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:30.164115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac0d5e-2797-4cec-9f43-863a34d78350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:30.164115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac0d5e-2797-4cec-9f43-863a34d78350 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:30.164115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac0d5e-2797-4cec-9f43-863a34d78350 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:30.164115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac0d5e-2797-4cec-9f43-863a34d78350 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:30.164710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac0d5e-2797-4cec-9f43-863a34d78350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:30.167340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ac0d5e-2797-4cec-9f43-863a34d78350 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:30.196163 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 965/1960] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:30.196163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac0d5e-2797-4cec-9f43-863a34d78350 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:30.213135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac0d5e-2797-4cec-9f43-863a34d78350 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:30.231987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2091f723-c9bc-4148-bc14-17f2ed2f0998 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '812177ca96534d55a6eedb917b146b66', 'request_id': 'req-2091f723-c9bc-4148-bc14-17f2ed2f0998'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:30.239739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2091f723-c9bc-4148-bc14-17f2ed2f0998 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '812177ca96534d55a6eedb917b146b66', 'request_id': 'req-2091f723-c9bc-4148-bc14-17f2ed2f0998'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:30.282568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5853f-7cf2-44c6-88b5-5ee205a19ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:30.283465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5853f-7cf2-44c6-88b5-5ee205a19ca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:30.283780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5853f-7cf2-44c6-88b5-5ee205a19ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:30.284067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5853f-7cf2-44c6-88b5-5ee205a19ca8 admin admin] PATH_INFO: `/v3/projects/324ff1955efe413bbf9f57f7f63dd1d2/users/6b0490e37a9244b69afbac9aa94bad05/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:30.286080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5853f-7cf2-44c6-88b5-5ee205a19ca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=324ff1955efe413bbf9f57f7f63dd1d2, user_id=6b0490e37a9244b69afbac9aa94bad05, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:30.288798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f5853f-7cf2-44c6-88b5-5ee205a19ca8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:30.306014 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 966/1961] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:30 2022] PUT /identity/v3/projects/324ff1955efe413bbf9f57f7f63dd1d2/users/6b0490e37a9244b69afbac9aa94bad05/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:30.307008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5853f-7cf2-44c6-88b5-5ee205a19ca8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:30.355291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2091f723-c9bc-4148-bc14-17f2ed2f0998 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '812177ca96534d55a6eedb917b146b66', 'request_id': 'req-2091f723-c9bc-4148-bc14-17f2ed2f0998'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:30.357370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:30.369773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2091f723-c9bc-4148-bc14-17f2ed2f0998 admin admin] The token cache is being invalidate because project 812177ca96534d55a6eedb917b146b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:30.370116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2091f723-c9bc-4148-bc14-17f2ed2f0998 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:30.371225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2091f723-c9bc-4148-bc14-17f2ed2f0998 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:30.373061 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 996/1962] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:29 2022] DELETE /identity/v3/projects/812177ca96534d55a6eedb917b146b66 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:30.374720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d3f1b-9bf6-4d1c-a896-1960190b23a1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:30.374835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d3f1b-9bf6-4d1c-a896-1960190b23a1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:30.375047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d3f1b-9bf6-4d1c-a896-1960190b23a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:30.393147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5853f-7cf2-44c6-88b5-5ee205a19ca8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:30.398970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572502b-06fa-4faf-87d8-3525acf5251f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:30.400097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572502b-06fa-4faf-87d8-3525acf5251f admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:30.400408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572502b-06fa-4faf-87d8-3525acf5251f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:30.400686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2572502b-06fa-4faf-87d8-3525acf5251f admin admin] PATH_INFO: `/v3/projects/01b8421c635c440eaec446bfb659bd22/users/a43b061d4e864c289ac541d11e46c4de/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:30.405369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-112d3f1b-9bf6-4d1c-a896-1960190b23a1 None None] MFA Rules not processed for user `6b0490e37a9244b69afbac9aa94bad05`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:30.406700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-112d3f1b-9bf6-4d1c-a896-1960190b23a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:30.432419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572502b-06fa-4faf-87d8-3525acf5251f admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b8421c635c440eaec446bfb659bd22, user_id=a43b061d4e864c289ac541d11e46c4de, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:30.439671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2572502b-06fa-4faf-87d8-3525acf5251f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:30.505576 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 967/1963] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:30 2022] PUT /identity/v3/projects/01b8421c635c440eaec446bfb659bd22/users/a43b061d4e864c289ac541d11e46c4de/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:30.505576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572502b-06fa-4faf-87d8-3525acf5251f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:30.513260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2572502b-06fa-4faf-87d8-3525acf5251f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:30.513339 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 997/1964] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:30.513339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d3f1b-9bf6-4d1c-a896-1960190b23a1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:30.515486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-112d3f1b-9bf6-4d1c-a896-1960190b23a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:30.543140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d3f1b-9bf6-4d1c-a896-1960190b23a1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:30.613815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f27f2a-d5c3-428b-bc3a-351aa6aa93f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:30.614759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f27f2a-d5c3-428b-bc3a-351aa6aa93f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:30.615011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f27f2a-d5c3-428b-bc3a-351aa6aa93f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:30.615473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f27f2a-d5c3-428b-bc3a-351aa6aa93f5 admin admin] PATH_INFO: `/v3/users/5c05235c31c84ed99ed6e18a9f9d9c9c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:30.617703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f27f2a-d5c3-428b-bc3a-351aa6aa93f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c05235c31c84ed99ed6e18a9f9d9c9c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:30.621390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f27f2a-d5c3-428b-bc3a-351aa6aa93f5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:30.627794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2572502b-06fa-4faf-87d8-3525acf5251f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:30.638618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:30.639876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:30.639876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:30.640064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa admin admin] PATH_INFO: `/v3/projects/03dc4ac6b75d454786872add5a5c1cab` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:30.641301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=03dc4ac6b75d454786872add5a5c1cab)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:30.643707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:30.864530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-54f27f2a-d5c3-428b-bc3a-351aa6aa93f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c05235c31c84ed99ed6e18a9f9d9c9c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:30.865067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03dc4ac6b75d454786872add5a5c1cab', 'request_id': 'req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:30.871133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03dc4ac6b75d454786872add5a5c1cab', 'request_id': 'req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:30.872285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-54f27f2a-d5c3-428b-bc3a-351aa6aa93f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c05235c31c84ed99ed6e18a9f9d9c9c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:30.873067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-54f27f2a-d5c3-428b-bc3a-351aa6aa93f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c05235c31c84ed99ed6e18a9f9d9c9c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:30.873669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-54f27f2a-d5c3-428b-bc3a-351aa6aa93f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c05235c31c84ed99ed6e18a9f9d9c9c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:30.936733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03dc4ac6b75d454786872add5a5c1cab', 'request_id': 'req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:30.950031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa admin admin] The token cache is being invalidate because project 03dc4ac6b75d454786872add5a5c1cab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:30.950323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:30.950973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:30.952663 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 968/1965] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:30 2022] DELETE /identity/v3/projects/03dc4ac6b75d454786872add5a5c1cab => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:30.953429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:30.954734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:31.005243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-54f27f2a-d5c3-428b-bc3a-351aa6aa93f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c05235c31c84ed99ed6e18a9f9d9c9c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:31.014342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:31.018252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049537b9-cd9b-46e5-ab86-b3f96cf3e9aa admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:31.023101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cf9719-6fe2-4d64-b044-b86adc17cfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:31.023355 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 998/1966] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:30 2022] DELETE /identity/v3/users/5c05235c31c84ed99ed6e18a9f9d9c9c => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:31.023771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf9719-6fe2-4d64-b044-b86adc17cfe1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:31.024064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf9719-6fe2-4d64-b044-b86adc17cfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:31.024248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f27f2a-d5c3-428b-bc3a-351aa6aa93f5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:31.024361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cf9719-6fe2-4d64-b044-b86adc17cfe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:31.025214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf9719-6fe2-4d64-b044-b86adc17cfe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:31.025546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-54f27f2a-d5c3-428b-bc3a-351aa6aa93f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:31.026882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cf9719-6fe2-4d64-b044-b86adc17cfe1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:31.039526 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 969/1967] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:31.039893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d294792-a14a-4c93-b522-4098daea828a None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:31.046134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d294792-a14a-4c93-b522-4098daea828a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:31.046134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d294792-a14a-4c93-b522-4098daea828a None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:31.046134 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 970/1968] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:31.046134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76331cc-4668-46a8-813e-a4de2ad95c4d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:31.046134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76331cc-4668-46a8-813e-a4de2ad95c4d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:31.046134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76331cc-4668-46a8-813e-a4de2ad95c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:31.068547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e76331cc-4668-46a8-813e-a4de2ad95c4d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:31.072427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e76331cc-4668-46a8-813e-a4de2ad95c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:31.092082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:31.127687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f27f2a-d5c3-428b-bc3a-351aa6aa93f5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:31.133301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2692792-4ba7-45d7-ab2e-d1587d1f9401 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:31.134032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2692792-4ba7-45d7-ab2e-d1587d1f9401 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:31.134346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2692792-4ba7-45d7-ab2e-d1587d1f9401 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:31.134627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2692792-4ba7-45d7-ab2e-d1587d1f9401 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:31.136754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b2692792-4ba7-45d7-ab2e-d1587d1f9401 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:31.140837 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 971/1969] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:31.141597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76331cc-4668-46a8-813e-a4de2ad95c4d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:31.151431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76331cc-4668-46a8-813e-a4de2ad95c4d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:31.156301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d15099-c8d4-4ade-97aa-e540d2e81aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:31.157192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d15099-c8d4-4ade-97aa-e540d2e81aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:31.157480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d15099-c8d4-4ade-97aa-e540d2e81aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:31.158140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d15099-c8d4-4ade-97aa-e540d2e81aeb admin admin] PATH_INFO: `/v3/projects/01b8421c635c440eaec446bfb659bd22/users/a43b061d4e864c289ac541d11e46c4de/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:31.159957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d15099-c8d4-4ade-97aa-e540d2e81aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b8421c635c440eaec446bfb659bd22, user_id=a43b061d4e864c289ac541d11e46c4de, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:31.162704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d15099-c8d4-4ade-97aa-e540d2e81aeb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:31.174064 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 972/1970] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:31 2022] PUT /identity/v3/projects/01b8421c635c440eaec446bfb659bd22/users/a43b061d4e864c289ac541d11e46c4de/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:31.175351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58b554-1c35-43e4-8296-d5e293ecac41 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:31.175647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58b554-1c35-43e4-8296-d5e293ecac41 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:31.175926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a58b554-1c35-43e4-8296-d5e293ecac41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:31.201750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2692792-4ba7-45d7-ab2e-d1587d1f9401 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:31.202017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4a58b554-1c35-43e4-8296-d5e293ecac41 None None] MFA Rules not processed for user `1b2a9ebcd3f84b758322a80c6e590de0`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:31.203207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4a58b554-1c35-43e4-8296-d5e293ecac41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:31.203458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2692792-4ba7-45d7-ab2e-d1587d1f9401 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:31.213064 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 999/1971] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:31.214666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3e027-c7ed-4967-bc25-a8cefb5a40e7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:31.214952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3e027-c7ed-4967-bc25-a8cefb5a40e7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:31.215240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be3e027-c7ed-4967-bc25-a8cefb5a40e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:31.233524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1be3e027-c7ed-4967-bc25-a8cefb5a40e7 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:31.235048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1be3e027-c7ed-4967-bc25-a8cefb5a40e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:31.261748 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 973/1972] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:03:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:31.262870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751fc8c-90ef-4a30-a42e-59ee3b57ba8e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:31.263166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751fc8c-90ef-4a30-a42e-59ee3b57ba8e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:31.263441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751fc8c-90ef-4a30-a42e-59ee3b57ba8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:31.306971 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1000/1973] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:31.310142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3751fc8c-90ef-4a30-a42e-59ee3b57ba8e None None] MFA Rules not processed for user `a43b061d4e864c289ac541d11e46c4de`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:31.310142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3751fc8c-90ef-4a30-a42e-59ee3b57ba8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:31.310341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3e027-c7ed-4967-bc25-a8cefb5a40e7 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:31.312438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1be3e027-c7ed-4967-bc25-a8cefb5a40e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:31.325131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be3e027-c7ed-4967-bc25-a8cefb5a40e7 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:31.339443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08dac0-6a48-44ef-bcdc-e86208675ec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:31.339443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08dac0-6a48-44ef-bcdc-e86208675ec3 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:31.339443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08dac0-6a48-44ef-bcdc-e86208675ec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:31.339443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08dac0-6a48-44ef-bcdc-e86208675ec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:31.339723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08dac0-6a48-44ef-bcdc-e86208675ec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:31.341247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc08dac0-6a48-44ef-bcdc-e86208675ec3 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:31.351419 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1001/1974] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:31.352387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08dac0-6a48-44ef-bcdc-e86208675ec3 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:31.373540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08dac0-6a48-44ef-bcdc-e86208675ec3 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:31.379100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494270d-5a04-4d82-8562-aa1e17fb6d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:31.380026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494270d-5a04-4d82-8562-aa1e17fb6d73 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:31.380322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494270d-5a04-4d82-8562-aa1e17fb6d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:31.380717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494270d-5a04-4d82-8562-aa1e17fb6d73 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:31.382263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494270d-5a04-4d82-8562-aa1e17fb6d73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:03:31.382263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494270d-5a04-4d82-8562-aa1e17fb6d73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:31.384902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494270d-5a04-4d82-8562-aa1e17fb6d73 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:31.400506 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1002/1975] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:31.406177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494270d-5a04-4d82-8562-aa1e17fb6d73 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:31.424212 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 974/1976] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:31.428002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494270d-5a04-4d82-8562-aa1e17fb6d73 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:31.437826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351be45-5f4e-4e63-a91a-c8d826165a28 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:31.440912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351be45-5f4e-4e63-a91a-c8d826165a28 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:31.441477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351be45-5f4e-4e63-a91a-c8d826165a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:31.453298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e8e58-4feb-4a51-8ea8-d561337f00f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:31.454165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e8e58-4feb-4a51-8ea8-d561337f00f8 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:31.454512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e8e58-4feb-4a51-8ea8-d561337f00f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:31.455037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e8e58-4feb-4a51-8ea8-d561337f00f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:31.455763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e8e58-4feb-4a51-8ea8-d561337f00f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:31.457892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e8e58-4feb-4a51-8ea8-d561337f00f8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:31.472841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2351be45-5f4e-4e63-a91a-c8d826165a28 None None] MFA Rules not processed for user `c14b965da64c40e382c69050ba01b183`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:31.474073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2351be45-5f4e-4e63-a91a-c8d826165a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:31.567623 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 975/1977] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:31.599140 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1003/1978] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:31 2022] POST /identity/v3/projects => generated 395 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:31.601311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351be45-5f4e-4e63-a91a-c8d826165a28 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:31.624788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351be45-5f4e-4e63-a91a-c8d826165a28 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:31.691547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7150b17-4734-43cb-b05e-406088e35fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:31.695533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7150b17-4734-43cb-b05e-406088e35fcc admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:31.695875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e8e58-4feb-4a51-8ea8-d561337f00f8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:31.695995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7150b17-4734-43cb-b05e-406088e35fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:31.696308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7150b17-4734-43cb-b05e-406088e35fcc admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:31.697100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7150b17-4734-43cb-b05e-406088e35fcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:31.698123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e54e8e58-4feb-4a51-8ea8-d561337f00f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:31.699112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7150b17-4734-43cb-b05e-406088e35fcc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:31.722966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e8e58-4feb-4a51-8ea8-d561337f00f8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:31.733050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0148ac61-4609-4d34-8dbe-39af7c3491ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:31.734218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148ac61-4609-4d34-8dbe-39af7c3491ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:31.735958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148ac61-4609-4d34-8dbe-39af7c3491ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:31.736248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0148ac61-4609-4d34-8dbe-39af7c3491ad admin admin] PATH_INFO: `/v3/projects/9397ab700e3b472597d1462ad128d87f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:31.739506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148ac61-4609-4d34-8dbe-39af7c3491ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=9397ab700e3b472597d1462ad128d87f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:31.742130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0148ac61-4609-4d34-8dbe-39af7c3491ad admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:31.779997 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 976/1979] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:31 2022] POST /identity/v3/users => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:31.785740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:31.787011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7150b17-4734-43cb-b05e-406088e35fcc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:31.800659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7150b17-4734-43cb-b05e-406088e35fcc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:31.806444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0006819-73f6-4b9d-b24e-35eaddf5199c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:31.807162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0006819-73f6-4b9d-b24e-35eaddf5199c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:31.807447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0006819-73f6-4b9d-b24e-35eaddf5199c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:31.807731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0006819-73f6-4b9d-b24e-35eaddf5199c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:31.808508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0006819-73f6-4b9d-b24e-35eaddf5199c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:31.810304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0006819-73f6-4b9d-b24e-35eaddf5199c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:31.821809 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 977/1980] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:31.825668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0006819-73f6-4b9d-b24e-35eaddf5199c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:31.836765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0006819-73f6-4b9d-b24e-35eaddf5199c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:31.842497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74ccf4-53a6-4789-9e24-6538e1e46e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:31.843309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74ccf4-53a6-4789-9e24-6538e1e46e8c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:31.843659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74ccf4-53a6-4789-9e24-6538e1e46e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:31.844004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74ccf4-53a6-4789-9e24-6538e1e46e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:31.844811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74ccf4-53a6-4789-9e24-6538e1e46e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:31.846572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74ccf4-53a6-4789-9e24-6538e1e46e8c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:31.858642 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 978/1981] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:31.863384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:31.864669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74ccf4-53a6-4789-9e24-6538e1e46e8c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:31.875399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74ccf4-53a6-4789-9e24-6538e1e46e8c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:31.880057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30747539-3041-46da-a7cf-2b083fc45f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:31.880941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30747539-3041-46da-a7cf-2b083fc45f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:31.881218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30747539-3041-46da-a7cf-2b083fc45f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:31.881490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30747539-3041-46da-a7cf-2b083fc45f74 admin admin] PATH_INFO: `/v3/projects/45a2b4e7e1484ea3807bc8d8bc31296c/users/fc6ee6dba5d24e90bff48384539fcd7f/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:31.896391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30747539-3041-46da-a7cf-2b083fc45f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a2b4e7e1484ea3807bc8d8bc31296c, user_id=fc6ee6dba5d24e90bff48384539fcd7f, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:31.899180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30747539-3041-46da-a7cf-2b083fc45f74 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:31.987408 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 979/1982] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:31 2022] PUT /identity/v3/projects/45a2b4e7e1484ea3807bc8d8bc31296c/users/fc6ee6dba5d24e90bff48384539fcd7f/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:31.988343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30747539-3041-46da-a7cf-2b083fc45f74 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:31.990200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-30747539-3041-46da-a7cf-2b083fc45f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:31.999405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0148ac61-4609-4d34-8dbe-39af7c3491ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9397ab700e3b472597d1462ad128d87f', 'request_id': 'req-0148ac61-4609-4d34-8dbe-39af7c3491ad'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:32.009280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0148ac61-4609-4d34-8dbe-39af7c3491ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9397ab700e3b472597d1462ad128d87f', 'request_id': 'req-0148ac61-4609-4d34-8dbe-39af7c3491ad'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:32.085519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0148ac61-4609-4d34-8dbe-39af7c3491ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9397ab700e3b472597d1462ad128d87f', 'request_id': 'req-0148ac61-4609-4d34-8dbe-39af7c3491ad'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:32.105252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0148ac61-4609-4d34-8dbe-39af7c3491ad admin admin] The token cache is being invalidate because project 9397ab700e3b472597d1462ad128d87f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:32.107780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0148ac61-4609-4d34-8dbe-39af7c3491ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:32.111028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0148ac61-4609-4d34-8dbe-39af7c3491ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:32.113453 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1004/1983] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:31 2022] DELETE /identity/v3/projects/9397ab700e3b472597d1462ad128d87f => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:03:32.121656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce61afc-660e-4a45-b11f-76fd67583ad7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:32.124250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce61afc-660e-4a45-b11f-76fd67583ad7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:32.124546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce61afc-660e-4a45-b11f-76fd67583ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:32.145439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30747539-3041-46da-a7cf-2b083fc45f74 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:32.154931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-dce61afc-660e-4a45-b11f-76fd67583ad7 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:32.156727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dce61afc-660e-4a45-b11f-76fd67583ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:32.215610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4783d706-e319-4015-b763-0892712bec44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:32.216229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783d706-e319-4015-b763-0892712bec44 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:32.216581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783d706-e319-4015-b763-0892712bec44 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:32.216840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4783d706-e319-4015-b763-0892712bec44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:32.222914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4783d706-e319-4015-b763-0892712bec44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:32.223100 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1005/1984] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:32.224733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800a2bd-93d9-434f-a6c1-564488d2192f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:32.225028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800a2bd-93d9-434f-a6c1-564488d2192f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:32.225308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800a2bd-93d9-434f-a6c1-564488d2192f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:32.253521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9800a2bd-93d9-434f-a6c1-564488d2192f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:32.254816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9800a2bd-93d9-434f-a6c1-564488d2192f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:32.264725 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1006/1985] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:32.266121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82277284-00cc-4076-8ce3-283711bdaf4f None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:32.266409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82277284-00cc-4076-8ce3-283711bdaf4f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:32.266715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82277284-00cc-4076-8ce3-283711bdaf4f None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:32.268379 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1007/1986] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:32.269129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82277284-00cc-4076-8ce3-283711bdaf4f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:32.274161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-82277284-00cc-4076-8ce3-283711bdaf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:32.313677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783d706-e319-4015-b763-0892712bec44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:32.315381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4783d706-e319-4015-b763-0892712bec44 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:32.326837 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 980/1987] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:32.328632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0999-9ecf-47d1-b167-239ab3b085fa None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:32.328632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0999-9ecf-47d1-b167-239ab3b085fa None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:32.331881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0999-9ecf-47d1-b167-239ab3b085fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:32.360330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82277284-00cc-4076-8ce3-283711bdaf4f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:32.360418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-909b0999-9ecf-47d1-b167-239ab3b085fa None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:32.360418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-909b0999-9ecf-47d1-b167-239ab3b085fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:32.361324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27a8e6-91ec-419c-b325-35293012c489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:32.362428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27a8e6-91ec-419c-b325-35293012c489 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:32.362946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27a8e6-91ec-419c-b325-35293012c489 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:32.363241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb27a8e6-91ec-419c-b325-35293012c489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:32.368949 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 981/1988] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:32.370267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71569097-c179-4c6f-a4b5-1558abc5b3b0 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:32.370822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71569097-c179-4c6f-a4b5-1558abc5b3b0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:32.371099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71569097-c179-4c6f-a4b5-1558abc5b3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:32.374829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb27a8e6-91ec-419c-b325-35293012c489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:32.377465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb27a8e6-91ec-419c-b325-35293012c489 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:32.390564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-71569097-c179-4c6f-a4b5-1558abc5b3b0 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:32.391756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-71569097-c179-4c6f-a4b5-1558abc5b3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:32.395671 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1008/1989] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:32.396508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27a8e6-91ec-419c-b325-35293012c489 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:32.408484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb27a8e6-91ec-419c-b325-35293012c489 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:32.409980 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 982/1990] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:03:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:32.416108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144b6e4-99a6-4bba-8867-f4d06bf3c5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:32.416108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144b6e4-99a6-4bba-8867-f4d06bf3c5a9 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:32.416108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144b6e4-99a6-4bba-8867-f4d06bf3c5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:32.416108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144b6e4-99a6-4bba-8867-f4d06bf3c5a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:32.416607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144b6e4-99a6-4bba-8867-f4d06bf3c5a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:03:32.416703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144b6e4-99a6-4bba-8867-f4d06bf3c5a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:32.418368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144b6e4-99a6-4bba-8867-f4d06bf3c5a9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:32.431775 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1009/1991] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:32.438928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71569097-c179-4c6f-a4b5-1558abc5b3b0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:32.442871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144b6e4-99a6-4bba-8867-f4d06bf3c5a9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:32.447768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8144b6e4-99a6-4bba-8867-f4d06bf3c5a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:32.453151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71569097-c179-4c6f-a4b5-1558abc5b3b0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:32.458844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee8f5a-dcaa-4721-b34d-55ec2b833c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:32.458844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee8f5a-dcaa-4721-b34d-55ec2b833c49 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:32.459055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee8f5a-dcaa-4721-b34d-55ec2b833c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:32.459231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ee8f5a-dcaa-4721-b34d-55ec2b833c49 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:32.460457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee8f5a-dcaa-4721-b34d-55ec2b833c49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:32.463075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ee8f5a-dcaa-4721-b34d-55ec2b833c49 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:32.480238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144b6e4-99a6-4bba-8867-f4d06bf3c5a9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:32.480420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:32.480541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d2da9-b5eb-4be3-8829-3a5324f3d5c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:32.480541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d2da9-b5eb-4be3-8829-3a5324f3d5c8 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:32.480541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d2da9-b5eb-4be3-8829-3a5324f3d5c8 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:32.480541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d2da9-b5eb-4be3-8829-3a5324f3d5c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:32.484457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d2da9-b5eb-4be3-8829-3a5324f3d5c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:32.486185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d2da9-b5eb-4be3-8829-3a5324f3d5c8 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:32.499656 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1010/1992] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:32.576003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d2da9-b5eb-4be3-8829-3a5324f3d5c8 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:32.586231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-123d2da9-b5eb-4be3-8829-3a5324f3d5c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:32.598039 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 983/1993] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:32 2022] POST /identity/v3/projects => generated 387 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:32.603140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee8f5a-dcaa-4721-b34d-55ec2b833c49 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:32.627221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d2da9-b5eb-4be3-8829-3a5324f3d5c8 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:32.696957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc3820-e87a-4f32-b1ff-26b07b5a55db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:32.697900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc3820-e87a-4f32-b1ff-26b07b5a55db admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:32.698336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc3820-e87a-4f32-b1ff-26b07b5a55db admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:32.698740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc3820-e87a-4f32-b1ff-26b07b5a55db admin admin] PATH_INFO: `/v3/users/10993092e2244c15b0ee389bdf39c92d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:32.700932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc3820-e87a-4f32-b1ff-26b07b5a55db admin admin] RBAC: Authorizing `identity:delete_user(user_id=10993092e2244c15b0ee389bdf39c92d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:32.718630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc3820-e87a-4f32-b1ff-26b07b5a55db admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:32.718732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ee8f5a-dcaa-4721-b34d-55ec2b833c49 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:32.735275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d3e29-5371-4344-9ec5-b7718b9bdf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:32.736157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d3e29-5371-4344-9ec5-b7718b9bdf8c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:32.736503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d3e29-5371-4344-9ec5-b7718b9bdf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:32.737191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d3e29-5371-4344-9ec5-b7718b9bdf8c admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:32.738066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d3e29-5371-4344-9ec5-b7718b9bdf8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:32.740702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d3e29-5371-4344-9ec5-b7718b9bdf8c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:32.755041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:32.848487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:32.857733 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 984/1994] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:32 2022] POST /identity/v3/users => generated 294 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:32.863248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d3e29-5371-4344-9ec5-b7718b9bdf8c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:32.880461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d3e29-5371-4344-9ec5-b7718b9bdf8c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:32.886466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79981fae-592a-4413-b7dd-19a8b302908d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:32.887181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79981fae-592a-4413-b7dd-19a8b302908d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:32.887469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79981fae-592a-4413-b7dd-19a8b302908d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:32.887855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79981fae-592a-4413-b7dd-19a8b302908d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:32.888775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79981fae-592a-4413-b7dd-19a8b302908d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:32.890543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79981fae-592a-4413-b7dd-19a8b302908d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:32.923134 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 985/1995] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:32.928133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79981fae-592a-4413-b7dd-19a8b302908d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:32.959580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79981fae-592a-4413-b7dd-19a8b302908d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:32.968149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8365a9-caaf-4a51-945b-0b7375415813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:32.971161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-28bc3820-e87a-4f32-b1ff-26b07b5a55db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10993092e2244c15b0ee389bdf39c92d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:32.971601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8365a9-caaf-4a51-945b-0b7375415813 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:32.971945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8365a9-caaf-4a51-945b-0b7375415813 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:32.972413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8365a9-caaf-4a51-945b-0b7375415813 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:32.974057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8365a9-caaf-4a51-945b-0b7375415813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:32.977693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8365a9-caaf-4a51-945b-0b7375415813 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:32.991936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-28bc3820-e87a-4f32-b1ff-26b07b5a55db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10993092e2244c15b0ee389bdf39c92d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:32.994327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-28bc3820-e87a-4f32-b1ff-26b07b5a55db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10993092e2244c15b0ee389bdf39c92d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:32.995149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-28bc3820-e87a-4f32-b1ff-26b07b5a55db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10993092e2244c15b0ee389bdf39c92d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:32.996455 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 986/1996] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:33.003743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8365a9-caaf-4a51-945b-0b7375415813 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:33.007290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fb8365a9-caaf-4a51-945b-0b7375415813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:33.025414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8365a9-caaf-4a51-945b-0b7375415813 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:33.032496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b78908-48c9-4b04-84f0-f0043924533e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:33.034646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b78908-48c9-4b04-84f0-f0043924533e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:33.035373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b78908-48c9-4b04-84f0-f0043924533e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:33.036915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b78908-48c9-4b04-84f0-f0043924533e admin admin] PATH_INFO: `/v3/projects/b6c7378af3e841f58fdf1a907d06bc08/users/0e733cc9497a48cba4ccdde860be104f/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:33.056581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b78908-48c9-4b04-84f0-f0043924533e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6c7378af3e841f58fdf1a907d06bc08, user_id=0e733cc9497a48cba4ccdde860be104f, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:33.059673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b78908-48c9-4b04-84f0-f0043924533e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:33.091039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-28bc3820-e87a-4f32-b1ff-26b07b5a55db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10993092e2244c15b0ee389bdf39c92d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:33.129954 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1011/1997] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:32 2022] DELETE /identity/v3/users/10993092e2244c15b0ee389bdf39c92d => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:03:33.133185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec49c8c-a3d4-44ec-996b-736995e8bb95 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:33.133490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec49c8c-a3d4-44ec-996b-736995e8bb95 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:33.133772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec49c8c-a3d4-44ec-996b-736995e8bb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:33.161955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6ec49c8c-a3d4-44ec-996b-736995e8bb95 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:33.161955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6ec49c8c-a3d4-44ec-996b-736995e8bb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:33.167524 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 987/1998] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:33 2022] PUT /identity/v3/projects/b6c7378af3e841f58fdf1a907d06bc08/users/0e733cc9497a48cba4ccdde860be104f/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:33.185929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1addf6d-c815-47af-9aad-49928e943171 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:33.186224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1addf6d-c815-47af-9aad-49928e943171 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:33.186506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1addf6d-c815-47af-9aad-49928e943171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:33.214135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b1addf6d-c815-47af-9aad-49928e943171 None None] MFA Rules not processed for user `0e733cc9497a48cba4ccdde860be104f`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:33.215393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b1addf6d-c815-47af-9aad-49928e943171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:33.234297 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1012/1999] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:33.245644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec49c8c-a3d4-44ec-996b-736995e8bb95 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:33.247403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6ec49c8c-a3d4-44ec-996b-736995e8bb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:33.293079 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 988/2000] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:03:33.294009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1addf6d-c815-47af-9aad-49928e943171 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:33.323421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec49c8c-a3d4-44ec-996b-736995e8bb95 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:33.412998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1addf6d-c815-47af-9aad-49928e943171 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:33.430060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4ba23-eb87-4bca-b8f2-8b0d573d3986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:33.430886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4ba23-eb87-4bca-b8f2-8b0d573d3986 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:33.437989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4ba23-eb87-4bca-b8f2-8b0d573d3986 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:33.437989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4ba23-eb87-4bca-b8f2-8b0d573d3986 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:33.437989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4ba23-eb87-4bca-b8f2-8b0d573d3986 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:33.438274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802c584-0325-46bd-b159-242b6aa60b39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:33.438274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802c584-0325-46bd-b159-242b6aa60b39 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:33.438274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802c584-0325-46bd-b159-242b6aa60b39 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:33.438274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802c584-0325-46bd-b159-242b6aa60b39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:33.438547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4ba23-eb87-4bca-b8f2-8b0d573d3986 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:33.451365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802c584-0325-46bd-b159-242b6aa60b39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:33.452857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802c584-0325-46bd-b159-242b6aa60b39 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:33.463698 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1013/2001] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:03:33.558898 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 989/2002] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:33 2022] POST /identity/v3/projects => generated 385 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:33.560007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802c584-0325-46bd-b159-242b6aa60b39 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:33.570966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802c584-0325-46bd-b159-242b6aa60b39 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:33.665311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:33.668626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aff178-5f57-4022-bd19-b95aad0323cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:33.668626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aff178-5f57-4022-bd19-b95aad0323cd admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:33.668626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aff178-5f57-4022-bd19-b95aad0323cd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:33.668962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aff178-5f57-4022-bd19-b95aad0323cd admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:33.672012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aff178-5f57-4022-bd19-b95aad0323cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:33.675617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aff178-5f57-4022-bd19-b95aad0323cd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:33.804338 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1014/2003] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:33 2022] POST /identity/v3/users => generated 299 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:03:33.807866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4ba23-eb87-4bca-b8f2-8b0d573d3986 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:33.822142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4ba23-eb87-4bca-b8f2-8b0d573d3986 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:33.828753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563749a-cb84-49c5-b01b-5f220c4de4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:33.829105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563749a-cb84-49c5-b01b-5f220c4de4c7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:33.829517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563749a-cb84-49c5-b01b-5f220c4de4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:33.829934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563749a-cb84-49c5-b01b-5f220c4de4c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:33.830783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563749a-cb84-49c5-b01b-5f220c4de4c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:33.848863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a563749a-cb84-49c5-b01b-5f220c4de4c7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:33.871744 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 990/2004] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:33.876433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aff178-5f57-4022-bd19-b95aad0323cd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:33.896346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aff178-5f57-4022-bd19-b95aad0323cd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:33.904914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60499b-3f23-4cc0-a457-9a01981c4ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:33.906136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60499b-3f23-4cc0-a457-9a01981c4ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:33.906652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60499b-3f23-4cc0-a457-9a01981c4ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:33.906968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60499b-3f23-4cc0-a457-9a01981c4ab7 admin admin] PATH_INFO: `/v3/projects/2a0bc9e83b434c4c93a514f9c0ee106b/users/a943a5a756c241379440c3ad9b937435/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:33.928547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60499b-3f23-4cc0-a457-9a01981c4ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a0bc9e83b434c4c93a514f9c0ee106b, user_id=a943a5a756c241379440c3ad9b937435, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:33.931479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60499b-3f23-4cc0-a457-9a01981c4ab7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:33.979166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563749a-cb84-49c5-b01b-5f220c4de4c7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:33.981680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a563749a-cb84-49c5-b01b-5f220c4de4c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:34.000773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563749a-cb84-49c5-b01b-5f220c4de4c7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:34.008517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9df2d-dde7-473b-9891-72167db9cd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:34.009637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9df2d-dde7-473b-9891-72167db9cd7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:34.010456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9df2d-dde7-473b-9891-72167db9cd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:34.010761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9df2d-dde7-473b-9891-72167db9cd7b admin admin] PATH_INFO: `/v3/projects/6ce3b7e72b974127a0b9f4c0db2d0bf6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:34.011984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9df2d-dde7-473b-9891-72167db9cd7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ce3b7e72b974127a0b9f4c0db2d0bf6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:34.013994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f9df2d-dde7-473b-9891-72167db9cd7b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:34.021467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:34.057240 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1015/2005] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:33 2022] PUT /identity/v3/projects/2a0bc9e83b434c4c93a514f9c0ee106b/users/a943a5a756c241379440c3ad9b937435/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:34.068888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60499b-3f23-4cc0-a457-9a01981c4ab7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:34.140895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b7f9df2d-dde7-473b-9891-72167db9cd7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ce3b7e72b974127a0b9f4c0db2d0bf6', 'request_id': 'req-b7f9df2d-dde7-473b-9891-72167db9cd7b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:34.153137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b7f9df2d-dde7-473b-9891-72167db9cd7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ce3b7e72b974127a0b9f4c0db2d0bf6', 'request_id': 'req-b7f9df2d-dde7-473b-9891-72167db9cd7b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:34.187168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60499b-3f23-4cc0-a457-9a01981c4ab7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:34.239598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b7f9df2d-dde7-473b-9891-72167db9cd7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ce3b7e72b974127a0b9f4c0db2d0bf6', 'request_id': 'req-b7f9df2d-dde7-473b-9891-72167db9cd7b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:34.247911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545a126-a478-41e6-9a60-c4658b6107d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:34.253128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545a126-a478-41e6-9a60-c4658b6107d2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:34.253128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545a126-a478-41e6-9a60-c4658b6107d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:34.253128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545a126-a478-41e6-9a60-c4658b6107d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:34.253128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545a126-a478-41e6-9a60-c4658b6107d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:34.256342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545a126-a478-41e6-9a60-c4658b6107d2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:34.256617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b7f9df2d-dde7-473b-9891-72167db9cd7b admin admin] The token cache is being invalidate because project 6ce3b7e72b974127a0b9f4c0db2d0bf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:34.257364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b7f9df2d-dde7-473b-9891-72167db9cd7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:34.258169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b7f9df2d-dde7-473b-9891-72167db9cd7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:34.260545 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 991/2006] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:33 2022] DELETE /identity/v3/projects/6ce3b7e72b974127a0b9f4c0db2d0bf6 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:34.264929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9df2d-dde7-473b-9891-72167db9cd7b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:34.267265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b7f9df2d-dde7-473b-9891-72167db9cd7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:34.270780 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1016/2007] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:34.274616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545a126-a478-41e6-9a60-c4658b6107d2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:34.276367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e545a126-a478-41e6-9a60-c4658b6107d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:34.283106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9df2d-dde7-473b-9891-72167db9cd7b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:34.292926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76618dbe-a1c5-4688-8b0c-1fd2fd53af30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:34.292926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76618dbe-a1c5-4688-8b0c-1fd2fd53af30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:34.292926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76618dbe-a1c5-4688-8b0c-1fd2fd53af30 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:34.292926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76618dbe-a1c5-4688-8b0c-1fd2fd53af30 admin admin] PATH_INFO: `/v3/users/6b0490e37a9244b69afbac9aa94bad05` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:34.294046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76618dbe-a1c5-4688-8b0c-1fd2fd53af30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b0490e37a9244b69afbac9aa94bad05)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:34.296670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76618dbe-a1c5-4688-8b0c-1fd2fd53af30 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:34.300958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545a126-a478-41e6-9a60-c4658b6107d2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:34.321228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78aaac7-5997-4d23-b8f6-5250cd4baf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:34.323802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78aaac7-5997-4d23-b8f6-5250cd4baf96 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:34.325438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78aaac7-5997-4d23-b8f6-5250cd4baf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:34.326440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78aaac7-5997-4d23-b8f6-5250cd4baf96 admin admin] PATH_INFO: `/v3/projects/2a0bc9e83b434c4c93a514f9c0ee106b/users/a943a5a756c241379440c3ad9b937435/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:34.328543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78aaac7-5997-4d23-b8f6-5250cd4baf96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a0bc9e83b434c4c93a514f9c0ee106b, user_id=a943a5a756c241379440c3ad9b937435, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:34.331773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78aaac7-5997-4d23-b8f6-5250cd4baf96 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:34.344315 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1017/2008] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:34 2022] PUT /identity/v3/projects/2a0bc9e83b434c4c93a514f9c0ee106b/users/a943a5a756c241379440c3ad9b937435/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:03:34.355082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b37d1-6c2c-4364-915a-95a6dbecde1d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:34.355534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b37d1-6c2c-4364-915a-95a6dbecde1d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:34.355863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442b37d1-6c2c-4364-915a-95a6dbecde1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:34.394199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-442b37d1-6c2c-4364-915a-95a6dbecde1d None None] MFA Rules not processed for user `a943a5a756c241379440c3ad9b937435`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:34.395622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-442b37d1-6c2c-4364-915a-95a6dbecde1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:34.502013 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1018/2009] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:03:34.504151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41278e-c6f9-4328-9c08-3bed82f77c81 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:34.504664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41278e-c6f9-4328-9c08-3bed82f77c81 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:34.504949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41278e-c6f9-4328-9c08-3bed82f77c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:34.525129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-76618dbe-a1c5-4688-8b0c-1fd2fd53af30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b0490e37a9244b69afbac9aa94bad05'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:34.527652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-76618dbe-a1c5-4688-8b0c-1fd2fd53af30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b0490e37a9244b69afbac9aa94bad05'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:34.543574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-76618dbe-a1c5-4688-8b0c-1fd2fd53af30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b0490e37a9244b69afbac9aa94bad05'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:34.560887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6b41278e-c6f9-4328-9c08-3bed82f77c81 None None] MFA Rules not processed for user `fc6ee6dba5d24e90bff48384539fcd7f`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:34.562150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6b41278e-c6f9-4328-9c08-3bed82f77c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:34.617442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-76618dbe-a1c5-4688-8b0c-1fd2fd53af30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b0490e37a9244b69afbac9aa94bad05'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:34.618205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-76618dbe-a1c5-4688-8b0c-1fd2fd53af30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b0490e37a9244b69afbac9aa94bad05'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:34.648884 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 992/2010] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:34 2022] DELETE /identity/v3/users/6b0490e37a9244b69afbac9aa94bad05 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:03:34.651690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aeb605-f0af-465a-86cc-7388a818c7d3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:34.652289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aeb605-f0af-465a-86cc-7388a818c7d3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:34.652796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aeb605-f0af-465a-86cc-7388a818c7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:34.683509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-44aeb605-f0af-465a-86cc-7388a818c7d3 None None] MFA Rules not processed for user `0e733cc9497a48cba4ccdde860be104f`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:34.685004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-44aeb605-f0af-465a-86cc-7388a818c7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:34.714168 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1019/2011] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:34.718463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41278e-c6f9-4328-9c08-3bed82f77c81 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:34.721501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6b41278e-c6f9-4328-9c08-3bed82f77c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:34.756668 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 993/2012] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:34.763568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:34.768691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aeb605-f0af-465a-86cc-7388a818c7d3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:34.770326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-44aeb605-f0af-465a-86cc-7388a818c7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:34.820122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:34.863434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41278e-c6f9-4328-9c08-3bed82f77c81 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:34.871049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aeb605-f0af-465a-86cc-7388a818c7d3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:34.881872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9ea61-5ffd-4b98-a070-2769d1455534 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:34.882575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9ea61-5ffd-4b98-a070-2769d1455534 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:34.882908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9ea61-5ffd-4b98-a070-2769d1455534 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:34.883150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9ea61-5ffd-4b98-a070-2769d1455534 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:34.885252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-33f9ea61-5ffd-4b98-a070-2769d1455534 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:34.897656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9ea61-5ffd-4b98-a070-2769d1455534 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:34.900987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9ea61-5ffd-4b98-a070-2769d1455534 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:34.911455 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 994/2013] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:34.965131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51424db-f1d1-4805-814b-b193686d99f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:34.966034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51424db-f1d1-4805-814b-b193686d99f9 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:34.966380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51424db-f1d1-4805-814b-b193686d99f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:34.966666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51424db-f1d1-4805-814b-b193686d99f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:34.967380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51424db-f1d1-4805-814b-b193686d99f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:34.970630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51424db-f1d1-4805-814b-b193686d99f9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:35.087475 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1020/2014] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:34 2022] POST /identity/v3/projects => generated 395 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:35.088696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9ea61-5ffd-4b98-a070-2769d1455534 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:35.103181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9ea61-5ffd-4b98-a070-2769d1455534 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:35.190758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647724d6-ee79-4d21-adb6-8baca1a9e1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:35.191718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647724d6-ee79-4d21-adb6-8baca1a9e1a5 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:35.192297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647724d6-ee79-4d21-adb6-8baca1a9e1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:35.193316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647724d6-ee79-4d21-adb6-8baca1a9e1a5 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:35.194756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647724d6-ee79-4d21-adb6-8baca1a9e1a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:35.196793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647724d6-ee79-4d21-adb6-8baca1a9e1a5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:35.350605 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 995/2015] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:35 2022] POST /identity/v3/users => generated 304 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:35.359047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51424db-f1d1-4805-814b-b193686d99f9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:35.370171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51424db-f1d1-4805-814b-b193686d99f9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:35.375508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f629f1-8da3-4d18-b674-1418ed89da4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:35.376063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f629f1-8da3-4d18-b674-1418ed89da4f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:35.376345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f629f1-8da3-4d18-b674-1418ed89da4f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:35.376613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f629f1-8da3-4d18-b674-1418ed89da4f admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:35.377350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f629f1-8da3-4d18-b674-1418ed89da4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:35.379035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f629f1-8da3-4d18-b674-1418ed89da4f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:35.395886 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1021/2016] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:35.396522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647724d6-ee79-4d21-adb6-8baca1a9e1a5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:35.409178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647724d6-ee79-4d21-adb6-8baca1a9e1a5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:35.414567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04756f4-6251-449d-bf38-77008af3cc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:35.415452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04756f4-6251-449d-bf38-77008af3cc7b admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:35.415735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04756f4-6251-449d-bf38-77008af3cc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:35.416009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04756f4-6251-449d-bf38-77008af3cc7b admin admin] PATH_INFO: `/v3/projects/737350d45b9544239e0736f663d80269/users/e268920fabee4dee9b00d77e0eb222df/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:35.432450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04756f4-6251-449d-bf38-77008af3cc7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=737350d45b9544239e0736f663d80269, user_id=e268920fabee4dee9b00d77e0eb222df, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:35.435273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04756f4-6251-449d-bf38-77008af3cc7b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:35.477957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f629f1-8da3-4d18-b674-1418ed89da4f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:35.479955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-25f629f1-8da3-4d18-b674-1418ed89da4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:35.495119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f629f1-8da3-4d18-b674-1418ed89da4f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:35.500489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70048b7-291d-4ef0-bbfa-eed854437640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:35.501530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70048b7-291d-4ef0-bbfa-eed854437640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:35.501917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70048b7-291d-4ef0-bbfa-eed854437640 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:35.502197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70048b7-291d-4ef0-bbfa-eed854437640 admin admin] PATH_INFO: `/v3/projects/324ff1955efe413bbf9f57f7f63dd1d2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:35.503547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70048b7-291d-4ef0-bbfa-eed854437640 admin admin] RBAC: Authorizing `identity:delete_project(project_id=324ff1955efe413bbf9f57f7f63dd1d2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:35.505552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70048b7-291d-4ef0-bbfa-eed854437640 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:35.549185 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 996/2017] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:35 2022] PUT /identity/v3/projects/737350d45b9544239e0736f663d80269/users/e268920fabee4dee9b00d77e0eb222df/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:03:35.552963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04756f4-6251-449d-bf38-77008af3cc7b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:35.658484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:35.662075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c70048b7-291d-4ef0-bbfa-eed854437640 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '324ff1955efe413bbf9f57f7f63dd1d2', 'request_id': 'req-c70048b7-291d-4ef0-bbfa-eed854437640'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:35.667745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04756f4-6251-449d-bf38-77008af3cc7b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:35.710771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c70048b7-291d-4ef0-bbfa-eed854437640 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '324ff1955efe413bbf9f57f7f63dd1d2', 'request_id': 'req-c70048b7-291d-4ef0-bbfa-eed854437640'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:35.758613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d757ac-2c59-4bfe-a3c0-b0a182412d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:35.759313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d757ac-2c59-4bfe-a3c0-b0a182412d89 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:35.759666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d757ac-2c59-4bfe-a3c0-b0a182412d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:35.759950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d757ac-2c59-4bfe-a3c0-b0a182412d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:35.761398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d757ac-2c59-4bfe-a3c0-b0a182412d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:35.763095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d757ac-2c59-4bfe-a3c0-b0a182412d89 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:35.783137 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 997/2018] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:03:35.783543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d757ac-2c59-4bfe-a3c0-b0a182412d89 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:35.798418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d757ac-2c59-4bfe-a3c0-b0a182412d89 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:35.804026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823979fa-9942-4f85-9425-e14db6263a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:35.804928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823979fa-9942-4f85-9425-e14db6263a8d admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:35.805230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823979fa-9942-4f85-9425-e14db6263a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:35.805485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823979fa-9942-4f85-9425-e14db6263a8d admin admin] PATH_INFO: `/v3/projects/737350d45b9544239e0736f663d80269/users/e268920fabee4dee9b00d77e0eb222df/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:35.807583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823979fa-9942-4f85-9425-e14db6263a8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=737350d45b9544239e0736f663d80269, user_id=e268920fabee4dee9b00d77e0eb222df, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:35.810422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823979fa-9942-4f85-9425-e14db6263a8d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:35.823582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c70048b7-291d-4ef0-bbfa-eed854437640 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '324ff1955efe413bbf9f57f7f63dd1d2', 'request_id': 'req-c70048b7-291d-4ef0-bbfa-eed854437640'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:35.827248 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 998/2019] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:35 2022] PUT /identity/v3/projects/737350d45b9544239e0736f663d80269/users/e268920fabee4dee9b00d77e0eb222df/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:35.841442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c70048b7-291d-4ef0-bbfa-eed854437640 admin admin] The token cache is being invalidate because project 324ff1955efe413bbf9f57f7f63dd1d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:35.841442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c70048b7-291d-4ef0-bbfa-eed854437640 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:35.842123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c70048b7-291d-4ef0-bbfa-eed854437640 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:35.844477 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1022/2020] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:35 2022] DELETE /identity/v3/projects/324ff1955efe413bbf9f57f7f63dd1d2 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:03:35.849327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0c6fb-a12a-415a-b3c8-94b187710e87 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:35.849628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0c6fb-a12a-415a-b3c8-94b187710e87 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:35.849951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b0c6fb-a12a-415a-b3c8-94b187710e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:35.884556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a9b0c6fb-a12a-415a-b3c8-94b187710e87 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:35.886340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a9b0c6fb-a12a-415a-b3c8-94b187710e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:36.061528 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 999/2021] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:36.069731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9fdc5-1303-4d6c-a4ad-52278ae42cf8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:36.069963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9fdc5-1303-4d6c-a4ad-52278ae42cf8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:36.070254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9fdc5-1303-4d6c-a4ad-52278ae42cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:36.126909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-80b9fdc5-1303-4d6c-a4ad-52278ae42cf8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:36.129554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-80b9fdc5-1303-4d6c-a4ad-52278ae42cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:36.143716 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1023/2022] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:36.150076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0c6fb-a12a-415a-b3c8-94b187710e87 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:36.160225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b0c6fb-a12a-415a-b3c8-94b187710e87 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:36.165638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ef777-f23a-4e86-b977-9590264655cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:36.166438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ef777-f23a-4e86-b977-9590264655cb admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:36.166835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ef777-f23a-4e86-b977-9590264655cb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:36.173284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ef777-f23a-4e86-b977-9590264655cb admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:36.174221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ef777-f23a-4e86-b977-9590264655cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:03:36.174960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ef777-f23a-4e86-b977-9590264655cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:36.176721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ef777-f23a-4e86-b977-9590264655cb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:36.177122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9fdc5-1303-4d6c-a4ad-52278ae42cf8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:36.184018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-80b9fdc5-1303-4d6c-a4ad-52278ae42cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:36.210478 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1000/2023] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:36.215819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ef777-f23a-4e86-b977-9590264655cb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:36.235553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ef777-f23a-4e86-b977-9590264655cb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:36.241440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd22b2-2cc5-40e5-95d8-c64327b4defb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:36.242188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd22b2-2cc5-40e5-95d8-c64327b4defb admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:36.242520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd22b2-2cc5-40e5-95d8-c64327b4defb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:36.242744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfd22b2-2cc5-40e5-95d8-c64327b4defb admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:36.243444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd22b2-2cc5-40e5-95d8-c64327b4defb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:36.250876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfd22b2-2cc5-40e5-95d8-c64327b4defb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:36.356242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9fdc5-1303-4d6c-a4ad-52278ae42cf8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:36.363097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb2a64-c9a6-49ef-a00c-96a21818ac74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:36.364061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb2a64-c9a6-49ef-a00c-96a21818ac74 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:36.364400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb2a64-c9a6-49ef-a00c-96a21818ac74 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:36.364785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb2a64-c9a6-49ef-a00c-96a21818ac74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:36.367944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6deb2a64-c9a6-49ef-a00c-96a21818ac74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:36.398518 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1001/2024] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:36 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:36.405208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd22b2-2cc5-40e5-95d8-c64327b4defb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:36.529246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb2a64-c9a6-49ef-a00c-96a21818ac74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:36.531474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb2a64-c9a6-49ef-a00c-96a21818ac74 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:36.538707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfd22b2-2cc5-40e5-95d8-c64327b4defb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:36.545986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29f8de-1963-4ee7-bbbf-5beaa0ac50fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:36.546778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29f8de-1963-4ee7-bbbf-5beaa0ac50fd admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:36.547299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29f8de-1963-4ee7-bbbf-5beaa0ac50fd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:36.547673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29f8de-1963-4ee7-bbbf-5beaa0ac50fd admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:36.548479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29f8de-1963-4ee7-bbbf-5beaa0ac50fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:36.550547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29f8de-1963-4ee7-bbbf-5beaa0ac50fd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:36.564849 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1024/2025] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:36.606450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:36.669353 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1002/2026] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:36 2022] POST /identity/v3/users => generated 288 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:36.669731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb2a64-c9a6-49ef-a00c-96a21818ac74 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:36.686886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb2a64-c9a6-49ef-a00c-96a21818ac74 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:36.693912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5612eba-cb50-43f4-84a1-d105c157b593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:36.697180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5612eba-cb50-43f4-84a1-d105c157b593 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:36.697525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5612eba-cb50-43f4-84a1-d105c157b593 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:36.697952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5612eba-cb50-43f4-84a1-d105c157b593 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:36.698711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5612eba-cb50-43f4-84a1-d105c157b593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:36.700615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5612eba-cb50-43f4-84a1-d105c157b593 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:36.723561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29f8de-1963-4ee7-bbbf-5beaa0ac50fd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:36.724759 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1025/2027] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:03:36.737874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29f8de-1963-4ee7-bbbf-5beaa0ac50fd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:36.746171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212d256-9e6f-4dad-9a8f-1e2dd38859d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:36.747348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212d256-9e6f-4dad-9a8f-1e2dd38859d8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:36.747833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212d256-9e6f-4dad-9a8f-1e2dd38859d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:36.748427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212d256-9e6f-4dad-9a8f-1e2dd38859d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:36.749443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212d256-9e6f-4dad-9a8f-1e2dd38859d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:36.752672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212d256-9e6f-4dad-9a8f-1e2dd38859d8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:36.766139 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1003/2028] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:36.776666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5612eba-cb50-43f4-84a1-d105c157b593 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:36.792947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5612eba-cb50-43f4-84a1-d105c157b593 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:36.800479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a9d52-3967-4ed7-84d5-676a5999a6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:36.802102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a9d52-3967-4ed7-84d5-676a5999a6af admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:36.802672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a9d52-3967-4ed7-84d5-676a5999a6af admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:36.803269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a9d52-3967-4ed7-84d5-676a5999a6af admin admin] PATH_INFO: `/v3/projects/c206785173854bd4abd20176a4f2ef45/users/e0b68a3f2a8b4984be2e1b81d37fd29a/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:36.866949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:36.874747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a9d52-3967-4ed7-84d5-676a5999a6af admin admin] RBAC: Authorizing `identity:create_grant(project_id=c206785173854bd4abd20176a4f2ef45, user_id=e0b68a3f2a8b4984be2e1b81d37fd29a, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:36.880154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a9d52-3967-4ed7-84d5-676a5999a6af admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:36.962383 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1026/2029] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:36 2022] PUT /identity/v3/projects/c206785173854bd4abd20176a4f2ef45/users/e0b68a3f2a8b4984be2e1b81d37fd29a/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:36.972653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e57a9e-30db-46ef-9ffa-8b6dc5db1539 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:36.973204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e57a9e-30db-46ef-9ffa-8b6dc5db1539 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:36.973873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e57a9e-30db-46ef-9ffa-8b6dc5db1539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:37.017408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-47e57a9e-30db-46ef-9ffa-8b6dc5db1539 None None] MFA Rules not processed for user `e0b68a3f2a8b4984be2e1b81d37fd29a`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:37.017408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-47e57a9e-30db-46ef-9ffa-8b6dc5db1539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:37.118780 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1004/2030] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:37.120928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a9d52-3967-4ed7-84d5-676a5999a6af admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:37.230208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a9d52-3967-4ed7-84d5-676a5999a6af admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:37.237207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a01c8-da7e-41c3-9e91-fe9de743e02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:37.237939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a01c8-da7e-41c3-9e91-fe9de743e02b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:37.238352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a01c8-da7e-41c3-9e91-fe9de743e02b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:37.238752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a01c8-da7e-41c3-9e91-fe9de743e02b admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:37.239533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a01c8-da7e-41c3-9e91-fe9de743e02b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:37.242527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a01c8-da7e-41c3-9e91-fe9de743e02b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:37.386059 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1027/2031] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:37 2022] POST /identity/v3/projects => generated 373 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:37.386059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:37.387252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e57a9e-30db-46ef-9ffa-8b6dc5db1539 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:37.399356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e57a9e-30db-46ef-9ffa-8b6dc5db1539 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:37.460880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee50fa-cce6-4233-b111-a9fc07bc6927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:37.461138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee50fa-cce6-4233-b111-a9fc07bc6927 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:37.462399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee50fa-cce6-4233-b111-a9fc07bc6927 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:37.462399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee50fa-cce6-4233-b111-a9fc07bc6927 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:37.462399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee50fa-cce6-4233-b111-a9fc07bc6927 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:37.464522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee50fa-cce6-4233-b111-a9fc07bc6927 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:37.540464 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1005/2032] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:37 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:03:37.544600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a01c8-da7e-41c3-9e91-fe9de743e02b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:37.562533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a01c8-da7e-41c3-9e91-fe9de743e02b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:37.568048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a7147-c22c-4d65-8df8-c2d5fb9a8cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:37.575223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a7147-c22c-4d65-8df8-c2d5fb9a8cdb admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:37.575223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a7147-c22c-4d65-8df8-c2d5fb9a8cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:37.575223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34a7147-c22c-4d65-8df8-c2d5fb9a8cdb admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:37.575223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a7147-c22c-4d65-8df8-c2d5fb9a8cdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:37.575223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34a7147-c22c-4d65-8df8-c2d5fb9a8cdb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:37.584590 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1028/2033] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:37.589309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee50fa-cce6-4233-b111-a9fc07bc6927 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:37.606049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee50fa-cce6-4233-b111-a9fc07bc6927 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:37.611969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779b488-7653-42f6-9885-aed3fc97b479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:37.612675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779b488-7653-42f6-9885-aed3fc97b479 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:37.613036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779b488-7653-42f6-9885-aed3fc97b479 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:37.613389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779b488-7653-42f6-9885-aed3fc97b479 admin admin] PATH_INFO: `/v3/projects/9342c0109f9744258d2cd1a6faa0f992/users/7cbbe216d1974ebea03f24df9bcedce9/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:37.638677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779b488-7653-42f6-9885-aed3fc97b479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9342c0109f9744258d2cd1a6faa0f992, user_id=7cbbe216d1974ebea03f24df9bcedce9, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:37.642731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8779b488-7653-42f6-9885-aed3fc97b479 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:37.749657 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1006/2034] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:37 2022] PUT /identity/v3/projects/9342c0109f9744258d2cd1a6faa0f992/users/7cbbe216d1974ebea03f24df9bcedce9/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:37.753692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a7147-c22c-4d65-8df8-c2d5fb9a8cdb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:37.756188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:37.764639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34a7147-c22c-4d65-8df8-c2d5fb9a8cdb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:37.834246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e399cb-e714-4475-ba07-45e37b43efb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:37.835483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e399cb-e714-4475-ba07-45e37b43efb7 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:37.835774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e399cb-e714-4475-ba07-45e37b43efb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:37.836059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e399cb-e714-4475-ba07-45e37b43efb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:37.836807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e399cb-e714-4475-ba07-45e37b43efb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:37.838558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e399cb-e714-4475-ba07-45e37b43efb7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:37.848899 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1029/2035] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:37.853231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779b488-7653-42f6-9885-aed3fc97b479 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:37.864179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779b488-7653-42f6-9885-aed3fc97b479 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:37.869126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bac24-6c3b-4540-aa05-7c7e72690e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:37.876677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bac24-6c3b-4540-aa05-7c7e72690e6a admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:37.876677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bac24-6c3b-4540-aa05-7c7e72690e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:37.876677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629bac24-6c3b-4540-aa05-7c7e72690e6a admin admin] PATH_INFO: `/v3/projects/9342c0109f9744258d2cd1a6faa0f992/users/7cbbe216d1974ebea03f24df9bcedce9/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:37.876677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bac24-6c3b-4540-aa05-7c7e72690e6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9342c0109f9744258d2cd1a6faa0f992, user_id=7cbbe216d1974ebea03f24df9bcedce9, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:37.876677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629bac24-6c3b-4540-aa05-7c7e72690e6a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:37.887380 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1007/2036] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:37 2022] PUT /identity/v3/projects/9342c0109f9744258d2cd1a6faa0f992/users/7cbbe216d1974ebea03f24df9bcedce9/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:03:37.898787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e09582-d285-4a4d-a4f9-5ca1c1f687e3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:37.899078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e09582-d285-4a4d-a4f9-5ca1c1f687e3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:37.899358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e09582-d285-4a4d-a4f9-5ca1c1f687e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:37.927088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-71e09582-d285-4a4d-a4f9-5ca1c1f687e3 None None] MFA Rules not processed for user `7cbbe216d1974ebea03f24df9bcedce9`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:37.928322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-71e09582-d285-4a4d-a4f9-5ca1c1f687e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:38.004547 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1030/2037] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:38.015937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bac24-6c3b-4540-aa05-7c7e72690e6a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:38.017785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-629bac24-6c3b-4540-aa05-7c7e72690e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:38.130474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629bac24-6c3b-4540-aa05-7c7e72690e6a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:38.138130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a669de7-65ad-4a91-97a6-aad82394adaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:38.138894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a669de7-65ad-4a91-97a6-aad82394adaa service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:38.139400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a669de7-65ad-4a91-97a6-aad82394adaa service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:38.139796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a669de7-65ad-4a91-97a6-aad82394adaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:38.155252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a669de7-65ad-4a91-97a6-aad82394adaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:38.157552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a669de7-65ad-4a91-97a6-aad82394adaa service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:38.187039 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1008/2038] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:38 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:38.424831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd6239-e8df-4612-968b-855738f28dd2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:38.425433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd6239-e8df-4612-968b-855738f28dd2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:38.425619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd6239-e8df-4612-968b-855738f28dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:38.489556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4ccd6239-e8df-4612-968b-855738f28dd2 None None] MFA Rules not processed for user `e268920fabee4dee9b00d77e0eb222df`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:38.491023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4ccd6239-e8df-4612-968b-855738f28dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:38.589686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a669de7-65ad-4a91-97a6-aad82394adaa service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:38.622726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a669de7-65ad-4a91-97a6-aad82394adaa service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:38.635348 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1031/2039] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:38.643693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152df32-2f04-402c-b247-29ee5705b0c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:38.644182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152df32-2f04-402c-b247-29ee5705b0c8 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:38.644557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152df32-2f04-402c-b247-29ee5705b0c8 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:38.644844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4152df32-2f04-402c-b247-29ee5705b0c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:38.654086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdc62c-cd44-4ec7-a3e6-5d8624c1342b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:38.656100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdc62c-cd44-4ec7-a3e6-5d8624c1342b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:38.659289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cdc62c-cd44-4ec7-a3e6-5d8624c1342b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:38.707503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b1cdc62c-cd44-4ec7-a3e6-5d8624c1342b None None] MFA Rules not processed for user `fc6ee6dba5d24e90bff48384539fcd7f`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:38.710110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b1cdc62c-cd44-4ec7-a3e6-5d8624c1342b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:38.790511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152df32-2f04-402c-b247-29ee5705b0c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:38.792639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4152df32-2f04-402c-b247-29ee5705b0c8 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:38.810764 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1009/2040] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:38.824160 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1032/2041] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:38.917417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152df32-2f04-402c-b247-29ee5705b0c8 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:38.932241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4152df32-2f04-402c-b247-29ee5705b0c8 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:38.938151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5dfbf-004a-4cf0-8e77-0da60d63a00a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:38.938816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5dfbf-004a-4cf0-8e77-0da60d63a00a service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:38.939107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5dfbf-004a-4cf0-8e77-0da60d63a00a service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:38.939501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5dfbf-004a-4cf0-8e77-0da60d63a00a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:38.941884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-03e5dfbf-004a-4cf0-8e77-0da60d63a00a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:38.952747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5dfbf-004a-4cf0-8e77-0da60d63a00a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:38.954439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e5dfbf-004a-4cf0-8e77-0da60d63a00a service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:38.965662 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1010/2042] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:38.978839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:39.031602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:39.693652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cdc62c-cd44-4ec7-a3e6-5d8624c1342b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:39.762642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:39.789545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cdc62c-cd44-4ec7-a3e6-5d8624c1342b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:39.795109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa2d18-5202-4742-bb0b-2061c716d753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:39.795984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa2d18-5202-4742-bb0b-2061c716d753 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:39.796270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa2d18-5202-4742-bb0b-2061c716d753 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:39.796547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa2d18-5202-4742-bb0b-2061c716d753 admin admin] PATH_INFO: `/v3/users/e0b68a3f2a8b4984be2e1b81d37fd29a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:39.797757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa2d18-5202-4742-bb0b-2061c716d753 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0b68a3f2a8b4984be2e1b81d37fd29a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:39.815043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa2d18-5202-4742-bb0b-2061c716d753 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:39.988341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f6aa2d18-5202-4742-bb0b-2061c716d753 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0b68a3f2a8b4984be2e1b81d37fd29a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:40.008844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f6aa2d18-5202-4742-bb0b-2061c716d753 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0b68a3f2a8b4984be2e1b81d37fd29a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:40.009630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f6aa2d18-5202-4742-bb0b-2061c716d753 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0b68a3f2a8b4984be2e1b81d37fd29a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:40.010416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f6aa2d18-5202-4742-bb0b-2061c716d753 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0b68a3f2a8b4984be2e1b81d37fd29a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:40.091295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f6aa2d18-5202-4742-bb0b-2061c716d753 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0b68a3f2a8b4984be2e1b81d37fd29a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:40.129110 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1033/2043] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:39 2022] DELETE /identity/v3/users/e0b68a3f2a8b4984be2e1b81d37fd29a => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:40.130979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491974a-f3f1-464e-9c03-030f47d6e9f6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:40.131577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491974a-f3f1-464e-9c03-030f47d6e9f6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:40.131577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491974a-f3f1-464e-9c03-030f47d6e9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:40.150243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8491974a-f3f1-464e-9c03-030f47d6e9f6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:40.151645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8491974a-f3f1-464e-9c03-030f47d6e9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:40.269340 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1011/2044] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:40.282282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa2d18-5202-4742-bb0b-2061c716d753 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:40.283835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f6aa2d18-5202-4742-bb0b-2061c716d753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:40.384278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa2d18-5202-4742-bb0b-2061c716d753 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:40.389120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281249a-70b7-4090-8826-83d84e4de5ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:40.390094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281249a-70b7-4090-8826-83d84e4de5ba service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:40.390446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281249a-70b7-4090-8826-83d84e4de5ba service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:40.391104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281249a-70b7-4090-8826-83d84e4de5ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:40.407538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281249a-70b7-4090-8826-83d84e4de5ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:40.409082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281249a-70b7-4090-8826-83d84e4de5ba service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:40.433873 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1034/2045] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:41.734464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598fecb-e0e9-4091-a146-71979e5a4b56 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:41.734941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598fecb-e0e9-4091-a146-71979e5a4b56 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:41.735118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598fecb-e0e9-4091-a146-71979e5a4b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:41.754779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5598fecb-e0e9-4091-a146-71979e5a4b56 None None] MFA Rules not processed for user `ad6f51b7b6834fefa06c58829399de85`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:41.755996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5598fecb-e0e9-4091-a146-71979e5a4b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:41.771570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281249a-70b7-4090-8826-83d84e4de5ba service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:41.774267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0281249a-70b7-4090-8826-83d84e4de5ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:41.798461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281249a-70b7-4090-8826-83d84e4de5ba service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:41.813422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:41.815479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:41.816379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:41.816676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:41.816966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1 admin admin] PATH_INFO: `/v3/projects/c206785173854bd4abd20176a4f2ef45` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:41.824828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c206785173854bd4abd20176a4f2ef45)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:41.827910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:41.874261 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1012/2046] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:41.960289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c206785173854bd4abd20176a4f2ef45', 'request_id': 'req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:41.965078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:41.967895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c206785173854bd4abd20176a4f2ef45', 'request_id': 'req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:42.022653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c206785173854bd4abd20176a4f2ef45', 'request_id': 'req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:42.038996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1 admin admin] The token cache is being invalidate because project c206785173854bd4abd20176a4f2ef45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:42.039370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:42.040100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:42.041958 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1035/2047] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:41 2022] DELETE /identity/v3/projects/c206785173854bd4abd20176a4f2ef45 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:42.045951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598fecb-e0e9-4091-a146-71979e5a4b56 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:42.064307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598fecb-e0e9-4091-a146-71979e5a4b56 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:42.122448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c551a-023f-4579-945b-1c070b69fe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:42.123158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c551a-023f-4579-945b-1c070b69fe2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:42.123862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c551a-023f-4579-945b-1c070b69fe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:42.124338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c551a-023f-4579-945b-1c070b69fe2d admin admin] PATH_INFO: `/v3/users/7cbbe216d1974ebea03f24df9bcedce9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:42.126491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c551a-023f-4579-945b-1c070b69fe2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cbbe216d1974ebea03f24df9bcedce9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:42.129138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c551a-023f-4579-945b-1c070b69fe2d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:42.455012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-638c551a-023f-4579-945b-1c070b69fe2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cbbe216d1974ebea03f24df9bcedce9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:42.456096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-638c551a-023f-4579-945b-1c070b69fe2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cbbe216d1974ebea03f24df9bcedce9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:42.467120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-638c551a-023f-4579-945b-1c070b69fe2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cbbe216d1974ebea03f24df9bcedce9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:42.555846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-638c551a-023f-4579-945b-1c070b69fe2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cbbe216d1974ebea03f24df9bcedce9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:42.561964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-638c551a-023f-4579-945b-1c070b69fe2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cbbe216d1974ebea03f24df9bcedce9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:42.591871 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1013/2048] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:42 2022] DELETE /identity/v3/users/7cbbe216d1974ebea03f24df9bcedce9 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:43.649854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:43.653167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:43.673206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec859aa-4501-4363-8a4a-e3f39b4a49f1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:43.888042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bf7902-bfe4-4690-b37e-6c6c7b12e4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:43.894674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf7902-bfe4-4690-b37e-6c6c7b12e4b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:43.895520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf7902-bfe4-4690-b37e-6c6c7b12e4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:43.895520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf7902-bfe4-4690-b37e-6c6c7b12e4b8 admin admin] PATH_INFO: `/v3/projects/9342c0109f9744258d2cd1a6faa0f992` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:43.896997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf7902-bfe4-4690-b37e-6c6c7b12e4b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9342c0109f9744258d2cd1a6faa0f992)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:43.906066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bf7902-bfe4-4690-b37e-6c6c7b12e4b8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:44.046902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057905df-4d33-4cde-adcb-3636ce52dac8 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:44.047421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057905df-4d33-4cde-adcb-3636ce52dac8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:44.047531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057905df-4d33-4cde-adcb-3636ce52dac8 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:44.049345 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1014/2049] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:03:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:44.065215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45d067-18cd-4e7c-baef-ae51bc791fc4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:44.065706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45d067-18cd-4e7c-baef-ae51bc791fc4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:44.066687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45d067-18cd-4e7c-baef-ae51bc791fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:44.101256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c0bf7902-bfe4-4690-b37e-6c6c7b12e4b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9342c0109f9744258d2cd1a6faa0f992', 'request_id': 'req-c0bf7902-bfe4-4690-b37e-6c6c7b12e4b8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:44.108998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0c45d067-18cd-4e7c-baef-ae51bc791fc4 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:44.114783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0c45d067-18cd-4e7c-baef-ae51bc791fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:44.116382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c0bf7902-bfe4-4690-b37e-6c6c7b12e4b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9342c0109f9744258d2cd1a6faa0f992', 'request_id': 'req-c0bf7902-bfe4-4690-b37e-6c6c7b12e4b8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:44.259480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:44.275202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c0bf7902-bfe4-4690-b37e-6c6c7b12e4b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9342c0109f9744258d2cd1a6faa0f992', 'request_id': 'req-c0bf7902-bfe4-4690-b37e-6c6c7b12e4b8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:44.294032 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1015/2050] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:03:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:03:44.301953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c0bf7902-bfe4-4690-b37e-6c6c7b12e4b8 admin admin] The token cache is being invalidate because project 9342c0109f9744258d2cd1a6faa0f992 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:44.303194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c0bf7902-bfe4-4690-b37e-6c6c7b12e4b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:44.305062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c0bf7902-bfe4-4690-b37e-6c6c7b12e4b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:44.311463 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1036/2051] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:43 2022] DELETE /identity/v3/projects/9342c0109f9744258d2cd1a6faa0f992 => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:44.312347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764238c6-ade6-4acc-8e9c-524ba140fd6b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:44.312645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764238c6-ade6-4acc-8e9c-524ba140fd6b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:44.312969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764238c6-ade6-4acc-8e9c-524ba140fd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:44.338636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-764238c6-ade6-4acc-8e9c-524ba140fd6b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:44.342226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-764238c6-ade6-4acc-8e9c-524ba140fd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:44.429914 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1016/2052] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:44.436344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e9049-4711-483b-bed4-4b6542bd679a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:44.436897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e9049-4711-483b-bed4-4b6542bd679a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:44.436897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e9049-4711-483b-bed4-4b6542bd679a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:44.456084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-dd0e9049-4711-483b-bed4-4b6542bd679a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:44.456854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dd0e9049-4711-483b-bed4-4b6542bd679a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:44.465862 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1037/2053] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:44.470075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764238c6-ade6-4acc-8e9c-524ba140fd6b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:44.482103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764238c6-ade6-4acc-8e9c-524ba140fd6b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:44.488092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25698e5b-74ff-47f0-b4a1-d21fee28990f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:44.489295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25698e5b-74ff-47f0-b4a1-d21fee28990f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:44.489411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25698e5b-74ff-47f0-b4a1-d21fee28990f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:44.491650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25698e5b-74ff-47f0-b4a1-d21fee28990f admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:44.491650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25698e5b-74ff-47f0-b4a1-d21fee28990f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:03:44.491650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25698e5b-74ff-47f0-b4a1-d21fee28990f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:44.493559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25698e5b-74ff-47f0-b4a1-d21fee28990f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:44.507188 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1017/2054] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:44.511136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e9049-4711-483b-bed4-4b6542bd679a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:44.521869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e9049-4711-483b-bed4-4b6542bd679a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:44.528622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00d9e1-b7c9-4619-8903-815def5ab5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:44.529779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00d9e1-b7c9-4619-8903-815def5ab5c7 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:44.530230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00d9e1-b7c9-4619-8903-815def5ab5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:44.540662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00d9e1-b7c9-4619-8903-815def5ab5c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:44.541570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00d9e1-b7c9-4619-8903-815def5ab5c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:44.544264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00d9e1-b7c9-4619-8903-815def5ab5c7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:44.567030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25698e5b-74ff-47f0-b4a1-d21fee28990f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:44.572245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-25698e5b-74ff-47f0-b4a1-d21fee28990f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:44.593542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25698e5b-74ff-47f0-b4a1-d21fee28990f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:44.600621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd07b74-8cf4-4464-9c6b-865211cae145 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:44.601565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd07b74-8cf4-4464-9c6b-865211cae145 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:44.602030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd07b74-8cf4-4464-9c6b-865211cae145 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:44.602454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd07b74-8cf4-4464-9c6b-865211cae145 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:44.604545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2cd07b74-8cf4-4464-9c6b-865211cae145 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:44.620061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd07b74-8cf4-4464-9c6b-865211cae145 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:44.622613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd07b74-8cf4-4464-9c6b-865211cae145 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:44.633756 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1038/2055] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:44 2022] POST /identity/v3/projects => generated 361 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:44.638070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00d9e1-b7c9-4619-8903-815def5ab5c7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:44.649559 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1018/2056] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:44.749390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:44.778866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00d9e1-b7c9-4619-8903-815def5ab5c7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:44.783968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707125b-4fc1-4350-aad1-f12bb003cd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:44.784853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707125b-4fc1-4350-aad1-f12bb003cd27 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:44.785229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707125b-4fc1-4350-aad1-f12bb003cd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:44.785632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707125b-4fc1-4350-aad1-f12bb003cd27 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:44.786468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707125b-4fc1-4350-aad1-f12bb003cd27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:44.789565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707125b-4fc1-4350-aad1-f12bb003cd27 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:44.858372 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1039/2057] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:44 2022] POST /identity/v3/users => generated 281 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:03:44.861642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd07b74-8cf4-4464-9c6b-865211cae145 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:44.874754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd07b74-8cf4-4464-9c6b-865211cae145 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:44.880895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddce980-0fed-404b-8435-a3d234542c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:44.881557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddce980-0fed-404b-8435-a3d234542c5b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:44.881890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddce980-0fed-404b-8435-a3d234542c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:44.882163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddce980-0fed-404b-8435-a3d234542c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:44.882893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddce980-0fed-404b-8435-a3d234542c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:44.884738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddce980-0fed-404b-8435-a3d234542c5b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:44.902616 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1019/2058] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:44.904775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707125b-4fc1-4350-aad1-f12bb003cd27 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:44.925393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707125b-4fc1-4350-aad1-f12bb003cd27 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:44.931243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d684f16-2cd0-4d9b-8f90-a6d9a9e892fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:44.932614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d684f16-2cd0-4d9b-8f90-a6d9a9e892fe admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:44.932999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d684f16-2cd0-4d9b-8f90-a6d9a9e892fe admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:44.933362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d684f16-2cd0-4d9b-8f90-a6d9a9e892fe admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:44.934494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d684f16-2cd0-4d9b-8f90-a6d9a9e892fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:44.936675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d684f16-2cd0-4d9b-8f90-a6d9a9e892fe admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:44.958946 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1040/2059] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:44.961174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddce980-0fed-404b-8435-a3d234542c5b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:44.972715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddce980-0fed-404b-8435-a3d234542c5b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:44.981397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfca924-5b30-4904-9785-c76ccea59fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:44.982385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfca924-5b30-4904-9785-c76ccea59fa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:44.982667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfca924-5b30-4904-9785-c76ccea59fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:44.983196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfca924-5b30-4904-9785-c76ccea59fa2 admin admin] PATH_INFO: `/v3/projects/5b7ae0ab157146cfafd9af48642be9a4/users/24d29dd8cf204032ab45c3bc52d09867/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:45.001111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfca924-5b30-4904-9785-c76ccea59fa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7ae0ab157146cfafd9af48642be9a4, user_id=24d29dd8cf204032ab45c3bc52d09867, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:45.004055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfca924-5b30-4904-9785-c76ccea59fa2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:45.101223 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1020/2060] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:44 2022] PUT /identity/v3/projects/5b7ae0ab157146cfafd9af48642be9a4/users/24d29dd8cf204032ab45c3bc52d09867/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:45.111771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f8afe-b7cd-4e3d-985e-a7db7b16e521 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:45.112062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f8afe-b7cd-4e3d-985e-a7db7b16e521 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:45.112340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f8afe-b7cd-4e3d-985e-a7db7b16e521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:45.154494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c15f8afe-b7cd-4e3d-985e-a7db7b16e521 None None] MFA Rules not processed for user `24d29dd8cf204032ab45c3bc52d09867`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:45.155748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c15f8afe-b7cd-4e3d-985e-a7db7b16e521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:45.297689 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1041/2061] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:45.302201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfca924-5b30-4904-9785-c76ccea59fa2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:45.403534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfca924-5b30-4904-9785-c76ccea59fa2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:45.425805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12676eed-9349-4e06-afa4-697a0d5d890c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:45.427093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676eed-9349-4e06-afa4-697a0d5d890c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:45.427403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676eed-9349-4e06-afa4-697a0d5d890c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:45.427772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12676eed-9349-4e06-afa4-697a0d5d890c admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:45.429284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12676eed-9349-4e06-afa4-697a0d5d890c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:45.432430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12676eed-9349-4e06-afa4-697a0d5d890c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:45.561025 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1021/2062] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:45 2022] POST /identity/v3/projects => generated 361 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:45.565107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f8afe-b7cd-4e3d-985e-a7db7b16e521 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:45.589556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f8afe-b7cd-4e3d-985e-a7db7b16e521 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:45.673578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953cda6-ca38-474a-ae95-56cfcbf51ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:45.674845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953cda6-ca38-474a-ae95-56cfcbf51ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:45.675713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953cda6-ca38-474a-ae95-56cfcbf51ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:45.676820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953cda6-ca38-474a-ae95-56cfcbf51ae1 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:45.678127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b953cda6-ca38-474a-ae95-56cfcbf51ae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:45.681299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b953cda6-ca38-474a-ae95-56cfcbf51ae1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:45.779848 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1042/2063] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:45 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:45.784665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12676eed-9349-4e06-afa4-697a0d5d890c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:45.796699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12676eed-9349-4e06-afa4-697a0d5d890c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:45.801631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b592d-4c0c-43d5-ba6d-5e3ff8fd5674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:45.802486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b592d-4c0c-43d5-ba6d-5e3ff8fd5674 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:45.802743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b592d-4c0c-43d5-ba6d-5e3ff8fd5674 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:45.803076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b592d-4c0c-43d5-ba6d-5e3ff8fd5674 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:45.804476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b592d-4c0c-43d5-ba6d-5e3ff8fd5674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:45.806537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b592d-4c0c-43d5-ba6d-5e3ff8fd5674 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:45.810003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:45.822307 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1022/2064] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:45.826621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953cda6-ca38-474a-ae95-56cfcbf51ae1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:45.838668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953cda6-ca38-474a-ae95-56cfcbf51ae1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:45.844079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7344d-8df3-49f0-bb88-b47bb6cbf72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:45.845001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7344d-8df3-49f0-bb88-b47bb6cbf72c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:45.845290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7344d-8df3-49f0-bb88-b47bb6cbf72c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:45.845569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7344d-8df3-49f0-bb88-b47bb6cbf72c admin admin] PATH_INFO: `/v3/projects/fa1484ab5c6d4af486b2c6b21cf488b1/users/bd03fecd26f34fb8b87102c1753423d7/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:45.874195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7344d-8df3-49f0-bb88-b47bb6cbf72c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa1484ab5c6d4af486b2c6b21cf488b1, user_id=bd03fecd26f34fb8b87102c1753423d7, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:45.877667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7344d-8df3-49f0-bb88-b47bb6cbf72c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:45.987826 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1043/2065] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:45 2022] PUT /identity/v3/projects/fa1484ab5c6d4af486b2c6b21cf488b1/users/bd03fecd26f34fb8b87102c1753423d7/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:03:45.989325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b592d-4c0c-43d5-ba6d-5e3ff8fd5674 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:46.001887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b592d-4c0c-43d5-ba6d-5e3ff8fd5674 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:46.060491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1be940-113a-4aee-9a75-b6279312c95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:46.061144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1be940-113a-4aee-9a75-b6279312c95a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:46.061437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1be940-113a-4aee-9a75-b6279312c95a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:46.061707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1be940-113a-4aee-9a75-b6279312c95a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:46.062514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1be940-113a-4aee-9a75-b6279312c95a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:46.064120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1be940-113a-4aee-9a75-b6279312c95a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:46.083391 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1023/2066] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:46.084936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7344d-8df3-49f0-bb88-b47bb6cbf72c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:46.097265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7344d-8df3-49f0-bb88-b47bb6cbf72c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:46.103248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0904cd3-91d4-44f1-b283-a1391f083900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:46.104340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0904cd3-91d4-44f1-b283-a1391f083900 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:46.104633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0904cd3-91d4-44f1-b283-a1391f083900 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:46.104989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0904cd3-91d4-44f1-b283-a1391f083900 admin admin] PATH_INFO: `/v3/projects/fa1484ab5c6d4af486b2c6b21cf488b1/users/bd03fecd26f34fb8b87102c1753423d7/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:46.107460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0904cd3-91d4-44f1-b283-a1391f083900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa1484ab5c6d4af486b2c6b21cf488b1, user_id=bd03fecd26f34fb8b87102c1753423d7, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:46.110556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0904cd3-91d4-44f1-b283-a1391f083900 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:46.135776 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1044/2067] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:46 2022] PUT /identity/v3/projects/fa1484ab5c6d4af486b2c6b21cf488b1/users/bd03fecd26f34fb8b87102c1753423d7/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:46.138602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7d081-d397-4e93-85ef-b6c0372d5110 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:46.139049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7d081-d397-4e93-85ef-b6c0372d5110 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:46.139199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7d081-d397-4e93-85ef-b6c0372d5110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:46.221632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f7a7d081-d397-4e93-85ef-b6c0372d5110 None None] MFA Rules not processed for user `bd03fecd26f34fb8b87102c1753423d7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:46.224789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f7a7d081-d397-4e93-85ef-b6c0372d5110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:46.325693 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1024/2068] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:46 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:46.431033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0904cd3-91d4-44f1-b283-a1391f083900 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:46.438925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:46.574069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0904cd3-91d4-44f1-b283-a1391f083900 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:46.590135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50c81e-a92b-4528-a13f-c70a5829ddf8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:46.591180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50c81e-a92b-4528-a13f-c70a5829ddf8 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:46.591670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50c81e-a92b-4528-a13f-c70a5829ddf8 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:46.592146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50c81e-a92b-4528-a13f-c70a5829ddf8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:46.615618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50c81e-a92b-4528-a13f-c70a5829ddf8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:46.618101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a50c81e-a92b-4528-a13f-c70a5829ddf8 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:46.637685 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1045/2069] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:46 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:47.981462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7d081-d397-4e93-85ef-b6c0372d5110 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:48.074704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7d081-d397-4e93-85ef-b6c0372d5110 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:48.080680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd8774-4e95-4380-8525-36a4b17941af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:48.081610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd8774-4e95-4380-8525-36a4b17941af admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:48.081962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd8774-4e95-4380-8525-36a4b17941af admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:48.082241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd8774-4e95-4380-8525-36a4b17941af admin admin] PATH_INFO: `/v3/users/24d29dd8cf204032ab45c3bc52d09867` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:48.087599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd8774-4e95-4380-8525-36a4b17941af admin admin] RBAC: Authorizing `identity:delete_user(user_id=24d29dd8cf204032ab45c3bc52d09867)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:48.087599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd8774-4e95-4380-8525-36a4b17941af admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:48.119067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:48.408527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-57cd8774-4e95-4380-8525-36a4b17941af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24d29dd8cf204032ab45c3bc52d09867'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:48.412246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-57cd8774-4e95-4380-8525-36a4b17941af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24d29dd8cf204032ab45c3bc52d09867'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:48.424930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-57cd8774-4e95-4380-8525-36a4b17941af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24d29dd8cf204032ab45c3bc52d09867'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:48.514004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-57cd8774-4e95-4380-8525-36a4b17941af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24d29dd8cf204032ab45c3bc52d09867'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:48.516823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-57cd8774-4e95-4380-8525-36a4b17941af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24d29dd8cf204032ab45c3bc52d09867'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:48.548944 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1025/2070] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:47 2022] DELETE /identity/v3/users/24d29dd8cf204032ab45c3bc52d09867 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:48.554440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7674f5-852a-4f7d-b5db-e7e671ae3b24 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:48.558622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7674f5-852a-4f7d-b5db-e7e671ae3b24 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:48.559069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7674f5-852a-4f7d-b5db-e7e671ae3b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:48.607780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6a7674f5-852a-4f7d-b5db-e7e671ae3b24 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:48.609174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6a7674f5-852a-4f7d-b5db-e7e671ae3b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:48.721455 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1046/2071] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:48.749573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd8774-4e95-4380-8525-36a4b17941af admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:48.751276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-57cd8774-4e95-4380-8525-36a4b17941af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:48.846491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd8774-4e95-4380-8525-36a4b17941af admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:48.851687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1044b6-79e7-4f66-a43d-7288247e7d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:48.852402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1044b6-79e7-4f66-a43d-7288247e7d50 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:48.852688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1044b6-79e7-4f66-a43d-7288247e7d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:48.853060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1044b6-79e7-4f66-a43d-7288247e7d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:48.863191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1044b6-79e7-4f66-a43d-7288247e7d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:48.864485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1044b6-79e7-4f66-a43d-7288247e7d50 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:48.875339 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1026/2072] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:49.915249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7674f5-852a-4f7d-b5db-e7e671ae3b24 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:49.920361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6a7674f5-852a-4f7d-b5db-e7e671ae3b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:49.944779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7674f5-852a-4f7d-b5db-e7e671ae3b24 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:49.951202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:49.953325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:49.954118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:49.954654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596 admin admin] PATH_INFO: `/v3/projects/5b7ae0ab157146cfafd9af48642be9a4` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:49.956980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b7ae0ab157146cfafd9af48642be9a4)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:49.964009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:50.146580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b7ae0ab157146cfafd9af48642be9a4', 'request_id': 'req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:50.157393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b7ae0ab157146cfafd9af48642be9a4', 'request_id': 'req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:50.188600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1044b6-79e7-4f66-a43d-7288247e7d50 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:50.190167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ab1044b6-79e7-4f66-a43d-7288247e7d50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:50.290713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1044b6-79e7-4f66-a43d-7288247e7d50 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:50.296247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b7ae0ab157146cfafd9af48642be9a4', 'request_id': 'req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:50.310832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596 admin admin] The token cache is being invalidate because project 5b7ae0ab157146cfafd9af48642be9a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:50.313732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:50.313732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:50.317418 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1047/2073] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:49 2022] DELETE /identity/v3/projects/5b7ae0ab157146cfafd9af48642be9a4 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:50.318607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:50.320787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:50.348112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:50.377017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:50.422206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024518a-a612-43a0-9932-4ebadfdc9d28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:50.426829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024518a-a612-43a0-9932-4ebadfdc9d28 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:50.426829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024518a-a612-43a0-9932-4ebadfdc9d28 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:50.426829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024518a-a612-43a0-9932-4ebadfdc9d28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:50.430353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7024518a-a612-43a0-9932-4ebadfdc9d28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:50.452636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f29df6-f2cb-4d8d-a6b6-8f95598ae596 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:50.458481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de9b27-685b-4ac2-97d4-233862510b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:50.460013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de9b27-685b-4ac2-97d4-233862510b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:50.460415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de9b27-685b-4ac2-97d4-233862510b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:50.460808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de9b27-685b-4ac2-97d4-233862510b74 admin admin] PATH_INFO: `/v3/users/bd03fecd26f34fb8b87102c1753423d7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:50.462378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de9b27-685b-4ac2-97d4-233862510b74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd03fecd26f34fb8b87102c1753423d7)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:50.464398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de9b27-685b-4ac2-97d4-233862510b74 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:50.534739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024518a-a612-43a0-9932-4ebadfdc9d28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:50.536401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7024518a-a612-43a0-9932-4ebadfdc9d28 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:50.548354 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1027/2074] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:03:50.679086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-76de9b27-685b-4ac2-97d4-233862510b74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd03fecd26f34fb8b87102c1753423d7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:50.691220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-76de9b27-685b-4ac2-97d4-233862510b74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd03fecd26f34fb8b87102c1753423d7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:50.692303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-76de9b27-685b-4ac2-97d4-233862510b74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd03fecd26f34fb8b87102c1753423d7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:50.694815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-76de9b27-685b-4ac2-97d4-233862510b74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd03fecd26f34fb8b87102c1753423d7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:50.770870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-76de9b27-685b-4ac2-97d4-233862510b74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd03fecd26f34fb8b87102c1753423d7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:50.797331 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1048/2075] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:50 2022] DELETE /identity/v3/users/bd03fecd26f34fb8b87102c1753423d7 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:51.783473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810aacb6-53e0-49db-8955-69384d982975 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:51.785893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810aacb6-53e0-49db-8955-69384d982975 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:51.786344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810aacb6-53e0-49db-8955-69384d982975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:51.812956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-810aacb6-53e0-49db-8955-69384d982975 None None] MFA Rules not processed for user `1b2a9ebcd3f84b758322a80c6e590de0`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:51.812956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-810aacb6-53e0-49db-8955-69384d982975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:51.835286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de9b27-685b-4ac2-97d4-233862510b74 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:51.836734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-76de9b27-685b-4ac2-97d4-233862510b74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:51.905204 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1028/2076] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:03:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:51.941765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810aacb6-53e0-49db-8955-69384d982975 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:51.946803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-810aacb6-53e0-49db-8955-69384d982975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:51.966502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810aacb6-53e0-49db-8955-69384d982975 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:51.966930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de9b27-685b-4ac2-97d4-233862510b74 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:51.975066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa300f4-b3f2-4ca0-940e-bf0180699ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:51.976984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa300f4-b3f2-4ca0-940e-bf0180699ebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:51.977747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa300f4-b3f2-4ca0-940e-bf0180699ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:51.978450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa300f4-b3f2-4ca0-940e-bf0180699ebf admin admin] PATH_INFO: `/v3/projects/fa1484ab5c6d4af486b2c6b21cf488b1` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:51.980688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa300f4-b3f2-4ca0-940e-bf0180699ebf admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa1484ab5c6d4af486b2c6b21cf488b1)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:51.984685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa300f4-b3f2-4ca0-940e-bf0180699ebf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:51.990838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d757156-ba5d-44b7-87b8-42fdceddd33b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:51.992972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d757156-ba5d-44b7-87b8-42fdceddd33b service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:51.992972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d757156-ba5d-44b7-87b8-42fdceddd33b service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:51.992972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d757156-ba5d-44b7-87b8-42fdceddd33b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:52.016695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d757156-ba5d-44b7-87b8-42fdceddd33b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:52.018627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d757156-ba5d-44b7-87b8-42fdceddd33b service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:52.036889 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1029/2077] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:52.082163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e1d18-8f63-4303-8d5c-e7ff29d2f8c8 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:52.082449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e1d18-8f63-4303-8d5c-e7ff29d2f8c8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:52.082731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e1d18-8f63-4303-8d5c-e7ff29d2f8c8 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:52.085146 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1030/2078] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:52.090470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775d22e-c1f2-4b89-b35a-b366e4d69552 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:52.090759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775d22e-c1f2-4b89-b35a-b366e4d69552 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:52.091050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1775d22e-c1f2-4b89-b35a-b366e4d69552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:52.111613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1775d22e-c1f2-4b89-b35a-b366e4d69552 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:52.112825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1775d22e-c1f2-4b89-b35a-b366e4d69552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:52.128473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:52.138855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9aa300f4-b3f2-4ca0-940e-bf0180699ebf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa1484ab5c6d4af486b2c6b21cf488b1', 'request_id': 'req-9aa300f4-b3f2-4ca0-940e-bf0180699ebf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:52.145588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9aa300f4-b3f2-4ca0-940e-bf0180699ebf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa1484ab5c6d4af486b2c6b21cf488b1', 'request_id': 'req-9aa300f4-b3f2-4ca0-940e-bf0180699ebf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:52.189657 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1031/2079] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:03:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:03:52.215975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775d22e-c1f2-4b89-b35a-b366e4d69552 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:52.217406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1775d22e-c1f2-4b89-b35a-b366e4d69552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:52.231385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9aa300f4-b3f2-4ca0-940e-bf0180699ebf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa1484ab5c6d4af486b2c6b21cf488b1', 'request_id': 'req-9aa300f4-b3f2-4ca0-940e-bf0180699ebf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:52.243751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:52.252691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9aa300f4-b3f2-4ca0-940e-bf0180699ebf admin admin] The token cache is being invalidate because project fa1484ab5c6d4af486b2c6b21cf488b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:52.252880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9aa300f4-b3f2-4ca0-940e-bf0180699ebf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:52.253057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1775d22e-c1f2-4b89-b35a-b366e4d69552 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:52.254981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9aa300f4-b3f2-4ca0-940e-bf0180699ebf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:52.265975 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1049/2080] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:51 2022] DELETE /identity/v3/projects/fa1484ab5c6d4af486b2c6b21cf488b1 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:52.272138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278507bf-b1c5-4544-95c5-4b27a9884a01 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:52.272654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278507bf-b1c5-4544-95c5-4b27a9884a01 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:52.273588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278507bf-b1c5-4544-95c5-4b27a9884a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:52.315818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-278507bf-b1c5-4544-95c5-4b27a9884a01 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:52.319531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-278507bf-b1c5-4544-95c5-4b27a9884a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:52.354553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a840338-1c82-4f3c-8b27-3660a2e639f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:52.356620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a840338-1c82-4f3c-8b27-3660a2e639f9 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:52.356620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a840338-1c82-4f3c-8b27-3660a2e639f9 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:52.356620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a840338-1c82-4f3c-8b27-3660a2e639f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:52.358394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9a840338-1c82-4f3c-8b27-3660a2e639f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:52.371255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a840338-1c82-4f3c-8b27-3660a2e639f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:52.372999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a840338-1c82-4f3c-8b27-3660a2e639f9 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:52.386150 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1032/2081] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:03:52.391022 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1050/2082] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:03:52.400677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511a31d-a613-4e0d-9ef0-138ec8c77c04 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:52.401257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511a31d-a613-4e0d-9ef0-138ec8c77c04 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:52.401699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a511a31d-a613-4e0d-9ef0-138ec8c77c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:52.421616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a511a31d-a613-4e0d-9ef0-138ec8c77c04 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:52.423259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a511a31d-a613-4e0d-9ef0-138ec8c77c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:52.435623 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1033/2083] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:03:52.439584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278507bf-b1c5-4544-95c5-4b27a9884a01 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:52.452479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278507bf-b1c5-4544-95c5-4b27a9884a01 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:52.471070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b818ed7-d282-4111-8aa7-c5fd692b2e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:52.471933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b818ed7-d282-4111-8aa7-c5fd692b2e01 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:52.472359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b818ed7-d282-4111-8aa7-c5fd692b2e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:52.472796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b818ed7-d282-4111-8aa7-c5fd692b2e01 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:52.473827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b818ed7-d282-4111-8aa7-c5fd692b2e01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:03:52.474301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b818ed7-d282-4111-8aa7-c5fd692b2e01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:52.479237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b818ed7-d282-4111-8aa7-c5fd692b2e01 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:52.502352 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1051/2084] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:03:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:03:52.507290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511a31d-a613-4e0d-9ef0-138ec8c77c04 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:52.522657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a511a31d-a613-4e0d-9ef0-138ec8c77c04 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:52.534538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ce384-d8d1-4350-889c-e7b04158fc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:52.536910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ce384-d8d1-4350-889c-e7b04158fc58 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:52.537200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ce384-d8d1-4350-889c-e7b04158fc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:52.537509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ce384-d8d1-4350-889c-e7b04158fc58 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:52.538270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ce384-d8d1-4350-889c-e7b04158fc58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:52.540217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ce384-d8d1-4350-889c-e7b04158fc58 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:52.643336 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1034/2085] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:52 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:03:52.648034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b818ed7-d282-4111-8aa7-c5fd692b2e01 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:52.665597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b818ed7-d282-4111-8aa7-c5fd692b2e01 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:52.761886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acda25f-dba0-4322-80bc-d095a9249783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:52.763122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acda25f-dba0-4322-80bc-d095a9249783 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:52.764191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acda25f-dba0-4322-80bc-d095a9249783 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:52.768396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acda25f-dba0-4322-80bc-d095a9249783 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:52.768396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acda25f-dba0-4322-80bc-d095a9249783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:52.769809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acda25f-dba0-4322-80bc-d095a9249783 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:52.900884 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1052/2086] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:52 2022] POST /identity/v3/users => generated 292 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:03:52.904802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ce384-d8d1-4350-889c-e7b04158fc58 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:52.920545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ce384-d8d1-4350-889c-e7b04158fc58 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:52.925644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb29f2-575e-47d8-aa71-0eadb4bcaddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:52.928906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb29f2-575e-47d8-aa71-0eadb4bcaddc admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:52.928906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb29f2-575e-47d8-aa71-0eadb4bcaddc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:52.928906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb29f2-575e-47d8-aa71-0eadb4bcaddc admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:52.928906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb29f2-575e-47d8-aa71-0eadb4bcaddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:52.929805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb29f2-575e-47d8-aa71-0eadb4bcaddc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:52.952340 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1035/2087] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:52.955660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acda25f-dba0-4322-80bc-d095a9249783 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:52.972717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acda25f-dba0-4322-80bc-d095a9249783 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:52.980865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4c207-a3df-4cec-98ef-79e4a61c8f6e None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:52.981265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4c207-a3df-4cec-98ef-79e4a61c8f6e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:52.981584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4c207-a3df-4cec-98ef-79e4a61c8f6e None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:52.983700 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1036/2088] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:03:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:52.986871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeab9ec-54af-49c5-ac43-2b7c36be98be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:52.988004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeab9ec-54af-49c5-ac43-2b7c36be98be admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:52.988302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeab9ec-54af-49c5-ac43-2b7c36be98be admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:52.988589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeab9ec-54af-49c5-ac43-2b7c36be98be admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:52.989339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeab9ec-54af-49c5-ac43-2b7c36be98be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:52.992297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a08f8c-105d-4b45-b386-dc747bffa936 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:52.992905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a08f8c-105d-4b45-b386-dc747bffa936 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:52.993825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a08f8c-105d-4b45-b386-dc747bffa936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:52.994756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeab9ec-54af-49c5-ac43-2b7c36be98be admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:53.022333 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1053/2089] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:03:53.025695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeab9ec-54af-49c5-ac43-2b7c36be98be admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:53.038025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-76a08f8c-105d-4b45-b386-dc747bffa936 None None] MFA Rules not processed for user `1b2a9ebcd3f84b758322a80c6e590de0`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:53.039431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-76a08f8c-105d-4b45-b386-dc747bffa936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:53.043638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeab9ec-54af-49c5-ac43-2b7c36be98be admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:53.064367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de080a-bc2e-4eb7-a09a-1f2082fdcc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:53.065301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de080a-bc2e-4eb7-a09a-1f2082fdcc9d admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:53.065592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de080a-bc2e-4eb7-a09a-1f2082fdcc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:53.065896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de080a-bc2e-4eb7-a09a-1f2082fdcc9d admin admin] PATH_INFO: `/v3/projects/678cc4f0eb5849beb1362526b8062fbd/users/e53cbcf1c1304ac9a0e9800a0c461553/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:53.107516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de080a-bc2e-4eb7-a09a-1f2082fdcc9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=678cc4f0eb5849beb1362526b8062fbd, user_id=e53cbcf1c1304ac9a0e9800a0c461553, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:53.110604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de080a-bc2e-4eb7-a09a-1f2082fdcc9d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:53.175749 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1037/2090] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:03:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:53.203065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a08f8c-105d-4b45-b386-dc747bffa936 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:53.203886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-76a08f8c-105d-4b45-b386-dc747bffa936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:53.231941 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1054/2091] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:53 2022] PUT /identity/v3/projects/678cc4f0eb5849beb1362526b8062fbd/users/e53cbcf1c1304ac9a0e9800a0c461553/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:53.242860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892d553-8e38-4074-bec1-4f66383fd9ef None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:53.243135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892d553-8e38-4074-bec1-4f66383fd9ef None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:53.243677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892d553-8e38-4074-bec1-4f66383fd9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:53.244086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a08f8c-105d-4b45-b386-dc747bffa936 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:53.254666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:53.292802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2892d553-8e38-4074-bec1-4f66383fd9ef None None] MFA Rules not processed for user `e53cbcf1c1304ac9a0e9800a0c461553`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:53.296417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2892d553-8e38-4074-bec1-4f66383fd9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:53.343657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290cb9ff-14c3-4193-b3eb-fe3b6f7268d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:53.345355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290cb9ff-14c3-4193-b3eb-fe3b6f7268d9 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:53.345793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290cb9ff-14c3-4193-b3eb-fe3b6f7268d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:53.346108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290cb9ff-14c3-4193-b3eb-fe3b6f7268d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:53.364148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290cb9ff-14c3-4193-b3eb-fe3b6f7268d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:53.366572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290cb9ff-14c3-4193-b3eb-fe3b6f7268d9 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:53.384256 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1038/2092] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:53.401435 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1055/2093] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:03:53.406420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290cb9ff-14c3-4193-b3eb-fe3b6f7268d9 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:53.420959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:53.514498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892d553-8e38-4074-bec1-4f66383fd9ef None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:53.516249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2892d553-8e38-4074-bec1-4f66383fd9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:53.518819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290cb9ff-14c3-4193-b3eb-fe3b6f7268d9 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:53.533691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2b189-78fe-4906-b780-628b31a9ac46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:53.534464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2b189-78fe-4906-b780-628b31a9ac46 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:53.535323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2b189-78fe-4906-b780-628b31a9ac46 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:53.539750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da2b189-78fe-4906-b780-628b31a9ac46 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:53.539750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da2b189-78fe-4906-b780-628b31a9ac46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:53.539750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da2b189-78fe-4906-b780-628b31a9ac46 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:53.661696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892d553-8e38-4074-bec1-4f66383fd9ef None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:53.676056 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1039/2094] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:03:53 2022] POST /identity/v3/projects => generated 387 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:53.676352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50f277-f382-4729-8959-0c75789800ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:53.676855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50f277-f382-4729-8959-0c75789800ff service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:53.677265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50f277-f382-4729-8959-0c75789800ff service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:53.677548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50f277-f382-4729-8959-0c75789800ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:53.679646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2b189-78fe-4906-b780-628b31a9ac46 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:53.679926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-db50f277-f382-4729-8959-0c75789800ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:53.763360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da2b189-78fe-4906-b780-628b31a9ac46 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:53.769128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd0c99-d56e-4759-a4e0-e9c02bb404f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:53.769971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd0c99-d56e-4759-a4e0-e9c02bb404f0 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:53.770288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd0c99-d56e-4759-a4e0-e9c02bb404f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:53.770576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd0c99-d56e-4759-a4e0-e9c02bb404f0 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:53.771329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd0c99-d56e-4759-a4e0-e9c02bb404f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:53.773347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd0c99-d56e-4759-a4e0-e9c02bb404f0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:53.814236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50f277-f382-4729-8959-0c75789800ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:53.815957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50f277-f382-4729-8959-0c75789800ff service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:53.826613 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1056/2095] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:53.827902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e0976-e1ae-4185-b467-86376f2e94e3 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:53.828185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e0976-e1ae-4185-b467-86376f2e94e3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:53.828462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e0976-e1ae-4185-b467-86376f2e94e3 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:53.829721 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1057/2096] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:53.835307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca39c1-03b7-40f8-b622-d5d33b6da2ff None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:53.836424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca39c1-03b7-40f8-b622-d5d33b6da2ff None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:53.836735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca39c1-03b7-40f8-b622-d5d33b6da2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:53.860708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-15ca39c1-03b7-40f8-b622-d5d33b6da2ff None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:53.862053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-15ca39c1-03b7-40f8-b622-d5d33b6da2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:53.901150 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1040/2097] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:03:53 2022] POST /identity/v3/users => generated 300 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:03:53.905034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd0c99-d56e-4759-a4e0-e9c02bb404f0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:53.917057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd0c99-d56e-4759-a4e0-e9c02bb404f0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:53.925808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145e4d5-8233-4cc3-9397-371866d1cee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:53.926764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145e4d5-8233-4cc3-9397-371866d1cee1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:53.927150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145e4d5-8233-4cc3-9397-371866d1cee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:53.927503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145e4d5-8233-4cc3-9397-371866d1cee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:53.929027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145e4d5-8233-4cc3-9397-371866d1cee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:53.931498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145e4d5-8233-4cc3-9397-371866d1cee1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:53.949779 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1041/2098] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:03:53.951055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145e4d5-8233-4cc3-9397-371866d1cee1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:53.973183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145e4d5-8233-4cc3-9397-371866d1cee1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:53.982263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f64f8-c446-4a1b-b1b7-cf4f38610da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:53.983434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f64f8-c446-4a1b-b1b7-cf4f38610da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:53.983961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f64f8-c446-4a1b-b1b7-cf4f38610da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:53.984486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f64f8-c446-4a1b-b1b7-cf4f38610da9 admin admin] PATH_INFO: `/v3/projects/7daa7a7b054a4caf8d81c5876f049ce8/users/3969be94757a4dee859abe308fb852b6/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:53.993187 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1058/2099] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:03:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:54.008660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f64f8-c446-4a1b-b1b7-cf4f38610da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7daa7a7b054a4caf8d81c5876f049ce8, user_id=3969be94757a4dee859abe308fb852b6, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:54.011346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f64f8-c446-4a1b-b1b7-cf4f38610da9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:54.014432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca39c1-03b7-40f8-b622-d5d33b6da2ff None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:54.015889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-15ca39c1-03b7-40f8-b622-d5d33b6da2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:54.039964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca39c1-03b7-40f8-b622-d5d33b6da2ff None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:54.055978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7265b-a222-4aa8-95d2-01cfc52ed4b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:54.058113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7265b-a222-4aa8-95d2-01cfc52ed4b4 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:54.058695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7265b-a222-4aa8-95d2-01cfc52ed4b4 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:54.059424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7265b-a222-4aa8-95d2-01cfc52ed4b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:54.070786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7265b-a222-4aa8-95d2-01cfc52ed4b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:54.072669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f7265b-a222-4aa8-95d2-01cfc52ed4b4 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:54.082735 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1042/2100] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:53 2022] PUT /identity/v3/projects/7daa7a7b054a4caf8d81c5876f049ce8/users/3969be94757a4dee859abe308fb852b6/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:54.085306 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1059/2101] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:54.087678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f64f8-c446-4a1b-b1b7-cf4f38610da9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:54.195689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:54.211873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f64f8-c446-4a1b-b1b7-cf4f38610da9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:54.217541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f2c21-5559-4c3f-af57-ea54a123deb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:54.218521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f2c21-5559-4c3f-af57-ea54a123deb1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:54.218973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f2c21-5559-4c3f-af57-ea54a123deb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:54.219570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f2c21-5559-4c3f-af57-ea54a123deb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:54.220504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f2c21-5559-4c3f-af57-ea54a123deb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:54.222394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6f2c21-5559-4c3f-af57-ea54a123deb1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:54.234104 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1043/2102] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:03:54.238196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7265b-a222-4aa8-95d2-01cfc52ed4b4 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:54.250089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7265b-a222-4aa8-95d2-01cfc52ed4b4 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:54.255400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942b855a-14ec-464d-986b-8822781f0645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:54.256317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b855a-14ec-464d-986b-8822781f0645 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:54.256605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b855a-14ec-464d-986b-8822781f0645 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:54.258940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942b855a-14ec-464d-986b-8822781f0645 admin admin] PATH_INFO: `/v3/projects/7daa7a7b054a4caf8d81c5876f049ce8/users/3969be94757a4dee859abe308fb852b6/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:54.258940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b855a-14ec-464d-986b-8822781f0645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7daa7a7b054a4caf8d81c5876f049ce8, user_id=3969be94757a4dee859abe308fb852b6, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:54.261850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942b855a-14ec-464d-986b-8822781f0645 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:54.272710 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1060/2103] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:03:54 2022] PUT /identity/v3/projects/7daa7a7b054a4caf8d81c5876f049ce8/users/3969be94757a4dee859abe308fb852b6/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:54.285381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:54.292865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec34e3-b4a9-43b7-8fe9-339075da0ed8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:54.294250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec34e3-b4a9-43b7-8fe9-339075da0ed8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:54.294778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec34e3-b4a9-43b7-8fe9-339075da0ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:54.355063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2300ad-de4d-4b36-ac67-b4e68e9ad77d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:54.355063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2300ad-de4d-4b36-ac67-b4e68e9ad77d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:54.355063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2300ad-de4d-4b36-ac67-b4e68e9ad77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:54.374341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-82ec34e3-b4a9-43b7-8fe9-339075da0ed8 None None] MFA Rules not processed for user `3969be94757a4dee859abe308fb852b6`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:54.377375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-82ec34e3-b4a9-43b7-8fe9-339075da0ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:54.394832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6b2300ad-de4d-4b36-ac67-b4e68e9ad77d None None] MFA Rules not processed for user `0e733cc9497a48cba4ccdde860be104f`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:54.399152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6b2300ad-de4d-4b36-ac67-b4e68e9ad77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:54.484714 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1061/2104] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:03:54.484714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956cbee-b185-447e-92cc-a394fafc4267 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:54.484714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956cbee-b185-447e-92cc-a394fafc4267 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:54.484714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956cbee-b185-447e-92cc-a394fafc4267 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:54.491261 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1062/2105] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:03:54 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:54.492680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcb3d6-7231-4e39-ab6f-d705773443ff None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:54.497022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcb3d6-7231-4e39-ab6f-d705773443ff None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:54.497022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcb3d6-7231-4e39-ab6f-d705773443ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:54.513614 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1044/2106] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:03:54.518320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-93fcb3d6-7231-4e39-ab6f-d705773443ff None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:54.520079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-93fcb3d6-7231-4e39-ab6f-d705773443ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:54.613438 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1063/2107] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:03:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:54.626259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec34e3-b4a9-43b7-8fe9-339075da0ed8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:54.701823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec34e3-b4a9-43b7-8fe9-339075da0ed8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:54.706304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669a59f-333b-4a73-9d7c-1baba88b2a17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:54.706849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669a59f-333b-4a73-9d7c-1baba88b2a17 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:54.707127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669a59f-333b-4a73-9d7c-1baba88b2a17 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:54.707528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669a59f-333b-4a73-9d7c-1baba88b2a17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:54.721699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669a59f-333b-4a73-9d7c-1baba88b2a17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:54.724111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669a59f-333b-4a73-9d7c-1baba88b2a17 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:54.742183 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1045/2108] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:54.870854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fcb3d6-7231-4e39-ab6f-d705773443ff None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:54.883847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fcb3d6-7231-4e39-ab6f-d705773443ff None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:54.889968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63204f2-ca94-4d4b-b434-a6520eda2555 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:54.890637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63204f2-ca94-4d4b-b434-a6520eda2555 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:54.891041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63204f2-ca94-4d4b-b434-a6520eda2555 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:54.891325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63204f2-ca94-4d4b-b434-a6520eda2555 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:54.901221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63204f2-ca94-4d4b-b434-a6520eda2555 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:54.903154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63204f2-ca94-4d4b-b434-a6520eda2555 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:54.914825 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1064/2109] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:56.081618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669a59f-333b-4a73-9d7c-1baba88b2a17 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:56.083154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8669a59f-333b-4a73-9d7c-1baba88b2a17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:56.099385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669a59f-333b-4a73-9d7c-1baba88b2a17 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:56.108719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a775ce-de87-4b0b-99d9-3b9aa159121b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:56.109384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a775ce-de87-4b0b-99d9-3b9aa159121b service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:56.109670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a775ce-de87-4b0b-99d9-3b9aa159121b service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:56.109985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a775ce-de87-4b0b-99d9-3b9aa159121b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:56.111787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-01a775ce-de87-4b0b-99d9-3b9aa159121b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:56.181805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a775ce-de87-4b0b-99d9-3b9aa159121b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:56.183808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a775ce-de87-4b0b-99d9-3b9aa159121b service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:56.194493 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1046/2110] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:57.399337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63204f2-ca94-4d4b-b434-a6520eda2555 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:57.401216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b63204f2-ca94-4d4b-b434-a6520eda2555 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:57.432885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:57.482480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63204f2-ca94-4d4b-b434-a6520eda2555 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:57.487687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021512dc-f2d1-4fc4-9753-8ed081756b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:57.488710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021512dc-f2d1-4fc4-9753-8ed081756b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:57.489167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021512dc-f2d1-4fc4-9753-8ed081756b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:57.489448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021512dc-f2d1-4fc4-9753-8ed081756b5d admin admin] PATH_INFO: `/v3/users/d870c609a91f4540bfbd7471106fc398` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:57.491538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021512dc-f2d1-4fc4-9753-8ed081756b5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d870c609a91f4540bfbd7471106fc398)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:57.493555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021512dc-f2d1-4fc4-9753-8ed081756b5d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:57.745318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-021512dc-f2d1-4fc4-9753-8ed081756b5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd870c609a91f4540bfbd7471106fc398'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:57.752191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-021512dc-f2d1-4fc4-9753-8ed081756b5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd870c609a91f4540bfbd7471106fc398'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:57.757623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-021512dc-f2d1-4fc4-9753-8ed081756b5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd870c609a91f4540bfbd7471106fc398'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:57.757623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-021512dc-f2d1-4fc4-9753-8ed081756b5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd870c609a91f4540bfbd7471106fc398'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:57.816336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-021512dc-f2d1-4fc4-9753-8ed081756b5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd870c609a91f4540bfbd7471106fc398'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:57.836672 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1065/2111] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:57 2022] DELETE /identity/v3/users/d870c609a91f4540bfbd7471106fc398 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:57.840929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3c869-17f3-45a8-93f4-48144e4594eb None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:57.841450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3c869-17f3-45a8-93f4-48144e4594eb None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:57.841450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3c869-17f3-45a8-93f4-48144e4594eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:57.848559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:57.860159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c3e3c869-17f3-45a8-93f4-48144e4594eb None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:57.862936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c3e3c869-17f3-45a8-93f4-48144e4594eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:57.933806 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1047/2112] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:57.943895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021512dc-f2d1-4fc4-9753-8ed081756b5d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:57.946919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-021512dc-f2d1-4fc4-9753-8ed081756b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:58.010486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021512dc-f2d1-4fc4-9753-8ed081756b5d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:58.014151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75368270-0a2d-4c59-b5f9-1f543962732c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:58.014892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75368270-0a2d-4c59-b5f9-1f543962732c service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:58.015092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75368270-0a2d-4c59-b5f9-1f543962732c service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:58.015373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75368270-0a2d-4c59-b5f9-1f543962732c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:58.025519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75368270-0a2d-4c59-b5f9-1f543962732c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:58.027156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75368270-0a2d-4c59-b5f9-1f543962732c service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:58.038613 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1066/2113] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:58.110891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f8588-1bba-4bec-980d-16e9d4783bd1 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:58.111654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f8588-1bba-4bec-980d-16e9d4783bd1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:58.111798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f8588-1bba-4bec-980d-16e9d4783bd1 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:58.113410 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1048/2114] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:03:58.119169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d85c8-f20c-440f-99b3-3790f6403140 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:58.119470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d85c8-f20c-440f-99b3-3790f6403140 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:58.119785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724d85c8-f20c-440f-99b3-3790f6403140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:58.144527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-724d85c8-f20c-440f-99b3-3790f6403140 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:58.145771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-724d85c8-f20c-440f-99b3-3790f6403140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:58.202868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:58.229854 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1067/2115] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:03:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:58.254501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f8588-1bba-4bec-980d-16e9d4783bd1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:58.256357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-118f8588-1bba-4bec-980d-16e9d4783bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:58.275096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f8588-1bba-4bec-980d-16e9d4783bd1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:58.281226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fb836-ee88-4c7d-bc4a-c2543f3550b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:58.282374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fb836-ee88-4c7d-bc4a-c2543f3550b1 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:58.282374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fb836-ee88-4c7d-bc4a-c2543f3550b1 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:58.282827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fb836-ee88-4c7d-bc4a-c2543f3550b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:58.299743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fb836-ee88-4c7d-bc4a-c2543f3550b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:58.302790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fb836-ee88-4c7d-bc4a-c2543f3550b1 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:58.342077 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1049/2116] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:58.533540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d85c8-f20c-440f-99b3-3790f6403140 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:58.536203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-724d85c8-f20c-440f-99b3-3790f6403140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:58.557439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724d85c8-f20c-440f-99b3-3790f6403140 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:58.566081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:58.566966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:58.567249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:58.567525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919 admin admin] PATH_INFO: `/v3/projects/7133f6301e034a2faf8215358bf3e691` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:58.569477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7133f6301e034a2faf8215358bf3e691)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:58.572103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:58.645722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fb836-ee88-4c7d-bc4a-c2543f3550b1 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:58.648534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c88fb836-ee88-4c7d-bc4a-c2543f3550b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:58.662902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fb836-ee88-4c7d-bc4a-c2543f3550b1 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:58.668536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4259fc-6538-441e-8433-5650a2e619c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:58.669432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4259fc-6538-441e-8433-5650a2e619c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:58.669709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4259fc-6538-441e-8433-5650a2e619c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:58.670050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4259fc-6538-441e-8433-5650a2e619c6 admin admin] PATH_INFO: `/v3/users/e53cbcf1c1304ac9a0e9800a0c461553` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:58.671271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4259fc-6538-441e-8433-5650a2e619c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e53cbcf1c1304ac9a0e9800a0c461553)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:58.673982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4259fc-6538-441e-8433-5650a2e619c6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:58.722695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7133f6301e034a2faf8215358bf3e691', 'request_id': 'req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:58.730679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7133f6301e034a2faf8215358bf3e691', 'request_id': 'req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:58.873314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7133f6301e034a2faf8215358bf3e691', 'request_id': 'req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:58.899982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919 admin admin] The token cache is being invalidate because project 7133f6301e034a2faf8215358bf3e691 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:03:58.900310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:58.901316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:58.905869 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1068/2117] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:03:58 2022] DELETE /identity/v3/projects/7133f6301e034a2faf8215358bf3e691 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:03:58.905869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:58.906910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:58.994931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c9fbd-cb59-4b9d-8fe5-dc64242ed919 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:59.000573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ff901-9cbb-46ba-bbfc-16a9ee94267d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:59.001388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ff901-9cbb-46ba-bbfc-16a9ee94267d service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:59.001685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ff901-9cbb-46ba-bbfc-16a9ee94267d service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:59.002018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ff901-9cbb-46ba-bbfc-16a9ee94267d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:59.003893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cd9ff901-9cbb-46ba-bbfc-16a9ee94267d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:59.020804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ff901-9cbb-46ba-bbfc-16a9ee94267d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:59.022743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9ff901-9cbb-46ba-bbfc-16a9ee94267d service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:59.027399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7f4259fc-6538-441e-8433-5650a2e619c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e53cbcf1c1304ac9a0e9800a0c461553'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:59.028423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7f4259fc-6538-441e-8433-5650a2e619c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e53cbcf1c1304ac9a0e9800a0c461553'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:59.037547 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1069/2118] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:59.037547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ff901-9cbb-46ba-bbfc-16a9ee94267d service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:59.037547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cd9ff901-9cbb-46ba-bbfc-16a9ee94267d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:59.070028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7f4259fc-6538-441e-8433-5650a2e619c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e53cbcf1c1304ac9a0e9800a0c461553'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:59.113406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:59.129963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ff901-9cbb-46ba-bbfc-16a9ee94267d service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:59.136894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b11d4-47c1-4111-be4c-81c2068d7fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:59.140572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b11d4-47c1-4111-be4c-81c2068d7fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:59.140572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b11d4-47c1-4111-be4c-81c2068d7fee admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:59.140572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b11d4-47c1-4111-be4c-81c2068d7fee admin admin] PATH_INFO: `/v3/users/2fb485e80a0c42e6bae2142de7c23338` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:59.140572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b11d4-47c1-4111-be4c-81c2068d7fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fb485e80a0c42e6bae2142de7c23338)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:59.141775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b11d4-47c1-4111-be4c-81c2068d7fee admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:59.188137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7f4259fc-6538-441e-8433-5650a2e619c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e53cbcf1c1304ac9a0e9800a0c461553'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:59.191527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7f4259fc-6538-441e-8433-5650a2e619c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e53cbcf1c1304ac9a0e9800a0c461553'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:59.216246 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1050/2119] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:58 2022] DELETE /identity/v3/users/e53cbcf1c1304ac9a0e9800a0c461553 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:03:59.220677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb529167-29ce-40d2-b5d9-6feac389257f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:59.220974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb529167-29ce-40d2-b5d9-6feac389257f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:59.221317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb529167-29ce-40d2-b5d9-6feac389257f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:59.244484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-cb529167-29ce-40d2-b5d9-6feac389257f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:03:59.246352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cb529167-29ce-40d2-b5d9-6feac389257f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:59.334268 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1051/2120] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:03:59.347318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb529167-29ce-40d2-b5d9-6feac389257f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:03:59.349670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cb529167-29ce-40d2-b5d9-6feac389257f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:59.359585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:59.389534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0a4b11d4-47c1-4111-be4c-81c2068d7fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fb485e80a0c42e6bae2142de7c23338'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:59.413228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0a4b11d4-47c1-4111-be4c-81c2068d7fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fb485e80a0c42e6bae2142de7c23338'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:59.413228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0a4b11d4-47c1-4111-be4c-81c2068d7fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fb485e80a0c42e6bae2142de7c23338'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:59.414550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0a4b11d4-47c1-4111-be4c-81c2068d7fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fb485e80a0c42e6bae2142de7c23338'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:59.456013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb529167-29ce-40d2-b5d9-6feac389257f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:03:59.460109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05729feb-1524-46f3-b032-e97803b6b5e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:03:59.460873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05729feb-1524-46f3-b032-e97803b6b5e5 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:03:59.462208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05729feb-1524-46f3-b032-e97803b6b5e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:03:59.462208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05729feb-1524-46f3-b032-e97803b6b5e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:03:59.463622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-05729feb-1524-46f3-b032-e97803b6b5e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:03:59.477199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05729feb-1524-46f3-b032-e97803b6b5e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:03:59.478926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05729feb-1524-46f3-b032-e97803b6b5e5 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:03:59.490077 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1052/2121] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:03:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:03:59.507534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0a4b11d4-47c1-4111-be4c-81c2068d7fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fb485e80a0c42e6bae2142de7c23338'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:03:59.535580 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1070/2122] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:03:59 2022] DELETE /identity/v3/users/2fb485e80a0c42e6bae2142de7c23338 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:00.008593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b463d7-dc98-48d2-bc3a-53ed646db0c0 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:00.008988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b463d7-dc98-48d2-bc3a-53ed646db0c0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:00.009369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b463d7-dc98-48d2-bc3a-53ed646db0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:00.029080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-90b463d7-dc98-48d2-bc3a-53ed646db0c0 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:00.030332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-90b463d7-dc98-48d2-bc3a-53ed646db0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:00.060438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b11d4-47c1-4111-be4c-81c2068d7fee admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:00.060438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0a4b11d4-47c1-4111-be4c-81c2068d7fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:00.199961 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1053/2123] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:04:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:00.200950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b463d7-dc98-48d2-bc3a-53ed646db0c0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:00.202573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-90b463d7-dc98-48d2-bc3a-53ed646db0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:00.210491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b11d4-47c1-4111-be4c-81c2068d7fee admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:00.217593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ed5fc-f732-4472-80cd-102bebeb048a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:00.218905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ed5fc-f732-4472-80cd-102bebeb048a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:00.219356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ed5fc-f732-4472-80cd-102bebeb048a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:00.219801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ed5fc-f732-4472-80cd-102bebeb048a admin admin] PATH_INFO: `/v3/projects/678cc4f0eb5849beb1362526b8062fbd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:00.220194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b463d7-dc98-48d2-bc3a-53ed646db0c0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:00.221492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ed5fc-f732-4472-80cd-102bebeb048a admin admin] RBAC: Authorizing `identity:delete_project(project_id=678cc4f0eb5849beb1362526b8062fbd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:00.224281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ed5fc-f732-4472-80cd-102bebeb048a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:00.226425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a618301-3887-4c47-92b9-35644c755d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:00.231032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a618301-3887-4c47-92b9-35644c755d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:00.231032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a618301-3887-4c47-92b9-35644c755d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:00.231032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a618301-3887-4c47-92b9-35644c755d00 admin admin] PATH_INFO: `/v3/projects/0eeea9efb9364971b8335deff54816d3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:00.231032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a618301-3887-4c47-92b9-35644c755d00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eeea9efb9364971b8335deff54816d3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:00.233807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a618301-3887-4c47-92b9-35644c755d00 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:00.379384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7a3ed5fc-f732-4472-80cd-102bebeb048a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '678cc4f0eb5849beb1362526b8062fbd', 'request_id': 'req-7a3ed5fc-f732-4472-80cd-102bebeb048a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:00.384798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a618301-3887-4c47-92b9-35644c755d00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eeea9efb9364971b8335deff54816d3', 'request_id': 'req-3a618301-3887-4c47-92b9-35644c755d00'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:00.388391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7a3ed5fc-f732-4472-80cd-102bebeb048a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '678cc4f0eb5849beb1362526b8062fbd', 'request_id': 'req-7a3ed5fc-f732-4472-80cd-102bebeb048a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:00.393921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a618301-3887-4c47-92b9-35644c755d00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eeea9efb9364971b8335deff54816d3', 'request_id': 'req-3a618301-3887-4c47-92b9-35644c755d00'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:00.468242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7a3ed5fc-f732-4472-80cd-102bebeb048a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '678cc4f0eb5849beb1362526b8062fbd', 'request_id': 'req-7a3ed5fc-f732-4472-80cd-102bebeb048a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:00.489208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:00.494452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7a3ed5fc-f732-4472-80cd-102bebeb048a admin admin] The token cache is being invalidate because project 678cc4f0eb5849beb1362526b8062fbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:00.494828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7a3ed5fc-f732-4472-80cd-102bebeb048a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:00.495873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7a3ed5fc-f732-4472-80cd-102bebeb048a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:00.497798 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1071/2124] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:00 2022] DELETE /identity/v3/projects/678cc4f0eb5849beb1362526b8062fbd => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:00.498495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ed5fc-f732-4472-80cd-102bebeb048a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:00.499883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7a3ed5fc-f732-4472-80cd-102bebeb048a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:00.525214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a618301-3887-4c47-92b9-35644c755d00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eeea9efb9364971b8335deff54816d3', 'request_id': 'req-3a618301-3887-4c47-92b9-35644c755d00'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:00.538213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a618301-3887-4c47-92b9-35644c755d00 admin admin] The token cache is being invalidate because project 0eeea9efb9364971b8335deff54816d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:00.538586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a618301-3887-4c47-92b9-35644c755d00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:00.539345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a618301-3887-4c47-92b9-35644c755d00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:00.540945 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1054/2125] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:00 2022] DELETE /identity/v3/projects/0eeea9efb9364971b8335deff54816d3 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:00.544778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a618301-3887-4c47-92b9-35644c755d00 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:00.544778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3a618301-3887-4c47-92b9-35644c755d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:00.580108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ed5fc-f732-4472-80cd-102bebeb048a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:00.588992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4686e800-9140-4818-863b-c9ca6d89d8e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:00.593766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686e800-9140-4818-863b-c9ca6d89d8e9 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:00.593766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686e800-9140-4818-863b-c9ca6d89d8e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:00.593766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4686e800-9140-4818-863b-c9ca6d89d8e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:00.593766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4686e800-9140-4818-863b-c9ca6d89d8e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:00.636197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a618301-3887-4c47-92b9-35644c755d00 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:00.678651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:00.713887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686e800-9140-4818-863b-c9ca6d89d8e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:00.716155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4686e800-9140-4818-863b-c9ca6d89d8e9 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:00.729575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ed390-9915-4994-9688-fd98c22dbe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:00.733255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ed390-9915-4994-9688-fd98c22dbe60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:00.733255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ed390-9915-4994-9688-fd98c22dbe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:00.733255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ed390-9915-4994-9688-fd98c22dbe60 admin admin] PATH_INFO: `/v3/users/3969be94757a4dee859abe308fb852b6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:00.733255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ed390-9915-4994-9688-fd98c22dbe60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3969be94757a4dee859abe308fb852b6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:00.734177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5ed390-9915-4994-9688-fd98c22dbe60 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:00.735541 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1072/2126] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:00.742191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155733b-c3c5-4058-89e6-826810a477e2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:00.742941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155733b-c3c5-4058-89e6-826810a477e2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:00.743511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155733b-c3c5-4058-89e6-826810a477e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:00.778123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5155733b-c3c5-4058-89e6-826810a477e2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:00.780366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5155733b-c3c5-4058-89e6-826810a477e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:00.793632 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1073/2127] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:00.801738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283cfa9-60d0-474d-a354-2815464e2520 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:00.801935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283cfa9-60d0-474d-a354-2815464e2520 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:00.802133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283cfa9-60d0-474d-a354-2815464e2520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:00.829224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d283cfa9-60d0-474d-a354-2815464e2520 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:00.830836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d283cfa9-60d0-474d-a354-2815464e2520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:00.842814 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1074/2128] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:04:00.846489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283cfa9-60d0-474d-a354-2815464e2520 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:00.857514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283cfa9-60d0-474d-a354-2815464e2520 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:00.863183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881b581-8eec-4256-85dd-6c95c1e77023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:00.863933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881b581-8eec-4256-85dd-6c95c1e77023 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:00.864296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881b581-8eec-4256-85dd-6c95c1e77023 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:00.864670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881b581-8eec-4256-85dd-6c95c1e77023 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:00.865595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881b581-8eec-4256-85dd-6c95c1e77023 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:00.866013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881b581-8eec-4256-85dd-6c95c1e77023 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:00.867976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881b581-8eec-4256-85dd-6c95c1e77023 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:00.883190 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1075/2129] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:04:00.886325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881b581-8eec-4256-85dd-6c95c1e77023 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:00.898479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881b581-8eec-4256-85dd-6c95c1e77023 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:00.904414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1f06e-e916-46d5-a2fc-c27e07383fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:00.905145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1f06e-e916-46d5-a2fc-c27e07383fba admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:00.905440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1f06e-e916-46d5-a2fc-c27e07383fba admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:00.905828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db1f06e-e916-46d5-a2fc-c27e07383fba admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:00.906576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db1f06e-e916-46d5-a2fc-c27e07383fba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:00.909048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db1f06e-e916-46d5-a2fc-c27e07383fba admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:00.942504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec5ed390-9915-4994-9688-fd98c22dbe60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3969be94757a4dee859abe308fb852b6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:00.943160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec5ed390-9915-4994-9688-fd98c22dbe60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3969be94757a4dee859abe308fb852b6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:00.950463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec5ed390-9915-4994-9688-fd98c22dbe60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3969be94757a4dee859abe308fb852b6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:01.018095 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1076/2130] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:00 2022] POST /identity/v3/projects => generated 393 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:04:01.022706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1f06e-e916-46d5-a2fc-c27e07383fba admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:01.067629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec5ed390-9915-4994-9688-fd98c22dbe60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3969be94757a4dee859abe308fb852b6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:01.068316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec5ed390-9915-4994-9688-fd98c22dbe60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3969be94757a4dee859abe308fb852b6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:01.086507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db1f06e-e916-46d5-a2fc-c27e07383fba admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:01.088155 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1055/2131] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:00 2022] DELETE /identity/v3/users/3969be94757a4dee859abe308fb852b6 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:01.150576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f6c5f-85d2-4d11-9b9f-d57a65de44b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:01.151209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f6c5f-85d2-4d11-9b9f-d57a65de44b8 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:01.151498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f6c5f-85d2-4d11-9b9f-d57a65de44b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:01.151774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f6c5f-85d2-4d11-9b9f-d57a65de44b8 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:01.152506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f6c5f-85d2-4d11-9b9f-d57a65de44b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:01.154516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f6c5f-85d2-4d11-9b9f-d57a65de44b8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:01.312691 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1077/2132] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:01 2022] POST /identity/v3/users => generated 297 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:04:01.314332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ed390-9915-4994-9688-fd98c22dbe60 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:01.316503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ec5ed390-9915-4994-9688-fd98c22dbe60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:01.330578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ed390-9915-4994-9688-fd98c22dbe60 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:01.335204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a7a21-b6c8-4067-8992-7ca3e93a681f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:01.335902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a7a21-b6c8-4067-8992-7ca3e93a681f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:01.339805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a7a21-b6c8-4067-8992-7ca3e93a681f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:01.339805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925a7a21-b6c8-4067-8992-7ca3e93a681f admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:01.339805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a7a21-b6c8-4067-8992-7ca3e93a681f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:01.339805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925a7a21-b6c8-4067-8992-7ca3e93a681f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:01.353817 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1056/2133] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:04:01.355083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f6c5f-85d2-4d11-9b9f-d57a65de44b8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:01.366989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f6c5f-85d2-4d11-9b9f-d57a65de44b8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:01.373675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26e0d1-c1e9-49a5-96d3-4c3ee00a9e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:01.374543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26e0d1-c1e9-49a5-96d3-4c3ee00a9e1f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:01.375041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26e0d1-c1e9-49a5-96d3-4c3ee00a9e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:01.375331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26e0d1-c1e9-49a5-96d3-4c3ee00a9e1f admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:01.376200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26e0d1-c1e9-49a5-96d3-4c3ee00a9e1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:01.377955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26e0d1-c1e9-49a5-96d3-4c3ee00a9e1f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:01.391149 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1078/2134] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:01.393249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a7a21-b6c8-4067-8992-7ca3e93a681f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:01.404867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925a7a21-b6c8-4067-8992-7ca3e93a681f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:01.410990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2a216-3c21-4379-b797-7dceedbbc861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:01.411879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2a216-3c21-4379-b797-7dceedbbc861 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:01.414374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2a216-3c21-4379-b797-7dceedbbc861 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:01.414374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2a216-3c21-4379-b797-7dceedbbc861 admin admin] PATH_INFO: `/v3/projects/14b229b388b845a7ae717c6d3238bdf1/users/f65e0f0097824cef8210a680b498e37a/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:01.429450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2a216-3c21-4379-b797-7dceedbbc861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14b229b388b845a7ae717c6d3238bdf1, user_id=f65e0f0097824cef8210a680b498e37a, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:01.432507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a2a216-3c21-4379-b797-7dceedbbc861 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:01.527792 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1057/2135] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:01 2022] PUT /identity/v3/projects/14b229b388b845a7ae717c6d3238bdf1/users/f65e0f0097824cef8210a680b498e37a/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:01.538926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e57ef8-ce33-4309-9137-d6327fe65af0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:01.539239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e57ef8-ce33-4309-9137-d6327fe65af0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:01.539517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e57ef8-ce33-4309-9137-d6327fe65af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:01.570880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-92e57ef8-ce33-4309-9137-d6327fe65af0 None None] MFA Rules not processed for user `f65e0f0097824cef8210a680b498e37a`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:01.573193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-92e57ef8-ce33-4309-9137-d6327fe65af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:01.573336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2a216-3c21-4379-b797-7dceedbbc861 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:01.575027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-61a2a216-3c21-4379-b797-7dceedbbc861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:01.611641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:01.714669 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1079/2136] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:04:01.719243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e57ef8-ce33-4309-9137-d6327fe65af0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:01.733824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a2a216-3c21-4379-b797-7dceedbbc861 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:01.737550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e57ef8-ce33-4309-9137-d6327fe65af0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:01.745606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be384de7-c10b-48b9-ba64-352b24d1da48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:01.746631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be384de7-c10b-48b9-ba64-352b24d1da48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:01.746921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be384de7-c10b-48b9-ba64-352b24d1da48 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:01.747234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be384de7-c10b-48b9-ba64-352b24d1da48 admin admin] PATH_INFO: `/v3/projects/7daa7a7b054a4caf8d81c5876f049ce8` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:01.748482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8cfeb-1b64-4c30-a59c-1d1353941725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:01.748844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be384de7-c10b-48b9-ba64-352b24d1da48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7daa7a7b054a4caf8d81c5876f049ce8)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:01.751491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8cfeb-1b64-4c30-a59c-1d1353941725 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:01.752079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be384de7-c10b-48b9-ba64-352b24d1da48 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:01.756182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:01.756699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8cfeb-1b64-4c30-a59c-1d1353941725 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:01.759522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad8cfeb-1b64-4c30-a59c-1d1353941725 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:01.759522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8cfeb-1b64-4c30-a59c-1d1353941725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:01.761570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad8cfeb-1b64-4c30-a59c-1d1353941725 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:01.894919 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1080/2137] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:01 2022] POST /identity/v3/projects => generated 391 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:04:01.896447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8cfeb-1b64-4c30-a59c-1d1353941725 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:01.903368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-be384de7-c10b-48b9-ba64-352b24d1da48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7daa7a7b054a4caf8d81c5876f049ce8', 'request_id': 'req-be384de7-c10b-48b9-ba64-352b24d1da48'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:01.912091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-be384de7-c10b-48b9-ba64-352b24d1da48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7daa7a7b054a4caf8d81c5876f049ce8', 'request_id': 'req-be384de7-c10b-48b9-ba64-352b24d1da48'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:01.970084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad8cfeb-1b64-4c30-a59c-1d1353941725 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:01.975526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a66b2d-6a9c-42ac-a3fb-9151ff2258e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:01.981232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a66b2d-6a9c-42ac-a3fb-9151ff2258e4 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:01.981232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a66b2d-6a9c-42ac-a3fb-9151ff2258e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:01.981232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a66b2d-6a9c-42ac-a3fb-9151ff2258e4 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:01.981232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a66b2d-6a9c-42ac-a3fb-9151ff2258e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:01.981232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a66b2d-6a9c-42ac-a3fb-9151ff2258e4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:02.016223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-be384de7-c10b-48b9-ba64-352b24d1da48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7daa7a7b054a4caf8d81c5876f049ce8', 'request_id': 'req-be384de7-c10b-48b9-ba64-352b24d1da48'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:02.032128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-be384de7-c10b-48b9-ba64-352b24d1da48 admin admin] The token cache is being invalidate because project 7daa7a7b054a4caf8d81c5876f049ce8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:02.032421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-be384de7-c10b-48b9-ba64-352b24d1da48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:02.033082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-be384de7-c10b-48b9-ba64-352b24d1da48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:02.034705 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1058/2138] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:01 2022] DELETE /identity/v3/projects/7daa7a7b054a4caf8d81c5876f049ce8 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:04:02.040283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af07fa3-9487-459e-b5d9-2495d9e69f1f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:02.040571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af07fa3-9487-459e-b5d9-2495d9e69f1f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:02.040845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af07fa3-9487-459e-b5d9-2495d9e69f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:02.059546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1af07fa3-9487-459e-b5d9-2495d9e69f1f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:02.061386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1af07fa3-9487-459e-b5d9-2495d9e69f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:02.076318 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1081/2139] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:01 2022] POST /identity/v3/users => generated 302 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:02.077880 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1059/2140] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:02.083165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a66b2d-6a9c-42ac-a3fb-9151ff2258e4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:02.084927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978abf10-757e-49d7-95bf-e9db9cc64914 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:02.085379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978abf10-757e-49d7-95bf-e9db9cc64914 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:02.085585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-62a66b2d-6a9c-42ac-a3fb-9151ff2258e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:02.085749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978abf10-757e-49d7-95bf-e9db9cc64914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:02.109660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a66b2d-6a9c-42ac-a3fb-9151ff2258e4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:02.116169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-978abf10-757e-49d7-95bf-e9db9cc64914 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:02.118212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-978abf10-757e-49d7-95bf-e9db9cc64914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:02.118360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1be0ef-8b17-46bb-904f-933adf6733f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:02.119124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1be0ef-8b17-46bb-904f-933adf6733f1 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:02.119408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1be0ef-8b17-46bb-904f-933adf6733f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:02.120495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1be0ef-8b17-46bb-904f-933adf6733f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:02.120768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1be0ef-8b17-46bb-904f-933adf6733f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:02.122438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1be0ef-8b17-46bb-904f-933adf6733f1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:02.133189 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1060/2141] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:04:02.134144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978abf10-757e-49d7-95bf-e9db9cc64914 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:02.139166 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1082/2142] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:02.140881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1be0ef-8b17-46bb-904f-933adf6733f1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:02.150769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978abf10-757e-49d7-95bf-e9db9cc64914 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:02.155852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1be0ef-8b17-46bb-904f-933adf6733f1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:02.156756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4c276-43a4-4e4e-80ec-3fdb0dbefefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:02.157768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4c276-43a4-4e4e-80ec-3fdb0dbefefb admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:02.158128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4c276-43a4-4e4e-80ec-3fdb0dbefefb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:02.158501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d4c276-43a4-4e4e-80ec-3fdb0dbefefb admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:02.159457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4c276-43a4-4e4e-80ec-3fdb0dbefefb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:02.159836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4c276-43a4-4e4e-80ec-3fdb0dbefefb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:02.162552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfab9b-5d84-4d54-88da-0071283254b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:02.162715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d4c276-43a4-4e4e-80ec-3fdb0dbefefb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:02.163413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfab9b-5d84-4d54-88da-0071283254b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:02.163801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfab9b-5d84-4d54-88da-0071283254b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:02.164151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfab9b-5d84-4d54-88da-0071283254b2 admin admin] PATH_INFO: `/v3/projects/e603e36673a74614932c52f9a007e4eb/users/52e93378697f424c855e5db84efadce8/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:02.171615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:02.179269 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1061/2143] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:02.184175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4c276-43a4-4e4e-80ec-3fdb0dbefefb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:02.191493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebfab9b-5d84-4d54-88da-0071283254b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e603e36673a74614932c52f9a007e4eb, user_id=52e93378697f424c855e5db84efadce8, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:02.196452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebfab9b-5d84-4d54-88da-0071283254b2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:02.199214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d4c276-43a4-4e4e-80ec-3fdb0dbefefb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:02.201947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77651843-ad0f-4cbb-9c1d-fa5977e2951a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:02.208154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77651843-ad0f-4cbb-9c1d-fa5977e2951a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:02.208154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77651843-ad0f-4cbb-9c1d-fa5977e2951a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:02.208154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77651843-ad0f-4cbb-9c1d-fa5977e2951a admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:02.208154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77651843-ad0f-4cbb-9c1d-fa5977e2951a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:02.208154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77651843-ad0f-4cbb-9c1d-fa5977e2951a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:02.270007 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1083/2144] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:02 2022] PUT /identity/v3/projects/e603e36673a74614932c52f9a007e4eb/users/52e93378697f424c855e5db84efadce8/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:02.271036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfab9b-5d84-4d54-88da-0071283254b2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:02.274073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-aebfab9b-5d84-4d54-88da-0071283254b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:02.284216 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1062/2145] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:02 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:02.284216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77651843-ad0f-4cbb-9c1d-fa5977e2951a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:02.343784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfab9b-5d84-4d54-88da-0071283254b2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:02.348554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd6cf8-a96b-43a8-8a20-c9ff14dc724a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:02.349466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd6cf8-a96b-43a8-8a20-c9ff14dc724a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:02.349761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd6cf8-a96b-43a8-8a20-c9ff14dc724a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:02.350050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd6cf8-a96b-43a8-8a20-c9ff14dc724a admin admin] PATH_INFO: `/v3/users/89ed91b9d56241d99b6df99ede3fb446` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:02.350933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77651843-ad0f-4cbb-9c1d-fa5977e2951a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:02.351288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd6cf8-a96b-43a8-8a20-c9ff14dc724a admin admin] RBAC: Authorizing `identity:delete_user(user_id=89ed91b9d56241d99b6df99ede3fb446)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:02.353966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd6cf8-a96b-43a8-8a20-c9ff14dc724a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:02.356070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7c992-93aa-47a9-8fe3-02294437d882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:02.360516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7c992-93aa-47a9-8fe3-02294437d882 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:02.360516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7c992-93aa-47a9-8fe3-02294437d882 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:02.360516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7c992-93aa-47a9-8fe3-02294437d882 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:02.360516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7c992-93aa-47a9-8fe3-02294437d882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:02.360516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7c992-93aa-47a9-8fe3-02294437d882 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:02.372621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:02.373732 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1063/2146] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:02.374608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7c992-93aa-47a9-8fe3-02294437d882 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:02.390308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7c992-93aa-47a9-8fe3-02294437d882 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:02.395651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec690808-e32f-4c00-9fc1-748f972d0444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:02.396311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec690808-e32f-4c00-9fc1-748f972d0444 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:02.396594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec690808-e32f-4c00-9fc1-748f972d0444 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:02.397019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec690808-e32f-4c00-9fc1-748f972d0444 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:02.398160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec690808-e32f-4c00-9fc1-748f972d0444 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:02.400078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec690808-e32f-4c00-9fc1-748f972d0444 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:02.480510 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1064/2147] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:02 2022] POST /identity/v3/users => generated 289 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:02.481120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec690808-e32f-4c00-9fc1-748f972d0444 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:02.491549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec690808-e32f-4c00-9fc1-748f972d0444 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:02.497280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91783e08-80b9-484e-82d5-e2c61008d2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:02.498212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91783e08-80b9-484e-82d5-e2c61008d2f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:02.501114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91783e08-80b9-484e-82d5-e2c61008d2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:02.501114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91783e08-80b9-484e-82d5-e2c61008d2f1 admin admin] PATH_INFO: `/v3/projects/e603e36673a74614932c52f9a007e4eb/users/52e93378697f424c855e5db84efadce8/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:02.501114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91783e08-80b9-484e-82d5-e2c61008d2f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e603e36673a74614932c52f9a007e4eb, user_id=52e93378697f424c855e5db84efadce8, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:02.503888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91783e08-80b9-484e-82d5-e2c61008d2f1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:02.505785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7fbd6cf8-a96b-43a8-8a20-c9ff14dc724a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89ed91b9d56241d99b6df99ede3fb446'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:02.514101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7fbd6cf8-a96b-43a8-8a20-c9ff14dc724a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89ed91b9d56241d99b6df99ede3fb446'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:02.515215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7fbd6cf8-a96b-43a8-8a20-c9ff14dc724a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89ed91b9d56241d99b6df99ede3fb446'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:02.515890 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1065/2148] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:02 2022] PUT /identity/v3/projects/e603e36673a74614932c52f9a007e4eb/users/52e93378697f424c855e5db84efadce8/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:02.516157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7fbd6cf8-a96b-43a8-8a20-c9ff14dc724a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89ed91b9d56241d99b6df99ede3fb446'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:02.516826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91783e08-80b9-484e-82d5-e2c61008d2f1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:02.518710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-91783e08-80b9-484e-82d5-e2c61008d2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:02.585141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91783e08-80b9-484e-82d5-e2c61008d2f1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:02.593400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7fbd6cf8-a96b-43a8-8a20-c9ff14dc724a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89ed91b9d56241d99b6df99ede3fb446'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:02.595020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff19d2-b912-46b7-b232-f988b1398fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:02.595723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff19d2-b912-46b7-b232-f988b1398fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:02.596009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff19d2-b912-46b7-b232-f988b1398fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:02.596283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff19d2-b912-46b7-b232-f988b1398fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:02.597043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff19d2-b912-46b7-b232-f988b1398fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:02.600810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caff19d2-b912-46b7-b232-f988b1398fe2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:02.614405 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1066/2149] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:02.617281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb7863-acc4-448f-8434-e59ce5e14838 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:02.617582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb7863-acc4-448f-8434-e59ce5e14838 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:02.618036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb7863-acc4-448f-8434-e59ce5e14838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:02.621308 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1084/2150] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:02 2022] DELETE /identity/v3/users/89ed91b9d56241d99b6df99ede3fb446 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:02.622885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd6cf8-a96b-43a8-8a20-c9ff14dc724a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:02.648147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-34eb7863-acc4-448f-8434-e59ce5e14838 None None] MFA Rules not processed for user `52e93378697f424c855e5db84efadce8`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:02.649759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-34eb7863-acc4-448f-8434-e59ce5e14838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:02.651234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:02.724019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd6cf8-a96b-43a8-8a20-c9ff14dc724a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:02.732144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd57e03-f2b0-42f1-b113-ad18b807d7aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:02.732756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd57e03-f2b0-42f1-b113-ad18b807d7aa admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:02.733042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd57e03-f2b0-42f1-b113-ad18b807d7aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:02.733322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd57e03-f2b0-42f1-b113-ad18b807d7aa admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:02.734162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd57e03-f2b0-42f1-b113-ad18b807d7aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:02.735777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd57e03-f2b0-42f1-b113-ad18b807d7aa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:02.748369 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1085/2151] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:02.749860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4281302-1d2d-48a2-8f9e-90f9487ae677 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:02.750304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4281302-1d2d-48a2-8f9e-90f9487ae677 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:02.750596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4281302-1d2d-48a2-8f9e-90f9487ae677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:02.778121 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1067/2152] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:02.778987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb7863-acc4-448f-8434-e59ce5e14838 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:02.788776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f4281302-1d2d-48a2-8f9e-90f9487ae677 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:02.790099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f4281302-1d2d-48a2-8f9e-90f9487ae677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:02.794678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb7863-acc4-448f-8434-e59ce5e14838 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:02.800707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6984dc-4aff-4858-972f-33cfdad7db22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:02.801372 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1086/2153] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:02.802155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6984dc-4aff-4858-972f-33cfdad7db22 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:02.802443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6984dc-4aff-4858-972f-33cfdad7db22 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:02.802784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6984dc-4aff-4858-972f-33cfdad7db22 admin admin] PATH_INFO: `/v3/projects/cde37d77fb1343f4af1dd8e735a41615/users/05ada83d82f648eeabca0dce3b8eb3cd/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:02.819243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4281302-1d2d-48a2-8f9e-90f9487ae677 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:02.820687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f4281302-1d2d-48a2-8f9e-90f9487ae677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:02.820803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6984dc-4aff-4858-972f-33cfdad7db22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cde37d77fb1343f4af1dd8e735a41615, user_id=05ada83d82f648eeabca0dce3b8eb3cd, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:02.823730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6984dc-4aff-4858-972f-33cfdad7db22 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:02.894717 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1068/2154] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:02 2022] PUT /identity/v3/projects/cde37d77fb1343f4af1dd8e735a41615/users/05ada83d82f648eeabca0dce3b8eb3cd/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:02.895878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6984dc-4aff-4858-972f-33cfdad7db22 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:02.900374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4f6984dc-4aff-4858-972f-33cfdad7db22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:02.913243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4281302-1d2d-48a2-8f9e-90f9487ae677 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:02.936914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:02.997322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e80bca-f993-407e-bc7f-809754598f46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:02.999495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e80bca-f993-407e-bc7f-809754598f46 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:03.000124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e80bca-f993-407e-bc7f-809754598f46 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:03.000698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e80bca-f993-407e-bc7f-809754598f46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:03.010191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6984dc-4aff-4858-972f-33cfdad7db22 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:03.016704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e80bca-f993-407e-bc7f-809754598f46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:03.016704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e80bca-f993-407e-bc7f-809754598f46 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:03.017950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c4d44-a0fe-41ad-ab54-c5066ff48774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:03.018875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c4d44-a0fe-41ad-ab54-c5066ff48774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:03.019258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c4d44-a0fe-41ad-ab54-c5066ff48774 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:03.019633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c4d44-a0fe-41ad-ab54-c5066ff48774 admin admin] PATH_INFO: `/v3/users/f65e0f0097824cef8210a680b498e37a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:03.021315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c4d44-a0fe-41ad-ab54-c5066ff48774 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f65e0f0097824cef8210a680b498e37a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:03.025151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c4d44-a0fe-41ad-ab54-c5066ff48774 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:03.035148 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1087/2155] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:04:03.038594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257a863-b5c0-4fb3-a598-f1fbdcdf2b25 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:03.039081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257a863-b5c0-4fb3-a598-f1fbdcdf2b25 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:03.039634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257a863-b5c0-4fb3-a598-f1fbdcdf2b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:03.094676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b257a863-b5c0-4fb3-a598-f1fbdcdf2b25 None None] MFA Rules not processed for user `05ada83d82f648eeabca0dce3b8eb3cd`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:03.096659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b257a863-b5c0-4fb3-a598-f1fbdcdf2b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:03.203680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:03.210098 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1088/2156] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:03.216282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257a863-b5c0-4fb3-a598-f1fbdcdf2b25 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:03.229005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257a863-b5c0-4fb3-a598-f1fbdcdf2b25 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:03.235987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900bd079-45e1-4cf2-ab62-2324db7c377b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:03.240576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bd079-45e1-4cf2-ab62-2324db7c377b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:03.240576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bd079-45e1-4cf2-ab62-2324db7c377b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:03.240576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900bd079-45e1-4cf2-ab62-2324db7c377b admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:03.240576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900bd079-45e1-4cf2-ab62-2324db7c377b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:03.240576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900bd079-45e1-4cf2-ab62-2324db7c377b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:03.293863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dc5c4d44-a0fe-41ad-ab54-c5066ff48774 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f65e0f0097824cef8210a680b498e37a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:03.294841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dc5c4d44-a0fe-41ad-ab54-c5066ff48774 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f65e0f0097824cef8210a680b498e37a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:03.303642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dc5c4d44-a0fe-41ad-ab54-c5066ff48774 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f65e0f0097824cef8210a680b498e37a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:03.323943 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1089/2157] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:03 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:03.328547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900bd079-45e1-4cf2-ab62-2324db7c377b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:03.383087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dc5c4d44-a0fe-41ad-ab54-c5066ff48774 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f65e0f0097824cef8210a680b498e37a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:03.383755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dc5c4d44-a0fe-41ad-ab54-c5066ff48774 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f65e0f0097824cef8210a680b498e37a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:03.406645 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1069/2158] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:02 2022] DELETE /identity/v3/users/f65e0f0097824cef8210a680b498e37a => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:03.411110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f689b62-613f-43ee-8e68-5ee9522b045e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:03.411398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f689b62-613f-43ee-8e68-5ee9522b045e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:03.411677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f689b62-613f-43ee-8e68-5ee9522b045e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:03.413461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900bd079-45e1-4cf2-ab62-2324db7c377b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:03.429691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0f689b62-613f-43ee-8e68-5ee9522b045e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:03.431218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0f689b62-613f-43ee-8e68-5ee9522b045e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:03.486757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb47b8-7a70-4fa5-ac6a-91f2d6aa9b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:03.487545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb47b8-7a70-4fa5-ac6a-91f2d6aa9b96 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:03.487965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb47b8-7a70-4fa5-ac6a-91f2d6aa9b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:03.488233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cb47b8-7a70-4fa5-ac6a-91f2d6aa9b96 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:03.489097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb47b8-7a70-4fa5-ac6a-91f2d6aa9b96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:03.492829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cb47b8-7a70-4fa5-ac6a-91f2d6aa9b96 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:03.516127 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1070/2159] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:04:03.517197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f689b62-613f-43ee-8e68-5ee9522b045e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:03.519048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0f689b62-613f-43ee-8e68-5ee9522b045e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:03.532297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f689b62-613f-43ee-8e68-5ee9522b045e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:03.537171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c157b-8056-4034-a14e-649650c310bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:03.538218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c157b-8056-4034-a14e-649650c310bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:03.538579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c157b-8056-4034-a14e-649650c310bc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:03.538927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c157b-8056-4034-a14e-649650c310bc admin admin] PATH_INFO: `/v3/projects/8c0200afe2ad4f6db3a7a21cf37f65c8` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:03.540315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c157b-8056-4034-a14e-649650c310bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c0200afe2ad4f6db3a7a21cf37f65c8)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:03.542515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80c157b-8056-4034-a14e-649650c310bc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:03.577116 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1090/2160] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:03 2022] POST /identity/v3/users => generated 295 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:03.578481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb47b8-7a70-4fa5-ac6a-91f2d6aa9b96 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:03.580945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f4cb47b8-7a70-4fa5-ac6a-91f2d6aa9b96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:03.640986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:03.641531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cb47b8-7a70-4fa5-ac6a-91f2d6aa9b96 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:03.654115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c80c157b-8056-4034-a14e-649650c310bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c0200afe2ad4f6db3a7a21cf37f65c8', 'request_id': 'req-c80c157b-8056-4034-a14e-649650c310bc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:03.659744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c80c157b-8056-4034-a14e-649650c310bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c0200afe2ad4f6db3a7a21cf37f65c8', 'request_id': 'req-c80c157b-8056-4034-a14e-649650c310bc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:03.670884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:03.698027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b8fff-c015-4a73-bfba-f12cc7564c5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:03.698765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b8fff-c015-4a73-bfba-f12cc7564c5c service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:03.699531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b8fff-c015-4a73-bfba-f12cc7564c5c service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:03.699531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b8fff-c015-4a73-bfba-f12cc7564c5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:03.747296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c80c157b-8056-4034-a14e-649650c310bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c0200afe2ad4f6db3a7a21cf37f65c8', 'request_id': 'req-c80c157b-8056-4034-a14e-649650c310bc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:03.764835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c80c157b-8056-4034-a14e-649650c310bc admin admin] The token cache is being invalidate because project 8c0200afe2ad4f6db3a7a21cf37f65c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:03.765366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c80c157b-8056-4034-a14e-649650c310bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:03.766405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c80c157b-8056-4034-a14e-649650c310bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:03.768531 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1071/2161] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:03 2022] DELETE /identity/v3/projects/8c0200afe2ad4f6db3a7a21cf37f65c8 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:03.772893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c157b-8056-4034-a14e-649650c310bc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:03.775273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c80c157b-8056-4034-a14e-649650c310bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:03.826831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379b8fff-c015-4a73-bfba-f12cc7564c5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:03.829078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379b8fff-c015-4a73-bfba-f12cc7564c5c service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:03.843712 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1091/2162] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:03.844896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b8fff-c015-4a73-bfba-f12cc7564c5c service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:03.866720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b8fff-c015-4a73-bfba-f12cc7564c5c service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:03.879465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c157b-8056-4034-a14e-649650c310bc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:03.900481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714bf933-6f6a-4669-9a64-ece76769db87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:03.901731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bf933-6f6a-4669-9a64-ece76769db87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:03.902189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bf933-6f6a-4669-9a64-ece76769db87 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:03.903224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714bf933-6f6a-4669-9a64-ece76769db87 admin admin] PATH_INFO: `/v3/users/d1b067ff75914828a49edd61f6a9318b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:03.909424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714bf933-6f6a-4669-9a64-ece76769db87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1b067ff75914828a49edd61f6a9318b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:03.917456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714bf933-6f6a-4669-9a64-ece76769db87 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:03.922185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc890b6-3683-46ed-91a1-1f60a3b9bb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:03.922882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc890b6-3683-46ed-91a1-1f60a3b9bb35 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:03.923167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc890b6-3683-46ed-91a1-1f60a3b9bb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:03.923804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc890b6-3683-46ed-91a1-1f60a3b9bb35 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:03.925040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc890b6-3683-46ed-91a1-1f60a3b9bb35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:03.927331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc890b6-3683-46ed-91a1-1f60a3b9bb35 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:03.949928 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1072/2163] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:03 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:03.951352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc890b6-3683-46ed-91a1-1f60a3b9bb35 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:03.963020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc890b6-3683-46ed-91a1-1f60a3b9bb35 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:03.973505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc75975-c06e-4816-ba73-fd40082b30f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:03.981095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc75975-c06e-4816-ba73-fd40082b30f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:03.981095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc75975-c06e-4816-ba73-fd40082b30f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:03.981095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc75975-c06e-4816-ba73-fd40082b30f3 admin admin] PATH_INFO: `/v3/projects/1b93fc976e7c477494f36f429f136248/users/7c2a8a5e024e432084eab8c5eacb3637/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:04.013881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc75975-c06e-4816-ba73-fd40082b30f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b93fc976e7c477494f36f429f136248, user_id=7c2a8a5e024e432084eab8c5eacb3637, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:04.017667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc75975-c06e-4816-ba73-fd40082b30f3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:04.109871 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1073/2164] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:03 2022] PUT /identity/v3/projects/1b93fc976e7c477494f36f429f136248/users/7c2a8a5e024e432084eab8c5eacb3637/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:04.116356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc75975-c06e-4816-ba73-fd40082b30f3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:04.203334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc75975-c06e-4816-ba73-fd40082b30f3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:04.211923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c04edf-26a5-431c-89fd-141e4db3f66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:04.213299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c04edf-26a5-431c-89fd-141e4db3f66a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:04.213796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c04edf-26a5-431c-89fd-141e4db3f66a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:04.214519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c04edf-26a5-431c-89fd-141e4db3f66a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:04.215855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c04edf-26a5-431c-89fd-141e4db3f66a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:04.218600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c04edf-26a5-431c-89fd-141e4db3f66a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:04.233392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-714bf933-6f6a-4669-9a64-ece76769db87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1b067ff75914828a49edd61f6a9318b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:04.234787 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1074/2165] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:04:04.239009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c04edf-26a5-431c-89fd-141e4db3f66a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:04.251775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c04edf-26a5-431c-89fd-141e4db3f66a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:04.256962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-714bf933-6f6a-4669-9a64-ece76769db87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1b067ff75914828a49edd61f6a9318b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:04.257706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-714bf933-6f6a-4669-9a64-ece76769db87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1b067ff75914828a49edd61f6a9318b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:04.259687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-714bf933-6f6a-4669-9a64-ece76769db87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1b067ff75914828a49edd61f6a9318b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:04.259804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fba8476-9339-4751-be8f-2b66c1f8becc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:04.260226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba8476-9339-4751-be8f-2b66c1f8becc admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:04.260659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba8476-9339-4751-be8f-2b66c1f8becc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:04.261083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fba8476-9339-4751-be8f-2b66c1f8becc admin admin] PATH_INFO: `/v3/projects/1b93fc976e7c477494f36f429f136248/users/7c2a8a5e024e432084eab8c5eacb3637/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:04.263525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba8476-9339-4751-be8f-2b66c1f8becc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b93fc976e7c477494f36f429f136248, user_id=7c2a8a5e024e432084eab8c5eacb3637, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:04.267341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fba8476-9339-4751-be8f-2b66c1f8becc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:04.297822 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1075/2166] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:04 2022] PUT /identity/v3/projects/1b93fc976e7c477494f36f429f136248/users/7c2a8a5e024e432084eab8c5eacb3637/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:04.309127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb355fff-773c-422f-8c12-6fef57190033 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:04.309742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb355fff-773c-422f-8c12-6fef57190033 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:04.315020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb355fff-773c-422f-8c12-6fef57190033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:04.342986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bb355fff-773c-422f-8c12-6fef57190033 None None] MFA Rules not processed for user `7c2a8a5e024e432084eab8c5eacb3637`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:04.344360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bb355fff-773c-422f-8c12-6fef57190033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:04.352515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:04.366671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-714bf933-6f6a-4669-9a64-ece76769db87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1b067ff75914828a49edd61f6a9318b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:04.433147 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1092/2167] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:03 2022] DELETE /identity/v3/users/d1b067ff75914828a49edd61f6a9318b => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:04.464367 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1076/2168] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:04 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:04.471389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aabffb-c466-4174-86bf-ccf86b8605d1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:04.471685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aabffb-c466-4174-86bf-ccf86b8605d1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:04.471968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aabffb-c466-4174-86bf-ccf86b8605d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:04.491726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-35aabffb-c466-4174-86bf-ccf86b8605d1 None None] MFA Rules not processed for user `05ada83d82f648eeabca0dce3b8eb3cd`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:04.492966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-35aabffb-c466-4174-86bf-ccf86b8605d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:04.585883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:04.602491 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1093/2169] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:04.616976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb355fff-773c-422f-8c12-6fef57190033 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:04.618628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bb355fff-773c-422f-8c12-6fef57190033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:04.736900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb355fff-773c-422f-8c12-6fef57190033 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:04.884848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81dcefa-af06-47e2-b42c-31955b427c21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:04.885629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81dcefa-af06-47e2-b42c-31955b427c21 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:04.886022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81dcefa-af06-47e2-b42c-31955b427c21 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:04.886375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81dcefa-af06-47e2-b42c-31955b427c21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:04.888233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e81dcefa-af06-47e2-b42c-31955b427c21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:04.903200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81dcefa-af06-47e2-b42c-31955b427c21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:04.904833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81dcefa-af06-47e2-b42c-31955b427c21 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:04.914001 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1077/2170] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:04.955714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aabffb-c466-4174-86bf-ccf86b8605d1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:04.957232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-35aabffb-c466-4174-86bf-ccf86b8605d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:04.967533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81dcefa-af06-47e2-b42c-31955b427c21 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:04.969621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e81dcefa-af06-47e2-b42c-31955b427c21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:05.023941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:05.090941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81dcefa-af06-47e2-b42c-31955b427c21 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:05.096480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f224b4-5f12-4c40-9bc7-615dd08b508a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:05.097543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f224b4-5f12-4c40-9bc7-615dd08b508a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:05.098180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f224b4-5f12-4c40-9bc7-615dd08b508a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:05.098615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f224b4-5f12-4c40-9bc7-615dd08b508a admin admin] PATH_INFO: `/v3/projects/14b229b388b845a7ae717c6d3238bdf1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:05.100533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f224b4-5f12-4c40-9bc7-615dd08b508a admin admin] RBAC: Authorizing `identity:delete_project(project_id=14b229b388b845a7ae717c6d3238bdf1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:05.102796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f224b4-5f12-4c40-9bc7-615dd08b508a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:05.119285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aabffb-c466-4174-86bf-ccf86b8605d1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:05.126332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71fc931-d77a-431d-8c5d-994adf83c69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:05.127387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fc931-d77a-431d-8c5d-994adf83c69e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:05.127882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fc931-d77a-431d-8c5d-994adf83c69e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:05.128220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71fc931-d77a-431d-8c5d-994adf83c69e admin admin] PATH_INFO: `/v3/projects/6277287bf774466582bc214b11186b70` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:05.130623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71fc931-d77a-431d-8c5d-994adf83c69e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6277287bf774466582bc214b11186b70)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:05.144485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71fc931-d77a-431d-8c5d-994adf83c69e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:05.218901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-50f224b4-5f12-4c40-9bc7-615dd08b508a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14b229b388b845a7ae717c6d3238bdf1', 'request_id': 'req-50f224b4-5f12-4c40-9bc7-615dd08b508a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.226413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-50f224b4-5f12-4c40-9bc7-615dd08b508a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14b229b388b845a7ae717c6d3238bdf1', 'request_id': 'req-50f224b4-5f12-4c40-9bc7-615dd08b508a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.286918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a71fc931-d77a-431d-8c5d-994adf83c69e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6277287bf774466582bc214b11186b70', 'request_id': 'req-a71fc931-d77a-431d-8c5d-994adf83c69e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.301442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a71fc931-d77a-431d-8c5d-994adf83c69e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6277287bf774466582bc214b11186b70', 'request_id': 'req-a71fc931-d77a-431d-8c5d-994adf83c69e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.331322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-50f224b4-5f12-4c40-9bc7-615dd08b508a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14b229b388b845a7ae717c6d3238bdf1', 'request_id': 'req-50f224b4-5f12-4c40-9bc7-615dd08b508a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.346998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-50f224b4-5f12-4c40-9bc7-615dd08b508a admin admin] The token cache is being invalidate because project 14b229b388b845a7ae717c6d3238bdf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:05.347330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-50f224b4-5f12-4c40-9bc7-615dd08b508a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.347915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-50f224b4-5f12-4c40-9bc7-615dd08b508a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.349527 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1078/2171] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:04 2022] DELETE /identity/v3/projects/14b229b388b845a7ae717c6d3238bdf1 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:05.353267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f224b4-5f12-4c40-9bc7-615dd08b508a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:05.354672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-50f224b4-5f12-4c40-9bc7-615dd08b508a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:05.421579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a71fc931-d77a-431d-8c5d-994adf83c69e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6277287bf774466582bc214b11186b70', 'request_id': 'req-a71fc931-d77a-431d-8c5d-994adf83c69e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.425341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f224b4-5f12-4c40-9bc7-615dd08b508a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:05.435154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a71fc931-d77a-431d-8c5d-994adf83c69e admin admin] The token cache is being invalidate because project 6277287bf774466582bc214b11186b70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:05.435502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a71fc931-d77a-431d-8c5d-994adf83c69e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.436356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a71fc931-d77a-431d-8c5d-994adf83c69e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.438950 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1094/2172] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:04 2022] DELETE /identity/v3/projects/6277287bf774466582bc214b11186b70 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:05.444288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71fc931-d77a-431d-8c5d-994adf83c69e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:05.445726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a71fc931-d77a-431d-8c5d-994adf83c69e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:05.493297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9160c1d-b416-4a9f-b390-f3b32e7e133c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:05.494495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9160c1d-b416-4a9f-b390-f3b32e7e133c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:05.495275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9160c1d-b416-4a9f-b390-f3b32e7e133c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:05.495555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9160c1d-b416-4a9f-b390-f3b32e7e133c admin admin] PATH_INFO: `/v3/users/ad6f51b7b6834fefa06c58829399de85` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:05.497493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9160c1d-b416-4a9f-b390-f3b32e7e133c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad6f51b7b6834fefa06c58829399de85)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:05.499023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9160c1d-b416-4a9f-b390-f3b32e7e133c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:05.501560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:05.520476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71fc931-d77a-431d-8c5d-994adf83c69e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:05.531626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb61d72-504a-4212-b26d-087d91ebea9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:05.532557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb61d72-504a-4212-b26d-087d91ebea9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:05.532836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb61d72-504a-4212-b26d-087d91ebea9c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:05.533107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb61d72-504a-4212-b26d-087d91ebea9c admin admin] PATH_INFO: `/v3/users/52e93378697f424c855e5db84efadce8` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:05.534417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb61d72-504a-4212-b26d-087d91ebea9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=52e93378697f424c855e5db84efadce8)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:05.536421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb61d72-504a-4212-b26d-087d91ebea9c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:05.647490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:05.681002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c9160c1d-b416-4a9f-b390-f3b32e7e133c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad6f51b7b6834fefa06c58829399de85'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.681708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c9160c1d-b416-4a9f-b390-f3b32e7e133c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad6f51b7b6834fefa06c58829399de85'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.692586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c9160c1d-b416-4a9f-b390-f3b32e7e133c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad6f51b7b6834fefa06c58829399de85'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.724649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-efb61d72-504a-4212-b26d-087d91ebea9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52e93378697f424c855e5db84efadce8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.735367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-efb61d72-504a-4212-b26d-087d91ebea9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52e93378697f424c855e5db84efadce8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.736166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-efb61d72-504a-4212-b26d-087d91ebea9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52e93378697f424c855e5db84efadce8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.736803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-efb61d72-504a-4212-b26d-087d91ebea9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52e93378697f424c855e5db84efadce8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.775033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c9160c1d-b416-4a9f-b390-f3b32e7e133c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad6f51b7b6834fefa06c58829399de85'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.775804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c9160c1d-b416-4a9f-b390-f3b32e7e133c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad6f51b7b6834fefa06c58829399de85'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.808697 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1079/2173] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:05 2022] DELETE /identity/v3/users/ad6f51b7b6834fefa06c58829399de85 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:05.809968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e74ec7-d0de-41d0-b171-7220b844876c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:05.810225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e74ec7-d0de-41d0-b171-7220b844876c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:05.810695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e74ec7-d0de-41d0-b171-7220b844876c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:05.829585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-86e74ec7-d0de-41d0-b171-7220b844876c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:05.830854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-86e74ec7-d0de-41d0-b171-7220b844876c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:05.897176 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1080/2174] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:05.898836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50979ff5-64e0-4cee-88ac-1e4679c54555 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:05.899151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50979ff5-64e0-4cee-88ac-1e4679c54555 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:05.899500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50979ff5-64e0-4cee-88ac-1e4679c54555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:05.919127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-50979ff5-64e0-4cee-88ac-1e4679c54555 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:05.920618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-50979ff5-64e0-4cee-88ac-1e4679c54555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:05.935447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-efb61d72-504a-4212-b26d-087d91ebea9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52e93378697f424c855e5db84efadce8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:05.939635 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1081/2175] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:05.947715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c5db1-e940-4689-9321-619e8ad5dcc0 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:05.947715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c5db1-e940-4689-9321-619e8ad5dcc0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:05.947715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c5db1-e940-4689-9321-619e8ad5dcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:05.967892 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1095/2176] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:05 2022] DELETE /identity/v3/users/52e93378697f424c855e5db84efadce8 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:05.974248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb61d72-504a-4212-b26d-087d91ebea9c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:05.974248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-efb61d72-504a-4212-b26d-087d91ebea9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:05.979233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e62c5db1-e940-4689-9321-619e8ad5dcc0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:05.984835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e62c5db1-e940-4689-9321-619e8ad5dcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:06.088111 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1082/2177] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:06.092477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c5db1-e940-4689-9321-619e8ad5dcc0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:06.107238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c5db1-e940-4689-9321-619e8ad5dcc0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:06.111143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb61d72-504a-4212-b26d-087d91ebea9c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:06.113087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02315b00-6131-444e-ae34-a5282c6a4406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:06.114045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02315b00-6131-444e-ae34-a5282c6a4406 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:06.116278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02315b00-6131-444e-ae34-a5282c6a4406 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:06.116719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02315b00-6131-444e-ae34-a5282c6a4406 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:06.120677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02315b00-6131-444e-ae34-a5282c6a4406 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:06.121099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02315b00-6131-444e-ae34-a5282c6a4406 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:06.123218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02315b00-6131-444e-ae34-a5282c6a4406 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:06.123781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f2282-d7b6-4cbd-bf04-0588a10d6d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:06.124452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f2282-d7b6-4cbd-bf04-0588a10d6d17 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:06.124740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f2282-d7b6-4cbd-bf04-0588a10d6d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:06.125014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f2282-d7b6-4cbd-bf04-0588a10d6d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:06.159060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f2282-d7b6-4cbd-bf04-0588a10d6d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:06.161520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f2282-d7b6-4cbd-bf04-0588a10d6d17 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:06.172722 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1083/2178] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:04:06.185303 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1096/2179] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:06.186807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02315b00-6131-444e-ae34-a5282c6a4406 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:06.210888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02315b00-6131-444e-ae34-a5282c6a4406 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:06.226734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db172fe1-f135-4662-a4ee-f5fdb9dbdd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:06.227542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db172fe1-f135-4662-a4ee-f5fdb9dbdd63 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:06.227888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db172fe1-f135-4662-a4ee-f5fdb9dbdd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:06.228228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db172fe1-f135-4662-a4ee-f5fdb9dbdd63 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:06.229001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db172fe1-f135-4662-a4ee-f5fdb9dbdd63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:06.242024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db172fe1-f135-4662-a4ee-f5fdb9dbdd63 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:06.366233 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1084/2180] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:06 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:06.367594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f2282-d7b6-4cbd-bf04-0588a10d6d17 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:06.382579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f2282-d7b6-4cbd-bf04-0588a10d6d17 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:06.464851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718e59f-3997-4040-b4d4-bb1d41007fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:06.465578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718e59f-3997-4040-b4d4-bb1d41007fa7 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:06.466182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718e59f-3997-4040-b4d4-bb1d41007fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:06.466465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718e59f-3997-4040-b4d4-bb1d41007fa7 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:06.467362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718e59f-3997-4040-b4d4-bb1d41007fa7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:06.469509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e718e59f-3997-4040-b4d4-bb1d41007fa7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:06.593975 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1097/2181] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:06 2022] POST /identity/v3/users => generated 294 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:04:06.595189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718e59f-3997-4040-b4d4-bb1d41007fa7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:06.595928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db172fe1-f135-4662-a4ee-f5fdb9dbdd63 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:06.598428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e718e59f-3997-4040-b4d4-bb1d41007fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:06.611628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db172fe1-f135-4662-a4ee-f5fdb9dbdd63 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:06.615195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e718e59f-3997-4040-b4d4-bb1d41007fa7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:06.619678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646f53b-bf0d-45ee-868c-c3c083e196cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:06.620616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646f53b-bf0d-45ee-868c-c3c083e196cd admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:06.620905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646f53b-bf0d-45ee-868c-c3c083e196cd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:06.621183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f646f53b-bf0d-45ee-868c-c3c083e196cd admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:06.622514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcdf63-1529-48e5-8a4b-b4613bea03b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:06.622655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f646f53b-bf0d-45ee-868c-c3c083e196cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:06.627329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcdf63-1529-48e5-8a4b-b4613bea03b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:06.627329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcdf63-1529-48e5-8a4b-b4613bea03b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:06.627329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcdf63-1529-48e5-8a4b-b4613bea03b9 admin admin] PATH_INFO: `/v3/projects/0397fe0f76e24a83aa24ecec35dfb61e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:06.627736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f646f53b-bf0d-45ee-868c-c3c083e196cd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:06.627826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcdf63-1529-48e5-8a4b-b4613bea03b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0397fe0f76e24a83aa24ecec35dfb61e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:06.629102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcdf63-1529-48e5-8a4b-b4613bea03b9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:06.645652 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1085/2182] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:06.648750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646f53b-bf0d-45ee-868c-c3c083e196cd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:06.680174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f646f53b-bf0d-45ee-868c-c3c083e196cd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:06.689881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9903860-5cd7-49ae-8f4e-caa8045248ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:06.690837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9903860-5cd7-49ae-8f4e-caa8045248ff admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:06.691238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9903860-5cd7-49ae-8f4e-caa8045248ff admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:06.691616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9903860-5cd7-49ae-8f4e-caa8045248ff admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:06.692452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9903860-5cd7-49ae-8f4e-caa8045248ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:06.694808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9903860-5cd7-49ae-8f4e-caa8045248ff admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:06.707241 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1086/2183] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:06.710205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9903860-5cd7-49ae-8f4e-caa8045248ff admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:06.725229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9903860-5cd7-49ae-8f4e-caa8045248ff admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:06.730576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2ae82-bee0-44d9-b79f-74b091237844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:06.731523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2ae82-bee0-44d9-b79f-74b091237844 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:06.731874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2ae82-bee0-44d9-b79f-74b091237844 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:06.732154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f2ae82-bee0-44d9-b79f-74b091237844 admin admin] PATH_INFO: `/v3/projects/8019c4c6c22a432ab9b44601856967e8/users/299e6477185343bdbc2311528292fab1/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:06.748122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2ae82-bee0-44d9-b79f-74b091237844 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8019c4c6c22a432ab9b44601856967e8, user_id=299e6477185343bdbc2311528292fab1, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:06.755041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f2ae82-bee0-44d9-b79f-74b091237844 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:06.765744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8adcdf63-1529-48e5-8a4b-b4613bea03b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0397fe0f76e24a83aa24ecec35dfb61e', 'request_id': 'req-8adcdf63-1529-48e5-8a4b-b4613bea03b9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:06.771471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8adcdf63-1529-48e5-8a4b-b4613bea03b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0397fe0f76e24a83aa24ecec35dfb61e', 'request_id': 'req-8adcdf63-1529-48e5-8a4b-b4613bea03b9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:06.867865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8adcdf63-1529-48e5-8a4b-b4613bea03b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0397fe0f76e24a83aa24ecec35dfb61e', 'request_id': 'req-8adcdf63-1529-48e5-8a4b-b4613bea03b9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:06.870940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:06.877144 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1087/2184] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:06 2022] PUT /identity/v3/projects/8019c4c6c22a432ab9b44601856967e8/users/299e6477185343bdbc2311528292fab1/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:04:06.878019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2ae82-bee0-44d9-b79f-74b091237844 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:06.880017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-24f2ae82-bee0-44d9-b79f-74b091237844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:06.881810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8adcdf63-1529-48e5-8a4b-b4613bea03b9 admin admin] The token cache is being invalidate because project 0397fe0f76e24a83aa24ecec35dfb61e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:06.882200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8adcdf63-1529-48e5-8a4b-b4613bea03b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:06.882989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8adcdf63-1529-48e5-8a4b-b4613bea03b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:06.884850 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1098/2185] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:06 2022] DELETE /identity/v3/projects/0397fe0f76e24a83aa24ecec35dfb61e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:06.893101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfac0c5-fc7d-470c-81d3-994343bb57e1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:06.893463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfac0c5-fc7d-470c-81d3-994343bb57e1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:06.893822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfac0c5-fc7d-470c-81d3-994343bb57e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:06.932411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1bfac0c5-fc7d-470c-81d3-994343bb57e1 None None] MFA Rules not processed for user `299e6477185343bdbc2311528292fab1`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:06.934059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1bfac0c5-fc7d-470c-81d3-994343bb57e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:06.985405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f2ae82-bee0-44d9-b79f-74b091237844 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:06.991088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dc251-922d-49b7-9cb9-f4860907601e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:06.991968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dc251-922d-49b7-9cb9-f4860907601e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:06.992253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dc251-922d-49b7-9cb9-f4860907601e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:06.992528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dc251-922d-49b7-9cb9-f4860907601e admin admin] PATH_INFO: `/v3/projects/e603e36673a74614932c52f9a007e4eb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:06.994105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1dc251-922d-49b7-9cb9-f4860907601e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e603e36673a74614932c52f9a007e4eb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:06.996561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1dc251-922d-49b7-9cb9-f4860907601e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:07.015659 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1099/2186] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:07.023778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfac0c5-fc7d-470c-81d3-994343bb57e1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:07.023778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1bfac0c5-fc7d-470c-81d3-994343bb57e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:07.036121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfac0c5-fc7d-470c-81d3-994343bb57e1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:07.041898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea7658-9e5b-4d49-82c9-38d36b1e48b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:07.042787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea7658-9e5b-4d49-82c9-38d36b1e48b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:07.043071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea7658-9e5b-4d49-82c9-38d36b1e48b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:07.043343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea7658-9e5b-4d49-82c9-38d36b1e48b8 admin admin] PATH_INFO: `/v3/users/34c91c3ae5ce4ffe9ee08500d1aff221` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:07.044815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea7658-9e5b-4d49-82c9-38d36b1e48b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34c91c3ae5ce4ffe9ee08500d1aff221)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:07.047304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea7658-9e5b-4d49-82c9-38d36b1e48b8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:07.091652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:07.099293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5d1dc251-922d-49b7-9cb9-f4860907601e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e603e36673a74614932c52f9a007e4eb', 'request_id': 'req-5d1dc251-922d-49b7-9cb9-f4860907601e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:07.105961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5d1dc251-922d-49b7-9cb9-f4860907601e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e603e36673a74614932c52f9a007e4eb', 'request_id': 'req-5d1dc251-922d-49b7-9cb9-f4860907601e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:07.182279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5d1dc251-922d-49b7-9cb9-f4860907601e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e603e36673a74614932c52f9a007e4eb', 'request_id': 'req-5d1dc251-922d-49b7-9cb9-f4860907601e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:07.194928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5d1dc251-922d-49b7-9cb9-f4860907601e admin admin] The token cache is being invalidate because project e603e36673a74614932c52f9a007e4eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:07.195222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5d1dc251-922d-49b7-9cb9-f4860907601e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:07.195914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5d1dc251-922d-49b7-9cb9-f4860907601e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:07.197751 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1088/2187] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:06 2022] DELETE /identity/v3/projects/e603e36673a74614932c52f9a007e4eb => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:07.201976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dc251-922d-49b7-9cb9-f4860907601e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:07.203351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5d1dc251-922d-49b7-9cb9-f4860907601e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:07.265071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-05ea7658-9e5b-4d49-82c9-38d36b1e48b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34c91c3ae5ce4ffe9ee08500d1aff221'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:07.272100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dc251-922d-49b7-9cb9-f4860907601e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:07.281121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-05ea7658-9e5b-4d49-82c9-38d36b1e48b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34c91c3ae5ce4ffe9ee08500d1aff221'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:07.282322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-05ea7658-9e5b-4d49-82c9-38d36b1e48b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34c91c3ae5ce4ffe9ee08500d1aff221'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:07.282442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7310a2e-745a-4a0b-a933-964248ec91bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:07.283103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7310a2e-745a-4a0b-a933-964248ec91bb admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:07.283236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-05ea7658-9e5b-4d49-82c9-38d36b1e48b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34c91c3ae5ce4ffe9ee08500d1aff221'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:07.283393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7310a2e-745a-4a0b-a933-964248ec91bb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:07.283974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7310a2e-745a-4a0b-a933-964248ec91bb admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:07.284741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7310a2e-745a-4a0b-a933-964248ec91bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:07.286664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7310a2e-745a-4a0b-a933-964248ec91bb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:07.348420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-05ea7658-9e5b-4d49-82c9-38d36b1e48b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34c91c3ae5ce4ffe9ee08500d1aff221'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:07.368108 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1100/2188] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:07 2022] DELETE /identity/v3/users/34c91c3ae5ce4ffe9ee08500d1aff221 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:07.369575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e338e-3547-4bd6-8c18-d68068e2d58b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:07.370152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e338e-3547-4bd6-8c18-d68068e2d58b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:07.370557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e338e-3547-4bd6-8c18-d68068e2d58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:07.390933 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1089/2189] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:07 2022] POST /identity/v3/projects => generated 383 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:07.391425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-581e338e-3547-4bd6-8c18-d68068e2d58b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:07.391790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-581e338e-3547-4bd6-8c18-d68068e2d58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:07.397924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7310a2e-745a-4a0b-a933-964248ec91bb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:07.399702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d7310a2e-745a-4a0b-a933-964248ec91bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:07.464057 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1101/2190] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:07.473158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3fbc4-579c-40f0-81f8-e654a613d403 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:07.474317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3fbc4-579c-40f0-81f8-e654a613d403 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:07.475405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3fbc4-579c-40f0-81f8-e654a613d403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:07.479839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7310a2e-745a-4a0b-a933-964248ec91bb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:07.497219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521996e-4c49-4cab-93d0-47a5b2eecbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:07.498158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521996e-4c49-4cab-93d0-47a5b2eecbcb admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:07.498290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521996e-4c49-4cab-93d0-47a5b2eecbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:07.498618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521996e-4c49-4cab-93d0-47a5b2eecbcb admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:07.499353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521996e-4c49-4cab-93d0-47a5b2eecbcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:07.501713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521996e-4c49-4cab-93d0-47a5b2eecbcb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:07.510639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d1b3fbc4-579c-40f0-81f8-e654a613d403 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:07.511881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d1b3fbc4-579c-40f0-81f8-e654a613d403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:07.520462 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1102/2191] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:07.525024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3fbc4-579c-40f0-81f8-e654a613d403 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:07.534410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3fbc4-579c-40f0-81f8-e654a613d403 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:07.539826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ea992-3bab-408f-976d-7de965818259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:07.540581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ea992-3bab-408f-976d-7de965818259 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:07.541067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ea992-3bab-408f-976d-7de965818259 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:07.541582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ea992-3bab-408f-976d-7de965818259 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:07.542729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ea992-3bab-408f-976d-7de965818259 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:07.543103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ea992-3bab-408f-976d-7de965818259 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:07.544800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ea992-3bab-408f-976d-7de965818259 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:07.556365 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1103/2192] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:07.560662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ea992-3bab-408f-976d-7de965818259 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:07.568531 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1090/2193] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:07 2022] POST /identity/v3/users => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:04:07.569997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521996e-4c49-4cab-93d0-47a5b2eecbcb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:07.571259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ea992-3bab-408f-976d-7de965818259 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:07.576876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c1d19-d1bd-4507-899a-ed2882d0cef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:07.577546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c1d19-d1bd-4507-899a-ed2882d0cef8 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:07.577927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c1d19-d1bd-4507-899a-ed2882d0cef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:07.578205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652c1d19-d1bd-4507-899a-ed2882d0cef8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:07.578910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c1d19-d1bd-4507-899a-ed2882d0cef8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:07.580572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521996e-4c49-4cab-93d0-47a5b2eecbcb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:07.580872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652c1d19-d1bd-4507-899a-ed2882d0cef8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:07.587385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb58424-ee86-4157-954c-30fab76faab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:07.588159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb58424-ee86-4157-954c-30fab76faab0 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:07.588525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb58424-ee86-4157-954c-30fab76faab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:07.589040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb58424-ee86-4157-954c-30fab76faab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:07.589956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb58424-ee86-4157-954c-30fab76faab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:07.591725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb58424-ee86-4157-954c-30fab76faab0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:07.612150 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1091/2194] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:07.613905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb58424-ee86-4157-954c-30fab76faab0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:07.625673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb58424-ee86-4157-954c-30fab76faab0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:07.631498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207668a-3fd0-4d8a-9458-c456495ec216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:07.633380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207668a-3fd0-4d8a-9458-c456495ec216 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:07.633487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207668a-3fd0-4d8a-9458-c456495ec216 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:07.633909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6207668a-3fd0-4d8a-9458-c456495ec216 admin admin] PATH_INFO: `/v3/projects/b04f3b8ea4f2457aa59850428fea8ba8/users/d76932b7d2024134a2328f39683c72bc/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:07.650231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207668a-3fd0-4d8a-9458-c456495ec216 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04f3b8ea4f2457aa59850428fea8ba8, user_id=d76932b7d2024134a2328f39683c72bc, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:07.656706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6207668a-3fd0-4d8a-9458-c456495ec216 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:07.656829 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1104/2195] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:07 2022] POST /identity/v3/projects => generated 371 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:04:07.667245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c1d19-d1bd-4507-899a-ed2882d0cef8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:07.721033 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1092/2196] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:07 2022] PUT /identity/v3/projects/b04f3b8ea4f2457aa59850428fea8ba8/users/d76932b7d2024134a2328f39683c72bc/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:04:07.726715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207668a-3fd0-4d8a-9458-c456495ec216 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:07.745656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652c1d19-d1bd-4507-899a-ed2882d0cef8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:07.829281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55564718-49db-462b-bb7a-4e8fd7ffcdac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:07.830190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55564718-49db-462b-bb7a-4e8fd7ffcdac admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:07.830480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55564718-49db-462b-bb7a-4e8fd7ffcdac admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:07.830753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55564718-49db-462b-bb7a-4e8fd7ffcdac admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:07.831665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55564718-49db-462b-bb7a-4e8fd7ffcdac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:07.833685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55564718-49db-462b-bb7a-4e8fd7ffcdac admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:07.869144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6207668a-3fd0-4d8a-9458-c456495ec216 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:07.874501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac772aaf-858d-487a-bf68-79988eddb5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:07.875176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac772aaf-858d-487a-bf68-79988eddb5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:07.875815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac772aaf-858d-487a-bf68-79988eddb5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:07.876094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac772aaf-858d-487a-bf68-79988eddb5d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:07.876867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac772aaf-858d-487a-bf68-79988eddb5d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:07.878724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac772aaf-858d-487a-bf68-79988eddb5d8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:07.888170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:07.893731 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1093/2197] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:07.895534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac772aaf-858d-487a-bf68-79988eddb5d8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:07.899553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ac772aaf-858d-487a-bf68-79988eddb5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:07.917564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac772aaf-858d-487a-bf68-79988eddb5d8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:07.931538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a991152-9939-4571-be3d-cdcdbc9cd10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:07.932581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a991152-9939-4571-be3d-cdcdbc9cd10a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:07.933389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a991152-9939-4571-be3d-cdcdbc9cd10a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:07.933759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a991152-9939-4571-be3d-cdcdbc9cd10a admin admin] PATH_INFO: `/v3/projects/ce82bcaad9354b0d86718aeec0d5bf6f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:07.935265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a991152-9939-4571-be3d-cdcdbc9cd10a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce82bcaad9354b0d86718aeec0d5bf6f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:07.937324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a991152-9939-4571-be3d-cdcdbc9cd10a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:07.949935 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1105/2198] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:07 2022] POST /identity/v3/users => generated 286 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:07.950740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55564718-49db-462b-bb7a-4e8fd7ffcdac admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:07.962943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55564718-49db-462b-bb7a-4e8fd7ffcdac admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:07.966655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:07.970621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35722b60-7894-4fa1-a20f-a22845e55910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:07.975375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35722b60-7894-4fa1-a20f-a22845e55910 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:07.975375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35722b60-7894-4fa1-a20f-a22845e55910 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:07.975375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35722b60-7894-4fa1-a20f-a22845e55910 admin admin] PATH_INFO: `/v3/projects/b04f3b8ea4f2457aa59850428fea8ba8/users/d76932b7d2024134a2328f39683c72bc/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:07.975375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35722b60-7894-4fa1-a20f-a22845e55910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04f3b8ea4f2457aa59850428fea8ba8, user_id=d76932b7d2024134a2328f39683c72bc, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:07.977757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35722b60-7894-4fa1-a20f-a22845e55910 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:07.991037 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1106/2199] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:07 2022] PUT /identity/v3/projects/b04f3b8ea4f2457aa59850428fea8ba8/users/d76932b7d2024134a2328f39683c72bc/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:07.992070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35722b60-7894-4fa1-a20f-a22845e55910 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:08.083079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35722b60-7894-4fa1-a20f-a22845e55910 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:08.091467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a991152-9939-4571-be3d-cdcdbc9cd10a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce82bcaad9354b0d86718aeec0d5bf6f', 'request_id': 'req-3a991152-9939-4571-be3d-cdcdbc9cd10a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:08.098993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a991152-9939-4571-be3d-cdcdbc9cd10a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce82bcaad9354b0d86718aeec0d5bf6f', 'request_id': 'req-3a991152-9939-4571-be3d-cdcdbc9cd10a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:08.142413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043f4616-81ac-404b-adc6-e5a38d5ae65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:08.143008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f4616-81ac-404b-adc6-e5a38d5ae65c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:08.143314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f4616-81ac-404b-adc6-e5a38d5ae65c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:08.143576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043f4616-81ac-404b-adc6-e5a38d5ae65c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:08.144456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f4616-81ac-404b-adc6-e5a38d5ae65c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:08.146111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043f4616-81ac-404b-adc6-e5a38d5ae65c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:08.156216 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1107/2200] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:08.158033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e46cfa-b17b-4ad0-8fa8-99b852c0caff None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:08.169567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e46cfa-b17b-4ad0-8fa8-99b852c0caff None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:08.169567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e46cfa-b17b-4ad0-8fa8-99b852c0caff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:08.188398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a991152-9939-4571-be3d-cdcdbc9cd10a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce82bcaad9354b0d86718aeec0d5bf6f', 'request_id': 'req-3a991152-9939-4571-be3d-cdcdbc9cd10a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:08.197889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-43e46cfa-b17b-4ad0-8fa8-99b852c0caff None None] MFA Rules not processed for user `d76932b7d2024134a2328f39683c72bc`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:08.198956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-43e46cfa-b17b-4ad0-8fa8-99b852c0caff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:08.201980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a991152-9939-4571-be3d-cdcdbc9cd10a admin admin] The token cache is being invalidate because project ce82bcaad9354b0d86718aeec0d5bf6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:08.202456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a991152-9939-4571-be3d-cdcdbc9cd10a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:08.203469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a991152-9939-4571-be3d-cdcdbc9cd10a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:08.205784 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1094/2201] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:07 2022] DELETE /identity/v3/projects/ce82bcaad9354b0d86718aeec0d5bf6f => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:08.210512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a991152-9939-4571-be3d-cdcdbc9cd10a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:08.212085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3a991152-9939-4571-be3d-cdcdbc9cd10a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:08.226648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a991152-9939-4571-be3d-cdcdbc9cd10a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:08.231967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a25337-885d-41f0-968c-d4e4f280f1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:08.232802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a25337-885d-41f0-968c-d4e4f280f1ad admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:08.233152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a25337-885d-41f0-968c-d4e4f280f1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:08.233530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a25337-885d-41f0-968c-d4e4f280f1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:08.234676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a25337-885d-41f0-968c-d4e4f280f1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:08.237089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a25337-885d-41f0-968c-d4e4f280f1ad admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:08.251484 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1095/2202] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:08.251484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2e76f-34d4-41dc-a9c6-fe6ad25c1194 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:08.251484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2e76f-34d4-41dc-a9c6-fe6ad25c1194 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:08.251484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c2e76f-34d4-41dc-a9c6-fe6ad25c1194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:08.268747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-56c2e76f-34d4-41dc-a9c6-fe6ad25c1194 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:08.270000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-56c2e76f-34d4-41dc-a9c6-fe6ad25c1194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:08.275627 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1108/2203] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:08.276500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e46cfa-b17b-4ad0-8fa8-99b852c0caff None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:08.279242 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1096/2204] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:08.287266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca441bec-c00a-4f5a-9c5d-4af3a7809fb4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:08.287758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca441bec-c00a-4f5a-9c5d-4af3a7809fb4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:08.288187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca441bec-c00a-4f5a-9c5d-4af3a7809fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:08.293137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e46cfa-b17b-4ad0-8fa8-99b852c0caff None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:08.298194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd4137-cffe-4ba0-9a49-bfc28c5033bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:08.299100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd4137-cffe-4ba0-9a49-bfc28c5033bb admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:08.299388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd4137-cffe-4ba0-9a49-bfc28c5033bb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:08.299680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd4137-cffe-4ba0-9a49-bfc28c5033bb admin admin] PATH_INFO: `/v3/projects/2b0858c8d98743319b44336db2d6c6e5/users/78e6732bee9d4f86a26970a015c31c84/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:08.314240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ca441bec-c00a-4f5a-9c5d-4af3a7809fb4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:08.315562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ca441bec-c00a-4f5a-9c5d-4af3a7809fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:08.319688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd4137-cffe-4ba0-9a49-bfc28c5033bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b0858c8d98743319b44336db2d6c6e5, user_id=78e6732bee9d4f86a26970a015c31c84, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:08.322923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcd4137-cffe-4ba0-9a49-bfc28c5033bb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:08.327889 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1097/2205] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:08.332647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca441bec-c00a-4f5a-9c5d-4af3a7809fb4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:08.334114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ca441bec-c00a-4f5a-9c5d-4af3a7809fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:08.425017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca441bec-c00a-4f5a-9c5d-4af3a7809fb4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:08.430129 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1109/2206] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:08 2022] PUT /identity/v3/projects/2b0858c8d98743319b44336db2d6c6e5/users/78e6732bee9d4f86a26970a015c31c84/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:08.441564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd4137-cffe-4ba0-9a49-bfc28c5033bb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:08.521035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c2596-68d6-4e3a-8d04-a45dfb76c842 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:08.522026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c2596-68d6-4e3a-8d04-a45dfb76c842 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:08.522420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c2596-68d6-4e3a-8d04-a45dfb76c842 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:08.523224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c2596-68d6-4e3a-8d04-a45dfb76c842 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:08.539828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd4137-cffe-4ba0-9a49-bfc28c5033bb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:08.548198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d6cba-37b1-4c5d-a39d-b86d1d035d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:08.548198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d6cba-37b1-4c5d-a39d-b86d1d035d2b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:08.548198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d6cba-37b1-4c5d-a39d-b86d1d035d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:08.548198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d6cba-37b1-4c5d-a39d-b86d1d035d2b admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:08.549028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d6cba-37b1-4c5d-a39d-b86d1d035d2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:08.549829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d6cba-37b1-4c5d-a39d-b86d1d035d2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:08.551501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d6cba-37b1-4c5d-a39d-b86d1d035d2b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:08.568245 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1110/2207] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:08.569853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4229a-f4e1-40e7-b779-1fd522fc3c16 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:08.572783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4229a-f4e1-40e7-b779-1fd522fc3c16 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:08.572783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4229a-f4e1-40e7-b779-1fd522fc3c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:08.602500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-eaf4229a-f4e1-40e7-b779-1fd522fc3c16 None None] MFA Rules not processed for user `78e6732bee9d4f86a26970a015c31c84`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:08.603778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eaf4229a-f4e1-40e7-b779-1fd522fc3c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:08.623611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c2596-68d6-4e3a-8d04-a45dfb76c842 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:08.625315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c2596-68d6-4e3a-8d04-a45dfb76c842 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:08.640937 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1098/2208] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:08.642156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c2596-68d6-4e3a-8d04-a45dfb76c842 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:08.658877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c2596-68d6-4e3a-8d04-a45dfb76c842 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:08.665472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fa1d6-d534-4acb-9956-abe9b8bb3009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:08.668631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fa1d6-d534-4acb-9956-abe9b8bb3009 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:08.668631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fa1d6-d534-4acb-9956-abe9b8bb3009 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:08.668631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9fa1d6-d534-4acb-9956-abe9b8bb3009 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:08.668631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fa1d6-d534-4acb-9956-abe9b8bb3009 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:08.670954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9fa1d6-d534-4acb-9956-abe9b8bb3009 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:08.704294 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1111/2209] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:08.708757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4229a-f4e1-40e7-b779-1fd522fc3c16 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:08.722479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4229a-f4e1-40e7-b779-1fd522fc3c16 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:08.728428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b05e1-5428-44dd-9dcd-796c20b0597a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:08.729092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b05e1-5428-44dd-9dcd-796c20b0597a admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:08.729689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b05e1-5428-44dd-9dcd-796c20b0597a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:08.730035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b05e1-5428-44dd-9dcd-796c20b0597a admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:08.730740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b05e1-5428-44dd-9dcd-796c20b0597a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:08.732704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b05e1-5428-44dd-9dcd-796c20b0597a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:08.760737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:08.768166 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1099/2210] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:08 2022] POST /identity/v3/projects => generated 389 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:08.773592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fa1d6-d534-4acb-9956-abe9b8bb3009 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:08.855946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9fa1d6-d534-4acb-9956-abe9b8bb3009 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:08.861828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd7eea-19c5-4c46-bc65-e6c6a0353159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:08.862909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd7eea-19c5-4c46-bc65-e6c6a0353159 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:08.865348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd7eea-19c5-4c46-bc65-e6c6a0353159 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:08.865348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cd7eea-19c5-4c46-bc65-e6c6a0353159 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:08.865348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd7eea-19c5-4c46-bc65-e6c6a0353159 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:08.868676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cd7eea-19c5-4c46-bc65-e6c6a0353159 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:08.868830 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1112/2211] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:08 2022] POST /identity/v3/projects => generated 373 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:08.869645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b05e1-5428-44dd-9dcd-796c20b0597a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:08.939637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b05e1-5428-44dd-9dcd-796c20b0597a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:08.944436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4d118-db82-4e32-8b48-77a8dc5d1f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:08.945170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4d118-db82-4e32-8b48-77a8dc5d1f90 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:08.945468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4d118-db82-4e32-8b48-77a8dc5d1f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:08.945753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4d118-db82-4e32-8b48-77a8dc5d1f90 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:08.946534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4d118-db82-4e32-8b48-77a8dc5d1f90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:08.948490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4d118-db82-4e32-8b48-77a8dc5d1f90 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:08.985775 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1100/2212] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:08 2022] POST /identity/v3/users => generated 295 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:04:08.985775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd7eea-19c5-4c46-bc65-e6c6a0353159 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:08.987329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a9cd7eea-19c5-4c46-bc65-e6c6a0353159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:09.054857 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1113/2213] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:08 2022] POST /identity/v3/users => generated 293 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:09.055883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4d118-db82-4e32-8b48-77a8dc5d1f90 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:09.082952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4d118-db82-4e32-8b48-77a8dc5d1f90 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:09.092127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d31d7-df8a-49cd-b7f9-7e973eabb2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:09.093003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d31d7-df8a-49cd-b7f9-7e973eabb2b2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:09.093291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d31d7-df8a-49cd-b7f9-7e973eabb2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:09.093624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d31d7-df8a-49cd-b7f9-7e973eabb2b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:09.094614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d31d7-df8a-49cd-b7f9-7e973eabb2b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:09.096709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d31d7-df8a-49cd-b7f9-7e973eabb2b2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:09.108310 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1114/2214] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:09.109179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d31d7-df8a-49cd-b7f9-7e973eabb2b2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:09.115109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cd7eea-19c5-4c46-bc65-e6c6a0353159 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:09.121656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d31d7-df8a-49cd-b7f9-7e973eabb2b2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:09.127336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec0a56-7abf-40e3-98d2-f6af900b858d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:09.128015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec0a56-7abf-40e3-98d2-f6af900b858d admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:09.128301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec0a56-7abf-40e3-98d2-f6af900b858d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:09.128632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec0a56-7abf-40e3-98d2-f6af900b858d admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:09.129392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec0a56-7abf-40e3-98d2-f6af900b858d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:09.132238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec0a56-7abf-40e3-98d2-f6af900b858d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:09.135406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:09.149728 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1115/2215] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:09.152865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec0a56-7abf-40e3-98d2-f6af900b858d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:09.164503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec0a56-7abf-40e3-98d2-f6af900b858d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:09.170017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e294b58-9313-4761-bcac-fa467b7e8edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:09.170951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e294b58-9313-4761-bcac-fa467b7e8edf admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:09.171439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e294b58-9313-4761-bcac-fa467b7e8edf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:09.171818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e294b58-9313-4761-bcac-fa467b7e8edf admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:09.172678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e294b58-9313-4761-bcac-fa467b7e8edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:09.174584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e294b58-9313-4761-bcac-fa467b7e8edf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:09.185859 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1116/2216] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:09.187210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e294b58-9313-4761-bcac-fa467b7e8edf admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:09.203314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926891b3-d667-44d2-a797-184ee54703e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:09.204095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926891b3-d667-44d2-a797-184ee54703e1 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:09.211558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926891b3-d667-44d2-a797-184ee54703e1 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:09.211558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926891b3-d667-44d2-a797-184ee54703e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:09.211558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-926891b3-d667-44d2-a797-184ee54703e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:09.211854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e294b58-9313-4761-bcac-fa467b7e8edf admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:09.215760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc0e17-32e9-46e7-bc5f-e135c8e57c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:09.216315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc0e17-32e9-46e7-bc5f-e135c8e57c29 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:09.216608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc0e17-32e9-46e7-bc5f-e135c8e57c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:09.216894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc0e17-32e9-46e7-bc5f-e135c8e57c29 admin admin] PATH_INFO: `/v3/projects/de1f4b87aefe4f1eb4ac3d3f6934ef5e/users/10534fe7f08c44f6a6cd0433e4b91d96/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:09.234236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc0e17-32e9-46e7-bc5f-e135c8e57c29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1f4b87aefe4f1eb4ac3d3f6934ef5e, user_id=10534fe7f08c44f6a6cd0433e4b91d96, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:09.236850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc0e17-32e9-46e7-bc5f-e135c8e57c29 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:09.278470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926891b3-d667-44d2-a797-184ee54703e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:09.280122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926891b3-d667-44d2-a797-184ee54703e1 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:09.289962 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1101/2217] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:09.290763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926891b3-d667-44d2-a797-184ee54703e1 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:09.303506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926891b3-d667-44d2-a797-184ee54703e1 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:09.309375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe3936-ebd7-432f-8a77-063e8dc57dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:09.310368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe3936-ebd7-432f-8a77-063e8dc57dda admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:09.310682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe3936-ebd7-432f-8a77-063e8dc57dda admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:09.310948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe3936-ebd7-432f-8a77-063e8dc57dda admin admin] PATH_INFO: `/v3/projects/e0f50b8715e64beca36d4320ac0ca972/users/db0f47bc701d49f5a68fc6e26ed1629a/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:09.315635 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1117/2218] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:09 2022] PUT /identity/v3/projects/de1f4b87aefe4f1eb4ac3d3f6934ef5e/users/10534fe7f08c44f6a6cd0433e4b91d96/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:09.322206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc0e17-32e9-46e7-bc5f-e135c8e57c29 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:09.348980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe3936-ebd7-432f-8a77-063e8dc57dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f50b8715e64beca36d4320ac0ca972, user_id=db0f47bc701d49f5a68fc6e26ed1629a, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:09.353082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fe3936-ebd7-432f-8a77-063e8dc57dda admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:09.415924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc0e17-32e9-46e7-bc5f-e135c8e57c29 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:09.422014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02be2e7-1f8f-486f-957e-5a54ff5d68a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:09.423011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02be2e7-1f8f-486f-957e-5a54ff5d68a2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:09.428472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02be2e7-1f8f-486f-957e-5a54ff5d68a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:09.428472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02be2e7-1f8f-486f-957e-5a54ff5d68a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:09.428472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02be2e7-1f8f-486f-957e-5a54ff5d68a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:09.428472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02be2e7-1f8f-486f-957e-5a54ff5d68a2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:09.438371 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1118/2219] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:09.444169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02be2e7-1f8f-486f-957e-5a54ff5d68a2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:09.453329 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1102/2220] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:09 2022] PUT /identity/v3/projects/e0f50b8715e64beca36d4320ac0ca972/users/db0f47bc701d49f5a68fc6e26ed1629a/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:09.454742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05536649-560e-4df5-9f5d-47b25224daef None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:09.455040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05536649-560e-4df5-9f5d-47b25224daef None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:09.455318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05536649-560e-4df5-9f5d-47b25224daef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:09.462676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02be2e7-1f8f-486f-957e-5a54ff5d68a2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:09.502806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:09.509107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-05536649-560e-4df5-9f5d-47b25224daef None None] MFA Rules not processed for user `db0f47bc701d49f5a68fc6e26ed1629a`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:09.510456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-05536649-560e-4df5-9f5d-47b25224daef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:09.569904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b9799-0bd4-4a49-b692-a6d1835c008a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:09.571248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b9799-0bd4-4a49-b692-a6d1835c008a admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:09.571826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b9799-0bd4-4a49-b692-a6d1835c008a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:09.572118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b9799-0bd4-4a49-b692-a6d1835c008a admin admin] PATH_INFO: `/v3/projects/de1f4b87aefe4f1eb4ac3d3f6934ef5e/users/10534fe7f08c44f6a6cd0433e4b91d96/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:09.575085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b9799-0bd4-4a49-b692-a6d1835c008a admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1f4b87aefe4f1eb4ac3d3f6934ef5e, user_id=10534fe7f08c44f6a6cd0433e4b91d96, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:09.578294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b9799-0bd4-4a49-b692-a6d1835c008a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:09.599719 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1119/2221] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:09 2022] PUT /identity/v3/projects/de1f4b87aefe4f1eb4ac3d3f6934ef5e/users/10534fe7f08c44f6a6cd0433e4b91d96/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:09.610647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ceb481-f77e-44fc-a690-642d455d91b3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:09.611046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ceb481-f77e-44fc-a690-642d455d91b3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:09.611335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ceb481-f77e-44fc-a690-642d455d91b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:09.624661 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1103/2222] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:04:09.628886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05536649-560e-4df5-9f5d-47b25224daef None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:09.641884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05536649-560e-4df5-9f5d-47b25224daef None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:09.665325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b6ceb481-f77e-44fc-a690-642d455d91b3 None None] MFA Rules not processed for user `10534fe7f08c44f6a6cd0433e4b91d96`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:09.666881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b6ceb481-f77e-44fc-a690-642d455d91b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:09.674229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:09.727928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211ebc4-e805-4dec-9974-b7618aaaa643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:09.729878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211ebc4-e805-4dec-9974-b7618aaaa643 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:09.730189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211ebc4-e805-4dec-9974-b7618aaaa643 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:09.730876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211ebc4-e805-4dec-9974-b7618aaaa643 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:09.731899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211ebc4-e805-4dec-9974-b7618aaaa643 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:09.734864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211ebc4-e805-4dec-9974-b7618aaaa643 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:09.769627 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1120/2223] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:09.914197 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1104/2224] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:09 2022] POST /identity/v3/projects => generated 389 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:04:09.917627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ceb481-f77e-44fc-a690-642d455d91b3 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:09.946162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ceb481-f77e-44fc-a690-642d455d91b3 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:09.950068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211ebc4-e805-4dec-9974-b7618aaaa643 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:10.131081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2ae68-f753-45ac-bd39-b04a3b378a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:10.131847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2ae68-f753-45ac-bd39-b04a3b378a31 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:10.132066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2ae68-f753-45ac-bd39-b04a3b378a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:10.132408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2ae68-f753-45ac-bd39-b04a3b378a31 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:10.135468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2ae68-f753-45ac-bd39-b04a3b378a31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:10.138830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2ae68-f753-45ac-bd39-b04a3b378a31 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:10.148651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211ebc4-e805-4dec-9974-b7618aaaa643 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:10.164638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554974b-542e-4183-ab7a-5b87585e826e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:10.165321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554974b-542e-4183-ab7a-5b87585e826e service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:10.165618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554974b-542e-4183-ab7a-5b87585e826e service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:10.165915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554974b-542e-4183-ab7a-5b87585e826e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:10.264794 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1121/2225] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:09 2022] POST /identity/v3/users => generated 301 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:10.266008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2ae68-f753-45ac-bd39-b04a3b378a31 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:10.277056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:10.291347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554974b-542e-4183-ab7a-5b87585e826e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:10.296970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554974b-542e-4183-ab7a-5b87585e826e service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:10.307214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2ae68-f753-45ac-bd39-b04a3b378a31 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:10.313760 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1105/2226] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:04:10.317883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3f5b3-e53d-43b4-8032-350593a15c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:10.318749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3f5b3-e53d-43b4-8032-350593a15c25 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:10.319040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3f5b3-e53d-43b4-8032-350593a15c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:10.319320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db3f5b3-e53d-43b4-8032-350593a15c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:10.320058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3f5b3-e53d-43b4-8032-350593a15c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:10.321651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db3f5b3-e53d-43b4-8032-350593a15c25 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:10.324902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:10.333057 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1122/2227] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:10.337897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554974b-542e-4183-ab7a-5b87585e826e service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:10.348142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554974b-542e-4183-ab7a-5b87585e826e service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:10.361241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95106a3e-da3f-4339-8fa1-1d19d9f48d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:10.362438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95106a3e-da3f-4339-8fa1-1d19d9f48d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:10.362682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95106a3e-da3f-4339-8fa1-1d19d9f48d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:10.363117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95106a3e-da3f-4339-8fa1-1d19d9f48d6d admin admin] PATH_INFO: `/v3/projects/3555b83003064628a3778162f14c522b/users/a264123dcd6e455b945b54652bf218db/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:10.385228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95106a3e-da3f-4339-8fa1-1d19d9f48d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3555b83003064628a3778162f14c522b, user_id=a264123dcd6e455b945b54652bf218db, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:10.389605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95106a3e-da3f-4339-8fa1-1d19d9f48d6d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:10.413402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3f5b3-e53d-43b4-8032-350593a15c25 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:10.415515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2db3f5b3-e53d-43b4-8032-350593a15c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:10.431758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db3f5b3-e53d-43b4-8032-350593a15c25 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:10.440139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa5c51f-7936-4842-9bff-aae2999b4df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:10.441047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5c51f-7936-4842-9bff-aae2999b4df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:10.441339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5c51f-7936-4842-9bff-aae2999b4df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:10.441624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa5c51f-7936-4842-9bff-aae2999b4df8 admin admin] PATH_INFO: `/v3/users/299e6477185343bdbc2311528292fab1` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:10.443195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5c51f-7936-4842-9bff-aae2999b4df8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=299e6477185343bdbc2311528292fab1)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:10.445592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa5c51f-7936-4842-9bff-aae2999b4df8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:10.499670 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1106/2228] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:10 2022] PUT /identity/v3/projects/3555b83003064628a3778162f14c522b/users/a264123dcd6e455b945b54652bf218db/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:04:10.520033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95106a3e-da3f-4339-8fa1-1d19d9f48d6d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:10.604763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95106a3e-da3f-4339-8fa1-1d19d9f48d6d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:10.611378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93172d8-f205-44cf-9b5d-01b26c10df0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:10.612073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93172d8-f205-44cf-9b5d-01b26c10df0f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:10.612496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93172d8-f205-44cf-9b5d-01b26c10df0f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:10.613087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93172d8-f205-44cf-9b5d-01b26c10df0f admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:10.614226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93172d8-f205-44cf-9b5d-01b26c10df0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:10.617421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93172d8-f205-44cf-9b5d-01b26c10df0f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:10.638272 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1107/2229] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:10.642625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93172d8-f205-44cf-9b5d-01b26c10df0f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:10.657142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93172d8-f205-44cf-9b5d-01b26c10df0f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:10.662338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5aa5c51f-7936-4842-9bff-aae2999b4df8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '299e6477185343bdbc2311528292fab1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:10.663820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3cc61c-e9a6-456d-8fb5-46524f8b89fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:10.669423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cc61c-e9a6-456d-8fb5-46524f8b89fc admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:10.669423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cc61c-e9a6-456d-8fb5-46524f8b89fc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:10.669423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3cc61c-e9a6-456d-8fb5-46524f8b89fc admin admin] PATH_INFO: `/v3/projects/3555b83003064628a3778162f14c522b/users/a264123dcd6e455b945b54652bf218db/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:10.669423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3cc61c-e9a6-456d-8fb5-46524f8b89fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3555b83003064628a3778162f14c522b, user_id=a264123dcd6e455b945b54652bf218db, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:10.672789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3cc61c-e9a6-456d-8fb5-46524f8b89fc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:10.680901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5aa5c51f-7936-4842-9bff-aae2999b4df8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '299e6477185343bdbc2311528292fab1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:10.681720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5aa5c51f-7936-4842-9bff-aae2999b4df8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '299e6477185343bdbc2311528292fab1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:10.683076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5aa5c51f-7936-4842-9bff-aae2999b4df8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '299e6477185343bdbc2311528292fab1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:10.686326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:10.710097 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1108/2230] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:10 2022] PUT /identity/v3/projects/3555b83003064628a3778162f14c522b/users/a264123dcd6e455b945b54652bf218db/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:10.721518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:10.723562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a6587-282a-458b-b577-57cb4c503716 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:10.723870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a6587-282a-458b-b577-57cb4c503716 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:10.724220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a6587-282a-458b-b577-57cb4c503716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:10.775183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-df4a6587-282a-458b-b577-57cb4c503716 None None] MFA Rules not processed for user `a264123dcd6e455b945b54652bf218db`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:10.776391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-df4a6587-282a-458b-b577-57cb4c503716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:10.787210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5aa5c51f-7936-4842-9bff-aae2999b4df8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '299e6477185343bdbc2311528292fab1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:10.828347 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1123/2231] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:10 2022] DELETE /identity/v3/users/299e6477185343bdbc2311528292fab1 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:04:10.829791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa9f32-1069-4ad2-a464-d5c53f392cc1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:10.830370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa9f32-1069-4ad2-a464-d5c53f392cc1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:10.830370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa9f32-1069-4ad2-a464-d5c53f392cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:10.873905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7efa9f32-1069-4ad2-a464-d5c53f392cc1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:10.875323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7efa9f32-1069-4ad2-a464-d5c53f392cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:10.887473 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1109/2232] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:10 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:04:10.892033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a6587-282a-458b-b577-57cb4c503716 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:10.892033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-df4a6587-282a-458b-b577-57cb4c503716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:10.979656 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1124/2233] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:10.993235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa9f32-1069-4ad2-a464-d5c53f392cc1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:10.996744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7efa9f32-1069-4ad2-a464-d5c53f392cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:11.019652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a6587-282a-458b-b577-57cb4c503716 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:11.026160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4445990-acca-403c-aa39-c8e202157342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:11.028004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4445990-acca-403c-aa39-c8e202157342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:11.028004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4445990-acca-403c-aa39-c8e202157342 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:11.028004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4445990-acca-403c-aa39-c8e202157342 admin admin] PATH_INFO: `/v3/users/78e6732bee9d4f86a26970a015c31c84` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:11.029146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4445990-acca-403c-aa39-c8e202157342 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78e6732bee9d4f86a26970a015c31c84)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:11.031714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:11.032052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4445990-acca-403c-aa39-c8e202157342 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:11.096890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa9f32-1069-4ad2-a464-d5c53f392cc1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:11.101051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0c81c-c5f7-4c7f-9ac6-cf0f73785595 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:11.101783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0c81c-c5f7-4c7f-9ac6-cf0f73785595 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:11.102103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0c81c-c5f7-4c7f-9ac6-cf0f73785595 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:11.102412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0c81c-c5f7-4c7f-9ac6-cf0f73785595 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:11.113146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0c81c-c5f7-4c7f-9ac6-cf0f73785595 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:11.114441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0c81c-c5f7-4c7f-9ac6-cf0f73785595 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:11.127011 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1125/2234] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:04:11.243596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0c81c-c5f7-4c7f-9ac6-cf0f73785595 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:11.245375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:11.246708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5eb0c81c-c5f7-4c7f-9ac6-cf0f73785595 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:11.258943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a4445990-acca-403c-aa39-c8e202157342 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78e6732bee9d4f86a26970a015c31c84'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:11.260246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a4445990-acca-403c-aa39-c8e202157342 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78e6732bee9d4f86a26970a015c31c84'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:11.268812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a4445990-acca-403c-aa39-c8e202157342 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78e6732bee9d4f86a26970a015c31c84'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:11.333191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0c81c-c5f7-4c7f-9ac6-cf0f73785595 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:11.340010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760879e-7e91-4eb2-8882-94e131e16d4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:11.340682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760879e-7e91-4eb2-8882-94e131e16d4c service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:11.341132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760879e-7e91-4eb2-8882-94e131e16d4c service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:11.341259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760879e-7e91-4eb2-8882-94e131e16d4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:11.345197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a4445990-acca-403c-aa39-c8e202157342 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78e6732bee9d4f86a26970a015c31c84'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:11.346657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a4445990-acca-403c-aa39-c8e202157342 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78e6732bee9d4f86a26970a015c31c84'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:11.383931 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1110/2235] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:10 2022] DELETE /identity/v3/users/78e6732bee9d4f86a26970a015c31c84 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:11.386976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347109ab-eb74-4fe7-98e7-a20a974b79a2 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:11.387274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347109ab-eb74-4fe7-98e7-a20a974b79a2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:11.387561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347109ab-eb74-4fe7-98e7-a20a974b79a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:11.418366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760879e-7e91-4eb2-8882-94e131e16d4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:11.418505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-347109ab-eb74-4fe7-98e7-a20a974b79a2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:11.419382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-347109ab-eb74-4fe7-98e7-a20a974b79a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:11.419650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760879e-7e91-4eb2-8882-94e131e16d4c service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:11.432917 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1126/2236] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:04:11.504269 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1111/2237] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:11.520402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760879e-7e91-4eb2-8882-94e131e16d4c service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:11.535021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760879e-7e91-4eb2-8882-94e131e16d4c service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:11.609086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5dc6f-00dd-4410-a30a-73aa70402f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:11.610433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5dc6f-00dd-4410-a30a-73aa70402f40 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:11.610433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5dc6f-00dd-4410-a30a-73aa70402f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:11.610645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5dc6f-00dd-4410-a30a-73aa70402f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:11.624539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5dc6f-00dd-4410-a30a-73aa70402f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:11.626400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5dc6f-00dd-4410-a30a-73aa70402f40 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:11.639573 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1127/2238] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:11.781213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347109ab-eb74-4fe7-98e7-a20a974b79a2 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:11.783476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-347109ab-eb74-4fe7-98e7-a20a974b79a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:11.871036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347109ab-eb74-4fe7-98e7-a20a974b79a2 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:11.877803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60a217-2263-4f1e-a17a-f6b5f64a8cfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:11.878450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60a217-2263-4f1e-a17a-f6b5f64a8cfe service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:11.878739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60a217-2263-4f1e-a17a-f6b5f64a8cfe service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:11.879011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60a217-2263-4f1e-a17a-f6b5f64a8cfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:11.881076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3b60a217-2263-4f1e-a17a-f6b5f64a8cfe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:11.946798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60a217-2263-4f1e-a17a-f6b5f64a8cfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:11.948835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b60a217-2263-4f1e-a17a-f6b5f64a8cfe service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:11.962270 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1112/2239] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:12.099896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5dc6f-00dd-4410-a30a-73aa70402f40 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:12.102866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6ee5dc6f-00dd-4410-a30a-73aa70402f40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:12.120477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5dc6f-00dd-4410-a30a-73aa70402f40 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:12.127490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9be420-7b00-4fb4-bbb5-279333cea701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:12.128716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9be420-7b00-4fb4-bbb5-279333cea701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:12.140049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9be420-7b00-4fb4-bbb5-279333cea701 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:12.140374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9be420-7b00-4fb4-bbb5-279333cea701 admin admin] PATH_INFO: `/v3/projects/8019c4c6c22a432ab9b44601856967e8` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:12.141793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9be420-7b00-4fb4-bbb5-279333cea701 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8019c4c6c22a432ab9b44601856967e8)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:12.144609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9be420-7b00-4fb4-bbb5-279333cea701 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:12.331505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0d9be420-7b00-4fb4-bbb5-279333cea701 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8019c4c6c22a432ab9b44601856967e8', 'request_id': 'req-0d9be420-7b00-4fb4-bbb5-279333cea701'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.337817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0d9be420-7b00-4fb4-bbb5-279333cea701 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8019c4c6c22a432ab9b44601856967e8', 'request_id': 'req-0d9be420-7b00-4fb4-bbb5-279333cea701'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.347246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60a217-2263-4f1e-a17a-f6b5f64a8cfe service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:12.350133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3b60a217-2263-4f1e-a17a-f6b5f64a8cfe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:12.373169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60a217-2263-4f1e-a17a-f6b5f64a8cfe service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:12.405338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0d9be420-7b00-4fb4-bbb5-279333cea701 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8019c4c6c22a432ab9b44601856967e8', 'request_id': 'req-0d9be420-7b00-4fb4-bbb5-279333cea701'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.418216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0d9be420-7b00-4fb4-bbb5-279333cea701 admin admin] The token cache is being invalidate because project 8019c4c6c22a432ab9b44601856967e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:12.418509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0d9be420-7b00-4fb4-bbb5-279333cea701 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.419529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0d9be420-7b00-4fb4-bbb5-279333cea701 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.421342 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1128/2240] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:12 2022] DELETE /identity/v3/projects/8019c4c6c22a432ab9b44601856967e8 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:12.425595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9be420-7b00-4fb4-bbb5-279333cea701 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:12.427211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0d9be420-7b00-4fb4-bbb5-279333cea701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:12.448293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eece19-e005-4e2f-a115-dd7973eca2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:12.449819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eece19-e005-4e2f-a115-dd7973eca2da admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:12.450189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eece19-e005-4e2f-a115-dd7973eca2da admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:12.450564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eece19-e005-4e2f-a115-dd7973eca2da admin admin] PATH_INFO: `/v3/projects/2b0858c8d98743319b44336db2d6c6e5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:12.452360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eece19-e005-4e2f-a115-dd7973eca2da admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b0858c8d98743319b44336db2d6c6e5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:12.454511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eece19-e005-4e2f-a115-dd7973eca2da admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:12.513370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:12.517635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9be420-7b00-4fb4-bbb5-279333cea701 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:12.523472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b0f79-531b-408d-8a09-3752e78689bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:12.524354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b0f79-531b-408d-8a09-3752e78689bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:12.524664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b0f79-531b-408d-8a09-3752e78689bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:12.524952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487b0f79-531b-408d-8a09-3752e78689bd admin admin] PATH_INFO: `/v3/users/d76932b7d2024134a2328f39683c72bc` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:12.526232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b0f79-531b-408d-8a09-3752e78689bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d76932b7d2024134a2328f39683c72bc)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:12.528210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487b0f79-531b-408d-8a09-3752e78689bd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:12.575556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-47eece19-e005-4e2f-a115-dd7973eca2da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b0858c8d98743319b44336db2d6c6e5', 'request_id': 'req-47eece19-e005-4e2f-a115-dd7973eca2da'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.581089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-47eece19-e005-4e2f-a115-dd7973eca2da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b0858c8d98743319b44336db2d6c6e5', 'request_id': 'req-47eece19-e005-4e2f-a115-dd7973eca2da'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.638926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-47eece19-e005-4e2f-a115-dd7973eca2da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b0858c8d98743319b44336db2d6c6e5', 'request_id': 'req-47eece19-e005-4e2f-a115-dd7973eca2da'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.651103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-47eece19-e005-4e2f-a115-dd7973eca2da admin admin] The token cache is being invalidate because project 2b0858c8d98743319b44336db2d6c6e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:12.651396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-47eece19-e005-4e2f-a115-dd7973eca2da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.652034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-47eece19-e005-4e2f-a115-dd7973eca2da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.653576 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1113/2241] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:12 2022] DELETE /identity/v3/projects/2b0858c8d98743319b44336db2d6c6e5 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:12.654444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eece19-e005-4e2f-a115-dd7973eca2da admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:12.655729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-47eece19-e005-4e2f-a115-dd7973eca2da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:12.704002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-487b0f79-531b-408d-8a09-3752e78689bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd76932b7d2024134a2328f39683c72bc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.711341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-487b0f79-531b-408d-8a09-3752e78689bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd76932b7d2024134a2328f39683c72bc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.712109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-487b0f79-531b-408d-8a09-3752e78689bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd76932b7d2024134a2328f39683c72bc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.712742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-487b0f79-531b-408d-8a09-3752e78689bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd76932b7d2024134a2328f39683c72bc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.719318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eece19-e005-4e2f-a115-dd7973eca2da admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:12.723149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cecbf-ca43-479c-a15c-118fdb7fac74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:12.724005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cecbf-ca43-479c-a15c-118fdb7fac74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:12.724292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cecbf-ca43-479c-a15c-118fdb7fac74 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:12.724615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cecbf-ca43-479c-a15c-118fdb7fac74 admin admin] PATH_INFO: `/v3/users/db0f47bc701d49f5a68fc6e26ed1629a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:12.726372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cecbf-ca43-479c-a15c-118fdb7fac74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db0f47bc701d49f5a68fc6e26ed1629a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:12.728359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cecbf-ca43-479c-a15c-118fdb7fac74 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:12.776076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:12.792877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-487b0f79-531b-408d-8a09-3752e78689bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd76932b7d2024134a2328f39683c72bc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.811189 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1129/2242] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:12 2022] DELETE /identity/v3/users/d76932b7d2024134a2328f39683c72bc => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:12.812213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b0f79-531b-408d-8a09-3752e78689bd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:12.819918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-487b0f79-531b-408d-8a09-3752e78689bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:12.895232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487b0f79-531b-408d-8a09-3752e78689bd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:12.901423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a7a6aa-60af-45f8-a5e3-ee4fabf0ac29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:12.902351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7a6aa-60af-45f8-a5e3-ee4fabf0ac29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:12.902750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7a6aa-60af-45f8-a5e3-ee4fabf0ac29 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:12.903030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a7a6aa-60af-45f8-a5e3-ee4fabf0ac29 admin admin] PATH_INFO: `/v3/users/10534fe7f08c44f6a6cd0433e4b91d96` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:12.904339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7a6aa-60af-45f8-a5e3-ee4fabf0ac29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10534fe7f08c44f6a6cd0433e4b91d96)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:12.906661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a7a6aa-60af-45f8-a5e3-ee4fabf0ac29 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:12.907097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-415cecbf-ca43-479c-a15c-118fdb7fac74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db0f47bc701d49f5a68fc6e26ed1629a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.907897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-415cecbf-ca43-479c-a15c-118fdb7fac74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db0f47bc701d49f5a68fc6e26ed1629a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.921377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-415cecbf-ca43-479c-a15c-118fdb7fac74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db0f47bc701d49f5a68fc6e26ed1629a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.998820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-415cecbf-ca43-479c-a15c-118fdb7fac74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db0f47bc701d49f5a68fc6e26ed1629a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:12.999467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-415cecbf-ca43-479c-a15c-118fdb7fac74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db0f47bc701d49f5a68fc6e26ed1629a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:13.018533 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1114/2243] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:12 2022] DELETE /identity/v3/users/db0f47bc701d49f5a68fc6e26ed1629a => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:13.023606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec46cca-1c3d-424a-b3c8-97db02d58001 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:13.023896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec46cca-1c3d-424a-b3c8-97db02d58001 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:13.024172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec46cca-1c3d-424a-b3c8-97db02d58001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:13.043501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7ec46cca-1c3d-424a-b3c8-97db02d58001 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:13.045049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7ec46cca-1c3d-424a-b3c8-97db02d58001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:13.106608 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1115/2244] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:13.116210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec46cca-1c3d-424a-b3c8-97db02d58001 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:13.117552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7ec46cca-1c3d-424a-b3c8-97db02d58001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:13.127650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-99a7a6aa-60af-45f8-a5e3-ee4fabf0ac29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10534fe7f08c44f6a6cd0433e4b91d96'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:13.137015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-99a7a6aa-60af-45f8-a5e3-ee4fabf0ac29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10534fe7f08c44f6a6cd0433e4b91d96'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:13.137814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-99a7a6aa-60af-45f8-a5e3-ee4fabf0ac29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10534fe7f08c44f6a6cd0433e4b91d96'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:13.138555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-99a7a6aa-60af-45f8-a5e3-ee4fabf0ac29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10534fe7f08c44f6a6cd0433e4b91d96'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:13.182629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec46cca-1c3d-424a-b3c8-97db02d58001 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:13.186353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b63e37-6082-4b4f-aa8e-9f21e2d006e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:13.187008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b63e37-6082-4b4f-aa8e-9f21e2d006e7 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:13.187295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b63e37-6082-4b4f-aa8e-9f21e2d006e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:13.187574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b63e37-6082-4b4f-aa8e-9f21e2d006e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:13.189504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d2b63e37-6082-4b4f-aa8e-9f21e2d006e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:13.203039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b63e37-6082-4b4f-aa8e-9f21e2d006e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:13.204493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b63e37-6082-4b4f-aa8e-9f21e2d006e7 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:13.215201 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1116/2245] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:04:13.221497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-99a7a6aa-60af-45f8-a5e3-ee4fabf0ac29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10534fe7f08c44f6a6cd0433e4b91d96'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:13.246609 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1130/2246] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:12 2022] DELETE /identity/v3/users/10534fe7f08c44f6a6cd0433e4b91d96 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:13.382544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:13.411725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b63e37-6082-4b4f-aa8e-9f21e2d006e7 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:13.413689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d2b63e37-6082-4b4f-aa8e-9f21e2d006e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:13.430226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b63e37-6082-4b4f-aa8e-9f21e2d006e7 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:13.495740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94895f87-ef4d-41ed-a6a1-bccd39a3b766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:13.496610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94895f87-ef4d-41ed-a6a1-bccd39a3b766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:13.496916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94895f87-ef4d-41ed-a6a1-bccd39a3b766 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:13.497213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94895f87-ef4d-41ed-a6a1-bccd39a3b766 admin admin] PATH_INFO: `/v3/projects/b04f3b8ea4f2457aa59850428fea8ba8` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:13.498444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94895f87-ef4d-41ed-a6a1-bccd39a3b766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b04f3b8ea4f2457aa59850428fea8ba8)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:13.500463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94895f87-ef4d-41ed-a6a1-bccd39a3b766 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:13.630337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94895f87-ef4d-41ed-a6a1-bccd39a3b766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b04f3b8ea4f2457aa59850428fea8ba8', 'request_id': 'req-94895f87-ef4d-41ed-a6a1-bccd39a3b766'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:13.655981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:13.669320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94895f87-ef4d-41ed-a6a1-bccd39a3b766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b04f3b8ea4f2457aa59850428fea8ba8', 'request_id': 'req-94895f87-ef4d-41ed-a6a1-bccd39a3b766'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:13.742662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94895f87-ef4d-41ed-a6a1-bccd39a3b766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b04f3b8ea4f2457aa59850428fea8ba8', 'request_id': 'req-94895f87-ef4d-41ed-a6a1-bccd39a3b766'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:13.756727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94895f87-ef4d-41ed-a6a1-bccd39a3b766 admin admin] The token cache is being invalidate because project b04f3b8ea4f2457aa59850428fea8ba8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:13.756913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94895f87-ef4d-41ed-a6a1-bccd39a3b766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:13.758259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94895f87-ef4d-41ed-a6a1-bccd39a3b766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:13.760550 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1117/2247] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:13 2022] DELETE /identity/v3/projects/b04f3b8ea4f2457aa59850428fea8ba8 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:13.766043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69b7cf-5e2d-4dcf-81c8-23276711ab95 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:13.766349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69b7cf-5e2d-4dcf-81c8-23276711ab95 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:13.766674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69b7cf-5e2d-4dcf-81c8-23276711ab95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:13.810238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-eb69b7cf-5e2d-4dcf-81c8-23276711ab95 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:13.810238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eb69b7cf-5e2d-4dcf-81c8-23276711ab95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:13.875590 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1131/2248] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:13.884659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5a6cb-d0fd-4f62-b64f-0d87f03b185f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:13.884982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5a6cb-d0fd-4f62-b64f-0d87f03b185f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:13.885263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be5a6cb-d0fd-4f62-b64f-0d87f03b185f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:13.903864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6be5a6cb-d0fd-4f62-b64f-0d87f03b185f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:13.905012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6be5a6cb-d0fd-4f62-b64f-0d87f03b185f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:13.913713 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1118/2249] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:13.918238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69b7cf-5e2d-4dcf-81c8-23276711ab95 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:13.929188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69b7cf-5e2d-4dcf-81c8-23276711ab95 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:13.933988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326ca1d-f19a-45c8-9cc8-b7e7b7e7ae41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:13.934648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326ca1d-f19a-45c8-9cc8-b7e7b7e7ae41 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:13.934933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326ca1d-f19a-45c8-9cc8-b7e7b7e7ae41 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:13.935215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0326ca1d-f19a-45c8-9cc8-b7e7b7e7ae41 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:13.935947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0326ca1d-f19a-45c8-9cc8-b7e7b7e7ae41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:13.936258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0326ca1d-f19a-45c8-9cc8-b7e7b7e7ae41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:13.937944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0326ca1d-f19a-45c8-9cc8-b7e7b7e7ae41 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:13.949434 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1132/2250] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:13.953904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5a6cb-d0fd-4f62-b64f-0d87f03b185f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:13.963470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be5a6cb-d0fd-4f62-b64f-0d87f03b185f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:13.968338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe785f77-aa85-4bf3-bf68-597a2671d626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:13.969026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe785f77-aa85-4bf3-bf68-597a2671d626 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:13.969315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe785f77-aa85-4bf3-bf68-597a2671d626 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:13.969625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe785f77-aa85-4bf3-bf68-597a2671d626 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:13.970362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe785f77-aa85-4bf3-bf68-597a2671d626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:13.972283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe785f77-aa85-4bf3-bf68-597a2671d626 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:13.984470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326ca1d-f19a-45c8-9cc8-b7e7b7e7ae41 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:13.988437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0326ca1d-f19a-45c8-9cc8-b7e7b7e7ae41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:14.003123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0326ca1d-f19a-45c8-9cc8-b7e7b7e7ae41 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:14.007941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf570fa-19dc-478a-a197-0c42938a7d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:14.008838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf570fa-19dc-478a-a197-0c42938a7d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:14.009123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf570fa-19dc-478a-a197-0c42938a7d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:14.009435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf570fa-19dc-478a-a197-0c42938a7d47 admin admin] PATH_INFO: `/v3/projects/e0f50b8715e64beca36d4320ac0ca972` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:14.010670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf570fa-19dc-478a-a197-0c42938a7d47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0f50b8715e64beca36d4320ac0ca972)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:14.012606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf570fa-19dc-478a-a197-0c42938a7d47 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:14.099172 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1119/2251] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:13 2022] POST /identity/v3/projects => generated 371 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:14.100204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe785f77-aa85-4bf3-bf68-597a2671d626 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:14.103089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fe785f77-aa85-4bf3-bf68-597a2671d626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:14.154850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abf570fa-19dc-478a-a197-0c42938a7d47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0f50b8715e64beca36d4320ac0ca972', 'request_id': 'req-abf570fa-19dc-478a-a197-0c42938a7d47'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:14.163622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abf570fa-19dc-478a-a197-0c42938a7d47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0f50b8715e64beca36d4320ac0ca972', 'request_id': 'req-abf570fa-19dc-478a-a197-0c42938a7d47'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:14.204470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe785f77-aa85-4bf3-bf68-597a2671d626 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:14.245589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abf570fa-19dc-478a-a197-0c42938a7d47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0f50b8715e64beca36d4320ac0ca972', 'request_id': 'req-abf570fa-19dc-478a-a197-0c42938a7d47'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:14.270483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abf570fa-19dc-478a-a197-0c42938a7d47 admin admin] The token cache is being invalidate because project e0f50b8715e64beca36d4320ac0ca972 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:14.271063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abf570fa-19dc-478a-a197-0c42938a7d47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:14.272482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abf570fa-19dc-478a-a197-0c42938a7d47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:14.275229 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1133/2252] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:13 2022] DELETE /identity/v3/projects/e0f50b8715e64beca36d4320ac0ca972 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:04:14.276315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf570fa-19dc-478a-a197-0c42938a7d47 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:14.278066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-abf570fa-19dc-478a-a197-0c42938a7d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:14.279343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa536daf-b2eb-4638-9403-1824b54249e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:14.280631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa536daf-b2eb-4638-9403-1824b54249e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:14.281960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa536daf-b2eb-4638-9403-1824b54249e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:14.283728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa536daf-b2eb-4638-9403-1824b54249e4 admin admin] PATH_INFO: `/v3/projects/de1f4b87aefe4f1eb4ac3d3f6934ef5e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:14.286586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa536daf-b2eb-4638-9403-1824b54249e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de1f4b87aefe4f1eb4ac3d3f6934ef5e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:14.289120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa536daf-b2eb-4638-9403-1824b54249e4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:14.294312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf570fa-19dc-478a-a197-0c42938a7d47 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:14.300190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d1a62-b4c4-430e-8e90-7953b4d580fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:14.300968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d1a62-b4c4-430e-8e90-7953b4d580fb admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:14.301260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d1a62-b4c4-430e-8e90-7953b4d580fb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:14.301536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d1a62-b4c4-430e-8e90-7953b4d580fb admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:14.303399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d1a62-b4c4-430e-8e90-7953b4d580fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:14.306447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d1a62-b4c4-430e-8e90-7953b4d580fb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:14.416842 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1134/2253] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:14 2022] POST /identity/v3/users => generated 286 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:14.416842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d1a62-b4c4-430e-8e90-7953b4d580fb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:14.416842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d44d1a62-b4c4-430e-8e90-7953b4d580fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:14.431281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d1a62-b4c4-430e-8e90-7953b4d580fb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:14.434219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aa536daf-b2eb-4638-9403-1824b54249e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de1f4b87aefe4f1eb4ac3d3f6934ef5e', 'request_id': 'req-aa536daf-b2eb-4638-9403-1824b54249e4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:14.440164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aa536daf-b2eb-4638-9403-1824b54249e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de1f4b87aefe4f1eb4ac3d3f6934ef5e', 'request_id': 'req-aa536daf-b2eb-4638-9403-1824b54249e4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:14.490081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9643620-a297-4879-a254-c8e82a060056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:14.491510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9643620-a297-4879-a254-c8e82a060056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:14.491796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9643620-a297-4879-a254-c8e82a060056 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:14.492075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9643620-a297-4879-a254-c8e82a060056 admin admin] PATH_INFO: `/v3/users/a264123dcd6e455b945b54652bf218db` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:14.493329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9643620-a297-4879-a254-c8e82a060056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a264123dcd6e455b945b54652bf218db)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:14.495666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9643620-a297-4879-a254-c8e82a060056 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:14.496779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aa536daf-b2eb-4638-9403-1824b54249e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de1f4b87aefe4f1eb4ac3d3f6934ef5e', 'request_id': 'req-aa536daf-b2eb-4638-9403-1824b54249e4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:14.516269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aa536daf-b2eb-4638-9403-1824b54249e4 admin admin] The token cache is being invalidate because project de1f4b87aefe4f1eb4ac3d3f6934ef5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:14.516822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aa536daf-b2eb-4638-9403-1824b54249e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:14.517671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aa536daf-b2eb-4638-9403-1824b54249e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:14.519591 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1120/2254] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:14 2022] DELETE /identity/v3/projects/de1f4b87aefe4f1eb4ac3d3f6934ef5e => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:14.520263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa536daf-b2eb-4638-9403-1824b54249e4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:14.522059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-aa536daf-b2eb-4638-9403-1824b54249e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:14.534921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa536daf-b2eb-4638-9403-1824b54249e4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:14.540218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16054df9-104c-4f58-897f-8c2e82110189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:14.540881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16054df9-104c-4f58-897f-8c2e82110189 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:14.541173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16054df9-104c-4f58-897f-8c2e82110189 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:14.541451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16054df9-104c-4f58-897f-8c2e82110189 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:14.542357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16054df9-104c-4f58-897f-8c2e82110189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:14.544004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16054df9-104c-4f58-897f-8c2e82110189 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:14.555641 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1121/2255] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:14.557087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b759c5f-7b6a-4fb2-ba23-0566bd47c75e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:14.557379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b759c5f-7b6a-4fb2-ba23-0566bd47c75e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:14.557660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b759c5f-7b6a-4fb2-ba23-0566bd47c75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:14.575672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9b759c5f-7b6a-4fb2-ba23-0566bd47c75e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:14.576894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9b759c5f-7b6a-4fb2-ba23-0566bd47c75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:14.589591 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1122/2256] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:14.590802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b759c5f-7b6a-4fb2-ba23-0566bd47c75e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:14.606009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b759c5f-7b6a-4fb2-ba23-0566bd47c75e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:14.610796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450f4e8b-d539-43b5-84ce-b728a61cd268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:14.611628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f4e8b-d539-43b5-84ce-b728a61cd268 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:14.612157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f4e8b-d539-43b5-84ce-b728a61cd268 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:14.617798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450f4e8b-d539-43b5-84ce-b728a61cd268 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:14.617798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f4e8b-d539-43b5-84ce-b728a61cd268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:14.617798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450f4e8b-d539-43b5-84ce-b728a61cd268 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:14.625748 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1123/2257] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:14.627023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c267613-12fd-45e0-8cb5-1538d88dc9cf None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:14.629349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c267613-12fd-45e0-8cb5-1538d88dc9cf None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:14.629349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c267613-12fd-45e0-8cb5-1538d88dc9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:14.640822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:14.649215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6c267613-12fd-45e0-8cb5-1538d88dc9cf None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:14.650204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c9643620-a297-4879-a254-c8e82a060056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a264123dcd6e455b945b54652bf218db'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:14.652455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6c267613-12fd-45e0-8cb5-1538d88dc9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:14.658233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c9643620-a297-4879-a254-c8e82a060056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a264123dcd6e455b945b54652bf218db'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:14.659372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c9643620-a297-4879-a254-c8e82a060056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a264123dcd6e455b945b54652bf218db'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:14.664991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c9643620-a297-4879-a254-c8e82a060056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a264123dcd6e455b945b54652bf218db'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:14.665110 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1124/2258] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:14.665110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c267613-12fd-45e0-8cb5-1538d88dc9cf None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:14.673263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c267613-12fd-45e0-8cb5-1538d88dc9cf None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:14.678624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbdac55-3cdd-4626-bd90-0ce61471f385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:14.680987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdac55-3cdd-4626-bd90-0ce61471f385 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:14.680987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdac55-3cdd-4626-bd90-0ce61471f385 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:14.680987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbdac55-3cdd-4626-bd90-0ce61471f385 admin admin] PATH_INFO: `/v3/projects/2f5c3054461d4d07861a1685cfeba36a/users/7b44b524592e4a57a26681767b9c4541/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:14.695747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbdac55-3cdd-4626-bd90-0ce61471f385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f5c3054461d4d07861a1685cfeba36a, user_id=7b44b524592e4a57a26681767b9c4541, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:14.698509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbdac55-3cdd-4626-bd90-0ce61471f385 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:14.712972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c9643620-a297-4879-a254-c8e82a060056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a264123dcd6e455b945b54652bf218db'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:14.732574 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1135/2259] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:14 2022] DELETE /identity/v3/users/a264123dcd6e455b945b54652bf218db => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:14.733527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9643620-a297-4879-a254-c8e82a060056 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:14.734992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c9643620-a297-4879-a254-c8e82a060056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:14.793572 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1125/2260] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:14 2022] PUT /identity/v3/projects/2f5c3054461d4d07861a1685cfeba36a/users/7b44b524592e4a57a26681767b9c4541/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:14.795281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d924d-86f4-4b43-aba6-b017d4c9ec64 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:14.795750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d924d-86f4-4b43-aba6-b017d4c9ec64 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:14.796410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d924d-86f4-4b43-aba6-b017d4c9ec64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:14.802722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:14.820429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9643620-a297-4879-a254-c8e82a060056 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:14.846089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-025d924d-86f4-4b43-aba6-b017d4c9ec64 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:14.847644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-025d924d-86f4-4b43-aba6-b017d4c9ec64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:14.905049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dea030-7336-44bf-845f-3ba4fe154601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:14.905899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dea030-7336-44bf-845f-3ba4fe154601 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:14.906512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dea030-7336-44bf-845f-3ba4fe154601 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:14.906857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dea030-7336-44bf-845f-3ba4fe154601 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:14.907881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dea030-7336-44bf-845f-3ba4fe154601 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:14.910108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dea030-7336-44bf-845f-3ba4fe154601 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:14.910575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dea030-7336-44bf-845f-3ba4fe154601 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:14.938793 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1136/2261] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:14.942231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dea030-7336-44bf-845f-3ba4fe154601 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:14.957627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dea030-7336-44bf-845f-3ba4fe154601 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:14.959677 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1126/2262] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:14.964284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b0f75-923c-4788-81b7-9d95e3fb4b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:14.965151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b0f75-923c-4788-81b7-9d95e3fb4b90 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:14.965674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b0f75-923c-4788-81b7-9d95e3fb4b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:14.966075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355b0f75-923c-4788-81b7-9d95e3fb4b90 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:14.967445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b0f75-923c-4788-81b7-9d95e3fb4b90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:14.968916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355b0f75-923c-4788-81b7-9d95e3fb4b90 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:14.986818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d924d-86f4-4b43-aba6-b017d4c9ec64 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:14.993064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-025d924d-86f4-4b43-aba6-b017d4c9ec64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:15.116745 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1137/2263] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:14 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:15.119493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d924d-86f4-4b43-aba6-b017d4c9ec64 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:15.121443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b0f75-923c-4788-81b7-9d95e3fb4b90 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:15.277026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edcf12-25f7-44e8-9659-7db6a67f0e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:15.278169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edcf12-25f7-44e8-9659-7db6a67f0e8d service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:15.278461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edcf12-25f7-44e8-9659-7db6a67f0e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:15.278744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7edcf12-25f7-44e8-9659-7db6a67f0e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:15.279780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355b0f75-923c-4788-81b7-9d95e3fb4b90 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:15.285449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68fa3c-a137-42d1-b832-2869adda3f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:15.288559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68fa3c-a137-42d1-b832-2869adda3f6c admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:15.288559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68fa3c-a137-42d1-b832-2869adda3f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:15.288559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68fa3c-a137-42d1-b832-2869adda3f6c admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:15.288559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68fa3c-a137-42d1-b832-2869adda3f6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:15.289956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68fa3c-a137-42d1-b832-2869adda3f6c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:15.294450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edcf12-25f7-44e8-9659-7db6a67f0e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:15.296113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7edcf12-25f7-44e8-9659-7db6a67f0e8d service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:15.311483 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1127/2264] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:15.409923 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1138/2265] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:15 2022] POST /identity/v3/users => generated 291 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:04:15.413281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edcf12-25f7-44e8-9659-7db6a67f0e8d service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:15.429644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7edcf12-25f7-44e8-9659-7db6a67f0e8d service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:15.435249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d021821-7df7-453f-9205-964780073ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:15.436363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d021821-7df7-453f-9205-964780073ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:15.436648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d021821-7df7-453f-9205-964780073ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:15.437614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d021821-7df7-453f-9205-964780073ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:15.438445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d021821-7df7-453f-9205-964780073ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:15.440105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d021821-7df7-453f-9205-964780073ef2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:15.457292 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1128/2266] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:15.458816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68fa3c-a137-42d1-b832-2869adda3f6c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:15.471314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68fa3c-a137-42d1-b832-2869adda3f6c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:15.478463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399490b-3ee9-43c9-b001-418072e5c219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:15.479230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399490b-3ee9-43c9-b001-418072e5c219 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:15.479646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399490b-3ee9-43c9-b001-418072e5c219 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:15.479994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399490b-3ee9-43c9-b001-418072e5c219 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:15.481275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399490b-3ee9-43c9-b001-418072e5c219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:15.483423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399490b-3ee9-43c9-b001-418072e5c219 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:15.496816 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1139/2267] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:15.501064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d021821-7df7-453f-9205-964780073ef2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:15.513157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d021821-7df7-453f-9205-964780073ef2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:15.518353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4aef4-d9ec-48e0-a508-3e21601252f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:15.519241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4aef4-d9ec-48e0-a508-3e21601252f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:15.519524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4aef4-d9ec-48e0-a508-3e21601252f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:15.519800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4aef4-d9ec-48e0-a508-3e21601252f1 admin admin] PATH_INFO: `/v3/projects/065942f484d14140827befe78c108930/users/7204868cbb0a49aa83d9ccb85e610ea9/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:15.532271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399490b-3ee9-43c9-b001-418072e5c219 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:15.534148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f399490b-3ee9-43c9-b001-418072e5c219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:15.562671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399490b-3ee9-43c9-b001-418072e5c219 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:15.569145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c4560-7083-4713-8458-5fdfb8b69e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:15.570110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c4560-7083-4713-8458-5fdfb8b69e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:15.570784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c4560-7083-4713-8458-5fdfb8b69e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:15.571074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c4560-7083-4713-8458-5fdfb8b69e60 admin admin] PATH_INFO: `/v3/projects/3555b83003064628a3778162f14c522b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:15.572319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c4560-7083-4713-8458-5fdfb8b69e60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3555b83003064628a3778162f14c522b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:15.574943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c4560-7083-4713-8458-5fdfb8b69e60 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:15.588370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4aef4-d9ec-48e0-a508-3e21601252f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=065942f484d14140827befe78c108930, user_id=7204868cbb0a49aa83d9ccb85e610ea9, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:15.591504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4aef4-d9ec-48e0-a508-3e21601252f1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:15.668679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:15.687398 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1129/2268] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:15 2022] PUT /identity/v3/projects/065942f484d14140827befe78c108930/users/7204868cbb0a49aa83d9ccb85e610ea9/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:04:15.689508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333aac6-5cce-49e1-ad31-64c860dcc450 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:15.689973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333aac6-5cce-49e1-ad31-64c860dcc450 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:15.690256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333aac6-5cce-49e1-ad31-64c860dcc450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:15.704448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9e7c4560-7083-4713-8458-5fdfb8b69e60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3555b83003064628a3778162f14c522b', 'request_id': 'req-9e7c4560-7083-4713-8458-5fdfb8b69e60'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:15.710118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9e7c4560-7083-4713-8458-5fdfb8b69e60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3555b83003064628a3778162f14c522b', 'request_id': 'req-9e7c4560-7083-4713-8458-5fdfb8b69e60'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:15.723434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8333aac6-5cce-49e1-ad31-64c860dcc450 None None] MFA Rules not processed for user `7204868cbb0a49aa83d9ccb85e610ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:15.725004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8333aac6-5cce-49e1-ad31-64c860dcc450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:15.770741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9e7c4560-7083-4713-8458-5fdfb8b69e60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3555b83003064628a3778162f14c522b', 'request_id': 'req-9e7c4560-7083-4713-8458-5fdfb8b69e60'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:15.783869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9e7c4560-7083-4713-8458-5fdfb8b69e60 admin admin] The token cache is being invalidate because project 3555b83003064628a3778162f14c522b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:15.784159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9e7c4560-7083-4713-8458-5fdfb8b69e60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:15.785090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9e7c4560-7083-4713-8458-5fdfb8b69e60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:15.787046 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1140/2269] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:15 2022] DELETE /identity/v3/projects/3555b83003064628a3778162f14c522b => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:15.793423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4de962-8400-4f04-bb3c-a50cfa5aa67b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:15.793746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4de962-8400-4f04-bb3c-a50cfa5aa67b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:15.794144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4de962-8400-4f04-bb3c-a50cfa5aa67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:15.798283 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1130/2270] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:15.798711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8333aac6-5cce-49e1-ad31-64c860dcc450 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:15.817632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:15.819383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0f4de962-8400-4f04-bb3c-a50cfa5aa67b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:15.820510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0f4de962-8400-4f04-bb3c-a50cfa5aa67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:15.887897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8333aac6-5cce-49e1-ad31-64c860dcc450 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:15.890245 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1141/2271] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:15.893306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a621aff-431d-494b-82cb-a3dd7c0f0aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:15.894240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a621aff-431d-494b-82cb-a3dd7c0f0aa8 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:15.894532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a621aff-431d-494b-82cb-a3dd7c0f0aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:15.894815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a621aff-431d-494b-82cb-a3dd7c0f0aa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:15.895656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a621aff-431d-494b-82cb-a3dd7c0f0aa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:15.901691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a621aff-431d-494b-82cb-a3dd7c0f0aa8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:15.910975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39274f7d-cdd5-4138-a786-866b16389bba None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:15.911478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39274f7d-cdd5-4138-a786-866b16389bba None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:15.920171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39274f7d-cdd5-4138-a786-866b16389bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:15.939647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-39274f7d-cdd5-4138-a786-866b16389bba None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:15.941398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-39274f7d-cdd5-4138-a786-866b16389bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:15.953960 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1142/2272] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:15.955556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39274f7d-cdd5-4138-a786-866b16389bba None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:15.974198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39274f7d-cdd5-4138-a786-866b16389bba None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:15.979894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a28dc-1767-471d-94fb-fc5ec3a59f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:15.980724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a28dc-1767-471d-94fb-fc5ec3a59f64 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:15.981017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a28dc-1767-471d-94fb-fc5ec3a59f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:15.981446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a28dc-1767-471d-94fb-fc5ec3a59f64 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:15.982406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9a28dc-1767-471d-94fb-fc5ec3a59f64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:15.982720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9a28dc-1767-471d-94fb-fc5ec3a59f64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:15.984348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9a28dc-1767-471d-94fb-fc5ec3a59f64 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:16.013311 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1143/2273] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:04:16.018109 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1131/2274] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:15 2022] POST /identity/v3/projects => generated 381 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:16.018677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a28dc-1767-471d-94fb-fc5ec3a59f64 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:16.022517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a621aff-431d-494b-82cb-a3dd7c0f0aa8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:16.024214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9a621aff-431d-494b-82cb-a3dd7c0f0aa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:16.033609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a28dc-1767-471d-94fb-fc5ec3a59f64 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:16.140010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be29ca-0055-40e8-ae8b-d75cb112e76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:16.141731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be29ca-0055-40e8-ae8b-d75cb112e76d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:16.142258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be29ca-0055-40e8-ae8b-d75cb112e76d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:16.143402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be29ca-0055-40e8-ae8b-d75cb112e76d admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:16.144803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be29ca-0055-40e8-ae8b-d75cb112e76d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:16.147039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be29ca-0055-40e8-ae8b-d75cb112e76d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:16.153402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a621aff-431d-494b-82cb-a3dd7c0f0aa8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:16.163835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbdf63-52db-4626-93cf-de2619236ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:16.164526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbdf63-52db-4626-93cf-de2619236ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:16.164861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbdf63-52db-4626-93cf-de2619236ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:16.165146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbdf63-52db-4626-93cf-de2619236ea0 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:16.165875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbbdf63-52db-4626-93cf-de2619236ea0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:16.169285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbbdf63-52db-4626-93cf-de2619236ea0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:16.325267 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1144/2275] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:16 2022] POST /identity/v3/projects => generated 377 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:16.325475 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1132/2276] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:16 2022] POST /identity/v3/users => generated 297 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:16.333297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbdf63-52db-4626-93cf-de2619236ea0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:16.333703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be29ca-0055-40e8-ae8b-d75cb112e76d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:16.345420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbbdf63-52db-4626-93cf-de2619236ea0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:16.407266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be29ca-0055-40e8-ae8b-d75cb112e76d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:16.413558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfa628-9759-4d60-99fb-d18eeea60c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:16.414644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfa628-9759-4d60-99fb-d18eeea60c1e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:16.415082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfa628-9759-4d60-99fb-d18eeea60c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:16.415511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccfa628-9759-4d60-99fb-d18eeea60c1e admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:16.416480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccfa628-9759-4d60-99fb-d18eeea60c1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:16.418316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccfa628-9759-4d60-99fb-d18eeea60c1e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:16.445959 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1145/2277] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:16.455957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfa628-9759-4d60-99fb-d18eeea60c1e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:16.462556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e8695-3c29-4a30-ac8b-8afec8ea7d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:16.463267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e8695-3c29-4a30-ac8b-8afec8ea7d05 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:16.463554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e8695-3c29-4a30-ac8b-8afec8ea7d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:16.463886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585e8695-3c29-4a30-ac8b-8afec8ea7d05 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:16.464703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e8695-3c29-4a30-ac8b-8afec8ea7d05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:16.466302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccfa628-9759-4d60-99fb-d18eeea60c1e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:16.466757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585e8695-3c29-4a30-ac8b-8afec8ea7d05 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:16.471851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca855835-8da8-407c-a54c-afbf272f8b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:16.473020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca855835-8da8-407c-a54c-afbf272f8b76 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:16.473410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca855835-8da8-407c-a54c-afbf272f8b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:16.473792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca855835-8da8-407c-a54c-afbf272f8b76 admin admin] PATH_INFO: `/v3/projects/a4a541c4e769448ba1a8d194c9b33a27/users/a3dcdf84f381468c8cea281d0a451075/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:16.494283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca855835-8da8-407c-a54c-afbf272f8b76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a541c4e769448ba1a8d194c9b33a27, user_id=a3dcdf84f381468c8cea281d0a451075, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:16.497480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca855835-8da8-407c-a54c-afbf272f8b76 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:16.566312 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1133/2278] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:16 2022] POST /identity/v3/users => generated 289 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:16.576636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e8695-3c29-4a30-ac8b-8afec8ea7d05 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:16.587908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585e8695-3c29-4a30-ac8b-8afec8ea7d05 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:16.593385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd092ffe-c723-406b-ba2a-5a9692d4005c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:16.594622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd092ffe-c723-406b-ba2a-5a9692d4005c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:16.594959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd092ffe-c723-406b-ba2a-5a9692d4005c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:16.600095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd092ffe-c723-406b-ba2a-5a9692d4005c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:16.600095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd092ffe-c723-406b-ba2a-5a9692d4005c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:16.600095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd092ffe-c723-406b-ba2a-5a9692d4005c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:16.603299 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1146/2279] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:16 2022] PUT /identity/v3/projects/a4a541c4e769448ba1a8d194c9b33a27/users/a3dcdf84f381468c8cea281d0a451075/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:16.604237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca855835-8da8-407c-a54c-afbf272f8b76 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:16.613752 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1134/2280] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:04:16.618415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd092ffe-c723-406b-ba2a-5a9692d4005c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:16.630181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd092ffe-c723-406b-ba2a-5a9692d4005c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:16.746093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca855835-8da8-407c-a54c-afbf272f8b76 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:16.758404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900f0b3-4341-4432-a8f4-d66c2727029e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:16.766472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900f0b3-4341-4432-a8f4-d66c2727029e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:16.766472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900f0b3-4341-4432-a8f4-d66c2727029e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:16.766472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9900f0b3-4341-4432-a8f4-d66c2727029e admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:16.766472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900f0b3-4341-4432-a8f4-d66c2727029e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:16.766472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9900f0b3-4341-4432-a8f4-d66c2727029e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:16.800743 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1147/2281] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:16.804617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900f0b3-4341-4432-a8f4-d66c2727029e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:16.807201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701aae7-0ba0-41ea-9d3e-e71545dbc3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:16.807999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701aae7-0ba0-41ea-9d3e-e71545dbc3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:16.808193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701aae7-0ba0-41ea-9d3e-e71545dbc3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:16.808471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701aae7-0ba0-41ea-9d3e-e71545dbc3b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:16.809223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701aae7-0ba0-41ea-9d3e-e71545dbc3b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:16.812679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701aae7-0ba0-41ea-9d3e-e71545dbc3b8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:16.827659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9900f0b3-4341-4432-a8f4-d66c2727029e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:16.835428 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1135/2282] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:16.853324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701aae7-0ba0-41ea-9d3e-e71545dbc3b8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:16.864679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b6d68-2278-4006-8635-5a257539b60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:16.865692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b6d68-2278-4006-8635-5a257539b60a admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:16.866051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b6d68-2278-4006-8635-5a257539b60a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:16.866392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b6d68-2278-4006-8635-5a257539b60a admin admin] PATH_INFO: `/v3/projects/a4a541c4e769448ba1a8d194c9b33a27/users/a3dcdf84f381468c8cea281d0a451075/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:16.870017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b6d68-2278-4006-8635-5a257539b60a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4a541c4e769448ba1a8d194c9b33a27, user_id=a3dcdf84f381468c8cea281d0a451075, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:16.872727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b6d68-2278-4006-8635-5a257539b60a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:16.874188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701aae7-0ba0-41ea-9d3e-e71545dbc3b8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:16.880159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851cf5b-a962-48bd-8e6a-64e4d86a9482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:16.881942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851cf5b-a962-48bd-8e6a-64e4d86a9482 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:16.882954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851cf5b-a962-48bd-8e6a-64e4d86a9482 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:16.883312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851cf5b-a962-48bd-8e6a-64e4d86a9482 admin admin] PATH_INFO: `/v3/projects/890469b56ae04aa98dc5c0c406a6cd01/users/4918ffab682b4aefaa6180bc9e4aa6c3/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:16.897551 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1148/2283] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:16 2022] PUT /identity/v3/projects/a4a541c4e769448ba1a8d194c9b33a27/users/a3dcdf84f381468c8cea281d0a451075/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:16.899045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224631b1-5c23-47f2-91aa-b064b1eb7a1f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:16.899403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224631b1-5c23-47f2-91aa-b064b1eb7a1f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:16.899688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224631b1-5c23-47f2-91aa-b064b1eb7a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:16.909817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851cf5b-a962-48bd-8e6a-64e4d86a9482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=890469b56ae04aa98dc5c0c406a6cd01, user_id=4918ffab682b4aefaa6180bc9e4aa6c3, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:16.913048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851cf5b-a962-48bd-8e6a-64e4d86a9482 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:16.919831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:16.921174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-224631b1-5c23-47f2-91aa-b064b1eb7a1f None None] MFA Rules not processed for user `fc6ee6dba5d24e90bff48384539fcd7f`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:16.922453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-224631b1-5c23-47f2-91aa-b064b1eb7a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:16.995234 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1149/2284] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:16.996870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842a7af-968d-484f-b8b4-fe8920a343b1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:16.997227 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1136/2285] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:16 2022] PUT /identity/v3/projects/890469b56ae04aa98dc5c0c406a6cd01/users/4918ffab682b4aefaa6180bc9e4aa6c3/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:16.997340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842a7af-968d-484f-b8b4-fe8920a343b1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:16.997633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842a7af-968d-484f-b8b4-fe8920a343b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:17.007514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca43317-59cb-4978-8f42-723426ad942b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:17.008035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca43317-59cb-4978-8f42-723426ad942b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:17.008473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca43317-59cb-4978-8f42-723426ad942b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:17.028972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3842a7af-968d-484f-b8b4-fe8920a343b1 None None] MFA Rules not processed for user `a3dcdf84f381468c8cea281d0a451075`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:17.030410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3842a7af-968d-484f-b8b4-fe8920a343b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:17.041583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-aca43317-59cb-4978-8f42-723426ad942b None None] MFA Rules not processed for user `4918ffab682b4aefaa6180bc9e4aa6c3`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:17.042883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-aca43317-59cb-4978-8f42-723426ad942b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:17.117460 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1150/2286] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:16 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:17.135721 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1137/2287] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:04:17.140061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842a7af-968d-484f-b8b4-fe8920a343b1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:17.140216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca43317-59cb-4978-8f42-723426ad942b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:17.144453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3842a7af-968d-484f-b8b4-fe8920a343b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:17.225055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca43317-59cb-4978-8f42-723426ad942b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:17.229807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:17.233863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a62874f-0dc1-41f2-afd1-f400933f5312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:17.234665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62874f-0dc1-41f2-afd1-f400933f5312 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:17.235054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62874f-0dc1-41f2-afd1-f400933f5312 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:17.235602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a62874f-0dc1-41f2-afd1-f400933f5312 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:17.236406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a62874f-0dc1-41f2-afd1-f400933f5312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:17.236988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842a7af-968d-484f-b8b4-fe8920a343b1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:17.239441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a62874f-0dc1-41f2-afd1-f400933f5312 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:17.244634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20279e-ac89-4424-b9b5-39c03e02f447 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:17.245750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20279e-ac89-4424-b9b5-39c03e02f447 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:17.246129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20279e-ac89-4424-b9b5-39c03e02f447 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:17.246414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d20279e-ac89-4424-b9b5-39c03e02f447 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:17.256839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20279e-ac89-4424-b9b5-39c03e02f447 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:17.258407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d20279e-ac89-4424-b9b5-39c03e02f447 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:17.269433 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1151/2288] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:17.357527 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1138/2289] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:17 2022] POST /identity/v3/projects => generated 377 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:04:17.359166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20279e-ac89-4424-b9b5-39c03e02f447 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:17.383411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f8967-ea6b-4c15-9e5c-8f1b1b3ebe2d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:17.385819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f8967-ea6b-4c15-9e5c-8f1b1b3ebe2d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:17.385819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f8967-ea6b-4c15-9e5c-8f1b1b3ebe2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:17.387252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d20279e-ac89-4424-b9b5-39c03e02f447 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:17.446062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bb6f8967-ea6b-4c15-9e5c-8f1b1b3ebe2d None None] MFA Rules not processed for user `7b44b524592e4a57a26681767b9c4541`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:17.447320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bb6f8967-ea6b-4c15-9e5c-8f1b1b3ebe2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:17.518199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133500c-a54a-4a01-af6e-f1aba9843271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:17.518199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133500c-a54a-4a01-af6e-f1aba9843271 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:17.518773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133500c-a54a-4a01-af6e-f1aba9843271 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:17.518773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133500c-a54a-4a01-af6e-f1aba9843271 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:17.519254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133500c-a54a-4a01-af6e-f1aba9843271 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:17.524368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c133500c-a54a-4a01-af6e-f1aba9843271 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:17.564767 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1139/2290] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:17.571505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f8967-ea6b-4c15-9e5c-8f1b1b3ebe2d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:17.573147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bb6f8967-ea6b-4c15-9e5c-8f1b1b3ebe2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:17.600104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f8967-ea6b-4c15-9e5c-8f1b1b3ebe2d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:17.606143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b1c2d-8417-4ec1-a99b-e64e92bda469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:17.607129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b1c2d-8417-4ec1-a99b-e64e92bda469 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:17.607586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b1c2d-8417-4ec1-a99b-e64e92bda469 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:17.608014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b1c2d-8417-4ec1-a99b-e64e92bda469 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:17.609027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b1c2d-8417-4ec1-a99b-e64e92bda469 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:17.610038 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1152/2291] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:17 2022] POST /identity/v3/users => generated 295 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:04:17.611607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b1c2d-8417-4ec1-a99b-e64e92bda469 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:17.612667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133500c-a54a-4a01-af6e-f1aba9843271 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:17.624837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133500c-a54a-4a01-af6e-f1aba9843271 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:17.629978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802ae6b-ab7e-4e45-a53b-96d3df81e19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:17.630639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802ae6b-ab7e-4e45-a53b-96d3df81e19f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:17.631043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802ae6b-ab7e-4e45-a53b-96d3df81e19f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:17.631327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3802ae6b-ab7e-4e45-a53b-96d3df81e19f admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:17.632065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3802ae6b-ab7e-4e45-a53b-96d3df81e19f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:17.633685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3802ae6b-ab7e-4e45-a53b-96d3df81e19f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:17.645487 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1153/2292] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:04:17.649455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802ae6b-ab7e-4e45-a53b-96d3df81e19f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:17.659632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3802ae6b-ab7e-4e45-a53b-96d3df81e19f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:17.668231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa814c28-b609-4bf7-8666-0d7817c86ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:17.669123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa814c28-b609-4bf7-8666-0d7817c86ca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:17.669414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa814c28-b609-4bf7-8666-0d7817c86ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:17.669696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa814c28-b609-4bf7-8666-0d7817c86ca6 admin admin] PATH_INFO: `/v3/projects/99c01e770b5948d4b50929145a205280/users/29ef88bde23941d6a45cd4e2eb0c720d/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:17.685741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa814c28-b609-4bf7-8666-0d7817c86ca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c01e770b5948d4b50929145a205280, user_id=29ef88bde23941d6a45cd4e2eb0c720d, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:17.688536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa814c28-b609-4bf7-8666-0d7817c86ca6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:17.715513 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1140/2293] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:17 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:17.719331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b1c2d-8417-4ec1-a99b-e64e92bda469 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:17.782030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b1c2d-8417-4ec1-a99b-e64e92bda469 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:17.786767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f64f9-3597-4ab3-9b1e-c8466b775945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:17.788038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f64f9-3597-4ab3-9b1e-c8466b775945 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:17.788315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f64f9-3597-4ab3-9b1e-c8466b775945 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:17.788589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f64f9-3597-4ab3-9b1e-c8466b775945 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:17.789272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f64f9-3597-4ab3-9b1e-c8466b775945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:17.791230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24f64f9-3597-4ab3-9b1e-c8466b775945 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:17.800322 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1154/2294] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:17 2022] PUT /identity/v3/projects/99c01e770b5948d4b50929145a205280/users/29ef88bde23941d6a45cd4e2eb0c720d/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:17.804403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa814c28-b609-4bf7-8666-0d7817c86ca6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:17.867864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa814c28-b609-4bf7-8666-0d7817c86ca6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:17.873063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6312242-e026-429c-bc11-27b43f01e1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:17.873731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6312242-e026-429c-bc11-27b43f01e1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:17.874054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6312242-e026-429c-bc11-27b43f01e1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:17.874336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6312242-e026-429c-bc11-27b43f01e1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:17.875069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6312242-e026-429c-bc11-27b43f01e1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:17.876674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6312242-e026-429c-bc11-27b43f01e1f1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:17.889917 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1155/2295] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:04:17.891995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6312242-e026-429c-bc11-27b43f01e1f1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:17.903593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6312242-e026-429c-bc11-27b43f01e1f1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:17.916183 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1141/2296] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:17 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:04:17.920211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcd017-554d-431b-b51c-373a302c008c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:17.920211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcd017-554d-431b-b51c-373a302c008c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:17.920211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcd017-554d-431b-b51c-373a302c008c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:17.920871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcd017-554d-431b-b51c-373a302c008c admin admin] PATH_INFO: `/v3/projects/99c01e770b5948d4b50929145a205280/users/29ef88bde23941d6a45cd4e2eb0c720d/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:17.922386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcd017-554d-431b-b51c-373a302c008c admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c01e770b5948d4b50929145a205280, user_id=29ef88bde23941d6a45cd4e2eb0c720d, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:17.925259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcd017-554d-431b-b51c-373a302c008c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:17.927845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f64f9-3597-4ab3-9b1e-c8466b775945 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:17.952745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f64f9-3597-4ab3-9b1e-c8466b775945 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:17.957925 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1156/2297] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:17 2022] PUT /identity/v3/projects/99c01e770b5948d4b50929145a205280/users/29ef88bde23941d6a45cd4e2eb0c720d/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:04:17.966618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96d5f7-c28f-4859-82e6-cbac78466c0e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:17.966618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96d5f7-c28f-4859-82e6-cbac78466c0e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:17.966859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96d5f7-c28f-4859-82e6-cbac78466c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:18.008479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-dd96d5f7-c28f-4859-82e6-cbac78466c0e None None] MFA Rules not processed for user `29ef88bde23941d6a45cd4e2eb0c720d`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:18.020457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dd96d5f7-c28f-4859-82e6-cbac78466c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:18.113522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f7ba9-6a09-4736-aac1-9b5e1e634a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:18.114742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f7ba9-6a09-4736-aac1-9b5e1e634a72 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:18.115908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f7ba9-6a09-4736-aac1-9b5e1e634a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:18.116880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f7ba9-6a09-4736-aac1-9b5e1e634a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:18.118443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f7ba9-6a09-4736-aac1-9b5e1e634a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:18.122549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f7ba9-6a09-4736-aac1-9b5e1e634a72 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:18.139859 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1142/2298] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:18.152456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f7ba9-6a09-4736-aac1-9b5e1e634a72 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:18.163863 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1157/2299] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:04:18.170697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f7ba9-6a09-4736-aac1-9b5e1e634a72 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:18.176052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fe503-013e-48b6-944e-7ee42154b22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:18.177563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fe503-013e-48b6-944e-7ee42154b22b admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:18.178405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fe503-013e-48b6-944e-7ee42154b22b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:18.178703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936fe503-013e-48b6-944e-7ee42154b22b admin admin] PATH_INFO: `/v3/projects/43002fcef42c46b2aa326343c12a4126/users/f97b8d315694418db9c5915dc2c34c71/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:18.199507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936fe503-013e-48b6-944e-7ee42154b22b admin admin] RBAC: Authorizing `identity:create_grant(project_id=43002fcef42c46b2aa326343c12a4126, user_id=f97b8d315694418db9c5915dc2c34c71, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:18.201127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936fe503-013e-48b6-944e-7ee42154b22b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:18.276024 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1143/2300] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:18 2022] PUT /identity/v3/projects/43002fcef42c46b2aa326343c12a4126/users/f97b8d315694418db9c5915dc2c34c71/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:18.279913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96d5f7-c28f-4859-82e6-cbac78466c0e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:18.292668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96d5f7-c28f-4859-82e6-cbac78466c0e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:18.365856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99ffd4-59e7-46a7-b614-61174d9c7169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:18.366630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99ffd4-59e7-46a7-b614-61174d9c7169 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:18.366915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99ffd4-59e7-46a7-b614-61174d9c7169 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:18.367252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99ffd4-59e7-46a7-b614-61174d9c7169 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:18.368248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99ffd4-59e7-46a7-b614-61174d9c7169 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:18.373074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99ffd4-59e7-46a7-b614-61174d9c7169 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:18.385903 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1158/2301] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:18.389788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fe503-013e-48b6-944e-7ee42154b22b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:18.402612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936fe503-013e-48b6-944e-7ee42154b22b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:18.407697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaea955e-89be-47a2-9f04-3e9adcc47adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:18.408642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea955e-89be-47a2-9f04-3e9adcc47adf admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:18.408925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea955e-89be-47a2-9f04-3e9adcc47adf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:18.409197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaea955e-89be-47a2-9f04-3e9adcc47adf admin admin] PATH_INFO: `/v3/projects/43002fcef42c46b2aa326343c12a4126/users/f97b8d315694418db9c5915dc2c34c71/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:18.411756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea955e-89be-47a2-9f04-3e9adcc47adf admin admin] RBAC: Authorizing `identity:create_grant(project_id=43002fcef42c46b2aa326343c12a4126, user_id=f97b8d315694418db9c5915dc2c34c71, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:18.415983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaea955e-89be-47a2-9f04-3e9adcc47adf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:18.434213 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1144/2302] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:18 2022] PUT /identity/v3/projects/43002fcef42c46b2aa326343c12a4126/users/f97b8d315694418db9c5915dc2c34c71/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:18.475807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:18.658589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99ffd4-59e7-46a7-b614-61174d9c7169 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:18.661624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fa99ffd4-59e7-46a7-b614-61174d9c7169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:18.779473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99ffd4-59e7-46a7-b614-61174d9c7169 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:18.822243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:18.844088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03741ff-6a67-4adc-806d-e2f4da41a7da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:18.844743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03741ff-6a67-4adc-806d-e2f4da41a7da service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:18.845030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03741ff-6a67-4adc-806d-e2f4da41a7da service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:18.845308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03741ff-6a67-4adc-806d-e2f4da41a7da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:18.904367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03741ff-6a67-4adc-806d-e2f4da41a7da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:18.906048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03741ff-6a67-4adc-806d-e2f4da41a7da service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:18.916015 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1159/2303] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:19.731792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353f74c-48a1-4c68-a400-46e2b6c343a8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:19.732311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353f74c-48a1-4c68-a400-46e2b6c343a8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:19.732393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3353f74c-48a1-4c68-a400-46e2b6c343a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:19.743039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:19.766240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3353f74c-48a1-4c68-a400-46e2b6c343a8 None None] MFA Rules not processed for user `f97b8d315694418db9c5915dc2c34c71`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:19.767832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3353f74c-48a1-4c68-a400-46e2b6c343a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:19.879927 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1145/2304] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:19 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:19.891644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7caf4-3a15-46fb-9be4-c75cabf24529 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:19.892273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7caf4-3a15-46fb-9be4-c75cabf24529 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:19.892559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7caf4-3a15-46fb-9be4-c75cabf24529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:19.911067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a5d7caf4-3a15-46fb-9be4-c75cabf24529 None None] MFA Rules not processed for user `7b44b524592e4a57a26681767b9c4541`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:19.912924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a5d7caf4-3a15-46fb-9be4-c75cabf24529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:19.960934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353f74c-48a1-4c68-a400-46e2b6c343a8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:19.998637 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1160/2305] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:04:20.012397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7caf4-3a15-46fb-9be4-c75cabf24529 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:20.023760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:20.026969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7caf4-3a15-46fb-9be4-c75cabf24529 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:20.032723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01208fe7-9092-4216-93cb-379a19f3b151 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:20.033386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01208fe7-9092-4216-93cb-379a19f3b151 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:20.033672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01208fe7-9092-4216-93cb-379a19f3b151 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:20.033992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01208fe7-9092-4216-93cb-379a19f3b151 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:20.042970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01208fe7-9092-4216-93cb-379a19f3b151 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:20.044625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01208fe7-9092-4216-93cb-379a19f3b151 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:20.054421 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1161/2306] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:20.055825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3353f74c-48a1-4c68-a400-46e2b6c343a8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:20.060991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295cc7d-0af2-4b3f-98f5-036589638659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:20.062845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295cc7d-0af2-4b3f-98f5-036589638659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:20.066029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295cc7d-0af2-4b3f-98f5-036589638659 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:20.066029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295cc7d-0af2-4b3f-98f5-036589638659 admin admin] PATH_INFO: `/v3/users/7204868cbb0a49aa83d9ccb85e610ea9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:20.066029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295cc7d-0af2-4b3f-98f5-036589638659 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7204868cbb0a49aa83d9ccb85e610ea9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:20.067590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295cc7d-0af2-4b3f-98f5-036589638659 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:20.338876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c295cc7d-0af2-4b3f-98f5-036589638659 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7204868cbb0a49aa83d9ccb85e610ea9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:20.340044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c295cc7d-0af2-4b3f-98f5-036589638659 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7204868cbb0a49aa83d9ccb85e610ea9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:20.348952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c295cc7d-0af2-4b3f-98f5-036589638659 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7204868cbb0a49aa83d9ccb85e610ea9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:20.473399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c295cc7d-0af2-4b3f-98f5-036589638659 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7204868cbb0a49aa83d9ccb85e610ea9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:20.474688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c295cc7d-0af2-4b3f-98f5-036589638659 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7204868cbb0a49aa83d9ccb85e610ea9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:20.524930 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1146/2307] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:19 2022] DELETE /identity/v3/users/7204868cbb0a49aa83d9ccb85e610ea9 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:20.529552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24d635-18eb-45a0-b5dd-e68916d208b5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:20.530147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24d635-18eb-45a0-b5dd-e68916d208b5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:20.530609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24d635-18eb-45a0-b5dd-e68916d208b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:20.570004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-bb24d635-18eb-45a0-b5dd-e68916d208b5 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:20.572390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bb24d635-18eb-45a0-b5dd-e68916d208b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:20.668720 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1162/2308] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:20.676101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295cc7d-0af2-4b3f-98f5-036589638659 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:20.678365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c295cc7d-0af2-4b3f-98f5-036589638659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:20.683211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:20.807235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295cc7d-0af2-4b3f-98f5-036589638659 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:20.813208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932d488-a14f-427e-a880-54c964a018f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:20.814272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932d488-a14f-427e-a880-54c964a018f6 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:20.814441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932d488-a14f-427e-a880-54c964a018f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:20.814853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0932d488-a14f-427e-a880-54c964a018f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:20.826536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932d488-a14f-427e-a880-54c964a018f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:20.828713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0932d488-a14f-427e-a880-54c964a018f6 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:20.845574 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1147/2309] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:21.188829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:21.279711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24d635-18eb-45a0-b5dd-e68916d208b5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:21.281232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bb24d635-18eb-45a0-b5dd-e68916d208b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:21.318878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24d635-18eb-45a0-b5dd-e68916d208b5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:21.332542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c5465-51a9-4752-92b8-555e1fed671f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:21.333472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c5465-51a9-4752-92b8-555e1fed671f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:21.333997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c5465-51a9-4752-92b8-555e1fed671f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:21.334291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332c5465-51a9-4752-92b8-555e1fed671f admin admin] PATH_INFO: `/v3/users/4918ffab682b4aefaa6180bc9e4aa6c3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:21.335738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c5465-51a9-4752-92b8-555e1fed671f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4918ffab682b4aefaa6180bc9e4aa6c3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:21.337700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332c5465-51a9-4752-92b8-555e1fed671f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:21.636120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-332c5465-51a9-4752-92b8-555e1fed671f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4918ffab682b4aefaa6180bc9e4aa6c3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:21.658785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932d488-a14f-427e-a880-54c964a018f6 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:21.658785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0932d488-a14f-427e-a880-54c964a018f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:21.673283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-332c5465-51a9-4752-92b8-555e1fed671f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4918ffab682b4aefaa6180bc9e4aa6c3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:21.679562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-332c5465-51a9-4752-92b8-555e1fed671f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4918ffab682b4aefaa6180bc9e4aa6c3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:21.693615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-332c5465-51a9-4752-92b8-555e1fed671f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4918ffab682b4aefaa6180bc9e4aa6c3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:21.762839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0932d488-a14f-427e-a880-54c964a018f6 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:21.768591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d68d9-e2e3-40cb-91c6-5ed335fbf26a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:21.769447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d68d9-e2e3-40cb-91c6-5ed335fbf26a service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:21.769908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d68d9-e2e3-40cb-91c6-5ed335fbf26a service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:21.770328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d68d9-e2e3-40cb-91c6-5ed335fbf26a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:21.772841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5c5d68d9-e2e3-40cb-91c6-5ed335fbf26a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:21.817577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-332c5465-51a9-4752-92b8-555e1fed671f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4918ffab682b4aefaa6180bc9e4aa6c3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:21.847845 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1163/2310] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:21 2022] DELETE /identity/v3/users/4918ffab682b4aefaa6180bc9e4aa6c3 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:21.848805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c5465-51a9-4752-92b8-555e1fed671f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:21.851064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d68d9-e2e3-40cb-91c6-5ed335fbf26a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:21.853870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-332c5465-51a9-4752-92b8-555e1fed671f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:21.856581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d68d9-e2e3-40cb-91c6-5ed335fbf26a service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:21.873433 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1148/2311] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:04:21.875775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0a162-647e-4d83-910c-43ec72ede414 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:21.876222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0a162-647e-4d83-910c-43ec72ede414 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:21.876708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d0a162-647e-4d83-910c-43ec72ede414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:21.901087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-25d0a162-647e-4d83-910c-43ec72ede414 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:21.903828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-25d0a162-647e-4d83-910c-43ec72ede414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:22.004430 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1149/2312] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:22.046066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0a162-647e-4d83-910c-43ec72ede414 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:22.047645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-25d0a162-647e-4d83-910c-43ec72ede414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:22.084958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332c5465-51a9-4752-92b8-555e1fed671f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:22.104693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119c4b3-eb37-48fa-bd60-2163368c4474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:22.104693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119c4b3-eb37-48fa-bd60-2163368c4474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:22.104693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119c4b3-eb37-48fa-bd60-2163368c4474 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:22.104693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5119c4b3-eb37-48fa-bd60-2163368c4474 admin admin] PATH_INFO: `/v3/projects/065942f484d14140827befe78c108930` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:22.105065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119c4b3-eb37-48fa-bd60-2163368c4474 admin admin] RBAC: Authorizing `identity:delete_project(project_id=065942f484d14140827befe78c108930)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:22.105065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5119c4b3-eb37-48fa-bd60-2163368c4474 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:22.174761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d0a162-647e-4d83-910c-43ec72ede414 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:22.186621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796136f-25f4-4679-8ae7-f4d2d3e36a26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:22.187395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796136f-25f4-4679-8ae7-f4d2d3e36a26 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:22.188516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796136f-25f4-4679-8ae7-f4d2d3e36a26 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:22.188516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5796136f-25f4-4679-8ae7-f4d2d3e36a26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:22.198982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796136f-25f4-4679-8ae7-f4d2d3e36a26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:22.200898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5796136f-25f4-4679-8ae7-f4d2d3e36a26 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:22.221129 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1150/2313] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:04:22.221356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796136f-25f4-4679-8ae7-f4d2d3e36a26 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:22.227944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5796136f-25f4-4679-8ae7-f4d2d3e36a26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:22.241342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5796136f-25f4-4679-8ae7-f4d2d3e36a26 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:22.276881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbb720-f4b7-40db-9b10-f57eb608e0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:22.277803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbb720-f4b7-40db-9b10-f57eb608e0b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:22.278141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbb720-f4b7-40db-9b10-f57eb608e0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:22.278420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbb720-f4b7-40db-9b10-f57eb608e0b1 admin admin] PATH_INFO: `/v3/users/0e733cc9497a48cba4ccdde860be104f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:22.279861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbb720-f4b7-40db-9b10-f57eb608e0b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e733cc9497a48cba4ccdde860be104f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:22.282949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbb720-f4b7-40db-9b10-f57eb608e0b1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:22.289738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5119c4b3-eb37-48fa-bd60-2163368c4474 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '065942f484d14140827befe78c108930', 'request_id': 'req-5119c4b3-eb37-48fa-bd60-2163368c4474'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:22.301370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5119c4b3-eb37-48fa-bd60-2163368c4474 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '065942f484d14140827befe78c108930', 'request_id': 'req-5119c4b3-eb37-48fa-bd60-2163368c4474'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:22.399731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5119c4b3-eb37-48fa-bd60-2163368c4474 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '065942f484d14140827befe78c108930', 'request_id': 'req-5119c4b3-eb37-48fa-bd60-2163368c4474'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:22.403641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:22.420439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5119c4b3-eb37-48fa-bd60-2163368c4474 admin admin] The token cache is being invalidate because project 065942f484d14140827befe78c108930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:22.421182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5119c4b3-eb37-48fa-bd60-2163368c4474 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:22.423779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5119c4b3-eb37-48fa-bd60-2163368c4474 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:22.433507 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1164/2314] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:21 2022] DELETE /identity/v3/projects/065942f484d14140827befe78c108930 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:22.438689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119c4b3-eb37-48fa-bd60-2163368c4474 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:22.440917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5119c4b3-eb37-48fa-bd60-2163368c4474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:22.514731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7dcbb720-f4b7-40db-9b10-f57eb608e0b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e733cc9497a48cba4ccdde860be104f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:22.516099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7dcbb720-f4b7-40db-9b10-f57eb608e0b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e733cc9497a48cba4ccdde860be104f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:22.531451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7dcbb720-f4b7-40db-9b10-f57eb608e0b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e733cc9497a48cba4ccdde860be104f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:22.537875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5119c4b3-eb37-48fa-bd60-2163368c4474 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:22.543830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705807c-bbed-4b38-bcd3-da5d68c29de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:22.545116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705807c-bbed-4b38-bcd3-da5d68c29de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:22.545678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705807c-bbed-4b38-bcd3-da5d68c29de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:22.547243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6705807c-bbed-4b38-bcd3-da5d68c29de4 admin admin] PATH_INFO: `/v3/users/a3dcdf84f381468c8cea281d0a451075` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:22.551570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6705807c-bbed-4b38-bcd3-da5d68c29de4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3dcdf84f381468c8cea281d0a451075)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:22.552296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6705807c-bbed-4b38-bcd3-da5d68c29de4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:22.598602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7dcbb720-f4b7-40db-9b10-f57eb608e0b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e733cc9497a48cba4ccdde860be104f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:22.599365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7dcbb720-f4b7-40db-9b10-f57eb608e0b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e733cc9497a48cba4ccdde860be104f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:22.627606 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1151/2315] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:22 2022] DELETE /identity/v3/users/0e733cc9497a48cba4ccdde860be104f => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:22.633891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b281c-8ff1-4dcf-b3cf-05975a7a9d55 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:22.634650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b281c-8ff1-4dcf-b3cf-05975a7a9d55 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:22.635163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b281c-8ff1-4dcf-b3cf-05975a7a9d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:22.660077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b39b281c-8ff1-4dcf-b3cf-05975a7a9d55 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:22.661135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b39b281c-8ff1-4dcf-b3cf-05975a7a9d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:22.781083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6705807c-bbed-4b38-bcd3-da5d68c29de4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3dcdf84f381468c8cea281d0a451075'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:22.799005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6705807c-bbed-4b38-bcd3-da5d68c29de4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3dcdf84f381468c8cea281d0a451075'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:22.800291 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1152/2316] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:22.803588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6705807c-bbed-4b38-bcd3-da5d68c29de4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3dcdf84f381468c8cea281d0a451075'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:22.804555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6705807c-bbed-4b38-bcd3-da5d68c29de4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3dcdf84f381468c8cea281d0a451075'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:22.820963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b281c-8ff1-4dcf-b3cf-05975a7a9d55 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:22.823251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b39b281c-8ff1-4dcf-b3cf-05975a7a9d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:22.896448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b281c-8ff1-4dcf-b3cf-05975a7a9d55 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:22.900675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e08bc-f0b0-419b-bf5c-368af877b36f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:22.901397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e08bc-f0b0-419b-bf5c-368af877b36f service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:22.901753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e08bc-f0b0-419b-bf5c-368af877b36f service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:22.902172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e08bc-f0b0-419b-bf5c-368af877b36f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:22.904418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-229e08bc-f0b0-419b-bf5c-368af877b36f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:22.907672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6705807c-bbed-4b38-bcd3-da5d68c29de4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3dcdf84f381468c8cea281d0a451075'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:22.921794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e08bc-f0b0-419b-bf5c-368af877b36f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:22.923845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e08bc-f0b0-419b-bf5c-368af877b36f service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:22.934633 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1165/2317] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:22 2022] DELETE /identity/v3/users/a3dcdf84f381468c8cea281d0a451075 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:22.937056 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1153/2318] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:04:23.073463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:23.439188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705807c-bbed-4b38-bcd3-da5d68c29de4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:23.441399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6705807c-bbed-4b38-bcd3-da5d68c29de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:23.524620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e08bc-f0b0-419b-bf5c-368af877b36f service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:23.526113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-229e08bc-f0b0-419b-bf5c-368af877b36f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:23.548592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6705807c-bbed-4b38-bcd3-da5d68c29de4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:23.552140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e08bc-f0b0-419b-bf5c-368af877b36f service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:23.554663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcd7950-1cb8-47d3-b4ef-6987a65cc86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:23.555807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd7950-1cb8-47d3-b4ef-6987a65cc86e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:23.556340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd7950-1cb8-47d3-b4ef-6987a65cc86e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:23.556895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd7950-1cb8-47d3-b4ef-6987a65cc86e admin admin] PATH_INFO: `/v3/projects/b6c7378af3e841f58fdf1a907d06bc08` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:23.559814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd7950-1cb8-47d3-b4ef-6987a65cc86e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6c7378af3e841f58fdf1a907d06bc08)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:23.563227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcd7950-1cb8-47d3-b4ef-6987a65cc86e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:23.569380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6e170-0ac5-47cd-9c01-ad53654e0257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:23.570316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6e170-0ac5-47cd-9c01-ad53654e0257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:23.570975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6e170-0ac5-47cd-9c01-ad53654e0257 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:23.570975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6e170-0ac5-47cd-9c01-ad53654e0257 admin admin] PATH_INFO: `/v3/projects/a4a541c4e769448ba1a8d194c9b33a27` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:23.574955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6e170-0ac5-47cd-9c01-ad53654e0257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4a541c4e769448ba1a8d194c9b33a27)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:23.579132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6e170-0ac5-47cd-9c01-ad53654e0257 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:23.742682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2dcd7950-1cb8-47d3-b4ef-6987a65cc86e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6c7378af3e841f58fdf1a907d06bc08', 'request_id': 'req-2dcd7950-1cb8-47d3-b4ef-6987a65cc86e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:23.743093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aca6e170-0ac5-47cd-9c01-ad53654e0257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4a541c4e769448ba1a8d194c9b33a27', 'request_id': 'req-aca6e170-0ac5-47cd-9c01-ad53654e0257'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:23.750009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aca6e170-0ac5-47cd-9c01-ad53654e0257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4a541c4e769448ba1a8d194c9b33a27', 'request_id': 'req-aca6e170-0ac5-47cd-9c01-ad53654e0257'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:23.772925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2dcd7950-1cb8-47d3-b4ef-6987a65cc86e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6c7378af3e841f58fdf1a907d06bc08', 'request_id': 'req-2dcd7950-1cb8-47d3-b4ef-6987a65cc86e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:23.794140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aca6e170-0ac5-47cd-9c01-ad53654e0257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4a541c4e769448ba1a8d194c9b33a27', 'request_id': 'req-aca6e170-0ac5-47cd-9c01-ad53654e0257'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:23.809765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aca6e170-0ac5-47cd-9c01-ad53654e0257 admin admin] The token cache is being invalidate because project a4a541c4e769448ba1a8d194c9b33a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:23.810221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aca6e170-0ac5-47cd-9c01-ad53654e0257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:23.810871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aca6e170-0ac5-47cd-9c01-ad53654e0257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:23.812495 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1154/2319] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:23 2022] DELETE /identity/v3/projects/a4a541c4e769448ba1a8d194c9b33a27 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:23.813681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6e170-0ac5-47cd-9c01-ad53654e0257 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:23.814853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-aca6e170-0ac5-47cd-9c01-ad53654e0257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:23.924061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6e170-0ac5-47cd-9c01-ad53654e0257 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:23.965278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2dcd7950-1cb8-47d3-b4ef-6987a65cc86e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6c7378af3e841f58fdf1a907d06bc08', 'request_id': 'req-2dcd7950-1cb8-47d3-b4ef-6987a65cc86e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:23.984983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2dcd7950-1cb8-47d3-b4ef-6987a65cc86e admin admin] The token cache is being invalidate because project b6c7378af3e841f58fdf1a907d06bc08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:23.985414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2dcd7950-1cb8-47d3-b4ef-6987a65cc86e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:23.986032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2dcd7950-1cb8-47d3-b4ef-6987a65cc86e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:23.987995 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1166/2320] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:23 2022] DELETE /identity/v3/projects/b6c7378af3e841f58fdf1a907d06bc08 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:23.990880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d203061-9f00-45c2-8c75-90bad6084a4d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:23.991271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d203061-9f00-45c2-8c75-90bad6084a4d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:23.991441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d203061-9f00-45c2-8c75-90bad6084a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:24.015643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1610d9-89bd-4973-a807-3c04fb48da67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:24.016991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1610d9-89bd-4973-a807-3c04fb48da67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:24.017477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1610d9-89bd-4973-a807-3c04fb48da67 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:24.018030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1610d9-89bd-4973-a807-3c04fb48da67 admin admin] PATH_INFO: `/v3/projects/890469b56ae04aa98dc5c0c406a6cd01` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:24.018483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5d203061-9f00-45c2-8c75-90bad6084a4d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:24.019867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5d203061-9f00-45c2-8c75-90bad6084a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:24.020578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1610d9-89bd-4973-a807-3c04fb48da67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=890469b56ae04aa98dc5c0c406a6cd01)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:24.023285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1610d9-89bd-4973-a807-3c04fb48da67 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:24.038941 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1167/2321] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:24.039852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d203061-9f00-45c2-8c75-90bad6084a4d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:24.041909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5d203061-9f00-45c2-8c75-90bad6084a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:24.072123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d203061-9f00-45c2-8c75-90bad6084a4d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:24.077651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c65f4-9332-492a-b119-3fe1003551d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:24.078596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c65f4-9332-492a-b119-3fe1003551d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:24.078883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c65f4-9332-492a-b119-3fe1003551d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:24.079158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c65f4-9332-492a-b119-3fe1003551d6 admin admin] PATH_INFO: `/v3/users/a943a5a756c241379440c3ad9b937435` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:24.080428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c65f4-9332-492a-b119-3fe1003551d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a943a5a756c241379440c3ad9b937435)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:24.082650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c65f4-9332-492a-b119-3fe1003551d6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:24.188860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:24.205357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0b1610d9-89bd-4973-a807-3c04fb48da67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '890469b56ae04aa98dc5c0c406a6cd01', 'request_id': 'req-0b1610d9-89bd-4973-a807-3c04fb48da67'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:24.221172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0b1610d9-89bd-4973-a807-3c04fb48da67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '890469b56ae04aa98dc5c0c406a6cd01', 'request_id': 'req-0b1610d9-89bd-4973-a807-3c04fb48da67'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:24.318534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0b1610d9-89bd-4973-a807-3c04fb48da67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '890469b56ae04aa98dc5c0c406a6cd01', 'request_id': 'req-0b1610d9-89bd-4973-a807-3c04fb48da67'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:24.334591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b11c65f4-9332-492a-b119-3fe1003551d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a943a5a756c241379440c3ad9b937435'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:24.358005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b11c65f4-9332-492a-b119-3fe1003551d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a943a5a756c241379440c3ad9b937435'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:24.358785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b11c65f4-9332-492a-b119-3fe1003551d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a943a5a756c241379440c3ad9b937435'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:24.359577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b11c65f4-9332-492a-b119-3fe1003551d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a943a5a756c241379440c3ad9b937435'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:24.370362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0b1610d9-89bd-4973-a807-3c04fb48da67 admin admin] The token cache is being invalidate because project 890469b56ae04aa98dc5c0c406a6cd01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:24.372405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0b1610d9-89bd-4973-a807-3c04fb48da67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:24.375973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0b1610d9-89bd-4973-a807-3c04fb48da67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:24.378148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:24.381824 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1155/2322] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:23 2022] DELETE /identity/v3/projects/890469b56ae04aa98dc5c0c406a6cd01 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:24.383927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540104c-48e6-4bcd-affd-4e9fa4619c1f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:24.384928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540104c-48e6-4bcd-affd-4e9fa4619c1f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:24.385423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540104c-48e6-4bcd-affd-4e9fa4619c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:24.437525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f540104c-48e6-4bcd-affd-4e9fa4619c1f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:24.438952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f540104c-48e6-4bcd-affd-4e9fa4619c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:24.451677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b11c65f4-9332-492a-b119-3fe1003551d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a943a5a756c241379440c3ad9b937435'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:24.493099 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1168/2323] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:24 2022] DELETE /identity/v3/users/a943a5a756c241379440c3ad9b937435 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:24.506104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c65f4-9332-492a-b119-3fe1003551d6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:24.514995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b11c65f4-9332-492a-b119-3fe1003551d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:24.555279 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1156/2324] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:24.561577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540104c-48e6-4bcd-affd-4e9fa4619c1f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:24.577438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540104c-48e6-4bcd-affd-4e9fa4619c1f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:24.652571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c65f4-9332-492a-b119-3fe1003551d6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:24.657530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:24.681275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28342007-f385-49ce-a9de-59e63f7adee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:24.682688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28342007-f385-49ce-a9de-59e63f7adee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:24.683153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28342007-f385-49ce-a9de-59e63f7adee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:24.684398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28342007-f385-49ce-a9de-59e63f7adee1 admin admin] PATH_INFO: `/v3/users/29ef88bde23941d6a45cd4e2eb0c720d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:24.687226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28342007-f385-49ce-a9de-59e63f7adee1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29ef88bde23941d6a45cd4e2eb0c720d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:24.690158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28342007-f385-49ce-a9de-59e63f7adee1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:24.734478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc6959-cec1-413c-9f32-f2c00a903748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:24.735170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc6959-cec1-413c-9f32-f2c00a903748 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:24.735456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc6959-cec1-413c-9f32-f2c00a903748 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:24.735734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fc6959-cec1-413c-9f32-f2c00a903748 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:24.736488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc6959-cec1-413c-9f32-f2c00a903748 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:24.736805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc6959-cec1-413c-9f32-f2c00a903748 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:24.745239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fc6959-cec1-413c-9f32-f2c00a903748 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:24.770697 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1157/2325] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:24.772591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc6959-cec1-413c-9f32-f2c00a903748 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:24.783084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fc6959-cec1-413c-9f32-f2c00a903748 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:24.797694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555376f-6582-4977-95d8-6f825dc85871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:24.798666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555376f-6582-4977-95d8-6f825dc85871 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:24.799109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555376f-6582-4977-95d8-6f825dc85871 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:24.799539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555376f-6582-4977-95d8-6f825dc85871 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:24.800400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555376f-6582-4977-95d8-6f825dc85871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:24.803448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555376f-6582-4977-95d8-6f825dc85871 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:24.901618 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1158/2326] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:24 2022] POST /identity/v3/projects => generated 373 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:24.905813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555376f-6582-4977-95d8-6f825dc85871 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:24.958350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-28342007-f385-49ce-a9de-59e63f7adee1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29ef88bde23941d6a45cd4e2eb0c720d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:24.981936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-28342007-f385-49ce-a9de-59e63f7adee1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29ef88bde23941d6a45cd4e2eb0c720d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:24.984865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-28342007-f385-49ce-a9de-59e63f7adee1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29ef88bde23941d6a45cd4e2eb0c720d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:24.985738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-28342007-f385-49ce-a9de-59e63f7adee1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29ef88bde23941d6a45cd4e2eb0c720d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:25.052027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:25.077415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555376f-6582-4977-95d8-6f825dc85871 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:25.084781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1e531-dde7-4b34-b503-0f8972f351d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:25.085911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1e531-dde7-4b34-b503-0f8972f351d6 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:25.086427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1e531-dde7-4b34-b503-0f8972f351d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:25.086922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1e531-dde7-4b34-b503-0f8972f351d6 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:25.087933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1e531-dde7-4b34-b503-0f8972f351d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:25.090151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-28342007-f385-49ce-a9de-59e63f7adee1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29ef88bde23941d6a45cd4e2eb0c720d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:25.097407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1e531-dde7-4b34-b503-0f8972f351d6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:25.117714 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1169/2327] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:24 2022] DELETE /identity/v3/users/29ef88bde23941d6a45cd4e2eb0c720d => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:25.184620 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1159/2328] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:24 2022] POST /identity/v3/users => generated 287 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:25.185726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28342007-f385-49ce-a9de-59e63f7adee1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:25.187411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-28342007-f385-49ce-a9de-59e63f7adee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:25.259352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28342007-f385-49ce-a9de-59e63f7adee1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:25.264073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad964cf-eb3b-4452-a7bb-54546c84f26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:25.264744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad964cf-eb3b-4452-a7bb-54546c84f26d admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:25.265031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad964cf-eb3b-4452-a7bb-54546c84f26d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:25.265371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad964cf-eb3b-4452-a7bb-54546c84f26d admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:25.266175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad964cf-eb3b-4452-a7bb-54546c84f26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:25.267790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad964cf-eb3b-4452-a7bb-54546c84f26d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:25.280860 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1170/2329] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:25.282201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1e531-dde7-4b34-b503-0f8972f351d6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:25.292489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1e531-dde7-4b34-b503-0f8972f351d6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:25.297546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb929d8-3ebb-4f7a-a167-521f805cdf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:25.298268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb929d8-3ebb-4f7a-a167-521f805cdf6b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:25.298553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb929d8-3ebb-4f7a-a167-521f805cdf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:25.303038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb929d8-3ebb-4f7a-a167-521f805cdf6b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:25.303038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb929d8-3ebb-4f7a-a167-521f805cdf6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:25.303038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb929d8-3ebb-4f7a-a167-521f805cdf6b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:25.315222 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1160/2330] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:25.316419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad964cf-eb3b-4452-a7bb-54546c84f26d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:25.327463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad964cf-eb3b-4452-a7bb-54546c84f26d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:25.332355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1227b4-9ec2-4195-92a0-844221dcf2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:25.333254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1227b4-9ec2-4195-92a0-844221dcf2c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:25.333679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1227b4-9ec2-4195-92a0-844221dcf2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:25.333889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1227b4-9ec2-4195-92a0-844221dcf2c2 admin admin] PATH_INFO: `/v3/projects/16cb773714aa47299fa289c53ff1121f/users/bb7aa5f514d44c85b3c9863abf2d9d64/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:25.349534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1227b4-9ec2-4195-92a0-844221dcf2c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16cb773714aa47299fa289c53ff1121f, user_id=bb7aa5f514d44c85b3c9863abf2d9d64, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:25.353824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1227b4-9ec2-4195-92a0-844221dcf2c2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:25.434295 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1171/2331] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:25 2022] PUT /identity/v3/projects/16cb773714aa47299fa289c53ff1121f/users/bb7aa5f514d44c85b3c9863abf2d9d64/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:25.446488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba857ef-17c3-404f-ac8f-5983d0f90e7e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:25.446780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba857ef-17c3-404f-ac8f-5983d0f90e7e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:25.447062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba857ef-17c3-404f-ac8f-5983d0f90e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:25.456406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:25.476216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-fba857ef-17c3-404f-ac8f-5983d0f90e7e None None] MFA Rules not processed for user `bb7aa5f514d44c85b3c9863abf2d9d64`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:25.477545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fba857ef-17c3-404f-ac8f-5983d0f90e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:25.551473 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1161/2332] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:25.558807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1227b4-9ec2-4195-92a0-844221dcf2c2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:25.619042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba857ef-17c3-404f-ac8f-5983d0f90e7e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:25.620721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fba857ef-17c3-404f-ac8f-5983d0f90e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:25.648105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1227b4-9ec2-4195-92a0-844221dcf2c2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:25.654318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad0ed1-6f70-4552-9ee9-1432afeab662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:25.655047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad0ed1-6f70-4552-9ee9-1432afeab662 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:25.655464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad0ed1-6f70-4552-9ee9-1432afeab662 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:25.655674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad0ed1-6f70-4552-9ee9-1432afeab662 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:25.656464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad0ed1-6f70-4552-9ee9-1432afeab662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:25.658837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad0ed1-6f70-4552-9ee9-1432afeab662 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:25.705194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba857ef-17c3-404f-ac8f-5983d0f90e7e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:25.710663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5801c6-a29c-45e2-b8b1-a0c40c4f9cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:25.711572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5801c6-a29c-45e2-b8b1-a0c40c4f9cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:25.711861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5801c6-a29c-45e2-b8b1-a0c40c4f9cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:25.712145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5801c6-a29c-45e2-b8b1-a0c40c4f9cdc admin admin] PATH_INFO: `/v3/projects/2a0bc9e83b434c4c93a514f9c0ee106b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:25.713451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5801c6-a29c-45e2-b8b1-a0c40c4f9cdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a0bc9e83b434c4c93a514f9c0ee106b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:25.715415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5801c6-a29c-45e2-b8b1-a0c40c4f9cdc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:25.754825 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1172/2333] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:25 2022] POST /identity/v3/projects => generated 373 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:25.758740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad0ed1-6f70-4552-9ee9-1432afeab662 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:25.826912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad0ed1-6f70-4552-9ee9-1432afeab662 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:25.842059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec5801c6-a29c-45e2-b8b1-a0c40c4f9cdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a0bc9e83b434c4c93a514f9c0ee106b', 'request_id': 'req-ec5801c6-a29c-45e2-b8b1-a0c40c4f9cdc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:25.850450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec5801c6-a29c-45e2-b8b1-a0c40c4f9cdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a0bc9e83b434c4c93a514f9c0ee106b', 'request_id': 'req-ec5801c6-a29c-45e2-b8b1-a0c40c4f9cdc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:25.898991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5b3e5-39f8-4e60-9077-1a0b48c6f911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:25.900065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5b3e5-39f8-4e60-9077-1a0b48c6f911 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:25.901234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5b3e5-39f8-4e60-9077-1a0b48c6f911 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:25.901606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf5b3e5-39f8-4e60-9077-1a0b48c6f911 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:25.902525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf5b3e5-39f8-4e60-9077-1a0b48c6f911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:25.905329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf5b3e5-39f8-4e60-9077-1a0b48c6f911 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:25.919031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec5801c6-a29c-45e2-b8b1-a0c40c4f9cdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a0bc9e83b434c4c93a514f9c0ee106b', 'request_id': 'req-ec5801c6-a29c-45e2-b8b1-a0c40c4f9cdc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:25.937327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec5801c6-a29c-45e2-b8b1-a0c40c4f9cdc admin admin] The token cache is being invalidate because project 2a0bc9e83b434c4c93a514f9c0ee106b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:25.937968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec5801c6-a29c-45e2-b8b1-a0c40c4f9cdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:25.939672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec5801c6-a29c-45e2-b8b1-a0c40c4f9cdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:25.946992 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1162/2334] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:25 2022] DELETE /identity/v3/projects/2a0bc9e83b434c4c93a514f9c0ee106b => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:04:25.948821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e6517-5019-4fc0-95e6-502de00eb1de None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:25.949317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e6517-5019-4fc0-95e6-502de00eb1de None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:25.951220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e6517-5019-4fc0-95e6-502de00eb1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:25.972591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-854e6517-5019-4fc0-95e6-502de00eb1de None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:25.974798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-854e6517-5019-4fc0-95e6-502de00eb1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:25.985648 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1163/2335] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:25.995147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c7d88-9eb7-46e7-9217-61922d665542 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:25.995989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c7d88-9eb7-46e7-9217-61922d665542 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:25.996511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c7d88-9eb7-46e7-9217-61922d665542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:26.025941 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1173/2336] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:25 2022] POST /identity/v3/users => generated 293 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:04:26.027897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8a6c7d88-9eb7-46e7-9217-61922d665542 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:26.028020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5b3e5-39f8-4e60-9077-1a0b48c6f911 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:26.029657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8a6c7d88-9eb7-46e7-9217-61922d665542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:26.030599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fdf5b3e5-39f8-4e60-9077-1a0b48c6f911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:26.041642 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1164/2337] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:26.050231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c7d88-9eb7-46e7-9217-61922d665542 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:26.057648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf5b3e5-39f8-4e60-9077-1a0b48c6f911 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:26.067171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80502ae-e5b2-4f48-a0e6-1ef1b0b1f5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:26.067913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80502ae-e5b2-4f48-a0e6-1ef1b0b1f5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:26.073811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c7d88-9eb7-46e7-9217-61922d665542 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:26.074192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80502ae-e5b2-4f48-a0e6-1ef1b0b1f5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:26.074546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80502ae-e5b2-4f48-a0e6-1ef1b0b1f5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:26.075538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80502ae-e5b2-4f48-a0e6-1ef1b0b1f5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:26.077292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80502ae-e5b2-4f48-a0e6-1ef1b0b1f5a2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:26.080702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7445b7-ba2a-4faa-8cb9-732a8aa9c3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:26.081660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7445b7-ba2a-4faa-8cb9-732a8aa9c3a6 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:26.081982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7445b7-ba2a-4faa-8cb9-732a8aa9c3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:26.082266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7445b7-ba2a-4faa-8cb9-732a8aa9c3a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:26.083080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7445b7-ba2a-4faa-8cb9-732a8aa9c3a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:26.083399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7445b7-ba2a-4faa-8cb9-732a8aa9c3a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:26.086300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7445b7-ba2a-4faa-8cb9-732a8aa9c3a6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:26.096227 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1174/2338] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:26.111767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80502ae-e5b2-4f48-a0e6-1ef1b0b1f5a2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:26.114480 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1165/2339] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:26.115577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7445b7-ba2a-4faa-8cb9-732a8aa9c3a6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:26.134340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7445b7-ba2a-4faa-8cb9-732a8aa9c3a6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:26.139807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80502ae-e5b2-4f48-a0e6-1ef1b0b1f5a2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:26.140377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dd6cb-94ab-47c8-ab50-a5a7b4e54e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:26.141059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dd6cb-94ab-47c8-ab50-a5a7b4e54e00 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:26.141463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dd6cb-94ab-47c8-ab50-a5a7b4e54e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:26.141618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8dd6cb-94ab-47c8-ab50-a5a7b4e54e00 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:26.142469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8dd6cb-94ab-47c8-ab50-a5a7b4e54e00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:26.144613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8dd6cb-94ab-47c8-ab50-a5a7b4e54e00 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:26.155761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0efbf-b9d2-4fdb-81d0-a4d1c65f2a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:26.156859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0efbf-b9d2-4fdb-81d0-a4d1c65f2a2b admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:26.157240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0efbf-b9d2-4fdb-81d0-a4d1c65f2a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:26.157614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0efbf-b9d2-4fdb-81d0-a4d1c65f2a2b admin admin] PATH_INFO: `/v3/projects/68f8fc2f557e448d9ae5ce0743b23f86/users/ada3c5d65541405b80d97cd03cb507bf/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:26.233158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0efbf-b9d2-4fdb-81d0-a4d1c65f2a2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f8fc2f557e448d9ae5ce0743b23f86, user_id=ada3c5d65541405b80d97cd03cb507bf, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:26.238426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0efbf-b9d2-4fdb-81d0-a4d1c65f2a2b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:26.260920 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1166/2340] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:26 2022] POST /identity/v3/projects => generated 375 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:26.261970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dd6cb-94ab-47c8-ab50-a5a7b4e54e00 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:26.264731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5b8dd6cb-94ab-47c8-ab50-a5a7b4e54e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:26.307172 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1175/2341] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:26 2022] PUT /identity/v3/projects/68f8fc2f557e448d9ae5ce0743b23f86/users/ada3c5d65541405b80d97cd03cb507bf/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:04:26.308323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0efbf-b9d2-4fdb-81d0-a4d1c65f2a2b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:26.374869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8dd6cb-94ab-47c8-ab50-a5a7b4e54e00 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:26.444425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0efbf-b9d2-4fdb-81d0-a4d1c65f2a2b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:26.460990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb30ec3-6810-4f2c-b8ff-484861dc5340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:26.461683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb30ec3-6810-4f2c-b8ff-484861dc5340 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:26.462223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb30ec3-6810-4f2c-b8ff-484861dc5340 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:26.462376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb30ec3-6810-4f2c-b8ff-484861dc5340 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:26.463901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb30ec3-6810-4f2c-b8ff-484861dc5340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:26.468904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb30ec3-6810-4f2c-b8ff-484861dc5340 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:26.474781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:26.534607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:26.535986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:26.535986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:26.535986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192 admin admin] PATH_INFO: `/v3/projects/99c01e770b5948d4b50929145a205280` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:26.537303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99c01e770b5948d4b50929145a205280)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:26.542577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:26.599683 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1176/2342] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:26 2022] POST /identity/v3/users => generated 288 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:26.605553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb30ec3-6810-4f2c-b8ff-484861dc5340 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:26.618580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb30ec3-6810-4f2c-b8ff-484861dc5340 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:26.628522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5465fa-1ee8-4613-b133-60e66b307e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:26.629621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5465fa-1ee8-4613-b133-60e66b307e31 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:26.630091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5465fa-1ee8-4613-b133-60e66b307e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:26.631198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5465fa-1ee8-4613-b133-60e66b307e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:26.645456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5465fa-1ee8-4613-b133-60e66b307e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:26.650482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5465fa-1ee8-4613-b133-60e66b307e31 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:26.666295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99c01e770b5948d4b50929145a205280', 'request_id': 'req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:26.673332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:26.677129 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1177/2343] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:26.680005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5465fa-1ee8-4613-b133-60e66b307e31 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:26.681272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99c01e770b5948d4b50929145a205280', 'request_id': 'req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:26.700134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5465fa-1ee8-4613-b133-60e66b307e31 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:26.780440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9beb81-db80-403d-a0a2-fb17e0de4769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:26.781427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9beb81-db80-403d-a0a2-fb17e0de4769 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:26.781923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9beb81-db80-403d-a0a2-fb17e0de4769 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:26.782195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99c01e770b5948d4b50929145a205280', 'request_id': 'req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:26.782563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9beb81-db80-403d-a0a2-fb17e0de4769 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:26.783659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9beb81-db80-403d-a0a2-fb17e0de4769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:26.785478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9beb81-db80-403d-a0a2-fb17e0de4769 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:26.796482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192 admin admin] The token cache is being invalidate because project 99c01e770b5948d4b50929145a205280 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:26.796887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:26.797373 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1178/2344] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:26.797961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:26.798696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9beb81-db80-403d-a0a2-fb17e0de4769 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:26.801072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7c9beb81-db80-403d-a0a2-fb17e0de4769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:26.804923 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1167/2345] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:26 2022] DELETE /identity/v3/projects/99c01e770b5948d4b50929145a205280 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:26.806040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:26.808447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:26.834010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4bfdf-2e8b-4cd8-ab8d-257ca32bf192 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:26.835305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5297036f-c522-4eca-a38f-8286a4639a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:26.837014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297036f-c522-4eca-a38f-8286a4639a0f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:26.837375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297036f-c522-4eca-a38f-8286a4639a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:26.837729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5297036f-c522-4eca-a38f-8286a4639a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:26.839626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5297036f-c522-4eca-a38f-8286a4639a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:26.841533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5297036f-c522-4eca-a38f-8286a4639a0f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:26.850357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9beb81-db80-403d-a0a2-fb17e0de4769 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:26.855858 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1168/2346] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:26.858306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398de4e-b74b-45bd-b25c-353d8634890b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:26.859005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398de4e-b74b-45bd-b25c-353d8634890b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:26.859438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398de4e-b74b-45bd-b25c-353d8634890b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:26.870658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19195928-3855-4281-868e-c8b32286e376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:26.871728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19195928-3855-4281-868e-c8b32286e376 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:26.872122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19195928-3855-4281-868e-c8b32286e376 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:26.872585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19195928-3855-4281-868e-c8b32286e376 admin admin] PATH_INFO: `/v3/projects/68f8fc2f557e448d9ae5ce0743b23f86/users/ada3c5d65541405b80d97cd03cb507bf/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:26.876509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19195928-3855-4281-868e-c8b32286e376 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f8fc2f557e448d9ae5ce0743b23f86, user_id=ada3c5d65541405b80d97cd03cb507bf, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:26.882147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19195928-3855-4281-868e-c8b32286e376 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:26.893403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c398de4e-b74b-45bd-b25c-353d8634890b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:26.898169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c398de4e-b74b-45bd-b25c-353d8634890b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:26.909918 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1179/2347] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:26 2022] PUT /identity/v3/projects/68f8fc2f557e448d9ae5ce0743b23f86/users/ada3c5d65541405b80d97cd03cb507bf/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:26.909918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19195928-3855-4281-868e-c8b32286e376 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:26.981083 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1169/2348] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:26.983249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b051fe-01c3-49f1-ac68-64043e3dbfe5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:26.983547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b051fe-01c3-49f1-ac68-64043e3dbfe5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:26.983824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b051fe-01c3-49f1-ac68-64043e3dbfe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:27.001458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19195928-3855-4281-868e-c8b32286e376 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:27.007616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c5a80-2d74-419d-9796-3631f25bbd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:27.008843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c5a80-2d74-419d-9796-3631f25bbd76 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:27.009134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c5a80-2d74-419d-9796-3631f25bbd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:27.009409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c5a80-2d74-419d-9796-3631f25bbd76 admin admin] PATH_INFO: `/v3/projects/56e1dc5556624e2fb8de8a030c34c334/users/478d1609aa114d16b338e07d2d4c05c9/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:27.019924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a9b051fe-01c3-49f1-ac68-64043e3dbfe5 None None] MFA Rules not processed for user `ada3c5d65541405b80d97cd03cb507bf`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:27.021358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a9b051fe-01c3-49f1-ac68-64043e3dbfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:27.026482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c5a80-2d74-419d-9796-3631f25bbd76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56e1dc5556624e2fb8de8a030c34c334, user_id=478d1609aa114d16b338e07d2d4c05c9, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:27.029229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c5a80-2d74-419d-9796-3631f25bbd76 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:27.093690 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1170/2349] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:27.094989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfde65d-332e-46c4-ba12-274f620a895f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:27.095283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfde65d-332e-46c4-ba12-274f620a895f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:27.095564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfde65d-332e-46c4-ba12-274f620a895f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:27.108164 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1180/2350] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:26 2022] PUT /identity/v3/projects/56e1dc5556624e2fb8de8a030c34c334/users/478d1609aa114d16b338e07d2d4c05c9/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:27.109091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c5a80-2d74-419d-9796-3631f25bbd76 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:27.123148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2bfde65d-332e-46c4-ba12-274f620a895f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:27.124455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2bfde65d-332e-46c4-ba12-274f620a895f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:27.197666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c5a80-2d74-419d-9796-3631f25bbd76 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:27.202394 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1171/2351] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:27.203508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b4e31-c7f4-46a9-beb1-da110f200c0f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:27.203949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b4e31-c7f4-46a9-beb1-da110f200c0f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:27.204744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c6ba1-de00-4080-8ca9-3c98770c1f9b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:27.205502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c6ba1-de00-4080-8ca9-3c98770c1f9b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:27.205791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c6ba1-de00-4080-8ca9-3c98770c1f9b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:27.205961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b4e31-c7f4-46a9-beb1-da110f200c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:27.206103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1c6ba1-de00-4080-8ca9-3c98770c1f9b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:27.206845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c6ba1-de00-4080-8ca9-3c98770c1f9b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:27.207173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c6ba1-de00-4080-8ca9-3c98770c1f9b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:27.208924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1c6ba1-de00-4080-8ca9-3c98770c1f9b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:27.222637 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1181/2352] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:27.223751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c6ba1-de00-4080-8ca9-3c98770c1f9b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:27.238285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1c6ba1-de00-4080-8ca9-3c98770c1f9b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:27.245453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961dfcf-f2bf-478e-a674-03e5ebcdf378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:27.246451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961dfcf-f2bf-478e-a674-03e5ebcdf378 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:27.246912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961dfcf-f2bf-478e-a674-03e5ebcdf378 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:27.247442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961dfcf-f2bf-478e-a674-03e5ebcdf378 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:27.248797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961dfcf-f2bf-478e-a674-03e5ebcdf378 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:27.251804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961dfcf-f2bf-478e-a674-03e5ebcdf378 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:27.253096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c81b4e31-c7f4-46a9-beb1-da110f200c0f None None] MFA Rules not processed for user `478d1609aa114d16b338e07d2d4c05c9`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:27.254427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c81b4e31-c7f4-46a9-beb1-da110f200c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:27.255081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8961dfcf-f2bf-478e-a674-03e5ebcdf378 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:27.279459 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1182/2353] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:27.281041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961dfcf-f2bf-478e-a674-03e5ebcdf378 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:27.296655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961dfcf-f2bf-478e-a674-03e5ebcdf378 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:27.302478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbdc8c-85b4-441e-a5ca-14a17fa41124 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:27.304007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbdc8c-85b4-441e-a5ca-14a17fa41124 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:27.304750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbdc8c-85b4-441e-a5ca-14a17fa41124 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:27.305130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbdc8c-85b4-441e-a5ca-14a17fa41124 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:27.306217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbdc8c-85b4-441e-a5ca-14a17fa41124 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:27.345735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbdc8c-85b4-441e-a5ca-14a17fa41124 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:27.346185 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1172/2354] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:27.346185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b4e31-c7f4-46a9-beb1-da110f200c0f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:27.358809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b4e31-c7f4-46a9-beb1-da110f200c0f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:27.364727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707f8c0-6053-4156-8227-2b1e5fe524bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:27.366359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707f8c0-6053-4156-8227-2b1e5fe524bb admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:27.366711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707f8c0-6053-4156-8227-2b1e5fe524bb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:27.367285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9707f8c0-6053-4156-8227-2b1e5fe524bb admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:27.368083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707f8c0-6053-4156-8227-2b1e5fe524bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:27.371109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9707f8c0-6053-4156-8227-2b1e5fe524bb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:27.754298 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1183/2355] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:27 2022] POST /identity/v3/projects => generated 369 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:27.755276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbdc8c-85b4-441e-a5ca-14a17fa41124 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:27.781828 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1173/2356] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:27 2022] POST /identity/v3/projects => generated 383 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:27.783058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707f8c0-6053-4156-8227-2b1e5fe524bb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:27.832522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbdc8c-85b4-441e-a5ca-14a17fa41124 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:27.879712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9707f8c0-6053-4156-8227-2b1e5fe524bb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:27.887711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcd4b9-61a3-40c8-baa7-b2495bbd2af4 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:27.901861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a7ac7-b0b3-4a28-80bb-c0f4309d4f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:27.902019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcd4b9-61a3-40c8-baa7-b2495bbd2af4 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:27.902019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcd4b9-61a3-40c8-baa7-b2495bbd2af4 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:27.902019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bcd4b9-61a3-40c8-baa7-b2495bbd2af4 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:27.902247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a7ac7-b0b3-4a28-80bb-c0f4309d4f11 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:27.902327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bcd4b9-61a3-40c8-baa7-b2495bbd2af4 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:27.902327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bcd4b9-61a3-40c8-baa7-b2495bbd2af4 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:27.902480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a7ac7-b0b3-4a28-80bb-c0f4309d4f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:27.902553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:27.903327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a7ac7-b0b3-4a28-80bb-c0f4309d4f11 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:27.904118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a7ac7-b0b3-4a28-80bb-c0f4309d4f11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:27.906499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a7ac7-b0b3-4a28-80bb-c0f4309d4f11 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:27.909230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bcd4b9-61a3-40c8-baa7-b2495bbd2af4 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:27.939604 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1174/2357] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:27.943853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcd4b9-61a3-40c8-baa7-b2495bbd2af4 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:27.953368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bcd4b9-61a3-40c8-baa7-b2495bbd2af4 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:27.960108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e65f0-eea0-497e-9bbc-59c91457ad59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:27.960941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e65f0-eea0-497e-9bbc-59c91457ad59 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:27.961290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e65f0-eea0-497e-9bbc-59c91457ad59 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:27.961639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e65f0-eea0-497e-9bbc-59c91457ad59 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:27.972366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e65f0-eea0-497e-9bbc-59c91457ad59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:27.979527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e65f0-eea0-497e-9bbc-59c91457ad59 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:28.019572 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1184/2358] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:27 2022] POST /identity/v3/projects => generated 375 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:28.020649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a7ac7-b0b3-4a28-80bb-c0f4309d4f11 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:28.022932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-964a7ac7-b0b3-4a28-80bb-c0f4309d4f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:28.094241 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1175/2359] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:27 2022] POST /identity/v3/users => generated 292 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:28.095134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e65f0-eea0-497e-9bbc-59c91457ad59 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:28.117430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e65f0-eea0-497e-9bbc-59c91457ad59 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:28.130419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:28.183734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a7ac7-b0b3-4a28-80bb-c0f4309d4f11 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:28.190156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60cec5-74cf-4535-8d3e-e0a1e8276691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:28.191479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60cec5-74cf-4535-8d3e-e0a1e8276691 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:28.191479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60cec5-74cf-4535-8d3e-e0a1e8276691 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:28.191479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d60cec5-74cf-4535-8d3e-e0a1e8276691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:28.207715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc3e85-803b-4d05-aedc-19e309a6f8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:28.209401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc3e85-803b-4d05-aedc-19e309a6f8a9 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:28.209782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc3e85-803b-4d05-aedc-19e309a6f8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:28.210683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc3e85-803b-4d05-aedc-19e309a6f8a9 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:28.211777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc3e85-803b-4d05-aedc-19e309a6f8a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:28.214911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fc3e85-803b-4d05-aedc-19e309a6f8a9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:28.329059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60cec5-74cf-4535-8d3e-e0a1e8276691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:28.331563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d60cec5-74cf-4535-8d3e-e0a1e8276691 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:28.346136 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1176/2360] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:28 2022] POST /identity/v3/users => generated 294 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:28.346136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc3e85-803b-4d05-aedc-19e309a6f8a9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:28.354230 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1185/2361] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:28 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:28.355170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60cec5-74cf-4535-8d3e-e0a1e8276691 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:28.370492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc3e85-803b-4d05-aedc-19e309a6f8a9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:28.383368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acfc01d-14d1-45fe-a2ed-cec80c647de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:28.384454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfc01d-14d1-45fe-a2ed-cec80c647de2 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:28.386622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfc01d-14d1-45fe-a2ed-cec80c647de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:28.387028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfc01d-14d1-45fe-a2ed-cec80c647de2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:28.387458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d60cec5-74cf-4535-8d3e-e0a1e8276691 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:28.390478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acfc01d-14d1-45fe-a2ed-cec80c647de2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:28.392610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acfc01d-14d1-45fe-a2ed-cec80c647de2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:28.401188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7af8f-1091-43b9-8a12-59c866d4b377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:28.403304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7af8f-1091-43b9-8a12-59c866d4b377 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:28.403304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7af8f-1091-43b9-8a12-59c866d4b377 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:28.403304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7af8f-1091-43b9-8a12-59c866d4b377 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:28.404601 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7af8f-1091-43b9-8a12-59c866d4b377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:28.407439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7af8f-1091-43b9-8a12-59c866d4b377 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:28.410525 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1177/2362] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:28.412532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d0ae5-1d01-49b3-998a-222e9db1a89b None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:28.413019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d0ae5-1d01-49b3-998a-222e9db1a89b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:28.413613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d0ae5-1d01-49b3-998a-222e9db1a89b None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:28.415761 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1178/2363] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:04:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:04:28.417131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d0ae5-1d01-49b3-998a-222e9db1a89b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:28.432644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d0ae5-1d01-49b3-998a-222e9db1a89b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:28.438212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd43f48-c169-4d92-964d-627c5a7f59ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:28.439716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd43f48-c169-4d92-964d-627c5a7f59ed admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:28.440222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd43f48-c169-4d92-964d-627c5a7f59ed admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:28.440372 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1186/2364] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:28.440798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd43f48-c169-4d92-964d-627c5a7f59ed admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:28.441645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7af8f-1091-43b9-8a12-59c866d4b377 admin admin] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:04:28.441917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd43f48-c169-4d92-964d-627c5a7f59ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:28.444734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd43f48-c169-4d92-964d-627c5a7f59ed admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:28.458894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7af8f-1091-43b9-8a12-59c866d4b377 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:28.464814 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1179/2365] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:28.464814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd43f48-c169-4d92-964d-627c5a7f59ed admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:28.473996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7af8f-1091-43b9-8a12-59c866d4b377 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:28.475234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd43f48-c169-4d92-964d-627c5a7f59ed admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:28.479863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e8181-df79-493c-9ac4-f01865e74b66 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:28.481034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e8181-df79-493c-9ac4-f01865e74b66 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:28.481430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e8181-df79-493c-9ac4-f01865e74b66 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:28.481816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e8181-df79-493c-9ac4-f01865e74b66 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/projects/33c240b84f2a4ff3bc3680450a299110` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:28.482360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb5a577-3771-4512-8923-ab28db10c484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:28.483968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5a577-3771-4512-8923-ab28db10c484 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:28.484422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e8181-df79-493c-9ac4-f01865e74b66 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:get_project(project_id=33c240b84f2a4ff3bc3680450a299110)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:28.484610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5a577-3771-4512-8923-ab28db10c484 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:28.485083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb5a577-3771-4512-8923-ab28db10c484 admin admin] PATH_INFO: `/v3/projects/1283a7c356b74d8086558ea4217b0d20/users/ebbb3158771b420db1cc0827263ead5f/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:28.487961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e8181-df79-493c-9ac4-f01865e74b66 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:28.490905 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1187/2366] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:04:28 2022] GET /identity/v3/projects/33c240b84f2a4ff3bc3680450a299110 => generated 369 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:28.495862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e8181-df79-493c-9ac4-f01865e74b66 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:28.505411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5a577-3771-4512-8923-ab28db10c484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1283a7c356b74d8086558ea4217b0d20, user_id=ebbb3158771b420db1cc0827263ead5f, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:28.508107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e8181-df79-493c-9ac4-f01865e74b66 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:28.509965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb5a577-3771-4512-8923-ab28db10c484 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:28.515610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78569c85-db12-4680-85da-ef7de9bba158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:28.517693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78569c85-db12-4680-85da-ef7de9bba158 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:28.518114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78569c85-db12-4680-85da-ef7de9bba158 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:28.521678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78569c85-db12-4680-85da-ef7de9bba158 admin admin] PATH_INFO: `/v3/projects/d3ab11147ee34b828cdc96645cef519b/users/02ba016624f54a7896a1fcd9649d467e/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:28.558054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78569c85-db12-4680-85da-ef7de9bba158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3ab11147ee34b828cdc96645cef519b, user_id=02ba016624f54a7896a1fcd9649d467e, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:28.562383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78569c85-db12-4680-85da-ef7de9bba158 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:28.602052 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1180/2367] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:28 2022] PUT /identity/v3/projects/1283a7c356b74d8086558ea4217b0d20/users/ebbb3158771b420db1cc0827263ead5f/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:28.602052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed960ac9-f94b-4de3-adcb-6852ab49c334 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:28.602331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed960ac9-f94b-4de3-adcb-6852ab49c334 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:28.602416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed960ac9-f94b-4de3-adcb-6852ab49c334 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:28.616463 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1181/2368] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:04:28 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:04:28.618357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed960ac9-f94b-4de3-adcb-6852ab49c334 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:28.682193 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1188/2369] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:28 2022] PUT /identity/v3/projects/d3ab11147ee34b828cdc96645cef519b/users/02ba016624f54a7896a1fcd9649d467e/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:28.683614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78569c85-db12-4680-85da-ef7de9bba158 admin admin] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:04:28.685622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-78569c85-db12-4680-85da-ef7de9bba158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:28.721746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed960ac9-f94b-4de3-adcb-6852ab49c334 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:28.843144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7211747-b40c-428e-94c2-c2a586a8864d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:28.844310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7211747-b40c-428e-94c2-c2a586a8864d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:28.844782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7211747-b40c-428e-94c2-c2a586a8864d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:28.845236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7211747-b40c-428e-94c2-c2a586a8864d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:28.846376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7211747-b40c-428e-94c2-c2a586a8864d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:28.848179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7211747-b40c-428e-94c2-c2a586a8864d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:28.855987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78569c85-db12-4680-85da-ef7de9bba158 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:28.866551 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1182/2370] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:28.867550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6cbf1-6f11-42f2-9053-a5fb24c2b82d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:28.868320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6cbf1-6f11-42f2-9053-a5fb24c2b82d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:28.868766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6cbf1-6f11-42f2-9053-a5fb24c2b82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:28.904657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5ef6cbf1-6f11-42f2-9053-a5fb24c2b82d None None] MFA Rules not processed for user `02ba016624f54a7896a1fcd9649d467e`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:28.906301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5ef6cbf1-6f11-42f2-9053-a5fb24c2b82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:29.001874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78569c85-db12-4680-85da-ef7de9bba158 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:29.007746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35219109-48db-4f22-9a4c-06b7e9ed7398 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:29.008645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35219109-48db-4f22-9a4c-06b7e9ed7398 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:29.008934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35219109-48db-4f22-9a4c-06b7e9ed7398 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:29.009218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35219109-48db-4f22-9a4c-06b7e9ed7398 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/projects/33c240b84f2a4ff3bc3680450a299110` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:29.011146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35219109-48db-4f22-9a4c-06b7e9ed7398 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:get_project(project_id=33c240b84f2a4ff3bc3680450a299110)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:29.014000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35219109-48db-4f22-9a4c-06b7e9ed7398 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:29.030976 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1189/2371] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:04:28 2022] GET /identity/v3/projects/33c240b84f2a4ff3bc3680450a299110 => generated 369 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:29.031950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35219109-48db-4f22-9a4c-06b7e9ed7398 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:29.059825 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1183/2372] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:29.060779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6cbf1-6f11-42f2-9053-a5fb24c2b82d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:29.067464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35219109-48db-4f22-9a4c-06b7e9ed7398 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:29.081193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a4be72-92af-4a6b-893f-68055578420e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:29.082993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4be72-92af-4a6b-893f-68055578420e admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:29.086393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4be72-92af-4a6b-893f-68055578420e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:29.086596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a4be72-92af-4a6b-893f-68055578420e admin admin] PATH_INFO: `/v3/projects/1283a7c356b74d8086558ea4217b0d20/users/ebbb3158771b420db1cc0827263ead5f/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:29.090077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4be72-92af-4a6b-893f-68055578420e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1283a7c356b74d8086558ea4217b0d20, user_id=ebbb3158771b420db1cc0827263ead5f, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:29.095970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6cbf1-6f11-42f2-9053-a5fb24c2b82d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:29.099249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a4be72-92af-4a6b-893f-68055578420e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:29.108612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e5018-f826-4f6d-9ebe-f38cdabb6919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:29.110512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e5018-f826-4f6d-9ebe-f38cdabb6919 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:29.110800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e5018-f826-4f6d-9ebe-f38cdabb6919 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:29.111078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e5018-f826-4f6d-9ebe-f38cdabb6919 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:29.111919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e5018-f826-4f6d-9ebe-f38cdabb6919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:29.113890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e5018-f826-4f6d-9ebe-f38cdabb6919 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:29.138638 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1190/2373] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:29 2022] PUT /identity/v3/projects/1283a7c356b74d8086558ea4217b0d20/users/ebbb3158771b420db1cc0827263ead5f/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:29.155116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065abcaf-1a7f-4383-b3e5-b2462eba7bc1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:29.155407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065abcaf-1a7f-4383-b3e5-b2462eba7bc1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:29.155688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065abcaf-1a7f-4383-b3e5-b2462eba7bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:29.214887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-065abcaf-1a7f-4383-b3e5-b2462eba7bc1 None None] MFA Rules not processed for user `ebbb3158771b420db1cc0827263ead5f`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:29.216251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-065abcaf-1a7f-4383-b3e5-b2462eba7bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:29.278421 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1184/2374] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:29 2022] POST /identity/v3/projects => generated 385 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:29.279358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e5018-f826-4f6d-9ebe-f38cdabb6919 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:29.358132 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1191/2375] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:04:29.369078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210eb33-2c6e-4fd2-bbf2-a6e9a1776359 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:29.369377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210eb33-2c6e-4fd2-bbf2-a6e9a1776359 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:29.370275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210eb33-2c6e-4fd2-bbf2-a6e9a1776359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:29.411515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:29.415079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b210eb33-2c6e-4fd2-bbf2-a6e9a1776359 None None] MFA Rules not processed for user `478d1609aa114d16b338e07d2d4c05c9`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:29.416285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b210eb33-2c6e-4fd2-bbf2-a6e9a1776359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:29.422236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e5018-f826-4f6d-9ebe-f38cdabb6919 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:29.439273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee57bca-a738-4284-be1b-cac7772bf4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:29.439960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee57bca-a738-4284-be1b-cac7772bf4da admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:29.440407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee57bca-a738-4284-be1b-cac7772bf4da admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:29.440533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee57bca-a738-4284-be1b-cac7772bf4da admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:29.441207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee57bca-a738-4284-be1b-cac7772bf4da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:29.443401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee57bca-a738-4284-be1b-cac7772bf4da admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:29.518258 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1192/2376] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:29.519255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210eb33-2c6e-4fd2-bbf2-a6e9a1776359 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:29.526059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b210eb33-2c6e-4fd2-bbf2-a6e9a1776359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:29.548496 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1185/2377] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:29 2022] POST /identity/v3/users => generated 299 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:29.549954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee57bca-a738-4284-be1b-cac7772bf4da admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:29.552920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210eb33-2c6e-4fd2-bbf2-a6e9a1776359 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:29.562852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee57bca-a738-4284-be1b-cac7772bf4da admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:29.567502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e104f-2712-4a28-a5b9-b79a9e15e51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:29.568299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb6ef9-b0c6-4dd2-9ac6-052aa8537f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:29.569223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e104f-2712-4a28-a5b9-b79a9e15e51b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:29.569860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e104f-2712-4a28-a5b9-b79a9e15e51b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:29.570398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e104f-2712-4a28-a5b9-b79a9e15e51b admin admin] PATH_INFO: `/v3/users/05ada83d82f648eeabca0dce3b8eb3cd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:29.572035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e104f-2712-4a28-a5b9-b79a9e15e51b admin admin] RBAC: Authorizing `identity:delete_user(user_id=05ada83d82f648eeabca0dce3b8eb3cd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:29.574504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e104f-2712-4a28-a5b9-b79a9e15e51b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:29.578188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb6ef9-b0c6-4dd2-9ac6-052aa8537f73 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:29.581953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb6ef9-b0c6-4dd2-9ac6-052aa8537f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:29.581953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eb6ef9-b0c6-4dd2-9ac6-052aa8537f73 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:29.583787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb6ef9-b0c6-4dd2-9ac6-052aa8537f73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:29.586620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eb6ef9-b0c6-4dd2-9ac6-052aa8537f73 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:29.600805 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1186/2378] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:04:29.605698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb6ef9-b0c6-4dd2-9ac6-052aa8537f73 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:29.623892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eb6ef9-b0c6-4dd2-9ac6-052aa8537f73 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:29.630767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea98f8c-8b2b-482f-81c7-6161069b47a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:29.632396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea98f8c-8b2b-482f-81c7-6161069b47a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:29.632757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea98f8c-8b2b-482f-81c7-6161069b47a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:29.633472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea98f8c-8b2b-482f-81c7-6161069b47a4 admin admin] PATH_INFO: `/v3/projects/5672aa5675604f1c8e1c90219f5bdbce/users/737585c57e7b4e4ca8ae0ebf80ab4648/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:29.658336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea98f8c-8b2b-482f-81c7-6161069b47a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5672aa5675604f1c8e1c90219f5bdbce, user_id=737585c57e7b4e4ca8ae0ebf80ab4648, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:29.663561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea98f8c-8b2b-482f-81c7-6161069b47a4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:29.751036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:29.774903 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1187/2379] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:29 2022] PUT /identity/v3/projects/5672aa5675604f1c8e1c90219f5bdbce/users/737585c57e7b4e4ca8ae0ebf80ab4648/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:29.778048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-812e104f-2712-4a28-a5b9-b79a9e15e51b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05ada83d82f648eeabca0dce3b8eb3cd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:29.779868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163fe9a-f4f1-47c1-8b09-ad510ffb4395 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:29.780442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163fe9a-f4f1-47c1-8b09-ad510ffb4395 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:29.781042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163fe9a-f4f1-47c1-8b09-ad510ffb4395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:29.787965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-812e104f-2712-4a28-a5b9-b79a9e15e51b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05ada83d82f648eeabca0dce3b8eb3cd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:29.788803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-812e104f-2712-4a28-a5b9-b79a9e15e51b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05ada83d82f648eeabca0dce3b8eb3cd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:29.789422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-812e104f-2712-4a28-a5b9-b79a9e15e51b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05ada83d82f648eeabca0dce3b8eb3cd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:29.803232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9163fe9a-f4f1-47c1-8b09-ad510ffb4395 None None] MFA Rules not processed for user `cdeaa660f8ec465bba68096d0ed4b1ae`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:29.804446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9163fe9a-f4f1-47c1-8b09-ad510ffb4395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:29.870545 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1188/2380] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:29.871550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163fe9a-f4f1-47c1-8b09-ad510ffb4395 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:29.872969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9163fe9a-f4f1-47c1-8b09-ad510ffb4395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:29.879233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-812e104f-2712-4a28-a5b9-b79a9e15e51b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05ada83d82f648eeabca0dce3b8eb3cd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:29.906809 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1193/2381] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:29 2022] DELETE /identity/v3/users/05ada83d82f648eeabca0dce3b8eb3cd => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:29.908490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e104f-2712-4a28-a5b9-b79a9e15e51b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:29.909985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-812e104f-2712-4a28-a5b9-b79a9e15e51b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:29.956204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163fe9a-f4f1-47c1-8b09-ad510ffb4395 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:30.009365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e104f-2712-4a28-a5b9-b79a9e15e51b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:30.020827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ab323-23b1-4252-999e-1c62fe9f93be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:30.022325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ab323-23b1-4252-999e-1c62fe9f93be service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:30.022812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ab323-23b1-4252-999e-1c62fe9f93be service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:30.023195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ab323-23b1-4252-999e-1c62fe9f93be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:30.025177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e64ab323-23b1-4252-999e-1c62fe9f93be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:30.026747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177acfe-637d-44bc-a6e3-6be7204be8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:30.027784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177acfe-637d-44bc-a6e3-6be7204be8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:30.028077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177acfe-637d-44bc-a6e3-6be7204be8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:30.028362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177acfe-637d-44bc-a6e3-6be7204be8d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:30.029106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177acfe-637d-44bc-a6e3-6be7204be8d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:30.030826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177acfe-637d-44bc-a6e3-6be7204be8d9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:30.041763 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1189/2382] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:30.043666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01499cb-dfeb-4f4a-b8b6-9d9d30dbd64a None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:30.043972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01499cb-dfeb-4f4a-b8b6-9d9d30dbd64a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:30.044257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01499cb-dfeb-4f4a-b8b6-9d9d30dbd64a None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:30.045750 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1190/2383] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:04:30.047316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a9df1-9c2f-4c1a-99a8-beea12d16b86 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:30.047316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a9df1-9c2f-4c1a-99a8-beea12d16b86 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:30.047525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a9df1-9c2f-4c1a-99a8-beea12d16b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:30.066463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ed3a9df1-9c2f-4c1a-99a8-beea12d16b86 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:30.067717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ed3a9df1-9c2f-4c1a-99a8-beea12d16b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:30.080432 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1191/2384] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:30.081449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a9df1-9c2f-4c1a-99a8-beea12d16b86 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:30.103048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a9df1-9c2f-4c1a-99a8-beea12d16b86 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:30.109066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd16bf-490f-4c2a-8a9f-4a5fc6950a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:30.110513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd16bf-490f-4c2a-8a9f-4a5fc6950a5e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:30.110919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd16bf-490f-4c2a-8a9f-4a5fc6950a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:30.111298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd16bf-490f-4c2a-8a9f-4a5fc6950a5e admin admin] PATH_INFO: `/v3/projects/5672aa5675604f1c8e1c90219f5bdbce/users/737585c57e7b4e4ca8ae0ebf80ab4648/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:30.114197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd16bf-490f-4c2a-8a9f-4a5fc6950a5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5672aa5675604f1c8e1c90219f5bdbce, user_id=737585c57e7b4e4ca8ae0ebf80ab4648, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:30.118184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd16bf-490f-4c2a-8a9f-4a5fc6950a5e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:30.122364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ab323-23b1-4252-999e-1c62fe9f93be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:30.124906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64ab323-23b1-4252-999e-1c62fe9f93be service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:30.135350 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1192/2385] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:30 2022] PUT /identity/v3/projects/5672aa5675604f1c8e1c90219f5bdbce/users/737585c57e7b4e4ca8ae0ebf80ab4648/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:30.136348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd16bf-490f-4c2a-8a9f-4a5fc6950a5e admin admin] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:04:30.139338 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1194/2386] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:30.139621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3fdd16bf-490f-4c2a-8a9f-4a5fc6950a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:30.140705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ab323-23b1-4252-999e-1c62fe9f93be service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:30.142669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e64ab323-23b1-4252-999e-1c62fe9f93be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:30.224493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ab323-23b1-4252-999e-1c62fe9f93be service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:30.274489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd16bf-490f-4c2a-8a9f-4a5fc6950a5e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:30.290394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3fdd16bf-490f-4c2a-8a9f-4a5fc6950a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:30.295203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21eb20d-4876-4af0-8e0e-58cb440f1e97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:30.295880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21eb20d-4876-4af0-8e0e-58cb440f1e97 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:30.296168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21eb20d-4876-4af0-8e0e-58cb440f1e97 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:30.296448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21eb20d-4876-4af0-8e0e-58cb440f1e97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:30.386729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd16bf-490f-4c2a-8a9f-4a5fc6950a5e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:30.390488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21eb20d-4876-4af0-8e0e-58cb440f1e97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:30.392701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21eb20d-4876-4af0-8e0e-58cb440f1e97 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:30.393495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47dba8-64b4-432d-843d-d9d247139cc0 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:30.394938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47dba8-64b4-432d-843d-d9d247139cc0 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:30.395376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47dba8-64b4-432d-843d-d9d247139cc0 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:30.395934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47dba8-64b4-432d-843d-d9d247139cc0 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/projects/33c240b84f2a4ff3bc3680450a299110` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:30.397898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47dba8-64b4-432d-843d-d9d247139cc0 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:get_project(project_id=33c240b84f2a4ff3bc3680450a299110)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:30.400952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e47dba8-64b4-432d-843d-d9d247139cc0 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:30.403135 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1193/2387] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:04:30 2022] GET /identity/v3/projects/33c240b84f2a4ff3bc3680450a299110 => generated 369 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:30.406773 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1195/2388] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:30.408412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7199f-b8f0-43cb-a487-de5161cd64e3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:30.408709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7199f-b8f0-43cb-a487-de5161cd64e3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:30.408997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f7199f-b8f0-43cb-a487-de5161cd64e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:30.433689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21eb20d-4876-4af0-8e0e-58cb440f1e97 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:30.449480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-90f7199f-b8f0-43cb-a487-de5161cd64e3 None None] MFA Rules not processed for user `737585c57e7b4e4ca8ae0ebf80ab4648`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:30.451101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-90f7199f-b8f0-43cb-a487-de5161cd64e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:30.456285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21eb20d-4876-4af0-8e0e-58cb440f1e97 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:30.478797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932c6a5-1caa-48ff-878f-2f89ef71899a tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:30.479711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932c6a5-1caa-48ff-878f-2f89ef71899a tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:30.479999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932c6a5-1caa-48ff-878f-2f89ef71899a tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:30.480284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6932c6a5-1caa-48ff-878f-2f89ef71899a tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/projects/33c240b84f2a4ff3bc3680450a299110` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:30.487961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932c6a5-1caa-48ff-878f-2f89ef71899a tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:delete_project(project_id=33c240b84f2a4ff3bc3680450a299110)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:30.499560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6932c6a5-1caa-48ff-878f-2f89ef71899a tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:30.558987 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1194/2389] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:30.690933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6932c6a5-1caa-48ff-878f-2f89ef71899a tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33c240b84f2a4ff3bc3680450a299110', 'request_id': 'req-6932c6a5-1caa-48ff-878f-2f89ef71899a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:30.700214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6932c6a5-1caa-48ff-878f-2f89ef71899a tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33c240b84f2a4ff3bc3680450a299110', 'request_id': 'req-6932c6a5-1caa-48ff-878f-2f89ef71899a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:30.794516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:30.799723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6932c6a5-1caa-48ff-878f-2f89ef71899a tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33c240b84f2a4ff3bc3680450a299110', 'request_id': 'req-6932c6a5-1caa-48ff-878f-2f89ef71899a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:30.815522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780aeeb-4915-4ef6-b73c-30f68deca74a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:30.816068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780aeeb-4915-4ef6-b73c-30f68deca74a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:30.816502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780aeeb-4915-4ef6-b73c-30f68deca74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:30.840622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6932c6a5-1caa-48ff-878f-2f89ef71899a tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] The token cache is being invalidate because project 33c240b84f2a4ff3bc3680450a299110 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:30.841105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6932c6a5-1caa-48ff-878f-2f89ef71899a tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:30.842035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6932c6a5-1caa-48ff-878f-2f89ef71899a tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:30.851502 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1196/2390] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:30 2022] DELETE /identity/v3/projects/33c240b84f2a4ff3bc3680450a299110 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:04:30.866508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4780aeeb-4915-4ef6-b73c-30f68deca74a None None] MFA Rules not processed for user `478d1609aa114d16b338e07d2d4c05c9`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:30.867816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4780aeeb-4915-4ef6-b73c-30f68deca74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:30.947600 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1195/2391] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:30.949593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:30.963020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932c6a5-1caa-48ff-878f-2f89ef71899a tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:30.969627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6932c6a5-1caa-48ff-878f-2f89ef71899a tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:31.133411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6932c6a5-1caa-48ff-878f-2f89ef71899a tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:31.139351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31346656-3196-4420-b117-0ff4a3f7cb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:31.140533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31346656-3196-4420-b117-0ff4a3f7cb74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:31.141164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31346656-3196-4420-b117-0ff4a3f7cb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:31.141627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31346656-3196-4420-b117-0ff4a3f7cb74 admin admin] PATH_INFO: `/v3/projects/cde37d77fb1343f4af1dd8e735a41615` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:31.143102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31346656-3196-4420-b117-0ff4a3f7cb74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cde37d77fb1343f4af1dd8e735a41615)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:31.146917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31346656-3196-4420-b117-0ff4a3f7cb74 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:31.319900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-31346656-3196-4420-b117-0ff4a3f7cb74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cde37d77fb1343f4af1dd8e735a41615', 'request_id': 'req-31346656-3196-4420-b117-0ff4a3f7cb74'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:31.326946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-31346656-3196-4420-b117-0ff4a3f7cb74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cde37d77fb1343f4af1dd8e735a41615', 'request_id': 'req-31346656-3196-4420-b117-0ff4a3f7cb74'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:31.445717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-31346656-3196-4420-b117-0ff4a3f7cb74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cde37d77fb1343f4af1dd8e735a41615', 'request_id': 'req-31346656-3196-4420-b117-0ff4a3f7cb74'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:31.463409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-31346656-3196-4420-b117-0ff4a3f7cb74 admin admin] The token cache is being invalidate because project cde37d77fb1343f4af1dd8e735a41615 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:31.463851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-31346656-3196-4420-b117-0ff4a3f7cb74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:31.464634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-31346656-3196-4420-b117-0ff4a3f7cb74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:31.473494 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1197/2392] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:30 2022] DELETE /identity/v3/projects/cde37d77fb1343f4af1dd8e735a41615 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:04:31.475019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4780aeeb-4915-4ef6-b73c-30f68deca74a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:31.496225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4780aeeb-4915-4ef6-b73c-30f68deca74a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:31.580522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6cd70-a418-464f-bfef-c1bc7fd79914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:31.581484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6cd70-a418-464f-bfef-c1bc7fd79914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:31.581669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6cd70-a418-464f-bfef-c1bc7fd79914 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:31.581992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6cd70-a418-464f-bfef-c1bc7fd79914 admin admin] PATH_INFO: `/v3/users/7c2a8a5e024e432084eab8c5eacb3637` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:31.583226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6cd70-a418-464f-bfef-c1bc7fd79914 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c2a8a5e024e432084eab8c5eacb3637)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:31.587175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6cd70-a418-464f-bfef-c1bc7fd79914 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:31.683694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5711e6-ee1d-4c49-816c-1b8cb66b8b03 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:31.683982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5711e6-ee1d-4c49-816c-1b8cb66b8b03 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:31.694370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5711e6-ee1d-4c49-816c-1b8cb66b8b03 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:31.695806 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1198/2393] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:04:31 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:04:31.703647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5711e6-ee1d-4c49-816c-1b8cb66b8b03 None None] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:04:31.707180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1a5711e6-ee1d-4c49-816c-1b8cb66b8b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:31.825958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a7d6cd70-a418-464f-bfef-c1bc7fd79914 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c2a8a5e024e432084eab8c5eacb3637'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:31.827073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a7d6cd70-a418-464f-bfef-c1bc7fd79914 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c2a8a5e024e432084eab8c5eacb3637'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:31.830090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5711e6-ee1d-4c49-816c-1b8cb66b8b03 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:31.836889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1a5711e6-ee1d-4c49-816c-1b8cb66b8b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:31.845125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a7d6cd70-a418-464f-bfef-c1bc7fd79914 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c2a8a5e024e432084eab8c5eacb3637'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:31.928909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a7d6cd70-a418-464f-bfef-c1bc7fd79914 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c2a8a5e024e432084eab8c5eacb3637'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:31.928909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a7d6cd70-a418-464f-bfef-c1bc7fd79914 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c2a8a5e024e432084eab8c5eacb3637'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:31.953353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5711e6-ee1d-4c49-816c-1b8cb66b8b03 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:31.960783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c02d5-135d-48ba-8e24-5bfc255f0ca5 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:31.963201 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1196/2394] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:31 2022] DELETE /identity/v3/users/7c2a8a5e024e432084eab8c5eacb3637 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:04:31.965465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c02d5-135d-48ba-8e24-5bfc255f0ca5 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:31.966022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c02d5-135d-48ba-8e24-5bfc255f0ca5 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:31.966897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c02d5-135d-48ba-8e24-5bfc255f0ca5 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/projects/16cb773714aa47299fa289c53ff1121f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:31.968689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c02d5-135d-48ba-8e24-5bfc255f0ca5 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:get_project(project_id=16cb773714aa47299fa289c53ff1121f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:31.972243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c02d5-135d-48ba-8e24-5bfc255f0ca5 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:31.974265 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1199/2395] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:04:31 2022] GET /identity/v3/projects/16cb773714aa47299fa289c53ff1121f => generated 373 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:31.993623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6cd70-a418-464f-bfef-c1bc7fd79914 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:31.997636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a7d6cd70-a418-464f-bfef-c1bc7fd79914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:32.123295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6cd70-a418-464f-bfef-c1bc7fd79914 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:32.132199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc57c8-4464-40e1-8649-7c6465629238 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:32.133534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc57c8-4464-40e1-8649-7c6465629238 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:32.133927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc57c8-4464-40e1-8649-7c6465629238 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:32.134450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc57c8-4464-40e1-8649-7c6465629238 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:32.135474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc57c8-4464-40e1-8649-7c6465629238 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:32.135905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc57c8-4464-40e1-8649-7c6465629238 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:32.138233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc57c8-4464-40e1-8649-7c6465629238 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:32.154558 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1197/2396] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:32.158405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c02d5-135d-48ba-8e24-5bfc255f0ca5 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:32.168546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c02d5-135d-48ba-8e24-5bfc255f0ca5 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:32.173768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f6acc9-3bb9-4b73-bf6a-879d554a13e7 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:32.174521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6acc9-3bb9-4b73-bf6a-879d554a13e7 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:32.174814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6acc9-3bb9-4b73-bf6a-879d554a13e7 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:32.175258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f6acc9-3bb9-4b73-bf6a-879d554a13e7 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:32.175979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6acc9-3bb9-4b73-bf6a-879d554a13e7 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:32.178554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f6acc9-3bb9-4b73-bf6a-879d554a13e7 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:32.216261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:32.343106 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1200/2397] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:32 2022] POST /identity/v3/projects => generated 369 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:04:32.347045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc57c8-4464-40e1-8649-7c6465629238 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:32.386570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc57c8-4464-40e1-8649-7c6465629238 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:32.550715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2873672-7b06-49ce-8c3b-25c7c2c07863 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:32.553571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2873672-7b06-49ce-8c3b-25c7c2c07863 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:32.553885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2873672-7b06-49ce-8c3b-25c7c2c07863 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:32.554215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2873672-7b06-49ce-8c3b-25c7c2c07863 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:32.554973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2873672-7b06-49ce-8c3b-25c7c2c07863 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:32.555310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2873672-7b06-49ce-8c3b-25c7c2c07863 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:32.557132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2873672-7b06-49ce-8c3b-25c7c2c07863 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:32.571704 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1198/2398] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:32.867564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8382324-df5e-4584-baea-f2f6a38b147b None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:32.868221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8382324-df5e-4584-baea-f2f6a38b147b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:32.868652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8382324-df5e-4584-baea-f2f6a38b147b None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:32.872795 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1201/2399] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:04:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:04:32.876133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2873672-7b06-49ce-8c3b-25c7c2c07863 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:04:32.877945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a2873672-7b06-49ce-8c3b-25c7c2c07863 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:33.004551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2873672-7b06-49ce-8c3b-25c7c2c07863 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:33.017320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2873672-7b06-49ce-8c3b-25c7c2c07863 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:33.043441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f1e7b-9ee4-4d2b-a7c5-9e716e546eb1 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:33.044405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f1e7b-9ee4-4d2b-a7c5-9e716e546eb1 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:33.044693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f1e7b-9ee4-4d2b-a7c5-9e716e546eb1 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:33.045162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f1e7b-9ee4-4d2b-a7c5-9e716e546eb1 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/projects/991a68ad732f498e80a0a745646328af` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:33.049577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f1e7b-9ee4-4d2b-a7c5-9e716e546eb1 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:get_project(project_id=991a68ad732f498e80a0a745646328af)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:33.052163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f1e7b-9ee4-4d2b-a7c5-9e716e546eb1 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:33.054549 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1199/2400] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:04:32 2022] GET /identity/v3/projects/991a68ad732f498e80a0a745646328af => generated 369 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:33.055453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8382324-df5e-4584-baea-f2f6a38b147b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:33.057497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a8382324-df5e-4584-baea-f2f6a38b147b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:33.222154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8382324-df5e-4584-baea-f2f6a38b147b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:33.227984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7909edb4-021c-4b6d-8e2d-35902aa0c579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:33.229174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909edb4-021c-4b6d-8e2d-35902aa0c579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:33.229638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909edb4-021c-4b6d-8e2d-35902aa0c579 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:33.229960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7909edb4-021c-4b6d-8e2d-35902aa0c579 admin admin] PATH_INFO: `/v3/projects/1b93fc976e7c477494f36f429f136248` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:33.231435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909edb4-021c-4b6d-8e2d-35902aa0c579 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b93fc976e7c477494f36f429f136248)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:33.234056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7909edb4-021c-4b6d-8e2d-35902aa0c579 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:33.417247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7909edb4-021c-4b6d-8e2d-35902aa0c579 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b93fc976e7c477494f36f429f136248', 'request_id': 'req-7909edb4-021c-4b6d-8e2d-35902aa0c579'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:33.441927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7909edb4-021c-4b6d-8e2d-35902aa0c579 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b93fc976e7c477494f36f429f136248', 'request_id': 'req-7909edb4-021c-4b6d-8e2d-35902aa0c579'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:33.456627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a87b35-a09c-40d4-a108-24887708be03 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:33.456926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a87b35-a09c-40d4-a108-24887708be03 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:33.457278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a87b35-a09c-40d4-a108-24887708be03 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:33.459087 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1200/2401] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 21:04:33.472755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a87b35-a09c-40d4-a108-24887708be03 None None] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:04:33.477393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-22a87b35-a09c-40d4-a108-24887708be03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:33.513228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7909edb4-021c-4b6d-8e2d-35902aa0c579 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b93fc976e7c477494f36f429f136248', 'request_id': 'req-7909edb4-021c-4b6d-8e2d-35902aa0c579'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:33.548202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7909edb4-021c-4b6d-8e2d-35902aa0c579 admin admin] The token cache is being invalidate because project 1b93fc976e7c477494f36f429f136248 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:33.548782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7909edb4-021c-4b6d-8e2d-35902aa0c579 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:33.550721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7909edb4-021c-4b6d-8e2d-35902aa0c579 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:33.553751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:33.554464 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1202/2402] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:33 2022] DELETE /identity/v3/projects/1b93fc976e7c477494f36f429f136248 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:33.563700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97e934-0575-427a-86c2-a73809e3d465 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:33.564172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97e934-0575-427a-86c2-a73809e3d465 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:33.564723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97e934-0575-427a-86c2-a73809e3d465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:33.624984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5d97e934-0575-427a-86c2-a73809e3d465 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:33.626340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5d97e934-0575-427a-86c2-a73809e3d465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:33.766699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a87b35-a09c-40d4-a108-24887708be03 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:33.783844 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1203/2403] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:33.792042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a19fd2-c916-40a6-800f-54173f2a6abd None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:33.792042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a19fd2-c916-40a6-800f-54173f2a6abd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:33.795974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a19fd2-c916-40a6-800f-54173f2a6abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:33.881280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a3a19fd2-c916-40a6-800f-54173f2a6abd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:33.884182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a3a19fd2-c916-40a6-800f-54173f2a6abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:33.913758 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1204/2404] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:33.915052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a19fd2-c916-40a6-800f-54173f2a6abd None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:33.945453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a19fd2-c916-40a6-800f-54173f2a6abd None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:33.958385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661b97e-2f83-4104-a566-5ffbc729150c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:33.960448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661b97e-2f83-4104-a566-5ffbc729150c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:33.962392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661b97e-2f83-4104-a566-5ffbc729150c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:33.963245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661b97e-2f83-4104-a566-5ffbc729150c admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:33.967016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661b97e-2f83-4104-a566-5ffbc729150c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:33.975342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661b97e-2f83-4104-a566-5ffbc729150c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:33.975342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661b97e-2f83-4104-a566-5ffbc729150c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:33.988008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a87b35-a09c-40d4-a108-24887708be03 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:34.005304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07471586-22a4-4ec4-9d70-1c986b480ce6 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:34.006856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07471586-22a4-4ec4-9d70-1c986b480ce6 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:34.007224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07471586-22a4-4ec4-9d70-1c986b480ce6 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:34.007582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07471586-22a4-4ec4-9d70-1c986b480ce6 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/projects/991a68ad732f498e80a0a745646328af` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:34.008923 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1205/2405] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:34.010326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07471586-22a4-4ec4-9d70-1c986b480ce6 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:get_project(project_id=991a68ad732f498e80a0a745646328af)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:34.011369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661b97e-2f83-4104-a566-5ffbc729150c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:34.017062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07471586-22a4-4ec4-9d70-1c986b480ce6 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:34.033304 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1201/2406] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:04:33 2022] GET /identity/v3/projects/991a68ad732f498e80a0a745646328af => generated 369 bytes in 562 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:34.043741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661b97e-2f83-4104-a566-5ffbc729150c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:34.069091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7941bc-df9e-4526-8003-088ed078815d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:34.070436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7941bc-df9e-4526-8003-088ed078815d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:34.070728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7941bc-df9e-4526-8003-088ed078815d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:34.071011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7941bc-df9e-4526-8003-088ed078815d admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:34.071721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7941bc-df9e-4526-8003-088ed078815d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:34.073667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7941bc-df9e-4526-8003-088ed078815d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:34.112006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:34.131294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07471586-22a4-4ec4-9d70-1c986b480ce6 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:34.133041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-07471586-22a4-4ec4-9d70-1c986b480ce6 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:34.151477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07471586-22a4-4ec4-9d70-1c986b480ce6 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:34.165165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5997891-888b-4896-ba20-c58505f04956 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:34.166157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5997891-888b-4896-ba20-c58505f04956 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:34.167106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5997891-888b-4896-ba20-c58505f04956 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:34.167491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5997891-888b-4896-ba20-c58505f04956 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:34.168574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5997891-888b-4896-ba20-c58505f04956 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:34.169486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5997891-888b-4896-ba20-c58505f04956 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:34.172006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5997891-888b-4896-ba20-c58505f04956 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:34.194211 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1202/2407] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:04:34.195854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5997891-888b-4896-ba20-c58505f04956 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:34.215066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5997891-888b-4896-ba20-c58505f04956 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:34.222462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a6613-b4d6-4264-a4d5-16f25d33fe02 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:34.222980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a6613-b4d6-4264-a4d5-16f25d33fe02 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:34.223264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a6613-b4d6-4264-a4d5-16f25d33fe02 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:34.223542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a6613-b4d6-4264-a4d5-16f25d33fe02 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:34.225117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a6613-b4d6-4264-a4d5-16f25d33fe02 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:34.227892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a6613-b4d6-4264-a4d5-16f25d33fe02 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:34.252189 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1206/2408] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:34 2022] POST /identity/v3/projects => generated 385 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:34.252189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7941bc-df9e-4526-8003-088ed078815d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:34.385945 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1203/2409] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:34 2022] POST /identity/v3/users => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:34.390900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a6613-b4d6-4264-a4d5-16f25d33fe02 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:34.405072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7941bc-df9e-4526-8003-088ed078815d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:34.415939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920e611-378c-45c1-bd49-3bf2986500f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:34.416139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a6613-b4d6-4264-a4d5-16f25d33fe02 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:34.416251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920e611-378c-45c1-bd49-3bf2986500f4 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:34.416251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920e611-378c-45c1-bd49-3bf2986500f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:34.416251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920e611-378c-45c1-bd49-3bf2986500f4 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:34.416251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920e611-378c-45c1-bd49-3bf2986500f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:34.417184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920e611-378c-45c1-bd49-3bf2986500f4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:34.544914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823474c-23ca-47a8-b760-14620283ff68 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:34.545686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823474c-23ca-47a8-b760-14620283ff68 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:34.546350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823474c-23ca-47a8-b760-14620283ff68 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:34.546707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823474c-23ca-47a8-b760-14620283ff68 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:34.549550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823474c-23ca-47a8-b760-14620283ff68 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:34.549988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823474c-23ca-47a8-b760-14620283ff68 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:34.552759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823474c-23ca-47a8-b760-14620283ff68 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:34.564517 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1207/2410] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:34 2022] POST /identity/v3/users => generated 293 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:34.569342 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1204/2411] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:34.570069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920e611-378c-45c1-bd49-3bf2986500f4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:34.590732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920e611-378c-45c1-bd49-3bf2986500f4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:34.596133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb92810-2570-4bde-8eb0-c49c0804966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:34.597294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb92810-2570-4bde-8eb0-c49c0804966f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:34.597813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb92810-2570-4bde-8eb0-c49c0804966f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:34.598483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb92810-2570-4bde-8eb0-c49c0804966f admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:34.599978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb92810-2570-4bde-8eb0-c49c0804966f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:34.601878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb92810-2570-4bde-8eb0-c49c0804966f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:34.618284 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1208/2412] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:34.622734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823474c-23ca-47a8-b760-14620283ff68 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:34.636476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823474c-23ca-47a8-b760-14620283ff68 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:34.645796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43389967-94e0-48be-8d59-88b3f9e2dc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:34.647327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43389967-94e0-48be-8d59-88b3f9e2dc4e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:34.648225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43389967-94e0-48be-8d59-88b3f9e2dc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:34.648856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43389967-94e0-48be-8d59-88b3f9e2dc4e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:34.649884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43389967-94e0-48be-8d59-88b3f9e2dc4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:34.651628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43389967-94e0-48be-8d59-88b3f9e2dc4e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:34.668442 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1205/2413] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:34.669707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb92810-2570-4bde-8eb0-c49c0804966f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:34.684122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb92810-2570-4bde-8eb0-c49c0804966f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:34.692099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c9e4c1-a4ff-47dc-8815-0bbe7a68b20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:34.693983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9e4c1-a4ff-47dc-8815-0bbe7a68b20b admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:34.694435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9e4c1-a4ff-47dc-8815-0bbe7a68b20b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:34.694730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c9e4c1-a4ff-47dc-8815-0bbe7a68b20b admin admin] PATH_INFO: `/v3/projects/b059b8b9ea6e4afbb5d034c9d1a036d2/users/77a6068e4dc34fb6b8d1447472692e55/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:34.728834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9e4c1-a4ff-47dc-8815-0bbe7a68b20b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b059b8b9ea6e4afbb5d034c9d1a036d2, user_id=77a6068e4dc34fb6b8d1447472692e55, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:34.728834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c9e4c1-a4ff-47dc-8815-0bbe7a68b20b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:34.794410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a575f6-f59b-49d4-a8da-9d52e881f18b None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:34.795135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a575f6-f59b-49d4-a8da-9d52e881f18b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:34.795651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a575f6-f59b-49d4-a8da-9d52e881f18b None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:34.798987 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1206/2414] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:04:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:04:34.804808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a575f6-f59b-49d4-a8da-9d52e881f18b None None] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:04:34.806608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-35a575f6-f59b-49d4-a8da-9d52e881f18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:34.857813 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1209/2415] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:34 2022] PUT /identity/v3/projects/b059b8b9ea6e4afbb5d034c9d1a036d2/users/77a6068e4dc34fb6b8d1447472692e55/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:34.868185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef01eca-a3a6-49d4-b6e8-5bf80719d7bf None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:34.868773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef01eca-a3a6-49d4-b6e8-5bf80719d7bf None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:34.869203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef01eca-a3a6-49d4-b6e8-5bf80719d7bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:34.891368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6ef01eca-a3a6-49d4-b6e8-5bf80719d7bf None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:34.892592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6ef01eca-a3a6-49d4-b6e8-5bf80719d7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:34.902029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a575f6-f59b-49d4-a8da-9d52e881f18b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:34.912807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a575f6-f59b-49d4-a8da-9d52e881f18b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:34.961583 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1210/2416] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:34.973526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef01eca-a3a6-49d4-b6e8-5bf80719d7bf None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:34.976977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6ef01eca-a3a6-49d4-b6e8-5bf80719d7bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:34.979503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbb502-52f8-4e53-98a1-cccf2eaa6f7e tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:34.980387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbb502-52f8-4e53-98a1-cccf2eaa6f7e tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:34.980676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbb502-52f8-4e53-98a1-cccf2eaa6f7e tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:34.980956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbb502-52f8-4e53-98a1-cccf2eaa6f7e tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/projects/991a68ad732f498e80a0a745646328af` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:34.982201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbb502-52f8-4e53-98a1-cccf2eaa6f7e tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:get_project(project_id=991a68ad732f498e80a0a745646328af)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:34.984289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbb502-52f8-4e53-98a1-cccf2eaa6f7e tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:34.986382 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1207/2417] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:04:34 2022] GET /identity/v3/projects/991a68ad732f498e80a0a745646328af => generated 369 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:35.067562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef01eca-a3a6-49d4-b6e8-5bf80719d7bf None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:35.087294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3fbf3-4d82-486b-8201-3b6866761010 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:35.087955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3fbf3-4d82-486b-8201-3b6866761010 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:35.088191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3fbf3-4d82-486b-8201-3b6866761010 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:35.088473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3fbf3-4d82-486b-8201-3b6866761010 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:35.099793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3fbf3-4d82-486b-8201-3b6866761010 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:35.101244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3fbf3-4d82-486b-8201-3b6866761010 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:35.111402 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1211/2418] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:35.239234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5b621-7b54-4a9f-8b8a-29051edb35c2 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:35.248028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5b621-7b54-4a9f-8b8a-29051edb35c2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:35.248028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d5b621-7b54-4a9f-8b8a-29051edb35c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:35.255980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d34f6-e254-4812-83d6-47bebcc36783 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:35.256554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d34f6-e254-4812-83d6-47bebcc36783 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:35.257016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d34f6-e254-4812-83d6-47bebcc36783 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:35.258647 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1212/2419] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:04:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 21:04:35.266870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d34f6-e254-4812-83d6-47bebcc36783 None None] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:04:35.269318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c52d34f6-e254-4812-83d6-47bebcc36783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:35.297944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b4d5b621-7b54-4a9f-8b8a-29051edb35c2 None None] MFA Rules not processed for user `c14b965da64c40e382c69050ba01b183`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:35.299200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b4d5b621-7b54-4a9f-8b8a-29051edb35c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:35.386731 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1208/2420] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:35.406219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d34f6-e254-4812-83d6-47bebcc36783 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:35.435068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d34f6-e254-4812-83d6-47bebcc36783 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:35.440391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafa50b-a8f6-4f98-ba49-93c2efe9ac16 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:35.441348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafa50b-a8f6-4f98-ba49-93c2efe9ac16 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:35.441697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafa50b-a8f6-4f98-ba49-93c2efe9ac16 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:35.442018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bafa50b-a8f6-4f98-ba49-93c2efe9ac16 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/projects/991a68ad732f498e80a0a745646328af` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:35.443782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafa50b-a8f6-4f98-ba49-93c2efe9ac16 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:get_project(project_id=991a68ad732f498e80a0a745646328af)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:35.446170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bafa50b-a8f6-4f98-ba49-93c2efe9ac16 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:35.448943 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1213/2421] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:04:35 2022] GET /identity/v3/projects/991a68ad732f498e80a0a745646328af => generated 369 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:35.467725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:35.498363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5b621-7b54-4a9f-8b8a-29051edb35c2 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:35.513079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d5b621-7b54-4a9f-8b8a-29051edb35c2 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:35.524886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd25de2a-61f6-4d08-9d20-4e7109427286 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:35.526724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25de2a-61f6-4d08-9d20-4e7109427286 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:35.529546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25de2a-61f6-4d08-9d20-4e7109427286 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:35.529546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd25de2a-61f6-4d08-9d20-4e7109427286 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/users/4bda39bf693641ffaf1d977717476a19` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:35.556135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25de2a-61f6-4d08-9d20-4e7109427286 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:delete_user(user_id=4bda39bf693641ffaf1d977717476a19)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:35.559432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd25de2a-61f6-4d08-9d20-4e7109427286 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:35.674689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cd25de2a-61f6-4d08-9d20-4e7109427286 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bda39bf693641ffaf1d977717476a19'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:35.675507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cd25de2a-61f6-4d08-9d20-4e7109427286 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bda39bf693641ffaf1d977717476a19'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:35.684169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cd25de2a-61f6-4d08-9d20-4e7109427286 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bda39bf693641ffaf1d977717476a19'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:35.765230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cd25de2a-61f6-4d08-9d20-4e7109427286 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bda39bf693641ffaf1d977717476a19'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:35.765424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cd25de2a-61f6-4d08-9d20-4e7109427286 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bda39bf693641ffaf1d977717476a19'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:35.788827 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1209/2422] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:35 2022] DELETE /identity/v3/users/4bda39bf693641ffaf1d977717476a19 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:35.792904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafa50b-a8f6-4f98-ba49-93c2efe9ac16 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:35.794656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5bafa50b-a8f6-4f98-ba49-93c2efe9ac16 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:35.808376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bafa50b-a8f6-4f98-ba49-93c2efe9ac16 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:35.915435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d9f10-e922-4929-a45a-6172328a12aa tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:35.922733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d9f10-e922-4929-a45a-6172328a12aa tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:35.931085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d9f10-e922-4929-a45a-6172328a12aa tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:35.933431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d9f10-e922-4929-a45a-6172328a12aa tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/projects/991a68ad732f498e80a0a745646328af` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:35.937160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d9f10-e922-4929-a45a-6172328a12aa tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:delete_project(project_id=991a68ad732f498e80a0a745646328af)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:35.940062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d9f10-e922-4929-a45a-6172328a12aa tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:36.027256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:36.172193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3b4d9f10-e922-4929-a45a-6172328a12aa tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '991a68ad732f498e80a0a745646328af', 'request_id': 'req-3b4d9f10-e922-4929-a45a-6172328a12aa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:36.194348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3b4d9f10-e922-4929-a45a-6172328a12aa tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '991a68ad732f498e80a0a745646328af', 'request_id': 'req-3b4d9f10-e922-4929-a45a-6172328a12aa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:36.303010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3b4d9f10-e922-4929-a45a-6172328a12aa tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '991a68ad732f498e80a0a745646328af', 'request_id': 'req-3b4d9f10-e922-4929-a45a-6172328a12aa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:36.321875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3b4d9f10-e922-4929-a45a-6172328a12aa tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] The token cache is being invalidate because project 991a68ad732f498e80a0a745646328af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:36.322186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3b4d9f10-e922-4929-a45a-6172328a12aa tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:36.324348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3b4d9f10-e922-4929-a45a-6172328a12aa tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:36.343830 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1214/2423] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:35 2022] DELETE /identity/v3/projects/991a68ad732f498e80a0a745646328af => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:36.742124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95114cab-523a-4277-a3e1-6e8f4edacddc None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:36.742936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95114cab-523a-4277-a3e1-6e8f4edacddc None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:36.743378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95114cab-523a-4277-a3e1-6e8f4edacddc None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:36.746768 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1210/2424] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:04:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:04:36.756475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d9f10-e922-4929-a45a-6172328a12aa tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:04:36.758580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3b4d9f10-e922-4929-a45a-6172328a12aa tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:36.871078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d9f10-e922-4929-a45a-6172328a12aa tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:36.873206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3b4d9f10-e922-4929-a45a-6172328a12aa tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:36.965497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d9f10-e922-4929-a45a-6172328a12aa tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:36.970406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eab2a3-e050-4a68-a8bd-8843ed1ceb0b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:36.971316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eab2a3-e050-4a68-a8bd-8843ed1ceb0b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:36.971824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eab2a3-e050-4a68-a8bd-8843ed1ceb0b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:36.972109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eab2a3-e050-4a68-a8bd-8843ed1ceb0b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/projects/16cb773714aa47299fa289c53ff1121f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:36.974663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eab2a3-e050-4a68-a8bd-8843ed1ceb0b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:get_project(project_id=16cb773714aa47299fa289c53ff1121f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:36.977383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eab2a3-e050-4a68-a8bd-8843ed1ceb0b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:36.980035 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1215/2425] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:04:36 2022] GET /identity/v3/projects/16cb773714aa47299fa289c53ff1121f => generated 373 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:37.557050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837f8c4-fd89-496c-a961-b4f5038c720f None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:37.557344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837f8c4-fd89-496c-a961-b4f5038c720f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:37.557609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6837f8c4-fd89-496c-a961-b4f5038c720f None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:37.559506 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1211/2426] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:04:37.571582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eab2a3-e050-4a68-a8bd-8843ed1ceb0b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:04:37.605637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eab2a3-e050-4a68-a8bd-8843ed1ceb0b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:37.632530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eab2a3-e050-4a68-a8bd-8843ed1ceb0b tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:37.653707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d224a2-16fb-4c3e-a045-e271fced9d5d tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:37.654838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d224a2-16fb-4c3e-a045-e271fced9d5d tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:37.655285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d224a2-16fb-4c3e-a045-e271fced9d5d tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:37.655741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d224a2-16fb-4c3e-a045-e271fced9d5d tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/projects/16cb773714aa47299fa289c53ff1121f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:37.657237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d224a2-16fb-4c3e-a045-e271fced9d5d tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:get_project(project_id=16cb773714aa47299fa289c53ff1121f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:37.663069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d224a2-16fb-4c3e-a045-e271fced9d5d tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:37.663452 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1216/2427] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:04:37 2022] GET /identity/v3/projects/16cb773714aa47299fa289c53ff1121f => generated 373 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:37.764403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:37.897091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:39.753141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:39.872658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:39.932261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2e204-7713-45ae-a17d-639c734655e0 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:39.932465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2e204-7713-45ae-a17d-639c734655e0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:39.932758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2e204-7713-45ae-a17d-639c734655e0 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:39.935021 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1212/2428] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:04:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:04:39.945257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d224a2-16fb-4c3e-a045-e271fced9d5d tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:04:39.949797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b0d224a2-16fb-4c3e-a045-e271fced9d5d tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:39.971373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d224a2-16fb-4c3e-a045-e271fced9d5d tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:39.983399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d224a2-16fb-4c3e-a045-e271fced9d5d tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:39.988538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f51150-894e-42e6-81d9-24a759ba6635 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ada3c5d65541405b80d97cd03cb507bf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '68f8fc2f557e448d9ae5ce0743b23f86', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:39.989681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f51150-894e-42e6-81d9-24a759ba6635 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:39.990648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f51150-894e-42e6-81d9-24a759ba6635 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:39.991101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f51150-894e-42e6-81d9-24a759ba6635 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] PATH_INFO: `/v3/projects/16cb773714aa47299fa289c53ff1121f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:39.992499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f51150-894e-42e6-81d9-24a759ba6635 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorizing `identity:get_project(project_id=16cb773714aa47299fa289c53ff1121f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:39.995055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f51150-894e-42e6-81d9-24a759ba6635 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:39.997325 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1217/2429] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:04:39 2022] GET /identity/v3/projects/16cb773714aa47299fa289c53ff1121f => generated 373 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:40.091922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2e204-7713-45ae-a17d-639c734655e0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:40.118301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2e204-7713-45ae-a17d-639c734655e0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:40.126723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb55c31-f24a-4f8b-ac08-7c6a91201d77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:40.127736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb55c31-f24a-4f8b-ac08-7c6a91201d77 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:40.128015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb55c31-f24a-4f8b-ac08-7c6a91201d77 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:40.128529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb55c31-f24a-4f8b-ac08-7c6a91201d77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:40.130731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-afb55c31-f24a-4f8b-ac08-7c6a91201d77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:40.170639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:40.220592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb55c31-f24a-4f8b-ac08-7c6a91201d77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:40.223328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb55c31-f24a-4f8b-ac08-7c6a91201d77 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:40.242854 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1213/2430] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:40 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:41.667936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:41.782655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:42.148559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f51150-894e-42e6-81d9-24a759ba6635 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:42.150483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-46f51150-894e-42e6-81d9-24a759ba6635 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:42.242860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f51150-894e-42e6-81d9-24a759ba6635 tempest-QuotasAdminTestJSON-675682544 tempest-QuotasAdminTestJSON-675682544-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:42.250307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b8e01-9031-4f75-82c8-a69f1935b652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:42.251449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b8e01-9031-4f75-82c8-a69f1935b652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:42.251996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b8e01-9031-4f75-82c8-a69f1935b652 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:42.252432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b8e01-9031-4f75-82c8-a69f1935b652 admin admin] PATH_INFO: `/v3/users/bb7aa5f514d44c85b3c9863abf2d9d64` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:42.253904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b8e01-9031-4f75-82c8-a69f1935b652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb7aa5f514d44c85b3c9863abf2d9d64)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:42.256544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b8e01-9031-4f75-82c8-a69f1935b652 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:42.497230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-559b8e01-9031-4f75-82c8-a69f1935b652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb7aa5f514d44c85b3c9863abf2d9d64'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:42.503808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-559b8e01-9031-4f75-82c8-a69f1935b652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb7aa5f514d44c85b3c9863abf2d9d64'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:42.506378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-559b8e01-9031-4f75-82c8-a69f1935b652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb7aa5f514d44c85b3c9863abf2d9d64'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:42.507059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-559b8e01-9031-4f75-82c8-a69f1935b652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb7aa5f514d44c85b3c9863abf2d9d64'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:42.513533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb55c31-f24a-4f8b-ac08-7c6a91201d77 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:42.528707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb55c31-f24a-4f8b-ac08-7c6a91201d77 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:42.535692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631347b7-4111-41e1-bb72-24b210ae6f4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:42.536600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631347b7-4111-41e1-bb72-24b210ae6f4e service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:42.536895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631347b7-4111-41e1-bb72-24b210ae6f4e service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:42.537179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631347b7-4111-41e1-bb72-24b210ae6f4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:42.540115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-631347b7-4111-41e1-bb72-24b210ae6f4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:42.576058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-559b8e01-9031-4f75-82c8-a69f1935b652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb7aa5f514d44c85b3c9863abf2d9d64'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:42.606007 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1218/2431] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:42 2022] DELETE /identity/v3/users/bb7aa5f514d44c85b3c9863abf2d9d64 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:42.611159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ab899-48a8-4b77-bde7-16fb279b578c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:42.611785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ab899-48a8-4b77-bde7-16fb279b578c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:42.612238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ab899-48a8-4b77-bde7-16fb279b578c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:42.643783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631347b7-4111-41e1-bb72-24b210ae6f4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:42.646246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631347b7-4111-41e1-bb72-24b210ae6f4e service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:42.661406 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1214/2432] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:42.664233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a63ab899-48a8-4b77-bde7-16fb279b578c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:42.665495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a63ab899-48a8-4b77-bde7-16fb279b578c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:42.795539 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1219/2433] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:42.812089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:42.834027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631347b7-4111-41e1-bb72-24b210ae6f4e service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:42.835639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-631347b7-4111-41e1-bb72-24b210ae6f4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:42.943085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631347b7-4111-41e1-bb72-24b210ae6f4e service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:43.012746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339154d8-d5bc-4f8c-80c0-2f3b167d03bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:43.013627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339154d8-d5bc-4f8c-80c0-2f3b167d03bd service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:43.014044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339154d8-d5bc-4f8c-80c0-2f3b167d03bd service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:43.015746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339154d8-d5bc-4f8c-80c0-2f3b167d03bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:43.029809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339154d8-d5bc-4f8c-80c0-2f3b167d03bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:43.032847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339154d8-d5bc-4f8c-80c0-2f3b167d03bd service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:43.048975 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1215/2434] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:04:43.233685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:43.516415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb522314-e1ca-4634-b4cd-d315680e5001 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:43.517084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb522314-e1ca-4634-b4cd-d315680e5001 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:43.517598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb522314-e1ca-4634-b4cd-d315680e5001 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:43.519576 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1220/2435] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:04:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:04:43.531177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5586f4-8755-4b66-b8d5-36e80fe6e93a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:43.532289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5586f4-8755-4b66-b8d5-36e80fe6e93a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:43.532289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5586f4-8755-4b66-b8d5-36e80fe6e93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:43.586598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1a5586f4-8755-4b66-b8d5-36e80fe6e93a None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:43.586598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1a5586f4-8755-4b66-b8d5-36e80fe6e93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:43.701171 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1216/2436] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:04:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:43.721690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb522314-e1ca-4634-b4cd-d315680e5001 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:43.724301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eb522314-e1ca-4634-b4cd-d315680e5001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:43.756272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb522314-e1ca-4634-b4cd-d315680e5001 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:43.777695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a9550-2bb2-4dca-b2d4-07b1923c52b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:43.777695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a9550-2bb2-4dca-b2d4-07b1923c52b4 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:43.777695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a9550-2bb2-4dca-b2d4-07b1923c52b4 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:43.777695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a9550-2bb2-4dca-b2d4-07b1923c52b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:43.783816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a311911-68c3-435a-89a6-c760cd696b87 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:43.784532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a311911-68c3-435a-89a6-c760cd696b87 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:43.784970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a311911-68c3-435a-89a6-c760cd696b87 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:43.788183 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1217/2437] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:04:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 21:04:43.796061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a9550-2bb2-4dca-b2d4-07b1923c52b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:43.796607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b413d7f-b12d-43e3-bc2b-794c59ec54c9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:43.796607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b413d7f-b12d-43e3-bc2b-794c59ec54c9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:43.797184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b413d7f-b12d-43e3-bc2b-794c59ec54c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:43.801788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a9550-2bb2-4dca-b2d4-07b1923c52b4 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:43.817405 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1221/2438] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:43.855498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5b413d7f-b12d-43e3-bc2b-794c59ec54c9 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:43.858069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5b413d7f-b12d-43e3-bc2b-794c59ec54c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:43.876829 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1218/2439] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:04:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:43.913944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:43.921880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a9550-2bb2-4dca-b2d4-07b1923c52b4 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:43.923733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fa9a9550-2bb2-4dca-b2d4-07b1923c52b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:43.944340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a9550-2bb2-4dca-b2d4-07b1923c52b4 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:43.958584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9420ee-be45-4d03-8cd8-e3bb63ee9fa7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:43.959253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9420ee-be45-4d03-8cd8-e3bb63ee9fa7 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:43.959593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9420ee-be45-4d03-8cd8-e3bb63ee9fa7 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:43.959813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9420ee-be45-4d03-8cd8-e3bb63ee9fa7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:43.978077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9420ee-be45-4d03-8cd8-e3bb63ee9fa7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:43.979754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9420ee-be45-4d03-8cd8-e3bb63ee9fa7 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:44.001788 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1222/2440] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:44.035080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b413d7f-b12d-43e3-bc2b-794c59ec54c9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:44.037188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5b413d7f-b12d-43e3-bc2b-794c59ec54c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:44.064689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b413d7f-b12d-43e3-bc2b-794c59ec54c9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:44.081217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8185788a-9d0b-4a03-9be3-c34feae3eb00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:44.082159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185788a-9d0b-4a03-9be3-c34feae3eb00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:44.082509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185788a-9d0b-4a03-9be3-c34feae3eb00 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:44.082709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8185788a-9d0b-4a03-9be3-c34feae3eb00 admin admin] PATH_INFO: `/v3/projects/16cb773714aa47299fa289c53ff1121f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:44.087070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185788a-9d0b-4a03-9be3-c34feae3eb00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16cb773714aa47299fa289c53ff1121f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:44.089763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8185788a-9d0b-4a03-9be3-c34feae3eb00 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:44.259087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8185788a-9d0b-4a03-9be3-c34feae3eb00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16cb773714aa47299fa289c53ff1121f', 'request_id': 'req-8185788a-9d0b-4a03-9be3-c34feae3eb00'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:44.294329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8185788a-9d0b-4a03-9be3-c34feae3eb00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16cb773714aa47299fa289c53ff1121f', 'request_id': 'req-8185788a-9d0b-4a03-9be3-c34feae3eb00'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:44.374268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8185788a-9d0b-4a03-9be3-c34feae3eb00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16cb773714aa47299fa289c53ff1121f', 'request_id': 'req-8185788a-9d0b-4a03-9be3-c34feae3eb00'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:44.406890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8185788a-9d0b-4a03-9be3-c34feae3eb00 admin admin] The token cache is being invalidate because project 16cb773714aa47299fa289c53ff1121f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:44.406890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8185788a-9d0b-4a03-9be3-c34feae3eb00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:44.410718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8185788a-9d0b-4a03-9be3-c34feae3eb00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:44.418920 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1219/2441] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:44 2022] DELETE /identity/v3/projects/16cb773714aa47299fa289c53ff1121f => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:44.420669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9420ee-be45-4d03-8cd8-e3bb63ee9fa7 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:44.434541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9420ee-be45-4d03-8cd8-e3bb63ee9fa7 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:44.572365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a45cf7-9e79-4599-a017-2e73577eee00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:44.574664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a45cf7-9e79-4599-a017-2e73577eee00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:44.575198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a45cf7-9e79-4599-a017-2e73577eee00 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:44.575687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a45cf7-9e79-4599-a017-2e73577eee00 admin admin] PATH_INFO: `/v3/users/ada3c5d65541405b80d97cd03cb507bf` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:44.577407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a45cf7-9e79-4599-a017-2e73577eee00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ada3c5d65541405b80d97cd03cb507bf)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:44.579869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a45cf7-9e79-4599-a017-2e73577eee00 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:44.819361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-39a45cf7-9e79-4599-a017-2e73577eee00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ada3c5d65541405b80d97cd03cb507bf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:44.839427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-39a45cf7-9e79-4599-a017-2e73577eee00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ada3c5d65541405b80d97cd03cb507bf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:44.840717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-39a45cf7-9e79-4599-a017-2e73577eee00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ada3c5d65541405b80d97cd03cb507bf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:44.843523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-39a45cf7-9e79-4599-a017-2e73577eee00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ada3c5d65541405b80d97cd03cb507bf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:44.930258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:44.969780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-39a45cf7-9e79-4599-a017-2e73577eee00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ada3c5d65541405b80d97cd03cb507bf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:44.991150 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1223/2442] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:44 2022] DELETE /identity/v3/users/ada3c5d65541405b80d97cd03cb507bf => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:45.499280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40806df-7de5-44a6-84ed-977648d9ce7d None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:45.500242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40806df-7de5-44a6-84ed-977648d9ce7d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:45.500747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40806df-7de5-44a6-84ed-977648d9ce7d None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:45.502436 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1220/2443] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:04:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:04:45.514653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1cdb2-f4c8-436f-b6c9-b27ac3e9ae48 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:45.515333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1cdb2-f4c8-436f-b6c9-b27ac3e9ae48 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:45.516213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1cdb2-f4c8-436f-b6c9-b27ac3e9ae48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:45.549448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f6d1cdb2-f4c8-436f-b6c9-b27ac3e9ae48 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:45.550918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f6d1cdb2-f4c8-436f-b6c9-b27ac3e9ae48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:45.559750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:45.627593 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1224/2444] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:04:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:45.644734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40806df-7de5-44a6-84ed-977648d9ce7d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:45.647287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e40806df-7de5-44a6-84ed-977648d9ce7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:45.720848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40806df-7de5-44a6-84ed-977648d9ce7d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:45.728685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4796d-e445-4f50-a067-d58d808765c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:45.728826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4796d-e445-4f50-a067-d58d808765c1 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:45.729196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4796d-e445-4f50-a067-d58d808765c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:45.729645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4796d-e445-4f50-a067-d58d808765c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:45.741600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4796d-e445-4f50-a067-d58d808765c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:45.743106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4796d-e445-4f50-a067-d58d808765c1 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:45.756941 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1221/2445] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:04:45.759719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1cdb2-f4c8-436f-b6c9-b27ac3e9ae48 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:45.761678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f6d1cdb2-f4c8-436f-b6c9-b27ac3e9ae48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:45.777682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1be2d-57f7-42d1-aae1-a14deb7a9dcb None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:45.778197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1be2d-57f7-42d1-aae1-a14deb7a9dcb None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:45.778799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1be2d-57f7-42d1-aae1-a14deb7a9dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:45.806812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-eae1be2d-57f7-42d1-aae1-a14deb7a9dcb None None] MFA Rules not processed for user `1b2a9ebcd3f84b758322a80c6e590de0`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:45.808318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-eae1be2d-57f7-42d1-aae1-a14deb7a9dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:45.829363 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1222/2446] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:04:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:45.830444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1be2d-57f7-42d1-aae1-a14deb7a9dcb None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:45.844353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1be2d-57f7-42d1-aae1-a14deb7a9dcb None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:45.856246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aace9db6-130e-41fb-a394-ac5508c9e77d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:45.857222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace9db6-130e-41fb-a394-ac5508c9e77d service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:45.872546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace9db6-130e-41fb-a394-ac5508c9e77d service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:45.872546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aace9db6-130e-41fb-a394-ac5508c9e77d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:45.872793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace9db6-130e-41fb-a394-ac5508c9e77d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:45.874239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aace9db6-130e-41fb-a394-ac5508c9e77d service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:45.902078 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1223/2447] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:45.940854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1cdb2-f4c8-436f-b6c9-b27ac3e9ae48 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:45.946985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236455e9-fa5c-4028-8682-d7476030ca5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:45.948398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236455e9-fa5c-4028-8682-d7476030ca5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:45.948846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236455e9-fa5c-4028-8682-d7476030ca5b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:45.949318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236455e9-fa5c-4028-8682-d7476030ca5b admin admin] PATH_INFO: `/v3/projects/68f8fc2f557e448d9ae5ce0743b23f86` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:45.951347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236455e9-fa5c-4028-8682-d7476030ca5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=68f8fc2f557e448d9ae5ce0743b23f86)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:45.953614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236455e9-fa5c-4028-8682-d7476030ca5b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:46.109818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-236455e9-fa5c-4028-8682-d7476030ca5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68f8fc2f557e448d9ae5ce0743b23f86', 'request_id': 'req-236455e9-fa5c-4028-8682-d7476030ca5b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:46.118132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-236455e9-fa5c-4028-8682-d7476030ca5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68f8fc2f557e448d9ae5ce0743b23f86', 'request_id': 'req-236455e9-fa5c-4028-8682-d7476030ca5b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:46.135727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372dac5-a6e9-4d6f-9120-072fc1687a9f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:46.136030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372dac5-a6e9-4d6f-9120-072fc1687a9f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:46.136317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2372dac5-a6e9-4d6f-9120-072fc1687a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:46.172250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2372dac5-a6e9-4d6f-9120-072fc1687a9f None None] MFA Rules not processed for user `77a6068e4dc34fb6b8d1447472692e55`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:46.174511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2372dac5-a6e9-4d6f-9120-072fc1687a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:46.210845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-236455e9-fa5c-4028-8682-d7476030ca5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68f8fc2f557e448d9ae5ce0743b23f86', 'request_id': 'req-236455e9-fa5c-4028-8682-d7476030ca5b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:46.228484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-236455e9-fa5c-4028-8682-d7476030ca5b admin admin] The token cache is being invalidate because project 68f8fc2f557e448d9ae5ce0743b23f86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:46.229240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-236455e9-fa5c-4028-8682-d7476030ca5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:46.233209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-236455e9-fa5c-4028-8682-d7476030ca5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:46.238673 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1225/2448] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:45 2022] DELETE /identity/v3/projects/68f8fc2f557e448d9ae5ce0743b23f86 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:46.242954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692da066-ab2b-4188-a9ba-68966f0930f2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:46.243324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692da066-ab2b-4188-a9ba-68966f0930f2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:46.243675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692da066-ab2b-4188-a9ba-68966f0930f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:46.273573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-692da066-ab2b-4188-a9ba-68966f0930f2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:46.275276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-692da066-ab2b-4188-a9ba-68966f0930f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:46.291493 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1224/2449] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:04:46.301138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372dac5-a6e9-4d6f-9120-072fc1687a9f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:46.303748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2372dac5-a6e9-4d6f-9120-072fc1687a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:46.383506 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1226/2450] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:46.393302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdd67f-3e75-4f82-819d-c7c8a8f8f4c8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:46.396308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdd67f-3e75-4f82-819d-c7c8a8f8f4c8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:46.396308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdd67f-3e75-4f82-819d-c7c8a8f8f4c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:46.420641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-dcbdd67f-3e75-4f82-819d-c7c8a8f8f4c8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:46.422019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dcbdd67f-3e75-4f82-819d-c7c8a8f8f4c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:46.422571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2372dac5-a6e9-4d6f-9120-072fc1687a9f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:46.429986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4822b85-2534-4bf2-aba3-9e09a6b41b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:46.430886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4822b85-2534-4bf2-aba3-9e09a6b41b97 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:46.431178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4822b85-2534-4bf2-aba3-9e09a6b41b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:46.431469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4822b85-2534-4bf2-aba3-9e09a6b41b97 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:46.432183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4822b85-2534-4bf2-aba3-9e09a6b41b97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:46.434234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4822b85-2534-4bf2-aba3-9e09a6b41b97 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:46.436286 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1227/2451] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:04:46.440947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdd67f-3e75-4f82-819d-c7c8a8f8f4c8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:46.459746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbdd67f-3e75-4f82-819d-c7c8a8f8f4c8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:46.473663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2f47a-4072-4fb6-a36d-1f76c0b348f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:46.474567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2f47a-4072-4fb6-a36d-1f76c0b348f1 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:46.474851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2f47a-4072-4fb6-a36d-1f76c0b348f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:46.475129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2f47a-4072-4fb6-a36d-1f76c0b348f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:46.476507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2f47a-4072-4fb6-a36d-1f76c0b348f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:46.476816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2f47a-4072-4fb6-a36d-1f76c0b348f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:46.478480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2f47a-4072-4fb6-a36d-1f76c0b348f1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:46.496295 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1228/2452] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:46.500568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2f47a-4072-4fb6-a36d-1f76c0b348f1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:46.519560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2f47a-4072-4fb6-a36d-1f76c0b348f1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:46.527979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddfc99-a05e-41d2-a2f9-7e76ea3d467d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:46.528823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddfc99-a05e-41d2-a2f9-7e76ea3d467d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:46.529319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddfc99-a05e-41d2-a2f9-7e76ea3d467d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:46.529676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddfc99-a05e-41d2-a2f9-7e76ea3d467d admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:46.530180 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1225/2453] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:46 2022] POST /identity/v3/projects => generated 385 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:46.531219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4822b85-2534-4bf2-aba3-9e09a6b41b97 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:46.531437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddfc99-a05e-41d2-a2f9-7e76ea3d467d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:46.537767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddfc99-a05e-41d2-a2f9-7e76ea3d467d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:46.627453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4822b85-2534-4bf2-aba3-9e09a6b41b97 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:46.629575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362606b6-d66b-4282-a83e-b6064493dc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:46.631220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362606b6-d66b-4282-a83e-b6064493dc34 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:46.631515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362606b6-d66b-4282-a83e-b6064493dc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:46.631943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362606b6-d66b-4282-a83e-b6064493dc34 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:46.632667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362606b6-d66b-4282-a83e-b6064493dc34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:46.634797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362606b6-d66b-4282-a83e-b6064493dc34 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:46.658135 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1229/2454] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:46 2022] POST /identity/v3/projects => generated 391 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:46.659157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddfc99-a05e-41d2-a2f9-7e76ea3d467d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:46.734476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddfc99-a05e-41d2-a2f9-7e76ea3d467d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:46.738047 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1226/2455] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:46 2022] POST /identity/v3/users => generated 299 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:46.741225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362606b6-d66b-4282-a83e-b6064493dc34 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:46.753222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412aed7-5779-4398-bdf1-d6672cf25006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:46.753222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412aed7-5779-4398-bdf1-d6672cf25006 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:46.753222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412aed7-5779-4398-bdf1-d6672cf25006 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:46.753222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412aed7-5779-4398-bdf1-d6672cf25006 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:46.758362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412aed7-5779-4398-bdf1-d6672cf25006 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:46.759059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362606b6-d66b-4282-a83e-b6064493dc34 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:46.761194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412aed7-5779-4398-bdf1-d6672cf25006 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:46.765007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf030b57-88ca-4c45-8ddf-11feaef2522d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:46.765812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf030b57-88ca-4c45-8ddf-11feaef2522d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:46.766391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf030b57-88ca-4c45-8ddf-11feaef2522d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:46.766887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf030b57-88ca-4c45-8ddf-11feaef2522d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:46.767885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf030b57-88ca-4c45-8ddf-11feaef2522d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:46.770074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf030b57-88ca-4c45-8ddf-11feaef2522d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:46.783553 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1227/2456] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:46.799021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf030b57-88ca-4c45-8ddf-11feaef2522d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:46.821951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf030b57-88ca-4c45-8ddf-11feaef2522d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:46.822911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a6ea2-8246-459f-84bc-9d8a41ad79f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:46.824020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a6ea2-8246-459f-84bc-9d8a41ad79f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:46.824530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a6ea2-8246-459f-84bc-9d8a41ad79f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:46.825140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a6ea2-8246-459f-84bc-9d8a41ad79f4 admin admin] PATH_INFO: `/v3/projects/ea3cafb9299747efbe223b5302b76d47/users/1229cf9547964df18d40762b3f71a530/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:46.845468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a6ea2-8246-459f-84bc-9d8a41ad79f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea3cafb9299747efbe223b5302b76d47, user_id=1229cf9547964df18d40762b3f71a530, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:46.851581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a6ea2-8246-459f-84bc-9d8a41ad79f4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:46.889596 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1230/2457] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:46 2022] POST /identity/v3/users => generated 296 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:04:46.892895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412aed7-5779-4398-bdf1-d6672cf25006 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:46.926623 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1228/2458] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:46 2022] PUT /identity/v3/projects/ea3cafb9299747efbe223b5302b76d47/users/1229cf9547964df18d40762b3f71a530/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:46.930898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a6ea2-8246-459f-84bc-9d8a41ad79f4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:46.931287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412aed7-5779-4398-bdf1-d6672cf25006 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:47.078295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a6ea2-8246-459f-84bc-9d8a41ad79f4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:47.097895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82a123-f524-4922-a299-9294d3580263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:47.102675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82a123-f524-4922-a299-9294d3580263 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:47.102675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82a123-f524-4922-a299-9294d3580263 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:47.102675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82a123-f524-4922-a299-9294d3580263 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:47.102675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82a123-f524-4922-a299-9294d3580263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:47.103081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193b0b4-6e78-49f1-9304-0f81377d6454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:47.103081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193b0b4-6e78-49f1-9304-0f81377d6454 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:47.103587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193b0b4-6e78-49f1-9304-0f81377d6454 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:47.103688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d82a123-f524-4922-a299-9294d3580263 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:47.103777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193b0b4-6e78-49f1-9304-0f81377d6454 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:47.104415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e193b0b4-6e78-49f1-9304-0f81377d6454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:47.118653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e193b0b4-6e78-49f1-9304-0f81377d6454 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:47.124341 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1229/2459] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:04:47.133561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82a123-f524-4922-a299-9294d3580263 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:47.142856 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1231/2460] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:47.144941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193b0b4-6e78-49f1-9304-0f81377d6454 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:47.162219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193b0b4-6e78-49f1-9304-0f81377d6454 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:47.162525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82a123-f524-4922-a299-9294d3580263 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:47.172025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f60e1-fd23-43b7-8d91-1d60cbe7361a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:47.172827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f60e1-fd23-43b7-8d91-1d60cbe7361a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:47.173163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f60e1-fd23-43b7-8d91-1d60cbe7361a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:47.173473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147f60e1-fd23-43b7-8d91-1d60cbe7361a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:47.174521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f60e1-fd23-43b7-8d91-1d60cbe7361a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:47.176589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b555bdd9-2cbf-4d11-86fe-780a7a0e8759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:47.178015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555bdd9-2cbf-4d11-86fe-780a7a0e8759 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:47.178511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555bdd9-2cbf-4d11-86fe-780a7a0e8759 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:47.178937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b555bdd9-2cbf-4d11-86fe-780a7a0e8759 admin admin] PATH_INFO: `/v3/projects/ea3cafb9299747efbe223b5302b76d47/users/1229cf9547964df18d40762b3f71a530/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:47.183848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555bdd9-2cbf-4d11-86fe-780a7a0e8759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea3cafb9299747efbe223b5302b76d47, user_id=1229cf9547964df18d40762b3f71a530, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:47.184686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147f60e1-fd23-43b7-8d91-1d60cbe7361a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:47.191290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b555bdd9-2cbf-4d11-86fe-780a7a0e8759 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:47.210287 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1232/2461] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:47.212089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f60e1-fd23-43b7-8d91-1d60cbe7361a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:47.218200 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1230/2462] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:47 2022] PUT /identity/v3/projects/ea3cafb9299747efbe223b5302b76d47/users/1229cf9547964df18d40762b3f71a530/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:47.230911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147f60e1-fd23-43b7-8d91-1d60cbe7361a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:47.305536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66c385-1c63-43d9-afc6-7684311bb059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:47.306504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66c385-1c63-43d9-afc6-7684311bb059 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:47.307174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66c385-1c63-43d9-afc6-7684311bb059 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:47.315352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66c385-1c63-43d9-afc6-7684311bb059 admin admin] PATH_INFO: `/v3/projects/8058694a4ecb467783677e687e16ecb6/users/3b044baceddf49219485bd14aa914e35/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:47.329419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66c385-1c63-43d9-afc6-7684311bb059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8058694a4ecb467783677e687e16ecb6, user_id=3b044baceddf49219485bd14aa914e35, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:47.335958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b66c385-1c63-43d9-afc6-7684311bb059 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:47.545969 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1233/2463] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:47 2022] PUT /identity/v3/projects/8058694a4ecb467783677e687e16ecb6/users/3b044baceddf49219485bd14aa914e35/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:47.550624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a9b03-abb0-4592-be83-801f31046b2f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:47.551182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a9b03-abb0-4592-be83-801f31046b2f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:47.551182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76a9b03-abb0-4592-be83-801f31046b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:47.589217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d76a9b03-abb0-4592-be83-801f31046b2f None None] MFA Rules not processed for user `3b044baceddf49219485bd14aa914e35`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:47.591949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d76a9b03-abb0-4592-be83-801f31046b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:47.691810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66c385-1c63-43d9-afc6-7684311bb059 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:47.695194 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1231/2464] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:47.781240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66c385-1c63-43d9-afc6-7684311bb059 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:47.790148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c687a-99fa-474c-98aa-5300d8255988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:47.791481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c687a-99fa-474c-98aa-5300d8255988 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:47.791953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c687a-99fa-474c-98aa-5300d8255988 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:47.792409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436c687a-99fa-474c-98aa-5300d8255988 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:47.793309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c687a-99fa-474c-98aa-5300d8255988 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:47.796578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436c687a-99fa-474c-98aa-5300d8255988 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:47.850777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:47.926419 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1234/2465] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:47 2022] POST /identity/v3/projects => generated 393 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:47.931015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a9b03-abb0-4592-be83-801f31046b2f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:47.946451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76a9b03-abb0-4592-be83-801f31046b2f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:48.096313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b879f-e209-4107-b217-d47f0a8ce7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:48.096587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b879f-e209-4107-b217-d47f0a8ce7d8 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:48.097195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b879f-e209-4107-b217-d47f0a8ce7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:48.097308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b879f-e209-4107-b217-d47f0a8ce7d8 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:48.098008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b879f-e209-4107-b217-d47f0a8ce7d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:48.102266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2b879f-e209-4107-b217-d47f0a8ce7d8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:48.253283 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1232/2466] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:47 2022] POST /identity/v3/users => generated 303 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:48.259193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c687a-99fa-474c-98aa-5300d8255988 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:48.279329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436c687a-99fa-474c-98aa-5300d8255988 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:48.291768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d4884-45ea-4bc9-90ff-e8002ce72965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:48.293643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d4884-45ea-4bc9-90ff-e8002ce72965 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:48.296785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d4884-45ea-4bc9-90ff-e8002ce72965 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:48.297900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d4884-45ea-4bc9-90ff-e8002ce72965 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:48.298888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d4884-45ea-4bc9-90ff-e8002ce72965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:48.301717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d4884-45ea-4bc9-90ff-e8002ce72965 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:48.331937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b879f-e209-4107-b217-d47f0a8ce7d8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:48.333091 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1235/2467] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:48.342766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b879f-e209-4107-b217-d47f0a8ce7d8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:48.348522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de752528-6679-4712-9db1-dd54ae65ba62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:48.349489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de752528-6679-4712-9db1-dd54ae65ba62 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:48.349828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de752528-6679-4712-9db1-dd54ae65ba62 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:48.350206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de752528-6679-4712-9db1-dd54ae65ba62 admin admin] PATH_INFO: `/v3/projects/7c6c7fd995bf4b41873c30015cbc85a4/users/1edb01ef9d394c9da5086206d3033403/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:48.370241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de752528-6679-4712-9db1-dd54ae65ba62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c6c7fd995bf4b41873c30015cbc85a4, user_id=1edb01ef9d394c9da5086206d3033403, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:48.381426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de752528-6679-4712-9db1-dd54ae65ba62 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:48.454101 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1233/2468] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:48 2022] PUT /identity/v3/projects/7c6c7fd995bf4b41873c30015cbc85a4/users/1edb01ef9d394c9da5086206d3033403/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:48.460700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d4884-45ea-4bc9-90ff-e8002ce72965 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:48.475881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d4884-45ea-4bc9-90ff-e8002ce72965 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:48.497416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:48.595532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679e52b-1949-4d02-aa57-f4c6ded50d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:48.597000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679e52b-1949-4d02-aa57-f4c6ded50d55 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:48.597557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679e52b-1949-4d02-aa57-f4c6ded50d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:48.598099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679e52b-1949-4d02-aa57-f4c6ded50d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:48.599083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679e52b-1949-4d02-aa57-f4c6ded50d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:48.601054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679e52b-1949-4d02-aa57-f4c6ded50d55 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:48.616256 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1236/2469] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:48.620384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de752528-6679-4712-9db1-dd54ae65ba62 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:48.646237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de752528-6679-4712-9db1-dd54ae65ba62 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:48.660605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b3229-6961-40eb-8b75-c3d9edd2b4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:48.662714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b3229-6961-40eb-8b75-c3d9edd2b4dc admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:48.663009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b3229-6961-40eb-8b75-c3d9edd2b4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:48.665039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b3229-6961-40eb-8b75-c3d9edd2b4dc admin admin] PATH_INFO: `/v3/projects/7c6c7fd995bf4b41873c30015cbc85a4/users/1edb01ef9d394c9da5086206d3033403/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:48.671499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b3229-6961-40eb-8b75-c3d9edd2b4dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c6c7fd995bf4b41873c30015cbc85a4, user_id=1edb01ef9d394c9da5086206d3033403, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:48.678036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b3229-6961-40eb-8b75-c3d9edd2b4dc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:48.704278 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1234/2470] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:48 2022] PUT /identity/v3/projects/7c6c7fd995bf4b41873c30015cbc85a4/users/1edb01ef9d394c9da5086206d3033403/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:04:48.705123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac2c255-c73b-4d39-bd74-a9dae77dd57a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:48.705482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac2c255-c73b-4d39-bd74-a9dae77dd57a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:48.705784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac2c255-c73b-4d39-bd74-a9dae77dd57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:48.782026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-eac2c255-c73b-4d39-bd74-a9dae77dd57a None None] MFA Rules not processed for user `1edb01ef9d394c9da5086206d3033403`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:48.783949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eac2c255-c73b-4d39-bd74-a9dae77dd57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:48.885078 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1237/2471] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:04:48.900479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1826f-3ecb-44bc-8696-1ee6bcdab723 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:48.900808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1826f-3ecb-44bc-8696-1ee6bcdab723 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:48.901040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca1826f-3ecb-44bc-8696-1ee6bcdab723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:48.930869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-eca1826f-3ecb-44bc-8696-1ee6bcdab723 None None] MFA Rules not processed for user `3b044baceddf49219485bd14aa914e35`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:48.932538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-eca1826f-3ecb-44bc-8696-1ee6bcdab723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:49.078593 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1235/2472] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:49.496536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:49.742315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac2c255-c73b-4d39-bd74-a9dae77dd57a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:49.744084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eac2c255-c73b-4d39-bd74-a9dae77dd57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:49.852927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac2c255-c73b-4d39-bd74-a9dae77dd57a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:49.860443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96bc94-a83d-4662-9bab-64a0b79d1b64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:49.861739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96bc94-a83d-4662-9bab-64a0b79d1b64 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:49.862277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96bc94-a83d-4662-9bab-64a0b79d1b64 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:49.862777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96bc94-a83d-4662-9bab-64a0b79d1b64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:49.886058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96bc94-a83d-4662-9bab-64a0b79d1b64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:49.894832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96bc94-a83d-4662-9bab-64a0b79d1b64 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:49.906466 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1238/2473] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:50.049825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:50.213281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1826f-3ecb-44bc-8696-1ee6bcdab723 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:50.229116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-eca1826f-3ecb-44bc-8696-1ee6bcdab723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:50.414437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca1826f-3ecb-44bc-8696-1ee6bcdab723 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:50.432053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a222c-81d5-41c7-a82b-51953ffb88a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:50.433039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a222c-81d5-41c7-a82b-51953ffb88a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:50.433369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a222c-81d5-41c7-a82b-51953ffb88a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:50.433805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a222c-81d5-41c7-a82b-51953ffb88a4 admin admin] PATH_INFO: `/v3/users/02ba016624f54a7896a1fcd9649d467e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:50.435199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a222c-81d5-41c7-a82b-51953ffb88a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02ba016624f54a7896a1fcd9649d467e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:50.437269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a222c-81d5-41c7-a82b-51953ffb88a4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:50.711963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1b4a222c-81d5-41c7-a82b-51953ffb88a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02ba016624f54a7896a1fcd9649d467e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:50.712671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1b4a222c-81d5-41c7-a82b-51953ffb88a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02ba016624f54a7896a1fcd9649d467e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:50.734005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1b4a222c-81d5-41c7-a82b-51953ffb88a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02ba016624f54a7896a1fcd9649d467e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:50.838092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1b4a222c-81d5-41c7-a82b-51953ffb88a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02ba016624f54a7896a1fcd9649d467e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:50.839302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1b4a222c-81d5-41c7-a82b-51953ffb88a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02ba016624f54a7896a1fcd9649d467e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:50.861135 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1236/2474] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:50 2022] DELETE /identity/v3/users/02ba016624f54a7896a1fcd9649d467e => generated 0 bytes in 649 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:50.866300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b4ef0-9a00-4de2-b5d2-a2edb3fcff76 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:50.867131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b4ef0-9a00-4de2-b5d2-a2edb3fcff76 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:50.867570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b4ef0-9a00-4de2-b5d2-a2edb3fcff76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:50.906829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-528b4ef0-9a00-4de2-b5d2-a2edb3fcff76 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:50.908347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-528b4ef0-9a00-4de2-b5d2-a2edb3fcff76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:51.037476 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1239/2475] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:04:51.044531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:51.048446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a222c-81d5-41c7-a82b-51953ffb88a4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:51.050111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1b4a222c-81d5-41c7-a82b-51953ffb88a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:51.184675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a222c-81d5-41c7-a82b-51953ffb88a4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:51.191179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989cde5-a5ca-4cf9-84ed-1baf12a213d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:51.192193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989cde5-a5ca-4cf9-84ed-1baf12a213d1 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:51.192619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989cde5-a5ca-4cf9-84ed-1baf12a213d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:51.192996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989cde5-a5ca-4cf9-84ed-1baf12a213d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:51.205272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989cde5-a5ca-4cf9-84ed-1baf12a213d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:51.207549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989cde5-a5ca-4cf9-84ed-1baf12a213d1 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:51.252532 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1237/2476] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:51.530940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:52.052147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b4ef0-9a00-4de2-b5d2-a2edb3fcff76 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:52.055240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-528b4ef0-9a00-4de2-b5d2-a2edb3fcff76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:52.073566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b4ef0-9a00-4de2-b5d2-a2edb3fcff76 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:52.082300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:52.084038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:52.084792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:52.085148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1 admin admin] PATH_INFO: `/v3/projects/d3ab11147ee34b828cdc96645cef519b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:52.088390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3ab11147ee34b828cdc96645cef519b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:52.091278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:52.223585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3ab11147ee34b828cdc96645cef519b', 'request_id': 'req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:52.235776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3ab11147ee34b828cdc96645cef519b', 'request_id': 'req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:52.324470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3ab11147ee34b828cdc96645cef519b', 'request_id': 'req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:52.361977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1 admin admin] The token cache is being invalidate because project d3ab11147ee34b828cdc96645cef519b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:52.364105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:52.366526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:52.374084 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1240/2477] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:52 2022] DELETE /identity/v3/projects/d3ab11147ee34b828cdc96645cef519b => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:52.376130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989cde5-a5ca-4cf9-84ed-1baf12a213d1 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:52.394145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989cde5-a5ca-4cf9-84ed-1baf12a213d1 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:52.550443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b45bb-6a57-4de1-879a-7585a07ab255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:52.552758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b45bb-6a57-4de1-879a-7585a07ab255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:52.553367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b45bb-6a57-4de1-879a-7585a07ab255 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:52.553961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b45bb-6a57-4de1-879a-7585a07ab255 admin admin] PATH_INFO: `/v3/users/737585c57e7b4e4ca8ae0ebf80ab4648` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:52.555580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b45bb-6a57-4de1-879a-7585a07ab255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=737585c57e7b4e4ca8ae0ebf80ab4648)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:52.557878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b45bb-6a57-4de1-879a-7585a07ab255 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:52.923819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-447b45bb-6a57-4de1-879a-7585a07ab255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '737585c57e7b4e4ca8ae0ebf80ab4648'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:52.924926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-447b45bb-6a57-4de1-879a-7585a07ab255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '737585c57e7b4e4ca8ae0ebf80ab4648'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:52.937369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-447b45bb-6a57-4de1-879a-7585a07ab255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '737585c57e7b4e4ca8ae0ebf80ab4648'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:53.032948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-447b45bb-6a57-4de1-879a-7585a07ab255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '737585c57e7b4e4ca8ae0ebf80ab4648'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:53.033662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-447b45bb-6a57-4de1-879a-7585a07ab255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '737585c57e7b4e4ca8ae0ebf80ab4648'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:53.064530 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1238/2478] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:04:52 2022] DELETE /identity/v3/users/737585c57e7b4e4ca8ae0ebf80ab4648 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:53.330693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:53.587352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:54.553246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:54.559078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:54.587670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ee9f1-7764-4db9-82d5-03ff701f57e1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:54.663053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50364c-8025-4160-b59f-32fb1a04475c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:54.664073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50364c-8025-4160-b59f-32fb1a04475c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:54.664807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50364c-8025-4160-b59f-32fb1a04475c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:54.665135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50364c-8025-4160-b59f-32fb1a04475c admin admin] PATH_INFO: `/v3/projects/5672aa5675604f1c8e1c90219f5bdbce` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:54.666980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50364c-8025-4160-b59f-32fb1a04475c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5672aa5675604f1c8e1c90219f5bdbce)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:54.669289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50364c-8025-4160-b59f-32fb1a04475c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:54.814645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6d50364c-8025-4160-b59f-32fb1a04475c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5672aa5675604f1c8e1c90219f5bdbce', 'request_id': 'req-6d50364c-8025-4160-b59f-32fb1a04475c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:54.823147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6d50364c-8025-4160-b59f-32fb1a04475c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5672aa5675604f1c8e1c90219f5bdbce', 'request_id': 'req-6d50364c-8025-4160-b59f-32fb1a04475c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:54.889911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6d50364c-8025-4160-b59f-32fb1a04475c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5672aa5675604f1c8e1c90219f5bdbce', 'request_id': 'req-6d50364c-8025-4160-b59f-32fb1a04475c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:54.912336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6d50364c-8025-4160-b59f-32fb1a04475c admin admin] The token cache is being invalidate because project 5672aa5675604f1c8e1c90219f5bdbce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:04:54.912768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6d50364c-8025-4160-b59f-32fb1a04475c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:54.914080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6d50364c-8025-4160-b59f-32fb1a04475c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:04:54.915893 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1241/2479] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:04:54 2022] DELETE /identity/v3/projects/5672aa5675604f1c8e1c90219f5bdbce => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:54.921309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d958c6-7bd8-40f2-93db-9c18a3642335 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:54.921823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d958c6-7bd8-40f2-93db-9c18a3642335 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:54.922247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d958c6-7bd8-40f2-93db-9c18a3642335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:54.946233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-44d958c6-7bd8-40f2-93db-9c18a3642335 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:54.947650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-44d958c6-7bd8-40f2-93db-9c18a3642335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:55.042612 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1239/2480] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:04:55.055976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960a64f-53f5-4449-9ba2-05d00cfd7a3d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:55.056347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960a64f-53f5-4449-9ba2-05d00cfd7a3d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:55.057084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960a64f-53f5-4449-9ba2-05d00cfd7a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:55.078499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4960a64f-53f5-4449-9ba2-05d00cfd7a3d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:55.079730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4960a64f-53f5-4449-9ba2-05d00cfd7a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:55.089971 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1242/2481] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:04:55.094468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d958c6-7bd8-40f2-93db-9c18a3642335 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:55.114219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d958c6-7bd8-40f2-93db-9c18a3642335 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:55.123214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272d870-b676-478c-814e-d05d8893ffd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:55.124116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272d870-b676-478c-814e-d05d8893ffd7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:55.126590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272d870-b676-478c-814e-d05d8893ffd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:55.127567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7272d870-b676-478c-814e-d05d8893ffd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:55.128790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272d870-b676-478c-814e-d05d8893ffd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:55.129396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272d870-b676-478c-814e-d05d8893ffd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:55.132563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7272d870-b676-478c-814e-d05d8893ffd7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:55.155268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960a64f-53f5-4449-9ba2-05d00cfd7a3d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:55.156700 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1240/2482] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:55.159046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:55.168406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960a64f-53f5-4449-9ba2-05d00cfd7a3d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:55.187271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4342481-330a-43af-9582-eef7d938e662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:55.188672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4342481-330a-43af-9582-eef7d938e662 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:55.189131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4342481-330a-43af-9582-eef7d938e662 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:55.189575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4342481-330a-43af-9582-eef7d938e662 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:55.190496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4342481-330a-43af-9582-eef7d938e662 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:55.193378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4342481-330a-43af-9582-eef7d938e662 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:55.321874 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1243/2483] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:55 2022] POST /identity/v3/projects => generated 375 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:04:55.322777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272d870-b676-478c-814e-d05d8893ffd7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:55.336570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7272d870-b676-478c-814e-d05d8893ffd7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:55.342760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f06b8-700a-495f-95cf-399f084da2ce None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:55.343057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f06b8-700a-495f-95cf-399f084da2ce None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:55.343345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f06b8-700a-495f-95cf-399f084da2ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:55.364981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9e2f06b8-700a-495f-95cf-399f084da2ce None None] MFA Rules not processed for user `478d1609aa114d16b338e07d2d4c05c9`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:55.366260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9e2f06b8-700a-495f-95cf-399f084da2ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:55.393268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:55.446602 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1244/2484] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:55.468693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f8ad0-3dc7-481a-a641-dbcb062182be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:55.469746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f8ad0-3dc7-481a-a641-dbcb062182be admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:55.471292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f8ad0-3dc7-481a-a641-dbcb062182be admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:55.471791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f8ad0-3dc7-481a-a641-dbcb062182be admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:55.475459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f8ad0-3dc7-481a-a641-dbcb062182be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:55.480423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f8ad0-3dc7-481a-a641-dbcb062182be admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:55.592057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f06b8-700a-495f-95cf-399f084da2ce None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:55.600404 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1241/2485] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:55 2022] POST /identity/v3/users => generated 288 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:55.605805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f06b8-700a-495f-95cf-399f084da2ce None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:55.611871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efcc0f-d5fd-49f8-8d59-aa75ac05cc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:55.612782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efcc0f-d5fd-49f8-8d59-aa75ac05cc77 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:55.613257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efcc0f-d5fd-49f8-8d59-aa75ac05cc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:55.613706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efcc0f-d5fd-49f8-8d59-aa75ac05cc77 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:55.615487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6efcc0f-d5fd-49f8-8d59-aa75ac05cc77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:55.617295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6efcc0f-d5fd-49f8-8d59-aa75ac05cc77 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:55.635619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f8ad0-3dc7-481a-a641-dbcb062182be admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:55.636272 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1245/2486] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:55.647872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f8ad0-3dc7-481a-a641-dbcb062182be admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:55.654544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47438f64-50cf-48d6-a00a-40a1ec071388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:55.655418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47438f64-50cf-48d6-a00a-40a1ec071388 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:55.655859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47438f64-50cf-48d6-a00a-40a1ec071388 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:55.656287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47438f64-50cf-48d6-a00a-40a1ec071388 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:55.657191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47438f64-50cf-48d6-a00a-40a1ec071388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:55.659459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47438f64-50cf-48d6-a00a-40a1ec071388 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:55.677923 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1242/2487] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:55.679061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efcc0f-d5fd-49f8-8d59-aa75ac05cc77 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:55.691336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efcc0f-d5fd-49f8-8d59-aa75ac05cc77 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:55.699391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95957bd4-0895-4dcd-966f-0b55219bfefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:55.700611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95957bd4-0895-4dcd-966f-0b55219bfefe admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:55.701426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95957bd4-0895-4dcd-966f-0b55219bfefe admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:55.702070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95957bd4-0895-4dcd-966f-0b55219bfefe admin admin] PATH_INFO: `/v3/projects/8692c3aa46b74a2aaa1b0c742e7d6fb8/users/5d0536d93bc54013a921325d3a2213a6/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:55.729303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95957bd4-0895-4dcd-966f-0b55219bfefe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8692c3aa46b74a2aaa1b0c742e7d6fb8, user_id=5d0536d93bc54013a921325d3a2213a6, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:55.733995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95957bd4-0895-4dcd-966f-0b55219bfefe admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:55.857916 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1246/2488] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:55 2022] PUT /identity/v3/projects/8692c3aa46b74a2aaa1b0c742e7d6fb8/users/5d0536d93bc54013a921325d3a2213a6/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:55.872844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a721b-c342-4bc7-bc53-10f5e556aed9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:55.873371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a721b-c342-4bc7-bc53-10f5e556aed9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:55.874060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a721b-c342-4bc7-bc53-10f5e556aed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:55.935178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ba9a721b-c342-4bc7-bc53-10f5e556aed9 None None] MFA Rules not processed for user `5d0536d93bc54013a921325d3a2213a6`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:55.937174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ba9a721b-c342-4bc7-bc53-10f5e556aed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:56.072393 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1243/2489] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:56.076805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95957bd4-0895-4dcd-966f-0b55219bfefe admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:56.202155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95957bd4-0895-4dcd-966f-0b55219bfefe admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:56.214474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dbfb3-9e71-44ff-bafd-6933f44d4f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:56.215507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dbfb3-9e71-44ff-bafd-6933f44d4f63 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:56.215949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dbfb3-9e71-44ff-bafd-6933f44d4f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:56.216707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dbfb3-9e71-44ff-bafd-6933f44d4f63 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:56.217907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dbfb3-9e71-44ff-bafd-6933f44d4f63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:56.220713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287dbfb3-9e71-44ff-bafd-6933f44d4f63 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:56.350148 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1247/2490] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:56 2022] POST /identity/v3/projects => generated 373 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:56.351402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a721b-c342-4bc7-bc53-10f5e556aed9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:56.374886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a721b-c342-4bc7-bc53-10f5e556aed9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:56.542691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633f172-916b-45b5-be00-bd4ef90b705c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:56.543392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633f172-916b-45b5-be00-bd4ef90b705c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:56.543675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633f172-916b-45b5-be00-bd4ef90b705c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:56.543959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633f172-916b-45b5-be00-bd4ef90b705c admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:56.557490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633f172-916b-45b5-be00-bd4ef90b705c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:56.572132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633f172-916b-45b5-be00-bd4ef90b705c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:56.684938 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1244/2491] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:56 2022] POST /identity/v3/users => generated 293 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:04:56.688912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dbfb3-9e71-44ff-bafd-6933f44d4f63 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:56.704349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dbfb3-9e71-44ff-bafd-6933f44d4f63 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:56.714546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442a1ed-68c7-4b50-bbe8-2c1f2c3f5391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:56.715512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442a1ed-68c7-4b50-bbe8-2c1f2c3f5391 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:56.716971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442a1ed-68c7-4b50-bbe8-2c1f2c3f5391 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:56.719107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442a1ed-68c7-4b50-bbe8-2c1f2c3f5391 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:56.720120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442a1ed-68c7-4b50-bbe8-2c1f2c3f5391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:56.722012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6442a1ed-68c7-4b50-bbe8-2c1f2c3f5391 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:56.742578 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1248/2492] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:56.748827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633f172-916b-45b5-be00-bd4ef90b705c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:56.771679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633f172-916b-45b5-be00-bd4ef90b705c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:56.787217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cd54c-2205-470d-b727-fabf303dbf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:56.788144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cd54c-2205-470d-b727-fabf303dbf26 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:56.788434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cd54c-2205-470d-b727-fabf303dbf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:56.788713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75cd54c-2205-470d-b727-fabf303dbf26 admin admin] PATH_INFO: `/v3/projects/4881ea8451554b51baeab6e4abaea2ae/users/bc7cb31fb00943cdb9ca8e3919c32fff/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:56.825425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cd54c-2205-470d-b727-fabf303dbf26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4881ea8451554b51baeab6e4abaea2ae, user_id=bc7cb31fb00943cdb9ca8e3919c32fff, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:56.838001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75cd54c-2205-470d-b727-fabf303dbf26 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:56.920409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:56.928132 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1245/2493] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:56 2022] PUT /identity/v3/projects/4881ea8451554b51baeab6e4abaea2ae/users/bc7cb31fb00943cdb9ca8e3919c32fff/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:56.932106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442a1ed-68c7-4b50-bbe8-2c1f2c3f5391 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:56.947956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442a1ed-68c7-4b50-bbe8-2c1f2c3f5391 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:57.056319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258cee59-fb8b-4c93-a174-60b648722f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:57.057219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cee59-fb8b-4c93-a174-60b648722f0c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:57.057509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cee59-fb8b-4c93-a174-60b648722f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:57.057787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258cee59-fb8b-4c93-a174-60b648722f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:57.058761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258cee59-fb8b-4c93-a174-60b648722f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:57.060961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258cee59-fb8b-4c93-a174-60b648722f0c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:57.109353 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1249/2494] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:04:57.113923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cd54c-2205-470d-b727-fabf303dbf26 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:57.132764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75cd54c-2205-470d-b727-fabf303dbf26 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:57.146985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca815c-db93-489c-84e0-591081b494ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:57.148201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca815c-db93-489c-84e0-591081b494ca admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:57.150067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca815c-db93-489c-84e0-591081b494ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:57.150067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca815c-db93-489c-84e0-591081b494ca admin admin] PATH_INFO: `/v3/projects/4881ea8451554b51baeab6e4abaea2ae/users/bc7cb31fb00943cdb9ca8e3919c32fff/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:57.153339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca815c-db93-489c-84e0-591081b494ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=4881ea8451554b51baeab6e4abaea2ae, user_id=bc7cb31fb00943cdb9ca8e3919c32fff, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:57.159841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ca815c-db93-489c-84e0-591081b494ca admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:57.186572 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1246/2495] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:04:57 2022] PUT /identity/v3/projects/4881ea8451554b51baeab6e4abaea2ae/users/bc7cb31fb00943cdb9ca8e3919c32fff/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:04:57.197445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fd701-e920-45fa-936b-5bb519adbc40 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:57.197725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fd701-e920-45fa-936b-5bb519adbc40 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:57.198045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fd701-e920-45fa-936b-5bb519adbc40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:57.254944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7a9fd701-e920-45fa-936b-5bb519adbc40 None None] MFA Rules not processed for user `bc7cb31fb00943cdb9ca8e3919c32fff`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:57.260067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7a9fd701-e920-45fa-936b-5bb519adbc40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:57.359224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:57.428727 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1250/2496] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:57.435831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca815c-db93-489c-84e0-591081b494ca admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:57.457243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca815c-db93-489c-84e0-591081b494ca admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:57.470071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9863c4d-5996-46a1-b29a-4b3a1668b87b tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc7cb31fb00943cdb9ca8e3919c32fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4881ea8451554b51baeab6e4abaea2ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:57.470787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9863c4d-5996-46a1-b29a-4b3a1668b87b tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:57.471074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9863c4d-5996-46a1-b29a-4b3a1668b87b tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:57.471358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9863c4d-5996-46a1-b29a-4b3a1668b87b tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:57.472119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9863c4d-5996-46a1-b29a-4b3a1668b87b tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:57.472430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9863c4d-5996-46a1-b29a-4b3a1668b87b tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:57.484458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9863c4d-5996-46a1-b29a-4b3a1668b87b tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:57.516682 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1247/2497] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:04:57.530065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fd701-e920-45fa-936b-5bb519adbc40 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:57.584786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fd701-e920-45fa-936b-5bb519adbc40 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:57.600046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3b7d3-b11b-4144-b303-63d2dbceaf66 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc7cb31fb00943cdb9ca8e3919c32fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4881ea8451554b51baeab6e4abaea2ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:57.601207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3b7d3-b11b-4144-b303-63d2dbceaf66 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:57.603512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3b7d3-b11b-4144-b303-63d2dbceaf66 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:57.604027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3b7d3-b11b-4144-b303-63d2dbceaf66 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:57.607757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3b7d3-b11b-4144-b303-63d2dbceaf66 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:57.612555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3b7d3-b11b-4144-b303-63d2dbceaf66 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:57.733905 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1251/2498] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:04:57 2022] POST /identity/v3/projects => generated 369 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:04:57.737505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9863c4d-5996-46a1-b29a-4b3a1668b87b tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:57.753991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9863c4d-5996-46a1-b29a-4b3a1668b87b tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:57.873898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7739a594-f9b1-47e5-ba54-27711469aae9 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc7cb31fb00943cdb9ca8e3919c32fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4881ea8451554b51baeab6e4abaea2ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:57.875430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739a594-f9b1-47e5-ba54-27711469aae9 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:57.875731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739a594-f9b1-47e5-ba54-27711469aae9 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:57.876355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7739a594-f9b1-47e5-ba54-27711469aae9 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:57.877314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7739a594-f9b1-47e5-ba54-27711469aae9 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:57.877829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7739a594-f9b1-47e5-ba54-27711469aae9 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:57.880435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7739a594-f9b1-47e5-ba54-27711469aae9 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:57.901706 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1248/2499] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:58.581243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3b7d3-b11b-4144-b303-63d2dbceaf66 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:58.588397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-51e3b7d3-b11b-4144-b303-63d2dbceaf66 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:58.778052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3b7d3-b11b-4144-b303-63d2dbceaf66 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:58.787735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362aa03-3cb8-4cd4-a084-013b540fcdc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:58.790879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362aa03-3cb8-4cd4-a084-013b540fcdc1 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:58.790879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362aa03-3cb8-4cd4-a084-013b540fcdc1 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:58.790879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362aa03-3cb8-4cd4-a084-013b540fcdc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:58.801314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362aa03-3cb8-4cd4-a084-013b540fcdc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:58.803933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362aa03-3cb8-4cd4-a084-013b540fcdc1 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:58.830208 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1252/2500] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:58.877918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838a2b8-ab11-470a-9b33-a1e19eed9f3e None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:58.878781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838a2b8-ab11-470a-9b33-a1e19eed9f3e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:58.879262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838a2b8-ab11-470a-9b33-a1e19eed9f3e None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:58.881872 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1249/2501] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:04:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:04:58.896700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362aa03-3cb8-4cd4-a084-013b540fcdc1 service nova] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:04:58.926956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362aa03-3cb8-4cd4-a084-013b540fcdc1 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:58.939706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362aa03-3cb8-4cd4-a084-013b540fcdc1 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:58.945660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da9b4c-34a6-45cb-a214-e6f8d0c406c0 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc7cb31fb00943cdb9ca8e3919c32fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4881ea8451554b51baeab6e4abaea2ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:58.946826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da9b4c-34a6-45cb-a214-e6f8d0c406c0 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:58.947776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da9b4c-34a6-45cb-a214-e6f8d0c406c0 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:58.948229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da9b4c-34a6-45cb-a214-e6f8d0c406c0 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] PATH_INFO: `/v3/projects/97cc6006cd664e18a879e467b5165cc9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:58.956990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da9b4c-34a6-45cb-a214-e6f8d0c406c0 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorizing `identity:get_project(project_id=97cc6006cd664e18a879e467b5165cc9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:58.961402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da9b4c-34a6-45cb-a214-e6f8d0c406c0 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:58.964226 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1253/2502] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:04:58 2022] GET /identity/v3/projects/97cc6006cd664e18a879e467b5165cc9 => generated 369 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:59.257302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:59.315432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b7d17-1f01-4831-a169-07f7defc1b42 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:59.316003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b7d17-1f01-4831-a169-07f7defc1b42 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:59.316451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b7d17-1f01-4831-a169-07f7defc1b42 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:59.318743 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1250/2503] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:04:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:04:59.327827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5e787-bf75-455c-b8a3-8a9f3d6b8c86 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:59.329017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5e787-bf75-455c-b8a3-8a9f3d6b8c86 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:59.329467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba5e787-bf75-455c-b8a3-8a9f3d6b8c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:59.365028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5ba5e787-bf75-455c-b8a3-8a9f3d6b8c86 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:59.365028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5ba5e787-bf75-455c-b8a3-8a9f3d6b8c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:59.491090 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1254/2504] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:04:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:04:59.515009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b7d17-1f01-4831-a169-07f7defc1b42 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:59.517561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-666b7d17-1f01-4831-a169-07f7defc1b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:59.539717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b7d17-1f01-4831-a169-07f7defc1b42 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:59.550822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61668a0a-187f-434c-9622-509db02c4918 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:59.552270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61668a0a-187f-434c-9622-509db02c4918 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:59.553446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61668a0a-187f-434c-9622-509db02c4918 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:59.554056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61668a0a-187f-434c-9622-509db02c4918 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:59.583628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61668a0a-187f-434c-9622-509db02c4918 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:59.600692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61668a0a-187f-434c-9622-509db02c4918 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:59.639738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00739524-91fb-405b-8dee-9485db025c43 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:59.639999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00739524-91fb-405b-8dee-9485db025c43 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:59.640284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00739524-91fb-405b-8dee-9485db025c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:59.654951 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1251/2505] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:04:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:04:59.660249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07f381-3678-410b-b16c-a90a70bce6cd None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:59.660616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07f381-3678-410b-b16c-a90a70bce6cd None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:59.660836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07f381-3678-410b-b16c-a90a70bce6cd None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:59.662911 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1252/2506] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:04:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:04:59.674878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07f381-3678-410b-b16c-a90a70bce6cd None None] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:04:59.704119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07f381-3678-410b-b16c-a90a70bce6cd None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:59.721246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07f381-3678-410b-b16c-a90a70bce6cd None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:59.728966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345fc687-0833-4095-8d01-927d8bcf0b2a tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc7cb31fb00943cdb9ca8e3919c32fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4881ea8451554b51baeab6e4abaea2ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:59.730910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fc687-0833-4095-8d01-927d8bcf0b2a tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:59.731258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fc687-0833-4095-8d01-927d8bcf0b2a tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:59.731542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345fc687-0833-4095-8d01-927d8bcf0b2a tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] PATH_INFO: `/v3/projects/97cc6006cd664e18a879e467b5165cc9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:59.736600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-00739524-91fb-405b-8dee-9485db025c43 None None] MFA Rules not processed for user `1229cf9547964df18d40762b3f71a530`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:04:59.738172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-00739524-91fb-405b-8dee-9485db025c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:04:59.739801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345fc687-0833-4095-8d01-927d8bcf0b2a tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorizing `identity:get_project(project_id=97cc6006cd664e18a879e467b5165cc9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:59.743591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345fc687-0833-4095-8d01-927d8bcf0b2a tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:59.748044 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1253/2507] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:04:59 2022] GET /identity/v3/projects/97cc6006cd664e18a879e467b5165cc9 => generated 369 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:59.846236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345fc687-0833-4095-8d01-927d8bcf0b2a tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:59.881434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345fc687-0833-4095-8d01-927d8bcf0b2a tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:04:59.910240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:59.937671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273be4f-b720-472e-80a8-8fd96dc5fef5 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc7cb31fb00943cdb9ca8e3919c32fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4881ea8451554b51baeab6e4abaea2ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:04:59.940366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273be4f-b720-472e-80a8-8fd96dc5fef5 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:04:59.941561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273be4f-b720-472e-80a8-8fd96dc5fef5 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:04:59.942344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2273be4f-b720-472e-80a8-8fd96dc5fef5 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:04:59.944060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2273be4f-b720-472e-80a8-8fd96dc5fef5 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:04:59.944550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2273be4f-b720-472e-80a8-8fd96dc5fef5 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:04:59.946792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2273be4f-b720-472e-80a8-8fd96dc5fef5 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:04:59.975705 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1254/2508] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:04:59.980655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273be4f-b720-472e-80a8-8fd96dc5fef5 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:04:59.991135 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1255/2509] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:04:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:05:00.011909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00739524-91fb-405b-8dee-9485db025c43 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:00.013385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2273be4f-b720-472e-80a8-8fd96dc5fef5 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:00.014341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-00739524-91fb-405b-8dee-9485db025c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:00.063235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d172b28-ffe6-4b6b-bb88-5ada46ea56a6 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc7cb31fb00943cdb9ca8e3919c32fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4881ea8451554b51baeab6e4abaea2ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:00.065169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d172b28-ffe6-4b6b-bb88-5ada46ea56a6 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:00.065476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d172b28-ffe6-4b6b-bb88-5ada46ea56a6 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:00.065768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d172b28-ffe6-4b6b-bb88-5ada46ea56a6 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:00.066615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d172b28-ffe6-4b6b-bb88-5ada46ea56a6 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:00.071952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d172b28-ffe6-4b6b-bb88-5ada46ea56a6 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:00.231384 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1255/2510] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:04:59 2022] POST /identity/v3/users => generated 381 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:05:00.249408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d172b28-ffe6-4b6b-bb88-5ada46ea56a6 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:00.262367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00739524-91fb-405b-8dee-9485db025c43 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:00.268355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de9f51-b4c5-4da6-a053-e0f714be6546 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:00.273821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d172b28-ffe6-4b6b-bb88-5ada46ea56a6 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:00.278624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de9f51-b4c5-4da6-a053-e0f714be6546 service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:00.279289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de9f51-b4c5-4da6-a053-e0f714be6546 service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:00.279791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de9f51-b4c5-4da6-a053-e0f714be6546 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:00.280948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b0ef8-f59a-47b4-ab9f-dd14555f32a4 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc7cb31fb00943cdb9ca8e3919c32fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4881ea8451554b51baeab6e4abaea2ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:00.281939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b0ef8-f59a-47b4-ab9f-dd14555f32a4 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:00.282629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b0ef8-f59a-47b4-ab9f-dd14555f32a4 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:00.283122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b0ef8-f59a-47b4-ab9f-dd14555f32a4 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:00.284243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b0ef8-f59a-47b4-ab9f-dd14555f32a4 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:05:00.284861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b0ef8-f59a-47b4-ab9f-dd14555f32a4 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:00.287072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b0ef8-f59a-47b4-ab9f-dd14555f32a4 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:00.294415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de9f51-b4c5-4da6-a053-e0f714be6546 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:00.297387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de9f51-b4c5-4da6-a053-e0f714be6546 service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:00.303829 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1256/2511] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:05:00.306240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53083a39-c1e7-4ef2-a5d3-a36eb835370c None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:00.306952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53083a39-c1e7-4ef2-a5d3-a36eb835370c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:00.307445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53083a39-c1e7-4ef2-a5d3-a36eb835370c None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:00.309247 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1257/2512] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:05:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:05:00.315563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa54bca-8cd2-4bcd-a0ad-38ac9ef55bf5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:00.316144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa54bca-8cd2-4bcd-a0ad-38ac9ef55bf5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:00.316656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa54bca-8cd2-4bcd-a0ad-38ac9ef55bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:00.362712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:00.365712 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1256/2513] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:00.372947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6aa54bca-8cd2-4bcd-a0ad-38ac9ef55bf5 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:00.376976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6aa54bca-8cd2-4bcd-a0ad-38ac9ef55bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:00.405525 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1258/2514] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:05:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:00.474833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de9f51-b4c5-4da6-a053-e0f714be6546 service glance] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:00.476512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-87de9f51-b4c5-4da6-a053-e0f714be6546 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:00.759506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de9f51-b4c5-4da6-a053-e0f714be6546 service glance] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:00.787117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994f538-3f53-40b4-b370-eb3b9664a558 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:00.791151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994f538-3f53-40b4-b370-eb3b9664a558 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:00.791506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994f538-3f53-40b4-b370-eb3b9664a558 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:00.791795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994f538-3f53-40b4-b370-eb3b9664a558 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:00.829212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c994f538-3f53-40b4-b370-eb3b9664a558 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:00.839203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c994f538-3f53-40b4-b370-eb3b9664a558 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:00.854337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94296c6-6b8c-4683-8a5a-027c4ac806f9 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:00.854658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94296c6-6b8c-4683-8a5a-027c4ac806f9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:00.860374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94296c6-6b8c-4683-8a5a-027c4ac806f9 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:00.860374 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1259/2515] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 21:05:00.864746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94296c6-6b8c-4683-8a5a-027c4ac806f9 None None] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:05:00.866697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c94296c6-6b8c-4683-8a5a-027c4ac806f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:00.906984 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1257/2516] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:00.929573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94296c6-6b8c-4683-8a5a-027c4ac806f9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:00.981359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94296c6-6b8c-4683-8a5a-027c4ac806f9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:00.997340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3d7d82-8adf-4501-aed2-decc4c81a1da tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc7cb31fb00943cdb9ca8e3919c32fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4881ea8451554b51baeab6e4abaea2ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:00.998737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d7d82-8adf-4501-aed2-decc4c81a1da tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:01.003849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d7d82-8adf-4501-aed2-decc4c81a1da tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:01.003849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3d7d82-8adf-4501-aed2-decc4c81a1da tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] PATH_INFO: `/v3/projects/97cc6006cd664e18a879e467b5165cc9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:01.003849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d7d82-8adf-4501-aed2-decc4c81a1da tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorizing `identity:get_project(project_id=97cc6006cd664e18a879e467b5165cc9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:01.010141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3d7d82-8adf-4501-aed2-decc4c81a1da tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:01.012306 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1260/2517] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:05:00 2022] GET /identity/v3/projects/97cc6006cd664e18a879e467b5165cc9 => generated 369 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:05:01.456370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:02.072782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:02.363927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274732c-a46b-4713-8a06-8118cbae0a4b None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:02.364847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274732c-a46b-4713-8a06-8118cbae0a4b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:02.365351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d274732c-a46b-4713-8a06-8118cbae0a4b None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:02.373926 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1258/2518] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:05:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:05:02.393818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dbad6-7676-446c-b804-09b43c28ccdc None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:02.394490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dbad6-7676-446c-b804-09b43c28ccdc None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:02.394950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dbad6-7676-446c-b804-09b43c28ccdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:02.416309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274732c-a46b-4713-8a06-8118cbae0a4b None None] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:05:02.417620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-030dbad6-7676-446c-b804-09b43c28ccdc None None] MFA Rules not processed for user `fc6ee6dba5d24e90bff48384539fcd7f`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:02.419373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-030dbad6-7676-446c-b804-09b43c28ccdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:02.466611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274732c-a46b-4713-8a06-8118cbae0a4b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:02.513732 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1261/2519] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:02.539584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d274732c-a46b-4713-8a06-8118cbae0a4b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:02.573567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f2050-0714-4fc1-b810-961e63e4d584 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc7cb31fb00943cdb9ca8e3919c32fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4881ea8451554b51baeab6e4abaea2ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:02.578900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f2050-0714-4fc1-b810-961e63e4d584 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:02.582106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f2050-0714-4fc1-b810-961e63e4d584 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:02.584080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f2050-0714-4fc1-b810-961e63e4d584 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] PATH_INFO: `/v3/projects/97cc6006cd664e18a879e467b5165cc9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:02.588354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f2050-0714-4fc1-b810-961e63e4d584 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorizing `identity:get_project(project_id=97cc6006cd664e18a879e467b5165cc9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:02.597111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f2050-0714-4fc1-b810-961e63e4d584 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:02.617939 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1259/2520] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:05:02 2022] GET /identity/v3/projects/97cc6006cd664e18a879e467b5165cc9 => generated 369 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:05:02.706467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030dbad6-7676-446c-b804-09b43c28ccdc None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:02.732968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030dbad6-7676-446c-b804-09b43c28ccdc None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:02.755737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04f909-2fbb-4be6-85e8-e1226c9fc735 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc7cb31fb00943cdb9ca8e3919c32fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4881ea8451554b51baeab6e4abaea2ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:02.757981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04f909-2fbb-4be6-85e8-e1226c9fc735 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:02.758359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04f909-2fbb-4be6-85e8-e1226c9fc735 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:02.762166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04f909-2fbb-4be6-85e8-e1226c9fc735 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] PATH_INFO: `/v3/users/86ab50d819c84293ae34c398177ed855` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:02.794354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04f909-2fbb-4be6-85e8-e1226c9fc735 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorizing `identity:delete_user(user_id=86ab50d819c84293ae34c398177ed855)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:02.802721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04f909-2fbb-4be6-85e8-e1226c9fc735 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:03.069650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ac04f909-2fbb-4be6-85e8-e1226c9fc735 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86ab50d819c84293ae34c398177ed855'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:03.071549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ac04f909-2fbb-4be6-85e8-e1226c9fc735 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86ab50d819c84293ae34c398177ed855'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:03.139020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ac04f909-2fbb-4be6-85e8-e1226c9fc735 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86ab50d819c84293ae34c398177ed855'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:03.244647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ac04f909-2fbb-4be6-85e8-e1226c9fc735 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86ab50d819c84293ae34c398177ed855'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:03.245434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ac04f909-2fbb-4be6-85e8-e1226c9fc735 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86ab50d819c84293ae34c398177ed855'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:03.334255 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1262/2521] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:05:02 2022] DELETE /identity/v3/users/86ab50d819c84293ae34c398177ed855 => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:03.335678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f2050-0714-4fc1-b810-961e63e4d584 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:03.337817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e43f2050-0714-4fc1-b810-961e63e4d584 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:03.373236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f2050-0714-4fc1-b810-961e63e4d584 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:03.458588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:03.504473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524adfb8-f882-4c4b-b23e-4aaf0ef6e098 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc7cb31fb00943cdb9ca8e3919c32fff', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4881ea8451554b51baeab6e4abaea2ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:03.505752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524adfb8-f882-4c4b-b23e-4aaf0ef6e098 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:03.506115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524adfb8-f882-4c4b-b23e-4aaf0ef6e098 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:03.506435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524adfb8-f882-4c4b-b23e-4aaf0ef6e098 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] PATH_INFO: `/v3/projects/97cc6006cd664e18a879e467b5165cc9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:03.535159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524adfb8-f882-4c4b-b23e-4aaf0ef6e098 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorizing `identity:delete_project(project_id=97cc6006cd664e18a879e467b5165cc9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:03.540225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524adfb8-f882-4c4b-b23e-4aaf0ef6e098 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:03.751229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-524adfb8-f882-4c4b-b23e-4aaf0ef6e098 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97cc6006cd664e18a879e467b5165cc9', 'request_id': 'req-524adfb8-f882-4c4b-b23e-4aaf0ef6e098'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:03.759906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-524adfb8-f882-4c4b-b23e-4aaf0ef6e098 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97cc6006cd664e18a879e467b5165cc9', 'request_id': 'req-524adfb8-f882-4c4b-b23e-4aaf0ef6e098'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:03.880357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-524adfb8-f882-4c4b-b23e-4aaf0ef6e098 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97cc6006cd664e18a879e467b5165cc9', 'request_id': 'req-524adfb8-f882-4c4b-b23e-4aaf0ef6e098'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:03.900233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-524adfb8-f882-4c4b-b23e-4aaf0ef6e098 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] The token cache is being invalidate because project 97cc6006cd664e18a879e467b5165cc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:05:03.903934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-524adfb8-f882-4c4b-b23e-4aaf0ef6e098 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:03.904809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-524adfb8-f882-4c4b-b23e-4aaf0ef6e098 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:03.908752 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1260/2522] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:05:03 2022] DELETE /identity/v3/projects/97cc6006cd664e18a879e467b5165cc9 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:03.915205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04f909-2fbb-4be6-85e8-e1226c9fc735 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:03.917248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ac04f909-2fbb-4be6-85e8-e1226c9fc735 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:04.051290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04f909-2fbb-4be6-85e8-e1226c9fc735 tempest-QuotasAdminTestV257-843138382 tempest-QuotasAdminTestV257-843138382-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:04.216657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:04.223252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4e08b-a0d8-456b-80d9-f761d813dfc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:04.225738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4e08b-a0d8-456b-80d9-f761d813dfc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:04.226225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4e08b-a0d8-456b-80d9-f761d813dfc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:04.226680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4e08b-a0d8-456b-80d9-f761d813dfc6 admin admin] PATH_INFO: `/v3/users/5d0536d93bc54013a921325d3a2213a6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:04.229546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4e08b-a0d8-456b-80d9-f761d813dfc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d0536d93bc54013a921325d3a2213a6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:04.237812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4e08b-a0d8-456b-80d9-f761d813dfc6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:04.565472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d7a4e08b-a0d8-456b-80d9-f761d813dfc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d0536d93bc54013a921325d3a2213a6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:04.566960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d7a4e08b-a0d8-456b-80d9-f761d813dfc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d0536d93bc54013a921325d3a2213a6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:04.607377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d7a4e08b-a0d8-456b-80d9-f761d813dfc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d0536d93bc54013a921325d3a2213a6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:04.652133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:04.698914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d7a4e08b-a0d8-456b-80d9-f761d813dfc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d0536d93bc54013a921325d3a2213a6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:04.699956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d7a4e08b-a0d8-456b-80d9-f761d813dfc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d0536d93bc54013a921325d3a2213a6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:04.760935 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1263/2523] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:05:03 2022] DELETE /identity/v3/users/5d0536d93bc54013a921325d3a2213a6 => generated 0 bytes in 847 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:04.770776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c5119-2d10-4772-b1d8-2b516a21fee1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:04.771391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c5119-2d10-4772-b1d8-2b516a21fee1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:04.771854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c5119-2d10-4772-b1d8-2b516a21fee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:04.822658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-396c5119-2d10-4772-b1d8-2b516a21fee1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:04.825128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-396c5119-2d10-4772-b1d8-2b516a21fee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:05.003497 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1261/2524] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:05.011238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4e08b-a0d8-456b-80d9-f761d813dfc6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:05.012773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d7a4e08b-a0d8-456b-80d9-f761d813dfc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:05.164150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4e08b-a0d8-456b-80d9-f761d813dfc6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:05.178322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b86900-e07b-499b-94f1-7d952a4a3e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:05.180538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b86900-e07b-499b-94f1-7d952a4a3e83 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:05.180864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b86900-e07b-499b-94f1-7d952a4a3e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:05.181461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b86900-e07b-499b-94f1-7d952a4a3e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:05.203660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b86900-e07b-499b-94f1-7d952a4a3e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:05.205142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b86900-e07b-499b-94f1-7d952a4a3e83 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:05.239991 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1264/2525] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:05.347964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:05.759865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:05.985334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c5119-2d10-4772-b1d8-2b516a21fee1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:05.990439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-396c5119-2d10-4772-b1d8-2b516a21fee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:06.046753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c5119-2d10-4772-b1d8-2b516a21fee1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:06.084009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c07056-e54a-44fc-ad92-3c4caead51aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:06.085308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c07056-e54a-44fc-ad92-3c4caead51aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:06.086989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c07056-e54a-44fc-ad92-3c4caead51aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:06.088241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c07056-e54a-44fc-ad92-3c4caead51aa admin admin] PATH_INFO: `/v3/projects/8692c3aa46b74a2aaa1b0c742e7d6fb8` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:06.091439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c07056-e54a-44fc-ad92-3c4caead51aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8692c3aa46b74a2aaa1b0c742e7d6fb8)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:06.095464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c07056-e54a-44fc-ad92-3c4caead51aa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:06.170671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:06.340291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e1c07056-e54a-44fc-ad92-3c4caead51aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8692c3aa46b74a2aaa1b0c742e7d6fb8', 'request_id': 'req-e1c07056-e54a-44fc-ad92-3c4caead51aa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:06.362675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e1c07056-e54a-44fc-ad92-3c4caead51aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8692c3aa46b74a2aaa1b0c742e7d6fb8', 'request_id': 'req-e1c07056-e54a-44fc-ad92-3c4caead51aa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:06.497645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e1c07056-e54a-44fc-ad92-3c4caead51aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8692c3aa46b74a2aaa1b0c742e7d6fb8', 'request_id': 'req-e1c07056-e54a-44fc-ad92-3c4caead51aa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:06.523341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e1c07056-e54a-44fc-ad92-3c4caead51aa admin admin] The token cache is being invalidate because project 8692c3aa46b74a2aaa1b0c742e7d6fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:05:06.523621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e1c07056-e54a-44fc-ad92-3c4caead51aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:06.530780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e1c07056-e54a-44fc-ad92-3c4caead51aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:06.531493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b86900-e07b-499b-94f1-7d952a4a3e83 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:06.541510 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1262/2526] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:05:05 2022] DELETE /identity/v3/projects/8692c3aa46b74a2aaa1b0c742e7d6fb8 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:06.544272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b86900-e07b-499b-94f1-7d952a4a3e83 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:06.682305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e192c0-9f7d-4a6b-acb9-0011dbdd73b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:06.683204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e192c0-9f7d-4a6b-acb9-0011dbdd73b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:06.683493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e192c0-9f7d-4a6b-acb9-0011dbdd73b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:06.683772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e192c0-9f7d-4a6b-acb9-0011dbdd73b9 admin admin] PATH_INFO: `/v3/users/bc7cb31fb00943cdb9ca8e3919c32fff` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:06.686990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e192c0-9f7d-4a6b-acb9-0011dbdd73b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc7cb31fb00943cdb9ca8e3919c32fff)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:06.689274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e192c0-9f7d-4a6b-acb9-0011dbdd73b9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:06.737730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:07.010384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-04e192c0-9f7d-4a6b-acb9-0011dbdd73b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc7cb31fb00943cdb9ca8e3919c32fff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:07.011834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-04e192c0-9f7d-4a6b-acb9-0011dbdd73b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc7cb31fb00943cdb9ca8e3919c32fff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:07.102066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-04e192c0-9f7d-4a6b-acb9-0011dbdd73b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc7cb31fb00943cdb9ca8e3919c32fff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:07.137046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:07.215771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-04e192c0-9f7d-4a6b-acb9-0011dbdd73b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc7cb31fb00943cdb9ca8e3919c32fff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:07.217255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-04e192c0-9f7d-4a6b-acb9-0011dbdd73b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc7cb31fb00943cdb9ca8e3919c32fff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:07.265062 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1265/2527] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:05:06 2022] DELETE /identity/v3/users/bc7cb31fb00943cdb9ca8e3919c32fff => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:05:07.295819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:07.780290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8784232-25e5-4f5e-a0eb-c923bd71f2f9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:07.780859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8784232-25e5-4f5e-a0eb-c923bd71f2f9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:07.781301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8784232-25e5-4f5e-a0eb-c923bd71f2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:07.865338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f8784232-25e5-4f5e-a0eb-c923bd71f2f9 None None] MFA Rules not processed for user `7b44b524592e4a57a26681767b9c4541`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:07.865563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f8784232-25e5-4f5e-a0eb-c923bd71f2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:08.015917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:08.097130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e192c0-9f7d-4a6b-acb9-0011dbdd73b9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:08.107377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-04e192c0-9f7d-4a6b-acb9-0011dbdd73b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:08.148600 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1263/2528] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:08.363012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e192c0-9f7d-4a6b-acb9-0011dbdd73b9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:08.370882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a84315-76be-434f-990a-4f9241373d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:08.372946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a84315-76be-434f-990a-4f9241373d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:08.373347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a84315-76be-434f-990a-4f9241373d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:08.373805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a84315-76be-434f-990a-4f9241373d42 admin admin] PATH_INFO: `/v3/projects/4881ea8451554b51baeab6e4abaea2ae` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:08.375355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a84315-76be-434f-990a-4f9241373d42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4881ea8451554b51baeab6e4abaea2ae)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:08.378251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a84315-76be-434f-990a-4f9241373d42 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:08.524499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-16a84315-76be-434f-990a-4f9241373d42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4881ea8451554b51baeab6e4abaea2ae', 'request_id': 'req-16a84315-76be-434f-990a-4f9241373d42'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:08.535529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-16a84315-76be-434f-990a-4f9241373d42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4881ea8451554b51baeab6e4abaea2ae', 'request_id': 'req-16a84315-76be-434f-990a-4f9241373d42'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:08.559198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:08.599034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-16a84315-76be-434f-990a-4f9241373d42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4881ea8451554b51baeab6e4abaea2ae', 'request_id': 'req-16a84315-76be-434f-990a-4f9241373d42'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:08.641000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-16a84315-76be-434f-990a-4f9241373d42 admin admin] The token cache is being invalidate because project 4881ea8451554b51baeab6e4abaea2ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:05:08.641496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-16a84315-76be-434f-990a-4f9241373d42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:08.642853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-16a84315-76be-434f-990a-4f9241373d42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:08.649127 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1266/2529] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:05:08 2022] DELETE /identity/v3/projects/4881ea8451554b51baeab6e4abaea2ae => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:08.654243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cdd22-d0d1-46de-b060-1707344646d4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:08.655396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cdd22-d0d1-46de-b060-1707344646d4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:08.656638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92cdd22-d0d1-46de-b060-1707344646d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:08.709545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e92cdd22-d0d1-46de-b060-1707344646d4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:08.711269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e92cdd22-d0d1-46de-b060-1707344646d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:08.890108 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1264/2530] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:08.891571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c19c9-04b9-48c9-a6af-a4f3b3a04a4e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:08.891713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c19c9-04b9-48c9-a6af-a4f3b3a04a4e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:08.893273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c19c9-04b9-48c9-a6af-a4f3b3a04a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:08.979612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5a6c19c9-04b9-48c9-a6af-a4f3b3a04a4e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:08.981263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5a6c19c9-04b9-48c9-a6af-a4f3b3a04a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:09.004957 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1267/2531] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:09.017565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cdd22-d0d1-46de-b060-1707344646d4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:09.045920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92cdd22-d0d1-46de-b060-1707344646d4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:09.083912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427445ee-b959-4e24-aa29-2e426a6a6668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:09.086511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427445ee-b959-4e24-aa29-2e426a6a6668 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:09.086800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427445ee-b959-4e24-aa29-2e426a6a6668 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:09.087186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427445ee-b959-4e24-aa29-2e426a6a6668 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:09.090018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427445ee-b959-4e24-aa29-2e426a6a6668 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:05:09.091406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427445ee-b959-4e24-aa29-2e426a6a6668 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:09.093000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427445ee-b959-4e24-aa29-2e426a6a6668 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:09.130218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c19c9-04b9-48c9-a6af-a4f3b3a04a4e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:09.131338 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1265/2532] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:05:09.143084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c19c9-04b9-48c9-a6af-a4f3b3a04a4e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:09.169145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb9bbe-0e70-4c96-aff1-800e4df9d593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:09.170524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb9bbe-0e70-4c96-aff1-800e4df9d593 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:09.174160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb9bbe-0e70-4c96-aff1-800e4df9d593 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:09.174754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb9bbe-0e70-4c96-aff1-800e4df9d593 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:09.178343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb9bbe-0e70-4c96-aff1-800e4df9d593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:09.180645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb9bbe-0e70-4c96-aff1-800e4df9d593 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:09.408379 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1268/2533] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:05:09 2022] POST /identity/v3/projects => generated 383 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:05:09.413193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427445ee-b959-4e24-aa29-2e426a6a6668 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:09.448871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427445ee-b959-4e24-aa29-2e426a6a6668 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:09.550852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453d1c9-fc4b-4954-b71c-fa2654a331f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:09.552833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453d1c9-fc4b-4954-b71c-fa2654a331f1 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:09.553285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453d1c9-fc4b-4954-b71c-fa2654a331f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:09.553578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453d1c9-fc4b-4954-b71c-fa2654a331f1 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:09.554328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453d1c9-fc4b-4954-b71c-fa2654a331f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:09.572674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453d1c9-fc4b-4954-b71c-fa2654a331f1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:09.748801 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1266/2534] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:05:09 2022] POST /identity/v3/users => generated 292 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:05:09.750888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb9bbe-0e70-4c96-aff1-800e4df9d593 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:09.768449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb9bbe-0e70-4c96-aff1-800e4df9d593 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:09.788317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90519d-1f82-47ff-868b-d0a2366223ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:09.789013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90519d-1f82-47ff-868b-d0a2366223ff admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:09.789329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90519d-1f82-47ff-868b-d0a2366223ff admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:09.790352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c90519d-1f82-47ff-868b-d0a2366223ff admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:09.791342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90519d-1f82-47ff-868b-d0a2366223ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:09.793526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c90519d-1f82-47ff-868b-d0a2366223ff admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:09.819019 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1269/2535] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:05:09.827786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453d1c9-fc4b-4954-b71c-fa2654a331f1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:09.844595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453d1c9-fc4b-4954-b71c-fa2654a331f1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:09.862308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb52b6-8286-4b24-b8a5-2feb69ea5471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:09.866297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb52b6-8286-4b24-b8a5-2feb69ea5471 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:09.868695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb52b6-8286-4b24-b8a5-2feb69ea5471 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:09.869621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb52b6-8286-4b24-b8a5-2feb69ea5471 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:09.870669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb52b6-8286-4b24-b8a5-2feb69ea5471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:09.881181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb52b6-8286-4b24-b8a5-2feb69ea5471 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:09.916921 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1267/2536] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:05:09.921049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90519d-1f82-47ff-868b-d0a2366223ff admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:09.938128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c90519d-1f82-47ff-868b-d0a2366223ff admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:09.944007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881e73e-8495-4e94-bef3-39e86260c1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:09.945226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881e73e-8495-4e94-bef3-39e86260c1f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:09.945694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881e73e-8495-4e94-bef3-39e86260c1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:09.946168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881e73e-8495-4e94-bef3-39e86260c1f5 admin admin] PATH_INFO: `/v3/projects/df867107e50e446d8cbca00826c28cd0/users/ff0177f6b0bc486d89d2bd33dcfb62d7/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:10.000456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881e73e-8495-4e94-bef3-39e86260c1f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df867107e50e446d8cbca00826c28cd0, user_id=ff0177f6b0bc486d89d2bd33dcfb62d7, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:10.003639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881e73e-8495-4e94-bef3-39e86260c1f5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:10.147823 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1270/2537] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:05:09 2022] PUT /identity/v3/projects/df867107e50e446d8cbca00826c28cd0/users/ff0177f6b0bc486d89d2bd33dcfb62d7/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:10.164344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59711a6-b0af-4aab-8db1-9d0960c8b2f4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:10.165042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59711a6-b0af-4aab-8db1-9d0960c8b2f4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:10.165338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59711a6-b0af-4aab-8db1-9d0960c8b2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:10.203254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c59711a6-b0af-4aab-8db1-9d0960c8b2f4 None None] MFA Rules not processed for user `ff0177f6b0bc486d89d2bd33dcfb62d7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:10.217375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c59711a6-b0af-4aab-8db1-9d0960c8b2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:10.270043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:10.278004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:10.339165 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1268/2538] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:10.349171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881e73e-8495-4e94-bef3-39e86260c1f5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:10.453932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881e73e-8495-4e94-bef3-39e86260c1f5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:10.480350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343cd67-a6cb-4fad-aeb6-18b5e3881cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:10.481248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343cd67-a6cb-4fad-aeb6-18b5e3881cae admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:10.481662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343cd67-a6cb-4fad-aeb6-18b5e3881cae admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:10.482388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343cd67-a6cb-4fad-aeb6-18b5e3881cae admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:10.483843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343cd67-a6cb-4fad-aeb6-18b5e3881cae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:10.486051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3343cd67-a6cb-4fad-aeb6-18b5e3881cae admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:10.651053 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1271/2539] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:05:10 2022] POST /identity/v3/projects => generated 381 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:05:10.654375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59711a6-b0af-4aab-8db1-9d0960c8b2f4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:10.687231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59711a6-b0af-4aab-8db1-9d0960c8b2f4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:10.821971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d07d1a-01b6-428a-92dd-5865464f9856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:10.821971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d07d1a-01b6-428a-92dd-5865464f9856 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:10.821971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d07d1a-01b6-428a-92dd-5865464f9856 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:10.821971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d07d1a-01b6-428a-92dd-5865464f9856 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:10.822828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d07d1a-01b6-428a-92dd-5865464f9856 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:10.822828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d07d1a-01b6-428a-92dd-5865464f9856 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:10.912373 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1269/2540] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:05:10 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:05:10.916559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343cd67-a6cb-4fad-aeb6-18b5e3881cae admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:10.936285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3343cd67-a6cb-4fad-aeb6-18b5e3881cae admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:10.974616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c763d1f-2b93-4a1e-be21-24278251ea35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:10.975315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c763d1f-2b93-4a1e-be21-24278251ea35 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:10.975602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c763d1f-2b93-4a1e-be21-24278251ea35 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:10.975881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c763d1f-2b93-4a1e-be21-24278251ea35 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:10.976627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c763d1f-2b93-4a1e-be21-24278251ea35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:10.978285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c763d1f-2b93-4a1e-be21-24278251ea35 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:11.007890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d07d1a-01b6-428a-92dd-5865464f9856 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:11.008354 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1272/2541] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:05:11.019075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d07d1a-01b6-428a-92dd-5865464f9856 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:11.025585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b86a1-8544-4069-b20b-daab3b0dc4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:11.026879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b86a1-8544-4069-b20b-daab3b0dc4d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:11.027382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b86a1-8544-4069-b20b-daab3b0dc4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:11.027921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b86a1-8544-4069-b20b-daab3b0dc4d9 admin admin] PATH_INFO: `/v3/projects/67e25100d3894bdcaaa7358217e33604/users/634ed56feec24d329afe70247e897af4/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:11.088441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c763d1f-2b93-4a1e-be21-24278251ea35 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:11.090867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0c763d1f-2b93-4a1e-be21-24278251ea35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:11.103150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b86a1-8544-4069-b20b-daab3b0dc4d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67e25100d3894bdcaaa7358217e33604, user_id=634ed56feec24d329afe70247e897af4, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:11.114501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b86a1-8544-4069-b20b-daab3b0dc4d9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:11.200681 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1270/2542] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:05:11 2022] PUT /identity/v3/projects/67e25100d3894bdcaaa7358217e33604/users/634ed56feec24d329afe70247e897af4/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:05:11.204741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b86a1-8544-4069-b20b-daab3b0dc4d9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:11.226669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c763d1f-2b93-4a1e-be21-24278251ea35 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:11.313875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:11.329394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b86a1-8544-4069-b20b-daab3b0dc4d9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:11.338753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55b73a-298e-4eff-9d10-8722980a7137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:11.339887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55b73a-298e-4eff-9d10-8722980a7137 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:11.340003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d3389-ab09-4d38-9203-cddd1a56ea97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:11.340543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55b73a-298e-4eff-9d10-8722980a7137 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:11.340707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d3389-ab09-4d38-9203-cddd1a56ea97 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:11.341131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55b73a-298e-4eff-9d10-8722980a7137 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:11.341523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d3389-ab09-4d38-9203-cddd1a56ea97 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:11.341993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d3389-ab09-4d38-9203-cddd1a56ea97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:11.342212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55b73a-298e-4eff-9d10-8722980a7137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:11.344001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55b73a-298e-4eff-9d10-8722980a7137 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:11.344806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-262d3389-ab09-4d38-9203-cddd1a56ea97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:11.359695 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1271/2543] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:05:11.361395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55b73a-298e-4eff-9d10-8722980a7137 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:11.374924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55b73a-298e-4eff-9d10-8722980a7137 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:11.382067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5b981-4ffd-4782-abb9-cd5d90b9f4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:11.383160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5b981-4ffd-4782-abb9-cd5d90b9f4e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:11.383938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5b981-4ffd-4782-abb9-cd5d90b9f4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:11.384234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5b981-4ffd-4782-abb9-cd5d90b9f4e8 admin admin] PATH_INFO: `/v3/projects/67e25100d3894bdcaaa7358217e33604/users/634ed56feec24d329afe70247e897af4/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:11.387189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5b981-4ffd-4782-abb9-cd5d90b9f4e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67e25100d3894bdcaaa7358217e33604, user_id=634ed56feec24d329afe70247e897af4, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:11.391247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5b981-4ffd-4782-abb9-cd5d90b9f4e8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:11.429902 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1272/2544] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:05:11 2022] PUT /identity/v3/projects/67e25100d3894bdcaaa7358217e33604/users/634ed56feec24d329afe70247e897af4/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:11.435394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974814e1-d991-4cc8-b151-bc27ae11c14a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:11.435917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974814e1-d991-4cc8-b151-bc27ae11c14a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:11.436566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974814e1-d991-4cc8-b151-bc27ae11c14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:11.449788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d3389-ab09-4d38-9203-cddd1a56ea97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:11.452341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d3389-ab09-4d38-9203-cddd1a56ea97 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:11.481720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-974814e1-d991-4cc8-b151-bc27ae11c14a None None] MFA Rules not processed for user `634ed56feec24d329afe70247e897af4`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:11.483926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-974814e1-d991-4cc8-b151-bc27ae11c14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:11.491452 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1273/2545] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:05:11.496484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:11.540026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e23d7-373a-48fc-a2b0-217ad62d214d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:11.540335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e23d7-373a-48fc-a2b0-217ad62d214d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:11.540652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e23d7-373a-48fc-a2b0-217ad62d214d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:11.565675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a11e23d7-373a-48fc-a2b0-217ad62d214d None None] MFA Rules not processed for user `3b044baceddf49219485bd14aa914e35`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:11.567642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a11e23d7-373a-48fc-a2b0-217ad62d214d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:11.663986 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1273/2546] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:05:11.679915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974814e1-d991-4cc8-b151-bc27ae11c14a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:11.681854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-974814e1-d991-4cc8-b151-bc27ae11c14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:11.698634 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1274/2547] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:11.914243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974814e1-d991-4cc8-b151-bc27ae11c14a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:11.943108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4cde9-7871-4060-b66b-6ce61630aa2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:11.944844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4cde9-7871-4060-b66b-6ce61630aa2b service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:11.945341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4cde9-7871-4060-b66b-6ce61630aa2b service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:11.946506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4cde9-7871-4060-b66b-6ce61630aa2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:11.959650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b4cde9-7871-4060-b66b-6ce61630aa2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:11.962540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b4cde9-7871-4060-b66b-6ce61630aa2b service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:11.996175 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1274/2548] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:12.018443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bee5f-3d52-4e14-a02b-69e1d2da29b4 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:12.018777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bee5f-3d52-4e14-a02b-69e1d2da29b4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:12.019010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9bee5f-3d52-4e14-a02b-69e1d2da29b4 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:12.021376 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1275/2549] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:05:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:05:12.028049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4cde9-7871-4060-b66b-6ce61630aa2b service nova] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:05:12.044474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4cde9-7871-4060-b66b-6ce61630aa2b service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:12.054820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4cde9-7871-4060-b66b-6ce61630aa2b service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:12.061211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d531f-6740-4d9f-988b-9c6591fde850 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '634ed56feec24d329afe70247e897af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67e25100d3894bdcaaa7358217e33604', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:12.065394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d531f-6740-4d9f-988b-9c6591fde850 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:12.066001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d531f-6740-4d9f-988b-9c6591fde850 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:12.066494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d531f-6740-4d9f-988b-9c6591fde850 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] PATH_INFO: `/v3/projects/df867107e50e446d8cbca00826c28cd0` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:12.068614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d531f-6740-4d9f-988b-9c6591fde850 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorizing `identity:get_project(project_id=df867107e50e446d8cbca00826c28cd0)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:12.071047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d531f-6740-4d9f-988b-9c6591fde850 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:12.102305 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1275/2550] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:05:12 2022] GET /identity/v3/projects/df867107e50e446d8cbca00826c28cd0 => generated 383 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:05:12.612525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12892c04-f2ae-469e-8800-38f67fa9f2d1 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:12.613038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12892c04-f2ae-469e-8800-38f67fa9f2d1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:12.613230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12892c04-f2ae-469e-8800-38f67fa9f2d1 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:12.615546 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1276/2551] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:05:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:05:12.636739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d531f-6740-4d9f-988b-9c6591fde850 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:05:12.657570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d531f-6740-4d9f-988b-9c6591fde850 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:12.667234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d531f-6740-4d9f-988b-9c6591fde850 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:12.672221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3b2e3-0b56-4fbb-9573-a2f89bb0c64e tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '634ed56feec24d329afe70247e897af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67e25100d3894bdcaaa7358217e33604', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:12.673231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3b2e3-0b56-4fbb-9573-a2f89bb0c64e tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:12.673531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3b2e3-0b56-4fbb-9573-a2f89bb0c64e tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:12.673810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3b2e3-0b56-4fbb-9573-a2f89bb0c64e tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] PATH_INFO: `/v3/projects/df867107e50e446d8cbca00826c28cd0` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:12.675052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3b2e3-0b56-4fbb-9573-a2f89bb0c64e tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorizing `identity:get_project(project_id=df867107e50e446d8cbca00826c28cd0)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:12.677760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3b2e3-0b56-4fbb-9573-a2f89bb0c64e tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:12.687659 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1276/2552] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:05:12 2022] GET /identity/v3/projects/df867107e50e446d8cbca00826c28cd0 => generated 383 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:05:12.943397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252237a-c90a-471d-8bab-e932a5f02b03 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:12.943929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252237a-c90a-471d-8bab-e932a5f02b03 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:12.944483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b252237a-c90a-471d-8bab-e932a5f02b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:12.972561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b252237a-c90a-471d-8bab-e932a5f02b03 None None] MFA Rules not processed for user `ff0177f6b0bc486d89d2bd33dcfb62d7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:12.975372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b252237a-c90a-471d-8bab-e932a5f02b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:13.135522 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1277/2553] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:13.339152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3b2e3-0b56-4fbb-9573-a2f89bb0c64e tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:13.366607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3b2e3-0b56-4fbb-9573-a2f89bb0c64e tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:13.390193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7a19d-185f-42e0-a93c-1bec8aa0d803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:13.391246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7a19d-185f-42e0-a93c-1bec8aa0d803 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:13.392821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7a19d-185f-42e0-a93c-1bec8aa0d803 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:13.392821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7a19d-185f-42e0-a93c-1bec8aa0d803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:13.425672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7a19d-185f-42e0-a93c-1bec8aa0d803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:13.430389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7a19d-185f-42e0-a93c-1bec8aa0d803 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:13.450233 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1277/2554] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:14.116057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92dfa7-1892-45e8-8026-c849ac887ddb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:14.116215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a4de0-45ea-40e2-8b2f-b5574cebecd4 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:14.117229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a4de0-45ea-40e2-8b2f-b5574cebecd4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:14.117347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92dfa7-1892-45e8-8026-c849ac887ddb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:14.117443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a4de0-45ea-40e2-8b2f-b5574cebecd4 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:14.117531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b92dfa7-1892-45e8-8026-c849ac887ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:14.118248 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1278/2555] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:05:14.128329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a4de0-45ea-40e2-8b2f-b5574cebecd4 None None] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:05:14.150835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a4de0-45ea-40e2-8b2f-b5574cebecd4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:14.164877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a4de0-45ea-40e2-8b2f-b5574cebecd4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:14.175156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c3869-17eb-4950-a790-b17247f24478 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '634ed56feec24d329afe70247e897af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67e25100d3894bdcaaa7358217e33604', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:14.176188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3869-17eb-4950-a790-b17247f24478 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:14.177181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3869-17eb-4950-a790-b17247f24478 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:14.178774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c3869-17eb-4950-a790-b17247f24478 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] PATH_INFO: `/v3/projects/df867107e50e446d8cbca00826c28cd0` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:14.180204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c3869-17eb-4950-a790-b17247f24478 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorizing `identity:get_project(project_id=df867107e50e446d8cbca00826c28cd0)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:14.182674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c3869-17eb-4950-a790-b17247f24478 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:14.185392 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1279/2556] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:05:14 2022] GET /identity/v3/projects/df867107e50e446d8cbca00826c28cd0 => generated 383 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:05:14.186865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0b92dfa7-1892-45e8-8026-c849ac887ddb None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:14.188132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0b92dfa7-1892-45e8-8026-c849ac887ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:14.224444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:14.263438 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1278/2557] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:05:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:14.325130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c3869-17eb-4950-a790-b17247f24478 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:14.327114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8b0c3869-17eb-4950-a790-b17247f24478 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:14.379358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:14.475124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c3869-17eb-4950-a790-b17247f24478 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:14.489551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba50efe-93da-4248-9a93-ef068928d900 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:14.490326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba50efe-93da-4248-9a93-ef068928d900 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:14.490951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba50efe-93da-4248-9a93-ef068928d900 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:14.493191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba50efe-93da-4248-9a93-ef068928d900 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:14.504968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba50efe-93da-4248-9a93-ef068928d900 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:14.506482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7fa1b-ef11-4154-872f-b02a6d38f78e None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:14.506983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7fa1b-ef11-4154-872f-b02a6d38f78e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:14.507238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc7fa1b-ef11-4154-872f-b02a6d38f78e None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:14.507814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba50efe-93da-4248-9a93-ef068928d900 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:14.508784 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1279/2558] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:05:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 21:05:14.514351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7fa1b-ef11-4154-872f-b02a6d38f78e None None] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:05:14.529561 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1280/2559] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:05:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:05:14.572386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7fa1b-ef11-4154-872f-b02a6d38f78e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:14.606261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc7fa1b-ef11-4154-872f-b02a6d38f78e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:14.622592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e462926-9fa9-43e4-a5c4-334e8487adc4 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '634ed56feec24d329afe70247e897af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67e25100d3894bdcaaa7358217e33604', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:14.623622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e462926-9fa9-43e4-a5c4-334e8487adc4 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:14.624235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e462926-9fa9-43e4-a5c4-334e8487adc4 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:14.624727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e462926-9fa9-43e4-a5c4-334e8487adc4 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] PATH_INFO: `/v3/projects/df867107e50e446d8cbca00826c28cd0` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:14.628346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e462926-9fa9-43e4-a5c4-334e8487adc4 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorizing `identity:get_project(project_id=df867107e50e446d8cbca00826c28cd0)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:14.633054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e462926-9fa9-43e4-a5c4-334e8487adc4 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:14.639533 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1280/2560] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:05:14 2022] GET /identity/v3/projects/df867107e50e446d8cbca00826c28cd0 => generated 383 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:05:14.685750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d78e49-4ba4-481c-988d-75c937262dd7 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:14.686310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d78e49-4ba4-481c-988d-75c937262dd7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:14.686581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d78e49-4ba4-481c-988d-75c937262dd7 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:14.688008 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1281/2561] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:05:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:05:14.693466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e462926-9fa9-43e4-a5c4-334e8487adc4 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:05:14.715017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e462926-9fa9-43e4-a5c4-334e8487adc4 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:14.726879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e462926-9fa9-43e4-a5c4-334e8487adc4 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:14.732787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cf106-4765-44c6-84cc-115ab09acb2b tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '634ed56feec24d329afe70247e897af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67e25100d3894bdcaaa7358217e33604', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:14.733124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cf106-4765-44c6-84cc-115ab09acb2b tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:14.733620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cf106-4765-44c6-84cc-115ab09acb2b tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:14.734013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cf106-4765-44c6-84cc-115ab09acb2b tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] PATH_INFO: `/v3/projects/df867107e50e446d8cbca00826c28cd0` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:14.735613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0cf106-4765-44c6-84cc-115ab09acb2b tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorizing `identity:get_project(project_id=df867107e50e446d8cbca00826c28cd0)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:14.737814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0cf106-4765-44c6-84cc-115ab09acb2b tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:14.751964 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1281/2562] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:05:14 2022] GET /identity/v3/projects/df867107e50e446d8cbca00826c28cd0 => generated 383 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:05:14.956202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc227b-dcee-4a84-a875-b77a26d7869b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:14.956512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc227b-dcee-4a84-a875-b77a26d7869b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:14.956766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc227b-dcee-4a84-a875-b77a26d7869b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:15.000236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-20dc227b-dcee-4a84-a875-b77a26d7869b None None] MFA Rules not processed for user `ff0177f6b0bc486d89d2bd33dcfb62d7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:15.002680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-20dc227b-dcee-4a84-a875-b77a26d7869b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:15.032713 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1282/2563] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:16.138039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116c58a-10cd-438d-b1ef-07bddff86bfd None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:16.138657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116c58a-10cd-438d-b1ef-07bddff86bfd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:16.138657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b116c58a-10cd-438d-b1ef-07bddff86bfd None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:16.140590 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1282/2564] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:05:16.154367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc227b-dcee-4a84-a875-b77a26d7869b None None] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:05:16.172780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc227b-dcee-4a84-a875-b77a26d7869b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:16.188299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc227b-dcee-4a84-a875-b77a26d7869b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:16.195132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d7868-a7ef-4223-9157-ee9eb2c3dcde tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '634ed56feec24d329afe70247e897af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67e25100d3894bdcaaa7358217e33604', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:16.196292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d7868-a7ef-4223-9157-ee9eb2c3dcde tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:16.196785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d7868-a7ef-4223-9157-ee9eb2c3dcde tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:16.197258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d7868-a7ef-4223-9157-ee9eb2c3dcde tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] PATH_INFO: `/v3/projects/df867107e50e446d8cbca00826c28cd0` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:16.198712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d7868-a7ef-4223-9157-ee9eb2c3dcde tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorizing `identity:get_project(project_id=df867107e50e446d8cbca00826c28cd0)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:16.200783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d7868-a7ef-4223-9157-ee9eb2c3dcde tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:16.202883 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1283/2565] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:05:16 2022] GET /identity/v3/projects/df867107e50e446d8cbca00826c28cd0 => generated 383 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:05:16.429591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c76fb-9b95-47db-a41f-646b0fb0ca98 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:16.430157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c76fb-9b95-47db-a41f-646b0fb0ca98 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:16.430655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c76fb-9b95-47db-a41f-646b0fb0ca98 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:16.432193 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1283/2566] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:05:16.437068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d7868-a7ef-4223-9157-ee9eb2c3dcde tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:05:16.450715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d7868-a7ef-4223-9157-ee9eb2c3dcde tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:16.461395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d7868-a7ef-4223-9157-ee9eb2c3dcde tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:16.468082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792191b3-69ce-4a23-91ff-24e2824504af tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '634ed56feec24d329afe70247e897af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67e25100d3894bdcaaa7358217e33604', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:16.469240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792191b3-69ce-4a23-91ff-24e2824504af tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:16.469707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792191b3-69ce-4a23-91ff-24e2824504af tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:16.470234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792191b3-69ce-4a23-91ff-24e2824504af tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] PATH_INFO: `/v3/projects/df867107e50e446d8cbca00826c28cd0` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:16.472516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792191b3-69ce-4a23-91ff-24e2824504af tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorizing `identity:get_project(project_id=df867107e50e446d8cbca00826c28cd0)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:16.475201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792191b3-69ce-4a23-91ff-24e2824504af tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:16.476976 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1284/2567] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:05:16 2022] GET /identity/v3/projects/df867107e50e446d8cbca00826c28cd0 => generated 383 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:05:16.513631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c52e8-33f9-4705-b8a7-89da5c31e39f None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:16.514232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c52e8-33f9-4705-b8a7-89da5c31e39f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:16.514686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c52e8-33f9-4705-b8a7-89da5c31e39f None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:16.516595 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1284/2568] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:05:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:05:16.523004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792191b3-69ce-4a23-91ff-24e2824504af tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:05:16.539367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792191b3-69ce-4a23-91ff-24e2824504af tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:16.550186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792191b3-69ce-4a23-91ff-24e2824504af tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:16.559734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7112530-f4cf-47ae-9b40-68f333018bf9 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '634ed56feec24d329afe70247e897af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67e25100d3894bdcaaa7358217e33604', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:16.561092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7112530-f4cf-47ae-9b40-68f333018bf9 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:16.561620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7112530-f4cf-47ae-9b40-68f333018bf9 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:16.562115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7112530-f4cf-47ae-9b40-68f333018bf9 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] PATH_INFO: `/v3/projects/df867107e50e446d8cbca00826c28cd0` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:16.563611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7112530-f4cf-47ae-9b40-68f333018bf9 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorizing `identity:get_project(project_id=df867107e50e446d8cbca00826c28cd0)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:16.565748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7112530-f4cf-47ae-9b40-68f333018bf9 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:16.571809 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1285/2569] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:05:16 2022] GET /identity/v3/projects/df867107e50e446d8cbca00826c28cd0 => generated 383 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:05:16.747639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131124d-4a7a-4da4-b1c0-8bf4d58436a1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:16.747970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131124d-4a7a-4da4-b1c0-8bf4d58436a1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:16.748186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7131124d-4a7a-4da4-b1c0-8bf4d58436a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:16.786882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7131124d-4a7a-4da4-b1c0-8bf4d58436a1 None None] MFA Rules not processed for user `ff0177f6b0bc486d89d2bd33dcfb62d7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:16.788658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7131124d-4a7a-4da4-b1c0-8bf4d58436a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:16.797767 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1285/2570] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:17.979943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283d119-7719-44fc-8f58-3d908f34508a None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:17.981524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283d119-7719-44fc-8f58-3d908f34508a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:17.982043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283d119-7719-44fc-8f58-3d908f34508a None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:17.987819 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1286/2571] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:05:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:05:17.992960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131124d-4a7a-4da4-b1c0-8bf4d58436a1 None None] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:05:18.038796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131124d-4a7a-4da4-b1c0-8bf4d58436a1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:18.056336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7131124d-4a7a-4da4-b1c0-8bf4d58436a1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:18.061598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abddeac-4e6b-4707-8f11-cf2e6cfab110 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '634ed56feec24d329afe70247e897af4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '67e25100d3894bdcaaa7358217e33604', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:18.064679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abddeac-4e6b-4707-8f11-cf2e6cfab110 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:18.065227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abddeac-4e6b-4707-8f11-cf2e6cfab110 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:18.065512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abddeac-4e6b-4707-8f11-cf2e6cfab110 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] PATH_INFO: `/v3/projects/df867107e50e446d8cbca00826c28cd0` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:18.067147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abddeac-4e6b-4707-8f11-cf2e6cfab110 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorizing `identity:get_project(project_id=df867107e50e446d8cbca00826c28cd0)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:18.070311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abddeac-4e6b-4707-8f11-cf2e6cfab110 tempest-QuotasAdminNegativeTest-441086635 tempest-QuotasAdminNegativeTest-441086635-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:18.074071 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1286/2572] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:05:17 2022] GET /identity/v3/projects/df867107e50e446d8cbca00826c28cd0 => generated 383 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:05:18.842938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283d119-7719-44fc-8f58-3d908f34508a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:19.031080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283d119-7719-44fc-8f58-3d908f34508a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:19.037630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3d2fc-5900-4577-b0ff-a510251d4fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:19.039586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3d2fc-5900-4577-b0ff-a510251d4fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:19.040082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3d2fc-5900-4577-b0ff-a510251d4fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:19.040577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b3d2fc-5900-4577-b0ff-a510251d4fe2 admin admin] PATH_INFO: `/v3/users/ff0177f6b0bc486d89d2bd33dcfb62d7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:19.042072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3d2fc-5900-4577-b0ff-a510251d4fe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff0177f6b0bc486d89d2bd33dcfb62d7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:19.044524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b3d2fc-5900-4577-b0ff-a510251d4fe2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:19.357422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-64b3d2fc-5900-4577-b0ff-a510251d4fe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff0177f6b0bc486d89d2bd33dcfb62d7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:19.361197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-64b3d2fc-5900-4577-b0ff-a510251d4fe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff0177f6b0bc486d89d2bd33dcfb62d7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:19.382219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-64b3d2fc-5900-4577-b0ff-a510251d4fe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff0177f6b0bc486d89d2bd33dcfb62d7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:19.489232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-64b3d2fc-5900-4577-b0ff-a510251d4fe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff0177f6b0bc486d89d2bd33dcfb62d7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:19.490255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-64b3d2fc-5900-4577-b0ff-a510251d4fe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff0177f6b0bc486d89d2bd33dcfb62d7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:19.494294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:19.512314 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1287/2573] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:05:18 2022] DELETE /identity/v3/users/ff0177f6b0bc486d89d2bd33dcfb62d7 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:19.516730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bd605-60bf-4029-b65a-8274ebc5356f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:19.517254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bd605-60bf-4029-b65a-8274ebc5356f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:19.517431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9bd605-60bf-4029-b65a-8274ebc5356f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:19.551126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8b9bd605-60bf-4029-b65a-8274ebc5356f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:19.552876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8b9bd605-60bf-4029-b65a-8274ebc5356f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:19.639767 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1287/2574] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:19.662627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3d2fc-5900-4577-b0ff-a510251d4fe2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:19.664582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-64b3d2fc-5900-4577-b0ff-a510251d4fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:19.771175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b3d2fc-5900-4577-b0ff-a510251d4fe2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:19.782494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862d883-379d-4165-961b-f51f1697d434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:19.783186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862d883-379d-4165-961b-f51f1697d434 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:19.783474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862d883-379d-4165-961b-f51f1697d434 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:19.783758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862d883-379d-4165-961b-f51f1697d434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:19.796366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862d883-379d-4165-961b-f51f1697d434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:19.798067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862d883-379d-4165-961b-f51f1697d434 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:19.810739 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1288/2575] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:19.833754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bd605-60bf-4029-b65a-8274ebc5356f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:19.835892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8b9bd605-60bf-4029-b65a-8274ebc5356f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:19.976732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9bd605-60bf-4029-b65a-8274ebc5356f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:19.999195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e0f90-34b3-4e95-936d-579032e1200e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:19.999928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e0f90-34b3-4e95-936d-579032e1200e service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:20.000782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e0f90-34b3-4e95-936d-579032e1200e service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:20.001516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e0f90-34b3-4e95-936d-579032e1200e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:20.005104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-622e0f90-34b3-4e95-936d-579032e1200e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:20.090289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:20.162158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e0f90-34b3-4e95-936d-579032e1200e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:20.165139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e0f90-34b3-4e95-936d-579032e1200e service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:20.187282 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1288/2576] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:20.722052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862d883-379d-4165-961b-f51f1697d434 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:20.724234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3862d883-379d-4165-961b-f51f1697d434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:20.763526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862d883-379d-4165-961b-f51f1697d434 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:20.791011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf43f2e-376a-40ff-9076-94b58363f354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:20.793882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf43f2e-376a-40ff-9076-94b58363f354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:20.794741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf43f2e-376a-40ff-9076-94b58363f354 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:20.795513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf43f2e-376a-40ff-9076-94b58363f354 admin admin] PATH_INFO: `/v3/projects/df867107e50e446d8cbca00826c28cd0` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:20.800228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf43f2e-376a-40ff-9076-94b58363f354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df867107e50e446d8cbca00826c28cd0)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:20.805688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf43f2e-376a-40ff-9076-94b58363f354 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:21.070416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7bf43f2e-376a-40ff-9076-94b58363f354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df867107e50e446d8cbca00826c28cd0', 'request_id': 'req-7bf43f2e-376a-40ff-9076-94b58363f354'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:21.086340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7bf43f2e-376a-40ff-9076-94b58363f354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df867107e50e446d8cbca00826c28cd0', 'request_id': 'req-7bf43f2e-376a-40ff-9076-94b58363f354'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:21.175209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7bf43f2e-376a-40ff-9076-94b58363f354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df867107e50e446d8cbca00826c28cd0', 'request_id': 'req-7bf43f2e-376a-40ff-9076-94b58363f354'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:21.210300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7bf43f2e-376a-40ff-9076-94b58363f354 admin admin] The token cache is being invalidate because project df867107e50e446d8cbca00826c28cd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:05:21.211731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7bf43f2e-376a-40ff-9076-94b58363f354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:21.213368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7bf43f2e-376a-40ff-9076-94b58363f354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:21.215843 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1289/2577] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:05:20 2022] DELETE /identity/v3/projects/df867107e50e446d8cbca00826c28cd0 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:21.219947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e0f90-34b3-4e95-936d-579032e1200e service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:21.236291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e0f90-34b3-4e95-936d-579032e1200e service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:21.367682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2d7658-6ac8-4c97-a5d2-33da503ce9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:21.369198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d7658-6ac8-4c97-a5d2-33da503ce9dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:21.369635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d7658-6ac8-4c97-a5d2-33da503ce9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:21.370087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2d7658-6ac8-4c97-a5d2-33da503ce9dd admin admin] PATH_INFO: `/v3/users/634ed56feec24d329afe70247e897af4` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:21.372143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d7658-6ac8-4c97-a5d2-33da503ce9dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=634ed56feec24d329afe70247e897af4)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:21.376422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2d7658-6ac8-4c97-a5d2-33da503ce9dd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:21.655235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ea2d7658-6ac8-4c97-a5d2-33da503ce9dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '634ed56feec24d329afe70247e897af4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:21.668515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ea2d7658-6ac8-4c97-a5d2-33da503ce9dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '634ed56feec24d329afe70247e897af4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:21.669430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ea2d7658-6ac8-4c97-a5d2-33da503ce9dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '634ed56feec24d329afe70247e897af4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:21.670365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ea2d7658-6ac8-4c97-a5d2-33da503ce9dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '634ed56feec24d329afe70247e897af4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:21.761157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ea2d7658-6ac8-4c97-a5d2-33da503ce9dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '634ed56feec24d329afe70247e897af4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:21.793516 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1289/2578] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:05:21 2022] DELETE /identity/v3/users/634ed56feec24d329afe70247e897af4 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:22.327524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf43f2e-376a-40ff-9076-94b58363f354 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:22.330098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7bf43f2e-376a-40ff-9076-94b58363f354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:22.345390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf43f2e-376a-40ff-9076-94b58363f354 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:22.456150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e0b2f-1d9a-464a-a2ad-de0179b325a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:22.457234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e0b2f-1d9a-464a-a2ad-de0179b325a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:22.457634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e0b2f-1d9a-464a-a2ad-de0179b325a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:22.458336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e0b2f-1d9a-464a-a2ad-de0179b325a9 admin admin] PATH_INFO: `/v3/projects/67e25100d3894bdcaaa7358217e33604` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:22.459806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e0b2f-1d9a-464a-a2ad-de0179b325a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67e25100d3894bdcaaa7358217e33604)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:22.461941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e0b2f-1d9a-464a-a2ad-de0179b325a9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:22.480193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:22.603586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-332e0b2f-1d9a-464a-a2ad-de0179b325a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67e25100d3894bdcaaa7358217e33604', 'request_id': 'req-332e0b2f-1d9a-464a-a2ad-de0179b325a9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:22.620019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-332e0b2f-1d9a-464a-a2ad-de0179b325a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67e25100d3894bdcaaa7358217e33604', 'request_id': 'req-332e0b2f-1d9a-464a-a2ad-de0179b325a9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:22.700222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-332e0b2f-1d9a-464a-a2ad-de0179b325a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67e25100d3894bdcaaa7358217e33604', 'request_id': 'req-332e0b2f-1d9a-464a-a2ad-de0179b325a9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:22.724745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-332e0b2f-1d9a-464a-a2ad-de0179b325a9 admin admin] The token cache is being invalidate because project 67e25100d3894bdcaaa7358217e33604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:05:22.725055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-332e0b2f-1d9a-464a-a2ad-de0179b325a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:22.725793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-332e0b2f-1d9a-464a-a2ad-de0179b325a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:22.728145 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1290/2579] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:05:22 2022] DELETE /identity/v3/projects/67e25100d3894bdcaaa7358217e33604 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:22.739774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250283b8-93f6-4327-a0ad-4041e4b56286 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:22.740781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250283b8-93f6-4327-a0ad-4041e4b56286 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:22.741289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250283b8-93f6-4327-a0ad-4041e4b56286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:22.767912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-250283b8-93f6-4327-a0ad-4041e4b56286 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:22.769231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-250283b8-93f6-4327-a0ad-4041e4b56286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:22.841894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:22.869802 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1290/2580] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:22.876983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44dff8-690a-49ed-8553-135d5dcd5521 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:22.877272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44dff8-690a-49ed-8553-135d5dcd5521 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:22.877553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44dff8-690a-49ed-8553-135d5dcd5521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:22.911578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6c44dff8-690a-49ed-8553-135d5dcd5521 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:22.912856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6c44dff8-690a-49ed-8553-135d5dcd5521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:22.934017 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1291/2581] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:22.935267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250283b8-93f6-4327-a0ad-4041e4b56286 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:22.953757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250283b8-93f6-4327-a0ad-4041e4b56286 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:22.959378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6251b-d078-4ec0-9cc5-59242ccc89eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:22.961799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6251b-d078-4ec0-9cc5-59242ccc89eb admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:22.961799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6251b-d078-4ec0-9cc5-59242ccc89eb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:22.962505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6251b-d078-4ec0-9cc5-59242ccc89eb admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:22.963616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6251b-d078-4ec0-9cc5-59242ccc89eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:05:22.970447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6251b-d078-4ec0-9cc5-59242ccc89eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:22.974051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6251b-d078-4ec0-9cc5-59242ccc89eb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:22.992601 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1291/2582] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:05:22.997046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44dff8-690a-49ed-8553-135d5dcd5521 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:23.007858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44dff8-690a-49ed-8553-135d5dcd5521 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:23.014392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de529b-9ac1-4a2d-bda7-05dba19d46b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:23.015064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de529b-9ac1-4a2d-bda7-05dba19d46b1 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:23.015347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de529b-9ac1-4a2d-bda7-05dba19d46b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:23.015625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de529b-9ac1-4a2d-bda7-05dba19d46b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:23.016380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de529b-9ac1-4a2d-bda7-05dba19d46b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:23.018334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de529b-9ac1-4a2d-bda7-05dba19d46b1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:23.131078 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1292/2583] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:05:22 2022] POST /identity/v3/projects => generated 407 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:05:23.135262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6251b-d078-4ec0-9cc5-59242ccc89eb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:23.149294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6251b-d078-4ec0-9cc5-59242ccc89eb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:23.297052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc5e36-e0ea-403a-bc25-f0d858a3c09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:23.297724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc5e36-e0ea-403a-bc25-f0d858a3c09b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:23.298054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc5e36-e0ea-403a-bc25-f0d858a3c09b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:23.298485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc5e36-e0ea-403a-bc25-f0d858a3c09b admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:23.299384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc5e36-e0ea-403a-bc25-f0d858a3c09b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:23.301575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc5e36-e0ea-403a-bc25-f0d858a3c09b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:23.369192 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1292/2584] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:05:23 2022] POST /identity/v3/users => generated 304 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:05:23.370367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de529b-9ac1-4a2d-bda7-05dba19d46b1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:23.387519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de529b-9ac1-4a2d-bda7-05dba19d46b1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:23.397616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16698b09-3af6-4b1d-a5fc-08b95763cf15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:23.398783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16698b09-3af6-4b1d-a5fc-08b95763cf15 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:23.399394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16698b09-3af6-4b1d-a5fc-08b95763cf15 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:23.399896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16698b09-3af6-4b1d-a5fc-08b95763cf15 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:23.401571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16698b09-3af6-4b1d-a5fc-08b95763cf15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:23.404295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16698b09-3af6-4b1d-a5fc-08b95763cf15 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:23.424791 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1293/2585] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:05:23.429390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc5e36-e0ea-403a-bc25-f0d858a3c09b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:23.450377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc5e36-e0ea-403a-bc25-f0d858a3c09b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:23.458673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc230c-2852-4c10-862a-025ca58792a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:23.459662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc230c-2852-4c10-862a-025ca58792a5 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:23.460058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc230c-2852-4c10-862a-025ca58792a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:23.460340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc230c-2852-4c10-862a-025ca58792a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:23.461120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc230c-2852-4c10-862a-025ca58792a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:23.462864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc230c-2852-4c10-862a-025ca58792a5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:23.481684 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1293/2586] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:05:23.482480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16698b09-3af6-4b1d-a5fc-08b95763cf15 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:23.498993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16698b09-3af6-4b1d-a5fc-08b95763cf15 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:23.516244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791f2b4-cd83-4d47-abba-6ad2012de40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:23.517187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791f2b4-cd83-4d47-abba-6ad2012de40f admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:23.517447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791f2b4-cd83-4d47-abba-6ad2012de40f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:23.517726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791f2b4-cd83-4d47-abba-6ad2012de40f admin admin] PATH_INFO: `/v3/projects/5f25d7a921fc4cfcaa26d7644ed71148/users/0b9327b8e7b645318d0c9012c614ab77/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:23.550097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791f2b4-cd83-4d47-abba-6ad2012de40f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f25d7a921fc4cfcaa26d7644ed71148, user_id=0b9327b8e7b645318d0c9012c614ab77, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:23.552890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791f2b4-cd83-4d47-abba-6ad2012de40f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:23.639993 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1294/2587] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:05:23 2022] PUT /identity/v3/projects/5f25d7a921fc4cfcaa26d7644ed71148/users/0b9327b8e7b645318d0c9012c614ab77/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:23.724650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980606a8-feb9-40b0-ae52-0b35e8917727 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:23.724757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980606a8-feb9-40b0-ae52-0b35e8917727 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:23.725029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980606a8-feb9-40b0-ae52-0b35e8917727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:23.769134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-980606a8-feb9-40b0-ae52-0b35e8917727 None None] MFA Rules not processed for user `0b9327b8e7b645318d0c9012c614ab77`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:23.770656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-980606a8-feb9-40b0-ae52-0b35e8917727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:23.896651 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1294/2588] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:23 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:23.901600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791f2b4-cd83-4d47-abba-6ad2012de40f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:23.979025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791f2b4-cd83-4d47-abba-6ad2012de40f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:23.988455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cd6e5-1017-4c03-a67f-8da680dcc962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:23.989986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cd6e5-1017-4c03-a67f-8da680dcc962 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:23.990427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cd6e5-1017-4c03-a67f-8da680dcc962 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:23.990897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56cd6e5-1017-4c03-a67f-8da680dcc962 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:23.991976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56cd6e5-1017-4c03-a67f-8da680dcc962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:23.994320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56cd6e5-1017-4c03-a67f-8da680dcc962 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:24.125765 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1295/2589] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:05:23 2022] POST /identity/v3/projects => generated 407 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:05:24.130083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980606a8-feb9-40b0-ae52-0b35e8917727 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:24.143157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980606a8-feb9-40b0-ae52-0b35e8917727 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:24.193170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:24.214128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d124c-ab26-487e-867a-3b2b763e4455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:24.214128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d124c-ab26-487e-867a-3b2b763e4455 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:24.214128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d124c-ab26-487e-867a-3b2b763e4455 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:24.214128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d124c-ab26-487e-867a-3b2b763e4455 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:24.214580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d124c-ab26-487e-867a-3b2b763e4455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:24.214580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d124c-ab26-487e-867a-3b2b763e4455 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:24.570409 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1295/2590] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:05:24 2022] POST /identity/v3/users => generated 310 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:05:24.571789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cd6e5-1017-4c03-a67f-8da680dcc962 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:24.584503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56cd6e5-1017-4c03-a67f-8da680dcc962 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:24.591079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84049f-ddbb-4e98-8e89-61e48d12bda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:24.592091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84049f-ddbb-4e98-8e89-61e48d12bda2 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:24.592531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84049f-ddbb-4e98-8e89-61e48d12bda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:24.592967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84049f-ddbb-4e98-8e89-61e48d12bda2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:24.593753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84049f-ddbb-4e98-8e89-61e48d12bda2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:24.595412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84049f-ddbb-4e98-8e89-61e48d12bda2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:24.626875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d124c-ab26-487e-867a-3b2b763e4455 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:24.634270 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1296/2591] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:05:24.643067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d124c-ab26-487e-867a-3b2b763e4455 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:24.656194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8745f-b09d-45a1-a0f2-56e0e4b6fb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:24.657479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8745f-b09d-45a1-a0f2-56e0e4b6fb4c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:24.658119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8745f-b09d-45a1-a0f2-56e0e4b6fb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:24.658609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf8745f-b09d-45a1-a0f2-56e0e4b6fb4c admin admin] PATH_INFO: `/v3/projects/43008e97947e409e86b21624db90b2a2/users/f7aa8be6dd4f4b97bb9a4127d7c1db2b/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:24.685266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8745f-b09d-45a1-a0f2-56e0e4b6fb4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=43008e97947e409e86b21624db90b2a2, user_id=f7aa8be6dd4f4b97bb9a4127d7c1db2b, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:24.689400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf8745f-b09d-45a1-a0f2-56e0e4b6fb4c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:24.772345 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1296/2592] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:05:24 2022] PUT /identity/v3/projects/43008e97947e409e86b21624db90b2a2/users/f7aa8be6dd4f4b97bb9a4127d7c1db2b/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:05:24.776913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84049f-ddbb-4e98-8e89-61e48d12bda2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:24.791691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84049f-ddbb-4e98-8e89-61e48d12bda2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:24.815089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:24.971354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c746d7-ab6b-49dd-905c-55c317d237c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:24.972053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c746d7-ab6b-49dd-905c-55c317d237c4 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:24.972336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c746d7-ab6b-49dd-905c-55c317d237c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:24.972613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c746d7-ab6b-49dd-905c-55c317d237c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:24.973390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c746d7-ab6b-49dd-905c-55c317d237c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:24.980166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c746d7-ab6b-49dd-905c-55c317d237c4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:25.008625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8745f-b09d-45a1-a0f2-56e0e4b6fb4c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:25.015393 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1297/2593] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:05:25.025777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf8745f-b09d-45a1-a0f2-56e0e4b6fb4c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:25.029801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10516d-059a-4605-abb7-ca963c23c76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:25.032774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10516d-059a-4605-abb7-ca963c23c76d admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:25.033079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10516d-059a-4605-abb7-ca963c23c76d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:25.033375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10516d-059a-4605-abb7-ca963c23c76d admin admin] PATH_INFO: `/v3/projects/43008e97947e409e86b21624db90b2a2/users/f7aa8be6dd4f4b97bb9a4127d7c1db2b/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:25.050209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10516d-059a-4605-abb7-ca963c23c76d admin admin] RBAC: Authorizing `identity:create_grant(project_id=43008e97947e409e86b21624db90b2a2, user_id=f7aa8be6dd4f4b97bb9a4127d7c1db2b, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:25.053859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10516d-059a-4605-abb7-ca963c23c76d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:25.092106 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1297/2594] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:05:25 2022] PUT /identity/v3/projects/43008e97947e409e86b21624db90b2a2/users/f7aa8be6dd4f4b97bb9a4127d7c1db2b/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:05:25.096259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d42fe0-ff2a-40b8-8e5f-aa4e9d785126 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:25.096259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d42fe0-ff2a-40b8-8e5f-aa4e9d785126 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:25.096259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d42fe0-ff2a-40b8-8e5f-aa4e9d785126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:25.163256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-48d42fe0-ff2a-40b8-8e5f-aa4e9d785126 None None] MFA Rules not processed for user `f7aa8be6dd4f4b97bb9a4127d7c1db2b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:25.163256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-48d42fe0-ff2a-40b8-8e5f-aa4e9d785126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:25.328391 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1298/2595] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:25 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:05:25.339251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10516d-059a-4605-abb7-ca963c23c76d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:25.455283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10516d-059a-4605-abb7-ca963c23c76d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:25.462358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593eca3-2477-4d8d-b049-5bae64f7bf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:25.463389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593eca3-2477-4d8d-b049-5bae64f7bf30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:25.464209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593eca3-2477-4d8d-b049-5bae64f7bf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:25.464209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593eca3-2477-4d8d-b049-5bae64f7bf30 admin admin] PATH_INFO: `/v3/users/0b9327b8e7b645318d0c9012c614ab77` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:25.465401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593eca3-2477-4d8d-b049-5bae64f7bf30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b9327b8e7b645318d0c9012c614ab77)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:25.467710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593eca3-2477-4d8d-b049-5bae64f7bf30 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:25.733310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c593eca3-2477-4d8d-b049-5bae64f7bf30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b9327b8e7b645318d0c9012c614ab77'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:25.758198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c593eca3-2477-4d8d-b049-5bae64f7bf30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b9327b8e7b645318d0c9012c614ab77'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:25.759973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c593eca3-2477-4d8d-b049-5bae64f7bf30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b9327b8e7b645318d0c9012c614ab77'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:25.760745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c593eca3-2477-4d8d-b049-5bae64f7bf30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b9327b8e7b645318d0c9012c614ab77'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:25.866313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c593eca3-2477-4d8d-b049-5bae64f7bf30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b9327b8e7b645318d0c9012c614ab77'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:25.926770 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1298/2596] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:05:25 2022] DELETE /identity/v3/users/0b9327b8e7b645318d0c9012c614ab77 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:25.930589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63276fa4-6cb1-4a6b-ba15-a9f3796c4ac8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:25.931624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63276fa4-6cb1-4a6b-ba15-a9f3796c4ac8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:25.932123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63276fa4-6cb1-4a6b-ba15-a9f3796c4ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:25.969243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-63276fa4-6cb1-4a6b-ba15-a9f3796c4ac8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:25.970605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-63276fa4-6cb1-4a6b-ba15-a9f3796c4ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:26.130444 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1299/2597] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:26.145129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593eca3-2477-4d8d-b049-5bae64f7bf30 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:26.146992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c593eca3-2477-4d8d-b049-5bae64f7bf30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:26.204274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:26.281872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593eca3-2477-4d8d-b049-5bae64f7bf30 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:26.291453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efea18-2561-4c79-b0ac-7d861c9904a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:26.291453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efea18-2561-4c79-b0ac-7d861c9904a5 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:26.291880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efea18-2561-4c79-b0ac-7d861c9904a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:26.292073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50efea18-2561-4c79-b0ac-7d861c9904a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:26.306394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50efea18-2561-4c79-b0ac-7d861c9904a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:26.308258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50efea18-2561-4c79-b0ac-7d861c9904a5 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:26.320969 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1299/2598] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:27.380961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63276fa4-6cb1-4a6b-ba15-a9f3796c4ac8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:27.384297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-63276fa4-6cb1-4a6b-ba15-a9f3796c4ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:27.406609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63276fa4-6cb1-4a6b-ba15-a9f3796c4ac8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:27.421213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f14c1-894a-421f-9ddc-1e8d89d10435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:27.423100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f14c1-894a-421f-9ddc-1e8d89d10435 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:27.423482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f14c1-894a-421f-9ddc-1e8d89d10435 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:27.423854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f14c1-894a-421f-9ddc-1e8d89d10435 admin admin] PATH_INFO: `/v3/projects/5f25d7a921fc4cfcaa26d7644ed71148` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:27.426477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f14c1-894a-421f-9ddc-1e8d89d10435 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f25d7a921fc4cfcaa26d7644ed71148)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:27.430881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f14c1-894a-421f-9ddc-1e8d89d10435 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:27.614399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:27.660870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-096f14c1-894a-421f-9ddc-1e8d89d10435 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f25d7a921fc4cfcaa26d7644ed71148', 'request_id': 'req-096f14c1-894a-421f-9ddc-1e8d89d10435'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:27.675172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-096f14c1-894a-421f-9ddc-1e8d89d10435 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f25d7a921fc4cfcaa26d7644ed71148', 'request_id': 'req-096f14c1-894a-421f-9ddc-1e8d89d10435'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:27.757452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-096f14c1-894a-421f-9ddc-1e8d89d10435 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f25d7a921fc4cfcaa26d7644ed71148', 'request_id': 'req-096f14c1-894a-421f-9ddc-1e8d89d10435'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:27.777336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-096f14c1-894a-421f-9ddc-1e8d89d10435 admin admin] The token cache is being invalidate because project 5f25d7a921fc4cfcaa26d7644ed71148 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:05:27.777635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-096f14c1-894a-421f-9ddc-1e8d89d10435 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:27.778355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-096f14c1-894a-421f-9ddc-1e8d89d10435 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:27.781054 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1300/2599] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:05:27 2022] DELETE /identity/v3/projects/5f25d7a921fc4cfcaa26d7644ed71148 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:27.785724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efea18-2561-4c79-b0ac-7d861c9904a5 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:27.798950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50efea18-2561-4c79-b0ac-7d861c9904a5 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:27.885307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ded42-8b13-42bf-8865-c5b8459e2a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:27.886258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ded42-8b13-42bf-8865-c5b8459e2a4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:27.886543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ded42-8b13-42bf-8865-c5b8459e2a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:27.888594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ded42-8b13-42bf-8865-c5b8459e2a4b admin admin] PATH_INFO: `/v3/users/f7aa8be6dd4f4b97bb9a4127d7c1db2b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:27.893507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ded42-8b13-42bf-8865-c5b8459e2a4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7aa8be6dd4f4b97bb9a4127d7c1db2b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:27.895581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ded42-8b13-42bf-8865-c5b8459e2a4b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:28.131572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b97ded42-8b13-42bf-8865-c5b8459e2a4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7aa8be6dd4f4b97bb9a4127d7c1db2b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:28.153174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b97ded42-8b13-42bf-8865-c5b8459e2a4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7aa8be6dd4f4b97bb9a4127d7c1db2b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:28.156801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b97ded42-8b13-42bf-8865-c5b8459e2a4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7aa8be6dd4f4b97bb9a4127d7c1db2b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:28.157645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b97ded42-8b13-42bf-8865-c5b8459e2a4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7aa8be6dd4f4b97bb9a4127d7c1db2b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:28.260455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b97ded42-8b13-42bf-8865-c5b8459e2a4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7aa8be6dd4f4b97bb9a4127d7c1db2b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:28.320646 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1300/2600] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:05:27 2022] DELETE /identity/v3/users/f7aa8be6dd4f4b97bb9a4127d7c1db2b => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:29.354648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f14c1-894a-421f-9ddc-1e8d89d10435 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:29.356757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-096f14c1-894a-421f-9ddc-1e8d89d10435 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:29.386333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f14c1-894a-421f-9ddc-1e8d89d10435 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:29.615944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab5d73b-b61f-4cee-a780-6ef767d40014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:29.620174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5d73b-b61f-4cee-a780-6ef767d40014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:29.622509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5d73b-b61f-4cee-a780-6ef767d40014 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:29.623337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5d73b-b61f-4cee-a780-6ef767d40014 admin admin] PATH_INFO: `/v3/projects/43008e97947e409e86b21624db90b2a2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:29.628217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5d73b-b61f-4cee-a780-6ef767d40014 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43008e97947e409e86b21624db90b2a2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:29.632798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab5d73b-b61f-4cee-a780-6ef767d40014 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:29.871774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aab5d73b-b61f-4cee-a780-6ef767d40014 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43008e97947e409e86b21624db90b2a2', 'request_id': 'req-aab5d73b-b61f-4cee-a780-6ef767d40014'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:29.883868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aab5d73b-b61f-4cee-a780-6ef767d40014 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43008e97947e409e86b21624db90b2a2', 'request_id': 'req-aab5d73b-b61f-4cee-a780-6ef767d40014'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:30.367430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ded42-8b13-42bf-8865-c5b8459e2a4b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:30.369151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b97ded42-8b13-42bf-8865-c5b8459e2a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:30.439969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:30.466051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ded42-8b13-42bf-8865-c5b8459e2a4b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:30.482902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aab5d73b-b61f-4cee-a780-6ef767d40014 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43008e97947e409e86b21624db90b2a2', 'request_id': 'req-aab5d73b-b61f-4cee-a780-6ef767d40014'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:30.496797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aab5d73b-b61f-4cee-a780-6ef767d40014 admin admin] The token cache is being invalidate because project 43008e97947e409e86b21624db90b2a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:05:30.503992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aab5d73b-b61f-4cee-a780-6ef767d40014 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:30.503992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aab5d73b-b61f-4cee-a780-6ef767d40014 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:30.503992 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1301/2601] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:05:29 2022] DELETE /identity/v3/projects/43008e97947e409e86b21624db90b2a2 => generated 0 bytes in 1145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:30.506884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd904b-8f4f-400b-a9ad-afa3ae02882a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:30.507520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd904b-8f4f-400b-a9ad-afa3ae02882a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:30.507809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd904b-8f4f-400b-a9ad-afa3ae02882a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:30.525647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-06cd904b-8f4f-400b-a9ad-afa3ae02882a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:30.525825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a2010a-ff6e-452c-bb3e-51a77668569a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:30.526442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2010a-ff6e-452c-bb3e-51a77668569a service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:30.526732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2010a-ff6e-452c-bb3e-51a77668569a service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:30.526918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-06cd904b-8f4f-400b-a9ad-afa3ae02882a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:30.527030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a2010a-ff6e-452c-bb3e-51a77668569a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:30.528782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-60a2010a-ff6e-452c-bb3e-51a77668569a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:30.540095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2010a-ff6e-452c-bb3e-51a77668569a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:30.541727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a2010a-ff6e-452c-bb3e-51a77668569a service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:30.552477 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1301/2602] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:30.566732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:30.588931 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1302/2603] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:05:30.597447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddabd0-32b9-4232-b185-6ab559fc9c39 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:30.598107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddabd0-32b9-4232-b185-6ab559fc9c39 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:30.598837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fddabd0-32b9-4232-b185-6ab559fc9c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:30.625295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6fddabd0-32b9-4232-b185-6ab559fc9c39 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:30.625295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6fddabd0-32b9-4232-b185-6ab559fc9c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:30.628986 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1302/2604] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:05:30.637260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd904b-8f4f-400b-a9ad-afa3ae02882a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:30.648283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd904b-8f4f-400b-a9ad-afa3ae02882a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:30.655825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80aa9c0-1681-4d05-980c-236698134f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:30.657279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80aa9c0-1681-4d05-980c-236698134f35 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:30.657279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80aa9c0-1681-4d05-980c-236698134f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:30.657279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80aa9c0-1681-4d05-980c-236698134f35 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:30.658035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80aa9c0-1681-4d05-980c-236698134f35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:05:30.658349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80aa9c0-1681-4d05-980c-236698134f35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:30.659966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80aa9c0-1681-4d05-980c-236698134f35 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:30.671937 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1303/2605] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:05:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:05:30.677356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddabd0-32b9-4232-b185-6ab559fc9c39 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:30.688740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fddabd0-32b9-4232-b185-6ab559fc9c39 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:30.695362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c5b60-2014-4462-b95f-b3941018752d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:30.696068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c5b60-2014-4462-b95f-b3941018752d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:30.696354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c5b60-2014-4462-b95f-b3941018752d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:30.696631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635c5b60-2014-4462-b95f-b3941018752d admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:30.697390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c5b60-2014-4462-b95f-b3941018752d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:30.699382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635c5b60-2014-4462-b95f-b3941018752d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:30.796715 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1303/2606] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:05:30 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:05:30.797731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80aa9c0-1681-4d05-980c-236698134f35 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:30.823824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80aa9c0-1681-4d05-980c-236698134f35 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:30.900926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2ee7d-3a6a-4f6a-b7af-8d98df5a85ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:30.901604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2ee7d-3a6a-4f6a-b7af-8d98df5a85ea admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:30.901922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2ee7d-3a6a-4f6a-b7af-8d98df5a85ea admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:30.902205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2ee7d-3a6a-4f6a-b7af-8d98df5a85ea admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:30.902903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2ee7d-3a6a-4f6a-b7af-8d98df5a85ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:30.905460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2ee7d-3a6a-4f6a-b7af-8d98df5a85ea admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:31.000631 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1304/2607] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:05:30 2022] POST /identity/v3/users => generated 290 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:05:31.004719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c5b60-2014-4462-b95f-b3941018752d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:31.037750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635c5b60-2014-4462-b95f-b3941018752d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:31.042446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314de5a-6c24-4c15-839a-7e4e2c7865a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:31.043373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314de5a-6c24-4c15-839a-7e4e2c7865a6 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:31.043853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314de5a-6c24-4c15-839a-7e4e2c7865a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:31.044325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314de5a-6c24-4c15-839a-7e4e2c7865a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:31.045300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314de5a-6c24-4c15-839a-7e4e2c7865a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:31.047502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314de5a-6c24-4c15-839a-7e4e2c7865a6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:31.063741 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1304/2608] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:05:31.064894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2ee7d-3a6a-4f6a-b7af-8d98df5a85ea admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:31.075632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2ee7d-3a6a-4f6a-b7af-8d98df5a85ea admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:31.081124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26334c1a-7b64-4b49-8593-ea6f0031b4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:31.082079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26334c1a-7b64-4b49-8593-ea6f0031b4f3 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:31.082568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26334c1a-7b64-4b49-8593-ea6f0031b4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:31.083022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26334c1a-7b64-4b49-8593-ea6f0031b4f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:31.083922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26334c1a-7b64-4b49-8593-ea6f0031b4f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:31.085855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26334c1a-7b64-4b49-8593-ea6f0031b4f3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:31.097751 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1305/2609] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:05:31.102867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314de5a-6c24-4c15-839a-7e4e2c7865a6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:31.116165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314de5a-6c24-4c15-839a-7e4e2c7865a6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:31.122108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5c00c-23b2-47d6-959b-3888279588da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:31.123078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5c00c-23b2-47d6-959b-3888279588da admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:31.123343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5c00c-23b2-47d6-959b-3888279588da admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:31.123635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5c00c-23b2-47d6-959b-3888279588da admin admin] PATH_INFO: `/v3/projects/ed128675a82f4cc9b01ab452db18e51b/users/fe3e18c14d754955a33719394f690e0f/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:31.140548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa5c00c-23b2-47d6-959b-3888279588da admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed128675a82f4cc9b01ab452db18e51b, user_id=fe3e18c14d754955a33719394f690e0f, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:31.144407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa5c00c-23b2-47d6-959b-3888279588da admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:31.284188 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1305/2610] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:05:31 2022] PUT /identity/v3/projects/ed128675a82f4cc9b01ab452db18e51b/users/fe3e18c14d754955a33719394f690e0f/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:31.286655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeff3d5-f358-4226-bf14-fcde1597be88 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:31.286948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeff3d5-f358-4226-bf14-fcde1597be88 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:31.287263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeff3d5-f358-4226-bf14-fcde1597be88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:31.315231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4aeff3d5-f358-4226-bf14-fcde1597be88 None None] MFA Rules not processed for user `fe3e18c14d754955a33719394f690e0f`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:31.316463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4aeff3d5-f358-4226-bf14-fcde1597be88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:31.383145 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1306/2611] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:31.389054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5c00c-23b2-47d6-959b-3888279588da admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:31.449785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5c00c-23b2-47d6-959b-3888279588da admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:31.454502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcf9a9-5481-411b-9cd6-b443bbf3c505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:31.455156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcf9a9-5481-411b-9cd6-b443bbf3c505 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:31.455481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcf9a9-5481-411b-9cd6-b443bbf3c505 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:31.455746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcf9a9-5481-411b-9cd6-b443bbf3c505 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:31.456398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcf9a9-5481-411b-9cd6-b443bbf3c505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:31.458893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcf9a9-5481-411b-9cd6-b443bbf3c505 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:31.613541 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1306/2612] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:05:31 2022] POST /identity/v3/projects => generated 377 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:05:31.618092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeff3d5-f358-4226-bf14-fcde1597be88 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:31.634620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeff3d5-f358-4226-bf14-fcde1597be88 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:31.697625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046c55f-2be6-4ef3-9a01-7deb03e90d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:31.698540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046c55f-2be6-4ef3-9a01-7deb03e90d56 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:31.698927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046c55f-2be6-4ef3-9a01-7deb03e90d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:31.699249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046c55f-2be6-4ef3-9a01-7deb03e90d56 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:31.700000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046c55f-2be6-4ef3-9a01-7deb03e90d56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:31.702054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046c55f-2be6-4ef3-9a01-7deb03e90d56 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:31.809875 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1307/2613] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:05:31 2022] POST /identity/v3/users => generated 295 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:05:31.813404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcf9a9-5481-411b-9cd6-b443bbf3c505 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:31.824702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcf9a9-5481-411b-9cd6-b443bbf3c505 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:31.829660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae003d3a-6d2a-4fe5-9cc4-4406fc0d49ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:31.830555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae003d3a-6d2a-4fe5-9cc4-4406fc0d49ce admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:31.830840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae003d3a-6d2a-4fe5-9cc4-4406fc0d49ce admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:31.831132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae003d3a-6d2a-4fe5-9cc4-4406fc0d49ce admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:31.831851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae003d3a-6d2a-4fe5-9cc4-4406fc0d49ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:31.834048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae003d3a-6d2a-4fe5-9cc4-4406fc0d49ce admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:31.846146 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1307/2614] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:05:31.850255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046c55f-2be6-4ef3-9a01-7deb03e90d56 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:31.867468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046c55f-2be6-4ef3-9a01-7deb03e90d56 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:31.873576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c7a26-4f04-4793-b1f6-8acf64548af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:31.873916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c7a26-4f04-4793-b1f6-8acf64548af1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:31.874202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c7a26-4f04-4793-b1f6-8acf64548af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:31.874475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c7a26-4f04-4793-b1f6-8acf64548af1 admin admin] PATH_INFO: `/v3/projects/c4decf581404468ab8ca1873825b242e/users/cb8be78218064ee0a9319fc65ee59a75/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:31.937958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c7a26-4f04-4793-b1f6-8acf64548af1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4decf581404468ab8ca1873825b242e, user_id=cb8be78218064ee0a9319fc65ee59a75, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:31.943095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c7a26-4f04-4793-b1f6-8acf64548af1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:32.010102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6370e9-6b5f-467e-bba3-f99896781baa None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:32.011852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6370e9-6b5f-467e-bba3-f99896781baa None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:32.012563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6370e9-6b5f-467e-bba3-f99896781baa None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:32.016150 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1308/2615] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:05:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:05:32.019273 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1308/2616] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:05:31 2022] PUT /identity/v3/projects/c4decf581404468ab8ca1873825b242e/users/cb8be78218064ee0a9319fc65ee59a75/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:05:32.023304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6370e9-6b5f-467e-bba3-f99896781baa None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:32.024869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f10dd6-d5d1-4e60-9c05-bb5436e7228c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:32.025171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f10dd6-d5d1-4e60-9c05-bb5436e7228c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:32.025560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f10dd6-d5d1-4e60-9c05-bb5436e7228c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:32.047470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e9f10dd6-d5d1-4e60-9c05-bb5436e7228c None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:32.048717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e9f10dd6-d5d1-4e60-9c05-bb5436e7228c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:32.112485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:32.128778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6370e9-6b5f-467e-bba3-f99896781baa None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:32.135644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685f7cd-42b4-4088-979e-78157e4759b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:32.137054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685f7cd-42b4-4088-979e-78157e4759b2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:32.137360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685f7cd-42b4-4088-979e-78157e4759b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:32.137691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685f7cd-42b4-4088-979e-78157e4759b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:32.138434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685f7cd-42b4-4088-979e-78157e4759b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:32.144619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:32.144824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685f7cd-42b4-4088-979e-78157e4759b2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:32.158458 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1309/2617] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:05:32.162071 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1309/2618] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:05:32.167829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685f7cd-42b4-4088-979e-78157e4759b2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:32.175327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685f7cd-42b4-4088-979e-78157e4759b2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:32.180355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f10dd6-d5d1-4e60-9c05-bb5436e7228c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:32.181953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e9f10dd6-d5d1-4e60-9c05-bb5436e7228c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:32.184060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999202a2-606f-47d7-9161-79d859beb3d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:32.185229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999202a2-606f-47d7-9161-79d859beb3d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:32.185711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999202a2-606f-47d7-9161-79d859beb3d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:32.186485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999202a2-606f-47d7-9161-79d859beb3d4 admin admin] PATH_INFO: `/v3/projects/c4decf581404468ab8ca1873825b242e/users/cb8be78218064ee0a9319fc65ee59a75/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:32.190757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999202a2-606f-47d7-9161-79d859beb3d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4decf581404468ab8ca1873825b242e, user_id=cb8be78218064ee0a9319fc65ee59a75, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:32.192886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999202a2-606f-47d7-9161-79d859beb3d4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:32.198044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f10dd6-d5d1-4e60-9c05-bb5436e7228c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:32.204870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bda0ff-4ecc-46a0-9895-0975167bb2da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:32.205590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bda0ff-4ecc-46a0-9895-0975167bb2da service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:32.205898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bda0ff-4ecc-46a0-9895-0975167bb2da service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:32.206177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bda0ff-4ecc-46a0-9895-0975167bb2da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:32.207389 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1310/2619] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:05:32 2022] PUT /identity/v3/projects/c4decf581404468ab8ca1873825b242e/users/cb8be78218064ee0a9319fc65ee59a75/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:32.217323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e15062-542f-4c85-9682-9af182533387 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:32.217614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e15062-542f-4c85-9682-9af182533387 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:32.218049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e15062-542f-4c85-9682-9af182533387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:32.220054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bda0ff-4ecc-46a0-9895-0975167bb2da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:32.221763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bda0ff-4ecc-46a0-9895-0975167bb2da service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:32.233491 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1310/2620] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:05:32.248619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-84e15062-542f-4c85-9682-9af182533387 None None] MFA Rules not processed for user `cb8be78218064ee0a9319fc65ee59a75`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:32.250031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-84e15062-542f-4c85-9682-9af182533387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:32.336978 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1311/2621] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:05:33.039038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bda0ff-4ecc-46a0-9895-0975167bb2da service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:33.050176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bda0ff-4ecc-46a0-9895-0975167bb2da service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:33.118950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb086d-351f-475c-957a-57f9c77b71bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:33.119616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb086d-351f-475c-957a-57f9c77b71bf service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:33.119903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb086d-351f-475c-957a-57f9c77b71bf service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:33.120179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eb086d-351f-475c-957a-57f9c77b71bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:33.122030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-66eb086d-351f-475c-957a-57f9c77b71bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:33.135340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb086d-351f-475c-957a-57f9c77b71bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:33.136963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eb086d-351f-475c-957a-57f9c77b71bf service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:33.147779 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1311/2622] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:33.350095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e15062-542f-4c85-9682-9af182533387 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:33.350095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-84e15062-542f-4c85-9682-9af182533387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:33.401038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:33.425755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e15062-542f-4c85-9682-9af182533387 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:33.432777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b7688-1436-4b46-ab96-c8dd591c5bcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:33.433443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b7688-1436-4b46-ab96-c8dd591c5bcc service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:33.433732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b7688-1436-4b46-ab96-c8dd591c5bcc service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:33.434092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b7688-1436-4b46-ab96-c8dd591c5bcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:33.435950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e17b7688-1436-4b46-ab96-c8dd591c5bcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:33.498930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b7688-1436-4b46-ab96-c8dd591c5bcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:33.500866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b7688-1436-4b46-ab96-c8dd591c5bcc service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:33.513979 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1312/2623] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:33.640690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb086d-351f-475c-957a-57f9c77b71bf service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:33.654048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eb086d-351f-475c-957a-57f9c77b71bf service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:33.660174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3614983-e031-4c6e-ae14-de93a4906242 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:33.660865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3614983-e031-4c6e-ae14-de93a4906242 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:33.661312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3614983-e031-4c6e-ae14-de93a4906242 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:33.661450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3614983-e031-4c6e-ae14-de93a4906242 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:33.670299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3614983-e031-4c6e-ae14-de93a4906242 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:33.671880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3614983-e031-4c6e-ae14-de93a4906242 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:33.682187 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1312/2624] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:33 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:34.702157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b7688-1436-4b46-ab96-c8dd591c5bcc service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:34.884685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b7688-1436-4b46-ab96-c8dd591c5bcc service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:34.906319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957502d4-3458-4ad0-9c48-fe01932f55f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:34.906319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957502d4-3458-4ad0-9c48-fe01932f55f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:34.906319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957502d4-3458-4ad0-9c48-fe01932f55f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:34.906319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957502d4-3458-4ad0-9c48-fe01932f55f7 admin admin] PATH_INFO: `/v3/users/fe3e18c14d754955a33719394f690e0f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:34.906874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957502d4-3458-4ad0-9c48-fe01932f55f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe3e18c14d754955a33719394f690e0f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:34.908195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957502d4-3458-4ad0-9c48-fe01932f55f7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:35.222701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-957502d4-3458-4ad0-9c48-fe01932f55f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe3e18c14d754955a33719394f690e0f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:35.242639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-957502d4-3458-4ad0-9c48-fe01932f55f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe3e18c14d754955a33719394f690e0f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:35.243411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-957502d4-3458-4ad0-9c48-fe01932f55f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe3e18c14d754955a33719394f690e0f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:35.244292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-957502d4-3458-4ad0-9c48-fe01932f55f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe3e18c14d754955a33719394f690e0f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:35.346123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-957502d4-3458-4ad0-9c48-fe01932f55f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe3e18c14d754955a33719394f690e0f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:35.376044 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1313/2625] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:05:34 2022] DELETE /identity/v3/users/fe3e18c14d754955a33719394f690e0f => generated 0 bytes in 671 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:35.378606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b06b9-ee11-44b0-86ba-01794a0cfddd None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:35.380038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b06b9-ee11-44b0-86ba-01794a0cfddd None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:35.380497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b06b9-ee11-44b0-86ba-01794a0cfddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:35.389664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:35.400814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-451b06b9-ee11-44b0-86ba-01794a0cfddd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:35.402287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-451b06b9-ee11-44b0-86ba-01794a0cfddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:35.531662 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1313/2626] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:35.557117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957502d4-3458-4ad0-9c48-fe01932f55f7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:35.559042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-957502d4-3458-4ad0-9c48-fe01932f55f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:35.571773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:35.638265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957502d4-3458-4ad0-9c48-fe01932f55f7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:35.642291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cfd81-10fd-4411-a059-c4707fdbbe3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:35.642901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cfd81-10fd-4411-a059-c4707fdbbe3a service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:35.643364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cfd81-10fd-4411-a059-c4707fdbbe3a service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:35.643634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cfd81-10fd-4411-a059-c4707fdbbe3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:35.655239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cfd81-10fd-4411-a059-c4707fdbbe3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:35.656716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cfd81-10fd-4411-a059-c4707fdbbe3a service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:35.667605 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1314/2627] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:36.339381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b06b9-ee11-44b0-86ba-01794a0cfddd None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:36.341019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-451b06b9-ee11-44b0-86ba-01794a0cfddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:36.356000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b06b9-ee11-44b0-86ba-01794a0cfddd None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:36.362356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be857c76-6414-44b1-b367-ab414e076391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:36.363239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be857c76-6414-44b1-b367-ab414e076391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:36.364155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be857c76-6414-44b1-b367-ab414e076391 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:36.364506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be857c76-6414-44b1-b367-ab414e076391 admin admin] PATH_INFO: `/v3/projects/ed128675a82f4cc9b01ab452db18e51b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:36.365873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be857c76-6414-44b1-b367-ab414e076391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed128675a82f4cc9b01ab452db18e51b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:36.368281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be857c76-6414-44b1-b367-ab414e076391 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:36.476647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-be857c76-6414-44b1-b367-ab414e076391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed128675a82f4cc9b01ab452db18e51b', 'request_id': 'req-be857c76-6414-44b1-b367-ab414e076391'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:36.484766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-be857c76-6414-44b1-b367-ab414e076391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed128675a82f4cc9b01ab452db18e51b', 'request_id': 'req-be857c76-6414-44b1-b367-ab414e076391'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:36.563905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-be857c76-6414-44b1-b367-ab414e076391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed128675a82f4cc9b01ab452db18e51b', 'request_id': 'req-be857c76-6414-44b1-b367-ab414e076391'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:36.578687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da204447-9422-4851-8726-9cbdd98e98a5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:36.579576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da204447-9422-4851-8726-9cbdd98e98a5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:36.580040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da204447-9422-4851-8726-9cbdd98e98a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:36.581189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-be857c76-6414-44b1-b367-ab414e076391 admin admin] The token cache is being invalidate because project ed128675a82f4cc9b01ab452db18e51b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:05:36.581491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-be857c76-6414-44b1-b367-ab414e076391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:36.582755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-be857c76-6414-44b1-b367-ab414e076391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:36.584994 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1314/2628] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:05:36 2022] DELETE /identity/v3/projects/ed128675a82f4cc9b01ab452db18e51b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:36.588727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be857c76-6414-44b1-b367-ab414e076391 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:36.590239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-be857c76-6414-44b1-b367-ab414e076391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:36.610771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-da204447-9422-4851-8726-9cbdd98e98a5 None None] MFA Rules not processed for user `77a6068e4dc34fb6b8d1447472692e55`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:36.612088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-da204447-9422-4851-8726-9cbdd98e98a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:36.674435 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1315/2629] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:36.685683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be857c76-6414-44b1-b367-ab414e076391 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:36.692367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a82eea-5102-4169-acb8-4fa18915117a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:36.693556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a82eea-5102-4169-acb8-4fa18915117a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:36.693942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a82eea-5102-4169-acb8-4fa18915117a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:36.694424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a82eea-5102-4169-acb8-4fa18915117a admin admin] PATH_INFO: `/v3/users/cb8be78218064ee0a9319fc65ee59a75` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:36.695836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a82eea-5102-4169-acb8-4fa18915117a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb8be78218064ee0a9319fc65ee59a75)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:36.698223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a82eea-5102-4169-acb8-4fa18915117a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:36.923797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:36.937101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a5a82eea-5102-4169-acb8-4fa18915117a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb8be78218064ee0a9319fc65ee59a75'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:36.937777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a5a82eea-5102-4169-acb8-4fa18915117a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb8be78218064ee0a9319fc65ee59a75'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:36.945398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a5a82eea-5102-4169-acb8-4fa18915117a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb8be78218064ee0a9319fc65ee59a75'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:37.048758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a5a82eea-5102-4169-acb8-4fa18915117a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb8be78218064ee0a9319fc65ee59a75'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:37.050159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a5a82eea-5102-4169-acb8-4fa18915117a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb8be78218064ee0a9319fc65ee59a75'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:37.076617 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1315/2630] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:05:36 2022] DELETE /identity/v3/users/cb8be78218064ee0a9319fc65ee59a75 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:05:37.571576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da204447-9422-4851-8726-9cbdd98e98a5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:37.582692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da204447-9422-4851-8726-9cbdd98e98a5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:37.666407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17db50-eaac-4799-9212-0571ddce0b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:37.667715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17db50-eaac-4799-9212-0571ddce0b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:37.669023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17db50-eaac-4799-9212-0571ddce0b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:37.670532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17db50-eaac-4799-9212-0571ddce0b18 admin admin] PATH_INFO: `/v3/projects/c4decf581404468ab8ca1873825b242e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:37.673036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17db50-eaac-4799-9212-0571ddce0b18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4decf581404468ab8ca1873825b242e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:37.680039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17db50-eaac-4799-9212-0571ddce0b18 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:37.830577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2e17db50-eaac-4799-9212-0571ddce0b18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4decf581404468ab8ca1873825b242e', 'request_id': 'req-2e17db50-eaac-4799-9212-0571ddce0b18'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:37.836664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:37.837907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2e17db50-eaac-4799-9212-0571ddce0b18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4decf581404468ab8ca1873825b242e', 'request_id': 'req-2e17db50-eaac-4799-9212-0571ddce0b18'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:37.923356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2e17db50-eaac-4799-9212-0571ddce0b18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4decf581404468ab8ca1873825b242e', 'request_id': 'req-2e17db50-eaac-4799-9212-0571ddce0b18'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:37.942104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2e17db50-eaac-4799-9212-0571ddce0b18 admin admin] The token cache is being invalidate because project c4decf581404468ab8ca1873825b242e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:05:37.942406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2e17db50-eaac-4799-9212-0571ddce0b18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:37.943206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2e17db50-eaac-4799-9212-0571ddce0b18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:37.948454 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1316/2631] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:05:37 2022] DELETE /identity/v3/projects/c4decf581404468ab8ca1873825b242e => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:37.950396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa642c5c-afc4-4900-84de-5107983515f0 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:37.950886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa642c5c-afc4-4900-84de-5107983515f0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:37.951243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa642c5c-afc4-4900-84de-5107983515f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:37.983572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-aa642c5c-afc4-4900-84de-5107983515f0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:37.985288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-aa642c5c-afc4-4900-84de-5107983515f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:38.103720 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1316/2632] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:05:38.118230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea8ed6-40e5-42be-9069-daa7768372c8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:38.119401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea8ed6-40e5-42be-9069-daa7768372c8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:38.119765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea8ed6-40e5-42be-9069-daa7768372c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:38.171066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d6ea8ed6-40e5-42be-9069-daa7768372c8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:38.172953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d6ea8ed6-40e5-42be-9069-daa7768372c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:38.188127 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1317/2633] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:38.193540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa642c5c-afc4-4900-84de-5107983515f0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:38.210218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa642c5c-afc4-4900-84de-5107983515f0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:38.210218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964819c9-305f-4cf4-8b2d-7092a7c7d4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:38.210724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964819c9-305f-4cf4-8b2d-7092a7c7d4cd admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:38.211035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964819c9-305f-4cf4-8b2d-7092a7c7d4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:38.211482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964819c9-305f-4cf4-8b2d-7092a7c7d4cd admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:38.212440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964819c9-305f-4cf4-8b2d-7092a7c7d4cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:05:38.212779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964819c9-305f-4cf4-8b2d-7092a7c7d4cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:38.214430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964819c9-305f-4cf4-8b2d-7092a7c7d4cd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:38.228022 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1317/2634] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:05:38.232460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea8ed6-40e5-42be-9069-daa7768372c8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:38.259958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea8ed6-40e5-42be-9069-daa7768372c8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:38.266755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f22af-757c-4967-93b2-ed655ad999f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:38.267751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f22af-757c-4967-93b2-ed655ad999f4 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:38.268060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f22af-757c-4967-93b2-ed655ad999f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:38.268482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f22af-757c-4967-93b2-ed655ad999f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:38.269882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f22af-757c-4967-93b2-ed655ad999f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:38.272362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f22af-757c-4967-93b2-ed655ad999f4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:38.394178 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1318/2635] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:05:38 2022] POST /identity/v3/projects => generated 391 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:05:38.394649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964819c9-305f-4cf4-8b2d-7092a7c7d4cd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:38.408747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964819c9-305f-4cf4-8b2d-7092a7c7d4cd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:38.470419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085583e-695f-41ad-97ed-ba15883997f1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:38.470820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085583e-695f-41ad-97ed-ba15883997f1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:38.471412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1085583e-695f-41ad-97ed-ba15883997f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:38.529397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1085583e-695f-41ad-97ed-ba15883997f1 None None] MFA Rules not processed for user `c14b965da64c40e382c69050ba01b183`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:38.530861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1085583e-695f-41ad-97ed-ba15883997f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:38.585448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08998e74-1e03-45e5-8fcc-af49b978bf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:38.587485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08998e74-1e03-45e5-8fcc-af49b978bf4e admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:38.591811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08998e74-1e03-45e5-8fcc-af49b978bf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:38.591811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08998e74-1e03-45e5-8fcc-af49b978bf4e admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:38.591811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08998e74-1e03-45e5-8fcc-af49b978bf4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:38.592451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08998e74-1e03-45e5-8fcc-af49b978bf4e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:38.704897 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1318/2636] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:05:38 2022] POST /identity/v3/users => generated 296 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:05:38.708881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08998e74-1e03-45e5-8fcc-af49b978bf4e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:38.711428 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1319/2637] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:38.749537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08998e74-1e03-45e5-8fcc-af49b978bf4e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:38.761348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5e0e4-024b-406d-8b70-ac13fbb089d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:38.763593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5e0e4-024b-406d-8b70-ac13fbb089d1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:38.764412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5e0e4-024b-406d-8b70-ac13fbb089d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:38.764712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d5e0e4-024b-406d-8b70-ac13fbb089d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:38.766227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5e0e4-024b-406d-8b70-ac13fbb089d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:38.768230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d5e0e4-024b-406d-8b70-ac13fbb089d1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:38.788700 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1319/2638] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:05:38.789010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085583e-695f-41ad-97ed-ba15883997f1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:38.811377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1085583e-695f-41ad-97ed-ba15883997f1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:38.823235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b05f49-dfc5-4102-8436-2bb7f54d60d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:38.828126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b05f49-dfc5-4102-8436-2bb7f54d60d8 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:38.828126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b05f49-dfc5-4102-8436-2bb7f54d60d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:38.828126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b05f49-dfc5-4102-8436-2bb7f54d60d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:38.828126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b05f49-dfc5-4102-8436-2bb7f54d60d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:38.828126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b05f49-dfc5-4102-8436-2bb7f54d60d8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:38.841267 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1320/2639] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:05:38.851584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5e0e4-024b-406d-8b70-ac13fbb089d1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:38.862924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d5e0e4-024b-406d-8b70-ac13fbb089d1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:38.868282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555e0a2-56da-4882-978f-84914f434804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:38.869533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555e0a2-56da-4882-978f-84914f434804 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:38.869954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555e0a2-56da-4882-978f-84914f434804 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:38.870282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e555e0a2-56da-4882-978f-84914f434804 admin admin] PATH_INFO: `/v3/projects/60bc9878b797401baef0263dbce27503/users/d7448caf7fcf424c93e0450e251829aa/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:38.882478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:38.886657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555e0a2-56da-4882-978f-84914f434804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60bc9878b797401baef0263dbce27503, user_id=d7448caf7fcf424c93e0450e251829aa, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:38.890326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e555e0a2-56da-4882-978f-84914f434804 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:38.993914 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1320/2640] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:05:38 2022] PUT /identity/v3/projects/60bc9878b797401baef0263dbce27503/users/d7448caf7fcf424c93e0450e251829aa/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:38.999455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972987c-8a0e-4376-a638-20c6427485b6 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:38.999803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972987c-8a0e-4376-a638-20c6427485b6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:39.000088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972987c-8a0e-4376-a638-20c6427485b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:39.029214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5972987c-8a0e-4376-a638-20c6427485b6 None None] MFA Rules not processed for user `d7448caf7fcf424c93e0450e251829aa`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:39.030483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5972987c-8a0e-4376-a638-20c6427485b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:39.124156 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1321/2641] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:39.473446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555e0a2-56da-4882-978f-84914f434804 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:39.475642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e555e0a2-56da-4882-978f-84914f434804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:39.583528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e555e0a2-56da-4882-978f-84914f434804 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:39.590303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381016df-1f1c-40c0-a905-95a6c4cc4231 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:39.596361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381016df-1f1c-40c0-a905-95a6c4cc4231 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:39.596361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381016df-1f1c-40c0-a905-95a6c4cc4231 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:39.596361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381016df-1f1c-40c0-a905-95a6c4cc4231 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:39.602733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381016df-1f1c-40c0-a905-95a6c4cc4231 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:39.604542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381016df-1f1c-40c0-a905-95a6c4cc4231 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:39.618006 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1321/2642] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:39.944984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972987c-8a0e-4376-a638-20c6427485b6 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:39.997514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:40.019060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972987c-8a0e-4376-a638-20c6427485b6 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:40.025401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926ae82-35ff-4002-a3d7-9c9f65614b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:40.026360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926ae82-35ff-4002-a3d7-9c9f65614b72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:40.027435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926ae82-35ff-4002-a3d7-9c9f65614b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:40.027716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926ae82-35ff-4002-a3d7-9c9f65614b72 admin admin] PATH_INFO: `/v3/users/d7448caf7fcf424c93e0450e251829aa` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:40.028910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926ae82-35ff-4002-a3d7-9c9f65614b72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7448caf7fcf424c93e0450e251829aa)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:40.032070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926ae82-35ff-4002-a3d7-9c9f65614b72 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:40.274559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e926ae82-35ff-4002-a3d7-9c9f65614b72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7448caf7fcf424c93e0450e251829aa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:40.281228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e926ae82-35ff-4002-a3d7-9c9f65614b72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7448caf7fcf424c93e0450e251829aa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:40.281924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e926ae82-35ff-4002-a3d7-9c9f65614b72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7448caf7fcf424c93e0450e251829aa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:40.282592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e926ae82-35ff-4002-a3d7-9c9f65614b72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7448caf7fcf424c93e0450e251829aa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:40.414822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e926ae82-35ff-4002-a3d7-9c9f65614b72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7448caf7fcf424c93e0450e251829aa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:40.433353 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1322/2643] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:05:39 2022] DELETE /identity/v3/users/d7448caf7fcf424c93e0450e251829aa => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:40.438526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4503c88-f655-475f-8bb0-b6849bd8a1cb None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:40.439020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4503c88-f655-475f-8bb0-b6849bd8a1cb None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:40.439124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4503c88-f655-475f-8bb0-b6849bd8a1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:40.458992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a4503c88-f655-475f-8bb0-b6849bd8a1cb None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:40.500658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a4503c88-f655-475f-8bb0-b6849bd8a1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:40.535067 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1322/2644] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:05:40.545731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926ae82-35ff-4002-a3d7-9c9f65614b72 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:40.547282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e926ae82-35ff-4002-a3d7-9c9f65614b72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:40.611812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926ae82-35ff-4002-a3d7-9c9f65614b72 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:40.616792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a745a-49e3-4f4e-844d-083ecf8e1cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:40.617813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a745a-49e3-4f4e-844d-083ecf8e1cdd service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:40.618283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a745a-49e3-4f4e-844d-083ecf8e1cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:40.619095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a745a-49e3-4f4e-844d-083ecf8e1cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:40.636811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a745a-49e3-4f4e-844d-083ecf8e1cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:40.638594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a745a-49e3-4f4e-844d-083ecf8e1cdd service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:40.650343 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1323/2645] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:41.141998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd81dd-9ca2-4996-a2a8-fe6b5a0f511a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:41.142496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd81dd-9ca2-4996-a2a8-fe6b5a0f511a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:41.142966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd81dd-9ca2-4996-a2a8-fe6b5a0f511a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:41.162627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6fcd81dd-9ca2-4996-a2a8-fe6b5a0f511a None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:41.163851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6fcd81dd-9ca2-4996-a2a8-fe6b5a0f511a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:41.214919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a745a-49e3-4f4e-844d-083ecf8e1cdd service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:41.216474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-206a745a-49e3-4f4e-844d-083ecf8e1cdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:41.241821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a745a-49e3-4f4e-844d-083ecf8e1cdd service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:41.252185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c1efdd-7552-4fae-880f-d26ac154ffd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:41.255009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1efdd-7552-4fae-880f-d26ac154ffd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:41.256873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1efdd-7552-4fae-880f-d26ac154ffd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:41.257365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c1efdd-7552-4fae-880f-d26ac154ffd5 admin admin] PATH_INFO: `/v3/projects/60bc9878b797401baef0263dbce27503` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:41.261010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1efdd-7552-4fae-880f-d26ac154ffd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60bc9878b797401baef0263dbce27503)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:41.263605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c1efdd-7552-4fae-880f-d26ac154ffd5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:41.278487 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1323/2646] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:05:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:41.291824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd81dd-9ca2-4996-a2a8-fe6b5a0f511a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:41.293442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6fcd81dd-9ca2-4996-a2a8-fe6b5a0f511a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:41.324925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:41.375236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd81dd-9ca2-4996-a2a8-fe6b5a0f511a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:41.382135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7747e-e832-47b6-a7ee-1fd8a7e97ffc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:41.382913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7747e-e832-47b6-a7ee-1fd8a7e97ffc service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:41.383367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7747e-e832-47b6-a7ee-1fd8a7e97ffc service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:41.383726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7747e-e832-47b6-a7ee-1fd8a7e97ffc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:41.395765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7747e-e832-47b6-a7ee-1fd8a7e97ffc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:41.398439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7747e-e832-47b6-a7ee-1fd8a7e97ffc service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:41.408313 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1324/2647] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:05:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:05:41.429316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-13c1efdd-7552-4fae-880f-d26ac154ffd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60bc9878b797401baef0263dbce27503', 'request_id': 'req-13c1efdd-7552-4fae-880f-d26ac154ffd5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:41.440759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:41.442574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-13c1efdd-7552-4fae-880f-d26ac154ffd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60bc9878b797401baef0263dbce27503', 'request_id': 'req-13c1efdd-7552-4fae-880f-d26ac154ffd5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:41.523597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-13c1efdd-7552-4fae-880f-d26ac154ffd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60bc9878b797401baef0263dbce27503', 'request_id': 'req-13c1efdd-7552-4fae-880f-d26ac154ffd5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:41.537765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-13c1efdd-7552-4fae-880f-d26ac154ffd5 admin admin] The token cache is being invalidate because project 60bc9878b797401baef0263dbce27503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:05:41.538109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-13c1efdd-7552-4fae-880f-d26ac154ffd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:41.538961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-13c1efdd-7552-4fae-880f-d26ac154ffd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:41.540860 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1324/2648] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:05:41 2022] DELETE /identity/v3/projects/60bc9878b797401baef0263dbce27503 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:05:41.545462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819dad23-c52c-4f8e-bdd1-79ff9b5f86c5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:41.551434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819dad23-c52c-4f8e-bdd1-79ff9b5f86c5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:41.551434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819dad23-c52c-4f8e-bdd1-79ff9b5f86c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:41.567250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-819dad23-c52c-4f8e-bdd1-79ff9b5f86c5 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:41.568490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-819dad23-c52c-4f8e-bdd1-79ff9b5f86c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:41.834034 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1325/2649] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:05:41.835889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea456b0-7d2e-4b45-bae9-bd0ab9c964e2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:41.838212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea456b0-7d2e-4b45-bae9-bd0ab9c964e2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:41.838899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea456b0-7d2e-4b45-bae9-bd0ab9c964e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:41.905391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cea456b0-7d2e-4b45-bae9-bd0ab9c964e2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:41.906755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cea456b0-7d2e-4b45-bae9-bd0ab9c964e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:41.923040 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1325/2650] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:41.927289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819dad23-c52c-4f8e-bdd1-79ff9b5f86c5 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:41.946602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819dad23-c52c-4f8e-bdd1-79ff9b5f86c5 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:41.955544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bef134-5373-443a-bfbf-458547500c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:41.969978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bef134-5373-443a-bfbf-458547500c6a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:41.975430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bef134-5373-443a-bfbf-458547500c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:41.975430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bef134-5373-443a-bfbf-458547500c6a admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:41.980145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bef134-5373-443a-bfbf-458547500c6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:05:41.980472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bef134-5373-443a-bfbf-458547500c6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:41.987558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bef134-5373-443a-bfbf-458547500c6a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:42.036196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea456b0-7d2e-4b45-bae9-bd0ab9c964e2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:42.045085 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1326/2651] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:05:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:05:42.058734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea456b0-7d2e-4b45-bae9-bd0ab9c964e2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:42.068241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f188c71-e730-417d-9009-c98e00a75ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:42.069160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188c71-e730-417d-9009-c98e00a75ffc admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:42.069671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188c71-e730-417d-9009-c98e00a75ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:42.073204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f188c71-e730-417d-9009-c98e00a75ffc admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:42.073204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f188c71-e730-417d-9009-c98e00a75ffc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:42.079919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f188c71-e730-417d-9009-c98e00a75ffc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:42.262528 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1326/2652] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:05:42 2022] POST /identity/v3/projects => generated 397 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:05:42.267495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bef134-5373-443a-bfbf-458547500c6a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:42.282792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bef134-5373-443a-bfbf-458547500c6a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:42.373341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bcf10-c259-490c-9d57-e91c36ea27a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:42.375285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bcf10-c259-490c-9d57-e91c36ea27a9 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:42.384723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bcf10-c259-490c-9d57-e91c36ea27a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:42.385243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bcf10-c259-490c-9d57-e91c36ea27a9 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:42.386144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bcf10-c259-490c-9d57-e91c36ea27a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:42.388323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bcf10-c259-490c-9d57-e91c36ea27a9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:42.492085 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1327/2653] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:05:42 2022] POST /identity/v3/users => generated 299 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:05:42.496184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f188c71-e730-417d-9009-c98e00a75ffc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:42.516697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f188c71-e730-417d-9009-c98e00a75ffc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:42.520258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf219a3-9e9e-4ebd-b763-d2f7ff7e9d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:42.521229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf219a3-9e9e-4ebd-b763-d2f7ff7e9d1f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:42.521726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf219a3-9e9e-4ebd-b763-d2f7ff7e9d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:42.522259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf219a3-9e9e-4ebd-b763-d2f7ff7e9d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:42.523617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf219a3-9e9e-4ebd-b763-d2f7ff7e9d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:42.526126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf219a3-9e9e-4ebd-b763-d2f7ff7e9d1f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:42.542700 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1327/2654] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:05:42.547958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bcf10-c259-490c-9d57-e91c36ea27a9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:42.559899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bcf10-c259-490c-9d57-e91c36ea27a9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:42.565042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad5820-7a38-4530-a85a-28eff3f9b397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:42.565780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad5820-7a38-4530-a85a-28eff3f9b397 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:42.566109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad5820-7a38-4530-a85a-28eff3f9b397 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:42.566418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad5820-7a38-4530-a85a-28eff3f9b397 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:42.567864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad5820-7a38-4530-a85a-28eff3f9b397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:42.570379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad5820-7a38-4530-a85a-28eff3f9b397 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:42.586704 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1328/2655] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:05:42.587211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf219a3-9e9e-4ebd-b763-d2f7ff7e9d1f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:42.601136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf219a3-9e9e-4ebd-b763-d2f7ff7e9d1f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:42.622524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04a216-d521-4bd2-a469-1195c0dce237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:42.623468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04a216-d521-4bd2-a469-1195c0dce237 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:42.623752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04a216-d521-4bd2-a469-1195c0dce237 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:42.624128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de04a216-d521-4bd2-a469-1195c0dce237 admin admin] PATH_INFO: `/v3/projects/ab0c1d6a9fc44de4af61822849b3bf8d/users/dc48535b672d4d1190ecfefe6cb02720/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:42.659460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04a216-d521-4bd2-a469-1195c0dce237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0c1d6a9fc44de4af61822849b3bf8d, user_id=dc48535b672d4d1190ecfefe6cb02720, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:42.663740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de04a216-d521-4bd2-a469-1195c0dce237 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:42.738464 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1328/2656] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:05:42 2022] PUT /identity/v3/projects/ab0c1d6a9fc44de4af61822849b3bf8d/users/dc48535b672d4d1190ecfefe6cb02720/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:42.744075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb4df3-7ee4-43a9-ad03-9520919d30ec None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:42.744963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb4df3-7ee4-43a9-ad03-9520919d30ec None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:42.745511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb4df3-7ee4-43a9-ad03-9520919d30ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:42.775942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-aacb4df3-7ee4-43a9-ad03-9520919d30ec None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:42.777400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-aacb4df3-7ee4-43a9-ad03-9520919d30ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:42.791931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:42.865448 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1329/2657] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:42.881702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04a216-d521-4bd2-a469-1195c0dce237 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:42.883681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-de04a216-d521-4bd2-a469-1195c0dce237 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:42.967960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de04a216-d521-4bd2-a469-1195c0dce237 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:42.972897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64932b-e42d-463c-a995-4c82be2a7060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:42.973955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64932b-e42d-463c-a995-4c82be2a7060 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:42.974318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64932b-e42d-463c-a995-4c82be2a7060 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:42.974800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64932b-e42d-463c-a995-4c82be2a7060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:42.988436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64932b-e42d-463c-a995-4c82be2a7060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:42.990318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64932b-e42d-463c-a995-4c82be2a7060 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:43.002897 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1329/2658] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:43.092305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82035dbb-dd01-4342-9021-3c1a63d79a3f None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:43.101617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82035dbb-dd01-4342-9021-3c1a63d79a3f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:43.101617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82035dbb-dd01-4342-9021-3c1a63d79a3f None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:43.101617 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1330/2659] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:05:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:05:43.101965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfa841-8186-4a45-bb8c-6687c22d7ca9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:43.101965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfa841-8186-4a45-bb8c-6687c22d7ca9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:43.102453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfa841-8186-4a45-bb8c-6687c22d7ca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:43.137633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2abfa841-8186-4a45-bb8c-6687c22d7ca9 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:43.140034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2abfa841-8186-4a45-bb8c-6687c22d7ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:43.272666 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1330/2660] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:05:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:43.287507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82035dbb-dd01-4342-9021-3c1a63d79a3f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:43.289736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-82035dbb-dd01-4342-9021-3c1a63d79a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:43.329979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82035dbb-dd01-4342-9021-3c1a63d79a3f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:43.349956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b1303-e404-44bd-9ca2-644e8f698006 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:43.350590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b1303-e404-44bd-9ca2-644e8f698006 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:43.350877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b1303-e404-44bd-9ca2-644e8f698006 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:43.351150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b1303-e404-44bd-9ca2-644e8f698006 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:43.366605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b1303-e404-44bd-9ca2-644e8f698006 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:43.368350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50b1303-e404-44bd-9ca2-644e8f698006 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:43.386181 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1331/2661] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:44.888686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf35c9d-5713-4a15-be76-594034ebda0a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:44.889187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf35c9d-5713-4a15-be76-594034ebda0a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:44.889701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf35c9d-5713-4a15-be76-594034ebda0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:44.940250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9bf35c9d-5713-4a15-be76-594034ebda0a None None] MFA Rules not processed for user `dc48535b672d4d1190ecfefe6cb02720`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:44.941761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9bf35c9d-5713-4a15-be76-594034ebda0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:44.963790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:45.096448 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1331/2662] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:45.097377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2cb4c-3b61-4c49-95a3-ed18dca879b0 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:45.098475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2cb4c-3b61-4c49-95a3-ed18dca879b0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:45.098771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2cb4c-3b61-4c49-95a3-ed18dca879b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:45.124207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-94d2cb4c-3b61-4c49-95a3-ed18dca879b0 None None] MFA Rules not processed for user `dc48535b672d4d1190ecfefe6cb02720`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:45.125922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-94d2cb4c-3b61-4c49-95a3-ed18dca879b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:45.135760 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1332/2663] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:45 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:05:45.152593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf35c9d-5713-4a15-be76-594034ebda0a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:45.154164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9bf35c9d-5713-4a15-be76-594034ebda0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:45.248810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf35c9d-5713-4a15-be76-594034ebda0a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:45.256326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d022a0-9536-4f4d-a4b0-1308946d6d13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:45.257005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d022a0-9536-4f4d-a4b0-1308946d6d13 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:45.257296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d022a0-9536-4f4d-a4b0-1308946d6d13 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:45.257709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d022a0-9536-4f4d-a4b0-1308946d6d13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:45.269063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d022a0-9536-4f4d-a4b0-1308946d6d13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:45.272549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d022a0-9536-4f4d-a4b0-1308946d6d13 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:45.284446 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1332/2664] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:54.907227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2cb4c-3b61-4c49-95a3-ed18dca879b0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:54.908277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-94d2cb4c-3b61-4c49-95a3-ed18dca879b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:54.924847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2cb4c-3b61-4c49-95a3-ed18dca879b0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:54.930335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4405f06-5060-42b2-8adb-0572bc857145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:54.931412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4405f06-5060-42b2-8adb-0572bc857145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:54.931749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4405f06-5060-42b2-8adb-0572bc857145 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:54.932073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4405f06-5060-42b2-8adb-0572bc857145 admin admin] PATH_INFO: `/v3/users/3b044baceddf49219485bd14aa914e35` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:54.933509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4405f06-5060-42b2-8adb-0572bc857145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b044baceddf49219485bd14aa914e35)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:54.936231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4405f06-5060-42b2-8adb-0572bc857145 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:55.317416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=57 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:55.694163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c4405f06-5060-42b2-8adb-0572bc857145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b044baceddf49219485bd14aa914e35'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:55.695755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c4405f06-5060-42b2-8adb-0572bc857145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b044baceddf49219485bd14aa914e35'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:55.707841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c4405f06-5060-42b2-8adb-0572bc857145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b044baceddf49219485bd14aa914e35'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:55.797247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c4405f06-5060-42b2-8adb-0572bc857145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b044baceddf49219485bd14aa914e35'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:55.797987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c4405f06-5060-42b2-8adb-0572bc857145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b044baceddf49219485bd14aa914e35'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:55.821275 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1333/2665] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:05:54 2022] DELETE /identity/v3/users/3b044baceddf49219485bd14aa914e35 => generated 0 bytes in 915 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:55.825564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14d994-bc25-47dd-a960-0d6c6021d524 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:55.826137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14d994-bc25-47dd-a960-0d6c6021d524 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:55.826491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14d994-bc25-47dd-a960-0d6c6021d524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:55.846575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-dc14d994-bc25-47dd-a960-0d6c6021d524 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:55.848632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dc14d994-bc25-47dd-a960-0d6c6021d524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:55.990535 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1333/2666] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:05:56.021902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4405f06-5060-42b2-8adb-0572bc857145 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:56.024098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c4405f06-5060-42b2-8adb-0572bc857145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:56.133502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4405f06-5060-42b2-8adb-0572bc857145 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:56.141032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78996c-cad2-4f46-a6cc-b291305d5be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:56.144305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78996c-cad2-4f46-a6cc-b291305d5be7 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:56.144305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78996c-cad2-4f46-a6cc-b291305d5be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:56.144305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa78996c-cad2-4f46-a6cc-b291305d5be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:56.156579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78996c-cad2-4f46-a6cc-b291305d5be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:56.159145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa78996c-cad2-4f46-a6cc-b291305d5be7 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:56.189763 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1334/2667] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:05:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:05:56.763264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14d994-bc25-47dd-a960-0d6c6021d524 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:56.765303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dc14d994-bc25-47dd-a960-0d6c6021d524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:56.795310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14d994-bc25-47dd-a960-0d6c6021d524 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:56.815340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:56.816563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:56.817871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:56.823995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c admin admin] PATH_INFO: `/v3/projects/8058694a4ecb467783677e687e16ecb6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:56.823995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8058694a4ecb467783677e687e16ecb6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:56.825458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:57.044894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8058694a4ecb467783677e687e16ecb6', 'request_id': 'req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:57.063989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8058694a4ecb467783677e687e16ecb6', 'request_id': 'req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:57.219570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8058694a4ecb467783677e687e16ecb6', 'request_id': 'req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:57.240452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c admin admin] The token cache is being invalidate because project 8058694a4ecb467783677e687e16ecb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:05:57.250088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:57.252095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:57.262169 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1334/2668] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:05:56 2022] DELETE /identity/v3/projects/8058694a4ecb467783677e687e16ecb6 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:57.266261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78996c-cad2-4f46-a6cc-b291305d5be7 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:57.281124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa78996c-cad2-4f46-a6cc-b291305d5be7 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:57.299247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:57.395195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb34363a-a627-4a14-adf6-729352af28fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:57.396229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34363a-a627-4a14-adf6-729352af28fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:57.397639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34363a-a627-4a14-adf6-729352af28fc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:57.398056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb34363a-a627-4a14-adf6-729352af28fc admin admin] PATH_INFO: `/v3/users/1edb01ef9d394c9da5086206d3033403` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:57.399695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34363a-a627-4a14-adf6-729352af28fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1edb01ef9d394c9da5086206d3033403)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:57.403527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb34363a-a627-4a14-adf6-729352af28fc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:57.593548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-eb34363a-a627-4a14-adf6-729352af28fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1edb01ef9d394c9da5086206d3033403'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:57.596580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-eb34363a-a627-4a14-adf6-729352af28fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1edb01ef9d394c9da5086206d3033403'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:57.601960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-eb34363a-a627-4a14-adf6-729352af28fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1edb01ef9d394c9da5086206d3033403'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:57.692019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-eb34363a-a627-4a14-adf6-729352af28fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1edb01ef9d394c9da5086206d3033403'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:57.693087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-eb34363a-a627-4a14-adf6-729352af28fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1edb01ef9d394c9da5086206d3033403'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:57.738862 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1335/2669] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:05:57 2022] DELETE /identity/v3/users/1edb01ef9d394c9da5086206d3033403 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:58.282013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:58.283791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:58.308070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53926c2a-e9ee-4c4e-af95-912dc83c7c5c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:58.485816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a8fbe-5ef6-454f-9d35-2fc15e95f0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:58.487339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a8fbe-5ef6-454f-9d35-2fc15e95f0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:58.487851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a8fbe-5ef6-454f-9d35-2fc15e95f0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:58.488314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a8fbe-5ef6-454f-9d35-2fc15e95f0a9 admin admin] PATH_INFO: `/v3/projects/7c6c7fd995bf4b41873c30015cbc85a4` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:58.490490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a8fbe-5ef6-454f-9d35-2fc15e95f0a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c6c7fd995bf4b41873c30015cbc85a4)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:58.495303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428a8fbe-5ef6-454f-9d35-2fc15e95f0a9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:58.644933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:58.713566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-428a8fbe-5ef6-454f-9d35-2fc15e95f0a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c6c7fd995bf4b41873c30015cbc85a4', 'request_id': 'req-428a8fbe-5ef6-454f-9d35-2fc15e95f0a9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:58.730749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-428a8fbe-5ef6-454f-9d35-2fc15e95f0a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c6c7fd995bf4b41873c30015cbc85a4', 'request_id': 'req-428a8fbe-5ef6-454f-9d35-2fc15e95f0a9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:58.814027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-428a8fbe-5ef6-454f-9d35-2fc15e95f0a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c6c7fd995bf4b41873c30015cbc85a4', 'request_id': 'req-428a8fbe-5ef6-454f-9d35-2fc15e95f0a9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:58.839765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-428a8fbe-5ef6-454f-9d35-2fc15e95f0a9 admin admin] The token cache is being invalidate because project 7c6c7fd995bf4b41873c30015cbc85a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:05:58.840119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-428a8fbe-5ef6-454f-9d35-2fc15e95f0a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:58.842025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-428a8fbe-5ef6-454f-9d35-2fc15e95f0a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:05:58.844354 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1335/2670] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:05:58 2022] DELETE /identity/v3/projects/7c6c7fd995bf4b41873c30015cbc85a4 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:58.851445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328167f-6db0-428a-8f57-8af991c909cc None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:58.851956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328167f-6db0-428a-8f57-8af991c909cc None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:58.852165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328167f-6db0-428a-8f57-8af991c909cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:58.886953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0328167f-6db0-428a-8f57-8af991c909cc None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:58.893627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0328167f-6db0-428a-8f57-8af991c909cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:59.014253 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1336/2671] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:59.016868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cb997-0ff9-4765-85ee-66d218d8983c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:59.017627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cb997-0ff9-4765-85ee-66d218d8983c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:59.017973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505cb997-0ff9-4765-85ee-66d218d8983c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:59.043248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-505cb997-0ff9-4765-85ee-66d218d8983c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:59.045060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-505cb997-0ff9-4765-85ee-66d218d8983c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:05:59.064479 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1336/2672] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:05:59.067623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328167f-6db0-428a-8f57-8af991c909cc None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:59.084895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328167f-6db0-428a-8f57-8af991c909cc None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:59.097139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dd527-731c-4c03-a4e9-1fe2577ce88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:59.099575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dd527-731c-4c03-a4e9-1fe2577ce88b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:59.101444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dd527-731c-4c03-a4e9-1fe2577ce88b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:59.101811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67dd527-731c-4c03-a4e9-1fe2577ce88b admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:59.103969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67dd527-731c-4c03-a4e9-1fe2577ce88b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:05:59.104645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67dd527-731c-4c03-a4e9-1fe2577ce88b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:59.110840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67dd527-731c-4c03-a4e9-1fe2577ce88b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:59.145032 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1337/2673] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:05:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:05:59.148478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cb997-0ff9-4765-85ee-66d218d8983c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:59.190014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505cb997-0ff9-4765-85ee-66d218d8983c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:59.204191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14947882-2a1c-48c4-bf1f-d5b505f2295c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:59.204894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14947882-2a1c-48c4-bf1f-d5b505f2295c admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:59.205246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14947882-2a1c-48c4-bf1f-d5b505f2295c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:59.205464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14947882-2a1c-48c4-bf1f-d5b505f2295c admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:59.207641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14947882-2a1c-48c4-bf1f-d5b505f2295c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:59.213330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14947882-2a1c-48c4-bf1f-d5b505f2295c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:59.316318 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1337/2674] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:05:59 2022] POST /identity/v3/projects => generated 383 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:05:59.320199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dd527-731c-4c03-a4e9-1fe2577ce88b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:59.336746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67dd527-731c-4c03-a4e9-1fe2577ce88b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:59.376711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:59.498519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799324a-c696-4a86-8e08-e78491a461a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:59.499370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799324a-c696-4a86-8e08-e78491a461a5 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:59.499683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799324a-c696-4a86-8e08-e78491a461a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:59.499959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799324a-c696-4a86-8e08-e78491a461a5 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:59.500635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799324a-c696-4a86-8e08-e78491a461a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:59.502635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799324a-c696-4a86-8e08-e78491a461a5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:59.622029 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1338/2675] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:05:59 2022] POST /identity/v3/users => generated 292 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:05:59.626197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14947882-2a1c-48c4-bf1f-d5b505f2295c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:59.643098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14947882-2a1c-48c4-bf1f-d5b505f2295c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:59.657784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83795b71-4941-43d4-a039-5c025e722e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:59.658761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83795b71-4941-43d4-a039-5c025e722e20 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:59.658761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83795b71-4941-43d4-a039-5c025e722e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:59.659034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83795b71-4941-43d4-a039-5c025e722e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:59.659817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83795b71-4941-43d4-a039-5c025e722e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:59.661444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83795b71-4941-43d4-a039-5c025e722e20 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:59.688339 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1338/2676] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:05:59.692692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799324a-c696-4a86-8e08-e78491a461a5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:59.707662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799324a-c696-4a86-8e08-e78491a461a5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:59.725152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d003d8a-7457-497b-b910-e5bb0fafee79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:59.725381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d003d8a-7457-497b-b910-e5bb0fafee79 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:59.725578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d003d8a-7457-497b-b910-e5bb0fafee79 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:59.725825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d003d8a-7457-497b-b910-e5bb0fafee79 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:59.727142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d003d8a-7457-497b-b910-e5bb0fafee79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:59.728797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d003d8a-7457-497b-b910-e5bb0fafee79 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:59.754340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83795b71-4941-43d4-a039-5c025e722e20 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:05:59.755466 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1339/2677] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:05:59.767377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83795b71-4941-43d4-a039-5c025e722e20 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:05:59.782917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed4260-f886-4297-a119-4e9114e5a158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:05:59.783835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed4260-f886-4297-a119-4e9114e5a158 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:59.784127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed4260-f886-4297-a119-4e9114e5a158 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:59.784412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed4260-f886-4297-a119-4e9114e5a158 admin admin] PATH_INFO: `/v3/projects/348fbbc628b44152b05362657c11230d/users/5e54b51913be4a2bab74aae347785727/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:59.808888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed4260-f886-4297-a119-4e9114e5a158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=348fbbc628b44152b05362657c11230d, user_id=5e54b51913be4a2bab74aae347785727, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:05:59.811820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed4260-f886-4297-a119-4e9114e5a158 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:05:59.897552 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1339/2678] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:05:59 2022] PUT /identity/v3/projects/348fbbc628b44152b05362657c11230d/users/5e54b51913be4a2bab74aae347785727/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:05:59.908276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027051c-3254-4a6c-87a2-48a56eba7c7e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:05:59.908838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027051c-3254-4a6c-87a2-48a56eba7c7e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:05:59.908838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0027051c-3254-4a6c-87a2-48a56eba7c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:05:59.991360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0027051c-3254-4a6c-87a2-48a56eba7c7e None None] MFA Rules not processed for user `5e54b51913be4a2bab74aae347785727`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:05:59.992669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0027051c-3254-4a6c-87a2-48a56eba7c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:00.163663 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1340/2679] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:05:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:00.168857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed4260-f886-4297-a119-4e9114e5a158 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:00.295093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a4e1-23c7-48cd-ab53-d62e24ea20fd None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:00.295458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a4e1-23c7-48cd-ab53-d62e24ea20fd None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:00.295655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9087a4e1-23c7-48cd-ab53-d62e24ea20fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:00.317332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed4260-f886-4297-a119-4e9114e5a158 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:00.330715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46410f-bb7d-43d4-bd91-cc3d08f499b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:00.331398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46410f-bb7d-43d4-bd91-cc3d08f499b3 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:00.331774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46410f-bb7d-43d4-bd91-cc3d08f499b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:00.332114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46410f-bb7d-43d4-bd91-cc3d08f499b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:00.332817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46410f-bb7d-43d4-bd91-cc3d08f499b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:00.334807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b46410f-bb7d-43d4-bd91-cc3d08f499b3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:00.339981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9087a4e1-23c7-48cd-ab53-d62e24ea20fd None None] MFA Rules not processed for user `7b44b524592e4a57a26681767b9c4541`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:00.342470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9087a4e1-23c7-48cd-ab53-d62e24ea20fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:00.426178 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1341/2680] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:00.485615 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1340/2681] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:06:00 2022] POST /identity/v3/projects => generated 381 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:06:00.492444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087a4e1-23c7-48cd-ab53-d62e24ea20fd None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:00.506448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9087a4e1-23c7-48cd-ab53-d62e24ea20fd None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:00.584113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:00.604585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972f90e-c5c3-4790-8367-38085ed8db43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:00.606667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972f90e-c5c3-4790-8367-38085ed8db43 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:00.607213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972f90e-c5c3-4790-8367-38085ed8db43 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:00.607564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972f90e-c5c3-4790-8367-38085ed8db43 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:00.608484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1972f90e-c5c3-4790-8367-38085ed8db43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:00.610570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1972f90e-c5c3-4790-8367-38085ed8db43 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:00.717885 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1342/2682] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:06:00 2022] POST /identity/v3/users => generated 297 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:06:00.719459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46410f-bb7d-43d4-bd91-cc3d08f499b3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:00.733828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46410f-bb7d-43d4-bd91-cc3d08f499b3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:00.740210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a8bcc-b838-4912-8d4d-b3efe0226008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:00.746026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a8bcc-b838-4912-8d4d-b3efe0226008 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:00.746026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a8bcc-b838-4912-8d4d-b3efe0226008 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:00.746026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a8bcc-b838-4912-8d4d-b3efe0226008 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:00.746026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a8bcc-b838-4912-8d4d-b3efe0226008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:00.746026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a8bcc-b838-4912-8d4d-b3efe0226008 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:00.762437 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1341/2683] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:06:00.764418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972f90e-c5c3-4790-8367-38085ed8db43 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:00.779948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972f90e-c5c3-4790-8367-38085ed8db43 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:00.788378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b6eb2-f258-4e0c-b0a4-224a6da0dd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:00.789712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b6eb2-f258-4e0c-b0a4-224a6da0dd47 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:00.790060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b6eb2-f258-4e0c-b0a4-224a6da0dd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:00.790340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55b6eb2-f258-4e0c-b0a4-224a6da0dd47 admin admin] PATH_INFO: `/v3/projects/b83e29ca4af045138d1fbeff233f5e05/users/2807965e53b04a3086ef1b00ffb481af/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:00.819306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b6eb2-f258-4e0c-b0a4-224a6da0dd47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b83e29ca4af045138d1fbeff233f5e05, user_id=2807965e53b04a3086ef1b00ffb481af, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:00.822249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55b6eb2-f258-4e0c-b0a4-224a6da0dd47 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:00.913794 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1343/2684] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:06:00 2022] PUT /identity/v3/projects/b83e29ca4af045138d1fbeff233f5e05/users/2807965e53b04a3086ef1b00ffb481af/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:00.917443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a8bcc-b838-4912-8d4d-b3efe0226008 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:00.932186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a8bcc-b838-4912-8d4d-b3efe0226008 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:00.976604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:01.026184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b8d202-b6fb-44cf-83f1-5bfc6dbe28fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:01.026954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8d202-b6fb-44cf-83f1-5bfc6dbe28fd admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:01.027315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8d202-b6fb-44cf-83f1-5bfc6dbe28fd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:01.027732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b8d202-b6fb-44cf-83f1-5bfc6dbe28fd admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:01.028576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8d202-b6fb-44cf-83f1-5bfc6dbe28fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:01.033320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b8d202-b6fb-44cf-83f1-5bfc6dbe28fd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:01.047236 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1342/2685] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:00 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:06:01.050981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b6eb2-f258-4e0c-b0a4-224a6da0dd47 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:01.065196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55b6eb2-f258-4e0c-b0a4-224a6da0dd47 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:01.078038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f46a9-b173-4293-bb73-1ab7df595dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:01.079491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f46a9-b173-4293-bb73-1ab7df595dbd admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:01.079967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f46a9-b173-4293-bb73-1ab7df595dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:01.080460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f46a9-b173-4293-bb73-1ab7df595dbd admin admin] PATH_INFO: `/v3/projects/b83e29ca4af045138d1fbeff233f5e05/users/2807965e53b04a3086ef1b00ffb481af/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:01.086090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f46a9-b173-4293-bb73-1ab7df595dbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b83e29ca4af045138d1fbeff233f5e05, user_id=2807965e53b04a3086ef1b00ffb481af, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:01.094363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f46a9-b173-4293-bb73-1ab7df595dbd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:01.124281 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1344/2686] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:06:01 2022] PUT /identity/v3/projects/b83e29ca4af045138d1fbeff233f5e05/users/2807965e53b04a3086ef1b00ffb481af/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:06:01.135236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039f04c-5fde-4abd-8339-0947d27c2842 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:01.135819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039f04c-5fde-4abd-8339-0947d27c2842 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:01.136324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039f04c-5fde-4abd-8339-0947d27c2842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:01.199163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7039f04c-5fde-4abd-8339-0947d27c2842 None None] MFA Rules not processed for user `2807965e53b04a3086ef1b00ffb481af`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:01.201729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7039f04c-5fde-4abd-8339-0947d27c2842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:01.285788 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1343/2687] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:01.302767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c1e4e-4c99-4ace-9c4d-b5ada291589e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:01.303062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c1e4e-4c99-4ace-9c4d-b5ada291589e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:01.303369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c1e4e-4c99-4ace-9c4d-b5ada291589e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:01.325853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0a2c1e4e-4c99-4ace-9c4d-b5ada291589e None None] MFA Rules not processed for user `5e54b51913be4a2bab74aae347785727`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:01.330523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0a2c1e4e-4c99-4ace-9c4d-b5ada291589e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:01.444536 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1345/2688] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:01.652793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039f04c-5fde-4abd-8339-0947d27c2842 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:01.654409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7039f04c-5fde-4abd-8339-0947d27c2842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:01.758045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039f04c-5fde-4abd-8339-0947d27c2842 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:01.758045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b372d9f-4241-416f-84cd-824ec5d1511b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:01.758045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b372d9f-4241-416f-84cd-824ec5d1511b service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:01.758045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b372d9f-4241-416f-84cd-824ec5d1511b service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:01.759313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b372d9f-4241-416f-84cd-824ec5d1511b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:01.768151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b372d9f-4241-416f-84cd-824ec5d1511b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:01.769922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b372d9f-4241-416f-84cd-824ec5d1511b service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:01.779369 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1344/2689] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:06:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:06:10.189233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6d690-4836-40eb-844c-6600c4a209f0 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:10.190215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6d690-4836-40eb-844c-6600c4a209f0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:10.191204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6d690-4836-40eb-844c-6600c4a209f0 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:10.193078 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1346/2690] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:06:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:06:10.198687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f28a8-0eec-482a-a0fe-4f2a95c961ce None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:10.199228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f28a8-0eec-482a-a0fe-4f2a95c961ce None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:10.199330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f28a8-0eec-482a-a0fe-4f2a95c961ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:10.225029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-919f28a8-0eec-482a-a0fe-4f2a95c961ce None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:10.232278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-919f28a8-0eec-482a-a0fe-4f2a95c961ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:10.276923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:10.311808 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1345/2691] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:06:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:10.335349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6d690-4836-40eb-844c-6600c4a209f0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:10.337188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-dbe6d690-4836-40eb-844c-6600c4a209f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:10.357889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6d690-4836-40eb-844c-6600c4a209f0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:10.366378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c4236-b1e1-4a9b-bfcc-ec4ff1516317 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:10.367533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c4236-b1e1-4a9b-bfcc-ec4ff1516317 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:10.368026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c4236-b1e1-4a9b-bfcc-ec4ff1516317 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:10.368881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c4236-b1e1-4a9b-bfcc-ec4ff1516317 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:10.379467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c4236-b1e1-4a9b-bfcc-ec4ff1516317 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:10.384010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c4236-b1e1-4a9b-bfcc-ec4ff1516317 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:10.413987 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1347/2692] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:06:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:06:19.548726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f28a8-0eec-482a-a0fe-4f2a95c961ce None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:19.550432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-919f28a8-0eec-482a-a0fe-4f2a95c961ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:19.627218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f28a8-0eec-482a-a0fe-4f2a95c961ce None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:19.632603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911e33f-d791-4734-aa05-b8520436ebf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:19.633625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911e33f-d791-4734-aa05-b8520436ebf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:19.634190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911e33f-d791-4734-aa05-b8520436ebf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:19.634650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7911e33f-d791-4734-aa05-b8520436ebf4 admin admin] PATH_INFO: `/v3/users/cdeaa660f8ec465bba68096d0ed4b1ae` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:19.636183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911e33f-d791-4734-aa05-b8520436ebf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdeaa660f8ec465bba68096d0ed4b1ae)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:19.638186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7911e33f-d791-4734-aa05-b8520436ebf4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:19.833787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7911e33f-d791-4734-aa05-b8520436ebf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdeaa660f8ec465bba68096d0ed4b1ae'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:19.841048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7911e33f-d791-4734-aa05-b8520436ebf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdeaa660f8ec465bba68096d0ed4b1ae'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:19.841968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7911e33f-d791-4734-aa05-b8520436ebf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdeaa660f8ec465bba68096d0ed4b1ae'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:19.842584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7911e33f-d791-4734-aa05-b8520436ebf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdeaa660f8ec465bba68096d0ed4b1ae'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:19.932358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7911e33f-d791-4734-aa05-b8520436ebf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdeaa660f8ec465bba68096d0ed4b1ae'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:19.955022 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1346/2693] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:06:19 2022] DELETE /identity/v3/users/cdeaa660f8ec465bba68096d0ed4b1ae => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:20.394997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=61,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:20.442885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c4236-b1e1-4a9b-bfcc-ec4ff1516317 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:20.498634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c4236-b1e1-4a9b-bfcc-ec4ff1516317 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:20.613655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34429f71-98a9-4db8-8442-b6e2b91db4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:20.615040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34429f71-98a9-4db8-8442-b6e2b91db4f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:20.615513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34429f71-98a9-4db8-8442-b6e2b91db4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:20.615950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34429f71-98a9-4db8-8442-b6e2b91db4f5 admin admin] PATH_INFO: `/v3/projects/0e60b122ef584254b06fd8258c987f8d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:20.617550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34429f71-98a9-4db8-8442-b6e2b91db4f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e60b122ef584254b06fd8258c987f8d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:20.620153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34429f71-98a9-4db8-8442-b6e2b91db4f5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:20.746097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-34429f71-98a9-4db8-8442-b6e2b91db4f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e60b122ef584254b06fd8258c987f8d', 'request_id': 'req-34429f71-98a9-4db8-8442-b6e2b91db4f5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:20.755231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-34429f71-98a9-4db8-8442-b6e2b91db4f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e60b122ef584254b06fd8258c987f8d', 'request_id': 'req-34429f71-98a9-4db8-8442-b6e2b91db4f5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:20.814658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-34429f71-98a9-4db8-8442-b6e2b91db4f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e60b122ef584254b06fd8258c987f8d', 'request_id': 'req-34429f71-98a9-4db8-8442-b6e2b91db4f5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:20.839026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-34429f71-98a9-4db8-8442-b6e2b91db4f5 admin admin] The token cache is being invalidate because project 0e60b122ef584254b06fd8258c987f8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:06:20.839026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-34429f71-98a9-4db8-8442-b6e2b91db4f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:20.842803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-34429f71-98a9-4db8-8442-b6e2b91db4f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:20.848164 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1348/2694] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:06:20 2022] DELETE /identity/v3/projects/0e60b122ef584254b06fd8258c987f8d => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:20.850874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911e33f-d791-4734-aa05-b8520436ebf4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:20.853399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7911e33f-d791-4734-aa05-b8520436ebf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:20.877647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7911e33f-d791-4734-aa05-b8520436ebf4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:20.978324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:20.995764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82817e21-824d-401a-8da9-e9da51e8dbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:21.000298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82817e21-824d-401a-8da9-e9da51e8dbbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:21.000298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82817e21-824d-401a-8da9-e9da51e8dbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:21.000298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82817e21-824d-401a-8da9-e9da51e8dbbd admin admin] PATH_INFO: `/v3/users/16fedb3759d94951a633d58d8f91abd3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:21.000298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82817e21-824d-401a-8da9-e9da51e8dbbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=16fedb3759d94951a633d58d8f91abd3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:21.006349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82817e21-824d-401a-8da9-e9da51e8dbbd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:21.379682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-82817e21-824d-401a-8da9-e9da51e8dbbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16fedb3759d94951a633d58d8f91abd3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:21.388860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-82817e21-824d-401a-8da9-e9da51e8dbbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16fedb3759d94951a633d58d8f91abd3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:21.389797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-82817e21-824d-401a-8da9-e9da51e8dbbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16fedb3759d94951a633d58d8f91abd3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:21.390459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-82817e21-824d-401a-8da9-e9da51e8dbbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16fedb3759d94951a633d58d8f91abd3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:21.416689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6edc57-2c9f-4e8e-b34a-c3e87f442fdc None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:21.417224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6edc57-2c9f-4e8e-b34a-c3e87f442fdc None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:21.417224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6edc57-2c9f-4e8e-b34a-c3e87f442fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:21.452496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5b6edc57-2c9f-4e8e-b34a-c3e87f442fdc None None] MFA Rules not processed for user `dc48535b672d4d1190ecfefe6cb02720`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:21.453767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5b6edc57-2c9f-4e8e-b34a-c3e87f442fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:21.471031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-82817e21-824d-401a-8da9-e9da51e8dbbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16fedb3759d94951a633d58d8f91abd3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:21.508030 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1347/2695] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:06:20 2022] DELETE /identity/v3/users/16fedb3759d94951a633d58d8f91abd3 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:21.539682 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1349/2696] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:21 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:21.969016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82817e21-824d-401a-8da9-e9da51e8dbbd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:21.970610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-82817e21-824d-401a-8da9-e9da51e8dbbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:22.039040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82817e21-824d-401a-8da9-e9da51e8dbbd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:22.043973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f89561-b84d-4272-afda-6295754aef03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:22.045000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f89561-b84d-4272-afda-6295754aef03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:22.045357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f89561-b84d-4272-afda-6295754aef03 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:22.045734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f89561-b84d-4272-afda-6295754aef03 admin admin] PATH_INFO: `/v3/projects/db5eec07b6534c968b59a4b21cb75975` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:22.047701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f89561-b84d-4272-afda-6295754aef03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db5eec07b6534c968b59a4b21cb75975)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:22.049349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f89561-b84d-4272-afda-6295754aef03 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:22.164018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b3f89561-b84d-4272-afda-6295754aef03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db5eec07b6534c968b59a4b21cb75975', 'request_id': 'req-b3f89561-b84d-4272-afda-6295754aef03'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:22.170301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b3f89561-b84d-4272-afda-6295754aef03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db5eec07b6534c968b59a4b21cb75975', 'request_id': 'req-b3f89561-b84d-4272-afda-6295754aef03'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:22.224793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b3f89561-b84d-4272-afda-6295754aef03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db5eec07b6534c968b59a4b21cb75975', 'request_id': 'req-b3f89561-b84d-4272-afda-6295754aef03'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:22.227189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:22.238641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b3f89561-b84d-4272-afda-6295754aef03 admin admin] The token cache is being invalidate because project db5eec07b6534c968b59a4b21cb75975 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:06:22.238942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b3f89561-b84d-4272-afda-6295754aef03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:22.239618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b3f89561-b84d-4272-afda-6295754aef03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:22.241452 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1348/2697] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:06:21 2022] DELETE /identity/v3/projects/db5eec07b6534c968b59a4b21cb75975 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:22.246846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc162c5-6bdf-4d9c-9160-0e61488aca6f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:22.250649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc162c5-6bdf-4d9c-9160-0e61488aca6f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:22.250649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc162c5-6bdf-4d9c-9160-0e61488aca6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:22.277380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-fbc162c5-6bdf-4d9c-9160-0e61488aca6f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:22.278971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fbc162c5-6bdf-4d9c-9160-0e61488aca6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:22.312173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:22.341157 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1350/2698] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:22.349639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6fdc2-42f8-4191-961f-348f5de797a3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:22.354942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6fdc2-42f8-4191-961f-348f5de797a3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:22.354942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6fdc2-42f8-4191-961f-348f5de797a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:22.380402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2ae6fdc2-42f8-4191-961f-348f5de797a3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:22.382165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2ae6fdc2-42f8-4191-961f-348f5de797a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:22.394899 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1349/2699] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:22.396098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc162c5-6bdf-4d9c-9160-0e61488aca6f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:22.407662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc162c5-6bdf-4d9c-9160-0e61488aca6f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:22.413147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f8852-f19a-4dd8-9b14-6c6591c8f9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:22.416958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f8852-f19a-4dd8-9b14-6c6591c8f9af admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:22.416958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f8852-f19a-4dd8-9b14-6c6591c8f9af admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:22.416958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1f8852-f19a-4dd8-9b14-6c6591c8f9af admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:22.416958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f8852-f19a-4dd8-9b14-6c6591c8f9af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:06:22.416958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f8852-f19a-4dd8-9b14-6c6591c8f9af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:22.417882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1f8852-f19a-4dd8-9b14-6c6591c8f9af admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:22.430104 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1351/2700] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:06:22.434407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6fdc2-42f8-4191-961f-348f5de797a3 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:22.444677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6fdc2-42f8-4191-961f-348f5de797a3 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:22.449081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ea820-be8b-45f9-b4de-e3685325594e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:22.457444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ea820-be8b-45f9-b4de-e3685325594e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:22.457800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ea820-be8b-45f9-b4de-e3685325594e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:22.466250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ea820-be8b-45f9-b4de-e3685325594e admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:22.466250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ea820-be8b-45f9-b4de-e3685325594e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:22.466250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24ea820-be8b-45f9-b4de-e3685325594e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:22.538753 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1350/2701] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:06:22 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:06:22.543138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f8852-f19a-4dd8-9b14-6c6591c8f9af admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:22.553791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1f8852-f19a-4dd8-9b14-6c6591c8f9af admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:22.611801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd7d37-0d66-4f6b-97f1-7a1f4acb0ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:22.612480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd7d37-0d66-4f6b-97f1-7a1f4acb0ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:22.613306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd7d37-0d66-4f6b-97f1-7a1f4acb0ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:22.613584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd7d37-0d66-4f6b-97f1-7a1f4acb0ea9 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:22.614322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd7d37-0d66-4f6b-97f1-7a1f4acb0ea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:22.616249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd7d37-0d66-4f6b-97f1-7a1f4acb0ea9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:22.706118 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1352/2702] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:06:22 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:06:22.715633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ea820-be8b-45f9-b4de-e3685325594e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:22.730886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ea820-be8b-45f9-b4de-e3685325594e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:22.738845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff427d6-bf36-4b06-b71b-7ceb7ee8de6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:22.748983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff427d6-bf36-4b06-b71b-7ceb7ee8de6b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:22.748983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff427d6-bf36-4b06-b71b-7ceb7ee8de6b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:22.748983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff427d6-bf36-4b06-b71b-7ceb7ee8de6b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:22.748983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff427d6-bf36-4b06-b71b-7ceb7ee8de6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:22.748983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff427d6-bf36-4b06-b71b-7ceb7ee8de6b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:22.760256 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1351/2703] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:06:22.762545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd7d37-0d66-4f6b-97f1-7a1f4acb0ea9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:22.776538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd7d37-0d66-4f6b-97f1-7a1f4acb0ea9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:22.785456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6f5b0-a51e-49ba-9dcd-4ff1cc99b33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:22.786759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6f5b0-a51e-49ba-9dcd-4ff1cc99b33d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:22.787421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6f5b0-a51e-49ba-9dcd-4ff1cc99b33d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:22.787860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6f5b0-a51e-49ba-9dcd-4ff1cc99b33d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:22.789003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6f5b0-a51e-49ba-9dcd-4ff1cc99b33d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:22.791004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6f5b0-a51e-49ba-9dcd-4ff1cc99b33d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:22.830933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff427d6-bf36-4b06-b71b-7ceb7ee8de6b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:22.842561 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1353/2704] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:06:22.843142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff427d6-bf36-4b06-b71b-7ceb7ee8de6b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:22.852658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811f84c-a0a8-42b3-87e4-8dd0c185f2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:22.854208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811f84c-a0a8-42b3-87e4-8dd0c185f2ec admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:22.854507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811f84c-a0a8-42b3-87e4-8dd0c185f2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:22.854794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a811f84c-a0a8-42b3-87e4-8dd0c185f2ec admin admin] PATH_INFO: `/v3/projects/b6bd508b87e84bbea09ac7a1042eda02/users/ebe3922bb60c40a58cc5ae376ce9d220/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:22.893615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811f84c-a0a8-42b3-87e4-8dd0c185f2ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6bd508b87e84bbea09ac7a1042eda02, user_id=ebe3922bb60c40a58cc5ae376ce9d220, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:22.900615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a811f84c-a0a8-42b3-87e4-8dd0c185f2ec admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:22.996312 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1352/2705] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:06:22 2022] PUT /identity/v3/projects/b6bd508b87e84bbea09ac7a1042eda02/users/ebe3922bb60c40a58cc5ae376ce9d220/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:23.006892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa64131-f7f7-4a43-b1c6-390312555e26 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:23.007202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa64131-f7f7-4a43-b1c6-390312555e26 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:23.007493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa64131-f7f7-4a43-b1c6-390312555e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:23.036609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2fa64131-f7f7-4a43-b1c6-390312555e26 None None] MFA Rules not processed for user `ebe3922bb60c40a58cc5ae376ce9d220`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:23.044941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2fa64131-f7f7-4a43-b1c6-390312555e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:23.115298 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1354/2706] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:23.121116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811f84c-a0a8-42b3-87e4-8dd0c185f2ec admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:23.210280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a811f84c-a0a8-42b3-87e4-8dd0c185f2ec admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:23.215117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d094c-169e-4265-a940-55e92d5e69cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:23.215776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d094c-169e-4265-a940-55e92d5e69cf admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:23.216061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d094c-169e-4265-a940-55e92d5e69cf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:23.216340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d094c-169e-4265-a940-55e92d5e69cf admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:23.217031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d094c-169e-4265-a940-55e92d5e69cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:23.218977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d094c-169e-4265-a940-55e92d5e69cf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:23.353253 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1353/2707] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:06:23 2022] POST /identity/v3/projects => generated 377 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:06:23.357509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa64131-f7f7-4a43-b1c6-390312555e26 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:23.369103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa64131-f7f7-4a43-b1c6-390312555e26 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:23.414908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:23.451639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bde022-2ebd-422e-a021-9a32774ecba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:23.452335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bde022-2ebd-422e-a021-9a32774ecba6 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:23.452620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bde022-2ebd-422e-a021-9a32774ecba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:23.452898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bde022-2ebd-422e-a021-9a32774ecba6 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:23.453609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bde022-2ebd-422e-a021-9a32774ecba6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:23.457240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bde022-2ebd-422e-a021-9a32774ecba6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:23.574227 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1355/2708] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:06:23 2022] POST /identity/v3/users => generated 295 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:06:23.578202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d094c-169e-4265-a940-55e92d5e69cf admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:23.589199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d094c-169e-4265-a940-55e92d5e69cf admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:23.594913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58c3d0-bdcb-43bc-9d90-8ad159f9edfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:23.596356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58c3d0-bdcb-43bc-9d90-8ad159f9edfd admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:23.597006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58c3d0-bdcb-43bc-9d90-8ad159f9edfd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:23.597300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58c3d0-bdcb-43bc-9d90-8ad159f9edfd admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:23.598108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d58c3d0-bdcb-43bc-9d90-8ad159f9edfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:23.599757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d58c3d0-bdcb-43bc-9d90-8ad159f9edfd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:23.612089 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1354/2709] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:06:23.616463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bde022-2ebd-422e-a021-9a32774ecba6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:23.626991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bde022-2ebd-422e-a021-9a32774ecba6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:23.632243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4913ba-4bb5-47c1-9854-ec1f674dd18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:23.633133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4913ba-4bb5-47c1-9854-ec1f674dd18d admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:23.633398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4913ba-4bb5-47c1-9854-ec1f674dd18d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:23.635380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4913ba-4bb5-47c1-9854-ec1f674dd18d admin admin] PATH_INFO: `/v3/projects/3207da51cff4421d9f1098e14f982a27/users/ca45b8d64d6b492694409fa327b2fe24/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:23.650118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4913ba-4bb5-47c1-9854-ec1f674dd18d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3207da51cff4421d9f1098e14f982a27, user_id=ca45b8d64d6b492694409fa327b2fe24, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:23.652955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4913ba-4bb5-47c1-9854-ec1f674dd18d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:23.739760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58c3d0-bdcb-43bc-9d90-8ad159f9edfd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:23.744332 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1356/2710] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:06:23 2022] PUT /identity/v3/projects/3207da51cff4421d9f1098e14f982a27/users/ca45b8d64d6b492694409fa327b2fe24/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:23.764946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58c3d0-bdcb-43bc-9d90-8ad159f9edfd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:23.875850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea999d-8370-4702-9e75-52a02ee22ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:23.877575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea999d-8370-4702-9e75-52a02ee22ddb admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:23.878302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea999d-8370-4702-9e75-52a02ee22ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:23.879256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea999d-8370-4702-9e75-52a02ee22ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:23.881233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea999d-8370-4702-9e75-52a02ee22ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:23.883237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea999d-8370-4702-9e75-52a02ee22ddb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:23.924280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a515ef0-a453-4974-8e6a-e4750f74ff51 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:23.924602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a515ef0-a453-4974-8e6a-e4750f74ff51 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:23.924886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a515ef0-a453-4974-8e6a-e4750f74ff51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:23.942727 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1355/2711] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:06:23.951578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9a515ef0-a453-4974-8e6a-e4750f74ff51 None None] MFA Rules not processed for user `fc6ee6dba5d24e90bff48384539fcd7f`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:23.952067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea999d-8370-4702-9e75-52a02ee22ddb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:23.952818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9a515ef0-a453-4974-8e6a-e4750f74ff51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:23.958893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:23.966185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea999d-8370-4702-9e75-52a02ee22ddb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:23.996108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f3bef-26c6-448c-a218-fcc5cc0ad204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:23.997086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f3bef-26c6-448c-a218-fcc5cc0ad204 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:23.997681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f3bef-26c6-448c-a218-fcc5cc0ad204 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:23.997787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f3bef-26c6-448c-a218-fcc5cc0ad204 admin admin] PATH_INFO: `/v3/projects/3207da51cff4421d9f1098e14f982a27/users/ca45b8d64d6b492694409fa327b2fe24/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:24.000293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f3bef-26c6-448c-a218-fcc5cc0ad204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3207da51cff4421d9f1098e14f982a27, user_id=ca45b8d64d6b492694409fa327b2fe24, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:24.003121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f3bef-26c6-448c-a218-fcc5cc0ad204 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:24.017666 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1356/2712] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:06:23 2022] PUT /identity/v3/projects/3207da51cff4421d9f1098e14f982a27/users/ca45b8d64d6b492694409fa327b2fe24/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:24.029390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6de6a1-80bb-428f-889b-1c5e19d2e941 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:24.029673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6de6a1-80bb-428f-889b-1c5e19d2e941 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:24.030039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6de6a1-80bb-428f-889b-1c5e19d2e941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:24.044636 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1357/2713] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:06:24.071119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3b6de6a1-80bb-428f-889b-1c5e19d2e941 None None] MFA Rules not processed for user `ca45b8d64d6b492694409fa327b2fe24`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:24.072418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3b6de6a1-80bb-428f-889b-1c5e19d2e941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:24.167952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a515ef0-a453-4974-8e6a-e4750f74ff51 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:24.169498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9a515ef0-a453-4974-8e6a-e4750f74ff51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:24.194144 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1357/2714] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:06:24.207269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6de6a1-80bb-428f-889b-1c5e19d2e941 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:24.219550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3b6de6a1-80bb-428f-889b-1c5e19d2e941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:24.295406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a515ef0-a453-4974-8e6a-e4750f74ff51 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:24.304370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f4a68-e109-45e6-8409-91ffee9f3cea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:24.304861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f4a68-e109-45e6-8409-91ffee9f3cea service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:24.305142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f4a68-e109-45e6-8409-91ffee9f3cea service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:24.306269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f4a68-e109-45e6-8409-91ffee9f3cea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:24.307851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-994f4a68-e109-45e6-8409-91ffee9f3cea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:24.315655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6de6a1-80bb-428f-889b-1c5e19d2e941 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:24.323760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0810d9f-9b28-43c7-a641-242b7946cae4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:24.324241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0810d9f-9b28-43c7-a641-242b7946cae4 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:24.326673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0810d9f-9b28-43c7-a641-242b7946cae4 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:24.326673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0810d9f-9b28-43c7-a641-242b7946cae4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:24.336485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0810d9f-9b28-43c7-a641-242b7946cae4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:24.338611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0810d9f-9b28-43c7-a641-242b7946cae4 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:24.352076 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1358/2715] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:06:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:06:24.361091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:24.382204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f4a68-e109-45e6-8409-91ffee9f3cea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:24.384447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f4a68-e109-45e6-8409-91ffee9f3cea service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:24.399575 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1358/2716] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:06:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:06:25.036689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f42cc-f85a-4756-947e-a1d05434b4d9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:25.037830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f42cc-f85a-4756-947e-a1d05434b4d9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:25.038624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f42cc-f85a-4756-947e-a1d05434b4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:25.071279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c30f42cc-f85a-4756-947e-a1d05434b4d9 None None] MFA Rules not processed for user `5e54b51913be4a2bab74aae347785727`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:25.076885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c30f42cc-f85a-4756-947e-a1d05434b4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:25.198485 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1359/2717] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:27.525048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f4a68-e109-45e6-8409-91ffee9f3cea service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:27.538801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f4a68-e109-45e6-8409-91ffee9f3cea service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:27.547797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fce8a-e758-4979-b99e-4c4459418109 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:27.548503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fce8a-e758-4979-b99e-4c4459418109 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:27.548796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fce8a-e758-4979-b99e-4c4459418109 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:27.549103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fce8a-e758-4979-b99e-4c4459418109 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:27.625817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3fce8a-e758-4979-b99e-4c4459418109 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:27.627931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3fce8a-e758-4979-b99e-4c4459418109 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:27.667278 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1359/2718] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:06:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:06:29.913564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f42cc-f85a-4756-947e-a1d05434b4d9 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:29.919309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:29.980040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f42cc-f85a-4756-947e-a1d05434b4d9 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:29.984674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345b1bc-b796-4f74-8cc3-57aa0883549b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:29.985545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345b1bc-b796-4f74-8cc3-57aa0883549b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:29.985852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345b1bc-b796-4f74-8cc3-57aa0883549b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:29.986137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6345b1bc-b796-4f74-8cc3-57aa0883549b admin admin] PATH_INFO: `/v3/users/ebe3922bb60c40a58cc5ae376ce9d220` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:29.987351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345b1bc-b796-4f74-8cc3-57aa0883549b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebe3922bb60c40a58cc5ae376ce9d220)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:29.990008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6345b1bc-b796-4f74-8cc3-57aa0883549b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:30.183833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6345b1bc-b796-4f74-8cc3-57aa0883549b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebe3922bb60c40a58cc5ae376ce9d220'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:30.192867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6345b1bc-b796-4f74-8cc3-57aa0883549b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebe3922bb60c40a58cc5ae376ce9d220'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:30.193963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6345b1bc-b796-4f74-8cc3-57aa0883549b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebe3922bb60c40a58cc5ae376ce9d220'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:30.195615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6345b1bc-b796-4f74-8cc3-57aa0883549b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebe3922bb60c40a58cc5ae376ce9d220'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:30.316813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6345b1bc-b796-4f74-8cc3-57aa0883549b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebe3922bb60c40a58cc5ae376ce9d220'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:30.368356 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1360/2719] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:06:29 2022] DELETE /identity/v3/users/ebe3922bb60c40a58cc5ae376ce9d220 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:30.372898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bdb38-955d-4dd5-b64a-5e3f4c7073e0 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:30.373373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bdb38-955d-4dd5-b64a-5e3f4c7073e0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:30.373467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bdb38-955d-4dd5-b64a-5e3f4c7073e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:30.414713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f07bdb38-955d-4dd5-b64a-5e3f4c7073e0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:30.416313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f07bdb38-955d-4dd5-b64a-5e3f4c7073e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:30.516176 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1360/2720] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:30.526755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345b1bc-b796-4f74-8cc3-57aa0883549b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:30.528500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6345b1bc-b796-4f74-8cc3-57aa0883549b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:30.604838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6345b1bc-b796-4f74-8cc3-57aa0883549b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:30.609489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3efc710-d846-48d9-a686-95b86aa96078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:30.610184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3efc710-d846-48d9-a686-95b86aa96078 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:30.610499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3efc710-d846-48d9-a686-95b86aa96078 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:30.610774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3efc710-d846-48d9-a686-95b86aa96078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:30.656589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3efc710-d846-48d9-a686-95b86aa96078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:30.658318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3efc710-d846-48d9-a686-95b86aa96078 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:30.672138 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1361/2721] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:06:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:06:31.415875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7ff91-7dce-4e1c-a86f-98a73b380873 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:31.416418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7ff91-7dce-4e1c-a86f-98a73b380873 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:31.416418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7ff91-7dce-4e1c-a86f-98a73b380873 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:31.417828 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1361/2722] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:06:31.423417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e6c85-e680-4563-878a-04f4f312d6a0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:31.423905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e6c85-e680-4563-878a-04f4f312d6a0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:31.424027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e6c85-e680-4563-878a-04f4f312d6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:31.453330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-276e6c85-e680-4563-878a-04f4f312d6a0 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:31.454622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-276e6c85-e680-4563-878a-04f4f312d6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:31.485005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:31.526089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7ff91-7dce-4e1c-a86f-98a73b380873 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:31.528602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-61b7ff91-7dce-4e1c-a86f-98a73b380873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:31.546813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7ff91-7dce-4e1c-a86f-98a73b380873 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:31.552779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c076953-a04e-4a7f-b48d-e49665dcd34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:31.554099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c076953-a04e-4a7f-b48d-e49665dcd34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:31.554783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c076953-a04e-4a7f-b48d-e49665dcd34b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:31.555216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c076953-a04e-4a7f-b48d-e49665dcd34b admin admin] PATH_INFO: `/v3/projects/b6bd508b87e84bbea09ac7a1042eda02` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:31.557964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c076953-a04e-4a7f-b48d-e49665dcd34b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6bd508b87e84bbea09ac7a1042eda02)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:31.560211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c076953-a04e-4a7f-b48d-e49665dcd34b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:31.581036 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1362/2723] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:06:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:31.621727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e6c85-e680-4563-878a-04f4f312d6a0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:31.623686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-276e6c85-e680-4563-878a-04f4f312d6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:31.649929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e6c85-e680-4563-878a-04f4f312d6a0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:31.666650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eba379-190f-4ef7-abdc-981a5c6caada service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:31.668317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eba379-190f-4ef7-abdc-981a5c6caada service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:31.671595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eba379-190f-4ef7-abdc-981a5c6caada service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:31.671960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eba379-190f-4ef7-abdc-981a5c6caada service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:31.702751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eba379-190f-4ef7-abdc-981a5c6caada service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:31.708406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eba379-190f-4ef7-abdc-981a5c6caada service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:31.728406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5c076953-a04e-4a7f-b48d-e49665dcd34b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6bd508b87e84bbea09ac7a1042eda02', 'request_id': 'req-5c076953-a04e-4a7f-b48d-e49665dcd34b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:31.736426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5c076953-a04e-4a7f-b48d-e49665dcd34b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6bd508b87e84bbea09ac7a1042eda02', 'request_id': 'req-5c076953-a04e-4a7f-b48d-e49665dcd34b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:31.750717 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1363/2724] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:06:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:06:31.790698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5c076953-a04e-4a7f-b48d-e49665dcd34b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6bd508b87e84bbea09ac7a1042eda02', 'request_id': 'req-5c076953-a04e-4a7f-b48d-e49665dcd34b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:31.794414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:31.818720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5c076953-a04e-4a7f-b48d-e49665dcd34b admin admin] The token cache is being invalidate because project b6bd508b87e84bbea09ac7a1042eda02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:06:31.820302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5c076953-a04e-4a7f-b48d-e49665dcd34b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:31.821479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5c076953-a04e-4a7f-b48d-e49665dcd34b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:31.827544 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1362/2725] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:06:31 2022] DELETE /identity/v3/projects/b6bd508b87e84bbea09ac7a1042eda02 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:31.831598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eba379-190f-4ef7-abdc-981a5c6caada service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:31.845302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eba379-190f-4ef7-abdc-981a5c6caada service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:31.937876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2ffa9-e700-4b53-8c63-e31a23f536d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:31.939596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2ffa9-e700-4b53-8c63-e31a23f536d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:31.939596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2ffa9-e700-4b53-8c63-e31a23f536d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:31.939596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2ffa9-e700-4b53-8c63-e31a23f536d3 admin admin] PATH_INFO: `/v3/users/ca45b8d64d6b492694409fa327b2fe24` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:31.940256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2ffa9-e700-4b53-8c63-e31a23f536d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca45b8d64d6b492694409fa327b2fe24)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:31.942948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2ffa9-e700-4b53-8c63-e31a23f536d3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:32.162423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6bd2ffa9-e700-4b53-8c63-e31a23f536d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca45b8d64d6b492694409fa327b2fe24'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:32.171590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6bd2ffa9-e700-4b53-8c63-e31a23f536d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca45b8d64d6b492694409fa327b2fe24'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:32.176815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6bd2ffa9-e700-4b53-8c63-e31a23f536d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca45b8d64d6b492694409fa327b2fe24'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:32.176815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6bd2ffa9-e700-4b53-8c63-e31a23f536d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca45b8d64d6b492694409fa327b2fe24'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:32.287656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6bd2ffa9-e700-4b53-8c63-e31a23f536d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca45b8d64d6b492694409fa327b2fe24'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:32.331892 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1364/2726] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:06:31 2022] DELETE /identity/v3/users/ca45b8d64d6b492694409fa327b2fe24 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:33.498880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c076953-a04e-4a7f-b48d-e49665dcd34b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:33.500420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5c076953-a04e-4a7f-b48d-e49665dcd34b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:33.527193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c076953-a04e-4a7f-b48d-e49665dcd34b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:33.656764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c2819-5e10-4dd6-ae5c-c18d1bf4bb41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:33.659581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c2819-5e10-4dd6-ae5c-c18d1bf4bb41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:33.660798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c2819-5e10-4dd6-ae5c-c18d1bf4bb41 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:33.661982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c2819-5e10-4dd6-ae5c-c18d1bf4bb41 admin admin] PATH_INFO: `/v3/projects/3207da51cff4421d9f1098e14f982a27` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:33.665004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c2819-5e10-4dd6-ae5c-c18d1bf4bb41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3207da51cff4421d9f1098e14f982a27)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:33.667187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c2819-5e10-4dd6-ae5c-c18d1bf4bb41 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:33.858722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6f3c2819-5e10-4dd6-ae5c-c18d1bf4bb41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3207da51cff4421d9f1098e14f982a27', 'request_id': 'req-6f3c2819-5e10-4dd6-ae5c-c18d1bf4bb41'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:33.863020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6f3c2819-5e10-4dd6-ae5c-c18d1bf4bb41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3207da51cff4421d9f1098e14f982a27', 'request_id': 'req-6f3c2819-5e10-4dd6-ae5c-c18d1bf4bb41'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:33.980267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6f3c2819-5e10-4dd6-ae5c-c18d1bf4bb41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3207da51cff4421d9f1098e14f982a27', 'request_id': 'req-6f3c2819-5e10-4dd6-ae5c-c18d1bf4bb41'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:34.004408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6f3c2819-5e10-4dd6-ae5c-c18d1bf4bb41 admin admin] The token cache is being invalidate because project 3207da51cff4421d9f1098e14f982a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:06:34.004704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6f3c2819-5e10-4dd6-ae5c-c18d1bf4bb41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:34.005968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6f3c2819-5e10-4dd6-ae5c-c18d1bf4bb41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:34.009465 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1363/2727] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:06:33 2022] DELETE /identity/v3/projects/3207da51cff4421d9f1098e14f982a27 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:34.014184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211aa43-6b54-4b18-be42-695a719f6474 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:34.015110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211aa43-6b54-4b18-be42-695a719f6474 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:34.015545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0211aa43-6b54-4b18-be42-695a719f6474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:34.056611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0211aa43-6b54-4b18-be42-695a719f6474 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:34.058290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0211aa43-6b54-4b18-be42-695a719f6474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:34.099193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:34.149039 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1365/2728] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:34.157336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be2bd1-6524-47ec-b187-4545413a2cd7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:34.157648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be2bd1-6524-47ec-b187-4545413a2cd7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:34.157915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be2bd1-6524-47ec-b187-4545413a2cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:34.195933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-66be2bd1-6524-47ec-b187-4545413a2cd7 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:34.197905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-66be2bd1-6524-47ec-b187-4545413a2cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:34.226673 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1364/2729] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:34.230877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211aa43-6b54-4b18-be42-695a719f6474 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:34.253625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0211aa43-6b54-4b18-be42-695a719f6474 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:34.286116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bf203-8cda-4d97-85dd-0dfb1774aed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:34.290287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bf203-8cda-4d97-85dd-0dfb1774aed0 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:34.290769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bf203-8cda-4d97-85dd-0dfb1774aed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:34.292145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878bf203-8cda-4d97-85dd-0dfb1774aed0 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:34.295472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bf203-8cda-4d97-85dd-0dfb1774aed0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:06:34.296562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bf203-8cda-4d97-85dd-0dfb1774aed0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:34.303837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878bf203-8cda-4d97-85dd-0dfb1774aed0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:34.334967 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1366/2730] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:06:34.339167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be2bd1-6524-47ec-b187-4545413a2cd7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:34.373975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be2bd1-6524-47ec-b187-4545413a2cd7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:34.406337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ba427-e569-4660-8385-5b15a40efb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:34.407619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ba427-e569-4660-8385-5b15a40efb35 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:34.408062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ba427-e569-4660-8385-5b15a40efb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:34.409303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ba427-e569-4660-8385-5b15a40efb35 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:34.411889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9ba427-e569-4660-8385-5b15a40efb35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:34.417288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9ba427-e569-4660-8385-5b15a40efb35 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:34.585402 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1365/2731] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:06:34 2022] POST /identity/v3/projects => generated 401 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:06:34.589115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bf203-8cda-4d97-85dd-0dfb1774aed0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:34.604750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878bf203-8cda-4d97-85dd-0dfb1774aed0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:34.748876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc8809-59e4-4504-bf67-fa0cb74248ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:34.750158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc8809-59e4-4504-bf67-fa0cb74248ba admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:34.750610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc8809-59e4-4504-bf67-fa0cb74248ba admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:34.751339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc8809-59e4-4504-bf67-fa0cb74248ba admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:34.752748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc8809-59e4-4504-bf67-fa0cb74248ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:34.758474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc8809-59e4-4504-bf67-fa0cb74248ba admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:34.905328 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1367/2732] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:06:34 2022] POST /identity/v3/users => generated 301 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:06:34.906616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ba427-e569-4660-8385-5b15a40efb35 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:34.919287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ba427-e569-4660-8385-5b15a40efb35 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:34.925720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1f354-1dc4-4f9a-8912-bb82b923f938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:34.926481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1f354-1dc4-4f9a-8912-bb82b923f938 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:34.926777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1f354-1dc4-4f9a-8912-bb82b923f938 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:34.927054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f1f354-1dc4-4f9a-8912-bb82b923f938 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:34.927801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1f354-1dc4-4f9a-8912-bb82b923f938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:34.929446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f1f354-1dc4-4f9a-8912-bb82b923f938 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:34.953156 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1366/2733] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:06:34.955274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc8809-59e4-4504-bf67-fa0cb74248ba admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:34.976547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc8809-59e4-4504-bf67-fa0cb74248ba admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:34.987019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec266b6-4747-45d6-8ff8-f731e5249ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:34.987787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec266b6-4747-45d6-8ff8-f731e5249ada admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:34.988077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec266b6-4747-45d6-8ff8-f731e5249ada admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:34.988361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec266b6-4747-45d6-8ff8-f731e5249ada admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:34.989155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec266b6-4747-45d6-8ff8-f731e5249ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:34.991061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec266b6-4747-45d6-8ff8-f731e5249ada admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:35.012788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1f354-1dc4-4f9a-8912-bb82b923f938 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:35.024649 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1368/2734] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:06:35.029754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f1f354-1dc4-4f9a-8912-bb82b923f938 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:35.038948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f5353-4335-4b40-956b-e9f64bdb282b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:35.040812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f5353-4335-4b40-956b-e9f64bdb282b admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:35.041352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f5353-4335-4b40-956b-e9f64bdb282b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:35.041825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f5353-4335-4b40-956b-e9f64bdb282b admin admin] PATH_INFO: `/v3/projects/86bbd17227e84055827b4eff8bdac369/users/8910a38cbd6048f4a9ced613a0cea707/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:35.107951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f5353-4335-4b40-956b-e9f64bdb282b admin admin] RBAC: Authorizing `identity:create_grant(project_id=86bbd17227e84055827b4eff8bdac369, user_id=8910a38cbd6048f4a9ced613a0cea707, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:35.110963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f5353-4335-4b40-956b-e9f64bdb282b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:35.244957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0376ba1-4e5b-4de1-98bb-37c2068d9209 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:35.245251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0376ba1-4e5b-4de1-98bb-37c2068d9209 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:35.245537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0376ba1-4e5b-4de1-98bb-37c2068d9209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:35.251480 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1367/2735] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:06:35 2022] PUT /identity/v3/projects/86bbd17227e84055827b4eff8bdac369/users/8910a38cbd6048f4a9ced613a0cea707/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:35.282671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d0376ba1-4e5b-4de1-98bb-37c2068d9209 None None] MFA Rules not processed for user `8910a38cbd6048f4a9ced613a0cea707`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:35.284244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d0376ba1-4e5b-4de1-98bb-37c2068d9209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:35.377900 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1369/2736] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:35 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:35.382486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f5353-4335-4b40-956b-e9f64bdb282b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:35.431496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:35.523636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f5353-4335-4b40-956b-e9f64bdb282b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:35.530360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e62b8fb-168c-4535-85ee-1bf28bf907c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:35.531394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e62b8fb-168c-4535-85ee-1bf28bf907c4 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:35.533221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e62b8fb-168c-4535-85ee-1bf28bf907c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:35.533757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e62b8fb-168c-4535-85ee-1bf28bf907c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:35.535424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e62b8fb-168c-4535-85ee-1bf28bf907c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:35.537520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e62b8fb-168c-4535-85ee-1bf28bf907c4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:35.638207 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1368/2737] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:06:35 2022] POST /identity/v3/projects => generated 401 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:06:35.639753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0376ba1-4e5b-4de1-98bb-37c2068d9209 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:35.657114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0376ba1-4e5b-4de1-98bb-37c2068d9209 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:35.688217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:35.769064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c184b1f-f993-4567-bb2a-5b90d8c36183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:35.769859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c184b1f-f993-4567-bb2a-5b90d8c36183 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:35.770347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c184b1f-f993-4567-bb2a-5b90d8c36183 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:35.770727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c184b1f-f993-4567-bb2a-5b90d8c36183 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:35.771582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c184b1f-f993-4567-bb2a-5b90d8c36183 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:35.773965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c184b1f-f993-4567-bb2a-5b90d8c36183 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:35.917333 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1370/2738] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:06:35 2022] POST /identity/v3/users => generated 307 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:06:35.921024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e62b8fb-168c-4535-85ee-1bf28bf907c4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:35.934275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e62b8fb-168c-4535-85ee-1bf28bf907c4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:35.939607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a259c-2c76-401d-9435-cb3c97a61aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:35.945264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a259c-2c76-401d-9435-cb3c97a61aeb admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:35.945264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a259c-2c76-401d-9435-cb3c97a61aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:35.945264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a259c-2c76-401d-9435-cb3c97a61aeb admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:35.945264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a259c-2c76-401d-9435-cb3c97a61aeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:35.945264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a259c-2c76-401d-9435-cb3c97a61aeb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:35.968315 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1369/2739] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:06:35.969490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c184b1f-f993-4567-bb2a-5b90d8c36183 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:35.991135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c184b1f-f993-4567-bb2a-5b90d8c36183 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:36.003510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307f26b-3b94-46ab-a828-42ca2d071fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:36.003510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307f26b-3b94-46ab-a828-42ca2d071fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:36.003510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307f26b-3b94-46ab-a828-42ca2d071fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:36.003800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307f26b-3b94-46ab-a828-42ca2d071fc3 admin admin] PATH_INFO: `/v3/projects/77d7e5834b944f059157c4cf0e8a4017/users/0cf7dccf6c4442c58dbdca800b88fcfa/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:36.028638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307f26b-3b94-46ab-a828-42ca2d071fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d7e5834b944f059157c4cf0e8a4017, user_id=0cf7dccf6c4442c58dbdca800b88fcfa, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:36.032400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307f26b-3b94-46ab-a828-42ca2d071fc3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:36.144628 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1371/2740] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:06:35 2022] PUT /identity/v3/projects/77d7e5834b944f059157c4cf0e8a4017/users/0cf7dccf6c4442c58dbdca800b88fcfa/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:36.146012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a259c-2c76-401d-9435-cb3c97a61aeb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:36.160261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a259c-2c76-401d-9435-cb3c97a61aeb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:36.277032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffd052-159f-40ec-8867-2d17e29a0d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:36.277707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffd052-159f-40ec-8867-2d17e29a0d3a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:36.278022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffd052-159f-40ec-8867-2d17e29a0d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:36.278481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffd052-159f-40ec-8867-2d17e29a0d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:36.279444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffd052-159f-40ec-8867-2d17e29a0d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:36.281143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffd052-159f-40ec-8867-2d17e29a0d3a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:36.300228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307f26b-3b94-46ab-a828-42ca2d071fc3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:36.310812 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1370/2741] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:06:36.313686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307f26b-3b94-46ab-a828-42ca2d071fc3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:36.319832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19ad81-1264-49b1-aece-c10e81ff124b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:36.321606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19ad81-1264-49b1-aece-c10e81ff124b admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:36.327476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19ad81-1264-49b1-aece-c10e81ff124b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:36.327476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d19ad81-1264-49b1-aece-c10e81ff124b admin admin] PATH_INFO: `/v3/projects/77d7e5834b944f059157c4cf0e8a4017/users/0cf7dccf6c4442c58dbdca800b88fcfa/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:36.335207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19ad81-1264-49b1-aece-c10e81ff124b admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d7e5834b944f059157c4cf0e8a4017, user_id=0cf7dccf6c4442c58dbdca800b88fcfa, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:36.337236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d19ad81-1264-49b1-aece-c10e81ff124b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:36.366690 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1372/2742] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:06:36 2022] PUT /identity/v3/projects/77d7e5834b944f059157c4cf0e8a4017/users/0cf7dccf6c4442c58dbdca800b88fcfa/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:36.368865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b858181-4167-4fde-9d3e-8dfda75aca4b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:36.369384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b858181-4167-4fde-9d3e-8dfda75aca4b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:36.369958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b858181-4167-4fde-9d3e-8dfda75aca4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:36.407439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5b858181-4167-4fde-9d3e-8dfda75aca4b None None] MFA Rules not processed for user `0cf7dccf6c4442c58dbdca800b88fcfa`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:36.408664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5b858181-4167-4fde-9d3e-8dfda75aca4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:36.498397 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1371/2743] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:36 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:36.527211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19ad81-1264-49b1-aece-c10e81ff124b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:36.530609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1d19ad81-1264-49b1-aece-c10e81ff124b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:36.637957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d19ad81-1264-49b1-aece-c10e81ff124b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:36.643997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a5901-0149-4e72-8cd7-3922d8bf963b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:36.644659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a5901-0149-4e72-8cd7-3922d8bf963b service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:36.644946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a5901-0149-4e72-8cd7-3922d8bf963b service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:36.645219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a5901-0149-4e72-8cd7-3922d8bf963b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:36.657780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a5901-0149-4e72-8cd7-3922d8bf963b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:36.659674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a5901-0149-4e72-8cd7-3922d8bf963b service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:36.670603 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1373/2744] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:06:36 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:06:36.810156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b858181-4167-4fde-9d3e-8dfda75aca4b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:36.832468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b858181-4167-4fde-9d3e-8dfda75aca4b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:36.839342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50277f4-e262-4787-a1d6-5ce0c903161c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:36.841349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50277f4-e262-4787-a1d6-5ce0c903161c service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:36.842244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50277f4-e262-4787-a1d6-5ce0c903161c service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:36.843055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50277f4-e262-4787-a1d6-5ce0c903161c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:36.865069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:36.924372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50277f4-e262-4787-a1d6-5ce0c903161c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:36.929708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50277f4-e262-4787-a1d6-5ce0c903161c service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:36.962516 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1372/2745] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:06:36 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:06:37.797882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a5901-0149-4e72-8cd7-3922d8bf963b service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:37.998879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a5901-0149-4e72-8cd7-3922d8bf963b service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:38.030844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8f445-a30f-41bc-bbc4-edb4c7660822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:38.031757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8f445-a30f-41bc-bbc4-edb4c7660822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:38.033452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8f445-a30f-41bc-bbc4-edb4c7660822 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:38.034250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8f445-a30f-41bc-bbc4-edb4c7660822 admin admin] PATH_INFO: `/v3/users/8910a38cbd6048f4a9ced613a0cea707` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:38.040268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8f445-a30f-41bc-bbc4-edb4c7660822 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8910a38cbd6048f4a9ced613a0cea707)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:38.047663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8f445-a30f-41bc-bbc4-edb4c7660822 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:38.056984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:38.296530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f7a8f445-a30f-41bc-bbc4-edb4c7660822 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8910a38cbd6048f4a9ced613a0cea707'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:38.308158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f7a8f445-a30f-41bc-bbc4-edb4c7660822 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8910a38cbd6048f4a9ced613a0cea707'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:38.309446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f7a8f445-a30f-41bc-bbc4-edb4c7660822 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8910a38cbd6048f4a9ced613a0cea707'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:38.310239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f7a8f445-a30f-41bc-bbc4-edb4c7660822 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8910a38cbd6048f4a9ced613a0cea707'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:38.562445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f7a8f445-a30f-41bc-bbc4-edb4c7660822 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8910a38cbd6048f4a9ced613a0cea707'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:38.598843 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1374/2746] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:06:37 2022] DELETE /identity/v3/users/8910a38cbd6048f4a9ced613a0cea707 => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:38.603027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582abc8-4a87-45cb-b7f3-9315d2fb495e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:38.603521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582abc8-4a87-45cb-b7f3-9315d2fb495e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:38.603614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582abc8-4a87-45cb-b7f3-9315d2fb495e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:38.631741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c582abc8-4a87-45cb-b7f3-9315d2fb495e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:38.633239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c582abc8-4a87-45cb-b7f3-9315d2fb495e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:38.717035 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1373/2747] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:06:38.729169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8f445-a30f-41bc-bbc4-edb4c7660822 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:38.732358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f7a8f445-a30f-41bc-bbc4-edb4c7660822 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:38.802871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8f445-a30f-41bc-bbc4-edb4c7660822 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:38.808361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f54523-139d-439f-ba83-282f8d4e8d8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:38.809253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f54523-139d-439f-ba83-282f8d4e8d8a service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:38.809542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f54523-139d-439f-ba83-282f8d4e8d8a service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:38.809827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f54523-139d-439f-ba83-282f8d4e8d8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:38.820591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f54523-139d-439f-ba83-282f8d4e8d8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:38.822110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f54523-139d-439f-ba83-282f8d4e8d8a service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:38.834809 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1375/2748] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:06:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:06:39.423681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582abc8-4a87-45cb-b7f3-9315d2fb495e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:39.426663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c582abc8-4a87-45cb-b7f3-9315d2fb495e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:39.452838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582abc8-4a87-45cb-b7f3-9315d2fb495e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:39.459208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:39.460250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:39.460598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:39.460948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e admin admin] PATH_INFO: `/v3/projects/86bbd17227e84055827b4eff8bdac369` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:39.462556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=86bbd17227e84055827b4eff8bdac369)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:39.464596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:39.604670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86bbd17227e84055827b4eff8bdac369', 'request_id': 'req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:39.619844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86bbd17227e84055827b4eff8bdac369', 'request_id': 'req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:39.716078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86bbd17227e84055827b4eff8bdac369', 'request_id': 'req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:39.733280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e admin admin] The token cache is being invalidate because project 86bbd17227e84055827b4eff8bdac369 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:06:39.734405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:39.743017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:39.746624 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1374/2749] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:06:39 2022] DELETE /identity/v3/projects/86bbd17227e84055827b4eff8bdac369 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:39.750962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f54523-139d-439f-ba83-282f8d4e8d8a service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:39.763617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f54523-139d-439f-ba83-282f8d4e8d8a service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:39.845612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731b301-13bd-4bb8-b243-4ad68dbf8706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:39.846782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731b301-13bd-4bb8-b243-4ad68dbf8706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:39.847165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731b301-13bd-4bb8-b243-4ad68dbf8706 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:39.847725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731b301-13bd-4bb8-b243-4ad68dbf8706 admin admin] PATH_INFO: `/v3/users/0cf7dccf6c4442c58dbdca800b88fcfa` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:39.849179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731b301-13bd-4bb8-b243-4ad68dbf8706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cf7dccf6c4442c58dbdca800b88fcfa)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:39.851304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731b301-13bd-4bb8-b243-4ad68dbf8706 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:40.004376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:40.502226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2731b301-13bd-4bb8-b243-4ad68dbf8706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cf7dccf6c4442c58dbdca800b88fcfa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:40.517635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2731b301-13bd-4bb8-b243-4ad68dbf8706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cf7dccf6c4442c58dbdca800b88fcfa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:40.519758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2731b301-13bd-4bb8-b243-4ad68dbf8706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cf7dccf6c4442c58dbdca800b88fcfa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:40.526996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2731b301-13bd-4bb8-b243-4ad68dbf8706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cf7dccf6c4442c58dbdca800b88fcfa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:40.610493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2731b301-13bd-4bb8-b243-4ad68dbf8706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cf7dccf6c4442c58dbdca800b88fcfa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:40.642097 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1376/2750] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:06:39 2022] DELETE /identity/v3/users/0cf7dccf6c4442c58dbdca800b88fcfa => generated 0 bytes in 892 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:06:41.410313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:41.411515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:41.427445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a3d78-38f4-4d77-9906-8b8a234f4d8e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:41.496047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:41.533376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7042891e-8c90-4b40-a48d-c45ea7af3a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:41.533880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042891e-8c90-4b40-a48d-c45ea7af3a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:41.534839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042891e-8c90-4b40-a48d-c45ea7af3a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:41.535120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7042891e-8c90-4b40-a48d-c45ea7af3a14 admin admin] PATH_INFO: `/v3/projects/77d7e5834b944f059157c4cf0e8a4017` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:41.536417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042891e-8c90-4b40-a48d-c45ea7af3a14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d7e5834b944f059157c4cf0e8a4017)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:41.538572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7042891e-8c90-4b40-a48d-c45ea7af3a14 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:41.719912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7042891e-8c90-4b40-a48d-c45ea7af3a14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d7e5834b944f059157c4cf0e8a4017', 'request_id': 'req-7042891e-8c90-4b40-a48d-c45ea7af3a14'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:41.733080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7042891e-8c90-4b40-a48d-c45ea7af3a14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d7e5834b944f059157c4cf0e8a4017', 'request_id': 'req-7042891e-8c90-4b40-a48d-c45ea7af3a14'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:41.814022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7042891e-8c90-4b40-a48d-c45ea7af3a14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d7e5834b944f059157c4cf0e8a4017', 'request_id': 'req-7042891e-8c90-4b40-a48d-c45ea7af3a14'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:41.836283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7042891e-8c90-4b40-a48d-c45ea7af3a14 admin admin] The token cache is being invalidate because project 77d7e5834b944f059157c4cf0e8a4017 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:06:41.836585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7042891e-8c90-4b40-a48d-c45ea7af3a14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:41.839497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7042891e-8c90-4b40-a48d-c45ea7af3a14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:41.844179 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1375/2751] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:06:41 2022] DELETE /identity/v3/projects/77d7e5834b944f059157c4cf0e8a4017 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:41.850131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cde13-6127-4f6f-837a-8c35cbe7ebd0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:41.851160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cde13-6127-4f6f-837a-8c35cbe7ebd0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:41.851646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cde13-6127-4f6f-837a-8c35cbe7ebd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:41.878817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c42cde13-6127-4f6f-837a-8c35cbe7ebd0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:41.880844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c42cde13-6127-4f6f-837a-8c35cbe7ebd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:42.019061 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1377/2752] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:42.019465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6722a44-af1d-48d0-be34-1aa5a67c2bb8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:42.019663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6722a44-af1d-48d0-be34-1aa5a67c2bb8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:42.019967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6722a44-af1d-48d0-be34-1aa5a67c2bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:42.049121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e6722a44-af1d-48d0-be34-1aa5a67c2bb8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:42.051465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e6722a44-af1d-48d0-be34-1aa5a67c2bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:42.068484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42cde13-6127-4f6f-837a-8c35cbe7ebd0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:42.078004 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1376/2753] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:06:42.085664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42cde13-6127-4f6f-837a-8c35cbe7ebd0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:42.104464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474adfbc-f74b-4823-a67b-00b3219fb557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:42.106106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474adfbc-f74b-4823-a67b-00b3219fb557 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:42.106393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474adfbc-f74b-4823-a67b-00b3219fb557 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:42.106676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474adfbc-f74b-4823-a67b-00b3219fb557 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:42.109787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474adfbc-f74b-4823-a67b-00b3219fb557 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:06:42.109787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474adfbc-f74b-4823-a67b-00b3219fb557 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:42.119336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474adfbc-f74b-4823-a67b-00b3219fb557 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:42.136811 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1378/2754] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:06:42.141487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6722a44-af1d-48d0-be34-1aa5a67c2bb8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:42.152461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6722a44-af1d-48d0-be34-1aa5a67c2bb8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:42.158861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c211f73-5e3d-42ef-93ea-295842c7d978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:42.160807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c211f73-5e3d-42ef-93ea-295842c7d978 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:42.161099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c211f73-5e3d-42ef-93ea-295842c7d978 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:42.161378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c211f73-5e3d-42ef-93ea-295842c7d978 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:42.162156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c211f73-5e3d-42ef-93ea-295842c7d978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:42.164234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c211f73-5e3d-42ef-93ea-295842c7d978 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:42.932144 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1377/2755] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:06:42 2022] POST /identity/v3/projects => generated 391 bytes in 791 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:06:42.936646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474adfbc-f74b-4823-a67b-00b3219fb557 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:42.949194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474adfbc-f74b-4823-a67b-00b3219fb557 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:43.081671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db51aa2-2ba6-464e-b3bb-27f137a73fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:43.082391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db51aa2-2ba6-464e-b3bb-27f137a73fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:43.082674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db51aa2-2ba6-464e-b3bb-27f137a73fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:43.082951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db51aa2-2ba6-464e-b3bb-27f137a73fe7 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:43.083696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db51aa2-2ba6-464e-b3bb-27f137a73fe7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:43.090738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db51aa2-2ba6-464e-b3bb-27f137a73fe7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:43.287866 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1379/2756] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:06:42 2022] POST /identity/v3/users => generated 296 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:06:43.292304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c211f73-5e3d-42ef-93ea-295842c7d978 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:43.309442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c211f73-5e3d-42ef-93ea-295842c7d978 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:43.331370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d080e-f01d-4ea6-96ba-e040e9296ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:43.335927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d080e-f01d-4ea6-96ba-e040e9296ebd admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:43.336219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d080e-f01d-4ea6-96ba-e040e9296ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:43.336499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748d080e-f01d-4ea6-96ba-e040e9296ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:43.338154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d080e-f01d-4ea6-96ba-e040e9296ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:43.348541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748d080e-f01d-4ea6-96ba-e040e9296ebd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:43.376173 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1378/2757] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:06:43.380359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db51aa2-2ba6-464e-b3bb-27f137a73fe7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:43.392567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db51aa2-2ba6-464e-b3bb-27f137a73fe7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:43.399057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec22d7-03f1-44f3-bcc4-222fc03f7fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:43.401684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec22d7-03f1-44f3-bcc4-222fc03f7fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:43.401684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec22d7-03f1-44f3-bcc4-222fc03f7fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:43.401684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec22d7-03f1-44f3-bcc4-222fc03f7fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:43.402196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec22d7-03f1-44f3-bcc4-222fc03f7fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:43.404370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec22d7-03f1-44f3-bcc4-222fc03f7fe5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:43.425887 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1380/2758] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:06:43.430371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d080e-f01d-4ea6-96ba-e040e9296ebd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:43.440961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748d080e-f01d-4ea6-96ba-e040e9296ebd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:43.451797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba621710-53c1-486e-b521-13a4bb52c9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:43.452881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba621710-53c1-486e-b521-13a4bb52c9a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:43.453251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba621710-53c1-486e-b521-13a4bb52c9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:43.453865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba621710-53c1-486e-b521-13a4bb52c9a2 admin admin] PATH_INFO: `/v3/projects/ef2a8e541b9848baaeae4e353592060d/users/e1ea3123c1d049bf8345371b6778b7fc/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:43.500092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba621710-53c1-486e-b521-13a4bb52c9a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2a8e541b9848baaeae4e353592060d, user_id=e1ea3123c1d049bf8345371b6778b7fc, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:43.503500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba621710-53c1-486e-b521-13a4bb52c9a2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:43.622098 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1379/2759] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:06:43 2022] PUT /identity/v3/projects/ef2a8e541b9848baaeae4e353592060d/users/e1ea3123c1d049bf8345371b6778b7fc/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:43.629198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332643e-6725-4f76-8281-fec7c8169200 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:43.629488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332643e-6725-4f76-8281-fec7c8169200 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:43.629768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332643e-6725-4f76-8281-fec7c8169200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:43.664469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:43.678163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8332643e-6725-4f76-8281-fec7c8169200 None None] MFA Rules not processed for user `e1ea3123c1d049bf8345371b6778b7fc`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:43.679399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8332643e-6725-4f76-8281-fec7c8169200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:43.794050 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1381/2760] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:43.795540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba621710-53c1-486e-b521-13a4bb52c9a2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:43.921387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba621710-53c1-486e-b521-13a4bb52c9a2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:43.928983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7de5f-e15c-4b75-9c9e-ce606106ef60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:43.929664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7de5f-e15c-4b75-9c9e-ce606106ef60 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:43.929979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7de5f-e15c-4b75-9c9e-ce606106ef60 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:43.930261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7de5f-e15c-4b75-9c9e-ce606106ef60 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:43.930969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7de5f-e15c-4b75-9c9e-ce606106ef60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:43.933088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7de5f-e15c-4b75-9c9e-ce606106ef60 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:43.957501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:44.063239 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1380/2761] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:06:43 2022] POST /identity/v3/projects => generated 391 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:06:44.068169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332643e-6725-4f76-8281-fec7c8169200 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:44.100688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332643e-6725-4f76-8281-fec7c8169200 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:44.251985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d90f2-fa6a-4a23-b086-645f22d9c2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:44.253408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d90f2-fa6a-4a23-b086-645f22d9c2dc admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:44.253766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d90f2-fa6a-4a23-b086-645f22d9c2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:44.254240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d90f2-fa6a-4a23-b086-645f22d9c2dc admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:44.255021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d90f2-fa6a-4a23-b086-645f22d9c2dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:44.257284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d90f2-fa6a-4a23-b086-645f22d9c2dc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:44.386689 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1382/2762] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:06:44 2022] POST /identity/v3/users => generated 302 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:06:44.387680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7de5f-e15c-4b75-9c9e-ce606106ef60 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:44.409969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7de5f-e15c-4b75-9c9e-ce606106ef60 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:44.417483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec89e8d-bb4e-4218-9790-3c0c366a4509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:44.418199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec89e8d-bb4e-4218-9790-3c0c366a4509 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:44.418583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec89e8d-bb4e-4218-9790-3c0c366a4509 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:44.418953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec89e8d-bb4e-4218-9790-3c0c366a4509 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:44.419595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec89e8d-bb4e-4218-9790-3c0c366a4509 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:44.421227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec89e8d-bb4e-4218-9790-3c0c366a4509 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:44.446145 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1381/2763] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:06:44.446991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d90f2-fa6a-4a23-b086-645f22d9c2dc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:44.465359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d90f2-fa6a-4a23-b086-645f22d9c2dc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:44.470817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6372c-d2aa-4525-85b4-c7b82b1bd2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:44.472422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6372c-d2aa-4525-85b4-c7b82b1bd2fa admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:44.472717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6372c-d2aa-4525-85b4-c7b82b1bd2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:44.473003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf6372c-d2aa-4525-85b4-c7b82b1bd2fa admin admin] PATH_INFO: `/v3/projects/97ae11b4714548839444db01c0973f96/users/e1e46c2ee12248bd8b794453fe8fd818/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:44.491526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6372c-d2aa-4525-85b4-c7b82b1bd2fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ae11b4714548839444db01c0973f96, user_id=e1e46c2ee12248bd8b794453fe8fd818, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:44.495013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf6372c-d2aa-4525-85b4-c7b82b1bd2fa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:44.585356 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1383/2764] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:06:44 2022] PUT /identity/v3/projects/97ae11b4714548839444db01c0973f96/users/e1e46c2ee12248bd8b794453fe8fd818/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:44.589468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec89e8d-bb4e-4218-9790-3c0c366a4509 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:44.600617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec89e8d-bb4e-4218-9790-3c0c366a4509 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:44.720795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbf723-5bba-4dca-80a0-e5c8db22326a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:44.722957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbf723-5bba-4dca-80a0-e5c8db22326a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:44.723651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbf723-5bba-4dca-80a0-e5c8db22326a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:44.724159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbf723-5bba-4dca-80a0-e5c8db22326a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:44.726788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbf723-5bba-4dca-80a0-e5c8db22326a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:44.728922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbf723-5bba-4dca-80a0-e5c8db22326a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:44.749482 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1382/2765] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:06:44.752933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6372c-d2aa-4525-85b4-c7b82b1bd2fa admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:44.769721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf6372c-d2aa-4525-85b4-c7b82b1bd2fa admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:44.781794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedcc28-867a-4acb-9147-e2c355669c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:44.785240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedcc28-867a-4acb-9147-e2c355669c77 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:44.785561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedcc28-867a-4acb-9147-e2c355669c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:44.785870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fedcc28-867a-4acb-9147-e2c355669c77 admin admin] PATH_INFO: `/v3/projects/97ae11b4714548839444db01c0973f96/users/e1e46c2ee12248bd8b794453fe8fd818/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:44.798132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fedcc28-867a-4acb-9147-e2c355669c77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ae11b4714548839444db01c0973f96, user_id=e1e46c2ee12248bd8b794453fe8fd818, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:44.803030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fedcc28-867a-4acb-9147-e2c355669c77 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:44.843322 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1384/2766] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:06:44 2022] PUT /identity/v3/projects/97ae11b4714548839444db01c0973f96/users/e1e46c2ee12248bd8b794453fe8fd818/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:44.845134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf4a8c-ef11-4d31-b8c5-71b19711a2fe None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:44.845955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf4a8c-ef11-4d31-b8c5-71b19711a2fe None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:44.849821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf4a8c-ef11-4d31-b8c5-71b19711a2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:44.900762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b6cf4a8c-ef11-4d31-b8c5-71b19711a2fe None None] MFA Rules not processed for user `e1e46c2ee12248bd8b794453fe8fd818`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:44.902572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b6cf4a8c-ef11-4d31-b8c5-71b19711a2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:45.059786 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1383/2767] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:44 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:45.208979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedcc28-867a-4acb-9147-e2c355669c77 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:45.210893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7fedcc28-867a-4acb-9147-e2c355669c77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:45.344892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fedcc28-867a-4acb-9147-e2c355669c77 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:45.365015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5448a24-40f2-47cb-bdef-1c75420eafbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:45.367017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5448a24-40f2-47cb-bdef-1c75420eafbc service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:45.367017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5448a24-40f2-47cb-bdef-1c75420eafbc service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:45.367017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5448a24-40f2-47cb-bdef-1c75420eafbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:45.381135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5448a24-40f2-47cb-bdef-1c75420eafbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:45.383079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5448a24-40f2-47cb-bdef-1c75420eafbc service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:45.395050 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1385/2768] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:06:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:06:45.481575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:46.036458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf4a8c-ef11-4d31-b8c5-71b19711a2fe None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:46.144130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf4a8c-ef11-4d31-b8c5-71b19711a2fe None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:46.152096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a9942-e4b2-4df9-8095-1b45e76750d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:46.154105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a9942-e4b2-4df9-8095-1b45e76750d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:46.170145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a9942-e4b2-4df9-8095-1b45e76750d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:46.170145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a9942-e4b2-4df9-8095-1b45e76750d6 admin admin] PATH_INFO: `/v3/users/e1ea3123c1d049bf8345371b6778b7fc` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:46.170145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a9942-e4b2-4df9-8095-1b45e76750d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1ea3123c1d049bf8345371b6778b7fc)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:46.170145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a9942-e4b2-4df9-8095-1b45e76750d6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:46.170145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:46.397761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-556a9942-e4b2-4df9-8095-1b45e76750d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1ea3123c1d049bf8345371b6778b7fc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:46.403243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-556a9942-e4b2-4df9-8095-1b45e76750d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1ea3123c1d049bf8345371b6778b7fc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:46.418102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-556a9942-e4b2-4df9-8095-1b45e76750d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1ea3123c1d049bf8345371b6778b7fc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:46.493932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-556a9942-e4b2-4df9-8095-1b45e76750d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1ea3123c1d049bf8345371b6778b7fc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:46.494866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-556a9942-e4b2-4df9-8095-1b45e76750d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1ea3123c1d049bf8345371b6778b7fc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:46.546125 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1384/2769] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:06:46 2022] DELETE /identity/v3/users/e1ea3123c1d049bf8345371b6778b7fc => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:46.550563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cac42-185f-4403-b8d3-24dbb19e3b3a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:46.551379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cac42-185f-4403-b8d3-24dbb19e3b3a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:46.552947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962cac42-185f-4403-b8d3-24dbb19e3b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:46.593791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-962cac42-185f-4403-b8d3-24dbb19e3b3a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:46.595442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-962cac42-185f-4403-b8d3-24dbb19e3b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:46.775927 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1386/2770] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:46.801864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a9942-e4b2-4df9-8095-1b45e76750d6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:46.803882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-556a9942-e4b2-4df9-8095-1b45e76750d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:46.890506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a9942-e4b2-4df9-8095-1b45e76750d6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:46.895517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9cea9-815f-40f0-8caf-33b61a754a38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:46.896521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9cea9-815f-40f0-8caf-33b61a754a38 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:46.897106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9cea9-815f-40f0-8caf-33b61a754a38 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:46.897562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9cea9-815f-40f0-8caf-33b61a754a38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:46.908624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9cea9-815f-40f0-8caf-33b61a754a38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:46.910704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9cea9-815f-40f0-8caf-33b61a754a38 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:46.925651 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1385/2771] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:06:47.695433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cac42-185f-4403-b8d3-24dbb19e3b3a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:47.697412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-962cac42-185f-4403-b8d3-24dbb19e3b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:47.750420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962cac42-185f-4403-b8d3-24dbb19e3b3a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:47.768835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff9ac5-c702-4d68-a633-2c301490f916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:47.771587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff9ac5-c702-4d68-a633-2c301490f916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:47.771587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff9ac5-c702-4d68-a633-2c301490f916 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:47.771587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff9ac5-c702-4d68-a633-2c301490f916 admin admin] PATH_INFO: `/v3/projects/ef2a8e541b9848baaeae4e353592060d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:47.771587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff9ac5-c702-4d68-a633-2c301490f916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef2a8e541b9848baaeae4e353592060d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:47.773741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff9ac5-c702-4d68-a633-2c301490f916 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:47.914645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d0ff9ac5-c702-4d68-a633-2c301490f916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef2a8e541b9848baaeae4e353592060d', 'request_id': 'req-d0ff9ac5-c702-4d68-a633-2c301490f916'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:47.921557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d0ff9ac5-c702-4d68-a633-2c301490f916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef2a8e541b9848baaeae4e353592060d', 'request_id': 'req-d0ff9ac5-c702-4d68-a633-2c301490f916'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:48.002286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d0ff9ac5-c702-4d68-a633-2c301490f916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef2a8e541b9848baaeae4e353592060d', 'request_id': 'req-d0ff9ac5-c702-4d68-a633-2c301490f916'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:48.027751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d0ff9ac5-c702-4d68-a633-2c301490f916 admin admin] The token cache is being invalidate because project ef2a8e541b9848baaeae4e353592060d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:06:48.028053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d0ff9ac5-c702-4d68-a633-2c301490f916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:48.030306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d0ff9ac5-c702-4d68-a633-2c301490f916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:48.038651 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1387/2772] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:06:47 2022] DELETE /identity/v3/projects/ef2a8e541b9848baaeae4e353592060d => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:48.046327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9cea9-815f-40f0-8caf-33b61a754a38 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:48.060563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9cea9-815f-40f0-8caf-33b61a754a38 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:48.149966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3717eaa-e2ad-40a8-ae81-7a0ae555e51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:48.157044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3717eaa-e2ad-40a8-ae81-7a0ae555e51d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:48.157044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3717eaa-e2ad-40a8-ae81-7a0ae555e51d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:48.157044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3717eaa-e2ad-40a8-ae81-7a0ae555e51d admin admin] PATH_INFO: `/v3/users/e1e46c2ee12248bd8b794453fe8fd818` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:48.157044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3717eaa-e2ad-40a8-ae81-7a0ae555e51d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1e46c2ee12248bd8b794453fe8fd818)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:48.157044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3717eaa-e2ad-40a8-ae81-7a0ae555e51d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:48.266330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:48.368023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b3717eaa-e2ad-40a8-ae81-7a0ae555e51d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1e46c2ee12248bd8b794453fe8fd818'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:48.369007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b3717eaa-e2ad-40a8-ae81-7a0ae555e51d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1e46c2ee12248bd8b794453fe8fd818'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:48.378360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b3717eaa-e2ad-40a8-ae81-7a0ae555e51d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1e46c2ee12248bd8b794453fe8fd818'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:48.476902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b3717eaa-e2ad-40a8-ae81-7a0ae555e51d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1e46c2ee12248bd8b794453fe8fd818'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:48.477974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b3717eaa-e2ad-40a8-ae81-7a0ae555e51d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1e46c2ee12248bd8b794453fe8fd818'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:48.516607 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1386/2773] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:06:48 2022] DELETE /identity/v3/users/e1e46c2ee12248bd8b794453fe8fd818 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:49.060455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff9ac5-c702-4d68-a633-2c301490f916 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:49.062633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d0ff9ac5-c702-4d68-a633-2c301490f916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:49.102901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff9ac5-c702-4d68-a633-2c301490f916 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:49.229124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231e65c5-7878-42ad-acaf-3e371adb44f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:49.230380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e65c5-7878-42ad-acaf-3e371adb44f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:49.230848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e65c5-7878-42ad-acaf-3e371adb44f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:49.231147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e65c5-7878-42ad-acaf-3e371adb44f8 admin admin] PATH_INFO: `/v3/projects/97ae11b4714548839444db01c0973f96` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:49.232472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e65c5-7878-42ad-acaf-3e371adb44f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97ae11b4714548839444db01c0973f96)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:49.237875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231e65c5-7878-42ad-acaf-3e371adb44f8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:49.237875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:49.380610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-231e65c5-7878-42ad-acaf-3e371adb44f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97ae11b4714548839444db01c0973f96', 'request_id': 'req-231e65c5-7878-42ad-acaf-3e371adb44f8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:49.391259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-231e65c5-7878-42ad-acaf-3e371adb44f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97ae11b4714548839444db01c0973f96', 'request_id': 'req-231e65c5-7878-42ad-acaf-3e371adb44f8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:49.481800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-231e65c5-7878-42ad-acaf-3e371adb44f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97ae11b4714548839444db01c0973f96', 'request_id': 'req-231e65c5-7878-42ad-acaf-3e371adb44f8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:49.497195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-231e65c5-7878-42ad-acaf-3e371adb44f8 admin admin] The token cache is being invalidate because project 97ae11b4714548839444db01c0973f96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:06:49.497591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-231e65c5-7878-42ad-acaf-3e371adb44f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:49.498476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-231e65c5-7878-42ad-acaf-3e371adb44f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:49.500443 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1388/2774] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:06:49 2022] DELETE /identity/v3/projects/97ae11b4714548839444db01c0973f96 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:49.505765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605dead-3c33-42f0-b4ee-476008ca5e44 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:49.506265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605dead-3c33-42f0-b4ee-476008ca5e44 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:49.510100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e605dead-3c33-42f0-b4ee-476008ca5e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:49.527138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e605dead-3c33-42f0-b4ee-476008ca5e44 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:49.528508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e605dead-3c33-42f0-b4ee-476008ca5e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:49.590009 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1387/2775] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:49.597941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1dde2-a363-4a60-832a-3ee60298772d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:49.598235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1dde2-a363-4a60-832a-3ee60298772d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:49.598514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd1dde2-a363-4a60-832a-3ee60298772d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:49.618978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0dd1dde2-a363-4a60-832a-3ee60298772d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:49.620349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0dd1dde2-a363-4a60-832a-3ee60298772d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:49.657656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03af9-7782-42ce-8a3b-a53f98480bde None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:49.657975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03af9-7782-42ce-8a3b-a53f98480bde None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:49.658258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03af9-7782-42ce-8a3b-a53f98480bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:49.660151 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1389/2776] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:49.660878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1dde2-a363-4a60-832a-3ee60298772d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:49.680186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd1dde2-a363-4a60-832a-3ee60298772d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:49.688474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ee2d8-f6e8-4961-936e-ff942bb5c8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:49.690239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4da03af9-7782-42ce-8a3b-a53f98480bde None None] MFA Rules not processed for user `5e54b51913be4a2bab74aae347785727`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:49.691736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4da03af9-7782-42ce-8a3b-a53f98480bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:49.691967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ee2d8-f6e8-4961-936e-ff942bb5c8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:49.692248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ee2d8-f6e8-4961-936e-ff942bb5c8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:49.692539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ee2d8-f6e8-4961-936e-ff942bb5c8f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:49.696100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ee2d8-f6e8-4961-936e-ff942bb5c8f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:06:49.696631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ee2d8-f6e8-4961-936e-ff942bb5c8f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:49.698467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ee2d8-f6e8-4961-936e-ff942bb5c8f1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:49.727121 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1390/2777] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:06:49.731377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ee2d8-f6e8-4961-936e-ff942bb5c8f1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:49.744340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ee2d8-f6e8-4961-936e-ff942bb5c8f1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:49.749800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8cd07-28bd-41e6-a6a0-8b4e9cc4f5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:49.751635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8cd07-28bd-41e6-a6a0-8b4e9cc4f5fc admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:49.751635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8cd07-28bd-41e6-a6a0-8b4e9cc4f5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:49.751635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8cd07-28bd-41e6-a6a0-8b4e9cc4f5fc admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:49.752217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8cd07-28bd-41e6-a6a0-8b4e9cc4f5fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:49.757015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8cd07-28bd-41e6-a6a0-8b4e9cc4f5fc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:49.794959 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1388/2778] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:06:49.845608 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1391/2779] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:06:49 2022] POST /identity/v3/projects => generated 407 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:06:49.850088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da03af9-7782-42ce-8a3b-a53f98480bde None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:49.863939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da03af9-7782-42ce-8a3b-a53f98480bde None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:49.942352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:49.956776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a85ee5-69c5-4c26-9e9e-e15e8f7d34e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:49.957566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a85ee5-69c5-4c26-9e9e-e15e8f7d34e6 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:49.958013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a85ee5-69c5-4c26-9e9e-e15e8f7d34e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:49.958389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a85ee5-69c5-4c26-9e9e-e15e8f7d34e6 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:49.959628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a85ee5-69c5-4c26-9e9e-e15e8f7d34e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:49.961968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a85ee5-69c5-4c26-9e9e-e15e8f7d34e6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:50.043579 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1389/2780] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:06:49 2022] POST /identity/v3/users => generated 304 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:06:50.047927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8cd07-28bd-41e6-a6a0-8b4e9cc4f5fc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:50.082760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8cd07-28bd-41e6-a6a0-8b4e9cc4f5fc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:50.098941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2f3a3-d4a1-422d-bcda-f016ae61bf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:50.099875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2f3a3-d4a1-422d-bcda-f016ae61bf81 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:50.100338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2f3a3-d4a1-422d-bcda-f016ae61bf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:50.100775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2f3a3-d4a1-422d-bcda-f016ae61bf81 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:50.101669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2f3a3-d4a1-422d-bcda-f016ae61bf81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:50.103453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd2f3a3-d4a1-422d-bcda-f016ae61bf81 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:50.124830 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1392/2781] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:06:50.125646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a85ee5-69c5-4c26-9e9e-e15e8f7d34e6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:50.142348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a85ee5-69c5-4c26-9e9e-e15e8f7d34e6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:50.148320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f142052a-8e68-42b0-8b91-7c7d4336a35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:50.149195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142052a-8e68-42b0-8b91-7c7d4336a35d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:50.149623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142052a-8e68-42b0-8b91-7c7d4336a35d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:50.150257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f142052a-8e68-42b0-8b91-7c7d4336a35d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:50.151159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142052a-8e68-42b0-8b91-7c7d4336a35d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:50.152961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f142052a-8e68-42b0-8b91-7c7d4336a35d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:50.165252 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1390/2782] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:06:50.169516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2f3a3-d4a1-422d-bcda-f016ae61bf81 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:50.186944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2f3a3-d4a1-422d-bcda-f016ae61bf81 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:50.192875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb7396-2de1-4dd4-a521-7e683e0d3b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:50.194464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb7396-2de1-4dd4-a521-7e683e0d3b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:50.194922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb7396-2de1-4dd4-a521-7e683e0d3b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:50.195365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb7396-2de1-4dd4-a521-7e683e0d3b28 admin admin] PATH_INFO: `/v3/projects/0ee29dc0114c40db834365213c650e4f/users/2c5305ed9fd24f47837af1bba69fc591/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:50.216626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb7396-2de1-4dd4-a521-7e683e0d3b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee29dc0114c40db834365213c650e4f, user_id=2c5305ed9fd24f47837af1bba69fc591, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:50.218244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb7396-2de1-4dd4-a521-7e683e0d3b28 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:50.299379 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1393/2783] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:06:50 2022] PUT /identity/v3/projects/0ee29dc0114c40db834365213c650e4f/users/2c5305ed9fd24f47837af1bba69fc591/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:50.315481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4046b0b-38d2-4159-9ede-89d8e1f865f3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:50.315780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4046b0b-38d2-4159-9ede-89d8e1f865f3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:50.316065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4046b0b-38d2-4159-9ede-89d8e1f865f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:50.360431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e4046b0b-38d2-4159-9ede-89d8e1f865f3 None None] MFA Rules not processed for user `2c5305ed9fd24f47837af1bba69fc591`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:50.362430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e4046b0b-38d2-4159-9ede-89d8e1f865f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:50.461643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb7396-2de1-4dd4-a521-7e683e0d3b28 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:50.462769 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1391/2784] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:50 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:50.546728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb7396-2de1-4dd4-a521-7e683e0d3b28 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:50.552904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c64cd-7fec-4e14-aa2f-3ad130de25d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:50.553751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c64cd-7fec-4e14-aa2f-3ad130de25d4 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:50.554277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c64cd-7fec-4e14-aa2f-3ad130de25d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:50.554679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c64cd-7fec-4e14-aa2f-3ad130de25d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:50.555823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c64cd-7fec-4e14-aa2f-3ad130de25d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:50.558385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c64cd-7fec-4e14-aa2f-3ad130de25d4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:50.649893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:50.655895 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1394/2785] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:06:50 2022] POST /identity/v3/projects => generated 407 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:06:50.660007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4046b0b-38d2-4159-9ede-89d8e1f865f3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:50.683106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4046b0b-38d2-4159-9ede-89d8e1f865f3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:50.773947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1531e-8c7a-4b70-b56d-54bd0e58e94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:50.774631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1531e-8c7a-4b70-b56d-54bd0e58e94a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:50.774916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1531e-8c7a-4b70-b56d-54bd0e58e94a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:50.775714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e1531e-8c7a-4b70-b56d-54bd0e58e94a admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:50.775888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e1531e-8c7a-4b70-b56d-54bd0e58e94a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:50.778203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e1531e-8c7a-4b70-b56d-54bd0e58e94a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:50.865594 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1392/2786] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:06:50 2022] POST /identity/v3/users => generated 310 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:06:50.870407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c64cd-7fec-4e14-aa2f-3ad130de25d4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:50.886028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c64cd-7fec-4e14-aa2f-3ad130de25d4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:50.891211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04e691-ed93-4170-87d1-ca6c6b7c5dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:50.891954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04e691-ed93-4170-87d1-ca6c6b7c5dc4 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:50.892249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04e691-ed93-4170-87d1-ca6c6b7c5dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:50.892731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a04e691-ed93-4170-87d1-ca6c6b7c5dc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:50.893524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04e691-ed93-4170-87d1-ca6c6b7c5dc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:50.895176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a04e691-ed93-4170-87d1-ca6c6b7c5dc4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:50.907011 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1395/2787] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:06:50.911587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1531e-8c7a-4b70-b56d-54bd0e58e94a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:50.925942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e1531e-8c7a-4b70-b56d-54bd0e58e94a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:50.930978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eced0e-b109-40fc-9ba8-3080b4261cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:50.931966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eced0e-b109-40fc-9ba8-3080b4261cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:50.932322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eced0e-b109-40fc-9ba8-3080b4261cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:50.932676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eced0e-b109-40fc-9ba8-3080b4261cd6 admin admin] PATH_INFO: `/v3/projects/4d1e18a0cfbd4121a5ba7029a17a2b69/users/06a07acde5ee46d399370f77db5ba5c9/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:50.950760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eced0e-b109-40fc-9ba8-3080b4261cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d1e18a0cfbd4121a5ba7029a17a2b69, user_id=06a07acde5ee46d399370f77db5ba5c9, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:50.953630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eced0e-b109-40fc-9ba8-3080b4261cd6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:51.060107 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1393/2788] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:06:50 2022] PUT /identity/v3/projects/4d1e18a0cfbd4121a5ba7029a17a2b69/users/06a07acde5ee46d399370f77db5ba5c9/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:51.064132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04e691-ed93-4170-87d1-ca6c6b7c5dc4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:51.080403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a04e691-ed93-4170-87d1-ca6c6b7c5dc4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:51.163267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec756f26-d0b7-46f2-80cc-fbb7d178fad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:51.164148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec756f26-d0b7-46f2-80cc-fbb7d178fad6 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:51.164439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec756f26-d0b7-46f2-80cc-fbb7d178fad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:51.164734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec756f26-d0b7-46f2-80cc-fbb7d178fad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:51.165531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec756f26-d0b7-46f2-80cc-fbb7d178fad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:51.167627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec756f26-d0b7-46f2-80cc-fbb7d178fad6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:51.187951 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1396/2789] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:06:51.192984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eced0e-b109-40fc-9ba8-3080b4261cd6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:51.206366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eced0e-b109-40fc-9ba8-3080b4261cd6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:51.220671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33accf84-6ba1-47e8-915c-cca865bf8eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:51.222157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33accf84-6ba1-47e8-915c-cca865bf8eb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:51.222447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33accf84-6ba1-47e8-915c-cca865bf8eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:51.222727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33accf84-6ba1-47e8-915c-cca865bf8eb2 admin admin] PATH_INFO: `/v3/projects/4d1e18a0cfbd4121a5ba7029a17a2b69/users/06a07acde5ee46d399370f77db5ba5c9/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:51.227377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33accf84-6ba1-47e8-915c-cca865bf8eb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d1e18a0cfbd4121a5ba7029a17a2b69, user_id=06a07acde5ee46d399370f77db5ba5c9, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:51.242238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33accf84-6ba1-47e8-915c-cca865bf8eb2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:51.279236 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1394/2790] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:06:51 2022] PUT /identity/v3/projects/4d1e18a0cfbd4121a5ba7029a17a2b69/users/06a07acde5ee46d399370f77db5ba5c9/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:51.290148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceec14-0aa9-44b6-a8a7-c0d3c6a0c36f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:51.290454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceec14-0aa9-44b6-a8a7-c0d3c6a0c36f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:51.290728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dceec14-0aa9-44b6-a8a7-c0d3c6a0c36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:51.344956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0dceec14-0aa9-44b6-a8a7-c0d3c6a0c36f None None] MFA Rules not processed for user `06a07acde5ee46d399370f77db5ba5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:51.359391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0dceec14-0aa9-44b6-a8a7-c0d3c6a0c36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:51.551184 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1397/2791] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:51 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:51.644979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33accf84-6ba1-47e8-915c-cca865bf8eb2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:51.648099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-33accf84-6ba1-47e8-915c-cca865bf8eb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:51.664323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:51.799449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33accf84-6ba1-47e8-915c-cca865bf8eb2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:51.808220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480577d7-67a9-4b63-afd5-81c314b961d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:51.809015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480577d7-67a9-4b63-afd5-81c314b961d4 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:51.809708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480577d7-67a9-4b63-afd5-81c314b961d4 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:51.810238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480577d7-67a9-4b63-afd5-81c314b961d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:51.823524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480577d7-67a9-4b63-afd5-81c314b961d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:51.827300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480577d7-67a9-4b63-afd5-81c314b961d4 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:51.841322 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1395/2792] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:06:51 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:06:51.955265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dceec14-0aa9-44b6-a8a7-c0d3c6a0c36f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:51.975809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dceec14-0aa9-44b6-a8a7-c0d3c6a0c36f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:51.994137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dab8c8-3d23-4051-bc45-53df72ce385d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:51.995134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dab8c8-3d23-4051-bc45-53df72ce385d service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:51.995607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dab8c8-3d23-4051-bc45-53df72ce385d service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:51.996047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dab8c8-3d23-4051-bc45-53df72ce385d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:52.134851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dab8c8-3d23-4051-bc45-53df72ce385d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:52.134851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dab8c8-3d23-4051-bc45-53df72ce385d service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:52.134851 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1398/2793] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:06:51 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:06:52.311679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480577d7-67a9-4b63-afd5-81c314b961d4 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:52.394957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480577d7-67a9-4b63-afd5-81c314b961d4 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:52.399725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69e4ff-a07a-446d-a17b-e930abba0fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:52.400646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69e4ff-a07a-446d-a17b-e930abba0fb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:52.400927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69e4ff-a07a-446d-a17b-e930abba0fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:52.401201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69e4ff-a07a-446d-a17b-e930abba0fb5 admin admin] PATH_INFO: `/v3/users/2c5305ed9fd24f47837af1bba69fc591` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:52.402433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec69e4ff-a07a-446d-a17b-e930abba0fb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c5305ed9fd24f47837af1bba69fc591)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:52.404390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec69e4ff-a07a-446d-a17b-e930abba0fb5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:52.757510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec69e4ff-a07a-446d-a17b-e930abba0fb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c5305ed9fd24f47837af1bba69fc591'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:52.758255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec69e4ff-a07a-446d-a17b-e930abba0fb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c5305ed9fd24f47837af1bba69fc591'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:52.769501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec69e4ff-a07a-446d-a17b-e930abba0fb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c5305ed9fd24f47837af1bba69fc591'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:52.978924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec69e4ff-a07a-446d-a17b-e930abba0fb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c5305ed9fd24f47837af1bba69fc591'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:52.983099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec69e4ff-a07a-446d-a17b-e930abba0fb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c5305ed9fd24f47837af1bba69fc591'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:53.014489 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1396/2794] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:06:52 2022] DELETE /identity/v3/users/2c5305ed9fd24f47837af1bba69fc591 => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:06:53.020994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c72f1-f041-418c-babd-b0f281324ead None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:53.021537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c72f1-f041-418c-babd-b0f281324ead None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:53.021643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c72f1-f041-418c-babd-b0f281324ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:53.057789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:53.078530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-906c72f1-f041-418c-babd-b0f281324ead None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:53.092965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-906c72f1-f041-418c-babd-b0f281324ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:53.290809 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1399/2795] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:53.340713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69e4ff-a07a-446d-a17b-e930abba0fb5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:53.342337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ec69e4ff-a07a-446d-a17b-e930abba0fb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:53.369138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:53.461477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69e4ff-a07a-446d-a17b-e930abba0fb5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:53.470309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd5226-61fa-49f9-b490-970e694eb9a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:53.470835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd5226-61fa-49f9-b490-970e694eb9a1 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:53.471145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd5226-61fa-49f9-b490-970e694eb9a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:53.471476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd5226-61fa-49f9-b490-970e694eb9a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:53.490539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd5226-61fa-49f9-b490-970e694eb9a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:53.492020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedd5226-61fa-49f9-b490-970e694eb9a1 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:53.517585 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1397/2796] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:06:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:06:54.687781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c72f1-f041-418c-babd-b0f281324ead None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:54.689398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-906c72f1-f041-418c-babd-b0f281324ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:54.720363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c72f1-f041-418c-babd-b0f281324ead None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:54.740747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:54.741735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:54.742110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:54.743296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8 admin admin] PATH_INFO: `/v3/projects/0ee29dc0114c40db834365213c650e4f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:54.748331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ee29dc0114c40db834365213c650e4f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:54.751752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:54.989094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ee29dc0114c40db834365213c650e4f', 'request_id': 'req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:54.996797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ee29dc0114c40db834365213c650e4f', 'request_id': 'req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:55.066177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ee29dc0114c40db834365213c650e4f', 'request_id': 'req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:55.086389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8 admin admin] The token cache is being invalidate because project 0ee29dc0114c40db834365213c650e4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:06:55.086732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:55.087627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:55.090876 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1400/2797] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:06:54 2022] DELETE /identity/v3/projects/0ee29dc0114c40db834365213c650e4f => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:55.106635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd5226-61fa-49f9-b490-970e694eb9a1 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:55.150582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd5226-61fa-49f9-b490-970e694eb9a1 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:55.296747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a94776-ad4a-4392-a166-c99b454cb030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:55.297637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a94776-ad4a-4392-a166-c99b454cb030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:55.297959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a94776-ad4a-4392-a166-c99b454cb030 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:55.298601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a94776-ad4a-4392-a166-c99b454cb030 admin admin] PATH_INFO: `/v3/users/06a07acde5ee46d399370f77db5ba5c9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:55.300183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a94776-ad4a-4392-a166-c99b454cb030 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06a07acde5ee46d399370f77db5ba5c9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:55.302170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a94776-ad4a-4392-a166-c99b454cb030 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:55.532338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e9a94776-ad4a-4392-a166-c99b454cb030 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06a07acde5ee46d399370f77db5ba5c9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:55.533007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e9a94776-ad4a-4392-a166-c99b454cb030 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06a07acde5ee46d399370f77db5ba5c9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:55.553229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e9a94776-ad4a-4392-a166-c99b454cb030 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06a07acde5ee46d399370f77db5ba5c9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:55.630186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e9a94776-ad4a-4392-a166-c99b454cb030 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06a07acde5ee46d399370f77db5ba5c9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:55.630920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e9a94776-ad4a-4392-a166-c99b454cb030 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06a07acde5ee46d399370f77db5ba5c9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:55.649468 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1398/2798] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:06:55 2022] DELETE /identity/v3/users/06a07acde5ee46d399370f77db5ba5c9 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:56.516314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:56.517970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:56.532877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d44f37-4403-41e1-b7ce-ca3b8d9820d8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:56.599932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a1d17-16c1-45b3-ade8-4423fc350967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:56.600469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a1d17-16c1-45b3-ade8-4423fc350967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:56.600728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a1d17-16c1-45b3-ade8-4423fc350967 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:56.601197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a1d17-16c1-45b3-ade8-4423fc350967 admin admin] PATH_INFO: `/v3/projects/4d1e18a0cfbd4121a5ba7029a17a2b69` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:56.603057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a1d17-16c1-45b3-ade8-4423fc350967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d1e18a0cfbd4121a5ba7029a17a2b69)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:56.605164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a1d17-16c1-45b3-ade8-4423fc350967 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:56.765427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98430e0-e941-44d4-abcb-dde1d3f54d02 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:56.766021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98430e0-e941-44d4-abcb-dde1d3f54d02 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:56.766202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98430e0-e941-44d4-abcb-dde1d3f54d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:56.781692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:56.788768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e98430e0-e941-44d4-abcb-dde1d3f54d02 None None] MFA Rules not processed for user `dc48535b672d4d1190ecfefe6cb02720`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:56.790494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e98430e0-e941-44d4-abcb-dde1d3f54d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:56.810869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-767a1d17-16c1-45b3-ade8-4423fc350967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d1e18a0cfbd4121a5ba7029a17a2b69', 'request_id': 'req-767a1d17-16c1-45b3-ade8-4423fc350967'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:56.822017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-767a1d17-16c1-45b3-ade8-4423fc350967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d1e18a0cfbd4121a5ba7029a17a2b69', 'request_id': 'req-767a1d17-16c1-45b3-ade8-4423fc350967'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:56.874025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:56.908120 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1399/2799] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:06:56.957067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-767a1d17-16c1-45b3-ade8-4423fc350967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d1e18a0cfbd4121a5ba7029a17a2b69', 'request_id': 'req-767a1d17-16c1-45b3-ade8-4423fc350967'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:56.973613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-767a1d17-16c1-45b3-ade8-4423fc350967 admin admin] The token cache is being invalidate because project 4d1e18a0cfbd4121a5ba7029a17a2b69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:06:56.974021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-767a1d17-16c1-45b3-ade8-4423fc350967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:56.974728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-767a1d17-16c1-45b3-ade8-4423fc350967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:06:56.976918 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1401/2800] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:06:56 2022] DELETE /identity/v3/projects/4d1e18a0cfbd4121a5ba7029a17a2b69 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:06:56.984200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0cf6e-48ba-4836-bc5e-5d172fe742eb None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:56.984865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0cf6e-48ba-4836-bc5e-5d172fe742eb None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:56.985428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0cf6e-48ba-4836-bc5e-5d172fe742eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:57.016464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5df0cf6e-48ba-4836-bc5e-5d172fe742eb None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:57.030803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5df0cf6e-48ba-4836-bc5e-5d172fe742eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:57.136715 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1400/2801] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:57.146437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c48f70-567f-41ed-9253-cbfbca9329e3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:57.146721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c48f70-567f-41ed-9253-cbfbca9329e3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:57.147398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c48f70-567f-41ed-9253-cbfbca9329e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:57.168251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e0c48f70-567f-41ed-9253-cbfbca9329e3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:06:57.169494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e0c48f70-567f-41ed-9253-cbfbca9329e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:06:57.179974 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1402/2802] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:06:57.184517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0cf6e-48ba-4836-bc5e-5d172fe742eb None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:57.198207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0cf6e-48ba-4836-bc5e-5d172fe742eb None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:57.204531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354827d-8f5c-46e1-9515-15e81ee81383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:57.205324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354827d-8f5c-46e1-9515-15e81ee81383 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:57.206276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354827d-8f5c-46e1-9515-15e81ee81383 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:57.206658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354827d-8f5c-46e1-9515-15e81ee81383 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:57.207496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354827d-8f5c-46e1-9515-15e81ee81383 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:06:57.207878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354827d-8f5c-46e1-9515-15e81ee81383 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:57.210777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354827d-8f5c-46e1-9515-15e81ee81383 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:57.227159 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1401/2803] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:06:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:06:57.231605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c48f70-567f-41ed-9253-cbfbca9329e3 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:57.241598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c48f70-567f-41ed-9253-cbfbca9329e3 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:57.250830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003be2dd-c870-4dd4-a2a4-b1a553e91373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:57.252010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003be2dd-c870-4dd4-a2a4-b1a553e91373 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:57.266162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003be2dd-c870-4dd4-a2a4-b1a553e91373 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:57.266162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003be2dd-c870-4dd4-a2a4-b1a553e91373 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:57.266162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003be2dd-c870-4dd4-a2a4-b1a553e91373 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:57.266162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003be2dd-c870-4dd4-a2a4-b1a553e91373 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:06:59.798175 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1403/2804] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:06:57 2022] POST /identity/v3/projects => generated 409 bytes in 2567 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:06:59.803111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354827d-8f5c-46e1-9515-15e81ee81383 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:06:59.815260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354827d-8f5c-46e1-9515-15e81ee81383 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:06:59.872405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c3d3d-d466-4134-b0a7-3aa44f7e5700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:06:59.873084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c3d3d-d466-4134-b0a7-3aa44f7e5700 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:06:59.873397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c3d3d-d466-4134-b0a7-3aa44f7e5700 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:06:59.873676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c3d3d-d466-4134-b0a7-3aa44f7e5700 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:06:59.874430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c3d3d-d466-4134-b0a7-3aa44f7e5700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:06:59.876405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c3d3d-d466-4134-b0a7-3aa44f7e5700 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:00.179653 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1402/2805] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:06:59 2022] POST /identity/v3/users => generated 305 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:07:00.186525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003be2dd-c870-4dd4-a2a4-b1a553e91373 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:00.209609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003be2dd-c870-4dd4-a2a4-b1a553e91373 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:00.222575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd260946-f00f-4467-af6b-dd0447fc370c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:00.232158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd260946-f00f-4467-af6b-dd0447fc370c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:00.232158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd260946-f00f-4467-af6b-dd0447fc370c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:00.232158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd260946-f00f-4467-af6b-dd0447fc370c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:00.232158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd260946-f00f-4467-af6b-dd0447fc370c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:00.232158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd260946-f00f-4467-af6b-dd0447fc370c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:00.253092 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1404/2806] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:00.254399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c3d3d-d466-4134-b0a7-3aa44f7e5700 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:00.270886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c3d3d-d466-4134-b0a7-3aa44f7e5700 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:00.276140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0334ed-f201-42e6-8a06-ef93bff87109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:00.278569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0334ed-f201-42e6-8a06-ef93bff87109 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:00.278569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0334ed-f201-42e6-8a06-ef93bff87109 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:00.278569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0334ed-f201-42e6-8a06-ef93bff87109 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:00.278835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0334ed-f201-42e6-8a06-ef93bff87109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:00.280831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0334ed-f201-42e6-8a06-ef93bff87109 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:00.295587 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1403/2807] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:00.299647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd260946-f00f-4467-af6b-dd0447fc370c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:00.312889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd260946-f00f-4467-af6b-dd0447fc370c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:00.320437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfe30c-3b2f-4077-8beb-64576ffc4a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:00.321420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfe30c-3b2f-4077-8beb-64576ffc4a9a admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:00.323085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfe30c-3b2f-4077-8beb-64576ffc4a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:00.323396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfe30c-3b2f-4077-8beb-64576ffc4a9a admin admin] PATH_INFO: `/v3/projects/42c460c2c90640fa972c1c17fb41644a/users/5b8e5137419744f79eab638c4d3abff7/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:00.423260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfe30c-3b2f-4077-8beb-64576ffc4a9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=42c460c2c90640fa972c1c17fb41644a, user_id=5b8e5137419744f79eab638c4d3abff7, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:00.426309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dfe30c-3b2f-4077-8beb-64576ffc4a9a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:00.541979 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1405/2808] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:07:00 2022] PUT /identity/v3/projects/42c460c2c90640fa972c1c17fb41644a/users/5b8e5137419744f79eab638c4d3abff7/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:07:00.555162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4caa7-8906-4049-ad64-1bb4081925a4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:00.555703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4caa7-8906-4049-ad64-1bb4081925a4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:00.556087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4caa7-8906-4049-ad64-1bb4081925a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:00.602596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d3f4caa7-8906-4049-ad64-1bb4081925a4 None None] MFA Rules not processed for user `5b8e5137419744f79eab638c4d3abff7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:00.603842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d3f4caa7-8906-4049-ad64-1bb4081925a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:00.734811 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1404/2809] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:00 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:00.734969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfe30c-3b2f-4077-8beb-64576ffc4a9a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:00.901658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfe30c-3b2f-4077-8beb-64576ffc4a9a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:00.918271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b240e-0a1e-4fb3-ab8d-a4501ea8ae6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:00.919087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b240e-0a1e-4fb3-ab8d-a4501ea8ae6b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:00.920182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b240e-0a1e-4fb3-ab8d-a4501ea8ae6b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:00.920569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620b240e-0a1e-4fb3-ab8d-a4501ea8ae6b admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:00.925015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b240e-0a1e-4fb3-ab8d-a4501ea8ae6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:00.928636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620b240e-0a1e-4fb3-ab8d-a4501ea8ae6b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:01.073095 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1406/2810] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:07:00 2022] POST /identity/v3/projects => generated 409 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:07:01.077652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4caa7-8906-4049-ad64-1bb4081925a4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:01.114118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4caa7-8906-4049-ad64-1bb4081925a4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:01.226631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f5706-8696-4dba-9830-9904684af390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:01.228548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f5706-8696-4dba-9830-9904684af390 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:01.229157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f5706-8696-4dba-9830-9904684af390 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:01.229157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f5706-8696-4dba-9830-9904684af390 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:01.229889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f5706-8696-4dba-9830-9904684af390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:01.232448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f5706-8696-4dba-9830-9904684af390 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:01.382641 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1405/2811] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:07:01 2022] POST /identity/v3/users => generated 311 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:07:01.387302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b240e-0a1e-4fb3-ab8d-a4501ea8ae6b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:01.447225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620b240e-0a1e-4fb3-ab8d-a4501ea8ae6b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:01.470743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10411a0-2c59-4940-8f21-6971e25477ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:01.482942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10411a0-2c59-4940-8f21-6971e25477ac admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:01.484270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10411a0-2c59-4940-8f21-6971e25477ac admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:01.484270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10411a0-2c59-4940-8f21-6971e25477ac admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:01.484941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10411a0-2c59-4940-8f21-6971e25477ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:01.497127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10411a0-2c59-4940-8f21-6971e25477ac admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:01.516318 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1407/2812] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:01.524138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f5706-8696-4dba-9830-9904684af390 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:01.560989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f5706-8696-4dba-9830-9904684af390 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:01.573703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587ac32-c899-481d-bf98-751024e37b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:01.574825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587ac32-c899-481d-bf98-751024e37b31 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:01.575576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587ac32-c899-481d-bf98-751024e37b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:01.577286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587ac32-c899-481d-bf98-751024e37b31 admin admin] PATH_INFO: `/v3/projects/78a7f2aa3f7849f3b2f5dd0a06801899/users/717861b3356e469197cafbd14b71a02d/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:01.606776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5587ac32-c899-481d-bf98-751024e37b31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78a7f2aa3f7849f3b2f5dd0a06801899, user_id=717861b3356e469197cafbd14b71a02d, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:01.612598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5587ac32-c899-481d-bf98-751024e37b31 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:01.735839 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1406/2813] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:07:01 2022] PUT /identity/v3/projects/78a7f2aa3f7849f3b2f5dd0a06801899/users/717861b3356e469197cafbd14b71a02d/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:01.740051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10411a0-2c59-4940-8f21-6971e25477ac admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:01.757363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10411a0-2c59-4940-8f21-6971e25477ac admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:01.830009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:01.944340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39e13b-a254-4f8a-8d4c-918ebc7f2f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:01.946617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39e13b-a254-4f8a-8d4c-918ebc7f2f08 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:01.946617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39e13b-a254-4f8a-8d4c-918ebc7f2f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:01.946617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39e13b-a254-4f8a-8d4c-918ebc7f2f08 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:01.952175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39e13b-a254-4f8a-8d4c-918ebc7f2f08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:01.954255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39e13b-a254-4f8a-8d4c-918ebc7f2f08 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:01.993197 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1408/2814] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:01.997545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587ac32-c899-481d-bf98-751024e37b31 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:02.007059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:02.033512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587ac32-c899-481d-bf98-751024e37b31 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:02.057630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a60b3-9b54-4b7e-bb2e-256ed4218bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:02.058644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a60b3-9b54-4b7e-bb2e-256ed4218bae admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:02.058934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a60b3-9b54-4b7e-bb2e-256ed4218bae admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:02.059228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2a60b3-9b54-4b7e-bb2e-256ed4218bae admin admin] PATH_INFO: `/v3/projects/78a7f2aa3f7849f3b2f5dd0a06801899/users/717861b3356e469197cafbd14b71a02d/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:02.081673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a60b3-9b54-4b7e-bb2e-256ed4218bae admin admin] RBAC: Authorizing `identity:create_grant(project_id=78a7f2aa3f7849f3b2f5dd0a06801899, user_id=717861b3356e469197cafbd14b71a02d, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:02.105331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2a60b3-9b54-4b7e-bb2e-256ed4218bae admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:02.148378 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1407/2815] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:07:01 2022] PUT /identity/v3/projects/78a7f2aa3f7849f3b2f5dd0a06801899/users/717861b3356e469197cafbd14b71a02d/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:02.151719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71430f-685f-404a-be91-d31e5289bafc None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:02.152032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71430f-685f-404a-be91-d31e5289bafc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:02.152312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b71430f-685f-404a-be91-d31e5289bafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:02.229165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5b71430f-685f-404a-be91-d31e5289bafc None None] MFA Rules not processed for user `717861b3356e469197cafbd14b71a02d`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:02.230856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5b71430f-685f-404a-be91-d31e5289bafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:02.431480 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1409/2816] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:02 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:02.817121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a60b3-9b54-4b7e-bb2e-256ed4218bae admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:02.820325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fb2a60b3-9b54-4b7e-bb2e-256ed4218bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:02.986196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2a60b3-9b54-4b7e-bb2e-256ed4218bae admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:03.026720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd388fd-910d-4651-a5e8-6c172e99eed3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:03.029588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd388fd-910d-4651-a5e8-6c172e99eed3 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:03.031208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd388fd-910d-4651-a5e8-6c172e99eed3 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:03.032764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd388fd-910d-4651-a5e8-6c172e99eed3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:03.069232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd388fd-910d-4651-a5e8-6c172e99eed3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:03.069232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd388fd-910d-4651-a5e8-6c172e99eed3 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:03.089364 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1408/2817] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:02 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:07:03.457863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71430f-685f-404a-be91-d31e5289bafc None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:03.477000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b71430f-685f-404a-be91-d31e5289bafc None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:03.483022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed468944-4640-4976-9cae-61df1cfef3bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:03.483827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed468944-4640-4976-9cae-61df1cfef3bb service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:03.484211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed468944-4640-4976-9cae-61df1cfef3bb service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:03.484650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed468944-4640-4976-9cae-61df1cfef3bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:03.585941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed468944-4640-4976-9cae-61df1cfef3bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:03.587841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed468944-4640-4976-9cae-61df1cfef3bb service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:03.600359 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1410/2818] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:03 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:07:03.669305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd388fd-910d-4651-a5e8-6c172e99eed3 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:03.793128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd388fd-910d-4651-a5e8-6c172e99eed3 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:03.804018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c47bb-d8bb-4554-9015-a5c4378e7bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:03.805082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c47bb-d8bb-4554-9015-a5c4378e7bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:03.805361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c47bb-d8bb-4554-9015-a5c4378e7bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:03.805692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c47bb-d8bb-4554-9015-a5c4378e7bb0 admin admin] PATH_INFO: `/v3/users/5b8e5137419744f79eab638c4d3abff7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:03.806987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c47bb-d8bb-4554-9015-a5c4378e7bb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b8e5137419744f79eab638c4d3abff7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:03.809056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367c47bb-d8bb-4554-9015-a5c4378e7bb0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:04.120865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-367c47bb-d8bb-4554-9015-a5c4378e7bb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b8e5137419744f79eab638c4d3abff7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:04.125563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-367c47bb-d8bb-4554-9015-a5c4378e7bb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b8e5137419744f79eab638c4d3abff7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:04.145935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-367c47bb-d8bb-4554-9015-a5c4378e7bb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b8e5137419744f79eab638c4d3abff7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:04.339486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-367c47bb-d8bb-4554-9015-a5c4378e7bb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b8e5137419744f79eab638c4d3abff7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:04.341388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-367c47bb-d8bb-4554-9015-a5c4378e7bb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b8e5137419744f79eab638c4d3abff7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:04.364030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:04.442452 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1409/2819] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:07:03 2022] DELETE /identity/v3/users/5b8e5137419744f79eab638c4d3abff7 => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:04.442624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cfcbd-0d86-4a91-8dae-fc3fbb7525e8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:04.442624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cfcbd-0d86-4a91-8dae-fc3fbb7525e8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:04.442950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cfcbd-0d86-4a91-8dae-fc3fbb7525e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:04.491879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-717cfcbd-0d86-4a91-8dae-fc3fbb7525e8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:04.499728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-717cfcbd-0d86-4a91-8dae-fc3fbb7525e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:04.673903 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1411/2820] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:04.691087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c47bb-d8bb-4554-9015-a5c4378e7bb0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:04.692733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-367c47bb-d8bb-4554-9015-a5c4378e7bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:04.808391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367c47bb-d8bb-4554-9015-a5c4378e7bb0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:04.814265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25472395-5eee-4225-9b10-4ae4956abbbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:04.815518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25472395-5eee-4225-9b10-4ae4956abbbc service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:04.815801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25472395-5eee-4225-9b10-4ae4956abbbc service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:04.816295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25472395-5eee-4225-9b10-4ae4956abbbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:04.829145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25472395-5eee-4225-9b10-4ae4956abbbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:04.831563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25472395-5eee-4225-9b10-4ae4956abbbc service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:04.843709 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1410/2821] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:07:05.568396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:06.022518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d10aeb-bf32-4f10-934c-02c44b7ede7e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:06.022908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d10aeb-bf32-4f10-934c-02c44b7ede7e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:06.023207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d10aeb-bf32-4f10-934c-02c44b7ede7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:06.073405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c2d10aeb-bf32-4f10-934c-02c44b7ede7e None None] MFA Rules not processed for user `7b44b524592e4a57a26681767b9c4541`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:06.075668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c2d10aeb-bf32-4f10-934c-02c44b7ede7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:06.216899 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1412/2822] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:06.264780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25472395-5eee-4225-9b10-4ae4956abbbc service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:06.266268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-25472395-5eee-4225-9b10-4ae4956abbbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:06.284129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25472395-5eee-4225-9b10-4ae4956abbbc service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:06.291768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4b353-d010-490c-afed-6d66af749490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:06.292821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4b353-d010-490c-afed-6d66af749490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:06.293313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4b353-d010-490c-afed-6d66af749490 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:06.293757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4b353-d010-490c-afed-6d66af749490 admin admin] PATH_INFO: `/v3/projects/42c460c2c90640fa972c1c17fb41644a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:06.295375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b4b353-d010-490c-afed-6d66af749490 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42c460c2c90640fa972c1c17fb41644a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:06.298065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b4b353-d010-490c-afed-6d66af749490 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:06.471826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e4b4b353-d010-490c-afed-6d66af749490 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42c460c2c90640fa972c1c17fb41644a', 'request_id': 'req-e4b4b353-d010-490c-afed-6d66af749490'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:06.478139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e4b4b353-d010-490c-afed-6d66af749490 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42c460c2c90640fa972c1c17fb41644a', 'request_id': 'req-e4b4b353-d010-490c-afed-6d66af749490'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:06.577086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e4b4b353-d010-490c-afed-6d66af749490 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42c460c2c90640fa972c1c17fb41644a', 'request_id': 'req-e4b4b353-d010-490c-afed-6d66af749490'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:06.591648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e4b4b353-d010-490c-afed-6d66af749490 admin admin] The token cache is being invalidate because project 42c460c2c90640fa972c1c17fb41644a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:07:06.592229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e4b4b353-d010-490c-afed-6d66af749490 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:06.593384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e4b4b353-d010-490c-afed-6d66af749490 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:06.596450 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1411/2823] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:07:06 2022] DELETE /identity/v3/projects/42c460c2c90640fa972c1c17fb41644a => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:06.600948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d10aeb-bf32-4f10-934c-02c44b7ede7e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:06.615483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d10aeb-bf32-4f10-934c-02c44b7ede7e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:06.658786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:06.689341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e2926-a76b-41b4-aeda-8cfeefdbc83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:06.692081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e2926-a76b-41b4-aeda-8cfeefdbc83c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:06.692570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e2926-a76b-41b4-aeda-8cfeefdbc83c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:06.693818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e2926-a76b-41b4-aeda-8cfeefdbc83c admin admin] PATH_INFO: `/v3/users/717861b3356e469197cafbd14b71a02d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:06.697697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e2926-a76b-41b4-aeda-8cfeefdbc83c admin admin] RBAC: Authorizing `identity:delete_user(user_id=717861b3356e469197cafbd14b71a02d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:06.699716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e2926-a76b-41b4-aeda-8cfeefdbc83c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:06.992905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e57e2926-a76b-41b4-aeda-8cfeefdbc83c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '717861b3356e469197cafbd14b71a02d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:07.001403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e57e2926-a76b-41b4-aeda-8cfeefdbc83c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '717861b3356e469197cafbd14b71a02d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:07.002409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e57e2926-a76b-41b4-aeda-8cfeefdbc83c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '717861b3356e469197cafbd14b71a02d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:07.003153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e57e2926-a76b-41b4-aeda-8cfeefdbc83c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '717861b3356e469197cafbd14b71a02d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:07.138298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e57e2926-a76b-41b4-aeda-8cfeefdbc83c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '717861b3356e469197cafbd14b71a02d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:07.180567 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1413/2824] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:07:06 2022] DELETE /identity/v3/users/717861b3356e469197cafbd14b71a02d => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:08.147725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f61d4-3c5a-43c0-96ce-c257095badfa None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:08.152836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f61d4-3c5a-43c0-96ce-c257095badfa None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:08.152836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f61d4-3c5a-43c0-96ce-c257095badfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:08.189510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d11f61d4-3c5a-43c0-96ce-c257095badfa None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:08.192100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d11f61d4-3c5a-43c0-96ce-c257095badfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:08.282614 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1412/2825] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:07:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:08.325789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e2926-a76b-41b4-aeda-8cfeefdbc83c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:08.326770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e57e2926-a76b-41b4-aeda-8cfeefdbc83c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:08.347091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:08.401851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e2926-a76b-41b4-aeda-8cfeefdbc83c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:08.408183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7b98c-89ab-497a-a811-9c99404215f7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:08.409543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7b98c-89ab-497a-a811-9c99404215f7 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:08.409930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7b98c-89ab-497a-a811-9c99404215f7 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:08.410272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7b98c-89ab-497a-a811-9c99404215f7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:08.419829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7b98c-89ab-497a-a811-9c99404215f7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:08.421545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7b98c-89ab-497a-a811-9c99404215f7 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:08.431179 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1414/2826] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:07:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 21:07:08.497746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f61d4-3c5a-43c0-96ce-c257095badfa None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:08.499328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d11f61d4-3c5a-43c0-96ce-c257095badfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:08.591884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f61d4-3c5a-43c0-96ce-c257095badfa None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:08.597732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4aa120-b9b4-4432-8813-37366db2dc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:08.598841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4aa120-b9b4-4432-8813-37366db2dc0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:08.599270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4aa120-b9b4-4432-8813-37366db2dc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:08.601619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4aa120-b9b4-4432-8813-37366db2dc0b admin admin] PATH_INFO: `/v3/projects/78a7f2aa3f7849f3b2f5dd0a06801899` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:08.603279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4aa120-b9b4-4432-8813-37366db2dc0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=78a7f2aa3f7849f3b2f5dd0a06801899)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:08.606431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4aa120-b9b4-4432-8813-37366db2dc0b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:08.776444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dd4aa120-b9b4-4432-8813-37366db2dc0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78a7f2aa3f7849f3b2f5dd0a06801899', 'request_id': 'req-dd4aa120-b9b4-4432-8813-37366db2dc0b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:08.784711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dd4aa120-b9b4-4432-8813-37366db2dc0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78a7f2aa3f7849f3b2f5dd0a06801899', 'request_id': 'req-dd4aa120-b9b4-4432-8813-37366db2dc0b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:08.862969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dd4aa120-b9b4-4432-8813-37366db2dc0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78a7f2aa3f7849f3b2f5dd0a06801899', 'request_id': 'req-dd4aa120-b9b4-4432-8813-37366db2dc0b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:08.889796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dd4aa120-b9b4-4432-8813-37366db2dc0b admin admin] The token cache is being invalidate because project 78a7f2aa3f7849f3b2f5dd0a06801899 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:07:08.890293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dd4aa120-b9b4-4432-8813-37366db2dc0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:08.896195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dd4aa120-b9b4-4432-8813-37366db2dc0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:08.896195 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1413/2827] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:07:08 2022] DELETE /identity/v3/projects/78a7f2aa3f7849f3b2f5dd0a06801899 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:08.900099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ba150-e844-41de-8e35-ab5c1da3189b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:08.902145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ba150-e844-41de-8e35-ab5c1da3189b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:08.902577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ba150-e844-41de-8e35-ab5c1da3189b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:08.925319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cd8ba150-e844-41de-8e35-ab5c1da3189b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:08.927048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cd8ba150-e844-41de-8e35-ab5c1da3189b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:09.018815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:09.047375 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1415/2828] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:09.049479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79a5b2-e9b2-4d7d-acf8-07be0c28908d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:09.050147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79a5b2-e9b2-4d7d-acf8-07be0c28908d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:09.050740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79a5b2-e9b2-4d7d-acf8-07be0c28908d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:09.078960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ed79a5b2-e9b2-4d7d-acf8-07be0c28908d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:09.081647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ed79a5b2-e9b2-4d7d-acf8-07be0c28908d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:09.097194 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1414/2829] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:09.101394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ba150-e844-41de-8e35-ab5c1da3189b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:09.110959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ba150-e844-41de-8e35-ab5c1da3189b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:09.121540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f85965-8985-407b-90b1-e66a150bc185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:09.121540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f85965-8985-407b-90b1-e66a150bc185 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:09.121540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f85965-8985-407b-90b1-e66a150bc185 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:09.121540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f85965-8985-407b-90b1-e66a150bc185 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:09.122369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f85965-8985-407b-90b1-e66a150bc185 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:07:09.122369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f85965-8985-407b-90b1-e66a150bc185 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:09.122369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f85965-8985-407b-90b1-e66a150bc185 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:09.136563 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1416/2830] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:07:09.149601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79a5b2-e9b2-4d7d-acf8-07be0c28908d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:09.160642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79a5b2-e9b2-4d7d-acf8-07be0c28908d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:09.167310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa31eb-5b6b-42ce-9090-aa8a54eceab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:09.167996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa31eb-5b6b-42ce-9090-aa8a54eceab9 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:09.168332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa31eb-5b6b-42ce-9090-aa8a54eceab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:09.168783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaa31eb-5b6b-42ce-9090-aa8a54eceab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:09.171108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa31eb-5b6b-42ce-9090-aa8a54eceab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:09.173946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaa31eb-5b6b-42ce-9090-aa8a54eceab9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:09.315507 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1415/2831] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:07:09 2022] POST /identity/v3/projects => generated 395 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:07:09.315604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f85965-8985-407b-90b1-e66a150bc185 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:09.361477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f85965-8985-407b-90b1-e66a150bc185 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:09.538324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dad74-9887-46d0-b6bc-7664485df3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:09.539023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dad74-9887-46d0-b6bc-7664485df3ad admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:09.539452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dad74-9887-46d0-b6bc-7664485df3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:09.539915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dad74-9887-46d0-b6bc-7664485df3ad admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:09.541011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45dad74-9887-46d0-b6bc-7664485df3ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:09.543551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45dad74-9887-46d0-b6bc-7664485df3ad admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:09.650127 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1417/2832] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:07:09 2022] POST /identity/v3/users => generated 298 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:07:09.653282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa31eb-5b6b-42ce-9090-aa8a54eceab9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:09.667374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaa31eb-5b6b-42ce-9090-aa8a54eceab9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:09.682989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cb3f2-be48-4c1e-9d14-57730144bb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:09.683916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cb3f2-be48-4c1e-9d14-57730144bb27 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:09.684308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cb3f2-be48-4c1e-9d14-57730144bb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:09.685186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49cb3f2-be48-4c1e-9d14-57730144bb27 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:09.686218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49cb3f2-be48-4c1e-9d14-57730144bb27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:09.693194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49cb3f2-be48-4c1e-9d14-57730144bb27 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:09.724325 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1416/2833] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:09.726980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dad74-9887-46d0-b6bc-7664485df3ad admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:09.739321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dad74-9887-46d0-b6bc-7664485df3ad admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:09.748772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab0213-1975-489f-9a66-cc6cddfff067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:09.749622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab0213-1975-489f-9a66-cc6cddfff067 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:09.750052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab0213-1975-489f-9a66-cc6cddfff067 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:09.750458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab0213-1975-489f-9a66-cc6cddfff067 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:09.751312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab0213-1975-489f-9a66-cc6cddfff067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:09.753772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab0213-1975-489f-9a66-cc6cddfff067 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:09.772275 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1418/2834] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:09.774464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cb3f2-be48-4c1e-9d14-57730144bb27 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:09.788341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49cb3f2-be48-4c1e-9d14-57730144bb27 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:09.796401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d2af8-dae5-4a5d-a6e1-e08550af7b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:09.797761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d2af8-dae5-4a5d-a6e1-e08550af7b18 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:09.798359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d2af8-dae5-4a5d-a6e1-e08550af7b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:09.798877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577d2af8-dae5-4a5d-a6e1-e08550af7b18 admin admin] PATH_INFO: `/v3/projects/72d0adc19fe145caa50c8a0767765092/users/d1b349adf52040d1822953c96514ff7e/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:09.825222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d2af8-dae5-4a5d-a6e1-e08550af7b18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d0adc19fe145caa50c8a0767765092, user_id=d1b349adf52040d1822953c96514ff7e, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:09.829288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577d2af8-dae5-4a5d-a6e1-e08550af7b18 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:09.941719 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1417/2835] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:07:09 2022] PUT /identity/v3/projects/72d0adc19fe145caa50c8a0767765092/users/d1b349adf52040d1822953c96514ff7e/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:09.953487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a368c50-95e6-463a-8510-6a2224bb2aae None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:09.953883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a368c50-95e6-463a-8510-6a2224bb2aae None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:09.954171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a368c50-95e6-463a-8510-6a2224bb2aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:09.959044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d2af8-dae5-4a5d-a6e1-e08550af7b18 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:09.960746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-577d2af8-dae5-4a5d-a6e1-e08550af7b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:09.995497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5a368c50-95e6-463a-8510-6a2224bb2aae None None] MFA Rules not processed for user `d1b349adf52040d1822953c96514ff7e`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:09.997175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5a368c50-95e6-463a-8510-6a2224bb2aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:10.058863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577d2af8-dae5-4a5d-a6e1-e08550af7b18 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:10.073229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc5e08-89c6-4979-856f-1a29e4a0f30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:10.074846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc5e08-89c6-4979-856f-1a29e4a0f30d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:10.075390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc5e08-89c6-4979-856f-1a29e4a0f30d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:10.076251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc5e08-89c6-4979-856f-1a29e4a0f30d admin admin] PATH_INFO: `/v3/users/478d1609aa114d16b338e07d2d4c05c9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:10.078035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc5e08-89c6-4979-856f-1a29e4a0f30d admin admin] RBAC: Authorizing `identity:delete_user(user_id=478d1609aa114d16b338e07d2d4c05c9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:10.080963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc5e08-89c6-4979-856f-1a29e4a0f30d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:10.141265 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1419/2836] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:10.145827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a368c50-95e6-463a-8510-6a2224bb2aae None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:10.156416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a368c50-95e6-463a-8510-6a2224bb2aae None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:10.161587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780159c-ea28-4bf2-993a-3da959de8ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:10.162294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780159c-ea28-4bf2-993a-3da959de8ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:10.162581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780159c-ea28-4bf2-993a-3da959de8ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:10.162865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780159c-ea28-4bf2-993a-3da959de8ca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:10.163564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780159c-ea28-4bf2-993a-3da959de8ca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:10.165489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780159c-ea28-4bf2-993a-3da959de8ca1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:10.279871 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1420/2837] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:07:10 2022] POST /identity/v3/projects => generated 395 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:07:10.284185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780159c-ea28-4bf2-993a-3da959de8ca1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:10.359507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780159c-ea28-4bf2-993a-3da959de8ca1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:10.365925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6cb71-d7e0-40d4-b485-aac612d41585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:10.366890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6cb71-d7e0-40d4-b485-aac612d41585 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:10.367692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6cb71-d7e0-40d4-b485-aac612d41585 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:10.368066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6cb71-d7e0-40d4-b485-aac612d41585 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:10.369869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6cb71-d7e0-40d4-b485-aac612d41585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:10.372174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6cb71-d7e0-40d4-b485-aac612d41585 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:10.374143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5fdc5e08-89c6-4979-856f-1a29e4a0f30d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '478d1609aa114d16b338e07d2d4c05c9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:10.374966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5fdc5e08-89c6-4979-856f-1a29e4a0f30d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '478d1609aa114d16b338e07d2d4c05c9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:10.384769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5fdc5e08-89c6-4979-856f-1a29e4a0f30d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '478d1609aa114d16b338e07d2d4c05c9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:10.494156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5fdc5e08-89c6-4979-856f-1a29e4a0f30d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '478d1609aa114d16b338e07d2d4c05c9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:10.494983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5fdc5e08-89c6-4979-856f-1a29e4a0f30d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '478d1609aa114d16b338e07d2d4c05c9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:10.501693 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1421/2838] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:07:10 2022] POST /identity/v3/users => generated 304 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:07:10.506380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6cb71-d7e0-40d4-b485-aac612d41585 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:10.508121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d8f6cb71-d7e0-40d4-b485-aac612d41585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:10.518801 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1418/2839] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:07:09 2022] DELETE /identity/v3/users/478d1609aa114d16b338e07d2d4c05c9 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:10.520396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc58d9e-0dc1-4ab4-9330-f9c1b538067e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:10.520789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc58d9e-0dc1-4ab4-9330-f9c1b538067e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:10.521358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc58d9e-0dc1-4ab4-9330-f9c1b538067e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:10.528491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6cb71-d7e0-40d4-b485-aac612d41585 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:10.539760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ddc58d9e-0dc1-4ab4-9330-f9c1b538067e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:10.541241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ddc58d9e-0dc1-4ab4-9330-f9c1b538067e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:10.596825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15a1e5-29f7-4def-92da-9ac2463214fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:10.597504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15a1e5-29f7-4def-92da-9ac2463214fe admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:10.597789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15a1e5-29f7-4def-92da-9ac2463214fe admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:10.598107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e15a1e5-29f7-4def-92da-9ac2463214fe admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:10.598847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15a1e5-29f7-4def-92da-9ac2463214fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:10.600455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e15a1e5-29f7-4def-92da-9ac2463214fe admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:10.608477 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1419/2840] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:10.610456 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1422/2841] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:10.614468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc58d9e-0dc1-4ab4-9330-f9c1b538067e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:10.618766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15a1e5-29f7-4def-92da-9ac2463214fe admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:10.620536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8e15a1e5-29f7-4def-92da-9ac2463214fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:10.626240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc58d9e-0dc1-4ab4-9330-f9c1b538067e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:10.633298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdfd8f-705c-471a-a3ac-44412f336aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:10.633466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdfd8f-705c-471a-a3ac-44412f336aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:10.633709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdfd8f-705c-471a-a3ac-44412f336aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:10.634027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fdfd8f-705c-471a-a3ac-44412f336aeb admin admin] PATH_INFO: `/v3/projects/030d24620952431f8623548f8f1a51f6/users/5cbd272df98c4cf9b8860a35a13bbbd7/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:10.653157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdfd8f-705c-471a-a3ac-44412f336aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=030d24620952431f8623548f8f1a51f6, user_id=5cbd272df98c4cf9b8860a35a13bbbd7, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:10.657801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fdfd8f-705c-471a-a3ac-44412f336aeb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:10.704582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e15a1e5-29f7-4def-92da-9ac2463214fe admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:10.710762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91872b6b-497f-41b6-9f4b-f91be09c58b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:10.711794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91872b6b-497f-41b6-9f4b-f91be09c58b3 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:10.712178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91872b6b-497f-41b6-9f4b-f91be09c58b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:10.712740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91872b6b-497f-41b6-9f4b-f91be09c58b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:10.726534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91872b6b-497f-41b6-9f4b-f91be09c58b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:10.730533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91872b6b-497f-41b6-9f4b-f91be09c58b3 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:10.738732 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1423/2842] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:07:10.757317 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1420/2843] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:07:10 2022] PUT /identity/v3/projects/030d24620952431f8623548f8f1a51f6/users/5cbd272df98c4cf9b8860a35a13bbbd7/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:07:10.761363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91872b6b-497f-41b6-9f4b-f91be09c58b3 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:10.773095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91872b6b-497f-41b6-9f4b-f91be09c58b3 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:10.861530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f9d4af-f9a6-4db5-a8c1-83b695a603b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:10.862545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9d4af-f9a6-4db5-a8c1-83b695a603b4 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:10.862987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9d4af-f9a6-4db5-a8c1-83b695a603b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:10.863417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f9d4af-f9a6-4db5-a8c1-83b695a603b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:10.864353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9d4af-f9a6-4db5-a8c1-83b695a603b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:10.866189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f9d4af-f9a6-4db5-a8c1-83b695a603b4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:10.884786 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1424/2844] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:10.891285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdfd8f-705c-471a-a3ac-44412f336aeb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:10.908618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fdfd8f-705c-471a-a3ac-44412f336aeb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:10.920501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678057c8-3d6e-4abb-b944-31582009290d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:10.924735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678057c8-3d6e-4abb-b944-31582009290d admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:10.925087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678057c8-3d6e-4abb-b944-31582009290d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:10.925436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678057c8-3d6e-4abb-b944-31582009290d admin admin] PATH_INFO: `/v3/projects/030d24620952431f8623548f8f1a51f6/users/5cbd272df98c4cf9b8860a35a13bbbd7/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:10.928437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678057c8-3d6e-4abb-b944-31582009290d admin admin] RBAC: Authorizing `identity:create_grant(project_id=030d24620952431f8623548f8f1a51f6, user_id=5cbd272df98c4cf9b8860a35a13bbbd7, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:10.933772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678057c8-3d6e-4abb-b944-31582009290d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:10.975618 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1421/2845] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:07:10 2022] PUT /identity/v3/projects/030d24620952431f8623548f8f1a51f6/users/5cbd272df98c4cf9b8860a35a13bbbd7/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:10.976853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74445c-ba6c-4453-aae9-16e83c2cf905 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:10.977148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74445c-ba6c-4453-aae9-16e83c2cf905 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:10.977431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74445c-ba6c-4453-aae9-16e83c2cf905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:11.011861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cd74445c-ba6c-4453-aae9-16e83c2cf905 None None] MFA Rules not processed for user `5cbd272df98c4cf9b8860a35a13bbbd7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:11.013505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cd74445c-ba6c-4453-aae9-16e83c2cf905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:11.152914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e2cee-af07-4078-9ced-a7b119d0ad27 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:11.153184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e2cee-af07-4078-9ced-a7b119d0ad27 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:11.156293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41e2cee-af07-4078-9ced-a7b119d0ad27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:11.156394 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1425/2846] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:11.207724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a41e2cee-af07-4078-9ced-a7b119d0ad27 None None] MFA Rules not processed for user `d1b349adf52040d1822953c96514ff7e`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:11.209287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a41e2cee-af07-4078-9ced-a7b119d0ad27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:11.241368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8a4f1-f00e-466c-86b8-ac8a0cd00241 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:11.241777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8a4f1-f00e-466c-86b8-ac8a0cd00241 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:11.242014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc8a4f1-f00e-466c-86b8-ac8a0cd00241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:11.300111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ebc8a4f1-f00e-466c-86b8-ac8a0cd00241 None None] MFA Rules not processed for user `c14b965da64c40e382c69050ba01b183`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:11.312702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ebc8a4f1-f00e-466c-86b8-ac8a0cd00241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:11.372599 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1422/2847] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:11.373826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e2cee-af07-4078-9ced-a7b119d0ad27 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:11.375427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a41e2cee-af07-4078-9ced-a7b119d0ad27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:11.464446 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1426/2848] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:11.466674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8a4f1-f00e-466c-86b8-ac8a0cd00241 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:11.478746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ebc8a4f1-f00e-466c-86b8-ac8a0cd00241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:11.511953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41e2cee-af07-4078-9ced-a7b119d0ad27 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:11.527184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b9020-86d3-4d09-a985-f7773667b78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:11.527184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b9020-86d3-4d09-a985-f7773667b78c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:11.527184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b9020-86d3-4d09-a985-f7773667b78c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:11.527184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b9020-86d3-4d09-a985-f7773667b78c admin admin] PATH_INFO: `/v3/projects/56e1dc5556624e2fb8de8a030c34c334` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:11.527746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b9020-86d3-4d09-a985-f7773667b78c admin admin] RBAC: Authorizing `identity:delete_project(project_id=56e1dc5556624e2fb8de8a030c34c334)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:11.527746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b9020-86d3-4d09-a985-f7773667b78c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:11.595356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:11.626523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc8a4f1-f00e-466c-86b8-ac8a0cd00241 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:11.633354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8db40-e8f0-4100-abf3-601a04f1e68d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:11.634537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8db40-e8f0-4100-abf3-601a04f1e68d service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:11.634902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8db40-e8f0-4100-abf3-601a04f1e68d service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:11.635256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf8db40-e8f0-4100-abf3-601a04f1e68d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:11.637639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1cf8db40-e8f0-4100-abf3-601a04f1e68d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:11.652174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8db40-e8f0-4100-abf3-601a04f1e68d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:11.652174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf8db40-e8f0-4100-abf3-601a04f1e68d service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:11.662256 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1427/2849] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:07:11.716558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-779b9020-86d3-4d09-a985-f7773667b78c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56e1dc5556624e2fb8de8a030c34c334', 'request_id': 'req-779b9020-86d3-4d09-a985-f7773667b78c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:11.723137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-779b9020-86d3-4d09-a985-f7773667b78c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56e1dc5556624e2fb8de8a030c34c334', 'request_id': 'req-779b9020-86d3-4d09-a985-f7773667b78c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:11.766808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-779b9020-86d3-4d09-a985-f7773667b78c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56e1dc5556624e2fb8de8a030c34c334', 'request_id': 'req-779b9020-86d3-4d09-a985-f7773667b78c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:11.780457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-779b9020-86d3-4d09-a985-f7773667b78c admin admin] The token cache is being invalidate because project 56e1dc5556624e2fb8de8a030c34c334 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:07:11.780701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-779b9020-86d3-4d09-a985-f7773667b78c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:11.781402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-779b9020-86d3-4d09-a985-f7773667b78c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:11.787140 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1423/2850] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:07:11 2022] DELETE /identity/v3/projects/56e1dc5556624e2fb8de8a030c34c334 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:11.788405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8db40-e8f0-4100-abf3-601a04f1e68d service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:11.809208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf8db40-e8f0-4100-abf3-601a04f1e68d service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:11.883656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fbe9a9-b4eb-4c26-b53b-c0d04bc7a735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:11.884614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbe9a9-b4eb-4c26-b53b-c0d04bc7a735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:11.884900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbe9a9-b4eb-4c26-b53b-c0d04bc7a735 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:11.885180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fbe9a9-b4eb-4c26-b53b-c0d04bc7a735 admin admin] PATH_INFO: `/v3/users/ebbb3158771b420db1cc0827263ead5f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:11.886576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fbe9a9-b4eb-4c26-b53b-c0d04bc7a735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebbb3158771b420db1cc0827263ead5f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:11.889394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fbe9a9-b4eb-4c26-b53b-c0d04bc7a735 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:12.124132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c6fbe9a9-b4eb-4c26-b53b-c0d04bc7a735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebbb3158771b420db1cc0827263ead5f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:12.132874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c6fbe9a9-b4eb-4c26-b53b-c0d04bc7a735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebbb3158771b420db1cc0827263ead5f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:12.133660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c6fbe9a9-b4eb-4c26-b53b-c0d04bc7a735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebbb3158771b420db1cc0827263ead5f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:12.134381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c6fbe9a9-b4eb-4c26-b53b-c0d04bc7a735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebbb3158771b420db1cc0827263ead5f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:12.298262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c6fbe9a9-b4eb-4c26-b53b-c0d04bc7a735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebbb3158771b420db1cc0827263ead5f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:12.346418 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1428/2851] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:07:11 2022] DELETE /identity/v3/users/ebbb3158771b420db1cc0827263ead5f => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:12.452684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:12.958995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b9020-86d3-4d09-a985-f7773667b78c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:12.960513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-779b9020-86d3-4d09-a985-f7773667b78c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:12.978330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b9020-86d3-4d09-a985-f7773667b78c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:13.043360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5ea16-c2b1-4639-9afd-b23e95329bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:13.044402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5ea16-c2b1-4639-9afd-b23e95329bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:13.044703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5ea16-c2b1-4639-9afd-b23e95329bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:13.044981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5ea16-c2b1-4639-9afd-b23e95329bf9 admin admin] PATH_INFO: `/v3/projects/1283a7c356b74d8086558ea4217b0d20` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:13.046351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5ea16-c2b1-4639-9afd-b23e95329bf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1283a7c356b74d8086558ea4217b0d20)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:13.050013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5ea16-c2b1-4639-9afd-b23e95329bf9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:13.225797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f0e5ea16-c2b1-4639-9afd-b23e95329bf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1283a7c356b74d8086558ea4217b0d20', 'request_id': 'req-f0e5ea16-c2b1-4639-9afd-b23e95329bf9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:13.233818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f0e5ea16-c2b1-4639-9afd-b23e95329bf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1283a7c356b74d8086558ea4217b0d20', 'request_id': 'req-f0e5ea16-c2b1-4639-9afd-b23e95329bf9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:13.320926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f0e5ea16-c2b1-4639-9afd-b23e95329bf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1283a7c356b74d8086558ea4217b0d20', 'request_id': 'req-f0e5ea16-c2b1-4639-9afd-b23e95329bf9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:13.334203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f0e5ea16-c2b1-4639-9afd-b23e95329bf9 admin admin] The token cache is being invalidate because project 1283a7c356b74d8086558ea4217b0d20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:07:13.334532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f0e5ea16-c2b1-4639-9afd-b23e95329bf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:13.335202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f0e5ea16-c2b1-4639-9afd-b23e95329bf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:13.336766 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1424/2852] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:07:12 2022] DELETE /identity/v3/projects/1283a7c356b74d8086558ea4217b0d20 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:13.344018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90caefa1-4a1a-4aab-b76f-ba1d83fd5a4d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:13.344509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90caefa1-4a1a-4aab-b76f-ba1d83fd5a4d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:13.344737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90caefa1-4a1a-4aab-b76f-ba1d83fd5a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:13.364273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-90caefa1-4a1a-4aab-b76f-ba1d83fd5a4d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:13.365600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-90caefa1-4a1a-4aab-b76f-ba1d83fd5a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:13.425665 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1429/2853] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:13.434374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a9f75-783b-423d-bf42-d88fedc9dc6f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:13.434788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a9f75-783b-423d-bf42-d88fedc9dc6f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:13.435071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a9f75-783b-423d-bf42-d88fedc9dc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:13.455765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-450a9f75-783b-423d-bf42-d88fedc9dc6f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:13.457637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-450a9f75-783b-423d-bf42-d88fedc9dc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:13.466225 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1425/2854] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:13.470519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90caefa1-4a1a-4aab-b76f-ba1d83fd5a4d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:13.480735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90caefa1-4a1a-4aab-b76f-ba1d83fd5a4d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:13.485383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0b4f4-1b09-46c7-a970-96a2e7028324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:13.486071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0b4f4-1b09-46c7-a970-96a2e7028324 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:13.486357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0b4f4-1b09-46c7-a970-96a2e7028324 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:13.486665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0b4f4-1b09-46c7-a970-96a2e7028324 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:13.487421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0b4f4-1b09-46c7-a970-96a2e7028324 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:07:13.487733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0b4f4-1b09-46c7-a970-96a2e7028324 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:13.489321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0b4f4-1b09-46c7-a970-96a2e7028324 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:13.506691 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1430/2855] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:07:13.507580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a9f75-783b-423d-bf42-d88fedc9dc6f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:13.517454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a9f75-783b-423d-bf42-d88fedc9dc6f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:13.523488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eedb8d-d131-439e-9557-c2bda741d4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:13.524144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eedb8d-d131-439e-9557-c2bda741d4ab admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:13.524428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eedb8d-d131-439e-9557-c2bda741d4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:13.524700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eedb8d-d131-439e-9557-c2bda741d4ab admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:13.525396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eedb8d-d131-439e-9557-c2bda741d4ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:13.528067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eedb8d-d131-439e-9557-c2bda741d4ab admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:13.700440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0b4f4-1b09-46c7-a970-96a2e7028324 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:13.706559 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1426/2856] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:07:13 2022] POST /identity/v3/projects => generated 385 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:07:13.712117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0b4f4-1b09-46c7-a970-96a2e7028324 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:13.789903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa38691-8633-4aa3-a5ff-77898ebc7d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:13.792111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa38691-8633-4aa3-a5ff-77898ebc7d6d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:13.792455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa38691-8633-4aa3-a5ff-77898ebc7d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:13.792939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa38691-8633-4aa3-a5ff-77898ebc7d6d admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:13.793697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa38691-8633-4aa3-a5ff-77898ebc7d6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:13.796006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa38691-8633-4aa3-a5ff-77898ebc7d6d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:13.906541 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1431/2857] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:07:13 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:07:13.911360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eedb8d-d131-439e-9557-c2bda741d4ab admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:13.929495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eedb8d-d131-439e-9557-c2bda741d4ab admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:13.936776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e06a1-6dd3-41a3-a1fa-33750de9758f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:13.937549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e06a1-6dd3-41a3-a1fa-33750de9758f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:13.937978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e06a1-6dd3-41a3-a1fa-33750de9758f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:13.938350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e06a1-6dd3-41a3-a1fa-33750de9758f admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:13.939310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e06a1-6dd3-41a3-a1fa-33750de9758f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:13.941621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e06a1-6dd3-41a3-a1fa-33750de9758f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:13.954869 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1427/2858] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:13.959134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa38691-8633-4aa3-a5ff-77898ebc7d6d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:13.976256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa38691-8633-4aa3-a5ff-77898ebc7d6d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:13.981088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efcb126-866c-436e-9d91-2484763f70bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:13.981790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcb126-866c-436e-9d91-2484763f70bf admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:13.986155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcb126-866c-436e-9d91-2484763f70bf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:13.986155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcb126-866c-436e-9d91-2484763f70bf admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:13.986155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efcb126-866c-436e-9d91-2484763f70bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:13.986155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efcb126-866c-436e-9d91-2484763f70bf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:13.996104 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1432/2859] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:14.000311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e06a1-6dd3-41a3-a1fa-33750de9758f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:14.011716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e06a1-6dd3-41a3-a1fa-33750de9758f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:14.018364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4581b14-f4b2-4bcb-8a23-8f0f792fcc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:14.019452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4581b14-f4b2-4bcb-8a23-8f0f792fcc87 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:14.019739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4581b14-f4b2-4bcb-8a23-8f0f792fcc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:14.020016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4581b14-f4b2-4bcb-8a23-8f0f792fcc87 admin admin] PATH_INFO: `/v3/projects/f7a9c4408ec1432cb05c5256657886d4/users/be94d10c650645658c755b7fc1595370/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:14.036629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4581b14-f4b2-4bcb-8a23-8f0f792fcc87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7a9c4408ec1432cb05c5256657886d4, user_id=be94d10c650645658c755b7fc1595370, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:14.040303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4581b14-f4b2-4bcb-8a23-8f0f792fcc87 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:14.098527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:14.106983 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1428/2860] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:07:13 2022] PUT /identity/v3/projects/f7a9c4408ec1432cb05c5256657886d4/users/be94d10c650645658c755b7fc1595370/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:14.113582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149de35-edfc-4976-80a1-677d9c551ad7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:14.113918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149de35-edfc-4976-80a1-677d9c551ad7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:14.114201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149de35-edfc-4976-80a1-677d9c551ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:14.133387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8149de35-edfc-4976-80a1-677d9c551ad7 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:14.134654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8149de35-edfc-4976-80a1-677d9c551ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:14.193746 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1433/2861] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:14.204821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4581b14-f4b2-4bcb-8a23-8f0f792fcc87 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:14.206551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e4581b14-f4b2-4bcb-8a23-8f0f792fcc87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:14.264180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4581b14-f4b2-4bcb-8a23-8f0f792fcc87 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:14.268097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7f9af-ac22-4e54-9538-4dd1b3e72867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:14.270660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7f9af-ac22-4e54-9538-4dd1b3e72867 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:14.270660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7f9af-ac22-4e54-9538-4dd1b3e72867 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:14.270660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7f9af-ac22-4e54-9538-4dd1b3e72867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:14.278814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7f9af-ac22-4e54-9538-4dd1b3e72867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:14.280255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7f9af-ac22-4e54-9538-4dd1b3e72867 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:14.290150 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1429/2862] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:07:16.215626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511f4c4-6cfa-4b0f-aaf7-f967b1c3656b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:16.216137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511f4c4-6cfa-4b0f-aaf7-f967b1c3656b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:16.216364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d511f4c4-6cfa-4b0f-aaf7-f967b1c3656b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:16.248744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d511f4c4-6cfa-4b0f-aaf7-f967b1c3656b None None] MFA Rules not processed for user `be94d10c650645658c755b7fc1595370`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:16.250046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d511f4c4-6cfa-4b0f-aaf7-f967b1c3656b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:16.254616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:16.338771 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1434/2863] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:16.342842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7f9af-ac22-4e54-9538-4dd1b3e72867 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:16.354501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7f9af-ac22-4e54-9538-4dd1b3e72867 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:16.360368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414c733-c692-47e1-aab0-4af9c03f1d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:16.362232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414c733-c692-47e1-aab0-4af9c03f1d9f admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:16.362744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414c733-c692-47e1-aab0-4af9c03f1d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:16.363237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414c733-c692-47e1-aab0-4af9c03f1d9f admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:16.364395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414c733-c692-47e1-aab0-4af9c03f1d9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:16.366988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414c733-c692-47e1-aab0-4af9c03f1d9f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:16.409350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:16.498847 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1430/2864] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:07:16 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:07:16.504065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511f4c4-6cfa-4b0f-aaf7-f967b1c3656b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:16.519067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d511f4c4-6cfa-4b0f-aaf7-f967b1c3656b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:16.606231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44943b7d-9276-4531-819e-86cea8386ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:16.607949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44943b7d-9276-4531-819e-86cea8386ef4 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:16.608518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44943b7d-9276-4531-819e-86cea8386ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:16.609814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44943b7d-9276-4531-819e-86cea8386ef4 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:16.610808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44943b7d-9276-4531-819e-86cea8386ef4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:16.614163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44943b7d-9276-4531-819e-86cea8386ef4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:16.757497 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1435/2865] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:07:16 2022] POST /identity/v3/users => generated 298 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:07:16.763854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414c733-c692-47e1-aab0-4af9c03f1d9f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:16.780319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414c733-c692-47e1-aab0-4af9c03f1d9f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:16.794188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c928b-2785-44b7-9d03-92dbfef43146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:16.795191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c928b-2785-44b7-9d03-92dbfef43146 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:16.795920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c928b-2785-44b7-9d03-92dbfef43146 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:16.796388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c928b-2785-44b7-9d03-92dbfef43146 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:16.796605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39314a7a-8e13-48e9-87d8-58555ed35888 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:16.796764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39314a7a-8e13-48e9-87d8-58555ed35888 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:16.797054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39314a7a-8e13-48e9-87d8-58555ed35888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:16.797800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c928b-2785-44b7-9d03-92dbfef43146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:16.800739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c928b-2785-44b7-9d03-92dbfef43146 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:16.817763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-39314a7a-8e13-48e9-87d8-58555ed35888 None None] MFA Rules not processed for user `5e54b51913be4a2bab74aae347785727`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:16.819329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-39314a7a-8e13-48e9-87d8-58555ed35888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:16.828499 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1431/2866] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:16.829663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c928b-2785-44b7-9d03-92dbfef43146 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:16.841581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c928b-2785-44b7-9d03-92dbfef43146 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:16.847374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd59cc3-ff2f-4f79-8a6e-32a9720d4e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:16.848284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd59cc3-ff2f-4f79-8a6e-32a9720d4e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:16.848570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd59cc3-ff2f-4f79-8a6e-32a9720d4e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:16.848851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd59cc3-ff2f-4f79-8a6e-32a9720d4e09 admin admin] PATH_INFO: `/v3/projects/dcf954165a95433eb89884d6dc4c7feb/users/3b3b0064f12e43cd8069165fe4c370bc/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:16.878537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd59cc3-ff2f-4f79-8a6e-32a9720d4e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcf954165a95433eb89884d6dc4c7feb, user_id=3b3b0064f12e43cd8069165fe4c370bc, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:16.891324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd59cc3-ff2f-4f79-8a6e-32a9720d4e09 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:16.905097 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1436/2867] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:07:16.990928 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1432/2868] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:07:16 2022] PUT /identity/v3/projects/dcf954165a95433eb89884d6dc4c7feb/users/3b3b0064f12e43cd8069165fe4c370bc/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:16.995118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39314a7a-8e13-48e9-87d8-58555ed35888 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:17.012439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39314a7a-8e13-48e9-87d8-58555ed35888 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:17.036626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:17.111723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837be05-ce0a-4c19-b148-c881a3e1ed60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:17.112470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837be05-ce0a-4c19-b148-c881a3e1ed60 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:17.115885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837be05-ce0a-4c19-b148-c881a3e1ed60 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:17.115885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837be05-ce0a-4c19-b148-c881a3e1ed60 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:17.115885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837be05-ce0a-4c19-b148-c881a3e1ed60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:17.116128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837be05-ce0a-4c19-b148-c881a3e1ed60 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:17.129318 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1437/2869] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:17.134656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd59cc3-ff2f-4f79-8a6e-32a9720d4e09 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:17.154603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd59cc3-ff2f-4f79-8a6e-32a9720d4e09 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:17.166647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b6f0c-c43f-4027-b35f-e40043676d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:17.169998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b6f0c-c43f-4027-b35f-e40043676d40 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:17.169998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b6f0c-c43f-4027-b35f-e40043676d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:17.169998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b6f0c-c43f-4027-b35f-e40043676d40 admin admin] PATH_INFO: `/v3/projects/dcf954165a95433eb89884d6dc4c7feb/users/3b3b0064f12e43cd8069165fe4c370bc/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:17.174770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b6f0c-c43f-4027-b35f-e40043676d40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcf954165a95433eb89884d6dc4c7feb, user_id=3b3b0064f12e43cd8069165fe4c370bc, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:17.179962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b6f0c-c43f-4027-b35f-e40043676d40 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:17.211620 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1433/2870] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:07:17 2022] PUT /identity/v3/projects/dcf954165a95433eb89884d6dc4c7feb/users/3b3b0064f12e43cd8069165fe4c370bc/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:17.915877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:19.807567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245fa97-5754-4d9d-b208-4d4792c397e2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:19.808062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245fa97-5754-4d9d-b208-4d4792c397e2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:19.808175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245fa97-5754-4d9d-b208-4d4792c397e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:19.842265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5245fa97-5754-4d9d-b208-4d4792c397e2 None None] MFA Rules not processed for user `3b3b0064f12e43cd8069165fe4c370bc`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:19.843508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5245fa97-5754-4d9d-b208-4d4792c397e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:19.969198 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1438/2871] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:19.980870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8c6de-15fa-4f4c-87d1-bba4ea210660 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:19.981811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8c6de-15fa-4f4c-87d1-bba4ea210660 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:19.982146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a8c6de-15fa-4f4c-87d1-bba4ea210660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:20.000938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b5a8c6de-15fa-4f4c-87d1-bba4ea210660 None None] MFA Rules not processed for user `be94d10c650645658c755b7fc1595370`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:20.004875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b5a8c6de-15fa-4f4c-87d1-bba4ea210660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:20.023997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:20.061877 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1434/2872] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:20.081009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245fa97-5754-4d9d-b208-4d4792c397e2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:20.082445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5245fa97-5754-4d9d-b208-4d4792c397e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:20.159324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245fa97-5754-4d9d-b208-4d4792c397e2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:20.165015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d2847-20d5-4a51-a8a0-3126fd3ca964 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:20.165671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d2847-20d5-4a51-a8a0-3126fd3ca964 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:20.165993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d2847-20d5-4a51-a8a0-3126fd3ca964 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:20.166272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d2847-20d5-4a51-a8a0-3126fd3ca964 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:20.174451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d2847-20d5-4a51-a8a0-3126fd3ca964 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:20.176034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d2847-20d5-4a51-a8a0-3126fd3ca964 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:20.184885 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1439/2873] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:07:21.944080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:30.511288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:31.188557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8c6de-15fa-4f4c-87d1-bba4ea210660 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:31.194530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b5a8c6de-15fa-4f4c-87d1-bba4ea210660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:31.288465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a8c6de-15fa-4f4c-87d1-bba4ea210660 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:31.293662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41fb436-b670-4efa-83f3-d8197b470c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:31.294737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fb436-b670-4efa-83f3-d8197b470c6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:31.295156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fb436-b670-4efa-83f3-d8197b470c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:31.295541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41fb436-b670-4efa-83f3-d8197b470c6f admin admin] PATH_INFO: `/v3/users/dc48535b672d4d1190ecfefe6cb02720` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:31.296966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41fb436-b670-4efa-83f3-d8197b470c6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc48535b672d4d1190ecfefe6cb02720)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:31.299985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41fb436-b670-4efa-83f3-d8197b470c6f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:31.533898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b41fb436-b670-4efa-83f3-d8197b470c6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc48535b672d4d1190ecfefe6cb02720'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:31.535817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b41fb436-b670-4efa-83f3-d8197b470c6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc48535b672d4d1190ecfefe6cb02720'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:31.546935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b41fb436-b670-4efa-83f3-d8197b470c6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc48535b672d4d1190ecfefe6cb02720'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:31.615983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b41fb436-b670-4efa-83f3-d8197b470c6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc48535b672d4d1190ecfefe6cb02720'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:31.617978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b41fb436-b670-4efa-83f3-d8197b470c6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc48535b672d4d1190ecfefe6cb02720'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:31.643228 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1435/2874] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:07:31 2022] DELETE /identity/v3/users/dc48535b672d4d1190ecfefe6cb02720 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:31.716672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4a328-f342-4215-b1f3-6782ae7cb454 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:31.727486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4a328-f342-4215-b1f3-6782ae7cb454 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:31.727486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4a328-f342-4215-b1f3-6782ae7cb454 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:31.727486 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1440/2875] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:07:31.728299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a1396-0a78-455b-b12f-9e110de3e3e3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:31.728597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a1396-0a78-455b-b12f-9e110de3e3e3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:31.728881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a1396-0a78-455b-b12f-9e110de3e3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:31.755284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ff9a1396-0a78-455b-b12f-9e110de3e3e3 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:31.756944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ff9a1396-0a78-455b-b12f-9e110de3e3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:31.868891 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1436/2876] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:07:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:07:31.905140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4a328-f342-4215-b1f3-6782ae7cb454 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:31.907228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c8b4a328-f342-4215-b1f3-6782ae7cb454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:31.925686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4a328-f342-4215-b1f3-6782ae7cb454 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:31.946494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd01c4-5c21-4534-afb1-9a7dfafa636a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:31.946494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd01c4-5c21-4534-afb1-9a7dfafa636a service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:31.946494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd01c4-5c21-4534-afb1-9a7dfafa636a service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:31.946494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd01c4-5c21-4534-afb1-9a7dfafa636a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:31.947481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd01c4-5c21-4534-afb1-9a7dfafa636a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:31.947481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cd01c4-5c21-4534-afb1-9a7dfafa636a service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:31.970512 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1441/2877] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:07:32.213252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a1396-0a78-455b-b12f-9e110de3e3e3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:32.214777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ff9a1396-0a78-455b-b12f-9e110de3e3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:32.235884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:32.479991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a1396-0a78-455b-b12f-9e110de3e3e3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:32.506375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981eb87e-973b-43c3-8ecc-f1df9360337d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:32.509436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981eb87e-973b-43c3-8ecc-f1df9360337d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:32.511072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981eb87e-973b-43c3-8ecc-f1df9360337d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:32.511556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981eb87e-973b-43c3-8ecc-f1df9360337d admin admin] PATH_INFO: `/v3/projects/ab0c1d6a9fc44de4af61822849b3bf8d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:32.515464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981eb87e-973b-43c3-8ecc-f1df9360337d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab0c1d6a9fc44de4af61822849b3bf8d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:32.519218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981eb87e-973b-43c3-8ecc-f1df9360337d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:32.727540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-981eb87e-973b-43c3-8ecc-f1df9360337d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab0c1d6a9fc44de4af61822849b3bf8d', 'request_id': 'req-981eb87e-973b-43c3-8ecc-f1df9360337d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:32.750271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-981eb87e-973b-43c3-8ecc-f1df9360337d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab0c1d6a9fc44de4af61822849b3bf8d', 'request_id': 'req-981eb87e-973b-43c3-8ecc-f1df9360337d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:32.817376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85d795-ee41-4b3a-99de-f39deae78290 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:32.817885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85d795-ee41-4b3a-99de-f39deae78290 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:32.817975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85d795-ee41-4b3a-99de-f39deae78290 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:32.819370 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1442/2878] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:07:32.830341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-981eb87e-973b-43c3-8ecc-f1df9360337d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab0c1d6a9fc44de4af61822849b3bf8d', 'request_id': 'req-981eb87e-973b-43c3-8ecc-f1df9360337d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:32.838991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338aa7df-008b-438e-bd85-f5a5aa95bc97 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:32.839285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338aa7df-008b-438e-bd85-f5a5aa95bc97 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:32.841657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338aa7df-008b-438e-bd85-f5a5aa95bc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:32.847529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-981eb87e-973b-43c3-8ecc-f1df9360337d admin admin] The token cache is being invalidate because project ab0c1d6a9fc44de4af61822849b3bf8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:07:32.847821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-981eb87e-973b-43c3-8ecc-f1df9360337d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:32.848656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-981eb87e-973b-43c3-8ecc-f1df9360337d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:32.849047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:32.855908 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1437/2879] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:07:32 2022] DELETE /identity/v3/projects/ab0c1d6a9fc44de4af61822849b3bf8d => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:32.857761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17fd43-b33a-4b0d-842b-1fac80b319e4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:32.858234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17fd43-b33a-4b0d-842b-1fac80b319e4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:32.858687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c17fd43-b33a-4b0d-842b-1fac80b319e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:32.867967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-338aa7df-008b-438e-bd85-f5a5aa95bc97 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:32.869631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-338aa7df-008b-438e-bd85-f5a5aa95bc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:32.881970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5c17fd43-b33a-4b0d-842b-1fac80b319e4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:32.884199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5c17fd43-b33a-4b0d-842b-1fac80b319e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:32.966939 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1438/2880] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:32.967810 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1443/2881] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:07:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 28 21:07:32.969230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd212c5b-f26a-415a-84fa-e62f3d7733bf None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:32.969596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd212c5b-f26a-415a-84fa-e62f3d7733bf None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:32.970052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd212c5b-f26a-415a-84fa-e62f3d7733bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:32.982454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338aa7df-008b-438e-bd85-f5a5aa95bc97 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:32.983988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-338aa7df-008b-438e-bd85-f5a5aa95bc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:32.995383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bd212c5b-f26a-415a-84fa-e62f3d7733bf None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:32.997100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bd212c5b-f26a-415a-84fa-e62f3d7733bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:32.998676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338aa7df-008b-438e-bd85-f5a5aa95bc97 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:33.007117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3dc54-5446-4022-acf4-ee9dc844a5db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:33.007968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3dc54-5446-4022-acf4-ee9dc844a5db service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:33.008336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3dc54-5446-4022-acf4-ee9dc844a5db service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:33.008778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3dc54-5446-4022-acf4-ee9dc844a5db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:33.018175 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1439/2882] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:07:33.020079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd212c5b-f26a-415a-84fa-e62f3d7733bf None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:33.027267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3dc54-5446-4022-acf4-ee9dc844a5db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:33.030368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3dc54-5446-4022-acf4-ee9dc844a5db service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:33.035348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd212c5b-f26a-415a-84fa-e62f3d7733bf None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:33.041048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d530d4e-d0d3-47ad-9c85-b69bc924489c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:33.041737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d530d4e-d0d3-47ad-9c85-b69bc924489c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:33.042067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d530d4e-d0d3-47ad-9c85-b69bc924489c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:33.042350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d530d4e-d0d3-47ad-9c85-b69bc924489c admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:33.043096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d530d4e-d0d3-47ad-9c85-b69bc924489c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:07:33.043410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d530d4e-d0d3-47ad-9c85-b69bc924489c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:33.045526 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1444/2883] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:07:33.046384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d530d4e-d0d3-47ad-9c85-b69bc924489c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:33.063156 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1440/2884] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:07:33.072363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3dc54-5446-4022-acf4-ee9dc844a5db service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:33.090065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3dc54-5446-4022-acf4-ee9dc844a5db service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:33.114405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8a9a0-46df-4c61-974e-75f4b8c770d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:33.117233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8a9a0-46df-4c61-974e-75f4b8c770d0 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:33.117532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8a9a0-46df-4c61-974e-75f4b8c770d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:33.117820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8a9a0-46df-4c61-974e-75f4b8c770d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:33.120550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8a9a0-46df-4c61-974e-75f4b8c770d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:33.124680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8a9a0-46df-4c61-974e-75f4b8c770d0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:33.322318 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1445/2885] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:07:33 2022] POST /identity/v3/projects => generated 383 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:07:33.325702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d530d4e-d0d3-47ad-9c85-b69bc924489c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:33.338697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d530d4e-d0d3-47ad-9c85-b69bc924489c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:33.439790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9ad67-b9ce-40d8-86fd-b2b02bf5132f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:33.440667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9ad67-b9ce-40d8-86fd-b2b02bf5132f admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:33.440958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9ad67-b9ce-40d8-86fd-b2b02bf5132f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:33.441241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9ad67-b9ce-40d8-86fd-b2b02bf5132f admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:33.442016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f9ad67-b9ce-40d8-86fd-b2b02bf5132f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:33.443967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f9ad67-b9ce-40d8-86fd-b2b02bf5132f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:33.551014 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1441/2886] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:07:33 2022] POST /identity/v3/users => generated 292 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:07:33.555580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8a9a0-46df-4c61-974e-75f4b8c770d0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:33.572325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8a9a0-46df-4c61-974e-75f4b8c770d0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:33.577734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f76ad7-3c52-4eba-acbf-04787fb14501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:33.578704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f76ad7-3c52-4eba-acbf-04787fb14501 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:33.579296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f76ad7-3c52-4eba-acbf-04787fb14501 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:33.579778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f76ad7-3c52-4eba-acbf-04787fb14501 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:33.580703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f76ad7-3c52-4eba-acbf-04787fb14501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:33.582633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f76ad7-3c52-4eba-acbf-04787fb14501 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:33.594504 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1446/2887] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:33.598616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9ad67-b9ce-40d8-86fd-b2b02bf5132f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:33.617769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f9ad67-b9ce-40d8-86fd-b2b02bf5132f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:33.624184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce3029-a6b3-4c69-a7d4-dc1b429cb81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:33.625193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce3029-a6b3-4c69-a7d4-dc1b429cb81b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:33.625756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce3029-a6b3-4c69-a7d4-dc1b429cb81b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:33.627206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce3029-a6b3-4c69-a7d4-dc1b429cb81b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:33.628464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce3029-a6b3-4c69-a7d4-dc1b429cb81b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:33.631439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce3029-a6b3-4c69-a7d4-dc1b429cb81b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:33.655414 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1442/2888] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:07:33.664821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f76ad7-3c52-4eba-acbf-04787fb14501 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:33.685312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f76ad7-3c52-4eba-acbf-04787fb14501 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:33.699617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bd579-b00d-426b-91ce-3cce698096f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:33.701627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bd579-b00d-426b-91ce-3cce698096f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:33.706839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bd579-b00d-426b-91ce-3cce698096f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:33.706839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597bd579-b00d-426b-91ce-3cce698096f5 admin admin] PATH_INFO: `/v3/projects/caacba1610f44c5ca8057b10a2dd483a/users/bf67d3b90f7642c2b23020958f14e53b/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:33.732894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bd579-b00d-426b-91ce-3cce698096f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caacba1610f44c5ca8057b10a2dd483a, user_id=bf67d3b90f7642c2b23020958f14e53b, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:33.737750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597bd579-b00d-426b-91ce-3cce698096f5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:33.865875 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1447/2889] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:07:33 2022] PUT /identity/v3/projects/caacba1610f44c5ca8057b10a2dd483a/users/bf67d3b90f7642c2b23020958f14e53b/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:33.869557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac4d35-1d9f-48b1-b33c-140d4a6c37f3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:33.870057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac4d35-1d9f-48b1-b33c-140d4a6c37f3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:33.873555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ac4d35-1d9f-48b1-b33c-140d4a6c37f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:33.899251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b0ac4d35-1d9f-48b1-b33c-140d4a6c37f3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:33.900936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b0ac4d35-1d9f-48b1-b33c-140d4a6c37f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:34.006662 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1443/2890] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:34.027507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bd579-b00d-426b-91ce-3cce698096f5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:34.030408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-597bd579-b00d-426b-91ce-3cce698096f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:34.162608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597bd579-b00d-426b-91ce-3cce698096f5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:34.167406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582508f6-899c-49e5-bb85-f1ec877493f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:34.168157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582508f6-899c-49e5-bb85-f1ec877493f3 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:34.168525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582508f6-899c-49e5-bb85-f1ec877493f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:34.168897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582508f6-899c-49e5-bb85-f1ec877493f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:34.180268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582508f6-899c-49e5-bb85-f1ec877493f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:34.185140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582508f6-899c-49e5-bb85-f1ec877493f3 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:34.194262 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1448/2891] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:07:34.332940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:34.865898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b063ea-c8c8-4034-bc47-e0030348ceb6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:34.866138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b063ea-c8c8-4034-bc47-e0030348ceb6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:34.866418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b063ea-c8c8-4034-bc47-e0030348ceb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:34.887984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-93b063ea-c8c8-4034-bc47-e0030348ceb6 None None] MFA Rules not processed for user `5e54b51913be4a2bab74aae347785727`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:34.889195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-93b063ea-c8c8-4034-bc47-e0030348ceb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:34.962543 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1444/2892] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:35.084831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582508f6-899c-49e5-bb85-f1ec877493f3 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:35.086469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-582508f6-899c-49e5-bb85-f1ec877493f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:35.107441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582508f6-899c-49e5-bb85-f1ec877493f3 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:35.124394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d5b85-0d02-4ccc-8321-0559e65477d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:35.125894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d5b85-0d02-4ccc-8321-0559e65477d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:35.126827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d5b85-0d02-4ccc-8321-0559e65477d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:35.127597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d5b85-0d02-4ccc-8321-0559e65477d0 admin admin] PATH_INFO: `/v3/users/d1b349adf52040d1822953c96514ff7e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:35.130333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d5b85-0d02-4ccc-8321-0559e65477d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1b349adf52040d1822953c96514ff7e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:35.132731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d5b85-0d02-4ccc-8321-0559e65477d0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:35.155224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:35.377264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-102d5b85-0d02-4ccc-8321-0559e65477d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1b349adf52040d1822953c96514ff7e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:35.391925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-102d5b85-0d02-4ccc-8321-0559e65477d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1b349adf52040d1822953c96514ff7e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:35.394358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-102d5b85-0d02-4ccc-8321-0559e65477d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1b349adf52040d1822953c96514ff7e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:35.399294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-102d5b85-0d02-4ccc-8321-0559e65477d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1b349adf52040d1822953c96514ff7e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:35.607308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-102d5b85-0d02-4ccc-8321-0559e65477d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1b349adf52040d1822953c96514ff7e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:35.645972 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1449/2893] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:07:35 2022] DELETE /identity/v3/users/d1b349adf52040d1822953c96514ff7e => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:35.650653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3960c-fab1-4d3f-adf3-68b1ee8d636e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:35.656051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3960c-fab1-4d3f-adf3-68b1ee8d636e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:35.656051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec3960c-fab1-4d3f-adf3-68b1ee8d636e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:35.673652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8ec3960c-fab1-4d3f-adf3-68b1ee8d636e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:35.678653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8ec3960c-fab1-4d3f-adf3-68b1ee8d636e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:35.772202 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1445/2894] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:35.783146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d5b85-0d02-4ccc-8321-0559e65477d0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:35.784763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-102d5b85-0d02-4ccc-8321-0559e65477d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:35.899683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d5b85-0d02-4ccc-8321-0559e65477d0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:35.910584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1602f54-003f-4170-8ae1-2ee3dcbcf240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:35.911266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1602f54-003f-4170-8ae1-2ee3dcbcf240 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:35.911607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1602f54-003f-4170-8ae1-2ee3dcbcf240 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:35.911889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1602f54-003f-4170-8ae1-2ee3dcbcf240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:35.951087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1602f54-003f-4170-8ae1-2ee3dcbcf240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:35.954426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1602f54-003f-4170-8ae1-2ee3dcbcf240 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:35.967091 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1450/2895] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:07:36.687454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3960c-fab1-4d3f-adf3-68b1ee8d636e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:36.692727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8ec3960c-fab1-4d3f-adf3-68b1ee8d636e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:36.743872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec3960c-fab1-4d3f-adf3-68b1ee8d636e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:36.760670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794c9f1-5fef-4f56-beed-7280698b1d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:36.761577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794c9f1-5fef-4f56-beed-7280698b1d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:36.761891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794c9f1-5fef-4f56-beed-7280698b1d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:36.762178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c794c9f1-5fef-4f56-beed-7280698b1d13 admin admin] PATH_INFO: `/v3/projects/72d0adc19fe145caa50c8a0767765092` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:36.767105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794c9f1-5fef-4f56-beed-7280698b1d13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72d0adc19fe145caa50c8a0767765092)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:36.772185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c794c9f1-5fef-4f56-beed-7280698b1d13 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:37.135404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c794c9f1-5fef-4f56-beed-7280698b1d13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72d0adc19fe145caa50c8a0767765092', 'request_id': 'req-c794c9f1-5fef-4f56-beed-7280698b1d13'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:37.169192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c794c9f1-5fef-4f56-beed-7280698b1d13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72d0adc19fe145caa50c8a0767765092', 'request_id': 'req-c794c9f1-5fef-4f56-beed-7280698b1d13'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:37.372252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c794c9f1-5fef-4f56-beed-7280698b1d13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72d0adc19fe145caa50c8a0767765092', 'request_id': 'req-c794c9f1-5fef-4f56-beed-7280698b1d13'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:37.384621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c794c9f1-5fef-4f56-beed-7280698b1d13 admin admin] The token cache is being invalidate because project 72d0adc19fe145caa50c8a0767765092 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:07:37.385039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c794c9f1-5fef-4f56-beed-7280698b1d13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:37.385769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c794c9f1-5fef-4f56-beed-7280698b1d13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:37.387625 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1446/2896] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:07:36 2022] DELETE /identity/v3/projects/72d0adc19fe145caa50c8a0767765092 => generated 0 bytes in 704 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:37.391970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1602f54-003f-4170-8ae1-2ee3dcbcf240 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:37.406028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1602f54-003f-4170-8ae1-2ee3dcbcf240 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:37.465283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58228784-0ce7-4db9-8138-4a8898d744dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:37.465744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58228784-0ce7-4db9-8138-4a8898d744dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:37.466048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58228784-0ce7-4db9-8138-4a8898d744dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:37.466337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58228784-0ce7-4db9-8138-4a8898d744dc admin admin] PATH_INFO: `/v3/users/5cbd272df98c4cf9b8860a35a13bbbd7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:37.467779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58228784-0ce7-4db9-8138-4a8898d744dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cbd272df98c4cf9b8860a35a13bbbd7)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:37.473278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58228784-0ce7-4db9-8138-4a8898d744dc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:37.529194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794c9f1-5fef-4f56-beed-7280698b1d13 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:37.531179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c794c9f1-5fef-4f56-beed-7280698b1d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:37.552815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c794c9f1-5fef-4f56-beed-7280698b1d13 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:37.559283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cab7de-0378-43cf-b9d2-d952e68c8b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:37.560547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cab7de-0378-43cf-b9d2-d952e68c8b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:37.560838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cab7de-0378-43cf-b9d2-d952e68c8b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:37.561203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cab7de-0378-43cf-b9d2-d952e68c8b53 admin admin] PATH_INFO: `/v3/users/77a6068e4dc34fb6b8d1447472692e55` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:37.562390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cab7de-0378-43cf-b9d2-d952e68c8b53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77a6068e4dc34fb6b8d1447472692e55)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:37.564922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cab7de-0378-43cf-b9d2-d952e68c8b53 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:37.736286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-58228784-0ce7-4db9-8138-4a8898d744dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cbd272df98c4cf9b8860a35a13bbbd7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:37.746611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-58228784-0ce7-4db9-8138-4a8898d744dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cbd272df98c4cf9b8860a35a13bbbd7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:37.747319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-58228784-0ce7-4db9-8138-4a8898d744dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cbd272df98c4cf9b8860a35a13bbbd7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:37.748004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-58228784-0ce7-4db9-8138-4a8898d744dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cbd272df98c4cf9b8860a35a13bbbd7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:37.769397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:37.847293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-58228784-0ce7-4db9-8138-4a8898d744dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cbd272df98c4cf9b8860a35a13bbbd7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:37.874157 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1451/2897] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:07:37 2022] DELETE /identity/v3/users/5cbd272df98c4cf9b8860a35a13bbbd7 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:07:37.925294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-76cab7de-0378-43cf-b9d2-d952e68c8b53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77a6068e4dc34fb6b8d1447472692e55'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:37.928755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-76cab7de-0378-43cf-b9d2-d952e68c8b53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77a6068e4dc34fb6b8d1447472692e55'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:37.959503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-76cab7de-0378-43cf-b9d2-d952e68c8b53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77a6068e4dc34fb6b8d1447472692e55'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:38.052245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-76cab7de-0378-43cf-b9d2-d952e68c8b53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77a6068e4dc34fb6b8d1447472692e55'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:38.052819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-76cab7de-0378-43cf-b9d2-d952e68c8b53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77a6068e4dc34fb6b8d1447472692e55'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:38.106550 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1447/2898] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:07:37 2022] DELETE /identity/v3/users/77a6068e4dc34fb6b8d1447472692e55 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:38.557621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:38.849423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58228784-0ce7-4db9-8138-4a8898d744dc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:38.851142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-58228784-0ce7-4db9-8138-4a8898d744dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:38.923431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cab7de-0378-43cf-b9d2-d952e68c8b53 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:38.926262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-76cab7de-0378-43cf-b9d2-d952e68c8b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:38.973388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58228784-0ce7-4db9-8138-4a8898d744dc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:39.064578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cab7de-0378-43cf-b9d2-d952e68c8b53 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:39.083761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2d4c4-7ae2-4592-9cc9-be97bef8f1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:39.084996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2d4c4-7ae2-4592-9cc9-be97bef8f1c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:39.085279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2d4c4-7ae2-4592-9cc9-be97bef8f1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:39.090810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2d4c4-7ae2-4592-9cc9-be97bef8f1c5 admin admin] PATH_INFO: `/v3/projects/030d24620952431f8623548f8f1a51f6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:39.090810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d2d4c4-7ae2-4592-9cc9-be97bef8f1c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=030d24620952431f8623548f8f1a51f6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:39.090810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d2d4c4-7ae2-4592-9cc9-be97bef8f1c5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:39.121542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ff744-6eef-4632-a5b5-d59dc06bc677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:39.123207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ff744-6eef-4632-a5b5-d59dc06bc677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:39.123584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ff744-6eef-4632-a5b5-d59dc06bc677 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:39.123952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ff744-6eef-4632-a5b5-d59dc06bc677 admin admin] PATH_INFO: `/v3/projects/b059b8b9ea6e4afbb5d034c9d1a036d2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:39.125452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ff744-6eef-4632-a5b5-d59dc06bc677 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b059b8b9ea6e4afbb5d034c9d1a036d2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:39.129824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ff744-6eef-4632-a5b5-d59dc06bc677 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:39.278088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-61d2d4c4-7ae2-4592-9cc9-be97bef8f1c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '030d24620952431f8623548f8f1a51f6', 'request_id': 'req-61d2d4c4-7ae2-4592-9cc9-be97bef8f1c5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:39.288735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-61d2d4c4-7ae2-4592-9cc9-be97bef8f1c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '030d24620952431f8623548f8f1a51f6', 'request_id': 'req-61d2d4c4-7ae2-4592-9cc9-be97bef8f1c5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:39.315542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da1ff744-6eef-4632-a5b5-d59dc06bc677 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b059b8b9ea6e4afbb5d034c9d1a036d2', 'request_id': 'req-da1ff744-6eef-4632-a5b5-d59dc06bc677'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:39.322631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da1ff744-6eef-4632-a5b5-d59dc06bc677 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b059b8b9ea6e4afbb5d034c9d1a036d2', 'request_id': 'req-da1ff744-6eef-4632-a5b5-d59dc06bc677'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:39.333600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:39.361732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-61d2d4c4-7ae2-4592-9cc9-be97bef8f1c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '030d24620952431f8623548f8f1a51f6', 'request_id': 'req-61d2d4c4-7ae2-4592-9cc9-be97bef8f1c5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:39.374653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-61d2d4c4-7ae2-4592-9cc9-be97bef8f1c5 admin admin] The token cache is being invalidate because project 030d24620952431f8623548f8f1a51f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:07:39.375392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-61d2d4c4-7ae2-4592-9cc9-be97bef8f1c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:39.376279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-61d2d4c4-7ae2-4592-9cc9-be97bef8f1c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:39.379339 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1448/2899] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:07:38 2022] DELETE /identity/v3/projects/030d24620952431f8623548f8f1a51f6 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:39.384334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b65e9-8347-44a6-87b8-bdc080382dd2 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:39.384629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b65e9-8347-44a6-87b8-bdc080382dd2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:39.384905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b65e9-8347-44a6-87b8-bdc080382dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:39.405180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-674b65e9-8347-44a6-87b8-bdc080382dd2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:39.406601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-674b65e9-8347-44a6-87b8-bdc080382dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:39.448135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da1ff744-6eef-4632-a5b5-d59dc06bc677 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b059b8b9ea6e4afbb5d034c9d1a036d2', 'request_id': 'req-da1ff744-6eef-4632-a5b5-d59dc06bc677'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:39.478262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da1ff744-6eef-4632-a5b5-d59dc06bc677 admin admin] The token cache is being invalidate because project b059b8b9ea6e4afbb5d034c9d1a036d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:07:39.478262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da1ff744-6eef-4632-a5b5-d59dc06bc677 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:39.478592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da1ff744-6eef-4632-a5b5-d59dc06bc677 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:39.480748 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1452/2900] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:07:38 2022] DELETE /identity/v3/projects/b059b8b9ea6e4afbb5d034c9d1a036d2 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:07:39.485085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ff744-6eef-4632-a5b5-d59dc06bc677 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:39.488974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-da1ff744-6eef-4632-a5b5-d59dc06bc677 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:39.522694 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1449/2901] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:07:39.592854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43743bed-4f0f-4f87-9740-894ec8240320 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:39.593416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43743bed-4f0f-4f87-9740-894ec8240320 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:39.593934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43743bed-4f0f-4f87-9740-894ec8240320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:39.622220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-43743bed-4f0f-4f87-9740-894ec8240320 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:39.623764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-43743bed-4f0f-4f87-9740-894ec8240320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:39.638436 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1450/2902] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:39.639749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43743bed-4f0f-4f87-9740-894ec8240320 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:39.649695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43743bed-4f0f-4f87-9740-894ec8240320 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:39.665627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e041a9-7f97-4e84-96c3-b2e4b0ad3089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:39.666375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e041a9-7f97-4e84-96c3-b2e4b0ad3089 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:39.666668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e041a9-7f97-4e84-96c3-b2e4b0ad3089 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:39.666953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e041a9-7f97-4e84-96c3-b2e4b0ad3089 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:39.667886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e041a9-7f97-4e84-96c3-b2e4b0ad3089 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:07:39.668200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e041a9-7f97-4e84-96c3-b2e4b0ad3089 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:39.670553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e041a9-7f97-4e84-96c3-b2e4b0ad3089 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:39.703256 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1451/2903] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:07:39.704242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e041a9-7f97-4e84-96c3-b2e4b0ad3089 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:39.719924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ff744-6eef-4632-a5b5-d59dc06bc677 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:39.725994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e041a9-7f97-4e84-96c3-b2e4b0ad3089 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:39.741288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea6f86-f4f8-4d67-a1bf-e4dc74aab645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:39.743175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea6f86-f4f8-4d67-a1bf-e4dc74aab645 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:39.743526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea6f86-f4f8-4d67-a1bf-e4dc74aab645 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:39.743806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea6f86-f4f8-4d67-a1bf-e4dc74aab645 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:39.744524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea6f86-f4f8-4d67-a1bf-e4dc74aab645 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:39.746780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580ddc0-35fe-420e-b71a-b206853babe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:39.749005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea6f86-f4f8-4d67-a1bf-e4dc74aab645 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:39.750268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580ddc0-35fe-420e-b71a-b206853babe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:39.752863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580ddc0-35fe-420e-b71a-b206853babe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:39.753927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580ddc0-35fe-420e-b71a-b206853babe5 admin admin] PATH_INFO: `/v3/users/1229cf9547964df18d40762b3f71a530` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:39.759902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580ddc0-35fe-420e-b71a-b206853babe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1229cf9547964df18d40762b3f71a530)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:39.765997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580ddc0-35fe-420e-b71a-b206853babe5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:39.891118 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1452/2904] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:07:39 2022] POST /identity/v3/projects => generated 369 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:07:39.895257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea6f86-f4f8-4d67-a1bf-e4dc74aab645 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:40.061353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d580ddc0-35fe-420e-b71a-b206853babe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1229cf9547964df18d40762b3f71a530'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:40.081108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d580ddc0-35fe-420e-b71a-b206853babe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1229cf9547964df18d40762b3f71a530'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:40.083040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d580ddc0-35fe-420e-b71a-b206853babe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1229cf9547964df18d40762b3f71a530'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:40.095115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d580ddc0-35fe-420e-b71a-b206853babe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1229cf9547964df18d40762b3f71a530'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:40.128188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea6f86-f4f8-4d67-a1bf-e4dc74aab645 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:40.146187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636068cc-e67f-4b89-aad5-248a36538241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:40.146969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636068cc-e67f-4b89-aad5-248a36538241 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:40.147327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636068cc-e67f-4b89-aad5-248a36538241 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:40.147680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636068cc-e67f-4b89-aad5-248a36538241 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:40.148474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636068cc-e67f-4b89-aad5-248a36538241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:40.150581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636068cc-e67f-4b89-aad5-248a36538241 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:40.194671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d580ddc0-35fe-420e-b71a-b206853babe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1229cf9547964df18d40762b3f71a530'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:40.231347 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1453/2905] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:07:39 2022] DELETE /identity/v3/users/1229cf9547964df18d40762b3f71a530 => generated 0 bytes in 744 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:40.276585 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1453/2906] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:07:39 2022] POST /identity/v3/users => generated 285 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:07:40.280566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580ddc0-35fe-420e-b71a-b206853babe5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:40.282462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d580ddc0-35fe-420e-b71a-b206853babe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:40.420507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580ddc0-35fe-420e-b71a-b206853babe5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:40.422746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:40.435498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965904c-7049-437b-a3ed-8b263b1c47e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:40.435498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965904c-7049-437b-a3ed-8b263b1c47e1 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:40.435498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965904c-7049-437b-a3ed-8b263b1c47e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:40.435498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965904c-7049-437b-a3ed-8b263b1c47e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:40.436952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965904c-7049-437b-a3ed-8b263b1c47e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:40.439734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965904c-7049-437b-a3ed-8b263b1c47e1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:40.461468 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1454/2907] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:40.466385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636068cc-e67f-4b89-aad5-248a36538241 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:40.483373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636068cc-e67f-4b89-aad5-248a36538241 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:40.498766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291eee7b-8205-4b18-9015-97e452af5f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:40.504976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291eee7b-8205-4b18-9015-97e452af5f0d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:40.505277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291eee7b-8205-4b18-9015-97e452af5f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:40.505567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291eee7b-8205-4b18-9015-97e452af5f0d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:40.507614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291eee7b-8205-4b18-9015-97e452af5f0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:40.512147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291eee7b-8205-4b18-9015-97e452af5f0d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:40.538562 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1454/2908] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:40.543605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965904c-7049-437b-a3ed-8b263b1c47e1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:40.561817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965904c-7049-437b-a3ed-8b263b1c47e1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:40.578727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db38b91-c795-421f-a0fe-4adf7414ba9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:40.579415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db38b91-c795-421f-a0fe-4adf7414ba9c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:40.580256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db38b91-c795-421f-a0fe-4adf7414ba9c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:40.580540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db38b91-c795-421f-a0fe-4adf7414ba9c admin admin] PATH_INFO: `/v3/projects/60e34a44320848159602566aa69ca34f/users/fe2d757d3916465fba6b1c38057d4722/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:40.638431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db38b91-c795-421f-a0fe-4adf7414ba9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=60e34a44320848159602566aa69ca34f, user_id=fe2d757d3916465fba6b1c38057d4722, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:40.641576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db38b91-c795-421f-a0fe-4adf7414ba9c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:40.793613 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1455/2909] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:07:40 2022] PUT /identity/v3/projects/60e34a44320848159602566aa69ca34f/users/fe2d757d3916465fba6b1c38057d4722/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:40.794528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291eee7b-8205-4b18-9015-97e452af5f0d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:40.794528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-291eee7b-8205-4b18-9015-97e452af5f0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:40.795798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ac3ab-d395-422d-a7f6-d6b695899140 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:40.796093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ac3ab-d395-422d-a7f6-d6b695899140 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:40.796379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ac3ab-d395-422d-a7f6-d6b695899140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:40.871258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291eee7b-8205-4b18-9015-97e452af5f0d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:40.972243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-266ac3ab-d395-422d-a7f6-d6b695899140 None None] MFA Rules not processed for user `fe2d757d3916465fba6b1c38057d4722`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:40.978490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-266ac3ab-d395-422d-a7f6-d6b695899140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:41.068696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a99ef-9157-43ad-8ce4-34e4e38bcef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:41.069866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a99ef-9157-43ad-8ce4-34e4e38bcef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:41.070337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a99ef-9157-43ad-8ce4-34e4e38bcef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:41.070774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a99ef-9157-43ad-8ce4-34e4e38bcef0 admin admin] PATH_INFO: `/v3/projects/ea3cafb9299747efbe223b5302b76d47` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:41.072256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a99ef-9157-43ad-8ce4-34e4e38bcef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea3cafb9299747efbe223b5302b76d47)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:41.074523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a99ef-9157-43ad-8ce4-34e4e38bcef0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:41.198788 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1456/2910] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:40 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:41.221201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ac3ab-d395-422d-a7f6-d6b695899140 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:41.222790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-266ac3ab-d395-422d-a7f6-d6b695899140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:41.290719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-389a99ef-9157-43ad-8ce4-34e4e38bcef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea3cafb9299747efbe223b5302b76d47', 'request_id': 'req-389a99ef-9157-43ad-8ce4-34e4e38bcef0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:41.306783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-389a99ef-9157-43ad-8ce4-34e4e38bcef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea3cafb9299747efbe223b5302b76d47', 'request_id': 'req-389a99ef-9157-43ad-8ce4-34e4e38bcef0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:41.393998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ac3ab-d395-422d-a7f6-d6b695899140 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:41.417770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-389a99ef-9157-43ad-8ce4-34e4e38bcef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea3cafb9299747efbe223b5302b76d47', 'request_id': 'req-389a99ef-9157-43ad-8ce4-34e4e38bcef0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:41.438320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-389a99ef-9157-43ad-8ce4-34e4e38bcef0 admin admin] The token cache is being invalidate because project ea3cafb9299747efbe223b5302b76d47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:07:41.440469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-389a99ef-9157-43ad-8ce4-34e4e38bcef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:41.441997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-389a99ef-9157-43ad-8ce4-34e4e38bcef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:41.447581 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1455/2911] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:07:40 2022] DELETE /identity/v3/projects/ea3cafb9299747efbe223b5302b76d47 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:41.453992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce70714-2604-4da0-ac2c-3e80d9b02e77 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:41.455842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce70714-2604-4da0-ac2c-3e80d9b02e77 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:41.456668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce70714-2604-4da0-ac2c-3e80d9b02e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:41.498002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-cce70714-2604-4da0-ac2c-3e80d9b02e77 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:41.499426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cce70714-2604-4da0-ac2c-3e80d9b02e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:41.531715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97bdc0-001f-49d5-8afe-dbab55e7213d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:41.532892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97bdc0-001f-49d5-8afe-dbab55e7213d service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:41.534518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97bdc0-001f-49d5-8afe-dbab55e7213d service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:41.534518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97bdc0-001f-49d5-8afe-dbab55e7213d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:41.547071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-de97bdc0-001f-49d5-8afe-dbab55e7213d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:41.589787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:41.646816 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1456/2912] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:41.648809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c745e6-2f77-464e-afd1-7396f7d62d14 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:41.649125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c745e6-2f77-464e-afd1-7396f7d62d14 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:41.649390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c745e6-2f77-464e-afd1-7396f7d62d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:41.696938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-32c745e6-2f77-464e-afd1-7396f7d62d14 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:41.698324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-32c745e6-2f77-464e-afd1-7396f7d62d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:41.718287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97bdc0-001f-49d5-8afe-dbab55e7213d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:41.719907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97bdc0-001f-49d5-8afe-dbab55e7213d service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:41.732188 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1457/2913] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:07:41.732481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c745e6-2f77-464e-afd1-7396f7d62d14 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:41.733094 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1457/2914] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:41 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:07:41.758145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c745e6-2f77-464e-afd1-7396f7d62d14 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:41.763495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07884589-11ac-4c10-9a82-e34b935f77c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:41.764170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07884589-11ac-4c10-9a82-e34b935f77c1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:41.764502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07884589-11ac-4c10-9a82-e34b935f77c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:41.764780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07884589-11ac-4c10-9a82-e34b935f77c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:41.765807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07884589-11ac-4c10-9a82-e34b935f77c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:07:41.765807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07884589-11ac-4c10-9a82-e34b935f77c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:41.767487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07884589-11ac-4c10-9a82-e34b935f77c1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:41.787044 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1458/2915] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:07:41.788329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97bdc0-001f-49d5-8afe-dbab55e7213d service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:41.819087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97bdc0-001f-49d5-8afe-dbab55e7213d service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:41.829098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9154e-b9f6-4171-b622-9845c376374a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:41.830117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9154e-b9f6-4171-b622-9845c376374a admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:41.830117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9154e-b9f6-4171-b622-9845c376374a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:41.830427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9154e-b9f6-4171-b622-9845c376374a admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:41.838012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f9154e-b9f6-4171-b622-9845c376374a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:41.838012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f9154e-b9f6-4171-b622-9845c376374a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:41.984875 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1458/2916] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:07:41 2022] POST /identity/v3/projects => generated 363 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:07:41.989261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07884589-11ac-4c10-9a82-e34b935f77c1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:42.004450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07884589-11ac-4c10-9a82-e34b935f77c1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:42.215546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676fa8e-9688-4c94-a352-c75e2078176a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:42.216582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676fa8e-9688-4c94-a352-c75e2078176a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:42.217066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676fa8e-9688-4c94-a352-c75e2078176a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:42.217527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676fa8e-9688-4c94-a352-c75e2078176a admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:42.222978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676fa8e-9688-4c94-a352-c75e2078176a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:42.225356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676fa8e-9688-4c94-a352-c75e2078176a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:42.338427 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1459/2917] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:07:41 2022] POST /identity/v3/users => generated 282 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:07:42.342617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9154e-b9f6-4171-b622-9845c376374a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:42.369595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9154e-b9f6-4171-b622-9845c376374a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:42.386554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2d907-6bec-4f02-9682-f55c4d34ce26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:42.390487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2d907-6bec-4f02-9682-f55c4d34ce26 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:42.391109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2d907-6bec-4f02-9682-f55c4d34ce26 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:42.391667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2d907-6bec-4f02-9682-f55c4d34ce26 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:42.392774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2d907-6bec-4f02-9682-f55c4d34ce26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:42.396855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2d907-6bec-4f02-9682-f55c4d34ce26 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:42.457078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676fa8e-9688-4c94-a352-c75e2078176a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:42.457430 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1459/2918] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:42.493013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676fa8e-9688-4c94-a352-c75e2078176a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:42.499001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0bc0c-e7d8-4a0b-b79f-cf64779c71e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:42.499899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0bc0c-e7d8-4a0b-b79f-cf64779c71e7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:42.500786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0bc0c-e7d8-4a0b-b79f-cf64779c71e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:42.501084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0bc0c-e7d8-4a0b-b79f-cf64779c71e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:42.502441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0bc0c-e7d8-4a0b-b79f-cf64779c71e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:42.504306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0bc0c-e7d8-4a0b-b79f-cf64779c71e7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:42.530687 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1460/2919] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:42 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:42.535010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2d907-6bec-4f02-9682-f55c4d34ce26 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:42.553678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2d907-6bec-4f02-9682-f55c4d34ce26 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:42.569319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87b088-3ffd-4093-8d98-30fb5b582936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:42.570572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87b088-3ffd-4093-8d98-30fb5b582936 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:42.571075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87b088-3ffd-4093-8d98-30fb5b582936 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:42.571681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b87b088-3ffd-4093-8d98-30fb5b582936 admin admin] PATH_INFO: `/v3/projects/158c1665476643cda0976e68ab5e595a/users/04e5b86f94914df085b76e0d89b2230e/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:42.622320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87b088-3ffd-4093-8d98-30fb5b582936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=158c1665476643cda0976e68ab5e595a, user_id=04e5b86f94914df085b76e0d89b2230e, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:42.625623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b87b088-3ffd-4093-8d98-30fb5b582936 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:42.742491 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1460/2920] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:07:42 2022] PUT /identity/v3/projects/158c1665476643cda0976e68ab5e595a/users/04e5b86f94914df085b76e0d89b2230e/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:42.750952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ab10a-bc09-4583-b96f-07eacefd1184 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:42.753832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ab10a-bc09-4583-b96f-07eacefd1184 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:42.762365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ab10a-bc09-4583-b96f-07eacefd1184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:42.789999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-937ab10a-bc09-4583-b96f-07eacefd1184 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:42.792730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:42.816296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-937ab10a-bc09-4583-b96f-07eacefd1184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:42.832813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87b088-3ffd-4093-8d98-30fb5b582936 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:42.835297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5b87b088-3ffd-4093-8d98-30fb5b582936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:42.915388 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1461/2921] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:42.932036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ab10a-bc09-4583-b96f-07eacefd1184 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:42.934031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-937ab10a-bc09-4583-b96f-07eacefd1184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:42.967695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b87b088-3ffd-4093-8d98-30fb5b582936 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:42.985135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a745b-ba90-486e-8bf2-e5f54b8d55d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:42.986177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a745b-ba90-486e-8bf2-e5f54b8d55d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:42.986711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a745b-ba90-486e-8bf2-e5f54b8d55d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:42.987216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a745b-ba90-486e-8bf2-e5f54b8d55d0 admin admin] PATH_INFO: `/v3/users/fe2d757d3916465fba6b1c38057d4722` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:42.989130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a745b-ba90-486e-8bf2-e5f54b8d55d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe2d757d3916465fba6b1c38057d4722)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:42.991205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a745b-ba90-486e-8bf2-e5f54b8d55d0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:43.071856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ab10a-bc09-4583-b96f-07eacefd1184 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:43.078688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:43.078878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950b2be-3ee4-4204-b3e2-0eea1a9955a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:43.080188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950b2be-3ee4-4204-b3e2-0eea1a9955a1 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:43.080479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950b2be-3ee4-4204-b3e2-0eea1a9955a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:43.080756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950b2be-3ee4-4204-b3e2-0eea1a9955a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:43.100181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950b2be-3ee4-4204-b3e2-0eea1a9955a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:43.101784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950b2be-3ee4-4204-b3e2-0eea1a9955a1 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:43.117422 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1462/2922] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:07:43.282421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-950a745b-ba90-486e-8bf2-e5f54b8d55d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe2d757d3916465fba6b1c38057d4722'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:43.294088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-950a745b-ba90-486e-8bf2-e5f54b8d55d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe2d757d3916465fba6b1c38057d4722'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:43.295071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-950a745b-ba90-486e-8bf2-e5f54b8d55d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe2d757d3916465fba6b1c38057d4722'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:43.295857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-950a745b-ba90-486e-8bf2-e5f54b8d55d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe2d757d3916465fba6b1c38057d4722'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:43.378288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-950a745b-ba90-486e-8bf2-e5f54b8d55d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe2d757d3916465fba6b1c38057d4722'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:43.411261 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1461/2923] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:07:42 2022] DELETE /identity/v3/users/fe2d757d3916465fba6b1c38057d4722 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:43.419517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bd2e3-a9d5-4a38-b2af-6b6e7596af84 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:43.419517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bd2e3-a9d5-4a38-b2af-6b6e7596af84 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:43.419517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bd2e3-a9d5-4a38-b2af-6b6e7596af84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:43.452659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9a2bd2e3-a9d5-4a38-b2af-6b6e7596af84 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:43.453809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9a2bd2e3-a9d5-4a38-b2af-6b6e7596af84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:43.546348 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1463/2924] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:07:43.565478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a745b-ba90-486e-8bf2-e5f54b8d55d0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:43.566911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-950a745b-ba90-486e-8bf2-e5f54b8d55d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:43.640268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a745b-ba90-486e-8bf2-e5f54b8d55d0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:43.646168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c596f-7a2b-461e-a50d-0292a6e3f901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:43.647158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c596f-7a2b-461e-a50d-0292a6e3f901 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:43.647603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c596f-7a2b-461e-a50d-0292a6e3f901 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:43.648116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c596f-7a2b-461e-a50d-0292a6e3f901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:43.660475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c596f-7a2b-461e-a50d-0292a6e3f901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:43.662376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c596f-7a2b-461e-a50d-0292a6e3f901 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:43.677914 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1462/2925] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:07:44.128014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00b506-3f69-40c8-bc9a-3583ed959530 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:44.128308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00b506-3f69-40c8-bc9a-3583ed959530 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:44.128586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a00b506-3f69-40c8-bc9a-3583ed959530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:44.156528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:44.188690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0a00b506-3f69-40c8-bc9a-3583ed959530 None None] MFA Rules not processed for user `bf67d3b90f7642c2b23020958f14e53b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:44.190795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0a00b506-3f69-40c8-bc9a-3583ed959530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:44.299124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c596f-7a2b-461e-a50d-0292a6e3f901 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:44.303548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e61c596f-7a2b-461e-a50d-0292a6e3f901 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:44.335921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c596f-7a2b-461e-a50d-0292a6e3f901 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:44.339903 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1464/2926] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:44.357640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b0b8e-54fb-4025-9dc1-740db9bd1d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:44.367168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b0b8e-54fb-4025-9dc1-740db9bd1d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:44.367168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b0b8e-54fb-4025-9dc1-740db9bd1d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:44.367168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b0b8e-54fb-4025-9dc1-740db9bd1d09 admin admin] PATH_INFO: `/v3/projects/60e34a44320848159602566aa69ca34f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:44.367168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b0b8e-54fb-4025-9dc1-740db9bd1d09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60e34a44320848159602566aa69ca34f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:44.368010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532b0b8e-54fb-4025-9dc1-740db9bd1d09 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:44.554342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-532b0b8e-54fb-4025-9dc1-740db9bd1d09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60e34a44320848159602566aa69ca34f', 'request_id': 'req-532b0b8e-54fb-4025-9dc1-740db9bd1d09'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:44.560252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-532b0b8e-54fb-4025-9dc1-740db9bd1d09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60e34a44320848159602566aa69ca34f', 'request_id': 'req-532b0b8e-54fb-4025-9dc1-740db9bd1d09'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:44.599513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00b506-3f69-40c8-bc9a-3583ed959530 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:44.601252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0a00b506-3f69-40c8-bc9a-3583ed959530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:44.679939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-532b0b8e-54fb-4025-9dc1-740db9bd1d09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60e34a44320848159602566aa69ca34f', 'request_id': 'req-532b0b8e-54fb-4025-9dc1-740db9bd1d09'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:44.692807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-532b0b8e-54fb-4025-9dc1-740db9bd1d09 admin admin] The token cache is being invalidate because project 60e34a44320848159602566aa69ca34f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:07:44.693107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-532b0b8e-54fb-4025-9dc1-740db9bd1d09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:44.693811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-532b0b8e-54fb-4025-9dc1-740db9bd1d09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:44.695487 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1463/2927] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:07:44 2022] DELETE /identity/v3/projects/60e34a44320848159602566aa69ca34f => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:44.700303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350964b2-e25f-4631-bfe0-8f5fe8701e1a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:44.700597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350964b2-e25f-4631-bfe0-8f5fe8701e1a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:44.700882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350964b2-e25f-4631-bfe0-8f5fe8701e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:44.722224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-350964b2-e25f-4631-bfe0-8f5fe8701e1a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:44.723528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-350964b2-e25f-4631-bfe0-8f5fe8701e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:44.724244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a00b506-3f69-40c8-bc9a-3583ed959530 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:44.788493 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1464/2928] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:44.797059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab19748-04ac-4ca5-b92d-61768d2b7e09 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:44.798057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab19748-04ac-4ca5-b92d-61768d2b7e09 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:44.798057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab19748-04ac-4ca5-b92d-61768d2b7e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:44.819509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0ab19748-04ac-4ca5-b92d-61768d2b7e09 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:44.820747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0ab19748-04ac-4ca5-b92d-61768d2b7e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:44.835270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80a621-8705-43eb-8458-1a5290bf0647 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:44.835981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80a621-8705-43eb-8458-1a5290bf0647 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:44.836266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80a621-8705-43eb-8458-1a5290bf0647 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:44.836537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb80a621-8705-43eb-8458-1a5290bf0647 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:44.838581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cb80a621-8705-43eb-8458-1a5290bf0647 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:44.844163 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1465/2929] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:07:44.849527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab19748-04ac-4ca5-b92d-61768d2b7e09 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:44.859954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab19748-04ac-4ca5-b92d-61768d2b7e09 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:44.865868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d770b53-be92-4f20-a5e7-3abd3b4c14cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:44.866669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d770b53-be92-4f20-a5e7-3abd3b4c14cd admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:44.867320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d770b53-be92-4f20-a5e7-3abd3b4c14cd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:44.867664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d770b53-be92-4f20-a5e7-3abd3b4c14cd admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:44.868717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d770b53-be92-4f20-a5e7-3abd3b4c14cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:07:44.869198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d770b53-be92-4f20-a5e7-3abd3b4c14cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:44.871025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d770b53-be92-4f20-a5e7-3abd3b4c14cd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:44.883804 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1466/2930] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:07:44.888535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d770b53-be92-4f20-a5e7-3abd3b4c14cd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:44.899621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d770b53-be92-4f20-a5e7-3abd3b4c14cd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:44.904529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274ce9e-2105-47d2-83c0-9517061ddb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:44.905239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274ce9e-2105-47d2-83c0-9517061ddb52 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:44.905603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274ce9e-2105-47d2-83c0-9517061ddb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:44.906041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274ce9e-2105-47d2-83c0-9517061ddb52 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:44.906743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274ce9e-2105-47d2-83c0-9517061ddb52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:44.909353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274ce9e-2105-47d2-83c0-9517061ddb52 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:44.932129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80a621-8705-43eb-8458-1a5290bf0647 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:44.934036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb80a621-8705-43eb-8458-1a5290bf0647 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:44.944980 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1465/2931] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:07:44.993030 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1467/2932] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:07:44 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:07:44.997256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80a621-8705-43eb-8458-1a5290bf0647 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:45.007851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb80a621-8705-43eb-8458-1a5290bf0647 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:45.112510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ceb5e-bd1e-4a62-9cea-02c12bcbdf75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:45.113596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ceb5e-bd1e-4a62-9cea-02c12bcbdf75 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:45.113974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ceb5e-bd1e-4a62-9cea-02c12bcbdf75 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:45.114254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ceb5e-bd1e-4a62-9cea-02c12bcbdf75 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:45.115005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ceb5e-bd1e-4a62-9cea-02c12bcbdf75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:45.117281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ceb5e-bd1e-4a62-9cea-02c12bcbdf75 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:45.201887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d72408-f8fd-4959-b427-fdb7f2fdb38d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:45.203344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d72408-f8fd-4959-b427-fdb7f2fdb38d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:45.203633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d72408-f8fd-4959-b427-fdb7f2fdb38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:45.242339 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1466/2933] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:07:44 2022] POST /identity/v3/users => generated 286 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:07:45.244008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ceb5e-bd1e-4a62-9cea-02c12bcbdf75 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:45.256984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-26d72408-f8fd-4959-b427-fdb7f2fdb38d None None] MFA Rules not processed for user `04e5b86f94914df085b76e0d89b2230e`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:45.258307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-26d72408-f8fd-4959-b427-fdb7f2fdb38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:45.258453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ceb5e-bd1e-4a62-9cea-02c12bcbdf75 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:45.268500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195e261-ee02-4776-bac0-f6cf7c99c948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:45.270060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195e261-ee02-4776-bac0-f6cf7c99c948 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:45.270624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195e261-ee02-4776-bac0-f6cf7c99c948 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:45.271185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195e261-ee02-4776-bac0-f6cf7c99c948 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:45.272048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195e261-ee02-4776-bac0-f6cf7c99c948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:45.274167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195e261-ee02-4776-bac0-f6cf7c99c948 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:45.288174 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1467/2934] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:45.292357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195e261-ee02-4776-bac0-f6cf7c99c948 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:45.315543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195e261-ee02-4776-bac0-f6cf7c99c948 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:45.324154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594aa6d-02a1-4efe-95a0-3a1fecbf4431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:45.325294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594aa6d-02a1-4efe-95a0-3a1fecbf4431 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:45.325710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594aa6d-02a1-4efe-95a0-3a1fecbf4431 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:45.326323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594aa6d-02a1-4efe-95a0-3a1fecbf4431 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:45.334162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d594aa6d-02a1-4efe-95a0-3a1fecbf4431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:45.334162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d594aa6d-02a1-4efe-95a0-3a1fecbf4431 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:45.357803 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1468/2935] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:45.359123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594aa6d-02a1-4efe-95a0-3a1fecbf4431 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:45.375566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594aa6d-02a1-4efe-95a0-3a1fecbf4431 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:45.379502 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1468/2936] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:45 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:07:45.390751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbdbca-a309-45f9-8952-d7c8df113588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:45.391009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b5250-98d3-4708-8f05-44e22a573c93 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:45.391560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b5250-98d3-4708-8f05-44e22a573c93 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:45.392686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b5250-98d3-4708-8f05-44e22a573c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:45.392924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbdbca-a309-45f9-8952-d7c8df113588 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:45.393947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbdbca-a309-45f9-8952-d7c8df113588 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:45.394341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbdbca-a309-45f9-8952-d7c8df113588 admin admin] PATH_INFO: `/v3/projects/1502581a54dd4e7d83785b715378536d/users/5cc0bf4d09594066872e7f934ddf69c1/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:45.417280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afbdbca-a309-45f9-8952-d7c8df113588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1502581a54dd4e7d83785b715378536d, user_id=5cc0bf4d09594066872e7f934ddf69c1, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:45.420225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afbdbca-a309-45f9-8952-d7c8df113588 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:45.422290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6f2b5250-98d3-4708-8f05-44e22a573c93 None None] MFA Rules not processed for user `04e5b86f94914df085b76e0d89b2230e`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:45.423935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6f2b5250-98d3-4708-8f05-44e22a573c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:45.434195 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1469/2937] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:45 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:07:45.448233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b5250-98d3-4708-8f05-44e22a573c93 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:45.450502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6f2b5250-98d3-4708-8f05-44e22a573c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:45.527509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:45.534194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b5250-98d3-4708-8f05-44e22a573c93 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:45.544448 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1469/2938] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:07:45 2022] PUT /identity/v3/projects/1502581a54dd4e7d83785b715378536d/users/5cc0bf4d09594066872e7f934ddf69c1/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:07:45.547988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10a279-1770-4b10-924d-479b29a8f847 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:45.549280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10a279-1770-4b10-924d-479b29a8f847 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:45.549280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10a279-1770-4b10-924d-479b29a8f847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:45.598738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-fa10a279-1770-4b10-924d-479b29a8f847 None None] MFA Rules not processed for user `5cc0bf4d09594066872e7f934ddf69c1`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:45.600015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fa10a279-1770-4b10-924d-479b29a8f847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:45.632432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bc050-1210-4683-99c6-0120e3334c72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:45.632654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bc050-1210-4683-99c6-0120e3334c72 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:45.633162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bc050-1210-4683-99c6-0120e3334c72 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:45.633410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bc050-1210-4683-99c6-0120e3334c72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:45.696454 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1470/2939] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:45.713119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bc050-1210-4683-99c6-0120e3334c72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:45.714789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bc050-1210-4683-99c6-0120e3334c72 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:45.724796 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1470/2940] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:45 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:07:46.136781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:46.639707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10a279-1770-4b10-924d-479b29a8f847 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:46.642188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fa10a279-1770-4b10-924d-479b29a8f847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:46.671350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10a279-1770-4b10-924d-479b29a8f847 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:46.704026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badd29d-3d89-42a0-aac3-b6a7cab432a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:46.706786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badd29d-3d89-42a0-aac3-b6a7cab432a8 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:46.708311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badd29d-3d89-42a0-aac3-b6a7cab432a8 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:46.713465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badd29d-3d89-42a0-aac3-b6a7cab432a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:46.731877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badd29d-3d89-42a0-aac3-b6a7cab432a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:46.734543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badd29d-3d89-42a0-aac3-b6a7cab432a8 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:46.780515 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1471/2941] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:07:47.266692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:47.402610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bc050-1210-4683-99c6-0120e3334c72 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:47.518827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bc050-1210-4683-99c6-0120e3334c72 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:47.523808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf224c-1426-42bb-aa2f-36588d568990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:47.524708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf224c-1426-42bb-aa2f-36588d568990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:47.525197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf224c-1426-42bb-aa2f-36588d568990 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:47.525646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf224c-1426-42bb-aa2f-36588d568990 admin admin] PATH_INFO: `/v3/users/5cc0bf4d09594066872e7f934ddf69c1` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:47.527453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf224c-1426-42bb-aa2f-36588d568990 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cc0bf4d09594066872e7f934ddf69c1)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:47.529581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbf224c-1426-42bb-aa2f-36588d568990 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:47.757214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6cbf224c-1426-42bb-aa2f-36588d568990 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cc0bf4d09594066872e7f934ddf69c1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:47.769471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6cbf224c-1426-42bb-aa2f-36588d568990 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cc0bf4d09594066872e7f934ddf69c1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:47.770283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6cbf224c-1426-42bb-aa2f-36588d568990 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cc0bf4d09594066872e7f934ddf69c1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:47.770892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6cbf224c-1426-42bb-aa2f-36588d568990 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cc0bf4d09594066872e7f934ddf69c1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:47.847477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6cbf224c-1426-42bb-aa2f-36588d568990 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cc0bf4d09594066872e7f934ddf69c1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:47.882933 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1471/2942] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:07:47 2022] DELETE /identity/v3/users/5cc0bf4d09594066872e7f934ddf69c1 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:47.887641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75a812-4279-4080-a791-a7b8c5a235df None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:47.888116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75a812-4279-4080-a791-a7b8c5a235df None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:47.888369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75a812-4279-4080-a791-a7b8c5a235df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:47.929400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ed75a812-4279-4080-a791-a7b8c5a235df None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:47.931905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ed75a812-4279-4080-a791-a7b8c5a235df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:48.018000 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1472/2943] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:07:48.025811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf224c-1426-42bb-aa2f-36588d568990 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:48.028727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6cbf224c-1426-42bb-aa2f-36588d568990 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:48.155189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf224c-1426-42bb-aa2f-36588d568990 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:48.159703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d16f66-023a-4e19-8a4b-2f588a943692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:48.160469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d16f66-023a-4e19-8a4b-2f588a943692 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:48.160756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d16f66-023a-4e19-8a4b-2f588a943692 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:48.163320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d16f66-023a-4e19-8a4b-2f588a943692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:48.185685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d16f66-023a-4e19-8a4b-2f588a943692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:48.186738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d16f66-023a-4e19-8a4b-2f588a943692 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:48.200601 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1472/2944] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:07:48.362062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:48.600334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d299437-c16f-4128-a796-fe03a65d526d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:48.600830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d299437-c16f-4128-a796-fe03a65d526d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:48.601448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d299437-c16f-4128-a796-fe03a65d526d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:48.633401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1d299437-c16f-4128-a796-fe03a65d526d None None] MFA Rules not processed for user `bf67d3b90f7642c2b23020958f14e53b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:48.635868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1d299437-c16f-4128-a796-fe03a65d526d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:48.715243 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1473/2945] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:07:48.877524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d16f66-023a-4e19-8a4b-2f588a943692 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:48.879113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-65d16f66-023a-4e19-8a4b-2f588a943692 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:48.896983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d16f66-023a-4e19-8a4b-2f588a943692 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:48.908320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4ced3-e58a-47d7-8813-1b22da385a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:48.909614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4ced3-e58a-47d7-8813-1b22da385a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:48.910089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4ced3-e58a-47d7-8813-1b22da385a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:48.910692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4ced3-e58a-47d7-8813-1b22da385a04 admin admin] PATH_INFO: `/v3/projects/1502581a54dd4e7d83785b715378536d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:48.912964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c4ced3-e58a-47d7-8813-1b22da385a04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1502581a54dd4e7d83785b715378536d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:48.915908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c4ced3-e58a-47d7-8813-1b22da385a04 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:49.054170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-48c4ced3-e58a-47d7-8813-1b22da385a04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1502581a54dd4e7d83785b715378536d', 'request_id': 'req-48c4ced3-e58a-47d7-8813-1b22da385a04'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:49.060813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-48c4ced3-e58a-47d7-8813-1b22da385a04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1502581a54dd4e7d83785b715378536d', 'request_id': 'req-48c4ced3-e58a-47d7-8813-1b22da385a04'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:49.127844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:49.135357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-48c4ced3-e58a-47d7-8813-1b22da385a04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1502581a54dd4e7d83785b715378536d', 'request_id': 'req-48c4ced3-e58a-47d7-8813-1b22da385a04'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:49.159149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-48c4ced3-e58a-47d7-8813-1b22da385a04 admin admin] The token cache is being invalidate because project 1502581a54dd4e7d83785b715378536d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:07:49.159589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-48c4ced3-e58a-47d7-8813-1b22da385a04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:49.160479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-48c4ced3-e58a-47d7-8813-1b22da385a04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:07:49.162114 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1473/2946] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:07:48 2022] DELETE /identity/v3/projects/1502581a54dd4e7d83785b715378536d => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:49.166918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10f78a-48a2-40dd-9760-834747eefb4e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:49.167513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10f78a-48a2-40dd-9760-834747eefb4e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:49.167513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10f78a-48a2-40dd-9760-834747eefb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:49.195953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5a10f78a-48a2-40dd-9760-834747eefb4e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:49.197522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5a10f78a-48a2-40dd-9760-834747eefb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:49.273641 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1474/2947] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:49.281409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a0809-d69f-4af3-b098-a8ccaeb657bd None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:49.281754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a0809-d69f-4af3-b098-a8ccaeb657bd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:49.282017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a0809-d69f-4af3-b098-a8ccaeb657bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:49.302056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-854a0809-d69f-4af3-b098-a8ccaeb657bd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:49.303313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-854a0809-d69f-4af3-b098-a8ccaeb657bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:49.312188 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1474/2948] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:07:49.316548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10f78a-48a2-40dd-9760-834747eefb4e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:49.326966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10f78a-48a2-40dd-9760-834747eefb4e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:49.332639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef78158-d7fe-4c1e-ac74-dafec1daf3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:49.333315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef78158-d7fe-4c1e-ac74-dafec1daf3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:49.333606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef78158-d7fe-4c1e-ac74-dafec1daf3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:49.333919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef78158-d7fe-4c1e-ac74-dafec1daf3b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:49.334662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef78158-d7fe-4c1e-ac74-dafec1daf3b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:07:49.334975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef78158-d7fe-4c1e-ac74-dafec1daf3b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:49.337048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef78158-d7fe-4c1e-ac74-dafec1daf3b8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:49.350476 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1475/2949] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:07:49.358526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a0809-d69f-4af3-b098-a8ccaeb657bd None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:49.368454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a0809-d69f-4af3-b098-a8ccaeb657bd None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:49.373924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bd898-f9d3-4554-8b73-ef5468e0a05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:49.374595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bd898-f9d3-4554-8b73-ef5468e0a05f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:49.374879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bd898-f9d3-4554-8b73-ef5468e0a05f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:49.375192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bd898-f9d3-4554-8b73-ef5468e0a05f admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:49.376084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780bd898-f9d3-4554-8b73-ef5468e0a05f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:49.378068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780bd898-f9d3-4554-8b73-ef5468e0a05f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:49.453014 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1475/2950] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:07:49 2022] POST /identity/v3/projects => generated 365 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:07:49.457782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef78158-d7fe-4c1e-ac74-dafec1daf3b8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:49.467936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef78158-d7fe-4c1e-ac74-dafec1daf3b8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:49.524847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e03dfc-8ba1-4108-b449-cb73e29826d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:49.525546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e03dfc-8ba1-4108-b449-cb73e29826d9 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:49.525831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e03dfc-8ba1-4108-b449-cb73e29826d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:49.526138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e03dfc-8ba1-4108-b449-cb73e29826d9 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:49.526888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e03dfc-8ba1-4108-b449-cb73e29826d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:49.529201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e03dfc-8ba1-4108-b449-cb73e29826d9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:49.629674 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1476/2951] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:07:49 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:07:49.633577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bd898-f9d3-4554-8b73-ef5468e0a05f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:49.650193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bd898-f9d3-4554-8b73-ef5468e0a05f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:49.667189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb1a55-05c5-4f52-b5c3-6c1290a14287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:49.667954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb1a55-05c5-4f52-b5c3-6c1290a14287 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:49.668306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb1a55-05c5-4f52-b5c3-6c1290a14287 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:49.668739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb1a55-05c5-4f52-b5c3-6c1290a14287 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:49.669550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb1a55-05c5-4f52-b5c3-6c1290a14287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:49.674329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb1a55-05c5-4f52-b5c3-6c1290a14287 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:49.693033 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1476/2952] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:49.700385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e03dfc-8ba1-4108-b449-cb73e29826d9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:49.715236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e03dfc-8ba1-4108-b449-cb73e29826d9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:49.722864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10db977-6dde-4cc6-9271-533597b85983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:49.723748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10db977-6dde-4cc6-9271-533597b85983 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:49.724042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10db977-6dde-4cc6-9271-533597b85983 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:49.726164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10db977-6dde-4cc6-9271-533597b85983 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:49.726164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10db977-6dde-4cc6-9271-533597b85983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:49.727077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10db977-6dde-4cc6-9271-533597b85983 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:49.755707 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1477/2953] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:07:49.758687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb1a55-05c5-4f52-b5c3-6c1290a14287 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:49.779097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb1a55-05c5-4f52-b5c3-6c1290a14287 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:49.786872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58039256-5577-4f70-b261-1c7bc67f6d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:49.787843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58039256-5577-4f70-b261-1c7bc67f6d27 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:49.788201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58039256-5577-4f70-b261-1c7bc67f6d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:49.788555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58039256-5577-4f70-b261-1c7bc67f6d27 admin admin] PATH_INFO: `/v3/projects/cde8dbaabafe47dc9aca37d9ef370819/users/121de134dc0c4e148d09ef9d764c29c6/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:49.824720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58039256-5577-4f70-b261-1c7bc67f6d27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cde8dbaabafe47dc9aca37d9ef370819, user_id=121de134dc0c4e148d09ef9d764c29c6, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:49.828340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58039256-5577-4f70-b261-1c7bc67f6d27 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:49.908814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:49.943937 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1477/2954] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:07:49 2022] PUT /identity/v3/projects/cde8dbaabafe47dc9aca37d9ef370819/users/121de134dc0c4e148d09ef9d764c29c6/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:07:49.949143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a4c00-26d0-4d4a-b50d-a58d65772e07 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:49.949437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a4c00-26d0-4d4a-b50d-a58d65772e07 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:49.949750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a4c00-26d0-4d4a-b50d-a58d65772e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:49.970368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bf5a4c00-26d0-4d4a-b50d-a58d65772e07 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:49.971604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bf5a4c00-26d0-4d4a-b50d-a58d65772e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:50.043251 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1478/2955] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:50.054278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58039256-5577-4f70-b261-1c7bc67f6d27 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:50.056581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-58039256-5577-4f70-b261-1c7bc67f6d27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:50.127868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58039256-5577-4f70-b261-1c7bc67f6d27 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:50.147452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033bf744-781d-45f7-aff9-16375fd124b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:50.148248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bf744-781d-45f7-aff9-16375fd124b2 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:50.148540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bf744-781d-45f7-aff9-16375fd124b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:50.149152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033bf744-781d-45f7-aff9-16375fd124b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:50.175052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033bf744-781d-45f7-aff9-16375fd124b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:50.178134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033bf744-781d-45f7-aff9-16375fd124b2 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:50.224764 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1478/2956] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:07:51.172338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:51.412379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a4c00-26d0-4d4a-b50d-a58d65772e07 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:51.414070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bf5a4c00-26d0-4d4a-b50d-a58d65772e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:51.452270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:51.538310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a4c00-26d0-4d4a-b50d-a58d65772e07 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:51.548709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf052216-dc54-4352-a0e5-cb4d5a9e476b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:51.550261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf052216-dc54-4352-a0e5-cb4d5a9e476b service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:51.550717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf052216-dc54-4352-a0e5-cb4d5a9e476b service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:51.551216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf052216-dc54-4352-a0e5-cb4d5a9e476b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:51.554148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bf052216-dc54-4352-a0e5-cb4d5a9e476b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:51.637040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf052216-dc54-4352-a0e5-cb4d5a9e476b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:51.638700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf052216-dc54-4352-a0e5-cb4d5a9e476b service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:51.648857 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1479/2957] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:07:52.286353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:53.887912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:54.000519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff3548-13b2-4a4b-b392-09dc28cef4ae None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:54.000856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff3548-13b2-4a4b-b392-09dc28cef4ae None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:54.001576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff3548-13b2-4a4b-b392-09dc28cef4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:54.051238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9dff3548-13b2-4a4b-b392-09dc28cef4ae None None] MFA Rules not processed for user `5e54b51913be4a2bab74aae347785727`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:54.051238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9dff3548-13b2-4a4b-b392-09dc28cef4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:54.391696 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1479/2958] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:59.385882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf3db2-3c34-4a80-9f5e-d41ab5b9869a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:59.386366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf3db2-3c34-4a80-9f5e-d41ab5b9869a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:59.386591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf3db2-3c34-4a80-9f5e-d41ab5b9869a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:59.461218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-93bf3db2-3c34-4a80-9f5e-d41ab5b9869a None None] MFA Rules not processed for user `121de134dc0c4e148d09ef9d764c29c6`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:07:59.467945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-93bf3db2-3c34-4a80-9f5e-d41ab5b9869a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:07:59.597885 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1480/2959] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:07:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:07:59.706894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff3548-13b2-4a4b-b392-09dc28cef4ae None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:07:59.725686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff3548-13b2-4a4b-b392-09dc28cef4ae None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:07:59.732546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c51374-e775-4747-8945-0fe0fe1e1405 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:07:59.733773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c51374-e775-4747-8945-0fe0fe1e1405 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:07:59.734095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c51374-e775-4747-8945-0fe0fe1e1405 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:07:59.734379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c51374-e775-4747-8945-0fe0fe1e1405 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:07:59.752348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c51374-e775-4747-8945-0fe0fe1e1405 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:07:59.755335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c51374-e775-4747-8945-0fe0fe1e1405 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:07:59.769557 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1480/2960] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:07:59 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:00.803447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf3db2-3c34-4a80-9f5e-d41ab5b9869a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:00.820628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf3db2-3c34-4a80-9f5e-d41ab5b9869a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:00.832445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552178eb-43f1-45b2-953f-9099c78955a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:00.838350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552178eb-43f1-45b2-953f-9099c78955a7 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:00.838350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552178eb-43f1-45b2-953f-9099c78955a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:00.838350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552178eb-43f1-45b2-953f-9099c78955a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:00.844568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-552178eb-43f1-45b2-953f-9099c78955a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:00.872038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552178eb-43f1-45b2-953f-9099c78955a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:00.873608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552178eb-43f1-45b2-953f-9099c78955a7 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:00.886593 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1481/2961] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:03.258942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c51374-e775-4747-8945-0fe0fe1e1405 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:03.271381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c51374-e775-4747-8945-0fe0fe1e1405 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:03.291043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a72a8-0797-46ff-bc14-727abfde15c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:03.292097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a72a8-0797-46ff-bc14-727abfde15c7 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:03.292538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a72a8-0797-46ff-bc14-727abfde15c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:03.292974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a72a8-0797-46ff-bc14-727abfde15c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:03.296049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ec3a72a8-0797-46ff-bc14-727abfde15c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:03.328913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a72a8-0797-46ff-bc14-727abfde15c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:03.333476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a72a8-0797-46ff-bc14-727abfde15c7 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:03.367374 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1481/2962] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:05.068931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77088e4-d022-4ad8-a27c-b561d45106da None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:05.070098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77088e4-d022-4ad8-a27c-b561d45106da None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:05.070649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77088e4-d022-4ad8-a27c-b561d45106da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:05.098972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d77088e4-d022-4ad8-a27c-b561d45106da None None] MFA Rules not processed for user `121de134dc0c4e148d09ef9d764c29c6`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:05.100829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d77088e4-d022-4ad8-a27c-b561d45106da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:05.113746 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1482/2963] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:08:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:07.176565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a72a8-0797-46ff-bc14-727abfde15c7 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:07.180074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ec3a72a8-0797-46ff-bc14-727abfde15c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:07.289871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a72a8-0797-46ff-bc14-727abfde15c7 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:07.296628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369722d-b6e9-42c1-9bf0-d6b2d3184f04 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:07.296628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369722d-b6e9-42c1-9bf0-d6b2d3184f04 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:07.298863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369722d-b6e9-42c1-9bf0-d6b2d3184f04 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:07.298863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369722d-b6e9-42c1-9bf0-d6b2d3184f04 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:07.299712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7369722d-b6e9-42c1-9bf0-d6b2d3184f04 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:07.315279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369722d-b6e9-42c1-9bf0-d6b2d3184f04 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:07.317055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369722d-b6e9-42c1-9bf0-d6b2d3184f04 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:07.328609 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1482/2964] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:10.852886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77088e4-d022-4ad8-a27c-b561d45106da None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:10.864965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77088e4-d022-4ad8-a27c-b561d45106da None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:10.869110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377db9da-bbb2-4751-b6d4-d828f643000e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:10.870219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377db9da-bbb2-4751-b6d4-d828f643000e service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:10.870688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377db9da-bbb2-4751-b6d4-d828f643000e service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:10.871130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377db9da-bbb2-4751-b6d4-d828f643000e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:10.873185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-377db9da-bbb2-4751-b6d4-d828f643000e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:10.885674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377db9da-bbb2-4751-b6d4-d828f643000e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:10.888348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377db9da-bbb2-4751-b6d4-d828f643000e service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:10.906828 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1483/2965] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:12.056930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce79c0-c6ac-4623-86c0-db97160e9759 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:12.057815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce79c0-c6ac-4623-86c0-db97160e9759 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:12.058413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce79c0-c6ac-4623-86c0-db97160e9759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:12.105451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cfce79c0-c6ac-4623-86c0-db97160e9759 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:12.107503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cfce79c0-c6ac-4623-86c0-db97160e9759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:12.160994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:12.267357 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1483/2966] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:08:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:12.567750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377db9da-bbb2-4751-b6d4-d828f643000e service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:12.569492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-377db9da-bbb2-4751-b6d4-d828f643000e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:12.586662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:12.673726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377db9da-bbb2-4751-b6d4-d828f643000e service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:12.680587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ae7d4-2afa-4f4a-959f-6de87b7da68f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:12.681303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ae7d4-2afa-4f4a-959f-6de87b7da68f service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:12.681751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ae7d4-2afa-4f4a-959f-6de87b7da68f service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:12.682206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ae7d4-2afa-4f4a-959f-6de87b7da68f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:12.692368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ae7d4-2afa-4f4a-959f-6de87b7da68f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:12.698339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ae7d4-2afa-4f4a-959f-6de87b7da68f service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:12.705195 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1484/2967] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:08:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:08:12.713786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce79c0-c6ac-4623-86c0-db97160e9759 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:12.715270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cfce79c0-c6ac-4623-86c0-db97160e9759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:12.742307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce79c0-c6ac-4623-86c0-db97160e9759 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:12.758001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe985a63-7795-4f91-9dd4-458843e78171 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:12.758001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe985a63-7795-4f91-9dd4-458843e78171 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:12.758642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe985a63-7795-4f91-9dd4-458843e78171 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:12.759002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe985a63-7795-4f91-9dd4-458843e78171 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:12.762074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fe985a63-7795-4f91-9dd4-458843e78171 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:12.794891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe985a63-7795-4f91-9dd4-458843e78171 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:12.796631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe985a63-7795-4f91-9dd4-458843e78171 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:12.833581 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1484/2968] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:16.743102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ae7d4-2afa-4f4a-959f-6de87b7da68f service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:16.754550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ae7d4-2afa-4f4a-959f-6de87b7da68f service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:16.759148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87085bc-11ab-4492-9ff0-2c6bb6dc267d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:16.760353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87085bc-11ab-4492-9ff0-2c6bb6dc267d service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:16.760651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87085bc-11ab-4492-9ff0-2c6bb6dc267d service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:16.760936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87085bc-11ab-4492-9ff0-2c6bb6dc267d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:16.763351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a87085bc-11ab-4492-9ff0-2c6bb6dc267d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:16.775611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87085bc-11ab-4492-9ff0-2c6bb6dc267d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:16.777366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87085bc-11ab-4492-9ff0-2c6bb6dc267d service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:16.788819 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1485/2969] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:19.146798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe985a63-7795-4f91-9dd4-458843e78171 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:19.149503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fe985a63-7795-4f91-9dd4-458843e78171 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:19.182573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe985a63-7795-4f91-9dd4-458843e78171 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:19.201877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec3dff-8415-43d0-a4b6-f36d782ca1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:19.202946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec3dff-8415-43d0-a4b6-f36d782ca1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:19.204265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec3dff-8415-43d0-a4b6-f36d782ca1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:19.204691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec3dff-8415-43d0-a4b6-f36d782ca1a6 admin admin] PATH_INFO: `/v3/users/be94d10c650645658c755b7fc1595370` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:19.206170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec3dff-8415-43d0-a4b6-f36d782ca1a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be94d10c650645658c755b7fc1595370)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:19.209769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec3dff-8415-43d0-a4b6-f36d782ca1a6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:19.527413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d8ec3dff-8415-43d0-a4b6-f36d782ca1a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be94d10c650645658c755b7fc1595370'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:19.553546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d8ec3dff-8415-43d0-a4b6-f36d782ca1a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be94d10c650645658c755b7fc1595370'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:19.556990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d8ec3dff-8415-43d0-a4b6-f36d782ca1a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be94d10c650645658c755b7fc1595370'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:19.556990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d8ec3dff-8415-43d0-a4b6-f36d782ca1a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be94d10c650645658c755b7fc1595370'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:19.651294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d8ec3dff-8415-43d0-a4b6-f36d782ca1a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be94d10c650645658c755b7fc1595370'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:19.677990 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1485/2970] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:08:19 2022] DELETE /identity/v3/users/be94d10c650645658c755b7fc1595370 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:08:20.047349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87085bc-11ab-4492-9ff0-2c6bb6dc267d service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:20.118717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87085bc-11ab-4492-9ff0-2c6bb6dc267d service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:20.298569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:20.299486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:20.299772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:20.300054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce admin admin] PATH_INFO: `/v3/projects/f7a9c4408ec1432cb05c5256657886d4` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:20.302258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7a9c4408ec1432cb05c5256657886d4)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:20.307417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:20.518950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7a9c4408ec1432cb05c5256657886d4', 'request_id': 'req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:20.529600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7a9c4408ec1432cb05c5256657886d4', 'request_id': 'req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:20.618833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7a9c4408ec1432cb05c5256657886d4', 'request_id': 'req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:20.639629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce admin admin] The token cache is being invalidate because project f7a9c4408ec1432cb05c5256657886d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:08:20.640838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:20.642871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:20.646259 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1486/2971] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:08:20 2022] DELETE /identity/v3/projects/f7a9c4408ec1432cb05c5256657886d4 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:08:20.649993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec3dff-8415-43d0-a4b6-f36d782ca1a6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:20.651727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d8ec3dff-8415-43d0-a4b6-f36d782ca1a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:20.673160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec3dff-8415-43d0-a4b6-f36d782ca1a6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:20.785333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f03f87-5fc9-46de-ab54-4b6f6b694e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:20.786118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f03f87-5fc9-46de-ab54-4b6f6b694e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:20.786410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f03f87-5fc9-46de-ab54-4b6f6b694e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:20.786696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f03f87-5fc9-46de-ab54-4b6f6b694e0b admin admin] PATH_INFO: `/v3/users/3b3b0064f12e43cd8069165fe4c370bc` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:20.788590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f03f87-5fc9-46de-ab54-4b6f6b694e0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b3b0064f12e43cd8069165fe4c370bc)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:20.790638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f03f87-5fc9-46de-ab54-4b6f6b694e0b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:21.002949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:21.019344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b0f03f87-5fc9-46de-ab54-4b6f6b694e0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b3b0064f12e43cd8069165fe4c370bc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:21.028085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b0f03f87-5fc9-46de-ab54-4b6f6b694e0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b3b0064f12e43cd8069165fe4c370bc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:21.028085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b0f03f87-5fc9-46de-ab54-4b6f6b694e0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b3b0064f12e43cd8069165fe4c370bc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:21.028688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b0f03f87-5fc9-46de-ab54-4b6f6b694e0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b3b0064f12e43cd8069165fe4c370bc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:21.119995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b0f03f87-5fc9-46de-ab54-4b6f6b694e0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b3b0064f12e43cd8069165fe4c370bc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:21.158802 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1486/2972] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:08:20 2022] DELETE /identity/v3/users/3b3b0064f12e43cd8069165fe4c370bc => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:08:21.533872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:21.535872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:21.560790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d16a3a-8ce8-4e7a-9251-99db17f3e1ce admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:21.582376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:21.690359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aeb5af-4dc5-4817-97ae-4fe125b82aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:21.691742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeb5af-4dc5-4817-97ae-4fe125b82aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:21.693902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeb5af-4dc5-4817-97ae-4fe125b82aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:21.695221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aeb5af-4dc5-4817-97ae-4fe125b82aeb admin admin] PATH_INFO: `/v3/projects/dcf954165a95433eb89884d6dc4c7feb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:21.697996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aeb5af-4dc5-4817-97ae-4fe125b82aeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcf954165a95433eb89884d6dc4c7feb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:21.701164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aeb5af-4dc5-4817-97ae-4fe125b82aeb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:21.899438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e9aeb5af-4dc5-4817-97ae-4fe125b82aeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcf954165a95433eb89884d6dc4c7feb', 'request_id': 'req-e9aeb5af-4dc5-4817-97ae-4fe125b82aeb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:21.907464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e9aeb5af-4dc5-4817-97ae-4fe125b82aeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcf954165a95433eb89884d6dc4c7feb', 'request_id': 'req-e9aeb5af-4dc5-4817-97ae-4fe125b82aeb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:21.988342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e9aeb5af-4dc5-4817-97ae-4fe125b82aeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcf954165a95433eb89884d6dc4c7feb', 'request_id': 'req-e9aeb5af-4dc5-4817-97ae-4fe125b82aeb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:22.032429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e9aeb5af-4dc5-4817-97ae-4fe125b82aeb admin admin] The token cache is being invalidate because project dcf954165a95433eb89884d6dc4c7feb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:08:22.033044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e9aeb5af-4dc5-4817-97ae-4fe125b82aeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:22.034284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e9aeb5af-4dc5-4817-97ae-4fe125b82aeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:22.037352 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1487/2973] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:08:21 2022] DELETE /identity/v3/projects/dcf954165a95433eb89884d6dc4c7feb => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:08:22.042848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f55a62-717f-4f35-8f88-680da0cc3656 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:22.043362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f55a62-717f-4f35-8f88-680da0cc3656 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:22.043448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f55a62-717f-4f35-8f88-680da0cc3656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:22.082852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f2f55a62-717f-4f35-8f88-680da0cc3656 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:22.087833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f2f55a62-717f-4f35-8f88-680da0cc3656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:22.224510 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1487/2974] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:22.232666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78f8b9-a553-4cb2-80d6-0697f0ef838a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:22.233733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78f8b9-a553-4cb2-80d6-0697f0ef838a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:22.234265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d78f8b9-a553-4cb2-80d6-0697f0ef838a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:22.264785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0d78f8b9-a553-4cb2-80d6-0697f0ef838a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:22.266827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0d78f8b9-a553-4cb2-80d6-0697f0ef838a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:22.291823 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1488/2975] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:22.295707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f55a62-717f-4f35-8f88-680da0cc3656 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:22.308153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f55a62-717f-4f35-8f88-680da0cc3656 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:22.319775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9f1da-c0cc-45ed-b234-331b6d6afe21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:22.320371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9f1da-c0cc-45ed-b234-331b6d6afe21 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:22.320654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9f1da-c0cc-45ed-b234-331b6d6afe21 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:22.320931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9f1da-c0cc-45ed-b234-331b6d6afe21 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:22.321666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9f1da-c0cc-45ed-b234-331b6d6afe21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:08:22.322004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9f1da-c0cc-45ed-b234-331b6d6afe21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:22.323687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9f1da-c0cc-45ed-b234-331b6d6afe21 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:22.363202 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1488/2976] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:08:22.367526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78f8b9-a553-4cb2-80d6-0697f0ef838a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:22.379789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d78f8b9-a553-4cb2-80d6-0697f0ef838a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:22.392712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dee09-e30b-4406-8701-80283e06b447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:22.393581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dee09-e30b-4406-8701-80283e06b447 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:22.397829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dee09-e30b-4406-8701-80283e06b447 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:22.397829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dee09-e30b-4406-8701-80283e06b447 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:22.397829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458dee09-e30b-4406-8701-80283e06b447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:22.402353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458dee09-e30b-4406-8701-80283e06b447 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:22.494415 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1489/2977] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:08:22 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:08:22.495689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9f1da-c0cc-45ed-b234-331b6d6afe21 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:22.513120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9f1da-c0cc-45ed-b234-331b6d6afe21 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:22.633358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47b3c9-b2ee-462a-8184-4bf70fad3573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:22.634450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47b3c9-b2ee-462a-8184-4bf70fad3573 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:22.634850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47b3c9-b2ee-462a-8184-4bf70fad3573 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:22.635204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47b3c9-b2ee-462a-8184-4bf70fad3573 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:22.636034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47b3c9-b2ee-462a-8184-4bf70fad3573 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:22.641241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47b3c9-b2ee-462a-8184-4bf70fad3573 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:22.732824 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1489/2978] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:08:22 2022] POST /identity/v3/users => generated 290 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:08:22.737330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dee09-e30b-4406-8701-80283e06b447 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:22.753696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458dee09-e30b-4406-8701-80283e06b447 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:22.767540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45dded0-4007-4e0e-8a5b-7bff32f79f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:22.768469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dded0-4007-4e0e-8a5b-7bff32f79f79 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:22.768949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dded0-4007-4e0e-8a5b-7bff32f79f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:22.769712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45dded0-4007-4e0e-8a5b-7bff32f79f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:22.770711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45dded0-4007-4e0e-8a5b-7bff32f79f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:22.772650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45dded0-4007-4e0e-8a5b-7bff32f79f79 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:22.801319 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1490/2979] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:08:22.802188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47b3c9-b2ee-462a-8184-4bf70fad3573 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:22.816577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47b3c9-b2ee-462a-8184-4bf70fad3573 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:22.822408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13538a-949c-43c2-ac79-4a723f37027a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:22.824351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13538a-949c-43c2-ac79-4a723f37027a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:22.824959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13538a-949c-43c2-ac79-4a723f37027a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:22.825420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13538a-949c-43c2-ac79-4a723f37027a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:22.826680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13538a-949c-43c2-ac79-4a723f37027a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:22.828489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13538a-949c-43c2-ac79-4a723f37027a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:22.848972 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1490/2980] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:08:22.852967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45dded0-4007-4e0e-8a5b-7bff32f79f79 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:22.878915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45dded0-4007-4e0e-8a5b-7bff32f79f79 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:22.913684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159eff88-984d-4ab9-91c8-0e2591596ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:22.914695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159eff88-984d-4ab9-91c8-0e2591596ae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:22.914979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159eff88-984d-4ab9-91c8-0e2591596ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:22.915255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159eff88-984d-4ab9-91c8-0e2591596ae6 admin admin] PATH_INFO: `/v3/projects/a11903b0e9b1485d9f27e6ee742d8b97/users/372d1e7ed7e44a2d80bd7b76be7c4c42/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:22.967183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159eff88-984d-4ab9-91c8-0e2591596ae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a11903b0e9b1485d9f27e6ee742d8b97, user_id=372d1e7ed7e44a2d80bd7b76be7c4c42, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:22.973822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159eff88-984d-4ab9-91c8-0e2591596ae6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:23.189901 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1491/2981] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:08:22 2022] PUT /identity/v3/projects/a11903b0e9b1485d9f27e6ee742d8b97/users/372d1e7ed7e44a2d80bd7b76be7c4c42/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:08:23.265428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0ce13-dabf-4099-b60f-9684a5186e70 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:23.265958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0ce13-dabf-4099-b60f-9684a5186e70 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:23.267765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0ce13-dabf-4099-b60f-9684a5186e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:23.329986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d5c0ce13-dabf-4099-b60f-9684a5186e70 None None] MFA Rules not processed for user `372d1e7ed7e44a2d80bd7b76be7c4c42`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:23.332560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d5c0ce13-dabf-4099-b60f-9684a5186e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:23.339207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:23.451941 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1491/2982] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:08:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:23.756473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159eff88-984d-4ab9-91c8-0e2591596ae6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:23.759174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-159eff88-984d-4ab9-91c8-0e2591596ae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:23.818805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:23.871993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159eff88-984d-4ab9-91c8-0e2591596ae6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:23.878137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753365a-7ab7-4332-b986-e8e447cfaedc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:23.879079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753365a-7ab7-4332-b986-e8e447cfaedc service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:23.879495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753365a-7ab7-4332-b986-e8e447cfaedc service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:23.879786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753365a-7ab7-4332-b986-e8e447cfaedc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:23.895205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753365a-7ab7-4332-b986-e8e447cfaedc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:23.896758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9753365a-7ab7-4332-b986-e8e447cfaedc service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:23.916493 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1492/2983] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:24.342632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cfde4-8ab0-4872-a060-4b34b205084f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:24.343638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cfde4-8ab0-4872-a060-4b34b205084f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:24.344434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14cfde4-8ab0-4872-a060-4b34b205084f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:24.378343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d14cfde4-8ab0-4872-a060-4b34b205084f None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:24.381638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d14cfde4-8ab0-4872-a060-4b34b205084f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:24.484442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf101c-b300-4816-82e4-1d186291256e None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:24.485361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf101c-b300-4816-82e4-1d186291256e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:24.485799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf101c-b300-4816-82e4-1d186291256e None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:24.487476 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1493/2984] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:08:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:08:24.493492 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1492/2985] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:08:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:08:24.497996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd8b5f-c613-499e-8e72-0078dd2ad6d1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:24.498296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd8b5f-c613-499e-8e72-0078dd2ad6d1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:24.498588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd8b5f-c613-499e-8e72-0078dd2ad6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:24.553013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6fbd8b5f-c613-499e-8e72-0078dd2ad6d1 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:24.554335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6fbd8b5f-c613-499e-8e72-0078dd2ad6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:24.559371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cfde4-8ab0-4872-a060-4b34b205084f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:24.561200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d14cfde4-8ab0-4872-a060-4b34b205084f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:24.720309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14cfde4-8ab0-4872-a060-4b34b205084f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:24.725689 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1494/2986] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:08:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:24.733441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b455a-ebe4-4b72-b3ce-53285f5b002d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:24.734170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b455a-ebe4-4b72-b3ce-53285f5b002d service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:24.737830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b455a-ebe4-4b72-b3ce-53285f5b002d service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:24.737830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b455a-ebe4-4b72-b3ce-53285f5b002d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:24.756143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd8b5f-c613-499e-8e72-0078dd2ad6d1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:24.758053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6fbd8b5f-c613-499e-8e72-0078dd2ad6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:24.759119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b455a-ebe4-4b72-b3ce-53285f5b002d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:24.760838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b455a-ebe4-4b72-b3ce-53285f5b002d service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:24.778144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd8b5f-c613-499e-8e72-0078dd2ad6d1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:24.781008 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1493/2987] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:08:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:08:24.785120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba98835-8f85-4a7e-b351-f184d487ae2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:24.786172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba98835-8f85-4a7e-b351-f184d487ae2d service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:24.786617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba98835-8f85-4a7e-b351-f184d487ae2d service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:24.787054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba98835-8f85-4a7e-b351-f184d487ae2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:24.808650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba98835-8f85-4a7e-b351-f184d487ae2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:24.810866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba98835-8f85-4a7e-b351-f184d487ae2d service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:24.837007 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1495/2988] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:25.313404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43784e4-509d-4833-bddd-a3773427d3ad None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:25.313693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43784e4-509d-4833-bddd-a3773427d3ad None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:25.314001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43784e4-509d-4833-bddd-a3773427d3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:25.360014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a43784e4-509d-4833-bddd-a3773427d3ad None None] MFA Rules not processed for user `372d1e7ed7e44a2d80bd7b76be7c4c42`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:25.362001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a43784e4-509d-4833-bddd-a3773427d3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:25.371186 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1494/2989] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:08:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:26.467063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b6022-4fd0-471f-ab64-a2d6b55b65b7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:26.470846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b6022-4fd0-471f-ab64-a2d6b55b65b7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:26.470846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b6022-4fd0-471f-ab64-a2d6b55b65b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:26.497117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-705b6022-4fd0-471f-ab64-a2d6b55b65b7 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:26.498665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-705b6022-4fd0-471f-ab64-a2d6b55b65b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:26.528027 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1496/2990] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:08:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:26.542692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43784e4-509d-4833-bddd-a3773427d3ad None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:26.566073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43784e4-509d-4833-bddd-a3773427d3ad None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:26.579641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d553d6d-e595-4f88-88c7-bfa755f2a081 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:26.580402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d553d6d-e595-4f88-88c7-bfa755f2a081 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:26.580805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d553d6d-e595-4f88-88c7-bfa755f2a081 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:26.581171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d553d6d-e595-4f88-88c7-bfa755f2a081 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:26.623377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d553d6d-e595-4f88-88c7-bfa755f2a081 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:26.626002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d553d6d-e595-4f88-88c7-bfa755f2a081 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:26.674073 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1495/2991] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:08:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:08:27.108906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b6022-4fd0-471f-ab64-a2d6b55b65b7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:27.202304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b6022-4fd0-471f-ab64-a2d6b55b65b7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:27.257310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec82c2e-761b-49ee-87f8-f6f5afaf9eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:27.257310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec82c2e-761b-49ee-87f8-f6f5afaf9eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:27.257310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec82c2e-761b-49ee-87f8-f6f5afaf9eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:27.257310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec82c2e-761b-49ee-87f8-f6f5afaf9eb8 admin admin] PATH_INFO: `/v3/users/372d1e7ed7e44a2d80bd7b76be7c4c42` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:27.258763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec82c2e-761b-49ee-87f8-f6f5afaf9eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=372d1e7ed7e44a2d80bd7b76be7c4c42)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:27.258763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec82c2e-761b-49ee-87f8-f6f5afaf9eb8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:27.564446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dec82c2e-761b-49ee-87f8-f6f5afaf9eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '372d1e7ed7e44a2d80bd7b76be7c4c42'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:27.566075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dec82c2e-761b-49ee-87f8-f6f5afaf9eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '372d1e7ed7e44a2d80bd7b76be7c4c42'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:27.569117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:27.575845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dec82c2e-761b-49ee-87f8-f6f5afaf9eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '372d1e7ed7e44a2d80bd7b76be7c4c42'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:27.695503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dec82c2e-761b-49ee-87f8-f6f5afaf9eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '372d1e7ed7e44a2d80bd7b76be7c4c42'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:27.697037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dec82c2e-761b-49ee-87f8-f6f5afaf9eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '372d1e7ed7e44a2d80bd7b76be7c4c42'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:27.737453 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1497/2992] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:08:27 2022] DELETE /identity/v3/users/372d1e7ed7e44a2d80bd7b76be7c4c42 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:08:27.742743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca3529-d930-40c7-af8b-8e976d38a063 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:27.743243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca3529-d930-40c7-af8b-8e976d38a063 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:27.743418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca3529-d930-40c7-af8b-8e976d38a063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:27.777264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-89ca3529-d930-40c7-af8b-8e976d38a063 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:27.780517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-89ca3529-d930-40c7-af8b-8e976d38a063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:27.807873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:27.877925 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1496/2993] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:08:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:27.885083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec82c2e-761b-49ee-87f8-f6f5afaf9eb8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:27.893352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-dec82c2e-761b-49ee-87f8-f6f5afaf9eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:27.993129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec82c2e-761b-49ee-87f8-f6f5afaf9eb8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:27.998460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42accd-158e-4819-a63b-a6e0d898ba8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:28.004050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42accd-158e-4819-a63b-a6e0d898ba8a service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:28.004050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42accd-158e-4819-a63b-a6e0d898ba8a service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:28.004050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42accd-158e-4819-a63b-a6e0d898ba8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:28.016516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42accd-158e-4819-a63b-a6e0d898ba8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:28.018126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42accd-158e-4819-a63b-a6e0d898ba8a service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:28.031315 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1498/2994] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:28.592579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca3529-d930-40c7-af8b-8e976d38a063 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:28.593332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-89ca3529-d930-40c7-af8b-8e976d38a063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:28.635542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca3529-d930-40c7-af8b-8e976d38a063 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:28.649494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d1320c-90ad-4eac-8b60-db570856c00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:28.654586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1320c-90ad-4eac-8b60-db570856c00b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:28.656128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1320c-90ad-4eac-8b60-db570856c00b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:28.657782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d1320c-90ad-4eac-8b60-db570856c00b admin admin] PATH_INFO: `/v3/projects/a11903b0e9b1485d9f27e6ee742d8b97` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:28.659281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1320c-90ad-4eac-8b60-db570856c00b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a11903b0e9b1485d9f27e6ee742d8b97)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:28.661254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d1320c-90ad-4eac-8b60-db570856c00b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:28.803365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3d1320c-90ad-4eac-8b60-db570856c00b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a11903b0e9b1485d9f27e6ee742d8b97', 'request_id': 'req-f3d1320c-90ad-4eac-8b60-db570856c00b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:28.810774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3d1320c-90ad-4eac-8b60-db570856c00b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a11903b0e9b1485d9f27e6ee742d8b97', 'request_id': 'req-f3d1320c-90ad-4eac-8b60-db570856c00b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:28.909665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3d1320c-90ad-4eac-8b60-db570856c00b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a11903b0e9b1485d9f27e6ee742d8b97', 'request_id': 'req-f3d1320c-90ad-4eac-8b60-db570856c00b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:28.926994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3d1320c-90ad-4eac-8b60-db570856c00b admin admin] The token cache is being invalidate because project a11903b0e9b1485d9f27e6ee742d8b97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:08:28.927293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3d1320c-90ad-4eac-8b60-db570856c00b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:28.927979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3d1320c-90ad-4eac-8b60-db570856c00b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:28.934548 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1497/2995] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:08:28 2022] DELETE /identity/v3/projects/a11903b0e9b1485d9f27e6ee742d8b97 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:08:28.935816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f5ef1-30be-424a-97cb-9ed4b7b6afcf None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:28.935816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f5ef1-30be-424a-97cb-9ed4b7b6afcf None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:28.936417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f5ef1-30be-424a-97cb-9ed4b7b6afcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:28.962878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-053f5ef1-30be-424a-97cb-9ed4b7b6afcf None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:28.968196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-053f5ef1-30be-424a-97cb-9ed4b7b6afcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:29.010771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a16e6-564f-4897-8d81-2a238b857216 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:29.011033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a16e6-564f-4897-8d81-2a238b857216 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:29.011426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a16e6-564f-4897-8d81-2a238b857216 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:29.013553 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1498/2996] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:08:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 21:08:29.021478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f7533-8a77-4b9f-bb58-9f30c829a105 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:29.021915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f7533-8a77-4b9f-bb58-9f30c829a105 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:29.022282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550f7533-8a77-4b9f-bb58-9f30c829a105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:29.065363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-550f7533-8a77-4b9f-bb58-9f30c829a105 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:29.068419 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1499/2997] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:08:29.075906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-550f7533-8a77-4b9f-bb58-9f30c829a105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:29.079811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2b0a6-d1b3-403b-8013-4df5b388fb7c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:29.080109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2b0a6-d1b3-403b-8013-4df5b388fb7c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:29.080390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2b0a6-d1b3-403b-8013-4df5b388fb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:29.107100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-76a2b0a6-d1b3-403b-8013-4df5b388fb7c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:29.108655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-76a2b0a6-d1b3-403b-8013-4df5b388fb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:29.131711 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1500/2998] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:29.134465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2b0a6-d1b3-403b-8013-4df5b388fb7c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:29.158271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2b0a6-d1b3-403b-8013-4df5b388fb7c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:29.163667 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1499/2999] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:08:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:29.173712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dab1d3-ed40-4450-98ed-53ec0faff2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:29.174425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dab1d3-ed40-4450-98ed-53ec0faff2a7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:29.175010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dab1d3-ed40-4450-98ed-53ec0faff2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:29.175298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dab1d3-ed40-4450-98ed-53ec0faff2a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:29.176103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dab1d3-ed40-4450-98ed-53ec0faff2a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:08:29.176417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dab1d3-ed40-4450-98ed-53ec0faff2a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:29.181059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dab1d3-ed40-4450-98ed-53ec0faff2a7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:29.198040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f7533-8a77-4b9f-bb58-9f30c829a105 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:29.198040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-550f7533-8a77-4b9f-bb58-9f30c829a105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:29.211464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550f7533-8a77-4b9f-bb58-9f30c829a105 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:29.218767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c697d2-2ebe-45d4-bf73-eb4364ec12cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:29.220285 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1501/3000] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:08:29.220645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c697d2-2ebe-45d4-bf73-eb4364ec12cc service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:29.221158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c697d2-2ebe-45d4-bf73-eb4364ec12cc service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:29.221548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c697d2-2ebe-45d4-bf73-eb4364ec12cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:29.224568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dab1d3-ed40-4450-98ed-53ec0faff2a7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:29.233072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c697d2-2ebe-45d4-bf73-eb4364ec12cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:29.236304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c697d2-2ebe-45d4-bf73-eb4364ec12cc service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:29.251996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dab1d3-ed40-4450-98ed-53ec0faff2a7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:29.266557 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1500/3001] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:29.266966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9ba9d-a8a0-4a44-9ac2-52c0717b2e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:29.266966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9ba9d-a8a0-4a44-9ac2-52c0717b2e3e admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:29.266966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9ba9d-a8a0-4a44-9ac2-52c0717b2e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:29.266966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9ba9d-a8a0-4a44-9ac2-52c0717b2e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:29.267316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9ba9d-a8a0-4a44-9ac2-52c0717b2e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:29.268441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c9ba9d-a8a0-4a44-9ac2-52c0717b2e3e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:29.388215 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1502/3002] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:08:29 2022] POST /identity/v3/projects => generated 395 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:08:29.392878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c697d2-2ebe-45d4-bf73-eb4364ec12cc service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:29.408741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c697d2-2ebe-45d4-bf73-eb4364ec12cc service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:29.441448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:29.490358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481a4fa-d80e-4121-94bb-319f9514b86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:29.491292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481a4fa-d80e-4121-94bb-319f9514b86d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:29.491591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481a4fa-d80e-4121-94bb-319f9514b86d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:29.491871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481a4fa-d80e-4121-94bb-319f9514b86d admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:29.492789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481a4fa-d80e-4121-94bb-319f9514b86d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:29.497001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481a4fa-d80e-4121-94bb-319f9514b86d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:29.669811 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1501/3003] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:08:29 2022] POST /identity/v3/users => generated 298 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:08:29.677076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9ba9d-a8a0-4a44-9ac2-52c0717b2e3e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:29.690378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9ba9d-a8a0-4a44-9ac2-52c0717b2e3e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:29.697346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9822284-40f8-4ed7-a1cf-a5ef5ed149d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:29.698322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9822284-40f8-4ed7-a1cf-a5ef5ed149d5 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:29.698698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9822284-40f8-4ed7-a1cf-a5ef5ed149d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:29.699044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9822284-40f8-4ed7-a1cf-a5ef5ed149d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:29.699837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9822284-40f8-4ed7-a1cf-a5ef5ed149d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:29.701470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9822284-40f8-4ed7-a1cf-a5ef5ed149d5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:29.715600 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1503/3004] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:08:29.716930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481a4fa-d80e-4121-94bb-319f9514b86d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:29.732186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481a4fa-d80e-4121-94bb-319f9514b86d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:29.738658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd410f-f9b2-4377-bf79-000f2e1dea65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:29.739412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd410f-f9b2-4377-bf79-000f2e1dea65 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:29.740365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd410f-f9b2-4377-bf79-000f2e1dea65 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:29.740365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd410f-f9b2-4377-bf79-000f2e1dea65 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:29.741337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd410f-f9b2-4377-bf79-000f2e1dea65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:29.743646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd410f-f9b2-4377-bf79-000f2e1dea65 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:29.762193 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1502/3005] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:08:29.768764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9822284-40f8-4ed7-a1cf-a5ef5ed149d5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:29.785108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9822284-40f8-4ed7-a1cf-a5ef5ed149d5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:29.801312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e331fd2-7165-47a1-8646-e2d1b1263fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:29.802348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e331fd2-7165-47a1-8646-e2d1b1263fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:29.802702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e331fd2-7165-47a1-8646-e2d1b1263fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:29.803546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e331fd2-7165-47a1-8646-e2d1b1263fbc admin admin] PATH_INFO: `/v3/projects/301f910ecee3465e8c6e8055ef99cf8d/users/b8d6261a1bea4c7a89df60b092f068ea/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:29.841067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e331fd2-7165-47a1-8646-e2d1b1263fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=301f910ecee3465e8c6e8055ef99cf8d, user_id=b8d6261a1bea4c7a89df60b092f068ea, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:29.844640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e331fd2-7165-47a1-8646-e2d1b1263fbc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:29.851949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:29.944051 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1504/3006] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:08:29 2022] PUT /identity/v3/projects/301f910ecee3465e8c6e8055ef99cf8d/users/b8d6261a1bea4c7a89df60b092f068ea/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:08:29.956500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e526b-7083-4084-8dac-8a29a8274f0c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:29.957071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e526b-7083-4084-8dac-8a29a8274f0c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:29.957071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e526b-7083-4084-8dac-8a29a8274f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:30.020229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b12e526b-7083-4084-8dac-8a29a8274f0c None None] MFA Rules not processed for user `b8d6261a1bea4c7a89df60b092f068ea`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:30.020229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b12e526b-7083-4084-8dac-8a29a8274f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:30.121356 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1503/3007] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:08:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:30.916383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e331fd2-7165-47a1-8646-e2d1b1263fbc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:30.918337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1e331fd2-7165-47a1-8646-e2d1b1263fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:31.013876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e331fd2-7165-47a1-8646-e2d1b1263fbc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:31.023723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebf855-a62c-4fc8-9055-fb6578dafaa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:31.025145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebf855-a62c-4fc8-9055-fb6578dafaa6 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:31.025600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebf855-a62c-4fc8-9055-fb6578dafaa6 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:31.026406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebf855-a62c-4fc8-9055-fb6578dafaa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:31.035256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebf855-a62c-4fc8-9055-fb6578dafaa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:31.036877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebf855-a62c-4fc8-9055-fb6578dafaa6 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:31.046890 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1505/3008] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:31.115149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1193c-58c3-40ca-a314-ac7d01da81d9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:31.115640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1193c-58c3-40ca-a314-ac7d01da81d9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:31.115743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1193c-58c3-40ca-a314-ac7d01da81d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:31.156249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-16e1193c-58c3-40ca-a314-ac7d01da81d9 None None] MFA Rules not processed for user `b8d6261a1bea4c7a89df60b092f068ea`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:31.157588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-16e1193c-58c3-40ca-a314-ac7d01da81d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:31.178542 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1504/3009] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:08:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:31.216668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1457f-cdc5-4dc5-9d41-aa1e5c49d892 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:31.217280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1457f-cdc5-4dc5-9d41-aa1e5c49d892 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:31.217772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1457f-cdc5-4dc5-9d41-aa1e5c49d892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:31.242402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-90f1457f-cdc5-4dc5-9d41-aa1e5c49d892 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:31.243660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-90f1457f-cdc5-4dc5-9d41-aa1e5c49d892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:31.344356 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1506/3010] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:08:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:31.358200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1193c-58c3-40ca-a314-ac7d01da81d9 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:31.365937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-16e1193c-58c3-40ca-a314-ac7d01da81d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:31.457479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1193c-58c3-40ca-a314-ac7d01da81d9 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:31.464054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451be500-bdcb-433e-8f19-860435ceaef3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:31.464799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451be500-bdcb-433e-8f19-860435ceaef3 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:31.465089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451be500-bdcb-433e-8f19-860435ceaef3 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:31.465522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451be500-bdcb-433e-8f19-860435ceaef3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:31.475586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451be500-bdcb-433e-8f19-860435ceaef3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:31.477188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451be500-bdcb-433e-8f19-860435ceaef3 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:31.486380 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1505/3011] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:08:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:08:32.154926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1457f-cdc5-4dc5-9d41-aa1e5c49d892 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:32.155401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451be500-bdcb-433e-8f19-860435ceaef3 service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:32.157572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-90f1457f-cdc5-4dc5-9d41-aa1e5c49d892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:32.176242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1457f-cdc5-4dc5-9d41-aa1e5c49d892 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:32.182627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd31e63-13b4-4f5e-85f8-0a6e38d0ccc1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:32.183735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31e63-13b4-4f5e-85f8-0a6e38d0ccc1 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:32.184244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31e63-13b4-4f5e-85f8-0a6e38d0ccc1 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:32.184555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd31e63-13b4-4f5e-85f8-0a6e38d0ccc1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:32.184842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:32.186908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6dd31e63-13b4-4f5e-85f8-0a6e38d0ccc1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:32.244310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451be500-bdcb-433e-8f19-860435ceaef3 service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:32.250739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a2feb-d364-422c-8d50-c3a97abca57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:32.251621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a2feb-d364-422c-8d50-c3a97abca57a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:32.252058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a2feb-d364-422c-8d50-c3a97abca57a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:32.252340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a2feb-d364-422c-8d50-c3a97abca57a admin admin] PATH_INFO: `/v3/users/b8d6261a1bea4c7a89df60b092f068ea` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:32.253710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2a2feb-d364-422c-8d50-c3a97abca57a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8d6261a1bea4c7a89df60b092f068ea)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:32.255684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2a2feb-d364-422c-8d50-c3a97abca57a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:32.261034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd31e63-13b4-4f5e-85f8-0a6e38d0ccc1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:32.264053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd31e63-13b4-4f5e-85f8-0a6e38d0ccc1 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:32.277299 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1507/3012] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:32 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:32.471620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3b2a2feb-d364-422c-8d50-c3a97abca57a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8d6261a1bea4c7a89df60b092f068ea'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:32.483345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3b2a2feb-d364-422c-8d50-c3a97abca57a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8d6261a1bea4c7a89df60b092f068ea'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:32.484840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3b2a2feb-d364-422c-8d50-c3a97abca57a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8d6261a1bea4c7a89df60b092f068ea'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:32.485627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3b2a2feb-d364-422c-8d50-c3a97abca57a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8d6261a1bea4c7a89df60b092f068ea'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:32.565170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3b2a2feb-d364-422c-8d50-c3a97abca57a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8d6261a1bea4c7a89df60b092f068ea'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:32.590962 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1506/3013] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:08:32 2022] DELETE /identity/v3/users/b8d6261a1bea4c7a89df60b092f068ea => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:08:32.596112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb442f0-1c40-416f-8b71-6603f3cf6191 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:32.596502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb442f0-1c40-416f-8b71-6603f3cf6191 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:32.599489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb442f0-1c40-416f-8b71-6603f3cf6191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:32.621481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4cb442f0-1c40-416f-8b71-6603f3cf6191 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:32.623534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4cb442f0-1c40-416f-8b71-6603f3cf6191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:32.644108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:32.691575 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1508/3014] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:32.702211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a2feb-d364-422c-8d50-c3a97abca57a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:32.704838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3b2a2feb-d364-422c-8d50-c3a97abca57a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:32.825881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a2feb-d364-422c-8d50-c3a97abca57a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:32.831768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b7e7ea-e866-41fc-8b56-ea870e85a81d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:32.831768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7e7ea-e866-41fc-8b56-ea870e85a81d service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:32.831768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7e7ea-e866-41fc-8b56-ea870e85a81d service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:32.831768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b7e7ea-e866-41fc-8b56-ea870e85a81d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:32.843795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7e7ea-e866-41fc-8b56-ea870e85a81d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:32.846363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b7e7ea-e866-41fc-8b56-ea870e85a81d service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:32.857257 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1507/3015] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:33.200690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb442f0-1c40-416f-8b71-6603f3cf6191 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:33.204639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b242fe-c013-4444-9f7c-498989765fd4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:33.205103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b242fe-c013-4444-9f7c-498989765fd4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:33.205188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b242fe-c013-4444-9f7c-498989765fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:33.206925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4cb442f0-1c40-416f-8b71-6603f3cf6191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:33.246848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-96b242fe-c013-4444-9f7c-498989765fd4 None None] MFA Rules not processed for user `04e5b86f94914df085b76e0d89b2230e`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:33.248091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-96b242fe-c013-4444-9f7c-498989765fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:33.294350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:33.324354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb442f0-1c40-416f-8b71-6603f3cf6191 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:33.332627 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1508/3016] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:08:33 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:33.338026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68d2419-dc1b-495d-894d-1a7cdfcfcf19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:33.338817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d2419-dc1b-495d-894d-1a7cdfcfcf19 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:33.339112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d2419-dc1b-495d-894d-1a7cdfcfcf19 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:33.339470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68d2419-dc1b-495d-894d-1a7cdfcfcf19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:33.341434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a68d2419-dc1b-495d-894d-1a7cdfcfcf19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:33.355058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d2419-dc1b-495d-894d-1a7cdfcfcf19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:33.356675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68d2419-dc1b-495d-894d-1a7cdfcfcf19 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:33.368885 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1509/3017] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:33.389528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b242fe-c013-4444-9f7c-498989765fd4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:33.391639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-96b242fe-c013-4444-9f7c-498989765fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:33.414615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b242fe-c013-4444-9f7c-498989765fd4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:33.419711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d034a383-ff07-491b-9fb2-b7ef5e72aac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:33.420762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034a383-ff07-491b-9fb2-b7ef5e72aac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:33.420883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034a383-ff07-491b-9fb2-b7ef5e72aac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:33.421198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d034a383-ff07-491b-9fb2-b7ef5e72aac3 admin admin] PATH_INFO: `/v3/projects/301f910ecee3465e8c6e8055ef99cf8d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:33.423480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034a383-ff07-491b-9fb2-b7ef5e72aac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=301f910ecee3465e8c6e8055ef99cf8d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:33.425899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d034a383-ff07-491b-9fb2-b7ef5e72aac3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:33.545219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d034a383-ff07-491b-9fb2-b7ef5e72aac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '301f910ecee3465e8c6e8055ef99cf8d', 'request_id': 'req-d034a383-ff07-491b-9fb2-b7ef5e72aac3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:33.551592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d034a383-ff07-491b-9fb2-b7ef5e72aac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '301f910ecee3465e8c6e8055ef99cf8d', 'request_id': 'req-d034a383-ff07-491b-9fb2-b7ef5e72aac3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:33.605794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d034a383-ff07-491b-9fb2-b7ef5e72aac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '301f910ecee3465e8c6e8055ef99cf8d', 'request_id': 'req-d034a383-ff07-491b-9fb2-b7ef5e72aac3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:33.621351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d034a383-ff07-491b-9fb2-b7ef5e72aac3 admin admin] The token cache is being invalidate because project 301f910ecee3465e8c6e8055ef99cf8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:08:33.621650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d034a383-ff07-491b-9fb2-b7ef5e72aac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:33.622362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d034a383-ff07-491b-9fb2-b7ef5e72aac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:08:33.623961 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1509/3018] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:08:33 2022] DELETE /identity/v3/projects/301f910ecee3465e8c6e8055ef99cf8d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:08:33.629602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3bf5b-002e-4c6f-971d-53ed55ba69fe None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:33.630229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3bf5b-002e-4c6f-971d-53ed55ba69fe None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:33.630720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3bf5b-002e-4c6f-971d-53ed55ba69fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:33.649710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6fd3bf5b-002e-4c6f-971d-53ed55ba69fe None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:33.651597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6fd3bf5b-002e-4c6f-971d-53ed55ba69fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:33.738822 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1510/3019] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:08:33.740452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbae048-3cc1-4762-a437-65b207ced8d0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:33.740751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbae048-3cc1-4762-a437-65b207ced8d0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:33.741333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbae048-3cc1-4762-a437-65b207ced8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:33.763681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cfbae048-3cc1-4762-a437-65b207ced8d0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:33.764991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cfbae048-3cc1-4762-a437-65b207ced8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:33.777161 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1510/3020] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:33.778560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3bf5b-002e-4c6f-971d-53ed55ba69fe None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:33.802361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3bf5b-002e-4c6f-971d-53ed55ba69fe None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:33.820405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc788d81-5518-443f-b94b-6621023abe2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:33.821238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc788d81-5518-443f-b94b-6621023abe2f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:33.821584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc788d81-5518-443f-b94b-6621023abe2f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:33.822021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc788d81-5518-443f-b94b-6621023abe2f admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:33.822653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc788d81-5518-443f-b94b-6621023abe2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:08:33.822962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc788d81-5518-443f-b94b-6621023abe2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:33.824593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc788d81-5518-443f-b94b-6621023abe2f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:33.839054 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1511/3021] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:08:33.843437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbae048-3cc1-4762-a437-65b207ced8d0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:33.853151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbae048-3cc1-4762-a437-65b207ced8d0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:33.858208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943320a-1258-49d9-9b50-e786caf6740b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:33.858882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943320a-1258-49d9-9b50-e786caf6740b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:33.859166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943320a-1258-49d9-9b50-e786caf6740b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:33.913186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7943320a-1258-49d9-9b50-e786caf6740b admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:33.913186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7943320a-1258-49d9-9b50-e786caf6740b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:33.913186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7943320a-1258-49d9-9b50-e786caf6740b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:34.002187 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1511/3022] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:08:33 2022] POST /identity/v3/projects => generated 379 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:08:34.007122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc788d81-5518-443f-b94b-6621023abe2f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:34.020467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc788d81-5518-443f-b94b-6621023abe2f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:34.078279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:34.102549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076670a-faeb-4e27-acd7-39516452cde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:34.103280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076670a-faeb-4e27-acd7-39516452cde4 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:34.103591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076670a-faeb-4e27-acd7-39516452cde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:34.104101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076670a-faeb-4e27-acd7-39516452cde4 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:34.104815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076670a-faeb-4e27-acd7-39516452cde4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:34.106949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076670a-faeb-4e27-acd7-39516452cde4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:34.223671 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1512/3023] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:08:34 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:08:34.230292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943320a-1258-49d9-9b50-e786caf6740b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:34.242280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7943320a-1258-49d9-9b50-e786caf6740b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:34.247143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd13d7-65b0-4dec-9617-5a89625e16bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:34.247811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd13d7-65b0-4dec-9617-5a89625e16bf admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:34.248098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd13d7-65b0-4dec-9617-5a89625e16bf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:34.248377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fd13d7-65b0-4dec-9617-5a89625e16bf admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:34.249124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd13d7-65b0-4dec-9617-5a89625e16bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:34.250780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fd13d7-65b0-4dec-9617-5a89625e16bf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:34.261475 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1512/3024] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:08:34.265653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076670a-faeb-4e27-acd7-39516452cde4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:34.276440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076670a-faeb-4e27-acd7-39516452cde4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:34.281206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcbe5a6-3e33-47a3-9b13-368fa98777d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:34.282452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbe5a6-3e33-47a3-9b13-368fa98777d9 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:34.282775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbe5a6-3e33-47a3-9b13-368fa98777d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:34.283057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcbe5a6-3e33-47a3-9b13-368fa98777d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:34.283782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbe5a6-3e33-47a3-9b13-368fa98777d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:34.289289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcbe5a6-3e33-47a3-9b13-368fa98777d9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:34.295405 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1513/3025] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:08:34.299685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd13d7-65b0-4dec-9617-5a89625e16bf admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:34.309670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fd13d7-65b0-4dec-9617-5a89625e16bf admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:34.314679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29e0e3-b1d6-43ea-813b-4b8ef197125f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:34.315554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29e0e3-b1d6-43ea-813b-4b8ef197125f admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:34.316048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29e0e3-b1d6-43ea-813b-4b8ef197125f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:34.316322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c29e0e3-b1d6-43ea-813b-4b8ef197125f admin admin] PATH_INFO: `/v3/projects/e96bf6518d66430399de823144f7bea8/users/f08e71479f9d4ef0a30681cdfdd6fef8/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:34.332558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29e0e3-b1d6-43ea-813b-4b8ef197125f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e96bf6518d66430399de823144f7bea8, user_id=f08e71479f9d4ef0a30681cdfdd6fef8, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:34.335825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c29e0e3-b1d6-43ea-813b-4b8ef197125f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:34.423473 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1513/3026] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:08:34 2022] PUT /identity/v3/projects/e96bf6518d66430399de823144f7bea8/users/f08e71479f9d4ef0a30681cdfdd6fef8/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:08:34.429439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c353ce-88f6-45ee-9015-0434aa312d04 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:34.429739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c353ce-88f6-45ee-9015-0434aa312d04 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:34.430296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c353ce-88f6-45ee-9015-0434aa312d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:34.450272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-31c353ce-88f6-45ee-9015-0434aa312d04 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:34.451508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-31c353ce-88f6-45ee-9015-0434aa312d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:34.534402 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1514/3027] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:34.547950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29e0e3-b1d6-43ea-813b-4b8ef197125f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:34.550643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9c29e0e3-b1d6-43ea-813b-4b8ef197125f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:34.615825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c29e0e3-b1d6-43ea-813b-4b8ef197125f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:34.620406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3210414-3bfd-4212-bfe1-7f974d60c404 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:34.621088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3210414-3bfd-4212-bfe1-7f974d60c404 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:34.621371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3210414-3bfd-4212-bfe1-7f974d60c404 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:34.621647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3210414-3bfd-4212-bfe1-7f974d60c404 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:34.638075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3210414-3bfd-4212-bfe1-7f974d60c404 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:34.638075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3210414-3bfd-4212-bfe1-7f974d60c404 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:34.647598 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1514/3028] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:35.980770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f288a-e23c-4889-8318-fa31964691db None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:35.981325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f288a-e23c-4889-8318-fa31964691db None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:35.981325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0f288a-e23c-4889-8318-fa31964691db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:36.043430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9f0f288a-e23c-4889-8318-fa31964691db None None] MFA Rules not processed for user `f08e71479f9d4ef0a30681cdfdd6fef8`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:36.043430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9f0f288a-e23c-4889-8318-fa31964691db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:36.168634 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1515/3029] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:08:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:36.319260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3210414-3bfd-4212-bfe1-7f974d60c404 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:36.321355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b3210414-3bfd-4212-bfe1-7f974d60c404 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:36.333931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:36.433351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3210414-3bfd-4212-bfe1-7f974d60c404 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:36.441361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bacc16-7f64-48db-8cff-802dceaf67f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:36.442063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bacc16-7f64-48db-8cff-802dceaf67f3 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:36.442347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bacc16-7f64-48db-8cff-802dceaf67f3 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:36.442785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bacc16-7f64-48db-8cff-802dceaf67f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:36.463692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bacc16-7f64-48db-8cff-802dceaf67f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:36.465308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bacc16-7f64-48db-8cff-802dceaf67f3 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:36.475831 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1515/3030] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:37.588640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15424922-25e3-44c9-9b49-ab65318d4373 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:37.589146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15424922-25e3-44c9-9b49-ab65318d4373 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:37.589402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15424922-25e3-44c9-9b49-ab65318d4373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:37.609176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-15424922-25e3-44c9-9b49-ab65318d4373 None None] MFA Rules not processed for user `f08e71479f9d4ef0a30681cdfdd6fef8`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:08:37.611343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-15424922-25e3-44c9-9b49-ab65318d4373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:37.636309 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1516/3031] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:08:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:08:38.770817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bacc16-7f64-48db-8cff-802dceaf67f3 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:38.773250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a2bacc16-7f64-48db-8cff-802dceaf67f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:38.804074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bacc16-7f64-48db-8cff-802dceaf67f3 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:38.842074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484c78f-f588-428f-b7bf-52e97e359d1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:38.847061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484c78f-f588-428f-b7bf-52e97e359d1b service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:38.867196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484c78f-f588-428f-b7bf-52e97e359d1b service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:38.868482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484c78f-f588-428f-b7bf-52e97e359d1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:38.874655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0484c78f-f588-428f-b7bf-52e97e359d1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:39.030901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484c78f-f588-428f-b7bf-52e97e359d1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:39.037552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484c78f-f588-428f-b7bf-52e97e359d1b service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:39.064360 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1516/3032] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:44.883089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15424922-25e3-44c9-9b49-ab65318d4373 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:08:44.899618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15424922-25e3-44c9-9b49-ab65318d4373 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:08:44.906264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bcaab-f152-44c3-9cf6-6a1d43ce63e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:08:44.907807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcaab-f152-44c3-9cf6-6a1d43ce63e6 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:08:44.908194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcaab-f152-44c3-9cf6-6a1d43ce63e6 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:08:44.908749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcaab-f152-44c3-9cf6-6a1d43ce63e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:08:44.910764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-872bcaab-f152-44c3-9cf6-6a1d43ce63e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:08:44.925888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872bcaab-f152-44c3-9cf6-6a1d43ce63e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:08:44.928399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872bcaab-f152-44c3-9cf6-6a1d43ce63e6 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:08:44.944498 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1517/3033] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:08:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:08:49.053221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=59 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:54.937272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:01.046793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484c78f-f588-428f-b7bf-52e97e359d1b service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:01.051041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0484c78f-f588-428f-b7bf-52e97e359d1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:01.089714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484c78f-f588-428f-b7bf-52e97e359d1b service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:01.105225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10943b0b-ca05-4b59-b1eb-851a403cb9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:01.106365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10943b0b-ca05-4b59-b1eb-851a403cb9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:01.109358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10943b0b-ca05-4b59-b1eb-851a403cb9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:01.109948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10943b0b-ca05-4b59-b1eb-851a403cb9a8 admin admin] PATH_INFO: `/v3/users/5e54b51913be4a2bab74aae347785727` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:01.116152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10943b0b-ca05-4b59-b1eb-851a403cb9a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e54b51913be4a2bab74aae347785727)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:01.127890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10943b0b-ca05-4b59-b1eb-851a403cb9a8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:01.498816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10943b0b-ca05-4b59-b1eb-851a403cb9a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e54b51913be4a2bab74aae347785727'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:01.517271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10943b0b-ca05-4b59-b1eb-851a403cb9a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e54b51913be4a2bab74aae347785727'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:01.518567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10943b0b-ca05-4b59-b1eb-851a403cb9a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e54b51913be4a2bab74aae347785727'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:01.519556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10943b0b-ca05-4b59-b1eb-851a403cb9a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e54b51913be4a2bab74aae347785727'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:01.646959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10943b0b-ca05-4b59-b1eb-851a403cb9a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e54b51913be4a2bab74aae347785727'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:01.692666 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1517/3034] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:09:01 2022] DELETE /identity/v3/users/5e54b51913be4a2bab74aae347785727 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:01.694517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46943484-d119-4d4a-b6e3-02cd65fca3a8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:01.695316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46943484-d119-4d4a-b6e3-02cd65fca3a8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:01.695760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46943484-d119-4d4a-b6e3-02cd65fca3a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:01.721601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-46943484-d119-4d4a-b6e3-02cd65fca3a8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:01.723917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-46943484-d119-4d4a-b6e3-02cd65fca3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:01.814311 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1518/3035] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:01.831254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10943b0b-ca05-4b59-b1eb-851a403cb9a8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:01.832721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-10943b0b-ca05-4b59-b1eb-851a403cb9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:01.925507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10943b0b-ca05-4b59-b1eb-851a403cb9a8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:01.930632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b540511-13a3-4bc3-90ad-79915b047336 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:01.930886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b540511-13a3-4bc3-90ad-79915b047336 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:01.931175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b540511-13a3-4bc3-90ad-79915b047336 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:01.931511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b540511-13a3-4bc3-90ad-79915b047336 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:01.946058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b540511-13a3-4bc3-90ad-79915b047336 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:01.947517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b540511-13a3-4bc3-90ad-79915b047336 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:01.959285 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1518/3036] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:02.599623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46943484-d119-4d4a-b6e3-02cd65fca3a8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:02.601285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-46943484-d119-4d4a-b6e3-02cd65fca3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:02.627029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46943484-d119-4d4a-b6e3-02cd65fca3a8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:02.642677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8aef82-a5cb-4865-b618-c2f1481c40e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:02.643738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8aef82-a5cb-4865-b618-c2f1481c40e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:02.644033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8aef82-a5cb-4865-b618-c2f1481c40e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:02.644314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8aef82-a5cb-4865-b618-c2f1481c40e3 admin admin] PATH_INFO: `/v3/projects/348fbbc628b44152b05362657c11230d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:02.653534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8aef82-a5cb-4865-b618-c2f1481c40e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=348fbbc628b44152b05362657c11230d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:02.653534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8aef82-a5cb-4865-b618-c2f1481c40e3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:02.828222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ee8aef82-a5cb-4865-b618-c2f1481c40e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '348fbbc628b44152b05362657c11230d', 'request_id': 'req-ee8aef82-a5cb-4865-b618-c2f1481c40e3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:02.838676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ee8aef82-a5cb-4865-b618-c2f1481c40e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '348fbbc628b44152b05362657c11230d', 'request_id': 'req-ee8aef82-a5cb-4865-b618-c2f1481c40e3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:02.905188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ee8aef82-a5cb-4865-b618-c2f1481c40e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '348fbbc628b44152b05362657c11230d', 'request_id': 'req-ee8aef82-a5cb-4865-b618-c2f1481c40e3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:02.939841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ee8aef82-a5cb-4865-b618-c2f1481c40e3 admin admin] The token cache is being invalidate because project 348fbbc628b44152b05362657c11230d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:09:02.940107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ee8aef82-a5cb-4865-b618-c2f1481c40e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:02.940922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ee8aef82-a5cb-4865-b618-c2f1481c40e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:02.943485 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1519/3037] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:09:02 2022] DELETE /identity/v3/projects/348fbbc628b44152b05362657c11230d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:02.947252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b540511-13a3-4bc3-90ad-79915b047336 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:02.961201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b540511-13a3-4bc3-90ad-79915b047336 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:03.039370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78701c-020c-4881-b764-65ccb3b2a50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:03.040259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78701c-020c-4881-b764-65ccb3b2a50f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:03.040546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78701c-020c-4881-b764-65ccb3b2a50f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:03.040827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d78701c-020c-4881-b764-65ccb3b2a50f admin admin] PATH_INFO: `/v3/users/2807965e53b04a3086ef1b00ffb481af` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:03.042147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d78701c-020c-4881-b764-65ccb3b2a50f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2807965e53b04a3086ef1b00ffb481af)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:03.044313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d78701c-020c-4881-b764-65ccb3b2a50f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:03.064972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:03.220044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d78701c-020c-4881-b764-65ccb3b2a50f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2807965e53b04a3086ef1b00ffb481af'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:03.227357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d78701c-020c-4881-b764-65ccb3b2a50f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2807965e53b04a3086ef1b00ffb481af'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:03.228067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d78701c-020c-4881-b764-65ccb3b2a50f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2807965e53b04a3086ef1b00ffb481af'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:03.228753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d78701c-020c-4881-b764-65ccb3b2a50f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2807965e53b04a3086ef1b00ffb481af'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:03.336127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d78701c-020c-4881-b764-65ccb3b2a50f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2807965e53b04a3086ef1b00ffb481af'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:03.365299 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1519/3038] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:09:02 2022] DELETE /identity/v3/users/2807965e53b04a3086ef1b00ffb481af => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:09:03.587534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8aef82-a5cb-4865-b618-c2f1481c40e3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:03.589028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ee8aef82-a5cb-4865-b618-c2f1481c40e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:03.605187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8aef82-a5cb-4865-b618-c2f1481c40e3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:03.684639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1043ffc-5b1b-4807-867d-61cbd40c394d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:03.686001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1043ffc-5b1b-4807-867d-61cbd40c394d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:03.686740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1043ffc-5b1b-4807-867d-61cbd40c394d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:03.687105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1043ffc-5b1b-4807-867d-61cbd40c394d admin admin] PATH_INFO: `/v3/users/c14b965da64c40e382c69050ba01b183` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:03.688557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1043ffc-5b1b-4807-867d-61cbd40c394d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c14b965da64c40e382c69050ba01b183)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:03.690627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1043ffc-5b1b-4807-867d-61cbd40c394d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:03.800940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:03.881904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78701c-020c-4881-b764-65ccb3b2a50f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:03.883469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5d78701c-020c-4881-b764-65ccb3b2a50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:03.907789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d78701c-020c-4881-b764-65ccb3b2a50f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:03.915664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86f1b5-94d0-42e4-95dd-1ff315b937b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:03.916648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86f1b5-94d0-42e4-95dd-1ff315b937b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:03.916935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86f1b5-94d0-42e4-95dd-1ff315b937b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:03.917221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86f1b5-94d0-42e4-95dd-1ff315b937b0 admin admin] PATH_INFO: `/v3/projects/b83e29ca4af045138d1fbeff233f5e05` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:03.918687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86f1b5-94d0-42e4-95dd-1ff315b937b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b83e29ca4af045138d1fbeff233f5e05)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:03.920686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86f1b5-94d0-42e4-95dd-1ff315b937b0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:03.922496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f1043ffc-5b1b-4807-867d-61cbd40c394d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c14b965da64c40e382c69050ba01b183'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:03.924476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f1043ffc-5b1b-4807-867d-61cbd40c394d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c14b965da64c40e382c69050ba01b183'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:03.952391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f1043ffc-5b1b-4807-867d-61cbd40c394d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c14b965da64c40e382c69050ba01b183'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:04.101052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f1043ffc-5b1b-4807-867d-61cbd40c394d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c14b965da64c40e382c69050ba01b183'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:04.101931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f1043ffc-5b1b-4807-867d-61cbd40c394d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c14b965da64c40e382c69050ba01b183'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:04.130407 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1520/3039] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:09:03 2022] DELETE /identity/v3/users/c14b965da64c40e382c69050ba01b183 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:04.135293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9a90d-55af-407d-ae39-3ec42fb974bb None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:04.135584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9a90d-55af-407d-ae39-3ec42fb974bb None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:04.136063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9a90d-55af-407d-ae39-3ec42fb974bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:04.158577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6c86f1b5-94d0-42e4-95dd-1ff315b937b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b83e29ca4af045138d1fbeff233f5e05', 'request_id': 'req-6c86f1b5-94d0-42e4-95dd-1ff315b937b0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:04.160549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-fce9a90d-55af-407d-ae39-3ec42fb974bb None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:04.162848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fce9a90d-55af-407d-ae39-3ec42fb974bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:04.168344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6c86f1b5-94d0-42e4-95dd-1ff315b937b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b83e29ca4af045138d1fbeff233f5e05', 'request_id': 'req-6c86f1b5-94d0-42e4-95dd-1ff315b937b0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:04.247812 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1521/3040] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:04.263024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9a90d-55af-407d-ae39-3ec42fb974bb None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:04.264934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fce9a90d-55af-407d-ae39-3ec42fb974bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:04.267281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6c86f1b5-94d0-42e4-95dd-1ff315b937b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b83e29ca4af045138d1fbeff233f5e05', 'request_id': 'req-6c86f1b5-94d0-42e4-95dd-1ff315b937b0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:04.279473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6c86f1b5-94d0-42e4-95dd-1ff315b937b0 admin admin] The token cache is being invalidate because project b83e29ca4af045138d1fbeff233f5e05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:09:04.279706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6c86f1b5-94d0-42e4-95dd-1ff315b937b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:04.281189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6c86f1b5-94d0-42e4-95dd-1ff315b937b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:04.283089 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1520/3041] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:09:03 2022] DELETE /identity/v3/projects/b83e29ca4af045138d1fbeff233f5e05 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:09:04.289775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00966a5-9b45-4ad7-9004-c567a4139264 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:04.290296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00966a5-9b45-4ad7-9004-c567a4139264 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:04.290579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00966a5-9b45-4ad7-9004-c567a4139264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:04.313598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f00966a5-9b45-4ad7-9004-c567a4139264 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:04.314885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f00966a5-9b45-4ad7-9004-c567a4139264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:04.325778 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1521/3042] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:09:04.333412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea282ca-0317-4df7-bbdb-4c8f167143c9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:04.333942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea282ca-0317-4df7-bbdb-4c8f167143c9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:04.334469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea282ca-0317-4df7-bbdb-4c8f167143c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:04.343501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9a90d-55af-407d-ae39-3ec42fb974bb None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:04.351230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62429db9-ef32-4978-aa74-ed1dab63437d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:04.353199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62429db9-ef32-4978-aa74-ed1dab63437d service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:04.353756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62429db9-ef32-4978-aa74-ed1dab63437d service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:04.354175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62429db9-ef32-4978-aa74-ed1dab63437d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:04.357723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-eea282ca-0317-4df7-bbdb-4c8f167143c9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:04.358945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-62429db9-ef32-4978-aa74-ed1dab63437d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:04.360343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eea282ca-0317-4df7-bbdb-4c8f167143c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:04.378242 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1522/3043] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:09:04.382522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea282ca-0317-4df7-bbdb-4c8f167143c9 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:04.393096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea282ca-0317-4df7-bbdb-4c8f167143c9 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:04.398513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624f1de-0a37-46b0-ab1f-83adcb1e62d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:04.399389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624f1de-0a37-46b0-ab1f-83adcb1e62d5 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:04.399856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624f1de-0a37-46b0-ab1f-83adcb1e62d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:04.400134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624f1de-0a37-46b0-ab1f-83adcb1e62d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:04.400868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624f1de-0a37-46b0-ab1f-83adcb1e62d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:09:04.401173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624f1de-0a37-46b0-ab1f-83adcb1e62d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:04.402818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f624f1de-0a37-46b0-ab1f-83adcb1e62d5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:04.403411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62429db9-ef32-4978-aa74-ed1dab63437d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:04.405010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62429db9-ef32-4978-aa74-ed1dab63437d service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:04.414885 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1523/3044] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:09:04.421213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624f1de-0a37-46b0-ab1f-83adcb1e62d5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:04.430968 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1522/3045] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:09:04.452279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624f1de-0a37-46b0-ab1f-83adcb1e62d5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:04.465124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18f7c4-a849-4235-a191-ea3843d4f7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:04.468243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18f7c4-a849-4235-a191-ea3843d4f7e4 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:04.469036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18f7c4-a849-4235-a191-ea3843d4f7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:04.469772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd18f7c4-a849-4235-a191-ea3843d4f7e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:04.471723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18f7c4-a849-4235-a191-ea3843d4f7e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:04.477570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd18f7c4-a849-4235-a191-ea3843d4f7e4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:04.658355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62429db9-ef32-4978-aa74-ed1dab63437d service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:04.668037 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1524/3046] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:09:04 2022] POST /identity/v3/projects => generated 375 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:04.672761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62429db9-ef32-4978-aa74-ed1dab63437d service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:04.747171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4915a-3f57-4871-b650-fb32143b0644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:04.747862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4915a-3f57-4871-b650-fb32143b0644 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:04.748280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4915a-3f57-4871-b650-fb32143b0644 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:04.748567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4915a-3f57-4871-b650-fb32143b0644 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:04.749467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4915a-3f57-4871-b650-fb32143b0644 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:04.753189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4915a-3f57-4871-b650-fb32143b0644 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:04.878936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18f7c4-a849-4235-a191-ea3843d4f7e4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:04.889975 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1523/3047] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:09:04 2022] POST /identity/v3/users => generated 288 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:04.895322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd18f7c4-a849-4235-a191-ea3843d4f7e4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:04.907132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0f31f-c704-4199-b012-cbe6a9297f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:04.908395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0f31f-c704-4199-b012-cbe6a9297f60 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:04.922932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0f31f-c704-4199-b012-cbe6a9297f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:04.924071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb0f31f-c704-4199-b012-cbe6a9297f60 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:04.924867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0f31f-c704-4199-b012-cbe6a9297f60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:04.927226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb0f31f-c704-4199-b012-cbe6a9297f60 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:04.939172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4915a-3f57-4871-b650-fb32143b0644 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:04.940980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-28a4915a-3f57-4871-b650-fb32143b0644 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:04.961712 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1525/3048] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:09:04.964762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4915a-3f57-4871-b650-fb32143b0644 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:04.970564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0f31f-c704-4199-b012-cbe6a9297f60 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:04.976150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac5295-aad8-45d5-91c3-cb196570c7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:04.977768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac5295-aad8-45d5-91c3-cb196570c7ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:04.982830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb0f31f-c704-4199-b012-cbe6a9297f60 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:04.987410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac5295-aad8-45d5-91c3-cb196570c7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:04.991097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb81290-76dd-485f-ba82-4f9abf14c2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:04.992029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb81290-76dd-485f-ba82-4f9abf14c2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:04.995766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb81290-76dd-485f-ba82-4f9abf14c2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:04.995766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb81290-76dd-485f-ba82-4f9abf14c2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:04.995766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb81290-76dd-485f-ba82-4f9abf14c2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:04.995766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb81290-76dd-485f-ba82-4f9abf14c2d9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:04.998526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac5295-aad8-45d5-91c3-cb196570c7ab admin admin] PATH_INFO: `/v3/projects/d6c38a96a96b4fba9e41ac58f5054803` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:05.007870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac5295-aad8-45d5-91c3-cb196570c7ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6c38a96a96b4fba9e41ac58f5054803)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:05.008519 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1526/3049] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:09:05.011172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac5295-aad8-45d5-91c3-cb196570c7ab admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:05.012233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb81290-76dd-485f-ba82-4f9abf14c2d9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:05.035792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb81290-76dd-485f-ba82-4f9abf14c2d9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:05.049440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73136025-f242-43ab-9db9-e7dc4a4344ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:05.050678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73136025-f242-43ab-9db9-e7dc4a4344ad admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:05.051007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73136025-f242-43ab-9db9-e7dc4a4344ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:05.051299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73136025-f242-43ab-9db9-e7dc4a4344ad admin admin] PATH_INFO: `/v3/projects/0dd2a795de30463db3d150e44df69489/users/5c65140c062f4eedb8bfe963137de014/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:05.077405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:05.096896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73136025-f242-43ab-9db9-e7dc4a4344ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dd2a795de30463db3d150e44df69489, user_id=5c65140c062f4eedb8bfe963137de014, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:05.099967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73136025-f242-43ab-9db9-e7dc4a4344ad admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:05.166330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:05.178043 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1527/3050] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:09:05 2022] PUT /identity/v3/projects/0dd2a795de30463db3d150e44df69489/users/5c65140c062f4eedb8bfe963137de014/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:05.183502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83699c5-4a64-48e4-8889-c7b3e159365e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:05.183874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83699c5-4a64-48e4-8889-c7b3e159365e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:05.186108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83699c5-4a64-48e4-8889-c7b3e159365e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:05.194101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aeac5295-aad8-45d5-91c3-cb196570c7ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6c38a96a96b4fba9e41ac58f5054803', 'request_id': 'req-aeac5295-aad8-45d5-91c3-cb196570c7ab'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:05.200986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aeac5295-aad8-45d5-91c3-cb196570c7ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6c38a96a96b4fba9e41ac58f5054803', 'request_id': 'req-aeac5295-aad8-45d5-91c3-cb196570c7ab'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:05.208221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a83699c5-4a64-48e4-8889-c7b3e159365e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:05.209751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a83699c5-4a64-48e4-8889-c7b3e159365e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:05.279211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aeac5295-aad8-45d5-91c3-cb196570c7ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6c38a96a96b4fba9e41ac58f5054803', 'request_id': 'req-aeac5295-aad8-45d5-91c3-cb196570c7ab'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:05.297895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aeac5295-aad8-45d5-91c3-cb196570c7ab admin admin] The token cache is being invalidate because project d6c38a96a96b4fba9e41ac58f5054803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:09:05.298378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aeac5295-aad8-45d5-91c3-cb196570c7ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:05.298713 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1528/3051] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:09:05.299228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aeac5295-aad8-45d5-91c3-cb196570c7ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:05.306165 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1524/3052] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:09:04 2022] DELETE /identity/v3/projects/d6c38a96a96b4fba9e41ac58f5054803 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:05.310156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83699c5-4a64-48e4-8889-c7b3e159365e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:05.316398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac5295-aad8-45d5-91c3-cb196570c7ab admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:05.317697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-aeac5295-aad8-45d5-91c3-cb196570c7ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:05.321998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83699c5-4a64-48e4-8889-c7b3e159365e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:05.331313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a56ca0-befe-442b-aeb7-dc259a2a88a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:05.332730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a56ca0-befe-442b-aeb7-dc259a2a88a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:05.333170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a56ca0-befe-442b-aeb7-dc259a2a88a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:05.333824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a56ca0-befe-442b-aeb7-dc259a2a88a4 admin admin] PATH_INFO: `/v3/users/a43b061d4e864c289ac541d11e46c4de` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:05.335764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a56ca0-befe-442b-aeb7-dc259a2a88a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a43b061d4e864c289ac541d11e46c4de)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:05.337737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a56ca0-befe-442b-aeb7-dc259a2a88a4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:05.408455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac5295-aad8-45d5-91c3-cb196570c7ab admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:05.413092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7116c-ff9e-4018-9098-6981ab2025f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:05.413910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7116c-ff9e-4018-9098-6981ab2025f4 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:05.414081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7116c-ff9e-4018-9098-6981ab2025f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:05.414391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7116c-ff9e-4018-9098-6981ab2025f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:05.416485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-15d7116c-ff9e-4018-9098-6981ab2025f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:05.440048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7116c-ff9e-4018-9098-6981ab2025f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:05.440048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7116c-ff9e-4018-9098-6981ab2025f4 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:05.444827 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1525/3053] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:05.596344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-83a56ca0-befe-442b-aeb7-dc259a2a88a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a43b061d4e864c289ac541d11e46c4de'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:05.605260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-83a56ca0-befe-442b-aeb7-dc259a2a88a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a43b061d4e864c289ac541d11e46c4de'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:05.609640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-83a56ca0-befe-442b-aeb7-dc259a2a88a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a43b061d4e864c289ac541d11e46c4de'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:05.609640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-83a56ca0-befe-442b-aeb7-dc259a2a88a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a43b061d4e864c289ac541d11e46c4de'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:05.725594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-83a56ca0-befe-442b-aeb7-dc259a2a88a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a43b061d4e864c289ac541d11e46c4de'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:05.753860 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1529/3054] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:09:05 2022] DELETE /identity/v3/users/a43b061d4e864c289ac541d11e46c4de => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:06.258020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7116c-ff9e-4018-9098-6981ab2025f4 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:06.259584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-15d7116c-ff9e-4018-9098-6981ab2025f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:06.273151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7116c-ff9e-4018-9098-6981ab2025f4 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:06.345066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee340dc-97bc-4f2f-b3bf-de538972bad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:06.346045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee340dc-97bc-4f2f-b3bf-de538972bad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:06.349881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee340dc-97bc-4f2f-b3bf-de538972bad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:06.350191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee340dc-97bc-4f2f-b3bf-de538972bad2 admin admin] PATH_INFO: `/v3/projects/01b8421c635c440eaec446bfb659bd22` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:06.352626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee340dc-97bc-4f2f-b3bf-de538972bad2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01b8421c635c440eaec446bfb659bd22)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:06.370165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee340dc-97bc-4f2f-b3bf-de538972bad2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:06.550501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9ee340dc-97bc-4f2f-b3bf-de538972bad2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01b8421c635c440eaec446bfb659bd22', 'request_id': 'req-9ee340dc-97bc-4f2f-b3bf-de538972bad2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:06.559918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9ee340dc-97bc-4f2f-b3bf-de538972bad2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01b8421c635c440eaec446bfb659bd22', 'request_id': 'req-9ee340dc-97bc-4f2f-b3bf-de538972bad2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:06.682407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9ee340dc-97bc-4f2f-b3bf-de538972bad2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01b8421c635c440eaec446bfb659bd22', 'request_id': 'req-9ee340dc-97bc-4f2f-b3bf-de538972bad2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:06.700758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9ee340dc-97bc-4f2f-b3bf-de538972bad2 admin admin] The token cache is being invalidate because project 01b8421c635c440eaec446bfb659bd22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:09:06.705199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9ee340dc-97bc-4f2f-b3bf-de538972bad2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:06.709527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9ee340dc-97bc-4f2f-b3bf-de538972bad2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:06.713246 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1526/3055] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:09:06 2022] DELETE /identity/v3/projects/01b8421c635c440eaec446bfb659bd22 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:06.720927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb17184-240b-4649-b765-e66a5899ad89 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:06.721366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb17184-240b-4649-b765-e66a5899ad89 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:06.721496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb17184-240b-4649-b765-e66a5899ad89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:06.768318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fbb17184-240b-4649-b765-e66a5899ad89 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:06.791046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fbb17184-240b-4649-b765-e66a5899ad89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:06.925991 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1530/3056] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:06.934878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a167b0-e8b9-4f31-8a31-112f26553f15 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:06.935177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a167b0-e8b9-4f31-8a31-112f26553f15 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:06.935989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a167b0-e8b9-4f31-8a31-112f26553f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:06.987131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-24a167b0-e8b9-4f31-8a31-112f26553f15 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:06.989875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-24a167b0-e8b9-4f31-8a31-112f26553f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:07.000252 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1527/3057] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:07.013051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb17184-240b-4649-b765-e66a5899ad89 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:07.040296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb17184-240b-4649-b765-e66a5899ad89 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:07.049281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3832baf-06f4-4782-80d2-eab548c258db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:07.050331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3832baf-06f4-4782-80d2-eab548c258db admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:07.050620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3832baf-06f4-4782-80d2-eab548c258db admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:07.050900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3832baf-06f4-4782-80d2-eab548c258db admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:07.051650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3832baf-06f4-4782-80d2-eab548c258db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:09:07.052018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3832baf-06f4-4782-80d2-eab548c258db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:07.053569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3832baf-06f4-4782-80d2-eab548c258db admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:07.073519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:07.098792 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1531/3058] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:09:07.103240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a167b0-e8b9-4f31-8a31-112f26553f15 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:07.116039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a167b0-e8b9-4f31-8a31-112f26553f15 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:07.133691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08838d-8f2f-48b0-b290-8e2a17f78761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:07.135074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08838d-8f2f-48b0-b290-8e2a17f78761 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:07.135074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08838d-8f2f-48b0-b290-8e2a17f78761 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:07.135437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d08838d-8f2f-48b0-b290-8e2a17f78761 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:07.136299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d08838d-8f2f-48b0-b290-8e2a17f78761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:07.142870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:07.145867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d08838d-8f2f-48b0-b290-8e2a17f78761 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:07.247622 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1528/3059] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:09:07 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:09:07.252618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3832baf-06f4-4782-80d2-eab548c258db admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:07.268846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3832baf-06f4-4782-80d2-eab548c258db admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:07.272036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08838d-8f2f-48b0-b290-8e2a17f78761 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:07.275068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3d08838d-8f2f-48b0-b290-8e2a17f78761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:07.355726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e1131-b727-4750-9eb3-7b0a06fbaced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:07.356857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e1131-b727-4750-9eb3-7b0a06fbaced admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:07.357426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e1131-b727-4750-9eb3-7b0a06fbaced admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:07.358068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0e1131-b727-4750-9eb3-7b0a06fbaced admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:07.363566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e1131-b727-4750-9eb3-7b0a06fbaced admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:07.363566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0e1131-b727-4750-9eb3-7b0a06fbaced admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:07.416147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d08838d-8f2f-48b0-b290-8e2a17f78761 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:07.422830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533321b5-cfa9-4400-aefd-056fd9a42e0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:07.424627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533321b5-cfa9-4400-aefd-056fd9a42e0c service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:07.425823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533321b5-cfa9-4400-aefd-056fd9a42e0c service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:07.426393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533321b5-cfa9-4400-aefd-056fd9a42e0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:07.428602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-533321b5-cfa9-4400-aefd-056fd9a42e0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:07.444438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533321b5-cfa9-4400-aefd-056fd9a42e0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:07.446869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533321b5-cfa9-4400-aefd-056fd9a42e0c service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:07.470799 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1529/3060] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:07.536494 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1532/3061] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:09:07 2022] POST /identity/v3/users => generated 288 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:07.541324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533321b5-cfa9-4400-aefd-056fd9a42e0c service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:07.562098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e1131-b727-4750-9eb3-7b0a06fbaced admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:07.564541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533321b5-cfa9-4400-aefd-056fd9a42e0c service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:07.565456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-be0e1131-b727-4750-9eb3-7b0a06fbaced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:07.571128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd9805-0248-4099-91e7-cafb78a7f2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:07.572132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd9805-0248-4099-91e7-cafb78a7f2bc admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:07.572663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd9805-0248-4099-91e7-cafb78a7f2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:07.573994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dd9805-0248-4099-91e7-cafb78a7f2bc admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:07.575176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd9805-0248-4099-91e7-cafb78a7f2bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:07.583821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dd9805-0248-4099-91e7-cafb78a7f2bc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:07.599157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0e1131-b727-4750-9eb3-7b0a06fbaced admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:07.604247 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1530/3062] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:09:07.606193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd9805-0248-4099-91e7-cafb78a7f2bc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:07.612291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4cbb7-89fe-4e63-a512-99d5694c108a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:07.612291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4cbb7-89fe-4e63-a512-99d5694c108a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:07.612291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4cbb7-89fe-4e63-a512-99d5694c108a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:07.612291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f4cbb7-89fe-4e63-a512-99d5694c108a admin admin] PATH_INFO: `/v3/users/bf67d3b90f7642c2b23020958f14e53b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:07.612735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4cbb7-89fe-4e63-a512-99d5694c108a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf67d3b90f7642c2b23020958f14e53b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:07.613778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f4cbb7-89fe-4e63-a512-99d5694c108a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:07.618656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dd9805-0248-4099-91e7-cafb78a7f2bc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:07.624386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27142189-3c7e-488d-81f9-184e93ae7cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:07.625337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27142189-3c7e-488d-81f9-184e93ae7cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:07.625875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27142189-3c7e-488d-81f9-184e93ae7cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:07.626367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27142189-3c7e-488d-81f9-184e93ae7cc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:07.627306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27142189-3c7e-488d-81f9-184e93ae7cc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:07.629200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27142189-3c7e-488d-81f9-184e93ae7cc8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:07.647940 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1531/3063] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:09:07.649769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27142189-3c7e-488d-81f9-184e93ae7cc8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:07.663672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27142189-3c7e-488d-81f9-184e93ae7cc8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:07.668578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acd76b-1c35-4836-a1b9-3909d593aae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:07.669483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acd76b-1c35-4836-a1b9-3909d593aae1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:07.675458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acd76b-1c35-4836-a1b9-3909d593aae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:07.675458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acd76b-1c35-4836-a1b9-3909d593aae1 admin admin] PATH_INFO: `/v3/projects/5ad5df2ad7944f939c48d2a769b112b5/users/7f4bb7a4b1664e8fa53d9559bff35ba9/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:07.686055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acd76b-1c35-4836-a1b9-3909d593aae1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad5df2ad7944f939c48d2a769b112b5, user_id=7f4bb7a4b1664e8fa53d9559bff35ba9, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:07.689959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acd76b-1c35-4836-a1b9-3909d593aae1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:07.821195 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1532/3064] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:09:07 2022] PUT /identity/v3/projects/5ad5df2ad7944f939c48d2a769b112b5/users/7f4bb7a4b1664e8fa53d9559bff35ba9/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:07.823449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a7d01-50c4-48e1-8149-b684c5fa27f0 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:07.823741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a7d01-50c4-48e1-8149-b684c5fa27f0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:07.824024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a7d01-50c4-48e1-8149-b684c5fa27f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:07.878568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-617a7d01-50c4-48e1-8149-b684c5fa27f0 None None] MFA Rules not processed for user `7f4bb7a4b1664e8fa53d9559bff35ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:07.879856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-617a7d01-50c4-48e1-8149-b684c5fa27f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:07.883920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-94f4cbb7-89fe-4e63-a512-99d5694c108a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf67d3b90f7642c2b23020958f14e53b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:07.893907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-94f4cbb7-89fe-4e63-a512-99d5694c108a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf67d3b90f7642c2b23020958f14e53b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:07.894662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-94f4cbb7-89fe-4e63-a512-99d5694c108a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf67d3b90f7642c2b23020958f14e53b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:07.895416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-94f4cbb7-89fe-4e63-a512-99d5694c108a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf67d3b90f7642c2b23020958f14e53b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:07.981821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-94f4cbb7-89fe-4e63-a512-99d5694c108a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf67d3b90f7642c2b23020958f14e53b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:08.000010 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1533/3065] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:09:08.001057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a7d01-50c4-48e1-8149-b684c5fa27f0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:08.003185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-617a7d01-50c4-48e1-8149-b684c5fa27f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:08.019362 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1533/3066] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:09:07 2022] DELETE /identity/v3/users/bf67d3b90f7642c2b23020958f14e53b => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:08.181770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a7d01-50c4-48e1-8149-b684c5fa27f0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:08.326234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63971f91-dafa-4851-a539-62f5f461b05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:08.328528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63971f91-dafa-4851-a539-62f5f461b05a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:08.328528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63971f91-dafa-4851-a539-62f5f461b05a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:08.328528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63971f91-dafa-4851-a539-62f5f461b05a admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:08.329268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63971f91-dafa-4851-a539-62f5f461b05a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:08.331580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63971f91-dafa-4851-a539-62f5f461b05a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:08.505696 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1534/3067] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:09:08 2022] POST /identity/v3/projects => generated 375 bytes in 505 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:08.506592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4cbb7-89fe-4e63-a512-99d5694c108a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:08.520679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f4cbb7-89fe-4e63-a512-99d5694c108a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:08.636579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:08.662181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63971f91-dafa-4851-a539-62f5f461b05a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:08.666341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-63971f91-dafa-4851-a539-62f5f461b05a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:08.688760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2340d99-5f08-4339-a69e-56975ecd222b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:08.689174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2340d99-5f08-4339-a69e-56975ecd222b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:08.690284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2340d99-5f08-4339-a69e-56975ecd222b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:08.690284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2340d99-5f08-4339-a69e-56975ecd222b admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:08.691801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2340d99-5f08-4339-a69e-56975ecd222b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:08.694860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2340d99-5f08-4339-a69e-56975ecd222b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:08.698089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63971f91-dafa-4851-a539-62f5f461b05a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:08.709914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68795813-e174-4c88-b626-2cd39092023f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:08.719508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68795813-e174-4c88-b626-2cd39092023f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:08.721261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:08.724836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68795813-e174-4c88-b626-2cd39092023f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:08.725662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68795813-e174-4c88-b626-2cd39092023f admin admin] PATH_INFO: `/v3/projects/caacba1610f44c5ca8057b10a2dd483a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:08.730471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68795813-e174-4c88-b626-2cd39092023f admin admin] RBAC: Authorizing `identity:delete_project(project_id=caacba1610f44c5ca8057b10a2dd483a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:08.737094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68795813-e174-4c88-b626-2cd39092023f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:08.777932 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1534/3068] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:09:08 2022] POST /identity/v3/users => generated 294 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:09:08.782247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2340d99-5f08-4339-a69e-56975ecd222b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:08.795839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2340d99-5f08-4339-a69e-56975ecd222b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:08.806217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaee07a-d0de-4a99-b2c3-ad3708d8faee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:08.807341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaee07a-d0de-4a99-b2c3-ad3708d8faee admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:08.807853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaee07a-d0de-4a99-b2c3-ad3708d8faee admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:08.808431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaee07a-d0de-4a99-b2c3-ad3708d8faee admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:08.810722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaee07a-d0de-4a99-b2c3-ad3708d8faee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:08.817684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaee07a-d0de-4a99-b2c3-ad3708d8faee admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:08.839213 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1535/3069] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:08 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:09:08.840849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaee07a-d0de-4a99-b2c3-ad3708d8faee admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:08.885103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaee07a-d0de-4a99-b2c3-ad3708d8faee admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:08.885672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389775c-aa1a-4768-8b9c-4cbe89660434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:08.888993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389775c-aa1a-4768-8b9c-4cbe89660434 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:08.889290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389775c-aa1a-4768-8b9c-4cbe89660434 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:08.889647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389775c-aa1a-4768-8b9c-4cbe89660434 admin admin] PATH_INFO: `/v3/projects/01d12922f63d4220b67c3c93278e3905/users/5941a0b68ed7445080c9d945641a49cf/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:08.913358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5389775c-aa1a-4768-8b9c-4cbe89660434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01d12922f63d4220b67c3c93278e3905, user_id=5941a0b68ed7445080c9d945641a49cf, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:08.915714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5389775c-aa1a-4768-8b9c-4cbe89660434 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:09.057538 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1536/3070] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:09:08 2022] PUT /identity/v3/projects/01d12922f63d4220b67c3c93278e3905/users/5941a0b68ed7445080c9d945641a49cf/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:09.062125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389775c-aa1a-4768-8b9c-4cbe89660434 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:09.076384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-68795813-e174-4c88-b626-2cd39092023f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caacba1610f44c5ca8057b10a2dd483a', 'request_id': 'req-68795813-e174-4c88-b626-2cd39092023f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:09.086012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-68795813-e174-4c88-b626-2cd39092023f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caacba1610f44c5ca8057b10a2dd483a', 'request_id': 'req-68795813-e174-4c88-b626-2cd39092023f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:09.174864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389775c-aa1a-4768-8b9c-4cbe89660434 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:09.180452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb25d5-7be0-42a9-a1d5-026c8742a595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:09.180452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb25d5-7be0-42a9-a1d5-026c8742a595 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:09.180743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb25d5-7be0-42a9-a1d5-026c8742a595 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:09.180995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb25d5-7be0-42a9-a1d5-026c8742a595 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:09.182004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb25d5-7be0-42a9-a1d5-026c8742a595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:09.183799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb25d5-7be0-42a9-a1d5-026c8742a595 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:09.194804 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1537/3071] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:09:09.209250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb25d5-7be0-42a9-a1d5-026c8742a595 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:09.217106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb25d5-7be0-42a9-a1d5-026c8742a595 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:09.223243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6c0af-a21d-4e5e-8d4a-c30cd2d8934c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:09.224170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6c0af-a21d-4e5e-8d4a-c30cd2d8934c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:09.224534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6c0af-a21d-4e5e-8d4a-c30cd2d8934c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:09.224820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6c0af-a21d-4e5e-8d4a-c30cd2d8934c admin admin] PATH_INFO: `/v3/projects/01d12922f63d4220b67c3c93278e3905/users/5941a0b68ed7445080c9d945641a49cf/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:09.227393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6c0af-a21d-4e5e-8d4a-c30cd2d8934c admin admin] RBAC: Authorizing `identity:create_grant(project_id=01d12922f63d4220b67c3c93278e3905, user_id=5941a0b68ed7445080c9d945641a49cf, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:09.228672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-68795813-e174-4c88-b626-2cd39092023f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caacba1610f44c5ca8057b10a2dd483a', 'request_id': 'req-68795813-e174-4c88-b626-2cd39092023f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:09.230591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f6c0af-a21d-4e5e-8d4a-c30cd2d8934c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:09.261358 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1538/3072] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:09:09 2022] PUT /identity/v3/projects/01d12922f63d4220b67c3c93278e3905/users/5941a0b68ed7445080c9d945641a49cf/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:09.261358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ac28a-dc29-46a7-8cfb-f17b03993c64 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:09.261358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ac28a-dc29-46a7-8cfb-f17b03993c64 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:09.261358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ac28a-dc29-46a7-8cfb-f17b03993c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:09.262950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-68795813-e174-4c88-b626-2cd39092023f admin admin] The token cache is being invalidate because project caacba1610f44c5ca8057b10a2dd483a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:09:09.263329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-68795813-e174-4c88-b626-2cd39092023f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:09.264252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-68795813-e174-4c88-b626-2cd39092023f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:09.271415 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1535/3073] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:09:08 2022] DELETE /identity/v3/projects/caacba1610f44c5ca8057b10a2dd483a => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:09:09.272915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ac6be-9638-4a96-b17a-483237243f32 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:09.273217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ac6be-9638-4a96-b17a-483237243f32 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:09.273629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340ac6be-9638-4a96-b17a-483237243f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:09.297231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-340ac6be-9638-4a96-b17a-483237243f32 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:09.299397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-340ac6be-9638-4a96-b17a-483237243f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:09.317603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-dc0ac28a-dc29-46a7-8cfb-f17b03993c64 None None] MFA Rules not processed for user `5941a0b68ed7445080c9d945641a49cf`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:09.318648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dc0ac28a-dc29-46a7-8cfb-f17b03993c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:09.422528 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1536/3074] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:09.431538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f675ec-f50c-4468-a30e-48a98d72ba50 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:09.432102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f675ec-f50c-4468-a30e-48a98d72ba50 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:09.432416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f675ec-f50c-4468-a30e-48a98d72ba50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:09.481292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-26f675ec-f50c-4468-a30e-48a98d72ba50 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:09.482598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-26f675ec-f50c-4468-a30e-48a98d72ba50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:09.498745 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1539/3075] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:09.500372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ac28a-dc29-46a7-8cfb-f17b03993c64 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:09.503979 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1537/3076] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:09.514545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f675ec-f50c-4468-a30e-48a98d72ba50 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:09.516591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ac28a-dc29-46a7-8cfb-f17b03993c64 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:09.523022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b4d0a-fcc8-40d6-8f63-3b4955582d1c tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5941a0b68ed7445080c9d945641a49cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01d12922f63d4220b67c3c93278e3905', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:09.523768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b4d0a-fcc8-40d6-8f63-3b4955582d1c tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:09.524055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b4d0a-fcc8-40d6-8f63-3b4955582d1c tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:09.524337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b4d0a-fcc8-40d6-8f63-3b4955582d1c tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:09.524636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f675ec-f50c-4468-a30e-48a98d72ba50 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:09.525146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b4d0a-fcc8-40d6-8f63-3b4955582d1c tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:09:09.525469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b4d0a-fcc8-40d6-8f63-3b4955582d1c tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:09.529582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b4d0a-fcc8-40d6-8f63-3b4955582d1c tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:09.535046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81daa6b-4bb1-426d-8017-f5b711b4ad51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:09.537276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81daa6b-4bb1-426d-8017-f5b711b4ad51 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:09.537732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81daa6b-4bb1-426d-8017-f5b711b4ad51 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:09.538194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81daa6b-4bb1-426d-8017-f5b711b4ad51 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:09.539198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81daa6b-4bb1-426d-8017-f5b711b4ad51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:09:09.539729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81daa6b-4bb1-426d-8017-f5b711b4ad51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:09.542793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81daa6b-4bb1-426d-8017-f5b711b4ad51 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:09.554508 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1540/3077] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:09:09.557971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b4d0a-fcc8-40d6-8f63-3b4955582d1c tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:09.571384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b4d0a-fcc8-40d6-8f63-3b4955582d1c tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:09.579944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0640cd-57b3-4767-8b06-6a4bb4d41c86 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5941a0b68ed7445080c9d945641a49cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01d12922f63d4220b67c3c93278e3905', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:09.586290 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1538/3078] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:09:09.586666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0640cd-57b3-4767-8b06-6a4bb4d41c86 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:09.586768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81daa6b-4bb1-426d-8017-f5b711b4ad51 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:09.586875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0640cd-57b3-4767-8b06-6a4bb4d41c86 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:09.586875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0640cd-57b3-4767-8b06-6a4bb4d41c86 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:09.586875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0640cd-57b3-4767-8b06-6a4bb4d41c86 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:09.599060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0640cd-57b3-4767-8b06-6a4bb4d41c86 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:09.605822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81daa6b-4bb1-426d-8017-f5b711b4ad51 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:09.613216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87141c88-075b-491a-95dd-b67f4185ed4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:09.614115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87141c88-075b-491a-95dd-b67f4185ed4f admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:09.614673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87141c88-075b-491a-95dd-b67f4185ed4f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:09.615056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87141c88-075b-491a-95dd-b67f4185ed4f admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:09.616137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87141c88-075b-491a-95dd-b67f4185ed4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:09.618263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87141c88-075b-491a-95dd-b67f4185ed4f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:09.704991 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1541/3079] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:09:09 2022] POST /identity/v3/projects => generated 371 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:09.707226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0bda1-087a-478f-ad24-f99ffa128c6b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:09.707628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0bda1-087a-478f-ad24-f99ffa128c6b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:09.708128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba0bda1-087a-478f-ad24-f99ffa128c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:09.739852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5ba0bda1-087a-478f-ad24-f99ffa128c6b None None] MFA Rules not processed for user `f08e71479f9d4ef0a30681cdfdd6fef8`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:09.741405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5ba0bda1-087a-478f-ad24-f99ffa128c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:09.761165 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1539/3080] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:09:09 2022] POST /identity/v3/projects => generated 375 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:09.764117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87141c88-075b-491a-95dd-b67f4185ed4f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:09.860075 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1542/3081] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:09.860075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0bda1-087a-478f-ad24-f99ffa128c6b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:09.922988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87141c88-075b-491a-95dd-b67f4185ed4f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:09.929894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1bd37c-50a5-46c1-95fa-1b1ebbb5c192 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5941a0b68ed7445080c9d945641a49cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01d12922f63d4220b67c3c93278e3905', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:09.931017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bd37c-50a5-46c1-95fa-1b1ebbb5c192 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:09.931319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bd37c-50a5-46c1-95fa-1b1ebbb5c192 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:09.934275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1bd37c-50a5-46c1-95fa-1b1ebbb5c192 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:09.934275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1bd37c-50a5-46c1-95fa-1b1ebbb5c192 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:09:09.934275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1bd37c-50a5-46c1-95fa-1b1ebbb5c192 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:09.934638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1bd37c-50a5-46c1-95fa-1b1ebbb5c192 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:09.952021 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1540/3082] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:09:10.072612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba0bda1-087a-478f-ad24-f99ffa128c6b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:10.186235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34b9b7-b451-43ad-9f72-ff00764be362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:10.187012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34b9b7-b451-43ad-9f72-ff00764be362 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:10.187976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34b9b7-b451-43ad-9f72-ff00764be362 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:10.188269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34b9b7-b451-43ad-9f72-ff00764be362 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:10.192026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34b9b7-b451-43ad-9f72-ff00764be362 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:10.192026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34b9b7-b451-43ad-9f72-ff00764be362 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:10.282780 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1543/3083] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:09:09 2022] POST /identity/v3/users => generated 288 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:10.287236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1bd37c-50a5-46c1-95fa-1b1ebbb5c192 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:10.305469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1bd37c-50a5-46c1-95fa-1b1ebbb5c192 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:10.313753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecf4a2-027a-40b6-ab74-7b5bac5d1769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:10.314529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecf4a2-027a-40b6-ab74-7b5bac5d1769 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:10.315045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecf4a2-027a-40b6-ab74-7b5bac5d1769 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:10.315485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecf4a2-027a-40b6-ab74-7b5bac5d1769 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:10.316406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecf4a2-027a-40b6-ab74-7b5bac5d1769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:10.321186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecf4a2-027a-40b6-ab74-7b5bac5d1769 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:10.340603 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1541/3084] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:09:10.343393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34b9b7-b451-43ad-9f72-ff00764be362 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:10.353020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ef34b9b7-b451-43ad-9f72-ff00764be362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:10.354158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecf4a2-027a-40b6-ab74-7b5bac5d1769 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:10.376566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecf4a2-027a-40b6-ab74-7b5bac5d1769 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:10.383707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cfbf4-b68b-49b9-b5b1-1f770eeddec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:10.385999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cfbf4-b68b-49b9-b5b1-1f770eeddec3 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:10.386940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cfbf4-b68b-49b9-b5b1-1f770eeddec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:10.387483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72cfbf4-b68b-49b9-b5b1-1f770eeddec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:10.389085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cfbf4-b68b-49b9-b5b1-1f770eeddec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:10.402210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72cfbf4-b68b-49b9-b5b1-1f770eeddec3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:10.427834 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1542/3085] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:10 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:09:10.432531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cfbf4-b68b-49b9-b5b1-1f770eeddec3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:10.449232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72cfbf4-b68b-49b9-b5b1-1f770eeddec3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:10.455549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19342879-d97d-439f-9b7c-ef146cdaadaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:10.457520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19342879-d97d-439f-9b7c-ef146cdaadaa admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:10.458044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19342879-d97d-439f-9b7c-ef146cdaadaa admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:10.458835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19342879-d97d-439f-9b7c-ef146cdaadaa admin admin] PATH_INFO: `/v3/projects/cf209a1ba94046f89ddf62c77b6b0f02/users/659bd3ebefc348b7a176379bd342f9ed/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:10.487837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19342879-d97d-439f-9b7c-ef146cdaadaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf209a1ba94046f89ddf62c77b6b0f02, user_id=659bd3ebefc348b7a176379bd342f9ed, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:10.492477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19342879-d97d-439f-9b7c-ef146cdaadaa admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:10.506211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34b9b7-b451-43ad-9f72-ff00764be362 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:10.522903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd4bdf-f4ef-45a0-afdf-445689d8b3a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:10.522903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd4bdf-f4ef-45a0-afdf-445689d8b3a0 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:10.523504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd4bdf-f4ef-45a0-afdf-445689d8b3a0 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:10.523856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd4bdf-f4ef-45a0-afdf-445689d8b3a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:10.545921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd4bdf-f4ef-45a0-afdf-445689d8b3a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:10.549331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd4bdf-f4ef-45a0-afdf-445689d8b3a0 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:10.576564 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1544/3086] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:10 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:10.577963 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1543/3087] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:09:10 2022] PUT /identity/v3/projects/cf209a1ba94046f89ddf62c77b6b0f02/users/659bd3ebefc348b7a176379bd342f9ed/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:10.581877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd4bdf-f4ef-45a0-afdf-445689d8b3a0 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:10.582466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6f525-eb78-4506-b733-02943dd22f7c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:10.582978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6f525-eb78-4506-b733-02943dd22f7c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:10.583355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a6f525-eb78-4506-b733-02943dd22f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:10.584016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-92fd4bdf-f4ef-45a0-afdf-445689d8b3a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:10.618938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-53a6f525-eb78-4506-b733-02943dd22f7c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:10.620491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-53a6f525-eb78-4506-b733-02943dd22f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:10.659390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd4bdf-f4ef-45a0-afdf-445689d8b3a0 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:10.758722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ae501-6981-431c-9301-93c3787d6633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:10.759847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ae501-6981-431c-9301-93c3787d6633 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:10.760354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ae501-6981-431c-9301-93c3787d6633 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:10.760853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ae501-6981-431c-9301-93c3787d6633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:10.761072 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1544/3088] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:09:10.763484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39188b27-210d-4ed9-9439-cf4993f1d689 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:10.767304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39188b27-210d-4ed9-9439-cf4993f1d689 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:10.767304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39188b27-210d-4ed9-9439-cf4993f1d689 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:10.767512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-052ae501-6981-431c-9301-93c3787d6633 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:10.767602 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1545/3089] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:09:10.767602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39188b27-210d-4ed9-9439-cf4993f1d689 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:10.784520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ae501-6981-431c-9301-93c3787d6633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:10.786835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ae501-6981-431c-9301-93c3787d6633 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:10.791607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39188b27-210d-4ed9-9439-cf4993f1d689 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:10.801290 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1545/3090] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:10.802157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ae501-6981-431c-9301-93c3787d6633 service neutron] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:09:10.803108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5500189f-7526-4bb5-884a-b24425257b09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:10.803765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500189f-7526-4bb5-884a-b24425257b09 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:10.805086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500189f-7526-4bb5-884a-b24425257b09 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:10.805086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5500189f-7526-4bb5-884a-b24425257b09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:10.823569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500189f-7526-4bb5-884a-b24425257b09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:10.828829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5500189f-7526-4bb5-884a-b24425257b09 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:10.846637 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1546/3091] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:09:10.907315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ae501-6981-431c-9301-93c3787d6633 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:10.934888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:10.988125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ae501-6981-431c-9301-93c3787d6633 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:10.994755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbc8fd-b68c-4182-a7b9-0373936fa804 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5941a0b68ed7445080c9d945641a49cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01d12922f63d4220b67c3c93278e3905', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:10.995654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbc8fd-b68c-4182-a7b9-0373936fa804 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:10.995944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbc8fd-b68c-4182-a7b9-0373936fa804 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:10.996229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cbc8fd-b68c-4182-a7b9-0373936fa804 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] PATH_INFO: `/v3/projects/472c72ac90f6431990955cade16e8587` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:10.997529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbc8fd-b68c-4182-a7b9-0373936fa804 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorizing `identity:get_project(project_id=472c72ac90f6431990955cade16e8587)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:10.999717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cbc8fd-b68c-4182-a7b9-0373936fa804 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:11.002350 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1546/3092] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:09:10 2022] GET /identity/v3/projects/472c72ac90f6431990955cade16e8587 => generated 371 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:09:11.109529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:11.817530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a680d2-610b-4f0b-8d68-a7cde7834505 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:11.818112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a680d2-610b-4f0b-8d68-a7cde7834505 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:11.818233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a680d2-610b-4f0b-8d68-a7cde7834505 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:11.820315 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1547/3093] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:09:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:09:11.825879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbc8fd-b68c-4182-a7b9-0373936fa804 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:09:11.830575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-88cbc8fd-b68c-4182-a7b9-0373936fa804 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:11.849292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbc8fd-b68c-4182-a7b9-0373936fa804 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:11.860351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cbc8fd-b68c-4182-a7b9-0373936fa804 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:11.867332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc01804-1b6e-44b2-889f-6eb955d90fea tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5941a0b68ed7445080c9d945641a49cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01d12922f63d4220b67c3c93278e3905', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:11.868581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc01804-1b6e-44b2-889f-6eb955d90fea tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:11.870471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc01804-1b6e-44b2-889f-6eb955d90fea tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:11.871336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc01804-1b6e-44b2-889f-6eb955d90fea tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] PATH_INFO: `/v3/projects/472c72ac90f6431990955cade16e8587` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:11.873772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc01804-1b6e-44b2-889f-6eb955d90fea tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorizing `identity:get_project(project_id=472c72ac90f6431990955cade16e8587)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:11.881573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc01804-1b6e-44b2-889f-6eb955d90fea tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:11.896871 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1547/3094] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:09:11 2022] GET /identity/v3/projects/472c72ac90f6431990955cade16e8587 => generated 371 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:09:11.948286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a680d2-610b-4f0b-8d68-a7cde7834505 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:11.972455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a680d2-610b-4f0b-8d68-a7cde7834505 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:11.989232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5cba4-c727-4940-8ca7-a32d484b0289 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5941a0b68ed7445080c9d945641a49cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01d12922f63d4220b67c3c93278e3905', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:11.989961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5cba4-c727-4940-8ca7-a32d484b0289 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:11.991005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5cba4-c727-4940-8ca7-a32d484b0289 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:11.991005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d5cba4-c727-4940-8ca7-a32d484b0289 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:11.991521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5cba4-c727-4940-8ca7-a32d484b0289 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:09:11.991889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5cba4-c727-4940-8ca7-a32d484b0289 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:11.993505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d5cba4-c727-4940-8ca7-a32d484b0289 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:12.022429 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1548/3095] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:09:12.027643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc01804-1b6e-44b2-889f-6eb955d90fea tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:12.039840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc01804-1b6e-44b2-889f-6eb955d90fea tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:12.054675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85f95cb-fec3-4d4d-ad86-2c51b3dbe80c tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5941a0b68ed7445080c9d945641a49cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01d12922f63d4220b67c3c93278e3905', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:12.055571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f95cb-fec3-4d4d-ad86-2c51b3dbe80c tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:12.056027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f95cb-fec3-4d4d-ad86-2c51b3dbe80c tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:12.056301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85f95cb-fec3-4d4d-ad86-2c51b3dbe80c tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:12.064391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f95cb-fec3-4d4d-ad86-2c51b3dbe80c tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:12.072785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85f95cb-fec3-4d4d-ad86-2c51b3dbe80c tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:12.145910 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1548/3096] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:09:12 2022] POST /identity/v3/users => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:12.149443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5cba4-c727-4940-8ca7-a32d484b0289 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:12.188340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d5cba4-c727-4940-8ca7-a32d484b0289 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:12.206974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9aa568-d0e7-4e87-b7ce-92dad6ab5f60 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5941a0b68ed7445080c9d945641a49cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01d12922f63d4220b67c3c93278e3905', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:12.207895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9aa568-d0e7-4e87-b7ce-92dad6ab5f60 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:12.210168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9aa568-d0e7-4e87-b7ce-92dad6ab5f60 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:12.210623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9aa568-d0e7-4e87-b7ce-92dad6ab5f60 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:12.211802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9aa568-d0e7-4e87-b7ce-92dad6ab5f60 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:09:12.212412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9aa568-d0e7-4e87-b7ce-92dad6ab5f60 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:12.217407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9aa568-d0e7-4e87-b7ce-92dad6ab5f60 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:12.250009 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1549/3097] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:09:12.565574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:12.842004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33911a5-d2ca-410e-9693-a0d876e0044e None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:12.842631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33911a5-d2ca-410e-9693-a0d876e0044e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:12.842631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33911a5-d2ca-410e-9693-a0d876e0044e None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:12.845784 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1549/3098] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:09:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:09:12.852431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9aa568-d0e7-4e87-b7ce-92dad6ab5f60 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:09:12.873778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9aa568-d0e7-4e87-b7ce-92dad6ab5f60 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:12.898612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9aa568-d0e7-4e87-b7ce-92dad6ab5f60 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:12.910114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb38ae-98ad-41e9-b525-93597ed4da9b tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5941a0b68ed7445080c9d945641a49cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01d12922f63d4220b67c3c93278e3905', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:12.911550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb38ae-98ad-41e9-b525-93597ed4da9b tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:12.912061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb38ae-98ad-41e9-b525-93597ed4da9b tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:12.912592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fb38ae-98ad-41e9-b525-93597ed4da9b tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] PATH_INFO: `/v3/projects/472c72ac90f6431990955cade16e8587` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:12.918452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb38ae-98ad-41e9-b525-93597ed4da9b tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorizing `identity:get_project(project_id=472c72ac90f6431990955cade16e8587)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:12.920962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fb38ae-98ad-41e9-b525-93597ed4da9b tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:12.928439 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1550/3099] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:09:12 2022] GET /identity/v3/projects/472c72ac90f6431990955cade16e8587 => generated 371 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:09:13.412958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fc0e1-7004-4e1a-9fd3-6af1752c1cda None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:13.413203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fc0e1-7004-4e1a-9fd3-6af1752c1cda None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:13.413443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fc0e1-7004-4e1a-9fd3-6af1752c1cda None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:13.414950 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1550/3100] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:09:13.432905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb38ae-98ad-41e9-b525-93597ed4da9b tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:09:13.458758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb38ae-98ad-41e9-b525-93597ed4da9b tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:13.473526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fb38ae-98ad-41e9-b525-93597ed4da9b tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:13.478857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea939aa-f195-4c2a-8a46-f40e83e75002 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5941a0b68ed7445080c9d945641a49cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01d12922f63d4220b67c3c93278e3905', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:13.480169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea939aa-f195-4c2a-8a46-f40e83e75002 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:13.480660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea939aa-f195-4c2a-8a46-f40e83e75002 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:13.481127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea939aa-f195-4c2a-8a46-f40e83e75002 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] PATH_INFO: `/v3/projects/472c72ac90f6431990955cade16e8587` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:13.484941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea939aa-f195-4c2a-8a46-f40e83e75002 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorizing `identity:get_project(project_id=472c72ac90f6431990955cade16e8587)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:13.487342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea939aa-f195-4c2a-8a46-f40e83e75002 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:13.490220 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1551/3101] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:09:13 2022] GET /identity/v3/projects/472c72ac90f6431990955cade16e8587 => generated 371 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:09:13.538975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fc0e1-7004-4e1a-9fd3-6af1752c1cda None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:13.559856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fc0e1-7004-4e1a-9fd3-6af1752c1cda None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:13.566904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29f91e-b656-4467-af56-74ae49ae5ddf tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5941a0b68ed7445080c9d945641a49cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01d12922f63d4220b67c3c93278e3905', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:13.589817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29f91e-b656-4467-af56-74ae49ae5ddf tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:13.590143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29f91e-b656-4467-af56-74ae49ae5ddf tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:13.592721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29f91e-b656-4467-af56-74ae49ae5ddf tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] PATH_INFO: `/v3/users/afd688afe7024b38ac279ed221c22de3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:13.646416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29f91e-b656-4467-af56-74ae49ae5ddf tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorizing `identity:delete_user(user_id=afd688afe7024b38ac279ed221c22de3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:13.649827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b29f91e-b656-4467-af56-74ae49ae5ddf tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:13.857448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9b29f91e-b656-4467-af56-74ae49ae5ddf tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afd688afe7024b38ac279ed221c22de3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:13.867573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:13.870151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9b29f91e-b656-4467-af56-74ae49ae5ddf tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afd688afe7024b38ac279ed221c22de3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:13.871143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9b29f91e-b656-4467-af56-74ae49ae5ddf tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afd688afe7024b38ac279ed221c22de3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:13.871964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9b29f91e-b656-4467-af56-74ae49ae5ddf tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afd688afe7024b38ac279ed221c22de3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:13.949583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9b29f91e-b656-4467-af56-74ae49ae5ddf tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afd688afe7024b38ac279ed221c22de3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:13.976736 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1551/3102] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:09:13 2022] DELETE /identity/v3/users/afd688afe7024b38ac279ed221c22de3 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:13.980808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea939aa-f195-4c2a-8a46-f40e83e75002 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:13.992231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6ea939aa-f195-4c2a-8a46-f40e83e75002 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:14.016801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea939aa-f195-4c2a-8a46-f40e83e75002 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:14.161660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c6e797-95bf-4164-8238-9bea8b58d4a1 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5941a0b68ed7445080c9d945641a49cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '01d12922f63d4220b67c3c93278e3905', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:14.162678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6e797-95bf-4164-8238-9bea8b58d4a1 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:14.162973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6e797-95bf-4164-8238-9bea8b58d4a1 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:14.163262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c6e797-95bf-4164-8238-9bea8b58d4a1 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] PATH_INFO: `/v3/projects/472c72ac90f6431990955cade16e8587` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:14.169320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6e797-95bf-4164-8238-9bea8b58d4a1 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorizing `identity:delete_project(project_id=472c72ac90f6431990955cade16e8587)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:14.174275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c6e797-95bf-4164-8238-9bea8b58d4a1 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:14.369077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f5c6e797-95bf-4164-8238-9bea8b58d4a1 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '472c72ac90f6431990955cade16e8587', 'request_id': 'req-f5c6e797-95bf-4164-8238-9bea8b58d4a1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:14.382889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f5c6e797-95bf-4164-8238-9bea8b58d4a1 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '472c72ac90f6431990955cade16e8587', 'request_id': 'req-f5c6e797-95bf-4164-8238-9bea8b58d4a1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:14.393792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:14.479078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f5c6e797-95bf-4164-8238-9bea8b58d4a1 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '472c72ac90f6431990955cade16e8587', 'request_id': 'req-f5c6e797-95bf-4164-8238-9bea8b58d4a1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:14.508063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f5c6e797-95bf-4164-8238-9bea8b58d4a1 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] The token cache is being invalidate because project 472c72ac90f6431990955cade16e8587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:09:14.510687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f5c6e797-95bf-4164-8238-9bea8b58d4a1 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:14.512013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f5c6e797-95bf-4164-8238-9bea8b58d4a1 tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:14.524801 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1552/3103] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:09:13 2022] DELETE /identity/v3/projects/472c72ac90f6431990955cade16e8587 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:09:14.539208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29f91e-b656-4467-af56-74ae49ae5ddf tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:14.544467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9b29f91e-b656-4467-af56-74ae49ae5ddf tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:14.797507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29f91e-b656-4467-af56-74ae49ae5ddf tempest-QuotasAdminTestV236-1953530557 tempest-QuotasAdminTestV236-1953530557-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:14.999217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6fa8a-ead6-4c87-a84f-a1766dd7789f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:15.001552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6fa8a-ead6-4c87-a84f-a1766dd7789f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:15.002027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6fa8a-ead6-4c87-a84f-a1766dd7789f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:15.002462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d6fa8a-ead6-4c87-a84f-a1766dd7789f admin admin] PATH_INFO: `/v3/users/7f4bb7a4b1664e8fa53d9559bff35ba9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:15.005297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d6fa8a-ead6-4c87-a84f-a1766dd7789f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f4bb7a4b1664e8fa53d9559bff35ba9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:15.015399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d6fa8a-ead6-4c87-a84f-a1766dd7789f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:15.387281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:15.390530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-12d6fa8a-ead6-4c87-a84f-a1766dd7789f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f4bb7a4b1664e8fa53d9559bff35ba9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:15.402143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-12d6fa8a-ead6-4c87-a84f-a1766dd7789f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f4bb7a4b1664e8fa53d9559bff35ba9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:15.403106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-12d6fa8a-ead6-4c87-a84f-a1766dd7789f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f4bb7a4b1664e8fa53d9559bff35ba9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:15.403934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-12d6fa8a-ead6-4c87-a84f-a1766dd7789f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f4bb7a4b1664e8fa53d9559bff35ba9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:15.514198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-12d6fa8a-ead6-4c87-a84f-a1766dd7789f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f4bb7a4b1664e8fa53d9559bff35ba9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:15.574733 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1552/3104] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:09:14 2022] DELETE /identity/v3/users/7f4bb7a4b1664e8fa53d9559bff35ba9 => generated 0 bytes in 1036 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:15.581354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdca27-9dbe-4294-b406-e91d246c0b9a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:15.582282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdca27-9dbe-4294-b406-e91d246c0b9a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:15.582798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdca27-9dbe-4294-b406-e91d246c0b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:15.612277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8abdca27-9dbe-4294-b406-e91d246c0b9a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:15.614072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8abdca27-9dbe-4294-b406-e91d246c0b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:15.695974 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1553/3105] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:15.759409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6fa8a-ead6-4c87-a84f-a1766dd7789f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:15.761407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-12d6fa8a-ead6-4c87-a84f-a1766dd7789f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:15.938843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d6fa8a-ead6-4c87-a84f-a1766dd7789f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:15.943294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824d161-4b41-44c9-93ba-0378947e48ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:15.944671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824d161-4b41-44c9-93ba-0378947e48ee service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:15.945686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824d161-4b41-44c9-93ba-0378947e48ee service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:15.946132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824d161-4b41-44c9-93ba-0378947e48ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:15.976309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824d161-4b41-44c9-93ba-0378947e48ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:15.978005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824d161-4b41-44c9-93ba-0378947e48ee service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:16.007397 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1553/3106] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:16.379943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e35471-b2ee-4c1d-94c9-4646929300ca None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:16.379943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e35471-b2ee-4c1d-94c9-4646929300ca None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:16.380168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e35471-b2ee-4c1d-94c9-4646929300ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:16.424635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-60e35471-b2ee-4c1d-94c9-4646929300ca None None] MFA Rules not processed for user `5c65140c062f4eedb8bfe963137de014`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:16.426868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-60e35471-b2ee-4c1d-94c9-4646929300ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:16.540956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:16.568193 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1554/3107] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:16.575352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824d161-4b41-44c9-93ba-0378947e48ee service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:16.577454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5824d161-4b41-44c9-93ba-0378947e48ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:16.721795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824d161-4b41-44c9-93ba-0378947e48ee service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:16.729822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b6869-12d3-460d-a367-e3e3401eb49d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:16.730909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b6869-12d3-460d-a367-e3e3401eb49d service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:16.731418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b6869-12d3-460d-a367-e3e3401eb49d service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:16.731915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b6869-12d3-460d-a367-e3e3401eb49d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:16.744396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b6869-12d3-460d-a367-e3e3401eb49d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:16.746910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b6869-12d3-460d-a367-e3e3401eb49d service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:16.759557 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1554/3108] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:17.679315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e35471-b2ee-4c1d-94c9-4646929300ca None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:17.681092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-60e35471-b2ee-4c1d-94c9-4646929300ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:17.731010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e35471-b2ee-4c1d-94c9-4646929300ca None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:17.755523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd044a63-3d7d-49fc-b33b-9249e6186d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:17.756702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd044a63-3d7d-49fc-b33b-9249e6186d7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:17.759810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd044a63-3d7d-49fc-b33b-9249e6186d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:17.759810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd044a63-3d7d-49fc-b33b-9249e6186d7d admin admin] PATH_INFO: `/v3/projects/5ad5df2ad7944f939c48d2a769b112b5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:17.762171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd044a63-3d7d-49fc-b33b-9249e6186d7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ad5df2ad7944f939c48d2a769b112b5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:17.766360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd044a63-3d7d-49fc-b33b-9249e6186d7d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:17.860326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:17.897119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:17.928533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fd044a63-3d7d-49fc-b33b-9249e6186d7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ad5df2ad7944f939c48d2a769b112b5', 'request_id': 'req-fd044a63-3d7d-49fc-b33b-9249e6186d7d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:17.940144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fd044a63-3d7d-49fc-b33b-9249e6186d7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ad5df2ad7944f939c48d2a769b112b5', 'request_id': 'req-fd044a63-3d7d-49fc-b33b-9249e6186d7d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:18.009952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fd044a63-3d7d-49fc-b33b-9249e6186d7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ad5df2ad7944f939c48d2a769b112b5', 'request_id': 'req-fd044a63-3d7d-49fc-b33b-9249e6186d7d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:18.035564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fd044a63-3d7d-49fc-b33b-9249e6186d7d admin admin] The token cache is being invalidate because project 5ad5df2ad7944f939c48d2a769b112b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:09:18.035564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fd044a63-3d7d-49fc-b33b-9249e6186d7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:18.037249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fd044a63-3d7d-49fc-b33b-9249e6186d7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:18.038970 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1555/3109] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:09:17 2022] DELETE /identity/v3/projects/5ad5df2ad7944f939c48d2a769b112b5 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:18.043092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b6869-12d3-460d-a367-e3e3401eb49d service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:18.064365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b6869-12d3-460d-a367-e3e3401eb49d service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:18.269446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799066c-162d-49f2-9146-9bd106d3f9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:18.273201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799066c-162d-49f2-9146-9bd106d3f9f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:18.273706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799066c-162d-49f2-9146-9bd106d3f9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:18.274213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799066c-162d-49f2-9146-9bd106d3f9f9 admin admin] PATH_INFO: `/v3/users/5941a0b68ed7445080c9d945641a49cf` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:18.280820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7799066c-162d-49f2-9146-9bd106d3f9f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5941a0b68ed7445080c9d945641a49cf)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:18.286680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7799066c-162d-49f2-9146-9bd106d3f9f9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:18.614171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7799066c-162d-49f2-9146-9bd106d3f9f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5941a0b68ed7445080c9d945641a49cf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:18.632202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7799066c-162d-49f2-9146-9bd106d3f9f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5941a0b68ed7445080c9d945641a49cf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:18.633333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7799066c-162d-49f2-9146-9bd106d3f9f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5941a0b68ed7445080c9d945641a49cf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:18.636076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7799066c-162d-49f2-9146-9bd106d3f9f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5941a0b68ed7445080c9d945641a49cf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:18.747477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7799066c-162d-49f2-9146-9bd106d3f9f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5941a0b68ed7445080c9d945641a49cf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:18.790936 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1555/3110] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:09:18 2022] DELETE /identity/v3/users/5941a0b68ed7445080c9d945641a49cf => generated 0 bytes in 745 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:20.382724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd044a63-3d7d-49fc-b33b-9249e6186d7d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:20.386235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fd044a63-3d7d-49fc-b33b-9249e6186d7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:20.433777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd044a63-3d7d-49fc-b33b-9249e6186d7d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:20.596421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329e930-869a-47cb-8914-285a4051a7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:20.597544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329e930-869a-47cb-8914-285a4051a7ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:20.597717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329e930-869a-47cb-8914-285a4051a7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:20.598230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329e930-869a-47cb-8914-285a4051a7ac admin admin] PATH_INFO: `/v3/projects/01d12922f63d4220b67c3c93278e3905` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:20.600918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329e930-869a-47cb-8914-285a4051a7ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=01d12922f63d4220b67c3c93278e3905)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:20.605303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329e930-869a-47cb-8914-285a4051a7ac admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:20.728467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d329e930-869a-47cb-8914-285a4051a7ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01d12922f63d4220b67c3c93278e3905', 'request_id': 'req-d329e930-869a-47cb-8914-285a4051a7ac'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:20.736685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d329e930-869a-47cb-8914-285a4051a7ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01d12922f63d4220b67c3c93278e3905', 'request_id': 'req-d329e930-869a-47cb-8914-285a4051a7ac'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:20.782644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d329e930-869a-47cb-8914-285a4051a7ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01d12922f63d4220b67c3c93278e3905', 'request_id': 'req-d329e930-869a-47cb-8914-285a4051a7ac'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:20.801876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d329e930-869a-47cb-8914-285a4051a7ac admin admin] The token cache is being invalidate because project 01d12922f63d4220b67c3c93278e3905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:09:20.802573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d329e930-869a-47cb-8914-285a4051a7ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:20.803497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d329e930-869a-47cb-8914-285a4051a7ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:20.812418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5296b-a05d-442e-bef8-3c9acdb3c526 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:20.813334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5296b-a05d-442e-bef8-3c9acdb3c526 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:20.813871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5296b-a05d-442e-bef8-3c9acdb3c526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:20.820896 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1556/3111] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:09:20 2022] DELETE /identity/v3/projects/01d12922f63d4220b67c3c93278e3905 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:20.821154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:20.836731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f3a5296b-a05d-442e-bef8-3c9acdb3c526 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:20.838384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f3a5296b-a05d-442e-bef8-3c9acdb3c526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:20.963464 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1556/3112] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:20.974259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c5b73-9d57-4458-9b43-67ed59388db9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:20.974559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c5b73-9d57-4458-9b43-67ed59388db9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:20.974842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c5b73-9d57-4458-9b43-67ed59388db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:21.007390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e09c5b73-9d57-4458-9b43-67ed59388db9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:21.009571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e09c5b73-9d57-4458-9b43-67ed59388db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:21.029973 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1557/3113] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:21.034581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5296b-a05d-442e-bef8-3c9acdb3c526 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:21.050594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a5296b-a05d-442e-bef8-3c9acdb3c526 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:21.061822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc186c-54ac-4d30-a9b7-c5aeced9213e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:21.062597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc186c-54ac-4d30-a9b7-c5aeced9213e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:21.063698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc186c-54ac-4d30-a9b7-c5aeced9213e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:21.064934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc186c-54ac-4d30-a9b7-c5aeced9213e admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:21.069024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc186c-54ac-4d30-a9b7-c5aeced9213e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:09:21.069024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc186c-54ac-4d30-a9b7-c5aeced9213e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:21.074382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc186c-54ac-4d30-a9b7-c5aeced9213e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:21.126449 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1557/3114] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:09:21.138582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c5b73-9d57-4458-9b43-67ed59388db9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:21.172175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c5b73-9d57-4458-9b43-67ed59388db9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:21.182881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae7511-6b72-42c3-b94a-039225c148e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:21.183683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae7511-6b72-42c3-b94a-039225c148e5 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:21.184045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae7511-6b72-42c3-b94a-039225c148e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:21.184390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ae7511-6b72-42c3-b94a-039225c148e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:21.185188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae7511-6b72-42c3-b94a-039225c148e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:21.188396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ae7511-6b72-42c3-b94a-039225c148e5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:21.191791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc186c-54ac-4d30-a9b7-c5aeced9213e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:21.195531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a3fc186c-54ac-4d30-a9b7-c5aeced9213e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:21.312578 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1558/3115] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:09:21 2022] POST /identity/v3/projects => generated 375 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:21.316227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae7511-6b72-42c3-b94a-039225c148e5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:21.333134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc186c-54ac-4d30-a9b7-c5aeced9213e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:21.341982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:21.465459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ae7511-6b72-42c3-b94a-039225c148e5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:21.477754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa7d70-5021-44fe-8e0c-2ef9cf9c493c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:21.478275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6334f-4130-4b4e-947f-035ee792a1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:21.487236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa7d70-5021-44fe-8e0c-2ef9cf9c493c service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:21.487236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa7d70-5021-44fe-8e0c-2ef9cf9c493c service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:21.487236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa7d70-5021-44fe-8e0c-2ef9cf9c493c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:21.487773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6334f-4130-4b4e-947f-035ee792a1a3 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:21.487773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6334f-4130-4b4e-947f-035ee792a1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:21.487773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6334f-4130-4b4e-947f-035ee792a1a3 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:21.487773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6334f-4130-4b4e-947f-035ee792a1a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:21.487773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6334f-4130-4b4e-947f-035ee792a1a3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:21.488693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e9aa7d70-5021-44fe-8e0c-2ef9cf9c493c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:21.625273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa7d70-5021-44fe-8e0c-2ef9cf9c493c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:21.627330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa7d70-5021-44fe-8e0c-2ef9cf9c493c service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:21.633617 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1559/3116] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:09:21 2022] POST /identity/v3/users => generated 288 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:09:21.637929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6334f-4130-4b4e-947f-035ee792a1a3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:21.656988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6334f-4130-4b4e-947f-035ee792a1a3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:21.664109 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1558/3117] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:21.669551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a5bc3-958e-4985-b2a0-1ab14446880e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:21.670553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a5bc3-958e-4985-b2a0-1ab14446880e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:21.670839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a5bc3-958e-4985-b2a0-1ab14446880e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:21.671124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a5bc3-958e-4985-b2a0-1ab14446880e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:21.675556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a5bc3-958e-4985-b2a0-1ab14446880e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:21.679330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a5bc3-958e-4985-b2a0-1ab14446880e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:21.706473 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1560/3118] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:09:21.708755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa7d70-5021-44fe-8e0c-2ef9cf9c493c service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:21.735117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa7d70-5021-44fe-8e0c-2ef9cf9c493c service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:21.754592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd7d75-640d-4902-a831-9c33d10f8328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:21.757534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd7d75-640d-4902-a831-9c33d10f8328 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:21.758077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd7d75-640d-4902-a831-9c33d10f8328 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:21.758799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd7d75-640d-4902-a831-9c33d10f8328 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:21.760064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd7d75-640d-4902-a831-9c33d10f8328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:21.761938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd7d75-640d-4902-a831-9c33d10f8328 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:21.791400 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1559/3119] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:09:21.796006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a5bc3-958e-4985-b2a0-1ab14446880e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:21.813191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a5bc3-958e-4985-b2a0-1ab14446880e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:21.818708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14266e0-19e5-41c3-b5dc-c8310f91d59e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:21.820838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14266e0-19e5-41c3-b5dc-c8310f91d59e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:21.821423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14266e0-19e5-41c3-b5dc-c8310f91d59e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:21.821803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14266e0-19e5-41c3-b5dc-c8310f91d59e admin admin] PATH_INFO: `/v3/projects/9ccf44d7826045cfa5d4c6e87290d9d2/users/0cec7231f1c1445f965d620ae70e8463/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:21.849473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14266e0-19e5-41c3-b5dc-c8310f91d59e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ccf44d7826045cfa5d4c6e87290d9d2, user_id=0cec7231f1c1445f965d620ae70e8463, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:21.853423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14266e0-19e5-41c3-b5dc-c8310f91d59e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:21.937020 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1561/3120] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:09:21 2022] PUT /identity/v3/projects/9ccf44d7826045cfa5d4c6e87290d9d2/users/0cec7231f1c1445f965d620ae70e8463/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:21.948834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76e9df-7f56-4e13-af3f-5a103005b9a7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:21.949277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76e9df-7f56-4e13-af3f-5a103005b9a7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:21.949400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76e9df-7f56-4e13-af3f-5a103005b9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:22.026388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-dc76e9df-7f56-4e13-af3f-5a103005b9a7 None None] MFA Rules not processed for user `0cec7231f1c1445f965d620ae70e8463`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:22.029782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dc76e9df-7f56-4e13-af3f-5a103005b9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:22.229486 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1560/3121] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:22.234763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14266e0-19e5-41c3-b5dc-c8310f91d59e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:22.430840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14266e0-19e5-41c3-b5dc-c8310f91d59e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:22.444738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9020e-7788-478b-8ffd-e95c1e6bae93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:22.445737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9020e-7788-478b-8ffd-e95c1e6bae93 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:22.446229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9020e-7788-478b-8ffd-e95c1e6bae93 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:22.446771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9020e-7788-478b-8ffd-e95c1e6bae93 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:22.447863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9020e-7788-478b-8ffd-e95c1e6bae93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:22.450350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9020e-7788-478b-8ffd-e95c1e6bae93 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:22.606769 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1562/3122] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:09:22 2022] POST /identity/v3/projects => generated 373 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:22.609778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76e9df-7f56-4e13-af3f-5a103005b9a7 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:22.638658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc76e9df-7f56-4e13-af3f-5a103005b9a7 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:22.746733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a40709-26af-44d2-ba0b-844a1317acc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:22.747468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a40709-26af-44d2-ba0b-844a1317acc1 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:22.747940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a40709-26af-44d2-ba0b-844a1317acc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:22.748442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a40709-26af-44d2-ba0b-844a1317acc1 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:22.750168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a40709-26af-44d2-ba0b-844a1317acc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:22.752468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a40709-26af-44d2-ba0b-844a1317acc1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:22.842925 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1561/3123] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:09:22 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:22.846879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9020e-7788-478b-8ffd-e95c1e6bae93 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:22.863991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9020e-7788-478b-8ffd-e95c1e6bae93 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:22.894193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa04d6-6f0f-46f7-b1a6-1efc9b207641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:22.896055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa04d6-6f0f-46f7-b1a6-1efc9b207641 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:22.896529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa04d6-6f0f-46f7-b1a6-1efc9b207641 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:22.897285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa04d6-6f0f-46f7-b1a6-1efc9b207641 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:22.898379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa04d6-6f0f-46f7-b1a6-1efc9b207641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:22.901224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa04d6-6f0f-46f7-b1a6-1efc9b207641 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:22.927402 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1563/3124] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:09:22.929425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a40709-26af-44d2-ba0b-844a1317acc1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:22.966876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a40709-26af-44d2-ba0b-844a1317acc1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:22.982874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9b83b-9916-47e2-8766-9b261beebb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:22.985878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9b83b-9916-47e2-8766-9b261beebb20 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:22.986180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9b83b-9916-47e2-8766-9b261beebb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:22.986468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9b83b-9916-47e2-8766-9b261beebb20 admin admin] PATH_INFO: `/v3/projects/af47af9b4c194d5cbfeef925a3681221/users/4134af80aae942d69fc3dca0b65ce08c/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:23.031770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:23.063315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9b83b-9916-47e2-8766-9b261beebb20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af47af9b4c194d5cbfeef925a3681221, user_id=4134af80aae942d69fc3dca0b65ce08c, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:23.072018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9b83b-9916-47e2-8766-9b261beebb20 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:23.183286 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1562/3125] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:09:22 2022] PUT /identity/v3/projects/af47af9b4c194d5cbfeef925a3681221/users/4134af80aae942d69fc3dca0b65ce08c/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:09:23.189970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa04d6-6f0f-46f7-b1a6-1efc9b207641 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:23.204654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa04d6-6f0f-46f7-b1a6-1efc9b207641 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:23.339481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad622fd-cb8f-4059-a831-71511e31e1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:23.340512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad622fd-cb8f-4059-a831-71511e31e1cc admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:23.341023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad622fd-cb8f-4059-a831-71511e31e1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:23.341557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad622fd-cb8f-4059-a831-71511e31e1cc admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:23.342731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad622fd-cb8f-4059-a831-71511e31e1cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:23.352887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad622fd-cb8f-4059-a831-71511e31e1cc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:23.360778 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1564/3126] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:23 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:09:23.362223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9b83b-9916-47e2-8766-9b261beebb20 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:23.377526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9b83b-9916-47e2-8766-9b261beebb20 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:23.381352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:23.387042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e18210-0ba8-4aef-91bb-f57b9e6b9439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:23.387992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e18210-0ba8-4aef-91bb-f57b9e6b9439 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:23.388275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e18210-0ba8-4aef-91bb-f57b9e6b9439 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:23.388542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e18210-0ba8-4aef-91bb-f57b9e6b9439 admin admin] PATH_INFO: `/v3/projects/af47af9b4c194d5cbfeef925a3681221/users/4134af80aae942d69fc3dca0b65ce08c/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:23.391334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e18210-0ba8-4aef-91bb-f57b9e6b9439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af47af9b4c194d5cbfeef925a3681221, user_id=4134af80aae942d69fc3dca0b65ce08c, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:23.394790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e18210-0ba8-4aef-91bb-f57b9e6b9439 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:23.415176 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1563/3127] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:09:23 2022] PUT /identity/v3/projects/af47af9b4c194d5cbfeef925a3681221/users/4134af80aae942d69fc3dca0b65ce08c/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:23.426328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5692e64-0100-4ad5-96e9-ec91711de548 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:23.427910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5692e64-0100-4ad5-96e9-ec91711de548 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:23.427910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5692e64-0100-4ad5-96e9-ec91711de548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:23.455797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a5692e64-0100-4ad5-96e9-ec91711de548 None None] MFA Rules not processed for user `4134af80aae942d69fc3dca0b65ce08c`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:23.457030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a5692e64-0100-4ad5-96e9-ec91711de548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:23.552700 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1565/3128] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:23.564985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6875a6b-3e2b-422c-877a-34a18381ac6d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:23.565281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6875a6b-3e2b-422c-877a-34a18381ac6d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:23.565567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6875a6b-3e2b-422c-877a-34a18381ac6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:23.584814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c6875a6b-3e2b-422c-877a-34a18381ac6d None None] MFA Rules not processed for user `0cec7231f1c1445f965d620ae70e8463`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:23.586371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c6875a6b-3e2b-422c-877a-34a18381ac6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:23.713831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:23.720365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5692e64-0100-4ad5-96e9-ec91711de548 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:23.722244 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1564/3129] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:23.723223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a5692e64-0100-4ad5-96e9-ec91711de548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:23.840573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5692e64-0100-4ad5-96e9-ec91711de548 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:23.846406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008e05a-c215-4eed-9e3f-b3d56d608491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:23.847753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008e05a-c215-4eed-9e3f-b3d56d608491 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:23.848911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008e05a-c215-4eed-9e3f-b3d56d608491 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:23.849435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008e05a-c215-4eed-9e3f-b3d56d608491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:23.853232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d008e05a-c215-4eed-9e3f-b3d56d608491 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:23.960240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008e05a-c215-4eed-9e3f-b3d56d608491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:23.963375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008e05a-c215-4eed-9e3f-b3d56d608491 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:23.980319 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1566/3130] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:24.031833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6875a6b-3e2b-422c-877a-34a18381ac6d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:24.241832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6875a6b-3e2b-422c-877a-34a18381ac6d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:24.253711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4746a66c-78b3-4771-baab-4eb06e4c3a62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:24.255344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746a66c-78b3-4771-baab-4eb06e4c3a62 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:24.255877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746a66c-78b3-4771-baab-4eb06e4c3a62 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:24.257092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4746a66c-78b3-4771-baab-4eb06e4c3a62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:24.272357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746a66c-78b3-4771-baab-4eb06e4c3a62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:24.274565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4746a66c-78b3-4771-baab-4eb06e4c3a62 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:24.288180 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1565/3131] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:24 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:25.330921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008e05a-c215-4eed-9e3f-b3d56d608491 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:25.332692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d008e05a-c215-4eed-9e3f-b3d56d608491 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:25.350321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:25.623590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008e05a-c215-4eed-9e3f-b3d56d608491 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:25.634298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20981e-b37c-41ec-a559-e0c64fd098ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:25.635278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20981e-b37c-41ec-a559-e0c64fd098ec service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:25.635720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20981e-b37c-41ec-a559-e0c64fd098ec service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:25.636230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20981e-b37c-41ec-a559-e0c64fd098ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:25.638454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9a20981e-b37c-41ec-a559-e0c64fd098ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:25.654607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20981e-b37c-41ec-a559-e0c64fd098ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:25.656676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a20981e-b37c-41ec-a559-e0c64fd098ec service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:25.671121 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1567/3132] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:26.839396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1191f9-a149-4c46-af5f-909be572ecf8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:26.839882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1191f9-a149-4c46-af5f-909be572ecf8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:26.839986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1191f9-a149-4c46-af5f-909be572ecf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:26.913887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9a1191f9-a149-4c46-af5f-909be572ecf8 None None] MFA Rules not processed for user `659bd3ebefc348b7a176379bd342f9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:26.915650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9a1191f9-a149-4c46-af5f-909be572ecf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:27.092078 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1566/3133] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:27.174809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20981e-b37c-41ec-a559-e0c64fd098ec service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:27.176463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9a20981e-b37c-41ec-a559-e0c64fd098ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:27.199588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20981e-b37c-41ec-a559-e0c64fd098ec service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:27.205706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2cc9a9-c58d-4fa7-8f7d-04f536b4d79f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:27.206691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cc9a9-c58d-4fa7-8f7d-04f536b4d79f service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:27.207164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cc9a9-c58d-4fa7-8f7d-04f536b4d79f service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:27.207991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2cc9a9-c58d-4fa7-8f7d-04f536b4d79f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:27.234346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2cc9a9-c58d-4fa7-8f7d-04f536b4d79f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:27.235920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2cc9a9-c58d-4fa7-8f7d-04f536b4d79f service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:27.263759 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1568/3134] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:27.509383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1191f9-a149-4c46-af5f-909be572ecf8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:27.510918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9a1191f9-a149-4c46-af5f-909be572ecf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:27.529959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1191f9-a149-4c46-af5f-909be572ecf8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:27.561457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb64f3-d22a-45f0-b5b1-7b3e8daec691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:27.563655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb64f3-d22a-45f0-b5b1-7b3e8daec691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:27.563947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb64f3-d22a-45f0-b5b1-7b3e8daec691 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:27.564232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb64f3-d22a-45f0-b5b1-7b3e8daec691 admin admin] PATH_INFO: `/v3/users/fc6ee6dba5d24e90bff48384539fcd7f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:27.565747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb64f3-d22a-45f0-b5b1-7b3e8daec691 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc6ee6dba5d24e90bff48384539fcd7f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:27.567765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb64f3-d22a-45f0-b5b1-7b3e8daec691 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:27.582954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:27.951048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4ccb64f3-d22a-45f0-b5b1-7b3e8daec691 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc6ee6dba5d24e90bff48384539fcd7f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:27.971679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4ccb64f3-d22a-45f0-b5b1-7b3e8daec691 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc6ee6dba5d24e90bff48384539fcd7f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:27.972439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4ccb64f3-d22a-45f0-b5b1-7b3e8daec691 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc6ee6dba5d24e90bff48384539fcd7f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:27.973287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4ccb64f3-d22a-45f0-b5b1-7b3e8daec691 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc6ee6dba5d24e90bff48384539fcd7f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:28.066893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4ccb64f3-d22a-45f0-b5b1-7b3e8daec691 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc6ee6dba5d24e90bff48384539fcd7f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:28.110002 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1567/3135] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:09:27 2022] DELETE /identity/v3/users/fc6ee6dba5d24e90bff48384539fcd7f => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:28.821354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2cc9a9-c58d-4fa7-8f7d-04f536b4d79f service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:28.856874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2cc9a9-c58d-4fa7-8f7d-04f536b4d79f service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:29.024018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb37d20-bb0c-4b14-8039-8aab85848cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:29.025041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb37d20-bb0c-4b14-8039-8aab85848cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:29.025397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb37d20-bb0c-4b14-8039-8aab85848cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:29.025942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb37d20-bb0c-4b14-8039-8aab85848cdd admin admin] PATH_INFO: `/v3/projects/45a2b4e7e1484ea3807bc8d8bc31296c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:29.027304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb37d20-bb0c-4b14-8039-8aab85848cdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=45a2b4e7e1484ea3807bc8d8bc31296c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:29.029322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb37d20-bb0c-4b14-8039-8aab85848cdd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:29.363691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1cb37d20-bb0c-4b14-8039-8aab85848cdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45a2b4e7e1484ea3807bc8d8bc31296c', 'request_id': 'req-1cb37d20-bb0c-4b14-8039-8aab85848cdd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:29.403426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1cb37d20-bb0c-4b14-8039-8aab85848cdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45a2b4e7e1484ea3807bc8d8bc31296c', 'request_id': 'req-1cb37d20-bb0c-4b14-8039-8aab85848cdd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:29.501174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1cb37d20-bb0c-4b14-8039-8aab85848cdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45a2b4e7e1484ea3807bc8d8bc31296c', 'request_id': 'req-1cb37d20-bb0c-4b14-8039-8aab85848cdd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:29.518130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1cb37d20-bb0c-4b14-8039-8aab85848cdd admin admin] The token cache is being invalidate because project 45a2b4e7e1484ea3807bc8d8bc31296c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:09:29.518569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1cb37d20-bb0c-4b14-8039-8aab85848cdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:29.519548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1cb37d20-bb0c-4b14-8039-8aab85848cdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:29.521400 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1569/3136] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:09:28 2022] DELETE /identity/v3/projects/45a2b4e7e1484ea3807bc8d8bc31296c => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:29.525581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb64f3-d22a-45f0-b5b1-7b3e8daec691 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:29.529069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4ccb64f3-d22a-45f0-b5b1-7b3e8daec691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:29.543563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb64f3-d22a-45f0-b5b1-7b3e8daec691 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:29.631823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c448db-1d9f-46d5-a525-4d8feb306304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:29.632947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c448db-1d9f-46d5-a525-4d8feb306304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:29.633399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c448db-1d9f-46d5-a525-4d8feb306304 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:29.633885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c448db-1d9f-46d5-a525-4d8feb306304 admin admin] PATH_INFO: `/v3/users/e268920fabee4dee9b00d77e0eb222df` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:29.635425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c448db-1d9f-46d5-a525-4d8feb306304 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e268920fabee4dee9b00d77e0eb222df)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:29.639567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c448db-1d9f-46d5-a525-4d8feb306304 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:29.897281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20c448db-1d9f-46d5-a525-4d8feb306304 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e268920fabee4dee9b00d77e0eb222df'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:29.909461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20c448db-1d9f-46d5-a525-4d8feb306304 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e268920fabee4dee9b00d77e0eb222df'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:29.917739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20c448db-1d9f-46d5-a525-4d8feb306304 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e268920fabee4dee9b00d77e0eb222df'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:29.919151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20c448db-1d9f-46d5-a525-4d8feb306304 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e268920fabee4dee9b00d77e0eb222df'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:30.012597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20c448db-1d9f-46d5-a525-4d8feb306304 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e268920fabee4dee9b00d77e0eb222df'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:30.066733 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1568/3137] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:09:29 2022] DELETE /identity/v3/users/e268920fabee4dee9b00d77e0eb222df => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:30.247526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54b7b1-95fd-4c5d-814a-27120235180b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:30.248092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54b7b1-95fd-4c5d-814a-27120235180b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:30.248092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f54b7b1-95fd-4c5d-814a-27120235180b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:30.273267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4f54b7b1-95fd-4c5d-814a-27120235180b None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:30.280458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4f54b7b1-95fd-4c5d-814a-27120235180b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:30.286114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:30.417246 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1570/3138] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:09:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:30.453309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c448db-1d9f-46d5-a525-4d8feb306304 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:30.454795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-20c448db-1d9f-46d5-a525-4d8feb306304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:30.471796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:30.579280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c448db-1d9f-46d5-a525-4d8feb306304 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:30.592430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade922f8-0909-4cd4-a2a2-25a13db4c113 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:30.594240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade922f8-0909-4cd4-a2a2-25a13db4c113 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:30.597817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade922f8-0909-4cd4-a2a2-25a13db4c113 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:30.598186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade922f8-0909-4cd4-a2a2-25a13db4c113 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:30.633874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade922f8-0909-4cd4-a2a2-25a13db4c113 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:30.636606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade922f8-0909-4cd4-a2a2-25a13db4c113 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:30.659654 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1569/3139] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:09:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:09:31.222283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54b7b1-95fd-4c5d-814a-27120235180b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:31.222283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4f54b7b1-95fd-4c5d-814a-27120235180b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:31.317952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a020072-8f44-422c-9da6-a9001d946e50 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:31.319063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a020072-8f44-422c-9da6-a9001d946e50 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:31.319563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a020072-8f44-422c-9da6-a9001d946e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:31.368566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8a020072-8f44-422c-9da6-a9001d946e50 None None] MFA Rules not processed for user `659bd3ebefc348b7a176379bd342f9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:31.372760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8a020072-8f44-422c-9da6-a9001d946e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:31.420241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f54b7b1-95fd-4c5d-814a-27120235180b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:31.434567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc0202-7f6f-487e-a049-1d1e82795e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:31.435733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc0202-7f6f-487e-a049-1d1e82795e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:31.436237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc0202-7f6f-487e-a049-1d1e82795e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:31.436739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc0202-7f6f-487e-a049-1d1e82795e1b admin admin] PATH_INFO: `/v3/projects/737350d45b9544239e0736f663d80269` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:31.438562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc0202-7f6f-487e-a049-1d1e82795e1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=737350d45b9544239e0736f663d80269)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:31.453887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc0202-7f6f-487e-a049-1d1e82795e1b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:31.594630 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1570/3140] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:31.690559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b2fc0202-7f6f-487e-a049-1d1e82795e1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '737350d45b9544239e0736f663d80269', 'request_id': 'req-b2fc0202-7f6f-487e-a049-1d1e82795e1b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:31.719373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b2fc0202-7f6f-487e-a049-1d1e82795e1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '737350d45b9544239e0736f663d80269', 'request_id': 'req-b2fc0202-7f6f-487e-a049-1d1e82795e1b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:31.839621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b2fc0202-7f6f-487e-a049-1d1e82795e1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '737350d45b9544239e0736f663d80269', 'request_id': 'req-b2fc0202-7f6f-487e-a049-1d1e82795e1b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:31.870143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b2fc0202-7f6f-487e-a049-1d1e82795e1b admin admin] The token cache is being invalidate because project 737350d45b9544239e0736f663d80269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:09:31.870742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b2fc0202-7f6f-487e-a049-1d1e82795e1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:31.871603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b2fc0202-7f6f-487e-a049-1d1e82795e1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:31.873959 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1571/3141] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:09:31 2022] DELETE /identity/v3/projects/737350d45b9544239e0736f663d80269 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:31.879756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa934d90-262e-4a31-8698-e40685f444bc None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:31.883855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa934d90-262e-4a31-8698-e40685f444bc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:31.883855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa934d90-262e-4a31-8698-e40685f444bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:31.920523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-aa934d90-262e-4a31-8698-e40685f444bc None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:31.921983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-aa934d90-262e-4a31-8698-e40685f444bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:32.058490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6213902-b77e-463e-869f-3f13cbff33d1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:32.058783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6213902-b77e-463e-869f-3f13cbff33d1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:32.059066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6213902-b77e-463e-869f-3f13cbff33d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:32.061220 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1571/3142] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:32.127383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f6213902-b77e-463e-869f-3f13cbff33d1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:32.128822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f6213902-b77e-463e-869f-3f13cbff33d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:32.162639 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1572/3143] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:32.164597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa934d90-262e-4a31-8698-e40685f444bc None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:32.194618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa934d90-262e-4a31-8698-e40685f444bc None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:32.205451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6213902-b77e-463e-869f-3f13cbff33d1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:32.206835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f6213902-b77e-463e-869f-3f13cbff33d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:32.227566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52592b2a-674f-468f-8478-c15765d8efad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:32.228197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6213902-b77e-463e-869f-3f13cbff33d1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:32.230660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52592b2a-674f-468f-8478-c15765d8efad admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:32.236060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52592b2a-674f-468f-8478-c15765d8efad admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:32.237579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52592b2a-674f-468f-8478-c15765d8efad admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:32.241533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52592b2a-674f-468f-8478-c15765d8efad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:09:32.243839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d0f59-af49-449d-b263-3cfc13f501c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:32.244514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52592b2a-674f-468f-8478-c15765d8efad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:32.244728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d0f59-af49-449d-b263-3cfc13f501c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:32.251604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d0f59-af49-449d-b263-3cfc13f501c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:32.251604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866d0f59-af49-449d-b263-3cfc13f501c8 admin admin] PATH_INFO: `/v3/users/121de134dc0c4e148d09ef9d764c29c6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:32.251604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d0f59-af49-449d-b263-3cfc13f501c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=121de134dc0c4e148d09ef9d764c29c6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:32.251604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866d0f59-af49-449d-b263-3cfc13f501c8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:32.252020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52592b2a-674f-468f-8478-c15765d8efad admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:32.294980 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1572/3144] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:09:32.303195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52592b2a-674f-468f-8478-c15765d8efad admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:32.342649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52592b2a-674f-468f-8478-c15765d8efad admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:32.365926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754ea9b-3f0a-473c-8f1a-107715bde256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:32.369238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754ea9b-3f0a-473c-8f1a-107715bde256 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:32.369238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754ea9b-3f0a-473c-8f1a-107715bde256 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:32.369238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754ea9b-3f0a-473c-8f1a-107715bde256 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:32.370607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754ea9b-3f0a-473c-8f1a-107715bde256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:32.373381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754ea9b-3f0a-473c-8f1a-107715bde256 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:32.542077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:32.564286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-866d0f59-af49-449d-b263-3cfc13f501c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '121de134dc0c4e148d09ef9d764c29c6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:32.566198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-866d0f59-af49-449d-b263-3cfc13f501c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '121de134dc0c4e148d09ef9d764c29c6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:32.586068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-866d0f59-af49-449d-b263-3cfc13f501c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '121de134dc0c4e148d09ef9d764c29c6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:32.741395 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1573/3145] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:09:32 2022] POST /identity/v3/projects => generated 389 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:32.742549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754ea9b-3f0a-473c-8f1a-107715bde256 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:32.746550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-866d0f59-af49-449d-b263-3cfc13f501c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '121de134dc0c4e148d09ef9d764c29c6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:32.751927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-866d0f59-af49-449d-b263-3cfc13f501c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '121de134dc0c4e148d09ef9d764c29c6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:32.768233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:32.793772 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1573/3146] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:09:32 2022] DELETE /identity/v3/users/121de134dc0c4e148d09ef9d764c29c6 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:32.969764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754ea9b-3f0a-473c-8f1a-107715bde256 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:33.105664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4c924-27ff-4275-b6c9-fea5791c8fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:33.106802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4c924-27ff-4275-b6c9-fea5791c8fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:33.107879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4c924-27ff-4275-b6c9-fea5791c8fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:33.108321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4c924-27ff-4275-b6c9-fea5791c8fd0 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:33.109877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4c924-27ff-4275-b6c9-fea5791c8fd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:33.113372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4c924-27ff-4275-b6c9-fea5791c8fd0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:33.248684 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1574/3147] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:09:32 2022] POST /identity/v3/users => generated 295 bytes in 507 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:33.250986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d0f59-af49-449d-b263-3cfc13f501c8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:33.252936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-866d0f59-af49-449d-b263-3cfc13f501c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:33.299710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866d0f59-af49-449d-b263-3cfc13f501c8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:33.333810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5acbd-9e8e-4ddf-a63c-125f7093cd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:33.338595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5acbd-9e8e-4ddf-a63c-125f7093cd9d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:33.340377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5acbd-9e8e-4ddf-a63c-125f7093cd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:33.343733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5acbd-9e8e-4ddf-a63c-125f7093cd9d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:33.348613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5acbd-9e8e-4ddf-a63c-125f7093cd9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:33.367159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5acbd-9e8e-4ddf-a63c-125f7093cd9d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:33.407594 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1574/3148] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:09:33.411738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4c924-27ff-4275-b6c9-fea5791c8fd0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:33.435522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4c924-27ff-4275-b6c9-fea5791c8fd0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:33.445585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b184cc0e-c675-4e47-8506-7bbf24ded42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:33.446362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184cc0e-c675-4e47-8506-7bbf24ded42b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:33.446670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184cc0e-c675-4e47-8506-7bbf24ded42b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:33.446952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184cc0e-c675-4e47-8506-7bbf24ded42b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:33.454904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184cc0e-c675-4e47-8506-7bbf24ded42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:33.454904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b184cc0e-c675-4e47-8506-7bbf24ded42b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:33.477414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5acbd-9e8e-4ddf-a63c-125f7093cd9d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:33.504572 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1575/3149] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:09:33.510793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5acbd-9e8e-4ddf-a63c-125f7093cd9d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:33.516174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4ad9a-9e21-4c7c-9a36-7223080f8498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:33.517085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4ad9a-9e21-4c7c-9a36-7223080f8498 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:33.518113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4ad9a-9e21-4c7c-9a36-7223080f8498 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:33.521282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b4ad9a-9e21-4c7c-9a36-7223080f8498 admin admin] PATH_INFO: `/v3/projects/1f4d318ab1144e5b96abd0654f9bf1b1/users/fbd176173cd0497bb3f5c6271a538cb8/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:33.554531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4ad9a-9e21-4c7c-9a36-7223080f8498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4d318ab1144e5b96abd0654f9bf1b1, user_id=fbd176173cd0497bb3f5c6271a538cb8, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:33.559547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b4ad9a-9e21-4c7c-9a36-7223080f8498 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:33.672552 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1575/3150] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:09:33 2022] PUT /identity/v3/projects/1f4d318ab1144e5b96abd0654f9bf1b1/users/fbd176173cd0497bb3f5c6271a538cb8/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:33.678016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ba9da-ec2d-4080-8622-ebd36eb617ee None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:33.678692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ba9da-ec2d-4080-8622-ebd36eb617ee None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:33.679157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ba9da-ec2d-4080-8622-ebd36eb617ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:33.809643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a60ba9da-ec2d-4080-8622-ebd36eb617ee None None] MFA Rules not processed for user `fbd176173cd0497bb3f5c6271a538cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:33.812268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a60ba9da-ec2d-4080-8622-ebd36eb617ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:33.849863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4ad9a-9e21-4c7c-9a36-7223080f8498 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:33.854094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a7b4ad9a-9e21-4c7c-9a36-7223080f8498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:33.992593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b4ad9a-9e21-4c7c-9a36-7223080f8498 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:33.996615 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1576/3151] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:34.000229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ba9da-ec2d-4080-8622-ebd36eb617ee None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:34.008736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4c3a3-eb17-4929-9cf1-43ce393c5b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:34.010083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4c3a3-eb17-4929-9cf1-43ce393c5b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:34.010574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4c3a3-eb17-4929-9cf1-43ce393c5b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:34.011125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4c3a3-eb17-4929-9cf1-43ce393c5b7b admin admin] PATH_INFO: `/v3/projects/cde8dbaabafe47dc9aca37d9ef370819` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:34.012549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4c3a3-eb17-4929-9cf1-43ce393c5b7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cde8dbaabafe47dc9aca37d9ef370819)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:34.014831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4c3a3-eb17-4929-9cf1-43ce393c5b7b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:34.016990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ba9da-ec2d-4080-8622-ebd36eb617ee None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:34.022905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4187539-898a-4eae-bde7-43b475b50823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:34.031579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4187539-898a-4eae-bde7-43b475b50823 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:34.031579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4187539-898a-4eae-bde7-43b475b50823 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:34.031579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4187539-898a-4eae-bde7-43b475b50823 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:34.031579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4187539-898a-4eae-bde7-43b475b50823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:34.031579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4187539-898a-4eae-bde7-43b475b50823 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:34.166886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:34.191728 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1577/3152] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:09:33 2022] POST /identity/v3/projects => generated 389 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:34.194557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4187539-898a-4eae-bde7-43b475b50823 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:34.200862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bfa4c3a3-eb17-4929-9cf1-43ce393c5b7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cde8dbaabafe47dc9aca37d9ef370819', 'request_id': 'req-bfa4c3a3-eb17-4929-9cf1-43ce393c5b7b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:34.227331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bfa4c3a3-eb17-4929-9cf1-43ce393c5b7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cde8dbaabafe47dc9aca37d9ef370819', 'request_id': 'req-bfa4c3a3-eb17-4929-9cf1-43ce393c5b7b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:34.323693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bfa4c3a3-eb17-4929-9cf1-43ce393c5b7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cde8dbaabafe47dc9aca37d9ef370819', 'request_id': 'req-bfa4c3a3-eb17-4929-9cf1-43ce393c5b7b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:34.347054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bfa4c3a3-eb17-4929-9cf1-43ce393c5b7b admin admin] The token cache is being invalidate because project cde8dbaabafe47dc9aca37d9ef370819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:09:34.347366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bfa4c3a3-eb17-4929-9cf1-43ce393c5b7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:34.355384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bfa4c3a3-eb17-4929-9cf1-43ce393c5b7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:34.357875 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1576/3153] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:09:33 2022] DELETE /identity/v3/projects/cde8dbaabafe47dc9aca37d9ef370819 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:34.369443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b26482-3cda-47fc-96c1-ec99a9545dad None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:34.370182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b26482-3cda-47fc-96c1-ec99a9545dad None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:34.371265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b26482-3cda-47fc-96c1-ec99a9545dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:34.386700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4187539-898a-4eae-bde7-43b475b50823 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:34.397574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcda13-e325-4803-a5d2-6d3601da5a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:34.398404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcda13-e325-4803-a5d2-6d3601da5a79 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:34.398658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcda13-e325-4803-a5d2-6d3601da5a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:34.399066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bcda13-e325-4803-a5d2-6d3601da5a79 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:34.399621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcda13-e325-4803-a5d2-6d3601da5a79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:34.401676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bcda13-e325-4803-a5d2-6d3601da5a79 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:34.450092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c2b26482-3cda-47fc-96c1-ec99a9545dad None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:34.455171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c2b26482-3cda-47fc-96c1-ec99a9545dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:34.495771 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1577/3154] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:34.500634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b14b27-4cea-40ab-88a9-efeb17acbea1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:34.503432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b14b27-4cea-40ab-88a9-efeb17acbea1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:34.503840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b14b27-4cea-40ab-88a9-efeb17acbea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:34.531958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:34.538546 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1578/3155] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:09:34 2022] POST /identity/v3/users => generated 301 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:34.544228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcda13-e325-4803-a5d2-6d3601da5a79 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:34.547063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c3bcda13-e325-4803-a5d2-6d3601da5a79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:34.563090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bcda13-e325-4803-a5d2-6d3601da5a79 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:34.569272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dacd40-bc8d-4ced-8a23-507d8e7fa1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:34.570131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dacd40-bc8d-4ced-8a23-507d8e7fa1f5 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:34.570483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dacd40-bc8d-4ced-8a23-507d8e7fa1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:34.570838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dacd40-bc8d-4ced-8a23-507d8e7fa1f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:34.571770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dacd40-bc8d-4ced-8a23-507d8e7fa1f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:34.575938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dacd40-bc8d-4ced-8a23-507d8e7fa1f5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:34.577978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e6b14b27-4cea-40ab-88a9-efeb17acbea1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:34.579251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e6b14b27-4cea-40ab-88a9-efeb17acbea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:34.595351 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1579/3156] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:09:34.600196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dacd40-bc8d-4ced-8a23-507d8e7fa1f5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:34.612008 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1578/3157] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:34.620577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b14b27-4cea-40ab-88a9-efeb17acbea1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:34.621701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dacd40-bc8d-4ced-8a23-507d8e7fa1f5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:34.632282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd617dc7-0bff-4e4c-9d63-5f2a9b8ba047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:34.633289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd617dc7-0bff-4e4c-9d63-5f2a9b8ba047 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:34.636236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd617dc7-0bff-4e4c-9d63-5f2a9b8ba047 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:34.636715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd617dc7-0bff-4e4c-9d63-5f2a9b8ba047 admin admin] PATH_INFO: `/v3/projects/31a5f1cb172e42689b6528e97b771c40/users/7574038f38fc402bb7097beb4ac93426/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:34.642864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b14b27-4cea-40ab-88a9-efeb17acbea1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:34.651240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff70e6-157c-4770-bab6-7e5a25b4509f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:34.652508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff70e6-157c-4770-bab6-7e5a25b4509f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:34.653043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff70e6-157c-4770-bab6-7e5a25b4509f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:34.654330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff70e6-157c-4770-bab6-7e5a25b4509f admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:34.656046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff70e6-157c-4770-bab6-7e5a25b4509f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:09:34.656645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff70e6-157c-4770-bab6-7e5a25b4509f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:34.658809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff70e6-157c-4770-bab6-7e5a25b4509f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:34.687104 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1579/3158] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:09:34.688642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd617dc7-0bff-4e4c-9d63-5f2a9b8ba047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31a5f1cb172e42689b6528e97b771c40, user_id=7574038f38fc402bb7097beb4ac93426, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:34.690161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff70e6-157c-4770-bab6-7e5a25b4509f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:34.693087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd617dc7-0bff-4e4c-9d63-5f2a9b8ba047 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:34.705282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff70e6-157c-4770-bab6-7e5a25b4509f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:34.712007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75d4c6-973d-4556-9d9e-f9b52bcdec4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:34.713597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75d4c6-973d-4556-9d9e-f9b52bcdec4b admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:34.714506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75d4c6-973d-4556-9d9e-f9b52bcdec4b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:34.714985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75d4c6-973d-4556-9d9e-f9b52bcdec4b admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:34.717897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75d4c6-973d-4556-9d9e-f9b52bcdec4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:34.724653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75d4c6-973d-4556-9d9e-f9b52bcdec4b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:34.778954 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1580/3159] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:09:34 2022] PUT /identity/v3/projects/31a5f1cb172e42689b6528e97b771c40/users/7574038f38fc402bb7097beb4ac93426/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:09:34.783373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd617dc7-0bff-4e4c-9d63-5f2a9b8ba047 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:34.858701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd617dc7-0bff-4e4c-9d63-5f2a9b8ba047 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:34.870301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b0efc-bc8f-455e-94c6-bc31270cdbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:34.871583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b0efc-bc8f-455e-94c6-bc31270cdbdc admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:34.871834 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1580/3160] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:09:34 2022] POST /identity/v3/projects => generated 377 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:34.875430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b0efc-bc8f-455e-94c6-bc31270cdbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:34.875959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849b0efc-bc8f-455e-94c6-bc31270cdbdc admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:34.876505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75d4c6-973d-4556-9d9e-f9b52bcdec4b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:34.876902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b0efc-bc8f-455e-94c6-bc31270cdbdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:34.878574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849b0efc-bc8f-455e-94c6-bc31270cdbdc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:34.908026 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1581/3161] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:09:34.909123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b0efc-bc8f-455e-94c6-bc31270cdbdc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:34.928644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849b0efc-bc8f-455e-94c6-bc31270cdbdc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:35.059519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75d4c6-973d-4556-9d9e-f9b52bcdec4b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:35.092879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ef317-edd2-40e2-8676-a3cbd0f7951a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:35.093696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ef317-edd2-40e2-8676-a3cbd0f7951a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:35.093901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ef317-edd2-40e2-8676-a3cbd0f7951a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:35.094182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ef317-edd2-40e2-8676-a3cbd0f7951a admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:35.094899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ef317-edd2-40e2-8676-a3cbd0f7951a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:35.095869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85a2b3-8f80-484d-b2e9-4caf32c1cd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:35.101280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85a2b3-8f80-484d-b2e9-4caf32c1cd85 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:35.101644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85a2b3-8f80-484d-b2e9-4caf32c1cd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:35.102831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85a2b3-8f80-484d-b2e9-4caf32c1cd85 admin admin] PATH_INFO: `/v3/projects/31a5f1cb172e42689b6528e97b771c40/users/7574038f38fc402bb7097beb4ac93426/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:35.105731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ef317-edd2-40e2-8676-a3cbd0f7951a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:35.106015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85a2b3-8f80-484d-b2e9-4caf32c1cd85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31a5f1cb172e42689b6528e97b771c40, user_id=7574038f38fc402bb7097beb4ac93426, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:35.109543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85a2b3-8f80-484d-b2e9-4caf32c1cd85 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:35.140146 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1582/3162] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:09:34 2022] PUT /identity/v3/projects/31a5f1cb172e42689b6528e97b771c40/users/7574038f38fc402bb7097beb4ac93426/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:35.144817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c9c8b-9f74-44a5-aa74-5cff968d9bfb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:35.145361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c9c8b-9f74-44a5-aa74-5cff968d9bfb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:35.145752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c9c8b-9f74-44a5-aa74-5cff968d9bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:35.202271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-697c9c8b-9f74-44a5-aa74-5cff968d9bfb None None] MFA Rules not processed for user `7574038f38fc402bb7097beb4ac93426`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:35.214676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-697c9c8b-9f74-44a5-aa74-5cff968d9bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:35.227042 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1581/3163] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:09:34 2022] POST /identity/v3/users => generated 289 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:35.230801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ef317-edd2-40e2-8676-a3cbd0f7951a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:35.258884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ef317-edd2-40e2-8676-a3cbd0f7951a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:35.303599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:35.406650 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1583/3164] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:35.425946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d76391-20a3-4f19-9abe-7645f0fab516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:35.426602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d76391-20a3-4f19-9abe-7645f0fab516 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:35.426755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d76391-20a3-4f19-9abe-7645f0fab516 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:35.427243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d76391-20a3-4f19-9abe-7645f0fab516 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:35.428268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d76391-20a3-4f19-9abe-7645f0fab516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:35.430180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d76391-20a3-4f19-9abe-7645f0fab516 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:35.477220 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1582/3165] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:09:35.478188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c9c8b-9f74-44a5-aa74-5cff968d9bfb None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:35.516347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c9c8b-9f74-44a5-aa74-5cff968d9bfb None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:35.526156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b284c-6860-4b74-b833-a76bc098f942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:35.527149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b284c-6860-4b74-b833-a76bc098f942 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:35.527760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b284c-6860-4b74-b833-a76bc098f942 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:35.528556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b284c-6860-4b74-b833-a76bc098f942 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:35.529645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b284c-6860-4b74-b833-a76bc098f942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:35.540949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b284c-6860-4b74-b833-a76bc098f942 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:35.591820 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1584/3166] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:09:35.594922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d76391-20a3-4f19-9abe-7645f0fab516 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:35.615741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d76391-20a3-4f19-9abe-7645f0fab516 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:35.618502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96d31d-871e-4f92-9679-3842870d679d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:35.619529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96d31d-871e-4f92-9679-3842870d679d admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:35.622527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96d31d-871e-4f92-9679-3842870d679d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:35.622956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96d31d-871e-4f92-9679-3842870d679d admin admin] PATH_INFO: `/v3/projects/7e1a1360253a413d8ee275cb1d1fac02/users/f5df3bc2a59b4d99b81932deac1ece64/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:35.662925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96d31d-871e-4f92-9679-3842870d679d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e1a1360253a413d8ee275cb1d1fac02, user_id=f5df3bc2a59b4d99b81932deac1ece64, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:35.666691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96d31d-871e-4f92-9679-3842870d679d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:35.813682 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1583/3167] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:09:35 2022] PUT /identity/v3/projects/7e1a1360253a413d8ee275cb1d1fac02/users/f5df3bc2a59b4d99b81932deac1ece64/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:09:35.819214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a356db-d0b3-46cd-8add-297ceb0f7a3e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:35.819741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a356db-d0b3-46cd-8add-297ceb0f7a3e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:35.820024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a356db-d0b3-46cd-8add-297ceb0f7a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:35.844919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f5a356db-d0b3-46cd-8add-297ceb0f7a3e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:35.846808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f5a356db-d0b3-46cd-8add-297ceb0f7a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:35.966218 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1585/3168] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:35.976200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96d31d-871e-4f92-9679-3842870d679d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:35.977863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4d96d31d-871e-4f92-9679-3842870d679d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:36.117056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96d31d-871e-4f92-9679-3842870d679d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:36.126763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713df59-960d-487a-9847-cb43636843fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:36.128026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713df59-960d-487a-9847-cb43636843fd service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:36.128385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713df59-960d-487a-9847-cb43636843fd service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:36.128673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0713df59-960d-487a-9847-cb43636843fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:36.161416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713df59-960d-487a-9847-cb43636843fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:36.166869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0713df59-960d-487a-9847-cb43636843fd service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:36.189302 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1584/3169] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:36.347955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:36.547393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a356db-d0b3-46cd-8add-297ceb0f7a3e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:36.549100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f5a356db-d0b3-46cd-8add-297ceb0f7a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:36.683805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:36.728536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a356db-d0b3-46cd-8add-297ceb0f7a3e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:36.742304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530314a-9eb6-4b32-a31e-1676ef7c8222 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:36.746933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530314a-9eb6-4b32-a31e-1676ef7c8222 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:36.747380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530314a-9eb6-4b32-a31e-1676ef7c8222 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:36.747568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530314a-9eb6-4b32-a31e-1676ef7c8222 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:36.750371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3530314a-9eb6-4b32-a31e-1676ef7c8222 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:36.871730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530314a-9eb6-4b32-a31e-1676ef7c8222 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:36.874104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3530314a-9eb6-4b32-a31e-1676ef7c8222 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:36.921635 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1586/3170] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:41.324888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713df59-960d-487a-9847-cb43636843fd service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:41.326553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0713df59-960d-487a-9847-cb43636843fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:41.350983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0713df59-960d-487a-9847-cb43636843fd service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:41.359847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e59ca-fde2-48bb-8e0a-7e6bdb4263f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:41.361671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e59ca-fde2-48bb-8e0a-7e6bdb4263f1 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:41.362241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e59ca-fde2-48bb-8e0a-7e6bdb4263f1 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:41.362771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e59ca-fde2-48bb-8e0a-7e6bdb4263f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:41.488582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e59ca-fde2-48bb-8e0a-7e6bdb4263f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:41.492145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e59ca-fde2-48bb-8e0a-7e6bdb4263f1 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:41.508093 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1585/3171] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:46.315013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530314a-9eb6-4b32-a31e-1676ef7c8222 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:46.317233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3530314a-9eb6-4b32-a31e-1676ef7c8222 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:46.360984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530314a-9eb6-4b32-a31e-1676ef7c8222 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:46.370474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d04129-70f5-42fb-82e4-3adc23e969c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:46.372231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d04129-70f5-42fb-82e4-3adc23e969c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:46.373469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d04129-70f5-42fb-82e4-3adc23e969c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:46.373927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d04129-70f5-42fb-82e4-3adc23e969c6 admin admin] PATH_INFO: `/v3/users/5c65140c062f4eedb8bfe963137de014` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:46.377575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d04129-70f5-42fb-82e4-3adc23e969c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c65140c062f4eedb8bfe963137de014)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:46.384073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d04129-70f5-42fb-82e4-3adc23e969c6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:46.671534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-60d04129-70f5-42fb-82e4-3adc23e969c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c65140c062f4eedb8bfe963137de014'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:46.688562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-60d04129-70f5-42fb-82e4-3adc23e969c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c65140c062f4eedb8bfe963137de014'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:46.691693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-60d04129-70f5-42fb-82e4-3adc23e969c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c65140c062f4eedb8bfe963137de014'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:46.692625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-60d04129-70f5-42fb-82e4-3adc23e969c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c65140c062f4eedb8bfe963137de014'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:46.774556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-60d04129-70f5-42fb-82e4-3adc23e969c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c65140c062f4eedb8bfe963137de014'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:46.823433 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1587/3172] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:09:46 2022] DELETE /identity/v3/users/5c65140c062f4eedb8bfe963137de014 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:49.153204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637af54d-a637-4df5-8a1b-4b070dbda4d4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:49.153707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637af54d-a637-4df5-8a1b-4b070dbda4d4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:49.153825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637af54d-a637-4df5-8a1b-4b070dbda4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:49.187135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d04129-70f5-42fb-82e4-3adc23e969c6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:49.188834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-60d04129-70f5-42fb-82e4-3adc23e969c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:49.202784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-637af54d-a637-4df5-8a1b-4b070dbda4d4 None None] MFA Rules not processed for user `f5df3bc2a59b4d99b81932deac1ece64`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:49.204495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-637af54d-a637-4df5-8a1b-4b070dbda4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:49.353468 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1586/3173] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:09:49.395502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d04129-70f5-42fb-82e4-3adc23e969c6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:49.408033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c928d4-fcbb-4883-b10e-559be56e86e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:49.409531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c928d4-fcbb-4883-b10e-559be56e86e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:49.410166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c928d4-fcbb-4883-b10e-559be56e86e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:49.410801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c928d4-fcbb-4883-b10e-559be56e86e2 admin admin] PATH_INFO: `/v3/projects/0dd2a795de30463db3d150e44df69489` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:49.413113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c928d4-fcbb-4883-b10e-559be56e86e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dd2a795de30463db3d150e44df69489)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:49.417649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c928d4-fcbb-4883-b10e-559be56e86e2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:49.456969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:49.610859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-09c928d4-fcbb-4883-b10e-559be56e86e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dd2a795de30463db3d150e44df69489', 'request_id': 'req-09c928d4-fcbb-4883-b10e-559be56e86e2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:49.629184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-09c928d4-fcbb-4883-b10e-559be56e86e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dd2a795de30463db3d150e44df69489', 'request_id': 'req-09c928d4-fcbb-4883-b10e-559be56e86e2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:49.700242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-09c928d4-fcbb-4883-b10e-559be56e86e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dd2a795de30463db3d150e44df69489', 'request_id': 'req-09c928d4-fcbb-4883-b10e-559be56e86e2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:49.723800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-09c928d4-fcbb-4883-b10e-559be56e86e2 admin admin] The token cache is being invalidate because project 0dd2a795de30463db3d150e44df69489 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:09:49.724117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-09c928d4-fcbb-4883-b10e-559be56e86e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:49.725009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-09c928d4-fcbb-4883-b10e-559be56e86e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:49.727748 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1588/3174] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:09:49 2022] DELETE /identity/v3/projects/0dd2a795de30463db3d150e44df69489 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:49.734216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03de30-c7ff-43ad-bfab-cfb2a61b0ed4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:49.735582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03de30-c7ff-43ad-bfab-cfb2a61b0ed4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:49.736067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b03de30-c7ff-43ad-bfab-cfb2a61b0ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:49.772195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0b03de30-c7ff-43ad-bfab-cfb2a61b0ed4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:49.774507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0b03de30-c7ff-43ad-bfab-cfb2a61b0ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:49.878702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:49.899340 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1587/3175] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:49.908457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbdf5e-23e1-423c-8d1c-2a525724b000 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:49.908853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbdf5e-23e1-423c-8d1c-2a525724b000 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:49.909189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdbdf5e-23e1-423c-8d1c-2a525724b000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:49.931649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1fdbdf5e-23e1-423c-8d1c-2a525724b000 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:49.933374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1fdbdf5e-23e1-423c-8d1c-2a525724b000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:49.944981 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1589/3176] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:49.964243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03de30-c7ff-43ad-bfab-cfb2a61b0ed4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:49.979399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b03de30-c7ff-43ad-bfab-cfb2a61b0ed4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:49.989677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9cd4f-4ee5-4403-93dc-6f0a5178f8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:49.989677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9cd4f-4ee5-4403-93dc-6f0a5178f8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:49.989677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9cd4f-4ee5-4403-93dc-6f0a5178f8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:49.989677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9cd4f-4ee5-4403-93dc-6f0a5178f8d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:49.992684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9cd4f-4ee5-4403-93dc-6f0a5178f8d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:09:49.992684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9cd4f-4ee5-4403-93dc-6f0a5178f8d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:49.993852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9cd4f-4ee5-4403-93dc-6f0a5178f8d3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:50.021475 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1588/3177] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:09:50.022769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbdf5e-23e1-423c-8d1c-2a525724b000 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:50.033993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdbdf5e-23e1-423c-8d1c-2a525724b000 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:50.041722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbeafb3-1a47-4392-8aeb-92d907b8a138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:50.045604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbeafb3-1a47-4392-8aeb-92d907b8a138 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:50.045604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbeafb3-1a47-4392-8aeb-92d907b8a138 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:50.045604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbeafb3-1a47-4392-8aeb-92d907b8a138 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:50.045604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbeafb3-1a47-4392-8aeb-92d907b8a138 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:50.046534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbeafb3-1a47-4392-8aeb-92d907b8a138 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:50.223039 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1590/3178] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:09:50 2022] POST /identity/v3/projects => generated 413 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:50.227358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9cd4f-4ee5-4403-93dc-6f0a5178f8d3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:50.242149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9cd4f-4ee5-4403-93dc-6f0a5178f8d3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:50.306555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbeafb3-1a47-4392-8aeb-92d907b8a138 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:50.309483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6bbeafb3-1a47-4392-8aeb-92d907b8a138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:50.411155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351088c0-9995-4df5-a14a-a2bf2a7e2ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:50.411863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351088c0-9995-4df5-a14a-a2bf2a7e2ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:50.412327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351088c0-9995-4df5-a14a-a2bf2a7e2ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:50.412745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351088c0-9995-4df5-a14a-a2bf2a7e2ae6 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:50.413619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351088c0-9995-4df5-a14a-a2bf2a7e2ae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:50.419849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351088c0-9995-4df5-a14a-a2bf2a7e2ae6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:50.501332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbeafb3-1a47-4392-8aeb-92d907b8a138 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:50.508835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fab805-e3e0-42b9-8d84-65b0a0a06161 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:50.509516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fab805-e3e0-42b9-8d84-65b0a0a06161 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:50.509809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fab805-e3e0-42b9-8d84-65b0a0a06161 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:50.510113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fab805-e3e0-42b9-8d84-65b0a0a06161 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:50.519390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-20fab805-e3e0-42b9-8d84-65b0a0a06161 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:50.551664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fab805-e3e0-42b9-8d84-65b0a0a06161 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:50.553240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fab805-e3e0-42b9-8d84-65b0a0a06161 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:50.560578 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1589/3179] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:09:50 2022] POST /identity/v3/users => generated 307 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:50.562585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351088c0-9995-4df5-a14a-a2bf2a7e2ae6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:50.583338 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1591/3180] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:50.611227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351088c0-9995-4df5-a14a-a2bf2a7e2ae6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:50.624993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03788c0f-28e7-4163-882c-051e5ba9da8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:50.625683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03788c0f-28e7-4163-882c-051e5ba9da8f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:50.631666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03788c0f-28e7-4163-882c-051e5ba9da8f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:50.631666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03788c0f-28e7-4163-882c-051e5ba9da8f admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:50.631666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03788c0f-28e7-4163-882c-051e5ba9da8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:50.633145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03788c0f-28e7-4163-882c-051e5ba9da8f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:50.666201 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1590/3181] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:09:50.666855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fab805-e3e0-42b9-8d84-65b0a0a06161 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:50.683066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fab805-e3e0-42b9-8d84-65b0a0a06161 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:50.698401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2293cd9-6ffa-4ece-a813-4002e21af6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:50.704292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2293cd9-6ffa-4ece-a813-4002e21af6d3 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:50.705664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2293cd9-6ffa-4ece-a813-4002e21af6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:50.706004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2293cd9-6ffa-4ece-a813-4002e21af6d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:50.706808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2293cd9-6ffa-4ece-a813-4002e21af6d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:50.708459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2293cd9-6ffa-4ece-a813-4002e21af6d3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:50.752652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03788c0f-28e7-4163-882c-051e5ba9da8f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:50.758170 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1592/3182] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:09:50.768251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03788c0f-28e7-4163-882c-051e5ba9da8f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:50.783076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49ba1e-70ae-42e0-a320-0f06e43b0aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:50.786560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49ba1e-70ae-42e0-a320-0f06e43b0aa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:50.786560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49ba1e-70ae-42e0-a320-0f06e43b0aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:50.786560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49ba1e-70ae-42e0-a320-0f06e43b0aa2 admin admin] PATH_INFO: `/v3/projects/3450ad991a3943e3a4c4233551a42729/users/4112534978ba4622b0fbbb9a92791876/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:50.824190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e49ba1e-70ae-42e0-a320-0f06e43b0aa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3450ad991a3943e3a4c4233551a42729, user_id=4112534978ba4622b0fbbb9a92791876, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:50.827217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e49ba1e-70ae-42e0-a320-0f06e43b0aa2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:50.942921 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1591/3183] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:09:50 2022] PUT /identity/v3/projects/3450ad991a3943e3a4c4233551a42729/users/4112534978ba4622b0fbbb9a92791876/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:50.948894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61992f9-8d3a-4dfe-aaba-b623d996f6c0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:50.949182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61992f9-8d3a-4dfe-aaba-b623d996f6c0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:50.949657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61992f9-8d3a-4dfe-aaba-b623d996f6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:51.014312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f61992f9-8d3a-4dfe-aaba-b623d996f6c0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:51.016884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f61992f9-8d3a-4dfe-aaba-b623d996f6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:51.319222 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1593/3184] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:51.353370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49ba1e-70ae-42e0-a320-0f06e43b0aa2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:51.443169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49ba1e-70ae-42e0-a320-0f06e43b0aa2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:51.456175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c97e0-6394-4da3-a2d9-60e68ba8b5cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:51.456892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c97e0-6394-4da3-a2d9-60e68ba8b5cc service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:51.457160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c97e0-6394-4da3-a2d9-60e68ba8b5cc service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:51.457502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4c97e0-6394-4da3-a2d9-60e68ba8b5cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:51.473302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c97e0-6394-4da3-a2d9-60e68ba8b5cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:51.474886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4c97e0-6394-4da3-a2d9-60e68ba8b5cc service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:51.486416 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1592/3185] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:51.556982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61992f9-8d3a-4dfe-aaba-b623d996f6c0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:51.576636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f61992f9-8d3a-4dfe-aaba-b623d996f6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:51.587063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:51.718307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61992f9-8d3a-4dfe-aaba-b623d996f6c0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:51.977152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1793379-30c8-4d34-bb36-1a65687eb5f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:51.977152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1793379-30c8-4d34-bb36-1a65687eb5f7 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:51.977152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1793379-30c8-4d34-bb36-1a65687eb5f7 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:51.977152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1793379-30c8-4d34-bb36-1a65687eb5f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:51.977938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b1793379-30c8-4d34-bb36-1a65687eb5f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:51.977938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1793379-30c8-4d34-bb36-1a65687eb5f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:51.977938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1793379-30c8-4d34-bb36-1a65687eb5f7 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:51.977938 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1594/3186] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:09:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:09:55.183741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c97e0-6394-4da3-a2d9-60e68ba8b5cc service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:55.186276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7e4c97e0-6394-4da3-a2d9-60e68ba8b5cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:55.206755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4c97e0-6394-4da3-a2d9-60e68ba8b5cc service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:55.221745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df6121-3117-4603-8cd5-2c2160e57d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:55.233475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df6121-3117-4603-8cd5-2c2160e57d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:55.233475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df6121-3117-4603-8cd5-2c2160e57d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:55.233475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00df6121-3117-4603-8cd5-2c2160e57d30 admin admin] PATH_INFO: `/v3/users/7b44b524592e4a57a26681767b9c4541` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:55.233475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df6121-3117-4603-8cd5-2c2160e57d30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b44b524592e4a57a26681767b9c4541)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:55.233475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00df6121-3117-4603-8cd5-2c2160e57d30 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:55.367317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:55.483942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-00df6121-3117-4603-8cd5-2c2160e57d30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b44b524592e4a57a26681767b9c4541'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:55.486457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-00df6121-3117-4603-8cd5-2c2160e57d30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b44b524592e4a57a26681767b9c4541'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:55.499106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-00df6121-3117-4603-8cd5-2c2160e57d30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b44b524592e4a57a26681767b9c4541'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:55.583701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-00df6121-3117-4603-8cd5-2c2160e57d30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b44b524592e4a57a26681767b9c4541'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:55.588294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-00df6121-3117-4603-8cd5-2c2160e57d30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b44b524592e4a57a26681767b9c4541'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:55.638943 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1593/3187] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:09:55 2022] DELETE /identity/v3/users/7b44b524592e4a57a26681767b9c4541 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:56.168856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4a356-55ea-4c7a-a56f-257b83940aea None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:56.173110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4a356-55ea-4c7a-a56f-257b83940aea None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:56.173110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4a356-55ea-4c7a-a56f-257b83940aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:56.196108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d1e4a356-55ea-4c7a-a56f-257b83940aea None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:56.198107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d1e4a356-55ea-4c7a-a56f-257b83940aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:56.287625 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1595/3188] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:09:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:56.309979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df6121-3117-4603-8cd5-2c2160e57d30 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:56.309979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-00df6121-3117-4603-8cd5-2c2160e57d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:56.409783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00df6121-3117-4603-8cd5-2c2160e57d30 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:56.417012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edf6fa-8195-45f1-9917-dad7a2978a8b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:56.417768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edf6fa-8195-45f1-9917-dad7a2978a8b service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:56.418151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edf6fa-8195-45f1-9917-dad7a2978a8b service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:56.418498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edf6fa-8195-45f1-9917-dad7a2978a8b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:56.436962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17edf6fa-8195-45f1-9917-dad7a2978a8b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:56.439348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17edf6fa-8195-45f1-9917-dad7a2978a8b service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:56.451457 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1594/3189] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:09:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:09:56.868599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4a356-55ea-4c7a-a56f-257b83940aea None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:56.870889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d1e4a356-55ea-4c7a-a56f-257b83940aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:56.967041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:57.012289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e4a356-55ea-4c7a-a56f-257b83940aea None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:57.026618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51133f0d-0a5c-4cea-acd3-67f847005d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:57.031266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51133f0d-0a5c-4cea-acd3-67f847005d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:57.032325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51133f0d-0a5c-4cea-acd3-67f847005d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:57.033144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51133f0d-0a5c-4cea-acd3-67f847005d87 admin admin] PATH_INFO: `/v3/projects/2f5c3054461d4d07861a1685cfeba36a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:57.034973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51133f0d-0a5c-4cea-acd3-67f847005d87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f5c3054461d4d07861a1685cfeba36a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:57.040663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51133f0d-0a5c-4cea-acd3-67f847005d87 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:57.182616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-51133f0d-0a5c-4cea-acd3-67f847005d87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f5c3054461d4d07861a1685cfeba36a', 'request_id': 'req-51133f0d-0a5c-4cea-acd3-67f847005d87'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:57.191643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-51133f0d-0a5c-4cea-acd3-67f847005d87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f5c3054461d4d07861a1685cfeba36a', 'request_id': 'req-51133f0d-0a5c-4cea-acd3-67f847005d87'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:57.312262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-51133f0d-0a5c-4cea-acd3-67f847005d87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f5c3054461d4d07861a1685cfeba36a', 'request_id': 'req-51133f0d-0a5c-4cea-acd3-67f847005d87'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:57.336631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-51133f0d-0a5c-4cea-acd3-67f847005d87 admin admin] The token cache is being invalidate because project 2f5c3054461d4d07861a1685cfeba36a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:09:57.337379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-51133f0d-0a5c-4cea-acd3-67f847005d87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:57.339071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-51133f0d-0a5c-4cea-acd3-67f847005d87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:57.344535 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1596/3190] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:09:56 2022] DELETE /identity/v3/projects/2f5c3054461d4d07861a1685cfeba36a => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:57.345477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edf6fa-8195-45f1-9917-dad7a2978a8b service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:57.356586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edf6fa-8195-45f1-9917-dad7a2978a8b service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:57.430693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c21a2a-e543-4837-8f6b-147fe28ed9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:57.431877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c21a2a-e543-4837-8f6b-147fe28ed9d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:57.433728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c21a2a-e543-4837-8f6b-147fe28ed9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:57.434527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c21a2a-e543-4837-8f6b-147fe28ed9d3 admin admin] PATH_INFO: `/v3/users/f97b8d315694418db9c5915dc2c34c71` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:57.437119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c21a2a-e543-4837-8f6b-147fe28ed9d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f97b8d315694418db9c5915dc2c34c71)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:57.439522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c21a2a-e543-4837-8f6b-147fe28ed9d3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:57.666754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:57.684985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-01c21a2a-e543-4837-8f6b-147fe28ed9d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f97b8d315694418db9c5915dc2c34c71'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:57.689026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-01c21a2a-e543-4837-8f6b-147fe28ed9d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f97b8d315694418db9c5915dc2c34c71'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:57.715926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-01c21a2a-e543-4837-8f6b-147fe28ed9d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f97b8d315694418db9c5915dc2c34c71'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:57.803421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-01c21a2a-e543-4837-8f6b-147fe28ed9d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f97b8d315694418db9c5915dc2c34c71'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:57.804408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-01c21a2a-e543-4837-8f6b-147fe28ed9d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f97b8d315694418db9c5915dc2c34c71'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:57.834068 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1595/3191] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:09:57 2022] DELETE /identity/v3/users/f97b8d315694418db9c5915dc2c34c71 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:58.321017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51133f0d-0a5c-4cea-acd3-67f847005d87 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:58.322591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-51133f0d-0a5c-4cea-acd3-67f847005d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:58.341445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51133f0d-0a5c-4cea-acd3-67f847005d87 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:58.470419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764f676-8b5d-44a8-99cc-bccd65c552a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:58.470419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764f676-8b5d-44a8-99cc-bccd65c552a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:58.470419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764f676-8b5d-44a8-99cc-bccd65c552a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:58.470419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764f676-8b5d-44a8-99cc-bccd65c552a5 admin admin] PATH_INFO: `/v3/projects/43002fcef42c46b2aa326343c12a4126` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:58.471463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764f676-8b5d-44a8-99cc-bccd65c552a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43002fcef42c46b2aa326343c12a4126)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:58.473414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764f676-8b5d-44a8-99cc-bccd65c552a5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:58.647264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8764f676-8b5d-44a8-99cc-bccd65c552a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43002fcef42c46b2aa326343c12a4126', 'request_id': 'req-8764f676-8b5d-44a8-99cc-bccd65c552a5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:58.657947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8764f676-8b5d-44a8-99cc-bccd65c552a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43002fcef42c46b2aa326343c12a4126', 'request_id': 'req-8764f676-8b5d-44a8-99cc-bccd65c552a5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:58.731297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8764f676-8b5d-44a8-99cc-bccd65c552a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43002fcef42c46b2aa326343c12a4126', 'request_id': 'req-8764f676-8b5d-44a8-99cc-bccd65c552a5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:58.753143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8764f676-8b5d-44a8-99cc-bccd65c552a5 admin admin] The token cache is being invalidate because project 43002fcef42c46b2aa326343c12a4126 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:09:58.753644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8764f676-8b5d-44a8-99cc-bccd65c552a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:58.756292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8764f676-8b5d-44a8-99cc-bccd65c552a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:58.767331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b9472-d875-44ea-9093-60d088211892 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:58.768914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b9472-d875-44ea-9093-60d088211892 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:58.769440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b9472-d875-44ea-9093-60d088211892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:58.778425 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1597/3192] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:09:58 2022] DELETE /identity/v3/projects/43002fcef42c46b2aa326343c12a4126 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:58.825463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-738b9472-d875-44ea-9093-60d088211892 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:58.828591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-738b9472-d875-44ea-9093-60d088211892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:58.843293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8734e8b-d7bf-482d-a9e7-8599024fc543 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:58.844194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8734e8b-d7bf-482d-a9e7-8599024fc543 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:58.844835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8734e8b-d7bf-482d-a9e7-8599024fc543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:58.855598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:58.899691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a8734e8b-d7bf-482d-a9e7-8599024fc543 None None] MFA Rules not processed for user `04e5b86f94914df085b76e0d89b2230e`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:58.907976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a8734e8b-d7bf-482d-a9e7-8599024fc543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:59.016099 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1596/3193] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:59.024411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0169c8f-316f-4fd3-805a-36459d7749c8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:59.025583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0169c8f-316f-4fd3-805a-36459d7749c8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:59.026143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0169c8f-316f-4fd3-805a-36459d7749c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:59.088144 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1598/3194] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:58 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:59.088764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c0169c8f-316f-4fd3-805a-36459d7749c8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:59.090146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c0169c8f-316f-4fd3-805a-36459d7749c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:59.148513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8734e8b-d7bf-482d-a9e7-8599024fc543 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:59.149925 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1597/3195] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:09:59.167658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8734e8b-d7bf-482d-a9e7-8599024fc543 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:59.173071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65810f24-943b-4414-b24c-1987f31c77c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:59.174163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65810f24-943b-4414-b24c-1987f31c77c2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:59.174713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65810f24-943b-4414-b24c-1987f31c77c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:59.175142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65810f24-943b-4414-b24c-1987f31c77c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:59.176115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65810f24-943b-4414-b24c-1987f31c77c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:09:59.177299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65810f24-943b-4414-b24c-1987f31c77c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:59.179408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65810f24-943b-4414-b24c-1987f31c77c2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:59.202618 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1599/3196] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:09:59.203093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0169c8f-316f-4fd3-805a-36459d7749c8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:59.226769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0169c8f-316f-4fd3-805a-36459d7749c8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:59.250009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a59177-de00-4344-a3d8-eee194fbea1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:59.250713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a59177-de00-4344-a3d8-eee194fbea1a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:59.250954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a59177-de00-4344-a3d8-eee194fbea1a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:59.251419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a59177-de00-4344-a3d8-eee194fbea1a admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:59.252451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a59177-de00-4344-a3d8-eee194fbea1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:59.254840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a59177-de00-4344-a3d8-eee194fbea1a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:59.382181 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1598/3197] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:09:59 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:09:59.385541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65810f24-943b-4414-b24c-1987f31c77c2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:59.402116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65810f24-943b-4414-b24c-1987f31c77c2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:59.545326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a26fe-3a54-4cb5-b5f4-654107a1b00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:59.546106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a26fe-3a54-4cb5-b5f4-654107a1b00f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:59.552190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a26fe-3a54-4cb5-b5f4-654107a1b00f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:59.552190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a26fe-3a54-4cb5-b5f4-654107a1b00f admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:59.552190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a26fe-3a54-4cb5-b5f4-654107a1b00f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:59.552190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a26fe-3a54-4cb5-b5f4-654107a1b00f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:59.634840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a59177-de00-4344-a3d8-eee194fbea1a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:59.636922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-13a59177-de00-4344-a3d8-eee194fbea1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:09:59.648070 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1600/3198] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:09:59 2022] POST /identity/v3/users => generated 294 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:09:59.659011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a26fe-3a54-4cb5-b5f4-654107a1b00f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:59.660221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a59177-de00-4344-a3d8-eee194fbea1a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:59.671839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aace7c2-13b4-4367-a330-3f1f74f1f999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:59.672739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aace7c2-13b4-4367-a330-3f1f74f1f999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:59.673027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aace7c2-13b4-4367-a330-3f1f74f1f999 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:59.673304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aace7c2-13b4-4367-a330-3f1f74f1f999 admin admin] PATH_INFO: `/v3/users/fbd176173cd0497bb3f5c6271a538cb8` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:59.674945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aace7c2-13b4-4367-a330-3f1f74f1f999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbd176173cd0497bb3f5c6271a538cb8)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:59.678715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aace7c2-13b4-4367-a330-3f1f74f1f999 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:59.690588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a26fe-3a54-4cb5-b5f4-654107a1b00f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:59.710733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7d141-9848-440f-bfa2-c7a757168d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:59.712119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7d141-9848-440f-bfa2-c7a757168d54 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:59.712616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7d141-9848-440f-bfa2-c7a757168d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:59.713083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e7d141-9848-440f-bfa2-c7a757168d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:59.714241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7d141-9848-440f-bfa2-c7a757168d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:59.716129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e7d141-9848-440f-bfa2-c7a757168d54 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:59.735942 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1601/3199] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:09:59.740870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7d141-9848-440f-bfa2-c7a757168d54 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:59.763999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e7d141-9848-440f-bfa2-c7a757168d54 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:59.772353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36022fe9-52a4-4067-9acd-145e6598245e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:59.773084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36022fe9-52a4-4067-9acd-145e6598245e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:59.773364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36022fe9-52a4-4067-9acd-145e6598245e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:59.773724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36022fe9-52a4-4067-9acd-145e6598245e admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:59.774617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36022fe9-52a4-4067-9acd-145e6598245e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:59.781067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36022fe9-52a4-4067-9acd-145e6598245e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:59.799462 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1602/3200] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:09:59.803442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36022fe9-52a4-4067-9acd-145e6598245e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:09:59.832614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36022fe9-52a4-4067-9acd-145e6598245e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:09:59.840220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fadc032-7253-4fb0-a7ed-ad82068fdf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:09:59.841436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadc032-7253-4fb0-a7ed-ad82068fdf8f admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:59.843511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadc032-7253-4fb0-a7ed-ad82068fdf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:59.843511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fadc032-7253-4fb0-a7ed-ad82068fdf8f admin admin] PATH_INFO: `/v3/projects/03390fc09db7409da2e77f2e430866f2/users/7f7bac0c0a2d4b1488d724f7d70e34e9/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:59.868040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fadc032-7253-4fb0-a7ed-ad82068fdf8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=03390fc09db7409da2e77f2e430866f2, user_id=7f7bac0c0a2d4b1488d724f7d70e34e9, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:09:59.872141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fadc032-7253-4fb0-a7ed-ad82068fdf8f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:09:59.916259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7aace7c2-13b4-4367-a330-3f1f74f1f999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbd176173cd0497bb3f5c6271a538cb8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:59.918209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7aace7c2-13b4-4367-a330-3f1f74f1f999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbd176173cd0497bb3f5c6271a538cb8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:59.924327 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1603/3201] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:09:59 2022] PUT /identity/v3/projects/03390fc09db7409da2e77f2e430866f2/users/7f7bac0c0a2d4b1488d724f7d70e34e9/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:09:59.929145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543c39c-c18a-4fae-a54e-4a8eaa916f8e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:09:59.929438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543c39c-c18a-4fae-a54e-4a8eaa916f8e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:09:59.929888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2543c39c-c18a-4fae-a54e-4a8eaa916f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:09:59.934995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7aace7c2-13b4-4367-a330-3f1f74f1f999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbd176173cd0497bb3f5c6271a538cb8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:09:59.952788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2543c39c-c18a-4fae-a54e-4a8eaa916f8e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:09:59.958383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2543c39c-c18a-4fae-a54e-4a8eaa916f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:00.008803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7aace7c2-13b4-4367-a330-3f1f74f1f999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbd176173cd0497bb3f5c6271a538cb8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:00.009520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7aace7c2-13b4-4367-a330-3f1f74f1f999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbd176173cd0497bb3f5c6271a538cb8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:00.039322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:00.050925 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1599/3202] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:09:59 2022] DELETE /identity/v3/users/fbd176173cd0497bb3f5c6271a538cb8 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:00.056321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79d9bc-c85e-47ae-a029-c642175b3b32 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:00.085473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79d9bc-c85e-47ae-a029-c642175b3b32 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:00.085771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79d9bc-c85e-47ae-a029-c642175b3b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:00.117057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-be79d9bc-c85e-47ae-a029-c642175b3b32 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:00.118780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-be79d9bc-c85e-47ae-a029-c642175b3b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:00.119482 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1604/3203] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:10:00.160575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543c39c-c18a-4fae-a54e-4a8eaa916f8e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:00.163359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2543c39c-c18a-4fae-a54e-4a8eaa916f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:00.230521 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1600/3204] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:00.280517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79d9bc-c85e-47ae-a029-c642175b3b32 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:00.294083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2543c39c-c18a-4fae-a54e-4a8eaa916f8e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:00.402641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e624d8e-1ccc-4c89-b5d7-f46b427fa68c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:00.403907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e624d8e-1ccc-4c89-b5d7-f46b427fa68c service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:00.403907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e624d8e-1ccc-4c89-b5d7-f46b427fa68c service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:00.404368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e624d8e-1ccc-4c89-b5d7-f46b427fa68c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:00.414591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79d9bc-c85e-47ae-a029-c642175b3b32 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:00.419265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e624d8e-1ccc-4c89-b5d7-f46b427fa68c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:00.422126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ca92f-11a7-468b-8e1e-4f0f9d3ea045 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:00.422625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e624d8e-1ccc-4c89-b5d7-f46b427fa68c service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:00.422814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ca92f-11a7-468b-8e1e-4f0f9d3ea045 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:00.423086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ca92f-11a7-468b-8e1e-4f0f9d3ea045 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:00.424018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ca92f-11a7-468b-8e1e-4f0f9d3ea045 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:00.438371 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1605/3205] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:00.445070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ca92f-11a7-468b-8e1e-4f0f9d3ea045 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:00.449402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ca92f-11a7-468b-8e1e-4f0f9d3ea045 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:00.466834 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1601/3206] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:00.683464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c9760-2d9d-4534-ad0f-c17f554ed939 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:00.683763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c9760-2d9d-4534-ad0f-c17f554ed939 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:00.684038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72c9760-2d9d-4534-ad0f-c17f554ed939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:00.704676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f72c9760-2d9d-4534-ad0f-c17f554ed939 None None] MFA Rules not processed for user `f5df3bc2a59b4d99b81932deac1ece64`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:00.706016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f72c9760-2d9d-4534-ad0f-c17f554ed939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:00.801280 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1606/3207] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:00.847617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e4a60-7773-4f35-a8bd-6a77b32afa58 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:00.847915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e4a60-7773-4f35-a8bd-6a77b32afa58 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:00.848202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e4a60-7773-4f35-a8bd-6a77b32afa58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:00.888728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:00.899344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7f6e4a60-7773-4f35-a8bd-6a77b32afa58 None None] MFA Rules not processed for user `4112534978ba4622b0fbbb9a92791876`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:00.900719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7f6e4a60-7773-4f35-a8bd-6a77b32afa58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:01.033511 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1602/3208] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:00 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:01.044287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a9789-cbb4-4003-92af-a5cc45c54923 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:01.044578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a9789-cbb4-4003-92af-a5cc45c54923 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:01.044812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a9789-cbb4-4003-92af-a5cc45c54923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:01.089564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b08a9789-cbb4-4003-92af-a5cc45c54923 None None] MFA Rules not processed for user `4112534978ba4622b0fbbb9a92791876`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:01.090863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b08a9789-cbb4-4003-92af-a5cc45c54923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:01.112137 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1607/3209] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:01 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:01.245776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e4a60-7773-4f35-a8bd-6a77b32afa58 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:01.247260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7f6e4a60-7773-4f35-a8bd-6a77b32afa58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:01.266150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e4a60-7773-4f35-a8bd-6a77b32afa58 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:01.274411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d069e-845a-4793-b8b3-0c2ea57b8a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:01.275307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d069e-845a-4793-b8b3-0c2ea57b8a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:01.275628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d069e-845a-4793-b8b3-0c2ea57b8a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:01.275872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d069e-845a-4793-b8b3-0c2ea57b8a94 admin admin] PATH_INFO: `/v3/projects/1f4d318ab1144e5b96abd0654f9bf1b1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:01.278420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d069e-845a-4793-b8b3-0c2ea57b8a94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f4d318ab1144e5b96abd0654f9bf1b1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:01.280612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d069e-845a-4793-b8b3-0c2ea57b8a94 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:01.442273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-811d069e-845a-4793-b8b3-0c2ea57b8a94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f4d318ab1144e5b96abd0654f9bf1b1', 'request_id': 'req-811d069e-845a-4793-b8b3-0c2ea57b8a94'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:01.453057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-811d069e-845a-4793-b8b3-0c2ea57b8a94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f4d318ab1144e5b96abd0654f9bf1b1', 'request_id': 'req-811d069e-845a-4793-b8b3-0c2ea57b8a94'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:01.536784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-811d069e-845a-4793-b8b3-0c2ea57b8a94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f4d318ab1144e5b96abd0654f9bf1b1', 'request_id': 'req-811d069e-845a-4793-b8b3-0c2ea57b8a94'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:01.561730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-811d069e-845a-4793-b8b3-0c2ea57b8a94 admin admin] The token cache is being invalidate because project 1f4d318ab1144e5b96abd0654f9bf1b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:10:01.561730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-811d069e-845a-4793-b8b3-0c2ea57b8a94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:01.562643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-811d069e-845a-4793-b8b3-0c2ea57b8a94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:01.565985 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1603/3210] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:10:01 2022] DELETE /identity/v3/projects/1f4d318ab1144e5b96abd0654f9bf1b1 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:01.568504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a9789-cbb4-4003-92af-a5cc45c54923 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:01.588932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a9789-cbb4-4003-92af-a5cc45c54923 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:01.664765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727bbbcb-eb7f-4316-be00-d4dd9ae68c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:01.665662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bbbcb-eb7f-4316-be00-d4dd9ae68c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:01.665998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bbbcb-eb7f-4316-be00-d4dd9ae68c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:01.671873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bbbcb-eb7f-4316-be00-d4dd9ae68c79 admin admin] PATH_INFO: `/v3/users/7574038f38fc402bb7097beb4ac93426` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:01.673652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727bbbcb-eb7f-4316-be00-d4dd9ae68c79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7574038f38fc402bb7097beb4ac93426)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:01.676712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727bbbcb-eb7f-4316-be00-d4dd9ae68c79 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:01.924262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-727bbbcb-eb7f-4316-be00-d4dd9ae68c79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7574038f38fc402bb7097beb4ac93426'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:01.933589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-727bbbcb-eb7f-4316-be00-d4dd9ae68c79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7574038f38fc402bb7097beb4ac93426'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:01.934759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-727bbbcb-eb7f-4316-be00-d4dd9ae68c79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7574038f38fc402bb7097beb4ac93426'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:01.935389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-727bbbcb-eb7f-4316-be00-d4dd9ae68c79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7574038f38fc402bb7097beb4ac93426'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:02.039579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:02.066291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-727bbbcb-eb7f-4316-be00-d4dd9ae68c79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7574038f38fc402bb7097beb4ac93426'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:02.097622 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1608/3211] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:10:01 2022] DELETE /identity/v3/users/7574038f38fc402bb7097beb4ac93426 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:02.123428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:02.421085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d069e-845a-4793-b8b3-0c2ea57b8a94 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:02.422653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-811d069e-845a-4793-b8b3-0c2ea57b8a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:02.450188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d069e-845a-4793-b8b3-0c2ea57b8a94 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:02.542444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e649f0-1b9a-4497-9dd2-ff0d1daa5659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:02.542713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727bbbcb-eb7f-4316-be00-d4dd9ae68c79 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:02.543458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e649f0-1b9a-4497-9dd2-ff0d1daa5659 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:02.544114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-727bbbcb-eb7f-4316-be00-d4dd9ae68c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:02.544299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e649f0-1b9a-4497-9dd2-ff0d1daa5659 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:02.545325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e649f0-1b9a-4497-9dd2-ff0d1daa5659 admin admin] PATH_INFO: `/v3/projects/31a5f1cb172e42689b6528e97b771c40` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:02.547019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e649f0-1b9a-4497-9dd2-ff0d1daa5659 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31a5f1cb172e42689b6528e97b771c40)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:02.549304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e649f0-1b9a-4497-9dd2-ff0d1daa5659 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:02.640568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727bbbcb-eb7f-4316-be00-d4dd9ae68c79 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:02.658012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d5e649f0-1b9a-4497-9dd2-ff0d1daa5659 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31a5f1cb172e42689b6528e97b771c40', 'request_id': 'req-d5e649f0-1b9a-4497-9dd2-ff0d1daa5659'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:02.665826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d5e649f0-1b9a-4497-9dd2-ff0d1daa5659 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31a5f1cb172e42689b6528e97b771c40', 'request_id': 'req-d5e649f0-1b9a-4497-9dd2-ff0d1daa5659'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:02.700326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485df02b-9fc8-4512-8381-47131e042ce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:02.700986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485df02b-9fc8-4512-8381-47131e042ce2 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:02.701273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485df02b-9fc8-4512-8381-47131e042ce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:02.702215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485df02b-9fc8-4512-8381-47131e042ce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:02.705074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-485df02b-9fc8-4512-8381-47131e042ce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:02.715288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485df02b-9fc8-4512-8381-47131e042ce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:02.717046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485df02b-9fc8-4512-8381-47131e042ce2 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:02.727325 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1609/3212] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:10:02.745630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d5e649f0-1b9a-4497-9dd2-ff0d1daa5659 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31a5f1cb172e42689b6528e97b771c40', 'request_id': 'req-d5e649f0-1b9a-4497-9dd2-ff0d1daa5659'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:02.763706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d5e649f0-1b9a-4497-9dd2-ff0d1daa5659 admin admin] The token cache is being invalidate because project 31a5f1cb172e42689b6528e97b771c40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:10:02.764017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d5e649f0-1b9a-4497-9dd2-ff0d1daa5659 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:02.764907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d5e649f0-1b9a-4497-9dd2-ff0d1daa5659 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:02.766746 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1604/3213] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:10:02 2022] DELETE /identity/v3/projects/31a5f1cb172e42689b6528e97b771c40 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:02.772470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685f59b-b7f5-4279-9574-001c021aac64 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:02.774594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685f59b-b7f5-4279-9574-001c021aac64 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:02.775018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685f59b-b7f5-4279-9574-001c021aac64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:02.799151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8685f59b-b7f5-4279-9574-001c021aac64 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:02.800817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8685f59b-b7f5-4279-9574-001c021aac64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:02.815411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:02.879422 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1610/3214] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:02.888696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732ad20-35e3-42ec-a6a0-225432f877f1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:02.889215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732ad20-35e3-42ec-a6a0-225432f877f1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:02.889321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9732ad20-35e3-42ec-a6a0-225432f877f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:02.915763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9732ad20-35e3-42ec-a6a0-225432f877f1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:02.918232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9732ad20-35e3-42ec-a6a0-225432f877f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:02.937864 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1605/3215] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:02.940791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685f59b-b7f5-4279-9574-001c021aac64 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:02.951455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685f59b-b7f5-4279-9574-001c021aac64 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:02.956480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f43b1f-9c8e-4d88-bb45-ee5890d2ec23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:02.959494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f43b1f-9c8e-4d88-bb45-ee5890d2ec23 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:02.959494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f43b1f-9c8e-4d88-bb45-ee5890d2ec23 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:02.959494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f43b1f-9c8e-4d88-bb45-ee5890d2ec23 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:02.959494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f43b1f-9c8e-4d88-bb45-ee5890d2ec23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:10:02.959494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f43b1f-9c8e-4d88-bb45-ee5890d2ec23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:02.960491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f43b1f-9c8e-4d88-bb45-ee5890d2ec23 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:02.973744 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1611/3216] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:10:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:10:02.978963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732ad20-35e3-42ec-a6a0-225432f877f1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:03.012716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9732ad20-35e3-42ec-a6a0-225432f877f1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:03.023561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e9dc6-f09c-4fed-a09b-9887ae237ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:03.027900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e9dc6-f09c-4fed-a09b-9887ae237ba7 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:03.027900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e9dc6-f09c-4fed-a09b-9887ae237ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:03.027900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8e9dc6-f09c-4fed-a09b-9887ae237ba7 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:03.027900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e9dc6-f09c-4fed-a09b-9887ae237ba7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:03.028430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8e9dc6-f09c-4fed-a09b-9887ae237ba7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:03.048993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f43b1f-9c8e-4d88-bb45-ee5890d2ec23 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:03.057889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-05f43b1f-9c8e-4d88-bb45-ee5890d2ec23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:03.147584 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1606/3217] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:10:02 2022] POST /identity/v3/projects => generated 393 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:10:03.151716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e9dc6-f09c-4fed-a09b-9887ae237ba7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:03.188787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:03.230853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f43b1f-9c8e-4d88-bb45-ee5890d2ec23 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:03.277381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8e9dc6-f09c-4fed-a09b-9887ae237ba7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:03.286128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0641a-de8b-4f00-bacb-0cdd9ceb06f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:03.287048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0641a-de8b-4f00-bacb-0cdd9ceb06f0 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:03.287499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0641a-de8b-4f00-bacb-0cdd9ceb06f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:03.290295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0641a-de8b-4f00-bacb-0cdd9ceb06f0 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:03.290533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0641a-de8b-4f00-bacb-0cdd9ceb06f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:03.315458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d0641a-de8b-4f00-bacb-0cdd9ceb06f0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:03.401062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d9371-4d50-46cf-bd14-338dc35c5da3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:03.404179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d9371-4d50-46cf-bd14-338dc35c5da3 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:03.405115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d9371-4d50-46cf-bd14-338dc35c5da3 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:03.405533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72d9371-4d50-46cf-bd14-338dc35c5da3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:03.414741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a72d9371-4d50-46cf-bd14-338dc35c5da3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:03.495000 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1607/3218] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:10:03 2022] POST /identity/v3/users => generated 297 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:10:03.499417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0641a-de8b-4f00-bacb-0cdd9ceb06f0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:03.512916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d9371-4d50-46cf-bd14-338dc35c5da3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:03.515315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72d9371-4d50-46cf-bd14-338dc35c5da3 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:03.523703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d0641a-de8b-4f00-bacb-0cdd9ceb06f0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:03.528241 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1612/3219] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:03 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:03.549483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bd369-6917-48cc-89d9-daec6a2fb6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:03.550291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd369-6917-48cc-89d9-daec6a2fb6eb admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:03.550583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd369-6917-48cc-89d9-daec6a2fb6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:03.550870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bd369-6917-48cc-89d9-daec6a2fb6eb admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:03.551624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bd369-6917-48cc-89d9-daec6a2fb6eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:03.557129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bd369-6917-48cc-89d9-daec6a2fb6eb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:03.587339 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1608/3220] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:10:03.595166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d9371-4d50-46cf-bd14-338dc35c5da3 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:03.609018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72d9371-4d50-46cf-bd14-338dc35c5da3 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:03.622964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b7131-5144-4d4b-9830-b1670e449403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:03.623853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b7131-5144-4d4b-9830-b1670e449403 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:03.624208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b7131-5144-4d4b-9830-b1670e449403 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:03.624561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b7131-5144-4d4b-9830-b1670e449403 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:03.627053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b7131-5144-4d4b-9830-b1670e449403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:03.628998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b7131-5144-4d4b-9830-b1670e449403 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:03.661947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bd369-6917-48cc-89d9-daec6a2fb6eb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:03.663702 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1613/3221] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:03 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:10:03.677291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bd369-6917-48cc-89d9-daec6a2fb6eb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:03.695391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2d954-41bc-49f0-a857-10ad73758582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:03.700986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2d954-41bc-49f0-a857-10ad73758582 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:03.703676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2d954-41bc-49f0-a857-10ad73758582 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:03.704018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2d954-41bc-49f0-a857-10ad73758582 admin admin] PATH_INFO: `/v3/projects/66c46bc156374e6fa5c0e6b514f110c5/users/66fe52f7f96e41a5840ec2d4da6a6a1c/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:03.751610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2d954-41bc-49f0-a857-10ad73758582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66c46bc156374e6fa5c0e6b514f110c5, user_id=66fe52f7f96e41a5840ec2d4da6a6a1c, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:03.755618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2d954-41bc-49f0-a857-10ad73758582 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:03.886677 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1609/3222] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:10:03 2022] PUT /identity/v3/projects/66c46bc156374e6fa5c0e6b514f110c5/users/66fe52f7f96e41a5840ec2d4da6a6a1c/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:03.897752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f89e39-513f-4ef6-a56e-f899e02ddc93 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:03.898405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f89e39-513f-4ef6-a56e-f899e02ddc93 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:03.898906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f89e39-513f-4ef6-a56e-f899e02ddc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:03.932414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f3f89e39-513f-4ef6-a56e-f899e02ddc93 None None] MFA Rules not processed for user `66fe52f7f96e41a5840ec2d4da6a6a1c`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:03.934379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f3f89e39-513f-4ef6-a56e-f899e02ddc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:04.027823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2d954-41bc-49f0-a857-10ad73758582 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:04.031350 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1614/3223] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:04.192023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2d954-41bc-49f0-a857-10ad73758582 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:04.200709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089dc9a-78da-4d9e-a468-06152e6013cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:04.202837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089dc9a-78da-4d9e-a468-06152e6013cc admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:04.203156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089dc9a-78da-4d9e-a468-06152e6013cc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:04.204566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089dc9a-78da-4d9e-a468-06152e6013cc admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:04.207692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089dc9a-78da-4d9e-a468-06152e6013cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:04.215124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c089dc9a-78da-4d9e-a468-06152e6013cc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:04.335443 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1610/3224] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:10:04 2022] POST /identity/v3/projects => generated 395 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:10:04.339673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f89e39-513f-4ef6-a56e-f899e02ddc93 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:04.353089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f89e39-513f-4ef6-a56e-f899e02ddc93 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:04.484978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:04.517032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e8aa7-7943-4821-ac5c-b9a4ef5f448d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:04.518076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e8aa7-7943-4821-ac5c-b9a4ef5f448d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:04.518231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e8aa7-7943-4821-ac5c-b9a4ef5f448d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:04.518682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e8aa7-7943-4821-ac5c-b9a4ef5f448d admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:04.519689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e8aa7-7943-4821-ac5c-b9a4ef5f448d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:04.524571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e8aa7-7943-4821-ac5c-b9a4ef5f448d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:04.643807 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1615/3225] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:10:04 2022] POST /identity/v3/users => generated 304 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:10:04.645543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089dc9a-78da-4d9e-a468-06152e6013cc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:04.666050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c089dc9a-78da-4d9e-a468-06152e6013cc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:04.687963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8b590-b071-4e46-bcf8-f5d1a0420a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:04.691220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8b590-b071-4e46-bcf8-f5d1a0420a97 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:04.692985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8b590-b071-4e46-bcf8-f5d1a0420a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:04.693331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d8b590-b071-4e46-bcf8-f5d1a0420a97 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:04.697999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8b590-b071-4e46-bcf8-f5d1a0420a97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:04.705269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d8b590-b071-4e46-bcf8-f5d1a0420a97 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:04.752821 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1611/3226] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:10:04.757133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e8aa7-7943-4821-ac5c-b9a4ef5f448d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:04.773422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e8aa7-7943-4821-ac5c-b9a4ef5f448d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:04.787176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f6d4f-d5e5-475a-9419-b87a61cd966a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:04.788178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f6d4f-d5e5-475a-9419-b87a61cd966a admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:04.788535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f6d4f-d5e5-475a-9419-b87a61cd966a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:04.788902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f6d4f-d5e5-475a-9419-b87a61cd966a admin admin] PATH_INFO: `/v3/projects/9a1c715fce5a4056a2152aa9af5190a6/users/482c92ce7acf4c59ac1a08ba52c56617/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:04.828397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8f6d4f-d5e5-475a-9419-b87a61cd966a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1c715fce5a4056a2152aa9af5190a6, user_id=482c92ce7acf4c59ac1a08ba52c56617, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:04.831315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8f6d4f-d5e5-475a-9419-b87a61cd966a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:04.973986 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1616/3227] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:10:04 2022] PUT /identity/v3/projects/9a1c715fce5a4056a2152aa9af5190a6/users/482c92ce7acf4c59ac1a08ba52c56617/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:04.975009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8b590-b071-4e46-bcf8-f5d1a0420a97 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:05.011136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d8b590-b071-4e46-bcf8-f5d1a0420a97 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:05.151131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:05.213679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf2d30-9ea2-43bd-b9e4-f01def2ed10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:05.216534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf2d30-9ea2-43bd-b9e4-f01def2ed10b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:05.216534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf2d30-9ea2-43bd-b9e4-f01def2ed10b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:05.216534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf2d30-9ea2-43bd-b9e4-f01def2ed10b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:05.219196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf2d30-9ea2-43bd-b9e4-f01def2ed10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:05.220887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf2d30-9ea2-43bd-b9e4-f01def2ed10b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:05.246597 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1612/3228] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:10:05.251214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f6d4f-d5e5-475a-9419-b87a61cd966a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:05.269667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8f6d4f-d5e5-475a-9419-b87a61cd966a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:05.304324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22272056-63bb-4c42-aee5-26c34bc637db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:05.305258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22272056-63bb-4c42-aee5-26c34bc637db admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:05.305542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22272056-63bb-4c42-aee5-26c34bc637db admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:05.305823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22272056-63bb-4c42-aee5-26c34bc637db admin admin] PATH_INFO: `/v3/projects/9a1c715fce5a4056a2152aa9af5190a6/users/482c92ce7acf4c59ac1a08ba52c56617/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:05.310954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22272056-63bb-4c42-aee5-26c34bc637db admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1c715fce5a4056a2152aa9af5190a6, user_id=482c92ce7acf4c59ac1a08ba52c56617, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:05.320503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22272056-63bb-4c42-aee5-26c34bc637db admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:05.373781 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1617/3229] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:10:05 2022] PUT /identity/v3/projects/9a1c715fce5a4056a2152aa9af5190a6/users/482c92ce7acf4c59ac1a08ba52c56617/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:05.383159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267ffc4-e634-4628-b159-a37d82ad7f00 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:05.383431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267ffc4-e634-4628-b159-a37d82ad7f00 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:05.383900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4267ffc4-e634-4628-b159-a37d82ad7f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:05.441052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4267ffc4-e634-4628-b159-a37d82ad7f00 None None] MFA Rules not processed for user `482c92ce7acf4c59ac1a08ba52c56617`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:05.441701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4267ffc4-e634-4628-b159-a37d82ad7f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:05.596441 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1613/3230] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:06.073431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22272056-63bb-4c42-aee5-26c34bc637db admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:06.076088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-22272056-63bb-4c42-aee5-26c34bc637db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:06.139646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:06.147874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267ffc4-e634-4628-b159-a37d82ad7f00 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:06.351275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4267ffc4-e634-4628-b159-a37d82ad7f00 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:06.367469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac9248-7838-46b9-af9b-31ec17f13cc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:06.368146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac9248-7838-46b9-af9b-31ec17f13cc4 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:06.374104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac9248-7838-46b9-af9b-31ec17f13cc4 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:06.374792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac9248-7838-46b9-af9b-31ec17f13cc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:06.376583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22272056-63bb-4c42-aee5-26c34bc637db admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:06.392900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cfb600-816e-4ee9-8b40-683f18114ddc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:06.394065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfb600-816e-4ee9-8b40-683f18114ddc service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:06.395332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfb600-816e-4ee9-8b40-683f18114ddc service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:06.396816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cfb600-816e-4ee9-8b40-683f18114ddc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:06.401103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-19cfb600-816e-4ee9-8b40-683f18114ddc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:06.404003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac9248-7838-46b9-af9b-31ec17f13cc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:06.408077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfac9248-7838-46b9-af9b-31ec17f13cc4 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:06.439743 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1614/3231] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:10:06.520915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cfb600-816e-4ee9-8b40-683f18114ddc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:06.524804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cfb600-816e-4ee9-8b40-683f18114ddc service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:06.568917 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1618/3232] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:10.467806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac9248-7838-46b9-af9b-31ec17f13cc4 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:10.483070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac9248-7838-46b9-af9b-31ec17f13cc4 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:10.490693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e35cd2-78ec-4ddb-9da6-39b889b0858f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:10.491559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e35cd2-78ec-4ddb-9da6-39b889b0858f service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:10.491945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e35cd2-78ec-4ddb-9da6-39b889b0858f service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:10.492319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e35cd2-78ec-4ddb-9da6-39b889b0858f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:10.584780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:10.603470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e35cd2-78ec-4ddb-9da6-39b889b0858f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:10.605383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e35cd2-78ec-4ddb-9da6-39b889b0858f service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:10.626886 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1615/3233] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:10:10.662009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cfb600-816e-4ee9-8b40-683f18114ddc service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:10.795459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cfb600-816e-4ee9-8b40-683f18114ddc service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:10.801901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b855232-ee1e-4149-984f-422a7da83ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:10.803401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b855232-ee1e-4149-984f-422a7da83ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:10.803922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b855232-ee1e-4149-984f-422a7da83ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:10.804614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b855232-ee1e-4149-984f-422a7da83ec8 admin admin] PATH_INFO: `/v3/users/66fe52f7f96e41a5840ec2d4da6a6a1c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:10.806280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b855232-ee1e-4149-984f-422a7da83ec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66fe52f7f96e41a5840ec2d4da6a6a1c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:10.808764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b855232-ee1e-4149-984f-422a7da83ec8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:11.043810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:11.051147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8b855232-ee1e-4149-984f-422a7da83ec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66fe52f7f96e41a5840ec2d4da6a6a1c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:11.074135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8b855232-ee1e-4149-984f-422a7da83ec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66fe52f7f96e41a5840ec2d4da6a6a1c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:11.076727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8b855232-ee1e-4149-984f-422a7da83ec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66fe52f7f96e41a5840ec2d4da6a6a1c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:11.077390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8b855232-ee1e-4149-984f-422a7da83ec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66fe52f7f96e41a5840ec2d4da6a6a1c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:11.157163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8b855232-ee1e-4149-984f-422a7da83ec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66fe52f7f96e41a5840ec2d4da6a6a1c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:11.213273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2a4f9-82cd-475c-92d6-9ed1084200d7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:11.213560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2a4f9-82cd-475c-92d6-9ed1084200d7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:11.213860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2a4f9-82cd-475c-92d6-9ed1084200d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:11.218119 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1619/3234] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:10:10 2022] DELETE /identity/v3/users/66fe52f7f96e41a5840ec2d4da6a6a1c => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:11.258802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e1e2a4f9-82cd-475c-92d6-9ed1084200d7 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:11.260055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e1e2a4f9-82cd-475c-92d6-9ed1084200d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:11.375208 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1616/3235] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:11.419126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b855232-ee1e-4149-984f-422a7da83ec8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:11.421203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8b855232-ee1e-4149-984f-422a7da83ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:11.547976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b855232-ee1e-4149-984f-422a7da83ec8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:11.553626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210ce1f-52e7-4e25-a9e9-2a3438519c1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:11.554619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210ce1f-52e7-4e25-a9e9-2a3438519c1c service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:11.556912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210ce1f-52e7-4e25-a9e9-2a3438519c1c service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:11.565395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210ce1f-52e7-4e25-a9e9-2a3438519c1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:11.589992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210ce1f-52e7-4e25-a9e9-2a3438519c1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:11.591510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a210ce1f-52e7-4e25-a9e9-2a3438519c1c service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:11.615723 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1620/3236] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:12.663665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae10b92-894c-409d-ad57-c7021521965f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:12.663665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae10b92-894c-409d-ad57-c7021521965f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:12.663665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae10b92-894c-409d-ad57-c7021521965f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:12.706866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2ae10b92-894c-409d-ad57-c7021521965f None None] MFA Rules not processed for user `7f7bac0c0a2d4b1488d724f7d70e34e9`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:12.708523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2ae10b92-894c-409d-ad57-c7021521965f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:12.828464 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1617/3237] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:12.868945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210ce1f-52e7-4e25-a9e9-2a3438519c1c service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:12.871530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a210ce1f-52e7-4e25-a9e9-2a3438519c1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:12.886804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a210ce1f-52e7-4e25-a9e9-2a3438519c1c service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:12.892174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d730ff7-26b0-493b-9a91-6cf0decc30df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:12.893053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d730ff7-26b0-493b-9a91-6cf0decc30df admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:12.893333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d730ff7-26b0-493b-9a91-6cf0decc30df admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:12.893909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d730ff7-26b0-493b-9a91-6cf0decc30df admin admin] PATH_INFO: `/v3/projects/66c46bc156374e6fa5c0e6b514f110c5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:12.895153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d730ff7-26b0-493b-9a91-6cf0decc30df admin admin] RBAC: Authorizing `identity:delete_project(project_id=66c46bc156374e6fa5c0e6b514f110c5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:12.897350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d730ff7-26b0-493b-9a91-6cf0decc30df admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:13.068751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8d730ff7-26b0-493b-9a91-6cf0decc30df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66c46bc156374e6fa5c0e6b514f110c5', 'request_id': 'req-8d730ff7-26b0-493b-9a91-6cf0decc30df'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:13.091092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8d730ff7-26b0-493b-9a91-6cf0decc30df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66c46bc156374e6fa5c0e6b514f110c5', 'request_id': 'req-8d730ff7-26b0-493b-9a91-6cf0decc30df'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:13.102157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae10b92-894c-409d-ad57-c7021521965f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:13.103623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2ae10b92-894c-409d-ad57-c7021521965f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:13.249135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8d730ff7-26b0-493b-9a91-6cf0decc30df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66c46bc156374e6fa5c0e6b514f110c5', 'request_id': 'req-8d730ff7-26b0-493b-9a91-6cf0decc30df'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:13.271505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae10b92-894c-409d-ad57-c7021521965f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:13.294217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:13.330882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8d730ff7-26b0-493b-9a91-6cf0decc30df admin admin] The token cache is being invalidate because project 66c46bc156374e6fa5c0e6b514f110c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:10:13.333670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8d730ff7-26b0-493b-9a91-6cf0decc30df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:13.337150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8d730ff7-26b0-493b-9a91-6cf0decc30df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:13.347486 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1621/3238] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:10:12 2022] DELETE /identity/v3/projects/66c46bc156374e6fa5c0e6b514f110c5 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:13.356179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d730ff7-26b0-493b-9a91-6cf0decc30df admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:13.359926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8d730ff7-26b0-493b-9a91-6cf0decc30df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:13.416271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438be56-3e52-4514-9258-170b73cf6a3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:13.416998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438be56-3e52-4514-9258-170b73cf6a3e service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:13.417473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438be56-3e52-4514-9258-170b73cf6a3e service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:13.417950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438be56-3e52-4514-9258-170b73cf6a3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:13.432063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c438be56-3e52-4514-9258-170b73cf6a3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:13.478585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:13.515535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d730ff7-26b0-493b-9a91-6cf0decc30df admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:13.527625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510a7c9-997c-44bd-adcb-6bb6c3f84bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:13.531737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510a7c9-997c-44bd-adcb-6bb6c3f84bc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:13.531737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510a7c9-997c-44bd-adcb-6bb6c3f84bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:13.531737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510a7c9-997c-44bd-adcb-6bb6c3f84bc4 admin admin] PATH_INFO: `/v3/users/482c92ce7acf4c59ac1a08ba52c56617` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:13.531737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510a7c9-997c-44bd-adcb-6bb6c3f84bc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=482c92ce7acf4c59ac1a08ba52c56617)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:13.533726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510a7c9-997c-44bd-adcb-6bb6c3f84bc4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:13.555896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c438be56-3e52-4514-9258-170b73cf6a3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:13.561307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c438be56-3e52-4514-9258-170b73cf6a3e service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:13.578799 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1618/3239] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:10:13.777453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2510a7c9-997c-44bd-adcb-6bb6c3f84bc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '482c92ce7acf4c59ac1a08ba52c56617'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:13.800296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2510a7c9-997c-44bd-adcb-6bb6c3f84bc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '482c92ce7acf4c59ac1a08ba52c56617'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:13.802014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2510a7c9-997c-44bd-adcb-6bb6c3f84bc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '482c92ce7acf4c59ac1a08ba52c56617'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:13.804908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2510a7c9-997c-44bd-adcb-6bb6c3f84bc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '482c92ce7acf4c59ac1a08ba52c56617'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:13.941792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2510a7c9-997c-44bd-adcb-6bb6c3f84bc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '482c92ce7acf4c59ac1a08ba52c56617'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:13.971158 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1622/3240] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:10:13 2022] DELETE /identity/v3/users/482c92ce7acf4c59ac1a08ba52c56617 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:14.542638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438be56-3e52-4514-9258-170b73cf6a3e service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:14.573049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438be56-3e52-4514-9258-170b73cf6a3e service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:14.690362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83a6a7-3280-4e41-a85b-78a4afc57d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:14.691417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83a6a7-3280-4e41-a85b-78a4afc57d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:14.691762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83a6a7-3280-4e41-a85b-78a4afc57d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:14.692050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83a6a7-3280-4e41-a85b-78a4afc57d8e admin admin] PATH_INFO: `/v3/projects/9a1c715fce5a4056a2152aa9af5190a6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:14.695790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83a6a7-3280-4e41-a85b-78a4afc57d8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a1c715fce5a4056a2152aa9af5190a6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:14.701794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83a6a7-3280-4e41-a85b-78a4afc57d8e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:14.900718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cc83a6a7-3280-4e41-a85b-78a4afc57d8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a1c715fce5a4056a2152aa9af5190a6', 'request_id': 'req-cc83a6a7-3280-4e41-a85b-78a4afc57d8e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:14.910975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:14.914087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cc83a6a7-3280-4e41-a85b-78a4afc57d8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a1c715fce5a4056a2152aa9af5190a6', 'request_id': 'req-cc83a6a7-3280-4e41-a85b-78a4afc57d8e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:15.025966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cc83a6a7-3280-4e41-a85b-78a4afc57d8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a1c715fce5a4056a2152aa9af5190a6', 'request_id': 'req-cc83a6a7-3280-4e41-a85b-78a4afc57d8e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:15.040001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cc83a6a7-3280-4e41-a85b-78a4afc57d8e admin admin] The token cache is being invalidate because project 9a1c715fce5a4056a2152aa9af5190a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:10:15.040231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cc83a6a7-3280-4e41-a85b-78a4afc57d8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:15.040923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cc83a6a7-3280-4e41-a85b-78a4afc57d8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:15.042581 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1619/3241] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:10:14 2022] DELETE /identity/v3/projects/9a1c715fce5a4056a2152aa9af5190a6 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:15.047928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d32e72-00cb-4e3c-a0c4-7f7c6d794e4f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:15.048444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d32e72-00cb-4e3c-a0c4-7f7c6d794e4f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:15.048663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d32e72-00cb-4e3c-a0c4-7f7c6d794e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:15.088524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-91d32e72-00cb-4e3c-a0c4-7f7c6d794e4f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:15.090374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-91d32e72-00cb-4e3c-a0c4-7f7c6d794e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:15.220140 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1623/3242] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:15.232267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6d11d-0af0-4b2c-9aaa-e966d15ca9cd None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:15.232592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6d11d-0af0-4b2c-9aaa-e966d15ca9cd None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:15.232878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c6d11d-0af0-4b2c-9aaa-e966d15ca9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:15.283093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a9c6d11d-0af0-4b2c-9aaa-e966d15ca9cd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:15.285687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a9c6d11d-0af0-4b2c-9aaa-e966d15ca9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:15.296717 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1620/3243] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:15.300991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d32e72-00cb-4e3c-a0c4-7f7c6d794e4f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:15.312589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d32e72-00cb-4e3c-a0c4-7f7c6d794e4f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:15.331012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb209548-c775-48d1-98b0-546b9c0ffe16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:15.331558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb209548-c775-48d1-98b0-546b9c0ffe16 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:15.331904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb209548-c775-48d1-98b0-546b9c0ffe16 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:15.332497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb209548-c775-48d1-98b0-546b9c0ffe16 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:15.333419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb209548-c775-48d1-98b0-546b9c0ffe16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:10:15.333953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb209548-c775-48d1-98b0-546b9c0ffe16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:15.339200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb209548-c775-48d1-98b0-546b9c0ffe16 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:15.369500 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1624/3244] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:10:15.374302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6d11d-0af0-4b2c-9aaa-e966d15ca9cd None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:15.387601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c6d11d-0af0-4b2c-9aaa-e966d15ca9cd None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:15.396837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1cb6e-c229-45f3-94b1-abaf38899bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:15.396837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1cb6e-c229-45f3-94b1-abaf38899bbc admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:15.396837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1cb6e-c229-45f3-94b1-abaf38899bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:15.396837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad1cb6e-c229-45f3-94b1-abaf38899bbc admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:15.397354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1cb6e-c229-45f3-94b1-abaf38899bbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:15.400351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad1cb6e-c229-45f3-94b1-abaf38899bbc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:15.539937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb209548-c775-48d1-98b0-546b9c0ffe16 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:15.562144 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1621/3245] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:10:15 2022] POST /identity/v3/projects => generated 401 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:10:15.571611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb209548-c775-48d1-98b0-546b9c0ffe16 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:15.636550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:15.648720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3555d40-57f5-420a-a40c-b69880775213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:15.649608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3555d40-57f5-420a-a40c-b69880775213 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:15.650035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3555d40-57f5-420a-a40c-b69880775213 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:15.650423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3555d40-57f5-420a-a40c-b69880775213 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:15.651956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3555d40-57f5-420a-a40c-b69880775213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:15.654113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3555d40-57f5-420a-a40c-b69880775213 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:15.771506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1cb6e-c229-45f3-94b1-abaf38899bbc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:15.780672 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1625/3246] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:10:15 2022] POST /identity/v3/users => generated 301 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:10:15.785635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad1cb6e-c229-45f3-94b1-abaf38899bbc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:15.792309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497a1c7-4881-46df-8965-b82c791c4504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:15.792309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497a1c7-4881-46df-8965-b82c791c4504 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:15.792309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497a1c7-4881-46df-8965-b82c791c4504 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:15.792623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497a1c7-4881-46df-8965-b82c791c4504 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:15.793168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497a1c7-4881-46df-8965-b82c791c4504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:15.795669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497a1c7-4881-46df-8965-b82c791c4504 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:15.823872 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1622/3247] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:10:15.835974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3555d40-57f5-420a-a40c-b69880775213 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:15.857061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3555d40-57f5-420a-a40c-b69880775213 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:15.863593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54919262-d31f-49a2-bcf0-77c26be60916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:15.864678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54919262-d31f-49a2-bcf0-77c26be60916 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:15.864913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54919262-d31f-49a2-bcf0-77c26be60916 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:15.865193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54919262-d31f-49a2-bcf0-77c26be60916 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:15.866022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54919262-d31f-49a2-bcf0-77c26be60916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:15.867651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54919262-d31f-49a2-bcf0-77c26be60916 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:15.895959 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1626/3248] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:15 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:10:15.896937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497a1c7-4881-46df-8965-b82c791c4504 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:15.912323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497a1c7-4881-46df-8965-b82c791c4504 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:15.929882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefbf27-e8c6-4d66-8c62-cd02ed136994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:15.931196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefbf27-e8c6-4d66-8c62-cd02ed136994 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:15.931526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefbf27-e8c6-4d66-8c62-cd02ed136994 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:15.931918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefbf27-e8c6-4d66-8c62-cd02ed136994 admin admin] PATH_INFO: `/v3/projects/d67c3a0c476941f6a277be14769d6208/users/8a131a79139848f9af2b3236cea70185/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:15.985723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefbf27-e8c6-4d66-8c62-cd02ed136994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d67c3a0c476941f6a277be14769d6208, user_id=8a131a79139848f9af2b3236cea70185, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:15.990262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfefbf27-e8c6-4d66-8c62-cd02ed136994 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:16.078743 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1623/3249] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:10:15 2022] PUT /identity/v3/projects/d67c3a0c476941f6a277be14769d6208/users/8a131a79139848f9af2b3236cea70185/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:16.089476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82122b10-136a-4a16-b9a7-037bee907b0c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:16.090059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82122b10-136a-4a16-b9a7-037bee907b0c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:16.090059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82122b10-136a-4a16-b9a7-037bee907b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:16.122730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-82122b10-136a-4a16-b9a7-037bee907b0c None None] MFA Rules not processed for user `8a131a79139848f9af2b3236cea70185`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:16.123994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-82122b10-136a-4a16-b9a7-037bee907b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:16.248967 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1627/3250] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:16 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:16.253320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefbf27-e8c6-4d66-8c62-cd02ed136994 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:16.359613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefbf27-e8c6-4d66-8c62-cd02ed136994 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:16.368941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff43b6-f06f-4fa5-b16a-92985e909325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:16.369631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff43b6-f06f-4fa5-b16a-92985e909325 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:16.369944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff43b6-f06f-4fa5-b16a-92985e909325 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:16.370227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff43b6-f06f-4fa5-b16a-92985e909325 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:16.370950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff43b6-f06f-4fa5-b16a-92985e909325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:16.377125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff43b6-f06f-4fa5-b16a-92985e909325 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:16.522542 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1624/3251] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:10:16 2022] POST /identity/v3/projects => generated 401 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:10:16.523497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82122b10-136a-4a16-b9a7-037bee907b0c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:16.538885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82122b10-136a-4a16-b9a7-037bee907b0c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:16.619176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a3216-ada2-481f-ac30-7aacf25ff2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:16.619871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a3216-ada2-481f-ac30-7aacf25ff2f9 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:16.620155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a3216-ada2-481f-ac30-7aacf25ff2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:16.620429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a3216-ada2-481f-ac30-7aacf25ff2f9 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:16.621154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a3216-ada2-481f-ac30-7aacf25ff2f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:16.623811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a3216-ada2-481f-ac30-7aacf25ff2f9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:16.720822 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1628/3252] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:10:16 2022] POST /identity/v3/users => generated 307 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:10:16.722034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff43b6-f06f-4fa5-b16a-92985e909325 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:16.733255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff43b6-f06f-4fa5-b16a-92985e909325 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:16.739198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713dd5ad-356c-4018-943f-72d7a4809878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:16.740675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dd5ad-356c-4018-943f-72d7a4809878 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:16.740675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dd5ad-356c-4018-943f-72d7a4809878 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:16.740903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713dd5ad-356c-4018-943f-72d7a4809878 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:16.741591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713dd5ad-356c-4018-943f-72d7a4809878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:16.743552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713dd5ad-356c-4018-943f-72d7a4809878 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:16.756337 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1625/3253] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:10:16.760430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a3216-ada2-481f-ac30-7aacf25ff2f9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:16.772069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a3216-ada2-481f-ac30-7aacf25ff2f9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:16.778574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb0229-5631-4dd8-9e31-8040544223cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:16.786877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb0229-5631-4dd8-9e31-8040544223cc admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:16.786877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb0229-5631-4dd8-9e31-8040544223cc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:16.786877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb0229-5631-4dd8-9e31-8040544223cc admin admin] PATH_INFO: `/v3/projects/0a1ad739378c4535b40e955f069ea639/users/f6b22ed0543b4ca286cb052fd8278a39/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:16.799113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb0229-5631-4dd8-9e31-8040544223cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a1ad739378c4535b40e955f069ea639, user_id=f6b22ed0543b4ca286cb052fd8278a39, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:16.804169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb0229-5631-4dd8-9e31-8040544223cc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:16.891256 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1629/3254] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:10:16 2022] PUT /identity/v3/projects/0a1ad739378c4535b40e955f069ea639/users/f6b22ed0543b4ca286cb052fd8278a39/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:10:16.895506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713dd5ad-356c-4018-943f-72d7a4809878 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:16.909325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713dd5ad-356c-4018-943f-72d7a4809878 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:16.969428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:16.978623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb0e3b-49ca-454d-8f84-931cec588f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:16.979471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb0e3b-49ca-454d-8f84-931cec588f18 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:16.979604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb0e3b-49ca-454d-8f84-931cec588f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:16.979840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb0e3b-49ca-454d-8f84-931cec588f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:16.980606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb0e3b-49ca-454d-8f84-931cec588f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:16.982931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb0e3b-49ca-454d-8f84-931cec588f18 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:16.994099 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1626/3255] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:10:16.999666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb0229-5631-4dd8-9e31-8040544223cc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:17.012686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb0229-5631-4dd8-9e31-8040544223cc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:17.017728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a3578-6d6a-4490-a212-668cb40c01c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:17.018671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a3578-6d6a-4490-a212-668cb40c01c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:17.018955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a3578-6d6a-4490-a212-668cb40c01c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:17.019235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a3578-6d6a-4490-a212-668cb40c01c8 admin admin] PATH_INFO: `/v3/projects/0a1ad739378c4535b40e955f069ea639/users/f6b22ed0543b4ca286cb052fd8278a39/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:17.021283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a3578-6d6a-4490-a212-668cb40c01c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a1ad739378c4535b40e955f069ea639, user_id=f6b22ed0543b4ca286cb052fd8278a39, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:17.024047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a3578-6d6a-4490-a212-668cb40c01c8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:17.036223 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1630/3256] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:10:16 2022] PUT /identity/v3/projects/0a1ad739378c4535b40e955f069ea639/users/f6b22ed0543b4ca286cb052fd8278a39/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:17.106856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656522e-71df-45ff-98dc-17088ea3abd5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:17.107148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656522e-71df-45ff-98dc-17088ea3abd5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:17.108240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656522e-71df-45ff-98dc-17088ea3abd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:17.137281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9656522e-71df-45ff-98dc-17088ea3abd5 None None] MFA Rules not processed for user `f6b22ed0543b4ca286cb052fd8278a39`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:17.139273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9656522e-71df-45ff-98dc-17088ea3abd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:17.226023 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1627/3257] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:17 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:17.601269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a3578-6d6a-4490-a212-668cb40c01c8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:17.604123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1c4a3578-6d6a-4490-a212-668cb40c01c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:17.633909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:17.702592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a3578-6d6a-4490-a212-668cb40c01c8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:17.708910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9b574-293c-4118-b4bf-493d7da329ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:17.709627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9b574-293c-4118-b4bf-493d7da329ae service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:17.709945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9b574-293c-4118-b4bf-493d7da329ae service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:17.710231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc9b574-293c-4118-b4bf-493d7da329ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:17.721190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9b574-293c-4118-b4bf-493d7da329ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:17.724685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc9b574-293c-4118-b4bf-493d7da329ae service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:17.734131 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1631/3258] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:17 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:23.082726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656522e-71df-45ff-98dc-17088ea3abd5 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:23.096732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656522e-71df-45ff-98dc-17088ea3abd5 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:23.103160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd348cab-c748-44d8-b695-6d934a8afbe3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:23.103629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd348cab-c748-44d8-b695-6d934a8afbe3 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:23.103876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd348cab-c748-44d8-b695-6d934a8afbe3 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:23.104157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd348cab-c748-44d8-b695-6d934a8afbe3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:23.170503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd348cab-c748-44d8-b695-6d934a8afbe3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:23.172500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd348cab-c748-44d8-b695-6d934a8afbe3 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:23.181767 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1628/3259] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:23 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:23.200792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9b574-293c-4118-b4bf-493d7da329ae service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:23.297196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc9b574-293c-4118-b4bf-493d7da329ae service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:23.302598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953b2b6-f314-43fe-b978-51fd9ed4fd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:23.303584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953b2b6-f314-43fe-b978-51fd9ed4fd4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:23.304056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953b2b6-f314-43fe-b978-51fd9ed4fd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:23.304427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953b2b6-f314-43fe-b978-51fd9ed4fd4f admin admin] PATH_INFO: `/v3/users/8a131a79139848f9af2b3236cea70185` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:23.305737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953b2b6-f314-43fe-b978-51fd9ed4fd4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a131a79139848f9af2b3236cea70185)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:23.308109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953b2b6-f314-43fe-b978-51fd9ed4fd4f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:23.462483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1953b2b6-f314-43fe-b978-51fd9ed4fd4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a131a79139848f9af2b3236cea70185'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:23.469449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1953b2b6-f314-43fe-b978-51fd9ed4fd4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a131a79139848f9af2b3236cea70185'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:23.470159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1953b2b6-f314-43fe-b978-51fd9ed4fd4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a131a79139848f9af2b3236cea70185'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:23.470755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1953b2b6-f314-43fe-b978-51fd9ed4fd4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a131a79139848f9af2b3236cea70185'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:23.528253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1953b2b6-f314-43fe-b978-51fd9ed4fd4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a131a79139848f9af2b3236cea70185'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:23.547781 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1632/3260] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:10:23 2022] DELETE /identity/v3/users/8a131a79139848f9af2b3236cea70185 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:23.552561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db03037-6a30-43fa-842e-1d5a6a3d5ef6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:23.552854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db03037-6a30-43fa-842e-1d5a6a3d5ef6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:23.553133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db03037-6a30-43fa-842e-1d5a6a3d5ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:23.571326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4db03037-6a30-43fa-842e-1d5a6a3d5ef6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:23.572576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4db03037-6a30-43fa-842e-1d5a6a3d5ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:23.652355 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1629/3261] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:10:23.662533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953b2b6-f314-43fe-b978-51fd9ed4fd4f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:23.666888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1953b2b6-f314-43fe-b978-51fd9ed4fd4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:23.690473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:23.733193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953b2b6-f314-43fe-b978-51fd9ed4fd4f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:23.737299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd582ca-238f-4839-8a2c-03ac0f151bfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:23.737778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd582ca-238f-4839-8a2c-03ac0f151bfb service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:23.738116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd582ca-238f-4839-8a2c-03ac0f151bfb service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:23.738386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd582ca-238f-4839-8a2c-03ac0f151bfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:23.748921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd582ca-238f-4839-8a2c-03ac0f151bfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:23.750431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd582ca-238f-4839-8a2c-03ac0f151bfb service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:23.760926 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1633/3262] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:24.455836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db03037-6a30-43fa-842e-1d5a6a3d5ef6 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:24.458313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4db03037-6a30-43fa-842e-1d5a6a3d5ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:24.478576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db03037-6a30-43fa-842e-1d5a6a3d5ef6 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:24.484992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:24.486061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:24.486525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:24.486986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd admin admin] PATH_INFO: `/v3/projects/d67c3a0c476941f6a277be14769d6208` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:24.488412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d67c3a0c476941f6a277be14769d6208)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:24.490925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:24.504068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:24.600077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd67c3a0c476941f6a277be14769d6208', 'request_id': 'req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:24.605725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd67c3a0c476941f6a277be14769d6208', 'request_id': 'req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:24.693545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd67c3a0c476941f6a277be14769d6208', 'request_id': 'req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:24.707655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd admin admin] The token cache is being invalidate because project d67c3a0c476941f6a277be14769d6208 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:10:24.708035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:24.708761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:24.710517 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1630/3263] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:10:24 2022] DELETE /identity/v3/projects/d67c3a0c476941f6a277be14769d6208 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:24.714628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd582ca-238f-4839-8a2c-03ac0f151bfb service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:24.725234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd582ca-238f-4839-8a2c-03ac0f151bfb service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:24.785406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cb97e-05dd-448f-9722-1f619be5b814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:24.786313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cb97e-05dd-448f-9722-1f619be5b814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:24.786594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cb97e-05dd-448f-9722-1f619be5b814 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:24.786883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cb97e-05dd-448f-9722-1f619be5b814 admin admin] PATH_INFO: `/v3/users/f6b22ed0543b4ca286cb052fd8278a39` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:24.788077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cb97e-05dd-448f-9722-1f619be5b814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6b22ed0543b4ca286cb052fd8278a39)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:24.790027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cb97e-05dd-448f-9722-1f619be5b814 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:24.975275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ce2cb97e-05dd-448f-9722-1f619be5b814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6b22ed0543b4ca286cb052fd8278a39'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:24.993787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ce2cb97e-05dd-448f-9722-1f619be5b814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6b22ed0543b4ca286cb052fd8278a39'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:24.994577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ce2cb97e-05dd-448f-9722-1f619be5b814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6b22ed0543b4ca286cb052fd8278a39'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:24.995212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ce2cb97e-05dd-448f-9722-1f619be5b814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6b22ed0543b4ca286cb052fd8278a39'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:25.042094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ce2cb97e-05dd-448f-9722-1f619be5b814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6b22ed0543b4ca286cb052fd8278a39'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:25.068218 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1634/3264] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:10:24 2022] DELETE /identity/v3/users/f6b22ed0543b4ca286cb052fd8278a39 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:25.580810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:25.583038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:25.598544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123641e-f1ce-4ea0-8d87-b70ab7ad35dd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:25.668424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f09553-b4d3-4f98-ac3a-81a3148d830a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:25.669234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f09553-b4d3-4f98-ac3a-81a3148d830a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:25.669564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f09553-b4d3-4f98-ac3a-81a3148d830a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:25.670559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f09553-b4d3-4f98-ac3a-81a3148d830a admin admin] PATH_INFO: `/v3/projects/0a1ad739378c4535b40e955f069ea639` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:25.671812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f09553-b4d3-4f98-ac3a-81a3148d830a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a1ad739378c4535b40e955f069ea639)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:25.674047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f09553-b4d3-4f98-ac3a-81a3148d830a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:25.770724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f2f09553-b4d3-4f98-ac3a-81a3148d830a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a1ad739378c4535b40e955f069ea639', 'request_id': 'req-f2f09553-b4d3-4f98-ac3a-81a3148d830a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:25.777413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f2f09553-b4d3-4f98-ac3a-81a3148d830a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a1ad739378c4535b40e955f069ea639', 'request_id': 'req-f2f09553-b4d3-4f98-ac3a-81a3148d830a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:25.830917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f2f09553-b4d3-4f98-ac3a-81a3148d830a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a1ad739378c4535b40e955f069ea639', 'request_id': 'req-f2f09553-b4d3-4f98-ac3a-81a3148d830a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:25.843840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f2f09553-b4d3-4f98-ac3a-81a3148d830a admin admin] The token cache is being invalidate because project 0a1ad739378c4535b40e955f069ea639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:10:25.844131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f2f09553-b4d3-4f98-ac3a-81a3148d830a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:25.844839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f2f09553-b4d3-4f98-ac3a-81a3148d830a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:25.846446 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1631/3265] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:10:25 2022] DELETE /identity/v3/projects/0a1ad739378c4535b40e955f069ea639 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:25.851855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c224fa-67b2-494e-8ed3-066bf9e83eab None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:25.852797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c224fa-67b2-494e-8ed3-066bf9e83eab None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:25.853115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c224fa-67b2-494e-8ed3-066bf9e83eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:25.874429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-41c224fa-67b2-494e-8ed3-066bf9e83eab None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:25.875740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-41c224fa-67b2-494e-8ed3-066bf9e83eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:25.913217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:25.939634 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1635/3266] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:25.947352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be99c7-41db-44e3-817c-7942a1e872a6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:25.947643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be99c7-41db-44e3-817c-7942a1e872a6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:25.947921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be99c7-41db-44e3-817c-7942a1e872a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:25.967888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-73be99c7-41db-44e3-817c-7942a1e872a6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:25.969240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-73be99c7-41db-44e3-817c-7942a1e872a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:25.980864 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1632/3267] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:25.982963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c224fa-67b2-494e-8ed3-066bf9e83eab None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:25.993865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c224fa-67b2-494e-8ed3-066bf9e83eab None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:25.998733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efb6ef-4cdd-42c6-ae0c-b56788611527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:25.999445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efb6ef-4cdd-42c6-ae0c-b56788611527 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:25.999791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efb6ef-4cdd-42c6-ae0c-b56788611527 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:26.000076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efb6ef-4cdd-42c6-ae0c-b56788611527 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:26.000821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efb6ef-4cdd-42c6-ae0c-b56788611527 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:10:26.001131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efb6ef-4cdd-42c6-ae0c-b56788611527 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:26.002796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efb6ef-4cdd-42c6-ae0c-b56788611527 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:26.016300 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1636/3268] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:10:26.020936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be99c7-41db-44e3-817c-7942a1e872a6 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:26.033428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be99c7-41db-44e3-817c-7942a1e872a6 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:26.039139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4255d-54f8-4044-b5e5-abdeb0ef4340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:26.039814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4255d-54f8-4044-b5e5-abdeb0ef4340 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:26.048694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4255d-54f8-4044-b5e5-abdeb0ef4340 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:26.048694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4255d-54f8-4044-b5e5-abdeb0ef4340 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:26.048694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb4255d-54f8-4044-b5e5-abdeb0ef4340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:26.048694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb4255d-54f8-4044-b5e5-abdeb0ef4340 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:26.146941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:26.158265 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1633/3269] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:10:26 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:10:26.162450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efb6ef-4cdd-42c6-ae0c-b56788611527 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:26.172918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efb6ef-4cdd-42c6-ae0c-b56788611527 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:26.274300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfff730-7269-45e7-a656-437ec479d189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:26.274869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfff730-7269-45e7-a656-437ec479d189 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:26.275511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfff730-7269-45e7-a656-437ec479d189 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:26.275799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfff730-7269-45e7-a656-437ec479d189 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:26.276588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfff730-7269-45e7-a656-437ec479d189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:26.280460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfff730-7269-45e7-a656-437ec479d189 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:26.395900 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1637/3270] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:10:26 2022] POST /identity/v3/users => generated 292 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:10:26.397905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4255d-54f8-4044-b5e5-abdeb0ef4340 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:26.426058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb4255d-54f8-4044-b5e5-abdeb0ef4340 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:26.433176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f1c02-fff7-4a87-b2c5-21576d115b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:26.433176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f1c02-fff7-4a87-b2c5-21576d115b9b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:26.433709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f1c02-fff7-4a87-b2c5-21576d115b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:26.434036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262f1c02-fff7-4a87-b2c5-21576d115b9b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:26.434810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f1c02-fff7-4a87-b2c5-21576d115b9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:26.437058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262f1c02-fff7-4a87-b2c5-21576d115b9b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:26.453522 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1634/3271] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:10:26.454911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfff730-7269-45e7-a656-437ec479d189 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:26.466083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfff730-7269-45e7-a656-437ec479d189 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:26.471042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a9cfa-371a-43c3-94f3-859adefa18d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:26.471807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a9cfa-371a-43c3-94f3-859adefa18d1 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:26.472095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a9cfa-371a-43c3-94f3-859adefa18d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:26.472378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a9cfa-371a-43c3-94f3-859adefa18d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:26.473169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a9cfa-371a-43c3-94f3-859adefa18d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:26.475041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007a9cfa-371a-43c3-94f3-859adefa18d1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:26.486348 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1638/3272] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:10:26.491090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f1c02-fff7-4a87-b2c5-21576d115b9b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:26.501881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262f1c02-fff7-4a87-b2c5-21576d115b9b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:26.506704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdba0a-ac27-4c3c-9493-c35e73ba5346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:26.507629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdba0a-ac27-4c3c-9493-c35e73ba5346 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:26.507931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdba0a-ac27-4c3c-9493-c35e73ba5346 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:26.508283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdba0a-ac27-4c3c-9493-c35e73ba5346 admin admin] PATH_INFO: `/v3/projects/b4be72b7f8c54528b1c0a2a1505416c1/users/94f224e3fd2142d0991cb173e2a2c880/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:26.524975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdba0a-ac27-4c3c-9493-c35e73ba5346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4be72b7f8c54528b1c0a2a1505416c1, user_id=94f224e3fd2142d0991cb173e2a2c880, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:26.530840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdba0a-ac27-4c3c-9493-c35e73ba5346 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:26.634348 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1635/3273] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:10:26 2022] PUT /identity/v3/projects/b4be72b7f8c54528b1c0a2a1505416c1/users/94f224e3fd2142d0991cb173e2a2c880/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:26.641979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01a57a-dc0c-4258-bdaa-1f2c7fb0c719 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:26.642439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01a57a-dc0c-4258-bdaa-1f2c7fb0c719 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:26.642830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c01a57a-dc0c-4258-bdaa-1f2c7fb0c719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:26.663015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9c01a57a-dc0c-4258-bdaa-1f2c7fb0c719 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:26.664740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9c01a57a-dc0c-4258-bdaa-1f2c7fb0c719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:26.816802 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1639/3274] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:26.834600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdba0a-ac27-4c3c-9493-c35e73ba5346 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:26.845947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-03fdba0a-ac27-4c3c-9493-c35e73ba5346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:26.958801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdba0a-ac27-4c3c-9493-c35e73ba5346 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:26.963826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f2f94-d1f6-467a-97ef-4f827d86b8bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:26.964424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f2f94-d1f6-467a-97ef-4f827d86b8bb service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:26.964757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f2f94-d1f6-467a-97ef-4f827d86b8bb service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:26.966329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f2f94-d1f6-467a-97ef-4f827d86b8bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:27.002824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f2f94-d1f6-467a-97ef-4f827d86b8bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:27.004466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f2f94-d1f6-467a-97ef-4f827d86b8bb service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:27.021772 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1636/3275] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:27.199320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c01a57a-dc0c-4258-bdaa-1f2c7fb0c719 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:27.200431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9c01a57a-dc0c-4258-bdaa-1f2c7fb0c719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:27.225825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c01a57a-dc0c-4258-bdaa-1f2c7fb0c719 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:27.237242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb2e35-e57f-4bd1-b716-5df3a90bca95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:27.240968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb2e35-e57f-4bd1-b716-5df3a90bca95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:27.240968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb2e35-e57f-4bd1-b716-5df3a90bca95 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:27.240968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb2e35-e57f-4bd1-b716-5df3a90bca95 admin admin] PATH_INFO: `/v3/users/f08e71479f9d4ef0a30681cdfdd6fef8` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:27.241782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb2e35-e57f-4bd1-b716-5df3a90bca95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f08e71479f9d4ef0a30681cdfdd6fef8)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:27.246274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb2e35-e57f-4bd1-b716-5df3a90bca95 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:27.458761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:27.465621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d9eb2e35-e57f-4bd1-b716-5df3a90bca95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f08e71479f9d4ef0a30681cdfdd6fef8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:27.475080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d9eb2e35-e57f-4bd1-b716-5df3a90bca95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f08e71479f9d4ef0a30681cdfdd6fef8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:27.475855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d9eb2e35-e57f-4bd1-b716-5df3a90bca95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f08e71479f9d4ef0a30681cdfdd6fef8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:27.476555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d9eb2e35-e57f-4bd1-b716-5df3a90bca95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f08e71479f9d4ef0a30681cdfdd6fef8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:27.567892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d9eb2e35-e57f-4bd1-b716-5df3a90bca95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f08e71479f9d4ef0a30681cdfdd6fef8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:27.587076 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1640/3276] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:10:27 2022] DELETE /identity/v3/users/f08e71479f9d4ef0a30681cdfdd6fef8 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:28.792059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f2f94-d1f6-467a-97ef-4f827d86b8bb service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:28.802924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f2f94-d1f6-467a-97ef-4f827d86b8bb service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:28.904544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd6088-e715-41b3-809d-68386e1b86e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:28.905520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd6088-e715-41b3-809d-68386e1b86e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:28.906379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd6088-e715-41b3-809d-68386e1b86e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:28.906379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd6088-e715-41b3-809d-68386e1b86e9 admin admin] PATH_INFO: `/v3/projects/e96bf6518d66430399de823144f7bea8` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:28.907782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd6088-e715-41b3-809d-68386e1b86e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e96bf6518d66430399de823144f7bea8)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:28.910322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd6088-e715-41b3-809d-68386e1b86e9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:29.057242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-61bd6088-e715-41b3-809d-68386e1b86e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e96bf6518d66430399de823144f7bea8', 'request_id': 'req-61bd6088-e715-41b3-809d-68386e1b86e9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:29.066086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-61bd6088-e715-41b3-809d-68386e1b86e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e96bf6518d66430399de823144f7bea8', 'request_id': 'req-61bd6088-e715-41b3-809d-68386e1b86e9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:29.169237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-61bd6088-e715-41b3-809d-68386e1b86e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e96bf6518d66430399de823144f7bea8', 'request_id': 'req-61bd6088-e715-41b3-809d-68386e1b86e9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:29.180353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:29.191661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-61bd6088-e715-41b3-809d-68386e1b86e9 admin admin] The token cache is being invalidate because project e96bf6518d66430399de823144f7bea8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:10:29.192201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-61bd6088-e715-41b3-809d-68386e1b86e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:29.193007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-61bd6088-e715-41b3-809d-68386e1b86e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:29.195429 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1637/3277] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:10:28 2022] DELETE /identity/v3/projects/e96bf6518d66430399de823144f7bea8 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:29.202933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55909c0-c7f7-4191-970f-1a24b5c83444 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:29.203435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55909c0-c7f7-4191-970f-1a24b5c83444 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:29.203556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55909c0-c7f7-4191-970f-1a24b5c83444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:29.246339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d55909c0-c7f7-4191-970f-1a24b5c83444 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:29.251519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d55909c0-c7f7-4191-970f-1a24b5c83444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:29.378540 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1641/3278] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:29.386533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1e3ee-c9c2-4a68-97c7-1547a995f703 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:29.386954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1e3ee-c9c2-4a68-97c7-1547a995f703 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:29.387558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1e3ee-c9c2-4a68-97c7-1547a995f703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:29.423212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d4b1e3ee-c9c2-4a68-97c7-1547a995f703 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:29.426363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d4b1e3ee-c9c2-4a68-97c7-1547a995f703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:29.448638 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1638/3279] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:29.453718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55909c0-c7f7-4191-970f-1a24b5c83444 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:29.471374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55909c0-c7f7-4191-970f-1a24b5c83444 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:29.484932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24244a43-98ef-499a-ad1a-137a99c82b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:29.486091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24244a43-98ef-499a-ad1a-137a99c82b57 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:29.490247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24244a43-98ef-499a-ad1a-137a99c82b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:29.492577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24244a43-98ef-499a-ad1a-137a99c82b57 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:29.493490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24244a43-98ef-499a-ad1a-137a99c82b57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:10:29.493931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24244a43-98ef-499a-ad1a-137a99c82b57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:29.498648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24244a43-98ef-499a-ad1a-137a99c82b57 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:29.522607 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1642/3280] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:10:29.527320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1e3ee-c9c2-4a68-97c7-1547a995f703 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:29.537242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1e3ee-c9c2-4a68-97c7-1547a995f703 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:29.542928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4511b0-a68c-4386-8e7b-6309eaa3815b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:29.544028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4511b0-a68c-4386-8e7b-6309eaa3815b admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:29.544408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4511b0-a68c-4386-8e7b-6309eaa3815b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:29.544813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4511b0-a68c-4386-8e7b-6309eaa3815b admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:29.545611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4511b0-a68c-4386-8e7b-6309eaa3815b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:29.559581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4511b0-a68c-4386-8e7b-6309eaa3815b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:29.662073 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1639/3281] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:10:29 2022] POST /identity/v3/projects => generated 361 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:10:29.666268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24244a43-98ef-499a-ad1a-137a99c82b57 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:29.713434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24244a43-98ef-499a-ad1a-137a99c82b57 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:29.829507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa8579-8917-4a57-ae31-acccd2129a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:29.832585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa8579-8917-4a57-ae31-acccd2129a05 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:29.832585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa8579-8917-4a57-ae31-acccd2129a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:29.832585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa8579-8917-4a57-ae31-acccd2129a05 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:29.832585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa8579-8917-4a57-ae31-acccd2129a05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:29.837260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa8579-8917-4a57-ae31-acccd2129a05 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:29.922192 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1643/3282] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:10:29 2022] POST /identity/v3/users => generated 281 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:10:29.926369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4511b0-a68c-4386-8e7b-6309eaa3815b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:29.943512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4511b0-a68c-4386-8e7b-6309eaa3815b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:29.958906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd5e43-2b95-4254-9759-49e7c0a1e380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:29.959984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd5e43-2b95-4254-9759-49e7c0a1e380 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:29.960286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd5e43-2b95-4254-9759-49e7c0a1e380 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:29.960566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd5e43-2b95-4254-9759-49e7c0a1e380 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:29.962171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd5e43-2b95-4254-9759-49e7c0a1e380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:29.964714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becd5e43-2b95-4254-9759-49e7c0a1e380 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:29.990390 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1640/3283] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:10:29.994176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa8579-8917-4a57-ae31-acccd2129a05 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:30.008671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa8579-8917-4a57-ae31-acccd2129a05 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:30.016119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70182902-1a65-4013-9ff1-79942e7b5996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:30.017136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70182902-1a65-4013-9ff1-79942e7b5996 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:30.017649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70182902-1a65-4013-9ff1-79942e7b5996 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:30.018194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70182902-1a65-4013-9ff1-79942e7b5996 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:30.019176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70182902-1a65-4013-9ff1-79942e7b5996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:30.021595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70182902-1a65-4013-9ff1-79942e7b5996 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:30.065366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd5e43-2b95-4254-9759-49e7c0a1e380 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:30.065739 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1644/3284] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:10:30.084241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd5e43-2b95-4254-9759-49e7c0a1e380 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:30.090800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22eb58-4ca9-4f36-acb8-ed62c6e55cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:30.092471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22eb58-4ca9-4f36-acb8-ed62c6e55cd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:30.092821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22eb58-4ca9-4f36-acb8-ed62c6e55cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:30.093167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22eb58-4ca9-4f36-acb8-ed62c6e55cd0 admin admin] PATH_INFO: `/v3/projects/615e24ab46f3491c858f0c6f0df87da5/users/ad8c1df3aa57484baade5068db36a731/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:30.114005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22eb58-4ca9-4f36-acb8-ed62c6e55cd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=615e24ab46f3491c858f0c6f0df87da5, user_id=ad8c1df3aa57484baade5068db36a731, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:30.117208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22eb58-4ca9-4f36-acb8-ed62c6e55cd0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:30.200377 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1641/3285] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:10:30 2022] PUT /identity/v3/projects/615e24ab46f3491c858f0c6f0df87da5/users/ad8c1df3aa57484baade5068db36a731/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:30.211189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb194e-264c-4f6d-a7f3-acd41ec9dc01 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:30.211741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb194e-264c-4f6d-a7f3-acd41ec9dc01 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:30.211983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb194e-264c-4f6d-a7f3-acd41ec9dc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:30.253876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6afb194e-264c-4f6d-a7f3-acd41ec9dc01 None None] MFA Rules not processed for user `ad8c1df3aa57484baade5068db36a731`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:30.255241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6afb194e-264c-4f6d-a7f3-acd41ec9dc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:30.286946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:30.401201 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1645/3286] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:31.004185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22eb58-4ca9-4f36-acb8-ed62c6e55cd0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:31.006931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ab22eb58-4ca9-4f36-acb8-ed62c6e55cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:31.198017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22eb58-4ca9-4f36-acb8-ed62c6e55cd0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:31.198017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7e527-6440-4b77-bafe-dacefc78ab6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:31.198017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7e527-6440-4b77-bafe-dacefc78ab6d service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:31.198017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7e527-6440-4b77-bafe-dacefc78ab6d service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:31.199768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7e527-6440-4b77-bafe-dacefc78ab6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:31.199768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7e527-6440-4b77-bafe-dacefc78ab6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:31.199768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7e527-6440-4b77-bafe-dacefc78ab6d service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:31.199768 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1642/3287] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:31.263825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb194e-264c-4f6d-a7f3-acd41ec9dc01 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:31.369348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb194e-264c-4f6d-a7f3-acd41ec9dc01 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:31.376077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521209d-0fe3-47de-b1bb-507dd13e251a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:31.380851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521209d-0fe3-47de-b1bb-507dd13e251a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:31.380851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521209d-0fe3-47de-b1bb-507dd13e251a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:31.380851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521209d-0fe3-47de-b1bb-507dd13e251a admin admin] PATH_INFO: `/v3/users/ad8c1df3aa57484baade5068db36a731` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:31.380851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521209d-0fe3-47de-b1bb-507dd13e251a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad8c1df3aa57484baade5068db36a731)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:31.382159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521209d-0fe3-47de-b1bb-507dd13e251a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:31.608841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f521209d-0fe3-47de-b1bb-507dd13e251a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad8c1df3aa57484baade5068db36a731'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:31.620484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f521209d-0fe3-47de-b1bb-507dd13e251a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad8c1df3aa57484baade5068db36a731'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:31.621595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f521209d-0fe3-47de-b1bb-507dd13e251a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad8c1df3aa57484baade5068db36a731'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:31.622708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f521209d-0fe3-47de-b1bb-507dd13e251a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad8c1df3aa57484baade5068db36a731'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:31.695135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f521209d-0fe3-47de-b1bb-507dd13e251a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad8c1df3aa57484baade5068db36a731'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:31.761685 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1646/3288] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:10:31 2022] DELETE /identity/v3/users/ad8c1df3aa57484baade5068db36a731 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:31.766203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48226a33-f49b-47c6-be52-5d37c8284c8f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:31.766818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48226a33-f49b-47c6-be52-5d37c8284c8f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:31.767578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48226a33-f49b-47c6-be52-5d37c8284c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:31.801100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-48226a33-f49b-47c6-be52-5d37c8284c8f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:31.804482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-48226a33-f49b-47c6-be52-5d37c8284c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:31.932009 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1643/3289] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:31.983154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521209d-0fe3-47de-b1bb-507dd13e251a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:31.984975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f521209d-0fe3-47de-b1bb-507dd13e251a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:32.030940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:32.055483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521209d-0fe3-47de-b1bb-507dd13e251a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:32.062623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5444a126-748c-4192-917e-af0f578ef77c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:32.064023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444a126-748c-4192-917e-af0f578ef77c service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:32.064509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444a126-748c-4192-917e-af0f578ef77c service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:32.064835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5444a126-748c-4192-917e-af0f578ef77c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:32.075477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5444a126-748c-4192-917e-af0f578ef77c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:32.077629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5444a126-748c-4192-917e-af0f578ef77c service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:32.091141 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1647/3290] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:33.910545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48226a33-f49b-47c6-be52-5d37c8284c8f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:33.915271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-48226a33-f49b-47c6-be52-5d37c8284c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:33.931130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48226a33-f49b-47c6-be52-5d37c8284c8f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:33.939382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f10945a-5bce-48fb-bef6-cf705f6475cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:33.940546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10945a-5bce-48fb-bef6-cf705f6475cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:33.941196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10945a-5bce-48fb-bef6-cf705f6475cf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:33.941482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f10945a-5bce-48fb-bef6-cf705f6475cf admin admin] PATH_INFO: `/v3/projects/615e24ab46f3491c858f0c6f0df87da5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:33.943884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10945a-5bce-48fb-bef6-cf705f6475cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=615e24ab46f3491c858f0c6f0df87da5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:33.945926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f10945a-5bce-48fb-bef6-cf705f6475cf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:33.959148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c5abf-f1cb-4c08-8ea9-1727b83be526 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:33.959733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c5abf-f1cb-4c08-8ea9-1727b83be526 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:33.959733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c5abf-f1cb-4c08-8ea9-1727b83be526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:33.988383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:33.995906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f48c5abf-f1cb-4c08-8ea9-1727b83be526 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:33.997340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f48c5abf-f1cb-4c08-8ea9-1727b83be526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:34.102961 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1648/3291] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:10:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:10:34.154838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c5abf-f1cb-4c08-8ea9-1727b83be526 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:34.156756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f48c5abf-f1cb-4c08-8ea9-1727b83be526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:34.170426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8f10945a-5bce-48fb-bef6-cf705f6475cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '615e24ab46f3491c858f0c6f0df87da5', 'request_id': 'req-8f10945a-5bce-48fb-bef6-cf705f6475cf'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:34.176721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8f10945a-5bce-48fb-bef6-cf705f6475cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '615e24ab46f3491c858f0c6f0df87da5', 'request_id': 'req-8f10945a-5bce-48fb-bef6-cf705f6475cf'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:34.265114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8f10945a-5bce-48fb-bef6-cf705f6475cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '615e24ab46f3491c858f0c6f0df87da5', 'request_id': 'req-8f10945a-5bce-48fb-bef6-cf705f6475cf'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:34.288831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8f10945a-5bce-48fb-bef6-cf705f6475cf admin admin] The token cache is being invalidate because project 615e24ab46f3491c858f0c6f0df87da5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:10:34.289159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8f10945a-5bce-48fb-bef6-cf705f6475cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:34.290310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8f10945a-5bce-48fb-bef6-cf705f6475cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:34.299418 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1644/3292] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:10:33 2022] DELETE /identity/v3/projects/615e24ab46f3491c858f0c6f0df87da5 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:34.306955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c5abf-f1cb-4c08-8ea9-1727b83be526 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:34.312446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4795818-26d9-43a4-bf07-02809d0a5745 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:34.313186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4795818-26d9-43a4-bf07-02809d0a5745 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:34.314526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4795818-26d9-43a4-bf07-02809d0a5745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:34.392566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d4795818-26d9-43a4-bf07-02809d0a5745 None None] MFA Rules not processed for user `94f224e3fd2142d0991cb173e2a2c880`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:34.395368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d4795818-26d9-43a4-bf07-02809d0a5745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:34.423251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de094656-1f4c-44ba-8e33-7bdf1df0f65e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:34.424252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de094656-1f4c-44ba-8e33-7bdf1df0f65e service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:34.424716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de094656-1f4c-44ba-8e33-7bdf1df0f65e service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:34.425338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de094656-1f4c-44ba-8e33-7bdf1df0f65e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:34.428037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-de094656-1f4c-44ba-8e33-7bdf1df0f65e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:34.485624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:34.547439 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1645/3293] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:34.549889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cd6e8-ef2e-45fb-a85d-7e277adb75ac None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:34.551682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cd6e8-ef2e-45fb-a85d-7e277adb75ac None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:34.552224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2cd6e8-ef2e-45fb-a85d-7e277adb75ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:34.579781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de094656-1f4c-44ba-8e33-7bdf1df0f65e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:34.585942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de094656-1f4c-44ba-8e33-7bdf1df0f65e service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:34.601143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7e2cd6e8-ef2e-45fb-a85d-7e277adb75ac None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:34.608305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7e2cd6e8-ef2e-45fb-a85d-7e277adb75ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:34.639128 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1649/3294] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:10:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 21:10:34.639128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de094656-1f4c-44ba-8e33-7bdf1df0f65e service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:34.639128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-de094656-1f4c-44ba-8e33-7bdf1df0f65e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:34.741199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de094656-1f4c-44ba-8e33-7bdf1df0f65e service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:34.750381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5e4b5-4a73-43fa-a5b7-acb358d20814 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:34.750381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5e4b5-4a73-43fa-a5b7-acb358d20814 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:34.750381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5e4b5-4a73-43fa-a5b7-acb358d20814 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:34.750381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5e4b5-4a73-43fa-a5b7-acb358d20814 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:34.752259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eef5e4b5-4a73-43fa-a5b7-acb358d20814 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:34.764540 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1646/3295] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:34.766566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3b2fc-0231-4452-a466-53e897b0d101 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:34.767109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3b2fc-0231-4452-a466-53e897b0d101 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:34.767558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df3b2fc-0231-4452-a466-53e897b0d101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:34.775109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5e4b5-4a73-43fa-a5b7-acb358d20814 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:34.797155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5e4b5-4a73-43fa-a5b7-acb358d20814 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:34.805973 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1650/3296] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:34.827555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6df3b2fc-0231-4452-a466-53e897b0d101 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:34.830446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6df3b2fc-0231-4452-a466-53e897b0d101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:34.867714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5e4b5-4a73-43fa-a5b7-acb358d20814 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:34.872619 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1647/3297] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:34.890516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5e4b5-4a73-43fa-a5b7-acb358d20814 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:34.904735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18356d22-81c5-4e03-8658-7893b0af2e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:34.905543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18356d22-81c5-4e03-8658-7893b0af2e14 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:34.905984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18356d22-81c5-4e03-8658-7893b0af2e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:34.906366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18356d22-81c5-4e03-8658-7893b0af2e14 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:34.907271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18356d22-81c5-4e03-8658-7893b0af2e14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:10:34.907697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18356d22-81c5-4e03-8658-7893b0af2e14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:34.916210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18356d22-81c5-4e03-8658-7893b0af2e14 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:34.945764 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1651/3298] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:10:34.949813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3b2fc-0231-4452-a466-53e897b0d101 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:34.965330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df3b2fc-0231-4452-a466-53e897b0d101 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:34.979503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71661a8-5075-464c-a69d-276147173cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:34.983912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71661a8-5075-464c-a69d-276147173cfb admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:34.983912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71661a8-5075-464c-a69d-276147173cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:34.983912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71661a8-5075-464c-a69d-276147173cfb admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:34.985253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71661a8-5075-464c-a69d-276147173cfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:34.988577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71661a8-5075-464c-a69d-276147173cfb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:35.102849 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1648/3299] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:10:34 2022] POST /identity/v3/projects => generated 377 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:10:35.108345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18356d22-81c5-4e03-8658-7893b0af2e14 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:35.125225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18356d22-81c5-4e03-8658-7893b0af2e14 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:35.260569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9ebda-ec2b-4507-8ce8-257d2924185b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:35.261469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9ebda-ec2b-4507-8ce8-257d2924185b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:35.262151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9ebda-ec2b-4507-8ce8-257d2924185b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:35.262630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9ebda-ec2b-4507-8ce8-257d2924185b admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:35.264418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9ebda-ec2b-4507-8ce8-257d2924185b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:35.267737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9ebda-ec2b-4507-8ce8-257d2924185b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:35.368883 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1652/3300] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:10:35 2022] POST /identity/v3/users => generated 289 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:10:35.376638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71661a8-5075-464c-a69d-276147173cfb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:35.396098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:35.407155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71661a8-5075-464c-a69d-276147173cfb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:35.440684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837c66a-6bf4-4970-a308-73e19a3377b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:35.441393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837c66a-6bf4-4970-a308-73e19a3377b9 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:35.441687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837c66a-6bf4-4970-a308-73e19a3377b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:35.442003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0837c66a-6bf4-4970-a308-73e19a3377b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:35.442760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0837c66a-6bf4-4970-a308-73e19a3377b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:35.444413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0837c66a-6bf4-4970-a308-73e19a3377b9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:35.503588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9ebda-ec2b-4507-8ce8-257d2924185b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:35.504866 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1649/3301] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:10:35.530273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9ebda-ec2b-4507-8ce8-257d2924185b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:35.556600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c472f2f-1f67-45ec-9d87-54402d621a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:35.563003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c472f2f-1f67-45ec-9d87-54402d621a9d admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:35.568517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c472f2f-1f67-45ec-9d87-54402d621a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:35.569632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c472f2f-1f67-45ec-9d87-54402d621a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:35.572072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c472f2f-1f67-45ec-9d87-54402d621a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:35.586110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c472f2f-1f67-45ec-9d87-54402d621a9d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:35.620733 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1653/3302] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:10:35.621382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837c66a-6bf4-4970-a308-73e19a3377b9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:35.633158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0837c66a-6bf4-4970-a308-73e19a3377b9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:35.662363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6617fcab-f41f-42c5-8ba0-443bdaf3b8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:35.663290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617fcab-f41f-42c5-8ba0-443bdaf3b8a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:35.663579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617fcab-f41f-42c5-8ba0-443bdaf3b8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:35.663853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6617fcab-f41f-42c5-8ba0-443bdaf3b8a6 admin admin] PATH_INFO: `/v3/projects/cf04fd760c54473f83980a385252fa3c/users/bf9c8f66e7d24ca9baca835e41546f22/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:35.718594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617fcab-f41f-42c5-8ba0-443bdaf3b8a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf04fd760c54473f83980a385252fa3c, user_id=bf9c8f66e7d24ca9baca835e41546f22, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:35.727690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6617fcab-f41f-42c5-8ba0-443bdaf3b8a6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:35.808769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c472f2f-1f67-45ec-9d87-54402d621a9d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:35.819104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3c472f2f-1f67-45ec-9d87-54402d621a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:35.836459 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1650/3303] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:10:35 2022] PUT /identity/v3/projects/cf04fd760c54473f83980a385252fa3c/users/bf9c8f66e7d24ca9baca835e41546f22/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:35.840055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe75eb-7378-4ada-9b57-5699af6881f0 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:35.841407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe75eb-7378-4ada-9b57-5699af6881f0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:35.842144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fe75eb-7378-4ada-9b57-5699af6881f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:35.883385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:35.897275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-24fe75eb-7378-4ada-9b57-5699af6881f0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:35.906769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-24fe75eb-7378-4ada-9b57-5699af6881f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:35.988933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c472f2f-1f67-45ec-9d87-54402d621a9d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:36.102984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8772768-3714-4ac5-b4c4-1a70525a91fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:36.106327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8772768-3714-4ac5-b4c4-1a70525a91fd service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:36.106769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8772768-3714-4ac5-b4c4-1a70525a91fd service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:36.107882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8772768-3714-4ac5-b4c4-1a70525a91fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:36.112445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a8772768-3714-4ac5-b4c4-1a70525a91fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:36.130261 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1651/3304] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:10:36.145261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8772768-3714-4ac5-b4c4-1a70525a91fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:36.148732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8772768-3714-4ac5-b4c4-1a70525a91fd service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:36.182892 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1654/3305] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:36.273664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe75eb-7378-4ada-9b57-5699af6881f0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:36.275562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-24fe75eb-7378-4ada-9b57-5699af6881f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:36.526868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fe75eb-7378-4ada-9b57-5699af6881f0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:36.542985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf2fe7-fc92-48b2-b9b6-0fb1f933bee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:36.546883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf2fe7-fc92-48b2-b9b6-0fb1f933bee0 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:36.548577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf2fe7-fc92-48b2-b9b6-0fb1f933bee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:36.548577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf2fe7-fc92-48b2-b9b6-0fb1f933bee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:36.589451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf2fe7-fc92-48b2-b9b6-0fb1f933bee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:36.591083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf2fe7-fc92-48b2-b9b6-0fb1f933bee0 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:36.618850 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1652/3306] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:38.399877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8772768-3714-4ac5-b4c4-1a70525a91fd service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:38.402574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a8772768-3714-4ac5-b4c4-1a70525a91fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:38.614347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8772768-3714-4ac5-b4c4-1a70525a91fd service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:38.638590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ee323-5d3e-46b3-b484-d25db04c9dd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:38.639274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee323-5d3e-46b3-b484-d25db04c9dd0 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:38.641513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee323-5d3e-46b3-b484-d25db04c9dd0 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:38.641969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee323-5d3e-46b3-b484-d25db04c9dd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:38.791170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:38.807423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ee323-5d3e-46b3-b484-d25db04c9dd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:38.809076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ee323-5d3e-46b3-b484-d25db04c9dd0 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:38.837468 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1655/3307] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:42.604221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2199d3-7b64-414c-94bb-c2cc9fde403d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:42.605296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2199d3-7b64-414c-94bb-c2cc9fde403d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:42.605792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2199d3-7b64-414c-94bb-c2cc9fde403d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:42.683411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bc2199d3-7b64-414c-94bb-c2cc9fde403d None None] MFA Rules not processed for user `bf9c8f66e7d24ca9baca835e41546f22`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:42.688617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bc2199d3-7b64-414c-94bb-c2cc9fde403d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:42.806488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac1fd4-c477-45ec-b5c2-fdca7fa5aff0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:42.806977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac1fd4-c477-45ec-b5c2-fdca7fa5aff0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:42.807085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac1fd4-c477-45ec-b5c2-fdca7fa5aff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:42.823059 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1653/3308] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:42.839288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a141142-b52c-49c5-94da-48765f030246 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:42.839958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a141142-b52c-49c5-94da-48765f030246 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:42.842039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a141142-b52c-49c5-94da-48765f030246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:42.866104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:42.879689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-66ac1fd4-c477-45ec-b5c2-fdca7fa5aff0 None None] MFA Rules not processed for user `659bd3ebefc348b7a176379bd342f9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:42.885171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-66ac1fd4-c477-45ec-b5c2-fdca7fa5aff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:42.891408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4a141142-b52c-49c5-94da-48765f030246 None None] MFA Rules not processed for user `bf9c8f66e7d24ca9baca835e41546f22`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:42.893978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4a141142-b52c-49c5-94da-48765f030246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:42.914510 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1654/3309] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:43.079010 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1656/3310] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:10:44.098182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a141142-b52c-49c5-94da-48765f030246 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:44.114593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a141142-b52c-49c5-94da-48765f030246 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:44.121152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a36cda-5408-4aef-957f-d60dc0ad38a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:44.122241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a36cda-5408-4aef-957f-d60dc0ad38a2 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:44.123272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a36cda-5408-4aef-957f-d60dc0ad38a2 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:44.123814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a36cda-5408-4aef-957f-d60dc0ad38a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:44.136212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a36cda-5408-4aef-957f-d60dc0ad38a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:44.138891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a36cda-5408-4aef-957f-d60dc0ad38a2 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:44.161919 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1655/3311] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:45.860039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63466300-2690-4a5a-a064-0aebed1bd165 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:45.860511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63466300-2690-4a5a-a064-0aebed1bd165 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:45.860624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63466300-2690-4a5a-a064-0aebed1bd165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:45.904794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-63466300-2690-4a5a-a064-0aebed1bd165 None None] MFA Rules not processed for user `94f224e3fd2142d0991cb173e2a2c880`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:45.906353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-63466300-2690-4a5a-a064-0aebed1bd165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:45.932764 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1657/3312] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:51.344459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a36cda-5408-4aef-957f-d60dc0ad38a2 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:51.348952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a9a36cda-5408-4aef-957f-d60dc0ad38a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:51.369829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a36cda-5408-4aef-957f-d60dc0ad38a2 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:51.376425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62452c-8e8c-442f-84f8-5f19dbe10178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:51.377707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62452c-8e8c-442f-84f8-5f19dbe10178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:51.378212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62452c-8e8c-442f-84f8-5f19dbe10178 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:51.378684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62452c-8e8c-442f-84f8-5f19dbe10178 admin admin] PATH_INFO: `/v3/users/0cec7231f1c1445f965d620ae70e8463` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:51.380332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae62452c-8e8c-442f-84f8-5f19dbe10178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cec7231f1c1445f965d620ae70e8463)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:51.382976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae62452c-8e8c-442f-84f8-5f19dbe10178 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:51.744837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ae62452c-8e8c-442f-84f8-5f19dbe10178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cec7231f1c1445f965d620ae70e8463'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:51.745855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ae62452c-8e8c-442f-84f8-5f19dbe10178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cec7231f1c1445f965d620ae70e8463'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:51.754229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ae62452c-8e8c-442f-84f8-5f19dbe10178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cec7231f1c1445f965d620ae70e8463'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:51.828100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63466300-2690-4a5a-a064-0aebed1bd165 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:51.830732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-63466300-2690-4a5a-a064-0aebed1bd165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:51.830982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ae62452c-8e8c-442f-84f8-5f19dbe10178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cec7231f1c1445f965d620ae70e8463'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:51.831839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ae62452c-8e8c-442f-84f8-5f19dbe10178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cec7231f1c1445f965d620ae70e8463'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:51.866756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63466300-2690-4a5a-a064-0aebed1bd165 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:51.878642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc66a8-deb8-472c-8932-8a931fd07eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:51.879033 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1656/3313] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:10:51 2022] DELETE /identity/v3/users/0cec7231f1c1445f965d620ae70e8463 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:51.880415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc66a8-deb8-472c-8932-8a931fd07eed admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:51.880869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc66a8-deb8-472c-8932-8a931fd07eed admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:51.881150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc66a8-deb8-472c-8932-8a931fd07eed admin admin] PATH_INFO: `/v3/users/7f7bac0c0a2d4b1488d724f7d70e34e9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:51.882475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc66a8-deb8-472c-8932-8a931fd07eed admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f7bac0c0a2d4b1488d724f7d70e34e9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:51.884526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc66a8-deb8-472c-8932-8a931fd07eed admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:51.886921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecb33e-f4bc-43d6-9671-0ac1a05e54e3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:51.887220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecb33e-f4bc-43d6-9671-0ac1a05e54e3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:51.887663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecb33e-f4bc-43d6-9671-0ac1a05e54e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:51.922012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a7ecb33e-f4bc-43d6-9671-0ac1a05e54e3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:51.923574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a7ecb33e-f4bc-43d6-9671-0ac1a05e54e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:52.083124 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1657/3314] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:52.093442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecb33e-f4bc-43d6-9671-0ac1a05e54e3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:52.096364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a7ecb33e-f4bc-43d6-9671-0ac1a05e54e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:52.166195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecb33e-f4bc-43d6-9671-0ac1a05e54e3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:52.170868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce469415-6306-4bc6-ad78-44e4ce34116e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:52.172058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce469415-6306-4bc6-ad78-44e4ce34116e service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:52.172597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce469415-6306-4bc6-ad78-44e4ce34116e service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:52.173115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce469415-6306-4bc6-ad78-44e4ce34116e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:52.195415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce469415-6306-4bc6-ad78-44e4ce34116e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:52.198392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3fc66a8-deb8-472c-8932-8a931fd07eed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f7bac0c0a2d4b1488d724f7d70e34e9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:52.198691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce469415-6306-4bc6-ad78-44e4ce34116e service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:52.206691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:52.214659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3fc66a8-deb8-472c-8932-8a931fd07eed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f7bac0c0a2d4b1488d724f7d70e34e9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:52.215929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3fc66a8-deb8-472c-8932-8a931fd07eed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f7bac0c0a2d4b1488d724f7d70e34e9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:52.217159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3fc66a8-deb8-472c-8932-8a931fd07eed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f7bac0c0a2d4b1488d724f7d70e34e9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:52.218684 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1658/3315] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:52.358373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3fc66a8-deb8-472c-8932-8a931fd07eed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f7bac0c0a2d4b1488d724f7d70e34e9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:52.386650 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1658/3316] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:10:51 2022] DELETE /identity/v3/users/7f7bac0c0a2d4b1488d724f7d70e34e9 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:52.886480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce469415-6306-4bc6-ad78-44e4ce34116e service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:52.888567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ce469415-6306-4bc6-ad78-44e4ce34116e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:52.913434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc66a8-deb8-472c-8932-8a931fd07eed admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:52.914934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f3fc66a8-deb8-472c-8932-8a931fd07eed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:52.926455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce469415-6306-4bc6-ad78-44e4ce34116e service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:53.046232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc66a8-deb8-472c-8932-8a931fd07eed admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:53.057867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff758413-0403-4d9e-816a-a0faf899baf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:53.059292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff758413-0403-4d9e-816a-a0faf899baf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:53.059855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff758413-0403-4d9e-816a-a0faf899baf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:53.060408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff758413-0403-4d9e-816a-a0faf899baf3 admin admin] PATH_INFO: `/v3/projects/03390fc09db7409da2e77f2e430866f2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:53.062138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff758413-0403-4d9e-816a-a0faf899baf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03390fc09db7409da2e77f2e430866f2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:53.065044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff758413-0403-4d9e-816a-a0faf899baf3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:53.070036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:53.079372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c4ab3-8119-49d0-8722-9a74098b4835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:53.080823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c4ab3-8119-49d0-8722-9a74098b4835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:53.081344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c4ab3-8119-49d0-8722-9a74098b4835 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:53.081710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c4ab3-8119-49d0-8722-9a74098b4835 admin admin] PATH_INFO: `/v3/projects/9ccf44d7826045cfa5d4c6e87290d9d2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:53.083555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c4ab3-8119-49d0-8722-9a74098b4835 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ccf44d7826045cfa5d4c6e87290d9d2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:53.090074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c4ab3-8119-49d0-8722-9a74098b4835 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:53.232708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ff758413-0403-4d9e-816a-a0faf899baf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03390fc09db7409da2e77f2e430866f2', 'request_id': 'req-ff758413-0403-4d9e-816a-a0faf899baf3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:53.252694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8a2c4ab3-8119-49d0-8722-9a74098b4835 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ccf44d7826045cfa5d4c6e87290d9d2', 'request_id': 'req-8a2c4ab3-8119-49d0-8722-9a74098b4835'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:53.253358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ff758413-0403-4d9e-816a-a0faf899baf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03390fc09db7409da2e77f2e430866f2', 'request_id': 'req-ff758413-0403-4d9e-816a-a0faf899baf3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:53.274075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8a2c4ab3-8119-49d0-8722-9a74098b4835 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ccf44d7826045cfa5d4c6e87290d9d2', 'request_id': 'req-8a2c4ab3-8119-49d0-8722-9a74098b4835'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:53.356540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ff758413-0403-4d9e-816a-a0faf899baf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03390fc09db7409da2e77f2e430866f2', 'request_id': 'req-ff758413-0403-4d9e-816a-a0faf899baf3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:53.385441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ff758413-0403-4d9e-816a-a0faf899baf3 admin admin] The token cache is being invalidate because project 03390fc09db7409da2e77f2e430866f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:10:53.385778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ff758413-0403-4d9e-816a-a0faf899baf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:53.390211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ff758413-0403-4d9e-816a-a0faf899baf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:53.393498 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1659/3317] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:10:52 2022] DELETE /identity/v3/projects/03390fc09db7409da2e77f2e430866f2 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:53.398584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a81624-b3ea-475b-b705-de3f06e8cd22 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:53.398917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a81624-b3ea-475b-b705-de3f06e8cd22 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:53.399145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a81624-b3ea-475b-b705-de3f06e8cd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:53.401751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8a2c4ab3-8119-49d0-8722-9a74098b4835 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ccf44d7826045cfa5d4c6e87290d9d2', 'request_id': 'req-8a2c4ab3-8119-49d0-8722-9a74098b4835'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:53.416195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8a2c4ab3-8119-49d0-8722-9a74098b4835 admin admin] The token cache is being invalidate because project 9ccf44d7826045cfa5d4c6e87290d9d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:10:53.421511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8a2c4ab3-8119-49d0-8722-9a74098b4835 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:53.421511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8a2c4ab3-8119-49d0-8722-9a74098b4835 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:53.421511 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1659/3318] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:10:52 2022] DELETE /identity/v3/projects/9ccf44d7826045cfa5d4c6e87290d9d2 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:53.424411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c4ab3-8119-49d0-8722-9a74098b4835 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:53.426013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8a2c4ab3-8119-49d0-8722-9a74098b4835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:53.446456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-12a81624-b3ea-475b-b705-de3f06e8cd22 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:53.447894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-12a81624-b3ea-475b-b705-de3f06e8cd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:53.540396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c4ab3-8119-49d0-8722-9a74098b4835 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:53.545946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798fa36-c15e-4201-8a62-d8c716f30ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:53.546916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798fa36-c15e-4201-8a62-d8c716f30ab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:53.547288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798fa36-c15e-4201-8a62-d8c716f30ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:53.547832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798fa36-c15e-4201-8a62-d8c716f30ab3 admin admin] PATH_INFO: `/v3/users/4134af80aae942d69fc3dca0b65ce08c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:53.549287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e798fa36-c15e-4201-8a62-d8c716f30ab3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4134af80aae942d69fc3dca0b65ce08c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:53.551829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e798fa36-c15e-4201-8a62-d8c716f30ab3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:53.593604 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1660/3319] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:10:53.603496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f02f96-1c2b-4895-9b4b-918f36496ac1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:53.603496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f02f96-1c2b-4895-9b4b-918f36496ac1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:53.603496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f02f96-1c2b-4895-9b4b-918f36496ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:53.634025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c3f02f96-1c2b-4895-9b4b-918f36496ac1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:53.635368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c3f02f96-1c2b-4895-9b4b-918f36496ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:53.646784 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1661/3320] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:53.653613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f02f96-1c2b-4895-9b4b-918f36496ac1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:53.669038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f02f96-1c2b-4895-9b4b-918f36496ac1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:53.676176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621ca14-f8a5-460a-87f4-758eaf854ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:53.676976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621ca14-f8a5-460a-87f4-758eaf854ede admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:53.677276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621ca14-f8a5-460a-87f4-758eaf854ede admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:53.677552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621ca14-f8a5-460a-87f4-758eaf854ede admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:53.678329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621ca14-f8a5-460a-87f4-758eaf854ede admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:10:53.678637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621ca14-f8a5-460a-87f4-758eaf854ede admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:53.681424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b621ca14-f8a5-460a-87f4-758eaf854ede admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:53.698991 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1662/3321] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:10:53.705981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621ca14-f8a5-460a-87f4-758eaf854ede admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:53.720484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621ca14-f8a5-460a-87f4-758eaf854ede admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:53.739784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:53.754286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7becd-ab50-4f79-abe4-f2ec0bf8ecf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:53.755653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7becd-ab50-4f79-abe4-f2ec0bf8ecf6 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:53.756330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7becd-ab50-4f79-abe4-f2ec0bf8ecf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:53.756844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7becd-ab50-4f79-abe4-f2ec0bf8ecf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:53.758183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab7becd-ab50-4f79-abe4-f2ec0bf8ecf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:53.760681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab7becd-ab50-4f79-abe4-f2ec0bf8ecf6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:53.779815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e798fa36-c15e-4201-8a62-d8c716f30ab3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4134af80aae942d69fc3dca0b65ce08c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:53.780526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e798fa36-c15e-4201-8a62-d8c716f30ab3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4134af80aae942d69fc3dca0b65ce08c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:53.798516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e798fa36-c15e-4201-8a62-d8c716f30ab3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4134af80aae942d69fc3dca0b65ce08c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:53.945934 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1663/3322] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:10:53 2022] POST /identity/v3/projects => generated 403 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:10:53.950033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7becd-ab50-4f79-abe4-f2ec0bf8ecf6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:53.980183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e798fa36-c15e-4201-8a62-d8c716f30ab3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4134af80aae942d69fc3dca0b65ce08c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:53.982388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e798fa36-c15e-4201-8a62-d8c716f30ab3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4134af80aae942d69fc3dca0b65ce08c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:54.025890 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1660/3323] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:10:53 2022] DELETE /identity/v3/users/4134af80aae942d69fc3dca0b65ce08c => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:54.040902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:54.098788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7becd-ab50-4f79-abe4-f2ec0bf8ecf6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:54.196057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af0d39-a922-4960-b34e-e27448491194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:54.198876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af0d39-a922-4960-b34e-e27448491194 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:54.200563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af0d39-a922-4960-b34e-e27448491194 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:54.204700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af0d39-a922-4960-b34e-e27448491194 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:54.204700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af0d39-a922-4960-b34e-e27448491194 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:54.209211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af0d39-a922-4960-b34e-e27448491194 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:54.321224 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1664/3324] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:10:53 2022] POST /identity/v3/users => generated 302 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:10:54.322404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798fa36-c15e-4201-8a62-d8c716f30ab3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:54.325158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e798fa36-c15e-4201-8a62-d8c716f30ab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:54.363097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798fa36-c15e-4201-8a62-d8c716f30ab3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:54.380723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efdd76-0c05-4d1f-ad53-2ea5a3aa6c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:54.380723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efdd76-0c05-4d1f-ad53-2ea5a3aa6c83 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:54.385272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efdd76-0c05-4d1f-ad53-2ea5a3aa6c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:54.385915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efdd76-0c05-4d1f-ad53-2ea5a3aa6c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:54.388447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6efdd76-0c05-4d1f-ad53-2ea5a3aa6c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:54.392509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6efdd76-0c05-4d1f-ad53-2ea5a3aa6c83 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:54.447004 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1661/3325] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:10:54.448431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af0d39-a922-4960-b34e-e27448491194 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:54.459722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af0d39-a922-4960-b34e-e27448491194 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:54.474770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd107c59-176d-48cc-b602-e170a585b14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:54.476381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd107c59-176d-48cc-b602-e170a585b14f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:54.476937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd107c59-176d-48cc-b602-e170a585b14f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:54.477393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd107c59-176d-48cc-b602-e170a585b14f admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:54.478464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd107c59-176d-48cc-b602-e170a585b14f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:54.483286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd107c59-176d-48cc-b602-e170a585b14f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:54.507549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efdd76-0c05-4d1f-ad53-2ea5a3aa6c83 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:54.508755 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1665/3326] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:10:54.520831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efdd76-0c05-4d1f-ad53-2ea5a3aa6c83 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:54.529426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c58ba4-f37f-4242-8677-321d8b2f5ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:54.530562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c58ba4-f37f-4242-8677-321d8b2f5ae9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:54.530964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c58ba4-f37f-4242-8677-321d8b2f5ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:54.531397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c58ba4-f37f-4242-8677-321d8b2f5ae9 admin admin] PATH_INFO: `/v3/projects/d6a685bc40014ea89edf1dfa6ac3c659/users/4176cf2d27e64e92b3078d96c82671dd/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:54.555070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c58ba4-f37f-4242-8677-321d8b2f5ae9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6a685bc40014ea89edf1dfa6ac3c659, user_id=4176cf2d27e64e92b3078d96c82671dd, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:54.559750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c58ba4-f37f-4242-8677-321d8b2f5ae9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:54.652111 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1662/3327] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:10:54 2022] PUT /identity/v3/projects/d6a685bc40014ea89edf1dfa6ac3c659/users/4176cf2d27e64e92b3078d96c82671dd/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:54.657546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bf5cd-532c-484d-a900-a2cda7073fb9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:54.658182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bf5cd-532c-484d-a900-a2cda7073fb9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:54.658331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bf5cd-532c-484d-a900-a2cda7073fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:54.687535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9c2bf5cd-532c-484d-a900-a2cda7073fb9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:54.689140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9c2bf5cd-532c-484d-a900-a2cda7073fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:54.733052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c58ba4-f37f-4242-8677-321d8b2f5ae9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:54.734959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e7c58ba4-f37f-4242-8677-321d8b2f5ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:54.848120 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1666/3328] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:54.860966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2bf5cd-532c-484d-a900-a2cda7073fb9 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:54.864927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9c2bf5cd-532c-484d-a900-a2cda7073fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:54.910315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c58ba4-f37f-4242-8677-321d8b2f5ae9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:54.923642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74924d-9f95-4eac-b499-bab718f1eef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:54.924813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74924d-9f95-4eac-b499-bab718f1eef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:54.926390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74924d-9f95-4eac-b499-bab718f1eef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:54.926697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74924d-9f95-4eac-b499-bab718f1eef8 admin admin] PATH_INFO: `/v3/projects/af47af9b4c194d5cbfeef925a3681221` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:54.928836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74924d-9f95-4eac-b499-bab718f1eef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af47af9b4c194d5cbfeef925a3681221)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:54.931414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74924d-9f95-4eac-b499-bab718f1eef8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:55.020276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2bf5cd-532c-484d-a900-a2cda7073fb9 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:55.033102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c927f-eff4-43e2-b9e7-72236b730e67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:55.034038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c927f-eff4-43e2-b9e7-72236b730e67 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:55.034333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c927f-eff4-43e2-b9e7-72236b730e67 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:55.034619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c927f-eff4-43e2-b9e7-72236b730e67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:55.073127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c927f-eff4-43e2-b9e7-72236b730e67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:55.076537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c927f-eff4-43e2-b9e7-72236b730e67 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:55.092774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb74924d-9f95-4eac-b499-bab718f1eef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af47af9b4c194d5cbfeef925a3681221', 'request_id': 'req-fb74924d-9f95-4eac-b499-bab718f1eef8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:55.100254 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1667/3329] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:55.103725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb74924d-9f95-4eac-b499-bab718f1eef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af47af9b4c194d5cbfeef925a3681221', 'request_id': 'req-fb74924d-9f95-4eac-b499-bab718f1eef8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:55.210217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb74924d-9f95-4eac-b499-bab718f1eef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af47af9b4c194d5cbfeef925a3681221', 'request_id': 'req-fb74924d-9f95-4eac-b499-bab718f1eef8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:55.224601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb74924d-9f95-4eac-b499-bab718f1eef8 admin admin] The token cache is being invalidate because project af47af9b4c194d5cbfeef925a3681221 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:10:55.224601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb74924d-9f95-4eac-b499-bab718f1eef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:55.226665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb74924d-9f95-4eac-b499-bab718f1eef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:55.229330 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1663/3330] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:10:54 2022] DELETE /identity/v3/projects/af47af9b4c194d5cbfeef925a3681221 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:55.235016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9897bd-0d94-4942-a9a9-e3c04c5c3696 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:55.235307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9897bd-0d94-4942-a9a9-e3c04c5c3696 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:55.235595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9897bd-0d94-4942-a9a9-e3c04c5c3696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:55.275463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8e9897bd-0d94-4942-a9a9-e3c04c5c3696 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:55.277020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8e9897bd-0d94-4942-a9a9-e3c04c5c3696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:55.320619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:55.482042 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1668/3331] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:55.490322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9908f3-e11a-4862-bc36-0daac49a3772 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:55.492341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9908f3-e11a-4862-bc36-0daac49a3772 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:55.492855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9908f3-e11a-4862-bc36-0daac49a3772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:55.529659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:55.585082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-de9908f3-e11a-4862-bc36-0daac49a3772 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:55.585082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-de9908f3-e11a-4862-bc36-0daac49a3772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:55.585082 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1664/3332] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:10:55.589599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9897bd-0d94-4942-a9a9-e3c04c5c3696 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:55.603271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9897bd-0d94-4942-a9a9-e3c04c5c3696 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:55.610004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b11ba4-9d81-4f86-acf5-86780fc23a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:55.610762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b11ba4-9d81-4f86-acf5-86780fc23a56 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:55.611078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b11ba4-9d81-4f86-acf5-86780fc23a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:55.611495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b11ba4-9d81-4f86-acf5-86780fc23a56 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:55.612262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b11ba4-9d81-4f86-acf5-86780fc23a56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:10:55.612610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b11ba4-9d81-4f86-acf5-86780fc23a56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:55.614990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b11ba4-9d81-4f86-acf5-86780fc23a56 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:55.633751 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1669/3333] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:10:55.634888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9908f3-e11a-4862-bc36-0daac49a3772 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:55.645902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9908f3-e11a-4862-bc36-0daac49a3772 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:55.653830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f89787-7efc-4893-aead-b92a9e49e440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:55.654328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f89787-7efc-4893-aead-b92a9e49e440 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:55.654623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f89787-7efc-4893-aead-b92a9e49e440 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:55.654914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f89787-7efc-4893-aead-b92a9e49e440 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:55.655628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f89787-7efc-4893-aead-b92a9e49e440 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:55.658240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f89787-7efc-4893-aead-b92a9e49e440 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:55.859264 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1665/3334] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:10:55 2022] POST /identity/v3/projects => generated 393 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:10:55.864158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b11ba4-9d81-4f86-acf5-86780fc23a56 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:55.878970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b11ba4-9d81-4f86-acf5-86780fc23a56 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:55.983104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6edde-c515-4d26-886e-25ee2f83b4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:55.984177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6edde-c515-4d26-886e-25ee2f83b4f2 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:55.984569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6edde-c515-4d26-886e-25ee2f83b4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:55.984950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6edde-c515-4d26-886e-25ee2f83b4f2 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:55.986191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e6edde-c515-4d26-886e-25ee2f83b4f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:55.988935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e6edde-c515-4d26-886e-25ee2f83b4f2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:56.097041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f89787-7efc-4893-aead-b92a9e49e440 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:56.097585 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1670/3335] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:10:55 2022] POST /identity/v3/users => generated 297 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:10:56.112452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f89787-7efc-4893-aead-b92a9e49e440 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:56.125368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43982f-9a3d-4a23-b822-a1e23b7fb2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:56.126107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43982f-9a3d-4a23-b822-a1e23b7fb2d7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:56.126395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43982f-9a3d-4a23-b822-a1e23b7fb2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:56.126675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43982f-9a3d-4a23-b822-a1e23b7fb2d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:56.127452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43982f-9a3d-4a23-b822-a1e23b7fb2d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:56.129086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f43982f-9a3d-4a23-b822-a1e23b7fb2d7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:56.154607 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1666/3336] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:56 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:10:56.158880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6edde-c515-4d26-886e-25ee2f83b4f2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:56.183965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6edde-c515-4d26-886e-25ee2f83b4f2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:56.196440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497ffed-4812-4ff2-ac39-71d7b014d18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:56.197445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497ffed-4812-4ff2-ac39-71d7b014d18d admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:56.197926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497ffed-4812-4ff2-ac39-71d7b014d18d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:56.198713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497ffed-4812-4ff2-ac39-71d7b014d18d admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:56.201213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497ffed-4812-4ff2-ac39-71d7b014d18d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:56.204755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497ffed-4812-4ff2-ac39-71d7b014d18d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:56.229610 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1671/3337] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:56 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:10:56.230548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43982f-9a3d-4a23-b822-a1e23b7fb2d7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:56.248447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f43982f-9a3d-4a23-b822-a1e23b7fb2d7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:56.260997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad423d6f-fe62-4263-865d-553792b82f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:56.261940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad423d6f-fe62-4263-865d-553792b82f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:56.262243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad423d6f-fe62-4263-865d-553792b82f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:56.262502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad423d6f-fe62-4263-865d-553792b82f71 admin admin] PATH_INFO: `/v3/projects/5d0c7dfc47114dffaf4c661dc3d519ab/users/293ff74be48d4ae2a5b59ace7bf620c6/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:56.290634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad423d6f-fe62-4263-865d-553792b82f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d0c7dfc47114dffaf4c661dc3d519ab, user_id=293ff74be48d4ae2a5b59ace7bf620c6, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:56.293727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad423d6f-fe62-4263-865d-553792b82f71 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:56.447851 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1667/3338] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:10:56 2022] PUT /identity/v3/projects/5d0c7dfc47114dffaf4c661dc3d519ab/users/293ff74be48d4ae2a5b59ace7bf620c6/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:10:56.459763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f865ec6-9831-441f-b3f7-bb77fddf3482 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:56.460460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f865ec6-9831-441f-b3f7-bb77fddf3482 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:56.460830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f865ec6-9831-441f-b3f7-bb77fddf3482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:56.500857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1f865ec6-9831-441f-b3f7-bb77fddf3482 None None] MFA Rules not processed for user `293ff74be48d4ae2a5b59ace7bf620c6`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:56.503490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1f865ec6-9831-441f-b3f7-bb77fddf3482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:56.680819 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1672/3339] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:56.685917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad423d6f-fe62-4263-865d-553792b82f71 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:56.821188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad423d6f-fe62-4263-865d-553792b82f71 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:56.827145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394fc35-60d8-4f3f-9cd8-f8682184e914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:56.827961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394fc35-60d8-4f3f-9cd8-f8682184e914 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:56.828318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394fc35-60d8-4f3f-9cd8-f8682184e914 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:56.828707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394fc35-60d8-4f3f-9cd8-f8682184e914 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:56.830363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394fc35-60d8-4f3f-9cd8-f8682184e914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:56.832522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394fc35-60d8-4f3f-9cd8-f8682184e914 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:57.008744 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1668/3340] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:10:56 2022] POST /identity/v3/projects => generated 391 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:10:57.010036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f865ec6-9831-441f-b3f7-bb77fddf3482 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:57.025957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f865ec6-9831-441f-b3f7-bb77fddf3482 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:57.040465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:57.158081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7323475-639d-4574-b2ca-f79edeeaa858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:57.159380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7323475-639d-4574-b2ca-f79edeeaa858 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:57.159687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7323475-639d-4574-b2ca-f79edeeaa858 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:57.159969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7323475-639d-4574-b2ca-f79edeeaa858 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:57.160674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7323475-639d-4574-b2ca-f79edeeaa858 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:57.162663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7323475-639d-4574-b2ca-f79edeeaa858 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:57.265938 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1673/3341] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:10:57 2022] POST /identity/v3/users => generated 302 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:10:57.271270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394fc35-60d8-4f3f-9cd8-f8682184e914 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:57.292008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394fc35-60d8-4f3f-9cd8-f8682184e914 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:57.305910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b38bb3-7b90-4f78-83e7-5070684ee4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:57.307184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b38bb3-7b90-4f78-83e7-5070684ee4ab admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:57.307409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b38bb3-7b90-4f78-83e7-5070684ee4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:57.307883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b38bb3-7b90-4f78-83e7-5070684ee4ab admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:57.309233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b38bb3-7b90-4f78-83e7-5070684ee4ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:57.312759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b38bb3-7b90-4f78-83e7-5070684ee4ab admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:57.326990 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1669/3342] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:10:57.331230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7323475-639d-4574-b2ca-f79edeeaa858 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:57.342504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7323475-639d-4574-b2ca-f79edeeaa858 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:57.347275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb62cd-f444-4841-835f-43f75a651a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:57.348259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb62cd-f444-4841-835f-43f75a651a65 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:57.348889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb62cd-f444-4841-835f-43f75a651a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:57.349176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb62cd-f444-4841-835f-43f75a651a65 admin admin] PATH_INFO: `/v3/projects/2aec49aa1d8347eb9d735d79edbdc0b6/users/61bcd16913c64e9faecb3dea768cd8e1/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:57.367781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb62cd-f444-4841-835f-43f75a651a65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aec49aa1d8347eb9d735d79edbdc0b6, user_id=61bcd16913c64e9faecb3dea768cd8e1, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:57.371515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb62cd-f444-4841-835f-43f75a651a65 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:57.477092 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1674/3343] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:10:57 2022] PUT /identity/v3/projects/2aec49aa1d8347eb9d735d79edbdc0b6/users/61bcd16913c64e9faecb3dea768cd8e1/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:57.480906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b38bb3-7b90-4f78-83e7-5070684ee4ab admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:57.495055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b38bb3-7b90-4f78-83e7-5070684ee4ab admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:57.565221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ecee9-6027-444b-86dd-3cecdd0fb1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:57.566286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ecee9-6027-444b-86dd-3cecdd0fb1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:57.566707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ecee9-6027-444b-86dd-3cecdd0fb1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:57.567027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ecee9-6027-444b-86dd-3cecdd0fb1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:57.567848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ecee9-6027-444b-86dd-3cecdd0fb1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:57.569555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9ecee9-6027-444b-86dd-3cecdd0fb1c1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:57.588407 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1670/3344] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:10:57.591940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb62cd-f444-4841-835f-43f75a651a65 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:57.627814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb62cd-f444-4841-835f-43f75a651a65 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:57.633640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d65f5f-35ca-4402-b043-90a30faa272c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:57.635212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d65f5f-35ca-4402-b043-90a30faa272c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:57.636090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d65f5f-35ca-4402-b043-90a30faa272c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:57.636435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d65f5f-35ca-4402-b043-90a30faa272c admin admin] PATH_INFO: `/v3/projects/2aec49aa1d8347eb9d735d79edbdc0b6/users/61bcd16913c64e9faecb3dea768cd8e1/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:57.639246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d65f5f-35ca-4402-b043-90a30faa272c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aec49aa1d8347eb9d735d79edbdc0b6, user_id=61bcd16913c64e9faecb3dea768cd8e1, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:57.642732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d65f5f-35ca-4402-b043-90a30faa272c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:57.663085 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1675/3345] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:10:57 2022] PUT /identity/v3/projects/2aec49aa1d8347eb9d735d79edbdc0b6/users/61bcd16913c64e9faecb3dea768cd8e1/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:10:57.673794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206ec0-65e0-487a-b9d7-96c5073ea867 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:57.674139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206ec0-65e0-487a-b9d7-96c5073ea867 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:57.674434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6206ec0-65e0-487a-b9d7-96c5073ea867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:57.695499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:57.705400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f6206ec0-65e0-487a-b9d7-96c5073ea867 None None] MFA Rules not processed for user `61bcd16913c64e9faecb3dea768cd8e1`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:57.707114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f6206ec0-65e0-487a-b9d7-96c5073ea867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:57.809475 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1671/3346] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:10:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:10:57.821721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d65f5f-35ca-4402-b043-90a30faa272c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:57.823523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-16d65f5f-35ca-4402-b043-90a30faa272c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:57.968235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d65f5f-35ca-4402-b043-90a30faa272c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:57.987768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79164ad1-957b-4229-a34c-6f37bf093cca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:57.988896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79164ad1-957b-4229-a34c-6f37bf093cca service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:57.989190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79164ad1-957b-4229-a34c-6f37bf093cca service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:57.989467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79164ad1-957b-4229-a34c-6f37bf093cca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:58.014352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79164ad1-957b-4229-a34c-6f37bf093cca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:58.016751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79164ad1-957b-4229-a34c-6f37bf093cca service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:58.043445 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1676/3347] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:59.365972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6206ec0-65e0-487a-b9d7-96c5073ea867 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:59.367505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f6206ec0-65e0-487a-b9d7-96c5073ea867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:59.393098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6206ec0-65e0-487a-b9d7-96c5073ea867 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:59.400952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d0271-cad8-4cea-92bd-a525d9cbf40a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:59.402165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d0271-cad8-4cea-92bd-a525d9cbf40a service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:59.402654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d0271-cad8-4cea-92bd-a525d9cbf40a service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:59.403647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d0271-cad8-4cea-92bd-a525d9cbf40a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:59.518057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d0271-cad8-4cea-92bd-a525d9cbf40a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:59.520248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d0271-cad8-4cea-92bd-a525d9cbf40a service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:59.549030 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1672/3348] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:10:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:10:59.553299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79164ad1-957b-4229-a34c-6f37bf093cca service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:59.664210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac5417-93fe-43fb-89f8-9c9b64f97a52 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:59.664414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:59.664914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac5417-93fe-43fb-89f8-9c9b64f97a52 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:59.665401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac5417-93fe-43fb-89f8-9c9b64f97a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:59.685964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79164ad1-957b-4229-a34c-6f37bf093cca service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:59.686852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-97ac5417-93fe-43fb-89f8-9c9b64f97a52 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:10:59.688489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-97ac5417-93fe-43fb-89f8-9c9b64f97a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:59.691121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcdcc6-b8ab-4411-942c-6c937c2ecce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:59.692077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcdcc6-b8ab-4411-942c-6c937c2ecce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:59.692370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcdcc6-b8ab-4411-942c-6c937c2ecce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:59.692647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfcdcc6-b8ab-4411-942c-6c937c2ecce5 admin admin] PATH_INFO: `/v3/users/293ff74be48d4ae2a5b59ace7bf620c6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:59.706640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcdcc6-b8ab-4411-942c-6c937c2ecce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=293ff74be48d4ae2a5b59ace7bf620c6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:59.706640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfcdcc6-b8ab-4411-942c-6c937c2ecce5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:59.785652 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1673/3349] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:10:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:10:59.787151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac5417-93fe-43fb-89f8-9c9b64f97a52 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:10:59.788825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-97ac5417-93fe-43fb-89f8-9c9b64f97a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:59.843927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:59.873089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dbfcdcc6-b8ab-4411-942c-6c937c2ecce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '293ff74be48d4ae2a5b59ace7bf620c6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:59.888409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dbfcdcc6-b8ab-4411-942c-6c937c2ecce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '293ff74be48d4ae2a5b59ace7bf620c6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:59.889158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dbfcdcc6-b8ab-4411-942c-6c937c2ecce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '293ff74be48d4ae2a5b59ace7bf620c6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:59.889309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac5417-93fe-43fb-89f8-9c9b64f97a52 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:10:59.893067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dbfcdcc6-b8ab-4411-942c-6c937c2ecce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '293ff74be48d4ae2a5b59ace7bf620c6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:10:59.900364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470f2bf7-f2d4-4f1f-b52c-1302b9c56424 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:10:59.901666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f2bf7-f2d4-4f1f-b52c-1302b9c56424 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:10:59.902262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f2bf7-f2d4-4f1f-b52c-1302b9c56424 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:10:59.902891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f2bf7-f2d4-4f1f-b52c-1302b9c56424 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:10:59.906936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-470f2bf7-f2d4-4f1f-b52c-1302b9c56424 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:10:59.922133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f2bf7-f2d4-4f1f-b52c-1302b9c56424 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:10:59.924550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470f2bf7-f2d4-4f1f-b52c-1302b9c56424 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:10:59.937518 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1674/3350] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:10:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:11:00.003198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dbfcdcc6-b8ab-4411-942c-6c937c2ecce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '293ff74be48d4ae2a5b59ace7bf620c6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:00.044806 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1677/3351] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:10:59 2022] DELETE /identity/v3/users/293ff74be48d4ae2a5b59ace7bf620c6 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:00.049467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35a83c-ff09-4b6d-8db4-041182d12fdf None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:00.049977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35a83c-ff09-4b6d-8db4-041182d12fdf None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:00.050723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d35a83c-ff09-4b6d-8db4-041182d12fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:00.096629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0d35a83c-ff09-4b6d-8db4-041182d12fdf None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:00.104264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0d35a83c-ff09-4b6d-8db4-041182d12fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:00.265408 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1675/3352] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:00.314835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcdcc6-b8ab-4411-942c-6c937c2ecce5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:00.316394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dbfcdcc6-b8ab-4411-942c-6c937c2ecce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:00.464997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfcdcc6-b8ab-4411-942c-6c937c2ecce5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:00.475779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fcdf4-085a-4d50-a20d-2587d26e6887 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:00.476144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fcdf4-085a-4d50-a20d-2587d26e6887 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:00.476507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fcdf4-085a-4d50-a20d-2587d26e6887 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:00.476854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fcdf4-085a-4d50-a20d-2587d26e6887 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:00.504101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9fcdf4-085a-4d50-a20d-2587d26e6887 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:00.504101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9fcdf4-085a-4d50-a20d-2587d26e6887 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:00.514805 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1678/3353] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:01.882486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159f814-070e-4084-bd80-29e5d63e5a63 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:01.882989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159f814-070e-4084-bd80-29e5d63e5a63 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:01.883104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159f814-070e-4084-bd80-29e5d63e5a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:01.917684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6159f814-070e-4084-bd80-29e5d63e5a63 None None] MFA Rules not processed for user `04e5b86f94914df085b76e0d89b2230e`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:01.919691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6159f814-070e-4084-bd80-29e5d63e5a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:02.050460 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1676/3354] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:01 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:02.520175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fcdf4-085a-4d50-a20d-2587d26e6887 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:02.522388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0d9fcdf4-085a-4d50-a20d-2587d26e6887 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:02.545163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fcdf4-085a-4d50-a20d-2587d26e6887 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:02.561404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681d7ef-8297-4a34-8695-22d81d955528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:02.562601 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681d7ef-8297-4a34-8695-22d81d955528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:02.563110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681d7ef-8297-4a34-8695-22d81d955528 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:02.563609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681d7ef-8297-4a34-8695-22d81d955528 admin admin] PATH_INFO: `/v3/projects/5d0c7dfc47114dffaf4c661dc3d519ab` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:02.565081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681d7ef-8297-4a34-8695-22d81d955528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d0c7dfc47114dffaf4c661dc3d519ab)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:02.576504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681d7ef-8297-4a34-8695-22d81d955528 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:02.695012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2681d7ef-8297-4a34-8695-22d81d955528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d0c7dfc47114dffaf4c661dc3d519ab', 'request_id': 'req-2681d7ef-8297-4a34-8695-22d81d955528'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:02.701100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2681d7ef-8297-4a34-8695-22d81d955528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d0c7dfc47114dffaf4c661dc3d519ab', 'request_id': 'req-2681d7ef-8297-4a34-8695-22d81d955528'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:02.774389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2681d7ef-8297-4a34-8695-22d81d955528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d0c7dfc47114dffaf4c661dc3d519ab', 'request_id': 'req-2681d7ef-8297-4a34-8695-22d81d955528'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:02.779514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:02.795684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2681d7ef-8297-4a34-8695-22d81d955528 admin admin] The token cache is being invalidate because project 5d0c7dfc47114dffaf4c661dc3d519ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:11:02.796243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2681d7ef-8297-4a34-8695-22d81d955528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:02.796920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2681d7ef-8297-4a34-8695-22d81d955528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:02.798734 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1679/3355] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:11:02 2022] DELETE /identity/v3/projects/5d0c7dfc47114dffaf4c661dc3d519ab => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:02.803087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159f814-070e-4084-bd80-29e5d63e5a63 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:02.820735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159f814-070e-4084-bd80-29e5d63e5a63 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:02.890508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e3dbe-308b-4500-b31b-147b8fc3a9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:02.892160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e3dbe-308b-4500-b31b-147b8fc3a9b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:02.892442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e3dbe-308b-4500-b31b-147b8fc3a9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:02.892718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e3dbe-308b-4500-b31b-147b8fc3a9b5 admin admin] PATH_INFO: `/v3/users/61bcd16913c64e9faecb3dea768cd8e1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:02.894057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e3dbe-308b-4500-b31b-147b8fc3a9b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61bcd16913c64e9faecb3dea768cd8e1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:02.896099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e3dbe-308b-4500-b31b-147b8fc3a9b5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:02.911413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:03.079277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b93e3dbe-308b-4500-b31b-147b8fc3a9b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61bcd16913c64e9faecb3dea768cd8e1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:03.082644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b93e3dbe-308b-4500-b31b-147b8fc3a9b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61bcd16913c64e9faecb3dea768cd8e1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:03.097422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b93e3dbe-308b-4500-b31b-147b8fc3a9b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61bcd16913c64e9faecb3dea768cd8e1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:03.202703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b93e3dbe-308b-4500-b31b-147b8fc3a9b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61bcd16913c64e9faecb3dea768cd8e1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:03.204042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b93e3dbe-308b-4500-b31b-147b8fc3a9b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61bcd16913c64e9faecb3dea768cd8e1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:03.262122 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1677/3356] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:11:02 2022] DELETE /identity/v3/users/61bcd16913c64e9faecb3dea768cd8e1 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:05.464283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681d7ef-8297-4a34-8695-22d81d955528 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:05.474282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2681d7ef-8297-4a34-8695-22d81d955528 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:05.509554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681d7ef-8297-4a34-8695-22d81d955528 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:05.664506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9539ea-b418-4dc6-bf80-be5eb400ee44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:05.665584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9539ea-b418-4dc6-bf80-be5eb400ee44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:05.666035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9539ea-b418-4dc6-bf80-be5eb400ee44 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:05.666421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9539ea-b418-4dc6-bf80-be5eb400ee44 admin admin] PATH_INFO: `/v3/projects/2aec49aa1d8347eb9d735d79edbdc0b6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:05.667968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9539ea-b418-4dc6-bf80-be5eb400ee44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aec49aa1d8347eb9d735d79edbdc0b6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:05.670732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9539ea-b418-4dc6-bf80-be5eb400ee44 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:05.895165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cf9539ea-b418-4dc6-bf80-be5eb400ee44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aec49aa1d8347eb9d735d79edbdc0b6', 'request_id': 'req-cf9539ea-b418-4dc6-bf80-be5eb400ee44'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:05.902421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cf9539ea-b418-4dc6-bf80-be5eb400ee44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aec49aa1d8347eb9d735d79edbdc0b6', 'request_id': 'req-cf9539ea-b418-4dc6-bf80-be5eb400ee44'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:05.987254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cf9539ea-b418-4dc6-bf80-be5eb400ee44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aec49aa1d8347eb9d735d79edbdc0b6', 'request_id': 'req-cf9539ea-b418-4dc6-bf80-be5eb400ee44'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:06.009974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cf9539ea-b418-4dc6-bf80-be5eb400ee44 admin admin] The token cache is being invalidate because project 2aec49aa1d8347eb9d735d79edbdc0b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:11:06.010611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cf9539ea-b418-4dc6-bf80-be5eb400ee44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:06.011638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cf9539ea-b418-4dc6-bf80-be5eb400ee44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:06.014145 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1680/3357] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:11:05 2022] DELETE /identity/v3/projects/2aec49aa1d8347eb9d735d79edbdc0b6 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:06.022451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72038019-468e-4c59-9a36-ec35b609874a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:06.022451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72038019-468e-4c59-9a36-ec35b609874a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:06.022451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72038019-468e-4c59-9a36-ec35b609874a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:06.042873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-72038019-468e-4c59-9a36-ec35b609874a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:06.044732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-72038019-468e-4c59-9a36-ec35b609874a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:06.171769 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1678/3358] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:06.180626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac0a6a-d376-41c1-80ea-06794ec5c64f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:06.181473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac0a6a-d376-41c1-80ea-06794ec5c64f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:06.181771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cac0a6a-d376-41c1-80ea-06794ec5c64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:06.220929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6cac0a6a-d376-41c1-80ea-06794ec5c64f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:06.223062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6cac0a6a-d376-41c1-80ea-06794ec5c64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:06.234911 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1681/3359] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:06.240509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72038019-468e-4c59-9a36-ec35b609874a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:06.253661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72038019-468e-4c59-9a36-ec35b609874a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:06.260986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7f9f5-385d-47e6-8c36-6e2ee1dfb7b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:06.262253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7f9f5-385d-47e6-8c36-6e2ee1dfb7b5 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:06.263022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7f9f5-385d-47e6-8c36-6e2ee1dfb7b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:06.267487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7f9f5-385d-47e6-8c36-6e2ee1dfb7b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:06.267487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7f9f5-385d-47e6-8c36-6e2ee1dfb7b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:11:06.267487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7f9f5-385d-47e6-8c36-6e2ee1dfb7b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:06.267487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7f9f5-385d-47e6-8c36-6e2ee1dfb7b5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:06.305390 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1679/3360] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:11:06.311629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac0a6a-d376-41c1-80ea-06794ec5c64f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:06.323384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cac0a6a-d376-41c1-80ea-06794ec5c64f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:06.329029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2e500-c438-4b03-81a4-221138ead218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:06.330067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2e500-c438-4b03-81a4-221138ead218 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:06.330461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2e500-c438-4b03-81a4-221138ead218 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:06.330986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2e500-c438-4b03-81a4-221138ead218 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:06.331865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2e500-c438-4b03-81a4-221138ead218 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:06.333968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2e500-c438-4b03-81a4-221138ead218 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:06.453459 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1682/3361] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:11:06 2022] POST /identity/v3/projects => generated 395 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:11:06.458462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7f9f5-385d-47e6-8c36-6e2ee1dfb7b5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:06.475653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7f9f5-385d-47e6-8c36-6e2ee1dfb7b5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:06.563572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01d907-9a53-47f3-8ed0-3049b0366e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:06.563572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01d907-9a53-47f3-8ed0-3049b0366e6c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:06.563572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01d907-9a53-47f3-8ed0-3049b0366e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:06.563876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01d907-9a53-47f3-8ed0-3049b0366e6c admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:06.564856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01d907-9a53-47f3-8ed0-3049b0366e6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:06.566282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01d907-9a53-47f3-8ed0-3049b0366e6c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:06.637198 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1680/3362] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:11:06 2022] POST /identity/v3/users => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:06.642856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2e500-c438-4b03-81a4-221138ead218 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:06.656498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2e500-c438-4b03-81a4-221138ead218 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:06.661642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd097f-d492-499d-999c-794a931c41ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:06.662747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd097f-d492-499d-999c-794a931c41ef admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:06.668743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd097f-d492-499d-999c-794a931c41ef admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:06.668743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd097f-d492-499d-999c-794a931c41ef admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:06.668743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd097f-d492-499d-999c-794a931c41ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:06.669505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd097f-d492-499d-999c-794a931c41ef admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:06.686033 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1683/3363] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:11:06.687138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01d907-9a53-47f3-8ed0-3049b0366e6c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:06.703821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01d907-9a53-47f3-8ed0-3049b0366e6c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:06.710727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f8211-ad26-4f1b-834c-73f4bfb56215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:06.712149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f8211-ad26-4f1b-834c-73f4bfb56215 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:06.712593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f8211-ad26-4f1b-834c-73f4bfb56215 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:06.712935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f8211-ad26-4f1b-834c-73f4bfb56215 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:06.714450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f8211-ad26-4f1b-834c-73f4bfb56215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:06.717427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f8211-ad26-4f1b-834c-73f4bfb56215 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:06.720852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:06.748421 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1681/3364] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:11:06.753392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd097f-d492-499d-999c-794a931c41ef admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:06.784938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd097f-d492-499d-999c-794a931c41ef admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:06.806180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad881be-e2a6-4b51-920a-a80325e4c4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:06.807121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad881be-e2a6-4b51-920a-a80325e4c4dd admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:06.807408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad881be-e2a6-4b51-920a-a80325e4c4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:06.807680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad881be-e2a6-4b51-920a-a80325e4c4dd admin admin] PATH_INFO: `/v3/projects/9d08be7a2d1e4b779987d75d87a5ef44/users/04ae3115a9194b47bce62ada4a2afdf2/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:06.862677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad881be-e2a6-4b51-920a-a80325e4c4dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d08be7a2d1e4b779987d75d87a5ef44, user_id=04ae3115a9194b47bce62ada4a2afdf2, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:06.865713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad881be-e2a6-4b51-920a-a80325e4c4dd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:07.084031 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1684/3365] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:11:06 2022] PUT /identity/v3/projects/9d08be7a2d1e4b779987d75d87a5ef44/users/04ae3115a9194b47bce62ada4a2afdf2/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:07.085631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a886163-667e-4f10-98d2-dbbfe758f9cd None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:07.086199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a886163-667e-4f10-98d2-dbbfe758f9cd None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:07.086199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a886163-667e-4f10-98d2-dbbfe758f9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:07.118651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0a886163-667e-4f10-98d2-dbbfe758f9cd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:07.120781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0a886163-667e-4f10-98d2-dbbfe758f9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:07.190740 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1682/3366] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:07.202969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad881be-e2a6-4b51-920a-a80325e4c4dd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:07.205664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-aad881be-e2a6-4b51-920a-a80325e4c4dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:07.243481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:07.291124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad881be-e2a6-4b51-920a-a80325e4c4dd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:07.296539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdd0ec-13a4-4c03-a772-4cfa4c51aa07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:07.297265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdd0ec-13a4-4c03-a772-4cfa4c51aa07 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:07.297554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdd0ec-13a4-4c03-a772-4cfa4c51aa07 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:07.297830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdd0ec-13a4-4c03-a772-4cfa4c51aa07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:07.312158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabdd0ec-13a4-4c03-a772-4cfa4c51aa07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:07.314705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabdd0ec-13a4-4c03-a772-4cfa4c51aa07 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:07.328555 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1685/3367] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:11:08.550664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a33917-3d40-4f33-9977-274723b014f4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:08.551708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a33917-3d40-4f33-9977-274723b014f4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:08.552251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a33917-3d40-4f33-9977-274723b014f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:08.617573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-93a33917-3d40-4f33-9977-274723b014f4 None None] MFA Rules not processed for user `4176cf2d27e64e92b3078d96c82671dd`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:08.619285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-93a33917-3d40-4f33-9977-274723b014f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:08.744239 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1683/3368] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:08.747100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdd0ec-13a4-4c03-a772-4cfa4c51aa07 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:08.748704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eabdd0ec-13a4-4c03-a772-4cfa4c51aa07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:08.879737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdd0ec-13a4-4c03-a772-4cfa4c51aa07 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:08.889572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea74967-5871-4df8-9d01-89dda34a7423 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:08.892495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea74967-5871-4df8-9d01-89dda34a7423 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:08.892900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea74967-5871-4df8-9d01-89dda34a7423 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:08.893428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea74967-5871-4df8-9d01-89dda34a7423 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:08.923724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea74967-5871-4df8-9d01-89dda34a7423 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:08.926455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea74967-5871-4df8-9d01-89dda34a7423 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:08.945653 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1686/3369] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:13.163900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17032cea-f090-468a-a962-fa16f896d36d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:13.163900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17032cea-f090-468a-a962-fa16f896d36d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:13.163900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17032cea-f090-468a-a962-fa16f896d36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:13.207982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-17032cea-f090-468a-a962-fa16f896d36d None None] MFA Rules not processed for user `bf9c8f66e7d24ca9baca835e41546f22`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:13.209361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-17032cea-f090-468a-a962-fa16f896d36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:13.408450 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1684/3370] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:18.936676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=50 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:20.764501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea74967-5871-4df8-9d01-89dda34a7423 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:20.766155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1ea74967-5871-4df8-9d01-89dda34a7423 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:20.789879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea74967-5871-4df8-9d01-89dda34a7423 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:20.800921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d2351-e805-4a09-8ae3-4b4b81fabf4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:20.802173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d2351-e805-4a09-8ae3-4b4b81fabf4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:20.802802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d2351-e805-4a09-8ae3-4b4b81fabf4f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:20.803267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d2351-e805-4a09-8ae3-4b4b81fabf4f admin admin] PATH_INFO: `/v3/users/4176cf2d27e64e92b3078d96c82671dd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:20.805688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d2351-e805-4a09-8ae3-4b4b81fabf4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4176cf2d27e64e92b3078d96c82671dd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:20.816543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d2351-e805-4a09-8ae3-4b4b81fabf4f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:21.094951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e78d2351-e805-4a09-8ae3-4b4b81fabf4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4176cf2d27e64e92b3078d96c82671dd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:21.110903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e78d2351-e805-4a09-8ae3-4b4b81fabf4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4176cf2d27e64e92b3078d96c82671dd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:21.111862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e78d2351-e805-4a09-8ae3-4b4b81fabf4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4176cf2d27e64e92b3078d96c82671dd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:21.112667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e78d2351-e805-4a09-8ae3-4b4b81fabf4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4176cf2d27e64e92b3078d96c82671dd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:21.218701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e78d2351-e805-4a09-8ae3-4b4b81fabf4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4176cf2d27e64e92b3078d96c82671dd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:21.265627 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1687/3371] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:11:20 2022] DELETE /identity/v3/users/4176cf2d27e64e92b3078d96c82671dd => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:21.371846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b208e7-bcba-45fa-98b7-085fa38cd8db None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:21.372312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b208e7-bcba-45fa-98b7-085fa38cd8db None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:21.372312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b208e7-bcba-45fa-98b7-085fa38cd8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:21.402903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f0b208e7-bcba-45fa-98b7-085fa38cd8db None None] MFA Rules not processed for user `04ae3115a9194b47bce62ada4a2afdf2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:21.404730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f0b208e7-bcba-45fa-98b7-085fa38cd8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:21.451564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:21.537968 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1685/3372] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:21.541170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1f862-172d-4663-95a9-b3f40d24b045 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:21.541650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1f862-172d-4663-95a9-b3f40d24b045 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:21.542111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1f862-172d-4663-95a9-b3f40d24b045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:21.583483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9ed1f862-172d-4663-95a9-b3f40d24b045 None None] MFA Rules not processed for user `04ae3115a9194b47bce62ada4a2afdf2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:21.586896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9ed1f862-172d-4663-95a9-b3f40d24b045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:21.606283 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1688/3373] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:21.821879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b208e7-bcba-45fa-98b7-085fa38cd8db None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:21.823439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f0b208e7-bcba-45fa-98b7-085fa38cd8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:22.005506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b208e7-bcba-45fa-98b7-085fa38cd8db None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:22.025400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d781ba75-71e1-49f8-9583-f1c2493be027 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:22.026111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781ba75-71e1-49f8-9583-f1c2493be027 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:22.026394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781ba75-71e1-49f8-9583-f1c2493be027 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:22.026670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d781ba75-71e1-49f8-9583-f1c2493be027 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:22.049453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d781ba75-71e1-49f8-9583-f1c2493be027 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:22.052858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d781ba75-71e1-49f8-9583-f1c2493be027 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:22.083561 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1686/3374] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:22.344645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1f862-172d-4663-95a9-b3f40d24b045 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:22.346167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9ed1f862-172d-4663-95a9-b3f40d24b045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:22.510489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1f862-172d-4663-95a9-b3f40d24b045 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:22.530315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a7bd2-fa5f-4f19-9ce0-c26a3092aaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:22.530315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a7bd2-fa5f-4f19-9ce0-c26a3092aaec admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:22.530862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a7bd2-fa5f-4f19-9ce0-c26a3092aaec admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:22.533912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a7bd2-fa5f-4f19-9ce0-c26a3092aaec admin admin] PATH_INFO: `/v3/projects/d6a685bc40014ea89edf1dfa6ac3c659` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:22.541498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a7bd2-fa5f-4f19-9ce0-c26a3092aaec admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6a685bc40014ea89edf1dfa6ac3c659)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:22.548512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a7bd2-fa5f-4f19-9ce0-c26a3092aaec admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:22.777517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c22a7bd2-fa5f-4f19-9ce0-c26a3092aaec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6a685bc40014ea89edf1dfa6ac3c659', 'request_id': 'req-c22a7bd2-fa5f-4f19-9ce0-c26a3092aaec'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:22.805202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c22a7bd2-fa5f-4f19-9ce0-c26a3092aaec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6a685bc40014ea89edf1dfa6ac3c659', 'request_id': 'req-c22a7bd2-fa5f-4f19-9ce0-c26a3092aaec'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:22.944262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c22a7bd2-fa5f-4f19-9ce0-c26a3092aaec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6a685bc40014ea89edf1dfa6ac3c659', 'request_id': 'req-c22a7bd2-fa5f-4f19-9ce0-c26a3092aaec'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:22.948812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:22.972764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c22a7bd2-fa5f-4f19-9ce0-c26a3092aaec admin admin] The token cache is being invalidate because project d6a685bc40014ea89edf1dfa6ac3c659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:11:22.973305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c22a7bd2-fa5f-4f19-9ce0-c26a3092aaec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:22.975075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c22a7bd2-fa5f-4f19-9ce0-c26a3092aaec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:22.977559 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1689/3375] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:11:22 2022] DELETE /identity/v3/projects/d6a685bc40014ea89edf1dfa6ac3c659 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:22.983821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec6878-0ea9-4c48-b345-4342b6524f39 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:22.984485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec6878-0ea9-4c48-b345-4342b6524f39 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:22.984611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec6878-0ea9-4c48-b345-4342b6524f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:23.018546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-adec6878-0ea9-4c48-b345-4342b6524f39 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:23.020678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-adec6878-0ea9-4c48-b345-4342b6524f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:23.187360 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1687/3376] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:23.197004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be72a7d-1d28-4f63-a9eb-961522fd7ee3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:23.197481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be72a7d-1d28-4f63-a9eb-961522fd7ee3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:23.197876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be72a7d-1d28-4f63-a9eb-961522fd7ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:23.239472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6be72a7d-1d28-4f63-a9eb-961522fd7ee3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:23.255489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6be72a7d-1d28-4f63-a9eb-961522fd7ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:23.266612 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1690/3377] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:23.270693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec6878-0ea9-4c48-b345-4342b6524f39 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:23.286249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec6878-0ea9-4c48-b345-4342b6524f39 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:23.301126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e539e-4795-4c49-800b-ef82e6d3e114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:23.304487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e539e-4795-4c49-800b-ef82e6d3e114 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:23.305023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e539e-4795-4c49-800b-ef82e6d3e114 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:23.305462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e539e-4795-4c49-800b-ef82e6d3e114 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:23.308453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e539e-4795-4c49-800b-ef82e6d3e114 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:11:23.308453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e539e-4795-4c49-800b-ef82e6d3e114 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:23.315352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e539e-4795-4c49-800b-ef82e6d3e114 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:23.355320 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1688/3378] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:11:23.361232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be72a7d-1d28-4f63-a9eb-961522fd7ee3 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:23.379107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be72a7d-1d28-4f63-a9eb-961522fd7ee3 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:23.397968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fa5b3-d3cd-492e-ae62-18488754769c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:23.398936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fa5b3-d3cd-492e-ae62-18488754769c admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:23.399741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fa5b3-d3cd-492e-ae62-18488754769c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:23.401581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012fa5b3-d3cd-492e-ae62-18488754769c admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:23.405491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012fa5b3-d3cd-492e-ae62-18488754769c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:23.409343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012fa5b3-d3cd-492e-ae62-18488754769c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:23.539620 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1691/3379] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:11:23 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:23.542426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e539e-4795-4c49-800b-ef82e6d3e114 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:23.560410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e539e-4795-4c49-800b-ef82e6d3e114 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:23.631416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:23.681668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0088b71d-5792-4224-9bf9-982871da14bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:23.682382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088b71d-5792-4224-9bf9-982871da14bc admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:23.682695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088b71d-5792-4224-9bf9-982871da14bc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:23.682965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0088b71d-5792-4224-9bf9-982871da14bc admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:23.683664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0088b71d-5792-4224-9bf9-982871da14bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:23.686056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0088b71d-5792-4224-9bf9-982871da14bc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:23.768307 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1689/3380] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:11:23 2022] POST /identity/v3/users => generated 291 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:23.772206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fa5b3-d3cd-492e-ae62-18488754769c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:23.809377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012fa5b3-d3cd-492e-ae62-18488754769c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:23.839406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a764f-cee2-4a1e-bdf2-c4bf8c9adb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:23.840791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a764f-cee2-4a1e-bdf2-c4bf8c9adb87 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:23.841119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a764f-cee2-4a1e-bdf2-c4bf8c9adb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:23.841400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a764f-cee2-4a1e-bdf2-c4bf8c9adb87 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:23.842176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a764f-cee2-4a1e-bdf2-c4bf8c9adb87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:23.854000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a764f-cee2-4a1e-bdf2-c4bf8c9adb87 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:23.892960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0088b71d-5792-4224-9bf9-982871da14bc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:23.893236 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1692/3381] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:23.908418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0088b71d-5792-4224-9bf9-982871da14bc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:23.921486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323aa37-bd7a-4ef9-be26-7fcb70a1e122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:23.922213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323aa37-bd7a-4ef9-be26-7fcb70a1e122 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:23.922500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323aa37-bd7a-4ef9-be26-7fcb70a1e122 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:23.922776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323aa37-bd7a-4ef9-be26-7fcb70a1e122 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:23.923533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323aa37-bd7a-4ef9-be26-7fcb70a1e122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:23.925136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323aa37-bd7a-4ef9-be26-7fcb70a1e122 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:23.959412 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1690/3382] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:23.961263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a764f-cee2-4a1e-bdf2-c4bf8c9adb87 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:23.982844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a764f-cee2-4a1e-bdf2-c4bf8c9adb87 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:23.990423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfeea0-1fc5-4773-9ece-60792f4d5931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:23.994504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfeea0-1fc5-4773-9ece-60792f4d5931 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:23.994504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfeea0-1fc5-4773-9ece-60792f4d5931 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:23.994504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfeea0-1fc5-4773-9ece-60792f4d5931 admin admin] PATH_INFO: `/v3/projects/e3b7c6618a074f8c97f2354096574844/users/f7355397c65b4af881865f09b6b488bd/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:24.017577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfeea0-1fc5-4773-9ece-60792f4d5931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3b7c6618a074f8c97f2354096574844, user_id=f7355397c65b4af881865f09b6b488bd, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:24.022287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfeea0-1fc5-4773-9ece-60792f4d5931 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:24.144452 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1693/3383] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:11:23 2022] PUT /identity/v3/projects/e3b7c6618a074f8c97f2354096574844/users/f7355397c65b4af881865f09b6b488bd/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:24.155302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a43612-e7fe-4b1a-a962-f1f9d3810286 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:24.157039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a43612-e7fe-4b1a-a962-f1f9d3810286 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:24.157039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a43612-e7fe-4b1a-a962-f1f9d3810286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:24.219555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-76a43612-e7fe-4b1a-a962-f1f9d3810286 None None] MFA Rules not processed for user `f7355397c65b4af881865f09b6b488bd`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:24.220824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-76a43612-e7fe-4b1a-a962-f1f9d3810286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:24.373271 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1691/3384] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:24.690177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfeea0-1fc5-4773-9ece-60792f4d5931 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:24.692079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-abcfeea0-1fc5-4773-9ece-60792f4d5931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:24.811580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfeea0-1fc5-4773-9ece-60792f4d5931 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:24.818873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ea919-aa57-4698-8697-e2ce590dcc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:24.820087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ea919-aa57-4698-8697-e2ce590dcc96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:24.820594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ea919-aa57-4698-8697-e2ce590dcc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:24.821092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7ea919-aa57-4698-8697-e2ce590dcc96 admin admin] PATH_INFO: `/v3/users/4112534978ba4622b0fbbb9a92791876` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:24.822703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ea919-aa57-4698-8697-e2ce590dcc96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4112534978ba4622b0fbbb9a92791876)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:24.825695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7ea919-aa57-4698-8697-e2ce590dcc96 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:25.010988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8e7ea919-aa57-4698-8697-e2ce590dcc96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4112534978ba4622b0fbbb9a92791876'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:25.039089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8e7ea919-aa57-4698-8697-e2ce590dcc96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4112534978ba4622b0fbbb9a92791876'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:25.040548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8e7ea919-aa57-4698-8697-e2ce590dcc96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4112534978ba4622b0fbbb9a92791876'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:25.042646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8e7ea919-aa57-4698-8697-e2ce590dcc96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4112534978ba4622b0fbbb9a92791876'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:25.115741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8e7ea919-aa57-4698-8697-e2ce590dcc96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4112534978ba4622b0fbbb9a92791876'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:25.156328 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1694/3385] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:11:24 2022] DELETE /identity/v3/users/4112534978ba4622b0fbbb9a92791876 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:25.203324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a43612-e7fe-4b1a-a962-f1f9d3810286 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:25.204917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-76a43612-e7fe-4b1a-a962-f1f9d3810286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:25.397004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a43612-e7fe-4b1a-a962-f1f9d3810286 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:25.491229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:25.529329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e86daa-b3c2-4d4d-b55c-9ead65c4b762 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:25.529329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e86daa-b3c2-4d4d-b55c-9ead65c4b762 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:25.529622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e86daa-b3c2-4d4d-b55c-9ead65c4b762 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:25.530266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e86daa-b3c2-4d4d-b55c-9ead65c4b762 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:25.532394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c5e86daa-b3c2-4d4d-b55c-9ead65c4b762 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:25.681635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e86daa-b3c2-4d4d-b55c-9ead65c4b762 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:25.684548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e86daa-b3c2-4d4d-b55c-9ead65c4b762 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:25.698069 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1692/3386] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:25.870470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ea919-aa57-4698-8697-e2ce590dcc96 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:25.873072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8e7ea919-aa57-4698-8697-e2ce590dcc96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:25.903932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:26.032718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7ea919-aa57-4698-8697-e2ce590dcc96 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:26.039437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9562327b-a06a-4c31-a972-192f19f75b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:26.040986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562327b-a06a-4c31-a972-192f19f75b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:26.041276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562327b-a06a-4c31-a972-192f19f75b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:26.041558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9562327b-a06a-4c31-a972-192f19f75b95 admin admin] PATH_INFO: `/v3/projects/3450ad991a3943e3a4c4233551a42729` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:26.042847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562327b-a06a-4c31-a972-192f19f75b95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3450ad991a3943e3a4c4233551a42729)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:26.046086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9562327b-a06a-4c31-a972-192f19f75b95 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:26.262757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9562327b-a06a-4c31-a972-192f19f75b95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3450ad991a3943e3a4c4233551a42729', 'request_id': 'req-9562327b-a06a-4c31-a972-192f19f75b95'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:26.276303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9562327b-a06a-4c31-a972-192f19f75b95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3450ad991a3943e3a4c4233551a42729', 'request_id': 'req-9562327b-a06a-4c31-a972-192f19f75b95'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:26.400858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9562327b-a06a-4c31-a972-192f19f75b95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3450ad991a3943e3a4c4233551a42729', 'request_id': 'req-9562327b-a06a-4c31-a972-192f19f75b95'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:26.414850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9562327b-a06a-4c31-a972-192f19f75b95 admin admin] The token cache is being invalidate because project 3450ad991a3943e3a4c4233551a42729 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:11:26.415149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9562327b-a06a-4c31-a972-192f19f75b95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:26.416156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9562327b-a06a-4c31-a972-192f19f75b95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:26.420783 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1695/3387] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:11:25 2022] DELETE /identity/v3/projects/3450ad991a3943e3a4c4233551a42729 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:26.427348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3465b61-8f8d-4d25-b23a-27b7080877b3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:26.427839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3465b61-8f8d-4d25-b23a-27b7080877b3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:26.427937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3465b61-8f8d-4d25-b23a-27b7080877b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:26.451172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b3465b61-8f8d-4d25-b23a-27b7080877b3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:26.452753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b3465b61-8f8d-4d25-b23a-27b7080877b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:26.546030 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1693/3388] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:26.553913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fd608-b033-47f3-a123-d7a36ddb4471 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:26.554385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fd608-b033-47f3-a123-d7a36ddb4471 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:26.554596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fd608-b033-47f3-a123-d7a36ddb4471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:26.583731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f61fd608-b033-47f3-a123-d7a36ddb4471 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:26.584967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f61fd608-b033-47f3-a123-d7a36ddb4471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:26.596606 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1696/3389] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:26.610818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3465b61-8f8d-4d25-b23a-27b7080877b3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:26.631629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3465b61-8f8d-4d25-b23a-27b7080877b3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:26.637608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d653f1-62fe-4e6b-94dd-5eb8d7952e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:26.638299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d653f1-62fe-4e6b-94dd-5eb8d7952e27 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:26.638935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d653f1-62fe-4e6b-94dd-5eb8d7952e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:26.639263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d653f1-62fe-4e6b-94dd-5eb8d7952e27 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:26.640147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d653f1-62fe-4e6b-94dd-5eb8d7952e27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:11:26.640464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d653f1-62fe-4e6b-94dd-5eb8d7952e27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:26.642518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d653f1-62fe-4e6b-94dd-5eb8d7952e27 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:26.687066 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1694/3390] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:11:26.696809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fd608-b033-47f3-a123-d7a36ddb4471 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:26.710660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fd608-b033-47f3-a123-d7a36ddb4471 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:26.724002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d5d33-5d3d-4de8-9937-a9a790823dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:26.728451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d5d33-5d3d-4de8-9937-a9a790823dad admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:26.728451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d5d33-5d3d-4de8-9937-a9a790823dad admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:26.728451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615d5d33-5d3d-4de8-9937-a9a790823dad admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:26.730024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d5d33-5d3d-4de8-9937-a9a790823dad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:26.735034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615d5d33-5d3d-4de8-9937-a9a790823dad admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:26.882333 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1697/3391] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:11:26 2022] POST /identity/v3/projects => generated 391 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:26.887745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d653f1-62fe-4e6b-94dd-5eb8d7952e27 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:26.918138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d653f1-62fe-4e6b-94dd-5eb8d7952e27 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:27.062181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1054ef7-8a91-4d1e-b62c-fd8c18c333f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:27.063197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1054ef7-8a91-4d1e-b62c-fd8c18c333f3 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:27.063549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1054ef7-8a91-4d1e-b62c-fd8c18c333f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:27.064532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1054ef7-8a91-4d1e-b62c-fd8c18c333f3 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:27.065370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1054ef7-8a91-4d1e-b62c-fd8c18c333f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:27.068105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1054ef7-8a91-4d1e-b62c-fd8c18c333f3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:27.094485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:27.202427 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1695/3392] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:11:26 2022] POST /identity/v3/users => generated 296 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:27.205237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d5d33-5d3d-4de8-9937-a9a790823dad admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:27.219899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615d5d33-5d3d-4de8-9937-a9a790823dad admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:27.224915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2412b-b918-43c5-bb46-e8f05b95bc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:27.225711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2412b-b918-43c5-bb46-e8f05b95bc66 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:27.226134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2412b-b918-43c5-bb46-e8f05b95bc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:27.226642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b2412b-b918-43c5-bb46-e8f05b95bc66 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:27.228712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2412b-b918-43c5-bb46-e8f05b95bc66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:27.230738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b2412b-b918-43c5-bb46-e8f05b95bc66 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:27.265205 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1698/3393] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:27.269792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1054ef7-8a91-4d1e-b62c-fd8c18c333f3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:27.283634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1054ef7-8a91-4d1e-b62c-fd8c18c333f3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:27.289986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bad089-f3bb-4b56-98b0-c881ec937949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:27.290882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bad089-f3bb-4b56-98b0-c881ec937949 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:27.291288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bad089-f3bb-4b56-98b0-c881ec937949 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:27.291638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bad089-f3bb-4b56-98b0-c881ec937949 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:27.292648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bad089-f3bb-4b56-98b0-c881ec937949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:27.294774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bad089-f3bb-4b56-98b0-c881ec937949 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:27.309342 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1696/3394] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:27.314165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2412b-b918-43c5-bb46-e8f05b95bc66 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:27.329902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b2412b-b918-43c5-bb46-e8f05b95bc66 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:27.335175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63caf8fa-3e54-461b-9291-d5a140a4547a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:27.336400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caf8fa-3e54-461b-9291-d5a140a4547a admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:27.337045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caf8fa-3e54-461b-9291-d5a140a4547a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:27.337554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63caf8fa-3e54-461b-9291-d5a140a4547a admin admin] PATH_INFO: `/v3/projects/c78c95b156dd4ee183230abc68093a4b/users/408a54e3416b40479fae5fba67d14a53/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:27.378790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63caf8fa-3e54-461b-9291-d5a140a4547a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c78c95b156dd4ee183230abc68093a4b, user_id=408a54e3416b40479fae5fba67d14a53, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:27.384036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63caf8fa-3e54-461b-9291-d5a140a4547a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:27.484118 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1699/3395] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:11:27 2022] PUT /identity/v3/projects/c78c95b156dd4ee183230abc68093a4b/users/408a54e3416b40479fae5fba67d14a53/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:27.495791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81114d7b-87ab-4639-86e6-f531e2323578 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:27.496964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81114d7b-87ab-4639-86e6-f531e2323578 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:27.500146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81114d7b-87ab-4639-86e6-f531e2323578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:27.546335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-81114d7b-87ab-4639-86e6-f531e2323578 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:27.547968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-81114d7b-87ab-4639-86e6-f531e2323578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:27.654017 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1697/3396] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:27.702893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63caf8fa-3e54-461b-9291-d5a140a4547a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:27.705282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-63caf8fa-3e54-461b-9291-d5a140a4547a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:27.785687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:27.806174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63caf8fa-3e54-461b-9291-d5a140a4547a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:27.813445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7056e61-3fa2-418a-a885-d58203f51999 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:27.814926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7056e61-3fa2-418a-a885-d58203f51999 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:27.815694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7056e61-3fa2-418a-a885-d58203f51999 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:27.816186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7056e61-3fa2-418a-a885-d58203f51999 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:27.832848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7056e61-3fa2-418a-a885-d58203f51999 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:27.835087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7056e61-3fa2-418a-a885-d58203f51999 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:27.849033 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1700/3397] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:28.004848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81114d7b-87ab-4639-86e6-f531e2323578 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:28.005163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-81114d7b-87ab-4639-86e6-f531e2323578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:28.024216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81114d7b-87ab-4639-86e6-f531e2323578 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:28.034266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595c6f4b-afc5-4d84-8d38-997021208a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:28.039823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c6f4b-afc5-4d84-8d38-997021208a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:28.039823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c6f4b-afc5-4d84-8d38-997021208a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:28.039823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595c6f4b-afc5-4d84-8d38-997021208a5b admin admin] PATH_INFO: `/v3/users/f7355397c65b4af881865f09b6b488bd` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:28.039823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c6f4b-afc5-4d84-8d38-997021208a5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7355397c65b4af881865f09b6b488bd)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:28.043616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595c6f4b-afc5-4d84-8d38-997021208a5b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:28.279571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7056e61-3fa2-418a-a885-d58203f51999 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:28.281229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f7056e61-3fa2-418a-a885-d58203f51999 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:28.298617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7056e61-3fa2-418a-a885-d58203f51999 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:28.303983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ab260-37ed-4439-8e26-635b210a911b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:28.305278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ab260-37ed-4439-8e26-635b210a911b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:28.305814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ab260-37ed-4439-8e26-635b210a911b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:28.306466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ab260-37ed-4439-8e26-635b210a911b admin admin] PATH_INFO: `/v3/users/f5df3bc2a59b4d99b81932deac1ece64` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:28.308399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ab260-37ed-4439-8e26-635b210a911b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5df3bc2a59b4d99b81932deac1ece64)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:28.315939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ab260-37ed-4439-8e26-635b210a911b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:28.325432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-595c6f4b-afc5-4d84-8d38-997021208a5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7355397c65b4af881865f09b6b488bd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:28.326252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-595c6f4b-afc5-4d84-8d38-997021208a5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7355397c65b4af881865f09b6b488bd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:28.338323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-595c6f4b-afc5-4d84-8d38-997021208a5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7355397c65b4af881865f09b6b488bd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:28.401270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-595c6f4b-afc5-4d84-8d38-997021208a5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7355397c65b4af881865f09b6b488bd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:28.402352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-595c6f4b-afc5-4d84-8d38-997021208a5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7355397c65b4af881865f09b6b488bd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:28.432459 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1698/3398] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:11:28 2022] DELETE /identity/v3/users/f7355397c65b4af881865f09b6b488bd => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:28.437181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bd771-2331-4870-8bcf-c49b56ff039b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:28.437684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bd771-2331-4870-8bcf-c49b56ff039b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:28.438015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bd771-2331-4870-8bcf-c49b56ff039b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:28.459805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-980bd771-2331-4870-8bcf-c49b56ff039b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:28.459805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-980bd771-2331-4870-8bcf-c49b56ff039b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:28.529079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:28.551776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cd5ab260-37ed-4439-8e26-635b210a911b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5df3bc2a59b4d99b81932deac1ece64'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:28.552548 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1699/3399] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:11:28.568155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cd5ab260-37ed-4439-8e26-635b210a911b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5df3bc2a59b4d99b81932deac1ece64'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:28.569318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cd5ab260-37ed-4439-8e26-635b210a911b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5df3bc2a59b4d99b81932deac1ece64'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:28.571456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cd5ab260-37ed-4439-8e26-635b210a911b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5df3bc2a59b4d99b81932deac1ece64'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:28.580641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bd771-2331-4870-8bcf-c49b56ff039b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:28.582967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-980bd771-2331-4870-8bcf-c49b56ff039b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:28.644473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cd5ab260-37ed-4439-8e26-635b210a911b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5df3bc2a59b4d99b81932deac1ece64'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:28.700647 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1701/3400] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:11:28 2022] DELETE /identity/v3/users/f5df3bc2a59b4d99b81932deac1ece64 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:11:28.720403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bd771-2331-4870-8bcf-c49b56ff039b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:28.820829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb31a74-28d7-4e7d-8657-b0671423ceb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:28.821812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb31a74-28d7-4e7d-8657-b0671423ceb4 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:28.822589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb31a74-28d7-4e7d-8657-b0671423ceb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:28.827243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb31a74-28d7-4e7d-8657-b0671423ceb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:28.830097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4eb31a74-28d7-4e7d-8657-b0671423ceb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:28.965820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb31a74-28d7-4e7d-8657-b0671423ceb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:28.968250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb31a74-28d7-4e7d-8657-b0671423ceb4 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:28.994632 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1700/3401] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:29.516136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ab260-37ed-4439-8e26-635b210a911b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:29.518694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cd5ab260-37ed-4439-8e26-635b210a911b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:29.535070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ab260-37ed-4439-8e26-635b210a911b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:29.541183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a2399-5c84-4d49-ba66-84020f79ae76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:29.542785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a2399-5c84-4d49-ba66-84020f79ae76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:29.544111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a2399-5c84-4d49-ba66-84020f79ae76 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:29.544554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a2399-5c84-4d49-ba66-84020f79ae76 admin admin] PATH_INFO: `/v3/projects/7e1a1360253a413d8ee275cb1d1fac02` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:29.546071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a2399-5c84-4d49-ba66-84020f79ae76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e1a1360253a413d8ee275cb1d1fac02)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:29.548394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a2399-5c84-4d49-ba66-84020f79ae76 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:29.695973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a59a2399-5c84-4d49-ba66-84020f79ae76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e1a1360253a413d8ee275cb1d1fac02', 'request_id': 'req-a59a2399-5c84-4d49-ba66-84020f79ae76'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:29.707146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a59a2399-5c84-4d49-ba66-84020f79ae76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e1a1360253a413d8ee275cb1d1fac02', 'request_id': 'req-a59a2399-5c84-4d49-ba66-84020f79ae76'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:29.795265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a59a2399-5c84-4d49-ba66-84020f79ae76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e1a1360253a413d8ee275cb1d1fac02', 'request_id': 'req-a59a2399-5c84-4d49-ba66-84020f79ae76'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:29.811143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a59a2399-5c84-4d49-ba66-84020f79ae76 admin admin] The token cache is being invalidate because project 7e1a1360253a413d8ee275cb1d1fac02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:11:29.813603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a59a2399-5c84-4d49-ba66-84020f79ae76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:29.813603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a59a2399-5c84-4d49-ba66-84020f79ae76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:29.815158 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1702/3402] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:11:29 2022] DELETE /identity/v3/projects/7e1a1360253a413d8ee275cb1d1fac02 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:29.820081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adecba9-e826-4a98-9885-f71854d07239 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:29.821117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adecba9-e826-4a98-9885-f71854d07239 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:29.821710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adecba9-e826-4a98-9885-f71854d07239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:29.864757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8adecba9-e826-4a98-9885-f71854d07239 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:29.872839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8adecba9-e826-4a98-9885-f71854d07239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:29.977668 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1701/3403] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:29.980081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a8309-e239-4a35-bacd-8faae199857d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:29.980677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a8309-e239-4a35-bacd-8faae199857d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:29.981170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a8309-e239-4a35-bacd-8faae199857d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:30.004529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-479a8309-e239-4a35-bacd-8faae199857d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:30.006502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-479a8309-e239-4a35-bacd-8faae199857d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:30.017108 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1703/3404] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:30.021644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adecba9-e826-4a98-9885-f71854d07239 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:30.035475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adecba9-e826-4a98-9885-f71854d07239 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:30.043129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f6f1c-e21e-4b4e-b7be-a6f8ee654e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:30.044282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f6f1c-e21e-4b4e-b7be-a6f8ee654e4f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:30.044805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f6f1c-e21e-4b4e-b7be-a6f8ee654e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:30.046181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f6f1c-e21e-4b4e-b7be-a6f8ee654e4f admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:30.047383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f6f1c-e21e-4b4e-b7be-a6f8ee654e4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:11:30.048094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f6f1c-e21e-4b4e-b7be-a6f8ee654e4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:30.050128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f6f1c-e21e-4b4e-b7be-a6f8ee654e4f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:30.070521 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1702/3405] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:11:30.071786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a8309-e239-4a35-bacd-8faae199857d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:30.083908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a8309-e239-4a35-bacd-8faae199857d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:30.095971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0263da-547f-42b7-be28-3cbf2dfd7c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:30.097119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0263da-547f-42b7-be28-3cbf2dfd7c87 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:30.098112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0263da-547f-42b7-be28-3cbf2dfd7c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:30.098261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0263da-547f-42b7-be28-3cbf2dfd7c87 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:30.100676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0263da-547f-42b7-be28-3cbf2dfd7c87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:30.102954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0263da-547f-42b7-be28-3cbf2dfd7c87 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:30.266030 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1704/3406] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:11:30 2022] POST /identity/v3/projects => generated 377 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:30.267412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f6f1c-e21e-4b4e-b7be-a6f8ee654e4f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:30.293048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f6f1c-e21e-4b4e-b7be-a6f8ee654e4f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:30.406527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f8547-95af-4109-b253-fe3614c22696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:30.407307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f8547-95af-4109-b253-fe3614c22696 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:30.410594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f8547-95af-4109-b253-fe3614c22696 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:30.415349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f8547-95af-4109-b253-fe3614c22696 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:30.416502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f8547-95af-4109-b253-fe3614c22696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:30.418984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f8547-95af-4109-b253-fe3614c22696 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:30.545029 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1703/3407] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:11:30 2022] POST /identity/v3/users => generated 289 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:30.549123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0263da-547f-42b7-be28-3cbf2dfd7c87 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:30.571238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f8547-95af-4109-b253-fe3614c22696 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:30.571493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0263da-547f-42b7-be28-3cbf2dfd7c87 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:30.573223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5a2f8547-95af-4109-b253-fe3614c22696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:30.584595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fc04d-94e5-45ea-845d-b3279733f7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:30.585788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fc04d-94e5-45ea-845d-b3279733f7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:30.589150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fc04d-94e5-45ea-845d-b3279733f7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:30.589543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902fc04d-94e5-45ea-845d-b3279733f7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:30.592205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902fc04d-94e5-45ea-845d-b3279733f7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:30.596565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f8547-95af-4109-b253-fe3614c22696 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:30.602122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902fc04d-94e5-45ea-845d-b3279733f7a9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:30.602122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:30.602358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893e07a3-0850-4a19-a219-4ac5a097d58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:30.603321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e07a3-0850-4a19-a219-4ac5a097d58e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:30.603448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e07a3-0850-4a19-a219-4ac5a097d58e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:30.603704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e07a3-0850-4a19-a219-4ac5a097d58e admin admin] PATH_INFO: `/v3/projects/e3b7c6618a074f8c97f2354096574844` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:30.606748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e07a3-0850-4a19-a219-4ac5a097d58e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3b7c6618a074f8c97f2354096574844)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:30.608702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893e07a3-0850-4a19-a219-4ac5a097d58e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:30.642094 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1705/3408] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:30.654334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fc04d-94e5-45ea-845d-b3279733f7a9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:30.693055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902fc04d-94e5-45ea-845d-b3279733f7a9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:30.703443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5b678-1592-4234-a53b-db43863712f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:30.704347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5b678-1592-4234-a53b-db43863712f8 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:30.704794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5b678-1592-4234-a53b-db43863712f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:30.705246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a5b678-1592-4234-a53b-db43863712f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:30.707067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5b678-1592-4234-a53b-db43863712f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:30.711012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a5b678-1592-4234-a53b-db43863712f8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:30.726288 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1706/3409] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:30 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:30.728933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5b678-1592-4234-a53b-db43863712f8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:30.752645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a5b678-1592-4234-a53b-db43863712f8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:30.765091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d9035-a476-4ae5-b588-b52aae8b3270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:30.766075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d9035-a476-4ae5-b588-b52aae8b3270 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:30.766590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d9035-a476-4ae5-b588-b52aae8b3270 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:30.767068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d9035-a476-4ae5-b588-b52aae8b3270 admin admin] PATH_INFO: `/v3/projects/8f20390f83d3407e93a3d8d2a196281b/users/cafc81a185c645e893a77b7fd297d950/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:30.803464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d9035-a476-4ae5-b588-b52aae8b3270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f20390f83d3407e93a3d8d2a196281b, user_id=cafc81a185c645e893a77b7fd297d950, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:30.806780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-893e07a3-0850-4a19-a219-4ac5a097d58e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3b7c6618a074f8c97f2354096574844', 'request_id': 'req-893e07a3-0850-4a19-a219-4ac5a097d58e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:30.809747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d9035-a476-4ae5-b588-b52aae8b3270 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:30.840426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-893e07a3-0850-4a19-a219-4ac5a097d58e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3b7c6618a074f8c97f2354096574844', 'request_id': 'req-893e07a3-0850-4a19-a219-4ac5a097d58e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:30.897493 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1707/3410] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:11:30 2022] PUT /identity/v3/projects/8f20390f83d3407e93a3d8d2a196281b/users/cafc81a185c645e893a77b7fd297d950/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:11:30.903818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c5935-9caa-4023-95bd-4212e608d612 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:30.904390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c5935-9caa-4023-95bd-4212e608d612 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:30.904879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c5935-9caa-4023-95bd-4212e608d612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:30.916824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:30.917967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-893e07a3-0850-4a19-a219-4ac5a097d58e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3b7c6618a074f8c97f2354096574844', 'request_id': 'req-893e07a3-0850-4a19-a219-4ac5a097d58e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:30.931996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-982c5935-9caa-4023-95bd-4212e608d612 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:30.933243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-893e07a3-0850-4a19-a219-4ac5a097d58e admin admin] The token cache is being invalidate because project e3b7c6618a074f8c97f2354096574844 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:11:30.933378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-982c5935-9caa-4023-95bd-4212e608d612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:30.934006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-893e07a3-0850-4a19-a219-4ac5a097d58e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:30.936685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-893e07a3-0850-4a19-a219-4ac5a097d58e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:30.939288 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1704/3411] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:11:30 2022] DELETE /identity/v3/projects/e3b7c6618a074f8c97f2354096574844 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:30.944160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3f7e8-0cfa-4f5c-93ac-8b073d017ec1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:30.945861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3f7e8-0cfa-4f5c-93ac-8b073d017ec1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:30.947488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3f7e8-0cfa-4f5c-93ac-8b073d017ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:30.994593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5ba3f7e8-0cfa-4f5c-93ac-8b073d017ec1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:30.998198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5ba3f7e8-0cfa-4f5c-93ac-8b073d017ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:31.091311 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1708/3412] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:31.146559 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1705/3413] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:31.155148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bb065-1f4f-45cf-93e3-e83a8cbbf6b2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:31.155459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bb065-1f4f-45cf-93e3-e83a8cbbf6b2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:31.157228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bb065-1f4f-45cf-93e3-e83a8cbbf6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:31.158876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3f7e8-0cfa-4f5c-93ac-8b073d017ec1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:31.163891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5ba3f7e8-0cfa-4f5c-93ac-8b073d017ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:31.194091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4f2bb065-1f4f-45cf-93e3-e83a8cbbf6b2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:31.195405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4f2bb065-1f4f-45cf-93e3-e83a8cbbf6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:31.217667 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1709/3414] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:11:31.218975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bb065-1f4f-45cf-93e3-e83a8cbbf6b2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:31.230514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bb065-1f4f-45cf-93e3-e83a8cbbf6b2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:31.243340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1528eb-019d-4e75-90a5-496843d6b39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:31.244563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1528eb-019d-4e75-90a5-496843d6b39b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:31.244908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1528eb-019d-4e75-90a5-496843d6b39b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:31.245174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1528eb-019d-4e75-90a5-496843d6b39b admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:31.246053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1528eb-019d-4e75-90a5-496843d6b39b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:11:31.246397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1528eb-019d-4e75-90a5-496843d6b39b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:31.249234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1528eb-019d-4e75-90a5-496843d6b39b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:31.279412 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1710/3415] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:11:31.289436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1528eb-019d-4e75-90a5-496843d6b39b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:31.308738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1528eb-019d-4e75-90a5-496843d6b39b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:31.311828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3f7e8-0cfa-4f5c-93ac-8b073d017ec1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:31.322523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b59224-7354-49ba-923a-61ae26f8854d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:31.322808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4287b4a-0055-42cf-871f-83e676d7f85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:31.323205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4287b4a-0055-42cf-871f-83e676d7f85e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:31.323493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4287b4a-0055-42cf-871f-83e676d7f85e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:31.323607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b59224-7354-49ba-923a-61ae26f8854d service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:31.323948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4287b4a-0055-42cf-871f-83e676d7f85e admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:31.325978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4287b4a-0055-42cf-871f-83e676d7f85e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:31.326742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b59224-7354-49ba-923a-61ae26f8854d service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:31.327140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b59224-7354-49ba-923a-61ae26f8854d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:31.328483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4287b4a-0055-42cf-871f-83e676d7f85e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:31.359432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b59224-7354-49ba-923a-61ae26f8854d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:31.362587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b59224-7354-49ba-923a-61ae26f8854d service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:31.390277 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1706/3416] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:31.501243 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1711/3417] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:11:31 2022] POST /identity/v3/projects => generated 395 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:31.505743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b59224-7354-49ba-923a-61ae26f8854d service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:31.517266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b59224-7354-49ba-923a-61ae26f8854d service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:31.585225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6497194-2ff5-4855-8051-ff67c831184d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:31.586333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6497194-2ff5-4855-8051-ff67c831184d admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:31.586679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6497194-2ff5-4855-8051-ff67c831184d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:31.587155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6497194-2ff5-4855-8051-ff67c831184d admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:31.587763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6497194-2ff5-4855-8051-ff67c831184d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:31.590641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6497194-2ff5-4855-8051-ff67c831184d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:31.715354 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1707/3418] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:11:31 2022] POST /identity/v3/users => generated 298 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:31.716684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4287b4a-0055-42cf-871f-83e676d7f85e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:31.730102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4287b4a-0055-42cf-871f-83e676d7f85e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:31.737510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefde3a-7ac9-4a35-9b44-420b3a4a90fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:31.738698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefde3a-7ac9-4a35-9b44-420b3a4a90fb admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:31.739191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefde3a-7ac9-4a35-9b44-420b3a4a90fb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:31.743429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faefde3a-7ac9-4a35-9b44-420b3a4a90fb admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:31.743429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefde3a-7ac9-4a35-9b44-420b3a4a90fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:31.743429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faefde3a-7ac9-4a35-9b44-420b3a4a90fb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:31.756683 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1712/3419] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:31.761258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6497194-2ff5-4855-8051-ff67c831184d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:31.775919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6497194-2ff5-4855-8051-ff67c831184d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:31.782279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e700b1-8472-4d5b-abf4-b5fd08a71a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:31.783318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e700b1-8472-4d5b-abf4-b5fd08a71a0f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:31.783666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e700b1-8472-4d5b-abf4-b5fd08a71a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:31.784010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e700b1-8472-4d5b-abf4-b5fd08a71a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:31.784894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e700b1-8472-4d5b-abf4-b5fd08a71a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:31.786683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e700b1-8472-4d5b-abf4-b5fd08a71a0f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:31.804642 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1708/3420] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:31.805165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefde3a-7ac9-4a35-9b44-420b3a4a90fb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:31.821871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faefde3a-7ac9-4a35-9b44-420b3a4a90fb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:31.828165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af9d5a-074b-499a-a4be-a6daa60e1cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:31.829144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af9d5a-074b-499a-a4be-a6daa60e1cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:31.829619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af9d5a-074b-499a-a4be-a6daa60e1cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:31.830199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af9d5a-074b-499a-a4be-a6daa60e1cc7 admin admin] PATH_INFO: `/v3/projects/c0dda471dbbd4c3093f8f362f4b2221a/users/64428727aaa145f69f6c009f36e9adbb/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:31.855573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af9d5a-074b-499a-a4be-a6daa60e1cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0dda471dbbd4c3093f8f362f4b2221a, user_id=64428727aaa145f69f6c009f36e9adbb, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:31.861684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63af9d5a-074b-499a-a4be-a6daa60e1cc7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:31.881753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:31.947365 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1713/3421] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:11:31 2022] PUT /identity/v3/projects/c0dda471dbbd4c3093f8f362f4b2221a/users/64428727aaa145f69f6c009f36e9adbb/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:31.958026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d663ce-c85d-4e2d-841b-07b8d703929b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:31.958026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d663ce-c85d-4e2d-841b-07b8d703929b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:31.958026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d663ce-c85d-4e2d-841b-07b8d703929b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:31.998555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-13d663ce-c85d-4e2d-841b-07b8d703929b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:32.000623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-13d663ce-c85d-4e2d-841b-07b8d703929b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:32.177593 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1709/3422] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:32.189772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af9d5a-074b-499a-a4be-a6daa60e1cc7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:32.192588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-63af9d5a-074b-499a-a4be-a6daa60e1cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:32.340932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af9d5a-074b-499a-a4be-a6daa60e1cc7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:32.342989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba3275-0377-40fd-b48e-3fec49f4255a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:32.344273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba3275-0377-40fd-b48e-3fec49f4255a service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:32.344839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba3275-0377-40fd-b48e-3fec49f4255a service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:32.345288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ba3275-0377-40fd-b48e-3fec49f4255a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:32.361036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ba3275-0377-40fd-b48e-3fec49f4255a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:32.363386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ba3275-0377-40fd-b48e-3fec49f4255a service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:32.380496 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1714/3423] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:32.859386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:37.015883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659d920-c18c-4421-9cf6-060200e3734e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:37.016324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659d920-c18c-4421-9cf6-060200e3734e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:37.016324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659d920-c18c-4421-9cf6-060200e3734e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:37.087404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7659d920-c18c-4421-9cf6-060200e3734e None None] MFA Rules not processed for user `64428727aaa145f69f6c009f36e9adbb`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:37.089207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7659d920-c18c-4421-9cf6-060200e3734e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:37.126889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:37.237551 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1710/3424] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:37.625346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba3275-0377-40fd-b48e-3fec49f4255a service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:37.625346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e4ba3275-0377-40fd-b48e-3fec49f4255a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:37.625346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ba3275-0377-40fd-b48e-3fec49f4255a service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:37.628369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995cae20-e505-4503-85ec-d6f27fd1d857 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:37.628369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cae20-e505-4503-85ec-d6f27fd1d857 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:37.628369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cae20-e505-4503-85ec-d6f27fd1d857 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:37.628369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995cae20-e505-4503-85ec-d6f27fd1d857 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:37.636021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995cae20-e505-4503-85ec-d6f27fd1d857 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:37.636021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995cae20-e505-4503-85ec-d6f27fd1d857 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:37.636021 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1715/3425] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:40.956435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659d920-c18c-4421-9cf6-060200e3734e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:41.003841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659d920-c18c-4421-9cf6-060200e3734e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:41.016785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a1a32-0f09-4a41-a8d0-ac94f3c319f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:41.016785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a1a32-0f09-4a41-a8d0-ac94f3c319f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:41.016785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a1a32-0f09-4a41-a8d0-ac94f3c319f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:41.016785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a1a32-0f09-4a41-a8d0-ac94f3c319f2 admin admin] PATH_INFO: `/v3/users/64428727aaa145f69f6c009f36e9adbb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:41.017052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a1a32-0f09-4a41-a8d0-ac94f3c319f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64428727aaa145f69f6c009f36e9adbb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:41.021300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a1a32-0f09-4a41-a8d0-ac94f3c319f2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:41.360074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a70a1a32-0f09-4a41-a8d0-ac94f3c319f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64428727aaa145f69f6c009f36e9adbb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:41.361917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a70a1a32-0f09-4a41-a8d0-ac94f3c319f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64428727aaa145f69f6c009f36e9adbb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:41.378789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a70a1a32-0f09-4a41-a8d0-ac94f3c319f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64428727aaa145f69f6c009f36e9adbb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:41.457871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a70a1a32-0f09-4a41-a8d0-ac94f3c319f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64428727aaa145f69f6c009f36e9adbb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:41.460509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a70a1a32-0f09-4a41-a8d0-ac94f3c319f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64428727aaa145f69f6c009f36e9adbb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:41.506863 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1711/3426] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:11:40 2022] DELETE /identity/v3/users/64428727aaa145f69f6c009f36e9adbb => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:41.626193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12469d-f9a0-46e0-9397-9095e5e6ed36 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:41.626689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12469d-f9a0-46e0-9397-9095e5e6ed36 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:41.626772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b12469d-f9a0-46e0-9397-9095e5e6ed36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:41.687284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6b12469d-f9a0-46e0-9397-9095e5e6ed36 None None] MFA Rules not processed for user `408a54e3416b40479fae5fba67d14a53`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:41.688768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6b12469d-f9a0-46e0-9397-9095e5e6ed36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:41.918748 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1716/3427] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:41.937892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a1a32-0f09-4a41-a8d0-ac94f3c319f2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:41.940319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a70a1a32-0f09-4a41-a8d0-ac94f3c319f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:42.154352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a1a32-0f09-4a41-a8d0-ac94f3c319f2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:42.173933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e49ff-31a3-40a0-9b4b-aa616b0a4118 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:42.176586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e49ff-31a3-40a0-9b4b-aa616b0a4118 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:42.177418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e49ff-31a3-40a0-9b4b-aa616b0a4118 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:42.177704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e49ff-31a3-40a0-9b4b-aa616b0a4118 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:42.231198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e49ff-31a3-40a0-9b4b-aa616b0a4118 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:42.242648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e49ff-31a3-40a0-9b4b-aa616b0a4118 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:42.265057 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1712/3428] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:42.375177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12469d-f9a0-46e0-9397-9095e5e6ed36 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:42.376867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6b12469d-f9a0-46e0-9397-9095e5e6ed36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:42.466369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e49ff-31a3-40a0-9b4b-aa616b0a4118 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:42.468575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c53e49ff-31a3-40a0-9b4b-aa616b0a4118 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:42.529274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:42.575698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b12469d-f9a0-46e0-9397-9095e5e6ed36 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:42.587222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01a1eb-646e-4648-9a7b-dacd07173e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:42.596005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01a1eb-646e-4648-9a7b-dacd07173e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:42.596435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01a1eb-646e-4648-9a7b-dacd07173e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:42.596853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01a1eb-646e-4648-9a7b-dacd07173e0d admin admin] PATH_INFO: `/v3/projects/c0dda471dbbd4c3093f8f362f4b2221a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:42.599811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01a1eb-646e-4648-9a7b-dacd07173e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0dda471dbbd4c3093f8f362f4b2221a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:42.618400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01a1eb-646e-4648-9a7b-dacd07173e0d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:42.627704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:42.739842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e49ff-31a3-40a0-9b4b-aa616b0a4118 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:42.756196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18af3cf7-8ab6-4446-b7be-92d950340289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:42.757202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af3cf7-8ab6-4446-b7be-92d950340289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:42.760509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af3cf7-8ab6-4446-b7be-92d950340289 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:42.760509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18af3cf7-8ab6-4446-b7be-92d950340289 admin admin] PATH_INFO: `/v3/users/04e5b86f94914df085b76e0d89b2230e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:42.761445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af3cf7-8ab6-4446-b7be-92d950340289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04e5b86f94914df085b76e0d89b2230e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:42.766551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18af3cf7-8ab6-4446-b7be-92d950340289 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:42.916688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b01a1eb-646e-4648-9a7b-dacd07173e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0dda471dbbd4c3093f8f362f4b2221a', 'request_id': 'req-0b01a1eb-646e-4648-9a7b-dacd07173e0d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:42.930947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b01a1eb-646e-4648-9a7b-dacd07173e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0dda471dbbd4c3093f8f362f4b2221a', 'request_id': 'req-0b01a1eb-646e-4648-9a7b-dacd07173e0d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:43.066559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b01a1eb-646e-4648-9a7b-dacd07173e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0dda471dbbd4c3093f8f362f4b2221a', 'request_id': 'req-0b01a1eb-646e-4648-9a7b-dacd07173e0d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:43.088903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b01a1eb-646e-4648-9a7b-dacd07173e0d admin admin] The token cache is being invalidate because project c0dda471dbbd4c3093f8f362f4b2221a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:11:43.089293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b01a1eb-646e-4648-9a7b-dacd07173e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:43.090424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b01a1eb-646e-4648-9a7b-dacd07173e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:43.097765 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1717/3429] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:11:42 2022] DELETE /identity/v3/projects/c0dda471dbbd4c3093f8f362f4b2221a => generated 0 bytes in 720 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:43.099490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302dbf1-f3bf-4296-bdf1-b1052492e814 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:43.099819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302dbf1-f3bf-4296-bdf1-b1052492e814 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:43.100105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4302dbf1-f3bf-4296-bdf1-b1052492e814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:43.163420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4302dbf1-f3bf-4296-bdf1-b1052492e814 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:43.166529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4302dbf1-f3bf-4296-bdf1-b1052492e814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:43.198823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-18af3cf7-8ab6-4446-b7be-92d950340289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04e5b86f94914df085b76e0d89b2230e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:43.200777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-18af3cf7-8ab6-4446-b7be-92d950340289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04e5b86f94914df085b76e0d89b2230e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:43.210082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-18af3cf7-8ab6-4446-b7be-92d950340289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04e5b86f94914df085b76e0d89b2230e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:43.304229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-18af3cf7-8ab6-4446-b7be-92d950340289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04e5b86f94914df085b76e0d89b2230e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:43.305479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-18af3cf7-8ab6-4446-b7be-92d950340289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04e5b86f94914df085b76e0d89b2230e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:43.352102 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1713/3430] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:11:42 2022] DELETE /identity/v3/users/04e5b86f94914df085b76e0d89b2230e => generated 0 bytes in 880 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:43.352282 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1718/3431] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:43.353941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83596ef0-655b-4fe6-80f4-170b00f356e2 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:43.354245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83596ef0-655b-4fe6-80f4-170b00f356e2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:43.354534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83596ef0-655b-4fe6-80f4-170b00f356e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:43.421407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-83596ef0-655b-4fe6-80f4-170b00f356e2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:43.422707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-83596ef0-655b-4fe6-80f4-170b00f356e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:43.593076 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1714/3432] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:43.594690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302dbf1-f3bf-4296-bdf1-b1052492e814 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:43.608009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4302dbf1-f3bf-4296-bdf1-b1052492e814 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:43.631032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6fbe6-a0dc-481b-b8db-4423ab5c1fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:43.632333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6fbe6-a0dc-481b-b8db-4423ab5c1fed admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:43.633682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6fbe6-a0dc-481b-b8db-4423ab5c1fed admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:43.633999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6fbe6-a0dc-481b-b8db-4423ab5c1fed admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:43.634768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6fbe6-a0dc-481b-b8db-4423ab5c1fed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:11:43.635084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6fbe6-a0dc-481b-b8db-4423ab5c1fed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:43.637951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6fbe6-a0dc-481b-b8db-4423ab5c1fed admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:43.689966 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1719/3433] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:11:43.691090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83596ef0-655b-4fe6-80f4-170b00f356e2 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:43.707119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83596ef0-655b-4fe6-80f4-170b00f356e2 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:43.718526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5faebaf-a159-432f-a75f-a61c1ef64198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:43.719216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faebaf-a159-432f-a75f-a61c1ef64198 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:43.719508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faebaf-a159-432f-a75f-a61c1ef64198 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:43.719787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5faebaf-a159-432f-a75f-a61c1ef64198 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:43.720492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5faebaf-a159-432f-a75f-a61c1ef64198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:43.733387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5faebaf-a159-432f-a75f-a61c1ef64198 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:43.887133 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1715/3434] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:11:43 2022] POST /identity/v3/projects => generated 395 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:43.889993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6fbe6-a0dc-481b-b8db-4423ab5c1fed admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:43.909457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6fbe6-a0dc-481b-b8db-4423ab5c1fed admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:44.091697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962aa90f-bc98-4bc5-9ec9-8f1e2fadb690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:44.091697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962aa90f-bc98-4bc5-9ec9-8f1e2fadb690 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:44.091697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962aa90f-bc98-4bc5-9ec9-8f1e2fadb690 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:44.091697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962aa90f-bc98-4bc5-9ec9-8f1e2fadb690 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:44.092556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962aa90f-bc98-4bc5-9ec9-8f1e2fadb690 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:44.092556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962aa90f-bc98-4bc5-9ec9-8f1e2fadb690 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:44.225803 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1720/3435] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:11:43 2022] POST /identity/v3/users => generated 298 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:44.230641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5faebaf-a159-432f-a75f-a61c1ef64198 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:44.246851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5faebaf-a159-432f-a75f-a61c1ef64198 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:44.267502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbee4ed-c175-4f36-97d1-7e1a1b2f0567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:44.268231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbee4ed-c175-4f36-97d1-7e1a1b2f0567 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:44.268519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbee4ed-c175-4f36-97d1-7e1a1b2f0567 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:44.268804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbee4ed-c175-4f36-97d1-7e1a1b2f0567 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:44.271277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbee4ed-c175-4f36-97d1-7e1a1b2f0567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:44.276374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbee4ed-c175-4f36-97d1-7e1a1b2f0567 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:44.305958 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1716/3436] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:44.309133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962aa90f-bc98-4bc5-9ec9-8f1e2fadb690 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:44.328389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962aa90f-bc98-4bc5-9ec9-8f1e2fadb690 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:44.337510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b57b9-e67f-4a72-8bc9-d315152d58a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:44.342704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b57b9-e67f-4a72-8bc9-d315152d58a7 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:44.342704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b57b9-e67f-4a72-8bc9-d315152d58a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:44.342704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517b57b9-e67f-4a72-8bc9-d315152d58a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:44.342704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b57b9-e67f-4a72-8bc9-d315152d58a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:44.342704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517b57b9-e67f-4a72-8bc9-d315152d58a7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:44.346330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:44.373075 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1721/3437] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:44.377408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbee4ed-c175-4f36-97d1-7e1a1b2f0567 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:44.390715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbee4ed-c175-4f36-97d1-7e1a1b2f0567 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:44.396802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5165889-ada0-4ad5-9a4d-74130928ba7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:44.399107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5165889-ada0-4ad5-9a4d-74130928ba7e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:44.399600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5165889-ada0-4ad5-9a4d-74130928ba7e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:44.400508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5165889-ada0-4ad5-9a4d-74130928ba7e admin admin] PATH_INFO: `/v3/projects/514d0cdb1f2641c9a259f7cf555486b7/users/799e63bdb159406286a90a5adeb16cc3/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:44.432569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b57b9-e67f-4a72-8bc9-d315152d58a7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:44.434673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-517b57b9-e67f-4a72-8bc9-d315152d58a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:44.444798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5165889-ada0-4ad5-9a4d-74130928ba7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=514d0cdb1f2641c9a259f7cf555486b7, user_id=799e63bdb159406286a90a5adeb16cc3, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:44.451128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517b57b9-e67f-4a72-8bc9-d315152d58a7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:44.453309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5165889-ada0-4ad5-9a4d-74130928ba7e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:44.466704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc025531-ebf8-4c73-a8d6-c5e8983bc71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:44.467610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc025531-ebf8-4c73-a8d6-c5e8983bc71a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:44.467899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc025531-ebf8-4c73-a8d6-c5e8983bc71a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:44.468185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc025531-ebf8-4c73-a8d6-c5e8983bc71a admin admin] PATH_INFO: `/v3/projects/158c1665476643cda0976e68ab5e595a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:44.477378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc025531-ebf8-4c73-a8d6-c5e8983bc71a admin admin] RBAC: Authorizing `identity:delete_project(project_id=158c1665476643cda0976e68ab5e595a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:44.481141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc025531-ebf8-4c73-a8d6-c5e8983bc71a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:44.608265 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1717/3438] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:11:44 2022] PUT /identity/v3/projects/514d0cdb1f2641c9a259f7cf555486b7/users/799e63bdb159406286a90a5adeb16cc3/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:44.619295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962b69b-89ce-4daf-9696-a06357625553 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:44.619751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962b69b-89ce-4daf-9696-a06357625553 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:44.620167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962b69b-89ce-4daf-9696-a06357625553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:44.668351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6962b69b-89ce-4daf-9696-a06357625553 None None] MFA Rules not processed for user `799e63bdb159406286a90a5adeb16cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:44.670098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6962b69b-89ce-4daf-9696-a06357625553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:44.672392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc025531-ebf8-4c73-a8d6-c5e8983bc71a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '158c1665476643cda0976e68ab5e595a', 'request_id': 'req-cc025531-ebf8-4c73-a8d6-c5e8983bc71a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:44.687988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc025531-ebf8-4c73-a8d6-c5e8983bc71a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '158c1665476643cda0976e68ab5e595a', 'request_id': 'req-cc025531-ebf8-4c73-a8d6-c5e8983bc71a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:44.715302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:44.772056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc025531-ebf8-4c73-a8d6-c5e8983bc71a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '158c1665476643cda0976e68ab5e595a', 'request_id': 'req-cc025531-ebf8-4c73-a8d6-c5e8983bc71a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:44.799902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc025531-ebf8-4c73-a8d6-c5e8983bc71a admin admin] The token cache is being invalidate because project 158c1665476643cda0976e68ab5e595a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:11:44.802078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc025531-ebf8-4c73-a8d6-c5e8983bc71a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:44.805611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc025531-ebf8-4c73-a8d6-c5e8983bc71a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:44.812207 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1722/3439] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:11:44 2022] DELETE /identity/v3/projects/158c1665476643cda0976e68ab5e595a => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:44.818054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa441c00-7583-47bb-8732-c269c1d778cf None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:44.818349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa441c00-7583-47bb-8732-c269c1d778cf None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:44.822013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa441c00-7583-47bb-8732-c269c1d778cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:44.838570 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1718/3440] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:44.907310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb52fe-19e6-4f61-bbe4-8ab610b3dedf None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:44.907621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb52fe-19e6-4f61-bbe4-8ab610b3dedf None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:44.907912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb52fe-19e6-4f61-bbe4-8ab610b3dedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:44.915465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-aa441c00-7583-47bb-8732-c269c1d778cf None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:44.919674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-aa441c00-7583-47bb-8732-c269c1d778cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:44.947828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-daeb52fe-19e6-4f61-bbe4-8ab610b3dedf None None] MFA Rules not processed for user `799e63bdb159406286a90a5adeb16cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:44.953029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-daeb52fe-19e6-4f61-bbe4-8ab610b3dedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:44.972805 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1719/3441] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:44.984405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb52fe-19e6-4f61-bbe4-8ab610b3dedf None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:44.986497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-daeb52fe-19e6-4f61-bbe4-8ab610b3dedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:45.081429 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1723/3442] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:45.083828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b51713-1f1d-4b59-a055-6a38a19c0fc9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:45.084535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b51713-1f1d-4b59-a055-6a38a19c0fc9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:45.085168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b51713-1f1d-4b59-a055-6a38a19c0fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:45.134476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a0b51713-1f1d-4b59-a055-6a38a19c0fc9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:45.136858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a0b51713-1f1d-4b59-a055-6a38a19c0fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:45.169596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb52fe-19e6-4f61-bbe4-8ab610b3dedf None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:45.173788 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1724/3443] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:45.176023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b51713-1f1d-4b59-a055-6a38a19c0fc9 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:45.188104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3982a1-bf2b-4c77-ac57-36ba82e0df6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:45.188932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3982a1-bf2b-4c77-ac57-36ba82e0df6c service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:45.189233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3982a1-bf2b-4c77-ac57-36ba82e0df6c service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:45.189514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3982a1-bf2b-4c77-ac57-36ba82e0df6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:45.196149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7c3982a1-bf2b-4c77-ac57-36ba82e0df6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:45.206771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b51713-1f1d-4b59-a055-6a38a19c0fc9 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:45.219019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3982a1-bf2b-4c77-ac57-36ba82e0df6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:45.220901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3982a1-bf2b-4c77-ac57-36ba82e0df6c service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:45.223996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7c76a-12d6-493a-b340-6ec65bf7551a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:45.224898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7c76a-12d6-493a-b340-6ec65bf7551a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:45.225575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7c76a-12d6-493a-b340-6ec65bf7551a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:45.226024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7c76a-12d6-493a-b340-6ec65bf7551a admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:45.226938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7c76a-12d6-493a-b340-6ec65bf7551a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:11:45.232368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7c76a-12d6-493a-b340-6ec65bf7551a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:45.239894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7c76a-12d6-493a-b340-6ec65bf7551a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:45.245356 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1720/3444] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:11:45.281904 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1725/3445] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:11:45.284387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3982a1-bf2b-4c77-ac57-36ba82e0df6c service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:45.305283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3982a1-bf2b-4c77-ac57-36ba82e0df6c service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:45.311191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55c0cd-6af5-4779-ba5c-74bf5a265bbe None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:45.311483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55c0cd-6af5-4779-ba5c-74bf5a265bbe None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:45.311789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55c0cd-6af5-4779-ba5c-74bf5a265bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:45.324885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f261c76-23c1-49fe-8b32-1dce6663aec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:45.329477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f261c76-23c1-49fe-8b32-1dce6663aec9 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:45.329768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f261c76-23c1-49fe-8b32-1dce6663aec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:45.330096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f261c76-23c1-49fe-8b32-1dce6663aec9 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:45.330820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f261c76-23c1-49fe-8b32-1dce6663aec9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:45.332804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f261c76-23c1-49fe-8b32-1dce6663aec9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:45.372005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2b55c0cd-6af5-4779-ba5c-74bf5a265bbe None None] MFA Rules not processed for user `799e63bdb159406286a90a5adeb16cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:45.373429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2b55c0cd-6af5-4779-ba5c-74bf5a265bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:45.402817 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1726/3446] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:45.449088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad9854-d720-47a0-804e-678899f7d90f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:45.449675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad9854-d720-47a0-804e-678899f7d90f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:45.451155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dad9854-d720-47a0-804e-678899f7d90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:45.453746 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1721/3447] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:11:45 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:45.458597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f261c76-23c1-49fe-8b32-1dce6663aec9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:45.494723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9dad9854-d720-47a0-804e-678899f7d90f None None] MFA Rules not processed for user `799e63bdb159406286a90a5adeb16cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:45.496142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9dad9854-d720-47a0-804e-678899f7d90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:45.664280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:45.673702 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1727/3448] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:45.675195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f261c76-23c1-49fe-8b32-1dce6663aec9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:45.742756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102417a-bfa9-4e45-a6fb-b83248b71dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:45.743305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd457d5-5329-44a0-b011-8dfcdadb5f40 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:45.743605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd457d5-5329-44a0-b011-8dfcdadb5f40 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:45.743909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd457d5-5329-44a0-b011-8dfcdadb5f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:45.746111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102417a-bfa9-4e45-a6fb-b83248b71dea admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:45.746474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102417a-bfa9-4e45-a6fb-b83248b71dea admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:45.746828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102417a-bfa9-4e45-a6fb-b83248b71dea admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:45.747652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102417a-bfa9-4e45-a6fb-b83248b71dea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:45.756159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102417a-bfa9-4e45-a6fb-b83248b71dea admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:45.801186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:45.815536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-afd457d5-5329-44a0-b011-8dfcdadb5f40 None None] MFA Rules not processed for user `799e63bdb159406286a90a5adeb16cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:45.817026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-afd457d5-5329-44a0-b011-8dfcdadb5f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:45.854103 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1722/3449] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:11:45 2022] POST /identity/v3/users => generated 287 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:11:45.858367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102417a-bfa9-4e45-a6fb-b83248b71dea admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:45.879723 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1728/3450] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:45.888599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102417a-bfa9-4e45-a6fb-b83248b71dea admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:45.909147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8827a-f276-4762-aa97-fe552a10d269 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:45.909443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8827a-f276-4762-aa97-fe552a10d269 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:45.909756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e8827a-f276-4762-aa97-fe552a10d269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:45.915290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a11997-80a6-481e-871b-0695babe64cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:45.915994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a11997-80a6-481e-871b-0695babe64cb admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:45.916281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a11997-80a6-481e-871b-0695babe64cb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:45.916563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a11997-80a6-481e-871b-0695babe64cb admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:45.923208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a11997-80a6-481e-871b-0695babe64cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:45.925379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a11997-80a6-481e-871b-0695babe64cb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:45.932200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c8e8827a-f276-4762-aa97-fe552a10d269 None None] MFA Rules not processed for user `799e63bdb159406286a90a5adeb16cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:45.934278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c8e8827a-f276-4762-aa97-fe552a10d269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:45.971522 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1729/3451] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:11:45.971522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b281b-886e-4ae6-8549-6c4d2183936a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:45.971522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b281b-886e-4ae6-8549-6c4d2183936a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:45.971522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b281b-886e-4ae6-8549-6c4d2183936a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:45.977347 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1723/3452] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:45.979011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a11997-80a6-481e-871b-0695babe64cb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:45.992908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a11997-80a6-481e-871b-0695babe64cb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:46.000001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e0ae0-aae3-4787-8064-961a57452c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:46.000826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e0ae0-aae3-4787-8064-961a57452c2a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:46.001113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e0ae0-aae3-4787-8064-961a57452c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:46.001395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e0ae0-aae3-4787-8064-961a57452c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:46.002771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e0ae0-aae3-4787-8064-961a57452c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:46.004825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e0ae0-aae3-4787-8064-961a57452c2a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:46.019601 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1724/3453] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:46.023186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e0ae0-aae3-4787-8064-961a57452c2a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:46.031019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-183e0ae0-aae3-4787-8064-961a57452c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:46.040738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4c8b281b-886e-4ae6-8549-6c4d2183936a None None] MFA Rules not processed for user `cafc81a185c645e893a77b7fd297d950`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:46.042069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4c8b281b-886e-4ae6-8549-6c4d2183936a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:46.051614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e0ae0-aae3-4787-8064-961a57452c2a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:46.065259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b8d0e-91c4-4d80-ba13-0d500baede06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:46.066239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b8d0e-91c4-4d80-ba13-0d500baede06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:46.066527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b8d0e-91c4-4d80-ba13-0d500baede06 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:46.066808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b8d0e-91c4-4d80-ba13-0d500baede06 admin admin] PATH_INFO: `/v3/users/799e63bdb159406286a90a5adeb16cc3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:46.071880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b8d0e-91c4-4d80-ba13-0d500baede06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=799e63bdb159406286a90a5adeb16cc3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:46.076828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b8d0e-91c4-4d80-ba13-0d500baede06 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:46.161508 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1730/3454] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:46.162625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b281b-886e-4ae6-8549-6c4d2183936a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:46.174691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b281b-886e-4ae6-8549-6c4d2183936a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:46.179920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51a03f-697c-4a20-8464-fb186e297346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:46.181054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51a03f-697c-4a20-8464-fb186e297346 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:46.181477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51a03f-697c-4a20-8464-fb186e297346 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:46.182320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51a03f-697c-4a20-8464-fb186e297346 admin admin] PATH_INFO: `/v3/projects/8e63642fba3b4b72bc6297f4b85a1119/users/32caeee3c02547e7bffce1e68ba81294/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:46.203943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51a03f-697c-4a20-8464-fb186e297346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e63642fba3b4b72bc6297f4b85a1119, user_id=32caeee3c02547e7bffce1e68ba81294, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:46.207528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51a03f-697c-4a20-8464-fb186e297346 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:46.351722 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1731/3455] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:11:46 2022] PUT /identity/v3/projects/8e63642fba3b4b72bc6297f4b85a1119/users/32caeee3c02547e7bffce1e68ba81294/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:46.366393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51a03f-697c-4a20-8464-fb186e297346 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:46.371421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3d51a03f-697c-4a20-8464-fb186e297346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:46.382596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-344b8d0e-91c4-4d80-ba13-0d500baede06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '799e63bdb159406286a90a5adeb16cc3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:46.383406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-344b8d0e-91c4-4d80-ba13-0d500baede06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '799e63bdb159406286a90a5adeb16cc3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:46.402681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-344b8d0e-91c4-4d80-ba13-0d500baede06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '799e63bdb159406286a90a5adeb16cc3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:46.464993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-344b8d0e-91c4-4d80-ba13-0d500baede06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '799e63bdb159406286a90a5adeb16cc3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:46.465923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-344b8d0e-91c4-4d80-ba13-0d500baede06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '799e63bdb159406286a90a5adeb16cc3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:46.478626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51a03f-697c-4a20-8464-fb186e297346 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:46.492159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6caca7-551b-4e80-b116-7cd5a0253f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:46.492895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6caca7-551b-4e80-b116-7cd5a0253f43 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:46.493185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6caca7-551b-4e80-b116-7cd5a0253f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:46.493474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6caca7-551b-4e80-b116-7cd5a0253f43 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:46.494209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6caca7-551b-4e80-b116-7cd5a0253f43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:46.499325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6caca7-551b-4e80-b116-7cd5a0253f43 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:46.508745 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1725/3456] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:11:46 2022] DELETE /identity/v3/users/799e63bdb159406286a90a5adeb16cc3 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:46.508745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccda13e-b4f4-41df-bad0-b2ff03581640 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:46.508745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccda13e-b4f4-41df-bad0-b2ff03581640 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:46.508745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccda13e-b4f4-41df-bad0-b2ff03581640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:46.547312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4ccda13e-b4f4-41df-bad0-b2ff03581640 None None] MFA Rules not processed for user `32caeee3c02547e7bffce1e68ba81294`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:46.548783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4ccda13e-b4f4-41df-bad0-b2ff03581640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:46.656952 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1726/3457] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:46.659150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816f748-d41f-4661-8896-4413fa63380a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:46.659626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816f748-d41f-4661-8896-4413fa63380a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:46.660573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816f748-d41f-4661-8896-4413fa63380a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:46.667518 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1732/3458] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:11:46 2022] POST /identity/v3/projects => generated 379 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:46.680738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6caca7-551b-4e80-b116-7cd5a0253f43 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:46.687346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b816f748-d41f-4661-8896-4413fa63380a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:46.689554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b816f748-d41f-4661-8896-4413fa63380a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:46.689659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ea6caca7-551b-4e80-b116-7cd5a0253f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:46.699245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:46.850123 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1727/3459] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:46.850742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6caca7-551b-4e80-b116-7cd5a0253f43 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:46.851613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816f748-d41f-4661-8896-4413fa63380a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:46.855133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b816f748-d41f-4661-8896-4413fa63380a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:46.867131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918c3a4-3943-40fe-aabb-7795c170217b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:46.867833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918c3a4-3943-40fe-aabb-7795c170217b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:46.868121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918c3a4-3943-40fe-aabb-7795c170217b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:46.868403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918c3a4-3943-40fe-aabb-7795c170217b admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:46.869117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918c3a4-3943-40fe-aabb-7795c170217b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:46.871252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918c3a4-3943-40fe-aabb-7795c170217b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:46.886543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:47.001664 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1733/3460] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:11:46 2022] POST /identity/v3/users => generated 296 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:47.027584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918c3a4-3943-40fe-aabb-7795c170217b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:47.034041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b918c3a4-3943-40fe-aabb-7795c170217b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:47.055004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816f748-d41f-4661-8896-4413fa63380a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:47.088441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e5983-8873-48fb-850d-517fd09b9447 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:47.089124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e5983-8873-48fb-850d-517fd09b9447 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:47.089456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e5983-8873-48fb-850d-517fd09b9447 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:47.090931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e5983-8873-48fb-850d-517fd09b9447 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:47.161868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918c3a4-3943-40fe-aabb-7795c170217b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:47.166808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648c846-3d1b-4951-ab92-cad83d5e4e5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:47.168490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648c846-3d1b-4951-ab92-cad83d5e4e5c service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:47.168986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648c846-3d1b-4951-ab92-cad83d5e4e5c service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:47.169486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648c846-3d1b-4951-ab92-cad83d5e4e5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:47.191298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648c846-3d1b-4951-ab92-cad83d5e4e5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:47.193715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e5983-8873-48fb-850d-517fd09b9447 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:47.196015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89e5983-8873-48fb-850d-517fd09b9447 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:47.196605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6648c846-3d1b-4951-ab92-cad83d5e4e5c service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:47.219925 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1734/3461] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:47.223960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648c846-3d1b-4951-ab92-cad83d5e4e5c service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:47.224438 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1728/3462] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:47.242006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648c846-3d1b-4951-ab92-cad83d5e4e5c service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:47.249565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfddbbe-9d6b-4b04-b721-571d4a42009a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:47.250868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfddbbe-9d6b-4b04-b721-571d4a42009a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:47.251298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfddbbe-9d6b-4b04-b721-571d4a42009a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:47.251958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfddbbe-9d6b-4b04-b721-571d4a42009a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:47.253286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfddbbe-9d6b-4b04-b721-571d4a42009a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:47.257856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfddbbe-9d6b-4b04-b721-571d4a42009a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:47.281432 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1735/3463] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:47.286190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e5983-8873-48fb-850d-517fd09b9447 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:47.317592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e5983-8873-48fb-850d-517fd09b9447 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:47.330528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655698bf-252a-4355-88f9-2cb6d328bb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:47.331588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655698bf-252a-4355-88f9-2cb6d328bb34 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:47.331939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655698bf-252a-4355-88f9-2cb6d328bb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:47.332293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655698bf-252a-4355-88f9-2cb6d328bb34 admin admin] PATH_INFO: `/v3/projects/cf23d1694b31430f9c7315c7c3195937/users/2d473cc820a94134a6fbd72bf6b1a0a6/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:47.382508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655698bf-252a-4355-88f9-2cb6d328bb34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf23d1694b31430f9c7315c7c3195937, user_id=2d473cc820a94134a6fbd72bf6b1a0a6, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:47.390035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655698bf-252a-4355-88f9-2cb6d328bb34 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:47.500028 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1729/3464] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:11:47 2022] PUT /identity/v3/projects/cf23d1694b31430f9c7315c7c3195937/users/2d473cc820a94134a6fbd72bf6b1a0a6/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:47.504182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfddbbe-9d6b-4b04-b721-571d4a42009a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:47.520515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfddbbe-9d6b-4b04-b721-571d4a42009a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:47.657470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b0883-408b-4de1-937f-aafc0d9fc560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:47.658222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b0883-408b-4de1-937f-aafc0d9fc560 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:47.658512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b0883-408b-4de1-937f-aafc0d9fc560 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:47.658786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356b0883-408b-4de1-937f-aafc0d9fc560 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:47.661005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b0883-408b-4de1-937f-aafc0d9fc560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:47.664485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356b0883-408b-4de1-937f-aafc0d9fc560 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:47.715242 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1736/3465] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:47.716957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655698bf-252a-4355-88f9-2cb6d328bb34 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:47.733229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655698bf-252a-4355-88f9-2cb6d328bb34 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:47.744723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78fb37-11b4-4e61-bc93-e6c6e1ec26ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:47.747359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78fb37-11b4-4e61-bc93-e6c6e1ec26ca admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:47.754075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78fb37-11b4-4e61-bc93-e6c6e1ec26ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:47.755047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78fb37-11b4-4e61-bc93-e6c6e1ec26ca admin admin] PATH_INFO: `/v3/projects/cf23d1694b31430f9c7315c7c3195937/users/2d473cc820a94134a6fbd72bf6b1a0a6/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:47.761091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78fb37-11b4-4e61-bc93-e6c6e1ec26ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf23d1694b31430f9c7315c7c3195937, user_id=2d473cc820a94134a6fbd72bf6b1a0a6, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:47.769581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be78fb37-11b4-4e61-bc93-e6c6e1ec26ca admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:47.791258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:47.796364 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1730/3466] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:11:47 2022] PUT /identity/v3/projects/cf23d1694b31430f9c7315c7c3195937/users/2d473cc820a94134a6fbd72bf6b1a0a6/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:48.973943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b0883-408b-4de1-937f-aafc0d9fc560 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:48.976023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-356b0883-408b-4de1-937f-aafc0d9fc560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:49.001081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356b0883-408b-4de1-937f-aafc0d9fc560 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:49.057506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78fb37-11b4-4e61-bc93-e6c6e1ec26ca admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:49.060543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-be78fb37-11b4-4e61-bc93-e6c6e1ec26ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:49.101824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:49.120903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85f3574-0591-45e0-91e0-de538f4d0d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:49.120903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f3574-0591-45e0-91e0-de538f4d0d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:49.120903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f3574-0591-45e0-91e0-de538f4d0d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:49.120903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85f3574-0591-45e0-91e0-de538f4d0d06 admin admin] PATH_INFO: `/v3/users/659bd3ebefc348b7a176379bd342f9ed` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:49.121236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f3574-0591-45e0-91e0-de538f4d0d06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=659bd3ebefc348b7a176379bd342f9ed)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:49.124669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85f3574-0591-45e0-91e0-de538f4d0d06 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:49.195362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be78fb37-11b4-4e61-bc93-e6c6e1ec26ca admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:49.208927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a50424-4ecb-474f-ab24-6a2ec5d09d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:49.210500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a50424-4ecb-474f-ab24-6a2ec5d09d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:49.210986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a50424-4ecb-474f-ab24-6a2ec5d09d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:49.211455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a50424-4ecb-474f-ab24-6a2ec5d09d9b admin admin] PATH_INFO: `/v3/projects/514d0cdb1f2641c9a259f7cf555486b7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:49.212905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a50424-4ecb-474f-ab24-6a2ec5d09d9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=514d0cdb1f2641c9a259f7cf555486b7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:49.216788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a50424-4ecb-474f-ab24-6a2ec5d09d9b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:49.505697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f0a50424-4ecb-474f-ab24-6a2ec5d09d9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '514d0cdb1f2641c9a259f7cf555486b7', 'request_id': 'req-f0a50424-4ecb-474f-ab24-6a2ec5d09d9b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:49.517128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f0a50424-4ecb-474f-ab24-6a2ec5d09d9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '514d0cdb1f2641c9a259f7cf555486b7', 'request_id': 'req-f0a50424-4ecb-474f-ab24-6a2ec5d09d9b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:49.522274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e85f3574-0591-45e0-91e0-de538f4d0d06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '659bd3ebefc348b7a176379bd342f9ed'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:49.543099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e85f3574-0591-45e0-91e0-de538f4d0d06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '659bd3ebefc348b7a176379bd342f9ed'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:49.545913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e85f3574-0591-45e0-91e0-de538f4d0d06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '659bd3ebefc348b7a176379bd342f9ed'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:49.547200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e85f3574-0591-45e0-91e0-de538f4d0d06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '659bd3ebefc348b7a176379bd342f9ed'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:49.623855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f0a50424-4ecb-474f-ab24-6a2ec5d09d9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '514d0cdb1f2641c9a259f7cf555486b7', 'request_id': 'req-f0a50424-4ecb-474f-ab24-6a2ec5d09d9b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:49.645493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f0a50424-4ecb-474f-ab24-6a2ec5d09d9b admin admin] The token cache is being invalidate because project 514d0cdb1f2641c9a259f7cf555486b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:11:49.645493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f0a50424-4ecb-474f-ab24-6a2ec5d09d9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:49.649354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f0a50424-4ecb-474f-ab24-6a2ec5d09d9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:49.660367 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1731/3467] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:11:49 2022] DELETE /identity/v3/projects/514d0cdb1f2641c9a259f7cf555486b7 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:49.664982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e4d58-7537-4dba-b717-8a30f22af177 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:49.665408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e4d58-7537-4dba-b717-8a30f22af177 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:49.665817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e4d58-7537-4dba-b717-8a30f22af177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:49.699605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b38e4d58-7537-4dba-b717-8a30f22af177 None None] MFA Rules not processed for user `04ae3115a9194b47bce62ada4a2afdf2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:49.702280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b38e4d58-7537-4dba-b717-8a30f22af177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:49.725227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e85f3574-0591-45e0-91e0-de538f4d0d06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '659bd3ebefc348b7a176379bd342f9ed'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:49.777571 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1737/3468] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:11:48 2022] DELETE /identity/v3/users/659bd3ebefc348b7a176379bd342f9ed => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:49.782094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302928b7-2815-41a1-812e-2b326683e012 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:49.782606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302928b7-2815-41a1-812e-2b326683e012 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:49.783093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302928b7-2815-41a1-812e-2b326683e012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:49.819088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-302928b7-2815-41a1-812e-2b326683e012 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:49.820712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-302928b7-2815-41a1-812e-2b326683e012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:49.876573 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1732/3469] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:49.967145 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1738/3470] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:49.978951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d1bdc-0304-46a3-b89f-2fbaa41b780e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:49.979391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d1bdc-0304-46a3-b89f-2fbaa41b780e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:49.979683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2d1bdc-0304-46a3-b89f-2fbaa41b780e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:50.051523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3d2d1bdc-0304-46a3-b89f-2fbaa41b780e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:50.053430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3d2d1bdc-0304-46a3-b89f-2fbaa41b780e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:50.083939 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1733/3471] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:50.087962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302928b7-2815-41a1-812e-2b326683e012 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:50.107243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302928b7-2815-41a1-812e-2b326683e012 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:50.117971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0586400-5d58-446f-81f6-51b9fe222e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:50.122179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0586400-5d58-446f-81f6-51b9fe222e9a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:50.122710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0586400-5d58-446f-81f6-51b9fe222e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:50.123136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0586400-5d58-446f-81f6-51b9fe222e9a admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:50.124048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0586400-5d58-446f-81f6-51b9fe222e9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:11:50.124504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0586400-5d58-446f-81f6-51b9fe222e9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:50.127171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0586400-5d58-446f-81f6-51b9fe222e9a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:50.142651 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1739/3472] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:11:50.147392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d1bdc-0304-46a3-b89f-2fbaa41b780e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:50.170824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2d1bdc-0304-46a3-b89f-2fbaa41b780e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:50.180182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6dfd7-298d-47ae-a131-e10459c78ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:50.181351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6dfd7-298d-47ae-a131-e10459c78ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:50.182502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6dfd7-298d-47ae-a131-e10459c78ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:50.182946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6dfd7-298d-47ae-a131-e10459c78ef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:50.184435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6dfd7-298d-47ae-a131-e10459c78ef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:50.190104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6dfd7-298d-47ae-a131-e10459c78ef0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:50.311116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:50.320757 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1734/3473] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:11:50 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:50.325374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0586400-5d58-446f-81f6-51b9fe222e9a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:50.336702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0586400-5d58-446f-81f6-51b9fe222e9a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:50.367477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6dfd7-298d-47ae-a131-e10459c78ef0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:50.388574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-33b6dfd7-298d-47ae-a131-e10459c78ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:50.453496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9959f9c-afba-4e4d-8ca7-dc9922194d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:50.454652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9959f9c-afba-4e4d-8ca7-dc9922194d8e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:50.455801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9959f9c-afba-4e4d-8ca7-dc9922194d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:50.456126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9959f9c-afba-4e4d-8ca7-dc9922194d8e admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:50.456995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9959f9c-afba-4e4d-8ca7-dc9922194d8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:50.460605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9959f9c-afba-4e4d-8ca7-dc9922194d8e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:50.526039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6dfd7-298d-47ae-a131-e10459c78ef0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:50.532336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423dfac-afde-4ed2-82e8-0e86503e219d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:50.533242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423dfac-afde-4ed2-82e8-0e86503e219d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:50.533526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423dfac-afde-4ed2-82e8-0e86503e219d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:50.533805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423dfac-afde-4ed2-82e8-0e86503e219d admin admin] PATH_INFO: `/v3/projects/cf209a1ba94046f89ddf62c77b6b0f02` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:50.535164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423dfac-afde-4ed2-82e8-0e86503e219d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf209a1ba94046f89ddf62c77b6b0f02)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:50.537360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423dfac-afde-4ed2-82e8-0e86503e219d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:50.570935 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1740/3474] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:11:50 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:50.571926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9959f9c-afba-4e4d-8ca7-dc9922194d8e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:50.574482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e9959f9c-afba-4e4d-8ca7-dc9922194d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:50.592723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9959f9c-afba-4e4d-8ca7-dc9922194d8e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:50.599519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19a0d0-214d-48eb-b54b-1e56a6f1e415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:50.600533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19a0d0-214d-48eb-b54b-1e56a6f1e415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:50.601340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19a0d0-214d-48eb-b54b-1e56a6f1e415 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:50.601889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19a0d0-214d-48eb-b54b-1e56a6f1e415 admin admin] PATH_INFO: `/v3/users/32caeee3c02547e7bffce1e68ba81294` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:50.603435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19a0d0-214d-48eb-b54b-1e56a6f1e415 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32caeee3c02547e7bffce1e68ba81294)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:50.614811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f19a0d0-214d-48eb-b54b-1e56a6f1e415 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:50.725883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b423dfac-afde-4ed2-82e8-0e86503e219d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf209a1ba94046f89ddf62c77b6b0f02', 'request_id': 'req-b423dfac-afde-4ed2-82e8-0e86503e219d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:50.736062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b423dfac-afde-4ed2-82e8-0e86503e219d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf209a1ba94046f89ddf62c77b6b0f02', 'request_id': 'req-b423dfac-afde-4ed2-82e8-0e86503e219d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:50.783423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:50.875056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b423dfac-afde-4ed2-82e8-0e86503e219d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf209a1ba94046f89ddf62c77b6b0f02', 'request_id': 'req-b423dfac-afde-4ed2-82e8-0e86503e219d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:50.894107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b423dfac-afde-4ed2-82e8-0e86503e219d admin admin] The token cache is being invalidate because project cf209a1ba94046f89ddf62c77b6b0f02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:11:50.894730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b423dfac-afde-4ed2-82e8-0e86503e219d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:50.895707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b423dfac-afde-4ed2-82e8-0e86503e219d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:50.897783 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1735/3475] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:11:50 2022] DELETE /identity/v3/projects/cf209a1ba94046f89ddf62c77b6b0f02 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:50.903398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423dfac-afde-4ed2-82e8-0e86503e219d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:50.903398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b423dfac-afde-4ed2-82e8-0e86503e219d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:50.979456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4f19a0d0-214d-48eb-b54b-1e56a6f1e415 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32caeee3c02547e7bffce1e68ba81294'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:50.999749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4f19a0d0-214d-48eb-b54b-1e56a6f1e415 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32caeee3c02547e7bffce1e68ba81294'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:51.001226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4f19a0d0-214d-48eb-b54b-1e56a6f1e415 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32caeee3c02547e7bffce1e68ba81294'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:51.003430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4f19a0d0-214d-48eb-b54b-1e56a6f1e415 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32caeee3c02547e7bffce1e68ba81294'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:51.029618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423dfac-afde-4ed2-82e8-0e86503e219d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:51.034881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a666175-693b-4809-b2c3-85a8c619619d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:51.035624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a666175-693b-4809-b2c3-85a8c619619d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:51.035907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a666175-693b-4809-b2c3-85a8c619619d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:51.036415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a666175-693b-4809-b2c3-85a8c619619d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:51.037229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a666175-693b-4809-b2c3-85a8c619619d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:51.039538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a666175-693b-4809-b2c3-85a8c619619d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:51.061995 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1736/3476] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:51.061995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ddcd4-d187-45e1-b843-3db52f18a524 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:51.061995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ddcd4-d187-45e1-b843-3db52f18a524 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:51.061995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7ddcd4-d187-45e1-b843-3db52f18a524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:51.093399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0b7ddcd4-d187-45e1-b843-3db52f18a524 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:51.094062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0b7ddcd4-d187-45e1-b843-3db52f18a524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:51.104440 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1737/3477] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:51.104926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ddcd4-d187-45e1-b843-3db52f18a524 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:51.106750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0b7ddcd4-d187-45e1-b843-3db52f18a524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:51.124742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7ddcd4-d187-45e1-b843-3db52f18a524 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:51.132491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4f19a0d0-214d-48eb-b54b-1e56a6f1e415 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32caeee3c02547e7bffce1e68ba81294'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:51.140118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53313448-6049-4aa6-9a43-28dff638b111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:51.143851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53313448-6049-4aa6-9a43-28dff638b111 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:51.144303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53313448-6049-4aa6-9a43-28dff638b111 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:51.147126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53313448-6049-4aa6-9a43-28dff638b111 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:51.148599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53313448-6049-4aa6-9a43-28dff638b111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:51.151980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53313448-6049-4aa6-9a43-28dff638b111 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:51.175204 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1738/3478] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:51.177254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b9528-44f9-471c-b8a1-7e5c564ff6b6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:51.178032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b9528-44f9-471c-b8a1-7e5c564ff6b6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:51.189314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217b9528-44f9-471c-b8a1-7e5c564ff6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:51.189404 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1741/3479] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:11:50 2022] DELETE /identity/v3/users/32caeee3c02547e7bffce1e68ba81294 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:51.189404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19a0d0-214d-48eb-b54b-1e56a6f1e415 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:51.208852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-217b9528-44f9-471c-b8a1-7e5c564ff6b6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:51.210516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-217b9528-44f9-471c-b8a1-7e5c564ff6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:51.303711 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1739/3480] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:51.303711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93323d-dc80-427b-a76e-0f6cd60dc780 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:51.303711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93323d-dc80-427b-a76e-0f6cd60dc780 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:51.303711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93323d-dc80-427b-a76e-0f6cd60dc780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:51.322535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19a0d0-214d-48eb-b54b-1e56a6f1e415 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:51.325810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1e93323d-dc80-427b-a76e-0f6cd60dc780 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:51.327660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1e93323d-dc80-427b-a76e-0f6cd60dc780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:51.329396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5061f-1719-4829-a03b-497623d77471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:51.330603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5061f-1719-4829-a03b-497623d77471 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:51.331145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5061f-1719-4829-a03b-497623d77471 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:51.331672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc5061f-1719-4829-a03b-497623d77471 admin admin] PATH_INFO: `/v3/projects/2e073d4d15a243eca3ea41985b8150b0/users/ce8ac48fe53b4080bf74c9788408f8c5/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:51.343206 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1740/3481] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:51.344296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93323d-dc80-427b-a76e-0f6cd60dc780 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:51.351758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5061f-1719-4829-a03b-497623d77471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e073d4d15a243eca3ea41985b8150b0, user_id=ce8ac48fe53b4080bf74c9788408f8c5, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:51.355525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc5061f-1719-4829-a03b-497623d77471 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:51.357121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93323d-dc80-427b-a76e-0f6cd60dc780 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:51.364316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b378b-a350-46aa-9567-f58a14276837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:51.365436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b378b-a350-46aa-9567-f58a14276837 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:51.365436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b378b-a350-46aa-9567-f58a14276837 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:51.365775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b378b-a350-46aa-9567-f58a14276837 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:51.366876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b378b-a350-46aa-9567-f58a14276837 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:11:51.367430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b378b-a350-46aa-9567-f58a14276837 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:51.369393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51b378b-a350-46aa-9567-f58a14276837 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:51.384792 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1741/3482] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:11:51.385814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b378b-a350-46aa-9567-f58a14276837 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:51.388920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e51b378b-a350-46aa-9567-f58a14276837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:51.437529 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1742/3483] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:11:51 2022] PUT /identity/v3/projects/2e073d4d15a243eca3ea41985b8150b0/users/ce8ac48fe53b4080bf74c9788408f8c5/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:51.439130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5061f-1719-4829-a03b-497623d77471 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:51.482346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b378b-a350-46aa-9567-f58a14276837 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:51.614376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc5061f-1719-4829-a03b-497623d77471 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:51.618757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23653d5d-9d3d-403a-ae9b-adc1b80858c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:51.619758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23653d5d-9d3d-403a-ae9b-adc1b80858c5 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:51.621279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23653d5d-9d3d-403a-ae9b-adc1b80858c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:51.621585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23653d5d-9d3d-403a-ae9b-adc1b80858c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:51.622948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2765d30-9b31-4192-b018-cba4ca3e5740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:51.624773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2765d30-9b31-4192-b018-cba4ca3e5740 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:51.625222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2765d30-9b31-4192-b018-cba4ca3e5740 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:51.625640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2765d30-9b31-4192-b018-cba4ca3e5740 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:51.630649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2765d30-9b31-4192-b018-cba4ca3e5740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:51.634096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23653d5d-9d3d-403a-ae9b-adc1b80858c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:51.635669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23653d5d-9d3d-403a-ae9b-adc1b80858c5 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:51.636440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2765d30-9b31-4192-b018-cba4ca3e5740 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:51.659241 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1742/3484] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:51.662232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0400f-6cc7-4709-8b26-88d2ff1c583f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:51.662542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0400f-6cc7-4709-8b26-88d2ff1c583f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:51.662830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0400f-6cc7-4709-8b26-88d2ff1c583f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:51.687381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6cb0400f-6cc7-4709-8b26-88d2ff1c583f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:51.689211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6cb0400f-6cc7-4709-8b26-88d2ff1c583f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:51.699078 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1743/3485] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:51.737617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0400f-6cc7-4709-8b26-88d2ff1c583f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:51.758364 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1743/3486] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:11:51 2022] POST /identity/v3/projects => generated 379 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:51.769238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2765d30-9b31-4192-b018-cba4ca3e5740 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:51.778949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0400f-6cc7-4709-8b26-88d2ff1c583f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:51.900722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d21c3-564a-42a4-8b9e-cd2826f2f3f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:51.901578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d21c3-564a-42a4-8b9e-cd2826f2f3f9 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:51.902586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d21c3-564a-42a4-8b9e-cd2826f2f3f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:51.902869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d21c3-564a-42a4-8b9e-cd2826f2f3f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:51.979239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2765d30-9b31-4192-b018-cba4ca3e5740 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:51.986666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e0090-d370-409a-a58a-0a7ae9e72342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:51.987317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e0090-d370-409a-a58a-0a7ae9e72342 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:51.989984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e0090-d370-409a-a58a-0a7ae9e72342 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:51.993865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e0090-d370-409a-a58a-0a7ae9e72342 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:51.993865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e0090-d370-409a-a58a-0a7ae9e72342 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:51.994586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e0090-d370-409a-a58a-0a7ae9e72342 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:52.088513 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1744/3487] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:11:51 2022] POST /identity/v3/users => generated 290 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:52.090224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d21c3-564a-42a4-8b9e-cd2826f2f3f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:52.092687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e0090-d370-409a-a58a-0a7ae9e72342 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:52.092891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d21c3-564a-42a4-8b9e-cd2826f2f3f9 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:52.109663 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1744/3488] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:52.114044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e0090-d370-409a-a58a-0a7ae9e72342 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:52.120799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e1d36-b1c1-48bc-a4a0-052fc5174ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:52.121433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e1d36-b1c1-48bc-a4a0-052fc5174ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:52.121658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e1d36-b1c1-48bc-a4a0-052fc5174ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:52.122064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e1d36-b1c1-48bc-a4a0-052fc5174ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:52.126273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e1d36-b1c1-48bc-a4a0-052fc5174ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:52.126273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e1d36-b1c1-48bc-a4a0-052fc5174ba9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:52.141503 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1745/3489] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:52.145480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d21c3-564a-42a4-8b9e-cd2826f2f3f9 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:52.158795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d21c3-564a-42a4-8b9e-cd2826f2f3f9 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:52.169424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce69ad-21f3-4e96-91a4-99547d7e115c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:52.171119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce69ad-21f3-4e96-91a4-99547d7e115c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:52.171119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce69ad-21f3-4e96-91a4-99547d7e115c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:52.171368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce69ad-21f3-4e96-91a4-99547d7e115c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:52.172326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce69ad-21f3-4e96-91a4-99547d7e115c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:52.175654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce69ad-21f3-4e96-91a4-99547d7e115c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:52.196480 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1745/3490] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:11:52.197735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e1d36-b1c1-48bc-a4a0-052fc5174ba9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:52.217911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e1d36-b1c1-48bc-a4a0-052fc5174ba9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:52.224777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc5d91-069b-432f-8edb-83f7a10de2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:52.226448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc5d91-069b-432f-8edb-83f7a10de2d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:52.226782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc5d91-069b-432f-8edb-83f7a10de2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:52.227405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc5d91-069b-432f-8edb-83f7a10de2d6 admin admin] PATH_INFO: `/v3/projects/e1f0267259d74463b29bda378389f5e4/users/28a6a9a1c0ed478eb54268fbb45b63d0/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:52.264671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc5d91-069b-432f-8edb-83f7a10de2d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1f0267259d74463b29bda378389f5e4, user_id=28a6a9a1c0ed478eb54268fbb45b63d0, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:52.267705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc5d91-069b-432f-8edb-83f7a10de2d6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:52.350756 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1746/3491] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:11:52 2022] PUT /identity/v3/projects/e1f0267259d74463b29bda378389f5e4/users/28a6a9a1c0ed478eb54268fbb45b63d0/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:52.352242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e73e7-c941-4ea0-8249-1b37c82dcf7b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:52.352800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e73e7-c941-4ea0-8249-1b37c82dcf7b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:52.352936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e73e7-c941-4ea0-8249-1b37c82dcf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:52.411637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-137e73e7-c941-4ea0-8249-1b37c82dcf7b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:52.413110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-137e73e7-c941-4ea0-8249-1b37c82dcf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:52.498888 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1746/3492] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:52.514320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc5d91-069b-432f-8edb-83f7a10de2d6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:52.514320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1edc5d91-069b-432f-8edb-83f7a10de2d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:52.638041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc5d91-069b-432f-8edb-83f7a10de2d6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:52.638041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1266bfc-6b01-4466-808d-c3953ce839f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:52.638041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1266bfc-6b01-4466-808d-c3953ce839f2 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:52.638041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1266bfc-6b01-4466-808d-c3953ce839f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:52.638555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1266bfc-6b01-4466-808d-c3953ce839f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:52.656883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1266bfc-6b01-4466-808d-c3953ce839f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:52.660352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1266bfc-6b01-4466-808d-c3953ce839f2 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:52.675766 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1747/3493] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:53.258448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:53.331424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e73e7-c941-4ea0-8249-1b37c82dcf7b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:53.333328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-137e73e7-c941-4ea0-8249-1b37c82dcf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:53.359643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e73e7-c941-4ea0-8249-1b37c82dcf7b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:53.373622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517342c3-4eb2-48f5-8bf9-2d2019496d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:53.376824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517342c3-4eb2-48f5-8bf9-2d2019496d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:53.394344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517342c3-4eb2-48f5-8bf9-2d2019496d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:53.394344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517342c3-4eb2-48f5-8bf9-2d2019496d17 admin admin] PATH_INFO: `/v3/projects/8e63642fba3b4b72bc6297f4b85a1119` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:53.396376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517342c3-4eb2-48f5-8bf9-2d2019496d17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e63642fba3b4b72bc6297f4b85a1119)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:53.404027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517342c3-4eb2-48f5-8bf9-2d2019496d17 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:53.671775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-517342c3-4eb2-48f5-8bf9-2d2019496d17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e63642fba3b4b72bc6297f4b85a1119', 'request_id': 'req-517342c3-4eb2-48f5-8bf9-2d2019496d17'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:53.681575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-517342c3-4eb2-48f5-8bf9-2d2019496d17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e63642fba3b4b72bc6297f4b85a1119', 'request_id': 'req-517342c3-4eb2-48f5-8bf9-2d2019496d17'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:53.804398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-517342c3-4eb2-48f5-8bf9-2d2019496d17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e63642fba3b4b72bc6297f4b85a1119', 'request_id': 'req-517342c3-4eb2-48f5-8bf9-2d2019496d17'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:53.846636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-517342c3-4eb2-48f5-8bf9-2d2019496d17 admin admin] The token cache is being invalidate because project 8e63642fba3b4b72bc6297f4b85a1119 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:11:53.863799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-517342c3-4eb2-48f5-8bf9-2d2019496d17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:53.863799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-517342c3-4eb2-48f5-8bf9-2d2019496d17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:53.863799 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1747/3494] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:11:53 2022] DELETE /identity/v3/projects/8e63642fba3b4b72bc6297f4b85a1119 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:53.864425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733f005-f132-43d1-bfd9-d148ea501d0f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:53.864425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733f005-f132-43d1-bfd9-d148ea501d0f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:53.864425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0733f005-f132-43d1-bfd9-d148ea501d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:53.892015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0733f005-f132-43d1-bfd9-d148ea501d0f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:53.893597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0733f005-f132-43d1-bfd9-d148ea501d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:54.000148 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1748/3495] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:54.007902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373ee21-d36a-4639-a823-06e76971eff9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:54.008203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373ee21-d36a-4639-a823-06e76971eff9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:54.008487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5373ee21-d36a-4639-a823-06e76971eff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:54.035664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:54.075614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5373ee21-d36a-4639-a823-06e76971eff9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:54.077013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5373ee21-d36a-4639-a823-06e76971eff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:54.102491 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1748/3496] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:54.109004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733f005-f132-43d1-bfd9-d148ea501d0f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:54.130856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0733f005-f132-43d1-bfd9-d148ea501d0f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:54.138759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3e2c1-615c-4cbb-8333-b074051fba30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:54.139666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3e2c1-615c-4cbb-8333-b074051fba30 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:54.140096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3e2c1-615c-4cbb-8333-b074051fba30 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:54.140656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3e2c1-615c-4cbb-8333-b074051fba30 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:54.141555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3e2c1-615c-4cbb-8333-b074051fba30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:11:54.142535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3e2c1-615c-4cbb-8333-b074051fba30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:54.144352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d3e2c1-615c-4cbb-8333-b074051fba30 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:54.177496 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1749/3497] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:11:54.182288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373ee21-d36a-4639-a823-06e76971eff9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:54.192635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb35f88-da63-4606-bb3c-db52be7fbfb1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:54.192968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb35f88-da63-4606-bb3c-db52be7fbfb1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:54.193256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb35f88-da63-4606-bb3c-db52be7fbfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:54.200158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5373ee21-d36a-4639-a823-06e76971eff9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:54.229160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1c945-80cd-46e1-b041-5813d14e389f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:54.229455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1c945-80cd-46e1-b041-5813d14e389f admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:54.229749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1c945-80cd-46e1-b041-5813d14e389f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:54.231831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1c945-80cd-46e1-b041-5813d14e389f admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:54.232562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc1c945-80cd-46e1-b041-5813d14e389f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:54.240368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc1c945-80cd-46e1-b041-5813d14e389f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:54.242425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6cb35f88-da63-4606-bb3c-db52be7fbfb1 None None] MFA Rules not processed for user `bf9c8f66e7d24ca9baca835e41546f22`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:54.243823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6cb35f88-da63-4606-bb3c-db52be7fbfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:54.394261 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1749/3498] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:11:54 2022] POST /identity/v3/projects => generated 383 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:54.399641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1c945-80cd-46e1-b041-5813d14e389f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:54.408289 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1750/3499] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:54.533244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1c945-80cd-46e1-b041-5813d14e389f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:54.538686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd852d2-9460-4b93-954d-b98794ec329e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:54.539334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd852d2-9460-4b93-954d-b98794ec329e admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:54.539786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd852d2-9460-4b93-954d-b98794ec329e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:54.540688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd852d2-9460-4b93-954d-b98794ec329e admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:54.541951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd852d2-9460-4b93-954d-b98794ec329e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:54.544420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd852d2-9460-4b93-954d-b98794ec329e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:54.726719 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1750/3500] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:11:54 2022] POST /identity/v3/users => generated 292 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:11:54.734314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb35f88-da63-4606-bb3c-db52be7fbfb1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:54.754651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb35f88-da63-4606-bb3c-db52be7fbfb1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:54.769021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1a6a4-8a06-4675-b547-dedbbdf6a070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:54.769718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1a6a4-8a06-4675-b547-dedbbdf6a070 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:54.770297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1a6a4-8a06-4675-b547-dedbbdf6a070 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:54.770585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1a6a4-8a06-4675-b547-dedbbdf6a070 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:54.771485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1a6a4-8a06-4675-b547-dedbbdf6a070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:54.775129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1a6a4-8a06-4675-b547-dedbbdf6a070 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:54.798633 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1751/3501] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:54.803801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd852d2-9460-4b93-954d-b98794ec329e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:54.829308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd852d2-9460-4b93-954d-b98794ec329e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:54.864230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d84036-74f3-446d-9d50-41e470e43c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:54.865098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d84036-74f3-446d-9d50-41e470e43c28 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:54.866025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d84036-74f3-446d-9d50-41e470e43c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:54.867182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d84036-74f3-446d-9d50-41e470e43c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:54.868160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d84036-74f3-446d-9d50-41e470e43c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:54.871431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d84036-74f3-446d-9d50-41e470e43c28 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:54.890314 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1751/3502] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:54.899322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1a6a4-8a06-4675-b547-dedbbdf6a070 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:54.905382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1a6a4-8a06-4675-b547-dedbbdf6a070 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:54.916102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040441c-efdf-4133-b2d3-d730d8d5f6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:54.917791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040441c-efdf-4133-b2d3-d730d8d5f6ae admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:54.918495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040441c-efdf-4133-b2d3-d730d8d5f6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:54.919115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040441c-efdf-4133-b2d3-d730d8d5f6ae admin admin] PATH_INFO: `/v3/projects/53ecc00f85b54d01aa7dbe2f0386dac6/users/25b2f1524f7f48149cbc5d2c694a9e2b/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:54.994376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7040441c-efdf-4133-b2d3-d730d8d5f6ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=53ecc00f85b54d01aa7dbe2f0386dac6, user_id=25b2f1524f7f48149cbc5d2c694a9e2b, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:54.998823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7040441c-efdf-4133-b2d3-d730d8d5f6ae admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:55.114555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737488d-1bbf-4937-a905-94b86b6e7a7f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:55.115116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737488d-1bbf-4937-a905-94b86b6e7a7f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:55.115805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737488d-1bbf-4937-a905-94b86b6e7a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:55.117651 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1752/3503] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:11:54 2022] PUT /identity/v3/projects/53ecc00f85b54d01aa7dbe2f0386dac6/users/25b2f1524f7f48149cbc5d2c694a9e2b/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:55.139574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1737488d-1bbf-4937-a905-94b86b6e7a7f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:55.141119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1737488d-1bbf-4937-a905-94b86b6e7a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:55.243723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040441c-efdf-4133-b2d3-d730d8d5f6ae admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:55.250851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7040441c-efdf-4133-b2d3-d730d8d5f6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:55.312776 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1752/3504] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:55.343170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737488d-1bbf-4937-a905-94b86b6e7a7f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:55.345172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1737488d-1bbf-4937-a905-94b86b6e7a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:55.398282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040441c-efdf-4133-b2d3-d730d8d5f6ae admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:55.404248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186c18f-d2fb-4dfa-ad8d-5bea5a2fb315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:55.405472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186c18f-d2fb-4dfa-ad8d-5bea5a2fb315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:55.405949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186c18f-d2fb-4dfa-ad8d-5bea5a2fb315 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:55.406390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186c18f-d2fb-4dfa-ad8d-5bea5a2fb315 admin admin] PATH_INFO: `/v3/users/408a54e3416b40479fae5fba67d14a53` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:55.408828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0186c18f-d2fb-4dfa-ad8d-5bea5a2fb315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=408a54e3416b40479fae5fba67d14a53)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:55.411355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0186c18f-d2fb-4dfa-ad8d-5bea5a2fb315 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:55.414444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737488d-1bbf-4937-a905-94b86b6e7a7f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:55.421621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a8535-a524-4667-827c-ddd6f5189d63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:55.422578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a8535-a524-4667-827c-ddd6f5189d63 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:55.423264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a8535-a524-4667-827c-ddd6f5189d63 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:55.423881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a8535-a524-4667-827c-ddd6f5189d63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:55.443021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a8535-a524-4667-827c-ddd6f5189d63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:55.446335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a8535-a524-4667-827c-ddd6f5189d63 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:55.464368 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1753/3505] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:11:55.750364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0186c18f-d2fb-4dfa-ad8d-5bea5a2fb315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '408a54e3416b40479fae5fba67d14a53'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:55.777383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0186c18f-d2fb-4dfa-ad8d-5bea5a2fb315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '408a54e3416b40479fae5fba67d14a53'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:55.779365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0186c18f-d2fb-4dfa-ad8d-5bea5a2fb315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '408a54e3416b40479fae5fba67d14a53'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:55.781110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0186c18f-d2fb-4dfa-ad8d-5bea5a2fb315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '408a54e3416b40479fae5fba67d14a53'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:55.874168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0186c18f-d2fb-4dfa-ad8d-5bea5a2fb315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '408a54e3416b40479fae5fba67d14a53'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:55.908370 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1753/3506] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:11:55 2022] DELETE /identity/v3/users/408a54e3416b40479fae5fba67d14a53 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:56.005446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c64ed-af25-4375-a316-315d09b44fe5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:56.005829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c64ed-af25-4375-a316-315d09b44fe5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:56.006227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253c64ed-af25-4375-a316-315d09b44fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:56.032402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fde8b0-5df1-4263-bad9-d6f8bc124f15 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:56.032843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fde8b0-5df1-4263-bad9-d6f8bc124f15 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:56.033133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fde8b0-5df1-4263-bad9-d6f8bc124f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:56.092007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f7fde8b0-5df1-4263-bad9-d6f8bc124f15 None None] MFA Rules not processed for user `28a6a9a1c0ed478eb54268fbb45b63d0`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:56.094120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f7fde8b0-5df1-4263-bad9-d6f8bc124f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:56.111296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-253c64ed-af25-4375-a316-315d09b44fe5 None None] MFA Rules not processed for user `ce8ac48fe53b4080bf74c9788408f8c5`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:56.113670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-253c64ed-af25-4375-a316-315d09b44fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:56.271413 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1754/3507] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:56.273062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f3896-d308-4163-b1d0-954f91bf7606 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:56.273495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f3896-d308-4163-b1d0-954f91bf7606 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:56.274012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630f3896-d308-4163-b1d0-954f91bf7606 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:56.290509 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1755/3508] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:11:56 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:11:56.291755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f3896-d308-4163-b1d0-954f91bf7606 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:56.298868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-630f3896-d308-4163-b1d0-954f91bf7606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:56.315653 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1754/3509] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:56.321296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef403a68-b214-4a92-98f0-c18bf1fc0cdf None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:56.323199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef403a68-b214-4a92-98f0-c18bf1fc0cdf None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:56.324209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef403a68-b214-4a92-98f0-c18bf1fc0cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:56.343830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:56.371989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ef403a68-b214-4a92-98f0-c18bf1fc0cdf None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:56.374673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ef403a68-b214-4a92-98f0-c18bf1fc0cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:56.486055 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1755/3510] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:11:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:56.494592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630f3896-d308-4163-b1d0-954f91bf7606 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:56.512754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef403a68-b214-4a92-98f0-c18bf1fc0cdf None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:56.514131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ef403a68-b214-4a92-98f0-c18bf1fc0cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:56.523263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117e120-d6c4-4084-b5c9-332fb189ea11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:56.524104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117e120-d6c4-4084-b5c9-332fb189ea11 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:56.524394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117e120-d6c4-4084-b5c9-332fb189ea11 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:56.524675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117e120-d6c4-4084-b5c9-332fb189ea11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:56.531523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef403a68-b214-4a92-98f0-c18bf1fc0cdf None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:56.538010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6418f24-cbe8-450d-864b-218618cad446 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:56.538970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6418f24-cbe8-450d-864b-218618cad446 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:56.539320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6418f24-cbe8-450d-864b-218618cad446 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:56.539670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6418f24-cbe8-450d-864b-218618cad446 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:56.546706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117e120-d6c4-4084-b5c9-332fb189ea11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:56.548432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117e120-d6c4-4084-b5c9-332fb189ea11 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:56.551923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6418f24-cbe8-450d-864b-218618cad446 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:56.553987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6418f24-cbe8-450d-864b-218618cad446 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:56.567977 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1756/3511] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:11:56.576151 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1756/3512] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:56.709080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6418f24-cbe8-450d-864b-218618cad446 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:56.710706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c6418f24-cbe8-450d-864b-218618cad446 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:56.741537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6418f24-cbe8-450d-864b-218618cad446 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:56.757106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92aabe-a881-442d-be65-ca5842312be5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:56.757763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92aabe-a881-442d-be65-ca5842312be5 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:56.758114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92aabe-a881-442d-be65-ca5842312be5 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:56.758398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92aabe-a881-442d-be65-ca5842312be5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:56.760827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ff92aabe-a881-442d-be65-ca5842312be5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:56.775027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92aabe-a881-442d-be65-ca5842312be5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:56.776738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92aabe-a881-442d-be65-ca5842312be5 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:56.789538 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1757/3513] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:56.813172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:56.830399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117e120-d6c4-4084-b5c9-332fb189ea11 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:56.831882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7117e120-d6c4-4084-b5c9-332fb189ea11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:56.863212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117e120-d6c4-4084-b5c9-332fb189ea11 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:56.882037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005e360-d325-4156-b141-683eec276c15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:56.882904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005e360-d325-4156-b141-683eec276c15 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:56.883393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005e360-d325-4156-b141-683eec276c15 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:56.883788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005e360-d325-4156-b141-683eec276c15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:56.902954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2005e360-d325-4156-b141-683eec276c15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:56.904687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2005e360-d325-4156-b141-683eec276c15 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:56.931186 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1757/3514] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:56.944169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257506e-5f3b-43b1-bf52-75634130476e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:56.944644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257506e-5f3b-43b1-bf52-75634130476e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:56.944718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257506e-5f3b-43b1-bf52-75634130476e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:56.969159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0257506e-5f3b-43b1-bf52-75634130476e None None] MFA Rules not processed for user `bf9c8f66e7d24ca9baca835e41546f22`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:56.970476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0257506e-5f3b-43b1-bf52-75634130476e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:57.069793 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1758/3515] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:57.263807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:57.406808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005e360-d325-4156-b141-683eec276c15 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:57.409068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2005e360-d325-4156-b141-683eec276c15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:57.523196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005e360-d325-4156-b141-683eec276c15 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:57.530531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73272e28-3fd9-4710-bb5b-84047675f97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:57.531725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73272e28-3fd9-4710-bb5b-84047675f97f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:57.532585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73272e28-3fd9-4710-bb5b-84047675f97f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:57.533141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73272e28-3fd9-4710-bb5b-84047675f97f admin admin] PATH_INFO: `/v3/projects/c78c95b156dd4ee183230abc68093a4b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:57.534626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73272e28-3fd9-4710-bb5b-84047675f97f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c78c95b156dd4ee183230abc68093a4b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:57.537050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73272e28-3fd9-4710-bb5b-84047675f97f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:57.772682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-73272e28-3fd9-4710-bb5b-84047675f97f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c78c95b156dd4ee183230abc68093a4b', 'request_id': 'req-73272e28-3fd9-4710-bb5b-84047675f97f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:57.786812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-73272e28-3fd9-4710-bb5b-84047675f97f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c78c95b156dd4ee183230abc68093a4b', 'request_id': 'req-73272e28-3fd9-4710-bb5b-84047675f97f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:57.875738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-73272e28-3fd9-4710-bb5b-84047675f97f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c78c95b156dd4ee183230abc68093a4b', 'request_id': 'req-73272e28-3fd9-4710-bb5b-84047675f97f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:57.919759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-73272e28-3fd9-4710-bb5b-84047675f97f admin admin] The token cache is being invalidate because project c78c95b156dd4ee183230abc68093a4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:11:57.920175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-73272e28-3fd9-4710-bb5b-84047675f97f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:57.920946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-73272e28-3fd9-4710-bb5b-84047675f97f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:11:57.928540 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1758/3516] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:11:57 2022] DELETE /identity/v3/projects/c78c95b156dd4ee183230abc68093a4b => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:57.929446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f39682-f1cc-4479-9aee-be8b49f0bb8a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:57.929742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f39682-f1cc-4479-9aee-be8b49f0bb8a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:57.930081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f39682-f1cc-4479-9aee-be8b49f0bb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:57.972974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fecfad-9655-4dec-995f-8c0a2e642ab6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:57.978462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-17f39682-f1cc-4479-9aee-be8b49f0bb8a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:57.979005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fecfad-9655-4dec-995f-8c0a2e642ab6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:57.979005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fecfad-9655-4dec-995f-8c0a2e642ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:57.984911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-17f39682-f1cc-4479-9aee-be8b49f0bb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:58.029809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c8fecfad-9655-4dec-995f-8c0a2e642ab6 None None] MFA Rules not processed for user `25b2f1524f7f48149cbc5d2c694a9e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:58.031474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c8fecfad-9655-4dec-995f-8c0a2e642ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:58.174493 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1759/3517] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:58.175665 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1759/3518] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:58.186496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c58635-772c-4020-afb6-35f4a98d42d0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:58.187107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c58635-772c-4020-afb6-35f4a98d42d0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:58.187628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c58635-772c-4020-afb6-35f4a98d42d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:58.196013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e781a58-a8e4-4031-a8e2-a1e4b9053fa9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:58.198207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e781a58-a8e4-4031-a8e2-a1e4b9053fa9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:58.199312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e781a58-a8e4-4031-a8e2-a1e4b9053fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:58.217506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a5c58635-772c-4020-afb6-35f4a98d42d0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:58.218891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a5c58635-772c-4020-afb6-35f4a98d42d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:58.250643 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1760/3519] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:11:58.253932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1e781a58-a8e4-4031-a8e2-a1e4b9053fa9 None None] MFA Rules not processed for user `25b2f1524f7f48149cbc5d2c694a9e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:58.253932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1e781a58-a8e4-4031-a8e2-a1e4b9053fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:58.255183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c58635-772c-4020-afb6-35f4a98d42d0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:58.268494 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1760/3520] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:58.276742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c58635-772c-4020-afb6-35f4a98d42d0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:58.294663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edc071-d12b-4662-a0ae-16b27a300142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:58.295513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edc071-d12b-4662-a0ae-16b27a300142 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:58.296034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edc071-d12b-4662-a0ae-16b27a300142 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:58.296499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7edc071-d12b-4662-a0ae-16b27a300142 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:58.302446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7edc071-d12b-4662-a0ae-16b27a300142 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:11:58.302446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7edc071-d12b-4662-a0ae-16b27a300142 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:58.302446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7edc071-d12b-4662-a0ae-16b27a300142 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:58.343917 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1761/3521] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:11:58.350222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e781a58-a8e4-4031-a8e2-a1e4b9053fa9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:58.365009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edc071-d12b-4662-a0ae-16b27a300142 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:58.366910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f7edc071-d12b-4662-a0ae-16b27a300142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:58.378206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e781a58-a8e4-4031-a8e2-a1e4b9053fa9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:58.386106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd4c1c-1600-4e4b-8fea-27df3476130f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:58.386833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd4c1c-1600-4e4b-8fea-27df3476130f admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:58.387127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd4c1c-1600-4e4b-8fea-27df3476130f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:58.387424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd4c1c-1600-4e4b-8fea-27df3476130f admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:58.388128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd4c1c-1600-4e4b-8fea-27df3476130f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:58.390125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd4c1c-1600-4e4b-8fea-27df3476130f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:58.513254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7edc071-d12b-4662-a0ae-16b27a300142 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:58.531557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f87dc-b5d7-45d7-b400-5ea43d31f50f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:58.534299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f87dc-b5d7-45d7-b400-5ea43d31f50f service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:58.534861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f87dc-b5d7-45d7-b400-5ea43d31f50f service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:58.535331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f87dc-b5d7-45d7-b400-5ea43d31f50f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:58.551352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f87dc-b5d7-45d7-b400-5ea43d31f50f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:58.553798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f87dc-b5d7-45d7-b400-5ea43d31f50f service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:58.572439 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1761/3522] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:11:58 2022] POST /identity/v3/projects => generated 397 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:11:58.576952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd4c1c-1600-4e4b-8fea-27df3476130f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:58.579494 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1762/3523] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:11:58.582268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:58.704978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd4c1c-1600-4e4b-8fea-27df3476130f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:58.715355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045fd48-b3c1-4aa0-9eca-371cce68ac57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:58.716580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045fd48-b3c1-4aa0-9eca-371cce68ac57 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:58.717166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045fd48-b3c1-4aa0-9eca-371cce68ac57 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:58.717829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045fd48-b3c1-4aa0-9eca-371cce68ac57 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:58.719096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045fd48-b3c1-4aa0-9eca-371cce68ac57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:58.723818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045fd48-b3c1-4aa0-9eca-371cce68ac57 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:58.844259 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1762/3524] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:11:58 2022] POST /identity/v3/users => generated 299 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:11:58.848574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f87dc-b5d7-45d7-b400-5ea43d31f50f service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:58.869370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f87dc-b5d7-45d7-b400-5ea43d31f50f service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:58.887357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32210bb8-ad49-4e80-a096-d99c5e2676bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:58.888081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32210bb8-ad49-4e80-a096-d99c5e2676bb admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:58.888371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32210bb8-ad49-4e80-a096-d99c5e2676bb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:58.888652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32210bb8-ad49-4e80-a096-d99c5e2676bb admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:58.889655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32210bb8-ad49-4e80-a096-d99c5e2676bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:58.891519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32210bb8-ad49-4e80-a096-d99c5e2676bb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:58.913818 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1763/3525] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:11:58.915691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045fd48-b3c1-4aa0-9eca-371cce68ac57 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:58.929892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045fd48-b3c1-4aa0-9eca-371cce68ac57 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:58.937411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170f5d6-e364-4de0-a095-b4c0a358402f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:58.938916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170f5d6-e364-4de0-a095-b4c0a358402f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:58.939349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170f5d6-e364-4de0-a095-b4c0a358402f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:58.939771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170f5d6-e364-4de0-a095-b4c0a358402f admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:58.941242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170f5d6-e364-4de0-a095-b4c0a358402f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:58.943847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170f5d6-e364-4de0-a095-b4c0a358402f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:58.955734 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1763/3526] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:11:58.959605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32210bb8-ad49-4e80-a096-d99c5e2676bb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:59.000256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32210bb8-ad49-4e80-a096-d99c5e2676bb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:59.017794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec5562-0e46-4100-b701-a36a26acb56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:59.022629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec5562-0e46-4100-b701-a36a26acb56e admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:59.022629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec5562-0e46-4100-b701-a36a26acb56e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:59.022629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec5562-0e46-4100-b701-a36a26acb56e admin admin] PATH_INFO: `/v3/projects/8c5c8f0a4a21466fa3704ddcf51a599d/users/e134b2546c7340c490aff72572059854/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:59.054625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec5562-0e46-4100-b701-a36a26acb56e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c5c8f0a4a21466fa3704ddcf51a599d, user_id=e134b2546c7340c490aff72572059854, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:59.057677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ec5562-0e46-4100-b701-a36a26acb56e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:59.134455 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1764/3527] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:11:58 2022] PUT /identity/v3/projects/8c5c8f0a4a21466fa3704ddcf51a599d/users/e134b2546c7340c490aff72572059854/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:11:59.146975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de70d1d-f77e-45dc-a1ff-f1b8fe362d7f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:59.147277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de70d1d-f77e-45dc-a1ff-f1b8fe362d7f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:59.147589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de70d1d-f77e-45dc-a1ff-f1b8fe362d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:59.204771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2de70d1d-f77e-45dc-a1ff-f1b8fe362d7f None None] MFA Rules not processed for user `e134b2546c7340c490aff72572059854`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:11:59.206333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2de70d1d-f77e-45dc-a1ff-f1b8fe362d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:59.295481 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1764/3528] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:11:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:11:59.371471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec5562-0e46-4100-b701-a36a26acb56e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:11:59.373375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-07ec5562-0e46-4100-b701-a36a26acb56e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:11:59.605507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec5562-0e46-4100-b701-a36a26acb56e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:11:59.621666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2ba79-e859-46b3-83a2-5669baaac7f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:11:59.626781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2ba79-e859-46b3-83a2-5669baaac7f3 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:11:59.626781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2ba79-e859-46b3-83a2-5669baaac7f3 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:11:59.626781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2ba79-e859-46b3-83a2-5669baaac7f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:11:59.641805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2ba79-e859-46b3-83a2-5669baaac7f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:11:59.657027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2ba79-e859-46b3-83a2-5669baaac7f3 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:11:59.687551 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1765/3529] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:11:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:12:00.006464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de70d1d-f77e-45dc-a1ff-f1b8fe362d7f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:00.008080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2de70d1d-f77e-45dc-a1ff-f1b8fe362d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:00.117379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de70d1d-f77e-45dc-a1ff-f1b8fe362d7f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:00.127374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ab1b2-968f-40b9-b9c5-d62ea036d7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:00.128324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ab1b2-968f-40b9-b9c5-d62ea036d7e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:00.132384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ab1b2-968f-40b9-b9c5-d62ea036d7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:00.132384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ab1b2-968f-40b9-b9c5-d62ea036d7e6 admin admin] PATH_INFO: `/v3/users/28a6a9a1c0ed478eb54268fbb45b63d0` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:00.132384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ab1b2-968f-40b9-b9c5-d62ea036d7e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28a6a9a1c0ed478eb54268fbb45b63d0)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:00.133540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ab1b2-968f-40b9-b9c5-d62ea036d7e6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:00.418283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-375ab1b2-968f-40b9-b9c5-d62ea036d7e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28a6a9a1c0ed478eb54268fbb45b63d0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:00.421954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-375ab1b2-968f-40b9-b9c5-d62ea036d7e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28a6a9a1c0ed478eb54268fbb45b63d0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:00.440762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-375ab1b2-968f-40b9-b9c5-d62ea036d7e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28a6a9a1c0ed478eb54268fbb45b63d0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:00.481970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:00.544302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-375ab1b2-968f-40b9-b9c5-d62ea036d7e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28a6a9a1c0ed478eb54268fbb45b63d0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:00.544302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-375ab1b2-968f-40b9-b9c5-d62ea036d7e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28a6a9a1c0ed478eb54268fbb45b63d0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:00.589106 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1765/3530] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:12:00 2022] DELETE /identity/v3/users/28a6a9a1c0ed478eb54268fbb45b63d0 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:00.878709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a09e7-808f-43c9-8725-dacdb85e8698 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:00.879830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a09e7-808f-43c9-8725-dacdb85e8698 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:00.880391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a09e7-808f-43c9-8725-dacdb85e8698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:00.945221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-778a09e7-808f-43c9-8725-dacdb85e8698 None None] MFA Rules not processed for user `ce8ac48fe53b4080bf74c9788408f8c5`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:00.945221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-778a09e7-808f-43c9-8725-dacdb85e8698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:01.028153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ab1b2-968f-40b9-b9c5-d62ea036d7e6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:01.030211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-375ab1b2-968f-40b9-b9c5-d62ea036d7e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:01.108529 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1766/3531] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:01.185183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ab1b2-968f-40b9-b9c5-d62ea036d7e6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:01.191729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a1fc3-27f8-4d3b-a4b4-d5bddfd31d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:01.193122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a1fc3-27f8-4d3b-a4b4-d5bddfd31d1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:01.193557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a1fc3-27f8-4d3b-a4b4-d5bddfd31d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:01.194098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a1fc3-27f8-4d3b-a4b4-d5bddfd31d1c admin admin] PATH_INFO: `/v3/users/e134b2546c7340c490aff72572059854` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:01.196361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a1fc3-27f8-4d3b-a4b4-d5bddfd31d1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e134b2546c7340c490aff72572059854)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:01.199621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a1fc3-27f8-4d3b-a4b4-d5bddfd31d1c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:01.377460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a09e7-808f-43c9-8725-dacdb85e8698 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:01.379609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-778a09e7-808f-43c9-8725-dacdb85e8698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:01.436418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a09e7-808f-43c9-8725-dacdb85e8698 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:01.448531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-629a1fc3-27f8-4d3b-a4b4-d5bddfd31d1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e134b2546c7340c490aff72572059854'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:01.449162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-629a1fc3-27f8-4d3b-a4b4-d5bddfd31d1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e134b2546c7340c490aff72572059854'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:01.456548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c552799-1971-467a-a62e-a5bda68c58c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:01.457672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c552799-1971-467a-a62e-a5bda68c58c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:01.464998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c552799-1971-467a-a62e-a5bda68c58c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:01.466138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c552799-1971-467a-a62e-a5bda68c58c2 admin admin] PATH_INFO: `/v3/projects/e1f0267259d74463b29bda378389f5e4` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:01.468273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c552799-1971-467a-a62e-a5bda68c58c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1f0267259d74463b29bda378389f5e4)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:01.470939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-629a1fc3-27f8-4d3b-a4b4-d5bddfd31d1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e134b2546c7340c490aff72572059854'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:01.478576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c552799-1971-467a-a62e-a5bda68c58c2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:01.600631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-629a1fc3-27f8-4d3b-a4b4-d5bddfd31d1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e134b2546c7340c490aff72572059854'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:01.601780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-629a1fc3-27f8-4d3b-a4b4-d5bddfd31d1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e134b2546c7340c490aff72572059854'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:01.627265 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1766/3532] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:12:01 2022] DELETE /identity/v3/users/e134b2546c7340c490aff72572059854 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:01.632672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb0be6-b134-4245-80d4-87bb4b6d0430 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:01.632969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb0be6-b134-4245-80d4-87bb4b6d0430 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:01.635630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bb0be6-b134-4245-80d4-87bb4b6d0430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:01.662098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-47bb0be6-b134-4245-80d4-87bb4b6d0430 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:01.663756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-47bb0be6-b134-4245-80d4-87bb4b6d0430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:01.699916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c552799-1971-467a-a62e-a5bda68c58c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1f0267259d74463b29bda378389f5e4', 'request_id': 'req-4c552799-1971-467a-a62e-a5bda68c58c2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:01.710351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c552799-1971-467a-a62e-a5bda68c58c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1f0267259d74463b29bda378389f5e4', 'request_id': 'req-4c552799-1971-467a-a62e-a5bda68c58c2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:01.721308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:01.764300 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1767/3533] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:01.778929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb0be6-b134-4245-80d4-87bb4b6d0430 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:01.783515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-47bb0be6-b134-4245-80d4-87bb4b6d0430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:01.814178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c552799-1971-467a-a62e-a5bda68c58c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1f0267259d74463b29bda378389f5e4', 'request_id': 'req-4c552799-1971-467a-a62e-a5bda68c58c2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:01.847070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c552799-1971-467a-a62e-a5bda68c58c2 admin admin] The token cache is being invalidate because project e1f0267259d74463b29bda378389f5e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:12:01.847393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c552799-1971-467a-a62e-a5bda68c58c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:01.858386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c552799-1971-467a-a62e-a5bda68c58c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:01.862814 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1767/3534] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:12:01 2022] DELETE /identity/v3/projects/e1f0267259d74463b29bda378389f5e4 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:12:01.871064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c142e91-4bd9-43f0-851e-42ea1f1a93f2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:01.874834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c142e91-4bd9-43f0-851e-42ea1f1a93f2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:01.875437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c142e91-4bd9-43f0-851e-42ea1f1a93f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:01.889000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bb0be6-b134-4245-80d4-87bb4b6d0430 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:01.893322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cbfcc-98ea-46a9-9f44-a439d5a5f49f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:01.894563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cbfcc-98ea-46a9-9f44-a439d5a5f49f service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:01.894563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cbfcc-98ea-46a9-9f44-a439d5a5f49f service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:01.894736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cbfcc-98ea-46a9-9f44-a439d5a5f49f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:01.896490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3b8cbfcc-98ea-46a9-9f44-a439d5a5f49f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:01.902697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9c142e91-4bd9-43f0-851e-42ea1f1a93f2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:01.904719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9c142e91-4bd9-43f0-851e-42ea1f1a93f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:01.914055 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1768/3535] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:01.927563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3cd89-092e-42db-91af-a55156ca6571 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:01.927563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3cd89-092e-42db-91af-a55156ca6571 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:01.927563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3cd89-092e-42db-91af-a55156ca6571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:01.930701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8cbfcc-98ea-46a9-9f44-a439d5a5f49f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:01.937802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8cbfcc-98ea-46a9-9f44-a439d5a5f49f service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:01.949241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-49e3cd89-092e-42db-91af-a55156ca6571 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:01.951019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-49e3cd89-092e-42db-91af-a55156ca6571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:01.960653 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1768/3536] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:12:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:12:01.978456 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1769/3537] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:01.983309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cbfcc-98ea-46a9-9f44-a439d5a5f49f service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:01.998798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cbfcc-98ea-46a9-9f44-a439d5a5f49f service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:02.010748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980cd93-182c-4847-9e2a-3aafcd520f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:02.012148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980cd93-182c-4847-9e2a-3aafcd520f95 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:02.012341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980cd93-182c-4847-9e2a-3aafcd520f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:02.012635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a980cd93-182c-4847-9e2a-3aafcd520f95 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:02.016903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a980cd93-182c-4847-9e2a-3aafcd520f95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:12:02.017683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a980cd93-182c-4847-9e2a-3aafcd520f95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:02.018964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a980cd93-182c-4847-9e2a-3aafcd520f95 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:02.047719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3cd89-092e-42db-91af-a55156ca6571 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:02.054259 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1769/3538] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:12:02.081212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3cd89-092e-42db-91af-a55156ca6571 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:02.095205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86a61b-3430-49d2-b110-a2cc4144a5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:02.096681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86a61b-3430-49d2-b110-a2cc4144a5ca admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:02.097257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86a61b-3430-49d2-b110-a2cc4144a5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:02.097741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec86a61b-3430-49d2-b110-a2cc4144a5ca admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:02.106370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86a61b-3430-49d2-b110-a2cc4144a5ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:02.115936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec86a61b-3430-49d2-b110-a2cc4144a5ca admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:02.233042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:02.253891 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1770/3539] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:12:02 2022] POST /identity/v3/projects => generated 379 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:12:02.257559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980cd93-182c-4847-9e2a-3aafcd520f95 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:02.275090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a980cd93-182c-4847-9e2a-3aafcd520f95 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:02.399133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645128-dbd6-4ca7-8776-c3ac0efa01a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:02.399861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645128-dbd6-4ca7-8776-c3ac0efa01a0 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:02.400146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645128-dbd6-4ca7-8776-c3ac0efa01a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:02.401044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0645128-dbd6-4ca7-8776-c3ac0efa01a0 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:02.402346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0645128-dbd6-4ca7-8776-c3ac0efa01a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:02.404375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0645128-dbd6-4ca7-8776-c3ac0efa01a0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:02.520920 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1770/3540] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:12:02 2022] POST /identity/v3/users => generated 290 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:12:02.524622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86a61b-3430-49d2-b110-a2cc4144a5ca admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:02.539482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec86a61b-3430-49d2-b110-a2cc4144a5ca admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:02.552907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604b1ab-b467-4568-a94f-a63f6a2ecf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:02.553900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604b1ab-b467-4568-a94f-a63f6a2ecf29 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:02.554434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604b1ab-b467-4568-a94f-a63f6a2ecf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:02.554904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5604b1ab-b467-4568-a94f-a63f6a2ecf29 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:02.555960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604b1ab-b467-4568-a94f-a63f6a2ecf29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:02.558042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5604b1ab-b467-4568-a94f-a63f6a2ecf29 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:02.573685 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1771/3541] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:12:02.578097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645128-dbd6-4ca7-8776-c3ac0efa01a0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:02.606754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0645128-dbd6-4ca7-8776-c3ac0efa01a0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:02.615883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df704302-7d30-4427-8cd1-1525bcd30e2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:02.618532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df704302-7d30-4427-8cd1-1525bcd30e2b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:02.619161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df704302-7d30-4427-8cd1-1525bcd30e2b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:02.619654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df704302-7d30-4427-8cd1-1525bcd30e2b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:02.620708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df704302-7d30-4427-8cd1-1525bcd30e2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:02.622692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df704302-7d30-4427-8cd1-1525bcd30e2b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:02.637244 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1771/3542] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:12:02.641410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604b1ab-b467-4568-a94f-a63f6a2ecf29 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:02.653335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5604b1ab-b467-4568-a94f-a63f6a2ecf29 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:02.659102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da18113-55f9-40b3-be12-921d8982e18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:02.660051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da18113-55f9-40b3-be12-921d8982e18b admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:02.660524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da18113-55f9-40b3-be12-921d8982e18b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:02.660998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da18113-55f9-40b3-be12-921d8982e18b admin admin] PATH_INFO: `/v3/projects/8047aaa6e1f7401fbcf4b4cd5dcd004e/users/2dd86b6dd9894aac971943b2fa859bd1/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:02.700972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da18113-55f9-40b3-be12-921d8982e18b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8047aaa6e1f7401fbcf4b4cd5dcd004e, user_id=2dd86b6dd9894aac971943b2fa859bd1, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:02.704288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da18113-55f9-40b3-be12-921d8982e18b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:02.740940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61427ca8-7172-4bbd-a97b-4cb2279bb006 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:02.741285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61427ca8-7172-4bbd-a97b-4cb2279bb006 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:02.741618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61427ca8-7172-4bbd-a97b-4cb2279bb006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:02.799771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-61427ca8-7172-4bbd-a97b-4cb2279bb006 None None] MFA Rules not processed for user `2d473cc820a94134a6fbd72bf6b1a0a6`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:02.801774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-61427ca8-7172-4bbd-a97b-4cb2279bb006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:02.821953 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1772/3543] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:12:02 2022] PUT /identity/v3/projects/8047aaa6e1f7401fbcf4b4cd5dcd004e/users/2dd86b6dd9894aac971943b2fa859bd1/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:02.827979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f82ff-cabf-4c91-b5a9-5893d4c834ea None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:02.828593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f82ff-cabf-4c91-b5a9-5893d4c834ea None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:02.829093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9f82ff-cabf-4c91-b5a9-5893d4c834ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:02.856750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8d9f82ff-cabf-4c91-b5a9-5893d4c834ea None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:02.858641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8d9f82ff-cabf-4c91-b5a9-5893d4c834ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:02.951370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:02.983976 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1773/3544] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:12:02.984858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f82ff-cabf-4c91-b5a9-5893d4c834ea None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:02.994368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8d9f82ff-cabf-4c91-b5a9-5893d4c834ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:03.028804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9f82ff-cabf-4c91-b5a9-5893d4c834ea None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:03.041088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735e1d5-adc3-494d-a997-599c342436eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:03.042321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735e1d5-adc3-494d-a997-599c342436eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:03.042608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735e1d5-adc3-494d-a997-599c342436eb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:03.042891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735e1d5-adc3-494d-a997-599c342436eb admin admin] PATH_INFO: `/v3/projects/8c5c8f0a4a21466fa3704ddcf51a599d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:03.044190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735e1d5-adc3-494d-a997-599c342436eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c5c8f0a4a21466fa3704ddcf51a599d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:03.046393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735e1d5-adc3-494d-a997-599c342436eb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:03.056888 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1772/3545] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:02 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:03.058675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61427ca8-7172-4bbd-a97b-4cb2279bb006 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:03.063339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-61427ca8-7172-4bbd-a97b-4cb2279bb006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:03.160820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61427ca8-7172-4bbd-a97b-4cb2279bb006 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:03.170659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c98f7a-c3b0-4265-a5c1-d65323cac19c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:03.171435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c98f7a-c3b0-4265-a5c1-d65323cac19c service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:03.172677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c98f7a-c3b0-4265-a5c1-d65323cac19c service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:03.173029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c98f7a-c3b0-4265-a5c1-d65323cac19c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:03.200805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c98f7a-c3b0-4265-a5c1-d65323cac19c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:03.202516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c98f7a-c3b0-4265-a5c1-d65323cac19c service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:03.217269 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1773/3546] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:12:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:12:03.217269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c98f7a-c3b0-4265-a5c1-d65323cac19c service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:03.218481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-62c98f7a-c3b0-4265-a5c1-d65323cac19c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:03.327863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d735e1d5-adc3-494d-a997-599c342436eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c5c8f0a4a21466fa3704ddcf51a599d', 'request_id': 'req-d735e1d5-adc3-494d-a997-599c342436eb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:03.338128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d735e1d5-adc3-494d-a997-599c342436eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c5c8f0a4a21466fa3704ddcf51a599d', 'request_id': 'req-d735e1d5-adc3-494d-a997-599c342436eb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:03.344500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c98f7a-c3b0-4265-a5c1-d65323cac19c service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:03.417397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d735e1d5-adc3-494d-a997-599c342436eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c5c8f0a4a21466fa3704ddcf51a599d', 'request_id': 'req-d735e1d5-adc3-494d-a997-599c342436eb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:03.451364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d735e1d5-adc3-494d-a997-599c342436eb admin admin] The token cache is being invalidate because project 8c5c8f0a4a21466fa3704ddcf51a599d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:12:03.452300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d735e1d5-adc3-494d-a997-599c342436eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:03.456588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d735e1d5-adc3-494d-a997-599c342436eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:03.469121 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1774/3547] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:12:02 2022] DELETE /identity/v3/projects/8c5c8f0a4a21466fa3704ddcf51a599d => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:03.472516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76711f32-932e-409a-8ef3-eee5d061efad None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:03.476790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76711f32-932e-409a-8ef3-eee5d061efad None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:03.477100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76711f32-932e-409a-8ef3-eee5d061efad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:03.502850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b2a79-7132-4236-84d4-8f4b7e91fbbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:03.502850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b2a79-7132-4236-84d4-8f4b7e91fbbc service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:03.503086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b2a79-7132-4236-84d4-8f4b7e91fbbc service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:03.503358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b2a79-7132-4236-84d4-8f4b7e91fbbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:03.508814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-470b2a79-7132-4236-84d4-8f4b7e91fbbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:03.516007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-76711f32-932e-409a-8ef3-eee5d061efad None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:03.517922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-76711f32-932e-409a-8ef3-eee5d061efad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:03.625172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b2a79-7132-4236-84d4-8f4b7e91fbbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:03.628151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b2a79-7132-4236-84d4-8f4b7e91fbbc service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:03.639704 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1774/3548] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:12:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:12:03.659566 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1775/3549] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:03.662278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93ef1d-3d5d-45a0-9d7e-16530a4f253e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:03.664529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93ef1d-3d5d-45a0-9d7e-16530a4f253e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:03.664529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93ef1d-3d5d-45a0-9d7e-16530a4f253e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:03.705258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7f93ef1d-3d5d-45a0-9d7e-16530a4f253e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:03.707420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7f93ef1d-3d5d-45a0-9d7e-16530a4f253e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:03.719787 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1775/3550] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:12:03.721230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76711f32-932e-409a-8ef3-eee5d061efad None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:03.736136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76711f32-932e-409a-8ef3-eee5d061efad None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:03.760609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f978ef-90ad-4e85-93e4-49979eaf5636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:03.761337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f978ef-90ad-4e85-93e4-49979eaf5636 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:03.761626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f978ef-90ad-4e85-93e4-49979eaf5636 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:03.761935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f978ef-90ad-4e85-93e4-49979eaf5636 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:03.762688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f978ef-90ad-4e85-93e4-49979eaf5636 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:12:03.763004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f978ef-90ad-4e85-93e4-49979eaf5636 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:03.769819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f978ef-90ad-4e85-93e4-49979eaf5636 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:03.814204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93ef1d-3d5d-45a0-9d7e-16530a4f253e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:03.825726 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1776/3551] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:12:03.850960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93ef1d-3d5d-45a0-9d7e-16530a4f253e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:03.869799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbbb27-1fd2-43c8-819d-1df98b2e70ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:03.870378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbbb27-1fd2-43c8-819d-1df98b2e70ee admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:03.877898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbbb27-1fd2-43c8-819d-1df98b2e70ee admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:03.878152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbbb27-1fd2-43c8-819d-1df98b2e70ee admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:03.879055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbbb27-1fd2-43c8-819d-1df98b2e70ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:03.881205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbbb27-1fd2-43c8-819d-1df98b2e70ee admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:04.042740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:04.058067 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1776/3552] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:12:03 2022] POST /identity/v3/projects => generated 381 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:12:04.060355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f978ef-90ad-4e85-93e4-49979eaf5636 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:04.079348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f978ef-90ad-4e85-93e4-49979eaf5636 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:04.213608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22455c01-f277-4a49-b1c3-48b0a8f93501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:04.216195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22455c01-f277-4a49-b1c3-48b0a8f93501 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:04.218982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22455c01-f277-4a49-b1c3-48b0a8f93501 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:04.219392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22455c01-f277-4a49-b1c3-48b0a8f93501 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:04.220178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22455c01-f277-4a49-b1c3-48b0a8f93501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:04.224222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22455c01-f277-4a49-b1c3-48b0a8f93501 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:04.400044 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1777/3553] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:12:04 2022] POST /identity/v3/users => generated 291 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:12:04.405594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbbb27-1fd2-43c8-819d-1df98b2e70ee admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:04.423406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbbb27-1fd2-43c8-819d-1df98b2e70ee admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:04.440323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a341ce-9ec6-4edb-83e2-c0c50a82fee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:04.449061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a341ce-9ec6-4edb-83e2-c0c50a82fee8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:04.449061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a341ce-9ec6-4edb-83e2-c0c50a82fee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:04.449061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a341ce-9ec6-4edb-83e2-c0c50a82fee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:04.449061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a341ce-9ec6-4edb-83e2-c0c50a82fee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:04.449418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a341ce-9ec6-4edb-83e2-c0c50a82fee8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:04.470234 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1777/3554] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:12:04.474973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22455c01-f277-4a49-b1c3-48b0a8f93501 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:04.492416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22455c01-f277-4a49-b1c3-48b0a8f93501 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:04.502744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f3f51-3d3e-40f4-88b7-6ef69e50c50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:04.503545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f3f51-3d3e-40f4-88b7-6ef69e50c50b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:04.503763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f3f51-3d3e-40f4-88b7-6ef69e50c50b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:04.504045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f3f51-3d3e-40f4-88b7-6ef69e50c50b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:04.504796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f3f51-3d3e-40f4-88b7-6ef69e50c50b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:04.506466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855f3f51-3d3e-40f4-88b7-6ef69e50c50b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:04.528782 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1778/3555] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:12:04.532826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a341ce-9ec6-4edb-83e2-c0c50a82fee8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:04.561453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a341ce-9ec6-4edb-83e2-c0c50a82fee8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:04.575725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177c975-0e57-49f7-bd5c-18d86217574a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:04.576890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177c975-0e57-49f7-bd5c-18d86217574a admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:04.577331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177c975-0e57-49f7-bd5c-18d86217574a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:04.577738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177c975-0e57-49f7-bd5c-18d86217574a admin admin] PATH_INFO: `/v3/projects/659cd6db8d6741329dded06813f0ef93/users/0e36e381aa5a4e4196d4e994e70f586e/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:04.613247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177c975-0e57-49f7-bd5c-18d86217574a admin admin] RBAC: Authorizing `identity:create_grant(project_id=659cd6db8d6741329dded06813f0ef93, user_id=0e36e381aa5a4e4196d4e994e70f586e, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:04.617305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5177c975-0e57-49f7-bd5c-18d86217574a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:04.923623 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1778/3556] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:12:04 2022] PUT /identity/v3/projects/659cd6db8d6741329dded06813f0ef93/users/0e36e381aa5a4e4196d4e994e70f586e/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:04.935682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094afb0-faad-4a67-b835-2ab510fbb632 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:04.936307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094afb0-faad-4a67-b835-2ab510fbb632 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:04.936777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094afb0-faad-4a67-b835-2ab510fbb632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:05.027234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b094afb0-faad-4a67-b835-2ab510fbb632 None None] MFA Rules not processed for user `0e36e381aa5a4e4196d4e994e70f586e`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:05.029762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b094afb0-faad-4a67-b835-2ab510fbb632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:05.224785 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1779/3557] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:05.235718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f0927-9210-4631-bee9-bf82838a95bb None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:05.236628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f0927-9210-4631-bee9-bf82838a95bb None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:05.236911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119f0927-9210-4631-bee9-bf82838a95bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:05.252203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:05.273961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-119f0927-9210-4631-bee9-bf82838a95bb None None] MFA Rules not processed for user `0e36e381aa5a4e4196d4e994e70f586e`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:05.279137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-119f0927-9210-4631-bee9-bf82838a95bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:05.305786 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1779/3558] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:05.319029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094afb0-faad-4a67-b835-2ab510fbb632 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:05.322467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b094afb0-faad-4a67-b835-2ab510fbb632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:05.529484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094afb0-faad-4a67-b835-2ab510fbb632 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:05.543674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09984b23-f931-42a8-ac7c-ac74388484a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:05.544784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09984b23-f931-42a8-ac7c-ac74388484a7 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:05.545250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09984b23-f931-42a8-ac7c-ac74388484a7 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:05.546937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09984b23-f931-42a8-ac7c-ac74388484a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:05.570098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09984b23-f931-42a8-ac7c-ac74388484a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:05.573231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09984b23-f931-42a8-ac7c-ac74388484a7 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:05.612167 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1780/3559] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:12:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:12:06.172736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:06.571224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd348b-5d0e-4447-b81b-ba5e7d7f4477 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:06.582082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd348b-5d0e-4447-b81b-ba5e7d7f4477 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:06.582082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd348b-5d0e-4447-b81b-ba5e7d7f4477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:06.646800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1ddd348b-5d0e-4447-b81b-ba5e7d7f4477 None None] MFA Rules not processed for user `2dd86b6dd9894aac971943b2fa859bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:06.650711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1ddd348b-5d0e-4447-b81b-ba5e7d7f4477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:06.844545 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1780/3560] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:06.856753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69471ef-9fd8-47c1-b21a-9429377b0e2c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:06.857330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69471ef-9fd8-47c1-b21a-9429377b0e2c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:06.857330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69471ef-9fd8-47c1-b21a-9429377b0e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:06.945338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d69471ef-9fd8-47c1-b21a-9429377b0e2c None None] MFA Rules not processed for user `2dd86b6dd9894aac971943b2fa859bd1`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:06.954236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d69471ef-9fd8-47c1-b21a-9429377b0e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:06.985033 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1781/3561] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:07.361038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd348b-5d0e-4447-b81b-ba5e7d7f4477 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:07.362841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1ddd348b-5d0e-4447-b81b-ba5e7d7f4477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:07.388516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd348b-5d0e-4447-b81b-ba5e7d7f4477 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:07.396168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3821978-f41b-47f5-9613-e3f3cdf41546 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:07.397617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3821978-f41b-47f5-9613-e3f3cdf41546 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:07.398103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3821978-f41b-47f5-9613-e3f3cdf41546 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:07.398565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3821978-f41b-47f5-9613-e3f3cdf41546 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:07.411339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3821978-f41b-47f5-9613-e3f3cdf41546 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:07.413051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3821978-f41b-47f5-9613-e3f3cdf41546 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:07.423579 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1781/3562] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:12:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:12:08.331888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:09.567257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:11.526402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:12.099208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:13.274578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:22.794675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=36 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:23.921297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=76,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:24.358304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3677e5-071a-40eb-abb1-58d7a8d96944 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:24.358826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3677e5-071a-40eb-abb1-58d7a8d96944 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:24.358902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3677e5-071a-40eb-abb1-58d7a8d96944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:24.381349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cb3677e5-071a-40eb-abb1-58d7a8d96944 None None] MFA Rules not processed for user `bf9c8f66e7d24ca9baca835e41546f22`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:24.384542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cb3677e5-071a-40eb-abb1-58d7a8d96944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:24.489101 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1782/3563] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:30.487839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36684ece-4618-49c9-9fe2-5a8515848a4f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:30.488359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36684ece-4618-49c9-9fe2-5a8515848a4f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:30.488359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36684ece-4618-49c9-9fe2-5a8515848a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:30.507312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-36684ece-4618-49c9-9fe2-5a8515848a4f None None] MFA Rules not processed for user `cafc81a185c645e893a77b7fd297d950`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:30.508562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-36684ece-4618-49c9-9fe2-5a8515848a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:30.566432 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1782/3564] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:30.577441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda760f1-3bd8-4573-8e20-45a763871f89 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:30.578005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda760f1-3bd8-4573-8e20-45a763871f89 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:30.578186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda760f1-3bd8-4573-8e20-45a763871f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:30.597407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-dda760f1-3bd8-4573-8e20-45a763871f89 None None] MFA Rules not processed for user `cafc81a185c645e893a77b7fd297d950`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:30.598819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dda760f1-3bd8-4573-8e20-45a763871f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:30.606731 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1783/3565] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:36.973664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36684ece-4618-49c9-9fe2-5a8515848a4f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:36.977556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-36684ece-4618-49c9-9fe2-5a8515848a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:37.084466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36684ece-4618-49c9-9fe2-5a8515848a4f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:37.093042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94aa6d-dd33-40dc-8554-12ecff5a1f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:37.094002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94aa6d-dd33-40dc-8554-12ecff5a1f31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:37.094297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94aa6d-dd33-40dc-8554-12ecff5a1f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:37.094576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94aa6d-dd33-40dc-8554-12ecff5a1f31 admin admin] PATH_INFO: `/v3/users/ce8ac48fe53b4080bf74c9788408f8c5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:37.095868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94aa6d-dd33-40dc-8554-12ecff5a1f31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce8ac48fe53b4080bf74c9788408f8c5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:37.097996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94aa6d-dd33-40dc-8554-12ecff5a1f31 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:37.321656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda760f1-3bd8-4573-8e20-45a763871f89 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:37.326728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9e94aa6d-dd33-40dc-8554-12ecff5a1f31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce8ac48fe53b4080bf74c9788408f8c5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:37.327401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9e94aa6d-dd33-40dc-8554-12ecff5a1f31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce8ac48fe53b4080bf74c9788408f8c5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:37.332577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda760f1-3bd8-4573-8e20-45a763871f89 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:37.335037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9e94aa6d-dd33-40dc-8554-12ecff5a1f31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce8ac48fe53b4080bf74c9788408f8c5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:37.337130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20eff5-9e4a-4106-b09e-20b635304dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:37.338031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20eff5-9e4a-4106-b09e-20b635304dde admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:37.338341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20eff5-9e4a-4106-b09e-20b635304dde admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:37.338618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e20eff5-9e4a-4106-b09e-20b635304dde admin admin] PATH_INFO: `/v3/users/0e36e381aa5a4e4196d4e994e70f586e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:37.339810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e20eff5-9e4a-4106-b09e-20b635304dde admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e36e381aa5a4e4196d4e994e70f586e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:37.341781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e20eff5-9e4a-4106-b09e-20b635304dde admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:37.436543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9e94aa6d-dd33-40dc-8554-12ecff5a1f31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce8ac48fe53b4080bf74c9788408f8c5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:37.438775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9e94aa6d-dd33-40dc-8554-12ecff5a1f31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce8ac48fe53b4080bf74c9788408f8c5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:37.472626 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1783/3566] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:12:36 2022] DELETE /identity/v3/users/ce8ac48fe53b4080bf74c9788408f8c5 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:37.474022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94aa6d-dd33-40dc-8554-12ecff5a1f31 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:37.475715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9e94aa6d-dd33-40dc-8554-12ecff5a1f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:37.512465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:37.539995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2e20eff5-9e4a-4106-b09e-20b635304dde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e36e381aa5a4e4196d4e994e70f586e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:37.559571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2e20eff5-9e4a-4106-b09e-20b635304dde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e36e381aa5a4e4196d4e994e70f586e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:37.560724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2e20eff5-9e4a-4106-b09e-20b635304dde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e36e381aa5a4e4196d4e994e70f586e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:37.562441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2e20eff5-9e4a-4106-b09e-20b635304dde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e36e381aa5a4e4196d4e994e70f586e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:37.596379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94aa6d-dd33-40dc-8554-12ecff5a1f31 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:37.601648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbdb9f-e11b-4fb2-ab0e-124155e7cfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:37.602738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbdb9f-e11b-4fb2-ab0e-124155e7cfb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:37.603622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbdb9f-e11b-4fb2-ab0e-124155e7cfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:37.603622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fbdb9f-e11b-4fb2-ab0e-124155e7cfb1 admin admin] PATH_INFO: `/v3/users/2dd86b6dd9894aac971943b2fa859bd1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:37.605218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbdb9f-e11b-4fb2-ab0e-124155e7cfb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dd86b6dd9894aac971943b2fa859bd1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:37.607241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fbdb9f-e11b-4fb2-ab0e-124155e7cfb1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:37.657274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2e20eff5-9e4a-4106-b09e-20b635304dde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e36e381aa5a4e4196d4e994e70f586e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:37.676039 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1784/3567] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:12:37 2022] DELETE /identity/v3/users/0e36e381aa5a4e4196d4e994e70f586e => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:12:37.676837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20eff5-9e4a-4106-b09e-20b635304dde admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:37.678189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2e20eff5-9e4a-4106-b09e-20b635304dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:37.738108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e20eff5-9e4a-4106-b09e-20b635304dde admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:37.742218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20ccfa-96eb-4915-a0c1-894ae76c2eb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:37.742859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20ccfa-96eb-4915-a0c1-894ae76c2eb4 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:37.743178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20ccfa-96eb-4915-a0c1-894ae76c2eb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:37.743456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20ccfa-96eb-4915-a0c1-894ae76c2eb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:37.745267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2f20ccfa-96eb-4915-a0c1-894ae76c2eb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:37.781436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:37.829173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d9fbdb9f-e11b-4fb2-ab0e-124155e7cfb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dd86b6dd9894aac971943b2fa859bd1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:37.830006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d9fbdb9f-e11b-4fb2-ab0e-124155e7cfb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dd86b6dd9894aac971943b2fa859bd1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:37.833698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20ccfa-96eb-4915-a0c1-894ae76c2eb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:37.835429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20ccfa-96eb-4915-a0c1-894ae76c2eb4 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:37.844169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d9fbdb9f-e11b-4fb2-ab0e-124155e7cfb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dd86b6dd9894aac971943b2fa859bd1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:37.851426 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1785/3568] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:12:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:12:37.852969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c0936-f4e9-4532-85bd-9f3b7d81bb47 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:37.853256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c0936-f4e9-4532-85bd-9f3b7d81bb47 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:37.853538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c0936-f4e9-4532-85bd-9f3b7d81bb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:37.873104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d00c0936-f4e9-4532-85bd-9f3b7d81bb47 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:37.874374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d00c0936-f4e9-4532-85bd-9f3b7d81bb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:37.943001 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1786/3569] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:37.943888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c0936-f4e9-4532-85bd-9f3b7d81bb47 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:37.945263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d00c0936-f4e9-4532-85bd-9f3b7d81bb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:37.957341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00c0936-f4e9-4532-85bd-9f3b7d81bb47 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:37.962481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bb0ba-9274-44eb-8341-4731836f34d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:37.963929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bb0ba-9274-44eb-8341-4731836f34d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:37.969059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bb0ba-9274-44eb-8341-4731836f34d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:37.969059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bb0ba-9274-44eb-8341-4731836f34d5 admin admin] PATH_INFO: `/v3/projects/2e073d4d15a243eca3ea41985b8150b0` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:37.969059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3bb0ba-9274-44eb-8341-4731836f34d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e073d4d15a243eca3ea41985b8150b0)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:37.969059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3bb0ba-9274-44eb-8341-4731836f34d5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:37.992022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d9fbdb9f-e11b-4fb2-ab0e-124155e7cfb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dd86b6dd9894aac971943b2fa859bd1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:37.992022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d9fbdb9f-e11b-4fb2-ab0e-124155e7cfb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dd86b6dd9894aac971943b2fa859bd1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:38.028120 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1784/3570] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:12:37 2022] DELETE /identity/v3/users/2dd86b6dd9894aac971943b2fa859bd1 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:38.029251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbdb9f-e11b-4fb2-ab0e-124155e7cfb1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:38.032417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d9fbdb9f-e11b-4fb2-ab0e-124155e7cfb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:38.134131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2a3bb0ba-9274-44eb-8341-4731836f34d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e073d4d15a243eca3ea41985b8150b0', 'request_id': 'req-2a3bb0ba-9274-44eb-8341-4731836f34d5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:38.141034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2a3bb0ba-9274-44eb-8341-4731836f34d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e073d4d15a243eca3ea41985b8150b0', 'request_id': 'req-2a3bb0ba-9274-44eb-8341-4731836f34d5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:38.144965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fbdb9f-e11b-4fb2-ab0e-124155e7cfb1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:38.208137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2258667-83aa-40fc-a929-7b343340ec92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:38.208797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2258667-83aa-40fc-a929-7b343340ec92 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:38.209450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2258667-83aa-40fc-a929-7b343340ec92 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:38.210192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2258667-83aa-40fc-a929-7b343340ec92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:38.212619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a2258667-83aa-40fc-a929-7b343340ec92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:38.307908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2a3bb0ba-9274-44eb-8341-4731836f34d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e073d4d15a243eca3ea41985b8150b0', 'request_id': 'req-2a3bb0ba-9274-44eb-8341-4731836f34d5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:38.330683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2a3bb0ba-9274-44eb-8341-4731836f34d5 admin admin] The token cache is being invalidate because project 2e073d4d15a243eca3ea41985b8150b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:12:38.331339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2a3bb0ba-9274-44eb-8341-4731836f34d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:38.333857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2a3bb0ba-9274-44eb-8341-4731836f34d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:38.341039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2258667-83aa-40fc-a929-7b343340ec92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:38.342715 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1787/3571] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:12:37 2022] DELETE /identity/v3/projects/2e073d4d15a243eca3ea41985b8150b0 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:38.343781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2258667-83aa-40fc-a929-7b343340ec92 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:38.350572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcedf7-8e2c-467f-8076-192c3f4de109 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:38.350954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcedf7-8e2c-467f-8076-192c3f4de109 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:38.351300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dcedf7-8e2c-467f-8076-192c3f4de109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:38.360455 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1785/3572] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:12:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:12:38.385641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-19dcedf7-8e2c-467f-8076-192c3f4de109 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:38.387377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-19dcedf7-8e2c-467f-8076-192c3f4de109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:38.396775 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1788/3573] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:12:38.405852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbfbdb-30af-4af1-be3b-a1b0306c9cb5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:38.406320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbfbdb-30af-4af1-be3b-a1b0306c9cb5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:38.406652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dbfbdb-30af-4af1-be3b-a1b0306c9cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:38.428932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b8dbfbdb-30af-4af1-be3b-a1b0306c9cb5 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:38.430865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b8dbfbdb-30af-4af1-be3b-a1b0306c9cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:38.442219 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1786/3574] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:38.445596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dcedf7-8e2c-467f-8076-192c3f4de109 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:38.461712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dcedf7-8e2c-467f-8076-192c3f4de109 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:38.469753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070eb5ea-15fa-4dca-be9d-be351079a8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:38.470462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eb5ea-15fa-4dca-be9d-be351079a8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:38.470752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eb5ea-15fa-4dca-be9d-be351079a8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:38.471034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eb5ea-15fa-4dca-be9d-be351079a8e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:38.471781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070eb5ea-15fa-4dca-be9d-be351079a8e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:12:38.472094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070eb5ea-15fa-4dca-be9d-be351079a8e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:38.473783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070eb5ea-15fa-4dca-be9d-be351079a8e7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:38.488042 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1789/3575] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:12:38.492433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbfbdb-30af-4af1-be3b-a1b0306c9cb5 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:38.503123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dbfbdb-30af-4af1-be3b-a1b0306c9cb5 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:38.511027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5540a-3600-493f-84ab-ba3f744c6949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:38.511711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5540a-3600-493f-84ab-ba3f744c6949 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:38.512080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5540a-3600-493f-84ab-ba3f744c6949 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:38.512335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5540a-3600-493f-84ab-ba3f744c6949 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:38.513191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5540a-3600-493f-84ab-ba3f744c6949 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:38.515039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5540a-3600-493f-84ab-ba3f744c6949 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:38.563929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070eb5ea-15fa-4dca-be9d-be351079a8e7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:38.567610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-070eb5ea-15fa-4dca-be9d-be351079a8e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:38.594895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070eb5ea-15fa-4dca-be9d-be351079a8e7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:38.611522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f0f87-b035-4951-8686-abf0f08a9f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:38.612808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f0f87-b035-4951-8686-abf0f08a9f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:38.613984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f0f87-b035-4951-8686-abf0f08a9f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:38.614372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f0f87-b035-4951-8686-abf0f08a9f3b admin admin] PATH_INFO: `/v3/projects/8047aaa6e1f7401fbcf4b4cd5dcd004e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:38.616679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f0f87-b035-4951-8686-abf0f08a9f3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8047aaa6e1f7401fbcf4b4cd5dcd004e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:38.619223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f0f87-b035-4951-8686-abf0f08a9f3b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:38.679357 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1787/3576] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:12:38 2022] POST /identity/v3/projects => generated 373 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:12:38.682427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5540a-3600-493f-84ab-ba3f744c6949 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:38.809336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5540a-3600-493f-84ab-ba3f744c6949 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:38.814864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d0407-60b4-407b-8c61-b400f24c0f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:38.815982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d0407-60b4-407b-8c61-b400f24c0f95 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:38.816678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d0407-60b4-407b-8c61-b400f24c0f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:38.816678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d0407-60b4-407b-8c61-b400f24c0f95 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:38.821053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d0407-60b4-407b-8c61-b400f24c0f95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:38.821053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d0407-60b4-407b-8c61-b400f24c0f95 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:38.858237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cd3f0f87-b035-4951-8686-abf0f08a9f3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8047aaa6e1f7401fbcf4b4cd5dcd004e', 'request_id': 'req-cd3f0f87-b035-4951-8686-abf0f08a9f3b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:38.871588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cd3f0f87-b035-4951-8686-abf0f08a9f3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8047aaa6e1f7401fbcf4b4cd5dcd004e', 'request_id': 'req-cd3f0f87-b035-4951-8686-abf0f08a9f3b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:39.080147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cd3f0f87-b035-4951-8686-abf0f08a9f3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8047aaa6e1f7401fbcf4b4cd5dcd004e', 'request_id': 'req-cd3f0f87-b035-4951-8686-abf0f08a9f3b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:39.087449 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1788/3577] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:12:38 2022] POST /identity/v3/users => generated 287 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:12:39.089499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d0407-60b4-407b-8c61-b400f24c0f95 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:39.091235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d13d0407-60b4-407b-8c61-b400f24c0f95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:39.099455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cd3f0f87-b035-4951-8686-abf0f08a9f3b admin admin] The token cache is being invalidate because project 8047aaa6e1f7401fbcf4b4cd5dcd004e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:12:39.099974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cd3f0f87-b035-4951-8686-abf0f08a9f3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:39.108081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cd3f0f87-b035-4951-8686-abf0f08a9f3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:39.108081 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1790/3578] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:12:38 2022] DELETE /identity/v3/projects/8047aaa6e1f7401fbcf4b4cd5dcd004e => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:39.108081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f0f87-b035-4951-8686-abf0f08a9f3b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:39.108081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cd3f0f87-b035-4951-8686-abf0f08a9f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:39.108692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d0407-60b4-407b-8c61-b400f24c0f95 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:39.195214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e9a14-6118-48de-8006-9296f1a86618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:39.196330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e9a14-6118-48de-8006-9296f1a86618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:39.196624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e9a14-6118-48de-8006-9296f1a86618 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:39.196901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463e9a14-6118-48de-8006-9296f1a86618 admin admin] PATH_INFO: `/v3/projects/659cd6db8d6741329dded06813f0ef93` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:39.200504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e9a14-6118-48de-8006-9296f1a86618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=659cd6db8d6741329dded06813f0ef93)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:39.202710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463e9a14-6118-48de-8006-9296f1a86618 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:39.214878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f0f87-b035-4951-8686-abf0f08a9f3b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:39.221372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920f6e1-a669-4795-ad85-d875fdacc9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:39.222599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920f6e1-a669-4795-ad85-d875fdacc9f7 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:39.223115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920f6e1-a669-4795-ad85-d875fdacc9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:39.223625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920f6e1-a669-4795-ad85-d875fdacc9f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:39.224755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920f6e1-a669-4795-ad85-d875fdacc9f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:39.227062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920f6e1-a669-4795-ad85-d875fdacc9f7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:39.246371 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1791/3579] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:12:39.247872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8aff2-03c5-4132-b232-851a8783637a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:39.249972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8aff2-03c5-4132-b232-851a8783637a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:39.249972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d8aff2-03c5-4132-b232-851a8783637a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:39.269338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-36d8aff2-03c5-4132-b232-851a8783637a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:39.270615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-36d8aff2-03c5-4132-b232-851a8783637a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:39.280394 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1792/3580] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:39.281189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8aff2-03c5-4132-b232-851a8783637a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:39.291164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d8aff2-03c5-4132-b232-851a8783637a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:39.295885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c7128-b812-451d-9449-9c6e798a24bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:39.296525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c7128-b812-451d-9449-9c6e798a24bf admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:39.296807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c7128-b812-451d-9449-9c6e798a24bf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:39.297084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c7128-b812-451d-9449-9c6e798a24bf admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:39.297813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c7128-b812-451d-9449-9c6e798a24bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:39.299437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c7128-b812-451d-9449-9c6e798a24bf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:39.310977 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1793/3581] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:12:39.312307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442d67c-b60f-4e40-a689-bae7f8132614 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:39.316901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442d67c-b60f-4e40-a689-bae7f8132614 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:39.316901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6442d67c-b60f-4e40-a689-bae7f8132614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:39.334443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6442d67c-b60f-4e40-a689-bae7f8132614 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:39.335664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6442d67c-b60f-4e40-a689-bae7f8132614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:39.343861 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1794/3582] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:39.344722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442d67c-b60f-4e40-a689-bae7f8132614 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:39.354061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6442d67c-b60f-4e40-a689-bae7f8132614 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:39.358591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d5538-aedb-40bf-8c44-60cc68448a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:39.359448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d5538-aedb-40bf-8c44-60cc68448a07 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:39.359732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d5538-aedb-40bf-8c44-60cc68448a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:39.360007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36d5538-aedb-40bf-8c44-60cc68448a07 admin admin] PATH_INFO: `/v3/projects/d64af7d364a74dc6b9c527532a5d8abd/users/752fbb20c21449bb80bc9de2da474865/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:39.376260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d5538-aedb-40bf-8c44-60cc68448a07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d64af7d364a74dc6b9c527532a5d8abd, user_id=752fbb20c21449bb80bc9de2da474865, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:39.379026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36d5538-aedb-40bf-8c44-60cc68448a07 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:39.382978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-463e9a14-6118-48de-8006-9296f1a86618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '659cd6db8d6741329dded06813f0ef93', 'request_id': 'req-463e9a14-6118-48de-8006-9296f1a86618'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:39.393411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-463e9a14-6118-48de-8006-9296f1a86618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '659cd6db8d6741329dded06813f0ef93', 'request_id': 'req-463e9a14-6118-48de-8006-9296f1a86618'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:39.464881 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1795/3583] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:12:39 2022] PUT /identity/v3/projects/d64af7d364a74dc6b9c527532a5d8abd/users/752fbb20c21449bb80bc9de2da474865/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:39.465882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d5538-aedb-40bf-8c44-60cc68448a07 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:39.534659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36d5538-aedb-40bf-8c44-60cc68448a07 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:39.540321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58987f90-60ea-4596-8d63-8d359b1a4666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:39.541888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58987f90-60ea-4596-8d63-8d359b1a4666 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:39.542271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58987f90-60ea-4596-8d63-8d359b1a4666 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:39.542647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58987f90-60ea-4596-8d63-8d359b1a4666 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:39.544056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58987f90-60ea-4596-8d63-8d359b1a4666 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:12:39.544368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58987f90-60ea-4596-8d63-8d359b1a4666 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:39.546851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58987f90-60ea-4596-8d63-8d359b1a4666 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:39.547611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-463e9a14-6118-48de-8006-9296f1a86618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '659cd6db8d6741329dded06813f0ef93', 'request_id': 'req-463e9a14-6118-48de-8006-9296f1a86618'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:39.566101 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1796/3584] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:12:39.567829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a3ec9-b820-449f-95f6-989e78c329db None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:39.568223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a3ec9-b820-449f-95f6-989e78c329db None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:39.569238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a3ec9-b820-449f-95f6-989e78c329db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:39.572085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-463e9a14-6118-48de-8006-9296f1a86618 admin admin] The token cache is being invalidate because project 659cd6db8d6741329dded06813f0ef93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:12:39.572373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-463e9a14-6118-48de-8006-9296f1a86618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:39.577197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-463e9a14-6118-48de-8006-9296f1a86618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:39.578971 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1789/3585] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:12:39 2022] DELETE /identity/v3/projects/659cd6db8d6741329dded06813f0ef93 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:39.579965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e9a14-6118-48de-8006-9296f1a86618 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:39.581700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-463e9a14-6118-48de-8006-9296f1a86618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:39.601684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8f9a3ec9-b820-449f-95f6-989e78c329db None None] MFA Rules not processed for user `752fbb20c21449bb80bc9de2da474865`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:39.603047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8f9a3ec9-b820-449f-95f6-989e78c329db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:39.669182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463e9a14-6118-48de-8006-9296f1a86618 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:39.674248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4f790-906a-4ce8-9eda-d975b1ae1205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:39.674988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4f790-906a-4ce8-9eda-d975b1ae1205 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:39.675347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4f790-906a-4ce8-9eda-d975b1ae1205 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:39.675841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4f790-906a-4ce8-9eda-d975b1ae1205 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:39.676606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4f790-906a-4ce8-9eda-d975b1ae1205 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:39.678763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4f790-906a-4ce8-9eda-d975b1ae1205 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:39.687519 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1797/3586] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:39.689931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b9c1c-27c7-45f5-a694-3e849b5afa44 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:39.689931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b9c1c-27c7-45f5-a694-3e849b5afa44 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:39.689931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4b9c1c-27c7-45f5-a694-3e849b5afa44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:39.717964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8d4b9c1c-27c7-45f5-a694-3e849b5afa44 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:39.719358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8d4b9c1c-27c7-45f5-a694-3e849b5afa44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:39.727780 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1798/3587] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:39.736629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb1592-5197-48e3-b7b7-cf74181925cf None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:39.736629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb1592-5197-48e3-b7b7-cf74181925cf None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:39.736629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb1592-5197-48e3-b7b7-cf74181925cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:39.757251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-55bb1592-5197-48e3-b7b7-cf74181925cf None None] MFA Rules not processed for user `752fbb20c21449bb80bc9de2da474865`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:39.758931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-55bb1592-5197-48e3-b7b7-cf74181925cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:39.768501 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1799/3588] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:39.781326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb3814-2109-4abf-8c83-39a60c7deefc None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:39.783434 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1790/3589] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:12:39 2022] POST /identity/v3/projects => generated 395 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:12:39.784424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4f790-906a-4ce8-9eda-d975b1ae1205 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:39.785023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb3814-2109-4abf-8c83-39a60c7deefc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:39.785827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb3814-2109-4abf-8c83-39a60c7deefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:39.787497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8aa4f790-906a-4ce8-9eda-d975b1ae1205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:39.821027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-43fb3814-2109-4abf-8c83-39a60c7deefc None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:39.821027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-43fb3814-2109-4abf-8c83-39a60c7deefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:39.884651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4f790-906a-4ce8-9eda-d975b1ae1205 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:39.889983 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1800/3590] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:39.889983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb3814-2109-4abf-8c83-39a60c7deefc None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:39.897675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eaf68e-2f9d-4fb6-a525-a8a61d99e190 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:39.898630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eaf68e-2f9d-4fb6-a525-a8a61d99e190 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:39.899055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb3814-2109-4abf-8c83-39a60c7deefc None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:39.899302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eaf68e-2f9d-4fb6-a525-a8a61d99e190 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:39.899725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eaf68e-2f9d-4fb6-a525-a8a61d99e190 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:39.905944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e97a02-27ac-4255-af19-d3bf630c258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:39.906645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e97a02-27ac-4255-af19-d3bf630c258b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:39.906934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e97a02-27ac-4255-af19-d3bf630c258b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:39.907240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e97a02-27ac-4255-af19-d3bf630c258b admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:39.908158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e97a02-27ac-4255-af19-d3bf630c258b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:39.910736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e97a02-27ac-4255-af19-d3bf630c258b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:39.964738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eaf68e-2f9d-4fb6-a525-a8a61d99e190 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:39.966500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eaf68e-2f9d-4fb6-a525-a8a61d99e190 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:39.975682 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1791/3591] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:12:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:12:39.976500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eaf68e-2f9d-4fb6-a525-a8a61d99e190 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:39.989865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eaf68e-2f9d-4fb6-a525-a8a61d99e190 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:39.995083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffb97b-0e7d-4077-9b94-67605fe2ebfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:39.995758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffb97b-0e7d-4077-9b94-67605fe2ebfe admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:39.996061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffb97b-0e7d-4077-9b94-67605fe2ebfe admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:39.996305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffb97b-0e7d-4077-9b94-67605fe2ebfe admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:39.997089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffb97b-0e7d-4077-9b94-67605fe2ebfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:12:39.997776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffb97b-0e7d-4077-9b94-67605fe2ebfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:39.999205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffb97b-0e7d-4077-9b94-67605fe2ebfe admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:40.013570 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1801/3592] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:12:39 2022] POST /identity/v3/users => generated 298 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:12:40.030837 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1792/3593] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:12:40.035021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e97a02-27ac-4255-af19-d3bf630c258b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:40.038648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffb97b-0e7d-4077-9b94-67605fe2ebfe admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:40.054361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffb97b-0e7d-4077-9b94-67605fe2ebfe admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:40.055294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e97a02-27ac-4255-af19-d3bf630c258b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:40.066789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757df20-05a0-447a-996b-520c37cce5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:40.073796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95145f-b0cb-4506-bbeb-3b9520fdd108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:40.079903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95145f-b0cb-4506-bbeb-3b9520fdd108 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:40.079903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95145f-b0cb-4506-bbeb-3b9520fdd108 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:40.079903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b95145f-b0cb-4506-bbeb-3b9520fdd108 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:40.079903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95145f-b0cb-4506-bbeb-3b9520fdd108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:40.079903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b95145f-b0cb-4506-bbeb-3b9520fdd108 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:40.082148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:40.083695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757df20-05a0-447a-996b-520c37cce5c4 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:40.084144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757df20-05a0-447a-996b-520c37cce5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:40.084575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757df20-05a0-447a-996b-520c37cce5c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:40.086256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757df20-05a0-447a-996b-520c37cce5c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:40.089617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757df20-05a0-447a-996b-520c37cce5c4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:40.100417 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1802/3594] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:12:40.101185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95145f-b0cb-4506-bbeb-3b9520fdd108 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:40.125767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:40.130460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b95145f-b0cb-4506-bbeb-3b9520fdd108 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:40.137050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8bd911-13b3-49db-b50e-dcc36cfa58d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:40.137746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bd911-13b3-49db-b50e-dcc36cfa58d2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:40.138518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bd911-13b3-49db-b50e-dcc36cfa58d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:40.138654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bd911-13b3-49db-b50e-dcc36cfa58d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:40.139493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8bd911-13b3-49db-b50e-dcc36cfa58d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:40.141003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8bd911-13b3-49db-b50e-dcc36cfa58d2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:40.157970 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1803/3595] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:12:40.161877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8bd911-13b3-49db-b50e-dcc36cfa58d2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:40.172951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8bd911-13b3-49db-b50e-dcc36cfa58d2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:40.178075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f9bcd0-e068-4acd-9eda-a08a0b0512bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:40.179163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9bcd0-e068-4acd-9eda-a08a0b0512bf admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:40.179451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9bcd0-e068-4acd-9eda-a08a0b0512bf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:40.179732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f9bcd0-e068-4acd-9eda-a08a0b0512bf admin admin] PATH_INFO: `/v3/projects/d26d403e254e4caa887f8155d74c84c4/users/ffb90cfa7828412f96f330dab49466cd/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:40.191660 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1793/3596] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:12:40 2022] POST /identity/v3/projects => generated 399 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:12:40.194128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757df20-05a0-447a-996b-520c37cce5c4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:40.197204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9bcd0-e068-4acd-9eda-a08a0b0512bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d26d403e254e4caa887f8155d74c84c4, user_id=ffb90cfa7828412f96f330dab49466cd, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:40.201078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f9bcd0-e068-4acd-9eda-a08a0b0512bf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:40.274698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757df20-05a0-447a-996b-520c37cce5c4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:40.280760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eea666-e05a-404b-8e6a-5bd109cb8e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:40.281672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eea666-e05a-404b-8e6a-5bd109cb8e19 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:40.282014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eea666-e05a-404b-8e6a-5bd109cb8e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:40.282298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eea666-e05a-404b-8e6a-5bd109cb8e19 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:40.283000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eea666-e05a-404b-8e6a-5bd109cb8e19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:40.285108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eea666-e05a-404b-8e6a-5bd109cb8e19 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:40.295040 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1804/3597] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:12:40 2022] PUT /identity/v3/projects/d26d403e254e4caa887f8155d74c84c4/users/ffb90cfa7828412f96f330dab49466cd/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:12:40.301090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fdd0ef-a519-4aac-90ed-83c610c2feb8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:40.301090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fdd0ef-a519-4aac-90ed-83c610c2feb8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:40.301090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fdd0ef-a519-4aac-90ed-83c610c2feb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:40.360984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-38fdd0ef-a519-4aac-90ed-83c610c2feb8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:40.362720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-38fdd0ef-a519-4aac-90ed-83c610c2feb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:40.425771 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1794/3598] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:12:40 2022] POST /identity/v3/users => generated 300 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:12:40.429381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eea666-e05a-404b-8e6a-5bd109cb8e19 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:40.464947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eea666-e05a-404b-8e6a-5bd109cb8e19 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:40.552270 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1805/3599] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:40.561235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fdd0ef-a519-4aac-90ed-83c610c2feb8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:40.563289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-38fdd0ef-a519-4aac-90ed-83c610c2feb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:40.627290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fec9f-7dc1-4eeb-8ab6-029c331bfe25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:40.636146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fec9f-7dc1-4eeb-8ab6-029c331bfe25 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:40.637513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fec9f-7dc1-4eeb-8ab6-029c331bfe25 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:40.644255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fec9f-7dc1-4eeb-8ab6-029c331bfe25 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:40.644255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8fec9f-7dc1-4eeb-8ab6-029c331bfe25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:40.644255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8fec9f-7dc1-4eeb-8ab6-029c331bfe25 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:40.654509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fdd0ef-a519-4aac-90ed-83c610c2feb8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:40.658365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b104318-03c8-4986-be5b-b26504326dfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:40.659016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b104318-03c8-4986-be5b-b26504326dfa service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:40.659301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b104318-03c8-4986-be5b-b26504326dfa service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:40.659580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b104318-03c8-4986-be5b-b26504326dfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:40.662263 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1795/3600] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:12:40.664187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fec9f-7dc1-4eeb-8ab6-029c331bfe25 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:40.670947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b104318-03c8-4986-be5b-b26504326dfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:40.679406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fec9f-7dc1-4eeb-8ab6-029c331bfe25 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:40.679762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b104318-03c8-4986-be5b-b26504326dfa service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:40.687959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41932aeb-7da5-48eb-b857-5bb7ec5f23dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:40.688942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41932aeb-7da5-48eb-b857-5bb7ec5f23dd admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:40.689232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41932aeb-7da5-48eb-b857-5bb7ec5f23dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:40.689571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41932aeb-7da5-48eb-b857-5bb7ec5f23dd admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:40.690383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41932aeb-7da5-48eb-b857-5bb7ec5f23dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:40.692190 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1806/3601] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:12:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:12:40.693208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41932aeb-7da5-48eb-b857-5bb7ec5f23dd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:40.706294 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1796/3602] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:12:40.710302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b104318-03c8-4986-be5b-b26504326dfa service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:40.720077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b104318-03c8-4986-be5b-b26504326dfa service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:40.724900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aed3bc-3a1e-4802-808c-6f309925920e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:40.725773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aed3bc-3a1e-4802-808c-6f309925920e admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:40.726141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aed3bc-3a1e-4802-808c-6f309925920e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:40.726372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aed3bc-3a1e-4802-808c-6f309925920e admin admin] PATH_INFO: `/v3/projects/7f665b043fd24ef986c43932f44d66d3/users/34509fc3349248e9acfee855487f92d6/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:40.741580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aed3bc-3a1e-4802-808c-6f309925920e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f665b043fd24ef986c43932f44d66d3, user_id=34509fc3349248e9acfee855487f92d6, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:40.744805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aed3bc-3a1e-4802-808c-6f309925920e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:40.832333 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1807/3603] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:12:40 2022] PUT /identity/v3/projects/7f665b043fd24ef986c43932f44d66d3/users/34509fc3349248e9acfee855487f92d6/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:40.844074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bd796-f291-4747-ba57-03011fc2e43b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:40.847615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bd796-f291-4747-ba57-03011fc2e43b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:40.848047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6bd796-f291-4747-ba57-03011fc2e43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:40.890651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7c6bd796-f291-4747-ba57-03011fc2e43b None None] MFA Rules not processed for user `34509fc3349248e9acfee855487f92d6`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:40.891883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7c6bd796-f291-4747-ba57-03011fc2e43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:40.987503 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1797/3604] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:12:41.002223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aed3bc-3a1e-4802-808c-6f309925920e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:41.004613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-71aed3bc-3a1e-4802-808c-6f309925920e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:41.143044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aed3bc-3a1e-4802-808c-6f309925920e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:41.149607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716270a-2f6e-4def-81d8-4c548b7c7e2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:41.150377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716270a-2f6e-4def-81d8-4c548b7c7e2b service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:41.150785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716270a-2f6e-4def-81d8-4c548b7c7e2b service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:41.151152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716270a-2f6e-4def-81d8-4c548b7c7e2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:41.161380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716270a-2f6e-4def-81d8-4c548b7c7e2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:41.163252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716270a-2f6e-4def-81d8-4c548b7c7e2b service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:41.173313 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1808/3605] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:12:41 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:12:41.214061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6bd796-f291-4747-ba57-03011fc2e43b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:41.349990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6bd796-f291-4747-ba57-03011fc2e43b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:41.364358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035feed-2c17-46dd-80d2-d08f367197d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:41.365556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035feed-2c17-46dd-80d2-d08f367197d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:41.365970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035feed-2c17-46dd-80d2-d08f367197d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:41.366262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7035feed-2c17-46dd-80d2-d08f367197d6 admin admin] PATH_INFO: `/v3/users/34509fc3349248e9acfee855487f92d6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:41.368138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035feed-2c17-46dd-80d2-d08f367197d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34509fc3349248e9acfee855487f92d6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:41.370505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7035feed-2c17-46dd-80d2-d08f367197d6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:41.739138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7035feed-2c17-46dd-80d2-d08f367197d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34509fc3349248e9acfee855487f92d6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:41.739822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7035feed-2c17-46dd-80d2-d08f367197d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34509fc3349248e9acfee855487f92d6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:41.753001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7035feed-2c17-46dd-80d2-d08f367197d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34509fc3349248e9acfee855487f92d6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:41.841579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7035feed-2c17-46dd-80d2-d08f367197d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34509fc3349248e9acfee855487f92d6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:41.842542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7035feed-2c17-46dd-80d2-d08f367197d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34509fc3349248e9acfee855487f92d6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:41.871846 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1798/3606] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:12:41 2022] DELETE /identity/v3/users/34509fc3349248e9acfee855487f92d6 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:41.876439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a90140-1d68-470a-8346-69c047092910 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:41.876962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a90140-1d68-470a-8346-69c047092910 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:41.876962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a90140-1d68-470a-8346-69c047092910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:41.898294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a1a90140-1d68-470a-8346-69c047092910 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:41.899821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a1a90140-1d68-470a-8346-69c047092910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:41.971244 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1809/3607] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:41.990791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035feed-2c17-46dd-80d2-d08f367197d6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:41.992729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7035feed-2c17-46dd-80d2-d08f367197d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:42.087932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7035feed-2c17-46dd-80d2-d08f367197d6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:42.098111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f92f10-9dea-4158-875a-ceeb85dd4db4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:42.099010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f92f10-9dea-4158-875a-ceeb85dd4db4 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:42.099265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f92f10-9dea-4158-875a-ceeb85dd4db4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:42.099631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f92f10-9dea-4158-875a-ceeb85dd4db4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:42.133010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f92f10-9dea-4158-875a-ceeb85dd4db4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:42.134959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f92f10-9dea-4158-875a-ceeb85dd4db4 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:42.154777 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1799/3608] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:12:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:12:42.554524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:42.743304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:43.426858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a90140-1d68-470a-8346-69c047092910 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:43.428303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a1a90140-1d68-470a-8346-69c047092910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:43.440986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a90140-1d68-470a-8346-69c047092910 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:43.447882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42f93d1-d085-4292-9043-bc77abcc159e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:43.449067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f93d1-d085-4292-9043-bc77abcc159e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:43.449550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f93d1-d085-4292-9043-bc77abcc159e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:43.450061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42f93d1-d085-4292-9043-bc77abcc159e admin admin] PATH_INFO: `/v3/projects/7f665b043fd24ef986c43932f44d66d3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:43.457040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f93d1-d085-4292-9043-bc77abcc159e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f665b043fd24ef986c43932f44d66d3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:43.457040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42f93d1-d085-4292-9043-bc77abcc159e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:43.647645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d42f93d1-d085-4292-9043-bc77abcc159e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f665b043fd24ef986c43932f44d66d3', 'request_id': 'req-d42f93d1-d085-4292-9043-bc77abcc159e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:43.663436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d42f93d1-d085-4292-9043-bc77abcc159e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f665b043fd24ef986c43932f44d66d3', 'request_id': 'req-d42f93d1-d085-4292-9043-bc77abcc159e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:43.770826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d42f93d1-d085-4292-9043-bc77abcc159e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f665b043fd24ef986c43932f44d66d3', 'request_id': 'req-d42f93d1-d085-4292-9043-bc77abcc159e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:43.790341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d42f93d1-d085-4292-9043-bc77abcc159e admin admin] The token cache is being invalidate because project 7f665b043fd24ef986c43932f44d66d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:12:43.790753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d42f93d1-d085-4292-9043-bc77abcc159e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:43.791794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d42f93d1-d085-4292-9043-bc77abcc159e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:43.794544 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1810/3609] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:12:43 2022] DELETE /identity/v3/projects/7f665b043fd24ef986c43932f44d66d3 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:43.799950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5be24d-f41a-4739-b53a-cbfdff47dda6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:43.800445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5be24d-f41a-4739-b53a-cbfdff47dda6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:43.800555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5be24d-f41a-4739-b53a-cbfdff47dda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:43.830807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7b5be24d-f41a-4739-b53a-cbfdff47dda6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:43.832524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7b5be24d-f41a-4739-b53a-cbfdff47dda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:43.955455 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1800/3610] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:43.963342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c048-1b1a-4926-a282-d377cd7a8121 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:43.963628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c048-1b1a-4926-a282-d377cd7a8121 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:43.963907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2678c048-1b1a-4926-a282-d377cd7a8121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:43.992473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2678c048-1b1a-4926-a282-d377cd7a8121 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:43.993762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2678c048-1b1a-4926-a282-d377cd7a8121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:44.003817 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1811/3611] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:44.007978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5be24d-f41a-4739-b53a-cbfdff47dda6 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:44.022718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5be24d-f41a-4739-b53a-cbfdff47dda6 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:44.028551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14e142-181d-43f9-96da-bb1ebf0954d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:44.033886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14e142-181d-43f9-96da-bb1ebf0954d9 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:44.033886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14e142-181d-43f9-96da-bb1ebf0954d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:44.033886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c14e142-181d-43f9-96da-bb1ebf0954d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:44.033886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14e142-181d-43f9-96da-bb1ebf0954d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:12:44.033886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14e142-181d-43f9-96da-bb1ebf0954d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:44.033886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c14e142-181d-43f9-96da-bb1ebf0954d9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:44.057441 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1801/3612] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:12:44.064121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678c048-1b1a-4926-a282-d377cd7a8121 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:44.088318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2678c048-1b1a-4926-a282-d377cd7a8121 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:44.094330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47713017-9b2e-4981-8f6e-b173a4df0636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:44.095230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47713017-9b2e-4981-8f6e-b173a4df0636 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:44.095673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47713017-9b2e-4981-8f6e-b173a4df0636 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:44.096138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47713017-9b2e-4981-8f6e-b173a4df0636 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:44.097018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47713017-9b2e-4981-8f6e-b173a4df0636 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:44.099207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47713017-9b2e-4981-8f6e-b173a4df0636 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:44.199969 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1812/3613] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:12:44 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:12:44.206549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14e142-181d-43f9-96da-bb1ebf0954d9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:44.218465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c14e142-181d-43f9-96da-bb1ebf0954d9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:44.273009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:44.313702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c717d59-93c0-456d-ae79-56bd4b1d0c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:44.315291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c717d59-93c0-456d-ae79-56bd4b1d0c7e admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:44.315680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c717d59-93c0-456d-ae79-56bd4b1d0c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:44.316114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c717d59-93c0-456d-ae79-56bd4b1d0c7e admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:44.316916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c717d59-93c0-456d-ae79-56bd4b1d0c7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:44.319439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c717d59-93c0-456d-ae79-56bd4b1d0c7e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:44.437888 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1802/3614] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:12:44 2022] POST /identity/v3/users => generated 290 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:12:44.447156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47713017-9b2e-4981-8f6e-b173a4df0636 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:44.465774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47713017-9b2e-4981-8f6e-b173a4df0636 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:44.472704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfb254-8767-40c7-9406-f5f1ed931ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:44.475600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfb254-8767-40c7-9406-f5f1ed931ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:44.475600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfb254-8767-40c7-9406-f5f1ed931ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:44.475600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfb254-8767-40c7-9406-f5f1ed931ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:44.479191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dfb254-8767-40c7-9406-f5f1ed931ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:44.481023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dfb254-8767-40c7-9406-f5f1ed931ce5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:44.493992 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1813/3615] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:12:44.498309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c717d59-93c0-456d-ae79-56bd4b1d0c7e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:44.510531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c717d59-93c0-456d-ae79-56bd4b1d0c7e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:44.515843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e85f5-45fb-432d-8a87-fab5d786d42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:44.516840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e85f5-45fb-432d-8a87-fab5d786d42b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:44.517231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e85f5-45fb-432d-8a87-fab5d786d42b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:44.517617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e85f5-45fb-432d-8a87-fab5d786d42b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:44.518721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e85f5-45fb-432d-8a87-fab5d786d42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:44.520614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e85f5-45fb-432d-8a87-fab5d786d42b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:44.536777 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1803/3616] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:12:44.537554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfb254-8767-40c7-9406-f5f1ed931ce5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:44.550182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfb254-8767-40c7-9406-f5f1ed931ce5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:44.557821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a741cef-9d5b-4b3c-aa8a-35ddf07e97cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:44.558785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a741cef-9d5b-4b3c-aa8a-35ddf07e97cb admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:44.559074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a741cef-9d5b-4b3c-aa8a-35ddf07e97cb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:44.559355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a741cef-9d5b-4b3c-aa8a-35ddf07e97cb admin admin] PATH_INFO: `/v3/projects/b7065418cf374caeb78ea178076af5ba/users/280a6ac1861343c1a52ac4886d3169e1/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:44.576709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a741cef-9d5b-4b3c-aa8a-35ddf07e97cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7065418cf374caeb78ea178076af5ba, user_id=280a6ac1861343c1a52ac4886d3169e1, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:44.580105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a741cef-9d5b-4b3c-aa8a-35ddf07e97cb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:44.686757 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1814/3617] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:12:44 2022] PUT /identity/v3/projects/b7065418cf374caeb78ea178076af5ba/users/280a6ac1861343c1a52ac4886d3169e1/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:44.698827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc276b04-96ce-427e-8d1c-7160fd326416 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:44.699137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc276b04-96ce-427e-8d1c-7160fd326416 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:44.699405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc276b04-96ce-427e-8d1c-7160fd326416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:44.741096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bc276b04-96ce-427e-8d1c-7160fd326416 None None] MFA Rules not processed for user `280a6ac1861343c1a52ac4886d3169e1`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:44.742326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bc276b04-96ce-427e-8d1c-7160fd326416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:44.835376 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1804/3618] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:44.847014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a741cef-9d5b-4b3c-aa8a-35ddf07e97cb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:44.848867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5a741cef-9d5b-4b3c-aa8a-35ddf07e97cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:44.969487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a741cef-9d5b-4b3c-aa8a-35ddf07e97cb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:44.979429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d4997-2bc4-486f-a7a8-1e216db9ace9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:44.980557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d4997-2bc4-486f-a7a8-1e216db9ace9 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:44.991389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:44.991389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d4997-2bc4-486f-a7a8-1e216db9ace9 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:44.991389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d4997-2bc4-486f-a7a8-1e216db9ace9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:45.000907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d4997-2bc4-486f-a7a8-1e216db9ace9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:45.005903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d4997-2bc4-486f-a7a8-1e216db9ace9 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:45.025010 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1815/3619] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:12:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:12:45.096591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc276b04-96ce-427e-8d1c-7160fd326416 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:45.182400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc276b04-96ce-427e-8d1c-7160fd326416 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:45.188592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90aaf0-e7e4-4b4d-a49a-5e5854c3242c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:45.190041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90aaf0-e7e4-4b4d-a49a-5e5854c3242c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:45.190487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90aaf0-e7e4-4b4d-a49a-5e5854c3242c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:45.191174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc90aaf0-e7e4-4b4d-a49a-5e5854c3242c admin admin] PATH_INFO: `/v3/users/280a6ac1861343c1a52ac4886d3169e1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:45.192580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90aaf0-e7e4-4b4d-a49a-5e5854c3242c admin admin] RBAC: Authorizing `identity:delete_user(user_id=280a6ac1861343c1a52ac4886d3169e1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:45.197897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc90aaf0-e7e4-4b4d-a49a-5e5854c3242c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:45.399609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bc90aaf0-e7e4-4b4d-a49a-5e5854c3242c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '280a6ac1861343c1a52ac4886d3169e1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:45.401201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bc90aaf0-e7e4-4b4d-a49a-5e5854c3242c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '280a6ac1861343c1a52ac4886d3169e1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:45.426304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bc90aaf0-e7e4-4b4d-a49a-5e5854c3242c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '280a6ac1861343c1a52ac4886d3169e1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:45.541511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bc90aaf0-e7e4-4b4d-a49a-5e5854c3242c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '280a6ac1861343c1a52ac4886d3169e1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:45.541511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bc90aaf0-e7e4-4b4d-a49a-5e5854c3242c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '280a6ac1861343c1a52ac4886d3169e1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:45.578188 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1805/3620] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:12:45 2022] DELETE /identity/v3/users/280a6ac1861343c1a52ac4886d3169e1 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:45.582768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb32e4-1a99-4be9-8a5e-7a7a7fb9fb9d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:45.583234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb32e4-1a99-4be9-8a5e-7a7a7fb9fb9d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:45.583324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb32e4-1a99-4be9-8a5e-7a7a7fb9fb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:45.605150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-20fb32e4-1a99-4be9-8a5e-7a7a7fb9fb9d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:45.606745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-20fb32e4-1a99-4be9-8a5e-7a7a7fb9fb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:45.724809 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1816/3621] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:45.755509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90aaf0-e7e4-4b4d-a49a-5e5854c3242c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:45.756829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bc90aaf0-e7e4-4b4d-a49a-5e5854c3242c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:45.837586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc90aaf0-e7e4-4b4d-a49a-5e5854c3242c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:45.845677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc554e-63ce-4271-9089-4a70d954fd92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:45.850782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc554e-63ce-4271-9089-4a70d954fd92 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:45.850782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc554e-63ce-4271-9089-4a70d954fd92 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:45.850782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc554e-63ce-4271-9089-4a70d954fd92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:45.872267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc554e-63ce-4271-9089-4a70d954fd92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:45.872267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc554e-63ce-4271-9089-4a70d954fd92 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:45.897741 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1806/3622] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:12:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:12:46.296942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:46.332465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:46.709816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb32e4-1a99-4be9-8a5e-7a7a7fb9fb9d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:46.712674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-20fb32e4-1a99-4be9-8a5e-7a7a7fb9fb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:46.853790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb32e4-1a99-4be9-8a5e-7a7a7fb9fb9d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:46.864918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbf1b7e-c5af-4231-a84b-b6bf6e107964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:46.867316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf1b7e-c5af-4231-a84b-b6bf6e107964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:46.868663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf1b7e-c5af-4231-a84b-b6bf6e107964 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:46.870355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbf1b7e-c5af-4231-a84b-b6bf6e107964 admin admin] PATH_INFO: `/v3/projects/b7065418cf374caeb78ea178076af5ba` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:46.872314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf1b7e-c5af-4231-a84b-b6bf6e107964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7065418cf374caeb78ea178076af5ba)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:46.877095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbf1b7e-c5af-4231-a84b-b6bf6e107964 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:47.125503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1fbf1b7e-c5af-4231-a84b-b6bf6e107964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7065418cf374caeb78ea178076af5ba', 'request_id': 'req-1fbf1b7e-c5af-4231-a84b-b6bf6e107964'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:47.165627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1fbf1b7e-c5af-4231-a84b-b6bf6e107964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7065418cf374caeb78ea178076af5ba', 'request_id': 'req-1fbf1b7e-c5af-4231-a84b-b6bf6e107964'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:47.272961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1fbf1b7e-c5af-4231-a84b-b6bf6e107964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7065418cf374caeb78ea178076af5ba', 'request_id': 'req-1fbf1b7e-c5af-4231-a84b-b6bf6e107964'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:47.292736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1fbf1b7e-c5af-4231-a84b-b6bf6e107964 admin admin] The token cache is being invalidate because project b7065418cf374caeb78ea178076af5ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:12:47.294740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1fbf1b7e-c5af-4231-a84b-b6bf6e107964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:47.296237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1fbf1b7e-c5af-4231-a84b-b6bf6e107964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:47.299667 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1817/3623] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:12:46 2022] DELETE /identity/v3/projects/b7065418cf374caeb78ea178076af5ba => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:47.308127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37b86f-5d6a-49ec-aea7-71bff138f161 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:47.309035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37b86f-5d6a-49ec-aea7-71bff138f161 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:47.309489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c37b86f-5d6a-49ec-aea7-71bff138f161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:47.448066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3c37b86f-5d6a-49ec-aea7-71bff138f161 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:47.449808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3c37b86f-5d6a-49ec-aea7-71bff138f161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:47.457816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:47.530546 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1807/3624] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:47.531115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:47.537750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b885204-e895-4e8f-b402-0a00dd5755cc None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:47.538064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b885204-e895-4e8f-b402-0a00dd5755cc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:47.538349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b885204-e895-4e8f-b402-0a00dd5755cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:47.569771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5b885204-e895-4e8f-b402-0a00dd5755cc None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:47.571022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5b885204-e895-4e8f-b402-0a00dd5755cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:47.581584 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1818/3625] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:47.585553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37b86f-5d6a-49ec-aea7-71bff138f161 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:47.599248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c37b86f-5d6a-49ec-aea7-71bff138f161 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:47.613756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ef9c5-19f0-4d1a-b9e1-61fc743d89bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:47.614488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ef9c5-19f0-4d1a-b9e1-61fc743d89bd admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:47.614762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ef9c5-19f0-4d1a-b9e1-61fc743d89bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:47.618384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382ef9c5-19f0-4d1a-b9e1-61fc743d89bd admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:47.619559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ef9c5-19f0-4d1a-b9e1-61fc743d89bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:12:47.620034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ef9c5-19f0-4d1a-b9e1-61fc743d89bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:47.625007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382ef9c5-19f0-4d1a-b9e1-61fc743d89bd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:47.650879 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1808/3626] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:12:47.655127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b885204-e895-4e8f-b402-0a00dd5755cc None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:47.670357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b885204-e895-4e8f-b402-0a00dd5755cc None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:47.679172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c93fe0-dd3a-420e-9f27-8e2e3e1075e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:47.679861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c93fe0-dd3a-420e-9f27-8e2e3e1075e7 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:47.680148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c93fe0-dd3a-420e-9f27-8e2e3e1075e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:47.680425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c93fe0-dd3a-420e-9f27-8e2e3e1075e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:47.681142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c93fe0-dd3a-420e-9f27-8e2e3e1075e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:47.683353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c93fe0-dd3a-420e-9f27-8e2e3e1075e7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:47.786579 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1819/3627] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:12:47 2022] POST /identity/v3/projects => generated 389 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:12:47.788089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ef9c5-19f0-4d1a-b9e1-61fc743d89bd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:47.800706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382ef9c5-19f0-4d1a-b9e1-61fc743d89bd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:47.923786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244d53d-7239-4d53-914f-b81751509da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:47.924760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244d53d-7239-4d53-914f-b81751509da2 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:47.925219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244d53d-7239-4d53-914f-b81751509da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:47.926372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244d53d-7239-4d53-914f-b81751509da2 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:47.927374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244d53d-7239-4d53-914f-b81751509da2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:47.932292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244d53d-7239-4d53-914f-b81751509da2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:48.085350 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1809/3628] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:12:47 2022] POST /identity/v3/users => generated 295 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:12:48.089688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c93fe0-dd3a-420e-9f27-8e2e3e1075e7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:48.108753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c93fe0-dd3a-420e-9f27-8e2e3e1075e7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:48.122992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bf209-d7d2-48cb-9a23-a8a0a599111d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:48.123944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bf209-d7d2-48cb-9a23-a8a0a599111d admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:48.127055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bf209-d7d2-48cb-9a23-a8a0a599111d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:48.127055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9bf209-d7d2-48cb-9a23-a8a0a599111d admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:48.127055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9bf209-d7d2-48cb-9a23-a8a0a599111d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:48.130586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9bf209-d7d2-48cb-9a23-a8a0a599111d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:48.152785 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1820/3629] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:12:48.156462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244d53d-7239-4d53-914f-b81751509da2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:48.167625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244d53d-7239-4d53-914f-b81751509da2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:48.173905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708990b4-5649-4b5d-9744-abcf8a5b8912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:48.174584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708990b4-5649-4b5d-9744-abcf8a5b8912 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:48.174858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708990b4-5649-4b5d-9744-abcf8a5b8912 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:48.175395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708990b4-5649-4b5d-9744-abcf8a5b8912 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:48.176289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708990b4-5649-4b5d-9744-abcf8a5b8912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:48.178740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708990b4-5649-4b5d-9744-abcf8a5b8912 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:48.192674 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1810/3630] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:12:48.198315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bf209-d7d2-48cb-9a23-a8a0a599111d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:48.216416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9bf209-d7d2-48cb-9a23-a8a0a599111d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:48.239912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3103226-3c58-4128-af7b-acfe61230e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:48.244619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708990b4-5649-4b5d-9744-abcf8a5b8912 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:48.250408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-708990b4-5649-4b5d-9744-abcf8a5b8912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:48.251906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3103226-3c58-4128-af7b-acfe61230e86 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:48.257631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3103226-3c58-4128-af7b-acfe61230e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:48.258384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3103226-3c58-4128-af7b-acfe61230e86 admin admin] PATH_INFO: `/v3/projects/ca680361a7b349e39f77a7f7c96934f8/users/eb7c3a01b54f4db2841f4836635eff2e/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:48.276063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708990b4-5649-4b5d-9744-abcf8a5b8912 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:48.285230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d834dc-df7b-458c-a0fa-ab2b3952a23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:48.287168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d834dc-df7b-458c-a0fa-ab2b3952a23f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:48.287646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d834dc-df7b-458c-a0fa-ab2b3952a23f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:48.289152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d834dc-df7b-458c-a0fa-ab2b3952a23f admin admin] PATH_INFO: `/v3/users/04ae3115a9194b47bce62ada4a2afdf2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:48.291540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d834dc-df7b-458c-a0fa-ab2b3952a23f admin admin] RBAC: Authorizing `identity:delete_user(user_id=04ae3115a9194b47bce62ada4a2afdf2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:48.296264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3103226-3c58-4128-af7b-acfe61230e86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca680361a7b349e39f77a7f7c96934f8, user_id=eb7c3a01b54f4db2841f4836635eff2e, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:48.297757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d834dc-df7b-458c-a0fa-ab2b3952a23f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:48.300659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3103226-3c58-4128-af7b-acfe61230e86 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:48.419838 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1821/3631] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:12:48 2022] PUT /identity/v3/projects/ca680361a7b349e39f77a7f7c96934f8/users/eb7c3a01b54f4db2841f4836635eff2e/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:48.425033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2d185-3c26-466a-9689-08ea8a528d00 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:48.425487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2d185-3c26-466a-9689-08ea8a528d00 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:48.426036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2d185-3c26-466a-9689-08ea8a528d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:48.445710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-00c2d185-3c26-466a-9689-08ea8a528d00 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:48.447855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-00c2d185-3c26-466a-9689-08ea8a528d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:48.572274 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1822/3632] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:48.579540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2d185-3c26-466a-9689-08ea8a528d00 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:48.581129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-00c2d185-3c26-466a-9689-08ea8a528d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:48.612634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a2d834dc-df7b-458c-a0fa-ab2b3952a23f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04ae3115a9194b47bce62ada4a2afdf2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:48.621492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a2d834dc-df7b-458c-a0fa-ab2b3952a23f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04ae3115a9194b47bce62ada4a2afdf2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:48.636176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a2d834dc-df7b-458c-a0fa-ab2b3952a23f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04ae3115a9194b47bce62ada4a2afdf2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:48.685253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2d185-3c26-466a-9689-08ea8a528d00 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:48.695138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a881c6-ff81-4e7c-8662-84e3c6fb67a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:48.695874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a881c6-ff81-4e7c-8662-84e3c6fb67a4 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:48.696158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a881c6-ff81-4e7c-8662-84e3c6fb67a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:48.696436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a881c6-ff81-4e7c-8662-84e3c6fb67a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:48.717781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a881c6-ff81-4e7c-8662-84e3c6fb67a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:48.719372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a881c6-ff81-4e7c-8662-84e3c6fb67a4 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:48.733896 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1823/3633] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:12:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:12:48.754816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a2d834dc-df7b-458c-a0fa-ab2b3952a23f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04ae3115a9194b47bce62ada4a2afdf2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:48.755877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a2d834dc-df7b-458c-a0fa-ab2b3952a23f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04ae3115a9194b47bce62ada4a2afdf2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:48.776877 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1811/3634] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:12:48 2022] DELETE /identity/v3/users/04ae3115a9194b47bce62ada4a2afdf2 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:12:49.489786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a881c6-ff81-4e7c-8662-84e3c6fb67a4 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:49.501180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a881c6-ff81-4e7c-8662-84e3c6fb67a4 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:49.625503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3e05e-105f-49af-8e5f-47d1006d5cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:49.626769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3e05e-105f-49af-8e5f-47d1006d5cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:49.627105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3e05e-105f-49af-8e5f-47d1006d5cab admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:49.629343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3e05e-105f-49af-8e5f-47d1006d5cab admin admin] PATH_INFO: `/v3/projects/9d08be7a2d1e4b779987d75d87a5ef44` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:49.629638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3e05e-105f-49af-8e5f-47d1006d5cab admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d08be7a2d1e4b779987d75d87a5ef44)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:49.631786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3e05e-105f-49af-8e5f-47d1006d5cab admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:49.829717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2cf3e05e-105f-49af-8e5f-47d1006d5cab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d08be7a2d1e4b779987d75d87a5ef44', 'request_id': 'req-2cf3e05e-105f-49af-8e5f-47d1006d5cab'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:49.841224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2cf3e05e-105f-49af-8e5f-47d1006d5cab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d08be7a2d1e4b779987d75d87a5ef44', 'request_id': 'req-2cf3e05e-105f-49af-8e5f-47d1006d5cab'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:49.980398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2cf3e05e-105f-49af-8e5f-47d1006d5cab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d08be7a2d1e4b779987d75d87a5ef44', 'request_id': 'req-2cf3e05e-105f-49af-8e5f-47d1006d5cab'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:50.005874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2cf3e05e-105f-49af-8e5f-47d1006d5cab admin admin] The token cache is being invalidate because project 9d08be7a2d1e4b779987d75d87a5ef44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:12:50.006367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2cf3e05e-105f-49af-8e5f-47d1006d5cab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:50.009238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2cf3e05e-105f-49af-8e5f-47d1006d5cab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:12:50.020159 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1824/3635] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:12:49 2022] DELETE /identity/v3/projects/9d08be7a2d1e4b779987d75d87a5ef44 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:50.030824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f2acf-5187-4511-9309-023ebad0ed4c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:50.031847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f2acf-5187-4511-9309-023ebad0ed4c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:50.032358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f2acf-5187-4511-9309-023ebad0ed4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:50.080361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d17f2acf-5187-4511-9309-023ebad0ed4c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:50.082282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d17f2acf-5187-4511-9309-023ebad0ed4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:50.175658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:50.179628 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1812/3636] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:50.187904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5a23e-9ce2-44e4-96b5-8f4bf7449843 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:50.188193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5a23e-9ce2-44e4-96b5-8f4bf7449843 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:50.188659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5a23e-9ce2-44e4-96b5-8f4bf7449843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:50.235335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f2c5a23e-9ce2-44e4-96b5-8f4bf7449843 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:50.237089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f2c5a23e-9ce2-44e4-96b5-8f4bf7449843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:50.259794 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1825/3637] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:50.260917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f2acf-5187-4511-9309-023ebad0ed4c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:50.274742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17f2acf-5187-4511-9309-023ebad0ed4c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:50.280702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cebc58-5238-4cc5-896c-3fcc0d2eb8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:50.282319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cebc58-5238-4cc5-896c-3fcc0d2eb8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:50.282793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cebc58-5238-4cc5-896c-3fcc0d2eb8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:50.283078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cebc58-5238-4cc5-896c-3fcc0d2eb8a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:50.283874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cebc58-5238-4cc5-896c-3fcc0d2eb8a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:12:50.284190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cebc58-5238-4cc5-896c-3fcc0d2eb8a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:50.285982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cebc58-5238-4cc5-896c-3fcc0d2eb8a4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:50.319555 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1813/3638] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:12:50.321173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5a23e-9ce2-44e4-96b5-8f4bf7449843 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:50.337463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5a23e-9ce2-44e4-96b5-8f4bf7449843 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:50.348197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ac163-d715-471c-9de7-99a5614d819f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:50.349157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ac163-d715-471c-9de7-99a5614d819f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:50.350145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ac163-d715-471c-9de7-99a5614d819f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:50.350608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ac163-d715-471c-9de7-99a5614d819f admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:50.355216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ac163-d715-471c-9de7-99a5614d819f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:50.357441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ac163-d715-471c-9de7-99a5614d819f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:50.472838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2a5e6-6ce8-4255-905b-8c4cfed18ca3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:50.474302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2a5e6-6ce8-4255-905b-8c4cfed18ca3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:50.474598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc2a5e6-6ce8-4255-905b-8c4cfed18ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:50.486199 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1826/3639] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:12:50 2022] POST /identity/v3/projects => generated 381 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:12:50.487449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ac163-d715-471c-9de7-99a5614d819f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:50.518019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:50.520652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1dc2a5e6-6ce8-4255-905b-8c4cfed18ca3 None None] MFA Rules not processed for user `ffb90cfa7828412f96f330dab49466cd`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:50.523354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1dc2a5e6-6ce8-4255-905b-8c4cfed18ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:50.627327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ac163-d715-471c-9de7-99a5614d819f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:50.630757 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1814/3640] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:50.636771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400a293-8c15-41df-b91c-3ad664900964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:50.637603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400a293-8c15-41df-b91c-3ad664900964 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:50.637976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400a293-8c15-41df-b91c-3ad664900964 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:50.638735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2400a293-8c15-41df-b91c-3ad664900964 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:50.639660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400a293-8c15-41df-b91c-3ad664900964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:50.641609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2400a293-8c15-41df-b91c-3ad664900964 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:50.647527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a8526-6234-4e59-ab71-ad93b0784967 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:50.647814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a8526-6234-4e59-ab71-ad93b0784967 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:50.648089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057a8526-6234-4e59-ab71-ad93b0784967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:50.670375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-057a8526-6234-4e59-ab71-ad93b0784967 None None] MFA Rules not processed for user `ffb90cfa7828412f96f330dab49466cd`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:50.671684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-057a8526-6234-4e59-ab71-ad93b0784967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:50.693159 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1815/3641] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:12:50.747916 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1827/3642] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:12:50 2022] POST /identity/v3/users => generated 291 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:12:50.751337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a8526-6234-4e59-ab71-ad93b0784967 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:50.780210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057a8526-6234-4e59-ab71-ad93b0784967 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:50.783367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6febe20-b2cb-484f-a9bc-c128531a6b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:50.788214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6febe20-b2cb-484f-a9bc-c128531a6b15 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:50.788214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6febe20-b2cb-484f-a9bc-c128531a6b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:50.788214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6febe20-b2cb-484f-a9bc-c128531a6b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:50.788214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6febe20-b2cb-484f-a9bc-c128531a6b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:50.788214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6febe20-b2cb-484f-a9bc-c128531a6b15 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:50.808832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400a293-8c15-41df-b91c-3ad664900964 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:50.813123 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1816/3643] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:12:50.821398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6febe20-b2cb-484f-a9bc-c128531a6b15 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:50.824827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2400a293-8c15-41df-b91c-3ad664900964 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:50.826425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e6febe20-b2cb-484f-a9bc-c128531a6b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:50.839067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873ef8c-d371-485d-9601-4afe228af477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:50.841194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873ef8c-d371-485d-9601-4afe228af477 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:50.841868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873ef8c-d371-485d-9601-4afe228af477 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:50.842732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873ef8c-d371-485d-9601-4afe228af477 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:50.848022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873ef8c-d371-485d-9601-4afe228af477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:50.851496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7873ef8c-d371-485d-9601-4afe228af477 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:50.877359 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1828/3644] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:12:50.879367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873ef8c-d371-485d-9601-4afe228af477 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:50.891783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7873ef8c-d371-485d-9601-4afe228af477 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:50.903380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dccdc4-4e20-4555-9436-7aa03c0549dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:50.904472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dccdc4-4e20-4555-9436-7aa03c0549dc admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:50.904766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dccdc4-4e20-4555-9436-7aa03c0549dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:50.905049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dccdc4-4e20-4555-9436-7aa03c0549dc admin admin] PATH_INFO: `/v3/projects/43f6ea47af5945bfada7f59cd1a7da28/users/9e5bb3a0b7c9438bbfdc22fac3f71995/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:50.930473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dccdc4-4e20-4555-9436-7aa03c0549dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=43f6ea47af5945bfada7f59cd1a7da28, user_id=9e5bb3a0b7c9438bbfdc22fac3f71995, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:50.934555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dccdc4-4e20-4555-9436-7aa03c0549dc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:51.018518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6febe20-b2cb-484f-a9bc-c128531a6b15 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:51.034483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e316c-876d-4b04-8206-0af176ac9779 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:51.035151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e316c-876d-4b04-8206-0af176ac9779 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:51.035439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e316c-876d-4b04-8206-0af176ac9779 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:51.047854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1e316c-876d-4b04-8206-0af176ac9779 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:51.048666 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1829/3645] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:12:50 2022] PUT /identity/v3/projects/43f6ea47af5945bfada7f59cd1a7da28/users/9e5bb3a0b7c9438bbfdc22fac3f71995/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:12:51.049655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe6eda-5e4a-4439-ad57-1ef19ddb463a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:51.050771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe6eda-5e4a-4439-ad57-1ef19ddb463a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:51.051128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe6eda-5e4a-4439-ad57-1ef19ddb463a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:51.066819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e316c-876d-4b04-8206-0af176ac9779 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:51.079381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1e316c-876d-4b04-8206-0af176ac9779 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:51.100927 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1817/3646] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:12:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:12:51.157483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-aafe6eda-5e4a-4439-ad57-1ef19ddb463a None None] MFA Rules not processed for user `9e5bb3a0b7c9438bbfdc22fac3f71995`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:51.170417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-aafe6eda-5e4a-4439-ad57-1ef19ddb463a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:51.324513 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1830/3647] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:12:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:51.345211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e316c-876d-4b04-8206-0af176ac9779 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:51.352554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cf1e316c-876d-4b04-8206-0af176ac9779 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:51.513987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1e316c-876d-4b04-8206-0af176ac9779 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:51.643439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185145b2-eb5e-464a-b6e2-b1aaf50c27f9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:51.644781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185145b2-eb5e-464a-b6e2-b1aaf50c27f9 service glance] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:51.645081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185145b2-eb5e-464a-b6e2-b1aaf50c27f9 service glance] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:51.647946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185145b2-eb5e-464a-b6e2-b1aaf50c27f9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:51.667856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185145b2-eb5e-464a-b6e2-b1aaf50c27f9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:51.669353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185145b2-eb5e-464a-b6e2-b1aaf50c27f9 service glance] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:51.698445 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1818/3648] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:12:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:12:52.031919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a1377-8906-4a93-854e-bc536b7f90e2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:52.032949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a1377-8906-4a93-854e-bc536b7f90e2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:52.033483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a1377-8906-4a93-854e-bc536b7f90e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:52.065298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-672a1377-8906-4a93-854e-bc536b7f90e2 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:12:52.067129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-672a1377-8906-4a93-854e-bc536b7f90e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:52.160995 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1831/3649] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:12:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:12:52.251648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:52.334633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185145b2-eb5e-464a-b6e2-b1aaf50c27f9 service glance] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:12:52.339317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-185145b2-eb5e-464a-b6e2-b1aaf50c27f9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:12:52.376308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:52.457211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185145b2-eb5e-464a-b6e2-b1aaf50c27f9 service glance] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:12:52.481216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999fea8-acba-4ba0-b118-aa102e15d061 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:12:52.482510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999fea8-acba-4ba0-b118-aa102e15d061 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:12:52.483009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999fea8-acba-4ba0-b118-aa102e15d061 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:12:52.483516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999fea8-acba-4ba0-b118-aa102e15d061 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:12:52.507567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999fea8-acba-4ba0-b118-aa102e15d061 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:12:52.518872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e999fea8-acba-4ba0-b118-aa102e15d061 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:12:52.550208 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1819/3650] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:12:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:12:54.777682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:54.913822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:56.216824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:58.137512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:00.040455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ae52d-4c28-4dd1-af03-edfd6e9e82b9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:00.041034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ae52d-4c28-4dd1-af03-edfd6e9e82b9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:00.041034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1ae52d-4c28-4dd1-af03-edfd6e9e82b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:00.097016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0f1ae52d-4c28-4dd1-af03-edfd6e9e82b9 None None] MFA Rules not processed for user `25b2f1524f7f48149cbc5d2c694a9e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:00.098700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0f1ae52d-4c28-4dd1-af03-edfd6e9e82b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:00.175041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:00.208858 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1832/3651] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:00.951614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999fea8-acba-4ba0-b118-aa102e15d061 service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:00.953661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e999fea8-acba-4ba0-b118-aa102e15d061 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:01.173776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999fea8-acba-4ba0-b118-aa102e15d061 service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:01.178435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c60f4-a530-4a55-b418-606e57107012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:01.181902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c60f4-a530-4a55-b418-606e57107012 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:01.181902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c60f4-a530-4a55-b418-606e57107012 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:01.181902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c60f4-a530-4a55-b418-606e57107012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:01.183265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-825c60f4-a530-4a55-b418-606e57107012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:01.319127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c60f4-a530-4a55-b418-606e57107012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:01.324542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c60f4-a530-4a55-b418-606e57107012 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:01.345719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:01.351991 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1820/3652] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:13:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:13:02.941442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd02ab-d68e-4a0c-8215-2ffcc4aa0b57 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:02.942996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd02ab-d68e-4a0c-8215-2ffcc4aa0b57 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:02.943441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd02ab-d68e-4a0c-8215-2ffcc4aa0b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:02.954874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c60f4-a530-4a55-b418-606e57107012 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:02.971450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c60f4-a530-4a55-b418-606e57107012 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:02.979265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c91d-799b-4725-837a-9d1a32f1b009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:02.979950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c91d-799b-4725-837a-9d1a32f1b009 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:02.982079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c91d-799b-4725-837a-9d1a32f1b009 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:02.983779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c91d-799b-4725-837a-9d1a32f1b009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:02.988950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f064c91d-799b-4725-837a-9d1a32f1b009 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:03.021481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-aebd02ab-d68e-4a0c-8215-2ffcc4aa0b57 None None] MFA Rules not processed for user `eb7c3a01b54f4db2841f4836635eff2e`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:03.024468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-aebd02ab-d68e-4a0c-8215-2ffcc4aa0b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:03.027894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064c91d-799b-4725-837a-9d1a32f1b009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:03.030184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064c91d-799b-4725-837a-9d1a32f1b009 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:03.071653 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1821/3653] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:13:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:13:03.224191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c91d-799b-4725-837a-9d1a32f1b009 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:03.225636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f064c91d-799b-4725-837a-9d1a32f1b009 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:03.233076 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1833/3654] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:03.407164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c91d-799b-4725-837a-9d1a32f1b009 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:03.421899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e1bd8-7825-4d60-93c5-0ba190d8d383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:03.422961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e1bd8-7825-4d60-93c5-0ba190d8d383 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:03.423446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e1bd8-7825-4d60-93c5-0ba190d8d383 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:03.423874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e1bd8-7825-4d60-93c5-0ba190d8d383 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:03.427119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e1bd8-7825-4d60-93c5-0ba190d8d383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:03.429478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e1bd8-7825-4d60-93c5-0ba190d8d383 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:03.576330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd02ab-d68e-4a0c-8215-2ffcc4aa0b57 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:03.586346 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1822/3655] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:13:03 2022] POST /identity/v3/projects => generated 389 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:13:03.590321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd02ab-d68e-4a0c-8215-2ffcc4aa0b57 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:03.723119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563c220-2f9a-46d8-8049-2c1d93207ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:03.724006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563c220-2f9a-46d8-8049-2c1d93207ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:03.724696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563c220-2f9a-46d8-8049-2c1d93207ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:03.724988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563c220-2f9a-46d8-8049-2c1d93207ed2 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:03.726141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563c220-2f9a-46d8-8049-2c1d93207ed2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:03.727848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563c220-2f9a-46d8-8049-2c1d93207ed2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:03.811599 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1834/3656] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:13:03 2022] POST /identity/v3/users => generated 301 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:13:03.817934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e1bd8-7825-4d60-93c5-0ba190d8d383 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:03.831120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e1bd8-7825-4d60-93c5-0ba190d8d383 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:03.838316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed1168-e4e8-4c93-812f-083b44c76684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:03.839896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed1168-e4e8-4c93-812f-083b44c76684 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:03.839896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed1168-e4e8-4c93-812f-083b44c76684 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:03.857789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed1168-e4e8-4c93-812f-083b44c76684 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:03.863087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed1168-e4e8-4c93-812f-083b44c76684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:03.863087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed1168-e4e8-4c93-812f-083b44c76684 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:03.893246 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1823/3657] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:13:03.897658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563c220-2f9a-46d8-8049-2c1d93207ed2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:03.926208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563c220-2f9a-46d8-8049-2c1d93207ed2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:03.935086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0a005-6462-4652-b8a9-0986a251eed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:03.936180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0a005-6462-4652-b8a9-0986a251eed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:03.936576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0a005-6462-4652-b8a9-0986a251eed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:03.936961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0a005-6462-4652-b8a9-0986a251eed1 admin admin] PATH_INFO: `/v3/projects/2911813847a3410c893dff40021c3099/users/a729b39321f642ae93c55019dbf07db9/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:03.967090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0a005-6462-4652-b8a9-0986a251eed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2911813847a3410c893dff40021c3099, user_id=a729b39321f642ae93c55019dbf07db9, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:03.971946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0a005-6462-4652-b8a9-0986a251eed1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:04.102117 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1835/3658] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:13:03 2022] PUT /identity/v3/projects/2911813847a3410c893dff40021c3099/users/a729b39321f642ae93c55019dbf07db9/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:13:04.104669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed1168-e4e8-4c93-812f-083b44c76684 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:04.144462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed1168-e4e8-4c93-812f-083b44c76684 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:04.265428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4bd32b-118a-49e6-a7f7-32fcd4a7ced0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:04.266423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bd32b-118a-49e6-a7f7-32fcd4a7ced0 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:04.278354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bd32b-118a-49e6-a7f7-32fcd4a7ced0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:04.278354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4bd32b-118a-49e6-a7f7-32fcd4a7ced0 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:04.278354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4bd32b-118a-49e6-a7f7-32fcd4a7ced0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:04.278354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4bd32b-118a-49e6-a7f7-32fcd4a7ced0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:04.289952 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1824/3659] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:13:04.293891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0a005-6462-4652-b8a9-0986a251eed1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:04.314489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0a005-6462-4652-b8a9-0986a251eed1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:04.339195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceed15f-e3ee-486a-8aa2-0376a3d0ca9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:04.347635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceed15f-e3ee-486a-8aa2-0376a3d0ca9c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:04.347960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceed15f-e3ee-486a-8aa2-0376a3d0ca9c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:04.348239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceed15f-e3ee-486a-8aa2-0376a3d0ca9c admin admin] PATH_INFO: `/v3/projects/2911813847a3410c893dff40021c3099/users/a729b39321f642ae93c55019dbf07db9/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:04.352084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceed15f-e3ee-486a-8aa2-0376a3d0ca9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2911813847a3410c893dff40021c3099, user_id=a729b39321f642ae93c55019dbf07db9, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:04.358264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceed15f-e3ee-486a-8aa2-0376a3d0ca9c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:04.377547 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1836/3660] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:13:04 2022] PUT /identity/v3/projects/2911813847a3410c893dff40021c3099/users/a729b39321f642ae93c55019dbf07db9/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:13:06.193628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38b735-bea2-4f3a-b6c2-c7719b3e34c0 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:06.196189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38b735-bea2-4f3a-b6c2-c7719b3e34c0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:06.196387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38b735-bea2-4f3a-b6c2-c7719b3e34c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:06.224365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bd38b735-bea2-4f3a-b6c2-c7719b3e34c0 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:06.226308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bd38b735-bea2-4f3a-b6c2-c7719b3e34c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:06.311164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:06.341358 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1825/3661] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:13:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:06.349679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceed15f-e3ee-486a-8aa2-0376a3d0ca9c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:06.398385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:06.479035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceed15f-e3ee-486a-8aa2-0376a3d0ca9c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:06.485271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20446883-45e0-4347-a315-08d6389f4b9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:06.486306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20446883-45e0-4347-a315-08d6389f4b9d service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:06.487084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20446883-45e0-4347-a315-08d6389f4b9d service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:06.487713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20446883-45e0-4347-a315-08d6389f4b9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:06.498389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20446883-45e0-4347-a315-08d6389f4b9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:06.502431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20446883-45e0-4347-a315-08d6389f4b9d service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:06.513767 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1837/3662] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:13:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:13:07.238572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38b735-bea2-4f3a-b6c2-c7719b3e34c0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:07.392446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38b735-bea2-4f3a-b6c2-c7719b3e34c0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:07.409228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfa0e3-2619-4528-b2a3-6a836a66538d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:07.410808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfa0e3-2619-4528-b2a3-6a836a66538d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:07.411454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfa0e3-2619-4528-b2a3-6a836a66538d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:07.411912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfa0e3-2619-4528-b2a3-6a836a66538d admin admin] PATH_INFO: `/v3/users/9e5bb3a0b7c9438bbfdc22fac3f71995` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:07.414184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dfa0e3-2619-4528-b2a3-6a836a66538d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e5bb3a0b7c9438bbfdc22fac3f71995)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:07.417524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dfa0e3-2619-4528-b2a3-6a836a66538d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:07.684294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c8dfa0e3-2619-4528-b2a3-6a836a66538d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e5bb3a0b7c9438bbfdc22fac3f71995'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:07.685212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c8dfa0e3-2619-4528-b2a3-6a836a66538d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e5bb3a0b7c9438bbfdc22fac3f71995'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:07.693047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c8dfa0e3-2619-4528-b2a3-6a836a66538d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e5bb3a0b7c9438bbfdc22fac3f71995'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:07.758372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c8dfa0e3-2619-4528-b2a3-6a836a66538d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e5bb3a0b7c9438bbfdc22fac3f71995'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:07.759049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c8dfa0e3-2619-4528-b2a3-6a836a66538d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e5bb3a0b7c9438bbfdc22fac3f71995'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:07.789362 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1826/3663] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:13:07 2022] DELETE /identity/v3/users/9e5bb3a0b7c9438bbfdc22fac3f71995 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:13:07.794936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4641d03-2978-4179-80a7-8ea742ffeae8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:07.795845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4641d03-2978-4179-80a7-8ea742ffeae8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:07.796138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4641d03-2978-4179-80a7-8ea742ffeae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:07.823652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a4641d03-2978-4179-80a7-8ea742ffeae8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:07.825071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a4641d03-2978-4179-80a7-8ea742ffeae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:07.918418 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1838/3664] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:07.936187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfa0e3-2619-4528-b2a3-6a836a66538d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:07.937639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c8dfa0e3-2619-4528-b2a3-6a836a66538d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:08.021046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfa0e3-2619-4528-b2a3-6a836a66538d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:08.027110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3512bf-83b9-415a-b7f4-1348eb932b4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:08.027776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3512bf-83b9-415a-b7f4-1348eb932b4c service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:08.028064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3512bf-83b9-415a-b7f4-1348eb932b4c service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:08.028368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3512bf-83b9-415a-b7f4-1348eb932b4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:08.041556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3512bf-83b9-415a-b7f4-1348eb932b4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:08.043170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3512bf-83b9-415a-b7f4-1348eb932b4c service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:08.054801 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1827/3665] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:13:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:13:08.820501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4641d03-2978-4179-80a7-8ea742ffeae8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:08.823299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a4641d03-2978-4179-80a7-8ea742ffeae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:08.847244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4641d03-2978-4179-80a7-8ea742ffeae8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:08.858231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e9699-9906-41cb-975e-4b20ecee119e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:08.859157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e9699-9906-41cb-975e-4b20ecee119e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:08.859444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e9699-9906-41cb-975e-4b20ecee119e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:08.861929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e9699-9906-41cb-975e-4b20ecee119e admin admin] PATH_INFO: `/v3/projects/43f6ea47af5945bfada7f59cd1a7da28` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:08.864245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e9699-9906-41cb-975e-4b20ecee119e admin admin] RBAC: Authorizing `identity:delete_project(project_id=43f6ea47af5945bfada7f59cd1a7da28)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:08.869762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e9699-9906-41cb-975e-4b20ecee119e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:09.030824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bd3e9699-9906-41cb-975e-4b20ecee119e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43f6ea47af5945bfada7f59cd1a7da28', 'request_id': 'req-bd3e9699-9906-41cb-975e-4b20ecee119e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:09.039898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bd3e9699-9906-41cb-975e-4b20ecee119e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43f6ea47af5945bfada7f59cd1a7da28', 'request_id': 'req-bd3e9699-9906-41cb-975e-4b20ecee119e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:09.127333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bd3e9699-9906-41cb-975e-4b20ecee119e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43f6ea47af5945bfada7f59cd1a7da28', 'request_id': 'req-bd3e9699-9906-41cb-975e-4b20ecee119e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:09.144877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bd3e9699-9906-41cb-975e-4b20ecee119e admin admin] The token cache is being invalidate because project 43f6ea47af5945bfada7f59cd1a7da28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:13:09.145178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bd3e9699-9906-41cb-975e-4b20ecee119e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:09.146500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bd3e9699-9906-41cb-975e-4b20ecee119e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:09.155163 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1839/3666] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:13:08 2022] DELETE /identity/v3/projects/43f6ea47af5945bfada7f59cd1a7da28 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:13:09.158214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f2950-b873-4f35-b0ba-6b1e2414fa66 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:09.158669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f2950-b873-4f35-b0ba-6b1e2414fa66 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:09.158956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928f2950-b873-4f35-b0ba-6b1e2414fa66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:09.197955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-928f2950-b873-4f35-b0ba-6b1e2414fa66 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:09.202273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-928f2950-b873-4f35-b0ba-6b1e2414fa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:09.237244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:09.302555 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1828/3667] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:09.312620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c096c-b34e-434f-83b3-5a605d008d5a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:09.313119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c096c-b34e-434f-83b3-5a605d008d5a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:09.313499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c096c-b34e-434f-83b3-5a605d008d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:09.335144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ad0c096c-b34e-434f-83b3-5a605d008d5a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:09.336371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ad0c096c-b34e-434f-83b3-5a605d008d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:09.350996 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1840/3668] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:13:09.356132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f2950-b873-4f35-b0ba-6b1e2414fa66 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:09.366749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928f2950-b873-4f35-b0ba-6b1e2414fa66 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:09.371893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ec5e0-5a68-4fca-8686-035890230790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:09.372568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ec5e0-5a68-4fca-8686-035890230790 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:09.372864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ec5e0-5a68-4fca-8686-035890230790 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:09.373149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ec5e0-5a68-4fca-8686-035890230790 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:09.374403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ec5e0-5a68-4fca-8686-035890230790 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:13:09.379782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ec5e0-5a68-4fca-8686-035890230790 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:09.379782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ec5e0-5a68-4fca-8686-035890230790 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:09.392322 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1829/3669] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:13:09.393535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c096c-b34e-434f-83b3-5a605d008d5a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:09.403755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c096c-b34e-434f-83b3-5a605d008d5a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:09.408634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59b535-e87f-46f8-bd2f-7150498734d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:09.409191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59b535-e87f-46f8-bd2f-7150498734d2 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:09.409477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59b535-e87f-46f8-bd2f-7150498734d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:09.409768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59b535-e87f-46f8-bd2f-7150498734d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:09.410532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59b535-e87f-46f8-bd2f-7150498734d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:09.412491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59b535-e87f-46f8-bd2f-7150498734d2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:09.497709 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1841/3670] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:13:09 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:13:09.505037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ec5e0-5a68-4fca-8686-035890230790 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:09.526830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ec5e0-5a68-4fca-8686-035890230790 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:09.615132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe29c0-c83d-4ae0-81c0-e3548ef8a8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:09.616066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe29c0-c83d-4ae0-81c0-e3548ef8a8fc admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:09.616488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe29c0-c83d-4ae0-81c0-e3548ef8a8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:09.616912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe29c0-c83d-4ae0-81c0-e3548ef8a8fc admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:09.618338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe29c0-c83d-4ae0-81c0-e3548ef8a8fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:09.621588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe29c0-c83d-4ae0-81c0-e3548ef8a8fc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:09.772955 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1830/3671] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:13:09 2022] POST /identity/v3/users => generated 290 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:13:09.774463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59b535-e87f-46f8-bd2f-7150498734d2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:09.794386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59b535-e87f-46f8-bd2f-7150498734d2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:09.811946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0170a93-6e74-4d4e-bd03-7535df052f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:09.812772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0170a93-6e74-4d4e-bd03-7535df052f07 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:09.813061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0170a93-6e74-4d4e-bd03-7535df052f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:09.813346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0170a93-6e74-4d4e-bd03-7535df052f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:09.814139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0170a93-6e74-4d4e-bd03-7535df052f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:09.816770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0170a93-6e74-4d4e-bd03-7535df052f07 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:09.840365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:09.856264 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1842/3672] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:13:09.859763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe29c0-c83d-4ae0-81c0-e3548ef8a8fc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:09.874970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe29c0-c83d-4ae0-81c0-e3548ef8a8fc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:09.891617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01980f8b-0202-40cc-b23c-3a5e504c3547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:09.892471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01980f8b-0202-40cc-b23c-3a5e504c3547 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:09.893131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01980f8b-0202-40cc-b23c-3a5e504c3547 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:09.893556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01980f8b-0202-40cc-b23c-3a5e504c3547 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:09.897773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01980f8b-0202-40cc-b23c-3a5e504c3547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:09.900670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01980f8b-0202-40cc-b23c-3a5e504c3547 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:09.940891 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1831/3673] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:13:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:13:09.944357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0170a93-6e74-4d4e-bd03-7535df052f07 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:09.961810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0170a93-6e74-4d4e-bd03-7535df052f07 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:09.974568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7ddaa-a2b6-4068-8ebd-8597d8c837bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:09.976535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7ddaa-a2b6-4068-8ebd-8597d8c837bb admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:09.976825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7ddaa-a2b6-4068-8ebd-8597d8c837bb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:09.977108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7ddaa-a2b6-4068-8ebd-8597d8c837bb admin admin] PATH_INFO: `/v3/projects/149d4937782d47f1b75fcdcfb4a89707/users/cf4bbfcd9bf64e3eabac163461e98f66/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:09.997395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7ddaa-a2b6-4068-8ebd-8597d8c837bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=149d4937782d47f1b75fcdcfb4a89707, user_id=cf4bbfcd9bf64e3eabac163461e98f66, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:10.000632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af7ddaa-a2b6-4068-8ebd-8597d8c837bb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:10.113215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7bbb0-5b33-4f7b-ada7-cc9e42e375ff None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:10.113516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7bbb0-5b33-4f7b-ada7-cc9e42e375ff None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:10.113823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7bbb0-5b33-4f7b-ada7-cc9e42e375ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:10.123783 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1843/3674] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:13:09 2022] PUT /identity/v3/projects/149d4937782d47f1b75fcdcfb4a89707/users/cf4bbfcd9bf64e3eabac163461e98f66/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:13:10.134979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131509b-8730-45d5-8c0a-b01ef874017b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:10.135573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131509b-8730-45d5-8c0a-b01ef874017b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:10.136274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131509b-8730-45d5-8c0a-b01ef874017b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:10.156712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-72a7bbb0-5b33-4f7b-ada7-cc9e42e375ff None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:10.158089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-72a7bbb0-5b33-4f7b-ada7-cc9e42e375ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:10.176975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9131509b-8730-45d5-8c0a-b01ef874017b None None] MFA Rules not processed for user `cf4bbfcd9bf64e3eabac163461e98f66`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:10.182241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9131509b-8730-45d5-8c0a-b01ef874017b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:10.270150 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1844/3675] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:13:10.275297 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1832/3676] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:13:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:13:10.281251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d9e9f-3c9e-484d-9227-2edf5450a3a0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:10.283920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d9e9f-3c9e-484d-9227-2edf5450a3a0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:10.296814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d9e9f-3c9e-484d-9227-2edf5450a3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:10.296966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7bbb0-5b33-4f7b-ada7-cc9e42e375ff None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:10.298417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-72a7bbb0-5b33-4f7b-ada7-cc9e42e375ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:10.338038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-891d9e9f-3c9e-484d-9227-2edf5450a3a0 None None] MFA Rules not processed for user `cf4bbfcd9bf64e3eabac163461e98f66`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:10.342398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-891d9e9f-3c9e-484d-9227-2edf5450a3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:10.372443 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1845/3677] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:13:10.448487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7bbb0-5b33-4f7b-ada7-cc9e42e375ff None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:10.455637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907343e0-99dd-4142-b9e5-6b45eacfb5cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:10.456497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907343e0-99dd-4142-b9e5-6b45eacfb5cc service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:10.456937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907343e0-99dd-4142-b9e5-6b45eacfb5cc service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:10.457364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907343e0-99dd-4142-b9e5-6b45eacfb5cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:10.470946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907343e0-99dd-4142-b9e5-6b45eacfb5cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:10.472991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907343e0-99dd-4142-b9e5-6b45eacfb5cc service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:10.487325 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1833/3678] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:13:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:13:10.748526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d9e9f-3c9e-484d-9227-2edf5450a3a0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:10.749129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-891d9e9f-3c9e-484d-9227-2edf5450a3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:10.799690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:10.912760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d9e9f-3c9e-484d-9227-2edf5450a3a0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:10.942264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49bc95-9272-4097-b91e-ecea96d21287 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:10.947489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49bc95-9272-4097-b91e-ecea96d21287 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:10.947489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49bc95-9272-4097-b91e-ecea96d21287 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:10.947489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49bc95-9272-4097-b91e-ecea96d21287 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:10.994045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49bc95-9272-4097-b91e-ecea96d21287 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:10.994045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49bc95-9272-4097-b91e-ecea96d21287 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:11.045004 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1846/3679] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:13:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:13:11.198765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907343e0-99dd-4142-b9e5-6b45eacfb5cc service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:11.200347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-907343e0-99dd-4142-b9e5-6b45eacfb5cc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:11.292283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49bc95-9272-4097-b91e-ecea96d21287 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:11.294292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7d49bc95-9272-4097-b91e-ecea96d21287 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:11.348803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907343e0-99dd-4142-b9e5-6b45eacfb5cc service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:11.359310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddbfd9-240b-451e-904e-5c2d32207f92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:11.360273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddbfd9-240b-451e-904e-5c2d32207f92 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:11.360564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddbfd9-240b-451e-904e-5c2d32207f92 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:11.360843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddbfd9-240b-451e-904e-5c2d32207f92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:11.364390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-32ddbfd9-240b-451e-904e-5c2d32207f92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:11.399421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddbfd9-240b-451e-904e-5c2d32207f92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:11.401457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddbfd9-240b-451e-904e-5c2d32207f92 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:11.425075 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1834/3680] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:13:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:13:11.439222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:11.454162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49bc95-9272-4097-b91e-ecea96d21287 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:11.458241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e6917-5719-4097-bb91-0ba372a49d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:11.458916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e6917-5719-4097-bb91-0ba372a49d50 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:11.459182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e6917-5719-4097-bb91-0ba372a49d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:11.459461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8e6917-5719-4097-bb91-0ba372a49d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:11.469381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e6917-5719-4097-bb91-0ba372a49d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:11.471093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8e6917-5719-4097-bb91-0ba372a49d50 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:11.485391 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1847/3681] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:13:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:13:11.890423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddbfd9-240b-451e-904e-5c2d32207f92 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:11.892447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-32ddbfd9-240b-451e-904e-5c2d32207f92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:12.085206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddbfd9-240b-451e-904e-5c2d32207f92 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:12.096189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66321195-9407-4fc8-8876-7f634c6eb78c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:12.097298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66321195-9407-4fc8-8876-7f634c6eb78c service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:12.097711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66321195-9407-4fc8-8876-7f634c6eb78c service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:12.098139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66321195-9407-4fc8-8876-7f634c6eb78c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:12.100889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-66321195-9407-4fc8-8876-7f634c6eb78c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:12.120591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66321195-9407-4fc8-8876-7f634c6eb78c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:12.123567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66321195-9407-4fc8-8876-7f634c6eb78c service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:12.150762 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1835/3682] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:13:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:13:18.532755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e6917-5719-4097-bb91-0ba372a49d50 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:18.556256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8e6917-5719-4097-bb91-0ba372a49d50 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:18.565212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2daf0f4-fdd3-46f9-80fa-bf8aadfcdfe7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:18.573867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2daf0f4-fdd3-46f9-80fa-bf8aadfcdfe7 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:18.573867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2daf0f4-fdd3-46f9-80fa-bf8aadfcdfe7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:18.573867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2daf0f4-fdd3-46f9-80fa-bf8aadfcdfe7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:18.575580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f2daf0f4-fdd3-46f9-80fa-bf8aadfcdfe7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:18.604041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2daf0f4-fdd3-46f9-80fa-bf8aadfcdfe7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:18.609002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2daf0f4-fdd3-46f9-80fa-bf8aadfcdfe7 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:18.638337 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1848/3683] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:13:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:13:19.212832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702f1f1-9f4d-487d-8ecd-10c8c59e9241 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:19.212832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702f1f1-9f4d-487d-8ecd-10c8c59e9241 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:19.212832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702f1f1-9f4d-487d-8ecd-10c8c59e9241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:19.305335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b702f1f1-9f4d-487d-8ecd-10c8c59e9241 None None] MFA Rules not processed for user `a729b39321f642ae93c55019dbf07db9`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:19.309022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b702f1f1-9f4d-487d-8ecd-10c8c59e9241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:19.352425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:19.457652 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1836/3684] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:19 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:19.469165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2daf0f4-fdd3-46f9-80fa-bf8aadfcdfe7 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:19.471546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f2daf0f4-fdd3-46f9-80fa-bf8aadfcdfe7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:19.598382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2daf0f4-fdd3-46f9-80fa-bf8aadfcdfe7 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:19.604485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bed94-2053-4f27-82e3-b5e024f7ae08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:19.605388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bed94-2053-4f27-82e3-b5e024f7ae08 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:19.605955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bed94-2053-4f27-82e3-b5e024f7ae08 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:19.606419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bed94-2053-4f27-82e3-b5e024f7ae08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:19.608608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-038bed94-2053-4f27-82e3-b5e024f7ae08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:19.718759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bed94-2053-4f27-82e3-b5e024f7ae08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:19.721337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038bed94-2053-4f27-82e3-b5e024f7ae08 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:19.742328 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1849/3685] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:13:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:13:21.257357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702f1f1-9f4d-487d-8ecd-10c8c59e9241 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:21.259987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b702f1f1-9f4d-487d-8ecd-10c8c59e9241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:21.308021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702f1f1-9f4d-487d-8ecd-10c8c59e9241 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:21.326091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40d9f2-b794-44c8-85ec-08fb50e4fab1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:21.329549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40d9f2-b794-44c8-85ec-08fb50e4fab1 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:21.329549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40d9f2-b794-44c8-85ec-08fb50e4fab1 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:21.329549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40d9f2-b794-44c8-85ec-08fb50e4fab1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:21.332799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fd40d9f2-b794-44c8-85ec-08fb50e4fab1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:21.375854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40d9f2-b794-44c8-85ec-08fb50e4fab1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:21.377746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40d9f2-b794-44c8-85ec-08fb50e4fab1 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:21.394560 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1837/3686] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:13:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:13:26.903019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bed94-2053-4f27-82e3-b5e024f7ae08 service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:26.906371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-038bed94-2053-4f27-82e3-b5e024f7ae08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:27.112406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bed94-2053-4f27-82e3-b5e024f7ae08 service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:27.132034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af96869-26bb-45e7-93aa-7e5f493d32c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:27.132034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af96869-26bb-45e7-93aa-7e5f493d32c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:27.132322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af96869-26bb-45e7-93aa-7e5f493d32c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:27.132544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af96869-26bb-45e7-93aa-7e5f493d32c8 admin admin] PATH_INFO: `/v3/users/752fbb20c21449bb80bc9de2da474865` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:27.135044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af96869-26bb-45e7-93aa-7e5f493d32c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=752fbb20c21449bb80bc9de2da474865)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:27.137203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af96869-26bb-45e7-93aa-7e5f493d32c8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:27.371341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:27.421045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8af96869-26bb-45e7-93aa-7e5f493d32c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '752fbb20c21449bb80bc9de2da474865'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:27.437112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8af96869-26bb-45e7-93aa-7e5f493d32c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '752fbb20c21449bb80bc9de2da474865'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:27.438633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8af96869-26bb-45e7-93aa-7e5f493d32c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '752fbb20c21449bb80bc9de2da474865'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:27.440986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8af96869-26bb-45e7-93aa-7e5f493d32c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '752fbb20c21449bb80bc9de2da474865'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:27.533042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8af96869-26bb-45e7-93aa-7e5f493d32c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '752fbb20c21449bb80bc9de2da474865'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:27.580403 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1850/3687] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:13:26 2022] DELETE /identity/v3/users/752fbb20c21449bb80bc9de2da474865 => generated 0 bytes in 678 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:13:27.585758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b63e5d-b24b-4e27-8088-4fa01c938425 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:27.586295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b63e5d-b24b-4e27-8088-4fa01c938425 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:27.586377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b63e5d-b24b-4e27-8088-4fa01c938425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:27.630708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-72b63e5d-b24b-4e27-8088-4fa01c938425 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:27.630708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-72b63e5d-b24b-4e27-8088-4fa01c938425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:27.904620 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1838/3688] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:27.921094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af96869-26bb-45e7-93aa-7e5f493d32c8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:27.922698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8af96869-26bb-45e7-93aa-7e5f493d32c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:27.989505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af96869-26bb-45e7-93aa-7e5f493d32c8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:27.994283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84f35d0-7749-4746-aac8-60fc3a385fc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:27.995028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f35d0-7749-4746-aac8-60fc3a385fc7 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:27.995380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f35d0-7749-4746-aac8-60fc3a385fc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:27.995905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84f35d0-7749-4746-aac8-60fc3a385fc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:28.007148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f35d0-7749-4746-aac8-60fc3a385fc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:28.010172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84f35d0-7749-4746-aac8-60fc3a385fc7 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:28.040903 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1851/3689] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:13:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:13:28.806296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b63e5d-b24b-4e27-8088-4fa01c938425 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:28.819088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-72b63e5d-b24b-4e27-8088-4fa01c938425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:28.824941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b63e5d-b24b-4e27-8088-4fa01c938425 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:28.840188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac65e72-d43c-4d88-b4fb-49a8f385b611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:28.840188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac65e72-d43c-4d88-b4fb-49a8f385b611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:28.840188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac65e72-d43c-4d88-b4fb-49a8f385b611 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:28.840188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac65e72-d43c-4d88-b4fb-49a8f385b611 admin admin] PATH_INFO: `/v3/projects/d64af7d364a74dc6b9c527532a5d8abd` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:28.840721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac65e72-d43c-4d88-b4fb-49a8f385b611 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d64af7d364a74dc6b9c527532a5d8abd)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:28.840721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac65e72-d43c-4d88-b4fb-49a8f385b611 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:28.984381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9ac65e72-d43c-4d88-b4fb-49a8f385b611 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd64af7d364a74dc6b9c527532a5d8abd', 'request_id': 'req-9ac65e72-d43c-4d88-b4fb-49a8f385b611'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:28.991136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9ac65e72-d43c-4d88-b4fb-49a8f385b611 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd64af7d364a74dc6b9c527532a5d8abd', 'request_id': 'req-9ac65e72-d43c-4d88-b4fb-49a8f385b611'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:29.089898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9ac65e72-d43c-4d88-b4fb-49a8f385b611 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd64af7d364a74dc6b9c527532a5d8abd', 'request_id': 'req-9ac65e72-d43c-4d88-b4fb-49a8f385b611'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:29.112509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9ac65e72-d43c-4d88-b4fb-49a8f385b611 admin admin] The token cache is being invalidate because project d64af7d364a74dc6b9c527532a5d8abd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:13:29.112808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9ac65e72-d43c-4d88-b4fb-49a8f385b611 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:29.113614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9ac65e72-d43c-4d88-b4fb-49a8f385b611 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:29.115878 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1839/3690] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:13:28 2022] DELETE /identity/v3/projects/d64af7d364a74dc6b9c527532a5d8abd => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:13:29.121931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527bb33-1db8-45ca-a291-08226dbb7ae4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:29.122831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527bb33-1db8-45ca-a291-08226dbb7ae4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:29.123367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4527bb33-1db8-45ca-a291-08226dbb7ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:29.144095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4527bb33-1db8-45ca-a291-08226dbb7ae4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:29.146040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4527bb33-1db8-45ca-a291-08226dbb7ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:29.214583 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1852/3691] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:29.222522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85643e2a-e91c-42fe-985f-05b74546e7f8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:29.222820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85643e2a-e91c-42fe-985f-05b74546e7f8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:29.223115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85643e2a-e91c-42fe-985f-05b74546e7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:29.244228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-85643e2a-e91c-42fe-985f-05b74546e7f8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:29.245585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-85643e2a-e91c-42fe-985f-05b74546e7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:29.254232 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1840/3692] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:29.259186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527bb33-1db8-45ca-a291-08226dbb7ae4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:29.270599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4527bb33-1db8-45ca-a291-08226dbb7ae4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:29.276136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca6ac7-8dd0-468c-ab52-e1a4a0cacf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:29.277145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca6ac7-8dd0-468c-ab52-e1a4a0cacf5d admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:29.277700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca6ac7-8dd0-468c-ab52-e1a4a0cacf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:29.278333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca6ac7-8dd0-468c-ab52-e1a4a0cacf5d admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:29.279261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca6ac7-8dd0-468c-ab52-e1a4a0cacf5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:13:29.279578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca6ac7-8dd0-468c-ab52-e1a4a0cacf5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:29.287010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca6ac7-8dd0-468c-ab52-e1a4a0cacf5d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:29.297439 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1853/3693] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:13:29.299338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85643e2a-e91c-42fe-985f-05b74546e7f8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:29.309595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85643e2a-e91c-42fe-985f-05b74546e7f8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:29.315535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fba890-17ff-4f98-9fb6-d96718564b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:29.316417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fba890-17ff-4f98-9fb6-d96718564b91 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:29.317307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fba890-17ff-4f98-9fb6-d96718564b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:29.318297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fba890-17ff-4f98-9fb6-d96718564b91 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:29.319015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fba890-17ff-4f98-9fb6-d96718564b91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:29.321816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fba890-17ff-4f98-9fb6-d96718564b91 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:29.411519 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1841/3694] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:13:29 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:13:29.415779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca6ac7-8dd0-468c-ab52-e1a4a0cacf5d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:29.427090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca6ac7-8dd0-468c-ab52-e1a4a0cacf5d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:29.462075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:29.484000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a0337-6354-477f-88bc-004eee583451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:29.484676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a0337-6354-477f-88bc-004eee583451 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:29.484965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a0337-6354-477f-88bc-004eee583451 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:29.485249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a0337-6354-477f-88bc-004eee583451 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:29.486241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a0337-6354-477f-88bc-004eee583451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:29.488358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a0337-6354-477f-88bc-004eee583451 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:29.603920 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1854/3695] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:13:29 2022] POST /identity/v3/users => generated 291 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:13:29.608231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fba890-17ff-4f98-9fb6-d96718564b91 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:29.614018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:29.619877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fba890-17ff-4f98-9fb6-d96718564b91 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:29.624825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2c075-c4d0-4e46-8531-9ff400adb093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:29.625262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2c075-c4d0-4e46-8531-9ff400adb093 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:29.625551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2c075-c4d0-4e46-8531-9ff400adb093 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:29.625855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2c075-c4d0-4e46-8531-9ff400adb093 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:29.626656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c2c075-c4d0-4e46-8531-9ff400adb093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:29.628303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c2c075-c4d0-4e46-8531-9ff400adb093 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:29.639991 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1842/3696] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:13:29.644168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a0337-6354-477f-88bc-004eee583451 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:29.654704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a0337-6354-477f-88bc-004eee583451 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:29.659961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074b856-18fd-41bc-a0d4-00679caef6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:29.660615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074b856-18fd-41bc-a0d4-00679caef6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:29.660894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074b856-18fd-41bc-a0d4-00679caef6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:29.661169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074b856-18fd-41bc-a0d4-00679caef6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:29.661943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074b856-18fd-41bc-a0d4-00679caef6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:29.663595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4074b856-18fd-41bc-a0d4-00679caef6b4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:29.677605 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1855/3697] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:13:29.679031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2c075-c4d0-4e46-8531-9ff400adb093 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:29.689456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2c075-c4d0-4e46-8531-9ff400adb093 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:29.694079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de929504-80ec-4e4f-92a7-23a0cb4ec8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:29.694979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de929504-80ec-4e4f-92a7-23a0cb4ec8c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:29.695262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de929504-80ec-4e4f-92a7-23a0cb4ec8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:29.695568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de929504-80ec-4e4f-92a7-23a0cb4ec8c9 admin admin] PATH_INFO: `/v3/projects/713293ceb9c44e8ea1a5d92668ad252a/users/041cd29a8fca40d0aab7e18c3b5e0dd6/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:29.710181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de929504-80ec-4e4f-92a7-23a0cb4ec8c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=713293ceb9c44e8ea1a5d92668ad252a, user_id=041cd29a8fca40d0aab7e18c3b5e0dd6, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:29.716696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de929504-80ec-4e4f-92a7-23a0cb4ec8c9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:29.810733 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1843/3698] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:13:29 2022] PUT /identity/v3/projects/713293ceb9c44e8ea1a5d92668ad252a/users/041cd29a8fca40d0aab7e18c3b5e0dd6/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:13:29.820762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb1467-44ce-44e8-848a-eddc99ac0830 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:29.821250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb1467-44ce-44e8-848a-eddc99ac0830 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:29.821561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb1467-44ce-44e8-848a-eddc99ac0830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:29.850470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-abcb1467-44ce-44e8-848a-eddc99ac0830 None None] MFA Rules not processed for user `041cd29a8fca40d0aab7e18c3b5e0dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:29.852209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-abcb1467-44ce-44e8-848a-eddc99ac0830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:29.925007 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1856/3699] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:30.089736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de929504-80ec-4e4f-92a7-23a0cb4ec8c9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:30.089736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-de929504-80ec-4e4f-92a7-23a0cb4ec8c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:30.173181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de929504-80ec-4e4f-92a7-23a0cb4ec8c9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:30.182207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb8349-b0b0-4ed5-acdb-cd18b2597d6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:30.182887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb8349-b0b0-4ed5-acdb-cd18b2597d6a service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:30.183170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb8349-b0b0-4ed5-acdb-cd18b2597d6a service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:30.183446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb8349-b0b0-4ed5-acdb-cd18b2597d6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:30.193433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb8349-b0b0-4ed5-acdb-cd18b2597d6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:30.195107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb8349-b0b0-4ed5-acdb-cd18b2597d6a service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:30.207815 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1844/3700] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:13:30 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:13:33.044946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5e63c-d2ab-4efc-84b9-45a4d2da565e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:33.045469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5e63c-d2ab-4efc-84b9-45a4d2da565e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:33.045589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5e63c-d2ab-4efc-84b9-45a4d2da565e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:33.077684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e4e5e63c-d2ab-4efc-84b9-45a4d2da565e None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:33.079035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e4e5e63c-d2ab-4efc-84b9-45a4d2da565e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:33.141886 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1857/3701] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:13:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:33.157355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb8349-b0b0-4ed5-acdb-cd18b2597d6a service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:33.159626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6ddb8349-b0b0-4ed5-acdb-cd18b2597d6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:33.269375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb8349-b0b0-4ed5-acdb-cd18b2597d6a service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:33.275253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912389eb-4b95-4eff-9155-f3f2ee3cd891 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:33.275902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912389eb-4b95-4eff-9155-f3f2ee3cd891 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:33.276182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912389eb-4b95-4eff-9155-f3f2ee3cd891 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:33.276576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912389eb-4b95-4eff-9155-f3f2ee3cd891 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:33.285224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912389eb-4b95-4eff-9155-f3f2ee3cd891 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:33.326008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912389eb-4b95-4eff-9155-f3f2ee3cd891 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:33.326008 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1845/3702] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:13:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:13:34.533035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5e63c-d2ab-4efc-84b9-45a4d2da565e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:34.552137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5e63c-d2ab-4efc-84b9-45a4d2da565e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:34.560047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10cafd-6ddd-4bf3-a242-fd2d8189f8dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:34.561218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10cafd-6ddd-4bf3-a242-fd2d8189f8dd service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:34.562397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10cafd-6ddd-4bf3-a242-fd2d8189f8dd service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:34.562860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10cafd-6ddd-4bf3-a242-fd2d8189f8dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:34.565784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9c10cafd-6ddd-4bf3-a242-fd2d8189f8dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:34.586034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10cafd-6ddd-4bf3-a242-fd2d8189f8dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:34.588835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10cafd-6ddd-4bf3-a242-fd2d8189f8dd service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:34.626119 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1858/3703] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:13:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:13:35.579989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fb990-39d4-49ef-b387-bbab67f70ffc None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:35.580983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fb990-39d4-49ef-b387-bbab67f70ffc None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:35.581244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fb990-39d4-49ef-b387-bbab67f70ffc None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:35.583100 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1846/3704] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:13:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 21:13:35.589585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3e602-9523-4109-97e3-3638ed6c47b6 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:35.590453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3e602-9523-4109-97e3-3638ed6c47b6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:35.590904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc3e602-9523-4109-97e3-3638ed6c47b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:35.650897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0dc3e602-9523-4109-97e3-3638ed6c47b6 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:35.652168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0dc3e602-9523-4109-97e3-3638ed6c47b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:35.684277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:35.752011 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1859/3705] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:13:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:35.776953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fb990-39d4-49ef-b387-bbab67f70ffc None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:35.778952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-728fb990-39d4-49ef-b387-bbab67f70ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:35.800879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fb990-39d4-49ef-b387-bbab67f70ffc None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:35.806175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3821dba-10d5-4d16-86ed-d55fbd96d4b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:35.806918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3821dba-10d5-4d16-86ed-d55fbd96d4b7 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:35.807346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3821dba-10d5-4d16-86ed-d55fbd96d4b7 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:35.807618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3821dba-10d5-4d16-86ed-d55fbd96d4b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:35.819965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3821dba-10d5-4d16-86ed-d55fbd96d4b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:35.819965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3821dba-10d5-4d16-86ed-d55fbd96d4b7 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:35.829303 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1847/3706] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:13:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:13:36.235567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ec3f9-1c9b-4e8d-872f-fc2f014920db None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:36.237040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ec3f9-1c9b-4e8d-872f-fc2f014920db None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:36.237479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ec3f9-1c9b-4e8d-872f-fc2f014920db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:36.260301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-810ec3f9-1c9b-4e8d-872f-fc2f014920db None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:36.262086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-810ec3f9-1c9b-4e8d-872f-fc2f014920db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:36.275935 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1860/3707] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:13:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:36.297908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3821dba-10d5-4d16-86ed-d55fbd96d4b7 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:36.327762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3821dba-10d5-4d16-86ed-d55fbd96d4b7 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:36.340622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c7116-9b6d-4a27-b589-3a03fddc474b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:36.340913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c7116-9b6d-4a27-b589-3a03fddc474b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:36.341378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c7116-9b6d-4a27-b589-3a03fddc474b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:36.343555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e4990-f380-4bce-8b7f-2f9ba110b5d9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:36.344283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e4990-f380-4bce-8b7f-2f9ba110b5d9 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:36.344729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e4990-f380-4bce-8b7f-2f9ba110b5d9 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:36.345061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143e4990-f380-4bce-8b7f-2f9ba110b5d9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:36.356298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e4990-f380-4bce-8b7f-2f9ba110b5d9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:36.361595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143e4990-f380-4bce-8b7f-2f9ba110b5d9 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:36.372830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-383c7116-9b6d-4a27-b589-3a03fddc474b None None] MFA Rules not processed for user `bf9c8f66e7d24ca9baca835e41546f22`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:36.373250 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1848/3708] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:13:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:13:36.374096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-383c7116-9b6d-4a27-b589-3a03fddc474b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:36.453190 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1861/3709] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:13:45.990314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e4990-f380-4bce-8b7f-2f9ba110b5d9 service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:45.996162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-143e4990-f380-4bce-8b7f-2f9ba110b5d9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:46.029593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143e4990-f380-4bce-8b7f-2f9ba110b5d9 service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:46.051637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf19b279-8716-439b-836a-6087fa17d7cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:46.054710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19b279-8716-439b-836a-6087fa17d7cb service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:46.057197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19b279-8716-439b-836a-6087fa17d7cb service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:46.058519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf19b279-8716-439b-836a-6087fa17d7cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:46.071778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cf19b279-8716-439b-836a-6087fa17d7cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:46.100383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19b279-8716-439b-836a-6087fa17d7cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:46.112270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf19b279-8716-439b-836a-6087fa17d7cb service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:46.149914 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1849/3710] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:13:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:13:46.439269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=46 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:47.379769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880daaa-707f-4707-90bc-f1a33a2e3c2e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:47.380946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880daaa-707f-4707-90bc-f1a33a2e3c2e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:47.382117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7880daaa-707f-4707-90bc-f1a33a2e3c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:47.441556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7880daaa-707f-4707-90bc-f1a33a2e3c2e None None] MFA Rules not processed for user `eb7c3a01b54f4db2841f4836635eff2e`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:47.443157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7880daaa-707f-4707-90bc-f1a33a2e3c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:47.581557 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1862/3711] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:47.614414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d6c20-3642-4f4c-9506-dcd60c3f0fb5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:47.615405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d6c20-3642-4f4c-9506-dcd60c3f0fb5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:47.615886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299d6c20-3642-4f4c-9506-dcd60c3f0fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:47.626242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:47.641455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-299d6c20-3642-4f4c-9506-dcd60c3f0fb5 None None] MFA Rules not processed for user `eb7c3a01b54f4db2841f4836635eff2e`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:47.650116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-299d6c20-3642-4f4c-9506-dcd60c3f0fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:47.661947 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1850/3712] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:54.262828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880daaa-707f-4707-90bc-f1a33a2e3c2e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:54.337387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7880daaa-707f-4707-90bc-f1a33a2e3c2e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:54.342817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e2fcd-a462-48bb-9e08-b7eba145972a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:54.343757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2fcd-a462-48bb-9e08-b7eba145972a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:54.344101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2fcd-a462-48bb-9e08-b7eba145972a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:54.344499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e2fcd-a462-48bb-9e08-b7eba145972a admin admin] PATH_INFO: `/v3/users/041cd29a8fca40d0aab7e18c3b5e0dd6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:54.353950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e2fcd-a462-48bb-9e08-b7eba145972a admin admin] RBAC: Authorizing `identity:delete_user(user_id=041cd29a8fca40d0aab7e18c3b5e0dd6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:54.356151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e2fcd-a462-48bb-9e08-b7eba145972a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:54.662869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-128e2fcd-a462-48bb-9e08-b7eba145972a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '041cd29a8fca40d0aab7e18c3b5e0dd6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:54.671802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-128e2fcd-a462-48bb-9e08-b7eba145972a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '041cd29a8fca40d0aab7e18c3b5e0dd6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:54.672560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-128e2fcd-a462-48bb-9e08-b7eba145972a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '041cd29a8fca40d0aab7e18c3b5e0dd6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:54.673226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-128e2fcd-a462-48bb-9e08-b7eba145972a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '041cd29a8fca40d0aab7e18c3b5e0dd6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:54.729404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-128e2fcd-a462-48bb-9e08-b7eba145972a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '041cd29a8fca40d0aab7e18c3b5e0dd6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:54.763787 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1863/3713] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:13:54 2022] DELETE /identity/v3/users/041cd29a8fca40d0aab7e18c3b5e0dd6 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:13:54.765901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca747258-78e0-41c7-bda2-5b28c403e505 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:54.766334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca747258-78e0-41c7-bda2-5b28c403e505 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:54.766462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca747258-78e0-41c7-bda2-5b28c403e505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:54.787377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ca747258-78e0-41c7-bda2-5b28c403e505 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:54.788615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ca747258-78e0-41c7-bda2-5b28c403e505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:54.854547 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1851/3714] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:13:54.865519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e2fcd-a462-48bb-9e08-b7eba145972a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:54.866998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-128e2fcd-a462-48bb-9e08-b7eba145972a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:54.903163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:54.948554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e2fcd-a462-48bb-9e08-b7eba145972a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:54.952420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286e69e-042f-48f8-a122-b9f947bdac14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:54.953319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286e69e-042f-48f8-a122-b9f947bdac14 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:54.953750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286e69e-042f-48f8-a122-b9f947bdac14 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:54.954079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286e69e-042f-48f8-a122-b9f947bdac14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:54.971032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286e69e-042f-48f8-a122-b9f947bdac14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:54.972882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7286e69e-042f-48f8-a122-b9f947bdac14 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:54.991493 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1864/3715] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:13:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:13:55.080487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaae2aa-5320-4d41-b158-25ee7ba56431 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:55.080782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaae2aa-5320-4d41-b158-25ee7ba56431 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:55.081066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaae2aa-5320-4d41-b158-25ee7ba56431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:55.102062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bcaae2aa-5320-4d41-b158-25ee7ba56431 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:55.103610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bcaae2aa-5320-4d41-b158-25ee7ba56431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:55.171698 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1852/3716] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:13:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:55.184621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286e69e-042f-48f8-a122-b9f947bdac14 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:55.186167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7286e69e-042f-48f8-a122-b9f947bdac14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:55.261438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286e69e-042f-48f8-a122-b9f947bdac14 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:55.266593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823272f0-444c-4ef1-b886-c3be5ba8d808 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:55.267257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823272f0-444c-4ef1-b886-c3be5ba8d808 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:55.267563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823272f0-444c-4ef1-b886-c3be5ba8d808 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:55.267847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823272f0-444c-4ef1-b886-c3be5ba8d808 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:55.276623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823272f0-444c-4ef1-b886-c3be5ba8d808 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:55.278274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823272f0-444c-4ef1-b886-c3be5ba8d808 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:55.287324 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1865/3717] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:13:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:13:55.622874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaae2aa-5320-4d41-b158-25ee7ba56431 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:55.624446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bcaae2aa-5320-4d41-b158-25ee7ba56431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:55.641470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaae2aa-5320-4d41-b158-25ee7ba56431 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:55.646833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1e000-65e0-4234-ae35-ea2418fdf19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:55.647791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1e000-65e0-4234-ae35-ea2418fdf19d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:55.648157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1e000-65e0-4234-ae35-ea2418fdf19d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:55.648558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1e000-65e0-4234-ae35-ea2418fdf19d admin admin] PATH_INFO: `/v3/projects/713293ceb9c44e8ea1a5d92668ad252a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:55.650069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1e000-65e0-4234-ae35-ea2418fdf19d admin admin] RBAC: Authorizing `identity:delete_project(project_id=713293ceb9c44e8ea1a5d92668ad252a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:55.652353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1e000-65e0-4234-ae35-ea2418fdf19d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:55.798134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e4d1e000-65e0-4234-ae35-ea2418fdf19d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '713293ceb9c44e8ea1a5d92668ad252a', 'request_id': 'req-e4d1e000-65e0-4234-ae35-ea2418fdf19d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:55.806346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e4d1e000-65e0-4234-ae35-ea2418fdf19d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '713293ceb9c44e8ea1a5d92668ad252a', 'request_id': 'req-e4d1e000-65e0-4234-ae35-ea2418fdf19d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:55.923691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e4d1e000-65e0-4234-ae35-ea2418fdf19d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '713293ceb9c44e8ea1a5d92668ad252a', 'request_id': 'req-e4d1e000-65e0-4234-ae35-ea2418fdf19d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:55.937727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e4d1e000-65e0-4234-ae35-ea2418fdf19d admin admin] The token cache is being invalidate because project 713293ceb9c44e8ea1a5d92668ad252a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:13:55.938194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e4d1e000-65e0-4234-ae35-ea2418fdf19d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:55.938865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e4d1e000-65e0-4234-ae35-ea2418fdf19d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:55.941221 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1853/3718] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:13:55 2022] DELETE /identity/v3/projects/713293ceb9c44e8ea1a5d92668ad252a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:13:55.948374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19ad5d-eea2-4a6f-82a0-6e896aa21ca1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:55.949510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19ad5d-eea2-4a6f-82a0-6e896aa21ca1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:55.949510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff19ad5d-eea2-4a6f-82a0-6e896aa21ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:55.969311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ff19ad5d-eea2-4a6f-82a0-6e896aa21ca1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:55.970837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ff19ad5d-eea2-4a6f-82a0-6e896aa21ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:56.058022 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1866/3719] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:56.075367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9169a4d-d5e2-41c3-9c76-11184b587ac4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:56.075716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9169a4d-d5e2-41c3-9c76-11184b587ac4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:56.080765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9169a4d-d5e2-41c3-9c76-11184b587ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:56.109141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b9169a4d-d5e2-41c3-9c76-11184b587ac4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:56.111988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b9169a4d-d5e2-41c3-9c76-11184b587ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:56.129517 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1854/3720] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:56.133009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff19ad5d-eea2-4a6f-82a0-6e896aa21ca1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:56.147238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff19ad5d-eea2-4a6f-82a0-6e896aa21ca1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:56.158130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347036a4-352c-430b-8d4f-6ff525298511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:56.158876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347036a4-352c-430b-8d4f-6ff525298511 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:56.159675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347036a4-352c-430b-8d4f-6ff525298511 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:56.160049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347036a4-352c-430b-8d4f-6ff525298511 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:56.161014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347036a4-352c-430b-8d4f-6ff525298511 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:13:56.161419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347036a4-352c-430b-8d4f-6ff525298511 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:56.163361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347036a4-352c-430b-8d4f-6ff525298511 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:56.177555 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1867/3721] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:13:56.184705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9169a4d-d5e2-41c3-9c76-11184b587ac4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:56.202150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9169a4d-d5e2-41c3-9c76-11184b587ac4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:56.221787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11d58d-bc3b-49d9-bdff-791bfb469d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:56.221787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11d58d-bc3b-49d9-bdff-791bfb469d2d admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:56.221787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11d58d-bc3b-49d9-bdff-791bfb469d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:56.221787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb11d58d-bc3b-49d9-bdff-791bfb469d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:56.222418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11d58d-bc3b-49d9-bdff-791bfb469d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:56.223584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb11d58d-bc3b-49d9-bdff-791bfb469d2d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:56.253765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:56.346298 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1855/3722] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:13:56 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:13:56.350148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347036a4-352c-430b-8d4f-6ff525298511 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:56.375598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347036a4-352c-430b-8d4f-6ff525298511 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:56.440061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:56.465810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fee02f-f7db-486b-843f-a344d8ecd849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:56.466963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fee02f-f7db-486b-843f-a344d8ecd849 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:56.467255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fee02f-f7db-486b-843f-a344d8ecd849 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:56.467539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fee02f-f7db-486b-843f-a344d8ecd849 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:56.468248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fee02f-f7db-486b-843f-a344d8ecd849 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:56.470583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fee02f-f7db-486b-843f-a344d8ecd849 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:56.625384 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1868/3723] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:13:56 2022] POST /identity/v3/users => generated 294 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:13:56.636238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11d58d-bc3b-49d9-bdff-791bfb469d2d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:56.648010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb11d58d-bc3b-49d9-bdff-791bfb469d2d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:56.656893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96651b-42bf-4d8c-987a-b98f23e39c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:56.662031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96651b-42bf-4d8c-987a-b98f23e39c1c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:56.662031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96651b-42bf-4d8c-987a-b98f23e39c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:56.662031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96651b-42bf-4d8c-987a-b98f23e39c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:56.662031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96651b-42bf-4d8c-987a-b98f23e39c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:56.662031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff96651b-42bf-4d8c-987a-b98f23e39c1c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:56.685731 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1856/3724] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:13:56.689746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fee02f-f7db-486b-843f-a344d8ecd849 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:56.709325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fee02f-f7db-486b-843f-a344d8ecd849 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:56.732167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8dc749-ac8e-4901-97a4-37ab65cf148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:56.734325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dc749-ac8e-4901-97a4-37ab65cf148b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:56.734325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dc749-ac8e-4901-97a4-37ab65cf148b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:56.735030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8dc749-ac8e-4901-97a4-37ab65cf148b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:56.736339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8dc749-ac8e-4901-97a4-37ab65cf148b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:56.738166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8dc749-ac8e-4901-97a4-37ab65cf148b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:56.754012 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1869/3725] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:13:56.759574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96651b-42bf-4d8c-987a-b98f23e39c1c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:56.771335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96651b-42bf-4d8c-987a-b98f23e39c1c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:56.789738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c58cf7-fac8-48b7-9935-fc3b26790d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:56.805495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c58cf7-fac8-48b7-9935-fc3b26790d30 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:56.805495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c58cf7-fac8-48b7-9935-fc3b26790d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:56.805495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c58cf7-fac8-48b7-9935-fc3b26790d30 admin admin] PATH_INFO: `/v3/projects/c7eb5cbea535490c977f54d7d587b668/users/562c62aa43b64f4386a02ce93ba3fecc/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:56.822748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c58cf7-fac8-48b7-9935-fc3b26790d30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7eb5cbea535490c977f54d7d587b668, user_id=562c62aa43b64f4386a02ce93ba3fecc, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:56.826530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c58cf7-fac8-48b7-9935-fc3b26790d30 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:56.899714 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1857/3726] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:13:56 2022] PUT /identity/v3/projects/c7eb5cbea535490c977f54d7d587b668/users/562c62aa43b64f4386a02ce93ba3fecc/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:13:56.912913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8c277-c5c8-46a7-b4da-098f6b943cdc None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:56.913211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8c277-c5c8-46a7-b4da-098f6b943cdc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:56.913503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b8c277-c5c8-46a7-b4da-098f6b943cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:56.970904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a4b8c277-c5c8-46a7-b4da-098f6b943cdc None None] MFA Rules not processed for user `562c62aa43b64f4386a02ce93ba3fecc`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:56.972355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a4b8c277-c5c8-46a7-b4da-098f6b943cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:57.088825 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1870/3727] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:57.093412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c58cf7-fac8-48b7-9935-fc3b26790d30 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:57.172836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c58cf7-fac8-48b7-9935-fc3b26790d30 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:57.182818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da84acf-6fe3-4362-8d6e-b5d90c3eb448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:57.183927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da84acf-6fe3-4362-8d6e-b5d90c3eb448 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:57.184368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da84acf-6fe3-4362-8d6e-b5d90c3eb448 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:57.192008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da84acf-6fe3-4362-8d6e-b5d90c3eb448 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:57.192008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da84acf-6fe3-4362-8d6e-b5d90c3eb448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:57.192008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da84acf-6fe3-4362-8d6e-b5d90c3eb448 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:57.362281 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1858/3728] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:13:57 2022] POST /identity/v3/projects => generated 385 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:13:57.366243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8c277-c5c8-46a7-b4da-098f6b943cdc None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:57.384124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b8c277-c5c8-46a7-b4da-098f6b943cdc None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:57.507177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8b2ef-eb8c-430e-b5c6-e573471327b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:57.507905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8b2ef-eb8c-430e-b5c6-e573471327b8 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:57.509500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8b2ef-eb8c-430e-b5c6-e573471327b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:57.509500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8b2ef-eb8c-430e-b5c6-e573471327b8 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:57.509721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8b2ef-eb8c-430e-b5c6-e573471327b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:57.511688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8b2ef-eb8c-430e-b5c6-e573471327b8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:57.668860 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1871/3729] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:13:57 2022] POST /identity/v3/users => generated 299 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:13:57.669671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da84acf-6fe3-4362-8d6e-b5d90c3eb448 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:57.684274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da84acf-6fe3-4362-8d6e-b5d90c3eb448 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:57.690390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eada6f-eae7-48d5-9259-291b28d45fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:57.691110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eada6f-eae7-48d5-9259-291b28d45fff admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:57.691414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eada6f-eae7-48d5-9259-291b28d45fff admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:57.691700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eada6f-eae7-48d5-9259-291b28d45fff admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:57.692783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eada6f-eae7-48d5-9259-291b28d45fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:57.694111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eada6f-eae7-48d5-9259-291b28d45fff admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:57.707417 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1859/3730] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:13:57.712166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8b2ef-eb8c-430e-b5c6-e573471327b8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:57.736597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8b2ef-eb8c-430e-b5c6-e573471327b8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:57.753327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d1b0a-3ed3-4980-9a24-7f8c05bc860f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:57.754282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d1b0a-3ed3-4980-9a24-7f8c05bc860f admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:57.754572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d1b0a-3ed3-4980-9a24-7f8c05bc860f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:57.754858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d1b0a-3ed3-4980-9a24-7f8c05bc860f admin admin] PATH_INFO: `/v3/projects/0d9322e4d12641a08b572d36faabe811/users/1b56cd967c7445ddad28ba2b3678a059/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:57.789820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d1b0a-3ed3-4980-9a24-7f8c05bc860f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d9322e4d12641a08b572d36faabe811, user_id=1b56cd967c7445ddad28ba2b3678a059, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:57.794170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d1b0a-3ed3-4980-9a24-7f8c05bc860f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:57.931672 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1872/3731] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:13:57 2022] PUT /identity/v3/projects/0d9322e4d12641a08b572d36faabe811/users/1b56cd967c7445ddad28ba2b3678a059/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:13:57.935439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eada6f-eae7-48d5-9259-291b28d45fff admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:57.947432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eada6f-eae7-48d5-9259-291b28d45fff admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:58.028860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0675d7-41d9-48a8-8e82-b7306768fbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:58.029575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0675d7-41d9-48a8-8e82-b7306768fbbb admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:58.030165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0675d7-41d9-48a8-8e82-b7306768fbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:58.030916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0675d7-41d9-48a8-8e82-b7306768fbbb admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:58.032130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0675d7-41d9-48a8-8e82-b7306768fbbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:58.034423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0675d7-41d9-48a8-8e82-b7306768fbbb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:58.079405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d1b0a-3ed3-4980-9a24-7f8c05bc860f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:58.079771 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1860/3732] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:13:58.097330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d1b0a-3ed3-4980-9a24-7f8c05bc860f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:58.115219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa57c6-361c-4ea2-87a3-4b4715fdc912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:58.116037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa57c6-361c-4ea2-87a3-4b4715fdc912 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:58.116253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa57c6-361c-4ea2-87a3-4b4715fdc912 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:58.119373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa57c6-361c-4ea2-87a3-4b4715fdc912 admin admin] PATH_INFO: `/v3/projects/0d9322e4d12641a08b572d36faabe811/users/1b56cd967c7445ddad28ba2b3678a059/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:58.123674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa57c6-361c-4ea2-87a3-4b4715fdc912 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d9322e4d12641a08b572d36faabe811, user_id=1b56cd967c7445ddad28ba2b3678a059, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:58.132163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fa57c6-361c-4ea2-87a3-4b4715fdc912 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:58.181067 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1873/3733] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:13:58 2022] PUT /identity/v3/projects/0d9322e4d12641a08b572d36faabe811/users/1b56cd967c7445ddad28ba2b3678a059/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:13:58.193191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01749b4-fdaf-4438-b1aa-ba18b2ffee24 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:58.194564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01749b4-fdaf-4438-b1aa-ba18b2ffee24 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:58.195136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01749b4-fdaf-4438-b1aa-ba18b2ffee24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:58.256244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c01749b4-fdaf-4438-b1aa-ba18b2ffee24 None None] MFA Rules not processed for user `1b56cd967c7445ddad28ba2b3678a059`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:13:58.257542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c01749b4-fdaf-4438-b1aa-ba18b2ffee24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:13:58.288202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:58.407585 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1861/3734] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:13:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:13:58.411676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa57c6-361c-4ea2-87a3-4b4715fdc912 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:58.420395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:58.428779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa57c6-361c-4ea2-87a3-4b4715fdc912 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:58.434958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208745b-4038-4a1e-9f5d-09ece3277533 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b56cd967c7445ddad28ba2b3678a059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d9322e4d12641a08b572d36faabe811', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:58.435935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208745b-4038-4a1e-9f5d-09ece3277533 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:58.436421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208745b-4038-4a1e-9f5d-09ece3277533 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:58.436896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208745b-4038-4a1e-9f5d-09ece3277533 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:58.438156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208745b-4038-4a1e-9f5d-09ece3277533 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:58.439530 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-f208745b-4038-4a1e-9f5d-09ece3277533 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:13:58.439530 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:13:58.439530 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:13:58.440961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208745b-4038-4a1e-9f5d-09ece3277533 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:58.540091 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1874/3735] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:13:58 2022] POST /identity/v3/services => generated 288 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:13:58.542754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01749b4-fdaf-4438-b1aa-ba18b2ffee24 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:58.557761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01749b4-fdaf-4438-b1aa-ba18b2ffee24 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:58.578623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111ac00-dd84-4742-ae79-35b772a12e9c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b56cd967c7445ddad28ba2b3678a059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d9322e4d12641a08b572d36faabe811', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:58.579446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111ac00-dd84-4742-ae79-35b772a12e9c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:58.579846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111ac00-dd84-4742-ae79-35b772a12e9c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:58.580255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111ac00-dd84-4742-ae79-35b772a12e9c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:58.581261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111ac00-dd84-4742-ae79-35b772a12e9c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:58.582575 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-6111ac00-dd84-4742-ae79-35b772a12e9c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:13:58.582575 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:13:58.582575 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:13:58.583844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111ac00-dd84-4742-ae79-35b772a12e9c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-6111ac00-dd84-4742-ae79-35b772a12e9c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: Failed validating 'type' in schema['properties']['enabled']: May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: {'enum': [True, False, None], 'type': 'boolean'} May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: On instance['enabled']: May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: Failed validating 'type' in schema['properties']['enabled']: May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: {'enum': [True, False, None], 'type': 'boolean'} May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: On instance['enabled']: May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: 'False' May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise error May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application On instance['enabled']: May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application 'False' May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:13:58.611013 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:13:58.612438 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:13:58.612438 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:13:58.612438 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:13:58.612438 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:13:58.612438 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:13:58.612438 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:13:58.612438 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:13:58.612438 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:13:58.612438 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:13:58.612438 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:13:58.612438 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:13:58.612438 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:13:58.612438 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post May 28 21:13:58.612438 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) May 28 21:13:58.612438 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate May 28 21:13:58.612438 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) May 28 21:13:58.612438 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate May 28 21:13:58.613961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) May 28 21:13:58.613961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' May 28 21:13:58.613961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:13:58.613961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: May 28 21:13:58.613961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} May 28 21:13:58.613961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:13:58.613961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application On instance['enabled']: May 28 21:13:58.613961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application 'False' May 28 21:13:58.613961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:13:58.619003 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1862/3736] 10.222.0.33 () {60 vars in 1155 bytes} [Sat May 28 21:13:58 2022] POST /identity/v3/endpoints => generated 283 bytes in 76 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) May 28 21:13:58.619484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208745b-4038-4a1e-9f5d-09ece3277533 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:58.636702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208745b-4038-4a1e-9f5d-09ece3277533 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:58.659783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bbc8e-cf0c-411d-bb6a-af9097128de4 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b56cd967c7445ddad28ba2b3678a059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d9322e4d12641a08b572d36faabe811', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:58.659783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bbc8e-cf0c-411d-bb6a-af9097128de4 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:58.659783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bbc8e-cf0c-411d-bb6a-af9097128de4 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:58.660038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bbc8e-cf0c-411d-bb6a-af9097128de4 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:58.660038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bbc8e-cf0c-411d-bb6a-af9097128de4 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:58.660038 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-bb4bbc8e-cf0c-411d-bb6a-af9097128de4 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:13:58.660038 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:13:58.660038 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:13:58.660353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bbc8e-cf0c-411d-bb6a-af9097128de4 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-bb4bbc8e-cf0c-411d-bb6a-af9097128de4 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: Failed validating 'type' in schema['properties']['enabled']: May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: {'enum': [True, False, None], 'type': 'boolean'} May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: On instance['enabled']: May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: Failed validating 'type' in schema['properties']['enabled']: May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: {'enum': [True, False, None], 'type': 'boolean'} May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: On instance['enabled']: May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: 'True' May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise error May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application On instance['enabled']: May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application 'True' May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:13:58.679102 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:13:58.686526 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:13:58.686526 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:13:58.686526 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:13:58.686526 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:13:58.686526 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:13:58.686526 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:13:58.686526 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:13:58.686526 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:13:58.686526 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:13:58.686526 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:13:58.686526 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:13:58.686526 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:13:58.686526 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 94, in post May 28 21:13:58.686526 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_create, endpoint) May 28 21:13:58.686526 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate May 28 21:13:58.686526 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) May 28 21:13:58.686526 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate May 28 21:13:58.688379 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) May 28 21:13:58.688379 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' May 28 21:13:58.688379 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:13:58.688379 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: May 28 21:13:58.688379 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} May 28 21:13:58.688379 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:13:58.688379 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application On instance['enabled']: May 28 21:13:58.688379 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application 'True' May 28 21:13:58.688379 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:13:58.688379 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1875/3737] 10.222.0.33 () {60 vars in 1155 bytes} [Sat May 28 21:13:58 2022] POST /identity/v3/endpoints => generated 281 bytes in 62 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) May 28 21:13:58.688942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111ac00-dd84-4742-ae79-35b772a12e9c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:58.713344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111ac00-dd84-4742-ae79-35b772a12e9c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:58.724796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835455e-01be-4a55-9477-a1aa68c84a28 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b56cd967c7445ddad28ba2b3678a059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d9322e4d12641a08b572d36faabe811', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:58.725651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835455e-01be-4a55-9477-a1aa68c84a28 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:58.730855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835455e-01be-4a55-9477-a1aa68c84a28 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:58.730855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835455e-01be-4a55-9477-a1aa68c84a28 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:58.730855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835455e-01be-4a55-9477-a1aa68c84a28 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:58.730855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835455e-01be-4a55-9477-a1aa68c84a28 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:58.826645 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-8835455e-01be-4a55-9477-a1aa68c84a28 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-383321109. To ensure interoperability, identifiers should be a valid uuid. May 28 21:13:58.826645 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' May 28 21:13:58.826645 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:13:58.979528 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1863/3738] 10.222.0.33 () {60 vars in 1155 bytes} [Sat May 28 21:13:58 2022] POST /identity/v3/endpoints => generated 359 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:13:58.981461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bbc8e-cf0c-411d-bb6a-af9097128de4 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:59.020233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bbc8e-cf0c-411d-bb6a-af9097128de4 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:59.056022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a536e0-753f-4b53-b787-988ec8639f14 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b56cd967c7445ddad28ba2b3678a059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d9322e4d12641a08b572d36faabe811', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:59.056882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a536e0-753f-4b53-b787-988ec8639f14 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:59.064485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a536e0-753f-4b53-b787-988ec8639f14 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:59.065010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a536e0-753f-4b53-b787-988ec8639f14 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] PATH_INFO: `/v3/regions/tempest-region-383321109` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:59.066176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a536e0-753f-4b53-b787-988ec8639f14 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-383321109)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:59.076293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a536e0-753f-4b53-b787-988ec8639f14 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:59.079563 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1876/3739] 10.222.0.33 () {58 vars in 1202 bytes} [Sat May 28 21:13:58 2022] GET /identity/v3/regions/tempest-region-383321109 => generated 178 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:13:59.083955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835455e-01be-4a55-9477-a1aa68c84a28 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:59.098488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835455e-01be-4a55-9477-a1aa68c84a28 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:59.105377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c23b8-3652-4468-ace8-7f1cf9779b93 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b56cd967c7445ddad28ba2b3678a059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d9322e4d12641a08b572d36faabe811', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:59.116144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c23b8-3652-4468-ace8-7f1cf9779b93 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:59.116144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c23b8-3652-4468-ace8-7f1cf9779b93 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:59.116144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10c23b8-3652-4468-ace8-7f1cf9779b93 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] PATH_INFO: `/v3/endpoints/3e7bbe7bb95b4b4c82de3bfa143f12e4` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:59.119617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c23b8-3652-4468-ace8-7f1cf9779b93 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=3e7bbe7bb95b4b4c82de3bfa143f12e4)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:59.120819 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-a10c23b8-3652-4468-ace8-7f1cf9779b93 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:13:59.120819 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:13:59.120819 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:13:59.123669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10c23b8-3652-4468-ace8-7f1cf9779b93 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-a10c23b8-3652-4468-ace8-7f1cf9779b93 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Invalid input for field 'enabled': 'False' is not of type 'boolean' May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: Failed validating 'type' in schema['properties']['enabled']: May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: {'enum': [True, False, None], 'type': 'boolean'} May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: On instance['enabled']: May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: 'False': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: Failed validating 'type' in schema['properties']['enabled']: May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: {'enum': [True, False, None], 'type': 'boolean'} May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: On instance['enabled']: May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: 'False' May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise error May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'False' is not of type 'boolean' May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application On instance['enabled']: May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application 'False' May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:13:59.138779 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:13:59.140609 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:13:59.140609 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:13:59.140609 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:13:59.140609 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:13:59.140609 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:13:59.140609 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:13:59.140609 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:13:59.140609 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:13:59.140609 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:13:59.140609 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:13:59.140609 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:13:59.140609 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:13:59.140609 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch May 28 21:13:59.140609 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) May 28 21:13:59.140609 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate May 28 21:13:59.140609 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) May 28 21:13:59.140609 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate May 28 21:13:59.141753 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) May 28 21:13:59.141753 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'False' is not of type 'boolean' May 28 21:13:59.141753 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:13:59.141753 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: May 28 21:13:59.141753 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} May 28 21:13:59.141753 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:13:59.141753 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application On instance['enabled']: May 28 21:13:59.141753 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application 'False' May 28 21:13:59.141753 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:13:59.144038 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1864/3740] 10.222.0.33 () {60 vars in 1254 bytes} [Sat May 28 21:13:59 2022] PATCH /identity/v3/endpoints/3e7bbe7bb95b4b4c82de3bfa143f12e4 => generated 283 bytes in 60 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) May 28 21:13:59.148481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a536e0-753f-4b53-b787-988ec8639f14 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:59.168259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a536e0-753f-4b53-b787-988ec8639f14 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:59.173193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b6d64-4b49-4319-a9a8-f9da273915ce tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b56cd967c7445ddad28ba2b3678a059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d9322e4d12641a08b572d36faabe811', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:59.174158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b6d64-4b49-4319-a9a8-f9da273915ce tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:59.174448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b6d64-4b49-4319-a9a8-f9da273915ce tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:59.174727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b6d64-4b49-4319-a9a8-f9da273915ce tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] PATH_INFO: `/v3/endpoints/3e7bbe7bb95b4b4c82de3bfa143f12e4` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:59.176289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73b6d64-4b49-4319-a9a8-f9da273915ce tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=3e7bbe7bb95b4b4c82de3bfa143f12e4)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:59.177521 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-e73b6d64-4b49-4319-a9a8-f9da273915ce tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:13:59.177521 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:13:59.177521 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:13:59.178872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73b6d64-4b49-4319-a9a8-f9da273915ce tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:59.306437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e73b6d64-4b49-4319-a9a8-f9da273915ce tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '3e7bbe7bb95b4b4c82de3bfa143f12e4', 'request_id': 'req-e73b6d64-4b49-4319-a9a8-f9da273915ce'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:59.325326 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1877/3741] 10.222.0.33 () {58 vars in 1235 bytes} [Sat May 28 21:13:59 2022] DELETE /identity/v3/endpoints/3e7bbe7bb95b4b4c82de3bfa143f12e4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:13:59.329563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c23b8-3652-4468-ace8-7f1cf9779b93 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:59.341210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10c23b8-3652-4468-ace8-7f1cf9779b93 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:59.368669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26b673-c8b0-4e5f-8ae5-2881ec4e5230 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b56cd967c7445ddad28ba2b3678a059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d9322e4d12641a08b572d36faabe811', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:59.369896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26b673-c8b0-4e5f-8ae5-2881ec4e5230 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:59.370211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26b673-c8b0-4e5f-8ae5-2881ec4e5230 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:59.371816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26b673-c8b0-4e5f-8ae5-2881ec4e5230 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] PATH_INFO: `/v3/regions/tempest-region-383321109` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:59.372567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26b673-c8b0-4e5f-8ae5-2881ec4e5230 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-383321109)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:59.374883 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-cb26b673-c8b0-4e5f-8ae5-2881ec4e5230 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:13:59.374883 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:13:59.374883 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:13:59.375886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26b673-c8b0-4e5f-8ae5-2881ec4e5230 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:59.491533 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1865/3742] 10.222.0.33 () {58 vars in 1205 bytes} [Sat May 28 21:13:59 2022] DELETE /identity/v3/regions/tempest-region-383321109 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:13:59.496470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b6d64-4b49-4319-a9a8-f9da273915ce tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:59.508886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b6d64-4b49-4319-a9a8-f9da273915ce tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:59.529896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2bc24c-9e02-4b5e-bfa6-2f389669d82c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b56cd967c7445ddad28ba2b3678a059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d9322e4d12641a08b572d36faabe811', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:59.540247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bc24c-9e02-4b5e-bfa6-2f389669d82c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:59.540247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bc24c-9e02-4b5e-bfa6-2f389669d82c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:59.540247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2bc24c-9e02-4b5e-bfa6-2f389669d82c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:59.540247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2bc24c-9e02-4b5e-bfa6-2f389669d82c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:59.540247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2bc24c-9e02-4b5e-bfa6-2f389669d82c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:59.620308 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-3b2bc24c-9e02-4b5e-bfa6-2f389669d82c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-630657403. To ensure interoperability, identifiers should be a valid uuid. May 28 21:13:59.620308 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' May 28 21:13:59.620308 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:13:59.717658 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1878/3743] 10.222.0.33 () {60 vars in 1155 bytes} [Sat May 28 21:13:59 2022] POST /identity/v3/endpoints => generated 359 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:13:59.722899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26b673-c8b0-4e5f-8ae5-2881ec4e5230 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:59.742058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26b673-c8b0-4e5f-8ae5-2881ec4e5230 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:59.774669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132dad8-ee63-4a87-bee2-02493819c3ff tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b56cd967c7445ddad28ba2b3678a059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d9322e4d12641a08b572d36faabe811', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:59.775574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132dad8-ee63-4a87-bee2-02493819c3ff tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:59.775849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132dad8-ee63-4a87-bee2-02493819c3ff tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:59.776116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132dad8-ee63-4a87-bee2-02493819c3ff tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] PATH_INFO: `/v3/regions/tempest-region-630657403` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:59.776774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132dad8-ee63-4a87-bee2-02493819c3ff tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-630657403)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:59.777933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1132dad8-ee63-4a87-bee2-02493819c3ff tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:59.783860 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1866/3744] 10.222.0.33 () {58 vars in 1202 bytes} [Sat May 28 21:13:59 2022] GET /identity/v3/regions/tempest-region-630657403 => generated 178 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:13:59.785919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2bc24c-9e02-4b5e-bfa6-2f389669d82c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:59.804695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2bc24c-9e02-4b5e-bfa6-2f389669d82c tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:59.815122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ac0ff-4e7e-4403-adaa-7e05d263b5f3 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b56cd967c7445ddad28ba2b3678a059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d9322e4d12641a08b572d36faabe811', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:59.816750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ac0ff-4e7e-4403-adaa-7e05d263b5f3 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:59.817236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ac0ff-4e7e-4403-adaa-7e05d263b5f3 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:59.817689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ac0ff-4e7e-4403-adaa-7e05d263b5f3 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] PATH_INFO: `/v3/endpoints/bddd1a394f614948a558b27a0e9ff752` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:59.828409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ac0ff-4e7e-4403-adaa-7e05d263b5f3 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=bddd1a394f614948a558b27a0e9ff752)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:59.830088 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-814ac0ff-4e7e-4403-adaa-7e05d263b5f3 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:13:59.830088 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:13:59.830088 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:13:59.830434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ac0ff-4e7e-4403-adaa-7e05d263b5f3 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-814ac0ff-4e7e-4403-adaa-7e05d263b5f3 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Invalid input for field 'enabled': 'True' is not of type 'boolean' May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: Failed validating 'type' in schema['properties']['enabled']: May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: {'enum': [True, False, None], 'type': 'boolean'} May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: On instance['enabled']: May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: 'True': keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: Failed validating 'type' in schema['properties']['enabled']: May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: {'enum': [True, False, None], 'type': 'boolean'} May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: On instance['enabled']: May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: 'True' May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise error May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'True' is not of type 'boolean' May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application On instance['enabled']: May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application 'True' May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:13:59.842082 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:13:59.843451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:13:59.843451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:13:59.843451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:13:59.843451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:13:59.843451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:13:59.843451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:13:59.843451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:13:59.843451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:13:59.843451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:13:59.843451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:13:59.843451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:13:59.843451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:13:59.843451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 105, in patch May 28 21:13:59.843451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application validation.lazy_validate(schema.endpoint_update, endpoint) May 28 21:13:59.843451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate May 28 21:13:59.843451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) May 28 21:13:59.843451 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate May 28 21:13:59.844507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) May 28 21:13:59.844507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'enabled': 'True' is not of type 'boolean' May 28 21:13:59.844507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:13:59.844507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Failed validating 'type' in schema['properties']['enabled']: May 28 21:13:59.844507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application {'enum': [True, False, None], 'type': 'boolean'} May 28 21:13:59.844507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:13:59.844507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application On instance['enabled']: May 28 21:13:59.844507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application 'True' May 28 21:13:59.844507 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:13:59.844507 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1879/3745] 10.222.0.33 () {60 vars in 1254 bytes} [Sat May 28 21:13:59 2022] PATCH /identity/v3/endpoints/bddd1a394f614948a558b27a0e9ff752 => generated 281 bytes in 59 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) May 28 21:13:59.848009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132dad8-ee63-4a87-bee2-02493819c3ff tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:13:59.858872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132dad8-ee63-4a87-bee2-02493819c3ff tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:13:59.864214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfd6e0-74a1-4238-8f59-04bcda66e225 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b56cd967c7445ddad28ba2b3678a059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d9322e4d12641a08b572d36faabe811', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:13:59.865223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfd6e0-74a1-4238-8f59-04bcda66e225 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:13:59.865521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfd6e0-74a1-4238-8f59-04bcda66e225 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:13:59.865807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfd6e0-74a1-4238-8f59-04bcda66e225 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] PATH_INFO: `/v3/endpoints/bddd1a394f614948a558b27a0e9ff752` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:13:59.867846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfd6e0-74a1-4238-8f59-04bcda66e225 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=bddd1a394f614948a558b27a0e9ff752)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:13:59.868873 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-b5cfd6e0-74a1-4238-8f59-04bcda66e225 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:13:59.868873 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:13:59.868873 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:13:59.869938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfd6e0-74a1-4238-8f59-04bcda66e225 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:13:59.980756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b5cfd6e0-74a1-4238-8f59-04bcda66e225 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'bddd1a394f614948a558b27a0e9ff752', 'request_id': 'req-b5cfd6e0-74a1-4238-8f59-04bcda66e225'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:13:59.998348 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1867/3746] 10.222.0.33 () {58 vars in 1235 bytes} [Sat May 28 21:13:59 2022] DELETE /identity/v3/endpoints/bddd1a394f614948a558b27a0e9ff752 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:14:00.002148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ac0ff-4e7e-4403-adaa-7e05d263b5f3 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:00.017398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ac0ff-4e7e-4403-adaa-7e05d263b5f3 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:00.039798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6e336-9abe-4108-9904-54cc4647d700 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b56cd967c7445ddad28ba2b3678a059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d9322e4d12641a08b572d36faabe811', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:00.041408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6e336-9abe-4108-9904-54cc4647d700 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:00.042160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6e336-9abe-4108-9904-54cc4647d700 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:00.042708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6e336-9abe-4108-9904-54cc4647d700 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] PATH_INFO: `/v3/regions/tempest-region-630657403` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:00.045624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6e336-9abe-4108-9904-54cc4647d700 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-630657403)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:00.045624 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-fcd6e336-9abe-4108-9904-54cc4647d700 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:14:00.045624 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:14:00.045624 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:14:00.049802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6e336-9abe-4108-9904-54cc4647d700 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:00.196222 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1880/3747] 10.222.0.33 () {58 vars in 1205 bytes} [Sat May 28 21:14:00 2022] DELETE /identity/v3/regions/tempest-region-630657403 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:00.201505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfd6e0-74a1-4238-8f59-04bcda66e225 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:00.224544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfd6e0-74a1-4238-8f59-04bcda66e225 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:00.247507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ab1d4-440a-48e3-aa36-0e7714747eff tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b56cd967c7445ddad28ba2b3678a059', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d9322e4d12641a08b572d36faabe811', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:00.248488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ab1d4-440a-48e3-aa36-0e7714747eff tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:00.248851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ab1d4-440a-48e3-aa36-0e7714747eff tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:00.249245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ab1d4-440a-48e3-aa36-0e7714747eff tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] PATH_INFO: `/v3/services/5ab6d384ac8e479ea326547c31c45c2f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:00.250253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ab1d4-440a-48e3-aa36-0e7714747eff tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorizing `identity:delete_service(service_id=5ab6d384ac8e479ea326547c31c45c2f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:00.251464 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-6d7ab1d4-440a-48e3-aa36-0e7714747eff tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:14:00.251464 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:14:00.251464 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:14:00.252871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ab1d4-440a-48e3-aa36-0e7714747eff tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:00.373754 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1868/3748] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:00 2022] DELETE /identity/v3/services/5ab6d384ac8e479ea326547c31c45c2f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:00.376374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6e336-9abe-4108-9904-54cc4647d700 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:00.491970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6e336-9abe-4108-9904-54cc4647d700 tempest-EndpointsNegativeTestJSON-406609575 tempest-EndpointsNegativeTestJSON-406609575-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:00.515749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabf8d6-acac-4cab-b872-099a7e778085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:00.517743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabf8d6-acac-4cab-b872-099a7e778085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:00.518842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabf8d6-acac-4cab-b872-099a7e778085 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:00.519675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaabf8d6-acac-4cab-b872-099a7e778085 admin admin] PATH_INFO: `/v3/users/562c62aa43b64f4386a02ce93ba3fecc` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:00.524424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabf8d6-acac-4cab-b872-099a7e778085 admin admin] RBAC: Authorizing `identity:delete_user(user_id=562c62aa43b64f4386a02ce93ba3fecc)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:00.526800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaabf8d6-acac-4cab-b872-099a7e778085 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:00.555364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:00.753154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eaabf8d6-acac-4cab-b872-099a7e778085 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '562c62aa43b64f4386a02ce93ba3fecc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:00.761862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eaabf8d6-acac-4cab-b872-099a7e778085 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '562c62aa43b64f4386a02ce93ba3fecc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:00.762851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eaabf8d6-acac-4cab-b872-099a7e778085 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '562c62aa43b64f4386a02ce93ba3fecc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:00.764155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eaabf8d6-acac-4cab-b872-099a7e778085 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '562c62aa43b64f4386a02ce93ba3fecc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:00.860202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eaabf8d6-acac-4cab-b872-099a7e778085 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '562c62aa43b64f4386a02ce93ba3fecc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:00.887941 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1881/3749] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:00 2022] DELETE /identity/v3/users/562c62aa43b64f4386a02ce93ba3fecc => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:00.892465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc02d84-c686-4122-9bb6-97515b85cb7d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:00.892799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc02d84-c686-4122-9bb6-97515b85cb7d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:00.893221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc02d84-c686-4122-9bb6-97515b85cb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:00.914883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8cc02d84-c686-4122-9bb6-97515b85cb7d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:00.916398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8cc02d84-c686-4122-9bb6-97515b85cb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:00.931348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:01.040662 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1869/3750] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:01.063567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabf8d6-acac-4cab-b872-099a7e778085 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:01.065267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eaabf8d6-acac-4cab-b872-099a7e778085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:01.211537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaabf8d6-acac-4cab-b872-099a7e778085 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:01.216737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a22627-c9cc-4ea1-ab29-c7e5ca7f4b63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:01.217612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a22627-c9cc-4ea1-ab29-c7e5ca7f4b63 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:01.218013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a22627-c9cc-4ea1-ab29-c7e5ca7f4b63 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:01.218368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a22627-c9cc-4ea1-ab29-c7e5ca7f4b63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:01.268354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a22627-c9cc-4ea1-ab29-c7e5ca7f4b63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:01.274143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a22627-c9cc-4ea1-ab29-c7e5ca7f4b63 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:01.317967 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1882/3751] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:14:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:14:01.951880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc02d84-c686-4122-9bb6-97515b85cb7d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:01.955067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8cc02d84-c686-4122-9bb6-97515b85cb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:01.970417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc02d84-c686-4122-9bb6-97515b85cb7d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:01.976236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:01.978993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:01.979494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:01.979997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930 admin admin] PATH_INFO: `/v3/projects/c7eb5cbea535490c977f54d7d587b668` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:01.981825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7eb5cbea535490c977f54d7d587b668)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:01.985804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:02.142020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7eb5cbea535490c977f54d7d587b668', 'request_id': 'req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:02.147669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7eb5cbea535490c977f54d7d587b668', 'request_id': 'req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:02.214915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7eb5cbea535490c977f54d7d587b668', 'request_id': 'req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:02.231534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930 admin admin] The token cache is being invalidate because project c7eb5cbea535490c977f54d7d587b668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:02.234435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:02.240561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:02.243095 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1870/3752] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:01 2022] DELETE /identity/v3/projects/c7eb5cbea535490c977f54d7d587b668 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:02.250263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a22627-c9cc-4ea1-ab29-c7e5ca7f4b63 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:02.264272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a22627-c9cc-4ea1-ab29-c7e5ca7f4b63 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:02.400694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5acce9-c07b-47eb-a29b-c133001df8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:02.407091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5acce9-c07b-47eb-a29b-c133001df8f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:02.407091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5acce9-c07b-47eb-a29b-c133001df8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:02.407091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5acce9-c07b-47eb-a29b-c133001df8f4 admin admin] PATH_INFO: `/v3/users/1b56cd967c7445ddad28ba2b3678a059` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:02.410784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5acce9-c07b-47eb-a29b-c133001df8f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b56cd967c7445ddad28ba2b3678a059)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:02.418948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5acce9-c07b-47eb-a29b-c133001df8f4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:02.565237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:02.761144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5b5acce9-c07b-47eb-a29b-c133001df8f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b56cd967c7445ddad28ba2b3678a059'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:02.777282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5b5acce9-c07b-47eb-a29b-c133001df8f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b56cd967c7445ddad28ba2b3678a059'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:02.779858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5b5acce9-c07b-47eb-a29b-c133001df8f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b56cd967c7445ddad28ba2b3678a059'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:02.781327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5b5acce9-c07b-47eb-a29b-c133001df8f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b56cd967c7445ddad28ba2b3678a059'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:02.924358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5b5acce9-c07b-47eb-a29b-c133001df8f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b56cd967c7445ddad28ba2b3678a059'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:02.955854 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1883/3753] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:02 2022] DELETE /identity/v3/users/1b56cd967c7445ddad28ba2b3678a059 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:03.676475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:03.677690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:03.697323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4bde2e-eb28-418c-9b34-e81f9d6ff930 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:03.801946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49425a8-ee3a-493a-a4c0-52946a6c7ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:03.801946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49425a8-ee3a-493a-a4c0-52946a6c7ef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:03.801946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49425a8-ee3a-493a-a4c0-52946a6c7ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:03.801946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49425a8-ee3a-493a-a4c0-52946a6c7ef0 admin admin] PATH_INFO: `/v3/projects/0d9322e4d12641a08b572d36faabe811` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:03.802523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49425a8-ee3a-493a-a4c0-52946a6c7ef0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d9322e4d12641a08b572d36faabe811)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:03.803123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49425a8-ee3a-493a-a4c0-52946a6c7ef0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:03.986143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:04.001705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a49425a8-ee3a-493a-a4c0-52946a6c7ef0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d9322e4d12641a08b572d36faabe811', 'request_id': 'req-a49425a8-ee3a-493a-a4c0-52946a6c7ef0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:04.009979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a49425a8-ee3a-493a-a4c0-52946a6c7ef0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d9322e4d12641a08b572d36faabe811', 'request_id': 'req-a49425a8-ee3a-493a-a4c0-52946a6c7ef0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:04.135372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a49425a8-ee3a-493a-a4c0-52946a6c7ef0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d9322e4d12641a08b572d36faabe811', 'request_id': 'req-a49425a8-ee3a-493a-a4c0-52946a6c7ef0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:04.150065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a49425a8-ee3a-493a-a4c0-52946a6c7ef0 admin admin] The token cache is being invalidate because project 0d9322e4d12641a08b572d36faabe811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:04.150638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a49425a8-ee3a-493a-a4c0-52946a6c7ef0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:04.151548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a49425a8-ee3a-493a-a4c0-52946a6c7ef0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:04.158885 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1871/3754] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:03 2022] DELETE /identity/v3/projects/0d9322e4d12641a08b572d36faabe811 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:04.159097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbc5e1-e4ad-4bbb-b573-12d4f2d870fb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:04.159531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbc5e1-e4ad-4bbb-b573-12d4f2d870fb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:04.159531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbc5e1-e4ad-4bbb-b573-12d4f2d870fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:04.197056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a3dbc5e1-e4ad-4bbb-b573-12d4f2d870fb None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:04.198402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a3dbc5e1-e4ad-4bbb-b573-12d4f2d870fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:04.410967 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1884/3755] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:04.418325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e5bf7-bcb2-4670-a77e-fbb2d82401ef None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:04.418957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e5bf7-bcb2-4670-a77e-fbb2d82401ef None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:04.419398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e5bf7-bcb2-4670-a77e-fbb2d82401ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:04.473036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d95e5bf7-bcb2-4670-a77e-fbb2d82401ef None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:04.474492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d95e5bf7-bcb2-4670-a77e-fbb2d82401ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:04.497160 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1872/3756] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:04.501380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbc5e1-e4ad-4bbb-b573-12d4f2d870fb None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:04.523202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbc5e1-e4ad-4bbb-b573-12d4f2d870fb None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:04.529297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc533da2-c292-42bd-926f-266f950cab05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:04.530136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc533da2-c292-42bd-926f-266f950cab05 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:04.530504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc533da2-c292-42bd-926f-266f950cab05 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:04.531377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc533da2-c292-42bd-926f-266f950cab05 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:04.532260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc533da2-c292-42bd-926f-266f950cab05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:14:04.532574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc533da2-c292-42bd-926f-266f950cab05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:04.534324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc533da2-c292-42bd-926f-266f950cab05 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:04.551354 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1885/3757] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:04.556265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e5bf7-bcb2-4670-a77e-fbb2d82401ef None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:04.566768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e5bf7-bcb2-4670-a77e-fbb2d82401ef None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:04.575889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68838aca-8670-4e4d-a721-5066aa52f254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:04.576659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68838aca-8670-4e4d-a721-5066aa52f254 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:04.577152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68838aca-8670-4e4d-a721-5066aa52f254 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:04.578050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68838aca-8670-4e4d-a721-5066aa52f254 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:04.579055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68838aca-8670-4e4d-a721-5066aa52f254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:04.581509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68838aca-8670-4e4d-a721-5066aa52f254 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:04.616772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf14ea-2002-4ebe-850d-aea856a8bb2f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:04.617030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf14ea-2002-4ebe-850d-aea856a8bb2f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:04.618064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf14ea-2002-4ebe-850d-aea856a8bb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:04.648479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-99bf14ea-2002-4ebe-850d-aea856a8bb2f None None] MFA Rules not processed for user `25b2f1524f7f48149cbc5d2c694a9e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:04.649927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-99bf14ea-2002-4ebe-850d-aea856a8bb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:04.697420 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1873/3758] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:14:04 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:04.705438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68838aca-8670-4e4d-a721-5066aa52f254 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:04.741513 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1886/3759] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:04.765431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794c922-6d34-4acf-b5a7-61145e4d1c33 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:04.765431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794c922-6d34-4acf-b5a7-61145e4d1c33 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:04.766413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f794c922-6d34-4acf-b5a7-61145e4d1c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:04.798062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f794c922-6d34-4acf-b5a7-61145e4d1c33 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:04.799395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f794c922-6d34-4acf-b5a7-61145e4d1c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:04.813215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68838aca-8670-4e4d-a721-5066aa52f254 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:04.827983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd47f8-0e1a-4ce4-8458-ea09e1bf7754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:04.828906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd47f8-0e1a-4ce4-8458-ea09e1bf7754 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:04.830164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd47f8-0e1a-4ce4-8458-ea09e1bf7754 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:04.830393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebd47f8-0e1a-4ce4-8458-ea09e1bf7754 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:04.831100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd47f8-0e1a-4ce4-8458-ea09e1bf7754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:04.833088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebd47f8-0e1a-4ce4-8458-ea09e1bf7754 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:04.842222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:04.902937 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1887/3760] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:14:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:14:04.910689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794c922-6d34-4acf-b5a7-61145e4d1c33 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:04.912310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f794c922-6d34-4acf-b5a7-61145e4d1c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:04.972841 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1874/3761] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:14:04 2022] POST /identity/v3/users => generated 294 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:04.980736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd47f8-0e1a-4ce4-8458-ea09e1bf7754 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:04.999366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebd47f8-0e1a-4ce4-8458-ea09e1bf7754 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:05.005215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856004b8-6d73-4941-b57b-d2003f0ddc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:05.005948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856004b8-6d73-4941-b57b-d2003f0ddc93 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:05.006510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856004b8-6d73-4941-b57b-d2003f0ddc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:05.007434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856004b8-6d73-4941-b57b-d2003f0ddc93 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:05.008773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856004b8-6d73-4941-b57b-d2003f0ddc93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:05.012786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856004b8-6d73-4941-b57b-d2003f0ddc93 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:05.040476 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1875/3762] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:05.042562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856004b8-6d73-4941-b57b-d2003f0ddc93 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:05.055368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856004b8-6d73-4941-b57b-d2003f0ddc93 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:05.061970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808a0c1-3d3d-4742-8bb0-15ec1f932a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:05.062649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808a0c1-3d3d-4742-8bb0-15ec1f932a3d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:05.062941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808a0c1-3d3d-4742-8bb0-15ec1f932a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:05.063220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808a0c1-3d3d-4742-8bb0-15ec1f932a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:05.065831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b808a0c1-3d3d-4742-8bb0-15ec1f932a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:05.067562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b808a0c1-3d3d-4742-8bb0-15ec1f932a3d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:05.084247 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1876/3763] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:05.088253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808a0c1-3d3d-4742-8bb0-15ec1f932a3d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:05.090777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f794c922-6d34-4acf-b5a7-61145e4d1c33 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:05.098412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec700fd-78f7-427c-b2e3-d7be13624631 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:05.099447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec700fd-78f7-427c-b2e3-d7be13624631 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:05.099767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec700fd-78f7-427c-b2e3-d7be13624631 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:05.099997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808a0c1-3d3d-4742-8bb0-15ec1f932a3d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:05.100124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec700fd-78f7-427c-b2e3-d7be13624631 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:05.112466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec700fd-78f7-427c-b2e3-d7be13624631 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:05.113276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934993a4-9875-4452-be9d-2901733a0cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:05.114199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec700fd-78f7-427c-b2e3-d7be13624631 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:05.117084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934993a4-9875-4452-be9d-2901733a0cbc admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:05.117084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934993a4-9875-4452-be9d-2901733a0cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:05.117084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934993a4-9875-4452-be9d-2901733a0cbc admin admin] PATH_INFO: `/v3/projects/cc8a398d1db349309404becc9a8ddd34/users/6a683e3618c5474699e5abc6560cce30/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:05.125362 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1888/3764] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:14:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 21:14:05.137147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934993a4-9875-4452-be9d-2901733a0cbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc8a398d1db349309404becc9a8ddd34, user_id=6a683e3618c5474699e5abc6560cce30, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:05.143959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934993a4-9875-4452-be9d-2901733a0cbc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:05.212964 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1877/3765] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:05 2022] PUT /identity/v3/projects/cc8a398d1db349309404becc9a8ddd34/users/6a683e3618c5474699e5abc6560cce30/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:05.225214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d727d4-8cca-46aa-b320-e2ff8bad37c4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:05.225740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d727d4-8cca-46aa-b320-e2ff8bad37c4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:05.226879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d727d4-8cca-46aa-b320-e2ff8bad37c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:05.295571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-72d727d4-8cca-46aa-b320-e2ff8bad37c4 None None] MFA Rules not processed for user `6a683e3618c5474699e5abc6560cce30`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:05.296851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-72d727d4-8cca-46aa-b320-e2ff8bad37c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:05.469552 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1889/3766] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:05.473973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934993a4-9875-4452-be9d-2901733a0cbc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:05.556108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=4,keystone:INSERT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:05.595080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934993a4-9875-4452-be9d-2901733a0cbc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:05.612795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544c889-de48-486d-8649-e02ae80d9032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:05.613824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544c889-de48-486d-8649-e02ae80d9032 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:05.614428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544c889-de48-486d-8649-e02ae80d9032 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:05.614879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544c889-de48-486d-8649-e02ae80d9032 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:05.617195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544c889-de48-486d-8649-e02ae80d9032 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:05.628666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544c889-de48-486d-8649-e02ae80d9032 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:05.776678 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1878/3767] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:14:05 2022] POST /identity/v3/projects => generated 387 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:05.779719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d727d4-8cca-46aa-b320-e2ff8bad37c4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:05.793120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d727d4-8cca-46aa-b320-e2ff8bad37c4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:05.842564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:05.890115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23260db4-2fe9-4128-ab3d-b4b439f4b79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:05.890806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23260db4-2fe9-4128-ab3d-b4b439f4b79e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:05.891195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23260db4-2fe9-4128-ab3d-b4b439f4b79e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:05.891476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23260db4-2fe9-4128-ab3d-b4b439f4b79e admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:05.892800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23260db4-2fe9-4128-ab3d-b4b439f4b79e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:05.894907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23260db4-2fe9-4128-ab3d-b4b439f4b79e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:06.017761 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1890/3768] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:14:05 2022] POST /identity/v3/users => generated 300 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:06.023176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544c889-de48-486d-8649-e02ae80d9032 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:06.040711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544c889-de48-486d-8649-e02ae80d9032 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:06.056882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91244404-e21b-41b5-89d6-e73c0decbc8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:06.057606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91244404-e21b-41b5-89d6-e73c0decbc8d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:06.057921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91244404-e21b-41b5-89d6-e73c0decbc8d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:06.058204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91244404-e21b-41b5-89d6-e73c0decbc8d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:06.058950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91244404-e21b-41b5-89d6-e73c0decbc8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:06.060591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91244404-e21b-41b5-89d6-e73c0decbc8d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:06.117577 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1879/3769] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:06.119777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23260db4-2fe9-4128-ab3d-b4b439f4b79e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:06.145868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23260db4-2fe9-4128-ab3d-b4b439f4b79e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:06.163342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85424fe0-3176-47af-9c03-254d71d862da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:06.164821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85424fe0-3176-47af-9c03-254d71d862da admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:06.165099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85424fe0-3176-47af-9c03-254d71d862da admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:06.165376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85424fe0-3176-47af-9c03-254d71d862da admin admin] PATH_INFO: `/v3/projects/aa527a522e5d442d81cf13d2748e6b84/users/cb3c9363526544ac9ca04d2c84c90b4f/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:06.223550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85424fe0-3176-47af-9c03-254d71d862da admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa527a522e5d442d81cf13d2748e6b84, user_id=cb3c9363526544ac9ca04d2c84c90b4f, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:06.231831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85424fe0-3176-47af-9c03-254d71d862da admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:06.372905 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1891/3770] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:06 2022] PUT /identity/v3/projects/aa527a522e5d442d81cf13d2748e6b84/users/cb3c9363526544ac9ca04d2c84c90b4f/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:06.376995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91244404-e21b-41b5-89d6-e73c0decbc8d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:06.393441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91244404-e21b-41b5-89d6-e73c0decbc8d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:06.518739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfac50-6770-46ff-add1-c69a91340186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:06.519975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfac50-6770-46ff-add1-c69a91340186 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:06.520428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfac50-6770-46ff-add1-c69a91340186 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:06.520897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfac50-6770-46ff-add1-c69a91340186 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:06.521787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfac50-6770-46ff-add1-c69a91340186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:06.525913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfac50-6770-46ff-add1-c69a91340186 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:06.540546 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1880/3771] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:06.544230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85424fe0-3176-47af-9c03-254d71d862da admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:06.563383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85424fe0-3176-47af-9c03-254d71d862da admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:06.570822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3a2eb-a304-4e10-a3ef-4cb96462d513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:06.571745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3a2eb-a304-4e10-a3ef-4cb96462d513 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:06.572055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3a2eb-a304-4e10-a3ef-4cb96462d513 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:06.572325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f3a2eb-a304-4e10-a3ef-4cb96462d513 admin admin] PATH_INFO: `/v3/projects/aa527a522e5d442d81cf13d2748e6b84/users/cb3c9363526544ac9ca04d2c84c90b4f/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:06.582358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f3a2eb-a304-4e10-a3ef-4cb96462d513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa527a522e5d442d81cf13d2748e6b84, user_id=cb3c9363526544ac9ca04d2c84c90b4f, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:06.585106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f3a2eb-a304-4e10-a3ef-4cb96462d513 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:06.611277 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1892/3772] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:06 2022] PUT /identity/v3/projects/aa527a522e5d442d81cf13d2748e6b84/users/cb3c9363526544ac9ca04d2c84c90b4f/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:06.624236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cade79-e3ee-4471-98db-4d4093f390ce None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:06.624236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cade79-e3ee-4471-98db-4d4093f390ce None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:06.624236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cade79-e3ee-4471-98db-4d4093f390ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:06.688786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a5cade79-e3ee-4471-98db-4d4093f390ce None None] MFA Rules not processed for user `cb3c9363526544ac9ca04d2c84c90b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:06.690204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a5cade79-e3ee-4471-98db-4d4093f390ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:06.826246 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1881/3773] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:06.831255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3a2eb-a304-4e10-a3ef-4cb96462d513 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:06.855131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f3a2eb-a304-4e10-a3ef-4cb96462d513 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:06.869783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e9565-0574-4f93-aaf4-c191bc327009 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb3c9363526544ac9ca04d2c84c90b4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa527a522e5d442d81cf13d2748e6b84', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:06.870964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e9565-0574-4f93-aaf4-c191bc327009 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:06.871410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e9565-0574-4f93-aaf4-c191bc327009 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:06.871999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e9565-0574-4f93-aaf4-c191bc327009 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:06.876293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e9565-0574-4f93-aaf4-c191bc327009 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:06.876293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e9565-0574-4f93-aaf4-c191bc327009 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:06.997277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cade79-e3ee-4471-98db-4d4093f390ce None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:06.997852 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1893/3774] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:14:06 2022] POST /identity/v3/projects => generated 359 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:07.020452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cade79-e3ee-4471-98db-4d4093f390ce None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:07.098190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:07.135527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9037b2-5aea-45d2-8bd4-b94525474d5c tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb3c9363526544ac9ca04d2c84c90b4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa527a522e5d442d81cf13d2748e6b84', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:07.136579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9037b2-5aea-45d2-8bd4-b94525474d5c tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:07.136872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9037b2-5aea-45d2-8bd4-b94525474d5c tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:07.137153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9037b2-5aea-45d2-8bd4-b94525474d5c tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] PATH_INFO: `/v3/projects/3903290dcb02480b8975fb9a5eed6d19/tags/tempest-tag-1977656370` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:07.138869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9037b2-5aea-45d2-8bd4-b94525474d5c tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorizing `identity:create_project_tag(project_id=3903290dcb02480b8975fb9a5eed6d19, value=tempest-tag-1977656370)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:07.142223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9037b2-5aea-45d2-8bd4-b94525474d5c tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:07.309329 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-5c9037b2-5aea-45d2-8bd4-b94525474d5c tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-1977656370. To ensure interoperability, identifiers should be a valid uuid. May 28 21:14:07.309329 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' May 28 21:14:07.309329 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:14:07.334274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e9565-0574-4f93-aaf4-c191bc327009 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:07.348526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e9565-0574-4f93-aaf4-c191bc327009 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:07.349152 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1882/3775] 10.222.0.33 () {60 vars in 1332 bytes} [Sat May 28 21:14:06 2022] PUT /identity/v3/projects/3903290dcb02480b8975fb9a5eed6d19/tags/tempest-tag-1977656370 => generated 0 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 291 bytes (1 switches on core 0) May 28 21:14:07.358959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac184aec-afe7-44e1-a4dc-ded3aaa2bb10 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb3c9363526544ac9ca04d2c84c90b4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa527a522e5d442d81cf13d2748e6b84', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:07.360009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac184aec-afe7-44e1-a4dc-ded3aaa2bb10 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:07.360694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac184aec-afe7-44e1-a4dc-ded3aaa2bb10 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:07.361068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac184aec-afe7-44e1-a4dc-ded3aaa2bb10 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] PATH_INFO: `/v3/projects/3903290dcb02480b8975fb9a5eed6d19/tags/tempest-tag-1977656370` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:07.394071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac184aec-afe7-44e1-a4dc-ded3aaa2bb10 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=3903290dcb02480b8975fb9a5eed6d19, value=tempest-tag-1977656370)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:07.394071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac184aec-afe7-44e1-a4dc-ded3aaa2bb10 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:07.426254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9037b2-5aea-45d2-8bd4-b94525474d5c tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:07.427001 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1894/3776] 10.222.0.33 () {58 vars in 1313 bytes} [Sat May 28 21:14:07 2022] GET /identity/v3/projects/3903290dcb02480b8975fb9a5eed6d19/tags/tempest-tag-1977656370 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:14:07.442804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9037b2-5aea-45d2-8bd4-b94525474d5c tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:07.449459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5bf41-77d9-445b-98ed-a443dcec48b8 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb3c9363526544ac9ca04d2c84c90b4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa527a522e5d442d81cf13d2748e6b84', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:07.450740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5bf41-77d9-445b-98ed-a443dcec48b8 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:07.451242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5bf41-77d9-445b-98ed-a443dcec48b8 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:07.452311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a5bf41-77d9-445b-98ed-a443dcec48b8 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] PATH_INFO: `/v3/projects/3903290dcb02480b8975fb9a5eed6d19/tags` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:07.453929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5bf41-77d9-445b-98ed-a443dcec48b8 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorizing `identity:update_project_tags(project_id=3903290dcb02480b8975fb9a5eed6d19)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:07.459791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a5bf41-77d9-445b-98ed-a443dcec48b8 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:07.622016 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1883/3777] 10.222.0.33 () {60 vars in 1264 bytes} [Sat May 28 21:14:07 2022] PUT /identity/v3/projects/3903290dcb02480b8975fb9a5eed6d19/tags => generated 141 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:07.626196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac184aec-afe7-44e1-a4dc-ded3aaa2bb10 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:07.642187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac184aec-afe7-44e1-a4dc-ded3aaa2bb10 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:07.652654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6027ff0-122d-4451-ab93-3439b391f7c2 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb3c9363526544ac9ca04d2c84c90b4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa527a522e5d442d81cf13d2748e6b84', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:07.655418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6027ff0-122d-4451-ab93-3439b391f7c2 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:07.655818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6027ff0-122d-4451-ab93-3439b391f7c2 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:07.657214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6027ff0-122d-4451-ab93-3439b391f7c2 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] PATH_INFO: `/v3/projects/3903290dcb02480b8975fb9a5eed6d19/tags` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:07.694613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6027ff0-122d-4451-ab93-3439b391f7c2 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=3903290dcb02480b8975fb9a5eed6d19)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:07.697036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6027ff0-122d-4451-ab93-3439b391f7c2 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:07.723821 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1895/3778] 10.222.0.33 () {58 vars in 1244 bytes} [Sat May 28 21:14:07 2022] GET /identity/v3/projects/3903290dcb02480b8975fb9a5eed6d19/tags => generated 141 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:07.728568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5bf41-77d9-445b-98ed-a443dcec48b8 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:07.753870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a5bf41-77d9-445b-98ed-a443dcec48b8 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:07.771795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21e027-ec4a-4947-b8f6-6d3ac5811427 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb3c9363526544ac9ca04d2c84c90b4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa527a522e5d442d81cf13d2748e6b84', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:07.771795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21e027-ec4a-4947-b8f6-6d3ac5811427 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:07.772162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21e027-ec4a-4947-b8f6-6d3ac5811427 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:07.772394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b21e027-ec4a-4947-b8f6-6d3ac5811427 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] PATH_INFO: `/v3/projects/3903290dcb02480b8975fb9a5eed6d19/tags/tempest-tag-412530565` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:07.775039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21e027-ec4a-4947-b8f6-6d3ac5811427 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorizing `identity:delete_project_tag(project_id=3903290dcb02480b8975fb9a5eed6d19, value=tempest-tag-412530565)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:07.779936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b21e027-ec4a-4947-b8f6-6d3ac5811427 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:07.967115 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-5b21e027-ec4a-4947-b8f6-6d3ac5811427 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-tag-412530565. To ensure interoperability, identifiers should be a valid uuid. May 28 21:14:07.967115 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' May 28 21:14:07.967115 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:14:07.977379 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1884/3779] 10.222.0.33 () {58 vars in 1313 bytes} [Sat May 28 21:14:07 2022] DELETE /identity/v3/projects/3903290dcb02480b8975fb9a5eed6d19/tags/tempest-tag-412530565 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:07.982055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6027ff0-122d-4451-ab93-3439b391f7c2 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:07.994675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6027ff0-122d-4451-ab93-3439b391f7c2 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:08.013399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb0ea3-d0ff-4af1-a9ef-163a277212ec tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb3c9363526544ac9ca04d2c84c90b4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa527a522e5d442d81cf13d2748e6b84', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:08.014950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb0ea3-d0ff-4af1-a9ef-163a277212ec tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:08.015478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb0ea3-d0ff-4af1-a9ef-163a277212ec tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:08.015864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb0ea3-d0ff-4af1-a9ef-163a277212ec tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] PATH_INFO: `/v3/projects/3903290dcb02480b8975fb9a5eed6d19/tags/tempest-tag-412530565` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:08.047897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb0ea3-d0ff-4af1-a9ef-163a277212ec tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorizing `identity:get_project_tag(project_id=3903290dcb02480b8975fb9a5eed6d19, value=tempest-tag-412530565)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:08.051012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb0ea3-d0ff-4af1-a9ef-163a277212ec tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:08.092713 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-d1fb0ea3-d0ff-4af1-a9ef-163a277212ec tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Could not find project tag: tempest-tag-412530565.: keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-412530565. May 28 21:14:08.092713 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:14:08.092713 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:14:08.092713 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:14:08.092713 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:14:08.092713 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:14:08.092713 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:14:08.092713 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:14:08.092713 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:14:08.092713 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:14:08.092713 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:14:08.092713 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:14:08.092713 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 284, in get May 28 21:14:08.092713 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_project_tag(project_id, value) May 28 21:14:08.101034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:14:08.101034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:14:08.101034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 960, in get_project_tag May 28 21:14:08.101034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectTagNotFound(project_tag=tag_name) May 28 21:14:08.101034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectTagNotFound: Could not find project tag: tempest-tag-412530565. May 28 21:14:08.101034 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:14:08.101034 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1896/3780] 10.222.0.33 () {58 vars in 1310 bytes} [Sat May 28 21:14:07 2022] GET /identity/v3/projects/3903290dcb02480b8975fb9a5eed6d19/tags/tempest-tag-412530565 => generated 106 bytes in 114 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) May 28 21:14:08.103267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21e027-ec4a-4947-b8f6-6d3ac5811427 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:08.118757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b21e027-ec4a-4947-b8f6-6d3ac5811427 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:08.123813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879e8ea-1eca-4f01-9f00-f8831c300e0f tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb3c9363526544ac9ca04d2c84c90b4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa527a522e5d442d81cf13d2748e6b84', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:08.124842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879e8ea-1eca-4f01-9f00-f8831c300e0f tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:08.125136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879e8ea-1eca-4f01-9f00-f8831c300e0f tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:08.125417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879e8ea-1eca-4f01-9f00-f8831c300e0f tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] PATH_INFO: `/v3/projects/3903290dcb02480b8975fb9a5eed6d19/tags` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:08.126712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8879e8ea-1eca-4f01-9f00-f8831c300e0f tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorizing `identity:delete_project_tags(project_id=3903290dcb02480b8975fb9a5eed6d19)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:08.129178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8879e8ea-1eca-4f01-9f00-f8831c300e0f tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:08.283054 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1885/3781] 10.222.0.33 () {58 vars in 1247 bytes} [Sat May 28 21:14:08 2022] DELETE /identity/v3/projects/3903290dcb02480b8975fb9a5eed6d19/tags => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:08.294574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb0ea3-d0ff-4af1-a9ef-163a277212ec tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:08.311625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb0ea3-d0ff-4af1-a9ef-163a277212ec tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:08.326097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74144e82-0f87-44a8-bf8a-de5072a7d840 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb3c9363526544ac9ca04d2c84c90b4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa527a522e5d442d81cf13d2748e6b84', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:08.327590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74144e82-0f87-44a8-bf8a-de5072a7d840 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:08.327950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74144e82-0f87-44a8-bf8a-de5072a7d840 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:08.328237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74144e82-0f87-44a8-bf8a-de5072a7d840 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] PATH_INFO: `/v3/projects/3903290dcb02480b8975fb9a5eed6d19/tags` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:08.359431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74144e82-0f87-44a8-bf8a-de5072a7d840 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorizing `identity:list_project_tags(project_id=3903290dcb02480b8975fb9a5eed6d19)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:08.363361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74144e82-0f87-44a8-bf8a-de5072a7d840 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:08.384222 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1897/3782] 10.222.0.33 () {58 vars in 1244 bytes} [Sat May 28 21:14:08 2022] GET /identity/v3/projects/3903290dcb02480b8975fb9a5eed6d19/tags => generated 66 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:14:08.385329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879e8ea-1eca-4f01-9f00-f8831c300e0f tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:08.404759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879e8ea-1eca-4f01-9f00-f8831c300e0f tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:08.417281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a08bdc-c9fa-4b9a-b4c0-a3b3df0137b3 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb3c9363526544ac9ca04d2c84c90b4f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aa527a522e5d442d81cf13d2748e6b84', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:08.418201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a08bdc-c9fa-4b9a-b4c0-a3b3df0137b3 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:08.418492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a08bdc-c9fa-4b9a-b4c0-a3b3df0137b3 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:08.418888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a08bdc-c9fa-4b9a-b4c0-a3b3df0137b3 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] PATH_INFO: `/v3/projects/3903290dcb02480b8975fb9a5eed6d19` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:08.420639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a08bdc-c9fa-4b9a-b4c0-a3b3df0137b3 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3903290dcb02480b8975fb9a5eed6d19)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:08.424180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a08bdc-c9fa-4b9a-b4c0-a3b3df0137b3 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:08.586461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:08.610232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0a08bdc-c9fa-4b9a-b4c0-a3b3df0137b3 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3903290dcb02480b8975fb9a5eed6d19', 'request_id': 'req-c0a08bdc-c9fa-4b9a-b4c0-a3b3df0137b3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:08.628105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0a08bdc-c9fa-4b9a-b4c0-a3b3df0137b3 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3903290dcb02480b8975fb9a5eed6d19', 'request_id': 'req-c0a08bdc-c9fa-4b9a-b4c0-a3b3df0137b3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:08.725707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0a08bdc-c9fa-4b9a-b4c0-a3b3df0137b3 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3903290dcb02480b8975fb9a5eed6d19', 'request_id': 'req-c0a08bdc-c9fa-4b9a-b4c0-a3b3df0137b3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:08.756147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0a08bdc-c9fa-4b9a-b4c0-a3b3df0137b3 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] The token cache is being invalidate because project 3903290dcb02480b8975fb9a5eed6d19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:08.756448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0a08bdc-c9fa-4b9a-b4c0-a3b3df0137b3 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:08.760538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0a08bdc-c9fa-4b9a-b4c0-a3b3df0137b3 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:08.760538 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1886/3783] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:08 2022] DELETE /identity/v3/projects/3903290dcb02480b8975fb9a5eed6d19 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:08.766815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74144e82-0f87-44a8-bf8a-de5072a7d840 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:08.771385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-74144e82-0f87-44a8-bf8a-de5072a7d840 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:08.908879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:08.985925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74144e82-0f87-44a8-bf8a-de5072a7d840 tempest-IdentityV3ProjectTagsTest-1116728576 tempest-IdentityV3ProjectTagsTest-1116728576-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:09.183908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafc1f8-3277-4bdd-9835-21b88d16ab55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:09.185095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafc1f8-3277-4bdd-9835-21b88d16ab55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:09.186363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafc1f8-3277-4bdd-9835-21b88d16ab55 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:09.188133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafc1f8-3277-4bdd-9835-21b88d16ab55 admin admin] PATH_INFO: `/v3/users/6a683e3618c5474699e5abc6560cce30` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:09.191220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cafc1f8-3277-4bdd-9835-21b88d16ab55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a683e3618c5474699e5abc6560cce30)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:09.195497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cafc1f8-3277-4bdd-9835-21b88d16ab55 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:09.481594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3cafc1f8-3277-4bdd-9835-21b88d16ab55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a683e3618c5474699e5abc6560cce30'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:09.524123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3cafc1f8-3277-4bdd-9835-21b88d16ab55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a683e3618c5474699e5abc6560cce30'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:09.525223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3cafc1f8-3277-4bdd-9835-21b88d16ab55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a683e3618c5474699e5abc6560cce30'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:09.526542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3cafc1f8-3277-4bdd-9835-21b88d16ab55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a683e3618c5474699e5abc6560cce30'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:09.604018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3cafc1f8-3277-4bdd-9835-21b88d16ab55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a683e3618c5474699e5abc6560cce30'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:09.640846 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1898/3784] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:08 2022] DELETE /identity/v3/users/6a683e3618c5474699e5abc6560cce30 => generated 0 bytes in 875 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:09.645520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d35f2-bfb8-40a3-8e0e-d83edc52e6a5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:09.646115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d35f2-bfb8-40a3-8e0e-d83edc52e6a5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:09.646115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d35f2-bfb8-40a3-8e0e-d83edc52e6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:09.677271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-813d35f2-bfb8-40a3-8e0e-d83edc52e6a5 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:09.678656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-813d35f2-bfb8-40a3-8e0e-d83edc52e6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:09.837783 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1887/3785] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:09.908210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafc1f8-3277-4bdd-9835-21b88d16ab55 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:09.909972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3cafc1f8-3277-4bdd-9835-21b88d16ab55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:10.027354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafc1f8-3277-4bdd-9835-21b88d16ab55 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:10.051610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6410b8-c21b-489b-8a9f-60dfde103bbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:10.051610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6410b8-c21b-489b-8a9f-60dfde103bbf service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:10.051610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6410b8-c21b-489b-8a9f-60dfde103bbf service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:10.051610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6410b8-c21b-489b-8a9f-60dfde103bbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:10.061471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6410b8-c21b-489b-8a9f-60dfde103bbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:10.061471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6410b8-c21b-489b-8a9f-60dfde103bbf service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:10.089340 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1899/3786] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:14:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:14:11.194138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d35f2-bfb8-40a3-8e0e-d83edc52e6a5 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:11.196228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-813d35f2-bfb8-40a3-8e0e-d83edc52e6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:11.215992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d35f2-bfb8-40a3-8e0e-d83edc52e6a5 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:11.236683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81096fb7-93d1-49f5-ab68-83ba2cfe1276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:11.238461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81096fb7-93d1-49f5-ab68-83ba2cfe1276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:11.244314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81096fb7-93d1-49f5-ab68-83ba2cfe1276 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:11.244314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81096fb7-93d1-49f5-ab68-83ba2cfe1276 admin admin] PATH_INFO: `/v3/projects/cc8a398d1db349309404becc9a8ddd34` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:11.244314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81096fb7-93d1-49f5-ab68-83ba2cfe1276 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc8a398d1db349309404becc9a8ddd34)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:11.245544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81096fb7-93d1-49f5-ab68-83ba2cfe1276 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:11.437497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-81096fb7-93d1-49f5-ab68-83ba2cfe1276 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc8a398d1db349309404becc9a8ddd34', 'request_id': 'req-81096fb7-93d1-49f5-ab68-83ba2cfe1276'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:11.455956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-81096fb7-93d1-49f5-ab68-83ba2cfe1276 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc8a398d1db349309404becc9a8ddd34', 'request_id': 'req-81096fb7-93d1-49f5-ab68-83ba2cfe1276'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:11.569185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-81096fb7-93d1-49f5-ab68-83ba2cfe1276 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc8a398d1db349309404becc9a8ddd34', 'request_id': 'req-81096fb7-93d1-49f5-ab68-83ba2cfe1276'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:11.593722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-81096fb7-93d1-49f5-ab68-83ba2cfe1276 admin admin] The token cache is being invalidate because project cc8a398d1db349309404becc9a8ddd34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:11.594332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-81096fb7-93d1-49f5-ab68-83ba2cfe1276 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:11.604276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-81096fb7-93d1-49f5-ab68-83ba2cfe1276 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:11.607221 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1888/3787] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:11 2022] DELETE /identity/v3/projects/cc8a398d1db349309404becc9a8ddd34 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:11.612317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6410b8-c21b-489b-8a9f-60dfde103bbf service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:11.631357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6410b8-c21b-489b-8a9f-60dfde103bbf service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:11.669268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:11.827676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3884631-33ea-4f2d-ba7f-3683b79249dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:11.828588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3884631-33ea-4f2d-ba7f-3683b79249dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:11.828877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3884631-33ea-4f2d-ba7f-3683b79249dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:11.829158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3884631-33ea-4f2d-ba7f-3683b79249dc admin admin] PATH_INFO: `/v3/users/cb3c9363526544ac9ca04d2c84c90b4f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:11.830532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3884631-33ea-4f2d-ba7f-3683b79249dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb3c9363526544ac9ca04d2c84c90b4f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:11.834712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3884631-33ea-4f2d-ba7f-3683b79249dc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:12.189572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e3884631-33ea-4f2d-ba7f-3683b79249dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb3c9363526544ac9ca04d2c84c90b4f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:12.212128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e3884631-33ea-4f2d-ba7f-3683b79249dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb3c9363526544ac9ca04d2c84c90b4f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:12.213001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e3884631-33ea-4f2d-ba7f-3683b79249dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb3c9363526544ac9ca04d2c84c90b4f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:12.213683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e3884631-33ea-4f2d-ba7f-3683b79249dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb3c9363526544ac9ca04d2c84c90b4f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:12.323045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e3884631-33ea-4f2d-ba7f-3683b79249dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb3c9363526544ac9ca04d2c84c90b4f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:12.359985 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1900/3788] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:11 2022] DELETE /identity/v3/users/cb3c9363526544ac9ca04d2c84c90b4f => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:14.069513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81096fb7-93d1-49f5-ab68-83ba2cfe1276 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:14.078487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-81096fb7-93d1-49f5-ab68-83ba2cfe1276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:14.110730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81096fb7-93d1-49f5-ab68-83ba2cfe1276 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:14.189255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:14.214211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac9f134-26d1-4ae1-b5da-1b1889f6f1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:14.217486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9f134-26d1-4ae1-b5da-1b1889f6f1a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:14.218103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9f134-26d1-4ae1-b5da-1b1889f6f1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:14.220768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac9f134-26d1-4ae1-b5da-1b1889f6f1a8 admin admin] PATH_INFO: `/v3/projects/aa527a522e5d442d81cf13d2748e6b84` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:14.229415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9f134-26d1-4ae1-b5da-1b1889f6f1a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa527a522e5d442d81cf13d2748e6b84)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:14.233331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac9f134-26d1-4ae1-b5da-1b1889f6f1a8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:14.439909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2ac9f134-26d1-4ae1-b5da-1b1889f6f1a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa527a522e5d442d81cf13d2748e6b84', 'request_id': 'req-2ac9f134-26d1-4ae1-b5da-1b1889f6f1a8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:14.457480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2ac9f134-26d1-4ae1-b5da-1b1889f6f1a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa527a522e5d442d81cf13d2748e6b84', 'request_id': 'req-2ac9f134-26d1-4ae1-b5da-1b1889f6f1a8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:14.562583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2ac9f134-26d1-4ae1-b5da-1b1889f6f1a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa527a522e5d442d81cf13d2748e6b84', 'request_id': 'req-2ac9f134-26d1-4ae1-b5da-1b1889f6f1a8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:14.575981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2ac9f134-26d1-4ae1-b5da-1b1889f6f1a8 admin admin] The token cache is being invalidate because project aa527a522e5d442d81cf13d2748e6b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:14.576718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2ac9f134-26d1-4ae1-b5da-1b1889f6f1a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:14.577689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2ac9f134-26d1-4ae1-b5da-1b1889f6f1a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:14.580141 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1889/3789] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:14 2022] DELETE /identity/v3/projects/aa527a522e5d442d81cf13d2748e6b84 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:14.585406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2819e8-113a-40d9-b351-043d53a448d7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:14.586527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2819e8-113a-40d9-b351-043d53a448d7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:14.586527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2819e8-113a-40d9-b351-043d53a448d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:14.617018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ac2819e8-113a-40d9-b351-043d53a448d7 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:14.619067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ac2819e8-113a-40d9-b351-043d53a448d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:14.769367 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1901/3790] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:14.779030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2c195-1b67-4e30-8933-bfb347a7bd55 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:14.786254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2c195-1b67-4e30-8933-bfb347a7bd55 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:14.786254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2c195-1b67-4e30-8933-bfb347a7bd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:14.824029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f5c2c195-1b67-4e30-8933-bfb347a7bd55 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:14.826967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f5c2c195-1b67-4e30-8933-bfb347a7bd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:14.842882 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1890/3791] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:14.844525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2819e8-113a-40d9-b351-043d53a448d7 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:14.865700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2819e8-113a-40d9-b351-043d53a448d7 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:14.876976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f568c-898c-4e72-8afc-21a80bd222a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:14.878192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f568c-898c-4e72-8afc-21a80bd222a0 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:14.878663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f568c-898c-4e72-8afc-21a80bd222a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:14.879202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f568c-898c-4e72-8afc-21a80bd222a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:14.880078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f568c-898c-4e72-8afc-21a80bd222a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:14:14.880667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f568c-898c-4e72-8afc-21a80bd222a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:14.883057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99f568c-898c-4e72-8afc-21a80bd222a0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:14.912856 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1902/3792] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:14.917582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2c195-1b67-4e30-8933-bfb347a7bd55 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:14.959803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2c195-1b67-4e30-8933-bfb347a7bd55 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:14.975935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616ed4e-7736-47e5-b6ce-3ee3288fbcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:14.977581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616ed4e-7736-47e5-b6ce-3ee3288fbcb8 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:14.980497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616ed4e-7736-47e5-b6ce-3ee3288fbcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:14.988324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616ed4e-7736-47e5-b6ce-3ee3288fbcb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:14.988324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616ed4e-7736-47e5-b6ce-3ee3288fbcb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:14.988324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616ed4e-7736-47e5-b6ce-3ee3288fbcb8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:15.149324 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1891/3793] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:14:14 2022] POST /identity/v3/projects => generated 367 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:15.151249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f568c-898c-4e72-8afc-21a80bd222a0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:15.164705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f568c-898c-4e72-8afc-21a80bd222a0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:15.192467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:15.323630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238356d4-2418-4c34-9cee-8acbad5efd47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:15.324328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238356d4-2418-4c34-9cee-8acbad5efd47 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:15.324647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238356d4-2418-4c34-9cee-8acbad5efd47 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:15.324901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238356d4-2418-4c34-9cee-8acbad5efd47 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:15.325608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238356d4-2418-4c34-9cee-8acbad5efd47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:15.327589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238356d4-2418-4c34-9cee-8acbad5efd47 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:15.457322 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1903/3794] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:14:15 2022] POST /identity/v3/users => generated 284 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:15.458835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616ed4e-7736-47e5-b6ce-3ee3288fbcb8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:15.476111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616ed4e-7736-47e5-b6ce-3ee3288fbcb8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:15.488579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93f6bf-6255-41ed-8804-0f413a6b8ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:15.489196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93f6bf-6255-41ed-8804-0f413a6b8ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:15.489481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93f6bf-6255-41ed-8804-0f413a6b8ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:15.489761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93f6bf-6255-41ed-8804-0f413a6b8ad9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:15.490847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93f6bf-6255-41ed-8804-0f413a6b8ad9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:15.492548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93f6bf-6255-41ed-8804-0f413a6b8ad9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:15.517768 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1892/3795] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:15.521870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238356d4-2418-4c34-9cee-8acbad5efd47 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:15.536822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238356d4-2418-4c34-9cee-8acbad5efd47 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:15.544756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51568bc0-d31a-476e-af09-7a7903eea7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:15.545813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51568bc0-d31a-476e-af09-7a7903eea7ef admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:15.546366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51568bc0-d31a-476e-af09-7a7903eea7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:15.546873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51568bc0-d31a-476e-af09-7a7903eea7ef admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:15.547832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51568bc0-d31a-476e-af09-7a7903eea7ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:15.551763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51568bc0-d31a-476e-af09-7a7903eea7ef admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:15.567713 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1904/3796] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:15.572099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93f6bf-6255-41ed-8804-0f413a6b8ad9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:15.584163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93f6bf-6255-41ed-8804-0f413a6b8ad9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:15.591149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b077ed7-7415-4d86-b196-bae4ab42bb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:15.591573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b077ed7-7415-4d86-b196-bae4ab42bb3e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:15.591999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b077ed7-7415-4d86-b196-bae4ab42bb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:15.592342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b077ed7-7415-4d86-b196-bae4ab42bb3e admin admin] PATH_INFO: `/v3/projects/c31d1575cef144c9862328e93af5a16b/users/50fb6adcdd94424eaee2a0d0154c08f0/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:15.613019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b077ed7-7415-4d86-b196-bae4ab42bb3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c31d1575cef144c9862328e93af5a16b, user_id=50fb6adcdd94424eaee2a0d0154c08f0, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:15.616402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b077ed7-7415-4d86-b196-bae4ab42bb3e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:15.759687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc8acb-5e68-4f2b-b66b-55495a12451e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:15.762503 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1893/3797] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:15 2022] PUT /identity/v3/projects/c31d1575cef144c9862328e93af5a16b/users/50fb6adcdd94424eaee2a0d0154c08f0/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:15.762609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc8acb-5e68-4f2b-b66b-55495a12451e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:15.766034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc8acb-5e68-4f2b-b66b-55495a12451e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:15.870221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-12dc8acb-5e68-4f2b-b66b-55495a12451e None None] MFA Rules not processed for user `50fb6adcdd94424eaee2a0d0154c08f0`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:15.871869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-12dc8acb-5e68-4f2b-b66b-55495a12451e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:16.052649 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1905/3798] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:15 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:16.053788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b077ed7-7415-4d86-b196-bae4ab42bb3e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:16.194166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b077ed7-7415-4d86-b196-bae4ab42bb3e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:16.201181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901caac-8e3c-4495-8e3b-6014cf09cdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:16.202733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901caac-8e3c-4495-8e3b-6014cf09cdbb admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:16.203239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901caac-8e3c-4495-8e3b-6014cf09cdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:16.203704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f901caac-8e3c-4495-8e3b-6014cf09cdbb admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:16.204588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901caac-8e3c-4495-8e3b-6014cf09cdbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:16.207028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f901caac-8e3c-4495-8e3b-6014cf09cdbb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:16.353521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:16.370741 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1894/3799] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:14:16 2022] POST /identity/v3/projects => generated 365 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:14:16.372231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc8acb-5e68-4f2b-b66b-55495a12451e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:16.414478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc8acb-5e68-4f2b-b66b-55495a12451e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:16.513777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3dc032-04b2-43f0-b011-56605e49a5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:16.514587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3dc032-04b2-43f0-b011-56605e49a5fa admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:16.530333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3dc032-04b2-43f0-b011-56605e49a5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:16.530333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3dc032-04b2-43f0-b011-56605e49a5fa admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:16.530333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3dc032-04b2-43f0-b011-56605e49a5fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:16.530333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3dc032-04b2-43f0-b011-56605e49a5fa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:16.676796 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1906/3800] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:14:16 2022] POST /identity/v3/users => generated 289 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:14:16.702300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901caac-8e3c-4495-8e3b-6014cf09cdbb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:16.740960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f901caac-8e3c-4495-8e3b-6014cf09cdbb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:16.748573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf704e-d5e1-4c1c-aa78-85ba4991acfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:16.753689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf704e-d5e1-4c1c-aa78-85ba4991acfa admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:16.755188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf704e-d5e1-4c1c-aa78-85ba4991acfa admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:16.755850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf704e-d5e1-4c1c-aa78-85ba4991acfa admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:16.756950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf704e-d5e1-4c1c-aa78-85ba4991acfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:16.774916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bf704e-d5e1-4c1c-aa78-85ba4991acfa admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:16.780792 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1895/3801] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:16.786114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3dc032-04b2-43f0-b011-56605e49a5fa admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:16.806363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3dc032-04b2-43f0-b011-56605e49a5fa admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:16.813697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a9c79-6e98-4808-b5a2-8f41f98ada0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:16.813697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a9c79-6e98-4808-b5a2-8f41f98ada0d admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:16.813697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a9c79-6e98-4808-b5a2-8f41f98ada0d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:16.814305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7a9c79-6e98-4808-b5a2-8f41f98ada0d admin admin] PATH_INFO: `/v3/projects/d976764d0e334fff83f70e0002713abb/users/06acbd50ebff421894be6666acb2f8e2/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:16.824360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:16.832147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a9c79-6e98-4808-b5a2-8f41f98ada0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d976764d0e334fff83f70e0002713abb, user_id=06acbd50ebff421894be6666acb2f8e2, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:16.835970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7a9c79-6e98-4808-b5a2-8f41f98ada0d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:17.334865 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1907/3802] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:16 2022] PUT /identity/v3/projects/d976764d0e334fff83f70e0002713abb/users/06acbd50ebff421894be6666acb2f8e2/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:17.340555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf704e-d5e1-4c1c-aa78-85ba4991acfa admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:17.361916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a9c79-6e98-4808-b5a2-8f41f98ada0d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:17.371755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2c7a9c79-6e98-4808-b5a2-8f41f98ada0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:17.374130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf704e-d5e1-4c1c-aa78-85ba4991acfa admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:17.497969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7a9c79-6e98-4808-b5a2-8f41f98ada0d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:17.515060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240a18f-5e69-4d7e-955d-a95ff8f9f369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:17.515965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240a18f-5e69-4d7e-955d-a95ff8f9f369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:17.516255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240a18f-5e69-4d7e-955d-a95ff8f9f369 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:17.517327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240a18f-5e69-4d7e-955d-a95ff8f9f369 admin admin] PATH_INFO: `/v3/users/ffb90cfa7828412f96f330dab49466cd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:17.521056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf2441-1f43-4fbd-a7eb-6b1776e82f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:17.521462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf2441-1f43-4fbd-a7eb-6b1776e82f59 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:17.521904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf2441-1f43-4fbd-a7eb-6b1776e82f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:17.522385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf2441-1f43-4fbd-a7eb-6b1776e82f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:17.522506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240a18f-5e69-4d7e-955d-a95ff8f9f369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffb90cfa7828412f96f330dab49466cd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:17.523962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf2441-1f43-4fbd-a7eb-6b1776e82f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:17.524609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240a18f-5e69-4d7e-955d-a95ff8f9f369 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:17.526498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf2441-1f43-4fbd-a7eb-6b1776e82f59 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:17.555331 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1896/3803] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:17.555331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf2441-1f43-4fbd-a7eb-6b1776e82f59 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:17.576289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf2441-1f43-4fbd-a7eb-6b1776e82f59 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:17.584518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be0710-b62b-45c4-b1d4-081e006eadb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:17.589246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be0710-b62b-45c4-b1d4-081e006eadb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:17.589246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be0710-b62b-45c4-b1d4-081e006eadb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:17.589246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be0710-b62b-45c4-b1d4-081e006eadb1 admin admin] PATH_INFO: `/v3/projects/d976764d0e334fff83f70e0002713abb/users/06acbd50ebff421894be6666acb2f8e2/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:17.589246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be0710-b62b-45c4-b1d4-081e006eadb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d976764d0e334fff83f70e0002713abb, user_id=06acbd50ebff421894be6666acb2f8e2, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:17.591224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be0710-b62b-45c4-b1d4-081e006eadb1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:17.605222 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1897/3804] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:17 2022] PUT /identity/v3/projects/d976764d0e334fff83f70e0002713abb/users/06acbd50ebff421894be6666acb2f8e2/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:17.621574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaf8ab-b2e6-4c75-8557-34ce3b2f66ee None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:17.621891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaf8ab-b2e6-4c75-8557-34ce3b2f66ee None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:17.622423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaf8ab-b2e6-4c75-8557-34ce3b2f66ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:17.695801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7cfaf8ab-b2e6-4c75-8557-34ce3b2f66ee None None] MFA Rules not processed for user `06acbd50ebff421894be6666acb2f8e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:17.697950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7cfaf8ab-b2e6-4c75-8557-34ce3b2f66ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:17.832425 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1898/3805] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:17 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:17.837955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaf8ab-b2e6-4c75-8557-34ce3b2f66ee None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:17.842118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7240a18f-5e69-4d7e-955d-a95ff8f9f369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffb90cfa7828412f96f330dab49466cd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:17.849807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaf8ab-b2e6-4c75-8557-34ce3b2f66ee None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:17.856382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28deb972-687f-4d5c-bc55-965f99bba558 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06acbd50ebff421894be6666acb2f8e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd976764d0e334fff83f70e0002713abb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:17.857202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28deb972-687f-4d5c-bc55-965f99bba558 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:17.857350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7240a18f-5e69-4d7e-955d-a95ff8f9f369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffb90cfa7828412f96f330dab49466cd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:17.857626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28deb972-687f-4d5c-bc55-965f99bba558 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:17.858012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28deb972-687f-4d5c-bc55-965f99bba558 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] PATH_INFO: `/v3/regions` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:17.858147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7240a18f-5e69-4d7e-955d-a95ff8f9f369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffb90cfa7828412f96f330dab49466cd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:17.858741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7240a18f-5e69-4d7e-955d-a95ff8f9f369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffb90cfa7828412f96f330dab49466cd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:17.859075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28deb972-687f-4d5c-bc55-965f99bba558 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:17.861108 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-28deb972-687f-4d5c-bc55-965f99bba558 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:14:17.861108 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:14:17.861108 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:14:17.862227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28deb972-687f-4d5c-bc55-965f99bba558 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:17.939806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7240a18f-5e69-4d7e-955d-a95ff8f9f369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffb90cfa7828412f96f330dab49466cd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:17.981310 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1908/3806] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:17 2022] DELETE /identity/v3/users/ffb90cfa7828412f96f330dab49466cd => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:17.993879 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1899/3807] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:14:17 2022] POST /identity/v3/regions => generated 224 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:17.997956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240a18f-5e69-4d7e-955d-a95ff8f9f369 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:17.999443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7240a18f-5e69-4d7e-955d-a95ff8f9f369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:18.105728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240a18f-5e69-4d7e-955d-a95ff8f9f369 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:18.123172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:18.126596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34bb4f0-e344-44e7-81d0-72d93a31eeae tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06acbd50ebff421894be6666acb2f8e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd976764d0e334fff83f70e0002713abb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:18.127394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bb4f0-e344-44e7-81d0-72d93a31eeae tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:18.127753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bb4f0-e344-44e7-81d0-72d93a31eeae tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:18.128173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bb4f0-e344-44e7-81d0-72d93a31eeae tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] PATH_INFO: `/v3/regions` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:18.128897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34bb4f0-e344-44e7-81d0-72d93a31eeae tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:18.130689 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-a34bb4f0-e344-44e7-81d0-72d93a31eeae tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:14:18.130689 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:14:18.130689 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:14:18.130971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34bb4f0-e344-44e7-81d0-72d93a31eeae tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:18.245297 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1909/3808] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:14:17 2022] POST /identity/v3/regions => generated 224 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:18.245830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28deb972-687f-4d5c-bc55-965f99bba558 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:18.261689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28deb972-687f-4d5c-bc55-965f99bba558 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:18.284524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8e3da-834f-487c-ad30-fef6c3f80cc5 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06acbd50ebff421894be6666acb2f8e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd976764d0e334fff83f70e0002713abb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:18.287920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8e3da-834f-487c-ad30-fef6c3f80cc5 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:18.287920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8e3da-834f-487c-ad30-fef6c3f80cc5 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:18.287920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da8e3da-834f-487c-ad30-fef6c3f80cc5 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] PATH_INFO: `/v3/regions/180b5ebb-1188-4463-8d97-82f3ca14f979` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:18.287920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8e3da-834f-487c-ad30-fef6c3f80cc5 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorizing `identity:create_region(region_id=180b5ebb-1188-4463-8d97-82f3ca14f979)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:18.290399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da8e3da-834f-487c-ad30-fef6c3f80cc5 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:18.318540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34bb4f0-e344-44e7-81d0-72d93a31eeae tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:18.318540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a34bb4f0-e344-44e7-81d0-72d93a31eeae tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:18.394199 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1900/3809] 10.222.0.33 () {60 vars in 1258 bytes} [Sat May 28 21:14:18 2022] PUT /identity/v3/regions/180b5ebb-1188-4463-8d97-82f3ca14f979 => generated 231 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:14:18.395737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8e3da-834f-487c-ad30-fef6c3f80cc5 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:18.460727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:18.474767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34bb4f0-e344-44e7-81d0-72d93a31eeae tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:18.476798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da8e3da-834f-487c-ad30-fef6c3f80cc5 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:18.496741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed622d2-9c3e-4c43-af3d-bade05e26445 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06acbd50ebff421894be6666acb2f8e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd976764d0e334fff83f70e0002713abb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:18.498088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed622d2-9c3e-4c43-af3d-bade05e26445 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:18.500349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed622d2-9c3e-4c43-af3d-bade05e26445 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:18.500349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed622d2-9c3e-4c43-af3d-bade05e26445 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] PATH_INFO: `/v3/regions/180b5ebb-1188-4463-8d97-82f3ca14f979` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:18.500349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed622d2-9c3e-4c43-af3d-bade05e26445 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorizing `identity:delete_region(region_id=180b5ebb-1188-4463-8d97-82f3ca14f979)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:18.503762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed622d2-9c3e-4c43-af3d-bade05e26445 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:18.508220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae7a0b8-33fe-4aa9-8056-9244d4c82618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:18.509107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7a0b8-33fe-4aa9-8056-9244d4c82618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:18.511683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7a0b8-33fe-4aa9-8056-9244d4c82618 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:18.512042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae7a0b8-33fe-4aa9-8056-9244d4c82618 admin admin] PATH_INFO: `/v3/projects/d26d403e254e4caa887f8155d74c84c4` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:18.515165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7a0b8-33fe-4aa9-8056-9244d4c82618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d26d403e254e4caa887f8155d74c84c4)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:18.519787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae7a0b8-33fe-4aa9-8056-9244d4c82618 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:18.633925 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1901/3810] 10.222.0.33 () {58 vars in 1241 bytes} [Sat May 28 21:14:18 2022] DELETE /identity/v3/regions/180b5ebb-1188-4463-8d97-82f3ca14f979 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:18.638807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed622d2-9c3e-4c43-af3d-bade05e26445 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:18.682510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-aae7a0b8-33fe-4aa9-8056-9244d4c82618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd26d403e254e4caa887f8155d74c84c4', 'request_id': 'req-aae7a0b8-33fe-4aa9-8056-9244d4c82618'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:18.692009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed622d2-9c3e-4c43-af3d-bade05e26445 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:18.694114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-aae7a0b8-33fe-4aa9-8056-9244d4c82618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd26d403e254e4caa887f8155d74c84c4', 'request_id': 'req-aae7a0b8-33fe-4aa9-8056-9244d4c82618'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:18.778832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa96a15-1f75-4caf-a59b-427f3609ff25 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06acbd50ebff421894be6666acb2f8e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd976764d0e334fff83f70e0002713abb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:18.779838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa96a15-1f75-4caf-a59b-427f3609ff25 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:18.780174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa96a15-1f75-4caf-a59b-427f3609ff25 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:18.780982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa96a15-1f75-4caf-a59b-427f3609ff25 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] PATH_INFO: `/v3/regions` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:18.781657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa96a15-1f75-4caf-a59b-427f3609ff25 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:18.783817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa96a15-1f75-4caf-a59b-427f3609ff25 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:18.799747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-aae7a0b8-33fe-4aa9-8056-9244d4c82618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd26d403e254e4caa887f8155d74c84c4', 'request_id': 'req-aae7a0b8-33fe-4aa9-8056-9244d4c82618'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:18.833166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-aae7a0b8-33fe-4aa9-8056-9244d4c82618 admin admin] The token cache is being invalidate because project d26d403e254e4caa887f8155d74c84c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:18.834915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-aae7a0b8-33fe-4aa9-8056-9244d4c82618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:18.847749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-aae7a0b8-33fe-4aa9-8056-9244d4c82618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:18.850798 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1910/3811] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:18 2022] DELETE /identity/v3/projects/d26d403e254e4caa887f8155d74c84c4 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:14:18.857708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9ef84-60f2-45bb-84d0-084cfb40e517 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:18.859585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9ef84-60f2-45bb-84d0-084cfb40e517 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:18.860000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9ef84-60f2-45bb-84d0-084cfb40e517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:18.905886 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1902/3812] 10.222.0.33 () {60 vars in 1149 bytes} [Sat May 28 21:14:18 2022] POST /identity/v3/regions => generated 253 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:18.910542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa96a15-1f75-4caf-a59b-427f3609ff25 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:18.912879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3fa96a15-1f75-4caf-a59b-427f3609ff25 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:18.916241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-97a9ef84-60f2-45bb-84d0-084cfb40e517 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:18.917623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-97a9ef84-60f2-45bb-84d0-084cfb40e517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:18.944233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa96a15-1f75-4caf-a59b-427f3609ff25 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:18.950224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd1382-9b91-4da1-b4b1-cc744ef34ace tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06acbd50ebff421894be6666acb2f8e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd976764d0e334fff83f70e0002713abb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:18.951093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd1382-9b91-4da1-b4b1-cc744ef34ace tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:18.951378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd1382-9b91-4da1-b4b1-cc744ef34ace tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:18.951667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd1382-9b91-4da1-b4b1-cc744ef34ace tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] PATH_INFO: `/v3/regions/e5b6a077545b44bc8ce0f9bacb88a0be` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:18.952532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd1382-9b91-4da1-b4b1-cc744ef34ace tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorizing `identity:update_region(region_id=e5b6a077545b44bc8ce0f9bacb88a0be)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:18.953576 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-c0dd1382-9b91-4da1-b4b1-cc744ef34ace tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_region": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:14:18.953576 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:14:18.953576 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:14:18.954520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd1382-9b91-4da1-b4b1-cc744ef34ace tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:19.024618 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1911/3814] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:19.024758 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1903/3814] 10.222.0.33 () {60 vars in 1249 bytes} [Sat May 28 21:14:18 2022] PATCH /identity/v3/regions/e5b6a077545b44bc8ce0f9bacb88a0be => generated 254 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:19.037830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c6e6c-3d65-4f53-a4ef-caeb29cfb42c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:19.037830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c6e6c-3d65-4f53-a4ef-caeb29cfb42c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:19.037830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c6e6c-3d65-4f53-a4ef-caeb29cfb42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:19.042653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9ef84-60f2-45bb-84d0-084cfb40e517 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:19.066823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9ef84-60f2-45bb-84d0-084cfb40e517 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:19.076843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9e5c6e6c-3d65-4f53-a4ef-caeb29cfb42c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:19.080691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9e5c6e6c-3d65-4f53-a4ef-caeb29cfb42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:19.111279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1d7b5-eb24-48f5-94df-0d89061f7137 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06acbd50ebff421894be6666acb2f8e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd976764d0e334fff83f70e0002713abb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:19.112776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1d7b5-eb24-48f5-94df-0d89061f7137 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:19.114139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1d7b5-eb24-48f5-94df-0d89061f7137 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:19.114614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1d7b5-eb24-48f5-94df-0d89061f7137 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] PATH_INFO: `/v3/regions/e5b6a077545b44bc8ce0f9bacb88a0be` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:19.116012 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1904/3815] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:19.116793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1d7b5-eb24-48f5-94df-0d89061f7137 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorizing `identity:get_region(region_id=e5b6a077545b44bc8ce0f9bacb88a0be)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:19.118346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1d7b5-eb24-48f5-94df-0d89061f7137 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:19.121349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c6e6c-3d65-4f53-a4ef-caeb29cfb42c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:19.131987 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1912/3816] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:14:19 2022] GET /identity/v3/regions/e5b6a077545b44bc8ce0f9bacb88a0be => generated 254 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:19.136400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1d7b5-eb24-48f5-94df-0d89061f7137 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:19.137164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c6e6c-3d65-4f53-a4ef-caeb29cfb42c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:19.144681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e801e3f-42e2-488b-a443-1e1f13e34b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:19.147836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1d7b5-eb24-48f5-94df-0d89061f7137 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:19.149243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e801e3f-42e2-488b-a443-1e1f13e34b10 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:19.149243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e801e3f-42e2-488b-a443-1e1f13e34b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:19.149487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e801e3f-42e2-488b-a443-1e1f13e34b10 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:19.150385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e801e3f-42e2-488b-a443-1e1f13e34b10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:14:19.150693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e801e3f-42e2-488b-a443-1e1f13e34b10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:19.152646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e801e3f-42e2-488b-a443-1e1f13e34b10 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:19.154247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb8fcd-b2e0-4eac-8b57-1744929ef908 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06acbd50ebff421894be6666acb2f8e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd976764d0e334fff83f70e0002713abb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:19.155553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb8fcd-b2e0-4eac-8b57-1744929ef908 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:19.156417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb8fcd-b2e0-4eac-8b57-1744929ef908 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:19.156771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb8fcd-b2e0-4eac-8b57-1744929ef908 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] PATH_INFO: `/v3/regions/e5b6a077545b44bc8ce0f9bacb88a0be` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:19.157522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb8fcd-b2e0-4eac-8b57-1744929ef908 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorizing `identity:delete_region(region_id=e5b6a077545b44bc8ce0f9bacb88a0be)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:19.159283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcb8fcd-b2e0-4eac-8b57-1744929ef908 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:19.208877 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1905/3817] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:19.212275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e801e3f-42e2-488b-a443-1e1f13e34b10 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:19.222854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e801e3f-42e2-488b-a443-1e1f13e34b10 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:19.229196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47faba04-9b3b-4cbf-9cd6-bb82abf2ef8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:19.230134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faba04-9b3b-4cbf-9cd6-bb82abf2ef8e admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:19.230502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faba04-9b3b-4cbf-9cd6-bb82abf2ef8e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:19.230868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47faba04-9b3b-4cbf-9cd6-bb82abf2ef8e admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:19.231650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47faba04-9b3b-4cbf-9cd6-bb82abf2ef8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:19.233675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47faba04-9b3b-4cbf-9cd6-bb82abf2ef8e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:19.253464 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1913/3818] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:14:19 2022] DELETE /identity/v3/regions/e5b6a077545b44bc8ce0f9bacb88a0be => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:19.257969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb8fcd-b2e0-4eac-8b57-1744929ef908 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:19.311778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb8fcd-b2e0-4eac-8b57-1744929ef908 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:19.317167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94d1ab-6866-4d14-b229-2190d9544dd9 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06acbd50ebff421894be6666acb2f8e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd976764d0e334fff83f70e0002713abb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:19.317887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94d1ab-6866-4d14-b229-2190d9544dd9 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:19.318178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94d1ab-6866-4d14-b229-2190d9544dd9 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:19.318466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94d1ab-6866-4d14-b229-2190d9544dd9 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] PATH_INFO: `/v3/regions` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:19.319210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94d1ab-6866-4d14-b229-2190d9544dd9 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:19.322674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b94d1ab-6866-4d14-b229-2190d9544dd9 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:19.337891 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1914/3819] 10.222.0.33 () {58 vars in 1127 bytes} [Sat May 28 21:14:19 2022] GET /identity/v3/regions => generated 670 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:19.342173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94d1ab-6866-4d14-b229-2190d9544dd9 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:19.342407 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1906/3820] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:14:19 2022] POST /identity/v3/projects => generated 393 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:14:19.342407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47faba04-9b3b-4cbf-9cd6-bb82abf2ef8e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:19.350037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b94d1ab-6866-4d14-b229-2190d9544dd9 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:19.434794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd616a3-e433-4c9e-a240-a99a71be9466 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06acbd50ebff421894be6666acb2f8e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd976764d0e334fff83f70e0002713abb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:19.438259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:19.439316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd616a3-e433-4c9e-a240-a99a71be9466 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:19.439676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd616a3-e433-4c9e-a240-a99a71be9466 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:19.440183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd616a3-e433-4c9e-a240-a99a71be9466 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] PATH_INFO: `/v3/regions/e5b6a077545b44bc8ce0f9bacb88a0be` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:19.440945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd616a3-e433-4c9e-a240-a99a71be9466 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorizing `identity:delete_region(region_id=e5b6a077545b44bc8ce0f9bacb88a0be)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:19.442898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd616a3-e433-4c9e-a240-a99a71be9466 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:19.462603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-1cd616a3-e433-4c9e-a240-a99a71be9466 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Could not find region: e5b6a077545b44bc8ce0f9bacb88a0be.: keystone.exception.RegionNotFound: Could not find region: e5b6a077545b44bc8ce0f9bacb88a0be. May 28 21:14:19.462603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:14:19.462603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 129, in delete_region May 28 21:14:19.462603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ret = self.driver.delete_region(region_id) May 28 21:14:19.462603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 152, in delete_region May 28 21:14:19.462603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = self._get_region(session, region_id) May 28 21:14:19.462603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 103, in _get_region May 28 21:14:19.462603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) May 28 21:14:19.462603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: e5b6a077545b44bc8ce0f9bacb88a0be. May 28 21:14:19.462603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:14:19.462603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:14:19.462603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:14:19.462603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:14:19.462603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:14:19.462603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:14:19.462603 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:14:19.463724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:14:19.463724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:14:19.463724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:14:19.463724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:14:19.463724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:14:19.463724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:14:19.463724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:14:19.463724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/regions.py", line 92, in delete May 28 21:14:19.463724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return PROVIDERS.catalog_api.delete_region( May 28 21:14:19.463724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:14:19.463724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:14:19.463724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 135, in delete_region May 28 21:14:19.463724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RegionNotFound(region_id=region_id) May 28 21:14:19.463724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RegionNotFound: Could not find region: e5b6a077545b44bc8ce0f9bacb88a0be. May 28 21:14:19.463724 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:14:19.463724 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1915/3821] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:14:19 2022] DELETE /identity/v3/regions/e5b6a077545b44bc8ce0f9bacb88a0be => generated 112 bytes in 125 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:14:19.470104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47faba04-9b3b-4cbf-9cd6-bb82abf2ef8e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:19.472642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd616a3-e433-4c9e-a240-a99a71be9466 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:19.495514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd616a3-e433-4c9e-a240-a99a71be9466 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:19.509895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b8a4e-893c-4db4-9713-b0316de4e8a4 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06acbd50ebff421894be6666acb2f8e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd976764d0e334fff83f70e0002713abb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:19.509895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b8a4e-893c-4db4-9713-b0316de4e8a4 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:19.509895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b8a4e-893c-4db4-9713-b0316de4e8a4 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:19.510221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35b8a4e-893c-4db4-9713-b0316de4e8a4 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] PATH_INFO: `/v3/regions` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:19.510221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b8a4e-893c-4db4-9713-b0316de4e8a4 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:19.511672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d879a35-df69-40ba-83a7-71aaeaddc60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:19.512699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d879a35-df69-40ba-83a7-71aaeaddc60d admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:19.513061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d879a35-df69-40ba-83a7-71aaeaddc60d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:19.513419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d879a35-df69-40ba-83a7-71aaeaddc60d admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:19.515095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35b8a4e-893c-4db4-9713-b0316de4e8a4 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:19.515907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d879a35-df69-40ba-83a7-71aaeaddc60d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:19.519279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d879a35-df69-40ba-83a7-71aaeaddc60d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:19.531800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:19.533143 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1916/3822] 10.222.0.33 () {58 vars in 1127 bytes} [Sat May 28 21:14:19 2022] GET /identity/v3/regions => generated 670 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:19.533143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b8a4e-893c-4db4-9713-b0316de4e8a4 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:19.545018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35b8a4e-893c-4db4-9713-b0316de4e8a4 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:19.550583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd40f1-a17c-4ebc-b46d-fbfca4339038 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06acbd50ebff421894be6666acb2f8e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd976764d0e334fff83f70e0002713abb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:19.555919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd40f1-a17c-4ebc-b46d-fbfca4339038 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:19.555919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd40f1-a17c-4ebc-b46d-fbfca4339038 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:19.555919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd40f1-a17c-4ebc-b46d-fbfca4339038 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] PATH_INFO: `/v3/regions` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:19.555919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd40f1-a17c-4ebc-b46d-fbfca4339038 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorizing `identity:create_region()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:19.555919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd40f1-a17c-4ebc-b46d-fbfca4339038 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:19.646356 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1907/3823] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:14:19 2022] POST /identity/v3/users => generated 297 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:19.648339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d879a35-df69-40ba-83a7-71aaeaddc60d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:19.651905 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1917/3824] 10.222.0.33 () {60 vars in 1149 bytes} [Sat May 28 21:14:19 2022] POST /identity/v3/regions => generated 254 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:19.655813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd40f1-a17c-4ebc-b46d-fbfca4339038 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:19.676201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d879a35-df69-40ba-83a7-71aaeaddc60d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:19.703287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd40f1-a17c-4ebc-b46d-fbfca4339038 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:19.710771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e929c6-e4e1-4f7c-8b2a-78532d815957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:19.712164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e929c6-e4e1-4f7c-8b2a-78532d815957 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:19.713623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e929c6-e4e1-4f7c-8b2a-78532d815957 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:19.714516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e929c6-e4e1-4f7c-8b2a-78532d815957 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:19.715358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1733ab-fcea-4ea9-866d-6ba0e1d46ada tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06acbd50ebff421894be6666acb2f8e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd976764d0e334fff83f70e0002713abb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:19.716045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1733ab-fcea-4ea9-866d-6ba0e1d46ada tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:19.716337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1733ab-fcea-4ea9-866d-6ba0e1d46ada tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:19.716676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e929c6-e4e1-4f7c-8b2a-78532d815957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:19.718131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1733ab-fcea-4ea9-866d-6ba0e1d46ada tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] PATH_INFO: `/v3/regions` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:19.718497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e929c6-e4e1-4f7c-8b2a-78532d815957 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:19.719302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1733ab-fcea-4ea9-866d-6ba0e1d46ada tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Adding query filter params (parent_region_id=c371b7b20a384c75bd7954fca5257a6d) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:14:19.719808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1733ab-fcea-4ea9-866d-6ba0e1d46ada tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorizing `identity:list_regions()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:19.722005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1733ab-fcea-4ea9-866d-6ba0e1d46ada tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:19.731831 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1918/3825] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:14:19 2022] GET /identity/v3/regions?parent_region_id=c371b7b20a384c75bd7954fca5257a6d => generated 400 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:19.734754 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1908/3826] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:19.741539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1733ab-fcea-4ea9-866d-6ba0e1d46ada tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:19.744764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e929c6-e4e1-4f7c-8b2a-78532d815957 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:19.752975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1733ab-fcea-4ea9-866d-6ba0e1d46ada tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:19.760641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e929c6-e4e1-4f7c-8b2a-78532d815957 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:19.769918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe3ed3-bb6e-4903-8ee0-f5374e4bf874 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06acbd50ebff421894be6666acb2f8e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd976764d0e334fff83f70e0002713abb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:19.771460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe3ed3-bb6e-4903-8ee0-f5374e4bf874 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:19.772640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe3ed3-bb6e-4903-8ee0-f5374e4bf874 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:19.773529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa546f4f-1cbb-4600-a57a-c39b8d4f4c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:19.774098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe3ed3-bb6e-4903-8ee0-f5374e4bf874 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] PATH_INFO: `/v3/regions/71438ff55307423c809ed652ba0f4726` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:19.775049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe3ed3-bb6e-4903-8ee0-f5374e4bf874 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorizing `identity:delete_region(region_id=71438ff55307423c809ed652ba0f4726)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:19.776079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa546f4f-1cbb-4600-a57a-c39b8d4f4c2d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:19.778074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa546f4f-1cbb-4600-a57a-c39b8d4f4c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:19.778576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa546f4f-1cbb-4600-a57a-c39b8d4f4c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:19.779548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa546f4f-1cbb-4600-a57a-c39b8d4f4c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:19.780060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfe3ed3-bb6e-4903-8ee0-f5374e4bf874 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:19.781948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa546f4f-1cbb-4600-a57a-c39b8d4f4c2d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:19.805998 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1909/3827] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:19.809203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa546f4f-1cbb-4600-a57a-c39b8d4f4c2d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:19.823646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa546f4f-1cbb-4600-a57a-c39b8d4f4c2d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:19.829796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aadc83-59ee-45cc-9277-040dd0012771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:19.831076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aadc83-59ee-45cc-9277-040dd0012771 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:19.831645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aadc83-59ee-45cc-9277-040dd0012771 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:19.832165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aadc83-59ee-45cc-9277-040dd0012771 admin admin] PATH_INFO: `/v3/projects/53b9cfacdbb74a058e52bed331a6f241/users/af446b3d802a42e3bd72b08bace8ea8d/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:19.878405 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1919/3828] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:14:19 2022] DELETE /identity/v3/regions/71438ff55307423c809ed652ba0f4726 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:19.884255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe3ed3-bb6e-4903-8ee0-f5374e4bf874 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:19.885358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aadc83-59ee-45cc-9277-040dd0012771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53b9cfacdbb74a058e52bed331a6f241, user_id=af446b3d802a42e3bd72b08bace8ea8d, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:19.888219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aadc83-59ee-45cc-9277-040dd0012771 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:19.943135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe3ed3-bb6e-4903-8ee0-f5374e4bf874 tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:19.950129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415ffa6-0c36-464a-99f6-aca025343b8f tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06acbd50ebff421894be6666acb2f8e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd976764d0e334fff83f70e0002713abb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:19.951124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415ffa6-0c36-464a-99f6-aca025343b8f tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:19.951896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415ffa6-0c36-464a-99f6-aca025343b8f tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:19.951896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415ffa6-0c36-464a-99f6-aca025343b8f tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] PATH_INFO: `/v3/regions/75897615d5024afe95d238d1218d5539` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:19.952667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415ffa6-0c36-464a-99f6-aca025343b8f tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorizing `identity:delete_region(region_id=75897615d5024afe95d238d1218d5539)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:19.954410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d415ffa6-0c36-464a-99f6-aca025343b8f tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:19.983736 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1910/3829] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:19 2022] PUT /identity/v3/projects/53b9cfacdbb74a058e52bed331a6f241/users/af446b3d802a42e3bd72b08bace8ea8d/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:19.989216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e499e4-e88b-43f4-a428-9e20233acd64 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:19.990599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e499e4-e88b-43f4-a428-9e20233acd64 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:19.990904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e499e4-e88b-43f4-a428-9e20233acd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:20.011728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c0e499e4-e88b-43f4-a428-9e20233acd64 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:20.014234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c0e499e4-e88b-43f4-a428-9e20233acd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:20.079707 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1920/3830] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:14:19 2022] DELETE /identity/v3/regions/75897615d5024afe95d238d1218d5539 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:14:20.084010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415ffa6-0c36-464a-99f6-aca025343b8f tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:20.123231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d415ffa6-0c36-464a-99f6-aca025343b8f tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:20.160715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:20.165280 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1911/3831] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:14:20.173466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e499e4-e88b-43f4-a428-9e20233acd64 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:20.177263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c0e499e4-e88b-43f4-a428-9e20233acd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:20.266544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416bb84a-bbd4-4a28-8534-5e4caf3c255d tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '06acbd50ebff421894be6666acb2f8e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd976764d0e334fff83f70e0002713abb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:20.267549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bb84a-bbd4-4a28-8534-5e4caf3c255d tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:20.270148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bb84a-bbd4-4a28-8534-5e4caf3c255d tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:20.270763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416bb84a-bbd4-4a28-8534-5e4caf3c255d tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] PATH_INFO: `/v3/regions/c371b7b20a384c75bd7954fca5257a6d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:20.271343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416bb84a-bbd4-4a28-8534-5e4caf3c255d tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorizing `identity:delete_region(region_id=c371b7b20a384c75bd7954fca5257a6d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:20.278648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416bb84a-bbd4-4a28-8534-5e4caf3c255d tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:20.354501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:20.358411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e499e4-e88b-43f4-a428-9e20233acd64 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:20.381107 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1921/3832] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:14:20 2022] DELETE /identity/v3/regions/c371b7b20a384c75bd7954fca5257a6d => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:14:20.382214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416bb84a-bbd4-4a28-8534-5e4caf3c255d tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:20.389087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67f14b-281a-447e-93cf-5985077ac559 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:20.394356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-416bb84a-bbd4-4a28-8534-5e4caf3c255d tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:20.397754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67f14b-281a-447e-93cf-5985077ac559 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:20.398660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67f14b-281a-447e-93cf-5985077ac559 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:20.398971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67f14b-281a-447e-93cf-5985077ac559 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:20.440766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67f14b-281a-447e-93cf-5985077ac559 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:20.442279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67f14b-281a-447e-93cf-5985077ac559 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:20.462072 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1912/3833] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:14:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:14:20.496266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416bb84a-bbd4-4a28-8534-5e4caf3c255d tempest-RegionsTestJSON-224838545 tempest-RegionsTestJSON-224838545-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:20.505077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f4f84-7a86-4e4e-afed-b072e2c5c4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:20.506118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f4f84-7a86-4e4e-afed-b072e2c5c4f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:20.506614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f4f84-7a86-4e4e-afed-b072e2c5c4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:20.507051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0f4f84-7a86-4e4e-afed-b072e2c5c4f8 admin admin] PATH_INFO: `/v3/users/50fb6adcdd94424eaee2a0d0154c08f0` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:20.508406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f4f84-7a86-4e4e-afed-b072e2c5c4f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50fb6adcdd94424eaee2a0d0154c08f0)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:20.510724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0f4f84-7a86-4e4e-afed-b072e2c5c4f8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:20.754680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ad0f4f84-7a86-4e4e-afed-b072e2c5c4f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50fb6adcdd94424eaee2a0d0154c08f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:20.777457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ad0f4f84-7a86-4e4e-afed-b072e2c5c4f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50fb6adcdd94424eaee2a0d0154c08f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:20.782411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ad0f4f84-7a86-4e4e-afed-b072e2c5c4f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50fb6adcdd94424eaee2a0d0154c08f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:20.783631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ad0f4f84-7a86-4e4e-afed-b072e2c5c4f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50fb6adcdd94424eaee2a0d0154c08f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:20.917924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ad0f4f84-7a86-4e4e-afed-b072e2c5c4f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50fb6adcdd94424eaee2a0d0154c08f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:20.959078 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1922/3834] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:20 2022] DELETE /identity/v3/users/50fb6adcdd94424eaee2a0d0154c08f0 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:20.964326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbcb98-db57-4d76-8cc0-cf1b97c15f0a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:20.964933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbcb98-db57-4d76-8cc0-cf1b97c15f0a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:20.965373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbcb98-db57-4d76-8cc0-cf1b97c15f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:21.050183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-dffbcb98-db57-4d76-8cc0-cf1b97c15f0a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:21.056319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-dffbcb98-db57-4d76-8cc0-cf1b97c15f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:21.261709 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1913/3835] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:21.277417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f4f84-7a86-4e4e-afed-b072e2c5c4f8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:21.278920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ad0f4f84-7a86-4e4e-afed-b072e2c5c4f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:21.357426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:21.421188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0f4f84-7a86-4e4e-afed-b072e2c5c4f8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:21.425411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f8f9b-4e8e-4785-b7a0-e08ecdeaaec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:21.427671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f8f9b-4e8e-4785-b7a0-e08ecdeaaec5 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:21.427671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f8f9b-4e8e-4785-b7a0-e08ecdeaaec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:21.427671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f8f9b-4e8e-4785-b7a0-e08ecdeaaec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:21.438007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f8f9b-4e8e-4785-b7a0-e08ecdeaaec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:21.439829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f8f9b-4e8e-4785-b7a0-e08ecdeaaec5 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:21.453870 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1923/3836] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:14:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:14:22.469566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbcb98-db57-4d76-8cc0-cf1b97c15f0a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:22.471223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-dffbcb98-db57-4d76-8cc0-cf1b97c15f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:22.488618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbcb98-db57-4d76-8cc0-cf1b97c15f0a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:22.496126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebffb6-f697-47ae-b33c-02e3dcd950ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:22.497013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebffb6-f697-47ae-b33c-02e3dcd950ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:22.497301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebffb6-f697-47ae-b33c-02e3dcd950ee admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:22.497582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebffb6-f697-47ae-b33c-02e3dcd950ee admin admin] PATH_INFO: `/v3/projects/c31d1575cef144c9862328e93af5a16b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:22.498947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ebffb6-f697-47ae-b33c-02e3dcd950ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c31d1575cef144c9862328e93af5a16b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:22.500948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ebffb6-f697-47ae-b33c-02e3dcd950ee admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:22.669397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-32ebffb6-f697-47ae-b33c-02e3dcd950ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c31d1575cef144c9862328e93af5a16b', 'request_id': 'req-32ebffb6-f697-47ae-b33c-02e3dcd950ee'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:22.675192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-32ebffb6-f697-47ae-b33c-02e3dcd950ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c31d1575cef144c9862328e93af5a16b', 'request_id': 'req-32ebffb6-f697-47ae-b33c-02e3dcd950ee'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:22.786512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-32ebffb6-f697-47ae-b33c-02e3dcd950ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c31d1575cef144c9862328e93af5a16b', 'request_id': 'req-32ebffb6-f697-47ae-b33c-02e3dcd950ee'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:22.802017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-32ebffb6-f697-47ae-b33c-02e3dcd950ee admin admin] The token cache is being invalidate because project c31d1575cef144c9862328e93af5a16b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:22.802317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-32ebffb6-f697-47ae-b33c-02e3dcd950ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:22.803369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-32ebffb6-f697-47ae-b33c-02e3dcd950ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:22.805897 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1914/3837] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:22 2022] DELETE /identity/v3/projects/c31d1575cef144c9862328e93af5a16b => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:22.809402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f8f9b-4e8e-4785-b7a0-e08ecdeaaec5 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:22.832514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f8f9b-4e8e-4785-b7a0-e08ecdeaaec5 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:22.976969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d69f2b-6829-4864-8220-61e980186523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:22.978973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d69f2b-6829-4864-8220-61e980186523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:22.979380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d69f2b-6829-4864-8220-61e980186523 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:22.980056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d69f2b-6829-4864-8220-61e980186523 admin admin] PATH_INFO: `/v3/users/06acbd50ebff421894be6666acb2f8e2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:22.982095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d69f2b-6829-4864-8220-61e980186523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06acbd50ebff421894be6666acb2f8e2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:22.984522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d69f2b-6829-4864-8220-61e980186523 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:23.356002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-82d69f2b-6829-4864-8220-61e980186523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06acbd50ebff421894be6666acb2f8e2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:23.365187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-82d69f2b-6829-4864-8220-61e980186523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06acbd50ebff421894be6666acb2f8e2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:23.366504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-82d69f2b-6829-4864-8220-61e980186523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06acbd50ebff421894be6666acb2f8e2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:23.367837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-82d69f2b-6829-4864-8220-61e980186523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06acbd50ebff421894be6666acb2f8e2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:23.471092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-82d69f2b-6829-4864-8220-61e980186523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06acbd50ebff421894be6666acb2f8e2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:23.494791 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1924/3838] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:22 2022] DELETE /identity/v3/users/06acbd50ebff421894be6666acb2f8e2 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:24.389614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebffb6-f697-47ae-b33c-02e3dcd950ee admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:24.391984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-32ebffb6-f697-47ae-b33c-02e3dcd950ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:24.406913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebffb6-f697-47ae-b33c-02e3dcd950ee admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:24.549788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75c4b6-cad7-4bef-9fd4-e64011c8c552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:24.551011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75c4b6-cad7-4bef-9fd4-e64011c8c552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:24.551488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75c4b6-cad7-4bef-9fd4-e64011c8c552 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:24.552281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75c4b6-cad7-4bef-9fd4-e64011c8c552 admin admin] PATH_INFO: `/v3/projects/d976764d0e334fff83f70e0002713abb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:24.553881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75c4b6-cad7-4bef-9fd4-e64011c8c552 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d976764d0e334fff83f70e0002713abb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:24.557653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75c4b6-cad7-4bef-9fd4-e64011c8c552 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:24.594167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:24.732462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a75c4b6-cad7-4bef-9fd4-e64011c8c552 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd976764d0e334fff83f70e0002713abb', 'request_id': 'req-3a75c4b6-cad7-4bef-9fd4-e64011c8c552'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:24.739949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a75c4b6-cad7-4bef-9fd4-e64011c8c552 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd976764d0e334fff83f70e0002713abb', 'request_id': 'req-3a75c4b6-cad7-4bef-9fd4-e64011c8c552'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:24.825192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a75c4b6-cad7-4bef-9fd4-e64011c8c552 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd976764d0e334fff83f70e0002713abb', 'request_id': 'req-3a75c4b6-cad7-4bef-9fd4-e64011c8c552'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:24.843983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a75c4b6-cad7-4bef-9fd4-e64011c8c552 admin admin] The token cache is being invalidate because project d976764d0e334fff83f70e0002713abb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:24.844366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a75c4b6-cad7-4bef-9fd4-e64011c8c552 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:24.847540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3a75c4b6-cad7-4bef-9fd4-e64011c8c552 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:24.850555 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1915/3839] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:24 2022] DELETE /identity/v3/projects/d976764d0e334fff83f70e0002713abb => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:24.856425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ea905-4724-4f8d-a049-c0c7dd7c934b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:24.857469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ea905-4724-4f8d-a049-c0c7dd7c934b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:24.857921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ea905-4724-4f8d-a049-c0c7dd7c934b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:24.883024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e62ea905-4724-4f8d-a049-c0c7dd7c934b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:24.889507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e62ea905-4724-4f8d-a049-c0c7dd7c934b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:24.949929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:25.000062 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1925/3840] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:25.002775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1399a-7da2-415c-8e4c-8a588230c778 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:25.002775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1399a-7da2-415c-8e4c-8a588230c778 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:25.003011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1399a-7da2-415c-8e4c-8a588230c778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:25.043768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1aa1399a-7da2-415c-8e4c-8a588230c778 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:25.045022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1aa1399a-7da2-415c-8e4c-8a588230c778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:25.075031 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1916/3841] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:25.079909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ea905-4724-4f8d-a049-c0c7dd7c934b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:25.097131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ea905-4724-4f8d-a049-c0c7dd7c934b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:25.108621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95353455-d68c-4d9f-8666-f25b6a33e54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:25.109679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95353455-d68c-4d9f-8666-f25b6a33e54b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:25.115878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533e389-5ad8-456c-920a-77abcc069dc4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:25.115878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533e389-5ad8-456c-920a-77abcc069dc4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:25.115878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533e389-5ad8-456c-920a-77abcc069dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:25.122592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95353455-d68c-4d9f-8666-f25b6a33e54b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:25.123064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95353455-d68c-4d9f-8666-f25b6a33e54b admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:25.123998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95353455-d68c-4d9f-8666-f25b6a33e54b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:14:25.124384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95353455-d68c-4d9f-8666-f25b6a33e54b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:25.127300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95353455-d68c-4d9f-8666-f25b6a33e54b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:25.159897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8533e389-5ad8-456c-920a-77abcc069dc4 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:25.161223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8533e389-5ad8-456c-920a-77abcc069dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:25.165766 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1926/3842] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:25.175288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95353455-d68c-4d9f-8666-f25b6a33e54b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:25.198375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95353455-d68c-4d9f-8666-f25b6a33e54b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:25.204325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c689e2-565a-491f-ae6c-5c510cf66ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:25.205107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c689e2-565a-491f-ae6c-5c510cf66ddb admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:25.205702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c689e2-565a-491f-ae6c-5c510cf66ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:25.206861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c689e2-565a-491f-ae6c-5c510cf66ddb admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:25.210643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c689e2-565a-491f-ae6c-5c510cf66ddb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:25.210643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c689e2-565a-491f-ae6c-5c510cf66ddb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:25.298548 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1917/3843] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:14:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:14:25.315934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533e389-5ad8-456c-920a-77abcc069dc4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:25.316184 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1927/3844] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:14:25 2022] POST /identity/v3/projects => generated 369 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:25.319252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8533e389-5ad8-456c-920a-77abcc069dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:25.320411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c689e2-565a-491f-ae6c-5c510cf66ddb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:25.456168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c689e2-565a-491f-ae6c-5c510cf66ddb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:25.464963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d58d6-8b13-4bbb-a400-666c029f5ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:25.465667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d58d6-8b13-4bbb-a400-666c029f5ecf admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:25.466023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d58d6-8b13-4bbb-a400-666c029f5ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:25.466316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d58d6-8b13-4bbb-a400-666c029f5ecf admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:25.467025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d58d6-8b13-4bbb-a400-666c029f5ecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:25.469001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d58d6-8b13-4bbb-a400-666c029f5ecf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:25.475193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533e389-5ad8-456c-920a-77abcc069dc4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:25.583333 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1928/3845] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:14:25 2022] POST /identity/v3/users => generated 285 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:25.590238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d58d6-8b13-4bbb-a400-666c029f5ecf admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:25.595923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4bb0c-a28d-448e-afc1-2b82df481645 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:25.596693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4bb0c-a28d-448e-afc1-2b82df481645 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:25.597033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4bb0c-a28d-448e-afc1-2b82df481645 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:25.597368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4bb0c-a28d-448e-afc1-2b82df481645 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:25.606552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d58d6-8b13-4bbb-a400-666c029f5ecf admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:25.631023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fc401-fbaf-497e-8f36-51c5e0ad1d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:25.631732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fc401-fbaf-497e-8f36-51c5e0ad1d2c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:25.632022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fc401-fbaf-497e-8f36-51c5e0ad1d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:25.632340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fc401-fbaf-497e-8f36-51c5e0ad1d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:25.633135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fc401-fbaf-497e-8f36-51c5e0ad1d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:25.634795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9fc401-fbaf-497e-8f36-51c5e0ad1d2c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:25.655893 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1929/3846] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:25.657977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fc401-fbaf-497e-8f36-51c5e0ad1d2c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:25.685081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fc401-fbaf-497e-8f36-51c5e0ad1d2c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:25.690899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c474b19-70ec-4927-b95c-d48b4dbf4f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:25.691855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c474b19-70ec-4927-b95c-d48b4dbf4f98 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:25.692392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c474b19-70ec-4927-b95c-d48b4dbf4f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:25.692785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c474b19-70ec-4927-b95c-d48b4dbf4f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:25.693666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c474b19-70ec-4927-b95c-d48b4dbf4f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:25.714846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c474b19-70ec-4927-b95c-d48b4dbf4f98 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:25.716974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4bb0c-a28d-448e-afc1-2b82df481645 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:25.719983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4bb0c-a28d-448e-afc1-2b82df481645 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:25.743634 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1930/3847] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:14:25.747359 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1918/3848] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:14:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 21:14:25.748468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c474b19-70ec-4927-b95c-d48b4dbf4f98 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:25.764927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c474b19-70ec-4927-b95c-d48b4dbf4f98 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:25.770309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f5a3f-faba-41dd-ad58-41d83647decb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:25.771554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f5a3f-faba-41dd-ad58-41d83647decb admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:25.771945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f5a3f-faba-41dd-ad58-41d83647decb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:25.772326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f5a3f-faba-41dd-ad58-41d83647decb admin admin] PATH_INFO: `/v3/projects/f4592fcdcdd94b8ea88f4b9ad5219c58/users/6ff700b7c46146f7bd2529a633352164/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:25.793192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f5a3f-faba-41dd-ad58-41d83647decb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4592fcdcdd94b8ea88f4b9ad5219c58, user_id=6ff700b7c46146f7bd2529a633352164, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:25.800970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f5a3f-faba-41dd-ad58-41d83647decb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:25.934007 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1931/3849] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:25 2022] PUT /identity/v3/projects/f4592fcdcdd94b8ea88f4b9ad5219c58/users/6ff700b7c46146f7bd2529a633352164/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:25.935636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c68fc8-72c0-4901-89f9-4d7063b6d8b0 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:25.936016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c68fc8-72c0-4901-89f9-4d7063b6d8b0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:25.936191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c68fc8-72c0-4901-89f9-4d7063b6d8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:25.972979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-25c68fc8-72c0-4901-89f9-4d7063b6d8b0 None None] MFA Rules not processed for user `6ff700b7c46146f7bd2529a633352164`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:25.974281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-25c68fc8-72c0-4901-89f9-4d7063b6d8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:26.072491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:26.097540 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1919/3850] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:25 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:26.107270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f5a3f-faba-41dd-ad58-41d83647decb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:26.195937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f5a3f-faba-41dd-ad58-41d83647decb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:26.216840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7aa056-efc1-47aa-b431-3776596c6d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:26.218121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aa056-efc1-47aa-b431-3776596c6d38 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:26.218619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aa056-efc1-47aa-b431-3776596c6d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:26.219053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aa056-efc1-47aa-b431-3776596c6d38 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:26.220076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7aa056-efc1-47aa-b431-3776596c6d38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:26.222046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7aa056-efc1-47aa-b431-3776596c6d38 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:26.265893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:26.328819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c68fc8-72c0-4901-89f9-4d7063b6d8b0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:26.337708 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1932/3851] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:14:26 2022] POST /identity/v3/projects => generated 369 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:26.340623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c68fc8-72c0-4901-89f9-4d7063b6d8b0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:26.406687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3dab00-0d48-4722-a22b-a43084075a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:26.407427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3dab00-0d48-4722-a22b-a43084075a57 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:26.407712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3dab00-0d48-4722-a22b-a43084075a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:26.407989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3dab00-0d48-4722-a22b-a43084075a57 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:26.408682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3dab00-0d48-4722-a22b-a43084075a57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:26.411113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3dab00-0d48-4722-a22b-a43084075a57 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:26.532703 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1920/3852] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:14:26 2022] POST /identity/v3/users => generated 291 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:26.533721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7aa056-efc1-47aa-b431-3776596c6d38 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:26.544229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7aa056-efc1-47aa-b431-3776596c6d38 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:26.550674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fbedf-324e-46d6-ae43-c557c7fe2fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:26.551376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fbedf-324e-46d6-ae43-c557c7fe2fcb admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:26.551982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fbedf-324e-46d6-ae43-c557c7fe2fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:26.554012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fbedf-324e-46d6-ae43-c557c7fe2fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:26.554012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053fbedf-324e-46d6-ae43-c557c7fe2fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:26.556182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053fbedf-324e-46d6-ae43-c557c7fe2fcb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:26.580425 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1933/3853] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:26.583674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3dab00-0d48-4722-a22b-a43084075a57 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:26.594609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3dab00-0d48-4722-a22b-a43084075a57 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:26.599786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e313b1-ac8a-42eb-ad23-757f729e1bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:26.600766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e313b1-ac8a-42eb-ad23-757f729e1bfe admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:26.601143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e313b1-ac8a-42eb-ad23-757f729e1bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:26.601481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e313b1-ac8a-42eb-ad23-757f729e1bfe admin admin] PATH_INFO: `/v3/projects/316491af215543789aad43617ed68878/users/6057efa7010c4d17a359d408338f7b44/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:26.619087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e313b1-ac8a-42eb-ad23-757f729e1bfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=316491af215543789aad43617ed68878, user_id=6057efa7010c4d17a359d408338f7b44, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:26.622128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e313b1-ac8a-42eb-ad23-757f729e1bfe admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:26.692123 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1921/3854] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:26 2022] PUT /identity/v3/projects/316491af215543789aad43617ed68878/users/6057efa7010c4d17a359d408338f7b44/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:26.696392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fbedf-324e-46d6-ae43-c557c7fe2fcb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:26.706687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fbedf-324e-46d6-ae43-c557c7fe2fcb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:26.786049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323b6c4-7bff-473d-aa30-8b03cc21349e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:26.786890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323b6c4-7bff-473d-aa30-8b03cc21349e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:26.787237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323b6c4-7bff-473d-aa30-8b03cc21349e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:26.787648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323b6c4-7bff-473d-aa30-8b03cc21349e admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:26.788911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323b6c4-7bff-473d-aa30-8b03cc21349e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:26.790980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323b6c4-7bff-473d-aa30-8b03cc21349e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:26.808272 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1934/3855] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:26.809037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e313b1-ac8a-42eb-ad23-757f729e1bfe admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:26.826537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e313b1-ac8a-42eb-ad23-757f729e1bfe admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:26.838150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de16f5-2f91-4c8c-83a5-c1a31133866e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:26.839046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de16f5-2f91-4c8c-83a5-c1a31133866e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:26.839331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de16f5-2f91-4c8c-83a5-c1a31133866e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:26.839607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de16f5-2f91-4c8c-83a5-c1a31133866e admin admin] PATH_INFO: `/v3/projects/316491af215543789aad43617ed68878/users/6057efa7010c4d17a359d408338f7b44/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:26.842430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de16f5-2f91-4c8c-83a5-c1a31133866e admin admin] RBAC: Authorizing `identity:create_grant(project_id=316491af215543789aad43617ed68878, user_id=6057efa7010c4d17a359d408338f7b44, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:26.847670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de16f5-2f91-4c8c-83a5-c1a31133866e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:26.869330 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1922/3856] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:26 2022] PUT /identity/v3/projects/316491af215543789aad43617ed68878/users/6057efa7010c4d17a359d408338f7b44/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:14:26.880234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402877c-3e9f-4de2-8613-e41ef3eae504 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:26.880234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402877c-3e9f-4de2-8613-e41ef3eae504 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:26.880630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d402877c-3e9f-4de2-8613-e41ef3eae504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:26.910417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d402877c-3e9f-4de2-8613-e41ef3eae504 None None] MFA Rules not processed for user `6057efa7010c4d17a359d408338f7b44`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:26.911670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d402877c-3e9f-4de2-8613-e41ef3eae504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:26.990794 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1935/3857] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:26 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:27.006265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de16f5-2f91-4c8c-83a5-c1a31133866e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:27.022244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de16f5-2f91-4c8c-83a5-c1a31133866e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:27.034124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f09bf1-d69b-4ca4-89c2-3e2230006c5d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:27.034945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f09bf1-d69b-4ca4-89c2-3e2230006c5d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:27.035337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f09bf1-d69b-4ca4-89c2-3e2230006c5d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:27.035746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f09bf1-d69b-4ca4-89c2-3e2230006c5d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:27.036605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f09bf1-d69b-4ca4-89c2-3e2230006c5d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:27.040295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f09bf1-d69b-4ca4-89c2-3e2230006c5d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:27.185197 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1923/3858] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:14:27 2022] POST /identity/v3/projects => generated 328 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:14:27.189509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402877c-3e9f-4de2-8613-e41ef3eae504 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:27.208680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d402877c-3e9f-4de2-8613-e41ef3eae504 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:27.332366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7dff3-6e12-49a3-9fd9-05bcfa0c2ed6 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:27.334317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7dff3-6e12-49a3-9fd9-05bcfa0c2ed6 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:27.334621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7dff3-6e12-49a3-9fd9-05bcfa0c2ed6 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:27.337741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f7dff3-6e12-49a3-9fd9-05bcfa0c2ed6 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:27.340973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7dff3-6e12-49a3-9fd9-05bcfa0c2ed6 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:27.345577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f7dff3-6e12-49a3-9fd9-05bcfa0c2ed6 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:27.457994 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1936/3859] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:14:27 2022] POST /identity/v3/users => generated 413 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:14:27.461311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f09bf1-d69b-4ca4-89c2-3e2230006c5d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:27.472786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f09bf1-d69b-4ca4-89c2-3e2230006c5d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:27.481086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45874838-b53e-4c3e-bb73-83c540643330 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:27.482644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45874838-b53e-4c3e-bb73-83c540643330 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:27.483141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45874838-b53e-4c3e-bb73-83c540643330 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:27.483754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45874838-b53e-4c3e-bb73-83c540643330 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:27.485653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45874838-b53e-4c3e-bb73-83c540643330 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:27.487168 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-45874838-b53e-4c3e-bb73-83c540643330 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:14:27.487168 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:14:27.487168 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:14:27.488398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45874838-b53e-4c3e-bb73-83c540643330 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:27.634746 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1924/3860] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:14:27 2022] POST /identity/v3/roles => generated 244 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:27.638885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7dff3-6e12-49a3-9fd9-05bcfa0c2ed6 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:27.656312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f7dff3-6e12-49a3-9fd9-05bcfa0c2ed6 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:27.663469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a4cdd-9101-43fa-b461-8e79f2813193 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:27.664717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a4cdd-9101-43fa-b461-8e79f2813193 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:27.665250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a4cdd-9101-43fa-b461-8e79f2813193 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:27.665730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43a4cdd-9101-43fa-b461-8e79f2813193 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:27.666945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a4cdd-9101-43fa-b461-8e79f2813193 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:27.668175 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-b43a4cdd-9101-43fa-b461-8e79f2813193 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:14:27.668175 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:14:27.668175 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:14:27.670476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43a4cdd-9101-43fa-b461-8e79f2813193 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:27.767745 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1937/3861] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:14:27 2022] POST /identity/v3/roles => generated 246 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:27.772572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45874838-b53e-4c3e-bb73-83c540643330 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:27.786929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:27.799790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45874838-b53e-4c3e-bb73-83c540643330 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:27.806218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759dc85f-d762-4dcf-8b35-2fbecb7faf00 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:27.807148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dc85f-d762-4dcf-8b35-2fbecb7faf00 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:27.807437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dc85f-d762-4dcf-8b35-2fbecb7faf00 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:27.807715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dc85f-d762-4dcf-8b35-2fbecb7faf00 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/fe12125c3aaf4809ad76ba7ed07ae4ec/users/1c737d3969b1466ba776b30d51b9d6aa/roles/26fc30561c3c4e41afdf01b779b6f2d1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:27.832450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759dc85f-d762-4dcf-8b35-2fbecb7faf00 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_grant(project_id=fe12125c3aaf4809ad76ba7ed07ae4ec, user_id=1c737d3969b1466ba776b30d51b9d6aa, role_id=26fc30561c3c4e41afdf01b779b6f2d1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:27.836149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759dc85f-d762-4dcf-8b35-2fbecb7faf00 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:27.928917 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1925/3862] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:27 2022] PUT /identity/v3/projects/fe12125c3aaf4809ad76ba7ed07ae4ec/users/1c737d3969b1466ba776b30d51b9d6aa/roles/26fc30561c3c4e41afdf01b779b6f2d1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:27.933186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a4cdd-9101-43fa-b461-8e79f2813193 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:27.946325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43a4cdd-9101-43fa-b461-8e79f2813193 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:28.087809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b1afb4-80a1-4906-b686-18d2f84b14bf tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:28.088800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1afb4-80a1-4906-b686-18d2f84b14bf tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:28.089162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1afb4-80a1-4906-b686-18d2f84b14bf tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:28.089508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b1afb4-80a1-4906-b686-18d2f84b14bf tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/fe12125c3aaf4809ad76ba7ed07ae4ec/users/1c737d3969b1466ba776b30d51b9d6aa/roles/664165a930174e30944358cfaf0912a9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:28.091950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b1afb4-80a1-4906-b686-18d2f84b14bf tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_grant(project_id=fe12125c3aaf4809ad76ba7ed07ae4ec, user_id=1c737d3969b1466ba776b30d51b9d6aa, role_id=664165a930174e30944358cfaf0912a9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:28.115966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b1afb4-80a1-4906-b686-18d2f84b14bf tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:28.194273 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1938/3863] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:27 2022] PUT /identity/v3/projects/fe12125c3aaf4809ad76ba7ed07ae4ec/users/1c737d3969b1466ba776b30d51b9d6aa/roles/664165a930174e30944358cfaf0912a9 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:28.198093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759dc85f-d762-4dcf-8b35-2fbecb7faf00 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:28.211284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759dc85f-d762-4dcf-8b35-2fbecb7faf00 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:28.343841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:28.354239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a130ebef-57ca-4b17-a680-da9608c6e765 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:28.358061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130ebef-57ca-4b17-a680-da9608c6e765 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:28.358522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130ebef-57ca-4b17-a680-da9608c6e765 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:28.358953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130ebef-57ca-4b17-a680-da9608c6e765 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:28.363224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130ebef-57ca-4b17-a680-da9608c6e765 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:28.365675 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-a130ebef-57ca-4b17-a680-da9608c6e765 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:14:28.365675 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:14:28.365675 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:14:28.368756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a130ebef-57ca-4b17-a680-da9608c6e765 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:28.470756 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1926/3864] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:28 2022] GET /identity/v3/users => generated 8639 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:28.476668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db350e7d-e3e7-4dca-9c72-fe858df9153d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:28.477549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db350e7d-e3e7-4dca-9c72-fe858df9153d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:28.477549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db350e7d-e3e7-4dca-9c72-fe858df9153d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:28.597884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-db350e7d-e3e7-4dca-9c72-fe858df9153d None None] MFA Rules not processed for user `1c737d3969b1466ba776b30d51b9d6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:28.599553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-db350e7d-e3e7-4dca-9c72-fe858df9153d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:28.600259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88a501-51b8-42ed-a3ec-28007c1c2662 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:28.600959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88a501-51b8-42ed-a3ec-28007c1c2662 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:28.601614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c88a501-51b8-42ed-a3ec-28007c1c2662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:28.640529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5c88a501-51b8-42ed-a3ec-28007c1c2662 None None] MFA Rules not processed for user `af446b3d802a42e3bd72b08bace8ea8d`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:28.642879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5c88a501-51b8-42ed-a3ec-28007c1c2662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:28.681527 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1939/3865] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:28.691790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ced809-5e78-4e77-9ead-ec1b708b54a5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:28.692092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ced809-5e78-4e77-9ead-ec1b708b54a5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:28.692384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ced809-5e78-4e77-9ead-ec1b708b54a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:28.718975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-44ced809-5e78-4e77-9ead-ec1b708b54a5 None None] MFA Rules not processed for user `1c737d3969b1466ba776b30d51b9d6aa`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:28.720320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-44ced809-5e78-4e77-9ead-ec1b708b54a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:28.729810 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1940/3866] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:14:28.734309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ced809-5e78-4e77-9ead-ec1b708b54a5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:28.745609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ced809-5e78-4e77-9ead-ec1b708b54a5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:28.752639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe455d-9875-44f4-a78e-7a673f66ba3d tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d3969b1466ba776b30d51b9d6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe12125c3aaf4809ad76ba7ed07ae4ec', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1199687108', 'tempest-NotDelegatedRole-873981143'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:28.753934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe455d-9875-44f4-a78e-7a673f66ba3d tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:28.754191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe455d-9875-44f4-a78e-7a673f66ba3d tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:28.754470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe455d-9875-44f4-a78e-7a673f66ba3d tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:28.755307 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1927/3867] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:14:28.755424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe455d-9875-44f4-a78e-7a673f66ba3d tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] RBAC: Authorizing `identity:create_trust()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:28.759040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfe455d-9875-44f4-a78e-7a673f66ba3d tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:28.765464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68384cf6-0e8a-46c2-b37a-de9092216def None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:28.765753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68384cf6-0e8a-46c2-b37a-de9092216def None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:28.766229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68384cf6-0e8a-46c2-b37a-de9092216def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:28.791417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-68384cf6-0e8a-46c2-b37a-de9092216def None None] MFA Rules not processed for user `af446b3d802a42e3bd72b08bace8ea8d`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:28.799753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:28.800921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-68384cf6-0e8a-46c2-b37a-de9092216def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:28.813999 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1928/3868] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:14:28.825488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68384cf6-0e8a-46c2-b37a-de9092216def None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:28.827123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-68384cf6-0e8a-46c2-b37a-de9092216def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:28.937203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68384cf6-0e8a-46c2-b37a-de9092216def None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:28.945780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e8c2c-7163-4720-8e71-75ffa376984a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:28.946743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e8c2c-7163-4720-8e71-75ffa376984a service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:28.947038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e8c2c-7163-4720-8e71-75ffa376984a service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:28.947429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e8c2c-7163-4720-8e71-75ffa376984a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:28.956367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e8c2c-7163-4720-8e71-75ffa376984a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:28.958097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e8c2c-7163-4720-8e71-75ffa376984a service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:28.975451 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1941/3869] 10.222.0.33 () {60 vars in 1173 bytes} [Sat May 28 21:14:28 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:28.980089 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1929/3870] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:14:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:14:28.981970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:28.983320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe455d-9875-44f4-a78e-7a673f66ba3d tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:28.998416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe455d-9875-44f4-a78e-7a673f66ba3d tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:29.003789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb77ad-80ba-4a15-b13e-02c1b41a7a06 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:29.004667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb77ad-80ba-4a15-b13e-02c1b41a7a06 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:29.008735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb77ad-80ba-4a15-b13e-02c1b41a7a06 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:29.008735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb77ad-80ba-4a15-b13e-02c1b41a7a06 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:29.008735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb77ad-80ba-4a15-b13e-02c1b41a7a06 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:list_trusts()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:29.008735 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-d4cb77ad-80ba-4a15-b13e-02c1b41a7a06 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_trusts": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:14:29.008735 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:14:29.008735 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:14:29.009199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cb77ad-80ba-4a15-b13e-02c1b41a7a06 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:29.020487 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1942/3871] 10.222.0.33 () {58 vars in 1154 bytes} [Sat May 28 21:14:28 2022] GET /identity/v3/OS-TRUST/trusts/ => generated 562 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:14:29.030909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e8c2c-7163-4720-8e71-75ffa376984a service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:29.043752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e8c2c-7163-4720-8e71-75ffa376984a service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:29.048120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e236389-2351-41e3-a271-0476a3d24ef6 tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d3969b1466ba776b30d51b9d6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe12125c3aaf4809ad76ba7ed07ae4ec', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1199687108', 'tempest-NotDelegatedRole-873981143'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:29.048953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e236389-2351-41e3-a271-0476a3d24ef6 tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:29.049341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e236389-2351-41e3-a271-0476a3d24ef6 tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:29.049638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e236389-2351-41e3-a271-0476a3d24ef6 tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] PATH_INFO: `/v3/OS-TRUST/trusts/ee5b32f42356451ead5518b34fa2a512` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:29.067199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e236389-2351-41e3-a271-0476a3d24ef6 tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] RBAC: Authorizing `identity:delete_trust(trust_id=ee5b32f42356451ead5518b34fa2a512)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:29.070255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e236389-2351-41e3-a271-0476a3d24ef6 tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:29.175859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4e236389-2351-41e3-a271-0476a3d24ef6 tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'ee5b32f42356451ead5518b34fa2a512', 'request_id': 'req-4e236389-2351-41e3-a271-0476a3d24ef6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:29.178740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4e236389-2351-41e3-a271-0476a3d24ef6 tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': 'ee5b32f42356451ead5518b34fa2a512', 'request_id': 'req-4e236389-2351-41e3-a271-0476a3d24ef6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:29.270906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4e236389-2351-41e3-a271-0476a3d24ef6 tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'ee5b32f42356451ead5518b34fa2a512', 'request_id': 'req-4e236389-2351-41e3-a271-0476a3d24ef6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:29.280698 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1930/3872] 10.222.0.33 () {58 vars in 1253 bytes} [Sat May 28 21:14:29 2022] DELETE /identity/v3/OS-TRUST/trusts/ee5b32f42356451ead5518b34fa2a512 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:29.285329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb77ad-80ba-4a15-b13e-02c1b41a7a06 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:29.287946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d4cb77ad-80ba-4a15-b13e-02c1b41a7a06 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:29.317588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb77ad-80ba-4a15-b13e-02c1b41a7a06 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:29.336123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ae8ea-dc46-449c-98cf-3f33d914c23c tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c737d3969b1466ba776b30d51b9d6aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe12125c3aaf4809ad76ba7ed07ae4ec', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1199687108', 'tempest-NotDelegatedRole-873981143'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:29.337671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ae8ea-dc46-449c-98cf-3f33d914c23c tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:29.338947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ae8ea-dc46-449c-98cf-3f33d914c23c tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:29.339394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ae8ea-dc46-449c-98cf-3f33d914c23c tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] PATH_INFO: `/v3/OS-TRUST/trusts/ee5b32f42356451ead5518b34fa2a512` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:29.384177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-316ae8ea-dc46-449c-98cf-3f33d914c23c tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] Could not find trust: ee5b32f42356451ead5518b34fa2a512.: keystone.exception.TrustNotFound: Could not find trust: ee5b32f42356451ead5518b34fa2a512. May 28 21:14:29.384177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:14:29.384177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:14:29.384177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:14:29.384177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:14:29.384177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:14:29.384177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:14:29.384177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:14:29.384177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:14:29.384177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:14:29.384177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:14:29.384177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:14:29.384177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get May 28 21:14:29.384177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) May 28 21:14:29.384177 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust May 28 21:14:29.385230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', May 28 21:14:29.385230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call May 28 21:14:29.385230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() May 28 21:14:29.385230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement May 28 21:14:29.385230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( May 28 21:14:29.385230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:14:29.385230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:14:29.385230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust May 28 21:14:29.385230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) May 28 21:14:29.385230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust May 28 21:14:29.385230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) May 28 21:14:29.385230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: ee5b32f42356451ead5518b34fa2a512. May 28 21:14:29.385230 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:14:29.386580 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1943/3873] 10.222.0.33 () {58 vars in 1250 bytes} [Sat May 28 21:14:29 2022] GET /identity/v3/OS-TRUST/trusts/ee5b32f42356451ead5518b34fa2a512 => generated 111 bytes in 101 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:14:29.390131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e236389-2351-41e3-a271-0476a3d24ef6 tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:29.393429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4e236389-2351-41e3-a271-0476a3d24ef6 tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:29.409759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e236389-2351-41e3-a271-0476a3d24ef6 tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:29.415516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ecabe0-8ad9-4da7-a03b-1e5f5b58111e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:29.416502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecabe0-8ad9-4da7-a03b-1e5f5b58111e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:29.416785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecabe0-8ad9-4da7-a03b-1e5f5b58111e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:29.417063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ecabe0-8ad9-4da7-a03b-1e5f5b58111e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles/664165a930174e30944358cfaf0912a9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:29.418831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ae8ea-dc46-449c-98cf-3f33d914c23c tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:29.418971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ecabe0-8ad9-4da7-a03b-1e5f5b58111e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_role(role_id=664165a930174e30944358cfaf0912a9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:29.419906 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-34ecabe0-8ad9-4da7-a03b-1e5f5b58111e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:14:29.419906 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:14:29.419906 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:14:29.420741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ecabe0-8ad9-4da7-a03b-1e5f5b58111e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:29.420963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-316ae8ea-dc46-449c-98cf-3f33d914c23c tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:29.488502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ae8ea-dc46-449c-98cf-3f33d914c23c tempest-TrustsV3TestJSON-553519044 tempest-user-277052401] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:29.493477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac14c9-0bd3-467d-a37d-730e23b746cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:29.506632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac14c9-0bd3-467d-a37d-730e23b746cb service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:29.506632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac14c9-0bd3-467d-a37d-730e23b746cb service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:29.506632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac14c9-0bd3-467d-a37d-730e23b746cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:29.506632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b9ac14c9-0bd3-467d-a37d-730e23b746cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:29.508183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac14c9-0bd3-467d-a37d-730e23b746cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:29.509759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac14c9-0bd3-467d-a37d-730e23b746cb service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:29.518948 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1944/3874] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:14:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:14:29.699430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-34ecabe0-8ad9-4da7-a03b-1e5f5b58111e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invalidating the token cache because role 664165a930174e30944358cfaf0912a9 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:29.699985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-34ecabe0-8ad9-4da7-a03b-1e5f5b58111e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:29.700457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-34ecabe0-8ad9-4da7-a03b-1e5f5b58111e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:29.702448 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1931/3875] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:29 2022] DELETE /identity/v3/roles/664165a930174e30944358cfaf0912a9 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:29.706635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac14c9-0bd3-467d-a37d-730e23b746cb service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:29.721319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac14c9-0bd3-467d-a37d-730e23b746cb service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:29.789417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a5370-a3ae-44f1-a5f9-2aceef1c0da7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:29.791867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a5370-a3ae-44f1-a5f9-2aceef1c0da7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:29.794843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a5370-a3ae-44f1-a5f9-2aceef1c0da7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:29.794843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a5370-a3ae-44f1-a5f9-2aceef1c0da7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles/26fc30561c3c4e41afdf01b779b6f2d1` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:29.794843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a5370-a3ae-44f1-a5f9-2aceef1c0da7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_role(role_id=26fc30561c3c4e41afdf01b779b6f2d1)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:29.795663 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-735a5370-a3ae-44f1-a5f9-2aceef1c0da7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:14:29.795663 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:14:29.795663 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:14:29.796578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a5370-a3ae-44f1-a5f9-2aceef1c0da7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:30.003735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-735a5370-a3ae-44f1-a5f9-2aceef1c0da7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invalidating the token cache because role 26fc30561c3c4e41afdf01b779b6f2d1 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:30.004542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-735a5370-a3ae-44f1-a5f9-2aceef1c0da7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:30.005225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-735a5370-a3ae-44f1-a5f9-2aceef1c0da7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:30.007708 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1945/3876] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:29 2022] DELETE /identity/v3/roles/26fc30561c3c4e41afdf01b779b6f2d1 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:14:30.012270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ecabe0-8ad9-4da7-a03b-1e5f5b58111e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:30.014891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-34ecabe0-8ad9-4da7-a03b-1e5f5b58111e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:30.044075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ecabe0-8ad9-4da7-a03b-1e5f5b58111e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:30.103574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:30.182922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f94757-454d-44b2-9482-07b207db48d0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:30.183474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f94757-454d-44b2-9482-07b207db48d0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:30.184057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f94757-454d-44b2-9482-07b207db48d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:30.195748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba7cd3-ca8a-488b-a9c8-4ae7ec60f947 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:30.197084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba7cd3-ca8a-488b-a9c8-4ae7ec60f947 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:30.197379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba7cd3-ca8a-488b-a9c8-4ae7ec60f947 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:30.197671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba7cd3-ca8a-488b-a9c8-4ae7ec60f947 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users/1c737d3969b1466ba776b30d51b9d6aa` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:30.206499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba7cd3-ca8a-488b-a9c8-4ae7ec60f947 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_user(user_id=1c737d3969b1466ba776b30d51b9d6aa)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:30.208156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba7cd3-ca8a-488b-a9c8-4ae7ec60f947 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:30.217406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-26f94757-454d-44b2-9482-07b207db48d0 None None] MFA Rules not processed for user `af446b3d802a42e3bd72b08bace8ea8d`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:30.219048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-26f94757-454d-44b2-9482-07b207db48d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:30.237346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:30.322847 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1946/3877] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:14:30.354738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6dba7cd3-ca8a-488b-a9c8-4ae7ec60f947 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c737d3969b1466ba776b30d51b9d6aa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:30.357953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6dba7cd3-ca8a-488b-a9c8-4ae7ec60f947 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c737d3969b1466ba776b30d51b9d6aa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:30.369657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6dba7cd3-ca8a-488b-a9c8-4ae7ec60f947 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c737d3969b1466ba776b30d51b9d6aa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:30.444529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6dba7cd3-ca8a-488b-a9c8-4ae7ec60f947 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c737d3969b1466ba776b30d51b9d6aa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:30.447600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6dba7cd3-ca8a-488b-a9c8-4ae7ec60f947 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c737d3969b1466ba776b30d51b9d6aa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:30.495747 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1932/3878] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:30 2022] DELETE /identity/v3/users/1c737d3969b1466ba776b30d51b9d6aa => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:30.505939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f94757-454d-44b2-9482-07b207db48d0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:30.508106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-26f94757-454d-44b2-9482-07b207db48d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:30.547969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f94757-454d-44b2-9482-07b207db48d0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:30.672198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8fe05-d036-467f-9447-0136159dbf79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:30.674040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8fe05-d036-467f-9447-0136159dbf79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:30.674475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8fe05-d036-467f-9447-0136159dbf79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:30.675184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8fe05-d036-467f-9447-0136159dbf79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/fe12125c3aaf4809ad76ba7ed07ae4ec` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:30.678518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f8fe05-d036-467f-9447-0136159dbf79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fe12125c3aaf4809ad76ba7ed07ae4ec)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:30.681479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f8fe05-d036-467f-9447-0136159dbf79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:31.003721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-90f8fe05-d036-467f-9447-0136159dbf79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe12125c3aaf4809ad76ba7ed07ae4ec', 'request_id': 'req-90f8fe05-d036-467f-9447-0136159dbf79'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:31.026865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-90f8fe05-d036-467f-9447-0136159dbf79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe12125c3aaf4809ad76ba7ed07ae4ec', 'request_id': 'req-90f8fe05-d036-467f-9447-0136159dbf79'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:31.146026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-90f8fe05-d036-467f-9447-0136159dbf79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe12125c3aaf4809ad76ba7ed07ae4ec', 'request_id': 'req-90f8fe05-d036-467f-9447-0136159dbf79'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:31.161357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-90f8fe05-d036-467f-9447-0136159dbf79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] The token cache is being invalidate because project fe12125c3aaf4809ad76ba7ed07ae4ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:31.161768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-90f8fe05-d036-467f-9447-0136159dbf79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:31.162626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-90f8fe05-d036-467f-9447-0136159dbf79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:31.164355 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1947/3879] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:30 2022] DELETE /identity/v3/projects/fe12125c3aaf4809ad76ba7ed07ae4ec => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:31.170996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba7cd3-ca8a-488b-a9c8-4ae7ec60f947 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:31.172755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6dba7cd3-ca8a-488b-a9c8-4ae7ec60f947 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:31.195321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba7cd3-ca8a-488b-a9c8-4ae7ec60f947 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:31.272479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310b110-001f-4c0c-b8e0-29074d461ef4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:31.273410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310b110-001f-4c0c-b8e0-29074d461ef4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:31.273882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310b110-001f-4c0c-b8e0-29074d461ef4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:31.274314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5310b110-001f-4c0c-b8e0-29074d461ef4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:31.276243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310b110-001f-4c0c-b8e0-29074d461ef4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:31.278779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5310b110-001f-4c0c-b8e0-29074d461ef4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:31.393276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8fe05-d036-467f-9447-0136159dbf79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:31.393829 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1933/3880] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:14:31 2022] POST /identity/v3/projects => generated 329 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:31.395582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-90f8fe05-d036-467f-9447-0136159dbf79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:31.411306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8fe05-d036-467f-9447-0136159dbf79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:31.514166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7d06c-31bd-479d-82f1-c3f2812b3633 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:31.515407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7d06c-31bd-479d-82f1-c3f2812b3633 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:31.516749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7d06c-31bd-479d-82f1-c3f2812b3633 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:31.517562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7d06c-31bd-479d-82f1-c3f2812b3633 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:31.518485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7d06c-31bd-479d-82f1-c3f2812b3633 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:31.523684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7d06c-31bd-479d-82f1-c3f2812b3633 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:31.664265 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1948/3881] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:14:31 2022] POST /identity/v3/users => generated 413 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:14:31.670134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310b110-001f-4c0c-b8e0-29074d461ef4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:31.701987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5310b110-001f-4c0c-b8e0-29074d461ef4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:31.711487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf993a-81d6-415b-b74a-c808dcc2be1b tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:31.712498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf993a-81d6-415b-b74a-c808dcc2be1b tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:31.712982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf993a-81d6-415b-b74a-c808dcc2be1b tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:31.713480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbf993a-81d6-415b-b74a-c808dcc2be1b tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:31.714613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf993a-81d6-415b-b74a-c808dcc2be1b tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:31.716427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbf993a-81d6-415b-b74a-c808dcc2be1b tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:31.848723 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1934/3882] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:14:31 2022] POST /identity/v3/roles => generated 243 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:31.861412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7d06c-31bd-479d-82f1-c3f2812b3633 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:31.877521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7d06c-31bd-479d-82f1-c3f2812b3633 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:31.884003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c32549-17bc-42b9-b30c-02f368c0110f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:31.884902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c32549-17bc-42b9-b30c-02f368c0110f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:31.885368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c32549-17bc-42b9-b30c-02f368c0110f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:31.886633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c32549-17bc-42b9-b30c-02f368c0110f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:31.887692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c32549-17bc-42b9-b30c-02f368c0110f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:31.891059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c32549-17bc-42b9-b30c-02f368c0110f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:32.022758 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1949/3883] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:14:31 2022] POST /identity/v3/roles => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:14:32.027934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf993a-81d6-415b-b74a-c808dcc2be1b tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:32.046604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbf993a-81d6-415b-b74a-c808dcc2be1b tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:32.073054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3346e-94b0-49bb-8354-ee4a87f329d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:32.073054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3346e-94b0-49bb-8354-ee4a87f329d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:32.073054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3346e-94b0-49bb-8354-ee4a87f329d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:32.073364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b3346e-94b0-49bb-8354-ee4a87f329d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/c77e01bce1d8485f928e3fff8cd6174d/users/71641ecd6c354becafeb749efdd49e11/roles/310f682586984e819815ae3c7bbfe811` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:32.138833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3346e-94b0-49bb-8354-ee4a87f329d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_grant(project_id=c77e01bce1d8485f928e3fff8cd6174d, user_id=71641ecd6c354becafeb749efdd49e11, role_id=310f682586984e819815ae3c7bbfe811)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:32.141695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b3346e-94b0-49bb-8354-ee4a87f329d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:32.153330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:32.306320 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1935/3884] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:32 2022] PUT /identity/v3/projects/c77e01bce1d8485f928e3fff8cd6174d/users/71641ecd6c354becafeb749efdd49e11/roles/310f682586984e819815ae3c7bbfe811 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:32.310948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c32549-17bc-42b9-b30c-02f368c0110f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:32.329433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c32549-17bc-42b9-b30c-02f368c0110f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:32.434658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:32.463865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be400329-aef1-468c-9acd-aa79e33009cb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:32.464967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be400329-aef1-468c-9acd-aa79e33009cb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:32.465385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be400329-aef1-468c-9acd-aa79e33009cb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:32.465727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be400329-aef1-468c-9acd-aa79e33009cb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/c77e01bce1d8485f928e3fff8cd6174d/users/71641ecd6c354becafeb749efdd49e11/roles/a9f0ff76efcf4c87abac8e4065b7c6cc` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:32.469030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be400329-aef1-468c-9acd-aa79e33009cb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_grant(project_id=c77e01bce1d8485f928e3fff8cd6174d, user_id=71641ecd6c354becafeb749efdd49e11, role_id=a9f0ff76efcf4c87abac8e4065b7c6cc)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:32.474457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be400329-aef1-468c-9acd-aa79e33009cb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:32.570154 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1950/3885] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:32 2022] PUT /identity/v3/projects/c77e01bce1d8485f928e3fff8cd6174d/users/71641ecd6c354becafeb749efdd49e11/roles/a9f0ff76efcf4c87abac8e4065b7c6cc => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:32.573722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3346e-94b0-49bb-8354-ee4a87f329d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:32.593468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b3346e-94b0-49bb-8354-ee4a87f329d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:32.672773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5f9ee-4979-4934-8642-0dd9763d74e7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:32.673627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5f9ee-4979-4934-8642-0dd9763d74e7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:32.674134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5f9ee-4979-4934-8642-0dd9763d74e7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:32.674422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5f9ee-4979-4934-8642-0dd9763d74e7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:32.675537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5f9ee-4979-4934-8642-0dd9763d74e7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:32.677569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5f9ee-4979-4934-8642-0dd9763d74e7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:32.710030 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1936/3886] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:32 2022] GET /identity/v3/users => generated 8639 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:32.715233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4c241-89f7-4e25-a916-aa4ed3962412 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:32.716808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4c241-89f7-4e25-a916-aa4ed3962412 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:32.717318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4c241-89f7-4e25-a916-aa4ed3962412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:32.796013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0af4c241-89f7-4e25-a916-aa4ed3962412 None None] MFA Rules not processed for user `71641ecd6c354becafeb749efdd49e11`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:32.798979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0af4c241-89f7-4e25-a916-aa4ed3962412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:32.935481 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1951/3887] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:32 2022] POST /identity/v3/auth/tokens => generated 3269 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:32.937874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7472b10-988f-4de4-90d3-319917984f49 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:32.938164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7472b10-988f-4de4-90d3-319917984f49 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:32.938443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7472b10-988f-4de4-90d3-319917984f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:32.960059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b7472b10-988f-4de4-90d3-319917984f49 None None] MFA Rules not processed for user `71641ecd6c354becafeb749efdd49e11`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:32.962266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b7472b10-988f-4de4-90d3-319917984f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:32.974587 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1937/3888] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:32 2022] POST /identity/v3/auth/tokens => generated 3269 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:32.979556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4c241-89f7-4e25-a916-aa4ed3962412 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:32.992157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4c241-89f7-4e25-a916-aa4ed3962412 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:32.998077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e07a7-d1e0-401b-93f0-38b67a9b0d56 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71641ecd6c354becafeb749efdd49e11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77e01bce1d8485f928e3fff8cd6174d', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1051993881', 'tempest-DelegatedRole-199825602'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:32.999624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e07a7-d1e0-401b-93f0-38b67a9b0d56 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:32.999624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e07a7-d1e0-401b-93f0-38b67a9b0d56 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:32.999624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e07a7-d1e0-401b-93f0-38b67a9b0d56 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:33.001133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e07a7-d1e0-401b-93f0-38b67a9b0d56 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] RBAC: Authorizing `identity:create_trust()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:33.002813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e07a7-d1e0-401b-93f0-38b67a9b0d56 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:33.375235 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1952/3889] 10.222.0.33 () {60 vars in 1173 bytes} [Sat May 28 21:14:32 2022] POST /identity/v3/OS-TRUST/trusts => generated 833 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:33.377808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7472b10-988f-4de4-90d3-319917984f49 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:33.387717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7472b10-988f-4de4-90d3-319917984f49 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:33.402300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9645c-30b9-4c3f-b8c9-cb2aad713e64 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71641ecd6c354becafeb749efdd49e11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77e01bce1d8485f928e3fff8cd6174d', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1051993881', 'tempest-DelegatedRole-199825602'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:33.403438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9645c-30b9-4c3f-b8c9-cb2aad713e64 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:33.403908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9645c-30b9-4c3f-b8c9-cb2aad713e64 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:33.404396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc9645c-30b9-4c3f-b8c9-cb2aad713e64 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:33.405610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9645c-30b9-4c3f-b8c9-cb2aad713e64 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] RBAC: Authorizing `identity:list_trusts_for_trustor()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:33.413575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc9645c-30b9-4c3f-b8c9-cb2aad713e64 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:33.425300 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1938/3890] 10.222.0.33 () {58 vars in 1251 bytes} [Sat May 28 21:14:33 2022] GET /identity/v3/OS-TRUST/trusts/?trustor_user_id=71641ecd6c354becafeb749efdd49e11 => generated 611 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:33.430021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e07a7-d1e0-401b-93f0-38b67a9b0d56 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:33.441121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e07a7-d1e0-401b-93f0-38b67a9b0d56 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:33.450971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7dd86-d041-4058-82c0-85b335b461c5 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71641ecd6c354becafeb749efdd49e11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77e01bce1d8485f928e3fff8cd6174d', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1051993881', 'tempest-DelegatedRole-199825602'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:33.451833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7dd86-d041-4058-82c0-85b335b461c5 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:33.452113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7dd86-d041-4058-82c0-85b335b461c5 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:33.452394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d7dd86-d041-4058-82c0-85b335b461c5 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] PATH_INFO: `/v3/OS-TRUST/trusts/3ecbf9f946f84f6fa8ba06846f64c818` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:33.462255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7dd86-d041-4058-82c0-85b335b461c5 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] RBAC: Authorizing `identity:delete_trust(trust_id=3ecbf9f946f84f6fa8ba06846f64c818)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:33.465688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d7dd86-d041-4058-82c0-85b335b461c5 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:33.676206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20d7dd86-d041-4058-82c0-85b335b461c5 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '3ecbf9f946f84f6fa8ba06846f64c818', 'request_id': 'req-20d7dd86-d041-4058-82c0-85b335b461c5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:33.681948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20d7dd86-d041-4058-82c0-85b335b461c5 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '3ecbf9f946f84f6fa8ba06846f64c818', 'request_id': 'req-20d7dd86-d041-4058-82c0-85b335b461c5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:33.832293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20d7dd86-d041-4058-82c0-85b335b461c5 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '3ecbf9f946f84f6fa8ba06846f64c818', 'request_id': 'req-20d7dd86-d041-4058-82c0-85b335b461c5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:33.858987 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1953/3891] 10.222.0.33 () {58 vars in 1253 bytes} [Sat May 28 21:14:33 2022] DELETE /identity/v3/OS-TRUST/trusts/3ecbf9f946f84f6fa8ba06846f64c818 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:33.860739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9645c-30b9-4c3f-b8c9-cb2aad713e64 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:33.862691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4dc9645c-30b9-4c3f-b8c9-cb2aad713e64 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:33.885349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc9645c-30b9-4c3f-b8c9-cb2aad713e64 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:33.890893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ab229-bc95-4515-b5dd-d2eb8a556e59 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71641ecd6c354becafeb749efdd49e11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c77e01bce1d8485f928e3fff8cd6174d', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1051993881', 'tempest-DelegatedRole-199825602'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:33.891884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ab229-bc95-4515-b5dd-d2eb8a556e59 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:33.892143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ab229-bc95-4515-b5dd-d2eb8a556e59 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:33.892452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ab229-bc95-4515-b5dd-d2eb8a556e59 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] PATH_INFO: `/v3/OS-TRUST/trusts/3ecbf9f946f84f6fa8ba06846f64c818` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:33.910961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-f89ab229-bc95-4515-b5dd-d2eb8a556e59 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] Could not find trust: 3ecbf9f946f84f6fa8ba06846f64c818.: keystone.exception.TrustNotFound: Could not find trust: 3ecbf9f946f84f6fa8ba06846f64c818. May 28 21:14:33.910961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:14:33.910961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:14:33.910961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:14:33.910961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:14:33.910961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:14:33.910961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:14:33.910961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:14:33.910961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:14:33.910961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:14:33.910961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:14:33.910961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:14:33.910961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get May 28 21:14:33.910961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) May 28 21:14:33.910961 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust May 28 21:14:33.917234 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', May 28 21:14:33.917234 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call May 28 21:14:33.917234 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() May 28 21:14:33.917234 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement May 28 21:14:33.917234 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( May 28 21:14:33.917234 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:14:33.917234 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:14:33.917234 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust May 28 21:14:33.917234 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) May 28 21:14:33.917234 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust May 28 21:14:33.917234 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) May 28 21:14:33.917234 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 3ecbf9f946f84f6fa8ba06846f64c818. May 28 21:14:33.917234 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:14:33.917234 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1939/3892] 10.222.0.33 () {58 vars in 1250 bytes} [Sat May 28 21:14:33 2022] GET /identity/v3/OS-TRUST/trusts/3ecbf9f946f84f6fa8ba06846f64c818 => generated 111 bytes in 55 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:14:33.919444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7dd86-d041-4058-82c0-85b335b461c5 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:33.920997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-20d7dd86-d041-4058-82c0-85b335b461c5 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:33.940225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d7dd86-d041-4058-82c0-85b335b461c5 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:33.969117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c061ea-6a84-4a7d-9b66-000eb378d522 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:33.971953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c061ea-6a84-4a7d-9b66-000eb378d522 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:33.972239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c061ea-6a84-4a7d-9b66-000eb378d522 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:33.973586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c061ea-6a84-4a7d-9b66-000eb378d522 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles/a9f0ff76efcf4c87abac8e4065b7c6cc` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:33.977609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c061ea-6a84-4a7d-9b66-000eb378d522 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_role(role_id=a9f0ff76efcf4c87abac8e4065b7c6cc)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:33.982554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c061ea-6a84-4a7d-9b66-000eb378d522 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:34.338572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-99c061ea-6a84-4a7d-9b66-000eb378d522 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invalidating the token cache because role a9f0ff76efcf4c87abac8e4065b7c6cc has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:34.339806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-99c061ea-6a84-4a7d-9b66-000eb378d522 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:34.340726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-99c061ea-6a84-4a7d-9b66-000eb378d522 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:34.344881 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1954/3893] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:33 2022] DELETE /identity/v3/roles/a9f0ff76efcf4c87abac8e4065b7c6cc => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:34.349398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ab229-bc95-4515-b5dd-d2eb8a556e59 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:34.356049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f89ab229-bc95-4515-b5dd-d2eb8a556e59 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:34.403335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ab229-bc95-4515-b5dd-d2eb8a556e59 tempest-TrustsV3TestJSON-2003698082 tempest-user-466669551] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:34.527398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bba669-05ea-4753-9ac5-36af762728b4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:34.528454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bba669-05ea-4753-9ac5-36af762728b4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:34.528812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bba669-05ea-4753-9ac5-36af762728b4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:34.529333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bba669-05ea-4753-9ac5-36af762728b4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles/310f682586984e819815ae3c7bbfe811` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:34.531196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bba669-05ea-4753-9ac5-36af762728b4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_role(role_id=310f682586984e819815ae3c7bbfe811)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:34.533303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bba669-05ea-4753-9ac5-36af762728b4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:34.760375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c061ea-6a84-4a7d-9b66-000eb378d522 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:34.762591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-99c061ea-6a84-4a7d-9b66-000eb378d522 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:34.803246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51bba669-05ea-4753-9ac5-36af762728b4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invalidating the token cache because role 310f682586984e819815ae3c7bbfe811 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:34.803598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51bba669-05ea-4753-9ac5-36af762728b4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:34.807195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51bba669-05ea-4753-9ac5-36af762728b4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:34.812853 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1940/3894] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:34 2022] DELETE /identity/v3/roles/310f682586984e819815ae3c7bbfe811 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:34.818176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bba669-05ea-4753-9ac5-36af762728b4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:34.820720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-51bba669-05ea-4753-9ac5-36af762728b4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:34.937761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c061ea-6a84-4a7d-9b66-000eb378d522 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:34.988630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bba669-05ea-4753-9ac5-36af762728b4 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:35.037796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a580c4-5da7-4094-a2c5-61a7ef1235d2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:35.044848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a580c4-5da7-4094-a2c5-61a7ef1235d2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:35.044965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a580c4-5da7-4094-a2c5-61a7ef1235d2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:35.045140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a580c4-5da7-4094-a2c5-61a7ef1235d2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users/71641ecd6c354becafeb749efdd49e11` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:35.070075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a580c4-5da7-4094-a2c5-61a7ef1235d2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_user(user_id=71641ecd6c354becafeb749efdd49e11)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:35.079342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a580c4-5da7-4094-a2c5-61a7ef1235d2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:35.089080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17c10c-dd8a-46a2-9890-4667e5a14b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:35.092527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17c10c-dd8a-46a2-9890-4667e5a14b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:35.092791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17c10c-dd8a-46a2-9890-4667e5a14b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:35.093080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17c10c-dd8a-46a2-9890-4667e5a14b3c admin admin] PATH_INFO: `/v3/users/cf4bbfcd9bf64e3eabac163461e98f66` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:35.094993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17c10c-dd8a-46a2-9890-4667e5a14b3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf4bbfcd9bf64e3eabac163461e98f66)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:35.117046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17c10c-dd8a-46a2-9890-4667e5a14b3c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:35.308493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=1,keystone:SELECT=96,keystone:DELETE=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:35.314768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-09a580c4-5da7-4094-a2c5-61a7ef1235d2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71641ecd6c354becafeb749efdd49e11'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:35.315791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-09a580c4-5da7-4094-a2c5-61a7ef1235d2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71641ecd6c354becafeb749efdd49e11'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:35.326685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-09a580c4-5da7-4094-a2c5-61a7ef1235d2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71641ecd6c354becafeb749efdd49e11'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:35.398837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7d17c10c-dd8a-46a2-9890-4667e5a14b3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf4bbfcd9bf64e3eabac163461e98f66'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:35.408957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7d17c10c-dd8a-46a2-9890-4667e5a14b3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf4bbfcd9bf64e3eabac163461e98f66'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:35.409709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7d17c10c-dd8a-46a2-9890-4667e5a14b3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf4bbfcd9bf64e3eabac163461e98f66'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:35.410366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7d17c10c-dd8a-46a2-9890-4667e5a14b3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf4bbfcd9bf64e3eabac163461e98f66'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:35.431164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-09a580c4-5da7-4094-a2c5-61a7ef1235d2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71641ecd6c354becafeb749efdd49e11'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:35.431818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-09a580c4-5da7-4094-a2c5-61a7ef1235d2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71641ecd6c354becafeb749efdd49e11'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:35.461522 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1941/3895] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:34 2022] DELETE /identity/v3/users/71641ecd6c354becafeb749efdd49e11 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:14:35.463028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f910eb-39bb-421c-9652-522c287c3718 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:35.463325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f910eb-39bb-421c-9652-522c287c3718 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:35.463610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f910eb-39bb-421c-9652-522c287c3718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:35.486479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e1f910eb-39bb-421c-9652-522c287c3718 None None] MFA Rules not processed for user `bf9c8f66e7d24ca9baca835e41546f22`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:35.488008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e1f910eb-39bb-421c-9652-522c287c3718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:35.575236 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1942/3896] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:35.576337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f910eb-39bb-421c-9652-522c287c3718 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:35.577636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e1f910eb-39bb-421c-9652-522c287c3718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:35.642213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7d17c10c-dd8a-46a2-9890-4667e5a14b3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf4bbfcd9bf64e3eabac163461e98f66'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:35.658818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f910eb-39bb-421c-9652-522c287c3718 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:35.667840 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1955/3897] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:34 2022] DELETE /identity/v3/users/cf4bbfcd9bf64e3eabac163461e98f66 => generated 0 bytes in 908 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:14:35.668862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cea5cb-916f-455a-9349-b9b36ea999d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:35.669875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cea5cb-916f-455a-9349-b9b36ea999d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:35.670578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cea5cb-916f-455a-9349-b9b36ea999d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:35.670954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cea5cb-916f-455a-9349-b9b36ea999d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/c77e01bce1d8485f928e3fff8cd6174d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:35.672294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cea5cb-916f-455a-9349-b9b36ea999d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c77e01bce1d8485f928e3fff8cd6174d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:35.673181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e5d18-24a4-49a8-8048-6b78f272f75c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:35.673478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e5d18-24a4-49a8-8048-6b78f272f75c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:35.673762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014e5d18-24a4-49a8-8048-6b78f272f75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:35.674716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cea5cb-916f-455a-9349-b9b36ea999d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:35.724363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-014e5d18-24a4-49a8-8048-6b78f272f75c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:35.726593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-014e5d18-24a4-49a8-8048-6b78f272f75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:35.833075 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1956/3898] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:35.859050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e5d18-24a4-49a8-8048-6b78f272f75c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:35.865603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-014e5d18-24a4-49a8-8048-6b78f272f75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:35.954786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-36cea5cb-916f-455a-9349-b9b36ea999d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c77e01bce1d8485f928e3fff8cd6174d', 'request_id': 'req-36cea5cb-916f-455a-9349-b9b36ea999d0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:35.961317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014e5d18-24a4-49a8-8048-6b78f272f75c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:35.979198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-36cea5cb-916f-455a-9349-b9b36ea999d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c77e01bce1d8485f928e3fff8cd6174d', 'request_id': 'req-36cea5cb-916f-455a-9349-b9b36ea999d0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:36.029113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed755219-5ca0-4863-98cd-860cce2b24aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:36.029786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed755219-5ca0-4863-98cd-860cce2b24aa service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:36.030114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed755219-5ca0-4863-98cd-860cce2b24aa service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:36.030397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed755219-5ca0-4863-98cd-860cce2b24aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:36.062021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-36cea5cb-916f-455a-9349-b9b36ea999d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c77e01bce1d8485f928e3fff8cd6174d', 'request_id': 'req-36cea5cb-916f-455a-9349-b9b36ea999d0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:36.088868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-36cea5cb-916f-455a-9349-b9b36ea999d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] The token cache is being invalidate because project c77e01bce1d8485f928e3fff8cd6174d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:36.090280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-36cea5cb-916f-455a-9349-b9b36ea999d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:36.092378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-36cea5cb-916f-455a-9349-b9b36ea999d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:36.095778 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1943/3899] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:35 2022] DELETE /identity/v3/projects/c77e01bce1d8485f928e3fff8cd6174d => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:36.101722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cea5cb-916f-455a-9349-b9b36ea999d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:36.103096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-36cea5cb-916f-455a-9349-b9b36ea999d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:36.155520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed755219-5ca0-4863-98cd-860cce2b24aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:36.157697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed755219-5ca0-4863-98cd-860cce2b24aa service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:36.173045 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1957/3900] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:14:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:14:36.224345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cea5cb-916f-455a-9349-b9b36ea999d0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:36.230120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84378dee-5184-4baa-a3dd-541da97a9cb7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:36.230809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84378dee-5184-4baa-a3dd-541da97a9cb7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:36.231099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84378dee-5184-4baa-a3dd-541da97a9cb7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:36.231381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84378dee-5184-4baa-a3dd-541da97a9cb7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:36.232685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84378dee-5184-4baa-a3dd-541da97a9cb7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:36.235255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84378dee-5184-4baa-a3dd-541da97a9cb7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:36.322240 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1944/3901] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:14:36 2022] POST /identity/v3/projects => generated 328 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:14:36.326985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed755219-5ca0-4863-98cd-860cce2b24aa service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:36.353979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed755219-5ca0-4863-98cd-860cce2b24aa service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:36.397363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:36.502926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037aae9a-e85c-4aaa-9582-4e1c2b547a0d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:36.503879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037aae9a-e85c-4aaa-9582-4e1c2b547a0d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:36.504446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037aae9a-e85c-4aaa-9582-4e1c2b547a0d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:36.504916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037aae9a-e85c-4aaa-9582-4e1c2b547a0d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:36.505813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037aae9a-e85c-4aaa-9582-4e1c2b547a0d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:36.507841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037aae9a-e85c-4aaa-9582-4e1c2b547a0d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:36.523641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84378dee-5184-4baa-a3dd-541da97a9cb7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:36.525492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-84378dee-5184-4baa-a3dd-541da97a9cb7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:36.630730 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1958/3902] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:14:36 2022] POST /identity/v3/users => generated 413 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:36.633188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037aae9a-e85c-4aaa-9582-4e1c2b547a0d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:36.646005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037aae9a-e85c-4aaa-9582-4e1c2b547a0d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:36.653510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36bea6-e42c-412c-aa71-23fd6108688d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:36.654904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36bea6-e42c-412c-aa71-23fd6108688d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:36.657855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36bea6-e42c-412c-aa71-23fd6108688d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:36.657855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36bea6-e42c-412c-aa71-23fd6108688d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:36.657855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36bea6-e42c-412c-aa71-23fd6108688d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:36.659043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36bea6-e42c-412c-aa71-23fd6108688d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:36.676356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84378dee-5184-4baa-a3dd-541da97a9cb7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:36.683915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8e0d9-eaad-4bad-800c-21f531715a81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:36.685014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8e0d9-eaad-4bad-800c-21f531715a81 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:36.685555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8e0d9-eaad-4bad-800c-21f531715a81 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:36.686164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8e0d9-eaad-4bad-800c-21f531715a81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:36.688456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d5f8e0d9-eaad-4bad-800c-21f531715a81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:36.720255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8e0d9-eaad-4bad-800c-21f531715a81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:36.722075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8e0d9-eaad-4bad-800c-21f531715a81 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:36.748801 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1945/3903] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:14:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:14:36.801447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8e0d9-eaad-4bad-800c-21f531715a81 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:36.803302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d5f8e0d9-eaad-4bad-800c-21f531715a81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:36.829236 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1959/3904] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:14:36 2022] POST /identity/v3/roles => generated 243 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:36.831395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36bea6-e42c-412c-aa71-23fd6108688d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:36.872864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36bea6-e42c-412c-aa71-23fd6108688d tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:36.893958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f650259-e1c4-4c76-8cb0-6509c2470a62 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:36.893958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f650259-e1c4-4c76-8cb0-6509c2470a62 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:36.893958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f650259-e1c4-4c76-8cb0-6509c2470a62 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:36.894456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f650259-e1c4-4c76-8cb0-6509c2470a62 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:36.894456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f650259-e1c4-4c76-8cb0-6509c2470a62 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:36.902802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f650259-e1c4-4c76-8cb0-6509c2470a62 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:36.941863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8e0d9-eaad-4bad-800c-21f531715a81 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:36.948010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef4c1d-b884-4487-9250-63b385bd8040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:36.948955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef4c1d-b884-4487-9250-63b385bd8040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:36.949248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef4c1d-b884-4487-9250-63b385bd8040 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:36.949581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef4c1d-b884-4487-9250-63b385bd8040 admin admin] PATH_INFO: `/v3/users/25b2f1524f7f48149cbc5d2c694a9e2b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:36.951254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef4c1d-b884-4487-9250-63b385bd8040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25b2f1524f7f48149cbc5d2c694a9e2b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:36.954016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef4c1d-b884-4487-9250-63b385bd8040 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:37.080737 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1960/3905] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:14:36 2022] POST /identity/v3/roles => generated 246 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:37.081891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f650259-e1c4-4c76-8cb0-6509c2470a62 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:37.113644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f650259-e1c4-4c76-8cb0-6509c2470a62 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:37.133770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07918e-3616-46c1-ab80-9fdede63baca tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:37.133770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07918e-3616-46c1-ab80-9fdede63baca tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:37.133770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07918e-3616-46c1-ab80-9fdede63baca tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:37.134371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07918e-3616-46c1-ab80-9fdede63baca tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/e1ca681e67644e6f99014e268b4b6fbf/users/dd1fb3b7573d4213a1e9f7cb83a336e5/roles/46f30a87e4304a2798298c0b83a61b3d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:37.173815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07918e-3616-46c1-ab80-9fdede63baca tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_grant(project_id=e1ca681e67644e6f99014e268b4b6fbf, user_id=dd1fb3b7573d4213a1e9f7cb83a336e5, role_id=46f30a87e4304a2798298c0b83a61b3d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:37.179630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07918e-3616-46c1-ab80-9fdede63baca tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:37.245975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-adef4c1d-b884-4487-9250-63b385bd8040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25b2f1524f7f48149cbc5d2c694a9e2b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:37.246666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-adef4c1d-b884-4487-9250-63b385bd8040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25b2f1524f7f48149cbc5d2c694a9e2b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:37.261436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-adef4c1d-b884-4487-9250-63b385bd8040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25b2f1524f7f48149cbc5d2c694a9e2b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:37.274635 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1961/3906] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:37 2022] PUT /identity/v3/projects/e1ca681e67644e6f99014e268b4b6fbf/users/dd1fb3b7573d4213a1e9f7cb83a336e5/roles/46f30a87e4304a2798298c0b83a61b3d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:37.277763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07918e-3616-46c1-ab80-9fdede63baca tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:37.281705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ba07918e-3616-46c1-ab80-9fdede63baca tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:37.345490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-adef4c1d-b884-4487-9250-63b385bd8040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25b2f1524f7f48149cbc5d2c694a9e2b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:37.346162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-adef4c1d-b884-4487-9250-63b385bd8040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25b2f1524f7f48149cbc5d2c694a9e2b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:37.401668 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1946/3907] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:36 2022] DELETE /identity/v3/users/25b2f1524f7f48149cbc5d2c694a9e2b => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:37.402639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef4c1d-b884-4487-9250-63b385bd8040 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:37.404029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-adef4c1d-b884-4487-9250-63b385bd8040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:37.426301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07918e-3616-46c1-ab80-9fdede63baca tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:37.585909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef4c1d-b884-4487-9250-63b385bd8040 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:37.591537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08895e-2b8b-46a3-8212-b4511ba3f55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:37.592763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08895e-2b8b-46a3-8212-b4511ba3f55b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:37.593432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08895e-2b8b-46a3-8212-b4511ba3f55b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:37.594188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08895e-2b8b-46a3-8212-b4511ba3f55b admin admin] PATH_INFO: `/v3/projects/149d4937782d47f1b75fcdcfb4a89707` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:37.595543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61b0d6-4075-49c5-9095-d2a0d77c37a0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:37.599606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61b0d6-4075-49c5-9095-d2a0d77c37a0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:37.599949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61b0d6-4075-49c5-9095-d2a0d77c37a0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:37.600394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac61b0d6-4075-49c5-9095-d2a0d77c37a0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/e1ca681e67644e6f99014e268b4b6fbf/users/dd1fb3b7573d4213a1e9f7cb83a336e5/roles/2bc1bfc233144a0eacfcefa331c30537` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:37.608870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea08895e-2b8b-46a3-8212-b4511ba3f55b admin admin] RBAC: Authorizing `identity:delete_project(project_id=149d4937782d47f1b75fcdcfb4a89707)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:37.612898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61b0d6-4075-49c5-9095-d2a0d77c37a0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_grant(project_id=e1ca681e67644e6f99014e268b4b6fbf, user_id=dd1fb3b7573d4213a1e9f7cb83a336e5, role_id=2bc1bfc233144a0eacfcefa331c30537)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:37.614602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea08895e-2b8b-46a3-8212-b4511ba3f55b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:37.615103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac61b0d6-4075-49c5-9095-d2a0d77c37a0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:37.662682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:37.725694 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1947/3908] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:37 2022] PUT /identity/v3/projects/e1ca681e67644e6f99014e268b4b6fbf/users/dd1fb3b7573d4213a1e9f7cb83a336e5/roles/2bc1bfc233144a0eacfcefa331c30537 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:37.729714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61b0d6-4075-49c5-9095-d2a0d77c37a0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:37.820970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac61b0d6-4075-49c5-9095-d2a0d77c37a0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:37.827908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2080b92f-b6f5-4446-aefc-0f4240a7816e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:37.834264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080b92f-b6f5-4446-aefc-0f4240a7816e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:37.834578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080b92f-b6f5-4446-aefc-0f4240a7816e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:37.840390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080b92f-b6f5-4446-aefc-0f4240a7816e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:37.840390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080b92f-b6f5-4446-aefc-0f4240a7816e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:37.840390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2080b92f-b6f5-4446-aefc-0f4240a7816e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:37.865223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ea08895e-2b8b-46a3-8212-b4511ba3f55b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '149d4937782d47f1b75fcdcfb4a89707', 'request_id': 'req-ea08895e-2b8b-46a3-8212-b4511ba3f55b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:37.873815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ea08895e-2b8b-46a3-8212-b4511ba3f55b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '149d4937782d47f1b75fcdcfb4a89707', 'request_id': 'req-ea08895e-2b8b-46a3-8212-b4511ba3f55b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:37.898738 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1948/3909] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:37 2022] GET /identity/v3/users => generated 8075 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:37.903374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae238d0-eede-435e-884c-ed0a5358e241 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:37.903902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae238d0-eede-435e-884c-ed0a5358e241 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:37.906786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae238d0-eede-435e-884c-ed0a5358e241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:37.964194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ea08895e-2b8b-46a3-8212-b4511ba3f55b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '149d4937782d47f1b75fcdcfb4a89707', 'request_id': 'req-ea08895e-2b8b-46a3-8212-b4511ba3f55b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:37.981566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ea08895e-2b8b-46a3-8212-b4511ba3f55b admin admin] The token cache is being invalidate because project 149d4937782d47f1b75fcdcfb4a89707 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:37.982304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ea08895e-2b8b-46a3-8212-b4511ba3f55b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:37.983451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ea08895e-2b8b-46a3-8212-b4511ba3f55b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:37.987692 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1962/3910] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:37 2022] DELETE /identity/v3/projects/149d4937782d47f1b75fcdcfb4a89707 => generated 0 bytes in 713 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:37.996161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9a322-21fb-4b5b-ba0c-456c71c260cf None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:37.998388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7ae238d0-eede-435e-884c-ed0a5358e241 None None] MFA Rules not processed for user `dd1fb3b7573d4213a1e9f7cb83a336e5`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:37.998805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9a322-21fb-4b5b-ba0c-456c71c260cf None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:37.999161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9a322-21fb-4b5b-ba0c-456c71c260cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:38.003641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7ae238d0-eede-435e-884c-ed0a5358e241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:38.057206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a0b9a322-21fb-4b5b-ba0c-456c71c260cf None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:38.058923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a0b9a322-21fb-4b5b-ba0c-456c71c260cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:38.157689 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1949/3911] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:37 2022] POST /identity/v3/auth/tokens => generated 3267 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:38.159685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089df6a-21af-4ad9-849e-0d7998845560 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:38.160081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089df6a-21af-4ad9-849e-0d7998845560 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:38.160877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c089df6a-21af-4ad9-849e-0d7998845560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:38.172613 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1963/3912] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:38.187416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9a322-21fb-4b5b-ba0c-456c71c260cf None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:38.188399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c089df6a-21af-4ad9-849e-0d7998845560 None None] MFA Rules not processed for user `dd1fb3b7573d4213a1e9f7cb83a336e5`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:38.188911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a0b9a322-21fb-4b5b-ba0c-456c71c260cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:38.190290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c089df6a-21af-4ad9-849e-0d7998845560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:38.207506 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1950/3913] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:38 2022] POST /identity/v3/auth/tokens => generated 3267 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:38.207506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08870051-e5bd-4c1c-94c3-4c7382c4e83f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:38.207506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08870051-e5bd-4c1c-94c3-4c7382c4e83f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:38.207506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08870051-e5bd-4c1c-94c3-4c7382c4e83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:38.224246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9a322-21fb-4b5b-ba0c-456c71c260cf None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:38.235302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-08870051-e5bd-4c1c-94c3-4c7382c4e83f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:38.236731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-08870051-e5bd-4c1c-94c3-4c7382c4e83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:38.237815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2572331-ae2c-4216-af13-272ccb11b9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:38.238879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2572331-ae2c-4216-af13-272ccb11b9a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:38.239150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2572331-ae2c-4216-af13-272ccb11b9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:38.239945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2572331-ae2c-4216-af13-272ccb11b9a5 admin admin] PATH_INFO: `/v3/projects/53ecc00f85b54d01aa7dbe2f0386dac6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:38.241938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2572331-ae2c-4216-af13-272ccb11b9a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53ecc00f85b54d01aa7dbe2f0386dac6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:38.243820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2572331-ae2c-4216-af13-272ccb11b9a5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:38.248056 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1951/3914] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:38.248762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08870051-e5bd-4c1c-94c3-4c7382c4e83f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:38.261454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08870051-e5bd-4c1c-94c3-4c7382c4e83f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:38.266512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd1ec3-ab5a-4d98-821b-3e356b5473fc tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1fb3b7573d4213a1e9f7cb83a336e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1ca681e67644e6f99014e268b4b6fbf', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-903447527', 'tempest-NotDelegatedRole-774504629'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:38.267298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd1ec3-ab5a-4d98-821b-3e356b5473fc tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:38.268206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd1ec3-ab5a-4d98-821b-3e356b5473fc tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:38.268206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd1ec3-ab5a-4d98-821b-3e356b5473fc tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:38.269058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd1ec3-ab5a-4d98-821b-3e356b5473fc tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: Authorizing `identity:create_trust()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:38.270325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd1ec3-ab5a-4d98-821b-3e356b5473fc tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:38.303653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:UPDATE=1,keystone:DELETE=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:38.374073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2572331-ae2c-4216-af13-272ccb11b9a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53ecc00f85b54d01aa7dbe2f0386dac6', 'request_id': 'req-e2572331-ae2c-4216-af13-272ccb11b9a5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:38.379587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2572331-ae2c-4216-af13-272ccb11b9a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53ecc00f85b54d01aa7dbe2f0386dac6', 'request_id': 'req-e2572331-ae2c-4216-af13-272ccb11b9a5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:38.446317 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1952/3915] 10.222.0.33 () {60 vars in 1173 bytes} [Sat May 28 21:14:38 2022] POST /identity/v3/OS-TRUST/trusts => generated 858 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:38.450259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd1ec3-ab5a-4d98-821b-3e356b5473fc tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:38.466744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd1ec3-ab5a-4d98-821b-3e356b5473fc tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:38.486027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2572331-ae2c-4216-af13-272ccb11b9a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53ecc00f85b54d01aa7dbe2f0386dac6', 'request_id': 'req-e2572331-ae2c-4216-af13-272ccb11b9a5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:38.499540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2572331-ae2c-4216-af13-272ccb11b9a5 admin admin] The token cache is being invalidate because project 53ecc00f85b54d01aa7dbe2f0386dac6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:38.499840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2572331-ae2c-4216-af13-272ccb11b9a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:38.500659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2572331-ae2c-4216-af13-272ccb11b9a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:38.503236 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1964/3916] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:38 2022] DELETE /identity/v3/projects/53ecc00f85b54d01aa7dbe2f0386dac6 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:38.507879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2572331-ae2c-4216-af13-272ccb11b9a5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:38.509591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e2572331-ae2c-4216-af13-272ccb11b9a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:38.533713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1efe6-bb0f-49cc-b121-aa1c988fa156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:38.535572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1efe6-bb0f-49cc-b121-aa1c988fa156 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:38.535863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1efe6-bb0f-49cc-b121-aa1c988fa156 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:38.536319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1efe6-bb0f-49cc-b121-aa1c988fa156 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:38.539770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1efe6-bb0f-49cc-b121-aa1c988fa156 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:14:38.542968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1efe6-bb0f-49cc-b121-aa1c988fa156 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:38.542968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b1efe6-bb0f-49cc-b121-aa1c988fa156 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:38.558578 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1953/3917] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:38.558578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223fd8d-a95a-49af-94e5-92987249aa12 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:38.558578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223fd8d-a95a-49af-94e5-92987249aa12 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:38.558578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223fd8d-a95a-49af-94e5-92987249aa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:38.577014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1223fd8d-a95a-49af-94e5-92987249aa12 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:38.578278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1223fd8d-a95a-49af-94e5-92987249aa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:38.587334 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1954/3918] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:38.588141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223fd8d-a95a-49af-94e5-92987249aa12 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:38.589429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1223fd8d-a95a-49af-94e5-92987249aa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:38.603735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223fd8d-a95a-49af-94e5-92987249aa12 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:38.608955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83c8ea-0f69-434d-bcc5-499450579239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:38.609705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83c8ea-0f69-434d-bcc5-499450579239 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:38.610279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83c8ea-0f69-434d-bcc5-499450579239 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:38.610554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d83c8ea-0f69-434d-bcc5-499450579239 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:38.611249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d83c8ea-0f69-434d-bcc5-499450579239 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:38.614158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2572331-ae2c-4216-af13-272ccb11b9a5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:38.623317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d83c8ea-0f69-434d-bcc5-499450579239 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:38.626652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa953e1-ccf9-4a1d-9f87-cb1034981dc5 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1fb3b7573d4213a1e9f7cb83a336e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1ca681e67644e6f99014e268b4b6fbf', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-903447527', 'tempest-NotDelegatedRole-774504629'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:38.628152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa953e1-ccf9-4a1d-9f87-cb1034981dc5 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:38.628630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa953e1-ccf9-4a1d-9f87-cb1034981dc5 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:38.629153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa953e1-ccf9-4a1d-9f87-cb1034981dc5 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] PATH_INFO: `/v3/OS-TRUST/trusts/4dd7a1bc94544b5f8b35f14470308f0a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:38.640647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa953e1-ccf9-4a1d-9f87-cb1034981dc5 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: Authorizing `identity:get_trust(trust_id=4dd7a1bc94544b5f8b35f14470308f0a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:38.642057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa953e1-ccf9-4a1d-9f87-cb1034981dc5 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:38.652555 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1965/3919] 10.222.0.33 () {58 vars in 1250 bytes} [Sat May 28 21:14:38 2022] GET /identity/v3/OS-TRUST/trusts/4dd7a1bc94544b5f8b35f14470308f0a => generated 858 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:38.654459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c3d36-7ea9-41c2-8901-d618bb85c6ea None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:38.654766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c3d36-7ea9-41c2-8901-d618bb85c6ea None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:38.655255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c3d36-7ea9-41c2-8901-d618bb85c6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:38.676287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cc4c3d36-7ea9-41c2-8901-d618bb85c6ea None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:38.677549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cc4c3d36-7ea9-41c2-8901-d618bb85c6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:38.686798 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1966/3920] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:38.687863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c3d36-7ea9-41c2-8901-d618bb85c6ea None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:38.699699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c3d36-7ea9-41c2-8901-d618bb85c6ea None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:38.705143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491eac4-34a6-4fbc-a923-b45d72cb2939 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1fb3b7573d4213a1e9f7cb83a336e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1ca681e67644e6f99014e268b4b6fbf', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-903447527', 'tempest-NotDelegatedRole-774504629'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:38.705993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491eac4-34a6-4fbc-a923-b45d72cb2939 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:38.706700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491eac4-34a6-4fbc-a923-b45d72cb2939 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:38.706991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491eac4-34a6-4fbc-a923-b45d72cb2939 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] PATH_INFO: `/v3/OS-TRUST/trusts/4dd7a1bc94544b5f8b35f14470308f0a/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:38.714145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491eac4-34a6-4fbc-a923-b45d72cb2939 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=4dd7a1bc94544b5f8b35f14470308f0a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:38.719479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d491eac4-34a6-4fbc-a923-b45d72cb2939 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:38.733683 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1967/3921] 10.222.0.33 () {58 vars in 1268 bytes} [Sat May 28 21:14:38 2022] GET /identity/v3/OS-TRUST/trusts/4dd7a1bc94544b5f8b35f14470308f0a/roles => generated 370 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:38.734374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491eac4-34a6-4fbc-a923-b45d72cb2939 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:38.753925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d491eac4-34a6-4fbc-a923-b45d72cb2939 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:38.758956 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1955/3922] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:14:38 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:38.759857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83c8ea-0f69-434d-bcc5-499450579239 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:38.766799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9ee19-5c88-4ba8-b324-d0497539a1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:38.767508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9ee19-5c88-4ba8-b324-d0497539a1a0 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:38.767894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9ee19-5c88-4ba8-b324-d0497539a1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:38.768186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9ee19-5c88-4ba8-b324-d0497539a1a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:38.769106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9ee19-5c88-4ba8-b324-d0497539a1a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:14:38.769413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9ee19-5c88-4ba8-b324-d0497539a1a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:38.771310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df9ee19-5c88-4ba8-b324-d0497539a1a0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:38.784444 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1968/3923] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:38.785271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9ee19-5c88-4ba8-b324-d0497539a1a0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:38.795840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9ee19-5c88-4ba8-b324-d0497539a1a0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:38.822218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d83c8ea-0f69-434d-bcc5-499450579239 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:38.826527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce5843-8c4c-4c4b-86f9-9ff5b3f2dd42 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1fb3b7573d4213a1e9f7cb83a336e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1ca681e67644e6f99014e268b4b6fbf', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-903447527', 'tempest-NotDelegatedRole-774504629'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:38.827344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce5843-8c4c-4c4b-86f9-9ff5b3f2dd42 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:38.827621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce5843-8c4c-4c4b-86f9-9ff5b3f2dd42 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:38.827894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bce5843-8c4c-4c4b-86f9-9ff5b3f2dd42 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] PATH_INFO: `/v3/OS-TRUST/trusts/4dd7a1bc94544b5f8b35f14470308f0a/roles/46f30a87e4304a2798298c0b83a61b3d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:38.852085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce5843-8c4c-4c4b-86f9-9ff5b3f2dd42 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: Authorizing `identity:get_role_for_trust(trust_id=4dd7a1bc94544b5f8b35f14470308f0a, role_id=46f30a87e4304a2798298c0b83a61b3d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:38.854691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bce5843-8c4c-4c4b-86f9-9ff5b3f2dd42 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:38.859216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abcfca-7b53-4e1c-9cbe-a25de5420ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:38.864468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abcfca-7b53-4e1c-9cbe-a25de5420ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:38.864468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abcfca-7b53-4e1c-9cbe-a25de5420ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:38.864468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abcfca-7b53-4e1c-9cbe-a25de5420ab4 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:38.864468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86abcfca-7b53-4e1c-9cbe-a25de5420ab4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:38.864468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86abcfca-7b53-4e1c-9cbe-a25de5420ab4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:38.872746 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1956/3924] 10.222.0.33 () {58 vars in 1367 bytes} [Sat May 28 21:14:38 2022] GET /identity/v3/OS-TRUST/trusts/4dd7a1bc94544b5f8b35f14470308f0a/roles/46f30a87e4304a2798298c0b83a61b3d => generated 243 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:38.875423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce5843-8c4c-4c4b-86f9-9ff5b3f2dd42 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:38.891696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bce5843-8c4c-4c4b-86f9-9ff5b3f2dd42 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:38.896684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff1ef3-09f9-4603-81e2-3748eedb7536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:38.897433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff1ef3-09f9-4603-81e2-3748eedb7536 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:38.897894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff1ef3-09f9-4603-81e2-3748eedb7536 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:38.898441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff1ef3-09f9-4603-81e2-3748eedb7536 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:38.899171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff1ef3-09f9-4603-81e2-3748eedb7536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:38.901590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff1ef3-09f9-4603-81e2-3748eedb7536 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:38.993081 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1969/3925] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:14:38 2022] POST /identity/v3/users => generated 292 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:38.994123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abcfca-7b53-4e1c-9cbe-a25de5420ab4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:39.009090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86abcfca-7b53-4e1c-9cbe-a25de5420ab4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:39.019051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0414a1-4c80-416b-af87-529da802e0ad tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1fb3b7573d4213a1e9f7cb83a336e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1ca681e67644e6f99014e268b4b6fbf', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-903447527', 'tempest-NotDelegatedRole-774504629'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:39.020537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0414a1-4c80-416b-af87-529da802e0ad tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] REQUEST_METHOD: `HEAD` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:39.021003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0414a1-4c80-416b-af87-529da802e0ad tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:39.021662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0414a1-4c80-416b-af87-529da802e0ad tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] PATH_INFO: `/v3/OS-TRUST/trusts/4dd7a1bc94544b5f8b35f14470308f0a/roles/46f30a87e4304a2798298c0b83a61b3d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:39.026315 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1957/3926] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:14:38 2022] POST /identity/v3/projects => generated 367 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:39.026315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff1ef3-09f9-4603-81e2-3748eedb7536 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:39.031482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0414a1-4c80-416b-af87-529da802e0ad tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: Authorizing `identity:get_role_for_trust(trust_id=4dd7a1bc94544b5f8b35f14470308f0a, role_id=46f30a87e4304a2798298c0b83a61b3d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:39.032944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0414a1-4c80-416b-af87-529da802e0ad tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:39.043941 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1970/3927] 10.222.0.33 () {58 vars in 1368 bytes} [Sat May 28 21:14:38 2022] HEAD /identity/v3/OS-TRUST/trusts/4dd7a1bc94544b5f8b35f14470308f0a/roles/46f30a87e4304a2798298c0b83a61b3d => generated 0 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) May 28 21:14:39.044713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0414a1-4c80-416b-af87-529da802e0ad tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:39.077715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0414a1-4c80-416b-af87-529da802e0ad tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:39.156865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff1ef3-09f9-4603-81e2-3748eedb7536 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:39.162119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e56b7b-6fd5-44f8-8488-ac27879a3831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:39.162802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e56b7b-6fd5-44f8-8488-ac27879a3831 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:39.164091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e56b7b-6fd5-44f8-8488-ac27879a3831 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:39.164381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e56b7b-6fd5-44f8-8488-ac27879a3831 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:39.165412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e56b7b-6fd5-44f8-8488-ac27879a3831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:39.167185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e56b7b-6fd5-44f8-8488-ac27879a3831 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:39.183832 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1958/3928] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:39 2022] GET /identity/v3/roles => generated 1891 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:39.185161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e56b7b-6fd5-44f8-8488-ac27879a3831 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:39.201050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0d3d2-fbc7-4b32-b638-62b375c425f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:39.205677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0d3d2-fbc7-4b32-b638-62b375c425f3 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:39.205677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0d3d2-fbc7-4b32-b638-62b375c425f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:39.205677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0d3d2-fbc7-4b32-b638-62b375c425f3 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:39.205677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0d3d2-fbc7-4b32-b638-62b375c425f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:39.207757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0d3d2-fbc7-4b32-b638-62b375c425f3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:39.256733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e56b7b-6fd5-44f8-8488-ac27879a3831 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:39.261394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f99680-cfa8-4036-9925-0a0163919c77 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1fb3b7573d4213a1e9f7cb83a336e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1ca681e67644e6f99014e268b4b6fbf', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-903447527', 'tempest-NotDelegatedRole-774504629'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:39.262321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f99680-cfa8-4036-9925-0a0163919c77 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:39.262609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f99680-cfa8-4036-9925-0a0163919c77 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:39.262892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f99680-cfa8-4036-9925-0a0163919c77 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] PATH_INFO: `/v3/OS-TRUST/trusts/4dd7a1bc94544b5f8b35f14470308f0a/roles/2bc1bfc233144a0eacfcefa331c30537` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:39.269514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f99680-cfa8-4036-9925-0a0163919c77 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: Authorizing `identity:get_role_for_trust(trust_id=4dd7a1bc94544b5f8b35f14470308f0a, role_id=2bc1bfc233144a0eacfcefa331c30537)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:39.271101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f99680-cfa8-4036-9925-0a0163919c77 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:39.290384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-f1f99680-cfa8-4036-9925-0a0163919c77 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Could not find role: 2bc1bfc233144a0eacfcefa331c30537.: keystone.exception.RoleNotFound: Could not find role: 2bc1bfc233144a0eacfcefa331c30537. May 28 21:14:39.290384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:14:39.290384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:14:39.290384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:14:39.290384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:14:39.290384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:14:39.290384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:14:39.290384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:14:39.290384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:14:39.290384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:14:39.290384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:14:39.290384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:14:39.290384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get May 28 21:14:39.290384 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 21:14:39.291341 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 2bc1bfc233144a0eacfcefa331c30537. May 28 21:14:39.291341 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:14:39.293172 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1959/3929] 10.222.0.33 () {58 vars in 1367 bytes} [Sat May 28 21:14:39 2022] GET /identity/v3/OS-TRUST/trusts/4dd7a1bc94544b5f8b35f14470308f0a/roles/2bc1bfc233144a0eacfcefa331c30537 => generated 110 bytes in 109 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:14:39.294312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f99680-cfa8-4036-9925-0a0163919c77 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:39.310712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f99680-cfa8-4036-9925-0a0163919c77 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:39.316363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d180d-62c7-40e0-9468-0aa7ee2e0ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:39.317188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d180d-62c7-40e0-9468-0aa7ee2e0ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:39.317482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d180d-62c7-40e0-9468-0aa7ee2e0ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:39.317765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d180d-62c7-40e0-9468-0aa7ee2e0ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:39.318554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d180d-62c7-40e0-9468-0aa7ee2e0ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:39.320934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d180d-62c7-40e0-9468-0aa7ee2e0ab0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:39.334441 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1971/3930] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:14:39 2022] POST /identity/v3/users => generated 284 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:39.334441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0d3d2-fbc7-4b32-b638-62b375c425f3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:39.347445 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1960/3931] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:39 2022] GET /identity/v3/roles => generated 1891 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:39.352734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0d3d2-fbc7-4b32-b638-62b375c425f3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:39.353390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d180d-62c7-40e0-9468-0aa7ee2e0ab0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:39.362563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03649d-6b4b-44f1-94bc-da156497d4f6 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1fb3b7573d4213a1e9f7cb83a336e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1ca681e67644e6f99014e268b4b6fbf', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-903447527', 'tempest-NotDelegatedRole-774504629'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:39.363413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03649d-6b4b-44f1-94bc-da156497d4f6 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] REQUEST_METHOD: `HEAD` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:39.363702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03649d-6b4b-44f1-94bc-da156497d4f6 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:39.363988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a03649d-6b4b-44f1-94bc-da156497d4f6 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] PATH_INFO: `/v3/OS-TRUST/trusts/4dd7a1bc94544b5f8b35f14470308f0a/roles/2bc1bfc233144a0eacfcefa331c30537` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:39.373735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03649d-6b4b-44f1-94bc-da156497d4f6 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: Authorizing `identity:get_role_for_trust(trust_id=4dd7a1bc94544b5f8b35f14470308f0a, role_id=2bc1bfc233144a0eacfcefa331c30537)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:39.374856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d180d-62c7-40e0-9468-0aa7ee2e0ab0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:39.375653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a03649d-6b4b-44f1-94bc-da156497d4f6 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:39.381307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a48f69-42b9-4ec0-ae53-0d2b644e319c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:39.382575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a48f69-42b9-4ec0-ae53-0d2b644e319c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:39.382965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a48f69-42b9-4ec0-ae53-0d2b644e319c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:39.383252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a48f69-42b9-4ec0-ae53-0d2b644e319c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:39.384049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a48f69-42b9-4ec0-ae53-0d2b644e319c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:39.385687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a48f69-42b9-4ec0-ae53-0d2b644e319c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:39.396012 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-0a03649d-6b4b-44f1-94bc-da156497d4f6 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Could not find role: 2bc1bfc233144a0eacfcefa331c30537.: keystone.exception.RoleNotFound: Could not find role: 2bc1bfc233144a0eacfcefa331c30537. May 28 21:14:39.396012 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:14:39.396012 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:14:39.396012 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:14:39.396012 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:14:39.396012 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:14:39.396012 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:14:39.396012 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:14:39.396012 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:14:39.396012 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:14:39.396012 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:14:39.396012 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:14:39.396012 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get May 28 21:14:39.396012 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 21:14:39.397178 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 2bc1bfc233144a0eacfcefa331c30537. May 28 21:14:39.397178 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:14:39.397736 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1972/3932] 10.222.0.33 () {58 vars in 1368 bytes} [Sat May 28 21:14:39 2022] HEAD /identity/v3/OS-TRUST/trusts/4dd7a1bc94544b5f8b35f14470308f0a/roles/2bc1bfc233144a0eacfcefa331c30537 => generated 0 bytes in 68 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) May 28 21:14:39.398333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03649d-6b4b-44f1-94bc-da156497d4f6 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:39.402363 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1961/3933] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:39 2022] GET /identity/v3/roles => generated 1891 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:39.403434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a48f69-42b9-4ec0-ae53-0d2b644e319c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:39.414624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a03649d-6b4b-44f1-94bc-da156497d4f6 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:39.417496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a48f69-42b9-4ec0-ae53-0d2b644e319c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:39.419650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a1199-137b-4847-89fb-15baa6c85239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:39.420547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a1199-137b-4847-89fb-15baa6c85239 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:39.420836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a1199-137b-4847-89fb-15baa6c85239 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:39.421117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a1199-137b-4847-89fb-15baa6c85239 admin admin] PATH_INFO: `/v3/projects/cf454e1b4e6a4c45af04865b154c41a9/users/98ab12b74029438a9a1d19e505d1fc6d/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:39.422648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aef49-ce6c-449a-8e52-cff1759c32c2 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1fb3b7573d4213a1e9f7cb83a336e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1ca681e67644e6f99014e268b4b6fbf', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-903447527', 'tempest-NotDelegatedRole-774504629'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:39.423753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aef49-ce6c-449a-8e52-cff1759c32c2 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:39.424252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aef49-ce6c-449a-8e52-cff1759c32c2 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:39.424979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aef49-ce6c-449a-8e52-cff1759c32c2 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] PATH_INFO: `/v3/OS-TRUST/trusts/4dd7a1bc94544b5f8b35f14470308f0a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:39.433968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aef49-ce6c-449a-8e52-cff1759c32c2 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: Authorizing `identity:delete_trust(trust_id=4dd7a1bc94544b5f8b35f14470308f0a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:39.439101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a1199-137b-4847-89fb-15baa6c85239 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf454e1b4e6a4c45af04865b154c41a9, user_id=98ab12b74029438a9a1d19e505d1fc6d, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:39.440542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aef49-ce6c-449a-8e52-cff1759c32c2 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:39.442632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a1199-137b-4847-89fb-15baa6c85239 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:39.540519 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1973/3934] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:39 2022] PUT /identity/v3/projects/cf454e1b4e6a4c45af04865b154c41a9/users/98ab12b74029438a9a1d19e505d1fc6d/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:39.540519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a1199-137b-4847-89fb-15baa6c85239 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:39.562947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-435aef49-ce6c-449a-8e52-cff1759c32c2 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '4dd7a1bc94544b5f8b35f14470308f0a', 'request_id': 'req-435aef49-ce6c-449a-8e52-cff1759c32c2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:39.567751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-435aef49-ce6c-449a-8e52-cff1759c32c2 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '4dd7a1bc94544b5f8b35f14470308f0a', 'request_id': 'req-435aef49-ce6c-449a-8e52-cff1759c32c2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:39.634627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a1199-137b-4847-89fb-15baa6c85239 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:39.639721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72b675-d6a0-406c-8815-7fc441b20855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:39.640563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72b675-d6a0-406c-8815-7fc441b20855 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:39.640873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72b675-d6a0-406c-8815-7fc441b20855 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:39.641157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72b675-d6a0-406c-8815-7fc441b20855 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:39.648343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72b675-d6a0-406c-8815-7fc441b20855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:39.648343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72b675-d6a0-406c-8815-7fc441b20855 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:39.655675 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1974/3935] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:39 2022] GET /identity/v3/roles => generated 1891 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:39.660402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e197940-4f4d-438b-a21a-5ebbbf882ae9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:39.660743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e197940-4f4d-438b-a21a-5ebbbf882ae9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:39.661027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e197940-4f4d-438b-a21a-5ebbbf882ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:39.695241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6e197940-4f4d-438b-a21a-5ebbbf882ae9 None None] MFA Rules not processed for user `98ab12b74029438a9a1d19e505d1fc6d`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:39.695241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6e197940-4f4d-438b-a21a-5ebbbf882ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:39.728001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-435aef49-ce6c-449a-8e52-cff1759c32c2 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '4dd7a1bc94544b5f8b35f14470308f0a', 'request_id': 'req-435aef49-ce6c-449a-8e52-cff1759c32c2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:39.740515 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1962/3936] 10.222.0.33 () {58 vars in 1253 bytes} [Sat May 28 21:14:39 2022] DELETE /identity/v3/OS-TRUST/trusts/4dd7a1bc94544b5f8b35f14470308f0a => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:39.740515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aef49-ce6c-449a-8e52-cff1759c32c2 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:39.740515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-435aef49-ce6c-449a-8e52-cff1759c32c2 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:39.769166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aef49-ce6c-449a-8e52-cff1759c32c2 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:39.777081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4572c0-0041-40f8-a335-425e7fbbc2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:39.783653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4572c0-0041-40f8-a335-425e7fbbc2e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:39.783653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4572c0-0041-40f8-a335-425e7fbbc2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:39.783653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4572c0-0041-40f8-a335-425e7fbbc2e0 admin admin] PATH_INFO: `/v3/projects/5905c4b42ecd40298340a8a2ce4d5e24/users/572170ccd67e4facbf10dad7cd1491c1/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:39.796038 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1975/3937] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:39.796904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e197940-4f4d-438b-a21a-5ebbbf882ae9 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:39.797878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4572c0-0041-40f8-a335-425e7fbbc2e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5905c4b42ecd40298340a8a2ce4d5e24, user_id=572170ccd67e4facbf10dad7cd1491c1, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:39.798282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6e197940-4f4d-438b-a21a-5ebbbf882ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:39.803074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4572c0-0041-40f8-a335-425e7fbbc2e0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:39.869323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e197940-4f4d-438b-a21a-5ebbbf882ae9 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:39.874405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23ccb3-0991-468c-a660-c586c4872d22 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd1fb3b7573d4213a1e9f7cb83a336e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e1ca681e67644e6f99014e268b4b6fbf', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-903447527', 'tempest-NotDelegatedRole-774504629'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:39.875573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23ccb3-0991-468c-a660-c586c4872d22 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:39.875930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23ccb3-0991-468c-a660-c586c4872d22 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:39.876275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23ccb3-0991-468c-a660-c586c4872d22 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] PATH_INFO: `/v3/OS-TRUST/trusts/4dd7a1bc94544b5f8b35f14470308f0a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:39.896525 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-6d23ccb3-0991-468c-a660-c586c4872d22 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Could not find trust: 4dd7a1bc94544b5f8b35f14470308f0a.: keystone.exception.TrustNotFound: Could not find trust: 4dd7a1bc94544b5f8b35f14470308f0a. May 28 21:14:39.896525 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:14:39.896525 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:14:39.896525 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:14:39.896525 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:14:39.896525 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:14:39.896525 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:14:39.896525 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:14:39.896525 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:14:39.896525 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:14:39.896525 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:14:39.896525 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:14:39.896525 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get May 28 21:14:39.896525 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) May 28 21:14:39.896525 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust May 28 21:14:39.898080 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', May 28 21:14:39.898080 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call May 28 21:14:39.898080 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() May 28 21:14:39.898080 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement May 28 21:14:39.898080 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( May 28 21:14:39.898080 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:14:39.898080 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:14:39.898080 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust May 28 21:14:39.898080 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) May 28 21:14:39.898080 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust May 28 21:14:39.898080 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) May 28 21:14:39.898080 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 4dd7a1bc94544b5f8b35f14470308f0a. May 28 21:14:39.898080 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:14:39.899920 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1976/3938] 10.222.0.33 () {58 vars in 1250 bytes} [Sat May 28 21:14:39 2022] GET /identity/v3/OS-TRUST/trusts/4dd7a1bc94544b5f8b35f14470308f0a => generated 111 bytes in 103 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:14:39.901003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23ccb3-0991-468c-a660-c586c4872d22 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:39.902879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6d23ccb3-0991-468c-a660-c586c4872d22 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:39.908035 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1963/3939] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:39 2022] PUT /identity/v3/projects/5905c4b42ecd40298340a8a2ce4d5e24/users/572170ccd67e4facbf10dad7cd1491c1/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:39.908877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4572c0-0041-40f8-a335-425e7fbbc2e0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:39.910544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fd4572c0-0041-40f8-a335-425e7fbbc2e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:39.996231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4572c0-0041-40f8-a335-425e7fbbc2e0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:40.003574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d23ccb3-0991-468c-a660-c586c4872d22 tempest-TrustsV3TestJSON-704562726 tempest-user-245262392] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:40.021329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8720924-8c55-4ae8-8e2b-64e0b8203d68 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:40.022626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8720924-8c55-4ae8-8e2b-64e0b8203d68 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:40.023184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8720924-8c55-4ae8-8e2b-64e0b8203d68 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:40.023854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8720924-8c55-4ae8-8e2b-64e0b8203d68 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles/2bc1bfc233144a0eacfcefa331c30537` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:40.026440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8720924-8c55-4ae8-8e2b-64e0b8203d68 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_role(role_id=2bc1bfc233144a0eacfcefa331c30537)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:40.031293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8720924-8c55-4ae8-8e2b-64e0b8203d68 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:40.083860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b55b37-0023-4c74-b022-0a532f45b27c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:40.084522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b55b37-0023-4c74-b022-0a532f45b27c service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:40.084809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b55b37-0023-4c74-b022-0a532f45b27c service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:40.085137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b55b37-0023-4c74-b022-0a532f45b27c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:40.147375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b55b37-0023-4c74-b022-0a532f45b27c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:40.151236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b55b37-0023-4c74-b022-0a532f45b27c service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:40.177420 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1977/3940] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:14:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:14:40.179822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0836ae6-450b-4dc6-b563-2fa900bd8b3a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:40.180221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0836ae6-450b-4dc6-b563-2fa900bd8b3a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:40.180609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0836ae6-450b-4dc6-b563-2fa900bd8b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:40.202276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c0836ae6-450b-4dc6-b563-2fa900bd8b3a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:40.204038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c0836ae6-450b-4dc6-b563-2fa900bd8b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:40.259110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f8720924-8c55-4ae8-8e2b-64e0b8203d68 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invalidating the token cache because role 2bc1bfc233144a0eacfcefa331c30537 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:40.259383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f8720924-8c55-4ae8-8e2b-64e0b8203d68 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:40.260072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f8720924-8c55-4ae8-8e2b-64e0b8203d68 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:40.261944 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1964/3941] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:39 2022] DELETE /identity/v3/roles/2bc1bfc233144a0eacfcefa331c30537 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:40.266029 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1978/3942] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:40.266161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8720924-8c55-4ae8-8e2b-64e0b8203d68 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:40.267446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f8720924-8c55-4ae8-8e2b-64e0b8203d68 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:40.275768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0836ae6-450b-4dc6-b563-2fa900bd8b3a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:40.278918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c0836ae6-450b-4dc6-b563-2fa900bd8b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:40.357416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8720924-8c55-4ae8-8e2b-64e0b8203d68 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:40.364750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de68067-82f0-415d-92d4-010ee1ae6785 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:40.365546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0836ae6-450b-4dc6-b563-2fa900bd8b3a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:40.367048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de68067-82f0-415d-92d4-010ee1ae6785 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:40.367712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de68067-82f0-415d-92d4-010ee1ae6785 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:40.368163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de68067-82f0-415d-92d4-010ee1ae6785 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles/46f30a87e4304a2798298c0b83a61b3d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:40.370511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de68067-82f0-415d-92d4-010ee1ae6785 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_role(role_id=46f30a87e4304a2798298c0b83a61b3d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:40.372903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de68067-82f0-415d-92d4-010ee1ae6785 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:40.423116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c0630-d114-4d52-8955-75d65b1f1545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:40.423987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c0630-d114-4d52-8955-75d65b1f1545 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:40.424276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c0630-d114-4d52-8955-75d65b1f1545 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:40.424751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c0630-d114-4d52-8955-75d65b1f1545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:40.488042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c0630-d114-4d52-8955-75d65b1f1545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:40.489706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c0630-d114-4d52-8955-75d65b1f1545 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:40.501255 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1979/3943] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:14:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:14:40.585547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3de68067-82f0-415d-92d4-010ee1ae6785 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invalidating the token cache because role 46f30a87e4304a2798298c0b83a61b3d has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:40.586042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3de68067-82f0-415d-92d4-010ee1ae6785 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:40.587303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3de68067-82f0-415d-92d4-010ee1ae6785 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:40.591799 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1965/3944] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:40 2022] DELETE /identity/v3/roles/46f30a87e4304a2798298c0b83a61b3d => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:14:40.593280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c0630-d114-4d52-8955-75d65b1f1545 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:40.608292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c0630-d114-4d52-8955-75d65b1f1545 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:40.654111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:UPDATE=1,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:40.671029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d4b2c-2cab-4b0d-ab80-de3392bc1bda tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:40.671917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d4b2c-2cab-4b0d-ab80-de3392bc1bda tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:40.672195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d4b2c-2cab-4b0d-ab80-de3392bc1bda tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:40.672475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d4b2c-2cab-4b0d-ab80-de3392bc1bda tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users/dd1fb3b7573d4213a1e9f7cb83a336e5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:40.673738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948d4b2c-2cab-4b0d-ab80-de3392bc1bda tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_user(user_id=dd1fb3b7573d4213a1e9f7cb83a336e5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:40.675762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948d4b2c-2cab-4b0d-ab80-de3392bc1bda tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:40.828966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-948d4b2c-2cab-4b0d-ab80-de3392bc1bda tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd1fb3b7573d4213a1e9f7cb83a336e5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:40.838024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-948d4b2c-2cab-4b0d-ab80-de3392bc1bda tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd1fb3b7573d4213a1e9f7cb83a336e5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:40.838834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-948d4b2c-2cab-4b0d-ab80-de3392bc1bda tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd1fb3b7573d4213a1e9f7cb83a336e5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:40.840800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-948d4b2c-2cab-4b0d-ab80-de3392bc1bda tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd1fb3b7573d4213a1e9f7cb83a336e5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:40.910668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-948d4b2c-2cab-4b0d-ab80-de3392bc1bda tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd1fb3b7573d4213a1e9f7cb83a336e5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:40.935010 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1980/3945] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:40 2022] DELETE /identity/v3/users/dd1fb3b7573d4213a1e9f7cb83a336e5 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:40.939303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de68067-82f0-415d-92d4-010ee1ae6785 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:40.940720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3de68067-82f0-415d-92d4-010ee1ae6785 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:40.954734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de68067-82f0-415d-92d4-010ee1ae6785 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:41.020575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ebcee-90a8-4479-bb42-4587fb9be247 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:41.021465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ebcee-90a8-4479-bb42-4587fb9be247 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:41.021757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ebcee-90a8-4479-bb42-4587fb9be247 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:41.022073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ebcee-90a8-4479-bb42-4587fb9be247 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/e1ca681e67644e6f99014e268b4b6fbf` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:41.023354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ebcee-90a8-4479-bb42-4587fb9be247 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e1ca681e67644e6f99014e268b4b6fbf)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:41.025461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ebcee-90a8-4479-bb42-4587fb9be247 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:41.239292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a68ebcee-90a8-4479-bb42-4587fb9be247 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1ca681e67644e6f99014e268b4b6fbf', 'request_id': 'req-a68ebcee-90a8-4479-bb42-4587fb9be247'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:41.248588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a68ebcee-90a8-4479-bb42-4587fb9be247 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1ca681e67644e6f99014e268b4b6fbf', 'request_id': 'req-a68ebcee-90a8-4479-bb42-4587fb9be247'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:41.344892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a68ebcee-90a8-4479-bb42-4587fb9be247 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1ca681e67644e6f99014e268b4b6fbf', 'request_id': 'req-a68ebcee-90a8-4479-bb42-4587fb9be247'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:41.387035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a68ebcee-90a8-4479-bb42-4587fb9be247 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] The token cache is being invalidate because project e1ca681e67644e6f99014e268b4b6fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:41.387335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a68ebcee-90a8-4479-bb42-4587fb9be247 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:41.388006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a68ebcee-90a8-4479-bb42-4587fb9be247 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:41.389757 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1966/3946] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:40 2022] DELETE /identity/v3/projects/e1ca681e67644e6f99014e268b4b6fbf => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:41.397070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d4b2c-2cab-4b0d-ab80-de3392bc1bda tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:41.399091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-948d4b2c-2cab-4b0d-ab80-de3392bc1bda tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:41.412136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:41.419713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d4b2c-2cab-4b0d-ab80-de3392bc1bda tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:41.479156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeff9e-e475-48f2-9af9-f26ee0877281 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:41.481153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeff9e-e475-48f2-9af9-f26ee0877281 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:41.481153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeff9e-e475-48f2-9af9-f26ee0877281 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:41.481153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeff9e-e475-48f2-9af9-f26ee0877281 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:41.481153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aeff9e-e475-48f2-9af9-f26ee0877281 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:41.483004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aeff9e-e475-48f2-9af9-f26ee0877281 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:41.524997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00db8ae-e1a8-4ccd-a325-3aae53460c04 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:41.525304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00db8ae-e1a8-4ccd-a325-3aae53460c04 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:41.525620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00db8ae-e1a8-4ccd-a325-3aae53460c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:41.544157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f00db8ae-e1a8-4ccd-a325-3aae53460c04 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:41.545377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f00db8ae-e1a8-4ccd-a325-3aae53460c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:41.614588 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1967/3947] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:14:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:14:41.632058 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1981/3948] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:14:41 2022] POST /identity/v3/projects => generated 328 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:41.632343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00db8ae-e1a8-4ccd-a325-3aae53460c04 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:41.632343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f00db8ae-e1a8-4ccd-a325-3aae53460c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:41.632870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeff9e-e475-48f2-9af9-f26ee0877281 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:41.636232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-04aeff9e-e475-48f2-9af9-f26ee0877281 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:41.728819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00db8ae-e1a8-4ccd-a325-3aae53460c04 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:41.745694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeff9e-e475-48f2-9af9-f26ee0877281 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:41.752208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395ff78-47aa-47bf-aef3-da49e1e79439 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:41.753442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395ff78-47aa-47bf-aef3-da49e1e79439 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:41.753963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395ff78-47aa-47bf-aef3-da49e1e79439 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:41.754419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395ff78-47aa-47bf-aef3-da49e1e79439 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:41.755282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395ff78-47aa-47bf-aef3-da49e1e79439 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:41.759042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395ff78-47aa-47bf-aef3-da49e1e79439 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:41.800022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30606480-020f-47e4-ba43-29d03e50855d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:41.800748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30606480-020f-47e4-ba43-29d03e50855d service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:41.801063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30606480-020f-47e4-ba43-29d03e50855d service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:41.801336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30606480-020f-47e4-ba43-29d03e50855d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:41.858463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30606480-020f-47e4-ba43-29d03e50855d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:41.860807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30606480-020f-47e4-ba43-29d03e50855d service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:41.870918 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1968/3949] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:14:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:14:41.878796 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1982/3950] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:14:41 2022] POST /identity/v3/users => generated 416 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:14:41.890778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30606480-020f-47e4-ba43-29d03e50855d service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:41.911442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30606480-020f-47e4-ba43-29d03e50855d service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:41.918011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc4fdf-1de2-4475-8fd2-8ea4dd017667 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:41.918831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc4fdf-1de2-4475-8fd2-8ea4dd017667 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:41.919293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc4fdf-1de2-4475-8fd2-8ea4dd017667 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:41.919715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fc4fdf-1de2-4475-8fd2-8ea4dd017667 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:41.920568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc4fdf-1de2-4475-8fd2-8ea4dd017667 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:41.922547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fc4fdf-1de2-4475-8fd2-8ea4dd017667 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:42.010676 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1969/3951] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:14:41 2022] POST /identity/v3/roles => generated 244 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:42.015002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395ff78-47aa-47bf-aef3-da49e1e79439 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:42.032195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395ff78-47aa-47bf-aef3-da49e1e79439 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:42.038901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f27032-8207-4afe-a07f-e6e53bb290d7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:42.039862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f27032-8207-4afe-a07f-e6e53bb290d7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:42.040226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f27032-8207-4afe-a07f-e6e53bb290d7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:42.040581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f27032-8207-4afe-a07f-e6e53bb290d7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:42.041725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f27032-8207-4afe-a07f-e6e53bb290d7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:42.043597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f27032-8207-4afe-a07f-e6e53bb290d7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:42.201917 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1983/3952] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:14:42 2022] POST /identity/v3/roles => generated 247 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:42.204263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc4fdf-1de2-4475-8fd2-8ea4dd017667 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:42.217323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fc4fdf-1de2-4475-8fd2-8ea4dd017667 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:42.223228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9742c-2471-4b52-92fc-041a569fe71f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:42.224214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9742c-2471-4b52-92fc-041a569fe71f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:42.224502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9742c-2471-4b52-92fc-041a569fe71f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:42.224933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9742c-2471-4b52-92fc-041a569fe71f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/7f987ca5ae7749578965a6e61c3bb230/users/24022da0b50344f48ca68ef29c8e0304/roles/f7eac243888649f098f50fbe40ed278a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:42.248028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9742c-2471-4b52-92fc-041a569fe71f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_grant(project_id=7f987ca5ae7749578965a6e61c3bb230, user_id=24022da0b50344f48ca68ef29c8e0304, role_id=f7eac243888649f098f50fbe40ed278a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:42.255694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9742c-2471-4b52-92fc-041a569fe71f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:42.404278 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1970/3953] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:42 2022] PUT /identity/v3/projects/7f987ca5ae7749578965a6e61c3bb230/users/24022da0b50344f48ca68ef29c8e0304/roles/f7eac243888649f098f50fbe40ed278a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:42.410783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f27032-8207-4afe-a07f-e6e53bb290d7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:42.436870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f27032-8207-4afe-a07f-e6e53bb290d7 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:42.530543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e8bba-4ff0-4eb1-aedb-e096cdc46e25 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:42.530783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e8bba-4ff0-4eb1-aedb-e096cdc46e25 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:42.531071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e8bba-4ff0-4eb1-aedb-e096cdc46e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:42.571096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33125d12-c9e9-4f0a-bd0e-441ac66ac870 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:42.572251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33125d12-c9e9-4f0a-bd0e-441ac66ac870 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:42.572585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33125d12-c9e9-4f0a-bd0e-441ac66ac870 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:42.572940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33125d12-c9e9-4f0a-bd0e-441ac66ac870 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/7f987ca5ae7749578965a6e61c3bb230/users/24022da0b50344f48ca68ef29c8e0304/roles/f4d8dd93c84b44bdb0976b152947cdbd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:42.577092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33125d12-c9e9-4f0a-bd0e-441ac66ac870 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_grant(project_id=7f987ca5ae7749578965a6e61c3bb230, user_id=24022da0b50344f48ca68ef29c8e0304, role_id=f4d8dd93c84b44bdb0976b152947cdbd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:42.580261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33125d12-c9e9-4f0a-bd0e-441ac66ac870 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:42.591178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2e3e8bba-4ff0-4eb1-aedb-e096cdc46e25 None None] MFA Rules not processed for user `572170ccd67e4facbf10dad7cd1491c1`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:42.592468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2e3e8bba-4ff0-4eb1-aedb-e096cdc46e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:42.671654 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1984/3954] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:42 2022] PUT /identity/v3/projects/7f987ca5ae7749578965a6e61c3bb230/users/24022da0b50344f48ca68ef29c8e0304/roles/f4d8dd93c84b44bdb0976b152947cdbd => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:42.672643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33125d12-c9e9-4f0a-bd0e-441ac66ac870 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:42.683201 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1971/3955] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:42 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:14:42.703956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bfa61-cce9-4068-96e2-338fca031e46 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:42.704970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bfa61-cce9-4068-96e2-338fca031e46 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:42.713579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05bfa61-cce9-4068-96e2-338fca031e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:42.738483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a05bfa61-cce9-4068-96e2-338fca031e46 None None] MFA Rules not processed for user `572170ccd67e4facbf10dad7cd1491c1`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:42.739794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a05bfa61-cce9-4068-96e2-338fca031e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:42.774942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33125d12-c9e9-4f0a-bd0e-441ac66ac870 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:42.780371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50df3f89-af5d-4132-999d-12316215875f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:42.781077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df3f89-af5d-4132-999d-12316215875f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:42.781366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df3f89-af5d-4132-999d-12316215875f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:42.781648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50df3f89-af5d-4132-999d-12316215875f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:42.782470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df3f89-af5d-4132-999d-12316215875f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:42.784300 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-50df3f89-af5d-4132-999d-12316215875f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:14:42.784300 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:14:42.784300 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:14:42.785480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50df3f89-af5d-4132-999d-12316215875f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:42.810551 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1972/3956] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:42 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:42.822706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bfa61-cce9-4068-96e2-338fca031e46 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:42.824653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a05bfa61-cce9-4068-96e2-338fca031e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:42.850189 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1985/3957] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:42 2022] GET /identity/v3/users => generated 8636 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:42.854660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6d43d-1b2a-4f98-a2a0-5f73a2cddc49 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:42.854975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6d43d-1b2a-4f98-a2a0-5f73a2cddc49 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:42.855414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6d43d-1b2a-4f98-a2a0-5f73a2cddc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:42.916335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e4d6d43d-1b2a-4f98-a2a0-5f73a2cddc49 None None] MFA Rules not processed for user `24022da0b50344f48ca68ef29c8e0304`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:42.917602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e4d6d43d-1b2a-4f98-a2a0-5f73a2cddc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:42.922487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05bfa61-cce9-4068-96e2-338fca031e46 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:42.929808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f66d5f-50a8-439c-87cf-3eddc112c088 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:42.930958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f66d5f-50a8-439c-87cf-3eddc112c088 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:42.930958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f66d5f-50a8-439c-87cf-3eddc112c088 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:42.933132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f66d5f-50a8-439c-87cf-3eddc112c088 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:42.941465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f66d5f-50a8-439c-87cf-3eddc112c088 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:42.943526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f66d5f-50a8-439c-87cf-3eddc112c088 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:42.954530 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1973/3958] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:14:42 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:14:42.994018 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1986/3959] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:43.004485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28546acf-6dd8-4545-99a0-65e8ab22b05e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:43.004650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28546acf-6dd8-4545-99a0-65e8ab22b05e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:43.004931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28546acf-6dd8-4545-99a0-65e8ab22b05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:43.026388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-28546acf-6dd8-4545-99a0-65e8ab22b05e None None] MFA Rules not processed for user `24022da0b50344f48ca68ef29c8e0304`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:43.027601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-28546acf-6dd8-4545-99a0-65e8ab22b05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:43.041297 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1974/3960] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:43.045721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6d43d-1b2a-4f98-a2a0-5f73a2cddc49 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:43.059017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6d43d-1b2a-4f98-a2a0-5f73a2cddc49 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:43.063923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da256a9d-c0bf-4fa2-b7fe-65028daa171c tempest-TrustsV3TestJSON-600537735 tempest-user-1607117832] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24022da0b50344f48ca68ef29c8e0304', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f987ca5ae7749578965a6e61c3bb230', 'project_domain_id': 'default', 'roles': ['tempest-NotDelegatedRole-1047912970', 'tempest-DelegatedRole-1082027711'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:43.064595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da256a9d-c0bf-4fa2-b7fe-65028daa171c tempest-TrustsV3TestJSON-600537735 tempest-user-1607117832] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:43.064881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da256a9d-c0bf-4fa2-b7fe-65028daa171c tempest-TrustsV3TestJSON-600537735 tempest-user-1607117832] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:43.065471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da256a9d-c0bf-4fa2-b7fe-65028daa171c tempest-TrustsV3TestJSON-600537735 tempest-user-1607117832] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:43.065956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da256a9d-c0bf-4fa2-b7fe-65028daa171c tempest-TrustsV3TestJSON-600537735 tempest-user-1607117832] RBAC: Authorizing `identity:create_trust()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:43.067150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da256a9d-c0bf-4fa2-b7fe-65028daa171c tempest-TrustsV3TestJSON-600537735 tempest-user-1607117832] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:43.199139 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-da256a9d-c0bf-4fa2-b7fe-65028daa171c tempest-TrustsV3TestJSON-600537735 tempest-user-1607117832] Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error.: keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. May 28 21:14:43.199139 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:14:43.199139 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 67, in parse_isotime May 28 21:14:43.199139 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return iso8601.parse_date(timestr) May 28 21:14:43.199139 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/iso8601/iso8601.py", line 127, in parse_date May 28 21:14:43.199139 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise ParseError(f"Unable to parse date string {datestring!r}") May 28 21:14:43.199139 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application iso8601.iso8601.ParseError: Unable to parse date string 'bad.123Z' May 28 21:14:43.199139 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:14:43.199139 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:14:43.199139 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:14:43.199139 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:14:43.199139 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 346, in parse_expiration_date May 28 21:14:43.199139 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application expiration_time = timeutils.parse_isotime(expiration_date) May 28 21:14:43.199139 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/timeutils.py", line 69, in parse_isotime May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise ValueError(str(e)) May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ValueError: Unable to parse date string 'bad.123Z' May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 296, in post May 28 21:14:43.200280 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application trust['expires_at'] = self._parse_expiration_date( May 28 21:14:43.201326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 127, in _parse_expiration_date May 28 21:14:43.201326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return utils.parse_expiration_date(expiration_date) May 28 21:14:43.201326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/utils.py", line 348, in parse_expiration_date May 28 21:14:43.201326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ValidationTimeStampError() May 28 21:14:43.201326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ValidationTimeStampError: Timestamp not in expected format. The server could not comply with the request since it is either malformed or otherwise incorrect. The client is assumed to be in error. May 28 21:14:43.201326 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:14:43.201326 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1987/3961] 10.222.0.33 () {60 vars in 1173 bytes} [Sat May 28 21:14:43 2022] POST /identity/v3/OS-TRUST/trusts => generated 227 bytes in 155 msecs (HTTP/1.1 400) 5 headers in 186 bytes (2 switches on core 0) May 28 21:14:43.205012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28546acf-6dd8-4545-99a0-65e8ab22b05e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:43.217358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28546acf-6dd8-4545-99a0-65e8ab22b05e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:43.222027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de39db4-58f5-46ff-b71e-9587000f4f64 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:43.222885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de39db4-58f5-46ff-b71e-9587000f4f64 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:43.223314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de39db4-58f5-46ff-b71e-9587000f4f64 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:43.223599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de39db4-58f5-46ff-b71e-9587000f4f64 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles/f4d8dd93c84b44bdb0976b152947cdbd` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:43.225143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de39db4-58f5-46ff-b71e-9587000f4f64 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_role(role_id=f4d8dd93c84b44bdb0976b152947cdbd)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:43.227625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de39db4-58f5-46ff-b71e-9587000f4f64 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:43.603843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3de39db4-58f5-46ff-b71e-9587000f4f64 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invalidating the token cache because role f4d8dd93c84b44bdb0976b152947cdbd has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:43.604226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3de39db4-58f5-46ff-b71e-9587000f4f64 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:43.604863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3de39db4-58f5-46ff-b71e-9587000f4f64 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:43.610805 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1975/3962] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:43 2022] DELETE /identity/v3/roles/f4d8dd93c84b44bdb0976b152947cdbd => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:43.612240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da256a9d-c0bf-4fa2-b7fe-65028daa171c tempest-TrustsV3TestJSON-600537735 tempest-user-1607117832] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:43.614195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-da256a9d-c0bf-4fa2-b7fe-65028daa171c tempest-TrustsV3TestJSON-600537735 tempest-user-1607117832] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:43.634488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da256a9d-c0bf-4fa2-b7fe-65028daa171c tempest-TrustsV3TestJSON-600537735 tempest-user-1607117832] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:43.659640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:43.702066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008e883-fab3-4bac-9024-672ed67c6f48 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:43.706832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008e883-fab3-4bac-9024-672ed67c6f48 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:43.706832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008e883-fab3-4bac-9024-672ed67c6f48 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:43.706832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008e883-fab3-4bac-9024-672ed67c6f48 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles/f7eac243888649f098f50fbe40ed278a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:43.708325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008e883-fab3-4bac-9024-672ed67c6f48 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_role(role_id=f7eac243888649f098f50fbe40ed278a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:43.710270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e008e883-fab3-4bac-9024-672ed67c6f48 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:43.782759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:43.870916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e939c-debc-47fe-92b4-4887e23aa6c3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:43.871506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e939c-debc-47fe-92b4-4887e23aa6c3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:43.871974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956e939c-debc-47fe-92b4-4887e23aa6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:43.883821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e008e883-fab3-4bac-9024-672ed67c6f48 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invalidating the token cache because role f7eac243888649f098f50fbe40ed278a has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:43.884235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e008e883-fab3-4bac-9024-672ed67c6f48 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:43.885670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e008e883-fab3-4bac-9024-672ed67c6f48 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:43.892489 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1988/3963] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:43 2022] DELETE /identity/v3/roles/f7eac243888649f098f50fbe40ed278a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:43.893558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008e883-fab3-4bac-9024-672ed67c6f48 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:43.896048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-956e939c-debc-47fe-92b4-4887e23aa6c3 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:43.896792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e008e883-fab3-4bac-9024-672ed67c6f48 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:43.897309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-956e939c-debc-47fe-92b4-4887e23aa6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:43.982349 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1976/3964] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:14:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:43.983863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e939c-debc-47fe-92b4-4887e23aa6c3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:43.986140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-956e939c-debc-47fe-92b4-4887e23aa6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:43.998970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008e883-fab3-4bac-9024-672ed67c6f48 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:44.006520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d70cb-f44f-40bc-bcff-d2b841d9a8eb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:44.007562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d70cb-f44f-40bc-bcff-d2b841d9a8eb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:44.007948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d70cb-f44f-40bc-bcff-d2b841d9a8eb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:44.008594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d70cb-f44f-40bc-bcff-d2b841d9a8eb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users/24022da0b50344f48ca68ef29c8e0304` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:44.009699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d70cb-f44f-40bc-bcff-d2b841d9a8eb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_user(user_id=24022da0b50344f48ca68ef29c8e0304)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:44.012220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d70cb-f44f-40bc-bcff-d2b841d9a8eb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:44.088649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956e939c-debc-47fe-92b4-4887e23aa6c3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:44.094140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe53bd-3fc8-4afc-ad31-8a45fe571397 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:44.102641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe53bd-3fc8-4afc-ad31-8a45fe571397 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:44.102641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe53bd-3fc8-4afc-ad31-8a45fe571397 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:44.102641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afe53bd-3fc8-4afc-ad31-8a45fe571397 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:44.111662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe53bd-3fc8-4afc-ad31-8a45fe571397 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:44.113546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afe53bd-3fc8-4afc-ad31-8a45fe571397 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:44.124612 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1977/3965] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:14:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:14:44.257649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d59d70cb-f44f-40bc-bcff-d2b841d9a8eb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24022da0b50344f48ca68ef29c8e0304'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:44.272978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d59d70cb-f44f-40bc-bcff-d2b841d9a8eb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24022da0b50344f48ca68ef29c8e0304'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:44.273727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d59d70cb-f44f-40bc-bcff-d2b841d9a8eb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24022da0b50344f48ca68ef29c8e0304'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:44.274396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d59d70cb-f44f-40bc-bcff-d2b841d9a8eb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24022da0b50344f48ca68ef29c8e0304'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:44.373663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d59d70cb-f44f-40bc-bcff-d2b841d9a8eb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24022da0b50344f48ca68ef29c8e0304'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:44.392505 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1989/3966] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:43 2022] DELETE /identity/v3/users/24022da0b50344f48ca68ef29c8e0304 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:44.396736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe53bd-3fc8-4afc-ad31-8a45fe571397 service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:44.408512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afe53bd-3fc8-4afc-ad31-8a45fe571397 service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:44.474030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b839738-f63f-4a34-bbb8-a2c56238d160 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:44.475287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b839738-f63f-4a34-bbb8-a2c56238d160 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:44.475616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b839738-f63f-4a34-bbb8-a2c56238d160 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:44.475898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b839738-f63f-4a34-bbb8-a2c56238d160 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/7f987ca5ae7749578965a6e61c3bb230` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:44.477104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b839738-f63f-4a34-bbb8-a2c56238d160 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7f987ca5ae7749578965a6e61c3bb230)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:44.479198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b839738-f63f-4a34-bbb8-a2c56238d160 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:44.619456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9b839738-f63f-4a34-bbb8-a2c56238d160 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f987ca5ae7749578965a6e61c3bb230', 'request_id': 'req-9b839738-f63f-4a34-bbb8-a2c56238d160'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:44.626710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9b839738-f63f-4a34-bbb8-a2c56238d160 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f987ca5ae7749578965a6e61c3bb230', 'request_id': 'req-9b839738-f63f-4a34-bbb8-a2c56238d160'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:44.722226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9b839738-f63f-4a34-bbb8-a2c56238d160 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f987ca5ae7749578965a6e61c3bb230', 'request_id': 'req-9b839738-f63f-4a34-bbb8-a2c56238d160'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:44.738575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9b839738-f63f-4a34-bbb8-a2c56238d160 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] The token cache is being invalidate because project 7f987ca5ae7749578965a6e61c3bb230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:44.739098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9b839738-f63f-4a34-bbb8-a2c56238d160 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:44.739978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9b839738-f63f-4a34-bbb8-a2c56238d160 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:44.742819 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1978/3967] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:44 2022] DELETE /identity/v3/projects/7f987ca5ae7749578965a6e61c3bb230 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:44.750011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d70cb-f44f-40bc-bcff-d2b841d9a8eb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:44.751582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d59d70cb-f44f-40bc-bcff-d2b841d9a8eb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:44.770673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d70cb-f44f-40bc-bcff-d2b841d9a8eb tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:44.852193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299aba5-3d59-4f75-945f-1dfc6f2d6cf2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:44.853270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299aba5-3d59-4f75-945f-1dfc6f2d6cf2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:44.853814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299aba5-3d59-4f75-945f-1dfc6f2d6cf2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:44.854332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299aba5-3d59-4f75-945f-1dfc6f2d6cf2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:44.855385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299aba5-3d59-4f75-945f-1dfc6f2d6cf2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:44.857756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b299aba5-3d59-4f75-945f-1dfc6f2d6cf2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:44.982978 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1990/3968] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:14:44 2022] POST /identity/v3/projects => generated 329 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:44.987340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b839738-f63f-4a34-bbb8-a2c56238d160 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:44.989055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9b839738-f63f-4a34-bbb8-a2c56238d160 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:45.003845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b839738-f63f-4a34-bbb8-a2c56238d160 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:45.086338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e6b07-f824-421a-8305-285b91e6f89a tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:45.088176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e6b07-f824-421a-8305-285b91e6f89a tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:45.093008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e6b07-f824-421a-8305-285b91e6f89a tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:45.093619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187e6b07-f824-421a-8305-285b91e6f89a tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:45.094083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e6b07-f824-421a-8305-285b91e6f89a tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:45.099693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187e6b07-f824-421a-8305-285b91e6f89a tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:45.231948 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1979/3969] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:14:44 2022] POST /identity/v3/users => generated 413 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:14:45.236107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299aba5-3d59-4f75-945f-1dfc6f2d6cf2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:45.249222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b299aba5-3d59-4f75-945f-1dfc6f2d6cf2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:45.261726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac07a7d-5e71-483e-a1c8-6fd0db5ce957 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:45.262749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac07a7d-5e71-483e-a1c8-6fd0db5ce957 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:45.262842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac07a7d-5e71-483e-a1c8-6fd0db5ce957 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:45.263060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac07a7d-5e71-483e-a1c8-6fd0db5ce957 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:45.263910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac07a7d-5e71-483e-a1c8-6fd0db5ce957 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:45.265537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac07a7d-5e71-483e-a1c8-6fd0db5ce957 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:45.368978 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1991/3970] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:14:45 2022] POST /identity/v3/roles => generated 244 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:45.370538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e6b07-f824-421a-8305-285b91e6f89a tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:45.384139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187e6b07-f824-421a-8305-285b91e6f89a tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:45.398112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160a011-5bd0-41d8-bf7a-ea25493ca33e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:45.398798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160a011-5bd0-41d8-bf7a-ea25493ca33e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:45.399088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160a011-5bd0-41d8-bf7a-ea25493ca33e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:45.399369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d160a011-5bd0-41d8-bf7a-ea25493ca33e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:45.400127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d160a011-5bd0-41d8-bf7a-ea25493ca33e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:45.401979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d160a011-5bd0-41d8-bf7a-ea25493ca33e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:45.503490 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1980/3971] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:14:45 2022] POST /identity/v3/roles => generated 246 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:45.504690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac07a7d-5e71-483e-a1c8-6fd0db5ce957 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:45.518620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac07a7d-5e71-483e-a1c8-6fd0db5ce957 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:45.523933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fdd4e-ad23-4b0e-91de-3a632ab01e57 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:45.524881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fdd4e-ad23-4b0e-91de-3a632ab01e57 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:45.525167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fdd4e-ad23-4b0e-91de-3a632ab01e57 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:45.525454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fdd4e-ad23-4b0e-91de-3a632ab01e57 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/06eb9b8d97204d8481d6e31a21fbc95c/users/21a509e865f844d69d27d400c7959178/roles/f1bab1ad7e4a42cd96310564a2e7aedb` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:45.541335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fdd4e-ad23-4b0e-91de-3a632ab01e57 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_grant(project_id=06eb9b8d97204d8481d6e31a21fbc95c, user_id=21a509e865f844d69d27d400c7959178, role_id=f1bab1ad7e4a42cd96310564a2e7aedb)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:45.544081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633fdd4e-ad23-4b0e-91de-3a632ab01e57 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:45.728578 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1992/3972] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:45 2022] PUT /identity/v3/projects/06eb9b8d97204d8481d6e31a21fbc95c/users/21a509e865f844d69d27d400c7959178/roles/f1bab1ad7e4a42cd96310564a2e7aedb => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:45.732876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160a011-5bd0-41d8-bf7a-ea25493ca33e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:45.743955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d160a011-5bd0-41d8-bf7a-ea25493ca33e tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:45.803573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b453d-8c7e-45e6-878f-bb451fc146e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:45.804486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b453d-8c7e-45e6-878f-bb451fc146e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:45.804773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b453d-8c7e-45e6-878f-bb451fc146e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:45.805054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b453d-8c7e-45e6-878f-bb451fc146e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/06eb9b8d97204d8481d6e31a21fbc95c/users/21a509e865f844d69d27d400c7959178/roles/8bc4910fa7fa4330ad0887be334d896e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:45.807386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b453d-8c7e-45e6-878f-bb451fc146e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_grant(project_id=06eb9b8d97204d8481d6e31a21fbc95c, user_id=21a509e865f844d69d27d400c7959178, role_id=8bc4910fa7fa4330ad0887be334d896e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:45.810218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b453d-8c7e-45e6-878f-bb451fc146e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:45.888295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:45.894202 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1981/3973] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:45 2022] PUT /identity/v3/projects/06eb9b8d97204d8481d6e31a21fbc95c/users/21a509e865f844d69d27d400c7959178/roles/8bc4910fa7fa4330ad0887be334d896e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:45.898595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fdd4e-ad23-4b0e-91de-3a632ab01e57 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:45.909431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633fdd4e-ad23-4b0e-91de-3a632ab01e57 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:45.969323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb273d1-ed54-4a28-a508-19c5e3ee3fd2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:45.970054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb273d1-ed54-4a28-a508-19c5e3ee3fd2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:45.970349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb273d1-ed54-4a28-a508-19c5e3ee3fd2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:45.970637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb273d1-ed54-4a28-a508-19c5e3ee3fd2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:45.971420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb273d1-ed54-4a28-a508-19c5e3ee3fd2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:45.973379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb273d1-ed54-4a28-a508-19c5e3ee3fd2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:46.004504 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1993/3974] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:45 2022] GET /identity/v3/users => generated 8633 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:46.009651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb13165-7863-4ab4-85a1-3f553734d989 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:46.009995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb13165-7863-4ab4-85a1-3f553734d989 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:46.010280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb13165-7863-4ab4-85a1-3f553734d989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:46.073690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9cb13165-7863-4ab4-85a1-3f553734d989 None None] MFA Rules not processed for user `21a509e865f844d69d27d400c7959178`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:46.086205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9cb13165-7863-4ab4-85a1-3f553734d989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:46.105657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa21159-7647-4e5a-bd40-afe8df7590dc None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:46.105971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa21159-7647-4e5a-bd40-afe8df7590dc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:46.106428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa21159-7647-4e5a-bd40-afe8df7590dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:46.133213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9fa21159-7647-4e5a-bd40-afe8df7590dc None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:46.134612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9fa21159-7647-4e5a-bd40-afe8df7590dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:46.171556 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1982/3975] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:46 2022] POST /identity/v3/auth/tokens => generated 3269 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:46.180887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54beda9f-300d-442b-b63c-f3901f47c3e7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:46.181266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54beda9f-300d-442b-b63c-f3901f47c3e7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:46.181558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54beda9f-300d-442b-b63c-f3901f47c3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:46.204126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-54beda9f-300d-442b-b63c-f3901f47c3e7 None None] MFA Rules not processed for user `21a509e865f844d69d27d400c7959178`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:46.209201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-54beda9f-300d-442b-b63c-f3901f47c3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:46.209342 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1994/3976] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:14:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:46.219941 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1983/3977] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:46 2022] POST /identity/v3/auth/tokens => generated 3269 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:14:46.223843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa21159-7647-4e5a-bd40-afe8df7590dc None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:46.225167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54beda9f-300d-442b-b63c-f3901f47c3e7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:46.226289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9fa21159-7647-4e5a-bd40-afe8df7590dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:46.244579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54beda9f-300d-442b-b63c-f3901f47c3e7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:46.250031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16098c0f-bc53-4225-9daa-6bda749dc667 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21a509e865f844d69d27d400c7959178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06eb9b8d97204d8481d6e31a21fbc95c', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2070237270', 'tempest-NotDelegatedRole-389058313'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:46.250537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16098c0f-bc53-4225-9daa-6bda749dc667 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:46.250958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16098c0f-bc53-4225-9daa-6bda749dc667 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:46.251250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16098c0f-bc53-4225-9daa-6bda749dc667 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:46.252705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16098c0f-bc53-4225-9daa-6bda749dc667 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: Authorizing `identity:create_trust()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:46.253635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16098c0f-bc53-4225-9daa-6bda749dc667 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:46.350942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa21159-7647-4e5a-bd40-afe8df7590dc None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:46.350942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1e345-fa89-49a4-b659-f9a65a008e77 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:46.351578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e345-fa89-49a4-b659-f9a65a008e77 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:46.351927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e345-fa89-49a4-b659-f9a65a008e77 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:46.352490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1e345-fa89-49a4-b659-f9a65a008e77 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:46.367708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1e345-fa89-49a4-b659-f9a65a008e77 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:46.369595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1e345-fa89-49a4-b659-f9a65a008e77 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:46.383490 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1995/3978] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:14:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:14:46.482441 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1984/3979] 10.222.0.33 () {60 vars in 1173 bytes} [Sat May 28 21:14:46 2022] POST /identity/v3/OS-TRUST/trusts => generated 834 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:46.484958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1e345-fa89-49a4-b659-f9a65a008e77 service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:46.497047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1e345-fa89-49a4-b659-f9a65a008e77 service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:46.502322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85268d0-65e2-44e1-abd7-4b7213c914d9 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21a509e865f844d69d27d400c7959178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06eb9b8d97204d8481d6e31a21fbc95c', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2070237270', 'tempest-NotDelegatedRole-389058313'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:46.503440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85268d0-65e2-44e1-abd7-4b7213c914d9 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:46.503739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85268d0-65e2-44e1-abd7-4b7213c914d9 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:46.504020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85268d0-65e2-44e1-abd7-4b7213c914d9 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] PATH_INFO: `/v3/OS-TRUST/trusts/c46b47f76fb94022adc337301332692f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:46.511261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85268d0-65e2-44e1-abd7-4b7213c914d9 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: Authorizing `identity:get_trust(trust_id=c46b47f76fb94022adc337301332692f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:46.512836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85268d0-65e2-44e1-abd7-4b7213c914d9 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:46.543846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16098c0f-bc53-4225-9daa-6bda749dc667 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:46.544358 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1996/3980] 10.222.0.33 () {58 vars in 1250 bytes} [Sat May 28 21:14:46 2022] GET /identity/v3/OS-TRUST/trusts/c46b47f76fb94022adc337301332692f => generated 834 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:46.556905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16098c0f-bc53-4225-9daa-6bda749dc667 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:46.571936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:46.574343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96336b8a-65a6-4bbc-ad67-c3f7e16a177a tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21a509e865f844d69d27d400c7959178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06eb9b8d97204d8481d6e31a21fbc95c', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2070237270', 'tempest-NotDelegatedRole-389058313'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:46.575192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96336b8a-65a6-4bbc-ad67-c3f7e16a177a tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:46.575474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96336b8a-65a6-4bbc-ad67-c3f7e16a177a tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:46.575760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96336b8a-65a6-4bbc-ad67-c3f7e16a177a tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] PATH_INFO: `/v3/OS-TRUST/trusts/c46b47f76fb94022adc337301332692f/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:46.590026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96336b8a-65a6-4bbc-ad67-c3f7e16a177a tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=c46b47f76fb94022adc337301332692f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:46.590026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96336b8a-65a6-4bbc-ad67-c3f7e16a177a tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:46.592004 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1985/3981] 10.222.0.33 () {58 vars in 1268 bytes} [Sat May 28 21:14:46 2022] GET /identity/v3/OS-TRUST/trusts/c46b47f76fb94022adc337301332692f/roles => generated 371 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:46.596229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85268d0-65e2-44e1-abd7-4b7213c914d9 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:46.607299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85268d0-65e2-44e1-abd7-4b7213c914d9 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:46.612430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812801ec-33b9-4ef5-9deb-0639250cb073 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21a509e865f844d69d27d400c7959178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06eb9b8d97204d8481d6e31a21fbc95c', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2070237270', 'tempest-NotDelegatedRole-389058313'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:46.613361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812801ec-33b9-4ef5-9deb-0639250cb073 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:46.614986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812801ec-33b9-4ef5-9deb-0639250cb073 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:46.614986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812801ec-33b9-4ef5-9deb-0639250cb073 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] PATH_INFO: `/v3/OS-TRUST/trusts/c46b47f76fb94022adc337301332692f/roles/f1bab1ad7e4a42cd96310564a2e7aedb` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:46.622178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812801ec-33b9-4ef5-9deb-0639250cb073 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: Authorizing `identity:get_role_for_trust(trust_id=c46b47f76fb94022adc337301332692f, role_id=f1bab1ad7e4a42cd96310564a2e7aedb)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:46.623648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812801ec-33b9-4ef5-9deb-0639250cb073 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:46.632469 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1997/3982] 10.222.0.33 () {58 vars in 1367 bytes} [Sat May 28 21:14:46 2022] GET /identity/v3/OS-TRUST/trusts/c46b47f76fb94022adc337301332692f/roles/f1bab1ad7e4a42cd96310564a2e7aedb => generated 244 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:46.636967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96336b8a-65a6-4bbc-ad67-c3f7e16a177a tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:46.646908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96336b8a-65a6-4bbc-ad67-c3f7e16a177a tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:46.651178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ecd8f-ee0c-47c4-b11f-464e7285d0dc tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21a509e865f844d69d27d400c7959178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06eb9b8d97204d8481d6e31a21fbc95c', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2070237270', 'tempest-NotDelegatedRole-389058313'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:46.652443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ecd8f-ee0c-47c4-b11f-464e7285d0dc tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] REQUEST_METHOD: `HEAD` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:46.654078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ecd8f-ee0c-47c4-b11f-464e7285d0dc tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:46.654078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ecd8f-ee0c-47c4-b11f-464e7285d0dc tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] PATH_INFO: `/v3/OS-TRUST/trusts/c46b47f76fb94022adc337301332692f/roles/f1bab1ad7e4a42cd96310564a2e7aedb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:46.659999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ecd8f-ee0c-47c4-b11f-464e7285d0dc tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: Authorizing `identity:get_role_for_trust(trust_id=c46b47f76fb94022adc337301332692f, role_id=f1bab1ad7e4a42cd96310564a2e7aedb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:46.661693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ecd8f-ee0c-47c4-b11f-464e7285d0dc tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:46.672930 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1986/3983] 10.222.0.33 () {58 vars in 1368 bytes} [Sat May 28 21:14:46 2022] HEAD /identity/v3/OS-TRUST/trusts/c46b47f76fb94022adc337301332692f/roles/f1bab1ad7e4a42cd96310564a2e7aedb => generated 0 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) May 28 21:14:46.673756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812801ec-33b9-4ef5-9deb-0639250cb073 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:46.699700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812801ec-33b9-4ef5-9deb-0639250cb073 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:46.705587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6ecd4-ef06-48b4-bcfd-365703a61e16 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21a509e865f844d69d27d400c7959178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06eb9b8d97204d8481d6e31a21fbc95c', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2070237270', 'tempest-NotDelegatedRole-389058313'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:46.706501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6ecd4-ef06-48b4-bcfd-365703a61e16 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:46.706912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6ecd4-ef06-48b4-bcfd-365703a61e16 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:46.707206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6ecd4-ef06-48b4-bcfd-365703a61e16 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] PATH_INFO: `/v3/OS-TRUST/trusts/c46b47f76fb94022adc337301332692f/roles/8bc4910fa7fa4330ad0887be334d896e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:46.715556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6ecd4-ef06-48b4-bcfd-365703a61e16 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: Authorizing `identity:get_role_for_trust(trust_id=c46b47f76fb94022adc337301332692f, role_id=8bc4910fa7fa4330ad0887be334d896e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:46.725486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6ecd4-ef06-48b4-bcfd-365703a61e16 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:46.737955 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-91f6ecd4-ef06-48b4-bcfd-365703a61e16 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Could not find role: 8bc4910fa7fa4330ad0887be334d896e.: keystone.exception.RoleNotFound: Could not find role: 8bc4910fa7fa4330ad0887be334d896e. May 28 21:14:46.737955 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:14:46.737955 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:14:46.737955 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:14:46.737955 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:14:46.737955 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:14:46.737955 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:14:46.737955 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:14:46.737955 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:14:46.737955 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:14:46.737955 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:14:46.737955 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:14:46.737955 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get May 28 21:14:46.737955 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 21:14:46.738975 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 8bc4910fa7fa4330ad0887be334d896e. May 28 21:14:46.738975 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:14:46.739388 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1998/3984] 10.222.0.33 () {58 vars in 1367 bytes} [Sat May 28 21:14:46 2022] GET /identity/v3/OS-TRUST/trusts/c46b47f76fb94022adc337301332692f/roles/8bc4910fa7fa4330ad0887be334d896e => generated 110 bytes in 66 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) May 28 21:14:46.744515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ecd8f-ee0c-47c4-b11f-464e7285d0dc tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:46.772022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ecd8f-ee0c-47c4-b11f-464e7285d0dc tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:46.779609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1540a1e-9cd8-47aa-9bcf-c58ae704970f tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21a509e865f844d69d27d400c7959178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06eb9b8d97204d8481d6e31a21fbc95c', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2070237270', 'tempest-NotDelegatedRole-389058313'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:46.785762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1540a1e-9cd8-47aa-9bcf-c58ae704970f tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] REQUEST_METHOD: `HEAD` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:46.785762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1540a1e-9cd8-47aa-9bcf-c58ae704970f tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:46.785762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1540a1e-9cd8-47aa-9bcf-c58ae704970f tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] PATH_INFO: `/v3/OS-TRUST/trusts/c46b47f76fb94022adc337301332692f/roles/8bc4910fa7fa4330ad0887be334d896e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:46.787615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1540a1e-9cd8-47aa-9bcf-c58ae704970f tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: Authorizing `identity:get_role_for_trust(trust_id=c46b47f76fb94022adc337301332692f, role_id=8bc4910fa7fa4330ad0887be334d896e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:46.789954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1540a1e-9cd8-47aa-9bcf-c58ae704970f tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:46.804452 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-d1540a1e-9cd8-47aa-9bcf-c58ae704970f tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Could not find role: 8bc4910fa7fa4330ad0887be334d896e.: keystone.exception.RoleNotFound: Could not find role: 8bc4910fa7fa4330ad0887be334d896e. May 28 21:14:46.804452 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:14:46.804452 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:14:46.804452 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:14:46.804452 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:14:46.804452 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:14:46.804452 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:14:46.804452 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:14:46.804452 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:14:46.804452 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:14:46.804452 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:14:46.804452 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:14:46.804452 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get May 28 21:14:46.804452 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 21:14:46.805505 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 8bc4910fa7fa4330ad0887be334d896e. May 28 21:14:46.805505 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:14:46.806981 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1987/3985] 10.222.0.33 () {58 vars in 1368 bytes} [Sat May 28 21:14:46 2022] HEAD /identity/v3/OS-TRUST/trusts/c46b47f76fb94022adc337301332692f/roles/8bc4910fa7fa4330ad0887be334d896e => generated 0 bytes in 63 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) May 28 21:14:46.812050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6ecd4-ef06-48b4-bcfd-365703a61e16 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:46.835361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6ecd4-ef06-48b4-bcfd-365703a61e16 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:46.838247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413665e1-a401-4c59-adbb-a3c29c0e3ac8 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21a509e865f844d69d27d400c7959178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06eb9b8d97204d8481d6e31a21fbc95c', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2070237270', 'tempest-NotDelegatedRole-389058313'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:46.839242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413665e1-a401-4c59-adbb-a3c29c0e3ac8 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:46.839653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413665e1-a401-4c59-adbb-a3c29c0e3ac8 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:46.839943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413665e1-a401-4c59-adbb-a3c29c0e3ac8 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] PATH_INFO: `/v3/OS-TRUST/trusts/c46b47f76fb94022adc337301332692f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:46.847262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413665e1-a401-4c59-adbb-a3c29c0e3ac8 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: Authorizing `identity:delete_trust(trust_id=c46b47f76fb94022adc337301332692f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:46.848734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413665e1-a401-4c59-adbb-a3c29c0e3ac8 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:46.944829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-413665e1-a401-4c59-adbb-a3c29c0e3ac8 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'c46b47f76fb94022adc337301332692f', 'request_id': 'req-413665e1-a401-4c59-adbb-a3c29c0e3ac8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:46.946389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-413665e1-a401-4c59-adbb-a3c29c0e3ac8 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': 'c46b47f76fb94022adc337301332692f', 'request_id': 'req-413665e1-a401-4c59-adbb-a3c29c0e3ac8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:47.031105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-413665e1-a401-4c59-adbb-a3c29c0e3ac8 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': 'c46b47f76fb94022adc337301332692f', 'request_id': 'req-413665e1-a401-4c59-adbb-a3c29c0e3ac8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:47.038565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bdb71-4bcf-4d27-b214-78845ea15103 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:47.047554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bdb71-4bcf-4d27-b214-78845ea15103 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:47.047554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652bdb71-4bcf-4d27-b214-78845ea15103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:47.048932 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 1999/3986] 10.222.0.33 () {58 vars in 1253 bytes} [Sat May 28 21:14:46 2022] DELETE /identity/v3/OS-TRUST/trusts/c46b47f76fb94022adc337301332692f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:47.050518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413665e1-a401-4c59-adbb-a3c29c0e3ac8 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:47.052110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-413665e1-a401-4c59-adbb-a3c29c0e3ac8 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:47.082277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-652bdb71-4bcf-4d27-b214-78845ea15103 None None] MFA Rules not processed for user `98ab12b74029438a9a1d19e505d1fc6d`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:47.083933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-652bdb71-4bcf-4d27-b214-78845ea15103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:47.093524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413665e1-a401-4c59-adbb-a3c29c0e3ac8 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:47.097923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4c071-5682-4bae-b861-c51cbb04bef7 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '21a509e865f844d69d27d400c7959178', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '06eb9b8d97204d8481d6e31a21fbc95c', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-2070237270', 'tempest-NotDelegatedRole-389058313'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:47.105236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4c071-5682-4bae-b861-c51cbb04bef7 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:47.105236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4c071-5682-4bae-b861-c51cbb04bef7 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:47.105236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4c071-5682-4bae-b861-c51cbb04bef7 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] PATH_INFO: `/v3/OS-TRUST/trusts/c46b47f76fb94022adc337301332692f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:47.116083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-3da4c071-5682-4bae-b861-c51cbb04bef7 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Could not find trust: c46b47f76fb94022adc337301332692f.: keystone.exception.TrustNotFound: Could not find trust: c46b47f76fb94022adc337301332692f. May 28 21:14:47.116083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:14:47.116083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:14:47.116083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:14:47.116083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:14:47.116083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:14:47.116083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:14:47.116083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:14:47.116083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:14:47.116083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:14:47.116083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:14:47.116083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:14:47.116083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get May 28 21:14:47.116083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) May 28 21:14:47.116083 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust May 28 21:14:47.117262 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', May 28 21:14:47.117262 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call May 28 21:14:47.117262 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() May 28 21:14:47.117262 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement May 28 21:14:47.117262 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( May 28 21:14:47.117262 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:14:47.117262 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:14:47.117262 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust May 28 21:14:47.117262 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) May 28 21:14:47.117262 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust May 28 21:14:47.117262 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) May 28 21:14:47.117262 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: c46b47f76fb94022adc337301332692f. May 28 21:14:47.117262 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:14:47.118117 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2000/3987] 10.222.0.33 () {58 vars in 1250 bytes} [Sat May 28 21:14:47 2022] GET /identity/v3/OS-TRUST/trusts/c46b47f76fb94022adc337301332692f => generated 111 bytes in 67 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) May 28 21:14:47.122956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4c071-5682-4bae-b861-c51cbb04bef7 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:47.125650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3da4c071-5682-4bae-b861-c51cbb04bef7 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:47.144386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4c071-5682-4bae-b861-c51cbb04bef7 tempest-TrustsV3TestJSON-1734352522 tempest-user-549849676] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:47.150684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f28731-09bc-4356-abc5-7de7810e7f45 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:47.151763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f28731-09bc-4356-abc5-7de7810e7f45 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:47.152158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f28731-09bc-4356-abc5-7de7810e7f45 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:47.152449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f28731-09bc-4356-abc5-7de7810e7f45 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles/8bc4910fa7fa4330ad0887be334d896e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:47.153978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f28731-09bc-4356-abc5-7de7810e7f45 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_role(role_id=8bc4910fa7fa4330ad0887be334d896e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:47.155580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f28731-09bc-4356-abc5-7de7810e7f45 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:47.164392 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1988/3988] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:14:47.418780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-00f28731-09bc-4356-abc5-7de7810e7f45 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invalidating the token cache because role 8bc4910fa7fa4330ad0887be334d896e has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:47.419151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-00f28731-09bc-4356-abc5-7de7810e7f45 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:47.419896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-00f28731-09bc-4356-abc5-7de7810e7f45 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:47.421857 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2001/3989] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:47 2022] DELETE /identity/v3/roles/8bc4910fa7fa4330ad0887be334d896e => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:14:47.426114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bdb71-4bcf-4d27-b214-78845ea15103 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:47.438693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652bdb71-4bcf-4d27-b214-78845ea15103 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:47.511076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90038bcb-019e-4b53-8c43-d1e7252f002c tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:47.511076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90038bcb-019e-4b53-8c43-d1e7252f002c tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:47.511076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90038bcb-019e-4b53-8c43-d1e7252f002c tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:47.511397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90038bcb-019e-4b53-8c43-d1e7252f002c tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles/f1bab1ad7e4a42cd96310564a2e7aedb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:47.512321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90038bcb-019e-4b53-8c43-d1e7252f002c tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_role(role_id=f1bab1ad7e4a42cd96310564a2e7aedb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:47.514003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90038bcb-019e-4b53-8c43-d1e7252f002c tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:47.719593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49552c93-d0fc-4fb3-8589-9021259b53c7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:47.719893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49552c93-d0fc-4fb3-8589-9021259b53c7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:47.720178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49552c93-d0fc-4fb3-8589-9021259b53c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:47.740110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-49552c93-d0fc-4fb3-8589-9021259b53c7 None None] MFA Rules not processed for user `98ab12b74029438a9a1d19e505d1fc6d`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:47.742524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-49552c93-d0fc-4fb3-8589-9021259b53c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:47.774102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-90038bcb-019e-4b53-8c43-d1e7252f002c tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invalidating the token cache because role f1bab1ad7e4a42cd96310564a2e7aedb has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:47.774380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-90038bcb-019e-4b53-8c43-d1e7252f002c tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:47.775653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-90038bcb-019e-4b53-8c43-d1e7252f002c tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:47.783597 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1989/3990] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:47 2022] DELETE /identity/v3/roles/f1bab1ad7e4a42cd96310564a2e7aedb => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:47.788463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90038bcb-019e-4b53-8c43-d1e7252f002c tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:47.790172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-90038bcb-019e-4b53-8c43-d1e7252f002c tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:47.818684 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2002/3991] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:14:47.916402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90038bcb-019e-4b53-8c43-d1e7252f002c tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:47.923466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff91f8-a158-4754-b869-fe3c507e2a79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:47.923855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff91f8-a158-4754-b869-fe3c507e2a79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:47.924267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff91f8-a158-4754-b869-fe3c507e2a79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:47.924707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff91f8-a158-4754-b869-fe3c507e2a79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users/21a509e865f844d69d27d400c7959178` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:47.926297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff91f8-a158-4754-b869-fe3c507e2a79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_user(user_id=21a509e865f844d69d27d400c7959178)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:47.929131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff91f8-a158-4754-b869-fe3c507e2a79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:48.101091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a5ff91f8-a158-4754-b869-fe3c507e2a79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21a509e865f844d69d27d400c7959178'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:48.102272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a5ff91f8-a158-4754-b869-fe3c507e2a79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21a509e865f844d69d27d400c7959178'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:48.117944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a5ff91f8-a158-4754-b869-fe3c507e2a79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21a509e865f844d69d27d400c7959178'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:48.208940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a5ff91f8-a158-4754-b869-fe3c507e2a79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21a509e865f844d69d27d400c7959178'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:48.211012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a5ff91f8-a158-4754-b869-fe3c507e2a79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21a509e865f844d69d27d400c7959178'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:48.255820 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1990/3992] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:47 2022] DELETE /identity/v3/users/21a509e865f844d69d27d400c7959178 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:48.259420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49552c93-d0fc-4fb3-8589-9021259b53c7 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:48.275972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49552c93-d0fc-4fb3-8589-9021259b53c7 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:48.368954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:48.370376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:48.370668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:48.371177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/06eb9b8d97204d8481d6e31a21fbc95c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:48.373326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_project(project_id=06eb9b8d97204d8481d6e31a21fbc95c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:48.374990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:48.385392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:48.665818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06eb9b8d97204d8481d6e31a21fbc95c', 'request_id': 'req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:48.673083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06eb9b8d97204d8481d6e31a21fbc95c', 'request_id': 'req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:48.762116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06eb9b8d97204d8481d6e31a21fbc95c', 'request_id': 'req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:48.786959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] The token cache is being invalidate because project 06eb9b8d97204d8481d6e31a21fbc95c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:48.787296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:48.788640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:48.792236 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2003/3993] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:48 2022] DELETE /identity/v3/projects/06eb9b8d97204d8481d6e31a21fbc95c => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:48.799494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff91f8-a158-4754-b869-fe3c507e2a79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:48.802360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a5ff91f8-a158-4754-b869-fe3c507e2a79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:48.819030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff91f8-a158-4754-b869-fe3c507e2a79 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:48.854371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:48.894448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979bf91-efb7-4d72-93a7-809f84a3c4e2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:48.895209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979bf91-efb7-4d72-93a7-809f84a3c4e2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:48.898135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979bf91-efb7-4d72-93a7-809f84a3c4e2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:48.898135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6979bf91-efb7-4d72-93a7-809f84a3c4e2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:48.898135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979bf91-efb7-4d72-93a7-809f84a3c4e2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:48.899722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6979bf91-efb7-4d72-93a7-809f84a3c4e2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:49.086604 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1991/3994] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:14:48 2022] POST /identity/v3/projects => generated 329 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:49.088153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:49.089793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:49.118277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2976f55f-bcd0-4977-88f4-0c3d9e97dea1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:49.206461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097e8cb-301a-4db2-b80e-c3f8b531eaa0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:49.207163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097e8cb-301a-4db2-b80e-c3f8b531eaa0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:49.207450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097e8cb-301a-4db2-b80e-c3f8b531eaa0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:49.207738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097e8cb-301a-4db2-b80e-c3f8b531eaa0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:49.208461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4097e8cb-301a-4db2-b80e-c3f8b531eaa0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:49.210979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4097e8cb-301a-4db2-b80e-c3f8b531eaa0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:49.410261 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2004/3995] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:14:49 2022] POST /identity/v3/users => generated 416 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:49.414061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979bf91-efb7-4d72-93a7-809f84a3c4e2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:49.426778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6979bf91-efb7-4d72-93a7-809f84a3c4e2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:49.432129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79da38b-12fc-40d7-bd3d-d1dfb084e045 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:49.432815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79da38b-12fc-40d7-bd3d-d1dfb084e045 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:49.433106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79da38b-12fc-40d7-bd3d-d1dfb084e045 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:49.433389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79da38b-12fc-40d7-bd3d-d1dfb084e045 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:49.434219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79da38b-12fc-40d7-bd3d-d1dfb084e045 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:49.435858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79da38b-12fc-40d7-bd3d-d1dfb084e045 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:49.543345 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1992/3996] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:14:49 2022] POST /identity/v3/roles => generated 244 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:14:49.547262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097e8cb-301a-4db2-b80e-c3f8b531eaa0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:49.564081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097e8cb-301a-4db2-b80e-c3f8b531eaa0 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:49.571331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6898f7-126d-4721-84cf-d976ef92f412 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:49.572345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6898f7-126d-4721-84cf-d976ef92f412 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:49.572979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6898f7-126d-4721-84cf-d976ef92f412 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:49.573269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6898f7-126d-4721-84cf-d976ef92f412 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:49.574230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6898f7-126d-4721-84cf-d976ef92f412 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:49.576231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6898f7-126d-4721-84cf-d976ef92f412 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:49.744973 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2005/3997] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:14:49 2022] POST /identity/v3/roles => generated 246 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:14:49.749391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79da38b-12fc-40d7-bd3d-d1dfb084e045 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:49.761445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79da38b-12fc-40d7-bd3d-d1dfb084e045 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:49.767255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c101ae4-2f3e-4697-9b16-40e4203f4f19 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:49.768180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c101ae4-2f3e-4697-9b16-40e4203f4f19 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:49.768645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c101ae4-2f3e-4697-9b16-40e4203f4f19 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:49.768934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c101ae4-2f3e-4697-9b16-40e4203f4f19 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/96a85c14fd5b4b768778ea2e07bc771b/users/1171e0c28bf54a1b9b2bca119aa6e42b/roles/5d3090b9e36c4173bd9518bcdd8a4e4b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:49.785395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c101ae4-2f3e-4697-9b16-40e4203f4f19 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_grant(project_id=96a85c14fd5b4b768778ea2e07bc771b, user_id=1171e0c28bf54a1b9b2bca119aa6e42b, role_id=5d3090b9e36c4173bd9518bcdd8a4e4b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:49.788223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c101ae4-2f3e-4697-9b16-40e4203f4f19 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:49.947327 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1993/3998] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:49 2022] PUT /identity/v3/projects/96a85c14fd5b4b768778ea2e07bc771b/users/1171e0c28bf54a1b9b2bca119aa6e42b/roles/5d3090b9e36c4173bd9518bcdd8a4e4b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:49.951362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6898f7-126d-4721-84cf-d976ef92f412 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:49.965457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6898f7-126d-4721-84cf-d976ef92f412 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:50.073007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfdc82-92d6-498d-95a3-d9e64905dff2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:50.074112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfdc82-92d6-498d-95a3-d9e64905dff2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:50.074471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfdc82-92d6-498d-95a3-d9e64905dff2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:50.075909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfdc82-92d6-498d-95a3-d9e64905dff2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/96a85c14fd5b4b768778ea2e07bc771b/users/1171e0c28bf54a1b9b2bca119aa6e42b/roles/38f43281638c4c2092f71db874bf4d77` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:50.078566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfdc82-92d6-498d-95a3-d9e64905dff2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:create_grant(project_id=96a85c14fd5b4b768778ea2e07bc771b, user_id=1171e0c28bf54a1b9b2bca119aa6e42b, role_id=38f43281638c4c2092f71db874bf4d77)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:50.081360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfdc82-92d6-498d-95a3-d9e64905dff2 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:50.197291 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2006/3999] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:49 2022] PUT /identity/v3/projects/96a85c14fd5b4b768778ea2e07bc771b/users/1171e0c28bf54a1b9b2bca119aa6e42b/roles/38f43281638c4c2092f71db874bf4d77 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:50.201088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c101ae4-2f3e-4697-9b16-40e4203f4f19 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:50.218367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c101ae4-2f3e-4697-9b16-40e4203f4f19 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:50.305491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6960b848-41f0-4ea3-ba61-b80c0f9f76c3 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:50.307002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960b848-41f0-4ea3-ba61-b80c0f9f76c3 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:50.307307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960b848-41f0-4ea3-ba61-b80c0f9f76c3 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:50.307594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6960b848-41f0-4ea3-ba61-b80c0f9f76c3 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:50.308514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960b848-41f0-4ea3-ba61-b80c0f9f76c3 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:50.313682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6960b848-41f0-4ea3-ba61-b80c0f9f76c3 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:50.349608 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1994/4000] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:50 2022] GET /identity/v3/users => generated 8636 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:50.354728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8edb62-df39-468a-9d53-0319a993c4fe None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:50.355052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8edb62-df39-468a-9d53-0319a993c4fe None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:50.355338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8edb62-df39-468a-9d53-0319a993c4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:50.395945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-de8edb62-df39-468a-9d53-0319a993c4fe None None] MFA Rules not processed for user `1171e0c28bf54a1b9b2bca119aa6e42b`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:50.397197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-de8edb62-df39-468a-9d53-0319a993c4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:50.461800 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2007/4001] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:50.472956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa216c6-5105-4183-bf08-d398a9c7f86a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:50.473264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa216c6-5105-4183-bf08-d398a9c7f86a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:50.473529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa216c6-5105-4183-bf08-d398a9c7f86a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:50.491492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-afa216c6-5105-4183-bf08-d398a9c7f86a None None] MFA Rules not processed for user `1171e0c28bf54a1b9b2bca119aa6e42b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:50.492801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-afa216c6-5105-4183-bf08-d398a9c7f86a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:50.501409 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1995/4002] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:50.507336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8edb62-df39-468a-9d53-0319a993c4fe None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:50.517563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8edb62-df39-468a-9d53-0319a993c4fe None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:50.522207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6f8e0-ce0b-4ff4-ad10-a9c0bd471b41 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1171e0c28bf54a1b9b2bca119aa6e42b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96a85c14fd5b4b768778ea2e07bc771b', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1716920579', 'tempest-NotDelegatedRole-721918843'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:50.522850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6f8e0-ce0b-4ff4-ad10-a9c0bd471b41 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:50.523133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6f8e0-ce0b-4ff4-ad10-a9c0bd471b41 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:50.523412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6f8e0-ce0b-4ff4-ad10-a9c0bd471b41 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] PATH_INFO: `/v3/OS-TRUST/trusts` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:50.524384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6f8e0-ce0b-4ff4-ad10-a9c0bd471b41 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: Authorizing `identity:create_trust()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:50.525552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6f8e0-ce0b-4ff4-ad10-a9c0bd471b41 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:50.681108 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2008/4003] 10.222.0.33 () {60 vars in 1173 bytes} [Sat May 28 21:14:50 2022] POST /identity/v3/OS-TRUST/trusts => generated 835 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:50.682364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa216c6-5105-4183-bf08-d398a9c7f86a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:50.691572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa216c6-5105-4183-bf08-d398a9c7f86a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:50.696726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25019b61-ec30-4785-8322-546bf5a29f7a tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1171e0c28bf54a1b9b2bca119aa6e42b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96a85c14fd5b4b768778ea2e07bc771b', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1716920579', 'tempest-NotDelegatedRole-721918843'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:50.697566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25019b61-ec30-4785-8322-546bf5a29f7a tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:50.697869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25019b61-ec30-4785-8322-546bf5a29f7a tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:50.698168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25019b61-ec30-4785-8322-546bf5a29f7a tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] PATH_INFO: `/v3/OS-TRUST/trusts/46a7ae158785452a90bcec61a733b213` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:50.704290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25019b61-ec30-4785-8322-546bf5a29f7a tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: Authorizing `identity:get_trust(trust_id=46a7ae158785452a90bcec61a733b213)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:50.705758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25019b61-ec30-4785-8322-546bf5a29f7a tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:50.713480 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1996/4004] 10.222.0.33 () {58 vars in 1250 bytes} [Sat May 28 21:14:50 2022] GET /identity/v3/OS-TRUST/trusts/46a7ae158785452a90bcec61a733b213 => generated 835 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:14:50.723228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6f8e0-ce0b-4ff4-ad10-a9c0bd471b41 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:50.735092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6f8e0-ce0b-4ff4-ad10-a9c0bd471b41 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:50.740333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faebfd-be0b-43db-bf08-3a4898faeab0 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1171e0c28bf54a1b9b2bca119aa6e42b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96a85c14fd5b4b768778ea2e07bc771b', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1716920579', 'tempest-NotDelegatedRole-721918843'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:50.745194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faebfd-be0b-43db-bf08-3a4898faeab0 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:50.745194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faebfd-be0b-43db-bf08-3a4898faeab0 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:50.745194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56faebfd-be0b-43db-bf08-3a4898faeab0 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] PATH_INFO: `/v3/OS-TRUST/trusts/46a7ae158785452a90bcec61a733b213/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:50.764955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56faebfd-be0b-43db-bf08-3a4898faeab0 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: Authorizing `identity:list_roles_for_trust(trust_id=46a7ae158785452a90bcec61a733b213)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:50.767064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56faebfd-be0b-43db-bf08-3a4898faeab0 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:50.794931 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2009/4005] 10.222.0.33 () {58 vars in 1268 bytes} [Sat May 28 21:14:50 2022] GET /identity/v3/OS-TRUST/trusts/46a7ae158785452a90bcec61a733b213/roles => generated 371 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:50.800233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25019b61-ec30-4785-8322-546bf5a29f7a tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:50.817528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25019b61-ec30-4785-8322-546bf5a29f7a tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:50.823042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982982b-c418-4056-9b08-fe7e72ff538a tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1171e0c28bf54a1b9b2bca119aa6e42b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96a85c14fd5b4b768778ea2e07bc771b', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1716920579', 'tempest-NotDelegatedRole-721918843'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:50.823876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982982b-c418-4056-9b08-fe7e72ff538a tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:50.824337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982982b-c418-4056-9b08-fe7e72ff538a tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:50.824796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982982b-c418-4056-9b08-fe7e72ff538a tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] PATH_INFO: `/v3/OS-TRUST/trusts/46a7ae158785452a90bcec61a733b213/roles/5d3090b9e36c4173bd9518bcdd8a4e4b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:50.832645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:UPDATE=1,keystone:INSERT=2,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:50.837363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982982b-c418-4056-9b08-fe7e72ff538a tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: Authorizing `identity:get_role_for_trust(trust_id=46a7ae158785452a90bcec61a733b213, role_id=5d3090b9e36c4173bd9518bcdd8a4e4b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:50.838878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982982b-c418-4056-9b08-fe7e72ff538a tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:50.846077 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1997/4006] 10.222.0.33 () {58 vars in 1367 bytes} [Sat May 28 21:14:50 2022] GET /identity/v3/OS-TRUST/trusts/46a7ae158785452a90bcec61a733b213/roles/5d3090b9e36c4173bd9518bcdd8a4e4b => generated 244 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:50.850230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faebfd-be0b-43db-bf08-3a4898faeab0 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:50.860879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56faebfd-be0b-43db-bf08-3a4898faeab0 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:50.866695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c220b-10f1-43d0-8900-1784d40f28fa tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1171e0c28bf54a1b9b2bca119aa6e42b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96a85c14fd5b4b768778ea2e07bc771b', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1716920579', 'tempest-NotDelegatedRole-721918843'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:50.867633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c220b-10f1-43d0-8900-1784d40f28fa tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] REQUEST_METHOD: `HEAD` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:50.870844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c220b-10f1-43d0-8900-1784d40f28fa tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:50.870844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c220b-10f1-43d0-8900-1784d40f28fa tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] PATH_INFO: `/v3/OS-TRUST/trusts/46a7ae158785452a90bcec61a733b213/roles/5d3090b9e36c4173bd9518bcdd8a4e4b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:50.884303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c220b-10f1-43d0-8900-1784d40f28fa tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: Authorizing `identity:get_role_for_trust(trust_id=46a7ae158785452a90bcec61a733b213, role_id=5d3090b9e36c4173bd9518bcdd8a4e4b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:50.886534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c220b-10f1-43d0-8900-1784d40f28fa tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:50.899638 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2010/4007] 10.222.0.33 () {58 vars in 1368 bytes} [Sat May 28 21:14:50 2022] HEAD /identity/v3/OS-TRUST/trusts/46a7ae158785452a90bcec61a733b213/roles/5d3090b9e36c4173bd9518bcdd8a4e4b => generated 0 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) May 28 21:14:50.900484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982982b-c418-4056-9b08-fe7e72ff538a tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:50.909987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982982b-c418-4056-9b08-fe7e72ff538a tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:50.914250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca627d-f7f1-4b8b-a848-bbdfaf9dedff tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1171e0c28bf54a1b9b2bca119aa6e42b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96a85c14fd5b4b768778ea2e07bc771b', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1716920579', 'tempest-NotDelegatedRole-721918843'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:50.915195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca627d-f7f1-4b8b-a848-bbdfaf9dedff tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:50.915478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca627d-f7f1-4b8b-a848-bbdfaf9dedff tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:50.915758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca627d-f7f1-4b8b-a848-bbdfaf9dedff tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] PATH_INFO: `/v3/OS-TRUST/trusts/46a7ae158785452a90bcec61a733b213/roles/38f43281638c4c2092f71db874bf4d77` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:50.921747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca627d-f7f1-4b8b-a848-bbdfaf9dedff tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: Authorizing `identity:get_role_for_trust(trust_id=46a7ae158785452a90bcec61a733b213, role_id=38f43281638c4c2092f71db874bf4d77)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:50.923504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca627d-f7f1-4b8b-a848-bbdfaf9dedff tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:50.938971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-e0ca627d-f7f1-4b8b-a848-bbdfaf9dedff tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Could not find role: 38f43281638c4c2092f71db874bf4d77.: keystone.exception.RoleNotFound: Could not find role: 38f43281638c4c2092f71db874bf4d77. May 28 21:14:50.938971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:14:50.938971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:14:50.938971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:14:50.938971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:14:50.938971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:14:50.938971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:14:50.938971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:14:50.938971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:14:50.938971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:14:50.938971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:14:50.938971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:14:50.938971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get May 28 21:14:50.938971 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 21:14:50.939880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 38f43281638c4c2092f71db874bf4d77. May 28 21:14:50.939880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:14:50.940417 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1998/4008] 10.222.0.33 () {58 vars in 1367 bytes} [Sat May 28 21:14:50 2022] GET /identity/v3/OS-TRUST/trusts/46a7ae158785452a90bcec61a733b213/roles/38f43281638c4c2092f71db874bf4d77 => generated 110 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:14:50.944982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c220b-10f1-43d0-8900-1784d40f28fa tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:50.955221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c220b-10f1-43d0-8900-1784d40f28fa tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:50.959743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2dbfb2-6979-48b9-9b86-8d46b9c9c7a4 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1171e0c28bf54a1b9b2bca119aa6e42b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96a85c14fd5b4b768778ea2e07bc771b', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1716920579', 'tempest-NotDelegatedRole-721918843'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:50.961253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2dbfb2-6979-48b9-9b86-8d46b9c9c7a4 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] REQUEST_METHOD: `HEAD` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:50.961539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2dbfb2-6979-48b9-9b86-8d46b9c9c7a4 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:50.961857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2dbfb2-6979-48b9-9b86-8d46b9c9c7a4 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] PATH_INFO: `/v3/OS-TRUST/trusts/46a7ae158785452a90bcec61a733b213/roles/38f43281638c4c2092f71db874bf4d77` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:50.967853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2dbfb2-6979-48b9-9b86-8d46b9c9c7a4 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: Authorizing `identity:get_role_for_trust(trust_id=46a7ae158785452a90bcec61a733b213, role_id=38f43281638c4c2092f71db874bf4d77)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:50.969370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2dbfb2-6979-48b9-9b86-8d46b9c9c7a4 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:50.985150 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-0d2dbfb2-6979-48b9-9b86-8d46b9c9c7a4 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Could not find role: 38f43281638c4c2092f71db874bf4d77.: keystone.exception.RoleNotFound: Could not find role: 38f43281638c4c2092f71db874bf4d77. May 28 21:14:50.985150 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:14:50.985150 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:14:50.985150 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:14:50.985150 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:14:50.985150 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:14:50.985150 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:14:50.985150 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:14:50.985150 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:14:50.985150 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:14:50.985150 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:14:50.985150 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:14:50.985150 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 438, in get May 28 21:14:50.985150 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 21:14:50.986221 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 38f43281638c4c2092f71db874bf4d77. May 28 21:14:50.986221 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:14:50.986815 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2011/4009] 10.222.0.33 () {58 vars in 1368 bytes} [Sat May 28 21:14:50 2022] HEAD /identity/v3/OS-TRUST/trusts/46a7ae158785452a90bcec61a733b213/roles/38f43281638c4c2092f71db874bf4d77 => generated 0 bytes in 43 msecs (HTTP/1.1 404) 5 headers in 184 bytes (0 switches on core 0) May 28 21:14:50.990892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca627d-f7f1-4b8b-a848-bbdfaf9dedff tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:51.000289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca627d-f7f1-4b8b-a848-bbdfaf9dedff tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:51.007148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb8c04-3d58-4255-be4b-c6cf00494a60 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1171e0c28bf54a1b9b2bca119aa6e42b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96a85c14fd5b4b768778ea2e07bc771b', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1716920579', 'tempest-NotDelegatedRole-721918843'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:51.007692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb8c04-3d58-4255-be4b-c6cf00494a60 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:51.008036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb8c04-3d58-4255-be4b-c6cf00494a60 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:51.008492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb8c04-3d58-4255-be4b-c6cf00494a60 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] PATH_INFO: `/v3/OS-TRUST/trusts/46a7ae158785452a90bcec61a733b213` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:51.015083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb8c04-3d58-4255-be4b-c6cf00494a60 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: Authorizing `identity:delete_trust(trust_id=46a7ae158785452a90bcec61a733b213)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:51.016688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cb8c04-3d58-4255-be4b-c6cf00494a60 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:51.150462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b5cb8c04-3d58-4255-be4b-c6cf00494a60 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Invoking callback _drop_receipt_cache for event identity OS-TRUST:trust deleted for {'resource_info': '46a7ae158785452a90bcec61a733b213', 'request_id': 'req-b5cb8c04-3d58-4255-be4b-c6cf00494a60'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:51.151341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b5cb8c04-3d58-4255-be4b-c6cf00494a60 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Invoking callback _trust_callback for event identity OS-TRUST:trust deleted for {'resource_info': '46a7ae158785452a90bcec61a733b213', 'request_id': 'req-b5cb8c04-3d58-4255-be4b-c6cf00494a60'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:51.230682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b5cb8c04-3d58-4255-be4b-c6cf00494a60 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Invoking callback _drop_token_cache for event identity OS-TRUST:trust deleted for {'resource_info': '46a7ae158785452a90bcec61a733b213', 'request_id': 'req-b5cb8c04-3d58-4255-be4b-c6cf00494a60'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:51.243568 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 1999/4010] 10.222.0.33 () {58 vars in 1253 bytes} [Sat May 28 21:14:50 2022] DELETE /identity/v3/OS-TRUST/trusts/46a7ae158785452a90bcec61a733b213 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:51.247895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2dbfb2-6979-48b9-9b86-8d46b9c9c7a4 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:51.249960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0d2dbfb2-6979-48b9-9b86-8d46b9c9c7a4 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:51.263389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2dbfb2-6979-48b9-9b86-8d46b9c9c7a4 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:51.267641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d298d9-4b5b-4372-b7dd-dde07c5126f0 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1171e0c28bf54a1b9b2bca119aa6e42b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '96a85c14fd5b4b768778ea2e07bc771b', 'project_domain_id': 'default', 'roles': ['tempest-DelegatedRole-1716920579', 'tempest-NotDelegatedRole-721918843'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:51.268479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d298d9-4b5b-4372-b7dd-dde07c5126f0 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:51.268812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d298d9-4b5b-4372-b7dd-dde07c5126f0 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:51.269091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d298d9-4b5b-4372-b7dd-dde07c5126f0 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] PATH_INFO: `/v3/OS-TRUST/trusts/46a7ae158785452a90bcec61a733b213` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:51.281896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-e2d298d9-4b5b-4372-b7dd-dde07c5126f0 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Could not find trust: 46a7ae158785452a90bcec61a733b213.: keystone.exception.TrustNotFound: Could not find trust: 46a7ae158785452a90bcec61a733b213. May 28 21:14:51.281896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:14:51.281896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:14:51.281896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:14:51.281896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:14:51.281896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:14:51.281896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:14:51.281896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:14:51.281896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:14:51.281896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:14:51.281896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:14:51.281896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:14:51.281896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 268, in get May 28 21:14:51.281896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_trust(trust_id=trust_id) May 28 21:14:51.281896 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 167, in _get_trust May 28 21:14:51.282903 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_trust', May 28 21:14:51.282903 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call May 28 21:14:51.282903 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() May 28 21:14:51.282903 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/trusts.py", line 54, in _build_trust_target_enforcement May 28 21:14:51.282903 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application target['trust'] = PROVIDERS.trust_api.get_trust( May 28 21:14:51.282903 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:14:51.282903 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:14:51.282903 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/core.py", line 117, in get_trust May 28 21:14:51.282903 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application trust = self.driver.get_trust(trust_id, deleted) May 28 21:14:51.282903 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/trust/backends/sql.py", line 148, in get_trust May 28 21:14:51.282903 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.TrustNotFound(trust_id=trust_id) May 28 21:14:51.282903 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.TrustNotFound: Could not find trust: 46a7ae158785452a90bcec61a733b213. May 28 21:14:51.282903 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:14:51.283649 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2012/4011] 10.222.0.33 () {58 vars in 1250 bytes} [Sat May 28 21:14:51 2022] GET /identity/v3/OS-TRUST/trusts/46a7ae158785452a90bcec61a733b213 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:14:51.289881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb8c04-3d58-4255-be4b-c6cf00494a60 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:51.291381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b5cb8c04-3d58-4255-be4b-c6cf00494a60 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:51.307022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb8c04-3d58-4255-be4b-c6cf00494a60 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:51.311781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe069d-99fc-41bc-b93b-6cf524c34108 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:51.312641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe069d-99fc-41bc-b93b-6cf524c34108 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:51.312931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe069d-99fc-41bc-b93b-6cf524c34108 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:51.313212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe069d-99fc-41bc-b93b-6cf524c34108 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles/38f43281638c4c2092f71db874bf4d77` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:51.314736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe069d-99fc-41bc-b93b-6cf524c34108 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_role(role_id=38f43281638c4c2092f71db874bf4d77)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:51.316361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe069d-99fc-41bc-b93b-6cf524c34108 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:51.597355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-88fe069d-99fc-41bc-b93b-6cf524c34108 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invalidating the token cache because role 38f43281638c4c2092f71db874bf4d77 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:51.597729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-88fe069d-99fc-41bc-b93b-6cf524c34108 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:51.598502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-88fe069d-99fc-41bc-b93b-6cf524c34108 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:51.604110 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2000/4012] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:51 2022] DELETE /identity/v3/roles/38f43281638c4c2092f71db874bf4d77 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:51.605463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d298d9-4b5b-4372-b7dd-dde07c5126f0 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:51.608004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e2d298d9-4b5b-4372-b7dd-dde07c5126f0 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:51.627905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d298d9-4b5b-4372-b7dd-dde07c5126f0 tempest-TrustsV3TestJSON-1453109348 tempest-user-1418611219] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:51.717258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92976a-3aae-4bf9-8f51-95261eead34f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:51.718355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92976a-3aae-4bf9-8f51-95261eead34f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:51.718652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92976a-3aae-4bf9-8f51-95261eead34f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:51.719072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92976a-3aae-4bf9-8f51-95261eead34f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/roles/5d3090b9e36c4173bd9518bcdd8a4e4b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:51.722315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92976a-3aae-4bf9-8f51-95261eead34f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_role(role_id=5d3090b9e36c4173bd9518bcdd8a4e4b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:51.724103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92976a-3aae-4bf9-8f51-95261eead34f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:51.945037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fb92976a-3aae-4bf9-8f51-95261eead34f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invalidating the token cache because role 5d3090b9e36c4173bd9518bcdd8a4e4b has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:51.945616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fb92976a-3aae-4bf9-8f51-95261eead34f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:51.946966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fb92976a-3aae-4bf9-8f51-95261eead34f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:51.955582 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2013/4013] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:51 2022] DELETE /identity/v3/roles/5d3090b9e36c4173bd9518bcdd8a4e4b => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:51.959939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe069d-99fc-41bc-b93b-6cf524c34108 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:51.974814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe069d-99fc-41bc-b93b-6cf524c34108 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:52.046628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747db9e3-9941-48b0-b48d-5e0a0719b6ff tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:52.047694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747db9e3-9941-48b0-b48d-5e0a0719b6ff tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:52.048050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747db9e3-9941-48b0-b48d-5e0a0719b6ff tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:52.048399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747db9e3-9941-48b0-b48d-5e0a0719b6ff tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/users/1171e0c28bf54a1b9b2bca119aa6e42b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:52.049750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747db9e3-9941-48b0-b48d-5e0a0719b6ff tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_user(user_id=1171e0c28bf54a1b9b2bca119aa6e42b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:52.078053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747db9e3-9941-48b0-b48d-5e0a0719b6ff tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:52.130262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:52.342957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-747db9e3-9941-48b0-b48d-5e0a0719b6ff tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1171e0c28bf54a1b9b2bca119aa6e42b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:52.346048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-747db9e3-9941-48b0-b48d-5e0a0719b6ff tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1171e0c28bf54a1b9b2bca119aa6e42b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:52.356335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-747db9e3-9941-48b0-b48d-5e0a0719b6ff tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1171e0c28bf54a1b9b2bca119aa6e42b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:52.442266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-747db9e3-9941-48b0-b48d-5e0a0719b6ff tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1171e0c28bf54a1b9b2bca119aa6e42b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:52.443599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-747db9e3-9941-48b0-b48d-5e0a0719b6ff tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1171e0c28bf54a1b9b2bca119aa6e42b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:52.485959 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2001/4014] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:51 2022] DELETE /identity/v3/users/1171e0c28bf54a1b9b2bca119aa6e42b => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:14:52.490363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92976a-3aae-4bf9-8f51-95261eead34f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:52.492693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fb92976a-3aae-4bf9-8f51-95261eead34f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:52.524188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92976a-3aae-4bf9-8f51-95261eead34f tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:52.642749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f67783-2315-4e3f-8098-76ab120265e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6057efa7010c4d17a359d408338f7b44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316491af215543789aad43617ed68878', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:52.643774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67783-2315-4e3f-8098-76ab120265e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:52.644361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67783-2315-4e3f-8098-76ab120265e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:52.644894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67783-2315-4e3f-8098-76ab120265e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] PATH_INFO: `/v3/projects/96a85c14fd5b4b768778ea2e07bc771b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:52.646557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f67783-2315-4e3f-8098-76ab120265e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorizing `identity:delete_project(project_id=96a85c14fd5b4b768778ea2e07bc771b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:52.649061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f67783-2315-4e3f-8098-76ab120265e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:52.837161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-90f67783-2315-4e3f-8098-76ab120265e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96a85c14fd5b4b768778ea2e07bc771b', 'request_id': 'req-90f67783-2315-4e3f-8098-76ab120265e1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:52.847860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-90f67783-2315-4e3f-8098-76ab120265e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96a85c14fd5b4b768778ea2e07bc771b', 'request_id': 'req-90f67783-2315-4e3f-8098-76ab120265e1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:52.911602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-90f67783-2315-4e3f-8098-76ab120265e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96a85c14fd5b4b768778ea2e07bc771b', 'request_id': 'req-90f67783-2315-4e3f-8098-76ab120265e1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:52.929910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-90f67783-2315-4e3f-8098-76ab120265e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] The token cache is being invalidate because project 96a85c14fd5b4b768778ea2e07bc771b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:52.934694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-90f67783-2315-4e3f-8098-76ab120265e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:52.936100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-90f67783-2315-4e3f-8098-76ab120265e1 tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:52.947196 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2014/4015] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:52 2022] DELETE /identity/v3/projects/96a85c14fd5b4b768778ea2e07bc771b => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:52.948617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747db9e3-9941-48b0-b48d-5e0a0719b6ff tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:52.950414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-747db9e3-9941-48b0-b48d-5e0a0719b6ff tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:53.087219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747db9e3-9941-48b0-b48d-5e0a0719b6ff tempest-TrustsV3TestJSON-1850160836 tempest-TrustsV3TestJSON-1850160836-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:53.152524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38324af5-ba44-4f12-a5bf-3f60c5c346d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:53.153971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38324af5-ba44-4f12-a5bf-3f60c5c346d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:53.154280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38324af5-ba44-4f12-a5bf-3f60c5c346d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:53.154554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38324af5-ba44-4f12-a5bf-3f60c5c346d9 admin admin] PATH_INFO: `/v3/users/6ff700b7c46146f7bd2529a633352164` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:53.156061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38324af5-ba44-4f12-a5bf-3f60c5c346d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff700b7c46146f7bd2529a633352164)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:53.158290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38324af5-ba44-4f12-a5bf-3f60c5c346d9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:53.455853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-38324af5-ba44-4f12-a5bf-3f60c5c346d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff700b7c46146f7bd2529a633352164'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:53.456547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-38324af5-ba44-4f12-a5bf-3f60c5c346d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff700b7c46146f7bd2529a633352164'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:53.470565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-38324af5-ba44-4f12-a5bf-3f60c5c346d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff700b7c46146f7bd2529a633352164'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:53.551291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:53.573212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-38324af5-ba44-4f12-a5bf-3f60c5c346d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff700b7c46146f7bd2529a633352164'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:53.577035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-38324af5-ba44-4f12-a5bf-3f60c5c346d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff700b7c46146f7bd2529a633352164'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:53.595545 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2002/4016] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:52 2022] DELETE /identity/v3/users/6ff700b7c46146f7bd2529a633352164 => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:53.600094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43643989-659f-4008-9045-61550073a785 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:53.601035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43643989-659f-4008-9045-61550073a785 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:53.601353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43643989-659f-4008-9045-61550073a785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:53.622977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-43643989-659f-4008-9045-61550073a785 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:53.624357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-43643989-659f-4008-9045-61550073a785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:53.711742 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2015/4017] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:53.720184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38324af5-ba44-4f12-a5bf-3f60c5c346d9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:53.723263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-38324af5-ba44-4f12-a5bf-3f60c5c346d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:53.792309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38324af5-ba44-4f12-a5bf-3f60c5c346d9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:53.796253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa77ab-d469-4771-82b9-5bb4cefc37a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:53.801116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa77ab-d469-4771-82b9-5bb4cefc37a6 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:53.801116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa77ab-d469-4771-82b9-5bb4cefc37a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:53.801116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa77ab-d469-4771-82b9-5bb4cefc37a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:53.808165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa77ab-d469-4771-82b9-5bb4cefc37a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:53.809733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa77ab-d469-4771-82b9-5bb4cefc37a6 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:53.822270 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2003/4018] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:14:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:14:54.136232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:54.437710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:54.922031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43643989-659f-4008-9045-61550073a785 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:54.923808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-43643989-659f-4008-9045-61550073a785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:54.939071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43643989-659f-4008-9045-61550073a785 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:54.944785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a04ca-5432-4faf-a48e-cca1332af66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:54.945942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a04ca-5432-4faf-a48e-cca1332af66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:54.946575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a04ca-5432-4faf-a48e-cca1332af66d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:54.946921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a04ca-5432-4faf-a48e-cca1332af66d admin admin] PATH_INFO: `/v3/projects/f4592fcdcdd94b8ea88f4b9ad5219c58` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:54.948204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a04ca-5432-4faf-a48e-cca1332af66d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4592fcdcdd94b8ea88f4b9ad5219c58)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:54.950507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a04ca-5432-4faf-a48e-cca1332af66d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:54.975674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:55.066538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-035a04ca-5432-4faf-a48e-cca1332af66d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4592fcdcdd94b8ea88f4b9ad5219c58', 'request_id': 'req-035a04ca-5432-4faf-a48e-cca1332af66d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:55.072875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-035a04ca-5432-4faf-a48e-cca1332af66d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4592fcdcdd94b8ea88f4b9ad5219c58', 'request_id': 'req-035a04ca-5432-4faf-a48e-cca1332af66d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:55.173357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-035a04ca-5432-4faf-a48e-cca1332af66d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4592fcdcdd94b8ea88f4b9ad5219c58', 'request_id': 'req-035a04ca-5432-4faf-a48e-cca1332af66d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:55.189345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-035a04ca-5432-4faf-a48e-cca1332af66d admin admin] The token cache is being invalidate because project f4592fcdcdd94b8ea88f4b9ad5219c58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:55.189774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-035a04ca-5432-4faf-a48e-cca1332af66d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:55.190650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-035a04ca-5432-4faf-a48e-cca1332af66d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:55.201692 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2016/4019] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:54 2022] DELETE /identity/v3/projects/f4592fcdcdd94b8ea88f4b9ad5219c58 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:14:55.205325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa77ab-d469-4771-82b9-5bb4cefc37a6 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:55.229788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa77ab-d469-4771-82b9-5bb4cefc37a6 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:55.319637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:55.321817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296aa43-edbe-431c-b5c3-5194a82239cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:55.322757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296aa43-edbe-431c-b5c3-5194a82239cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:55.323081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296aa43-edbe-431c-b5c3-5194a82239cc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:55.323377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296aa43-edbe-431c-b5c3-5194a82239cc admin admin] PATH_INFO: `/v3/users/6057efa7010c4d17a359d408338f7b44` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:55.324625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296aa43-edbe-431c-b5c3-5194a82239cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6057efa7010c4d17a359d408338f7b44)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:55.327065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296aa43-edbe-431c-b5c3-5194a82239cc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:55.594762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b296aa43-edbe-431c-b5c3-5194a82239cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6057efa7010c4d17a359d408338f7b44'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:55.596233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b296aa43-edbe-431c-b5c3-5194a82239cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6057efa7010c4d17a359d408338f7b44'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:55.608753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b296aa43-edbe-431c-b5c3-5194a82239cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6057efa7010c4d17a359d408338f7b44'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:55.704803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b296aa43-edbe-431c-b5c3-5194a82239cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6057efa7010c4d17a359d408338f7b44'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:55.705707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b296aa43-edbe-431c-b5c3-5194a82239cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6057efa7010c4d17a359d408338f7b44'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:55.741728 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2004/4020] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:55 2022] DELETE /identity/v3/users/6057efa7010c4d17a359d408338f7b44 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:14:56.442139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a04ca-5432-4faf-a48e-cca1332af66d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:56.446250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-035a04ca-5432-4faf-a48e-cca1332af66d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:56.469874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a04ca-5432-4faf-a48e-cca1332af66d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:56.596168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751a85a-2733-4fcb-8df7-6af039907ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:56.597584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751a85a-2733-4fcb-8df7-6af039907ca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:56.598056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751a85a-2733-4fcb-8df7-6af039907ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:56.598487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751a85a-2733-4fcb-8df7-6af039907ca1 admin admin] PATH_INFO: `/v3/projects/316491af215543789aad43617ed68878` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:56.599951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5751a85a-2733-4fcb-8df7-6af039907ca1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=316491af215543789aad43617ed68878)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:56.603281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5751a85a-2733-4fcb-8df7-6af039907ca1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:56.792973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5751a85a-2733-4fcb-8df7-6af039907ca1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '316491af215543789aad43617ed68878', 'request_id': 'req-5751a85a-2733-4fcb-8df7-6af039907ca1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:56.799856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5751a85a-2733-4fcb-8df7-6af039907ca1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '316491af215543789aad43617ed68878', 'request_id': 'req-5751a85a-2733-4fcb-8df7-6af039907ca1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:56.927887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5751a85a-2733-4fcb-8df7-6af039907ca1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '316491af215543789aad43617ed68878', 'request_id': 'req-5751a85a-2733-4fcb-8df7-6af039907ca1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:56.940666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5751a85a-2733-4fcb-8df7-6af039907ca1 admin admin] The token cache is being invalidate because project 316491af215543789aad43617ed68878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:56.940962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5751a85a-2733-4fcb-8df7-6af039907ca1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:56.941605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5751a85a-2733-4fcb-8df7-6af039907ca1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:56.943214 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2017/4021] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:56 2022] DELETE /identity/v3/projects/316491af215543789aad43617ed68878 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:56.948686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2205be6-c43d-4a7e-82b1-593f08132ed4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:56.949166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2205be6-c43d-4a7e-82b1-593f08132ed4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:56.949273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2205be6-c43d-4a7e-82b1-593f08132ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:56.969496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d2205be6-c43d-4a7e-82b1-593f08132ed4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:56.970706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d2205be6-c43d-4a7e-82b1-593f08132ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:57.041888 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2005/4022] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:57.120899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360ab8d-dc23-4f33-a529-faac833684fa None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:57.125994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360ab8d-dc23-4f33-a529-faac833684fa None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:57.125994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e360ab8d-dc23-4f33-a529-faac833684fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:57.125994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:57.144420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e360ab8d-dc23-4f33-a529-faac833684fa None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:57.145699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e360ab8d-dc23-4f33-a529-faac833684fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:57.155197 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2018/4023] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:57.159426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2205be6-c43d-4a7e-82b1-593f08132ed4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:57.169739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2205be6-c43d-4a7e-82b1-593f08132ed4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:57.175103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23cb34-6dc1-4a27-91f9-06bf48db375b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:57.176175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23cb34-6dc1-4a27-91f9-06bf48db375b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:57.176563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23cb34-6dc1-4a27-91f9-06bf48db375b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:57.176941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23cb34-6dc1-4a27-91f9-06bf48db375b admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:57.178808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23cb34-6dc1-4a27-91f9-06bf48db375b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:14:57.179539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23cb34-6dc1-4a27-91f9-06bf48db375b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:57.181287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23cb34-6dc1-4a27-91f9-06bf48db375b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:57.198208 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2006/4024] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:57.202519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360ab8d-dc23-4f33-a529-faac833684fa None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:57.214087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e360ab8d-dc23-4f33-a529-faac833684fa None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:57.220946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6e46b5-d97d-47b9-91e3-cb2663595763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:57.221684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e46b5-d97d-47b9-91e3-cb2663595763 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:57.222119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e46b5-d97d-47b9-91e3-cb2663595763 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:57.222638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6e46b5-d97d-47b9-91e3-cb2663595763 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:57.223453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e46b5-d97d-47b9-91e3-cb2663595763 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:57.225644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6e46b5-d97d-47b9-91e3-cb2663595763 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:57.318081 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2019/4025] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:14:57 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:57.322873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23cb34-6dc1-4a27-91f9-06bf48db375b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:57.333583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23cb34-6dc1-4a27-91f9-06bf48db375b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:57.394566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be073687-5115-45aa-a615-2ba65000bdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:57.395339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be073687-5115-45aa-a615-2ba65000bdfb admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:57.395627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be073687-5115-45aa-a615-2ba65000bdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:57.395932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be073687-5115-45aa-a615-2ba65000bdfb admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:57.396628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be073687-5115-45aa-a615-2ba65000bdfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:57.398988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be073687-5115-45aa-a615-2ba65000bdfb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:57.456745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132644a-3ded-4217-8f09-c40137bff5f5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:57.457253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132644a-3ded-4217-8f09-c40137bff5f5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:57.457351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1132644a-3ded-4217-8f09-c40137bff5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:57.475696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1132644a-3ded-4217-8f09-c40137bff5f5 None None] MFA Rules not processed for user `572170ccd67e4facbf10dad7cd1491c1`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:57.477000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1132644a-3ded-4217-8f09-c40137bff5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:57.531930 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2020/4026] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:14:57.533869 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2007/4027] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:14:57 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:14:57.538216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132644a-3ded-4217-8f09-c40137bff5f5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:57.549325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1132644a-3ded-4217-8f09-c40137bff5f5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:57.554844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a02607-fe8e-44d8-88a8-d9878df7808c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:57.555551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a02607-fe8e-44d8-88a8-d9878df7808c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:57.555836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a02607-fe8e-44d8-88a8-d9878df7808c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:57.556104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a02607-fe8e-44d8-88a8-d9878df7808c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:57.556841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a02607-fe8e-44d8-88a8-d9878df7808c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:57.558552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a02607-fe8e-44d8-88a8-d9878df7808c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:57.569392 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2021/4028] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:14:57.572941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be073687-5115-45aa-a615-2ba65000bdfb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:57.583527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be073687-5115-45aa-a615-2ba65000bdfb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:57.589420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc9c83b-b52e-416f-b23c-6eff932b75e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:57.589577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9c83b-b52e-416f-b23c-6eff932b75e9 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:57.589824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9c83b-b52e-416f-b23c-6eff932b75e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:57.593392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc9c83b-b52e-416f-b23c-6eff932b75e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:57.593392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9c83b-b52e-416f-b23c-6eff932b75e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:57.593392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc9c83b-b52e-416f-b23c-6eff932b75e9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:57.609253 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2008/4029] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:14:57.610708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a02607-fe8e-44d8-88a8-d9878df7808c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:57.621490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a02607-fe8e-44d8-88a8-d9878df7808c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:57.626138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee86cd-acb2-4bdc-ad46-ea08adeea9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:57.627025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee86cd-acb2-4bdc-ad46-ea08adeea9ef admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:57.627310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee86cd-acb2-4bdc-ad46-ea08adeea9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:57.627593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee86cd-acb2-4bdc-ad46-ea08adeea9ef admin admin] PATH_INFO: `/v3/projects/2ed6422f2fe2486887da3a7adabe5db3/users/bdbc46d0dbac4c04ad9b05fa5907d285/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:57.644924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee86cd-acb2-4bdc-ad46-ea08adeea9ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ed6422f2fe2486887da3a7adabe5db3, user_id=bdbc46d0dbac4c04ad9b05fa5907d285, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:57.647679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee86cd-acb2-4bdc-ad46-ea08adeea9ef admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:57.750844 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2022/4030] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:14:57 2022] PUT /identity/v3/projects/2ed6422f2fe2486887da3a7adabe5db3/users/bdbc46d0dbac4c04ad9b05fa5907d285/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:57.762374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40f48c-7462-4cd1-8b0a-c8bb50d537e0 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:57.762936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40f48c-7462-4cd1-8b0a-c8bb50d537e0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:57.766653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40f48c-7462-4cd1-8b0a-c8bb50d537e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:57.797097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7f40f48c-7462-4cd1-8b0a-c8bb50d537e0 None None] MFA Rules not processed for user `bdbc46d0dbac4c04ad9b05fa5907d285`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:57.797097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7f40f48c-7462-4cd1-8b0a-c8bb50d537e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:57.901610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee86cd-acb2-4bdc-ad46-ea08adeea9ef admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:57.915966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:57.915966 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2009/4031] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:57.922880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee86cd-acb2-4bdc-ad46-ea08adeea9ef admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:57.927592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e94a6-3991-4508-9b20-eeb012a067c4 tempest-DefaultDomainTestJSON-814258234 tempest-DefaultDomainTestJSON-814258234-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bdbc46d0dbac4c04ad9b05fa5907d285', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ed6422f2fe2486887da3a7adabe5db3', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:57.928973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e94a6-3991-4508-9b20-eeb012a067c4 tempest-DefaultDomainTestJSON-814258234 tempest-DefaultDomainTestJSON-814258234-project] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:57.929267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e94a6-3991-4508-9b20-eeb012a067c4 tempest-DefaultDomainTestJSON-814258234 tempest-DefaultDomainTestJSON-814258234-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:57.929553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e94a6-3991-4508-9b20-eeb012a067c4 tempest-DefaultDomainTestJSON-814258234 tempest-DefaultDomainTestJSON-814258234-project] PATH_INFO: `/v3/domains/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:57.930893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e94a6-3991-4508-9b20-eeb012a067c4 tempest-DefaultDomainTestJSON-814258234 tempest-DefaultDomainTestJSON-814258234-project] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:57.933276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e94a6-3991-4508-9b20-eeb012a067c4 tempest-DefaultDomainTestJSON-814258234 tempest-DefaultDomainTestJSON-814258234-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:57.936621 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2023/4032] 10.222.0.33 () {58 vars in 1151 bytes} [Sat May 28 21:14:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:57.945158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40f48c-7462-4cd1-8b0a-c8bb50d537e0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:57.992148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:58.029770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40f48c-7462-4cd1-8b0a-c8bb50d537e0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:58.035333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6767e62-9f99-4fe8-bb4e-25554829c2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:58.036331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6767e62-9f99-4fe8-bb4e-25554829c2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:58.036605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6767e62-9f99-4fe8-bb4e-25554829c2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:58.036919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6767e62-9f99-4fe8-bb4e-25554829c2f7 admin admin] PATH_INFO: `/v3/users/bdbc46d0dbac4c04ad9b05fa5907d285` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:58.038357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6767e62-9f99-4fe8-bb4e-25554829c2f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdbc46d0dbac4c04ad9b05fa5907d285)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:58.040303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6767e62-9f99-4fe8-bb4e-25554829c2f7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:58.283637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e6767e62-9f99-4fe8-bb4e-25554829c2f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdbc46d0dbac4c04ad9b05fa5907d285'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:58.284625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e6767e62-9f99-4fe8-bb4e-25554829c2f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdbc46d0dbac4c04ad9b05fa5907d285'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:58.320075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e6767e62-9f99-4fe8-bb4e-25554829c2f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdbc46d0dbac4c04ad9b05fa5907d285'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:58.329375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:58.442938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e6767e62-9f99-4fe8-bb4e-25554829c2f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdbc46d0dbac4c04ad9b05fa5907d285'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:58.443656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e6767e62-9f99-4fe8-bb4e-25554829c2f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdbc46d0dbac4c04ad9b05fa5907d285'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:58.472140 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2010/4033] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:14:57 2022] DELETE /identity/v3/users/bdbc46d0dbac4c04ad9b05fa5907d285 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:58.483834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f5bff-6bb7-4c30-a768-20bbe3363db2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:58.485227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f5bff-6bb7-4c30-a768-20bbe3363db2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:58.487267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f5bff-6bb7-4c30-a768-20bbe3363db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:58.514345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a48f5bff-6bb7-4c30-a768-20bbe3363db2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:58.516006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a48f5bff-6bb7-4c30-a768-20bbe3363db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:58.585320 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2024/4034] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:58.594767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6767e62-9f99-4fe8-bb4e-25554829c2f7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:58.596518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e6767e62-9f99-4fe8-bb4e-25554829c2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:58.643121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:58.745461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6767e62-9f99-4fe8-bb4e-25554829c2f7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:58.749603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e85b27-9d73-410b-97f0-e17ddde4bb56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:58.750598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e85b27-9d73-410b-97f0-e17ddde4bb56 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:58.750969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e85b27-9d73-410b-97f0-e17ddde4bb56 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:58.751302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e85b27-9d73-410b-97f0-e17ddde4bb56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:58.764475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e85b27-9d73-410b-97f0-e17ddde4bb56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:58.766541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e85b27-9d73-410b-97f0-e17ddde4bb56 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:58.786480 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2011/4035] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:14:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:14:58.982179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5644c-1f51-45bb-ad31-190e35e41598 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:58.982611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5644c-1f51-45bb-ad31-190e35e41598 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:58.982848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd5644c-1f51-45bb-ad31-190e35e41598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:59.002059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ddd5644c-1f51-45bb-ad31-190e35e41598 None None] MFA Rules not processed for user `572170ccd67e4facbf10dad7cd1491c1`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:59.003599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ddd5644c-1f51-45bb-ad31-190e35e41598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:59.120556 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2025/4036] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:59.313699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e85b27-9d73-410b-97f0-e17ddde4bb56 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:59.315266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-46e85b27-9d73-410b-97f0-e17ddde4bb56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:59.349727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e85b27-9d73-410b-97f0-e17ddde4bb56 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:59.356147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad8fca-a808-42c1-a4dc-b234756a2441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:59.356767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad8fca-a808-42c1-a4dc-b234756a2441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:59.357297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad8fca-a808-42c1-a4dc-b234756a2441 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:59.357956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad8fca-a808-42c1-a4dc-b234756a2441 admin admin] PATH_INFO: `/v3/projects/2ed6422f2fe2486887da3a7adabe5db3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:59.362862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad8fca-a808-42c1-a4dc-b234756a2441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ed6422f2fe2486887da3a7adabe5db3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:59.364977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad8fca-a808-42c1-a4dc-b234756a2441 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:59.496346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-23ad8fca-a808-42c1-a4dc-b234756a2441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ed6422f2fe2486887da3a7adabe5db3', 'request_id': 'req-23ad8fca-a808-42c1-a4dc-b234756a2441'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:59.501854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-23ad8fca-a808-42c1-a4dc-b234756a2441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ed6422f2fe2486887da3a7adabe5db3', 'request_id': 'req-23ad8fca-a808-42c1-a4dc-b234756a2441'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:59.625124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-23ad8fca-a808-42c1-a4dc-b234756a2441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ed6422f2fe2486887da3a7adabe5db3', 'request_id': 'req-23ad8fca-a808-42c1-a4dc-b234756a2441'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:59.638802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-23ad8fca-a808-42c1-a4dc-b234756a2441 admin admin] The token cache is being invalidate because project 2ed6422f2fe2486887da3a7adabe5db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:14:59.639095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-23ad8fca-a808-42c1-a4dc-b234756a2441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:59.639818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-23ad8fca-a808-42c1-a4dc-b234756a2441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:14:59.641347 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2012/4037] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:14:59 2022] DELETE /identity/v3/projects/2ed6422f2fe2486887da3a7adabe5db3 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:14:59.647571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6af53-4f61-4a4b-9d30-86a6b492f825 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:59.648957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6af53-4f61-4a4b-9d30-86a6b492f825 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:59.649249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6af53-4f61-4a4b-9d30-86a6b492f825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:59.668982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4df6af53-4f61-4a4b-9d30-86a6b492f825 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:59.670225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4df6af53-4f61-4a4b-9d30-86a6b492f825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:59.737456 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2026/4038] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:59.745353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc8ffa-c410-46c8-9bb0-035cd38c6491 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:59.747487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc8ffa-c410-46c8-9bb0-035cd38c6491 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:59.747487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dc8ffa-c410-46c8-9bb0-035cd38c6491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:59.766082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b3dc8ffa-c410-46c8-9bb0-035cd38c6491 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:14:59.770566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b3dc8ffa-c410-46c8-9bb0-035cd38c6491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:14:59.776613 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2013/4039] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:14:59.780934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6af53-4f61-4a4b-9d30-86a6b492f825 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:59.791395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6af53-4f61-4a4b-9d30-86a6b492f825 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:59.797166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba29760-1f33-4630-b958-31fa04641fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:59.798052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba29760-1f33-4630-b958-31fa04641fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:59.798397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba29760-1f33-4630-b958-31fa04641fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:59.798713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba29760-1f33-4630-b958-31fa04641fe4 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:59.799584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba29760-1f33-4630-b958-31fa04641fe4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:14:59.799895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba29760-1f33-4630-b958-31fa04641fe4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:59.801579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba29760-1f33-4630-b958-31fa04641fe4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:59.813658 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2027/4040] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:14:59.818026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc8ffa-c410-46c8-9bb0-035cd38c6491 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:59.829483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dc8ffa-c410-46c8-9bb0-035cd38c6491 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:14:59.834208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa0b2f-d0cf-4990-b9ec-805adbe826a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:14:59.835539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa0b2f-d0cf-4990-b9ec-805adbe826a6 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:14:59.835820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa0b2f-d0cf-4990-b9ec-805adbe826a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:14:59.836098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfa0b2f-d0cf-4990-b9ec-805adbe826a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:14:59.836801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa0b2f-d0cf-4990-b9ec-805adbe826a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:14:59.838810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfa0b2f-d0cf-4990-b9ec-805adbe826a6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:14:59.938840 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2014/4041] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:14:59 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:14:59.943038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba29760-1f33-4630-b958-31fa04641fe4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:14:59.954830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba29760-1f33-4630-b958-31fa04641fe4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:00.013293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52829399-b35c-459c-b7d0-02b30c9671b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:00.014040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52829399-b35c-459c-b7d0-02b30c9671b6 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:00.014468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52829399-b35c-459c-b7d0-02b30c9671b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:00.014753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52829399-b35c-459c-b7d0-02b30c9671b6 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:00.015459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52829399-b35c-459c-b7d0-02b30c9671b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:00.017649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52829399-b35c-459c-b7d0-02b30c9671b6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:00.173332 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2028/4042] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:14:59 2022] POST /identity/v3/users => generated 296 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:00.177445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa0b2f-d0cf-4990-b9ec-805adbe826a6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:00.187891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfa0b2f-d0cf-4990-b9ec-805adbe826a6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:00.195170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7c81e-df35-4ffd-b7d9-a54b4fba519a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:00.195972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7c81e-df35-4ffd-b7d9-a54b4fba519a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:00.196255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7c81e-df35-4ffd-b7d9-a54b4fba519a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:00.196562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7c81e-df35-4ffd-b7d9-a54b4fba519a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:00.197297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7c81e-df35-4ffd-b7d9-a54b4fba519a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:00.198963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7c81e-df35-4ffd-b7d9-a54b4fba519a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:00.211495 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2015/4043] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:00.215498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52829399-b35c-459c-b7d0-02b30c9671b6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:00.227072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52829399-b35c-459c-b7d0-02b30c9671b6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:00.232323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacd996-8021-4179-bee3-5506e94297ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:00.233215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacd996-8021-4179-bee3-5506e94297ae admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:00.233501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacd996-8021-4179-bee3-5506e94297ae admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:00.233777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacd996-8021-4179-bee3-5506e94297ae admin admin] PATH_INFO: `/v3/projects/76f52ecba1234c34a4489d7141a32338/users/be3010e7f4d647119ff01541a733d5ac/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:00.249861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacd996-8021-4179-bee3-5506e94297ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=76f52ecba1234c34a4489d7141a32338, user_id=be3010e7f4d647119ff01541a733d5ac, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:00.252938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacd996-8021-4179-bee3-5506e94297ae admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:00.300478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:00.305279 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2029/4044] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:00 2022] PUT /identity/v3/projects/76f52ecba1234c34a4489d7141a32338/users/be3010e7f4d647119ff01541a733d5ac/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:00.309641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7c81e-df35-4ffd-b7d9-a54b4fba519a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:00.319828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7c81e-df35-4ffd-b7d9-a54b4fba519a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:00.381817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0000037-a8ec-483f-9945-0c41a3466595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:00.382532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0000037-a8ec-483f-9945-0c41a3466595 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:00.383125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0000037-a8ec-483f-9945-0c41a3466595 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:00.383241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0000037-a8ec-483f-9945-0c41a3466595 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:00.384016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0000037-a8ec-483f-9945-0c41a3466595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:00.385664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0000037-a8ec-483f-9945-0c41a3466595 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:00.396820 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2016/4045] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:00.401247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacd996-8021-4179-bee3-5506e94297ae admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:00.413672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacd996-8021-4179-bee3-5506e94297ae admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:00.420803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854200e-e36b-4b0a-94e7-7e0ba5dcff36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:00.422550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854200e-e36b-4b0a-94e7-7e0ba5dcff36 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:00.423075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854200e-e36b-4b0a-94e7-7e0ba5dcff36 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:00.423622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854200e-e36b-4b0a-94e7-7e0ba5dcff36 admin admin] PATH_INFO: `/v3/projects/76f52ecba1234c34a4489d7141a32338/users/be3010e7f4d647119ff01541a733d5ac/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:00.425978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854200e-e36b-4b0a-94e7-7e0ba5dcff36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76f52ecba1234c34a4489d7141a32338, user_id=be3010e7f4d647119ff01541a733d5ac, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:00.429031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854200e-e36b-4b0a-94e7-7e0ba5dcff36 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:00.442954 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2030/4046] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:00 2022] PUT /identity/v3/projects/76f52ecba1234c34a4489d7141a32338/users/be3010e7f4d647119ff01541a733d5ac/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:00.454657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be93e32-1db8-4897-96ae-21774f73af3e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:00.454951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be93e32-1db8-4897-96ae-21774f73af3e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:00.455689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be93e32-1db8-4897-96ae-21774f73af3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:00.487806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4be93e32-1db8-4897-96ae-21774f73af3e None None] MFA Rules not processed for user `be3010e7f4d647119ff01541a733d5ac`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:00.489043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4be93e32-1db8-4897-96ae-21774f73af3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:00.560926 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2017/4047] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:00.565539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854200e-e36b-4b0a-94e7-7e0ba5dcff36 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:00.660476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854200e-e36b-4b0a-94e7-7e0ba5dcff36 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:00.668338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90685e2-3f0b-4785-9d73-1c7d85a2b0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:00.669027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90685e2-3f0b-4785-9d73-1c7d85a2b0c7 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:00.669315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90685e2-3f0b-4785-9d73-1c7d85a2b0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:00.669598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90685e2-3f0b-4785-9d73-1c7d85a2b0c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:00.670477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90685e2-3f0b-4785-9d73-1c7d85a2b0c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:00.673346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90685e2-3f0b-4785-9d73-1c7d85a2b0c7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:00.754017 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2031/4048] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:00 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:00.758847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be93e32-1db8-4897-96ae-21774f73af3e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:00.768725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:00.772160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be93e32-1db8-4897-96ae-21774f73af3e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:00.924431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327d31a-6e5f-491a-a521-5007f6eb61f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:00.927344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327d31a-6e5f-491a-a521-5007f6eb61f6 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:00.929348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327d31a-6e5f-491a-a521-5007f6eb61f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:00.930474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327d31a-6e5f-491a-a521-5007f6eb61f6 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:00.931921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327d31a-6e5f-491a-a521-5007f6eb61f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:00.934164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327d31a-6e5f-491a-a521-5007f6eb61f6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:01.111202 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2018/4049] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:00 2022] POST /identity/v3/users => generated 291 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:01.116960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90685e2-3f0b-4785-9d73-1c7d85a2b0c7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:01.127960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90685e2-3f0b-4785-9d73-1c7d85a2b0c7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:01.136171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1aacc-c542-4671-8422-fc9727f7a828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:01.137397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1aacc-c542-4671-8422-fc9727f7a828 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:01.137898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1aacc-c542-4671-8422-fc9727f7a828 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:01.138284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c1aacc-c542-4671-8422-fc9727f7a828 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:01.139094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1aacc-c542-4671-8422-fc9727f7a828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:01.140853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c1aacc-c542-4671-8422-fc9727f7a828 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:01.152370 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2032/4050] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:01.156660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327d31a-6e5f-491a-a521-5007f6eb61f6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:01.170269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327d31a-6e5f-491a-a521-5007f6eb61f6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:01.175972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bc696-53d2-4f4b-a506-941d4041f848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:01.176638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bc696-53d2-4f4b-a506-941d4041f848 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:01.176967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bc696-53d2-4f4b-a506-941d4041f848 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:01.177245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bc696-53d2-4f4b-a506-941d4041f848 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:01.178014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bc696-53d2-4f4b-a506-941d4041f848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:01.180517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bc696-53d2-4f4b-a506-941d4041f848 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:01.195628 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2019/4051] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:01.196885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1aacc-c542-4671-8422-fc9727f7a828 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:01.208530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c1aacc-c542-4671-8422-fc9727f7a828 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:01.213456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e118bc9-79c4-4833-a6d0-d1b5fca42688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:01.214410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e118bc9-79c4-4833-a6d0-d1b5fca42688 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:01.214692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e118bc9-79c4-4833-a6d0-d1b5fca42688 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:01.215059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e118bc9-79c4-4833-a6d0-d1b5fca42688 admin admin] PATH_INFO: `/v3/projects/e83a3a0546c64a689a70d06c58a0d2a9/users/d449b9b92a94491db6a558c6d27ac2d3/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:01.234796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e118bc9-79c4-4833-a6d0-d1b5fca42688 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e83a3a0546c64a689a70d06c58a0d2a9, user_id=d449b9b92a94491db6a558c6d27ac2d3, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:01.237615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e118bc9-79c4-4833-a6d0-d1b5fca42688 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:01.350156 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2033/4052] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:01 2022] PUT /identity/v3/projects/e83a3a0546c64a689a70d06c58a0d2a9/users/d449b9b92a94491db6a558c6d27ac2d3/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:01.361038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147edffe-a206-4ac1-8479-d2a7d4351054 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:01.361538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147edffe-a206-4ac1-8479-d2a7d4351054 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:01.361658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147edffe-a206-4ac1-8479-d2a7d4351054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:01.388495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-147edffe-a206-4ac1-8479-d2a7d4351054 None None] MFA Rules not processed for user `d449b9b92a94491db6a558c6d27ac2d3`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:01.389730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-147edffe-a206-4ac1-8479-d2a7d4351054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:01.460653 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2020/4053] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:01.472742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e118bc9-79c4-4833-a6d0-d1b5fca42688 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:01.474797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0e118bc9-79c4-4833-a6d0-d1b5fca42688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:01.549378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e118bc9-79c4-4833-a6d0-d1b5fca42688 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:01.554428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02876514-39d0-450f-ab93-b700705733a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:01.555088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02876514-39d0-450f-ab93-b700705733a3 service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:01.555373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02876514-39d0-450f-ab93-b700705733a3 service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:01.555653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02876514-39d0-450f-ab93-b700705733a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:01.620603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02876514-39d0-450f-ab93-b700705733a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:01.622775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02876514-39d0-450f-ab93-b700705733a3 service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:01.633615 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2034/4054] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:02.088380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:UPDATE=1,keystone:INSERT=1,keystone:DELETE=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:02.405906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:02.544038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147edffe-a206-4ac1-8479-d2a7d4351054 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:02.619128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147edffe-a206-4ac1-8479-d2a7d4351054 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:02.633709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2be73-c038-4815-9a5c-5ffd2af06b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:02.634669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2be73-c038-4815-9a5c-5ffd2af06b8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:02.634950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2be73-c038-4815-9a5c-5ffd2af06b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:02.635250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2be73-c038-4815-9a5c-5ffd2af06b8f admin admin] PATH_INFO: `/v3/users/be3010e7f4d647119ff01541a733d5ac` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:02.637287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2be73-c038-4815-9a5c-5ffd2af06b8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=be3010e7f4d647119ff01541a733d5ac)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:02.639340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd2be73-c038-4815-9a5c-5ffd2af06b8f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:02.818233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bcd2be73-c038-4815-9a5c-5ffd2af06b8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be3010e7f4d647119ff01541a733d5ac'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:02.819003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bcd2be73-c038-4815-9a5c-5ffd2af06b8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be3010e7f4d647119ff01541a733d5ac'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:02.827355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bcd2be73-c038-4815-9a5c-5ffd2af06b8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be3010e7f4d647119ff01541a733d5ac'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:02.909706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bcd2be73-c038-4815-9a5c-5ffd2af06b8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be3010e7f4d647119ff01541a733d5ac'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:02.910379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bcd2be73-c038-4815-9a5c-5ffd2af06b8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be3010e7f4d647119ff01541a733d5ac'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:02.929981 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2021/4055] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:15:02 2022] DELETE /identity/v3/users/be3010e7f4d647119ff01541a733d5ac => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:02.934856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2e69e-0335-4e35-9979-767b5f83c77a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:02.935148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2e69e-0335-4e35-9979-767b5f83c77a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:02.935439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d2e69e-0335-4e35-9979-767b5f83c77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:02.955336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-86d2e69e-0335-4e35-9979-767b5f83c77a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:02.961510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-86d2e69e-0335-4e35-9979-767b5f83c77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:03.022840 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2035/4056] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:03.032649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2be73-c038-4815-9a5c-5ffd2af06b8f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:03.034275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bcd2be73-c038-4815-9a5c-5ffd2af06b8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:03.138573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd2be73-c038-4815-9a5c-5ffd2af06b8f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:03.144282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba20da4-477f-4fb0-b46d-d6ee518b33c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:03.145182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba20da4-477f-4fb0-b46d-d6ee518b33c5 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:03.145623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba20da4-477f-4fb0-b46d-d6ee518b33c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:03.146841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba20da4-477f-4fb0-b46d-d6ee518b33c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:03.158846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba20da4-477f-4fb0-b46d-d6ee518b33c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:03.160624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba20da4-477f-4fb0-b46d-d6ee518b33c5 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:03.173392 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2022/4057] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:03.636405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:03.771525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2e69e-0335-4e35-9979-767b5f83c77a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:03.771525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-86d2e69e-0335-4e35-9979-767b5f83c77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:03.782616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d2e69e-0335-4e35-9979-767b5f83c77a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:03.788937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:03.790090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:03.798994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:03.799317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55 admin admin] PATH_INFO: `/v3/projects/76f52ecba1234c34a4489d7141a32338` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:03.801301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76f52ecba1234c34a4489d7141a32338)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:03.803441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:03.934259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76f52ecba1234c34a4489d7141a32338', 'request_id': 'req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:03.942007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76f52ecba1234c34a4489d7141a32338', 'request_id': 'req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:04.012837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76f52ecba1234c34a4489d7141a32338', 'request_id': 'req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:04.025433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55 admin admin] The token cache is being invalidate because project 76f52ecba1234c34a4489d7141a32338 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:15:04.025750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:04.026435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:04.028053 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2036/4058] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:03 2022] DELETE /identity/v3/projects/76f52ecba1234c34a4489d7141a32338 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:04.032307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba20da4-477f-4fb0-b46d-d6ee518b33c5 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:04.043644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba20da4-477f-4fb0-b46d-d6ee518b33c5 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:04.136944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e958b30-d3bb-43eb-b82a-fc0f48effc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:04.138335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e958b30-d3bb-43eb-b82a-fc0f48effc56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:04.138834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e958b30-d3bb-43eb-b82a-fc0f48effc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:04.139118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e958b30-d3bb-43eb-b82a-fc0f48effc56 admin admin] PATH_INFO: `/v3/users/d449b9b92a94491db6a558c6d27ac2d3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:04.140366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e958b30-d3bb-43eb-b82a-fc0f48effc56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d449b9b92a94491db6a558c6d27ac2d3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:04.142411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e958b30-d3bb-43eb-b82a-fc0f48effc56 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:04.390991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:04.405724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9e958b30-d3bb-43eb-b82a-fc0f48effc56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd449b9b92a94491db6a558c6d27ac2d3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:04.406529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9e958b30-d3bb-43eb-b82a-fc0f48effc56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd449b9b92a94491db6a558c6d27ac2d3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:04.436927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9e958b30-d3bb-43eb-b82a-fc0f48effc56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd449b9b92a94491db6a558c6d27ac2d3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:04.481921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9e958b30-d3bb-43eb-b82a-fc0f48effc56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd449b9b92a94491db6a558c6d27ac2d3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:04.482669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9e958b30-d3bb-43eb-b82a-fc0f48effc56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd449b9b92a94491db6a558c6d27ac2d3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:04.505437 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2023/4059] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:15:04 2022] DELETE /identity/v3/users/d449b9b92a94491db6a558c6d27ac2d3 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:04.545286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:04.917159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:04.917159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:04.940804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bf1c6-ffda-4fca-ac1a-d8289b6eed55 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:05.007741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b04a5-d641-456c-b47e-6e4a2ab58455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:05.008624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b04a5-d641-456c-b47e-6e4a2ab58455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:05.008946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b04a5-d641-456c-b47e-6e4a2ab58455 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:05.009200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b04a5-d641-456c-b47e-6e4a2ab58455 admin admin] PATH_INFO: `/v3/projects/e83a3a0546c64a689a70d06c58a0d2a9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:05.010458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b04a5-d641-456c-b47e-6e4a2ab58455 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e83a3a0546c64a689a70d06c58a0d2a9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:05.012425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b04a5-d641-456c-b47e-6e4a2ab58455 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:05.130120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e44b04a5-d641-456c-b47e-6e4a2ab58455 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e83a3a0546c64a689a70d06c58a0d2a9', 'request_id': 'req-e44b04a5-d641-456c-b47e-6e4a2ab58455'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:05.136757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e44b04a5-d641-456c-b47e-6e4a2ab58455 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e83a3a0546c64a689a70d06c58a0d2a9', 'request_id': 'req-e44b04a5-d641-456c-b47e-6e4a2ab58455'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:05.236133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e44b04a5-d641-456c-b47e-6e4a2ab58455 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e83a3a0546c64a689a70d06c58a0d2a9', 'request_id': 'req-e44b04a5-d641-456c-b47e-6e4a2ab58455'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:05.249788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e44b04a5-d641-456c-b47e-6e4a2ab58455 admin admin] The token cache is being invalidate because project e83a3a0546c64a689a70d06c58a0d2a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:15:05.250119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e44b04a5-d641-456c-b47e-6e4a2ab58455 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:05.250800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e44b04a5-d641-456c-b47e-6e4a2ab58455 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:05.252441 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2037/4060] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:04 2022] DELETE /identity/v3/projects/e83a3a0546c64a689a70d06c58a0d2a9 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:05.258523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6899c7-462d-4b7f-a6da-bb31fd630313 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:05.259012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6899c7-462d-4b7f-a6da-bb31fd630313 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:05.259266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6899c7-462d-4b7f-a6da-bb31fd630313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:05.281235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-cc6899c7-462d-4b7f-a6da-bb31fd630313 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:05.282479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cc6899c7-462d-4b7f-a6da-bb31fd630313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:05.322929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:05.350732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:05.358877 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2024/4061] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:15:05.368596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b0f3b-7d64-4d09-a052-26e450a47778 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:05.368891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b0f3b-7d64-4d09-a052-26e450a47778 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:05.369176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b0f3b-7d64-4d09-a052-26e450a47778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:05.393264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6c9b0f3b-7d64-4d09-a052-26e450a47778 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:05.394716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6c9b0f3b-7d64-4d09-a052-26e450a47778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:05.404249 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2038/4062] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:05.408516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6899c7-462d-4b7f-a6da-bb31fd630313 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:05.420703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6899c7-462d-4b7f-a6da-bb31fd630313 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:05.426722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb1cfb-09a3-4966-94b2-c2d7723457e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:05.427406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb1cfb-09a3-4966-94b2-c2d7723457e0 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:05.427815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb1cfb-09a3-4966-94b2-c2d7723457e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:05.428109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb1cfb-09a3-4966-94b2-c2d7723457e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:05.429412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb1cfb-09a3-4966-94b2-c2d7723457e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:15:05.429725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb1cfb-09a3-4966-94b2-c2d7723457e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:05.431403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb1cfb-09a3-4966-94b2-c2d7723457e0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:05.443996 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2025/4063] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:05.448732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b0f3b-7d64-4d09-a052-26e450a47778 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:05.460338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b0f3b-7d64-4d09-a052-26e450a47778 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:05.465432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcf24a-3627-4b47-9f96-f14057eb311e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:05.466128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcf24a-3627-4b47-9f96-f14057eb311e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:05.466481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcf24a-3627-4b47-9f96-f14057eb311e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:05.466823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfcf24a-3627-4b47-9f96-f14057eb311e admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:05.467596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcf24a-3627-4b47-9f96-f14057eb311e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:05.469536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfcf24a-3627-4b47-9f96-f14057eb311e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:05.560519 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2039/4064] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:05 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:15:05.567677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb1cfb-09a3-4966-94b2-c2d7723457e0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:05.579799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb1cfb-09a3-4966-94b2-c2d7723457e0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:05.659005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:05.663403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f7548-7098-487a-ad40-30186b1c6b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:05.664093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f7548-7098-487a-ad40-30186b1c6b9a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:05.664383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f7548-7098-487a-ad40-30186b1c6b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:05.664666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f7548-7098-487a-ad40-30186b1c6b9a admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:05.665366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f7548-7098-487a-ad40-30186b1c6b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:05.667355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f7548-7098-487a-ad40-30186b1c6b9a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:05.722919 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2026/4065] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:05 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:05.723790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcf24a-3627-4b47-9f96-f14057eb311e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:05.738571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfcf24a-3627-4b47-9f96-f14057eb311e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:05.745464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbbf2f-e4be-4e56-bfa2-dba6a4f2392a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:05.746347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbbf2f-e4be-4e56-bfa2-dba6a4f2392a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:05.746824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbbf2f-e4be-4e56-bfa2-dba6a4f2392a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:05.747264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfbbf2f-e4be-4e56-bfa2-dba6a4f2392a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:05.748335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbbf2f-e4be-4e56-bfa2-dba6a4f2392a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:05.750046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfbbf2f-e4be-4e56-bfa2-dba6a4f2392a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:05.762282 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2040/4066] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:15:05.767312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f7548-7098-487a-ad40-30186b1c6b9a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:05.778462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f7548-7098-487a-ad40-30186b1c6b9a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:05.783890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ccee2-efdf-4d4e-97e1-294d7cae071b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:05.784745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ccee2-efdf-4d4e-97e1-294d7cae071b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:05.785036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ccee2-efdf-4d4e-97e1-294d7cae071b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:05.785318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ccee2-efdf-4d4e-97e1-294d7cae071b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:05.786131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ccee2-efdf-4d4e-97e1-294d7cae071b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:05.788515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ccee2-efdf-4d4e-97e1-294d7cae071b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:05.800657 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2027/4067] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:05.805875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbbf2f-e4be-4e56-bfa2-dba6a4f2392a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:05.819063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfbbf2f-e4be-4e56-bfa2-dba6a4f2392a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:05.822184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfaedad-9df1-42d1-887d-d20e27df3c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:05.823075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaedad-9df1-42d1-887d-d20e27df3c24 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:05.823362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaedad-9df1-42d1-887d-d20e27df3c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:05.823644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfaedad-9df1-42d1-887d-d20e27df3c24 admin admin] PATH_INFO: `/v3/projects/1f856c8f99b94657a546fdeb116bb9dd/users/037acb335663496a9edbac13bd0519d7/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:05.838609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfaedad-9df1-42d1-887d-d20e27df3c24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f856c8f99b94657a546fdeb116bb9dd, user_id=037acb335663496a9edbac13bd0519d7, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:05.841653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfaedad-9df1-42d1-887d-d20e27df3c24 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:05.939045 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2041/4068] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:05 2022] PUT /identity/v3/projects/1f856c8f99b94657a546fdeb116bb9dd/users/037acb335663496a9edbac13bd0519d7/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:05.949403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717e171-ff37-43cb-878b-1ecd749645dd None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:05.949696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717e171-ff37-43cb-878b-1ecd749645dd None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:05.950037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717e171-ff37-43cb-878b-1ecd749645dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:05.978770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9717e171-ff37-43cb-878b-1ecd749645dd None None] MFA Rules not processed for user `037acb335663496a9edbac13bd0519d7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:05.980009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9717e171-ff37-43cb-878b-1ecd749645dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:06.047009 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2028/4069] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:06.051591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfaedad-9df1-42d1-887d-d20e27df3c24 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:06.115829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfaedad-9df1-42d1-887d-d20e27df3c24 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:06.123289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c14242-d6d2-4988-b096-e051c3861410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:06.126060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c14242-d6d2-4988-b096-e051c3861410 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:06.126060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c14242-d6d2-4988-b096-e051c3861410 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:06.126060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c14242-d6d2-4988-b096-e051c3861410 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:06.126060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c14242-d6d2-4988-b096-e051c3861410 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:06.127684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c14242-d6d2-4988-b096-e051c3861410 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:06.127969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:06.206038 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2042/4070] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:06 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:06.207483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717e171-ff37-43cb-878b-1ecd749645dd None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:06.219025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717e171-ff37-43cb-878b-1ecd749645dd None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:06.254711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:06.280198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bff227-86b1-4560-b375-052036d76bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:06.280886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bff227-86b1-4560-b375-052036d76bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:06.281170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bff227-86b1-4560-b375-052036d76bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:06.281447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bff227-86b1-4560-b375-052036d76bf2 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:06.282205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bff227-86b1-4560-b375-052036d76bf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:06.284182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bff227-86b1-4560-b375-052036d76bf2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:06.351235 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2029/4071] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:06 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:06.355491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c14242-d6d2-4988-b096-e051c3861410 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:06.362076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:06.366648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c14242-d6d2-4988-b096-e051c3861410 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:06.371465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f70a11-df28-4d2a-99f7-6968538ff133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:06.372126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f70a11-df28-4d2a-99f7-6968538ff133 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:06.372413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f70a11-df28-4d2a-99f7-6968538ff133 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:06.372695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f70a11-df28-4d2a-99f7-6968538ff133 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:06.373427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f70a11-df28-4d2a-99f7-6968538ff133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:06.375058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f70a11-df28-4d2a-99f7-6968538ff133 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:06.385907 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2043/4072] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:06.389824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bff227-86b1-4560-b375-052036d76bf2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:06.401119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bff227-86b1-4560-b375-052036d76bf2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:06.405995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5d666-cadd-43bf-ad80-dbfd31e08bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:06.406657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5d666-cadd-43bf-ad80-dbfd31e08bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:06.406974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5d666-cadd-43bf-ad80-dbfd31e08bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:06.407252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5d666-cadd-43bf-ad80-dbfd31e08bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:06.407985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5d666-cadd-43bf-ad80-dbfd31e08bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:06.409585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5d666-cadd-43bf-ad80-dbfd31e08bf1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:06.421324 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2030/4073] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:06.433530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f70a11-df28-4d2a-99f7-6968538ff133 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:06.444402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f70a11-df28-4d2a-99f7-6968538ff133 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:06.449138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2da21-7a1c-4cfc-93c9-26a69221c7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:06.450060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2da21-7a1c-4cfc-93c9-26a69221c7e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:06.450364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2da21-7a1c-4cfc-93c9-26a69221c7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:06.450644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2da21-7a1c-4cfc-93c9-26a69221c7e3 admin admin] PATH_INFO: `/v3/projects/9add546fdfbb41d2987194ebfe04066a/users/15dfc8bbf27043878f9243b4995653a8/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:06.467034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2da21-7a1c-4cfc-93c9-26a69221c7e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9add546fdfbb41d2987194ebfe04066a, user_id=15dfc8bbf27043878f9243b4995653a8, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:06.469753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2da21-7a1c-4cfc-93c9-26a69221c7e3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:06.590132 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2044/4074] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:06 2022] PUT /identity/v3/projects/9add546fdfbb41d2987194ebfe04066a/users/15dfc8bbf27043878f9243b4995653a8/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:06.600539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093f5d5-b658-49ad-a59b-99d5939bae6d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:06.600773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093f5d5-b658-49ad-a59b-99d5939bae6d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:06.601059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0093f5d5-b658-49ad-a59b-99d5939bae6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:06.606764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:06.627275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0093f5d5-b658-49ad-a59b-99d5939bae6d None None] MFA Rules not processed for user `15dfc8bbf27043878f9243b4995653a8`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:06.628513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0093f5d5-b658-49ad-a59b-99d5939bae6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:06.648953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:06.695368 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2031/4075] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:06.711200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2da21-7a1c-4cfc-93c9-26a69221c7e3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:06.712962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-67d2da21-7a1c-4cfc-93c9-26a69221c7e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:06.751573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:06.767261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:06.777329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2da21-7a1c-4cfc-93c9-26a69221c7e3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:06.781065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5ac8f-f804-49d2-b02d-88f2ee950acb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:06.781715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5ac8f-f804-49d2-b02d-88f2ee950acb service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:06.782150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5ac8f-f804-49d2-b02d-88f2ee950acb service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:06.782434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5ac8f-f804-49d2-b02d-88f2ee950acb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:06.839350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5ac8f-f804-49d2-b02d-88f2ee950acb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:06.840846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d5ac8f-f804-49d2-b02d-88f2ee950acb service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:06.849993 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2045/4076] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:15:07.128710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:07.154268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36657e8-ddc5-4e53-9f46-a0b05ad2f5c4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:07.154564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36657e8-ddc5-4e53-9f46-a0b05ad2f5c4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:07.156301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36657e8-ddc5-4e53-9f46-a0b05ad2f5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:07.174258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a36657e8-ddc5-4e53-9f46-a0b05ad2f5c4 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:07.179366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a36657e8-ddc5-4e53-9f46-a0b05ad2f5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:07.231570 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2032/4077] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:15:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:07.254041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5ac8f-f804-49d2-b02d-88f2ee950acb service glance] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:07.255582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-53d5ac8f-f804-49d2-b02d-88f2ee950acb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:07.329548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d5ac8f-f804-49d2-b02d-88f2ee950acb service glance] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:07.335432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916378b-6401-419a-b85c-9078767d7283 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:07.336093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916378b-6401-419a-b85c-9078767d7283 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:07.336409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916378b-6401-419a-b85c-9078767d7283 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:07.336689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916378b-6401-419a-b85c-9078767d7283 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:07.346137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916378b-6401-419a-b85c-9078767d7283 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:07.347848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916378b-6401-419a-b85c-9078767d7283 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:07.349796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:07.357776 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2046/4078] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:15:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:15:07.401513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:07.682437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:07.790984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:07.989435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36657e8-ddc5-4e53-9f46-a0b05ad2f5c4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:07.991252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a36657e8-ddc5-4e53-9f46-a0b05ad2f5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:08.005110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36657e8-ddc5-4e53-9f46-a0b05ad2f5c4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:08.011489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996e18b-bba2-4ae1-add3-48e1f3898ccb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:08.011998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996e18b-bba2-4ae1-add3-48e1f3898ccb service glance] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:08.012276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996e18b-bba2-4ae1-add3-48e1f3898ccb service glance] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:08.012564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996e18b-bba2-4ae1-add3-48e1f3898ccb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:08.032382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996e18b-bba2-4ae1-add3-48e1f3898ccb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:08.034116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996e18b-bba2-4ae1-add3-48e1f3898ccb service glance] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:08.046770 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2033/4079] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:08.129950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e4004-3292-4021-9832-aea41be971ae None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:08.130414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e4004-3292-4021-9832-aea41be971ae None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:08.130532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e4004-3292-4021-9832-aea41be971ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:08.150604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-820e4004-3292-4021-9832-aea41be971ae None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:08.152006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-820e4004-3292-4021-9832-aea41be971ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:08.161532 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2047/4080] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:15:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:08.173582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996e18b-bba2-4ae1-add3-48e1f3898ccb service glance] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:08.186434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996e18b-bba2-4ae1-add3-48e1f3898ccb service glance] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:08.191679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac332d-9ef0-4b61-97cd-01fb4209999c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:08.192390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac332d-9ef0-4b61-97cd-01fb4209999c service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:08.192741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac332d-9ef0-4b61-97cd-01fb4209999c service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:08.193083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac332d-9ef0-4b61-97cd-01fb4209999c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:08.204344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac332d-9ef0-4b61-97cd-01fb4209999c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:08.204344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac332d-9ef0-4b61-97cd-01fb4209999c service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:08.213110 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2034/4081] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:15:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:15:08.227292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:08.272396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e4004-3292-4021-9832-aea41be971ae None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:08.274675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-820e4004-3292-4021-9832-aea41be971ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:08.353467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e4004-3292-4021-9832-aea41be971ae None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:08.359479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74911dce-13c2-4904-9f56-4704b10b2e75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:08.360134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74911dce-13c2-4904-9f56-4704b10b2e75 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:08.360417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74911dce-13c2-4904-9f56-4704b10b2e75 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:08.360695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74911dce-13c2-4904-9f56-4704b10b2e75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:08.362485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-74911dce-13c2-4904-9f56-4704b10b2e75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:08.375589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74911dce-13c2-4904-9f56-4704b10b2e75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:08.377201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74911dce-13c2-4904-9f56-4704b10b2e75 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:08.389767 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2048/4082] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:08.476233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac332d-9ef0-4b61-97cd-01fb4209999c service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:08.477816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5aac332d-9ef0-4b61-97cd-01fb4209999c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:08.516600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:08.549537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac332d-9ef0-4b61-97cd-01fb4209999c service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:08.554178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e447b-0e3e-458c-8e82-5bae1a84d01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:08.555583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e447b-0e3e-458c-8e82-5bae1a84d01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:08.555872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e447b-0e3e-458c-8e82-5bae1a84d01a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:08.556149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e447b-0e3e-458c-8e82-5bae1a84d01a admin admin] PATH_INFO: `/v3/users/bf9c8f66e7d24ca9baca835e41546f22` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:08.557365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e447b-0e3e-458c-8e82-5bae1a84d01a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf9c8f66e7d24ca9baca835e41546f22)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:08.559364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e447b-0e3e-458c-8e82-5bae1a84d01a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:08.759413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3f0e447b-0e3e-458c-8e82-5bae1a84d01a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf9c8f66e7d24ca9baca835e41546f22'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:08.763286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3f0e447b-0e3e-458c-8e82-5bae1a84d01a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf9c8f66e7d24ca9baca835e41546f22'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:08.764788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74911dce-13c2-4904-9f56-4704b10b2e75 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:08.770251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3f0e447b-0e3e-458c-8e82-5bae1a84d01a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf9c8f66e7d24ca9baca835e41546f22'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:08.778131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74911dce-13c2-4904-9f56-4704b10b2e75 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:08.789528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022e5f2-9b2e-4cb8-b393-43f18388503e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:08.790540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022e5f2-9b2e-4cb8-b393-43f18388503e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:08.791044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022e5f2-9b2e-4cb8-b393-43f18388503e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:08.791502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022e5f2-9b2e-4cb8-b393-43f18388503e admin admin] PATH_INFO: `/v3/users/037acb335663496a9edbac13bd0519d7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:08.792789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022e5f2-9b2e-4cb8-b393-43f18388503e admin admin] RBAC: Authorizing `identity:delete_user(user_id=037acb335663496a9edbac13bd0519d7)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:08.795183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022e5f2-9b2e-4cb8-b393-43f18388503e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:08.899171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3f0e447b-0e3e-458c-8e82-5bae1a84d01a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf9c8f66e7d24ca9baca835e41546f22'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:08.900051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3f0e447b-0e3e-458c-8e82-5bae1a84d01a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf9c8f66e7d24ca9baca835e41546f22'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:08.918030 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2035/4083] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:15:08 2022] DELETE /identity/v3/users/bf9c8f66e7d24ca9baca835e41546f22 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:09.040540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2022e5f2-9b2e-4cb8-b393-43f18388503e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '037acb335663496a9edbac13bd0519d7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:09.041922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:09.049263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2022e5f2-9b2e-4cb8-b393-43f18388503e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '037acb335663496a9edbac13bd0519d7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:09.050483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2022e5f2-9b2e-4cb8-b393-43f18388503e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '037acb335663496a9edbac13bd0519d7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:09.051200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2022e5f2-9b2e-4cb8-b393-43f18388503e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '037acb335663496a9edbac13bd0519d7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:09.123095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2022e5f2-9b2e-4cb8-b393-43f18388503e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '037acb335663496a9edbac13bd0519d7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:09.141438 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2049/4084] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:15:08 2022] DELETE /identity/v3/users/037acb335663496a9edbac13bd0519d7 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:09.147544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:09.156179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a6def-7006-454e-91c8-9fda7695e2e3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:09.156428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022e5f2-9b2e-4cb8-b393-43f18388503e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:09.156583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a6def-7006-454e-91c8-9fda7695e2e3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:09.156815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a6def-7006-454e-91c8-9fda7695e2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:09.158385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2022e5f2-9b2e-4cb8-b393-43f18388503e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:09.176280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bf8a6def-7006-454e-91c8-9fda7695e2e3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:09.177693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bf8a6def-7006-454e-91c8-9fda7695e2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:09.235515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022e5f2-9b2e-4cb8-b393-43f18388503e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:09.240622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5413b84-e6de-4fd2-a9a6-e7f9665c5be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:09.241568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5413b84-e6de-4fd2-a9a6-e7f9665c5be0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:09.241891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5413b84-e6de-4fd2-a9a6-e7f9665c5be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:09.242588 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2036/4085] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:09.242756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5413b84-e6de-4fd2-a9a6-e7f9665c5be0 admin admin] PATH_INFO: `/v3/projects/cf04fd760c54473f83980a385252fa3c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:09.244120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5413b84-e6de-4fd2-a9a6-e7f9665c5be0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf04fd760c54473f83980a385252fa3c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:09.246283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5413b84-e6de-4fd2-a9a6-e7f9665c5be0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:09.253191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a6def-7006-454e-91c8-9fda7695e2e3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:09.254884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bf8a6def-7006-454e-91c8-9fda7695e2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:09.316974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a6def-7006-454e-91c8-9fda7695e2e3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:09.321367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b71f4-0501-4b81-98da-b007281bb0a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:09.322077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b71f4-0501-4b81-98da-b007281bb0a1 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:09.322364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b71f4-0501-4b81-98da-b007281bb0a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:09.325766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b71f4-0501-4b81-98da-b007281bb0a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:09.332172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b71f4-0501-4b81-98da-b007281bb0a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:09.333619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b71f4-0501-4b81-98da-b007281bb0a1 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:09.343978 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2037/4086] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:15:09.386362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5413b84-e6de-4fd2-a9a6-e7f9665c5be0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf04fd760c54473f83980a385252fa3c', 'request_id': 'req-a5413b84-e6de-4fd2-a9a6-e7f9665c5be0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:09.393739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5413b84-e6de-4fd2-a9a6-e7f9665c5be0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf04fd760c54473f83980a385252fa3c', 'request_id': 'req-a5413b84-e6de-4fd2-a9a6-e7f9665c5be0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:09.494011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5413b84-e6de-4fd2-a9a6-e7f9665c5be0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf04fd760c54473f83980a385252fa3c', 'request_id': 'req-a5413b84-e6de-4fd2-a9a6-e7f9665c5be0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:09.506749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5413b84-e6de-4fd2-a9a6-e7f9665c5be0 admin admin] The token cache is being invalidate because project cf04fd760c54473f83980a385252fa3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:15:09.507168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5413b84-e6de-4fd2-a9a6-e7f9665c5be0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:09.507970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5413b84-e6de-4fd2-a9a6-e7f9665c5be0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:09.510085 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2050/4087] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:09 2022] DELETE /identity/v3/projects/cf04fd760c54473f83980a385252fa3c => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:15:09.515112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edac917-5fd0-437e-beb9-1135f3266818 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:09.515571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edac917-5fd0-437e-beb9-1135f3266818 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:09.516196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edac917-5fd0-437e-beb9-1135f3266818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:09.537263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8edac917-5fd0-437e-beb9-1135f3266818 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:09.538651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8edac917-5fd0-437e-beb9-1135f3266818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:09.604645 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2038/4088] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:09.613482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb047082-c191-4ec4-b32b-e725381629c8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:09.613830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb047082-c191-4ec4-b32b-e725381629c8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:09.614163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb047082-c191-4ec4-b32b-e725381629c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:09.635457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-bb047082-c191-4ec4-b32b-e725381629c8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:09.636695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bb047082-c191-4ec4-b32b-e725381629c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:09.645047 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2051/4089] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:09.649286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edac917-5fd0-437e-beb9-1135f3266818 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:09.658910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edac917-5fd0-437e-beb9-1135f3266818 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:09.663698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3c544-9fd2-4751-b616-ebc0563c23a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:09.664904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3c544-9fd2-4751-b616-ebc0563c23a8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:09.668629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3c544-9fd2-4751-b616-ebc0563c23a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:09.668629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3c544-9fd2-4751-b616-ebc0563c23a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:09.668629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3c544-9fd2-4751-b616-ebc0563c23a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:15:09.668629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3c544-9fd2-4751-b616-ebc0563c23a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:09.668629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3c544-9fd2-4751-b616-ebc0563c23a8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:09.683947 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2039/4090] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:09.685341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb047082-c191-4ec4-b32b-e725381629c8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:09.695656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb047082-c191-4ec4-b32b-e725381629c8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:09.700426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebf242-d67a-4606-9eee-6f3e5939ad99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:09.701084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebf242-d67a-4606-9eee-6f3e5939ad99 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:09.701393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebf242-d67a-4606-9eee-6f3e5939ad99 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:09.701671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebf242-d67a-4606-9eee-6f3e5939ad99 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:09.702416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebf242-d67a-4606-9eee-6f3e5939ad99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:09.704332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebf242-d67a-4606-9eee-6f3e5939ad99 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:09.861637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3c544-9fd2-4751-b616-ebc0563c23a8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:09.863409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2ac3c544-9fd2-4751-b616-ebc0563c23a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:09.879078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3c544-9fd2-4751-b616-ebc0563c23a8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:09.890154 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2052/4091] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:09 2022] POST /identity/v3/projects => generated 379 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:09.891312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebf242-d67a-4606-9eee-6f3e5939ad99 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:09.937818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:09.970422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb19b6-caca-474f-89aa-68d181795cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:09.971598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb19b6-caca-474f-89aa-68d181795cbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:09.972175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb19b6-caca-474f-89aa-68d181795cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:09.978046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb19b6-caca-474f-89aa-68d181795cbb admin admin] PATH_INFO: `/v3/projects/1f856c8f99b94657a546fdeb116bb9dd` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:09.978046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb19b6-caca-474f-89aa-68d181795cbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f856c8f99b94657a546fdeb116bb9dd)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:09.978046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb19b6-caca-474f-89aa-68d181795cbb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:09.978416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebf242-d67a-4606-9eee-6f3e5939ad99 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:09.983391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfb181-4d90-4335-a0ec-a623c4aea66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:09.984440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfb181-4d90-4335-a0ec-a623c4aea66e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:09.984699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfb181-4d90-4335-a0ec-a623c4aea66e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:09.984983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bfb181-4d90-4335-a0ec-a623c4aea66e admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:09.985770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bfb181-4d90-4335-a0ec-a623c4aea66e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:09.987923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bfb181-4d90-4335-a0ec-a623c4aea66e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:10.102805 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2053/4092] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:09 2022] POST /identity/v3/users => generated 290 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:15:10.103907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfb181-4d90-4335-a0ec-a623c4aea66e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:10.115579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bfb181-4d90-4335-a0ec-a623c4aea66e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:10.120927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b26ce-b439-4c70-bd59-706fa6013033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:10.121606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b26ce-b439-4c70-bd59-706fa6013033 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:10.121937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b26ce-b439-4c70-bd59-706fa6013033 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:10.122254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7b26ce-b439-4c70-bd59-706fa6013033 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:10.122951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b26ce-b439-4c70-bd59-706fa6013033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:10.124564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7b26ce-b439-4c70-bd59-706fa6013033 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:10.135553 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2054/4093] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:10.154625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b26ce-b439-4c70-bd59-706fa6013033 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:10.160907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-06fb19b6-caca-474f-89aa-68d181795cbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f856c8f99b94657a546fdeb116bb9dd', 'request_id': 'req-06fb19b6-caca-474f-89aa-68d181795cbb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:10.166108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7b26ce-b439-4c70-bd59-706fa6013033 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:10.168546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-06fb19b6-caca-474f-89aa-68d181795cbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f856c8f99b94657a546fdeb116bb9dd', 'request_id': 'req-06fb19b6-caca-474f-89aa-68d181795cbb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:10.229551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fa343-0336-4590-b3c4-7b8e9001f436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:10.230363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fa343-0336-4590-b3c4-7b8e9001f436 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:10.230670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fa343-0336-4590-b3c4-7b8e9001f436 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:10.230948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82fa343-0336-4590-b3c4-7b8e9001f436 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:10.231730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82fa343-0336-4590-b3c4-7b8e9001f436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:10.233309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82fa343-0336-4590-b3c4-7b8e9001f436 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:10.244477 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2055/4094] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:10.249056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fa343-0336-4590-b3c4-7b8e9001f436 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:10.259300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-06fb19b6-caca-474f-89aa-68d181795cbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f856c8f99b94657a546fdeb116bb9dd', 'request_id': 'req-06fb19b6-caca-474f-89aa-68d181795cbb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:10.261785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82fa343-0336-4590-b3c4-7b8e9001f436 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:10.266735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70cd39-bc6d-48b4-aa24-9f3df058e681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:10.267746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70cd39-bc6d-48b4-aa24-9f3df058e681 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:10.268046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70cd39-bc6d-48b4-aa24-9f3df058e681 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:10.268345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70cd39-bc6d-48b4-aa24-9f3df058e681 admin admin] PATH_INFO: `/v3/projects/6846f7be3c13484e92e2bfbe9df3810f/users/c9d2988731d14049b27283cda69bec93/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:10.291390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70cd39-bc6d-48b4-aa24-9f3df058e681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6846f7be3c13484e92e2bfbe9df3810f, user_id=c9d2988731d14049b27283cda69bec93, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:10.292531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-06fb19b6-caca-474f-89aa-68d181795cbb admin admin] The token cache is being invalidate because project 1f856c8f99b94657a546fdeb116bb9dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:15:10.293007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-06fb19b6-caca-474f-89aa-68d181795cbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:10.293678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-06fb19b6-caca-474f-89aa-68d181795cbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:10.294481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70cd39-bc6d-48b4-aa24-9f3df058e681 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:10.295359 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2040/4095] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:09 2022] DELETE /identity/v3/projects/1f856c8f99b94657a546fdeb116bb9dd => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:15:10.299451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb19b6-caca-474f-89aa-68d181795cbb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:10.300828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-06fb19b6-caca-474f-89aa-68d181795cbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:10.314166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb19b6-caca-474f-89aa-68d181795cbb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:10.319182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b664acc4-e8ed-4e8c-aff0-548bcb31fef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:10.320044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664acc4-e8ed-4e8c-aff0-548bcb31fef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:10.320943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664acc4-e8ed-4e8c-aff0-548bcb31fef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:10.321283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b664acc4-e8ed-4e8c-aff0-548bcb31fef9 admin admin] PATH_INFO: `/v3/users/15dfc8bbf27043878f9243b4995653a8` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:10.322654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664acc4-e8ed-4e8c-aff0-548bcb31fef9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15dfc8bbf27043878f9243b4995653a8)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:10.324593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b664acc4-e8ed-4e8c-aff0-548bcb31fef9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:10.368011 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2056/4096] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:10 2022] PUT /identity/v3/projects/6846f7be3c13484e92e2bfbe9df3810f/users/c9d2988731d14049b27283cda69bec93/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:10.378245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c173a-2468-4fbf-913d-0e245e6da16b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:10.378543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c173a-2468-4fbf-913d-0e245e6da16b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:10.378825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c173a-2468-4fbf-913d-0e245e6da16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:10.413389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1e2c173a-2468-4fbf-913d-0e245e6da16b None None] MFA Rules not processed for user `c9d2988731d14049b27283cda69bec93`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:10.414769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1e2c173a-2468-4fbf-913d-0e245e6da16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:10.511735 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2057/4097] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:10.515473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803af39-2e3c-44cd-ae82-6139ccff2511 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:10.515775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803af39-2e3c-44cd-ae82-6139ccff2511 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:10.516060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803af39-2e3c-44cd-ae82-6139ccff2511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:10.539025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b664acc4-e8ed-4e8c-aff0-548bcb31fef9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15dfc8bbf27043878f9243b4995653a8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:10.540107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b664acc4-e8ed-4e8c-aff0-548bcb31fef9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15dfc8bbf27043878f9243b4995653a8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:10.545140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d803af39-2e3c-44cd-ae82-6139ccff2511 None None] MFA Rules not processed for user `c9d2988731d14049b27283cda69bec93`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:10.547127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d803af39-2e3c-44cd-ae82-6139ccff2511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:10.552202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b664acc4-e8ed-4e8c-aff0-548bcb31fef9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15dfc8bbf27043878f9243b4995653a8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:10.555892 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2058/4098] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:15:10.578814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803af39-2e3c-44cd-ae82-6139ccff2511 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:10.580361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d803af39-2e3c-44cd-ae82-6139ccff2511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:10.625122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b664acc4-e8ed-4e8c-aff0-548bcb31fef9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15dfc8bbf27043878f9243b4995653a8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:10.625797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b664acc4-e8ed-4e8c-aff0-548bcb31fef9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15dfc8bbf27043878f9243b4995653a8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:10.649446 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2041/4099] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:15:10 2022] DELETE /identity/v3/users/15dfc8bbf27043878f9243b4995653a8 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:15:10.681484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d803af39-2e3c-44cd-ae82-6139ccff2511 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:10.737600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bbd13-dc14-4cbd-a3db-ed4dd5537b31 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:10.738443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bbd13-dc14-4cbd-a3db-ed4dd5537b31 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:10.738959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bbd13-dc14-4cbd-a3db-ed4dd5537b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:10.767775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-867bbd13-dc14-4cbd-a3db-ed4dd5537b31 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:10.769983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-867bbd13-dc14-4cbd-a3db-ed4dd5537b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:10.810429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:10.820995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3fc71-c07a-46fb-af08-aac186495953 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:10.821677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3fc71-c07a-46fb-af08-aac186495953 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:10.822124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3fc71-c07a-46fb-af08-aac186495953 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:10.822370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3fc71-c07a-46fb-af08-aac186495953 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:10.826556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5ff3fc71-c07a-46fb-af08-aac186495953 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:10.890994 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2042/4100] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:15:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:15:10.906782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867bbd13-dc14-4cbd-a3db-ed4dd5537b31 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:10.908998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-867bbd13-dc14-4cbd-a3db-ed4dd5537b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:10.923292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff3fc71-c07a-46fb-af08-aac186495953 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:10.925589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff3fc71-c07a-46fb-af08-aac186495953 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:10.958650 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2059/4101] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:15:11.048551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867bbd13-dc14-4cbd-a3db-ed4dd5537b31 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:11.055074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96181a4f-bb96-4346-a584-7a35506ca9f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:11.056115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96181a4f-bb96-4346-a584-7a35506ca9f2 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:11.056407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96181a4f-bb96-4346-a584-7a35506ca9f2 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:11.057347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96181a4f-bb96-4346-a584-7a35506ca9f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:11.090251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96181a4f-bb96-4346-a584-7a35506ca9f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:11.091473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96181a4f-bb96-4346-a584-7a35506ca9f2 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:11.109591 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2043/4102] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:15:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 21:15:11.267437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3fc71-c07a-46fb-af08-aac186495953 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:11.269633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5ff3fc71-c07a-46fb-af08-aac186495953 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:11.481650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3fc71-c07a-46fb-af08-aac186495953 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:11.490382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff5754-43c8-4d76-965e-9bcc49aa8396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:11.491535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff5754-43c8-4d76-965e-9bcc49aa8396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:11.493185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff5754-43c8-4d76-965e-9bcc49aa8396 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:11.493629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff5754-43c8-4d76-965e-9bcc49aa8396 admin admin] PATH_INFO: `/v3/projects/9add546fdfbb41d2987194ebfe04066a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:11.495100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff5754-43c8-4d76-965e-9bcc49aa8396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9add546fdfbb41d2987194ebfe04066a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:11.497620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff5754-43c8-4d76-965e-9bcc49aa8396 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:11.580876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:11.654672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9eff5754-43c8-4d76-965e-9bcc49aa8396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9add546fdfbb41d2987194ebfe04066a', 'request_id': 'req-9eff5754-43c8-4d76-965e-9bcc49aa8396'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:11.663147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9eff5754-43c8-4d76-965e-9bcc49aa8396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9add546fdfbb41d2987194ebfe04066a', 'request_id': 'req-9eff5754-43c8-4d76-965e-9bcc49aa8396'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:11.752975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:11.778582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9eff5754-43c8-4d76-965e-9bcc49aa8396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9add546fdfbb41d2987194ebfe04066a', 'request_id': 'req-9eff5754-43c8-4d76-965e-9bcc49aa8396'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:11.791576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9eff5754-43c8-4d76-965e-9bcc49aa8396 admin admin] The token cache is being invalidate because project 9add546fdfbb41d2987194ebfe04066a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:15:11.792022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9eff5754-43c8-4d76-965e-9bcc49aa8396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:11.792769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9eff5754-43c8-4d76-965e-9bcc49aa8396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:11.794491 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2060/4103] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:11 2022] DELETE /identity/v3/projects/9add546fdfbb41d2987194ebfe04066a => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:11.800622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebaeefd-f1bc-46d1-81bd-e98d31c20390 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:11.800941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebaeefd-f1bc-46d1-81bd-e98d31c20390 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:11.801224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebaeefd-f1bc-46d1-81bd-e98d31c20390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:11.826302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bebaeefd-f1bc-46d1-81bd-e98d31c20390 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:11.828150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bebaeefd-f1bc-46d1-81bd-e98d31c20390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:11.904536 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2044/4104] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:11.914495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a896d-e5c2-498a-ab75-8869898416fb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:11.914723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a896d-e5c2-498a-ab75-8869898416fb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:11.915012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a896d-e5c2-498a-ab75-8869898416fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:11.946849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d52a896d-e5c2-498a-ab75-8869898416fb None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:11.948859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d52a896d-e5c2-498a-ab75-8869898416fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:11.957492 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2061/4105] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:11.962880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebaeefd-f1bc-46d1-81bd-e98d31c20390 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:11.973206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebaeefd-f1bc-46d1-81bd-e98d31c20390 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:11.985402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b61fb-a231-460d-bd85-e578d681c4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:11.985402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b61fb-a231-460d-bd85-e578d681c4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:11.985402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b61fb-a231-460d-bd85-e578d681c4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:11.985402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b61fb-a231-460d-bd85-e578d681c4b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:11.985947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b61fb-a231-460d-bd85-e578d681c4b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:15:11.985947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b61fb-a231-460d-bd85-e578d681c4b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:11.986684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b61fb-a231-460d-bd85-e578d681c4b9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:12.005878 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2045/4106] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:15:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:12.008958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a896d-e5c2-498a-ab75-8869898416fb None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:12.024923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a896d-e5c2-498a-ab75-8869898416fb None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:12.032497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278249c-24b1-4309-9aa1-90241e690194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:12.033280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278249c-24b1-4309-9aa1-90241e690194 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:12.033669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278249c-24b1-4309-9aa1-90241e690194 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:12.034104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278249c-24b1-4309-9aa1-90241e690194 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:12.035551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278249c-24b1-4309-9aa1-90241e690194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:12.038679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278249c-24b1-4309-9aa1-90241e690194 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:12.136659 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2062/4107] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:12 2022] POST /identity/v3/projects => generated 371 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:12.138151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b61fb-a231-460d-bd85-e578d681c4b9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:12.149627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b61fb-a231-460d-bd85-e578d681c4b9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:12.223446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1636c4c-ddc1-463a-ac90-081249a64ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:12.224425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1636c4c-ddc1-463a-ac90-081249a64ffc admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:12.224875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1636c4c-ddc1-463a-ac90-081249a64ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:12.225148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1636c4c-ddc1-463a-ac90-081249a64ffc admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:12.225868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1636c4c-ddc1-463a-ac90-081249a64ffc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:12.228596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1636c4c-ddc1-463a-ac90-081249a64ffc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:12.315655 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2046/4108] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:12 2022] POST /identity/v3/users => generated 286 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:12.316935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278249c-24b1-4309-9aa1-90241e690194 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:12.333372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278249c-24b1-4309-9aa1-90241e690194 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:12.337119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:12.343935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414d108-1e96-460f-bf3d-a3a83e3dbcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:12.343935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414d108-1e96-460f-bf3d-a3a83e3dbcdf admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:12.344561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414d108-1e96-460f-bf3d-a3a83e3dbcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:12.345208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f414d108-1e96-460f-bf3d-a3a83e3dbcdf admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:12.346288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414d108-1e96-460f-bf3d-a3a83e3dbcdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:12.348420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f414d108-1e96-460f-bf3d-a3a83e3dbcdf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:12.390720 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2063/4109] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:12.398981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1636c4c-ddc1-463a-ac90-081249a64ffc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:12.407225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1636c4c-ddc1-463a-ac90-081249a64ffc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:12.420989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849285c4-4b58-476d-acfc-d2d1c2be0722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:12.421673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849285c4-4b58-476d-acfc-d2d1c2be0722 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:12.422208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849285c4-4b58-476d-acfc-d2d1c2be0722 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:12.427296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849285c4-4b58-476d-acfc-d2d1c2be0722 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:12.427296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849285c4-4b58-476d-acfc-d2d1c2be0722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:12.427296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849285c4-4b58-476d-acfc-d2d1c2be0722 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:12.468901 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2047/4110] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:12.475060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414d108-1e96-460f-bf3d-a3a83e3dbcdf admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:12.488105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f414d108-1e96-460f-bf3d-a3a83e3dbcdf admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:12.495188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95335de-d73e-47fa-81c3-ee7ff3dcc186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:12.496592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95335de-d73e-47fa-81c3-ee7ff3dcc186 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:12.497055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95335de-d73e-47fa-81c3-ee7ff3dcc186 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:12.497505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95335de-d73e-47fa-81c3-ee7ff3dcc186 admin admin] PATH_INFO: `/v3/projects/1d9399d1b6a542049d7dc3a5c53dc705/users/f76143b0eeac475eaa67b5baa1a282df/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:12.522064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95335de-d73e-47fa-81c3-ee7ff3dcc186 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9399d1b6a542049d7dc3a5c53dc705, user_id=f76143b0eeac475eaa67b5baa1a282df, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:12.526792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95335de-d73e-47fa-81c3-ee7ff3dcc186 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:12.567423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:12.647933 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2064/4111] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:12 2022] PUT /identity/v3/projects/1d9399d1b6a542049d7dc3a5c53dc705/users/f76143b0eeac475eaa67b5baa1a282df/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:12.659691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abf9a0-feeb-406f-8b2a-a79d6290fa36 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:12.661314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abf9a0-feeb-406f-8b2a-a79d6290fa36 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:12.661314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abf9a0-feeb-406f-8b2a-a79d6290fa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:12.703704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-73abf9a0-feeb-406f-8b2a-a79d6290fa36 None None] MFA Rules not processed for user `f76143b0eeac475eaa67b5baa1a282df`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:12.705195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-73abf9a0-feeb-406f-8b2a-a79d6290fa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:12.805434 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2048/4112] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:12.815390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95335de-d73e-47fa-81c3-ee7ff3dcc186 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:12.816719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e95335de-d73e-47fa-81c3-ee7ff3dcc186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:12.914857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95335de-d73e-47fa-81c3-ee7ff3dcc186 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:12.918581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f36d2-8677-4a77-9c85-b3c5cac4cb4b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:12.919272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f36d2-8677-4a77-9c85-b3c5cac4cb4b service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:12.920094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f36d2-8677-4a77-9c85-b3c5cac4cb4b service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:12.920094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f36d2-8677-4a77-9c85-b3c5cac4cb4b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:12.930395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f36d2-8677-4a77-9c85-b3c5cac4cb4b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:12.931905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f36d2-8677-4a77-9c85-b3c5cac4cb4b service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:12.943604 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2065/4113] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:14.242430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:14.385023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:14.813407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:15.523545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abf9a0-feeb-406f-8b2a-a79d6290fa36 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:15.607925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abf9a0-feeb-406f-8b2a-a79d6290fa36 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:15.614507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b88b8d-7ae8-4d6f-8b8d-b32c30faf133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:15.615785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b88b8d-7ae8-4d6f-8b8d-b32c30faf133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:15.616398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b88b8d-7ae8-4d6f-8b8d-b32c30faf133 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:15.616868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b88b8d-7ae8-4d6f-8b8d-b32c30faf133 admin admin] PATH_INFO: `/v3/users/f76143b0eeac475eaa67b5baa1a282df` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:15.619130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b88b8d-7ae8-4d6f-8b8d-b32c30faf133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f76143b0eeac475eaa67b5baa1a282df)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:15.621674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b88b8d-7ae8-4d6f-8b8d-b32c30faf133 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:15.844646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-02b88b8d-7ae8-4d6f-8b8d-b32c30faf133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f76143b0eeac475eaa67b5baa1a282df'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:15.845749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-02b88b8d-7ae8-4d6f-8b8d-b32c30faf133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f76143b0eeac475eaa67b5baa1a282df'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:15.855015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-02b88b8d-7ae8-4d6f-8b8d-b32c30faf133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f76143b0eeac475eaa67b5baa1a282df'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:15.948766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-02b88b8d-7ae8-4d6f-8b8d-b32c30faf133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f76143b0eeac475eaa67b5baa1a282df'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:15.949709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-02b88b8d-7ae8-4d6f-8b8d-b32c30faf133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f76143b0eeac475eaa67b5baa1a282df'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:15.992129 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2049/4114] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:15:15 2022] DELETE /identity/v3/users/f76143b0eeac475eaa67b5baa1a282df => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:15.996461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8f159-c464-478c-8a6e-5ca695534286 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:15.996987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8f159-c464-478c-8a6e-5ca695534286 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:15.997074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8f159-c464-478c-8a6e-5ca695534286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:16.006221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:16.020435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-24d8f159-c464-478c-8a6e-5ca695534286 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:16.022985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-24d8f159-c464-478c-8a6e-5ca695534286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:16.167948 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2066/4115] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:16.180467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b88b8d-7ae8-4d6f-8b8d-b32c30faf133 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:16.182469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-02b88b8d-7ae8-4d6f-8b8d-b32c30faf133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:16.310812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b88b8d-7ae8-4d6f-8b8d-b32c30faf133 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:16.315397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72583c-7189-4f60-82cd-0752e8f29d76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:16.316262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72583c-7189-4f60-82cd-0752e8f29d76 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:16.316757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72583c-7189-4f60-82cd-0752e8f29d76 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:16.317203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72583c-7189-4f60-82cd-0752e8f29d76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:16.352732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72583c-7189-4f60-82cd-0752e8f29d76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:16.355519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72583c-7189-4f60-82cd-0752e8f29d76 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:16.362098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:16.371000 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2050/4116] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:17.404984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8f159-c464-478c-8a6e-5ca695534286 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:17.406506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-24d8f159-c464-478c-8a6e-5ca695534286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:17.418990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8f159-c464-478c-8a6e-5ca695534286 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:17.425224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd43a2e-1870-407a-bc6c-cc5d60f7b8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:17.426406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd43a2e-1870-407a-bc6c-cc5d60f7b8e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:17.426828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd43a2e-1870-407a-bc6c-cc5d60f7b8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:17.427194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd43a2e-1870-407a-bc6c-cc5d60f7b8e8 admin admin] PATH_INFO: `/v3/projects/1d9399d1b6a542049d7dc3a5c53dc705` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:17.428564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd43a2e-1870-407a-bc6c-cc5d60f7b8e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d9399d1b6a542049d7dc3a5c53dc705)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:17.431328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd43a2e-1870-407a-bc6c-cc5d60f7b8e8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:17.578949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cdd43a2e-1870-407a-bc6c-cc5d60f7b8e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d9399d1b6a542049d7dc3a5c53dc705', 'request_id': 'req-cdd43a2e-1870-407a-bc6c-cc5d60f7b8e8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:17.589912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cdd43a2e-1870-407a-bc6c-cc5d60f7b8e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d9399d1b6a542049d7dc3a5c53dc705', 'request_id': 'req-cdd43a2e-1870-407a-bc6c-cc5d60f7b8e8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:17.694493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cdd43a2e-1870-407a-bc6c-cc5d60f7b8e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d9399d1b6a542049d7dc3a5c53dc705', 'request_id': 'req-cdd43a2e-1870-407a-bc6c-cc5d60f7b8e8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:17.707863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cdd43a2e-1870-407a-bc6c-cc5d60f7b8e8 admin admin] The token cache is being invalidate because project 1d9399d1b6a542049d7dc3a5c53dc705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:15:17.708591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cdd43a2e-1870-407a-bc6c-cc5d60f7b8e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:17.711562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cdd43a2e-1870-407a-bc6c-cc5d60f7b8e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:17.715076 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2067/4117] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:17 2022] DELETE /identity/v3/projects/1d9399d1b6a542049d7dc3a5c53dc705 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:17.722364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42283980-8398-48ba-af30-1fb51f2db200 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:17.722891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42283980-8398-48ba-af30-1fb51f2db200 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:17.723005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42283980-8398-48ba-af30-1fb51f2db200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:17.742970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-42283980-8398-48ba-af30-1fb51f2db200 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:17.744219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-42283980-8398-48ba-af30-1fb51f2db200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:17.814358 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2051/4118] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:15:17.823704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd1d1f-59bf-4584-8c87-107d9f6c9920 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:17.824080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd1d1f-59bf-4584-8c87-107d9f6c9920 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:17.824380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd1d1f-59bf-4584-8c87-107d9f6c9920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:17.845011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0cdd1d1f-59bf-4584-8c87-107d9f6c9920 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:17.884392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0cdd1d1f-59bf-4584-8c87-107d9f6c9920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:17.884392 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2068/4119] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:17.887862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42283980-8398-48ba-af30-1fb51f2db200 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:17.898838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42283980-8398-48ba-af30-1fb51f2db200 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:17.903653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c79dd0-808f-40bf-9462-8f0d9ac32359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:17.904345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c79dd0-808f-40bf-9462-8f0d9ac32359 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:17.904629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c79dd0-808f-40bf-9462-8f0d9ac32359 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:17.904907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c79dd0-808f-40bf-9462-8f0d9ac32359 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:17.905639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c79dd0-808f-40bf-9462-8f0d9ac32359 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:15:17.905981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c79dd0-808f-40bf-9462-8f0d9ac32359 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:17.907598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c79dd0-808f-40bf-9462-8f0d9ac32359 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:17.924545 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2052/4120] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:17.925787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd1d1f-59bf-4584-8c87-107d9f6c9920 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:17.935934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd1d1f-59bf-4584-8c87-107d9f6c9920 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:17.940901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb9c64-bb6b-43f5-a2b0-051e5668db2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:17.941678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb9c64-bb6b-43f5-a2b0-051e5668db2e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:17.942021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb9c64-bb6b-43f5-a2b0-051e5668db2e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:17.942431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb9c64-bb6b-43f5-a2b0-051e5668db2e admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:17.943150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb9c64-bb6b-43f5-a2b0-051e5668db2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:17.945111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb9c64-bb6b-43f5-a2b0-051e5668db2e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:18.090972 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2069/4121] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:17 2022] POST /identity/v3/projects => generated 407 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:18.095341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c79dd0-808f-40bf-9462-8f0d9ac32359 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:18.107401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c79dd0-808f-40bf-9462-8f0d9ac32359 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:18.184094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a854025-42a3-4228-8ca9-6a41bdfe50e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:18.184805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a854025-42a3-4228-8ca9-6a41bdfe50e1 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:18.185086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a854025-42a3-4228-8ca9-6a41bdfe50e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:18.185364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a854025-42a3-4228-8ca9-6a41bdfe50e1 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:18.186395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a854025-42a3-4228-8ca9-6a41bdfe50e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:18.188396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a854025-42a3-4228-8ca9-6a41bdfe50e1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:18.307282 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2053/4122] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:18 2022] POST /identity/v3/users => generated 304 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:15:18.311222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb9c64-bb6b-43f5-a2b0-051e5668db2e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:18.326576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb9c64-bb6b-43f5-a2b0-051e5668db2e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:18.337117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48fbfe-1234-4dc3-a09e-3d2caa0c13e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:18.338693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48fbfe-1234-4dc3-a09e-3d2caa0c13e6 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:18.339088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48fbfe-1234-4dc3-a09e-3d2caa0c13e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:18.339751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48fbfe-1234-4dc3-a09e-3d2caa0c13e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:18.340770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48fbfe-1234-4dc3-a09e-3d2caa0c13e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:18.342586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48fbfe-1234-4dc3-a09e-3d2caa0c13e6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:18.361016 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2070/4123] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:18.362378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a854025-42a3-4228-8ca9-6a41bdfe50e1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:18.375428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a854025-42a3-4228-8ca9-6a41bdfe50e1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:18.381988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d68fa8-6c51-47db-8bd6-024d16ef322e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:18.383145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d68fa8-6c51-47db-8bd6-024d16ef322e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:18.383837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d68fa8-6c51-47db-8bd6-024d16ef322e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:18.384370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d68fa8-6c51-47db-8bd6-024d16ef322e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:18.386944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d68fa8-6c51-47db-8bd6-024d16ef322e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:18.388756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d68fa8-6c51-47db-8bd6-024d16ef322e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:18.405884 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2054/4124] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:18.410782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48fbfe-1234-4dc3-a09e-3d2caa0c13e6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:18.425126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48fbfe-1234-4dc3-a09e-3d2caa0c13e6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:18.436068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292812e-b4de-421e-8488-242c916bb872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:18.438858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292812e-b4de-421e-8488-242c916bb872 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:18.439352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292812e-b4de-421e-8488-242c916bb872 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:18.439820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292812e-b4de-421e-8488-242c916bb872 admin admin] PATH_INFO: `/v3/projects/bd992f2f3ab9401492cff286b12de76d/users/5761db70f7c140029f495bb1cc79b55e/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:18.470939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292812e-b4de-421e-8488-242c916bb872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd992f2f3ab9401492cff286b12de76d, user_id=5761db70f7c140029f495bb1cc79b55e, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:18.481932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292812e-b4de-421e-8488-242c916bb872 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:18.579897 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2071/4125] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:18 2022] PUT /identity/v3/projects/bd992f2f3ab9401492cff286b12de76d/users/5761db70f7c140029f495bb1cc79b55e/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:15:18.594346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23378b85-9eb8-4c30-be33-60875c21174c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:18.599644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23378b85-9eb8-4c30-be33-60875c21174c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:18.599644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23378b85-9eb8-4c30-be33-60875c21174c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:18.646077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-23378b85-9eb8-4c30-be33-60875c21174c None None] MFA Rules not processed for user `5761db70f7c140029f495bb1cc79b55e`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:18.647410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-23378b85-9eb8-4c30-be33-60875c21174c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:18.785421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:18.796983 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2055/4126] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:18 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:15:18.801325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292812e-b4de-421e-8488-242c916bb872 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:18.851931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:18.902121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292812e-b4de-421e-8488-242c916bb872 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:18.907089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457af0d-018d-47c1-86c0-303929accecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:18.907761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457af0d-018d-47c1-86c0-303929accecb admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:18.908043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457af0d-018d-47c1-86c0-303929accecb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:18.908321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457af0d-018d-47c1-86c0-303929accecb admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:18.909073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457af0d-018d-47c1-86c0-303929accecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:18.911029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457af0d-018d-47c1-86c0-303929accecb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:19.001796 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2072/4127] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:18 2022] POST /identity/v3/projects => generated 407 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:19.005055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23378b85-9eb8-4c30-be33-60875c21174c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:19.017767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23378b85-9eb8-4c30-be33-60875c21174c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:19.134215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b42d61-c994-484b-9597-a0bfab5121bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:19.134907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b42d61-c994-484b-9597-a0bfab5121bf admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:19.135211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b42d61-c994-484b-9597-a0bfab5121bf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:19.135471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b42d61-c994-484b-9597-a0bfab5121bf admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:19.136185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b42d61-c994-484b-9597-a0bfab5121bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:19.138452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b42d61-c994-484b-9597-a0bfab5121bf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:19.249319 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2056/4128] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:19 2022] POST /identity/v3/users => generated 310 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:15:19.250588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457af0d-018d-47c1-86c0-303929accecb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:19.272055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457af0d-018d-47c1-86c0-303929accecb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:19.287157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b55db-c10a-491d-a9ea-c09c7fd48f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:19.287909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b55db-c10a-491d-a9ea-c09c7fd48f33 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:19.288199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b55db-c10a-491d-a9ea-c09c7fd48f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:19.288533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b55db-c10a-491d-a9ea-c09c7fd48f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:19.289425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b55db-c10a-491d-a9ea-c09c7fd48f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:19.291401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b55db-c10a-491d-a9ea-c09c7fd48f33 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:19.307358 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2073/4129] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:19.311494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b42d61-c994-484b-9597-a0bfab5121bf admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:19.334029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b42d61-c994-484b-9597-a0bfab5121bf admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:19.339758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212ee0c-55fe-437a-955c-9335f3190928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:19.340733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212ee0c-55fe-437a-955c-9335f3190928 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:19.341084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212ee0c-55fe-437a-955c-9335f3190928 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:19.341425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212ee0c-55fe-437a-955c-9335f3190928 admin admin] PATH_INFO: `/v3/projects/344d9a047aff414381eb1e17f7dd76de/users/4166f72f3a184b29bdab60a81b8cb52f/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:19.373663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212ee0c-55fe-437a-955c-9335f3190928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=344d9a047aff414381eb1e17f7dd76de, user_id=4166f72f3a184b29bdab60a81b8cb52f, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:19.374917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212ee0c-55fe-437a-955c-9335f3190928 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:19.518750 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2057/4130] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:19 2022] PUT /identity/v3/projects/344d9a047aff414381eb1e17f7dd76de/users/4166f72f3a184b29bdab60a81b8cb52f/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:19.519994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b55db-c10a-491d-a9ea-c09c7fd48f33 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:19.533179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b55db-c10a-491d-a9ea-c09c7fd48f33 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:19.622521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5745a3d-cc6a-4e42-80be-377c968f14fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:19.623226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5745a3d-cc6a-4e42-80be-377c968f14fa admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:19.623514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5745a3d-cc6a-4e42-80be-377c968f14fa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:19.623843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5745a3d-cc6a-4e42-80be-377c968f14fa admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:19.624523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5745a3d-cc6a-4e42-80be-377c968f14fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:19.626273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5745a3d-cc6a-4e42-80be-377c968f14fa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:19.642550 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2074/4131] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:19.643871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212ee0c-55fe-437a-955c-9335f3190928 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:19.674451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212ee0c-55fe-437a-955c-9335f3190928 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:19.698020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b1c3f-e726-49d3-b31c-a51dfb0d5b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:19.710217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b1c3f-e726-49d3-b31c-a51dfb0d5b35 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:19.710529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b1c3f-e726-49d3-b31c-a51dfb0d5b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:19.710826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b1c3f-e726-49d3-b31c-a51dfb0d5b35 admin admin] PATH_INFO: `/v3/projects/344d9a047aff414381eb1e17f7dd76de/users/4166f72f3a184b29bdab60a81b8cb52f/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:19.722780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b1c3f-e726-49d3-b31c-a51dfb0d5b35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=344d9a047aff414381eb1e17f7dd76de, user_id=4166f72f3a184b29bdab60a81b8cb52f, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:19.725493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b1c3f-e726-49d3-b31c-a51dfb0d5b35 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:19.760934 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2058/4132] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:19 2022] PUT /identity/v3/projects/344d9a047aff414381eb1e17f7dd76de/users/4166f72f3a184b29bdab60a81b8cb52f/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:19.771759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e8683-8143-4fd9-b2de-a79babffd2f7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:19.772650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e8683-8143-4fd9-b2de-a79babffd2f7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:19.773076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e8683-8143-4fd9-b2de-a79babffd2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:19.833130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7b6e8683-8143-4fd9-b2de-a79babffd2f7 None None] MFA Rules not processed for user `4166f72f3a184b29bdab60a81b8cb52f`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:19.834535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7b6e8683-8143-4fd9-b2de-a79babffd2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:19.933444 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2075/4133] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:19 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:19.945719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b1c3f-e726-49d3-b31c-a51dfb0d5b35 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:19.956630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2a0b1c3f-e726-49d3-b31c-a51dfb0d5b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:20.059927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee29a48-6027-4c5f-a21a-f7d30e2ab875 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:20.059927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee29a48-6027-4c5f-a21a-f7d30e2ab875 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:20.059927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee29a48-6027-4c5f-a21a-f7d30e2ab875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:20.090797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cee29a48-6027-4c5f-a21a-f7d30e2ab875 None None] MFA Rules not processed for user `572170ccd67e4facbf10dad7cd1491c1`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:20.091879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b1c3f-e726-49d3-b31c-a51dfb0d5b35 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:20.092303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cee29a48-6027-4c5f-a21a-f7d30e2ab875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:20.103925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985753e4-a00e-4fc6-aef6-3f90ed9d3feb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:20.104940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985753e4-a00e-4fc6-aef6-3f90ed9d3feb service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:20.105427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985753e4-a00e-4fc6-aef6-3f90ed9d3feb service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:20.105926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985753e4-a00e-4fc6-aef6-3f90ed9d3feb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:20.113522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:20.200552 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2076/4134] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:20 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:20.238237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985753e4-a00e-4fc6-aef6-3f90ed9d3feb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:20.240550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985753e4-a00e-4fc6-aef6-3f90ed9d3feb service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:20.254748 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2059/4135] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:19 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:20.638725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee29a48-6027-4c5f-a21a-f7d30e2ab875 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:20.641007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cee29a48-6027-4c5f-a21a-f7d30e2ab875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:20.656576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee29a48-6027-4c5f-a21a-f7d30e2ab875 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:20.663022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0deb985-0e1d-44c6-aa74-d0c19d108acf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:20.664524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0deb985-0e1d-44c6-aa74-d0c19d108acf service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:20.664875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0deb985-0e1d-44c6-aa74-d0c19d108acf service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:20.665219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0deb985-0e1d-44c6-aa74-d0c19d108acf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:20.667548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c0deb985-0e1d-44c6-aa74-d0c19d108acf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:20.682109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0deb985-0e1d-44c6-aa74-d0c19d108acf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:20.683659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0deb985-0e1d-44c6-aa74-d0c19d108acf service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:20.703585 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2077/4136] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:20.936610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985753e4-a00e-4fc6-aef6-3f90ed9d3feb service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:20.938185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-985753e4-a00e-4fc6-aef6-3f90ed9d3feb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:20.945399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:21.044119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985753e4-a00e-4fc6-aef6-3f90ed9d3feb service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:21.051115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ca43d-3c98-47e9-aa26-afd0fa749a07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:21.053683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ca43d-3c98-47e9-aa26-afd0fa749a07 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:21.054753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ca43d-3c98-47e9-aa26-afd0fa749a07 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:21.055113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ca43d-3c98-47e9-aa26-afd0fa749a07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:21.065174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ca43d-3c98-47e9-aa26-afd0fa749a07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:21.069545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1ca43d-3c98-47e9-aa26-afd0fa749a07 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:21.082763 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2060/4137] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:20 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:15:23.665772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0deb985-0e1d-44c6-aa74-d0c19d108acf service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:23.666500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c0deb985-0e1d-44c6-aa74-d0c19d108acf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:23.767295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0deb985-0e1d-44c6-aa74-d0c19d108acf service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:23.772057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d086c-4339-42f6-a0c7-1c03716ccc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:23.773008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d086c-4339-42f6-a0c7-1c03716ccc40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:23.773300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d086c-4339-42f6-a0c7-1c03716ccc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:23.773585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d086c-4339-42f6-a0c7-1c03716ccc40 admin admin] PATH_INFO: `/v3/users/c9d2988731d14049b27283cda69bec93` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:23.774853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d086c-4339-42f6-a0c7-1c03716ccc40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9d2988731d14049b27283cda69bec93)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:23.776878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d086c-4339-42f6-a0c7-1c03716ccc40 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:23.972683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cf7d086c-4339-42f6-a0c7-1c03716ccc40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9d2988731d14049b27283cda69bec93'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:23.981884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cf7d086c-4339-42f6-a0c7-1c03716ccc40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9d2988731d14049b27283cda69bec93'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:23.985933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cf7d086c-4339-42f6-a0c7-1c03716ccc40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9d2988731d14049b27283cda69bec93'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:23.986824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cf7d086c-4339-42f6-a0c7-1c03716ccc40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9d2988731d14049b27283cda69bec93'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:24.076896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cf7d086c-4339-42f6-a0c7-1c03716ccc40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9d2988731d14049b27283cda69bec93'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:24.098906 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2078/4138] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:15:23 2022] DELETE /identity/v3/users/c9d2988731d14049b27283cda69bec93 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:24.104444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2e113-72c7-4b13-923e-0ef0d7cb5a6a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:24.106600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2e113-72c7-4b13-923e-0ef0d7cb5a6a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:24.106960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde2e113-72c7-4b13-923e-0ef0d7cb5a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:24.126114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-fde2e113-72c7-4b13-923e-0ef0d7cb5a6a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:24.128186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fde2e113-72c7-4b13-923e-0ef0d7cb5a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:24.204743 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2061/4139] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:24.214870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d086c-4339-42f6-a0c7-1c03716ccc40 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:24.216357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cf7d086c-4339-42f6-a0c7-1c03716ccc40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:24.251405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:24.279325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d086c-4339-42f6-a0c7-1c03716ccc40 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:24.288415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26beb04-0b8d-4a5b-85f9-082c6c24e9fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:24.289665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26beb04-0b8d-4a5b-85f9-082c6c24e9fb service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:24.289989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26beb04-0b8d-4a5b-85f9-082c6c24e9fb service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:24.290272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26beb04-0b8d-4a5b-85f9-082c6c24e9fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:24.300821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26beb04-0b8d-4a5b-85f9-082c6c24e9fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:24.302331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26beb04-0b8d-4a5b-85f9-082c6c24e9fb service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:24.314000 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2079/4140] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:25.235388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2e113-72c7-4b13-923e-0ef0d7cb5a6a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:25.238350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fde2e113-72c7-4b13-923e-0ef0d7cb5a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:25.262037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde2e113-72c7-4b13-923e-0ef0d7cb5a6a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:25.266367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfa511-8338-4557-b5f8-e23a8b23d143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:25.268028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfa511-8338-4557-b5f8-e23a8b23d143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:25.269066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfa511-8338-4557-b5f8-e23a8b23d143 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:25.269658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfa511-8338-4557-b5f8-e23a8b23d143 admin admin] PATH_INFO: `/v3/projects/6846f7be3c13484e92e2bfbe9df3810f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:25.271119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfa511-8338-4557-b5f8-e23a8b23d143 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6846f7be3c13484e92e2bfbe9df3810f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:25.275311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfa511-8338-4557-b5f8-e23a8b23d143 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:25.465556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-00bfa511-8338-4557-b5f8-e23a8b23d143 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6846f7be3c13484e92e2bfbe9df3810f', 'request_id': 'req-00bfa511-8338-4557-b5f8-e23a8b23d143'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:25.471536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-00bfa511-8338-4557-b5f8-e23a8b23d143 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6846f7be3c13484e92e2bfbe9df3810f', 'request_id': 'req-00bfa511-8338-4557-b5f8-e23a8b23d143'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:25.582433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-00bfa511-8338-4557-b5f8-e23a8b23d143 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6846f7be3c13484e92e2bfbe9df3810f', 'request_id': 'req-00bfa511-8338-4557-b5f8-e23a8b23d143'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:25.598014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-00bfa511-8338-4557-b5f8-e23a8b23d143 admin admin] The token cache is being invalidate because project 6846f7be3c13484e92e2bfbe9df3810f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:15:25.598211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-00bfa511-8338-4557-b5f8-e23a8b23d143 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:25.598856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-00bfa511-8338-4557-b5f8-e23a8b23d143 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:25.600481 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2062/4141] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:25 2022] DELETE /identity/v3/projects/6846f7be3c13484e92e2bfbe9df3810f => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:25.610546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a24e7-c5fe-4100-b3d5-a1f14dbc89cb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:25.611440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a24e7-c5fe-4100-b3d5-a1f14dbc89cb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:25.611440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a24e7-c5fe-4100-b3d5-a1f14dbc89cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:25.634309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cb0a24e7-c5fe-4100-b3d5-a1f14dbc89cb None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:25.635882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cb0a24e7-c5fe-4100-b3d5-a1f14dbc89cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:25.709242 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2080/4142] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:15:25.718031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf515e5-7bbb-4a74-b848-700a57ba0b8d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:25.718502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf515e5-7bbb-4a74-b848-700a57ba0b8d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:25.718895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf515e5-7bbb-4a74-b848-700a57ba0b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:25.753885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bdf515e5-7bbb-4a74-b848-700a57ba0b8d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:25.755467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bdf515e5-7bbb-4a74-b848-700a57ba0b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:25.773480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a24e7-c5fe-4100-b3d5-a1f14dbc89cb None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:25.774644 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2063/4143] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:15:25.822175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a24e7-c5fe-4100-b3d5-a1f14dbc89cb None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:25.847276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41562b-aa4b-4b2b-9b84-895e29d5d531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:25.851075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41562b-aa4b-4b2b-9b84-895e29d5d531 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:25.851075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41562b-aa4b-4b2b-9b84-895e29d5d531 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:25.852875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41562b-aa4b-4b2b-9b84-895e29d5d531 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:25.853733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41562b-aa4b-4b2b-9b84-895e29d5d531 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:15:25.856833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41562b-aa4b-4b2b-9b84-895e29d5d531 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:25.859616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41562b-aa4b-4b2b-9b84-895e29d5d531 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:25.893877 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2081/4144] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:25.897076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf515e5-7bbb-4a74-b848-700a57ba0b8d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:25.904129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:25.914683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf515e5-7bbb-4a74-b848-700a57ba0b8d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:25.920212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d1be9-b6e0-491b-aa57-cfb31e68ab6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:25.921205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d1be9-b6e0-491b-aa57-cfb31e68ab6b admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:25.921815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d1be9-b6e0-491b-aa57-cfb31e68ab6b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:25.922370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d1be9-b6e0-491b-aa57-cfb31e68ab6b admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:25.923555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d1be9-b6e0-491b-aa57-cfb31e68ab6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:25.926613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1d1be9-b6e0-491b-aa57-cfb31e68ab6b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:26.055176 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2064/4145] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:25 2022] POST /identity/v3/projects => generated 399 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:26.059921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41562b-aa4b-4b2b-9b84-895e29d5d531 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:26.078727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41562b-aa4b-4b2b-9b84-895e29d5d531 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:26.147130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc9eed-59f6-4e1e-a037-5bea23c85638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:26.147712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc9eed-59f6-4e1e-a037-5bea23c85638 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:26.148588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc9eed-59f6-4e1e-a037-5bea23c85638 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:26.148872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc9eed-59f6-4e1e-a037-5bea23c85638 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:26.149595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc9eed-59f6-4e1e-a037-5bea23c85638 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:26.151573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc9eed-59f6-4e1e-a037-5bea23c85638 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:26.220049 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2082/4146] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:26 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:26.229832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d1be9-b6e0-491b-aa57-cfb31e68ab6b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:26.240979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1d1be9-b6e0-491b-aa57-cfb31e68ab6b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:26.248646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041af03-1d5e-41c2-be15-db03de937451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:26.249663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041af03-1d5e-41c2-be15-db03de937451 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:26.250090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041af03-1d5e-41c2-be15-db03de937451 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:26.250557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041af03-1d5e-41c2-be15-db03de937451 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:26.251511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041af03-1d5e-41c2-be15-db03de937451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:26.253397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041af03-1d5e-41c2-be15-db03de937451 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:26.264226 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2065/4147] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:26.268354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc9eed-59f6-4e1e-a037-5bea23c85638 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:26.278823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc9eed-59f6-4e1e-a037-5bea23c85638 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:26.284552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0f019-27d6-498f-9f5d-6a2fd24c69c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:26.285310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0f019-27d6-498f-9f5d-6a2fd24c69c1 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:26.285670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0f019-27d6-498f-9f5d-6a2fd24c69c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:26.286048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0f019-27d6-498f-9f5d-6a2fd24c69c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:26.286848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0f019-27d6-498f-9f5d-6a2fd24c69c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:26.289192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0f019-27d6-498f-9f5d-6a2fd24c69c1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:26.299802 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2083/4148] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:26.304270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041af03-1d5e-41c2-be15-db03de937451 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:26.315262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041af03-1d5e-41c2-be15-db03de937451 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:26.320320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d2d08-3c8b-4db3-8bb9-66c6c1f6e0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:26.321211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d2d08-3c8b-4db3-8bb9-66c6c1f6e0e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:26.321494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d2d08-3c8b-4db3-8bb9-66c6c1f6e0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:26.321766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d2d08-3c8b-4db3-8bb9-66c6c1f6e0e5 admin admin] PATH_INFO: `/v3/projects/12671a8f96044bc88ad2ac14e0220d77/users/ea8697bfa82c4333abda2133581b9f33/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:26.340340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d2d08-3c8b-4db3-8bb9-66c6c1f6e0e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12671a8f96044bc88ad2ac14e0220d77, user_id=ea8697bfa82c4333abda2133581b9f33, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:26.343500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d2d08-3c8b-4db3-8bb9-66c6c1f6e0e5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:26.454085 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2066/4149] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:26 2022] PUT /identity/v3/projects/12671a8f96044bc88ad2ac14e0220d77/users/ea8697bfa82c4333abda2133581b9f33/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:26.465042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c8591-b005-422a-a1d9-dfb6b6054f15 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:26.465419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c8591-b005-422a-a1d9-dfb6b6054f15 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:26.465863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c8591-b005-422a-a1d9-dfb6b6054f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:26.507495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0c5c8591-b005-422a-a1d9-dfb6b6054f15 None None] MFA Rules not processed for user `ea8697bfa82c4333abda2133581b9f33`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:26.508935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0c5c8591-b005-422a-a1d9-dfb6b6054f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:26.635036 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2084/4150] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:26.653776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d2d08-3c8b-4db3-8bb9-66c6c1f6e0e5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:26.727025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d2d08-3c8b-4db3-8bb9-66c6c1f6e0e5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:26.733091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05cd1f-9daf-4072-b332-35a1c64fac99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:26.735668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05cd1f-9daf-4072-b332-35a1c64fac99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:26.736294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05cd1f-9daf-4072-b332-35a1c64fac99 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:26.736294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05cd1f-9daf-4072-b332-35a1c64fac99 admin admin] PATH_INFO: `/v3/users/ea8697bfa82c4333abda2133581b9f33` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:26.740012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05cd1f-9daf-4072-b332-35a1c64fac99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea8697bfa82c4333abda2133581b9f33)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:26.740012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05cd1f-9daf-4072-b332-35a1c64fac99 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:26.904304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:26.964191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0c05cd1f-9daf-4072-b332-35a1c64fac99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea8697bfa82c4333abda2133581b9f33'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:26.965429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0c05cd1f-9daf-4072-b332-35a1c64fac99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea8697bfa82c4333abda2133581b9f33'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:26.982554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0c05cd1f-9daf-4072-b332-35a1c64fac99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea8697bfa82c4333abda2133581b9f33'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:27.095395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0c05cd1f-9daf-4072-b332-35a1c64fac99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea8697bfa82c4333abda2133581b9f33'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:27.095395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0c05cd1f-9daf-4072-b332-35a1c64fac99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea8697bfa82c4333abda2133581b9f33'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:27.125953 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2067/4151] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:15:26 2022] DELETE /identity/v3/users/ea8697bfa82c4333abda2133581b9f33 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:27.130211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae4ab7-bfce-4553-b458-d757a5790ed4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:27.130503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae4ab7-bfce-4553-b458-d757a5790ed4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:27.130786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ae4ab7-bfce-4553-b458-d757a5790ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:27.155259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-30ae4ab7-bfce-4553-b458-d757a5790ed4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:27.156623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-30ae4ab7-bfce-4553-b458-d757a5790ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:27.229959 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2085/4152] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:27.240779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05cd1f-9daf-4072-b332-35a1c64fac99 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:27.242767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0c05cd1f-9daf-4072-b332-35a1c64fac99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:27.311374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05cd1f-9daf-4072-b332-35a1c64fac99 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:27.315168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30eb9c-9420-4635-aa3b-d0514fe638fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:27.315835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30eb9c-9420-4635-aa3b-d0514fe638fe service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:27.316144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30eb9c-9420-4635-aa3b-d0514fe638fe service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:27.316427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30eb9c-9420-4635-aa3b-d0514fe638fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:27.327162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe30eb9c-9420-4635-aa3b-d0514fe638fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:27.328609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe30eb9c-9420-4635-aa3b-d0514fe638fe service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:27.339769 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2068/4153] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:28.084063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae4ab7-bfce-4553-b458-d757a5790ed4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:28.086201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-30ae4ab7-bfce-4553-b458-d757a5790ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:28.099774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ae4ab7-bfce-4553-b458-d757a5790ed4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:28.106387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00b72db-4722-4079-90f7-63fcd2bf8296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:28.107521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b72db-4722-4079-90f7-63fcd2bf8296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:28.107933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b72db-4722-4079-90f7-63fcd2bf8296 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:28.108300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00b72db-4722-4079-90f7-63fcd2bf8296 admin admin] PATH_INFO: `/v3/projects/12671a8f96044bc88ad2ac14e0220d77` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:28.110130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b72db-4722-4079-90f7-63fcd2bf8296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12671a8f96044bc88ad2ac14e0220d77)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:28.112306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00b72db-4722-4079-90f7-63fcd2bf8296 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:28.286110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b00b72db-4722-4079-90f7-63fcd2bf8296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12671a8f96044bc88ad2ac14e0220d77', 'request_id': 'req-b00b72db-4722-4079-90f7-63fcd2bf8296'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:28.291686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b00b72db-4722-4079-90f7-63fcd2bf8296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12671a8f96044bc88ad2ac14e0220d77', 'request_id': 'req-b00b72db-4722-4079-90f7-63fcd2bf8296'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:28.407869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b00b72db-4722-4079-90f7-63fcd2bf8296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12671a8f96044bc88ad2ac14e0220d77', 'request_id': 'req-b00b72db-4722-4079-90f7-63fcd2bf8296'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:28.422810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b00b72db-4722-4079-90f7-63fcd2bf8296 admin admin] The token cache is being invalidate because project 12671a8f96044bc88ad2ac14e0220d77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:15:28.423237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b00b72db-4722-4079-90f7-63fcd2bf8296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:28.424125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b00b72db-4722-4079-90f7-63fcd2bf8296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:28.426644 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2086/4154] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:28 2022] DELETE /identity/v3/projects/12671a8f96044bc88ad2ac14e0220d77 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:28.431793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d3f5a-9635-4a83-bfb0-2328b6ccdbff None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:28.432308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d3f5a-9635-4a83-bfb0-2328b6ccdbff None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:28.432406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d3f5a-9635-4a83-bfb0-2328b6ccdbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:28.459623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b20d3f5a-9635-4a83-bfb0-2328b6ccdbff None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:28.461935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b20d3f5a-9635-4a83-bfb0-2328b6ccdbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:28.551788 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2069/4155] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:28.560572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975436f-c9b8-457b-9d73-02a634303e69 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:28.560926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975436f-c9b8-457b-9d73-02a634303e69 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:28.561211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975436f-c9b8-457b-9d73-02a634303e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:28.590833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2975436f-c9b8-457b-9d73-02a634303e69 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:28.590833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2975436f-c9b8-457b-9d73-02a634303e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:28.600317 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2087/4156] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:28.604993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d3f5a-9635-4a83-bfb0-2328b6ccdbff None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:28.641220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d3f5a-9635-4a83-bfb0-2328b6ccdbff None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:28.644094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:28.653786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df978b5a-8893-4b1b-b244-681d3e5ccdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:28.654562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df978b5a-8893-4b1b-b244-681d3e5ccdaf admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:28.655165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df978b5a-8893-4b1b-b244-681d3e5ccdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:28.655515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df978b5a-8893-4b1b-b244-681d3e5ccdaf admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:28.656267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df978b5a-8893-4b1b-b244-681d3e5ccdaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:15:28.656590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df978b5a-8893-4b1b-b244-681d3e5ccdaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:28.659114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df978b5a-8893-4b1b-b244-681d3e5ccdaf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:28.674023 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2070/4157] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:28.677588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975436f-c9b8-457b-9d73-02a634303e69 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:28.689248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975436f-c9b8-457b-9d73-02a634303e69 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:28.694325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf705ea-48bc-406f-8403-be771dcfffdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:28.695964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf705ea-48bc-406f-8403-be771dcfffdf admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:28.696509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf705ea-48bc-406f-8403-be771dcfffdf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:28.697123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf705ea-48bc-406f-8403-be771dcfffdf admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:28.698065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf705ea-48bc-406f-8403-be771dcfffdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:28.700521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf705ea-48bc-406f-8403-be771dcfffdf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:28.837311 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2088/4158] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:28 2022] POST /identity/v3/projects => generated 387 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:28.841222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df978b5a-8893-4b1b-b244-681d3e5ccdaf admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:28.851388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df978b5a-8893-4b1b-b244-681d3e5ccdaf admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:28.907989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ae926-2fb0-4239-bcd8-4417d617b06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:28.908656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ae926-2fb0-4239-bcd8-4417d617b06d admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:28.908943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ae926-2fb0-4239-bcd8-4417d617b06d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:28.909220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ae926-2fb0-4239-bcd8-4417d617b06d admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:28.909964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ae926-2fb0-4239-bcd8-4417d617b06d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:28.911882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ae926-2fb0-4239-bcd8-4417d617b06d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:28.997577 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2071/4159] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:28 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:29.003301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf705ea-48bc-406f-8403-be771dcfffdf admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:29.015106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf705ea-48bc-406f-8403-be771dcfffdf admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:29.021594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2def6f4-3280-42c8-86f5-fe0b1c657a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:29.022423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2def6f4-3280-42c8-86f5-fe0b1c657a9b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:29.030810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2def6f4-3280-42c8-86f5-fe0b1c657a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:29.030810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2def6f4-3280-42c8-86f5-fe0b1c657a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:29.030810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2def6f4-3280-42c8-86f5-fe0b1c657a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:29.030810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2def6f4-3280-42c8-86f5-fe0b1c657a9b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:29.041671 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2089/4160] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:29.042564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ae926-2fb0-4239-bcd8-4417d617b06d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:29.053256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ae926-2fb0-4239-bcd8-4417d617b06d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:29.062448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9859e-5ed7-4b7a-8684-7e78bcc73f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:29.063330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9859e-5ed7-4b7a-8684-7e78bcc73f30 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:29.063631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9859e-5ed7-4b7a-8684-7e78bcc73f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:29.063902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9859e-5ed7-4b7a-8684-7e78bcc73f30 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:29.064631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9859e-5ed7-4b7a-8684-7e78bcc73f30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:29.066475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9859e-5ed7-4b7a-8684-7e78bcc73f30 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:29.080196 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2072/4161] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:15:29.084619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2def6f4-3280-42c8-86f5-fe0b1c657a9b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:29.098920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2def6f4-3280-42c8-86f5-fe0b1c657a9b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:29.102605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc9800-9708-47ef-b31f-43a4e4845af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:29.103511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc9800-9708-47ef-b31f-43a4e4845af9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:29.103965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc9800-9708-47ef-b31f-43a4e4845af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:29.104318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dc9800-9708-47ef-b31f-43a4e4845af9 admin admin] PATH_INFO: `/v3/projects/44b41a2aedcd4e45a4e96cfc9228fdd6/users/81ccd8c6e84040ca86ed3abc42c53fed/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:29.119090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:29.122324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc9800-9708-47ef-b31f-43a4e4845af9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b41a2aedcd4e45a4e96cfc9228fdd6, user_id=81ccd8c6e84040ca86ed3abc42c53fed, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:29.125020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dc9800-9708-47ef-b31f-43a4e4845af9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:29.202828 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2090/4162] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:29 2022] PUT /identity/v3/projects/44b41a2aedcd4e45a4e96cfc9228fdd6/users/81ccd8c6e84040ca86ed3abc42c53fed/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:29.211144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a5b40-5ce2-4ac8-a3f8-df01f9e68140 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:29.212199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a5b40-5ce2-4ac8-a3f8-df01f9e68140 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:29.213082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a5b40-5ce2-4ac8-a3f8-df01f9e68140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:29.237506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-335a5b40-5ce2-4ac8-a3f8-df01f9e68140 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:29.238689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-335a5b40-5ce2-4ac8-a3f8-df01f9e68140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:29.302496 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2073/4163] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:29.313027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc9800-9708-47ef-b31f-43a4e4845af9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:29.314955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-09dc9800-9708-47ef-b31f-43a4e4845af9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:29.374074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dc9800-9708-47ef-b31f-43a4e4845af9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:29.378102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d5c15a-5029-45f7-b724-159ce48d3414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:29.378761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5c15a-5029-45f7-b724-159ce48d3414 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:29.379046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5c15a-5029-45f7-b724-159ce48d3414 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:29.379328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5c15a-5029-45f7-b724-159ce48d3414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:29.399484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d5c15a-5029-45f7-b724-159ce48d3414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:29.399484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d5c15a-5029-45f7-b724-159ce48d3414 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:29.402830 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2091/4164] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:30.359033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347ade2-d6aa-46df-b9e6-a5cf9aa15327 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:30.360639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347ade2-d6aa-46df-b9e6-a5cf9aa15327 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:30.361132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347ade2-d6aa-46df-b9e6-a5cf9aa15327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:30.401463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3347ade2-d6aa-46df-b9e6-a5cf9aa15327 None None] MFA Rules not processed for user `81ccd8c6e84040ca86ed3abc42c53fed`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:30.404542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3347ade2-d6aa-46df-b9e6-a5cf9aa15327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:30.484784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d5c15a-5029-45f7-b724-159ce48d3414 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:30.486585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-27d5c15a-5029-45f7-b724-159ce48d3414 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:30.504666 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2074/4165] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:30.506588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d5c15a-5029-45f7-b724-159ce48d3414 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:30.515554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed40ced-54f8-458a-a212-6bc2b888f7b5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:30.516229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed40ced-54f8-458a-a212-6bc2b888f7b5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:30.516609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed40ced-54f8-458a-a212-6bc2b888f7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:30.516753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758a43d-6cc1-4fac-aad3-94f0a9e06407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:30.517603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758a43d-6cc1-4fac-aad3-94f0a9e06407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:30.517913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758a43d-6cc1-4fac-aad3-94f0a9e06407 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:30.518202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758a43d-6cc1-4fac-aad3-94f0a9e06407 admin admin] PATH_INFO: `/v3/users/5761db70f7c140029f495bb1cc79b55e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:30.520526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758a43d-6cc1-4fac-aad3-94f0a9e06407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5761db70f7c140029f495bb1cc79b55e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:30.522636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758a43d-6cc1-4fac-aad3-94f0a9e06407 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:30.539161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0ed40ced-54f8-458a-a212-6bc2b888f7b5 None None] MFA Rules not processed for user `81ccd8c6e84040ca86ed3abc42c53fed`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:30.540702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0ed40ced-54f8-458a-a212-6bc2b888f7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:30.554028 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2075/4166] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:15:30.569290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed40ced-54f8-458a-a212-6bc2b888f7b5 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:30.571963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0ed40ced-54f8-458a-a212-6bc2b888f7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:30.655241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed40ced-54f8-458a-a212-6bc2b888f7b5 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:30.661388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80709a01-94d6-4d77-a739-36b1928d96c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:30.662097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80709a01-94d6-4d77-a739-36b1928d96c8 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:30.665586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80709a01-94d6-4d77-a739-36b1928d96c8 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:30.665586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80709a01-94d6-4d77-a739-36b1928d96c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:30.671797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80709a01-94d6-4d77-a739-36b1928d96c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:30.674365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80709a01-94d6-4d77-a739-36b1928d96c8 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:30.686095 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2076/4167] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:15:30.727347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:30.805223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca3c20-4e9a-410a-8247-0bf615ace10f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:30.805535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca3c20-4e9a-410a-8247-0bf615ace10f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:30.805813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca3c20-4e9a-410a-8247-0bf615ace10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:30.834642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9758a43d-6cc1-4fac-aad3-94f0a9e06407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5761db70f7c140029f495bb1cc79b55e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:30.843912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9758a43d-6cc1-4fac-aad3-94f0a9e06407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5761db70f7c140029f495bb1cc79b55e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:30.845105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9758a43d-6cc1-4fac-aad3-94f0a9e06407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5761db70f7c140029f495bb1cc79b55e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:30.846321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9758a43d-6cc1-4fac-aad3-94f0a9e06407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5761db70f7c140029f495bb1cc79b55e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:30.871492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0bca3c20-4e9a-410a-8247-0bf615ace10f None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:30.872817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0bca3c20-4e9a-410a-8247-0bf615ace10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:30.934463 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2077/4168] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:15:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:15:30.947484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca3c20-4e9a-410a-8247-0bf615ace10f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:30.952782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9758a43d-6cc1-4fac-aad3-94f0a9e06407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5761db70f7c140029f495bb1cc79b55e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:30.955350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0bca3c20-4e9a-410a-8247-0bf615ace10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:30.976733 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2092/4169] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:15:30 2022] DELETE /identity/v3/users/5761db70f7c140029f495bb1cc79b55e => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:15:30.981465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1688a-7454-4903-8750-4a0111e596d2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:30.981706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1688a-7454-4903-8750-4a0111e596d2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:30.982134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1688a-7454-4903-8750-4a0111e596d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:31.003393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f9d1688a-7454-4903-8750-4a0111e596d2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:31.004629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f9d1688a-7454-4903-8750-4a0111e596d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:31.072759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca3c20-4e9a-410a-8247-0bf615ace10f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:31.083449 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2093/4170] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:31.098502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1688a-7454-4903-8750-4a0111e596d2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:31.099649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f9d1688a-7454-4903-8750-4a0111e596d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:31.184013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:31.209498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1688a-7454-4903-8750-4a0111e596d2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:31.216009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21528244-0dbf-4f5a-8493-de1deafa31a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:31.224995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21528244-0dbf-4f5a-8493-de1deafa31a9 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:31.224995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21528244-0dbf-4f5a-8493-de1deafa31a9 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:31.224995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21528244-0dbf-4f5a-8493-de1deafa31a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:31.225294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36dd9f-6f60-4504-9cae-9df36b9fa170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:31.225294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36dd9f-6f60-4504-9cae-9df36b9fa170 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:31.225294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36dd9f-6f60-4504-9cae-9df36b9fa170 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:31.225294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36dd9f-6f60-4504-9cae-9df36b9fa170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:31.242483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36dd9f-6f60-4504-9cae-9df36b9fa170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:31.244844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36dd9f-6f60-4504-9cae-9df36b9fa170 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:31.262939 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2094/4171] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:15:31.325469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21528244-0dbf-4f5a-8493-de1deafa31a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:31.327902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21528244-0dbf-4f5a-8493-de1deafa31a9 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:31.341125 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2078/4172] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:15:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:15:31.603964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36dd9f-6f60-4504-9cae-9df36b9fa170 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:31.606275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8a36dd9f-6f60-4504-9cae-9df36b9fa170 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:31.628591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36dd9f-6f60-4504-9cae-9df36b9fa170 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:31.635332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4134d8-49c2-45fb-8fb6-a8e28308f663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:31.636739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4134d8-49c2-45fb-8fb6-a8e28308f663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:31.637024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4134d8-49c2-45fb-8fb6-a8e28308f663 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:31.637691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4134d8-49c2-45fb-8fb6-a8e28308f663 admin admin] PATH_INFO: `/v3/projects/bd992f2f3ab9401492cff286b12de76d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:31.639198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4134d8-49c2-45fb-8fb6-a8e28308f663 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd992f2f3ab9401492cff286b12de76d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:31.644803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4134d8-49c2-45fb-8fb6-a8e28308f663 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:31.708241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21528244-0dbf-4f5a-8493-de1deafa31a9 service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:31.709271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-21528244-0dbf-4f5a-8493-de1deafa31a9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:31.732562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21528244-0dbf-4f5a-8493-de1deafa31a9 service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:31.737746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be3f12-e187-48d0-84e2-2506c162fb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:31.738665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be3f12-e187-48d0-84e2-2506c162fb5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:31.738948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be3f12-e187-48d0-84e2-2506c162fb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:31.739221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be3f12-e187-48d0-84e2-2506c162fb5c admin admin] PATH_INFO: `/v3/users/81ccd8c6e84040ca86ed3abc42c53fed` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:31.740432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be3f12-e187-48d0-84e2-2506c162fb5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=81ccd8c6e84040ca86ed3abc42c53fed)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:31.742675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be3f12-e187-48d0-84e2-2506c162fb5c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:31.807232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5c4134d8-49c2-45fb-8fb6-a8e28308f663 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd992f2f3ab9401492cff286b12de76d', 'request_id': 'req-5c4134d8-49c2-45fb-8fb6-a8e28308f663'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:31.815168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5c4134d8-49c2-45fb-8fb6-a8e28308f663 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd992f2f3ab9401492cff286b12de76d', 'request_id': 'req-5c4134d8-49c2-45fb-8fb6-a8e28308f663'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:31.924195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5c4134d8-49c2-45fb-8fb6-a8e28308f663 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd992f2f3ab9401492cff286b12de76d', 'request_id': 'req-5c4134d8-49c2-45fb-8fb6-a8e28308f663'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:31.937058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5c4134d8-49c2-45fb-8fb6-a8e28308f663 admin admin] The token cache is being invalidate because project bd992f2f3ab9401492cff286b12de76d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:15:31.937348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5c4134d8-49c2-45fb-8fb6-a8e28308f663 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:31.938055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5c4134d8-49c2-45fb-8fb6-a8e28308f663 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:31.942109 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2095/4173] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:31 2022] DELETE /identity/v3/projects/bd992f2f3ab9401492cff286b12de76d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:31.946545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4134d8-49c2-45fb-8fb6-a8e28308f663 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:31.947995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5c4134d8-49c2-45fb-8fb6-a8e28308f663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:31.958844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:31.978185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-30be3f12-e187-48d0-84e2-2506c162fb5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81ccd8c6e84040ca86ed3abc42c53fed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:31.979069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-30be3f12-e187-48d0-84e2-2506c162fb5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81ccd8c6e84040ca86ed3abc42c53fed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:31.986814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-30be3f12-e187-48d0-84e2-2506c162fb5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81ccd8c6e84040ca86ed3abc42c53fed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:32.041937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4134d8-49c2-45fb-8fb6-a8e28308f663 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:32.046931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbeca55-48c2-4220-845f-fffc540f55f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:32.047968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeca55-48c2-4220-845f-fffc540f55f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:32.048363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeca55-48c2-4220-845f-fffc540f55f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:32.048706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbeca55-48c2-4220-845f-fffc540f55f7 admin admin] PATH_INFO: `/v3/users/4166f72f3a184b29bdab60a81b8cb52f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:32.050174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbeca55-48c2-4220-845f-fffc540f55f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4166f72f3a184b29bdab60a81b8cb52f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:32.052562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbeca55-48c2-4220-845f-fffc540f55f7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:32.104407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-30be3f12-e187-48d0-84e2-2506c162fb5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81ccd8c6e84040ca86ed3abc42c53fed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:32.105283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-30be3f12-e187-48d0-84e2-2506c162fb5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81ccd8c6e84040ca86ed3abc42c53fed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:32.127232 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2079/4174] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:15:31 2022] DELETE /identity/v3/users/81ccd8c6e84040ca86ed3abc42c53fed => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:15:32.308758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abbeca55-48c2-4220-845f-fffc540f55f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4166f72f3a184b29bdab60a81b8cb52f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:32.316686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abbeca55-48c2-4220-845f-fffc540f55f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4166f72f3a184b29bdab60a81b8cb52f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:32.317440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abbeca55-48c2-4220-845f-fffc540f55f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4166f72f3a184b29bdab60a81b8cb52f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:32.318116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abbeca55-48c2-4220-845f-fffc540f55f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4166f72f3a184b29bdab60a81b8cb52f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:32.414265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abbeca55-48c2-4220-845f-fffc540f55f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4166f72f3a184b29bdab60a81b8cb52f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:32.458138 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2096/4175] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:15:31 2022] DELETE /identity/v3/users/4166f72f3a184b29bdab60a81b8cb52f => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:15:32.603712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be3f12-e187-48d0-84e2-2506c162fb5c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:32.609299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:32.611027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-30be3f12-e187-48d0-84e2-2506c162fb5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:32.769993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be3f12-e187-48d0-84e2-2506c162fb5c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:32.812697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbeca55-48c2-4220-845f-fffc540f55f7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:32.815272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-abbeca55-48c2-4220-845f-fffc540f55f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:32.966512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314eee38-5343-49e1-a4c3-4a0fd873be59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:32.968330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314eee38-5343-49e1-a4c3-4a0fd873be59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:32.968801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314eee38-5343-49e1-a4c3-4a0fd873be59 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:32.969230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314eee38-5343-49e1-a4c3-4a0fd873be59 admin admin] PATH_INFO: `/v3/projects/44b41a2aedcd4e45a4e96cfc9228fdd6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:32.970734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314eee38-5343-49e1-a4c3-4a0fd873be59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44b41a2aedcd4e45a4e96cfc9228fdd6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:32.973439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314eee38-5343-49e1-a4c3-4a0fd873be59 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:32.993869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbeca55-48c2-4220-845f-fffc540f55f7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:32.999631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dfdf42-621f-4b61-a2c7-92e7361a585d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:33.000864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfdf42-621f-4b61-a2c7-92e7361a585d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:33.001025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfdf42-621f-4b61-a2c7-92e7361a585d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:33.001212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dfdf42-621f-4b61-a2c7-92e7361a585d admin admin] PATH_INFO: `/v3/projects/344d9a047aff414381eb1e17f7dd76de` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:33.002493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dfdf42-621f-4b61-a2c7-92e7361a585d admin admin] RBAC: Authorizing `identity:delete_project(project_id=344d9a047aff414381eb1e17f7dd76de)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:33.004457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dfdf42-621f-4b61-a2c7-92e7361a585d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:33.145255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:33.159917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-314eee38-5343-49e1-a4c3-4a0fd873be59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44b41a2aedcd4e45a4e96cfc9228fdd6', 'request_id': 'req-314eee38-5343-49e1-a4c3-4a0fd873be59'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:33.169599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-314eee38-5343-49e1-a4c3-4a0fd873be59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44b41a2aedcd4e45a4e96cfc9228fdd6', 'request_id': 'req-314eee38-5343-49e1-a4c3-4a0fd873be59'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:33.177818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c4dfdf42-621f-4b61-a2c7-92e7361a585d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '344d9a047aff414381eb1e17f7dd76de', 'request_id': 'req-c4dfdf42-621f-4b61-a2c7-92e7361a585d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:33.185337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c4dfdf42-621f-4b61-a2c7-92e7361a585d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '344d9a047aff414381eb1e17f7dd76de', 'request_id': 'req-c4dfdf42-621f-4b61-a2c7-92e7361a585d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:33.229952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:33.286339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-314eee38-5343-49e1-a4c3-4a0fd873be59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44b41a2aedcd4e45a4e96cfc9228fdd6', 'request_id': 'req-314eee38-5343-49e1-a4c3-4a0fd873be59'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:33.300113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-314eee38-5343-49e1-a4c3-4a0fd873be59 admin admin] The token cache is being invalidate because project 44b41a2aedcd4e45a4e96cfc9228fdd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:15:33.300407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-314eee38-5343-49e1-a4c3-4a0fd873be59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:33.301072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-314eee38-5343-49e1-a4c3-4a0fd873be59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:33.302972 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2080/4176] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:32 2022] DELETE /identity/v3/projects/44b41a2aedcd4e45a4e96cfc9228fdd6 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:33.313684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6637679-3b82-4330-8ab4-e3cbd8c4dc45 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:33.314011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6637679-3b82-4330-8ab4-e3cbd8c4dc45 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:33.314294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6637679-3b82-4330-8ab4-e3cbd8c4dc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:33.342396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b6637679-3b82-4330-8ab4-e3cbd8c4dc45 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:33.342396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b6637679-3b82-4330-8ab4-e3cbd8c4dc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:33.406868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c4dfdf42-621f-4b61-a2c7-92e7361a585d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '344d9a047aff414381eb1e17f7dd76de', 'request_id': 'req-c4dfdf42-621f-4b61-a2c7-92e7361a585d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:33.421491 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2081/4177] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:33.431720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c4dfdf42-621f-4b61-a2c7-92e7361a585d admin admin] The token cache is being invalidate because project 344d9a047aff414381eb1e17f7dd76de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:15:33.431720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c4dfdf42-621f-4b61-a2c7-92e7361a585d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:33.431720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c4dfdf42-621f-4b61-a2c7-92e7361a585d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:33.434174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1fcd8-d366-4154-8204-e3f80b25584f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:33.434628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1fcd8-d366-4154-8204-e3f80b25584f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:33.435056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b1fcd8-d366-4154-8204-e3f80b25584f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:33.436235 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2097/4178] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:32 2022] DELETE /identity/v3/projects/344d9a047aff414381eb1e17f7dd76de => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:33.437629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068eb4a5-ed25-475a-bd5f-0d204ccbc54f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:33.437945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068eb4a5-ed25-475a-bd5f-0d204ccbc54f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:33.438229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068eb4a5-ed25-475a-bd5f-0d204ccbc54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:33.475452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-068eb4a5-ed25-475a-bd5f-0d204ccbc54f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:33.476568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-068eb4a5-ed25-475a-bd5f-0d204ccbc54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:33.502017 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2098/4179] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:33.503190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368abe5-d279-4745-b58f-ad4a66d698db None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:33.503482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368abe5-d279-4745-b58f-ad4a66d698db None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:33.503762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c368abe5-d279-4745-b58f-ad4a66d698db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:33.510860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-53b1fcd8-d366-4154-8204-e3f80b25584f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:33.513212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-53b1fcd8-d366-4154-8204-e3f80b25584f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:33.528519 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2082/4180] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:33.537589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1fcd8-d366-4154-8204-e3f80b25584f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:33.540076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c368abe5-d279-4745-b58f-ad4a66d698db None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:33.541303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c368abe5-d279-4745-b58f-ad4a66d698db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:33.553469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b1fcd8-d366-4154-8204-e3f80b25584f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:33.559133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310e4db-2857-49c7-9452-455552163394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:33.560067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310e4db-2857-49c7-9452-455552163394 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:33.560438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310e4db-2857-49c7-9452-455552163394 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:33.560790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6310e4db-2857-49c7-9452-455552163394 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:33.561886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310e4db-2857-49c7-9452-455552163394 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:15:33.562252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310e4db-2857-49c7-9452-455552163394 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:33.563348 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2099/4181] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:33.564560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6310e4db-2857-49c7-9452-455552163394 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:33.569192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368abe5-d279-4745-b58f-ad4a66d698db None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:33.583835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c368abe5-d279-4745-b58f-ad4a66d698db None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:33.589777 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2083/4182] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:33.590807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310e4db-2857-49c7-9452-455552163394 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:33.598146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd3cf3-eb81-44fa-87ba-055e9c521244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:33.598833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd3cf3-eb81-44fa-87ba-055e9c521244 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:33.599136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd3cf3-eb81-44fa-87ba-055e9c521244 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:33.599497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd3cf3-eb81-44fa-87ba-055e9c521244 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:33.600235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd3cf3-eb81-44fa-87ba-055e9c521244 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:15:33.600548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd3cf3-eb81-44fa-87ba-055e9c521244 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:33.602567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd3cf3-eb81-44fa-87ba-055e9c521244 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:33.605660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6310e4db-2857-49c7-9452-455552163394 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:33.613861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815bd9b-9912-415c-8304-529461ddad37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:33.613861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815bd9b-9912-415c-8304-529461ddad37 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:33.613861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815bd9b-9912-415c-8304-529461ddad37 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:33.613861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b815bd9b-9912-415c-8304-529461ddad37 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:33.614670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815bd9b-9912-415c-8304-529461ddad37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:33.616691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b815bd9b-9912-415c-8304-529461ddad37 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:33.621319 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2100/4183] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:33.626270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd3cf3-eb81-44fa-87ba-055e9c521244 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:33.640684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd3cf3-eb81-44fa-87ba-055e9c521244 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:33.653368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb84c9-922e-472e-9b6c-109b9d3ddc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:33.654275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb84c9-922e-472e-9b6c-109b9d3ddc40 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:33.654665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb84c9-922e-472e-9b6c-109b9d3ddc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:33.655119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb84c9-922e-472e-9b6c-109b9d3ddc40 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:33.655926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb84c9-922e-472e-9b6c-109b9d3ddc40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:33.658067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb84c9-922e-472e-9b6c-109b9d3ddc40 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:33.710827 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2084/4184] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:33 2022] POST /identity/v3/projects => generated 369 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:33.713283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815bd9b-9912-415c-8304-529461ddad37 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:33.777234 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2101/4185] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:33 2022] POST /identity/v3/projects => generated 373 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:33.782399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb84c9-922e-472e-9b6c-109b9d3ddc40 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:33.856544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b815bd9b-9912-415c-8304-529461ddad37 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:33.900353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb84c9-922e-472e-9b6c-109b9d3ddc40 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:33.908461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5233179-85f4-48b6-9243-e888f6cf76d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:33.909497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5233179-85f4-48b6-9243-e888f6cf76d3 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:33.910112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5233179-85f4-48b6-9243-e888f6cf76d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:33.911107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5233179-85f4-48b6-9243-e888f6cf76d3 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:33.912141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5233179-85f4-48b6-9243-e888f6cf76d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:33.914434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5233179-85f4-48b6-9243-e888f6cf76d3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:33.969874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b771cff-3e79-4e28-9457-12aa5539f3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:33.970895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b771cff-3e79-4e28-9457-12aa5539f3ca admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:33.971073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b771cff-3e79-4e28-9457-12aa5539f3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:33.971370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b771cff-3e79-4e28-9457-12aa5539f3ca admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:33.981819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b771cff-3e79-4e28-9457-12aa5539f3ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:33.984250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b771cff-3e79-4e28-9457-12aa5539f3ca admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:34.026536 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2102/4186] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:33 2022] POST /identity/v3/users => generated 287 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:15:34.030706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5233179-85f4-48b6-9243-e888f6cf76d3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:34.048304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5233179-85f4-48b6-9243-e888f6cf76d3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:34.056940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468f1e3-b4ed-4372-bc19-181ff44a36c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:34.057688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468f1e3-b4ed-4372-bc19-181ff44a36c8 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:34.059006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468f1e3-b4ed-4372-bc19-181ff44a36c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:34.059006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468f1e3-b4ed-4372-bc19-181ff44a36c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:34.061157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468f1e3-b4ed-4372-bc19-181ff44a36c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:34.062364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468f1e3-b4ed-4372-bc19-181ff44a36c8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:34.093948 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2085/4187] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:33 2022] POST /identity/v3/users => generated 285 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:34.094183 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2103/4188] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:15:34.096337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b771cff-3e79-4e28-9457-12aa5539f3ca admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:34.097972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468f1e3-b4ed-4372-bc19-181ff44a36c8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:34.109069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468f1e3-b4ed-4372-bc19-181ff44a36c8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:34.114641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edd2f8-efe0-4925-b126-9f9195ccad7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:34.115640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b771cff-3e79-4e28-9457-12aa5539f3ca admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:34.115842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edd2f8-efe0-4925-b126-9f9195ccad7b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:34.117016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edd2f8-efe0-4925-b126-9f9195ccad7b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:34.117491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0edd2f8-efe0-4925-b126-9f9195ccad7b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:34.118372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0edd2f8-efe0-4925-b126-9f9195ccad7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:34.120197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0edd2f8-efe0-4925-b126-9f9195ccad7b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:34.122777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410762b7-8ec7-43e3-9b63-fa42b17c184c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:34.123793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410762b7-8ec7-43e3-9b63-fa42b17c184c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:34.124648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410762b7-8ec7-43e3-9b63-fa42b17c184c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:34.125028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410762b7-8ec7-43e3-9b63-fa42b17c184c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:34.126365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410762b7-8ec7-43e3-9b63-fa42b17c184c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:34.128577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410762b7-8ec7-43e3-9b63-fa42b17c184c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:34.134180 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2104/4189] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:34.147786 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2086/4190] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:34.153001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410762b7-8ec7-43e3-9b63-fa42b17c184c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:34.153320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edd2f8-efe0-4925-b126-9f9195ccad7b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:34.170629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0edd2f8-efe0-4925-b126-9f9195ccad7b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:34.171535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410762b7-8ec7-43e3-9b63-fa42b17c184c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:34.177349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4518b-719b-44dc-8187-acaf36fab49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:34.177746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5edcb-9657-4678-bbd7-a17bf1c1c5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:34.178221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4518b-719b-44dc-8187-acaf36fab49b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:34.178563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4518b-719b-44dc-8187-acaf36fab49b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:34.178968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4518b-719b-44dc-8187-acaf36fab49b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:34.180323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4518b-719b-44dc-8187-acaf36fab49b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:34.180438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5edcb-9657-4678-bbd7-a17bf1c1c5b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:34.180888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5edcb-9657-4678-bbd7-a17bf1c1c5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:34.181360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5edcb-9657-4678-bbd7-a17bf1c1c5b9 admin admin] PATH_INFO: `/v3/projects/abf6ccfe1c384df69375ebfb4aebefbc/users/72685ce28f7e42b5b725946627ff5291/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:34.182681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4518b-719b-44dc-8187-acaf36fab49b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:34.200829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5edcb-9657-4678-bbd7-a17bf1c1c5b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abf6ccfe1c384df69375ebfb4aebefbc, user_id=72685ce28f7e42b5b725946627ff5291, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:34.204002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5edcb-9657-4678-bbd7-a17bf1c1c5b9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:34.212493 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2087/4191] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:34.214895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4518b-719b-44dc-8187-acaf36fab49b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:34.228220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4518b-719b-44dc-8187-acaf36fab49b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:34.235287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68f440-6a40-493b-a347-61354757c232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:34.236366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68f440-6a40-493b-a347-61354757c232 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:34.237118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68f440-6a40-493b-a347-61354757c232 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:34.237263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68f440-6a40-493b-a347-61354757c232 admin admin] PATH_INFO: `/v3/projects/f3f2ea38d3274d8e82ab37aded1a5c07/users/8a68a25f5db94edd8d0cf84c812207ea/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:34.256203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68f440-6a40-493b-a347-61354757c232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f2ea38d3274d8e82ab37aded1a5c07, user_id=8a68a25f5db94edd8d0cf84c812207ea, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:34.259381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68f440-6a40-493b-a347-61354757c232 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:34.317494 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2105/4192] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:34 2022] PUT /identity/v3/projects/abf6ccfe1c384df69375ebfb4aebefbc/users/72685ce28f7e42b5b725946627ff5291/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:34.319124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debf89a-f187-4972-99ef-d980383e2760 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:34.319512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debf89a-f187-4972-99ef-d980383e2760 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:34.319796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2debf89a-f187-4972-99ef-d980383e2760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:34.356845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2debf89a-f187-4972-99ef-d980383e2760 None None] MFA Rules not processed for user `72685ce28f7e42b5b725946627ff5291`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:34.358314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2debf89a-f187-4972-99ef-d980383e2760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:34.371345 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2088/4193] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:34 2022] PUT /identity/v3/projects/f3f2ea38d3274d8e82ab37aded1a5c07/users/8a68a25f5db94edd8d0cf84c812207ea/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:34.468284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff6f63-4b53-446a-a8db-fe6050ec963d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:34.468579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff6f63-4b53-446a-a8db-fe6050ec963d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:34.468860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff6f63-4b53-446a-a8db-fe6050ec963d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:34.519625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1bff6f63-4b53-446a-a8db-fe6050ec963d None None] MFA Rules not processed for user `8a68a25f5db94edd8d0cf84c812207ea`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:34.521060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1bff6f63-4b53-446a-a8db-fe6050ec963d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:34.529130 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2106/4194] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:34.533150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debf89a-f187-4972-99ef-d980383e2760 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:34.539013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2debf89a-f187-4972-99ef-d980383e2760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:34.634681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2debf89a-f187-4972-99ef-d980383e2760 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:34.640240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b355076-a3e6-41a2-87b3-83a74379d215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:34.641124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b355076-a3e6-41a2-87b3-83a74379d215 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:34.641416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b355076-a3e6-41a2-87b3-83a74379d215 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:34.641745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b355076-a3e6-41a2-87b3-83a74379d215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:34.654126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b355076-a3e6-41a2-87b3-83a74379d215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:34.658687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b355076-a3e6-41a2-87b3-83a74379d215 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:34.658824 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2089/4195] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:15:34.660629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff6f63-4b53-446a-a8db-fe6050ec963d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:34.668969 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2107/4196] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:34.780396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff6f63-4b53-446a-a8db-fe6050ec963d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:34.794708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62587e6-9153-4f58-8d19-65c9d297e69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:34.794708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62587e6-9153-4f58-8d19-65c9d297e69a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:34.794708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62587e6-9153-4f58-8d19-65c9d297e69a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:34.794708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62587e6-9153-4f58-8d19-65c9d297e69a admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:34.795022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62587e6-9153-4f58-8d19-65c9d297e69a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:34.795022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62587e6-9153-4f58-8d19-65c9d297e69a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:35.045889 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2090/4197] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:34 2022] POST /identity/v3/projects => generated 365 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:35.049494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b355076-a3e6-41a2-87b3-83a74379d215 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:35.083984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b355076-a3e6-41a2-87b3-83a74379d215 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:35.121282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:35.202462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:35.260928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43319ee4-16d6-4d83-8e02-a04349ad983b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:35.261664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43319ee4-16d6-4d83-8e02-a04349ad983b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:35.262125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43319ee4-16d6-4d83-8e02-a04349ad983b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:35.262420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43319ee4-16d6-4d83-8e02-a04349ad983b admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:35.263165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43319ee4-16d6-4d83-8e02-a04349ad983b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:35.265229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43319ee4-16d6-4d83-8e02-a04349ad983b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:35.450282 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2108/4198] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:35 2022] POST /identity/v3/users => generated 289 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:35.458824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62587e6-9153-4f58-8d19-65c9d297e69a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:35.524913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62587e6-9153-4f58-8d19-65c9d297e69a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:35.535439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2a326-2b4a-4d00-963a-f42d39a00a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:35.536160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2a326-2b4a-4d00-963a-f42d39a00a9d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:35.536530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2a326-2b4a-4d00-963a-f42d39a00a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:35.536819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2a326-2b4a-4d00-963a-f42d39a00a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:35.537616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2a326-2b4a-4d00-963a-f42d39a00a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:35.540470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2a326-2b4a-4d00-963a-f42d39a00a9d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:35.558064 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2091/4199] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:35.560804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43319ee4-16d6-4d83-8e02-a04349ad983b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:35.575454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43319ee4-16d6-4d83-8e02-a04349ad983b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:35.582666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78345b4a-95f7-4319-b7ec-653c4c871b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:35.583865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78345b4a-95f7-4319-b7ec-653c4c871b87 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:35.584266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78345b4a-95f7-4319-b7ec-653c4c871b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:35.585890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78345b4a-95f7-4319-b7ec-653c4c871b87 admin admin] PATH_INFO: `/v3/projects/cadf0ccd043e456281965885fac7747a/users/990e6e0dee3d4a61a484c34c8077f485/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:35.603748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78345b4a-95f7-4319-b7ec-653c4c871b87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cadf0ccd043e456281965885fac7747a, user_id=990e6e0dee3d4a61a484c34c8077f485, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:35.609181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78345b4a-95f7-4319-b7ec-653c4c871b87 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:35.703156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2a326-2b4a-4d00-963a-f42d39a00a9d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:35.714777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2a326-2b4a-4d00-963a-f42d39a00a9d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:35.720150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875d6bfc-b529-486b-a690-c88d21f56fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:35.721113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d6bfc-b529-486b-a690-c88d21f56fe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:35.721395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d6bfc-b529-486b-a690-c88d21f56fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:35.721781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875d6bfc-b529-486b-a690-c88d21f56fe6 admin admin] PATH_INFO: `/v3/users/72685ce28f7e42b5b725946627ff5291` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:35.723088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d6bfc-b529-486b-a690-c88d21f56fe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72685ce28f7e42b5b725946627ff5291)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:35.725070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875d6bfc-b529-486b-a690-c88d21f56fe6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:35.754611 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2109/4200] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:35 2022] PUT /identity/v3/projects/cadf0ccd043e456281965885fac7747a/users/990e6e0dee3d4a61a484c34c8077f485/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:35.755739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78345b4a-95f7-4319-b7ec-653c4c871b87 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:35.862784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78345b4a-95f7-4319-b7ec-653c4c871b87 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:35.868809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015ef8a-9f91-47eb-b8ff-c1c25e40ce5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:35.873118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015ef8a-9f91-47eb-b8ff-c1c25e40ce5d admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:35.873118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015ef8a-9f91-47eb-b8ff-c1c25e40ce5d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:35.873118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8015ef8a-9f91-47eb-b8ff-c1c25e40ce5d admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:35.873118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015ef8a-9f91-47eb-b8ff-c1c25e40ce5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:35.873118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8015ef8a-9f91-47eb-b8ff-c1c25e40ce5d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:35.884693 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2110/4201] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:35.889077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015ef8a-9f91-47eb-b8ff-c1c25e40ce5d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:35.903869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8015ef8a-9f91-47eb-b8ff-c1c25e40ce5d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:35.910181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424057d9-1d59-4665-871b-0d293abe4490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:35.911466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424057d9-1d59-4665-871b-0d293abe4490 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:35.911957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424057d9-1d59-4665-871b-0d293abe4490 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:35.912394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424057d9-1d59-4665-871b-0d293abe4490 admin admin] PATH_INFO: `/v3/projects/cadf0ccd043e456281965885fac7747a/users/990e6e0dee3d4a61a484c34c8077f485/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:35.915237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424057d9-1d59-4665-871b-0d293abe4490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cadf0ccd043e456281965885fac7747a, user_id=990e6e0dee3d4a61a484c34c8077f485, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:35.919013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424057d9-1d59-4665-871b-0d293abe4490 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:35.934779 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2111/4202] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:35 2022] PUT /identity/v3/projects/cadf0ccd043e456281965885fac7747a/users/990e6e0dee3d4a61a484c34c8077f485/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:35.951503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41aa5db-bdfd-4ab4-b291-5b90ba37491c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:35.951734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41aa5db-bdfd-4ab4-b291-5b90ba37491c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:35.952018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41aa5db-bdfd-4ab4-b291-5b90ba37491c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:36.004534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e41aa5db-bdfd-4ab4-b291-5b90ba37491c None None] MFA Rules not processed for user `990e6e0dee3d4a61a484c34c8077f485`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:36.006504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e41aa5db-bdfd-4ab4-b291-5b90ba37491c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:36.078793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-875d6bfc-b529-486b-a690-c88d21f56fe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72685ce28f7e42b5b725946627ff5291'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:36.079838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-875d6bfc-b529-486b-a690-c88d21f56fe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72685ce28f7e42b5b725946627ff5291'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:36.109621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-875d6bfc-b529-486b-a690-c88d21f56fe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72685ce28f7e42b5b725946627ff5291'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:36.128100 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2112/4203] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:35 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:15:36.133492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41aa5db-bdfd-4ab4-b291-5b90ba37491c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:36.147605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41aa5db-bdfd-4ab4-b291-5b90ba37491c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:36.155200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f2ec7-ce0f-4852-af54-50106703fec6 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990e6e0dee3d4a61a484c34c8077f485', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cadf0ccd043e456281965885fac7747a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:36.156152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f2ec7-ce0f-4852-af54-50106703fec6 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:36.156796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f2ec7-ce0f-4852-af54-50106703fec6 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:36.157124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f2ec7-ce0f-4852-af54-50106703fec6 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:36.158141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f2ec7-ce0f-4852-af54-50106703fec6 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:36.160738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-875d6bfc-b529-486b-a690-c88d21f56fe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72685ce28f7e42b5b725946627ff5291'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:36.161584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-875d6bfc-b529-486b-a690-c88d21f56fe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72685ce28f7e42b5b725946627ff5291'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:36.167677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f2ec7-ce0f-4852-af54-50106703fec6 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:36.197543 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2092/4204] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:15:35 2022] DELETE /identity/v3/users/72685ce28f7e42b5b725946627ff5291 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:36.203007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f272f-4b1a-4083-8497-634c0b029848 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:36.203185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f272f-4b1a-4083-8497-634c0b029848 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:36.204845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575f272f-4b1a-4083-8497-634c0b029848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:36.249270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-575f272f-4b1a-4083-8497-634c0b029848 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:36.252086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-575f272f-4b1a-4083-8497-634c0b029848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:36.276812 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2113/4205] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:15:36 2022] POST /identity/v3/services => generated 239 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:36.282108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f2ec7-ce0f-4852-af54-50106703fec6 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:36.285655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-434f2ec7-ce0f-4852-af54-50106703fec6 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:36.336206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f2ec7-ce0f-4852-af54-50106703fec6 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:36.441445 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2093/4206] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:36 2022] POST /identity/v3/auth/tokens => generated 3361 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:15:36.456712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f272f-4b1a-4083-8497-634c0b029848 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:36.460104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-575f272f-4b1a-4083-8497-634c0b029848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:36.495408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d38ef-f9da-4b64-9fe5-11b430f6209f tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990e6e0dee3d4a61a484c34c8077f485', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cadf0ccd043e456281965885fac7747a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:36.496450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d38ef-f9da-4b64-9fe5-11b430f6209f tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:36.496578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d38ef-f9da-4b64-9fe5-11b430f6209f tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:36.497280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d38ef-f9da-4b64-9fe5-11b430f6209f tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] PATH_INFO: `/v3/services/9cb2ac794e5a42889ae1713d6d913de8` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:36.498066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d38ef-f9da-4b64-9fe5-11b430f6209f tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorizing `identity:delete_service(service_id=9cb2ac794e5a42889ae1713d6d913de8)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:36.499219 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-7d2d38ef-f9da-4b64-9fe5-11b430f6209f tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:15:36.499219 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:15:36.499219 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:15:36.500054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d38ef-f9da-4b64-9fe5-11b430f6209f tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:36.535077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:36.602665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:36.638783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575f272f-4b1a-4083-8497-634c0b029848 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:36.649044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5e040-3fbd-40d6-8059-70ea6a6a55a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:36.649435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5e040-3fbd-40d6-8059-70ea6a6a55a8 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:36.649690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5e040-3fbd-40d6-8059-70ea6a6a55a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:36.650554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5e040-3fbd-40d6-8059-70ea6a6a55a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:36.662337 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2114/4207] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:36 2022] DELETE /identity/v3/services/9cb2ac794e5a42889ae1713d6d913de8 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:36.669641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d38ef-f9da-4b64-9fe5-11b430f6209f tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:36.701573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5e040-3fbd-40d6-8059-70ea6a6a55a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:36.703753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5e040-3fbd-40d6-8059-70ea6a6a55a8 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:36.718971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d38ef-f9da-4b64-9fe5-11b430f6209f tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:36.722713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c9b17-2a41-4bb3-a18d-ef42418c0524 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990e6e0dee3d4a61a484c34c8077f485', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cadf0ccd043e456281965885fac7747a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:36.724109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c9b17-2a41-4bb3-a18d-ef42418c0524 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:36.724627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c9b17-2a41-4bb3-a18d-ef42418c0524 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:36.725208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122c9b17-2a41-4bb3-a18d-ef42418c0524 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:36.726732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c9b17-2a41-4bb3-a18d-ef42418c0524 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:36.729013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122c9b17-2a41-4bb3-a18d-ef42418c0524 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:36.733144 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2094/4208] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:36 2022] GET /identity/v3/auth/tokens => generated 3361 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:15:36.840296 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2115/4209] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:36 2022] POST /identity/v3/services => generated 290 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:36.841599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5e040-3fbd-40d6-8059-70ea6a6a55a8 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:36.856288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5e040-3fbd-40d6-8059-70ea6a6a55a8 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:36.877745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76305f9-a88a-42c0-be90-fa2541c7fb20 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990e6e0dee3d4a61a484c34c8077f485', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cadf0ccd043e456281965885fac7747a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:36.878833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76305f9-a88a-42c0-be90-fa2541c7fb20 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:36.879297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76305f9-a88a-42c0-be90-fa2541c7fb20 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:36.879809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76305f9-a88a-42c0-be90-fa2541c7fb20 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] PATH_INFO: `/v3/services/a7a8221362b44a8fac02b056e3c344a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:36.880717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76305f9-a88a-42c0-be90-fa2541c7fb20 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorizing `identity:update_service(service_id=a7a8221362b44a8fac02b056e3c344a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:36.882221 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-d76305f9-a88a-42c0-be90-fa2541c7fb20 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:15:36.882221 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:15:36.882221 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:15:36.883604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76305f9-a88a-42c0-be90-fa2541c7fb20 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:36.998585 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2095/4210] 10.222.0.33 () {60 vars in 1251 bytes} [Sat May 28 21:15:36 2022] PATCH /identity/v3/services/a7a8221362b44a8fac02b056e3c344a5 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:37.002415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c9b17-2a41-4bb3-a18d-ef42418c0524 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:37.018341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122c9b17-2a41-4bb3-a18d-ef42418c0524 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:37.041274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df640b5-4c52-4eeb-8845-825eef5c37f1 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990e6e0dee3d4a61a484c34c8077f485', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cadf0ccd043e456281965885fac7747a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:37.043776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df640b5-4c52-4eeb-8845-825eef5c37f1 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:37.044366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df640b5-4c52-4eeb-8845-825eef5c37f1 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:37.044932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df640b5-4c52-4eeb-8845-825eef5c37f1 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] PATH_INFO: `/v3/services/a7a8221362b44a8fac02b056e3c344a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:37.046476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df640b5-4c52-4eeb-8845-825eef5c37f1 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorizing `identity:get_service(service_id=a7a8221362b44a8fac02b056e3c344a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:37.048117 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-2df640b5-4c52-4eeb-8845-825eef5c37f1 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_service": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:15:37.048117 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:15:37.048117 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:15:37.049638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df640b5-4c52-4eeb-8845-825eef5c37f1 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:37.080263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76305f9-a88a-42c0-be90-fa2541c7fb20 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:37.080724 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2116/4211] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:15:37 2022] GET /identity/v3/services/a7a8221362b44a8fac02b056e3c344a5 => generated 283 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:37.087909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76305f9-a88a-42c0-be90-fa2541c7fb20 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:37.105975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddce391-2311-43d7-89dc-ec8a6c12a286 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990e6e0dee3d4a61a484c34c8077f485', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cadf0ccd043e456281965885fac7747a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:37.106867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddce391-2311-43d7-89dc-ec8a6c12a286 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:37.107160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddce391-2311-43d7-89dc-ec8a6c12a286 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:37.107439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddce391-2311-43d7-89dc-ec8a6c12a286 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] PATH_INFO: `/v3/services/a7a8221362b44a8fac02b056e3c344a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:37.108566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddce391-2311-43d7-89dc-ec8a6c12a286 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorizing `identity:delete_service(service_id=a7a8221362b44a8fac02b056e3c344a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:37.110297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddce391-2311-43d7-89dc-ec8a6c12a286 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:37.254890 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2096/4212] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:37 2022] DELETE /identity/v3/services/a7a8221362b44a8fac02b056e3c344a5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:15:37.258214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:37.265614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df640b5-4c52-4eeb-8845-825eef5c37f1 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:37.280013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df640b5-4c52-4eeb-8845-825eef5c37f1 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:37.309679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:37.316721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eee44d-f1f2-47c9-8d12-bf58a93ef0f9 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990e6e0dee3d4a61a484c34c8077f485', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cadf0ccd043e456281965885fac7747a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:37.317749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eee44d-f1f2-47c9-8d12-bf58a93ef0f9 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:37.319794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eee44d-f1f2-47c9-8d12-bf58a93ef0f9 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:37.319794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eee44d-f1f2-47c9-8d12-bf58a93ef0f9 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] PATH_INFO: `/v3/services/a7a8221362b44a8fac02b056e3c344a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:37.321769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eee44d-f1f2-47c9-8d12-bf58a93ef0f9 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorizing `identity:get_service(service_id=a7a8221362b44a8fac02b056e3c344a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:37.324291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eee44d-f1f2-47c9-8d12-bf58a93ef0f9 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:37.344983 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-a0eee44d-f1f2-47c9-8d12-bf58a93ef0f9 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Could not find service: a7a8221362b44a8fac02b056e3c344a5.: keystone.exception.ServiceNotFound: Could not find service: a7a8221362b44a8fac02b056e3c344a5. May 28 21:15:37.344983 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:15:37.344983 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 28 21:15:37.344983 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 28 21:15:37.344983 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 28 21:15:37.344983 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 28 21:15:37.344983 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 28 21:15:37.344983 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 21:15:37.344983 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: a7a8221362b44a8fac02b056e3c344a5. May 28 21:15:37.344983 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:15:37.344983 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:15:37.344983 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:15:37.344983 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:15:37.344983 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:15:37.344983 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:15:37.344983 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:15:37.346308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:15:37.346308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:15:37.346308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:15:37.346308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:15:37.346308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:15:37.346308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:15:37.346308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:15:37.346308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 28 21:15:37.346308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_service(service_id) May 28 21:15:37.346308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 28 21:15:37.346308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 28 21:15:37.346308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:15:37.346308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:15:37.346308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:15:37.346308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:15:37.346308 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:15:37.347501 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:15:37.347501 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:15:37.347501 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 21:15:37.347501 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:15:37.347501 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 21:15:37.347501 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:15:37.347501 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:15:37.347501 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:15:37.347501 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 21:15:37.347501 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:15:37.347501 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 21:15:37.347501 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 28 21:15:37.347501 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 28 21:15:37.347501 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: a7a8221362b44a8fac02b056e3c344a5. May 28 21:15:37.347501 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:15:37.347501 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2117/4213] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:15:37 2022] GET /identity/v3/services/a7a8221362b44a8fac02b056e3c344a5 => generated 113 bytes in 81 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:15:37.351685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddce391-2311-43d7-89dc-ec8a6c12a286 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:37.370127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddce391-2311-43d7-89dc-ec8a6c12a286 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:37.379710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09f55c-1823-478b-8038-fcf51e509d82 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990e6e0dee3d4a61a484c34c8077f485', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cadf0ccd043e456281965885fac7747a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:37.380410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09f55c-1823-478b-8038-fcf51e509d82 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:37.380697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09f55c-1823-478b-8038-fcf51e509d82 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:37.381195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe09f55c-1823-478b-8038-fcf51e509d82 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:37.382023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09f55c-1823-478b-8038-fcf51e509d82 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:37.383401 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-fe09f55c-1823-478b-8038-fcf51e509d82 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_service": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:15:37.383401 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:15:37.383401 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:15:37.384286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe09f55c-1823-478b-8038-fcf51e509d82 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:37.501738 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2097/4214] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:37 2022] POST /identity/v3/services => generated 274 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:37.506531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eee44d-f1f2-47c9-8d12-bf58a93ef0f9 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:37.518060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eee44d-f1f2-47c9-8d12-bf58a93ef0f9 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:37.540035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995c1dc-85a8-499c-816b-ba7e6e7b8ac8 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990e6e0dee3d4a61a484c34c8077f485', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cadf0ccd043e456281965885fac7747a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:37.541511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995c1dc-85a8-499c-816b-ba7e6e7b8ac8 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:37.542093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995c1dc-85a8-499c-816b-ba7e6e7b8ac8 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:37.547718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995c1dc-85a8-499c-816b-ba7e6e7b8ac8 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:37.547718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6995c1dc-85a8-499c-816b-ba7e6e7b8ac8 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:37.547718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6995c1dc-85a8-499c-816b-ba7e6e7b8ac8 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:37.575269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09f55c-1823-478b-8038-fcf51e509d82 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:37.577353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fe09f55c-1823-478b-8038-fcf51e509d82 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:37.635769 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2118/4215] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:37 2022] POST /identity/v3/services => generated 274 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:37.636251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe09f55c-1823-478b-8038-fcf51e509d82 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:37.636520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995c1dc-85a8-499c-816b-ba7e6e7b8ac8 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:37.658779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b9e6d-d08e-4e77-96d8-5b665679ffee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:37.659719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b9e6d-d08e-4e77-96d8-5b665679ffee admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:37.660003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b9e6d-d08e-4e77-96d8-5b665679ffee admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:37.660284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b9e6d-d08e-4e77-96d8-5b665679ffee admin admin] PATH_INFO: `/v3/projects/abf6ccfe1c384df69375ebfb4aebefbc` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:37.661490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b9e6d-d08e-4e77-96d8-5b665679ffee admin admin] RBAC: Authorizing `identity:delete_project(project_id=abf6ccfe1c384df69375ebfb4aebefbc)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:37.664405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b9e6d-d08e-4e77-96d8-5b665679ffee admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:37.674869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995c1dc-85a8-499c-816b-ba7e6e7b8ac8 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:37.683338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ab491-73cb-4433-a15f-c1a7a98bb4f8 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990e6e0dee3d4a61a484c34c8077f485', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cadf0ccd043e456281965885fac7747a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:37.684522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ab491-73cb-4433-a15f-c1a7a98bb4f8 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:37.685107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ab491-73cb-4433-a15f-c1a7a98bb4f8 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:37.685548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ab491-73cb-4433-a15f-c1a7a98bb4f8 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:37.686464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ab491-73cb-4433-a15f-c1a7a98bb4f8 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:37.688393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ab491-73cb-4433-a15f-c1a7a98bb4f8 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:37.792141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3c8b9e6d-d08e-4e77-96d8-5b665679ffee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abf6ccfe1c384df69375ebfb4aebefbc', 'request_id': 'req-3c8b9e6d-d08e-4e77-96d8-5b665679ffee'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:37.798915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3c8b9e6d-d08e-4e77-96d8-5b665679ffee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abf6ccfe1c384df69375ebfb4aebefbc', 'request_id': 'req-3c8b9e6d-d08e-4e77-96d8-5b665679ffee'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:37.812694 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2119/4216] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:37 2022] POST /identity/v3/services => generated 275 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:37.813809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ab491-73cb-4433-a15f-c1a7a98bb4f8 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:37.852424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ab491-73cb-4433-a15f-c1a7a98bb4f8 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:37.892566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3c8b9e6d-d08e-4e77-96d8-5b665679ffee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abf6ccfe1c384df69375ebfb4aebefbc', 'request_id': 'req-3c8b9e6d-d08e-4e77-96d8-5b665679ffee'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:37.904017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:37.908666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3c8b9e6d-d08e-4e77-96d8-5b665679ffee admin admin] The token cache is being invalidate because project abf6ccfe1c384df69375ebfb4aebefbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:15:37.908666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3c8b9e6d-d08e-4e77-96d8-5b665679ffee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:37.909234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3c8b9e6d-d08e-4e77-96d8-5b665679ffee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:37.919183 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2098/4217] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:37 2022] DELETE /identity/v3/projects/abf6ccfe1c384df69375ebfb4aebefbc => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:15:37.925164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5a9a6-add4-4509-8012-d4fff24b4cae None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:37.926533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5a9a6-add4-4509-8012-d4fff24b4cae None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:37.927241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5a9a6-add4-4509-8012-d4fff24b4cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:37.952435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe9e1d-2be7-4ab0-92f4-789714f5f985 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990e6e0dee3d4a61a484c34c8077f485', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cadf0ccd043e456281965885fac7747a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:37.953941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe9e1d-2be7-4ab0-92f4-789714f5f985 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:37.954381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe9e1d-2be7-4ab0-92f4-789714f5f985 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:37.954766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfe9e1d-2be7-4ab0-92f4-789714f5f985 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:37.955904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe9e1d-2be7-4ab0-92f4-789714f5f985 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:37.956910 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-7dfe9e1d-2be7-4ab0-92f4-789714f5f985 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_services": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:15:37.956910 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:15:37.956910 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:15:37.958245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfe9e1d-2be7-4ab0-92f4-789714f5f985 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:37.968754 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2120/4218] 10.222.0.33 () {58 vars in 1130 bytes} [Sat May 28 21:15:37 2022] GET /identity/v3/services => generated 2978 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:37.973285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c1b5a9a6-add4-4509-8012-d4fff24b4cae None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:37.973457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe9e1d-2be7-4ab0-92f4-789714f5f985 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:37.976427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c1b5a9a6-add4-4509-8012-d4fff24b4cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:37.986020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7dfe9e1d-2be7-4ab0-92f4-789714f5f985 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:38.000289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfe9e1d-2be7-4ab0-92f4-789714f5f985 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:38.009240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ee0cc-85bd-4b36-b1b0-ac4098f354c5 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990e6e0dee3d4a61a484c34c8077f485', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cadf0ccd043e456281965885fac7747a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:38.014388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ee0cc-85bd-4b36-b1b0-ac4098f354c5 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:38.014838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ee0cc-85bd-4b36-b1b0-ac4098f354c5 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:38.015226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ee0cc-85bd-4b36-b1b0-ac4098f354c5 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:38.017031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ee0cc-85bd-4b36-b1b0-ac4098f354c5 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-378036887) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:15:38.018493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ee0cc-85bd-4b36-b1b0-ac4098f354c5 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:38.020228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ee0cc-85bd-4b36-b1b0-ac4098f354c5 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:38.036638 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2121/4219] 10.222.0.33 () {58 vars in 1219 bytes} [Sat May 28 21:15:37 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-378036887 => generated 416 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:38.038083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ee0cc-85bd-4b36-b1b0-ac4098f354c5 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:38.064143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ee0cc-85bd-4b36-b1b0-ac4098f354c5 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:38.071362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233f88c-46d6-4104-bc62-db606c46170c tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990e6e0dee3d4a61a484c34c8077f485', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cadf0ccd043e456281965885fac7747a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:38.072055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233f88c-46d6-4104-bc62-db606c46170c tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:38.072348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233f88c-46d6-4104-bc62-db606c46170c tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:38.072631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233f88c-46d6-4104-bc62-db606c46170c tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:38.073777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233f88c-46d6-4104-bc62-db606c46170c tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-798569035) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:15:38.074132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233f88c-46d6-4104-bc62-db606c46170c tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:38.086914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233f88c-46d6-4104-bc62-db606c46170c tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:38.093487 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2122/4220] 10.222.0.33 () {58 vars in 1219 bytes} [Sat May 28 21:15:38 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-798569035 => generated 416 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:38.098395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233f88c-46d6-4104-bc62-db606c46170c tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:38.114500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233f88c-46d6-4104-bc62-db606c46170c tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:38.131121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:UPDATE=1,keystone:DELETE=9,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:38.131121 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2099/4221] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:37 2022] POST /identity/v3/auth/tokens => generated 3731 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:38.133038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcc262-75a6-4bd3-8de9-9fb5437175b7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:38.133328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcc262-75a6-4bd3-8de9-9fb5437175b7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:38.133605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedcc262-75a6-4bd3-8de9-9fb5437175b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:38.133901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612f4d9-4fe8-47db-aebd-a06fb2b135e3 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990e6e0dee3d4a61a484c34c8077f485', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cadf0ccd043e456281965885fac7747a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:38.134598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612f4d9-4fe8-47db-aebd-a06fb2b135e3 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:38.134889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612f4d9-4fe8-47db-aebd-a06fb2b135e3 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:38.135171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9612f4d9-4fe8-47db-aebd-a06fb2b135e3 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:38.137788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612f4d9-4fe8-47db-aebd-a06fb2b135e3 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Adding query filter params (type=tempest-ServicesTestJSON-Type-1335316529) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:15:38.138157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612f4d9-4fe8-47db-aebd-a06fb2b135e3 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:38.139800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9612f4d9-4fe8-47db-aebd-a06fb2b135e3 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:38.151355 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2123/4222] 10.222.0.33 () {58 vars in 1221 bytes} [Sat May 28 21:15:38 2022] GET /identity/v3/services?type=tempest-ServicesTestJSON-Type-1335316529 => generated 418 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:38.157476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612f4d9-4fe8-47db-aebd-a06fb2b135e3 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:38.170083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-aedcc262-75a6-4bd3-8de9-9fb5437175b7 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:38.172072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-aedcc262-75a6-4bd3-8de9-9fb5437175b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:38.176669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9612f4d9-4fe8-47db-aebd-a06fb2b135e3 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:38.182110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7faef-ffe8-4b65-a857-a3e6b4de8443 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990e6e0dee3d4a61a484c34c8077f485', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cadf0ccd043e456281965885fac7747a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:38.184343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7faef-ffe8-4b65-a857-a3e6b4de8443 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:38.184473 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2100/4223] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:38 2022] POST /identity/v3/auth/tokens => generated 3731 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:38.184572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7faef-ffe8-4b65-a857-a3e6b4de8443 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:38.184572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e7faef-ffe8-4b65-a857-a3e6b4de8443 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] PATH_INFO: `/v3/services/000a4fb70f124f45a59beab6559bf01c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:38.184572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e7faef-ffe8-4b65-a857-a3e6b4de8443 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorizing `identity:delete_service(service_id=000a4fb70f124f45a59beab6559bf01c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:38.190372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e7faef-ffe8-4b65-a857-a3e6b4de8443 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:38.191581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcc262-75a6-4bd3-8de9-9fb5437175b7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:38.203348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedcc262-75a6-4bd3-8de9-9fb5437175b7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:38.209576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02fad3-b597-47ab-bd94-d1294325fc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:38.211526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02fad3-b597-47ab-bd94-d1294325fc42 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:38.211526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02fad3-b597-47ab-bd94-d1294325fc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:38.211526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02fad3-b597-47ab-bd94-d1294325fc42 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:38.212189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02fad3-b597-47ab-bd94-d1294325fc42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:15:38.212508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02fad3-b597-47ab-bd94-d1294325fc42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:38.214391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02fad3-b597-47ab-bd94-d1294325fc42 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:38.235785 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2101/4224] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:15:38.236965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02fad3-b597-47ab-bd94-d1294325fc42 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:38.249047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02fad3-b597-47ab-bd94-d1294325fc42 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:38.255050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6283e7e-495a-4d2b-a76f-42dc4620bcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:38.255924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6283e7e-495a-4d2b-a76f-42dc4620bcc7 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:38.256219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6283e7e-495a-4d2b-a76f-42dc4620bcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:38.256487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6283e7e-495a-4d2b-a76f-42dc4620bcc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:38.258041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6283e7e-495a-4d2b-a76f-42dc4620bcc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:38.260250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6283e7e-495a-4d2b-a76f-42dc4620bcc7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:38.274463 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2124/4225] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:38 2022] DELETE /identity/v3/services/000a4fb70f124f45a59beab6559bf01c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:38.280447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7faef-ffe8-4b65-a857-a3e6b4de8443 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:38.335502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e7faef-ffe8-4b65-a857-a3e6b4de8443 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:38.343768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8eec0-834b-484d-9625-56a0130202bb tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990e6e0dee3d4a61a484c34c8077f485', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cadf0ccd043e456281965885fac7747a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:38.344819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8eec0-834b-484d-9625-56a0130202bb tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:38.345202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8eec0-834b-484d-9625-56a0130202bb tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:38.345560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8eec0-834b-484d-9625-56a0130202bb tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] PATH_INFO: `/v3/services/79b4e548322b47fe8b960f897ca222a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:38.346755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8eec0-834b-484d-9625-56a0130202bb tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorizing `identity:delete_service(service_id=79b4e548322b47fe8b960f897ca222a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:38.348652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e8eec0-834b-484d-9625-56a0130202bb tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:38.363271 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2102/4226] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:38 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:38.368085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6283e7e-495a-4d2b-a76f-42dc4620bcc7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:38.442046 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2125/4227] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:38 2022] DELETE /identity/v3/services/79b4e548322b47fe8b960f897ca222a5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:15:38.449486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8eec0-834b-484d-9625-56a0130202bb tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:38.477812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6283e7e-495a-4d2b-a76f-42dc4620bcc7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:38.500252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e8eec0-834b-484d-9625-56a0130202bb tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:38.504454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909aef53-7df0-4e88-a023-8ae3659aa6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:38.505954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909aef53-7df0-4e88-a023-8ae3659aa6b8 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:38.505954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909aef53-7df0-4e88-a023-8ae3659aa6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:38.505954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909aef53-7df0-4e88-a023-8ae3659aa6b8 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:38.506483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909aef53-7df0-4e88-a023-8ae3659aa6b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:38.509044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909aef53-7df0-4e88-a023-8ae3659aa6b8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:38.519809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:38.572420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ceda5-1822-490a-bb99-6bbb0eba2909 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '990e6e0dee3d4a61a484c34c8077f485', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cadf0ccd043e456281965885fac7747a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:38.573419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ceda5-1822-490a-bb99-6bbb0eba2909 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:38.573779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ceda5-1822-490a-bb99-6bbb0eba2909 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:38.574164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ceda5-1822-490a-bb99-6bbb0eba2909 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] PATH_INFO: `/v3/services/d32272cbcae14b63a8235dea28e52c36` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:38.574992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ceda5-1822-490a-bb99-6bbb0eba2909 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorizing `identity:delete_service(service_id=d32272cbcae14b63a8235dea28e52c36)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:38.577823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39ceda5-1822-490a-bb99-6bbb0eba2909 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:38.628168 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2103/4228] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:38 2022] POST /identity/v3/users => generated 291 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:38.630855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909aef53-7df0-4e88-a023-8ae3659aa6b8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:38.643325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909aef53-7df0-4e88-a023-8ae3659aa6b8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:38.649280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b7190-fc3f-405e-9284-d43a22653827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:38.650098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b7190-fc3f-405e-9284-d43a22653827 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:38.650459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b7190-fc3f-405e-9284-d43a22653827 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:38.650873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054b7190-fc3f-405e-9284-d43a22653827 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:38.651824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b7190-fc3f-405e-9284-d43a22653827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:38.653515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054b7190-fc3f-405e-9284-d43a22653827 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:38.671161 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2126/4229] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:38 2022] DELETE /identity/v3/services/d32272cbcae14b63a8235dea28e52c36 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:15:38.677021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ceda5-1822-490a-bb99-6bbb0eba2909 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:38.679829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a39ceda5-1822-490a-bb99-6bbb0eba2909 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:38.692317 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2104/4230] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:38.698516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b7190-fc3f-405e-9284-d43a22653827 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:38.712972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054b7190-fc3f-405e-9284-d43a22653827 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:38.732039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967cab1-21b4-44a4-bdac-cf89c8ee4e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:38.732584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ceda5-1822-490a-bb99-6bbb0eba2909 tempest-ServicesTestJSON-40365605 tempest-ServicesTestJSON-40365605-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:38.732859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967cab1-21b4-44a4-bdac-cf89c8ee4e1c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:38.733148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967cab1-21b4-44a4-bdac-cf89c8ee4e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:38.733427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967cab1-21b4-44a4-bdac-cf89c8ee4e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:38.734464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967cab1-21b4-44a4-bdac-cf89c8ee4e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:38.736101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967cab1-21b4-44a4-bdac-cf89c8ee4e1c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:38.748238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90495b12-6ac2-47c6-8853-bb19b9c3d462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:38.756319 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2105/4231] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:38 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:38.757401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967cab1-21b4-44a4-bdac-cf89c8ee4e1c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:38.760307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90495b12-6ac2-47c6-8853-bb19b9c3d462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:38.760605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90495b12-6ac2-47c6-8853-bb19b9c3d462 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:38.760892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90495b12-6ac2-47c6-8853-bb19b9c3d462 admin admin] PATH_INFO: `/v3/users/8a68a25f5db94edd8d0cf84c812207ea` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:38.764688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90495b12-6ac2-47c6-8853-bb19b9c3d462 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a68a25f5db94edd8d0cf84c812207ea)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:38.764688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90495b12-6ac2-47c6-8853-bb19b9c3d462 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:38.769883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967cab1-21b4-44a4-bdac-cf89c8ee4e1c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:38.775657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c79ce-4d69-4dd4-926b-e991489dd6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:38.777051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c79ce-4d69-4dd4-926b-e991489dd6c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:38.777345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c79ce-4d69-4dd4-926b-e991489dd6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:38.777624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c79ce-4d69-4dd4-926b-e991489dd6c6 admin admin] PATH_INFO: `/v3/projects/b04e5d3e331c4c45b309b5041a99b681/users/fcfb9ccd45dd44bb9747fb68ac912c7c/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:38.795806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c79ce-4d69-4dd4-926b-e991489dd6c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04e5d3e331c4c45b309b5041a99b681, user_id=fcfb9ccd45dd44bb9747fb68ac912c7c, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:38.801276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c79ce-4d69-4dd4-926b-e991489dd6c6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:38.905920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:38.912158 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2106/4232] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:38 2022] PUT /identity/v3/projects/b04e5d3e331c4c45b309b5041a99b681/users/fcfb9ccd45dd44bb9747fb68ac912c7c/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:38.912997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c79ce-4d69-4dd4-926b-e991489dd6c6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:38.918713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-128c79ce-4d69-4dd4-926b-e991489dd6c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:38.985335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-90495b12-6ac2-47c6-8853-bb19b9c3d462 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a68a25f5db94edd8d0cf84c812207ea'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:39.003973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-90495b12-6ac2-47c6-8853-bb19b9c3d462 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a68a25f5db94edd8d0cf84c812207ea'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:39.005493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-90495b12-6ac2-47c6-8853-bb19b9c3d462 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a68a25f5db94edd8d0cf84c812207ea'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:39.007295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-90495b12-6ac2-47c6-8853-bb19b9c3d462 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a68a25f5db94edd8d0cf84c812207ea'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:39.105280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-90495b12-6ac2-47c6-8853-bb19b9c3d462 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a68a25f5db94edd8d0cf84c812207ea'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:39.130771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c79ce-4d69-4dd4-926b-e991489dd6c6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:39.134455 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2127/4233] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:15:38 2022] DELETE /identity/v3/users/8a68a25f5db94edd8d0cf84c812207ea => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:15:39.142342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177f88d-2771-4528-aa32-7a1a085fbf19 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:39.142342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177f88d-2771-4528-aa32-7a1a085fbf19 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:39.142342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1177f88d-2771-4528-aa32-7a1a085fbf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:39.142650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0098a-9aec-4a28-b748-9d6c9468862c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:39.142650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0098a-9aec-4a28-b748-9d6c9468862c service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:39.142650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0098a-9aec-4a28-b748-9d6c9468862c service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:39.142650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0098a-9aec-4a28-b748-9d6c9468862c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:39.143309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b1d0098a-9aec-4a28-b748-9d6c9468862c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:39.184902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1177f88d-2771-4528-aa32-7a1a085fbf19 None None] MFA Rules not processed for user `fcfb9ccd45dd44bb9747fb68ac912c7c`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:39.185204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0098a-9aec-4a28-b748-9d6c9468862c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:39.186905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d0098a-9aec-4a28-b748-9d6c9468862c service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:39.187569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1177f88d-2771-4528-aa32-7a1a085fbf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:39.246647 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2107/4234] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:39.250230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cd4d7-b974-443c-8982-b8b1de01223d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:39.251853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cd4d7-b974-443c-8982-b8b1de01223d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:39.252590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057cd4d7-b974-443c-8982-b8b1de01223d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:39.300038 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2128/4235] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:39.303951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:39.314555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-057cd4d7-b974-443c-8982-b8b1de01223d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:39.315917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177f88d-2771-4528-aa32-7a1a085fbf19 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:39.317891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-057cd4d7-b974-443c-8982-b8b1de01223d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:39.322063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1177f88d-2771-4528-aa32-7a1a085fbf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:39.473496 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2108/4236] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:39.484456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cd4d7-b974-443c-8982-b8b1de01223d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:39.485921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-057cd4d7-b974-443c-8982-b8b1de01223d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:39.512907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1177f88d-2771-4528-aa32-7a1a085fbf19 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:39.527136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d1a3b-6a00-4479-964b-ef8e5036c96a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:39.528094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d1a3b-6a00-4479-964b-ef8e5036c96a service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:39.528245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d1a3b-6a00-4479-964b-ef8e5036c96a service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:39.528535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d1a3b-6a00-4479-964b-ef8e5036c96a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:39.529675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057cd4d7-b974-443c-8982-b8b1de01223d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:39.543846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f7031-0202-4ec0-8f4d-ab516db369f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:39.544521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f7031-0202-4ec0-8f4d-ab516db369f6 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:39.547132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f7031-0202-4ec0-8f4d-ab516db369f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:39.547132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f7031-0202-4ec0-8f4d-ab516db369f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:39.548732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d1a3b-6a00-4479-964b-ef8e5036c96a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:39.553407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d1a3b-6a00-4479-964b-ef8e5036c96a service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:39.593688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f7031-0202-4ec0-8f4d-ab516db369f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:39.598269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f7031-0202-4ec0-8f4d-ab516db369f6 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:39.599885 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2129/4237] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:39.627035 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2109/4238] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:15:40.620820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d1a3b-6a00-4479-964b-ef8e5036c96a service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:40.623688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d86d1a3b-6a00-4479-964b-ef8e5036c96a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:40.667209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d1a3b-6a00-4479-964b-ef8e5036c96a service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:40.685434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec5f75-27f1-42a9-a771-acf70cba9452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:40.690322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec5f75-27f1-42a9-a771-acf70cba9452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:40.690322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec5f75-27f1-42a9-a771-acf70cba9452 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:40.690322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec5f75-27f1-42a9-a771-acf70cba9452 admin admin] PATH_INFO: `/v3/projects/f3f2ea38d3274d8e82ab37aded1a5c07` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:40.690322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec5f75-27f1-42a9-a771-acf70cba9452 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3f2ea38d3274d8e82ab37aded1a5c07)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:40.694477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec5f75-27f1-42a9-a771-acf70cba9452 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:40.892235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d2ec5f75-27f1-42a9-a771-acf70cba9452 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3f2ea38d3274d8e82ab37aded1a5c07', 'request_id': 'req-d2ec5f75-27f1-42a9-a771-acf70cba9452'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:40.908219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d2ec5f75-27f1-42a9-a771-acf70cba9452 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3f2ea38d3274d8e82ab37aded1a5c07', 'request_id': 'req-d2ec5f75-27f1-42a9-a771-acf70cba9452'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:41.040209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d2ec5f75-27f1-42a9-a771-acf70cba9452 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3f2ea38d3274d8e82ab37aded1a5c07', 'request_id': 'req-d2ec5f75-27f1-42a9-a771-acf70cba9452'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:41.082472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d2ec5f75-27f1-42a9-a771-acf70cba9452 admin admin] The token cache is being invalidate because project f3f2ea38d3274d8e82ab37aded1a5c07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:15:41.082881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d2ec5f75-27f1-42a9-a771-acf70cba9452 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:41.083890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d2ec5f75-27f1-42a9-a771-acf70cba9452 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:41.089188 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2130/4239] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:40 2022] DELETE /identity/v3/projects/f3f2ea38d3274d8e82ab37aded1a5c07 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:41.090224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f7031-0202-4ec0-8f4d-ab516db369f6 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:41.118550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f7031-0202-4ec0-8f4d-ab516db369f6 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:41.219721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316fbc1-4399-4e79-8808-0208bc934890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:41.220796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316fbc1-4399-4e79-8808-0208bc934890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:41.221251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316fbc1-4399-4e79-8808-0208bc934890 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:41.221566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0316fbc1-4399-4e79-8808-0208bc934890 admin admin] PATH_INFO: `/v3/users/990e6e0dee3d4a61a484c34c8077f485` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:41.223698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0316fbc1-4399-4e79-8808-0208bc934890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=990e6e0dee3d4a61a484c34c8077f485)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:41.225823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0316fbc1-4399-4e79-8808-0208bc934890 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:41.232806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:41.572143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0316fbc1-4399-4e79-8808-0208bc934890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '990e6e0dee3d4a61a484c34c8077f485'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:41.573816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0316fbc1-4399-4e79-8808-0208bc934890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '990e6e0dee3d4a61a484c34c8077f485'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:41.584194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0316fbc1-4399-4e79-8808-0208bc934890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '990e6e0dee3d4a61a484c34c8077f485'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:41.721681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0316fbc1-4399-4e79-8808-0208bc934890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '990e6e0dee3d4a61a484c34c8077f485'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:41.722408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0316fbc1-4399-4e79-8808-0208bc934890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '990e6e0dee3d4a61a484c34c8077f485'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:41.761510 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2110/4240] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:15:41 2022] DELETE /identity/v3/users/990e6e0dee3d4a61a484c34c8077f485 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:15:42.275081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e3a63-6394-4996-bf73-ff2ed41ae130 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:42.276264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e3a63-6394-4996-bf73-ff2ed41ae130 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:42.276823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e3a63-6394-4996-bf73-ff2ed41ae130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:42.307112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cf7e3a63-6394-4996-bf73-ff2ed41ae130 None None] MFA Rules not processed for user `572170ccd67e4facbf10dad7cd1491c1`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:42.315225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cf7e3a63-6394-4996-bf73-ff2ed41ae130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:42.442263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316fbc1-4399-4e79-8808-0208bc934890 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:42.454260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0316fbc1-4399-4e79-8808-0208bc934890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:42.478866 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2131/4241] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:15:42.659109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0316fbc1-4399-4e79-8808-0208bc934890 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:42.668765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23900d-4a36-46ce-8d34-0a198f13a760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:42.670360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23900d-4a36-46ce-8d34-0a198f13a760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:42.670798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23900d-4a36-46ce-8d34-0a198f13a760 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:42.671242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23900d-4a36-46ce-8d34-0a198f13a760 admin admin] PATH_INFO: `/v3/projects/cadf0ccd043e456281965885fac7747a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:42.673226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23900d-4a36-46ce-8d34-0a198f13a760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cadf0ccd043e456281965885fac7747a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:42.675378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23900d-4a36-46ce-8d34-0a198f13a760 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:42.848447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0c23900d-4a36-46ce-8d34-0a198f13a760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cadf0ccd043e456281965885fac7747a', 'request_id': 'req-0c23900d-4a36-46ce-8d34-0a198f13a760'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:42.867541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0c23900d-4a36-46ce-8d34-0a198f13a760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cadf0ccd043e456281965885fac7747a', 'request_id': 'req-0c23900d-4a36-46ce-8d34-0a198f13a760'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:42.927939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0c23900d-4a36-46ce-8d34-0a198f13a760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cadf0ccd043e456281965885fac7747a', 'request_id': 'req-0c23900d-4a36-46ce-8d34-0a198f13a760'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:42.945668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0c23900d-4a36-46ce-8d34-0a198f13a760 admin admin] The token cache is being invalidate because project cadf0ccd043e456281965885fac7747a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:15:42.946441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0c23900d-4a36-46ce-8d34-0a198f13a760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:42.947449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0c23900d-4a36-46ce-8d34-0a198f13a760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:42.955914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e3a63-6394-4996-bf73-ff2ed41ae130 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:42.960779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cf7e3a63-6394-4996-bf73-ff2ed41ae130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:42.973705 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2111/4242] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:42 2022] DELETE /identity/v3/projects/cadf0ccd043e456281965885fac7747a => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:42.981475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b73f00-ac0c-493b-bfd2-0fe4e2790ab6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:42.986819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b73f00-ac0c-493b-bfd2-0fe4e2790ab6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:42.987256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b73f00-ac0c-493b-bfd2-0fe4e2790ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:42.989998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:43.040344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a3b73f00-ac0c-493b-bfd2-0fe4e2790ab6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:43.043619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a3b73f00-ac0c-493b-bfd2-0fe4e2790ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:43.071198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:43.138940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e3a63-6394-4996-bf73-ff2ed41ae130 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:43.181819 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2112/4243] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:43.183540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988be0e-0964-473e-ad91-0e95d41a925d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:43.183929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988be0e-0964-473e-ad91-0e95d41a925d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:43.184306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0988be0e-0964-473e-ad91-0e95d41a925d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:43.227197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0988be0e-0964-473e-ad91-0e95d41a925d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:43.228787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0988be0e-0964-473e-ad91-0e95d41a925d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:43.238259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61840d-7d16-498a-9166-c3dcf988e15f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:43.240327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61840d-7d16-498a-9166-c3dcf988e15f service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:43.241004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61840d-7d16-498a-9166-c3dcf988e15f service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:43.242280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61840d-7d16-498a-9166-c3dcf988e15f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:43.244783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ad61840d-7d16-498a-9166-c3dcf988e15f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:43.251352 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2113/4244] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:43.255869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988be0e-0964-473e-ad91-0e95d41a925d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:43.264525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61840d-7d16-498a-9166-c3dcf988e15f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:43.264939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0988be0e-0964-473e-ad91-0e95d41a925d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:43.267234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61840d-7d16-498a-9166-c3dcf988e15f service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:43.278392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a2e8b-d32f-40a5-9682-f3a31289f5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:43.279085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a2e8b-d32f-40a5-9682-f3a31289f5db admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:43.279459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a2e8b-d32f-40a5-9682-f3a31289f5db admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:43.279738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541a2e8b-d32f-40a5-9682-f3a31289f5db admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:43.281526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a2e8b-d32f-40a5-9682-f3a31289f5db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:15:43.282055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a2e8b-d32f-40a5-9682-f3a31289f5db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:43.283911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541a2e8b-d32f-40a5-9682-f3a31289f5db admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:43.297047 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2132/4245] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:43.302944 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2114/4246] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:43.307679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61840d-7d16-498a-9166-c3dcf988e15f service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:43.321070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61840d-7d16-498a-9166-c3dcf988e15f service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:43.342687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af4388-445e-4e30-9ef3-43539c2b381f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:43.347726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af4388-445e-4e30-9ef3-43539c2b381f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:43.347726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af4388-445e-4e30-9ef3-43539c2b381f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:43.347726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af4388-445e-4e30-9ef3-43539c2b381f admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:43.347726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af4388-445e-4e30-9ef3-43539c2b381f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:43.354075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af4388-445e-4e30-9ef3-43539c2b381f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:43.481402 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2133/4247] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:43 2022] POST /identity/v3/projects => generated 369 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:43.483042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a2e8b-d32f-40a5-9682-f3a31289f5db admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:43.495969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541a2e8b-d32f-40a5-9682-f3a31289f5db admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:43.669595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66c2cc-13b9-41b0-9e5a-9c3d73ea9ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:43.670330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66c2cc-13b9-41b0-9e5a-9c3d73ea9ba1 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:43.671124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66c2cc-13b9-41b0-9e5a-9c3d73ea9ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:43.672111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66c2cc-13b9-41b0-9e5a-9c3d73ea9ba1 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:43.675617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66c2cc-13b9-41b0-9e5a-9c3d73ea9ba1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:43.680049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66c2cc-13b9-41b0-9e5a-9c3d73ea9ba1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:43.765303 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2115/4248] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:43 2022] POST /identity/v3/users => generated 285 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:43.769676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af4388-445e-4e30-9ef3-43539c2b381f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:43.786653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af4388-445e-4e30-9ef3-43539c2b381f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:43.802528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bc2b9-7785-4079-91c8-f5dbfca1993f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:43.803136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bc2b9-7785-4079-91c8-f5dbfca1993f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:43.803415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bc2b9-7785-4079-91c8-f5dbfca1993f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:43.804241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76bc2b9-7785-4079-91c8-f5dbfca1993f admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:43.805154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bc2b9-7785-4079-91c8-f5dbfca1993f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:43.807584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76bc2b9-7785-4079-91c8-f5dbfca1993f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:43.842151 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2134/4249] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:43.844010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66c2cc-13b9-41b0-9e5a-9c3d73ea9ba1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:43.857319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66c2cc-13b9-41b0-9e5a-9c3d73ea9ba1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:43.878220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4b587-5590-4c0a-858a-3c7a84539d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:43.878947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4b587-5590-4c0a-858a-3c7a84539d3b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:43.879250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4b587-5590-4c0a-858a-3c7a84539d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:43.879527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4b587-5590-4c0a-858a-3c7a84539d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:43.880278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4b587-5590-4c0a-858a-3c7a84539d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:43.882513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4b587-5590-4c0a-858a-3c7a84539d3b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:43.913535 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2116/4250] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:43.916404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bc2b9-7785-4079-91c8-f5dbfca1993f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:43.935530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76bc2b9-7785-4079-91c8-f5dbfca1993f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:43.945550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b9cee-2708-44f6-823d-64d7d66f4754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:43.946560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b9cee-2708-44f6-823d-64d7d66f4754 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:43.946851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b9cee-2708-44f6-823d-64d7d66f4754 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:43.947126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b9cee-2708-44f6-823d-64d7d66f4754 admin admin] PATH_INFO: `/v3/projects/38b694eef3334496a1788d8053e130ef/users/e3e708884af044ef87d01004f5d03967/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:43.972551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b9cee-2708-44f6-823d-64d7d66f4754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b694eef3334496a1788d8053e130ef, user_id=e3e708884af044ef87d01004f5d03967, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:43.978002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b9cee-2708-44f6-823d-64d7d66f4754 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:44.107732 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2135/4251] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:43 2022] PUT /identity/v3/projects/38b694eef3334496a1788d8053e130ef/users/e3e708884af044ef87d01004f5d03967/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:44.112311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594f545-0ea6-445d-93bb-f433613122e7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:44.112607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594f545-0ea6-445d-93bb-f433613122e7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:44.112889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594f545-0ea6-445d-93bb-f433613122e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:44.177807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d594f545-0ea6-445d-93bb-f433613122e7 None None] MFA Rules not processed for user `e3e708884af044ef87d01004f5d03967`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:44.184834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d594f545-0ea6-445d-93bb-f433613122e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:44.383120 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2117/4252] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:44.388318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b9cee-2708-44f6-823d-64d7d66f4754 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:44.417929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b9cee-2708-44f6-823d-64d7d66f4754 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:44.424444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b67e3e-ee11-4425-9957-0fdc33b54364 tempest-TestApiDiscovery-1089507995 tempest-TestApiDiscovery-1089507995-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3e708884af044ef87d01004f5d03967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38b694eef3334496a1788d8053e130ef', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:44.425143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b67e3e-ee11-4425-9957-0fdc33b54364 tempest-TestApiDiscovery-1089507995 tempest-TestApiDiscovery-1089507995-project] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:44.426418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b67e3e-ee11-4425-9957-0fdc33b54364 tempest-TestApiDiscovery-1089507995 tempest-TestApiDiscovery-1089507995-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:44.426418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b67e3e-ee11-4425-9957-0fdc33b54364 tempest-TestApiDiscovery-1089507995 tempest-TestApiDiscovery-1089507995-project] PATH_INFO: `/v3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:44.435259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594f545-0ea6-445d-93bb-f433613122e7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:44.443269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:44.445328 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2136/4253] 10.222.0.33 () {58 vars in 1103 bytes} [Sat May 28 21:15:44 2022] GET /identity/v3 => generated 255 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:15:44.452358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594f545-0ea6-445d-93bb-f433613122e7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:44.469702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76766e94-3247-4cb1-8f32-62860a097c25 tempest-TestApiDiscovery-1089507995 tempest-TestApiDiscovery-1089507995-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3e708884af044ef87d01004f5d03967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38b694eef3334496a1788d8053e130ef', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:44.477420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76766e94-3247-4cb1-8f32-62860a097c25 tempest-TestApiDiscovery-1089507995 tempest-TestApiDiscovery-1089507995-project] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:44.477420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76766e94-3247-4cb1-8f32-62860a097c25 tempest-TestApiDiscovery-1089507995 tempest-TestApiDiscovery-1089507995-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:44.477420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76766e94-3247-4cb1-8f32-62860a097c25 tempest-TestApiDiscovery-1089507995 tempest-TestApiDiscovery-1089507995-project] PATH_INFO: `/v3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:44.478137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b67e3e-ee11-4425-9957-0fdc33b54364 tempest-TestApiDiscovery-1089507995 tempest-TestApiDiscovery-1089507995-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:44.483339 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2118/4254] 10.222.0.33 () {58 vars in 1103 bytes} [Sat May 28 21:15:44 2022] GET /identity/v3 => generated 255 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:44.491539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b67e3e-ee11-4425-9957-0fdc33b54364 tempest-TestApiDiscovery-1089507995 tempest-TestApiDiscovery-1089507995-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:44.499877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4085c69-738f-4e76-b5af-df68c3cbd4a9 tempest-TestApiDiscovery-1089507995 tempest-TestApiDiscovery-1089507995-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3e708884af044ef87d01004f5d03967', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '38b694eef3334496a1788d8053e130ef', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:44.501333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4085c69-738f-4e76-b5af-df68c3cbd4a9 tempest-TestApiDiscovery-1089507995 tempest-TestApiDiscovery-1089507995-project] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:44.501333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4085c69-738f-4e76-b5af-df68c3cbd4a9 tempest-TestApiDiscovery-1089507995 tempest-TestApiDiscovery-1089507995-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:44.501333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4085c69-738f-4e76-b5af-df68c3cbd4a9 tempest-TestApiDiscovery-1089507995 tempest-TestApiDiscovery-1089507995-project] PATH_INFO: `/v3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:44.503036 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2137/4255] 10.222.0.33 () {58 vars in 1103 bytes} [Sat May 28 21:15:44 2022] GET /identity/v3 => generated 255 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:44.510104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d34bfd-21d4-4d74-a8a5-5662a57e9961 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:44.510104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d34bfd-21d4-4d74-a8a5-5662a57e9961 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:44.510104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d34bfd-21d4-4d74-a8a5-5662a57e9961 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:44.510386 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2119/4256] 10.222.0.33 () {56 vars in 894 bytes} [Sat May 28 21:15:44 2022] GET /identity/ => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:15:44.517226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02ae84-d0f4-4473-8d37-ede0abca15f7 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:44.517527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02ae84-d0f4-4473-8d37-ede0abca15f7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:44.517826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02ae84-d0f4-4473-8d37-ede0abca15f7 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:44.519281 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2138/4257] 10.222.0.33 () {56 vars in 894 bytes} [Sat May 28 21:15:44 2022] GET /identity/ => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:15:44.527103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d34bfd-21d4-4d74-a8a5-5662a57e9961 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:44.685022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d34bfd-21d4-4d74-a8a5-5662a57e9961 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:44.693595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643a355-1ac8-46e6-8de8-0d58db557737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:44.695490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643a355-1ac8-46e6-8de8-0d58db557737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:44.695827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643a355-1ac8-46e6-8de8-0d58db557737 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:44.696218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643a355-1ac8-46e6-8de8-0d58db557737 admin admin] PATH_INFO: `/v3/users/e3e708884af044ef87d01004f5d03967` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:44.698083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2643a355-1ac8-46e6-8de8-0d58db557737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3e708884af044ef87d01004f5d03967)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:44.703076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2643a355-1ac8-46e6-8de8-0d58db557737 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:45.269637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2643a355-1ac8-46e6-8de8-0d58db557737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3e708884af044ef87d01004f5d03967'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:45.271860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2643a355-1ac8-46e6-8de8-0d58db557737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3e708884af044ef87d01004f5d03967'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:45.282898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2643a355-1ac8-46e6-8de8-0d58db557737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3e708884af044ef87d01004f5d03967'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:45.400587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2643a355-1ac8-46e6-8de8-0d58db557737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3e708884af044ef87d01004f5d03967'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:45.401280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2643a355-1ac8-46e6-8de8-0d58db557737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3e708884af044ef87d01004f5d03967'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:45.428790 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2120/4258] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:15:44 2022] DELETE /identity/v3/users/e3e708884af044ef87d01004f5d03967 => generated 0 bytes in 902 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:45.433617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee3362-3542-492f-8bca-fbf4cf7807fc None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:45.434173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee3362-3542-492f-8bca-fbf4cf7807fc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:45.434173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee3362-3542-492f-8bca-fbf4cf7807fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:45.460072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c7ee3362-3542-492f-8bca-fbf4cf7807fc None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:45.462423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c7ee3362-3542-492f-8bca-fbf4cf7807fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:45.508044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:45.545572 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2139/4259] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:45.555769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643a355-1ac8-46e6-8de8-0d58db557737 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:45.557283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2643a355-1ac8-46e6-8de8-0d58db557737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:45.628005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643a355-1ac8-46e6-8de8-0d58db557737 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:45.632118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d060e6-6eb6-46d4-a4c8-95d34160a241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:45.632779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d060e6-6eb6-46d4-a4c8-95d34160a241 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:45.633201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d060e6-6eb6-46d4-a4c8-95d34160a241 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:45.633739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d060e6-6eb6-46d4-a4c8-95d34160a241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:45.643967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d060e6-6eb6-46d4-a4c8-95d34160a241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:45.645437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d060e6-6eb6-46d4-a4c8-95d34160a241 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:45.655554 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2121/4260] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:46.230778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee3362-3542-492f-8bca-fbf4cf7807fc None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:46.232141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c7ee3362-3542-492f-8bca-fbf4cf7807fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:46.247665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee3362-3542-492f-8bca-fbf4cf7807fc None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:46.252609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf4470-abd6-44c3-9265-91871598345e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:46.254174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf4470-abd6-44c3-9265-91871598345e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:46.254812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf4470-abd6-44c3-9265-91871598345e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:46.255095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf4470-abd6-44c3-9265-91871598345e admin admin] PATH_INFO: `/v3/projects/38b694eef3334496a1788d8053e130ef` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:46.256604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf4470-abd6-44c3-9265-91871598345e admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b694eef3334496a1788d8053e130ef)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:46.259678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf4470-abd6-44c3-9265-91871598345e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:46.413101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-61bf4470-abd6-44c3-9265-91871598345e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b694eef3334496a1788d8053e130ef', 'request_id': 'req-61bf4470-abd6-44c3-9265-91871598345e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:46.420326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-61bf4470-abd6-44c3-9265-91871598345e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b694eef3334496a1788d8053e130ef', 'request_id': 'req-61bf4470-abd6-44c3-9265-91871598345e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:46.515167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-61bf4470-abd6-44c3-9265-91871598345e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b694eef3334496a1788d8053e130ef', 'request_id': 'req-61bf4470-abd6-44c3-9265-91871598345e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:46.529620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-61bf4470-abd6-44c3-9265-91871598345e admin admin] The token cache is being invalidate because project 38b694eef3334496a1788d8053e130ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:15:46.530419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-61bf4470-abd6-44c3-9265-91871598345e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:46.531263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-61bf4470-abd6-44c3-9265-91871598345e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:46.542678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfba6b-7757-4be9-95a0-c7e172a32420 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:46.542964 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2140/4261] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:46 2022] DELETE /identity/v3/projects/38b694eef3334496a1788d8053e130ef => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:46.543062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfba6b-7757-4be9-95a0-c7e172a32420 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:46.543062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfba6b-7757-4be9-95a0-c7e172a32420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:46.565656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-52dfba6b-7757-4be9-95a0-c7e172a32420 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:46.567080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-52dfba6b-7757-4be9-95a0-c7e172a32420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:46.586897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:46.642640 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2122/4262] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:46.650546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f0bbc-eb3a-4564-9975-f36a703faaf1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:46.650841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f0bbc-eb3a-4564-9975-f36a703faaf1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:46.651125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3f0bbc-eb3a-4564-9975-f36a703faaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:46.670574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4c3f0bbc-eb3a-4564-9975-f36a703faaf1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:46.671813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4c3f0bbc-eb3a-4564-9975-f36a703faaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:46.680686 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2141/4263] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:46.684885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfba6b-7757-4be9-95a0-c7e172a32420 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:46.695365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfba6b-7757-4be9-95a0-c7e172a32420 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:46.701659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93211922-457f-4a5a-adfb-0cffe41ed993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:46.702411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93211922-457f-4a5a-adfb-0cffe41ed993 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:46.710190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93211922-457f-4a5a-adfb-0cffe41ed993 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:46.711905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93211922-457f-4a5a-adfb-0cffe41ed993 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:46.712801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93211922-457f-4a5a-adfb-0cffe41ed993 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:15:46.713606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93211922-457f-4a5a-adfb-0cffe41ed993 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:46.716448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93211922-457f-4a5a-adfb-0cffe41ed993 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:46.735181 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2123/4264] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:15:46.736622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f0bbc-eb3a-4564-9975-f36a703faaf1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:46.748881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3f0bbc-eb3a-4564-9975-f36a703faaf1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:46.753916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974988ff-d53d-46d1-bc27-0b8851e0f8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:46.754644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974988ff-d53d-46d1-bc27-0b8851e0f8cb admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:46.754954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974988ff-d53d-46d1-bc27-0b8851e0f8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:46.755236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974988ff-d53d-46d1-bc27-0b8851e0f8cb admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:46.755938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974988ff-d53d-46d1-bc27-0b8851e0f8cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:46.758515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974988ff-d53d-46d1-bc27-0b8851e0f8cb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:46.884144 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2142/4265] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:46 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:46.888622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93211922-457f-4a5a-adfb-0cffe41ed993 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:46.899037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93211922-457f-4a5a-adfb-0cffe41ed993 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:46.967859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2762401-c819-4362-87b1-e1bde874ecf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:46.968533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2762401-c819-4362-87b1-e1bde874ecf7 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:46.968980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2762401-c819-4362-87b1-e1bde874ecf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:46.969257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2762401-c819-4362-87b1-e1bde874ecf7 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:46.970014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2762401-c819-4362-87b1-e1bde874ecf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:46.972340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2762401-c819-4362-87b1-e1bde874ecf7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:47.073435 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2124/4266] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:46 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:15:47.074903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974988ff-d53d-46d1-bc27-0b8851e0f8cb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:47.086842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974988ff-d53d-46d1-bc27-0b8851e0f8cb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:47.091668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac124c8-8dc8-4c47-b19b-c5be19e2c712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:47.092346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac124c8-8dc8-4c47-b19b-c5be19e2c712 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:47.092631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac124c8-8dc8-4c47-b19b-c5be19e2c712 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:47.092910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac124c8-8dc8-4c47-b19b-c5be19e2c712 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:47.093648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac124c8-8dc8-4c47-b19b-c5be19e2c712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:47.095301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac124c8-8dc8-4c47-b19b-c5be19e2c712 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:47.111704 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2143/4267] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:15:47.113050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2762401-c819-4362-87b1-e1bde874ecf7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:47.125501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2762401-c819-4362-87b1-e1bde874ecf7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:47.146047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6fc378-7011-4556-b141-1c3a9f23375f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:47.146938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fc378-7011-4556-b141-1c3a9f23375f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:47.147355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fc378-7011-4556-b141-1c3a9f23375f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:47.147726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6fc378-7011-4556-b141-1c3a9f23375f admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:47.149254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fc378-7011-4556-b141-1c3a9f23375f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:47.151631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6fc378-7011-4556-b141-1c3a9f23375f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:47.171738 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2125/4268] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:47.173093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac124c8-8dc8-4c47-b19b-c5be19e2c712 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:47.186545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac124c8-8dc8-4c47-b19b-c5be19e2c712 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:47.196125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e228d7-bd87-41d9-8304-5709ea7568a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:47.197742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e228d7-bd87-41d9-8304-5709ea7568a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:47.197742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e228d7-bd87-41d9-8304-5709ea7568a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:47.197742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e228d7-bd87-41d9-8304-5709ea7568a5 admin admin] PATH_INFO: `/v3/projects/9eccfc35c6a14ccbbc278d7cc9b02ea8/users/c434eb3cc69a421c817163bc3e14f560/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:47.252978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e228d7-bd87-41d9-8304-5709ea7568a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eccfc35c6a14ccbbc278d7cc9b02ea8, user_id=c434eb3cc69a421c817163bc3e14f560, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:47.256112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e228d7-bd87-41d9-8304-5709ea7568a5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:47.384966 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2144/4269] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:47 2022] PUT /identity/v3/projects/9eccfc35c6a14ccbbc278d7cc9b02ea8/users/c434eb3cc69a421c817163bc3e14f560/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:47.395730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69b556-048e-4577-8eb3-261862a6dddd None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:47.399243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69b556-048e-4577-8eb3-261862a6dddd None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:47.399597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d69b556-048e-4577-8eb3-261862a6dddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:47.436978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1d69b556-048e-4577-8eb3-261862a6dddd None None] MFA Rules not processed for user `c434eb3cc69a421c817163bc3e14f560`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:47.438264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1d69b556-048e-4577-8eb3-261862a6dddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:47.509997 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2126/4270] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:47.512372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e228d7-bd87-41d9-8304-5709ea7568a5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:47.523324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e228d7-bd87-41d9-8304-5709ea7568a5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:47.529832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5606704-8d45-4aaa-847a-103ee0431e5f tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c434eb3cc69a421c817163bc3e14f560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eccfc35c6a14ccbbc278d7cc9b02ea8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:47.537557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5606704-8d45-4aaa-847a-103ee0431e5f tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:47.537557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5606704-8d45-4aaa-847a-103ee0431e5f tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:47.537557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5606704-8d45-4aaa-847a-103ee0431e5f tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] PATH_INFO: `/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:47.537557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5606704-8d45-4aaa-847a-103ee0431e5f tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=c434eb3cc69a421c817163bc3e14f560)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:47.538768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5606704-8d45-4aaa-847a-103ee0431e5f tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:47.667312 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2145/4271] 10.222.0.33 () {60 vars in 1298 bytes} [Sat May 28 21:15:47 2022] POST /identity/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2 => generated 367 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:47.672829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69b556-048e-4577-8eb3-261862a6dddd None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:47.683715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d69b556-048e-4577-8eb3-261862a6dddd None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:47.688170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fb550-0bd0-466a-a0d0-2c6c4446dbdf tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c434eb3cc69a421c817163bc3e14f560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eccfc35c6a14ccbbc278d7cc9b02ea8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:47.689075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fb550-0bd0-466a-a0d0-2c6c4446dbdf tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:47.689348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fb550-0bd0-466a-a0d0-2c6c4446dbdf tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:47.689623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1fb550-0bd0-466a-a0d0-2c6c4446dbdf tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] PATH_INFO: `/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2/e4b3af8aa6fd48d0b64a4d23d30ae1e4` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:47.692070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fb550-0bd0-466a-a0d0-2c6c4446dbdf tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=c434eb3cc69a421c817163bc3e14f560, credential_id=e4b3af8aa6fd48d0b64a4d23d30ae1e4)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:47.693737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1fb550-0bd0-466a-a0d0-2c6c4446dbdf tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:47.793312 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2127/4272] 10.222.0.33 () {58 vars in 1379 bytes} [Sat May 28 21:15:47 2022] DELETE /identity/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2/e4b3af8aa6fd48d0b64a4d23d30ae1e4 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:47.798698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5606704-8d45-4aaa-847a-103ee0431e5f tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:47.822840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5606704-8d45-4aaa-847a-103ee0431e5f tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:47.832521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30706-1634-4d72-9d7b-231e3600c9bc tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c434eb3cc69a421c817163bc3e14f560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eccfc35c6a14ccbbc278d7cc9b02ea8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:47.833491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30706-1634-4d72-9d7b-231e3600c9bc tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:47.833917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30706-1634-4d72-9d7b-231e3600c9bc tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:47.834341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30706-1634-4d72-9d7b-231e3600c9bc tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] PATH_INFO: `/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:47.836220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe30706-1634-4d72-9d7b-231e3600c9bc tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=c434eb3cc69a421c817163bc3e14f560)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:47.837441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe30706-1634-4d72-9d7b-231e3600c9bc tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:47.913306 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2146/4273] 10.222.0.33 () {60 vars in 1298 bytes} [Sat May 28 21:15:47 2022] POST /identity/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2 => generated 367 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:47.917080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fb550-0bd0-466a-a0d0-2c6c4446dbdf tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:47.923104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:47.928457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1fb550-0bd0-466a-a0d0-2c6c4446dbdf tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:47.934528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe32812-2341-4194-ac30-14e21e02fddf tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c434eb3cc69a421c817163bc3e14f560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eccfc35c6a14ccbbc278d7cc9b02ea8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:47.935748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe32812-2341-4194-ac30-14e21e02fddf tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:47.935994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe32812-2341-4194-ac30-14e21e02fddf tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:47.936277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe32812-2341-4194-ac30-14e21e02fddf tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] PATH_INFO: `/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2/92d81be7463c44a09c18ad6d7a2dfdf7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:47.938389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe32812-2341-4194-ac30-14e21e02fddf tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=c434eb3cc69a421c817163bc3e14f560, credential_id=92d81be7463c44a09c18ad6d7a2dfdf7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:47.939888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe32812-2341-4194-ac30-14e21e02fddf tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:48.025608 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2128/4274] 10.222.0.33 () {58 vars in 1379 bytes} [Sat May 28 21:15:47 2022] DELETE /identity/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2/92d81be7463c44a09c18ad6d7a2dfdf7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:48.029808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30706-1634-4d72-9d7b-231e3600c9bc tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:48.039235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30706-1634-4d72-9d7b-231e3600c9bc tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:48.043365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de8b07-81f8-437d-8c86-2b66877a1861 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c434eb3cc69a421c817163bc3e14f560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eccfc35c6a14ccbbc278d7cc9b02ea8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:48.044238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de8b07-81f8-437d-8c86-2b66877a1861 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:48.044510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de8b07-81f8-437d-8c86-2b66877a1861 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:48.044791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de8b07-81f8-437d-8c86-2b66877a1861 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] PATH_INFO: `/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2/92d81be7463c44a09c18ad6d7a2dfdf7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:48.077993 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-d7de8b07-81f8-437d-8c86-2b66877a1861 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Could not find credential: 61e718b4f4c8d50a3e71ce39d29b3c0f92951a3b2bc2fb5c9ac20a98d3be28df.: keystone.exception.CredentialNotFound: Could not find credential: 61e718b4f4c8d50a3e71ce39d29b3c0f92951a3b2bc2fb5c9ac20a98d3be28df. May 28 21:15:48.077993 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:15:48.077993 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:15:48.077993 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:15:48.077993 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:15:48.077993 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:15:48.077993 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:15:48.077993 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:15:48.077993 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:15:48.077993 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:15:48.077993 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:15:48.077993 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:15:48.077993 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 426, in get May 28 21:15:48.077993 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call( May 28 21:15:48.079205 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 431, in enforce_call May 28 21:15:48.079205 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict['target'] = target_attr or build_target() May 28 21:15:48.079205 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 119, in _build_enforcer_target_data_owner_and_user_id_match May 28 21:15:48.079205 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref['credential'] = PROVIDERS.credential_api.get_credential( May 28 21:15:48.079205 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:15:48.079205 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:15:48.079205 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 119, in get_credential May 28 21:15:48.079205 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application credential = self._get_credential(credential_id) May 28 21:15:48.079205 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:15:48.079205 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:15:48.079205 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:15:48.079205 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:15:48.079205 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:15:48.079205 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 21:15:48.079205 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:15:48.079205 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 21:15:48.080380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:15:48.080380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:15:48.080380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:15:48.080380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 21:15:48.080380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:15:48.080380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 21:15:48.080380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/core.py", line 124, in _get_credential May 28 21:15:48.080380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_credential(credential_id) May 28 21:15:48.080380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 88, in get_credential May 28 21:15:48.080380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_credential(session, credential_id).to_dict() May 28 21:15:48.080380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/credential/backends/sql.py", line 83, in _get_credential May 28 21:15:48.080380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.CredentialNotFound(credential_id=credential_id) May 28 21:15:48.080380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.CredentialNotFound: Could not find credential: 61e718b4f4c8d50a3e71ce39d29b3c0f92951a3b2bc2fb5c9ac20a98d3be28df. May 28 21:15:48.080380 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:15:48.082588 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2147/4275] 10.222.0.33 () {58 vars in 1376 bytes} [Sat May 28 21:15:48 2022] GET /identity/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2/92d81be7463c44a09c18ad6d7a2dfdf7 => generated 148 bytes in 53 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:15:48.087348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe32812-2341-4194-ac30-14e21e02fddf tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:48.099538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe32812-2341-4194-ac30-14e21e02fddf tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:48.105655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038e682-311f-468f-977b-c2e4947062bc tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c434eb3cc69a421c817163bc3e14f560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eccfc35c6a14ccbbc278d7cc9b02ea8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:48.107130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038e682-311f-468f-977b-c2e4947062bc tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:48.107530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038e682-311f-468f-977b-c2e4947062bc tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:48.108055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038e682-311f-468f-977b-c2e4947062bc tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] PATH_INFO: `/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:48.109309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038e682-311f-468f-977b-c2e4947062bc tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=c434eb3cc69a421c817163bc3e14f560)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:48.110946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5038e682-311f-468f-977b-c2e4947062bc tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:48.250199 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2129/4276] 10.222.0.33 () {60 vars in 1298 bytes} [Sat May 28 21:15:48 2022] POST /identity/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2 => generated 367 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:15:48.254492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de8b07-81f8-437d-8c86-2b66877a1861 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:48.268151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7de8b07-81f8-437d-8c86-2b66877a1861 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:48.272771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b044e1f-18cc-4af9-b40d-8fcab4df70ed tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c434eb3cc69a421c817163bc3e14f560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eccfc35c6a14ccbbc278d7cc9b02ea8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:48.273777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b044e1f-18cc-4af9-b40d-8fcab4df70ed tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:48.274163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b044e1f-18cc-4af9-b40d-8fcab4df70ed tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:48.280246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b044e1f-18cc-4af9-b40d-8fcab4df70ed tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] PATH_INFO: `/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:48.280246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b044e1f-18cc-4af9-b40d-8fcab4df70ed tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=c434eb3cc69a421c817163bc3e14f560)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:48.280246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b044e1f-18cc-4af9-b40d-8fcab4df70ed tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:48.363639 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2148/4277] 10.222.0.33 () {60 vars in 1298 bytes} [Sat May 28 21:15:48 2022] POST /identity/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2 => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:48.369549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038e682-311f-468f-977b-c2e4947062bc tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:48.388621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038e682-311f-468f-977b-c2e4947062bc tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:48.394278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfb629-e985-4687-8914-37754a1fada1 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c434eb3cc69a421c817163bc3e14f560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eccfc35c6a14ccbbc278d7cc9b02ea8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:48.395403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfb629-e985-4687-8914-37754a1fada1 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:48.396323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfb629-e985-4687-8914-37754a1fada1 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:48.396965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cfb629-e985-4687-8914-37754a1fada1 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] PATH_INFO: `/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:48.398066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cfb629-e985-4687-8914-37754a1fada1 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorizing `identity:ec2_list_credentials(user_id=c434eb3cc69a421c817163bc3e14f560)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:48.400713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cfb629-e985-4687-8914-37754a1fada1 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:48.419075 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2130/4278] 10.222.0.33 () {58 vars in 1277 bytes} [Sat May 28 21:15:48 2022] GET /identity/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2 => generated 865 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:48.420399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b044e1f-18cc-4af9-b40d-8fcab4df70ed tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:48.429595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:48.435609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b044e1f-18cc-4af9-b40d-8fcab4df70ed tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:48.445550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557c67d-2d3e-46c0-8b94-2df188453dc1 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c434eb3cc69a421c817163bc3e14f560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eccfc35c6a14ccbbc278d7cc9b02ea8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:48.446415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557c67d-2d3e-46c0-8b94-2df188453dc1 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:48.446703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557c67d-2d3e-46c0-8b94-2df188453dc1 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:48.446984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557c67d-2d3e-46c0-8b94-2df188453dc1 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] PATH_INFO: `/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2/fe340ed93b4e403fa9f6cbee66980f3c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:48.451674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557c67d-2d3e-46c0-8b94-2df188453dc1 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=c434eb3cc69a421c817163bc3e14f560, credential_id=fe340ed93b4e403fa9f6cbee66980f3c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:48.453302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557c67d-2d3e-46c0-8b94-2df188453dc1 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:48.534980 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2149/4279] 10.222.0.33 () {58 vars in 1379 bytes} [Sat May 28 21:15:48 2022] DELETE /identity/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2/fe340ed93b4e403fa9f6cbee66980f3c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:48.539914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfb629-e985-4687-8914-37754a1fada1 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:48.552742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cfb629-e985-4687-8914-37754a1fada1 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:48.566602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935fd51-7a1f-4007-b978-92ff7fac9c18 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c434eb3cc69a421c817163bc3e14f560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eccfc35c6a14ccbbc278d7cc9b02ea8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:48.567532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935fd51-7a1f-4007-b978-92ff7fac9c18 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:48.567820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935fd51-7a1f-4007-b978-92ff7fac9c18 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:48.568125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935fd51-7a1f-4007-b978-92ff7fac9c18 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] PATH_INFO: `/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2/8af5749e970343afae91e03cf219e389` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:48.570652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935fd51-7a1f-4007-b978-92ff7fac9c18 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=c434eb3cc69a421c817163bc3e14f560, credential_id=8af5749e970343afae91e03cf219e389)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:48.572497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935fd51-7a1f-4007-b978-92ff7fac9c18 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:48.683930 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2131/4280] 10.222.0.33 () {58 vars in 1379 bytes} [Sat May 28 21:15:48 2022] DELETE /identity/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2/8af5749e970343afae91e03cf219e389 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:15:48.691873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557c67d-2d3e-46c0-8b94-2df188453dc1 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:48.719786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557c67d-2d3e-46c0-8b94-2df188453dc1 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:48.735831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae913755-480f-461f-b5c9-11e6d514250b tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c434eb3cc69a421c817163bc3e14f560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eccfc35c6a14ccbbc278d7cc9b02ea8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:48.736871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae913755-480f-461f-b5c9-11e6d514250b tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:48.737197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae913755-480f-461f-b5c9-11e6d514250b tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:48.737995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae913755-480f-461f-b5c9-11e6d514250b tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] PATH_INFO: `/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:48.739083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae913755-480f-461f-b5c9-11e6d514250b tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorizing `identity:ec2_create_credential(user_id=c434eb3cc69a421c817163bc3e14f560)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:48.740257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae913755-480f-461f-b5c9-11e6d514250b tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:48.872180 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2150/4281] 10.222.0.33 () {60 vars in 1298 bytes} [Sat May 28 21:15:48 2022] POST /identity/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2 => generated 367 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:48.876102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935fd51-7a1f-4007-b978-92ff7fac9c18 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:48.889069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935fd51-7a1f-4007-b978-92ff7fac9c18 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:48.893952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d7de2-83c9-4d73-86f2-b16dd6bfb944 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c434eb3cc69a421c817163bc3e14f560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eccfc35c6a14ccbbc278d7cc9b02ea8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:48.895139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d7de2-83c9-4d73-86f2-b16dd6bfb944 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:48.895595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d7de2-83c9-4d73-86f2-b16dd6bfb944 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:48.896038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8d7de2-83c9-4d73-86f2-b16dd6bfb944 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] PATH_INFO: `/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2/f6cf939154c0408d837760bdeaa26710` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:48.898785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d7de2-83c9-4d73-86f2-b16dd6bfb944 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorizing `identity:ec2_get_credential(user_id=c434eb3cc69a421c817163bc3e14f560, credential_id=f6cf939154c0408d837760bdeaa26710)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:48.901187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8d7de2-83c9-4d73-86f2-b16dd6bfb944 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:48.905069 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2132/4282] 10.222.0.33 () {58 vars in 1376 bytes} [Sat May 28 21:15:48 2022] GET /identity/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2/f6cf939154c0408d837760bdeaa26710 => generated 367 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:48.909221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae913755-480f-461f-b5c9-11e6d514250b tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:48.921907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae913755-480f-461f-b5c9-11e6d514250b tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:48.943144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ab86b-aaaa-4b3f-9eb6-74103411125f tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c434eb3cc69a421c817163bc3e14f560', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9eccfc35c6a14ccbbc278d7cc9b02ea8', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:48.945651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ab86b-aaaa-4b3f-9eb6-74103411125f tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:48.946117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ab86b-aaaa-4b3f-9eb6-74103411125f tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:48.947655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ab86b-aaaa-4b3f-9eb6-74103411125f tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] PATH_INFO: `/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2/f6cf939154c0408d837760bdeaa26710` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:48.953509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ab86b-aaaa-4b3f-9eb6-74103411125f tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorizing `identity:ec2_delete_credential(user_id=c434eb3cc69a421c817163bc3e14f560, credential_id=f6cf939154c0408d837760bdeaa26710)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:48.957213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ab86b-aaaa-4b3f-9eb6-74103411125f tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:49.044920 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2151/4283] 10.222.0.33 () {58 vars in 1379 bytes} [Sat May 28 21:15:48 2022] DELETE /identity/v3/users/c434eb3cc69a421c817163bc3e14f560/credentials/OS-EC2/f6cf939154c0408d837760bdeaa26710 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:49.049550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d7de2-83c9-4d73-86f2-b16dd6bfb944 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:49.197968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8d7de2-83c9-4d73-86f2-b16dd6bfb944 tempest-EC2CredentialsTest-1948001315 tempest-EC2CredentialsTest-1948001315-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:49.206055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a06a0-acf8-49e5-b4e9-bdb43d076900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:49.207046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a06a0-acf8-49e5-b4e9-bdb43d076900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:49.207431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a06a0-acf8-49e5-b4e9-bdb43d076900 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:49.207816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a06a0-acf8-49e5-b4e9-bdb43d076900 admin admin] PATH_INFO: `/v3/users/c434eb3cc69a421c817163bc3e14f560` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:49.209224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a06a0-acf8-49e5-b4e9-bdb43d076900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c434eb3cc69a421c817163bc3e14f560)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:49.211320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a06a0-acf8-49e5-b4e9-bdb43d076900 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:49.599305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f0a06a0-acf8-49e5-b4e9-bdb43d076900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c434eb3cc69a421c817163bc3e14f560'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:49.599962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f0a06a0-acf8-49e5-b4e9-bdb43d076900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c434eb3cc69a421c817163bc3e14f560'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:49.607427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f0a06a0-acf8-49e5-b4e9-bdb43d076900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c434eb3cc69a421c817163bc3e14f560'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:49.772706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f0a06a0-acf8-49e5-b4e9-bdb43d076900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c434eb3cc69a421c817163bc3e14f560'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:49.773738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f0a06a0-acf8-49e5-b4e9-bdb43d076900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c434eb3cc69a421c817163bc3e14f560'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:49.800138 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2133/4284] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:15:49 2022] DELETE /identity/v3/users/c434eb3cc69a421c817163bc3e14f560 => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:49.801330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed7e93-6a06-4f0a-86b5-da2bb97628ad None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:49.801774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed7e93-6a06-4f0a-86b5-da2bb97628ad None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:49.802015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed7e93-6a06-4f0a-86b5-da2bb97628ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:49.874345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-78ed7e93-6a06-4f0a-86b5-da2bb97628ad None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:49.876689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-78ed7e93-6a06-4f0a-86b5-da2bb97628ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:49.952204 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2152/4285] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:15:49.972501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a06a0-acf8-49e5-b4e9-bdb43d076900 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:49.972501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0f0a06a0-acf8-49e5-b4e9-bdb43d076900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:50.003786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:50.082551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a06a0-acf8-49e5-b4e9-bdb43d076900 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:50.087469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaec3788-9c92-4e99-909c-c37fdb2b89c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:50.088791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec3788-9c92-4e99-909c-c37fdb2b89c1 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:50.089437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec3788-9c92-4e99-909c-c37fdb2b89c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:50.101201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaec3788-9c92-4e99-909c-c37fdb2b89c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:50.108334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec3788-9c92-4e99-909c-c37fdb2b89c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:50.110409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaec3788-9c92-4e99-909c-c37fdb2b89c1 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:50.122309 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2134/4286] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:50.737076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed7e93-6a06-4f0a-86b5-da2bb97628ad None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:50.738774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-78ed7e93-6a06-4f0a-86b5-da2bb97628ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:50.761346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed7e93-6a06-4f0a-86b5-da2bb97628ad None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:50.779040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628977c-e24f-4da3-81da-babf706b4f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:50.779040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628977c-e24f-4da3-81da-babf706b4f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:50.779040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628977c-e24f-4da3-81da-babf706b4f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:50.779040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628977c-e24f-4da3-81da-babf706b4f2c admin admin] PATH_INFO: `/v3/projects/9eccfc35c6a14ccbbc278d7cc9b02ea8` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:50.781962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628977c-e24f-4da3-81da-babf706b4f2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eccfc35c6a14ccbbc278d7cc9b02ea8)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:50.787080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628977c-e24f-4da3-81da-babf706b4f2c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:50.941582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3628977c-e24f-4da3-81da-babf706b4f2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eccfc35c6a14ccbbc278d7cc9b02ea8', 'request_id': 'req-3628977c-e24f-4da3-81da-babf706b4f2c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:50.949555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3628977c-e24f-4da3-81da-babf706b4f2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eccfc35c6a14ccbbc278d7cc9b02ea8', 'request_id': 'req-3628977c-e24f-4da3-81da-babf706b4f2c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:51.080428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3628977c-e24f-4da3-81da-babf706b4f2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eccfc35c6a14ccbbc278d7cc9b02ea8', 'request_id': 'req-3628977c-e24f-4da3-81da-babf706b4f2c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:51.094090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3628977c-e24f-4da3-81da-babf706b4f2c admin admin] The token cache is being invalidate because project 9eccfc35c6a14ccbbc278d7cc9b02ea8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:15:51.094615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3628977c-e24f-4da3-81da-babf706b4f2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:51.096302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3628977c-e24f-4da3-81da-babf706b4f2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:51.098358 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2153/4287] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:50 2022] DELETE /identity/v3/projects/9eccfc35c6a14ccbbc278d7cc9b02ea8 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:51.107395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa8f09-3688-4240-bf50-1bc14a2c5484 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:51.108305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa8f09-3688-4240-bf50-1bc14a2c5484 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:51.109067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa8f09-3688-4240-bf50-1bc14a2c5484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:51.132535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-35fa8f09-3688-4240-bf50-1bc14a2c5484 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:51.134166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-35fa8f09-3688-4240-bf50-1bc14a2c5484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:51.232992 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2135/4288] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:51.244454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e41131-f8d3-42e7-9b65-089db7b792cf None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:51.244769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e41131-f8d3-42e7-9b65-089db7b792cf None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:51.245026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e41131-f8d3-42e7-9b65-089db7b792cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:51.266240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e2e41131-f8d3-42e7-9b65-089db7b792cf None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:51.267477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e2e41131-f8d3-42e7-9b65-089db7b792cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:51.280233 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2154/4289] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:51.281317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa8f09-3688-4240-bf50-1bc14a2c5484 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:51.291380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa8f09-3688-4240-bf50-1bc14a2c5484 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:51.296342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71c2d8-7de3-4bb6-93db-9b79ef940d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:51.297407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71c2d8-7de3-4bb6-93db-9b79ef940d5f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:51.302032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71c2d8-7de3-4bb6-93db-9b79ef940d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:51.302032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71c2d8-7de3-4bb6-93db-9b79ef940d5f admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:51.302032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71c2d8-7de3-4bb6-93db-9b79ef940d5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:15:51.302032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71c2d8-7de3-4bb6-93db-9b79ef940d5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:51.302032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71c2d8-7de3-4bb6-93db-9b79ef940d5f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:51.316908 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2136/4290] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:51.321787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e41131-f8d3-42e7-9b65-089db7b792cf None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:51.333659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e41131-f8d3-42e7-9b65-089db7b792cf None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:51.338597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ac0f2-7286-4b96-b15b-0b2f5ec086d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:51.342355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ac0f2-7286-4b96-b15b-0b2f5ec086d9 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:51.342355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ac0f2-7286-4b96-b15b-0b2f5ec086d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:51.342355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ac0f2-7286-4b96-b15b-0b2f5ec086d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:51.342355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ac0f2-7286-4b96-b15b-0b2f5ec086d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:51.343353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ac0f2-7286-4b96-b15b-0b2f5ec086d9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:51.451161 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2155/4291] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:51 2022] POST /identity/v3/projects => generated 361 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:15:51.452415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71c2d8-7de3-4bb6-93db-9b79ef940d5f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:51.463917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71c2d8-7de3-4bb6-93db-9b79ef940d5f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:51.530880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28391bc-9454-4160-9bbf-fd90c3e6a701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:51.531582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28391bc-9454-4160-9bbf-fd90c3e6a701 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:51.531867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28391bc-9454-4160-9bbf-fd90c3e6a701 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:51.532139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28391bc-9454-4160-9bbf-fd90c3e6a701 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:51.532821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28391bc-9454-4160-9bbf-fd90c3e6a701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:51.534791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28391bc-9454-4160-9bbf-fd90c3e6a701 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:51.607659 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2137/4292] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:51 2022] POST /identity/v3/users => generated 281 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:51.612386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ac0f2-7286-4b96-b15b-0b2f5ec086d9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:51.625533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ac0f2-7286-4b96-b15b-0b2f5ec086d9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:51.630728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9760a-a078-4ad1-a660-3ad854737fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:51.631422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9760a-a078-4ad1-a660-3ad854737fbf admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:51.631873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9760a-a078-4ad1-a660-3ad854737fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:51.632267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9760a-a078-4ad1-a660-3ad854737fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:51.633017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a9760a-a078-4ad1-a660-3ad854737fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:51.634927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a9760a-a078-4ad1-a660-3ad854737fbf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:51.645979 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2156/4293] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:51.650198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28391bc-9454-4160-9bbf-fd90c3e6a701 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:51.661202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28391bc-9454-4160-9bbf-fd90c3e6a701 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:51.666951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a8886-1d1e-4df4-a6a0-25dd273a8603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:51.667722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a8886-1d1e-4df4-a6a0-25dd273a8603 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:51.668006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a8886-1d1e-4df4-a6a0-25dd273a8603 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:51.668286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a8886-1d1e-4df4-a6a0-25dd273a8603 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:51.669219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a8886-1d1e-4df4-a6a0-25dd273a8603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:51.670961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3a8886-1d1e-4df4-a6a0-25dd273a8603 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:51.685349 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2138/4294] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:51.686501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9760a-a078-4ad1-a660-3ad854737fbf admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:51.696115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:51.700297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9760a-a078-4ad1-a660-3ad854737fbf admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:51.705394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a18daf-1bc0-4816-8e76-1a2e27c234eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:51.708749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a18daf-1bc0-4816-8e76-1a2e27c234eb admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:51.708749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a18daf-1bc0-4816-8e76-1a2e27c234eb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:51.708749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a18daf-1bc0-4816-8e76-1a2e27c234eb admin admin] PATH_INFO: `/v3/projects/d000d8228c574d4fb52d9786ba6efb97/users/cd347727b749450fbf09c592df897e64/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:51.722521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a18daf-1bc0-4816-8e76-1a2e27c234eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d000d8228c574d4fb52d9786ba6efb97, user_id=cd347727b749450fbf09c592df897e64, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:51.725875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a18daf-1bc0-4816-8e76-1a2e27c234eb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:51.829197 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2157/4295] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:51 2022] PUT /identity/v3/projects/d000d8228c574d4fb52d9786ba6efb97/users/cd347727b749450fbf09c592df897e64/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:51.840277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a943527-eec4-4d84-bfd7-af8318ece291 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:51.840506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a943527-eec4-4d84-bfd7-af8318ece291 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:51.840785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a943527-eec4-4d84-bfd7-af8318ece291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:51.868513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6a943527-eec4-4d84-bfd7-af8318ece291 None None] MFA Rules not processed for user `cd347727b749450fbf09c592df897e64`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:51.869718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6a943527-eec4-4d84-bfd7-af8318ece291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:51.890081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:51.936618 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2139/4296] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:51 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:51.948819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a18daf-1bc0-4816-8e76-1a2e27c234eb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:52.016442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a18daf-1bc0-4816-8e76-1a2e27c234eb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:52.021143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b3878-c429-48d1-8e7a-a1598b946fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:52.022805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b3878-c429-48d1-8e7a-a1598b946fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:52.025101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b3878-c429-48d1-8e7a-a1598b946fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:52.025101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b3878-c429-48d1-8e7a-a1598b946fd3 admin admin] PATH_INFO: `/v3/users/cd347727b749450fbf09c592df897e64` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:52.025101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b3878-c429-48d1-8e7a-a1598b946fd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd347727b749450fbf09c592df897e64)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:52.026613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56b3878-c429-48d1-8e7a-a1598b946fd3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:52.152771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c56b3878-c429-48d1-8e7a-a1598b946fd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd347727b749450fbf09c592df897e64'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:52.160136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c56b3878-c429-48d1-8e7a-a1598b946fd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd347727b749450fbf09c592df897e64'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:52.160887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c56b3878-c429-48d1-8e7a-a1598b946fd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd347727b749450fbf09c592df897e64'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:52.161515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c56b3878-c429-48d1-8e7a-a1598b946fd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd347727b749450fbf09c592df897e64'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:52.228951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c56b3878-c429-48d1-8e7a-a1598b946fd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd347727b749450fbf09c592df897e64'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:52.249567 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2158/4297] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:15:51 2022] DELETE /identity/v3/users/cd347727b749450fbf09c592df897e64 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:52.257437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e2b86-e219-425d-bfe8-3667cf84e805 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:52.257942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e2b86-e219-425d-bfe8-3667cf84e805 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:52.258175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567e2b86-e219-425d-bfe8-3667cf84e805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:52.276477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-567e2b86-e219-425d-bfe8-3667cf84e805 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:52.277676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-567e2b86-e219-425d-bfe8-3667cf84e805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:52.349424 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2140/4298] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:15:52.360293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b3878-c429-48d1-8e7a-a1598b946fd3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:52.362285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c56b3878-c429-48d1-8e7a-a1598b946fd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:52.431991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b3878-c429-48d1-8e7a-a1598b946fd3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:52.436334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d9136-ebd5-427c-8ce5-965b8a68bf68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:52.437292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d9136-ebd5-427c-8ce5-965b8a68bf68 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:52.437644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d9136-ebd5-427c-8ce5-965b8a68bf68 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:52.437962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d9136-ebd5-427c-8ce5-965b8a68bf68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:52.456870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d9136-ebd5-427c-8ce5-965b8a68bf68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:52.458712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d9136-ebd5-427c-8ce5-965b8a68bf68 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:52.472555 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2159/4299] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:53.048978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e2b86-e219-425d-bfe8-3667cf84e805 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:53.050669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-567e2b86-e219-425d-bfe8-3667cf84e805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:53.065438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567e2b86-e219-425d-bfe8-3667cf84e805 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:53.073207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0b49b-8e27-4603-bd6f-3375b613969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:53.074402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0b49b-8e27-4603-bd6f-3375b613969c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:53.074751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0b49b-8e27-4603-bd6f-3375b613969c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:53.075146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0b49b-8e27-4603-bd6f-3375b613969c admin admin] PATH_INFO: `/v3/projects/d000d8228c574d4fb52d9786ba6efb97` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:53.076445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0b49b-8e27-4603-bd6f-3375b613969c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d000d8228c574d4fb52d9786ba6efb97)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:53.078509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0b49b-8e27-4603-bd6f-3375b613969c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:53.195391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cbf0b49b-8e27-4603-bd6f-3375b613969c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd000d8228c574d4fb52d9786ba6efb97', 'request_id': 'req-cbf0b49b-8e27-4603-bd6f-3375b613969c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:53.202182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cbf0b49b-8e27-4603-bd6f-3375b613969c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd000d8228c574d4fb52d9786ba6efb97', 'request_id': 'req-cbf0b49b-8e27-4603-bd6f-3375b613969c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:53.272824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cbf0b49b-8e27-4603-bd6f-3375b613969c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd000d8228c574d4fb52d9786ba6efb97', 'request_id': 'req-cbf0b49b-8e27-4603-bd6f-3375b613969c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:53.288591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cbf0b49b-8e27-4603-bd6f-3375b613969c admin admin] The token cache is being invalidate because project d000d8228c574d4fb52d9786ba6efb97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:15:53.289072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cbf0b49b-8e27-4603-bd6f-3375b613969c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:53.289747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cbf0b49b-8e27-4603-bd6f-3375b613969c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:15:53.291586 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2141/4300] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:15:53 2022] DELETE /identity/v3/projects/d000d8228c574d4fb52d9786ba6efb97 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:53.296485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe76049-d35a-4304-b4ce-85f362de4dd3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:53.299919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe76049-d35a-4304-b4ce-85f362de4dd3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:53.299919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe76049-d35a-4304-b4ce-85f362de4dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:53.309242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:53.329506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ebe76049-d35a-4304-b4ce-85f362de4dd3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:53.330831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ebe76049-d35a-4304-b4ce-85f362de4dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:53.384732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b7899-66a3-4ede-95ed-f6d977004ddd None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:53.385025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b7899-66a3-4ede-95ed-f6d977004ddd None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:53.385310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b7899-66a3-4ede-95ed-f6d977004ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:53.403506 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2160/4301] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:53.411104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e54b7899-66a3-4ede-95ed-f6d977004ddd None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:53.411770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790479db-6468-4465-bf4e-a39e693974ce None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:53.412129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790479db-6468-4465-bf4e-a39e693974ce None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:53.412326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e54b7899-66a3-4ede-95ed-f6d977004ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:53.414964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790479db-6468-4465-bf4e-a39e693974ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:53.454905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-790479db-6468-4465-bf4e-a39e693974ce None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:53.458611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-790479db-6468-4465-bf4e-a39e693974ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:53.470997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:53.477999 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2161/4302] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:15:53.480467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790479db-6468-4465-bf4e-a39e693974ce None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:53.493161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790479db-6468-4465-bf4e-a39e693974ce None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:53.499224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc9118-fdd7-4683-9382-f52a0994ba25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:53.500439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc9118-fdd7-4683-9382-f52a0994ba25 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:53.500744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc9118-fdd7-4683-9382-f52a0994ba25 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:53.501144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc9118-fdd7-4683-9382-f52a0994ba25 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:53.502446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc9118-fdd7-4683-9382-f52a0994ba25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:15:53.502777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc9118-fdd7-4683-9382-f52a0994ba25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:53.506800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc9118-fdd7-4683-9382-f52a0994ba25 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:53.527928 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2162/4303] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:15:53.529095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc9118-fdd7-4683-9382-f52a0994ba25 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:53.534839 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2142/4304] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:15:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:53.549716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b7899-66a3-4ede-95ed-f6d977004ddd None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:53.549939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc9118-fdd7-4683-9382-f52a0994ba25 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:53.552217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e54b7899-66a3-4ede-95ed-f6d977004ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:53.557250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26aab3-22c0-4652-9d9a-8cd823fee6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:53.558703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26aab3-22c0-4652-9d9a-8cd823fee6d5 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:53.558703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26aab3-22c0-4652-9d9a-8cd823fee6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:53.558703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26aab3-22c0-4652-9d9a-8cd823fee6d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:53.559690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26aab3-22c0-4652-9d9a-8cd823fee6d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:53.561700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26aab3-22c0-4652-9d9a-8cd823fee6d5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:53.707460 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2163/4305] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:53 2022] POST /identity/v3/projects => generated 411 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:53.711368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26aab3-22c0-4652-9d9a-8cd823fee6d5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:53.722650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b7899-66a3-4ede-95ed-f6d977004ddd None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:53.821140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a2416-426c-49f6-aa6f-4a88da25ecec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:53.821817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a2416-426c-49f6-aa6f-4a88da25ecec service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:53.822175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a2416-426c-49f6-aa6f-4a88da25ecec service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:53.822458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7a2416-426c-49f6-aa6f-4a88da25ecec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:53.838258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26aab3-22c0-4652-9d9a-8cd823fee6d5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:53.844732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb9237-b087-4392-89f5-bda80489fd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:53.846036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb9237-b087-4392-89f5-bda80489fd69 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:53.846437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb9237-b087-4392-89f5-bda80489fd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:53.846990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb9237-b087-4392-89f5-bda80489fd69 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:53.847984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb9237-b087-4392-89f5-bda80489fd69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:53.850556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb9237-b087-4392-89f5-bda80489fd69 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:53.901346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a2416-426c-49f6-aa6f-4a88da25ecec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:53.903602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7a2416-426c-49f6-aa6f-4a88da25ecec service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:53.915243 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2143/4306] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:15:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 21:15:53.995731 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2164/4307] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:53 2022] POST /identity/v3/users => generated 306 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:15:54.000429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a2416-426c-49f6-aa6f-4a88da25ecec service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:54.011680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7a2416-426c-49f6-aa6f-4a88da25ecec service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:54.016489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982d037-8a86-409f-813d-f3baf3eadac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:54.018231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982d037-8a86-409f-813d-f3baf3eadac1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:54.018231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982d037-8a86-409f-813d-f3baf3eadac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:54.018231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3982d037-8a86-409f-813d-f3baf3eadac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:54.018553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982d037-8a86-409f-813d-f3baf3eadac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:54.020046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3982d037-8a86-409f-813d-f3baf3eadac1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:54.035997 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2144/4308] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:54.049760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb9237-b087-4392-89f5-bda80489fd69 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:54.102137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb9237-b087-4392-89f5-bda80489fd69 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:54.120907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688fbb2-da50-4aa5-8916-9c053eb5d95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:54.121667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688fbb2-da50-4aa5-8916-9c053eb5d95f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:54.123067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688fbb2-da50-4aa5-8916-9c053eb5d95f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:54.123559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688fbb2-da50-4aa5-8916-9c053eb5d95f admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:54.124485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688fbb2-da50-4aa5-8916-9c053eb5d95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:54.127417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688fbb2-da50-4aa5-8916-9c053eb5d95f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:54.150653 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2165/4309] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:54.151842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982d037-8a86-409f-813d-f3baf3eadac1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:54.162392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3982d037-8a86-409f-813d-f3baf3eadac1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:54.168552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e6291-966e-4dca-829a-0a6b867ba13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:54.170065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e6291-966e-4dca-829a-0a6b867ba13a admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:54.170065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e6291-966e-4dca-829a-0a6b867ba13a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:54.170370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e6291-966e-4dca-829a-0a6b867ba13a admin admin] PATH_INFO: `/v3/projects/65d7e6533b364355bba7bfd71918ba7a/users/33613abc5f2240f7977d957a422547ba/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:54.172915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:54.193395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e6291-966e-4dca-829a-0a6b867ba13a admin admin] RBAC: Authorizing `identity:create_grant(project_id=65d7e6533b364355bba7bfd71918ba7a, user_id=33613abc5f2240f7977d957a422547ba, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:54.199364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e6291-966e-4dca-829a-0a6b867ba13a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:54.277052 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2145/4310] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:54 2022] PUT /identity/v3/projects/65d7e6533b364355bba7bfd71918ba7a/users/33613abc5f2240f7977d957a422547ba/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:54.288726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f721fe4-bb67-4c61-a0f3-8cb98f981671 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:54.289020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f721fe4-bb67-4c61-a0f3-8cb98f981671 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:54.292623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f721fe4-bb67-4c61-a0f3-8cb98f981671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:54.343157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5f721fe4-bb67-4c61-a0f3-8cb98f981671 None None] MFA Rules not processed for user `33613abc5f2240f7977d957a422547ba`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:54.344413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5f721fe4-bb67-4c61-a0f3-8cb98f981671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:54.434795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:54.509617 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2166/4311] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:54 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:54.517985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e6291-966e-4dca-829a-0a6b867ba13a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:54.647565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e6291-966e-4dca-829a-0a6b867ba13a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:54.653443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61471d37-e02f-493b-b5d3-eab58bc6a2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:54.654169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61471d37-e02f-493b-b5d3-eab58bc6a2e1 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:54.654456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61471d37-e02f-493b-b5d3-eab58bc6a2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:54.654734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61471d37-e02f-493b-b5d3-eab58bc6a2e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:54.655435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61471d37-e02f-493b-b5d3-eab58bc6a2e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:54.657545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61471d37-e02f-493b-b5d3-eab58bc6a2e1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:54.768283 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2146/4312] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:15:54 2022] POST /identity/v3/projects => generated 411 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:54.769490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f721fe4-bb67-4c61-a0f3-8cb98f981671 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:54.782953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f721fe4-bb67-4c61-a0f3-8cb98f981671 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:54.921989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf5e27-1083-4194-b5aa-36348cdd83a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:54.922699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf5e27-1083-4194-b5aa-36348cdd83a9 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:54.922993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf5e27-1083-4194-b5aa-36348cdd83a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:54.923279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf5e27-1083-4194-b5aa-36348cdd83a9 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:54.925285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf5e27-1083-4194-b5aa-36348cdd83a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:54.927290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf5e27-1083-4194-b5aa-36348cdd83a9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:55.071996 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2167/4313] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:15:54 2022] POST /identity/v3/users => generated 312 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:15:55.076183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61471d37-e02f-493b-b5d3-eab58bc6a2e1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:55.097203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61471d37-e02f-493b-b5d3-eab58bc6a2e1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:55.122636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2ed7f-8762-4636-a0a4-d7d627eec452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:55.124466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2ed7f-8762-4636-a0a4-d7d627eec452 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:55.124848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2ed7f-8762-4636-a0a4-d7d627eec452 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:55.126123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2ed7f-8762-4636-a0a4-d7d627eec452 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:55.127388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2ed7f-8762-4636-a0a4-d7d627eec452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:55.130936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2ed7f-8762-4636-a0a4-d7d627eec452 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:55.158501 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2147/4314] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:55.163066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf5e27-1083-4194-b5aa-36348cdd83a9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:55.178303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf5e27-1083-4194-b5aa-36348cdd83a9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:55.184612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ddcae-6d9c-44c7-a266-def31def38fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:55.188381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ddcae-6d9c-44c7-a266-def31def38fd admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:55.188381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ddcae-6d9c-44c7-a266-def31def38fd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:55.188381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ddcae-6d9c-44c7-a266-def31def38fd admin admin] PATH_INFO: `/v3/projects/7ecca54d58874c96aab5c41c5e5fda67/users/acec3b32e8034eae8e112a647e93bce0/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:55.225733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ddcae-6d9c-44c7-a266-def31def38fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ecca54d58874c96aab5c41c5e5fda67, user_id=acec3b32e8034eae8e112a647e93bce0, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:55.230141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ddcae-6d9c-44c7-a266-def31def38fd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:55.309724 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2168/4315] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:55 2022] PUT /identity/v3/projects/7ecca54d58874c96aab5c41c5e5fda67/users/acec3b32e8034eae8e112a647e93bce0/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:55.311522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2ed7f-8762-4636-a0a4-d7d627eec452 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:55.331259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2ed7f-8762-4636-a0a4-d7d627eec452 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:55.572644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37478f5d-ddb2-48a2-8333-9ddd52a8b57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:55.573052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37478f5d-ddb2-48a2-8333-9ddd52a8b57a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:55.573342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37478f5d-ddb2-48a2-8333-9ddd52a8b57a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:55.573626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37478f5d-ddb2-48a2-8333-9ddd52a8b57a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:55.574418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37478f5d-ddb2-48a2-8333-9ddd52a8b57a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:55.576816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37478f5d-ddb2-48a2-8333-9ddd52a8b57a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:55.592269 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2148/4316] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:15:55.597041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ddcae-6d9c-44c7-a266-def31def38fd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:55.617896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ddcae-6d9c-44c7-a266-def31def38fd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:55.625763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d65eb-a6fc-4d25-935d-bdd1dd9ac7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:55.627118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d65eb-a6fc-4d25-935d-bdd1dd9ac7d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:55.627784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d65eb-a6fc-4d25-935d-bdd1dd9ac7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:55.628202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d65eb-a6fc-4d25-935d-bdd1dd9ac7d8 admin admin] PATH_INFO: `/v3/projects/7ecca54d58874c96aab5c41c5e5fda67/users/acec3b32e8034eae8e112a647e93bce0/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:55.634376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d65eb-a6fc-4d25-935d-bdd1dd9ac7d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ecca54d58874c96aab5c41c5e5fda67, user_id=acec3b32e8034eae8e112a647e93bce0, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:55.641689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d65eb-a6fc-4d25-935d-bdd1dd9ac7d8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:55.656409 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2169/4317] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:15:55 2022] PUT /identity/v3/projects/7ecca54d58874c96aab5c41c5e5fda67/users/acec3b32e8034eae8e112a647e93bce0/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:15:55.666709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705eeb1-7c72-42fc-b043-beee5e2da532 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:55.667011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705eeb1-7c72-42fc-b043-beee5e2da532 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:55.667347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705eeb1-7c72-42fc-b043-beee5e2da532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:55.764579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d705eeb1-7c72-42fc-b043-beee5e2da532 None None] MFA Rules not processed for user `acec3b32e8034eae8e112a647e93bce0`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:55.766539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d705eeb1-7c72-42fc-b043-beee5e2da532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:55.884458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:55.935805 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2149/4318] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:55 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:15:55.954582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d65eb-a6fc-4d25-935d-bdd1dd9ac7d8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:15:55.956486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cd0d65eb-a6fc-4d25-935d-bdd1dd9ac7d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:56.068775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d65eb-a6fc-4d25-935d-bdd1dd9ac7d8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:15:56.076470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34edd7d-221a-4766-b6bc-458263191872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:15:56.077781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34edd7d-221a-4766-b6bc-458263191872 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:56.078620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34edd7d-221a-4766-b6bc-458263191872 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:56.079292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34edd7d-221a-4766-b6bc-458263191872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:56.096380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34edd7d-221a-4766-b6bc-458263191872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:15:56.110299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34edd7d-221a-4766-b6bc-458263191872 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:15:56.126098 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2170/4319] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:15:55 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:15:57.391316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed301de-9191-43ea-9070-74786ccb9548 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:15:57.392305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed301de-9191-43ea-9070-74786ccb9548 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:15:57.392830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed301de-9191-43ea-9070-74786ccb9548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:15:57.415707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8ed301de-9191-43ea-9070-74786ccb9548 None None] MFA Rules not processed for user `572170ccd67e4facbf10dad7cd1491c1`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:15:57.418112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8ed301de-9191-43ea-9070-74786ccb9548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:15:57.531408 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2150/4320] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:15:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:00.485813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34edd7d-221a-4766-b6bc-458263191872 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:00.488072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d34edd7d-221a-4766-b6bc-458263191872 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:00.524488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:00.597558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34edd7d-221a-4766-b6bc-458263191872 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:00.605488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1cece4-f845-4f81-ab35-ddd95ccc03bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:00.606484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cece4-f845-4f81-ab35-ddd95ccc03bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:00.606811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cece4-f845-4f81-ab35-ddd95ccc03bf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:00.607109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1cece4-f845-4f81-ab35-ddd95ccc03bf admin admin] PATH_INFO: `/v3/users/fcfb9ccd45dd44bb9747fb68ac912c7c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:00.608611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1cece4-f845-4f81-ab35-ddd95ccc03bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcfb9ccd45dd44bb9747fb68ac912c7c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:00.610954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1cece4-f845-4f81-ab35-ddd95ccc03bf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:00.879377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0a1cece4-f845-4f81-ab35-ddd95ccc03bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcfb9ccd45dd44bb9747fb68ac912c7c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:00.894758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed301de-9191-43ea-9070-74786ccb9548 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:00.899448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0a1cece4-f845-4f81-ab35-ddd95ccc03bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcfb9ccd45dd44bb9747fb68ac912c7c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:00.900337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0a1cece4-f845-4f81-ab35-ddd95ccc03bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcfb9ccd45dd44bb9747fb68ac912c7c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:00.901258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0a1cece4-f845-4f81-ab35-ddd95ccc03bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcfb9ccd45dd44bb9747fb68ac912c7c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:00.907237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed301de-9191-43ea-9070-74786ccb9548 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:00.925233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e91c67-dcd3-4e22-9be5-c97dd51d6263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:00.926534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e91c67-dcd3-4e22-9be5-c97dd51d6263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:00.926817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e91c67-dcd3-4e22-9be5-c97dd51d6263 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:00.927085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e91c67-dcd3-4e22-9be5-c97dd51d6263 admin admin] PATH_INFO: `/v3/users/33613abc5f2240f7977d957a422547ba` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:00.932119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e91c67-dcd3-4e22-9be5-c97dd51d6263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33613abc5f2240f7977d957a422547ba)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:00.934117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e91c67-dcd3-4e22-9be5-c97dd51d6263 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:00.976829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0a1cece4-f845-4f81-ab35-ddd95ccc03bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcfb9ccd45dd44bb9747fb68ac912c7c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:01.004259 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2171/4321] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:16:00 2022] DELETE /identity/v3/users/fcfb9ccd45dd44bb9747fb68ac912c7c => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:01.009028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0faf44-02f0-4f85-9b08-12f86da96572 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:01.009404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0faf44-02f0-4f85-9b08-12f86da96572 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:01.009766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0faf44-02f0-4f85-9b08-12f86da96572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:01.055583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ba0faf44-02f0-4f85-9b08-12f86da96572 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:01.059095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ba0faf44-02f0-4f85-9b08-12f86da96572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:01.209728 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2172/4322] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:16:01.219627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0faf44-02f0-4f85-9b08-12f86da96572 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:01.221029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ba0faf44-02f0-4f85-9b08-12f86da96572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:01.263451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-88e91c67-dcd3-4e22-9be5-c97dd51d6263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33613abc5f2240f7977d957a422547ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:01.265030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-88e91c67-dcd3-4e22-9be5-c97dd51d6263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33613abc5f2240f7977d957a422547ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:01.277456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-88e91c67-dcd3-4e22-9be5-c97dd51d6263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33613abc5f2240f7977d957a422547ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:01.307140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0faf44-02f0-4f85-9b08-12f86da96572 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:01.314330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9730f-a752-4a5d-9846-71acc8b0f5bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:01.315225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9730f-a752-4a5d-9846-71acc8b0f5bb service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:01.315741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9730f-a752-4a5d-9846-71acc8b0f5bb service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:01.316185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9730f-a752-4a5d-9846-71acc8b0f5bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:01.329421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9730f-a752-4a5d-9846-71acc8b0f5bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:01.331115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9730f-a752-4a5d-9846-71acc8b0f5bb service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:01.343740 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2173/4323] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:16:01.383455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-88e91c67-dcd3-4e22-9be5-c97dd51d6263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33613abc5f2240f7977d957a422547ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:01.384848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-88e91c67-dcd3-4e22-9be5-c97dd51d6263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33613abc5f2240f7977d957a422547ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:01.429405 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2151/4324] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:16:00 2022] DELETE /identity/v3/users/33613abc5f2240f7977d957a422547ba => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:16:01.430105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec848190-930e-4996-ab5a-feeb1c4a671d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:01.430477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec848190-930e-4996-ab5a-feeb1c4a671d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:01.430833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec848190-930e-4996-ab5a-feeb1c4a671d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:01.473497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ec848190-930e-4996-ab5a-feeb1c4a671d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:01.479609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ec848190-930e-4996-ab5a-feeb1c4a671d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:01.487602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:01.565319 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2174/4325] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:01.575543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e91c67-dcd3-4e22-9be5-c97dd51d6263 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:01.580918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-88e91c67-dcd3-4e22-9be5-c97dd51d6263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:01.654682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e91c67-dcd3-4e22-9be5-c97dd51d6263 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:01.658226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c2dd5-ff82-409b-b460-3fd4d661b96f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:01.661628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c2dd5-ff82-409b-b460-3fd4d661b96f service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:01.661628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c2dd5-ff82-409b-b460-3fd4d661b96f service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:01.661628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c2dd5-ff82-409b-b460-3fd4d661b96f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:01.672343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c2dd5-ff82-409b-b460-3fd4d661b96f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:01.674318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c2dd5-ff82-409b-b460-3fd4d661b96f service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:01.679247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:01.705830 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2152/4326] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:16:01.706765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec848190-930e-4996-ab5a-feeb1c4a671d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:01.715261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ec848190-930e-4996-ab5a-feeb1c4a671d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:01.735061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec848190-930e-4996-ab5a-feeb1c4a671d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:01.750457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a9380-246e-4113-8d23-e2f40edef5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:01.751424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a9380-246e-4113-8d23-e2f40edef5b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:01.751717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a9380-246e-4113-8d23-e2f40edef5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:01.752232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a9380-246e-4113-8d23-e2f40edef5b5 admin admin] PATH_INFO: `/v3/projects/b04e5d3e331c4c45b309b5041a99b681` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:01.755108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04a9380-246e-4113-8d23-e2f40edef5b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b04e5d3e331c4c45b309b5041a99b681)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:01.761054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04a9380-246e-4113-8d23-e2f40edef5b5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:01.924852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f04a9380-246e-4113-8d23-e2f40edef5b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b04e5d3e331c4c45b309b5041a99b681', 'request_id': 'req-f04a9380-246e-4113-8d23-e2f40edef5b5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:01.946608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f04a9380-246e-4113-8d23-e2f40edef5b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b04e5d3e331c4c45b309b5041a99b681', 'request_id': 'req-f04a9380-246e-4113-8d23-e2f40edef5b5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:02.048518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f04a9380-246e-4113-8d23-e2f40edef5b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b04e5d3e331c4c45b309b5041a99b681', 'request_id': 'req-f04a9380-246e-4113-8d23-e2f40edef5b5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:02.083832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f04a9380-246e-4113-8d23-e2f40edef5b5 admin admin] The token cache is being invalidate because project b04e5d3e331c4c45b309b5041a99b681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:16:02.084280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f04a9380-246e-4113-8d23-e2f40edef5b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:02.086236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f04a9380-246e-4113-8d23-e2f40edef5b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:02.092656 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2175/4327] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:16:01 2022] DELETE /identity/v3/projects/b04e5d3e331c4c45b309b5041a99b681 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:02.096204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6357b4-3673-4a56-b296-8a94b9d67e9e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:02.103337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6357b4-3673-4a56-b296-8a94b9d67e9e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:02.103337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6357b4-3673-4a56-b296-8a94b9d67e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:02.126616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4f6357b4-3673-4a56-b296-8a94b9d67e9e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:02.128469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4f6357b4-3673-4a56-b296-8a94b9d67e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:02.230183 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2153/4328] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:02.237741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6da81-c5eb-44bb-86bf-5348289225be None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:02.238155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6da81-c5eb-44bb-86bf-5348289225be None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:02.238468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6da81-c5eb-44bb-86bf-5348289225be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:02.259170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-92b6da81-c5eb-44bb-86bf-5348289225be None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:02.260649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-92b6da81-c5eb-44bb-86bf-5348289225be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:02.281921 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2176/4329] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:02.283684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6357b4-3673-4a56-b296-8a94b9d67e9e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:02.294859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6357b4-3673-4a56-b296-8a94b9d67e9e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:02.304832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decb9b0-677c-4e9d-80e8-ba6637d65916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:02.305722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decb9b0-677c-4e9d-80e8-ba6637d65916 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:02.306141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decb9b0-677c-4e9d-80e8-ba6637d65916 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:02.306575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decb9b0-677c-4e9d-80e8-ba6637d65916 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:02.308958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decb9b0-677c-4e9d-80e8-ba6637d65916 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:16:02.309768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decb9b0-677c-4e9d-80e8-ba6637d65916 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:02.312068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decb9b0-677c-4e9d-80e8-ba6637d65916 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:02.326052 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2154/4330] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:16:02.330059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6da81-c5eb-44bb-86bf-5348289225be None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:02.341801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6da81-c5eb-44bb-86bf-5348289225be None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:02.347791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf09b00-9f09-49fe-9d0a-1a19c72c743c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:02.348620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf09b00-9f09-49fe-9d0a-1a19c72c743c admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:02.349132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf09b00-9f09-49fe-9d0a-1a19c72c743c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:02.349513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf09b00-9f09-49fe-9d0a-1a19c72c743c admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:02.350901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf09b00-9f09-49fe-9d0a-1a19c72c743c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:02.352568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf09b00-9f09-49fe-9d0a-1a19c72c743c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:02.444507 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2177/4331] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:16:02 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:16:02.449494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decb9b0-677c-4e9d-80e8-ba6637d65916 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:02.465173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decb9b0-677c-4e9d-80e8-ba6637d65916 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:02.579752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d105280-0080-4d5b-a007-e0b07c9af902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:02.580538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d105280-0080-4d5b-a007-e0b07c9af902 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:02.581039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d105280-0080-4d5b-a007-e0b07c9af902 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:02.581393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d105280-0080-4d5b-a007-e0b07c9af902 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:02.582261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d105280-0080-4d5b-a007-e0b07c9af902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:02.586593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d105280-0080-4d5b-a007-e0b07c9af902 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:02.714665 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2155/4332] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:16:02 2022] POST /identity/v3/users => generated 283 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:16:02.716901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf09b00-9f09-49fe-9d0a-1a19c72c743c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:02.730651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf09b00-9f09-49fe-9d0a-1a19c72c743c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:02.736324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fe6dc-b95e-49f5-aca4-b1280ccc59bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:02.737017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fe6dc-b95e-49f5-aca4-b1280ccc59bd admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:02.737306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fe6dc-b95e-49f5-aca4-b1280ccc59bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:02.737589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226fe6dc-b95e-49f5-aca4-b1280ccc59bd admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:02.738687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fe6dc-b95e-49f5-aca4-b1280ccc59bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:02.740392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226fe6dc-b95e-49f5-aca4-b1280ccc59bd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:02.759982 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2178/4333] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:16:02.760210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d105280-0080-4d5b-a007-e0b07c9af902 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:02.778801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d105280-0080-4d5b-a007-e0b07c9af902 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:02.788570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa66546-82dc-4309-bc5b-1b3748d8fa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:02.791122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa66546-82dc-4309-bc5b-1b3748d8fa6e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:02.791418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa66546-82dc-4309-bc5b-1b3748d8fa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:02.791895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa66546-82dc-4309-bc5b-1b3748d8fa6e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:02.792705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa66546-82dc-4309-bc5b-1b3748d8fa6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:02.807265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa66546-82dc-4309-bc5b-1b3748d8fa6e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:02.825927 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2156/4334] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:16:02.830108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fe6dc-b95e-49f5-aca4-b1280ccc59bd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:02.846114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226fe6dc-b95e-49f5-aca4-b1280ccc59bd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:02.861175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa66546-82dc-4309-bc5b-1b3748d8fa6e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:02.865450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-dfa66546-82dc-4309-bc5b-1b3748d8fa6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:02.874569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac8290f-cf25-447d-ae66-677103af6781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:02.884916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8290f-cf25-447d-ae66-677103af6781 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:02.885233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8290f-cf25-447d-ae66-677103af6781 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:02.892777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac8290f-cf25-447d-ae66-677103af6781 admin admin] PATH_INFO: `/v3/projects/b5bb286579e34b67b1102dd86cc1fd0a/users/269519c04f264104aef296030828caec/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:02.901608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa66546-82dc-4309-bc5b-1b3748d8fa6e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:02.914856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8290f-cf25-447d-ae66-677103af6781 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5bb286579e34b67b1102dd86cc1fd0a, user_id=269519c04f264104aef296030828caec, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:02.921209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac8290f-cf25-447d-ae66-677103af6781 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:02.923418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f32f12-56a2-4cb3-bb48-abc91ae74139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:02.924826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f32f12-56a2-4cb3-bb48-abc91ae74139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:02.925126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f32f12-56a2-4cb3-bb48-abc91ae74139 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:02.925408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f32f12-56a2-4cb3-bb48-abc91ae74139 admin admin] PATH_INFO: `/v3/projects/65d7e6533b364355bba7bfd71918ba7a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:02.931900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f32f12-56a2-4cb3-bb48-abc91ae74139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65d7e6533b364355bba7bfd71918ba7a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:02.934177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f32f12-56a2-4cb3-bb48-abc91ae74139 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:02.998521 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2179/4335] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:16:02 2022] PUT /identity/v3/projects/b5bb286579e34b67b1102dd86cc1fd0a/users/269519c04f264104aef296030828caec/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:03.018053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ea83d-3abf-4dff-aa98-eff7deef2af0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:03.018422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ea83d-3abf-4dff-aa98-eff7deef2af0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:03.018779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ea83d-3abf-4dff-aa98-eff7deef2af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:03.038714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:03.059750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-10f32f12-56a2-4cb3-bb48-abc91ae74139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65d7e6533b364355bba7bfd71918ba7a', 'request_id': 'req-10f32f12-56a2-4cb3-bb48-abc91ae74139'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:03.080012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-778ea83d-3abf-4dff-aa98-eff7deef2af0 None None] MFA Rules not processed for user `269519c04f264104aef296030828caec`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:03.083398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-778ea83d-3abf-4dff-aa98-eff7deef2af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:03.086521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-10f32f12-56a2-4cb3-bb48-abc91ae74139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65d7e6533b364355bba7bfd71918ba7a', 'request_id': 'req-10f32f12-56a2-4cb3-bb48-abc91ae74139'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:03.088384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:03.156306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-10f32f12-56a2-4cb3-bb48-abc91ae74139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65d7e6533b364355bba7bfd71918ba7a', 'request_id': 'req-10f32f12-56a2-4cb3-bb48-abc91ae74139'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:03.175648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-10f32f12-56a2-4cb3-bb48-abc91ae74139 admin admin] The token cache is being invalidate because project 65d7e6533b364355bba7bfd71918ba7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:16:03.176633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-10f32f12-56a2-4cb3-bb48-abc91ae74139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:03.177460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-10f32f12-56a2-4cb3-bb48-abc91ae74139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:03.180697 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2157/4336] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:16:02 2022] DELETE /identity/v3/projects/65d7e6533b364355bba7bfd71918ba7a => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:03.189878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f32f12-56a2-4cb3-bb48-abc91ae74139 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:03.191530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-10f32f12-56a2-4cb3-bb48-abc91ae74139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:03.245049 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2180/4337] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:03.262749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ea83d-3abf-4dff-aa98-eff7deef2af0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:03.268370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-778ea83d-3abf-4dff-aa98-eff7deef2af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:03.367725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f32f12-56a2-4cb3-bb48-abc91ae74139 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:03.376789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cf828e-d11d-4c68-abb9-c09b9fce4653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:03.377007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf828e-d11d-4c68-abb9-c09b9fce4653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:03.377176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf828e-d11d-4c68-abb9-c09b9fce4653 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:03.377463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cf828e-d11d-4c68-abb9-c09b9fce4653 admin admin] PATH_INFO: `/v3/users/acec3b32e8034eae8e112a647e93bce0` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:03.378873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf828e-d11d-4c68-abb9-c09b9fce4653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acec3b32e8034eae8e112a647e93bce0)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:03.381178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cf828e-d11d-4c68-abb9-c09b9fce4653 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:03.396144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ea83d-3abf-4dff-aa98-eff7deef2af0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:03.404767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77a188-31f0-46a3-8806-8a7b021fb27c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:03.405456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77a188-31f0-46a3-8806-8a7b021fb27c service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:03.405744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77a188-31f0-46a3-8806-8a7b021fb27c service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:03.406094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77a188-31f0-46a3-8806-8a7b021fb27c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:03.416282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77a188-31f0-46a3-8806-8a7b021fb27c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:03.418244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77a188-31f0-46a3-8806-8a7b021fb27c service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:03.432061 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2181/4338] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:16:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:16:04.086602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-98cf828e-d11d-4c68-abb9-c09b9fce4653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acec3b32e8034eae8e112a647e93bce0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:04.087784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-98cf828e-d11d-4c68-abb9-c09b9fce4653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acec3b32e8034eae8e112a647e93bce0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:04.115339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-98cf828e-d11d-4c68-abb9-c09b9fce4653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acec3b32e8034eae8e112a647e93bce0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:04.240164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-98cf828e-d11d-4c68-abb9-c09b9fce4653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acec3b32e8034eae8e112a647e93bce0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:04.241765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-98cf828e-d11d-4c68-abb9-c09b9fce4653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acec3b32e8034eae8e112a647e93bce0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:04.283221 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2158/4339] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:16:03 2022] DELETE /identity/v3/users/acec3b32e8034eae8e112a647e93bce0 => generated 0 bytes in 1094 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:04.685208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77a188-31f0-46a3-8806-8a7b021fb27c service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:04.707839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77a188-31f0-46a3-8806-8a7b021fb27c service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:04.808731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e5375-677f-4d14-933b-4de6f16c04fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:04.809884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e5375-677f-4d14-933b-4de6f16c04fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:04.810449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e5375-677f-4d14-933b-4de6f16c04fc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:04.810832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e5375-677f-4d14-933b-4de6f16c04fc admin admin] PATH_INFO: `/v3/projects/7ecca54d58874c96aab5c41c5e5fda67` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:04.812384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e5375-677f-4d14-933b-4de6f16c04fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ecca54d58874c96aab5c41c5e5fda67)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:04.815711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e5375-677f-4d14-933b-4de6f16c04fc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:04.946514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:04.956046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc2e5375-677f-4d14-933b-4de6f16c04fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ecca54d58874c96aab5c41c5e5fda67', 'request_id': 'req-cc2e5375-677f-4d14-933b-4de6f16c04fc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:04.965496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc2e5375-677f-4d14-933b-4de6f16c04fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ecca54d58874c96aab5c41c5e5fda67', 'request_id': 'req-cc2e5375-677f-4d14-933b-4de6f16c04fc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:05.025184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc2e5375-677f-4d14-933b-4de6f16c04fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ecca54d58874c96aab5c41c5e5fda67', 'request_id': 'req-cc2e5375-677f-4d14-933b-4de6f16c04fc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:05.042170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc2e5375-677f-4d14-933b-4de6f16c04fc admin admin] The token cache is being invalidate because project 7ecca54d58874c96aab5c41c5e5fda67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:16:05.042583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc2e5375-677f-4d14-933b-4de6f16c04fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:05.043161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc2e5375-677f-4d14-933b-4de6f16c04fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:05.048300 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2182/4340] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:16:04 2022] DELETE /identity/v3/projects/7ecca54d58874c96aab5c41c5e5fda67 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:05.049795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fa9ff-bc78-429d-8435-f2e56f7e8a94 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:05.050143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fa9ff-bc78-429d-8435-f2e56f7e8a94 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:05.050432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fa9ff-bc78-429d-8435-f2e56f7e8a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:05.100544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7b3fa9ff-bc78-429d-8435-f2e56f7e8a94 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:05.102267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7b3fa9ff-bc78-429d-8435-f2e56f7e8a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:05.212465 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2159/4341] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:05.221675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54591f7c-f965-47dc-9c7b-f3bebe4907f2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:05.221989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54591f7c-f965-47dc-9c7b-f3bebe4907f2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:05.222301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54591f7c-f965-47dc-9c7b-f3bebe4907f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:05.246543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-54591f7c-f965-47dc-9c7b-f3bebe4907f2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:05.247770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-54591f7c-f965-47dc-9c7b-f3bebe4907f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:05.259444 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2183/4342] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:05.265138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fa9ff-bc78-429d-8435-f2e56f7e8a94 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:05.281093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fa9ff-bc78-429d-8435-f2e56f7e8a94 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:05.286414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40c966-73dc-4193-b651-d7cba25bba7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:05.287299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40c966-73dc-4193-b651-d7cba25bba7c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:05.288770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40c966-73dc-4193-b651-d7cba25bba7c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:05.289219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40c966-73dc-4193-b651-d7cba25bba7c admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:05.290157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40c966-73dc-4193-b651-d7cba25bba7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:16:05.290616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40c966-73dc-4193-b651-d7cba25bba7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:05.292397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40c966-73dc-4193-b651-d7cba25bba7c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:05.294378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:05.329571 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2160/4343] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:16:05.331813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54591f7c-f965-47dc-9c7b-f3bebe4907f2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:05.376713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54591f7c-f965-47dc-9c7b-f3bebe4907f2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:05.394191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fff49-b2a4-4c04-86ff-9ce7a8649a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:05.394191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fff49-b2a4-4c04-86ff-9ce7a8649a5d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:05.394191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fff49-b2a4-4c04-86ff-9ce7a8649a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:05.394191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65fff49-b2a4-4c04-86ff-9ce7a8649a5d admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:05.394537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fff49-b2a4-4c04-86ff-9ce7a8649a5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:05.398254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65fff49-b2a4-4c04-86ff-9ce7a8649a5d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:05.552514 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2184/4344] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:16:05 2022] POST /identity/v3/projects => generated 397 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:16:05.556763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40c966-73dc-4193-b651-d7cba25bba7c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:05.572948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40c966-73dc-4193-b651-d7cba25bba7c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:05.647573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81054005-0a9a-47cb-94a9-960c1275555c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:05.648385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81054005-0a9a-47cb-94a9-960c1275555c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:05.648745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81054005-0a9a-47cb-94a9-960c1275555c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:05.649102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81054005-0a9a-47cb-94a9-960c1275555c admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:05.650036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81054005-0a9a-47cb-94a9-960c1275555c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:05.652305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81054005-0a9a-47cb-94a9-960c1275555c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:05.822383 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2161/4345] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:16:05 2022] POST /identity/v3/users => generated 299 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:16:05.826617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fff49-b2a4-4c04-86ff-9ce7a8649a5d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:05.880692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65fff49-b2a4-4c04-86ff-9ce7a8649a5d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:05.895316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e00f2-d74f-43f7-9f8a-0404bf6446d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:05.896157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e00f2-d74f-43f7-9f8a-0404bf6446d3 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:05.896517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e00f2-d74f-43f7-9f8a-0404bf6446d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:05.897435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e00f2-d74f-43f7-9f8a-0404bf6446d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:05.899008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e00f2-d74f-43f7-9f8a-0404bf6446d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:05.901739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e00f2-d74f-43f7-9f8a-0404bf6446d3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:05.927591 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2185/4346] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:16:05.933118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81054005-0a9a-47cb-94a9-960c1275555c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:05.951206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81054005-0a9a-47cb-94a9-960c1275555c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:05.964721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4944e-01fe-4185-8601-84d96670b442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:05.969131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4944e-01fe-4185-8601-84d96670b442 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:05.973957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4944e-01fe-4185-8601-84d96670b442 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:05.973957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4944e-01fe-4185-8601-84d96670b442 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:05.980697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a4944e-01fe-4185-8601-84d96670b442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:05.987160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a4944e-01fe-4185-8601-84d96670b442 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:06.015462 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2162/4347] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:16:06.018275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e00f2-d74f-43f7-9f8a-0404bf6446d3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:06.040128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e00f2-d74f-43f7-9f8a-0404bf6446d3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:06.040128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171d1a7-2468-4c5b-8be6-d5122c2caeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:06.040511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171d1a7-2468-4c5b-8be6-d5122c2caeb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:06.040810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171d1a7-2468-4c5b-8be6-d5122c2caeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:06.041279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171d1a7-2468-4c5b-8be6-d5122c2caeb9 admin admin] PATH_INFO: `/v3/projects/5201ad6f7ad54d028fd0cb152cecdc41/users/a3ef7c5e78ee497b9aaf5d199962f75e/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:06.071701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171d1a7-2468-4c5b-8be6-d5122c2caeb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5201ad6f7ad54d028fd0cb152cecdc41, user_id=a3ef7c5e78ee497b9aaf5d199962f75e, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:06.083903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171d1a7-2468-4c5b-8be6-d5122c2caeb9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:06.197325 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2186/4348] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:16:06 2022] PUT /identity/v3/projects/5201ad6f7ad54d028fd0cb152cecdc41/users/a3ef7c5e78ee497b9aaf5d199962f75e/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:06.207793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c9a62-5cba-4a7e-998c-d47933d5c2f7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:06.208159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c9a62-5cba-4a7e-998c-d47933d5c2f7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:06.208514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28c9a62-5cba-4a7e-998c-d47933d5c2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:06.282774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d28c9a62-5cba-4a7e-998c-d47933d5c2f7 None None] MFA Rules not processed for user `a3ef7c5e78ee497b9aaf5d199962f75e`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:06.284658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d28c9a62-5cba-4a7e-998c-d47933d5c2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:06.424958 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2163/4349] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:06.428812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171d1a7-2468-4c5b-8be6-d5122c2caeb9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:06.553422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171d1a7-2468-4c5b-8be6-d5122c2caeb9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:06.565491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf0578-cc7a-45e1-965e-b2f681ebcc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:06.568578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf0578-cc7a-45e1-965e-b2f681ebcc11 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:06.570740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf0578-cc7a-45e1-965e-b2f681ebcc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:06.571049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf0578-cc7a-45e1-965e-b2f681ebcc11 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:06.572707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf0578-cc7a-45e1-965e-b2f681ebcc11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:06.583053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf0578-cc7a-45e1-965e-b2f681ebcc11 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:06.683163 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2187/4350] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:16:06 2022] POST /identity/v3/projects => generated 399 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:16:06.684425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c9a62-5cba-4a7e-998c-d47933d5c2f7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:06.723010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28c9a62-5cba-4a7e-998c-d47933d5c2f7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:06.813272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2274a-7b4e-4ada-9553-3e470a74e72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:06.813971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2274a-7b4e-4ada-9553-3e470a74e72d admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:06.814264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2274a-7b4e-4ada-9553-3e470a74e72d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:06.814546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2274a-7b4e-4ada-9553-3e470a74e72d admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:06.815542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2274a-7b4e-4ada-9553-3e470a74e72d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:06.817605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2274a-7b4e-4ada-9553-3e470a74e72d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:06.830686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:06.951856 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2164/4351] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:16:06 2022] POST /identity/v3/users => generated 306 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:16:06.954177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf0578-cc7a-45e1-965e-b2f681ebcc11 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:06.970891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf0578-cc7a-45e1-965e-b2f681ebcc11 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:06.986870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada29c72-e2be-4e96-a7b8-f5ead1ce0ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:06.987869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada29c72-e2be-4e96-a7b8-f5ead1ce0ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:06.988328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada29c72-e2be-4e96-a7b8-f5ead1ce0ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:06.988602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada29c72-e2be-4e96-a7b8-f5ead1ce0ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:06.989339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada29c72-e2be-4e96-a7b8-f5ead1ce0ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:06.991515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada29c72-e2be-4e96-a7b8-f5ead1ce0ee7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:07.013099 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2188/4352] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:16:07.018389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2274a-7b4e-4ada-9553-3e470a74e72d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:07.031416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2274a-7b4e-4ada-9553-3e470a74e72d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:07.040918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef80001-9bfb-4c31-b6e9-673bf8ca19c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:07.042243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef80001-9bfb-4c31-b6e9-673bf8ca19c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:07.043832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef80001-9bfb-4c31-b6e9-673bf8ca19c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:07.044346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef80001-9bfb-4c31-b6e9-673bf8ca19c8 admin admin] PATH_INFO: `/v3/projects/9edd0e9b412c47a1b85e47c89f0056c8/users/2414aa781a16426b960520ae5c44c12e/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:07.082978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef80001-9bfb-4c31-b6e9-673bf8ca19c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9edd0e9b412c47a1b85e47c89f0056c8, user_id=2414aa781a16426b960520ae5c44c12e, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:07.087319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef80001-9bfb-4c31-b6e9-673bf8ca19c8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:07.180146 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2165/4353] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:16:07 2022] PUT /identity/v3/projects/9edd0e9b412c47a1b85e47c89f0056c8/users/2414aa781a16426b960520ae5c44c12e/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:07.183961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada29c72-e2be-4e96-a7b8-f5ead1ce0ee7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:07.199931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada29c72-e2be-4e96-a7b8-f5ead1ce0ee7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:07.270942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:07.292108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd780b-6d17-46e6-b751-c8878ce30e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:07.292782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd780b-6d17-46e6-b751-c8878ce30e36 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:07.293067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd780b-6d17-46e6-b751-c8878ce30e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:07.293349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd780b-6d17-46e6-b751-c8878ce30e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:07.294182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd780b-6d17-46e6-b751-c8878ce30e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:07.295783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd780b-6d17-46e6-b751-c8878ce30e36 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:07.312998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef80001-9bfb-4c31-b6e9-673bf8ca19c8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:07.314141 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2189/4354] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:16:07.341194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef80001-9bfb-4c31-b6e9-673bf8ca19c8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:07.359167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c73c9a-f523-4730-95fc-c866d9737d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:07.362518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c73c9a-f523-4730-95fc-c866d9737d4d admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:07.362876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c73c9a-f523-4730-95fc-c866d9737d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:07.364377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c73c9a-f523-4730-95fc-c866d9737d4d admin admin] PATH_INFO: `/v3/projects/9edd0e9b412c47a1b85e47c89f0056c8/users/2414aa781a16426b960520ae5c44c12e/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:07.369294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c73c9a-f523-4730-95fc-c866d9737d4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9edd0e9b412c47a1b85e47c89f0056c8, user_id=2414aa781a16426b960520ae5c44c12e, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:07.376792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c73c9a-f523-4730-95fc-c866d9737d4d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:07.393388 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2166/4355] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:16:07 2022] PUT /identity/v3/projects/9edd0e9b412c47a1b85e47c89f0056c8/users/2414aa781a16426b960520ae5c44c12e/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:07.404355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919caa0-acff-4132-bc5d-6ddcb1e5ec25 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:07.404646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919caa0-acff-4132-bc5d-6ddcb1e5ec25 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:07.404942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1919caa0-acff-4132-bc5d-6ddcb1e5ec25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:07.484155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1919caa0-acff-4132-bc5d-6ddcb1e5ec25 None None] MFA Rules not processed for user `2414aa781a16426b960520ae5c44c12e`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:07.598671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1919caa0-acff-4132-bc5d-6ddcb1e5ec25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:07.641300 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2190/4356] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:07 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:16:07.657736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c73c9a-f523-4730-95fc-c866d9737d4d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:07.659808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d8c73c9a-f523-4730-95fc-c866d9737d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:07.804970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c73c9a-f523-4730-95fc-c866d9737d4d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:07.811225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2b64d-739a-4bf3-a95c-841bd76d7ec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:07.812768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2b64d-739a-4bf3-a95c-841bd76d7ec4 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:07.813160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2b64d-739a-4bf3-a95c-841bd76d7ec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:07.813817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc2b64d-739a-4bf3-a95c-841bd76d7ec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:07.977068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2b64d-739a-4bf3-a95c-841bd76d7ec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:07.981825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc2b64d-739a-4bf3-a95c-841bd76d7ec4 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:08.002938 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2167/4357] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:16:07 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:16:09.514885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8dcb4a-9571-48b5-a854-b6ad0b79693c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:09.515438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8dcb4a-9571-48b5-a854-b6ad0b79693c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:09.515438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8dcb4a-9571-48b5-a854-b6ad0b79693c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:09.534036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ce8dcb4a-9571-48b5-a854-b6ad0b79693c None None] MFA Rules not processed for user `af446b3d802a42e3bd72b08bace8ea8d`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:09.535250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ce8dcb4a-9571-48b5-a854-b6ad0b79693c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:09.578632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2b64d-739a-4bf3-a95c-841bd76d7ec4 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:09.581251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6bc2b64d-739a-4bf3-a95c-841bd76d7ec4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:09.652527 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2191/4358] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:16:09.760497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:09.772995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc2b64d-739a-4bf3-a95c-841bd76d7ec4 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:09.783020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad058d7e-1487-4b60-8007-d617bef83cc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:09.784568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad058d7e-1487-4b60-8007-d617bef83cc3 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:09.786042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad058d7e-1487-4b60-8007-d617bef83cc3 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:09.786467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad058d7e-1487-4b60-8007-d617bef83cc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:09.799109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad058d7e-1487-4b60-8007-d617bef83cc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:09.801206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad058d7e-1487-4b60-8007-d617bef83cc3 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:09.832072 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2168/4359] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:16:09 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:16:15.844487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bb003-bdc2-467b-9b7f-0454b0e05514 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:15.845324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bb003-bdc2-467b-9b7f-0454b0e05514 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:15.845752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3bb003-bdc2-467b-9b7f-0454b0e05514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:15.906566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0e3bb003-bdc2-467b-9b7f-0454b0e05514 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:15.908672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0e3bb003-bdc2-467b-9b7f-0454b0e05514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:16.039072 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2192/4360] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:16:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:16.052594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad058d7e-1487-4b60-8007-d617bef83cc3 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:16.054338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ad058d7e-1487-4b60-8007-d617bef83cc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:16.179988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad058d7e-1487-4b60-8007-d617bef83cc3 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:16.186045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff4196-7644-4de4-aa10-de86728b06de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:16.186969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff4196-7644-4de4-aa10-de86728b06de service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:16.187419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff4196-7644-4de4-aa10-de86728b06de service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:16.188040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff4196-7644-4de4-aa10-de86728b06de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:16.198088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff4196-7644-4de4-aa10-de86728b06de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:16.201244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff4196-7644-4de4-aa10-de86728b06de service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:16.210274 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2169/4361] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:16:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:16:17.695995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e27de-d151-433c-b2cf-6b926be687d9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:17.696501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e27de-d151-433c-b2cf-6b926be687d9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:17.696586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e27de-d151-433c-b2cf-6b926be687d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:17.705643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:17.721210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-dd3e27de-d151-433c-b2cf-6b926be687d9 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:17.722680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dd3e27de-d151-433c-b2cf-6b926be687d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:17.740207 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2193/4362] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:16:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:16:17.757753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff4196-7644-4de4-aa10-de86728b06de service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:17.771465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff4196-7644-4de4-aa10-de86728b06de service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:17.783547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958ca7a-d9fd-497f-b075-2d60acc03170 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:17.784758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958ca7a-d9fd-497f-b075-2d60acc03170 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:17.786454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958ca7a-d9fd-497f-b075-2d60acc03170 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:17.786454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4958ca7a-d9fd-497f-b075-2d60acc03170 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:17.795970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958ca7a-d9fd-497f-b075-2d60acc03170 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:17.797736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4958ca7a-d9fd-497f-b075-2d60acc03170 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:17.807587 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2170/4363] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:16:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:16:19.462671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9dfdc-45cc-4710-a8a7-f47616ffd54a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:19.463182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9dfdc-45cc-4710-a8a7-f47616ffd54a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:19.463786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9dfdc-45cc-4710-a8a7-f47616ffd54a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:19.489608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-69f9dfdc-45cc-4710-a8a7-f47616ffd54a None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:19.490426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-69f9dfdc-45cc-4710-a8a7-f47616ffd54a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:19.500667 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2194/4364] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:16:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:19.526641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958ca7a-d9fd-497f-b075-2d60acc03170 service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:19.570390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4958ca7a-d9fd-497f-b075-2d60acc03170 service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:19.591543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c92a0-d6e6-4979-b23d-a3567e6d5090 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:19.592484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c92a0-d6e6-4979-b23d-a3567e6d5090 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:19.592821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c92a0-d6e6-4979-b23d-a3567e6d5090 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:19.593056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c92a0-d6e6-4979-b23d-a3567e6d5090 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:19.610561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c92a0-d6e6-4979-b23d-a3567e6d5090 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:19.612555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c92a0-d6e6-4979-b23d-a3567e6d5090 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:19.624807 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2171/4365] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:16:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:16:21.981399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9dfdc-45cc-4710-a8a7-f47616ffd54a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:22.049141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9dfdc-45cc-4710-a8a7-f47616ffd54a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:22.056279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaa173-d3f6-45e8-8864-9bf2cdb60c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:22.058940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaa173-d3f6-45e8-8864-9bf2cdb60c4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:22.059286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaa173-d3f6-45e8-8864-9bf2cdb60c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:22.059623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaa173-d3f6-45e8-8864-9bf2cdb60c4a admin admin] PATH_INFO: `/v3/users/a3ef7c5e78ee497b9aaf5d199962f75e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:22.061100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aaa173-d3f6-45e8-8864-9bf2cdb60c4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3ef7c5e78ee497b9aaf5d199962f75e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:22.063138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aaa173-d3f6-45e8-8864-9bf2cdb60c4a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:22.354948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f4aaa173-d3f6-45e8-8864-9bf2cdb60c4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3ef7c5e78ee497b9aaf5d199962f75e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:22.365352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f4aaa173-d3f6-45e8-8864-9bf2cdb60c4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3ef7c5e78ee497b9aaf5d199962f75e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:22.366411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f4aaa173-d3f6-45e8-8864-9bf2cdb60c4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3ef7c5e78ee497b9aaf5d199962f75e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:22.371228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f4aaa173-d3f6-45e8-8864-9bf2cdb60c4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3ef7c5e78ee497b9aaf5d199962f75e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:22.429400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f4aaa173-d3f6-45e8-8864-9bf2cdb60c4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3ef7c5e78ee497b9aaf5d199962f75e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:22.460685 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2195/4366] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:16:21 2022] DELETE /identity/v3/users/a3ef7c5e78ee497b9aaf5d199962f75e => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:22.465234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aee0b3-c958-4e1a-9415-5395b25ff66b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:22.466102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aee0b3-c958-4e1a-9415-5395b25ff66b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:22.466540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aee0b3-c958-4e1a-9415-5395b25ff66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:22.488617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-88aee0b3-c958-4e1a-9415-5395b25ff66b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:22.490179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-88aee0b3-c958-4e1a-9415-5395b25ff66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:22.567259 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2172/4367] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:22.577735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaa173-d3f6-45e8-8864-9bf2cdb60c4a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:22.579220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f4aaa173-d3f6-45e8-8864-9bf2cdb60c4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:22.646966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaa173-d3f6-45e8-8864-9bf2cdb60c4a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:22.650813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc3721-e36a-41a1-8f1b-86879e28b697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:22.655031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc3721-e36a-41a1-8f1b-86879e28b697 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:22.655031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc3721-e36a-41a1-8f1b-86879e28b697 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:22.655031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cc3721-e36a-41a1-8f1b-86879e28b697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:22.662412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cc3721-e36a-41a1-8f1b-86879e28b697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:22.664845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cc3721-e36a-41a1-8f1b-86879e28b697 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:22.679699 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2196/4368] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:16:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:16:22.943649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aee0b3-c958-4e1a-9415-5395b25ff66b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:22.946097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-88aee0b3-c958-4e1a-9415-5395b25ff66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:22.979360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aee0b3-c958-4e1a-9415-5395b25ff66b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:22.993223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:22.994675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:22.995162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:22.995602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e admin admin] PATH_INFO: `/v3/projects/5201ad6f7ad54d028fd0cb152cecdc41` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:22.997083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5201ad6f7ad54d028fd0cb152cecdc41)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:23.004107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:23.181615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:23.181615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5201ad6f7ad54d028fd0cb152cecdc41', 'request_id': 'req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:23.187398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5201ad6f7ad54d028fd0cb152cecdc41', 'request_id': 'req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:23.256914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5201ad6f7ad54d028fd0cb152cecdc41', 'request_id': 'req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:23.274381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e admin admin] The token cache is being invalidate because project 5201ad6f7ad54d028fd0cb152cecdc41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:16:23.274910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:23.275915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:23.277506 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2173/4369] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:16:22 2022] DELETE /identity/v3/projects/5201ad6f7ad54d028fd0cb152cecdc41 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:23.281875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc3721-e36a-41a1-8f1b-86879e28b697 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:23.297453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cc3721-e36a-41a1-8f1b-86879e28b697 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:23.309282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:23.379057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e74f696-57a7-42b3-8617-9721f8414dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:23.380068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74f696-57a7-42b3-8617-9721f8414dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:23.380489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74f696-57a7-42b3-8617-9721f8414dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:23.380892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e74f696-57a7-42b3-8617-9721f8414dc3 admin admin] PATH_INFO: `/v3/users/2414aa781a16426b960520ae5c44c12e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:23.382440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74f696-57a7-42b3-8617-9721f8414dc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2414aa781a16426b960520ae5c44c12e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:23.384562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e74f696-57a7-42b3-8617-9721f8414dc3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:23.616546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8e74f696-57a7-42b3-8617-9721f8414dc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2414aa781a16426b960520ae5c44c12e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:23.624841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8e74f696-57a7-42b3-8617-9721f8414dc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2414aa781a16426b960520ae5c44c12e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:23.626720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8e74f696-57a7-42b3-8617-9721f8414dc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2414aa781a16426b960520ae5c44c12e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:23.627526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8e74f696-57a7-42b3-8617-9721f8414dc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2414aa781a16426b960520ae5c44c12e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:23.721505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8e74f696-57a7-42b3-8617-9721f8414dc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2414aa781a16426b960520ae5c44c12e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:23.756722 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2197/4370] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:16:23 2022] DELETE /identity/v3/users/2414aa781a16426b960520ae5c44c12e => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:24.181428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:24.183998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:24.214643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d3765-c5d8-4cb7-adf0-78a2f1c7681e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:24.303975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e4580-88b5-428d-a93c-68fd8119ec27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:24.305311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e4580-88b5-428d-a93c-68fd8119ec27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:24.305596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e4580-88b5-428d-a93c-68fd8119ec27 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:24.305897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e4580-88b5-428d-a93c-68fd8119ec27 admin admin] PATH_INFO: `/v3/projects/9edd0e9b412c47a1b85e47c89f0056c8` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:24.307169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e4580-88b5-428d-a93c-68fd8119ec27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9edd0e9b412c47a1b85e47c89f0056c8)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:24.309310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e4580-88b5-428d-a93c-68fd8119ec27 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:24.422509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e1e4580-88b5-428d-a93c-68fd8119ec27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9edd0e9b412c47a1b85e47c89f0056c8', 'request_id': 'req-7e1e4580-88b5-428d-a93c-68fd8119ec27'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:24.431614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e1e4580-88b5-428d-a93c-68fd8119ec27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9edd0e9b412c47a1b85e47c89f0056c8', 'request_id': 'req-7e1e4580-88b5-428d-a93c-68fd8119ec27'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:24.486303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e1e4580-88b5-428d-a93c-68fd8119ec27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9edd0e9b412c47a1b85e47c89f0056c8', 'request_id': 'req-7e1e4580-88b5-428d-a93c-68fd8119ec27'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:24.501490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e1e4580-88b5-428d-a93c-68fd8119ec27 admin admin] The token cache is being invalidate because project 9edd0e9b412c47a1b85e47c89f0056c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:16:24.502107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e1e4580-88b5-428d-a93c-68fd8119ec27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:24.502939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e1e4580-88b5-428d-a93c-68fd8119ec27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:24.505155 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2174/4371] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:16:24 2022] DELETE /identity/v3/projects/9edd0e9b412c47a1b85e47c89f0056c8 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:24.510165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016be4cb-887e-4c60-99cb-e8fdce350340 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:24.510677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016be4cb-887e-4c60-99cb-e8fdce350340 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:24.510802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016be4cb-887e-4c60-99cb-e8fdce350340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:24.539029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-016be4cb-887e-4c60-99cb-e8fdce350340 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:24.540617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-016be4cb-887e-4c60-99cb-e8fdce350340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:24.644122 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2198/4372] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:24.652164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181415c-25b5-453d-8a8a-685cfe16a731 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:24.652472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181415c-25b5-453d-8a8a-685cfe16a731 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:24.652753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c181415c-25b5-453d-8a8a-685cfe16a731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:24.701965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c181415c-25b5-453d-8a8a-685cfe16a731 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:24.703818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c181415c-25b5-453d-8a8a-685cfe16a731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:24.714522 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2175/4373] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:24.718937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016be4cb-887e-4c60-99cb-e8fdce350340 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:24.731256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016be4cb-887e-4c60-99cb-e8fdce350340 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:24.739114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1991233-a2bd-4365-ac19-90df7ec76ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:24.739800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1991233-a2bd-4365-ac19-90df7ec76ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:24.740640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1991233-a2bd-4365-ac19-90df7ec76ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:24.741165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1991233-a2bd-4365-ac19-90df7ec76ef2 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:24.741819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1991233-a2bd-4365-ac19-90df7ec76ef2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:16:24.746281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1991233-a2bd-4365-ac19-90df7ec76ef2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:24.746281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1991233-a2bd-4365-ac19-90df7ec76ef2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:24.758003 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2199/4374] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:16:24.761329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181415c-25b5-453d-8a8a-685cfe16a731 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:24.774018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c181415c-25b5-453d-8a8a-685cfe16a731 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:24.779938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922870a-fa52-4645-a8a3-52853558f2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:24.781890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922870a-fa52-4645-a8a3-52853558f2a5 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:24.782292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922870a-fa52-4645-a8a3-52853558f2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:24.782677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922870a-fa52-4645-a8a3-52853558f2a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:24.783633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922870a-fa52-4645-a8a3-52853558f2a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:24.785962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922870a-fa52-4645-a8a3-52853558f2a5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:24.895889 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2176/4375] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:16:24 2022] POST /identity/v3/projects => generated 395 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:16:24.897695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1991233-a2bd-4365-ac19-90df7ec76ef2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:24.907909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1991233-a2bd-4365-ac19-90df7ec76ef2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:24.969236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79838fbe-923e-461b-936e-b017a14e4403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:24.969965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79838fbe-923e-461b-936e-b017a14e4403 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:24.970252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79838fbe-923e-461b-936e-b017a14e4403 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:24.970530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79838fbe-923e-461b-936e-b017a14e4403 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:24.971236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79838fbe-923e-461b-936e-b017a14e4403 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:24.973170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79838fbe-923e-461b-936e-b017a14e4403 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:25.053108 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2200/4376] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:16:24 2022] POST /identity/v3/users => generated 298 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:16:25.057343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922870a-fa52-4645-a8a3-52853558f2a5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:25.076608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922870a-fa52-4645-a8a3-52853558f2a5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:25.081830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70baa27-b0e9-446e-b0f1-382e631f5a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:25.082605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70baa27-b0e9-446e-b0f1-382e631f5a61 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:25.082893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70baa27-b0e9-446e-b0f1-382e631f5a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:25.083167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70baa27-b0e9-446e-b0f1-382e631f5a61 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:25.084466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70baa27-b0e9-446e-b0f1-382e631f5a61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:25.086136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70baa27-b0e9-446e-b0f1-382e631f5a61 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:25.097241 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2177/4377] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:16:25.101865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79838fbe-923e-461b-936e-b017a14e4403 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:25.112612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79838fbe-923e-461b-936e-b017a14e4403 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:25.120294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8146a-4aa7-4cb1-8ed1-608ab53dda22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:25.120294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8146a-4aa7-4cb1-8ed1-608ab53dda22 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:25.120294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8146a-4aa7-4cb1-8ed1-608ab53dda22 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:25.120294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8146a-4aa7-4cb1-8ed1-608ab53dda22 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:25.120675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8146a-4aa7-4cb1-8ed1-608ab53dda22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:25.121111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8146a-4aa7-4cb1-8ed1-608ab53dda22 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:25.132967 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2201/4378] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:16:25.137233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70baa27-b0e9-446e-b0f1-382e631f5a61 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:25.148056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70baa27-b0e9-446e-b0f1-382e631f5a61 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:25.153148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4d6af-155a-4178-8bf3-907816e5e8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:25.154054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4d6af-155a-4178-8bf3-907816e5e8f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:25.154340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4d6af-155a-4178-8bf3-907816e5e8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:25.154618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f4d6af-155a-4178-8bf3-907816e5e8f4 admin admin] PATH_INFO: `/v3/projects/5c394e3c7ecb4975ab4cc95dc2e50eee/users/a19e8510b2664d4bb4d1ea92ac5e3fe0/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:25.174979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4d6af-155a-4178-8bf3-907816e5e8f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c394e3c7ecb4975ab4cc95dc2e50eee, user_id=a19e8510b2664d4bb4d1ea92ac5e3fe0, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:25.177924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f4d6af-155a-4178-8bf3-907816e5e8f4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:25.263240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:25.286481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:25.292875 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2178/4379] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:16:25 2022] PUT /identity/v3/projects/5c394e3c7ecb4975ab4cc95dc2e50eee/users/a19e8510b2664d4bb4d1ea92ac5e3fe0/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:25.304193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14ac1e-7b87-4026-8b20-d3f48da9e88d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:25.304503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14ac1e-7b87-4026-8b20-d3f48da9e88d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:25.304784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d14ac1e-7b87-4026-8b20-d3f48da9e88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:25.331184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1d14ac1e-7b87-4026-8b20-d3f48da9e88d None None] MFA Rules not processed for user `a19e8510b2664d4bb4d1ea92ac5e3fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:25.332425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1d14ac1e-7b87-4026-8b20-d3f48da9e88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:25.399301 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2202/4380] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:25.409985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4d6af-155a-4178-8bf3-907816e5e8f4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:25.412460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-41f4d6af-155a-4178-8bf3-907816e5e8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:25.475822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f4d6af-155a-4178-8bf3-907816e5e8f4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:25.480560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e76c74-b646-4592-82fc-01aecee2552d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:25.481278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e76c74-b646-4592-82fc-01aecee2552d service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:25.481569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e76c74-b646-4592-82fc-01aecee2552d service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:25.482262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e76c74-b646-4592-82fc-01aecee2552d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:25.491572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e76c74-b646-4592-82fc-01aecee2552d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:25.493242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e76c74-b646-4592-82fc-01aecee2552d service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:25.502656 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2179/4381] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:16:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:16:30.828378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18a89a-a3ad-4964-b098-cfea4cd83004 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:30.828884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18a89a-a3ad-4964-b098-cfea4cd83004 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:30.828981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18a89a-a3ad-4964-b098-cfea4cd83004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:30.875738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9b18a89a-a3ad-4964-b098-cfea4cd83004 None None] MFA Rules not processed for user `572170ccd67e4facbf10dad7cd1491c1`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:30.879480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9b18a89a-a3ad-4964-b098-cfea4cd83004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:30.975084 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2203/4382] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:30 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:32.391800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecaca8-51c2-45dd-b993-1e4cb3570d2e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:32.392329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecaca8-51c2-45dd-b993-1e4cb3570d2e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:32.392543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecaca8-51c2-45dd-b993-1e4cb3570d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:32.414459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f0ecaca8-51c2-45dd-b993-1e4cb3570d2e None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:32.415812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f0ecaca8-51c2-45dd-b993-1e4cb3570d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:32.481911 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2180/4383] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:16:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:32.501518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18a89a-a3ad-4964-b098-cfea4cd83004 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:32.503004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9b18a89a-a3ad-4964-b098-cfea4cd83004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:32.547051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:32.604256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18a89a-a3ad-4964-b098-cfea4cd83004 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:32.609754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127fbb0-3db3-415f-82b2-4f9d0c687e27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:32.610551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127fbb0-3db3-415f-82b2-4f9d0c687e27 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:32.610892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127fbb0-3db3-415f-82b2-4f9d0c687e27 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:32.611241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127fbb0-3db3-415f-82b2-4f9d0c687e27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:32.622605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127fbb0-3db3-415f-82b2-4f9d0c687e27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:32.624524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127fbb0-3db3-415f-82b2-4f9d0c687e27 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:32.634991 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2204/4384] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:16:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:16:33.200069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659b7a3-3f00-43f6-8961-44245a0a610d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:33.200435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659b7a3-3f00-43f6-8961-44245a0a610d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:33.200804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e659b7a3-3f00-43f6-8961-44245a0a610d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:33.241155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e659b7a3-3f00-43f6-8961-44245a0a610d None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:33.244090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e659b7a3-3f00-43f6-8961-44245a0a610d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:33.254013 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2181/4385] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:16:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:33.267861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127fbb0-3db3-415f-82b2-4f9d0c687e27 service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:33.298891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127fbb0-3db3-415f-82b2-4f9d0c687e27 service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:33.306057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a3831-b618-41f9-afb8-cc45870089d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:33.306953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a3831-b618-41f9-afb8-cc45870089d2 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:33.307424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a3831-b618-41f9-afb8-cc45870089d2 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:33.307947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a3831-b618-41f9-afb8-cc45870089d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:33.321471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a3831-b618-41f9-afb8-cc45870089d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:33.323413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a3831-b618-41f9-afb8-cc45870089d2 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:33.343124 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2205/4386] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:16:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:16:33.843709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce976d1-0896-4ecb-bcfc-1e2cb09598c7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:33.844662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce976d1-0896-4ecb-bcfc-1e2cb09598c7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:33.844946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce976d1-0896-4ecb-bcfc-1e2cb09598c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:33.872804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7ce976d1-0896-4ecb-bcfc-1e2cb09598c7 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:33.874145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7ce976d1-0896-4ecb-bcfc-1e2cb09598c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:33.892701 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2182/4387] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:16:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:33.901257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a3831-b618-41f9-afb8-cc45870089d2 service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:33.936362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a3831-b618-41f9-afb8-cc45870089d2 service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:33.945579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a05f72-6ec3-444e-bca8-1cf9c321123d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:33.948112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a05f72-6ec3-444e-bca8-1cf9c321123d service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:33.948112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a05f72-6ec3-444e-bca8-1cf9c321123d service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:33.948768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a05f72-6ec3-444e-bca8-1cf9c321123d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:33.963008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a05f72-6ec3-444e-bca8-1cf9c321123d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:33.964887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a05f72-6ec3-444e-bca8-1cf9c321123d service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:33.975855 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2206/4388] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:16:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:16:34.748432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce976d1-0896-4ecb-bcfc-1e2cb09598c7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:34.750617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7ce976d1-0896-4ecb-bcfc-1e2cb09598c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:34.872431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce976d1-0896-4ecb-bcfc-1e2cb09598c7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:34.886660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b686b3-b330-421a-9169-0eeba7709053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:34.888624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b686b3-b330-421a-9169-0eeba7709053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:34.889211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b686b3-b330-421a-9169-0eeba7709053 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:34.889664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b686b3-b330-421a-9169-0eeba7709053 admin admin] PATH_INFO: `/v3/users/98ab12b74029438a9a1d19e505d1fc6d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:34.892198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b686b3-b330-421a-9169-0eeba7709053 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ab12b74029438a9a1d19e505d1fc6d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:34.895206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b686b3-b330-421a-9169-0eeba7709053 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:35.146031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:35.176566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e2b686b3-b330-421a-9169-0eeba7709053 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ab12b74029438a9a1d19e505d1fc6d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:35.176566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e2b686b3-b330-421a-9169-0eeba7709053 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ab12b74029438a9a1d19e505d1fc6d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:35.194313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e2b686b3-b330-421a-9169-0eeba7709053 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ab12b74029438a9a1d19e505d1fc6d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:35.313536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e2b686b3-b330-421a-9169-0eeba7709053 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ab12b74029438a9a1d19e505d1fc6d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:35.314230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e2b686b3-b330-421a-9169-0eeba7709053 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ab12b74029438a9a1d19e505d1fc6d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:35.353551 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2183/4389] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:16:34 2022] DELETE /identity/v3/users/98ab12b74029438a9a1d19e505d1fc6d => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:35.355505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc8451-029a-4401-8a60-0573928f454e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:35.355984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc8451-029a-4401-8a60-0573928f454e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:35.356092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc8451-029a-4401-8a60-0573928f454e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:35.386051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0ddc8451-029a-4401-8a60-0573928f454e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:35.387791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0ddc8451-029a-4401-8a60-0573928f454e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:35.491209 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2207/4390] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:35.512848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b686b3-b330-421a-9169-0eeba7709053 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:35.516391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e2b686b3-b330-421a-9169-0eeba7709053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:35.620390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b686b3-b330-421a-9169-0eeba7709053 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:35.651569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d2dd5-09ca-4d4d-beed-096bbf97e94c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:35.655519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d2dd5-09ca-4d4d-beed-096bbf97e94c service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:35.655519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d2dd5-09ca-4d4d-beed-096bbf97e94c service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:35.655519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d2dd5-09ca-4d4d-beed-096bbf97e94c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:35.676540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d2dd5-09ca-4d4d-beed-096bbf97e94c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:35.678754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d2dd5-09ca-4d4d-beed-096bbf97e94c service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:35.695603 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2184/4391] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:16:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:16:36.305531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc8451-029a-4401-8a60-0573928f454e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:36.307477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0ddc8451-029a-4401-8a60-0573928f454e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:36.322586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc8451-029a-4401-8a60-0573928f454e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:36.328401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16172c-5d2a-4b40-8632-fbc1c13849cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:36.329366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16172c-5d2a-4b40-8632-fbc1c13849cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:36.329756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16172c-5d2a-4b40-8632-fbc1c13849cc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:36.330305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16172c-5d2a-4b40-8632-fbc1c13849cc admin admin] PATH_INFO: `/v3/projects/cf454e1b4e6a4c45af04865b154c41a9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:36.332052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16172c-5d2a-4b40-8632-fbc1c13849cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf454e1b4e6a4c45af04865b154c41a9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:36.334086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16172c-5d2a-4b40-8632-fbc1c13849cc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:36.478480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7e16172c-5d2a-4b40-8632-fbc1c13849cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf454e1b4e6a4c45af04865b154c41a9', 'request_id': 'req-7e16172c-5d2a-4b40-8632-fbc1c13849cc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:36.486348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7e16172c-5d2a-4b40-8632-fbc1c13849cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf454e1b4e6a4c45af04865b154c41a9', 'request_id': 'req-7e16172c-5d2a-4b40-8632-fbc1c13849cc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:36.600150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7e16172c-5d2a-4b40-8632-fbc1c13849cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf454e1b4e6a4c45af04865b154c41a9', 'request_id': 'req-7e16172c-5d2a-4b40-8632-fbc1c13849cc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:36.619225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7e16172c-5d2a-4b40-8632-fbc1c13849cc admin admin] The token cache is being invalidate because project cf454e1b4e6a4c45af04865b154c41a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:16:36.619532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7e16172c-5d2a-4b40-8632-fbc1c13849cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:36.620255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7e16172c-5d2a-4b40-8632-fbc1c13849cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:36.622058 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2208/4392] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:16:36 2022] DELETE /identity/v3/projects/cf454e1b4e6a4c45af04865b154c41a9 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:36.628078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5d548-74d7-4a51-a365-ca40046cb233 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:36.634060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5d548-74d7-4a51-a365-ca40046cb233 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:36.634060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5d548-74d7-4a51-a365-ca40046cb233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:36.646818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3ed5d548-74d7-4a51-a365-ca40046cb233 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:36.648313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3ed5d548-74d7-4a51-a365-ca40046cb233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:36.736952 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2185/4393] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:36.745203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6043091-dafe-4116-acc0-5efbdf1d9228 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:36.745500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6043091-dafe-4116-acc0-5efbdf1d9228 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:36.745795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6043091-dafe-4116-acc0-5efbdf1d9228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:36.765498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b6043091-dafe-4116-acc0-5efbdf1d9228 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:36.766997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b6043091-dafe-4116-acc0-5efbdf1d9228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:36.776169 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2209/4394] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:36.781340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5d548-74d7-4a51-a365-ca40046cb233 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:36.793360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5d548-74d7-4a51-a365-ca40046cb233 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:36.798847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df0e01-731e-4b63-b831-33d8c657ce79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:36.799535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df0e01-731e-4b63-b831-33d8c657ce79 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:36.802148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df0e01-731e-4b63-b831-33d8c657ce79 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:36.802148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df0e01-731e-4b63-b831-33d8c657ce79 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:36.802148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df0e01-731e-4b63-b831-33d8c657ce79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:16:36.802148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df0e01-731e-4b63-b831-33d8c657ce79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:36.804196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df0e01-731e-4b63-b831-33d8c657ce79 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:36.817945 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2186/4395] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:16:36.822704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6043091-dafe-4116-acc0-5efbdf1d9228 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:36.834001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6043091-dafe-4116-acc0-5efbdf1d9228 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:36.839341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8152c7-08d2-49b6-9525-3f2450632501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:36.840831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8152c7-08d2-49b6-9525-3f2450632501 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:36.841318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8152c7-08d2-49b6-9525-3f2450632501 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:36.841602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8152c7-08d2-49b6-9525-3f2450632501 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:36.842420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8152c7-08d2-49b6-9525-3f2450632501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:36.844877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8152c7-08d2-49b6-9525-3f2450632501 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:36.855564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:36.974807 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2210/4396] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:16:36 2022] POST /identity/v3/projects => generated 373 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:16:36.979325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df0e01-731e-4b63-b831-33d8c657ce79 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:36.992950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df0e01-731e-4b63-b831-33d8c657ce79 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:37.043222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:37.070381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceb822-028f-45a2-886d-31db688c0343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:37.071080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceb822-028f-45a2-886d-31db688c0343 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:37.071362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceb822-028f-45a2-886d-31db688c0343 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:37.071823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceb822-028f-45a2-886d-31db688c0343 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:37.073308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aceb822-028f-45a2-886d-31db688c0343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:37.076127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aceb822-028f-45a2-886d-31db688c0343 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:37.200610 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2187/4397] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:16:36 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:16:37.205384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8152c7-08d2-49b6-9525-3f2450632501 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:37.218716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8152c7-08d2-49b6-9525-3f2450632501 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:37.224382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fdde4-27b9-40d2-bead-76f7b70f445a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:37.225884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fdde4-27b9-40d2-bead-76f7b70f445a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:37.225884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fdde4-27b9-40d2-bead-76f7b70f445a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:37.225884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fdde4-27b9-40d2-bead-76f7b70f445a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:37.227231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fdde4-27b9-40d2-bead-76f7b70f445a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:37.229203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fdde4-27b9-40d2-bead-76f7b70f445a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:37.251198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceb822-028f-45a2-886d-31db688c0343 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:37.258466 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2211/4398] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:16:37.263343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceb822-028f-45a2-886d-31db688c0343 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:37.268624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fce1a-e3ae-4b6d-85e8-d44a4aacc7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:37.269467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fce1a-e3ae-4b6d-85e8-d44a4aacc7ce admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:37.269820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fce1a-e3ae-4b6d-85e8-d44a4aacc7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:37.270342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fce1a-e3ae-4b6d-85e8-d44a4aacc7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:37.271766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25fce1a-e3ae-4b6d-85e8-d44a4aacc7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:37.273699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25fce1a-e3ae-4b6d-85e8-d44a4aacc7ce admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:37.289162 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2188/4399] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:16:37.293786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fdde4-27b9-40d2-bead-76f7b70f445a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:37.305598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fdde4-27b9-40d2-bead-76f7b70f445a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:37.311644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12f772-94af-4d84-9fe5-ea396163b6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:37.313193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12f772-94af-4d84-9fe5-ea396163b6bd admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:37.313672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12f772-94af-4d84-9fe5-ea396163b6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:37.314418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12f772-94af-4d84-9fe5-ea396163b6bd admin admin] PATH_INFO: `/v3/projects/f97101c788064513a47e7a3ad8611ce5/users/45c035c80888410ba53f9834fda06f2c/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:37.342325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12f772-94af-4d84-9fe5-ea396163b6bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f97101c788064513a47e7a3ad8611ce5, user_id=45c035c80888410ba53f9834fda06f2c, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:37.346231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12f772-94af-4d84-9fe5-ea396163b6bd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:37.499609 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2212/4400] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:16:37 2022] PUT /identity/v3/projects/f97101c788064513a47e7a3ad8611ce5/users/45c035c80888410ba53f9834fda06f2c/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:37.528344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7ef00-f585-4873-8edf-c257d672860d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:37.529479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7ef00-f585-4873-8edf-c257d672860d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:37.529875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7ef00-f585-4873-8edf-c257d672860d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:37.601064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c5a7ef00-f585-4873-8edf-c257d672860d None None] MFA Rules not processed for user `45c035c80888410ba53f9834fda06f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:37.602903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c5a7ef00-f585-4873-8edf-c257d672860d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:37.708279 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2189/4401] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:37.716118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12f772-94af-4d84-9fe5-ea396163b6bd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:37.718964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cb12f772-94af-4d84-9fe5-ea396163b6bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:37.848771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12f772-94af-4d84-9fe5-ea396163b6bd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:37.863640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c363db0-3642-4855-952c-c01f290a36e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:37.867322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c363db0-3642-4855-952c-c01f290a36e4 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:37.867322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c363db0-3642-4855-952c-c01f290a36e4 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:37.867322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c363db0-3642-4855-952c-c01f290a36e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:37.884666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c363db0-3642-4855-952c-c01f290a36e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:37.887769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c363db0-3642-4855-952c-c01f290a36e4 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:37.908868 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2213/4402] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:16:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:16:41.431127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7ef00-f585-4873-8edf-c257d672860d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:41.518601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7ef00-f585-4873-8edf-c257d672860d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:41.524498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ba405-92dd-4c9e-b74e-ae0bc13e08e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:41.525690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ba405-92dd-4c9e-b74e-ae0bc13e08e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:41.526068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ba405-92dd-4c9e-b74e-ae0bc13e08e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:41.526357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ba405-92dd-4c9e-b74e-ae0bc13e08e7 admin admin] PATH_INFO: `/v3/users/45c035c80888410ba53f9834fda06f2c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:41.527576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ba405-92dd-4c9e-b74e-ae0bc13e08e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45c035c80888410ba53f9834fda06f2c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:41.529684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ba405-92dd-4c9e-b74e-ae0bc13e08e7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:41.740547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-959ba405-92dd-4c9e-b74e-ae0bc13e08e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45c035c80888410ba53f9834fda06f2c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:41.741394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-959ba405-92dd-4c9e-b74e-ae0bc13e08e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45c035c80888410ba53f9834fda06f2c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:41.747089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:41.752557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-959ba405-92dd-4c9e-b74e-ae0bc13e08e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45c035c80888410ba53f9834fda06f2c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:41.811119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-959ba405-92dd-4c9e-b74e-ae0bc13e08e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45c035c80888410ba53f9834fda06f2c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:41.811904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-959ba405-92dd-4c9e-b74e-ae0bc13e08e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45c035c80888410ba53f9834fda06f2c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:41.837711 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2190/4403] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:16:41 2022] DELETE /identity/v3/users/45c035c80888410ba53f9834fda06f2c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:41.842337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbeb2d-7583-4d3e-9977-b6319cbbe266 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:41.843246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbeb2d-7583-4d3e-9977-b6319cbbe266 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:41.843900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fbeb2d-7583-4d3e-9977-b6319cbbe266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:41.866119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f1fbeb2d-7583-4d3e-9977-b6319cbbe266 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:41.867466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f1fbeb2d-7583-4d3e-9977-b6319cbbe266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:41.936073 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2214/4404] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:41.946605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ba405-92dd-4c9e-b74e-ae0bc13e08e7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:41.954364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-959ba405-92dd-4c9e-b74e-ae0bc13e08e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:42.034838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ba405-92dd-4c9e-b74e-ae0bc13e08e7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:42.038417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b78bd-d135-415a-9504-930412fef38f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:42.039105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b78bd-d135-415a-9504-930412fef38f service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:42.039393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b78bd-d135-415a-9504-930412fef38f service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:42.039722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b78bd-d135-415a-9504-930412fef38f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:42.050531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b78bd-d135-415a-9504-930412fef38f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:42.062919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b78bd-d135-415a-9504-930412fef38f service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:42.070537 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2191/4405] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:16:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:16:42.629709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbeb2d-7583-4d3e-9977-b6319cbbe266 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:42.631571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f1fbeb2d-7583-4d3e-9977-b6319cbbe266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:42.645697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fbeb2d-7583-4d3e-9977-b6319cbbe266 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:42.650759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec57983-3e84-49f9-b097-b4cc84b27a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:42.651844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec57983-3e84-49f9-b097-b4cc84b27a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:42.652129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec57983-3e84-49f9-b097-b4cc84b27a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:42.652410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec57983-3e84-49f9-b097-b4cc84b27a98 admin admin] PATH_INFO: `/v3/projects/f97101c788064513a47e7a3ad8611ce5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:42.654335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec57983-3e84-49f9-b097-b4cc84b27a98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f97101c788064513a47e7a3ad8611ce5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:42.656731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec57983-3e84-49f9-b097-b4cc84b27a98 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:42.774038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:42.790375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4ec57983-3e84-49f9-b097-b4cc84b27a98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f97101c788064513a47e7a3ad8611ce5', 'request_id': 'req-4ec57983-3e84-49f9-b097-b4cc84b27a98'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:42.796710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4ec57983-3e84-49f9-b097-b4cc84b27a98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f97101c788064513a47e7a3ad8611ce5', 'request_id': 'req-4ec57983-3e84-49f9-b097-b4cc84b27a98'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:42.867786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4ec57983-3e84-49f9-b097-b4cc84b27a98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f97101c788064513a47e7a3ad8611ce5', 'request_id': 'req-4ec57983-3e84-49f9-b097-b4cc84b27a98'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:42.880702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4ec57983-3e84-49f9-b097-b4cc84b27a98 admin admin] The token cache is being invalidate because project f97101c788064513a47e7a3ad8611ce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:16:42.881304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4ec57983-3e84-49f9-b097-b4cc84b27a98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:42.882119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4ec57983-3e84-49f9-b097-b4cc84b27a98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:42.883874 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2215/4406] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:16:42 2022] DELETE /identity/v3/projects/f97101c788064513a47e7a3ad8611ce5 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:42.888793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d36603-af38-4c12-a0f9-112892897c6d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:42.889298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d36603-af38-4c12-a0f9-112892897c6d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:42.889471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d36603-af38-4c12-a0f9-112892897c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:42.910443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-72d36603-af38-4c12-a0f9-112892897c6d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:42.912367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-72d36603-af38-4c12-a0f9-112892897c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:43.004069 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2192/4407] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:16:43.014148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3819f9a-2677-43ef-b85a-3fbbf7bba204 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:43.015621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3819f9a-2677-43ef-b85a-3fbbf7bba204 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:43.015621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3819f9a-2677-43ef-b85a-3fbbf7bba204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:43.050607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c3819f9a-2677-43ef-b85a-3fbbf7bba204 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:43.052558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c3819f9a-2677-43ef-b85a-3fbbf7bba204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:43.062701 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2216/4408] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:43.067360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d36603-af38-4c12-a0f9-112892897c6d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:43.079059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d36603-af38-4c12-a0f9-112892897c6d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:43.088694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788a429-81e9-4fdd-ba7a-0783265afc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:43.089826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788a429-81e9-4fdd-ba7a-0783265afc6d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:43.090382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788a429-81e9-4fdd-ba7a-0783265afc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:43.090870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788a429-81e9-4fdd-ba7a-0783265afc6d admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:43.091846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788a429-81e9-4fdd-ba7a-0783265afc6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:16:43.092335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788a429-81e9-4fdd-ba7a-0783265afc6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:43.094064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788a429-81e9-4fdd-ba7a-0783265afc6d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:43.112682 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2193/4409] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:16:43.114074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3819f9a-2677-43ef-b85a-3fbbf7bba204 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:43.125888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3819f9a-2677-43ef-b85a-3fbbf7bba204 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:43.131125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76756a-284c-4d98-aebb-2d56db387b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:43.132010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76756a-284c-4d98-aebb-2d56db387b5d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:43.132432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76756a-284c-4d98-aebb-2d56db387b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:43.132814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b76756a-284c-4d98-aebb-2d56db387b5d admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:43.133693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76756a-284c-4d98-aebb-2d56db387b5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:43.135803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b76756a-284c-4d98-aebb-2d56db387b5d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:43.272770 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2217/4410] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:16:43 2022] POST /identity/v3/projects => generated 389 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:16:43.275868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788a429-81e9-4fdd-ba7a-0783265afc6d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:43.287190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788a429-81e9-4fdd-ba7a-0783265afc6d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:43.361110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45ac28-630a-41f9-876d-37668af982ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:43.361797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45ac28-630a-41f9-876d-37668af982ca admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:43.362118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45ac28-630a-41f9-876d-37668af982ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:43.362392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c45ac28-630a-41f9-876d-37668af982ca admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:43.363086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45ac28-630a-41f9-876d-37668af982ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:43.365142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c45ac28-630a-41f9-876d-37668af982ca admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:43.491803 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2194/4411] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:16:43 2022] POST /identity/v3/users => generated 295 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:16:43.495871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76756a-284c-4d98-aebb-2d56db387b5d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:43.514335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b76756a-284c-4d98-aebb-2d56db387b5d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:43.520506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfdf8e-aa9c-404b-a402-90da12870724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:43.521766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfdf8e-aa9c-404b-a402-90da12870724 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:43.523429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfdf8e-aa9c-404b-a402-90da12870724 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:43.523429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acfdf8e-aa9c-404b-a402-90da12870724 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:43.524097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfdf8e-aa9c-404b-a402-90da12870724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:43.525988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acfdf8e-aa9c-404b-a402-90da12870724 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:43.539546 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2218/4412] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:16:43.543749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45ac28-630a-41f9-876d-37668af982ca admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:43.557610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c45ac28-630a-41f9-876d-37668af982ca admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:43.562950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab81128-0a83-42fa-b34f-e5ceec786d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:43.563629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab81128-0a83-42fa-b34f-e5ceec786d0b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:43.563916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab81128-0a83-42fa-b34f-e5ceec786d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:43.564249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab81128-0a83-42fa-b34f-e5ceec786d0b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:43.565149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab81128-0a83-42fa-b34f-e5ceec786d0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:43.567201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab81128-0a83-42fa-b34f-e5ceec786d0b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:43.588354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfdf8e-aa9c-404b-a402-90da12870724 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:43.588557 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2195/4413] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:16:43.599341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acfdf8e-aa9c-404b-a402-90da12870724 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:43.604588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5219bd-451b-496a-ab2d-1298204e8dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:43.605487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5219bd-451b-496a-ab2d-1298204e8dde admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:43.605780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5219bd-451b-496a-ab2d-1298204e8dde admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:43.606099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5219bd-451b-496a-ab2d-1298204e8dde admin admin] PATH_INFO: `/v3/projects/5e2aed4ec2e74e69bb5eea2e65ab92bb/users/665ce91603f24affb78017790130f733/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:43.623437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5219bd-451b-496a-ab2d-1298204e8dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e2aed4ec2e74e69bb5eea2e65ab92bb, user_id=665ce91603f24affb78017790130f733, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:43.626338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5219bd-451b-496a-ab2d-1298204e8dde admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:43.725459 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2219/4414] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:16:43 2022] PUT /identity/v3/projects/5e2aed4ec2e74e69bb5eea2e65ab92bb/users/665ce91603f24affb78017790130f733/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:16:43.730578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dea6a2-f2c4-46fa-b01a-d238a8f84e92 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:43.731249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dea6a2-f2c4-46fa-b01a-d238a8f84e92 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:43.731690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dea6a2-f2c4-46fa-b01a-d238a8f84e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:43.756245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-30dea6a2-f2c4-46fa-b01a-d238a8f84e92 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:43.757688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-30dea6a2-f2c4-46fa-b01a-d238a8f84e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:43.792785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:43.845930 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2196/4415] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:43.855630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5219bd-451b-496a-ab2d-1298204e8dde admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:43.857400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3d5219bd-451b-496a-ab2d-1298204e8dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:43.965924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5219bd-451b-496a-ab2d-1298204e8dde admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:43.971296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b304b-7d9c-49a1-8c89-e001f1382b21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:43.972346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b304b-7d9c-49a1-8c89-e001f1382b21 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:43.972953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b304b-7d9c-49a1-8c89-e001f1382b21 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:43.973246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b304b-7d9c-49a1-8c89-e001f1382b21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:43.987672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b304b-7d9c-49a1-8c89-e001f1382b21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:43.989166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b304b-7d9c-49a1-8c89-e001f1382b21 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:44.001371 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2220/4416] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:16:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:16:45.844501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6baa53-3f9a-44b5-b223-6b0f2168e55c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:45.845643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6baa53-3f9a-44b5-b223-6b0f2168e55c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:45.846747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6baa53-3f9a-44b5-b223-6b0f2168e55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:45.888820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1c6baa53-3f9a-44b5-b223-6b0f2168e55c None None] MFA Rules not processed for user `665ce91603f24affb78017790130f733`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:45.890724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1c6baa53-3f9a-44b5-b223-6b0f2168e55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:46.019745 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2197/4417] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:46.039530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b304b-7d9c-49a1-8c89-e001f1382b21 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:46.043670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c94b304b-7d9c-49a1-8c89-e001f1382b21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:46.128105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:46.147433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b304b-7d9c-49a1-8c89-e001f1382b21 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:46.157812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7c5c7-9df4-4bb9-88ee-eec038316537 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:46.157812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7c5c7-9df4-4bb9-88ee-eec038316537 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:46.157812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7c5c7-9df4-4bb9-88ee-eec038316537 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:46.157812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7c5c7-9df4-4bb9-88ee-eec038316537 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:46.185989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7c5c7-9df4-4bb9-88ee-eec038316537 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:46.188296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7c5c7-9df4-4bb9-88ee-eec038316537 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:46.200544 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2221/4418] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:16:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:16:47.227729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6baa53-3f9a-44b5-b223-6b0f2168e55c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:47.227729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1c6baa53-3f9a-44b5-b223-6b0f2168e55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:47.239468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6baa53-3f9a-44b5-b223-6b0f2168e55c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:47.245686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba7526-8829-4f8b-a751-a892b6444589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:47.246864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba7526-8829-4f8b-a751-a892b6444589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:47.247152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba7526-8829-4f8b-a751-a892b6444589 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:47.247430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ba7526-8829-4f8b-a751-a892b6444589 admin admin] PATH_INFO: `/v3/users/a19e8510b2664d4bb4d1ea92ac5e3fe0` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:47.251035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba7526-8829-4f8b-a751-a892b6444589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a19e8510b2664d4bb4d1ea92ac5e3fe0)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:47.260879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ba7526-8829-4f8b-a751-a892b6444589 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:47.447406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-64ba7526-8829-4f8b-a751-a892b6444589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a19e8510b2664d4bb4d1ea92ac5e3fe0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:47.448753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-64ba7526-8829-4f8b-a751-a892b6444589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a19e8510b2664d4bb4d1ea92ac5e3fe0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:47.460446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-64ba7526-8829-4f8b-a751-a892b6444589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a19e8510b2664d4bb4d1ea92ac5e3fe0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:47.554062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-64ba7526-8829-4f8b-a751-a892b6444589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a19e8510b2664d4bb4d1ea92ac5e3fe0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:47.554953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-64ba7526-8829-4f8b-a751-a892b6444589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a19e8510b2664d4bb4d1ea92ac5e3fe0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:47.556645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:47.576631 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2198/4419] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:16:47 2022] DELETE /identity/v3/users/a19e8510b2664d4bb4d1ea92ac5e3fe0 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:47.581217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecdde2-72b6-4d3f-a647-b7b2b180bf02 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:47.582520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecdde2-72b6-4d3f-a647-b7b2b180bf02 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:47.583201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecdde2-72b6-4d3f-a647-b7b2b180bf02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:47.603538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0fecdde2-72b6-4d3f-a647-b7b2b180bf02 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:47.604917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0fecdde2-72b6-4d3f-a647-b7b2b180bf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:47.690331 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2222/4420] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:47.700230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba7526-8829-4f8b-a751-a892b6444589 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:47.701789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-64ba7526-8829-4f8b-a751-a892b6444589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:47.779292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ba7526-8829-4f8b-a751-a892b6444589 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:47.792047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a2f6c-6f35-4adf-9acd-293e056419b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:47.792047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a2f6c-6f35-4adf-9acd-293e056419b0 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:47.792047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a2f6c-6f35-4adf-9acd-293e056419b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:47.792047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a2f6c-6f35-4adf-9acd-293e056419b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:47.802970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a2f6c-6f35-4adf-9acd-293e056419b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:47.804451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5a2f6c-6f35-4adf-9acd-293e056419b0 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:47.816790 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2199/4421] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:16:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:16:48.132263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fecdde2-72b6-4d3f-a647-b7b2b180bf02 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:48.133813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0fecdde2-72b6-4d3f-a647-b7b2b180bf02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:48.160169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fecdde2-72b6-4d3f-a647-b7b2b180bf02 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:48.172072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2a0889-dab4-41bf-b5e3-b8d4a2d672cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:48.173228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a0889-dab4-41bf-b5e3-b8d4a2d672cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:48.173658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a0889-dab4-41bf-b5e3-b8d4a2d672cc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:48.174093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2a0889-dab4-41bf-b5e3-b8d4a2d672cc admin admin] PATH_INFO: `/v3/projects/5c394e3c7ecb4975ab4cc95dc2e50eee` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:48.175603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a0889-dab4-41bf-b5e3-b8d4a2d672cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c394e3c7ecb4975ab4cc95dc2e50eee)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:48.177783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2a0889-dab4-41bf-b5e3-b8d4a2d672cc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:48.379526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da2a0889-dab4-41bf-b5e3-b8d4a2d672cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c394e3c7ecb4975ab4cc95dc2e50eee', 'request_id': 'req-da2a0889-dab4-41bf-b5e3-b8d4a2d672cc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:48.385873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da2a0889-dab4-41bf-b5e3-b8d4a2d672cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c394e3c7ecb4975ab4cc95dc2e50eee', 'request_id': 'req-da2a0889-dab4-41bf-b5e3-b8d4a2d672cc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:48.473538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da2a0889-dab4-41bf-b5e3-b8d4a2d672cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c394e3c7ecb4975ab4cc95dc2e50eee', 'request_id': 'req-da2a0889-dab4-41bf-b5e3-b8d4a2d672cc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:48.492847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da2a0889-dab4-41bf-b5e3-b8d4a2d672cc admin admin] The token cache is being invalidate because project 5c394e3c7ecb4975ab4cc95dc2e50eee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:16:48.493452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da2a0889-dab4-41bf-b5e3-b8d4a2d672cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:48.502293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da2a0889-dab4-41bf-b5e3-b8d4a2d672cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:48.509281 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2223/4422] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:16:48 2022] DELETE /identity/v3/projects/5c394e3c7ecb4975ab4cc95dc2e50eee => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:48.511108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f241a1-76e1-4d2d-b689-e62bf2fa0c6f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:48.511590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f241a1-76e1-4d2d-b689-e62bf2fa0c6f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:48.511769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f241a1-76e1-4d2d-b689-e62bf2fa0c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:48.533054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-18f241a1-76e1-4d2d-b689-e62bf2fa0c6f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:48.535150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-18f241a1-76e1-4d2d-b689-e62bf2fa0c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:48.603383 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2200/4423] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:48.611111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d22e59-4764-4f6e-a015-2d7a9b50f153 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:48.611635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d22e59-4764-4f6e-a015-2d7a9b50f153 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:48.612171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d22e59-4764-4f6e-a015-2d7a9b50f153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:48.632020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-89d22e59-4764-4f6e-a015-2d7a9b50f153 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:48.633290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-89d22e59-4764-4f6e-a015-2d7a9b50f153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:48.641305 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2224/4424] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:48.645576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f241a1-76e1-4d2d-b689-e62bf2fa0c6f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:48.655818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f241a1-76e1-4d2d-b689-e62bf2fa0c6f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:48.660988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bcad4-500d-437e-b07c-b6a06e1f344c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:48.664851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bcad4-500d-437e-b07c-b6a06e1f344c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:48.664851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bcad4-500d-437e-b07c-b6a06e1f344c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:48.664851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bcad4-500d-437e-b07c-b6a06e1f344c admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:48.664851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bcad4-500d-437e-b07c-b6a06e1f344c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:16:48.664851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bcad4-500d-437e-b07c-b6a06e1f344c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:48.665312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991bcad4-500d-437e-b07c-b6a06e1f344c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:48.678179 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2201/4425] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:16:48.682611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d22e59-4764-4f6e-a015-2d7a9b50f153 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:48.694768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d22e59-4764-4f6e-a015-2d7a9b50f153 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:48.699639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9564f-6706-41a5-a408-6f4d526bfb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:48.700353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9564f-6706-41a5-a408-6f4d526bfb89 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:48.700644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9564f-6706-41a5-a408-6f4d526bfb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:48.700920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9564f-6706-41a5-a408-6f4d526bfb89 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:48.701822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9564f-6706-41a5-a408-6f4d526bfb89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:48.703789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9564f-6706-41a5-a408-6f4d526bfb89 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:48.865406 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2225/4426] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:16:48 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:16:48.866618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bcad4-500d-437e-b07c-b6a06e1f344c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:48.879004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bcad4-500d-437e-b07c-b6a06e1f344c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:48.961383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151a69c-33cd-4142-9f64-f93034e01429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:48.962429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151a69c-33cd-4142-9f64-f93034e01429 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:48.962885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151a69c-33cd-4142-9f64-f93034e01429 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:48.963319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0151a69c-33cd-4142-9f64-f93034e01429 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:48.965087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0151a69c-33cd-4142-9f64-f93034e01429 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:48.967455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0151a69c-33cd-4142-9f64-f93034e01429 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:49.030926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:49.039897 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2202/4427] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:16:48 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:16:49.050934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9564f-6706-41a5-a408-6f4d526bfb89 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:49.063233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9564f-6706-41a5-a408-6f4d526bfb89 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:49.074797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651b4e0-25e2-4db1-8d6a-9334fec79148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:49.076138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651b4e0-25e2-4db1-8d6a-9334fec79148 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:49.076594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651b4e0-25e2-4db1-8d6a-9334fec79148 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:49.080815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651b4e0-25e2-4db1-8d6a-9334fec79148 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:49.080815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651b4e0-25e2-4db1-8d6a-9334fec79148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:49.080815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651b4e0-25e2-4db1-8d6a-9334fec79148 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:49.092521 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2226/4428] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:16:49.096659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151a69c-33cd-4142-9f64-f93034e01429 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:49.107803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0151a69c-33cd-4142-9f64-f93034e01429 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:49.112529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4312980d-b167-4f49-a1cc-6e0463683951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:49.113190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312980d-b167-4f49-a1cc-6e0463683951 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:49.113474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312980d-b167-4f49-a1cc-6e0463683951 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:49.113754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4312980d-b167-4f49-a1cc-6e0463683951 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:49.115046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312980d-b167-4f49-a1cc-6e0463683951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:49.116677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4312980d-b167-4f49-a1cc-6e0463683951 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:49.127461 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2203/4429] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:16:49.137899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651b4e0-25e2-4db1-8d6a-9334fec79148 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:49.148620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651b4e0-25e2-4db1-8d6a-9334fec79148 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:49.153789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff1fe3-a3e7-4395-9b5f-611fc877b6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:49.154733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff1fe3-a3e7-4395-9b5f-611fc877b6d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:49.155014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff1fe3-a3e7-4395-9b5f-611fc877b6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:49.155289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff1fe3-a3e7-4395-9b5f-611fc877b6d6 admin admin] PATH_INFO: `/v3/projects/c170941c16e3489c9f2db2ff2883e5e4/users/a9c0c9f3a90648148023c4b8038f7d23/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:49.179209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff1fe3-a3e7-4395-9b5f-611fc877b6d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c170941c16e3489c9f2db2ff2883e5e4, user_id=a9c0c9f3a90648148023c4b8038f7d23, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:49.181786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff1fe3-a3e7-4395-9b5f-611fc877b6d6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:49.255594 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2227/4430] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:16:49 2022] PUT /identity/v3/projects/c170941c16e3489c9f2db2ff2883e5e4/users/a9c0c9f3a90648148023c4b8038f7d23/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:49.266333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a48f12-254f-41a0-89f2-8c73df1214be None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:49.266628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a48f12-254f-41a0-89f2-8c73df1214be None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:49.267410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a48f12-254f-41a0-89f2-8c73df1214be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:49.297043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-13a48f12-254f-41a0-89f2-8c73df1214be None None] MFA Rules not processed for user `a9c0c9f3a90648148023c4b8038f7d23`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:49.298319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-13a48f12-254f-41a0-89f2-8c73df1214be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:49.375293 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2204/4431] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:49.386116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff1fe3-a3e7-4395-9b5f-611fc877b6d6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:49.388979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dcff1fe3-a3e7-4395-9b5f-611fc877b6d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:49.403556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:49.461302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff1fe3-a3e7-4395-9b5f-611fc877b6d6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:49.465733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba278a-9671-4119-b566-aaacb543c05a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:49.466625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba278a-9671-4119-b566-aaacb543c05a service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:49.467003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba278a-9671-4119-b566-aaacb543c05a service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:49.467507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba278a-9671-4119-b566-aaacb543c05a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:49.479527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba278a-9671-4119-b566-aaacb543c05a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:49.480983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ba278a-9671-4119-b566-aaacb543c05a service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:49.495775 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2228/4432] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:16:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:16:50.585041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29857cc5-8c8b-47e3-8ee0-be8cad46cec7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:50.585698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29857cc5-8c8b-47e3-8ee0-be8cad46cec7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:50.585698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29857cc5-8c8b-47e3-8ee0-be8cad46cec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:50.605561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-29857cc5-8c8b-47e3-8ee0-be8cad46cec7 None None] MFA Rules not processed for user `572170ccd67e4facbf10dad7cd1491c1`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:50.606846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-29857cc5-8c8b-47e3-8ee0-be8cad46cec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:50.670131 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2205/4433] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:55.604641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba278a-9671-4119-b566-aaacb543c05a service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:55.606790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-05ba278a-9671-4119-b566-aaacb543c05a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:55.690933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba278a-9671-4119-b566-aaacb543c05a service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:55.696166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46bdb8c-0de6-4944-b6e8-3ee8241e24d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:55.697223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bdb8c-0de6-4944-b6e8-3ee8241e24d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:55.697509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bdb8c-0de6-4944-b6e8-3ee8241e24d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:55.698163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bdb8c-0de6-4944-b6e8-3ee8241e24d0 admin admin] PATH_INFO: `/v3/users/269519c04f264104aef296030828caec` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:55.699480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46bdb8c-0de6-4944-b6e8-3ee8241e24d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=269519c04f264104aef296030828caec)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:55.701539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46bdb8c-0de6-4944-b6e8-3ee8241e24d0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:55.907624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b46bdb8c-0de6-4944-b6e8-3ee8241e24d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269519c04f264104aef296030828caec'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:55.917119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b46bdb8c-0de6-4944-b6e8-3ee8241e24d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269519c04f264104aef296030828caec'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:55.921692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b46bdb8c-0de6-4944-b6e8-3ee8241e24d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269519c04f264104aef296030828caec'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:55.921692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b46bdb8c-0de6-4944-b6e8-3ee8241e24d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269519c04f264104aef296030828caec'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:56.004769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b46bdb8c-0de6-4944-b6e8-3ee8241e24d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269519c04f264104aef296030828caec'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:56.024766 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2229/4434] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:16:55 2022] DELETE /identity/v3/users/269519c04f264104aef296030828caec => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:56.029357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb0fde-1310-456f-a5f1-9b59b2f09a0f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:56.029936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb0fde-1310-456f-a5f1-9b59b2f09a0f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:56.030062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeb0fde-1310-456f-a5f1-9b59b2f09a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:56.048525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-cbeb0fde-1310-456f-a5f1-9b59b2f09a0f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:56.049753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cbeb0fde-1310-456f-a5f1-9b59b2f09a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:56.119940 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2206/4435] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:56.128071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46bdb8c-0de6-4944-b6e8-3ee8241e24d0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:56.129781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b46bdb8c-0de6-4944-b6e8-3ee8241e24d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:56.167860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:56.198021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46bdb8c-0de6-4944-b6e8-3ee8241e24d0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:56.201951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c834a4-7f94-4cba-99db-e92c72d7ad0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:56.202615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c834a4-7f94-4cba-99db-e92c72d7ad0b service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:56.202910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c834a4-7f94-4cba-99db-e92c72d7ad0b service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:56.203331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c834a4-7f94-4cba-99db-e92c72d7ad0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:56.214856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c834a4-7f94-4cba-99db-e92c72d7ad0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:56.216225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c834a4-7f94-4cba-99db-e92c72d7ad0b service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:56.229606 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2230/4436] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:16:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:16:56.960567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb0fde-1310-456f-a5f1-9b59b2f09a0f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:56.962138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cbeb0fde-1310-456f-a5f1-9b59b2f09a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:56.978234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeb0fde-1310-456f-a5f1-9b59b2f09a0f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:57.012777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fd6172-aa5f-45c0-8682-cd694f5fa4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:57.014731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd6172-aa5f-45c0-8682-cd694f5fa4a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:57.021002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd6172-aa5f-45c0-8682-cd694f5fa4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:57.021002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fd6172-aa5f-45c0-8682-cd694f5fa4a1 admin admin] PATH_INFO: `/v3/projects/b5bb286579e34b67b1102dd86cc1fd0a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:57.021002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd6172-aa5f-45c0-8682-cd694f5fa4a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5bb286579e34b67b1102dd86cc1fd0a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:57.021002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fd6172-aa5f-45c0-8682-cd694f5fa4a1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:57.071140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:57.186815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-75fd6172-aa5f-45c0-8682-cd694f5fa4a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5bb286579e34b67b1102dd86cc1fd0a', 'request_id': 'req-75fd6172-aa5f-45c0-8682-cd694f5fa4a1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:57.193369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-75fd6172-aa5f-45c0-8682-cd694f5fa4a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5bb286579e34b67b1102dd86cc1fd0a', 'request_id': 'req-75fd6172-aa5f-45c0-8682-cd694f5fa4a1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:57.302046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-75fd6172-aa5f-45c0-8682-cd694f5fa4a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5bb286579e34b67b1102dd86cc1fd0a', 'request_id': 'req-75fd6172-aa5f-45c0-8682-cd694f5fa4a1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:57.316689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-75fd6172-aa5f-45c0-8682-cd694f5fa4a1 admin admin] The token cache is being invalidate because project b5bb286579e34b67b1102dd86cc1fd0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:16:57.317084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-75fd6172-aa5f-45c0-8682-cd694f5fa4a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:57.317980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-75fd6172-aa5f-45c0-8682-cd694f5fa4a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:16:57.319734 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2207/4437] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:16:56 2022] DELETE /identity/v3/projects/b5bb286579e34b67b1102dd86cc1fd0a => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:57.324747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f57f9-2bd2-4617-b3ce-1454cc024d1e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:57.325257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f57f9-2bd2-4617-b3ce-1454cc024d1e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:57.325374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790f57f9-2bd2-4617-b3ce-1454cc024d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:57.344639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-790f57f9-2bd2-4617-b3ce-1454cc024d1e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:57.346483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-790f57f9-2bd2-4617-b3ce-1454cc024d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:57.411787 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2231/4438] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:57.419985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385519f-469c-46ed-9457-4b64945655be None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:57.420376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385519f-469c-46ed-9457-4b64945655be None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:57.420725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b385519f-469c-46ed-9457-4b64945655be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:57.439774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b385519f-469c-46ed-9457-4b64945655be None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:57.445321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b385519f-469c-46ed-9457-4b64945655be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:57.450220 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2208/4439] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:16:57.454591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f57f9-2bd2-4617-b3ce-1454cc024d1e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:57.472372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790f57f9-2bd2-4617-b3ce-1454cc024d1e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:57.472372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98128f45-6c60-4bc9-9f9d-3d5a9d2c628e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:57.472372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98128f45-6c60-4bc9-9f9d-3d5a9d2c628e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:57.472372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98128f45-6c60-4bc9-9f9d-3d5a9d2c628e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:57.473060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98128f45-6c60-4bc9-9f9d-3d5a9d2c628e admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:57.473060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98128f45-6c60-4bc9-9f9d-3d5a9d2c628e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:16:57.473060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98128f45-6c60-4bc9-9f9d-3d5a9d2c628e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:57.474037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98128f45-6c60-4bc9-9f9d-3d5a9d2c628e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:57.486654 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2232/4440] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:16:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:16:57.497938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385519f-469c-46ed-9457-4b64945655be None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:57.508120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b385519f-469c-46ed-9457-4b64945655be None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:57.514145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ee030-2b1b-4bef-a754-dc8718a11c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:57.518126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ee030-2b1b-4bef-a754-dc8718a11c1f admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:57.518126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ee030-2b1b-4bef-a754-dc8718a11c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:57.518126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ee030-2b1b-4bef-a754-dc8718a11c1f admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:57.518126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ee030-2b1b-4bef-a754-dc8718a11c1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:57.518126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ee030-2b1b-4bef-a754-dc8718a11c1f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:57.627314 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2209/4441] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:16:57 2022] POST /identity/v3/projects => generated 357 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:16:57.628492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98128f45-6c60-4bc9-9f9d-3d5a9d2c628e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:57.639459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98128f45-6c60-4bc9-9f9d-3d5a9d2c628e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:57.707806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090856ac-cd52-4805-a193-a56e88adc086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:57.709203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090856ac-cd52-4805-a193-a56e88adc086 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:57.709521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090856ac-cd52-4805-a193-a56e88adc086 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:57.709805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090856ac-cd52-4805-a193-a56e88adc086 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:57.710557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090856ac-cd52-4805-a193-a56e88adc086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:57.712714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090856ac-cd52-4805-a193-a56e88adc086 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:57.831747 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2233/4442] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:16:57 2022] POST /identity/v3/users => generated 279 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:16:57.832804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ee030-2b1b-4bef-a754-dc8718a11c1f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:57.845595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ee030-2b1b-4bef-a754-dc8718a11c1f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:57.850352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d4352-37fb-4724-b79a-6e623488ba7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:57.851021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d4352-37fb-4724-b79a-6e623488ba7d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:57.851304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d4352-37fb-4724-b79a-6e623488ba7d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:57.851582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d4352-37fb-4724-b79a-6e623488ba7d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:57.852349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d4352-37fb-4724-b79a-6e623488ba7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:57.854037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d4352-37fb-4724-b79a-6e623488ba7d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:57.865533 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2210/4443] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:16:57.869509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090856ac-cd52-4805-a193-a56e88adc086 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:57.880519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090856ac-cd52-4805-a193-a56e88adc086 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:57.893360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2bf51d-6aa9-475e-948b-7801017e0781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:57.895897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bf51d-6aa9-475e-948b-7801017e0781 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:57.896276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bf51d-6aa9-475e-948b-7801017e0781 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:57.896640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2bf51d-6aa9-475e-948b-7801017e0781 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:57.897953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2bf51d-6aa9-475e-948b-7801017e0781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:57.899937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2bf51d-6aa9-475e-948b-7801017e0781 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:57.921976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d4352-37fb-4724-b79a-6e623488ba7d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:57.934818 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2234/4444] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:16:57.940343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d4352-37fb-4724-b79a-6e623488ba7d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:57.946081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866958b-fe85-467f-82e9-685a85d38817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:57.946983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866958b-fe85-467f-82e9-685a85d38817 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:57.947268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866958b-fe85-467f-82e9-685a85d38817 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:57.947547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2866958b-fe85-467f-82e9-685a85d38817 admin admin] PATH_INFO: `/v3/projects/6c438a847524489a8c273b283bceafbe/users/14b9e4bebdd949319785e03cda251f1c/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:57.964211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866958b-fe85-467f-82e9-685a85d38817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c438a847524489a8c273b283bceafbe, user_id=14b9e4bebdd949319785e03cda251f1c, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:57.966925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2866958b-fe85-467f-82e9-685a85d38817 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:58.021794 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2211/4445] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:16:57 2022] PUT /identity/v3/projects/6c438a847524489a8c273b283bceafbe/users/14b9e4bebdd949319785e03cda251f1c/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:16:58.032426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67661c-063a-448a-989a-95fe3b3a24d1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:58.032736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67661c-063a-448a-989a-95fe3b3a24d1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:58.033004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a67661c-063a-448a-989a-95fe3b3a24d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:58.062397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7a67661c-063a-448a-989a-95fe3b3a24d1 None None] MFA Rules not processed for user `14b9e4bebdd949319785e03cda251f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:16:58.064398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7a67661c-063a-448a-989a-95fe3b3a24d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:58.097413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:58.137155 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2235/4446] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:16:58 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:16:58.148824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866958b-fe85-467f-82e9-685a85d38817 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:16:58.150608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2866958b-fe85-467f-82e9-685a85d38817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:16:58.215121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2866958b-fe85-467f-82e9-685a85d38817 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:16:58.219351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8359e48-a0a6-475e-9dfd-8f003f6f5675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:16:58.220001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8359e48-a0a6-475e-9dfd-8f003f6f5675 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:16:58.220318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8359e48-a0a6-475e-9dfd-8f003f6f5675 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:16:58.220628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8359e48-a0a6-475e-9dfd-8f003f6f5675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:16:58.230896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8359e48-a0a6-475e-9dfd-8f003f6f5675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:16:58.232713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8359e48-a0a6-475e-9dfd-8f003f6f5675 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:16:58.243201 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2212/4447] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:16:58 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:17:02.564126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67661c-063a-448a-989a-95fe3b3a24d1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:02.567245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7a67661c-063a-448a-989a-95fe3b3a24d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:02.690034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a67661c-063a-448a-989a-95fe3b3a24d1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:02.719365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458bbf72-595e-4d51-b088-5cbafbdc641f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:02.720467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bbf72-595e-4d51-b088-5cbafbdc641f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:02.721014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bbf72-595e-4d51-b088-5cbafbdc641f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:02.721392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458bbf72-595e-4d51-b088-5cbafbdc641f admin admin] PATH_INFO: `/v3/users/665ce91603f24affb78017790130f733` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:02.726904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458bbf72-595e-4d51-b088-5cbafbdc641f admin admin] RBAC: Authorizing `identity:delete_user(user_id=665ce91603f24affb78017790130f733)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:02.734225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458bbf72-595e-4d51-b088-5cbafbdc641f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:03.039881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-458bbf72-595e-4d51-b088-5cbafbdc641f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '665ce91603f24affb78017790130f733'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:03.050579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-458bbf72-595e-4d51-b088-5cbafbdc641f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '665ce91603f24affb78017790130f733'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:03.055816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-458bbf72-595e-4d51-b088-5cbafbdc641f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '665ce91603f24affb78017790130f733'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:03.075228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-458bbf72-595e-4d51-b088-5cbafbdc641f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '665ce91603f24affb78017790130f733'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:03.166424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-458bbf72-595e-4d51-b088-5cbafbdc641f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '665ce91603f24affb78017790130f733'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:03.214414 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2236/4448] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:17:02 2022] DELETE /identity/v3/users/665ce91603f24affb78017790130f733 => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:03.707360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8359e48-a0a6-475e-9dfd-8f003f6f5675 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:03.723077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8359e48-a0a6-475e-9dfd-8f003f6f5675 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:03.852502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:03.881015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff209a-f2b0-4a17-af14-639681f3a6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:03.882988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff209a-f2b0-4a17-af14-639681f3a6dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:03.887255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff209a-f2b0-4a17-af14-639681f3a6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:03.887255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff209a-f2b0-4a17-af14-639681f3a6dc admin admin] PATH_INFO: `/v3/projects/5e2aed4ec2e74e69bb5eea2e65ab92bb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:03.887255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff209a-f2b0-4a17-af14-639681f3a6dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e2aed4ec2e74e69bb5eea2e65ab92bb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:03.888263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff209a-f2b0-4a17-af14-639681f3a6dc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:04.103205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-08ff209a-f2b0-4a17-af14-639681f3a6dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e2aed4ec2e74e69bb5eea2e65ab92bb', 'request_id': 'req-08ff209a-f2b0-4a17-af14-639681f3a6dc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:04.111190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-08ff209a-f2b0-4a17-af14-639681f3a6dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e2aed4ec2e74e69bb5eea2e65ab92bb', 'request_id': 'req-08ff209a-f2b0-4a17-af14-639681f3a6dc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:04.216260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-08ff209a-f2b0-4a17-af14-639681f3a6dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e2aed4ec2e74e69bb5eea2e65ab92bb', 'request_id': 'req-08ff209a-f2b0-4a17-af14-639681f3a6dc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:04.243945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-08ff209a-f2b0-4a17-af14-639681f3a6dc admin admin] The token cache is being invalidate because project 5e2aed4ec2e74e69bb5eea2e65ab92bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:17:04.246964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-08ff209a-f2b0-4a17-af14-639681f3a6dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:04.249224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-08ff209a-f2b0-4a17-af14-639681f3a6dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:04.255510 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2213/4449] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:17:03 2022] DELETE /identity/v3/projects/5e2aed4ec2e74e69bb5eea2e65ab92bb => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:04.262033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70552d2-8a84-4a87-af0b-0da384324279 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:04.262882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70552d2-8a84-4a87-af0b-0da384324279 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:04.263492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70552d2-8a84-4a87-af0b-0da384324279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:04.324861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d70552d2-8a84-4a87-af0b-0da384324279 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:04.334309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d70552d2-8a84-4a87-af0b-0da384324279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:04.384748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:04.411425 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2237/4450] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:04.414232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86d008-e234-4129-986e-f8faeabdcb63 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:04.414681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86d008-e234-4129-986e-f8faeabdcb63 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:04.415039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a86d008-e234-4129-986e-f8faeabdcb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:04.444283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7a86d008-e234-4129-986e-f8faeabdcb63 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:04.446235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7a86d008-e234-4129-986e-f8faeabdcb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:04.461886 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2214/4451] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:04.465371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70552d2-8a84-4a87-af0b-0da384324279 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:04.477488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70552d2-8a84-4a87-af0b-0da384324279 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:04.482784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c601284-d0d4-488c-a92a-cd8f2f3bf2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:04.485101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c601284-d0d4-488c-a92a-cd8f2f3bf2aa admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:04.485604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c601284-d0d4-488c-a92a-cd8f2f3bf2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:04.485921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c601284-d0d4-488c-a92a-cd8f2f3bf2aa admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:04.487169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c601284-d0d4-488c-a92a-cd8f2f3bf2aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:17:04.487275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c601284-d0d4-488c-a92a-cd8f2f3bf2aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:04.488873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c601284-d0d4-488c-a92a-cd8f2f3bf2aa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:04.509588 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2238/4452] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:17:04.510781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86d008-e234-4129-986e-f8faeabdcb63 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:04.521371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a86d008-e234-4129-986e-f8faeabdcb63 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:04.531176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea595b60-29ff-4393-858c-b9aee32fce95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:04.531176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea595b60-29ff-4393-858c-b9aee32fce95 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:04.531376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea595b60-29ff-4393-858c-b9aee32fce95 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:04.531957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea595b60-29ff-4393-858c-b9aee32fce95 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:04.534309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea595b60-29ff-4393-858c-b9aee32fce95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:04.538975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea595b60-29ff-4393-858c-b9aee32fce95 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:04.658772 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2215/4453] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:17:04 2022] POST /identity/v3/projects => generated 403 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:17:04.660620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c601284-d0d4-488c-a92a-cd8f2f3bf2aa admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:04.696828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c601284-d0d4-488c-a92a-cd8f2f3bf2aa admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:04.797986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3fb32-3e21-4b55-8e7e-dd144b7da762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:04.799323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3fb32-3e21-4b55-8e7e-dd144b7da762 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:04.799759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3fb32-3e21-4b55-8e7e-dd144b7da762 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:04.799953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3fb32-3e21-4b55-8e7e-dd144b7da762 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:04.800772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3fb32-3e21-4b55-8e7e-dd144b7da762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:04.803691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3fb32-3e21-4b55-8e7e-dd144b7da762 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:04.940230 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2239/4454] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:17:04 2022] POST /identity/v3/users => generated 302 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:17:04.944484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea595b60-29ff-4393-858c-b9aee32fce95 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:04.956216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea595b60-29ff-4393-858c-b9aee32fce95 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:04.961653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118e270-bc16-4ef2-8418-6e0343136a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:04.962531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118e270-bc16-4ef2-8418-6e0343136a4d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:04.962827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118e270-bc16-4ef2-8418-6e0343136a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:04.963107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118e270-bc16-4ef2-8418-6e0343136a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:04.964004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118e270-bc16-4ef2-8418-6e0343136a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:04.965627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118e270-bc16-4ef2-8418-6e0343136a4d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:04.978565 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2216/4455] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:17:04.983168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3fb32-3e21-4b55-8e7e-dd144b7da762 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:04.995251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3fb32-3e21-4b55-8e7e-dd144b7da762 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:05.008646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2c4dfc-7718-4a7a-9f89-48c2855d11cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:05.009572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c4dfc-7718-4a7a-9f89-48c2855d11cb admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:05.010409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c4dfc-7718-4a7a-9f89-48c2855d11cb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:05.010692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2c4dfc-7718-4a7a-9f89-48c2855d11cb admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:05.011575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c4dfc-7718-4a7a-9f89-48c2855d11cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:05.014310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2c4dfc-7718-4a7a-9f89-48c2855d11cb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:05.029462 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2240/4456] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:17:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:17:05.031219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118e270-bc16-4ef2-8418-6e0343136a4d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:05.042653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118e270-bc16-4ef2-8418-6e0343136a4d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:05.047361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68cabe-b613-4e13-b21e-42e61373c2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:05.048276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68cabe-b613-4e13-b21e-42e61373c2ee admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:05.048545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68cabe-b613-4e13-b21e-42e61373c2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:05.048845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68cabe-b613-4e13-b21e-42e61373c2ee admin admin] PATH_INFO: `/v3/projects/2be6782605ab4d8f8e4289aaa8b19bcd/users/4c425cfffaea4e86ab29da7bb1f4c2ff/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:05.067593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68cabe-b613-4e13-b21e-42e61373c2ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=2be6782605ab4d8f8e4289aaa8b19bcd, user_id=4c425cfffaea4e86ab29da7bb1f4c2ff, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:05.070549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68cabe-b613-4e13-b21e-42e61373c2ee admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:05.526923 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2217/4457] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:17:05 2022] PUT /identity/v3/projects/2be6782605ab4d8f8e4289aaa8b19bcd/users/4c425cfffaea4e86ab29da7bb1f4c2ff/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:05.539691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c40bf5-e61c-45c0-8b4b-d60e136e0014 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:05.539691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c40bf5-e61c-45c0-8b4b-d60e136e0014 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:05.543425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c40bf5-e61c-45c0-8b4b-d60e136e0014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:05.570268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-23c40bf5-e61c-45c0-8b4b-d60e136e0014 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:05.574123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-23c40bf5-e61c-45c0-8b4b-d60e136e0014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:05.704644 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2241/4458] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:05.715559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68cabe-b613-4e13-b21e-42e61373c2ee admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:05.717400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3d68cabe-b613-4e13-b21e-42e61373c2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:05.804055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68cabe-b613-4e13-b21e-42e61373c2ee admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:05.810191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f8266-052d-45c5-8f2c-32b0c8322357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:05.810191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f8266-052d-45c5-8f2c-32b0c8322357 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:05.810191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f8266-052d-45c5-8f2c-32b0c8322357 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:05.810191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319f8266-052d-45c5-8f2c-32b0c8322357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:05.823636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f8266-052d-45c5-8f2c-32b0c8322357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:05.825106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319f8266-052d-45c5-8f2c-32b0c8322357 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:05.851495 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2218/4459] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:17:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:17:08.241517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64138d50-2cd0-4315-9cc3-08e05b4a2304 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:08.243239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64138d50-2cd0-4315-9cc3-08e05b4a2304 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:08.243551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64138d50-2cd0-4315-9cc3-08e05b4a2304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:08.288647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-64138d50-2cd0-4315-9cc3-08e05b4a2304 None None] MFA Rules not processed for user `4c425cfffaea4e86ab29da7bb1f4c2ff`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:08.290904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-64138d50-2cd0-4315-9cc3-08e05b4a2304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:08.359343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:08.389489 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2242/4460] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:08.405424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f8266-052d-45c5-8f2c-32b0c8322357 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:08.407063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-319f8266-052d-45c5-8f2c-32b0c8322357 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:08.442970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:08.543534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319f8266-052d-45c5-8f2c-32b0c8322357 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:08.551188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b020efc1-6d0d-490f-8c13-ac5bbac9e928 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:08.552129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020efc1-6d0d-490f-8c13-ac5bbac9e928 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:08.552522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020efc1-6d0d-490f-8c13-ac5bbac9e928 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:08.552861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b020efc1-6d0d-490f-8c13-ac5bbac9e928 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:08.561831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020efc1-6d0d-490f-8c13-ac5bbac9e928 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:08.563730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b020efc1-6d0d-490f-8c13-ac5bbac9e928 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:08.577661 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2219/4461] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:17:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:17:13.861434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64138d50-2cd0-4315-9cc3-08e05b4a2304 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:13.862980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-64138d50-2cd0-4315-9cc3-08e05b4a2304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:13.883874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64138d50-2cd0-4315-9cc3-08e05b4a2304 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:13.893178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3a74d-c1fd-4a41-a280-f5c88f178045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:13.898018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3a74d-c1fd-4a41-a280-f5c88f178045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:13.898755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3a74d-c1fd-4a41-a280-f5c88f178045 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:13.899907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b3a74d-c1fd-4a41-a280-f5c88f178045 admin admin] PATH_INFO: `/v3/users/a9c0c9f3a90648148023c4b8038f7d23` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:13.912068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3a74d-c1fd-4a41-a280-f5c88f178045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9c0c9f3a90648148023c4b8038f7d23)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:13.914080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b3a74d-c1fd-4a41-a280-f5c88f178045 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:14.185497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c0b3a74d-c1fd-4a41-a280-f5c88f178045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9c0c9f3a90648148023c4b8038f7d23'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:14.196091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c0b3a74d-c1fd-4a41-a280-f5c88f178045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9c0c9f3a90648148023c4b8038f7d23'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:14.197579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c0b3a74d-c1fd-4a41-a280-f5c88f178045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9c0c9f3a90648148023c4b8038f7d23'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:14.198286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c0b3a74d-c1fd-4a41-a280-f5c88f178045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9c0c9f3a90648148023c4b8038f7d23'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:14.346693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c0b3a74d-c1fd-4a41-a280-f5c88f178045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9c0c9f3a90648148023c4b8038f7d23'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:14.370748 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2243/4462] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:17:13 2022] DELETE /identity/v3/users/a9c0c9f3a90648148023c4b8038f7d23 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:14.375390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258fe41-fc61-4aec-a632-625873ac737d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:14.376375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258fe41-fc61-4aec-a632-625873ac737d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:14.376824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0258fe41-fc61-4aec-a632-625873ac737d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:14.396581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0258fe41-fc61-4aec-a632-625873ac737d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:14.398256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0258fe41-fc61-4aec-a632-625873ac737d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:14.495541 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2220/4463] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:14.501734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3a74d-c1fd-4a41-a280-f5c88f178045 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:14.505080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c0b3a74d-c1fd-4a41-a280-f5c88f178045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:14.579089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b3a74d-c1fd-4a41-a280-f5c88f178045 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:14.583829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaddf4c-4cc9-4ed9-aa21-72bda851d0e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:14.584838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaddf4c-4cc9-4ed9-aa21-72bda851d0e9 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:14.585327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaddf4c-4cc9-4ed9-aa21-72bda851d0e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:14.585826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaddf4c-4cc9-4ed9-aa21-72bda851d0e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:14.599952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaddf4c-4cc9-4ed9-aa21-72bda851d0e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:14.602040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaddf4c-4cc9-4ed9-aa21-72bda851d0e9 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:14.627090 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2244/4464] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:17:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:17:14.991915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258fe41-fc61-4aec-a632-625873ac737d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:14.993975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0258fe41-fc61-4aec-a632-625873ac737d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:15.016129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0258fe41-fc61-4aec-a632-625873ac737d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:15.022009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe98d833-1829-4767-8c79-761195672c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:15.023224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98d833-1829-4767-8c79-761195672c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:15.023720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98d833-1829-4767-8c79-761195672c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:15.024363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe98d833-1829-4767-8c79-761195672c3d admin admin] PATH_INFO: `/v3/projects/c170941c16e3489c9f2db2ff2883e5e4` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:15.042293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98d833-1829-4767-8c79-761195672c3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c170941c16e3489c9f2db2ff2883e5e4)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:15.042293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe98d833-1829-4767-8c79-761195672c3d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:15.260086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fe98d833-1829-4767-8c79-761195672c3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c170941c16e3489c9f2db2ff2883e5e4', 'request_id': 'req-fe98d833-1829-4767-8c79-761195672c3d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:15.267443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fe98d833-1829-4767-8c79-761195672c3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c170941c16e3489c9f2db2ff2883e5e4', 'request_id': 'req-fe98d833-1829-4767-8c79-761195672c3d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:15.370085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fe98d833-1829-4767-8c79-761195672c3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c170941c16e3489c9f2db2ff2883e5e4', 'request_id': 'req-fe98d833-1829-4767-8c79-761195672c3d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:15.382190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fe98d833-1829-4767-8c79-761195672c3d admin admin] The token cache is being invalidate because project c170941c16e3489c9f2db2ff2883e5e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:17:15.382884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fe98d833-1829-4767-8c79-761195672c3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:15.383377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fe98d833-1829-4767-8c79-761195672c3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:15.385019 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2221/4465] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:17:14 2022] DELETE /identity/v3/projects/c170941c16e3489c9f2db2ff2883e5e4 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:15.391989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6339c56-9469-4f43-8bc7-d460a64a1690 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:15.392862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6339c56-9469-4f43-8bc7-d460a64a1690 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:15.393151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6339c56-9469-4f43-8bc7-d460a64a1690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:15.412492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c6339c56-9469-4f43-8bc7-d460a64a1690 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:15.413707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c6339c56-9469-4f43-8bc7-d460a64a1690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:15.478035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:15.494019 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2245/4466] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:17:15.500664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62be97-c267-4e92-b72a-b8a8c6855454 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:15.501041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62be97-c267-4e92-b72a-b8a8c6855454 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:15.501239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62be97-c267-4e92-b72a-b8a8c6855454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:15.530469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9c62be97-c267-4e92-b72a-b8a8c6855454 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:15.534244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9c62be97-c267-4e92-b72a-b8a8c6855454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:15.541635 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2222/4467] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:15.546124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6339c56-9469-4f43-8bc7-d460a64a1690 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:15.557386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6339c56-9469-4f43-8bc7-d460a64a1690 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:15.562787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1e8b6-2805-449d-80a1-475bacfbeea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:15.563462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1e8b6-2805-449d-80a1-475bacfbeea0 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:15.563878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1e8b6-2805-449d-80a1-475bacfbeea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:15.564167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d1e8b6-2805-449d-80a1-475bacfbeea0 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:15.565057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1e8b6-2805-449d-80a1-475bacfbeea0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:17:15.565617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1e8b6-2805-449d-80a1-475bacfbeea0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:15.568464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d1e8b6-2805-449d-80a1-475bacfbeea0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:15.583252 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2246/4468] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:17:15.587095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62be97-c267-4e92-b72a-b8a8c6855454 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:15.597203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62be97-c267-4e92-b72a-b8a8c6855454 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:15.604714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d968b38-e799-4f78-8f59-88b9c1412ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:15.608930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d968b38-e799-4f78-8f59-88b9c1412ff3 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:15.608930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d968b38-e799-4f78-8f59-88b9c1412ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:15.608930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d968b38-e799-4f78-8f59-88b9c1412ff3 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:15.608930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d968b38-e799-4f78-8f59-88b9c1412ff3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:15.609531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d968b38-e799-4f78-8f59-88b9c1412ff3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:15.627474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:15.712553 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2223/4469] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:17:15 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:17:15.713973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1e8b6-2805-449d-80a1-475bacfbeea0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:15.730065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d1e8b6-2805-449d-80a1-475bacfbeea0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:15.789451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796d39e-def7-41f4-84b6-35ad29acbbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:15.790235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796d39e-def7-41f4-84b6-35ad29acbbf5 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:15.790472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796d39e-def7-41f4-84b6-35ad29acbbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:15.790847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796d39e-def7-41f4-84b6-35ad29acbbf5 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:15.791552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796d39e-def7-41f4-84b6-35ad29acbbf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:15.793483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796d39e-def7-41f4-84b6-35ad29acbbf5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:15.899667 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2247/4470] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:17:15 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:17:15.901869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d968b38-e799-4f78-8f59-88b9c1412ff3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:15.930669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d968b38-e799-4f78-8f59-88b9c1412ff3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:15.936529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc34bd9-a1e7-4c4b-9e07-7086f0f7331b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:15.938731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc34bd9-a1e7-4c4b-9e07-7086f0f7331b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:15.939096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc34bd9-a1e7-4c4b-9e07-7086f0f7331b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:15.939620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc34bd9-a1e7-4c4b-9e07-7086f0f7331b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:15.940393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc34bd9-a1e7-4c4b-9e07-7086f0f7331b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:15.942119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc34bd9-a1e7-4c4b-9e07-7086f0f7331b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:15.956674 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2224/4471] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:17:15.960605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796d39e-def7-41f4-84b6-35ad29acbbf5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:15.973261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796d39e-def7-41f4-84b6-35ad29acbbf5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:15.978341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff364c88-5917-4a90-bde4-02b97d0a2714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:15.979236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff364c88-5917-4a90-bde4-02b97d0a2714 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:15.979524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff364c88-5917-4a90-bde4-02b97d0a2714 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:15.979808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff364c88-5917-4a90-bde4-02b97d0a2714 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:15.980545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff364c88-5917-4a90-bde4-02b97d0a2714 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:15.982484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff364c88-5917-4a90-bde4-02b97d0a2714 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:15.996139 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2248/4472] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:17:16.000345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc34bd9-a1e7-4c4b-9e07-7086f0f7331b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:16.011629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc34bd9-a1e7-4c4b-9e07-7086f0f7331b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:16.017199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f793bb-bcca-4f83-89c7-e35bdabbcc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:16.018131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f793bb-bcca-4f83-89c7-e35bdabbcc14 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:16.018419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f793bb-bcca-4f83-89c7-e35bdabbcc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:16.018760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f793bb-bcca-4f83-89c7-e35bdabbcc14 admin admin] PATH_INFO: `/v3/projects/161b7adf0b4a4d489d11c62ad0f03a3f/users/abab220deb8943f0b9c7e5559d8e4440/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:16.035140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f793bb-bcca-4f83-89c7-e35bdabbcc14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=161b7adf0b4a4d489d11c62ad0f03a3f, user_id=abab220deb8943f0b9c7e5559d8e4440, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:16.038102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f793bb-bcca-4f83-89c7-e35bdabbcc14 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:16.153273 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2225/4473] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:17:15 2022] PUT /identity/v3/projects/161b7adf0b4a4d489d11c62ad0f03a3f/users/abab220deb8943f0b9c7e5559d8e4440/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:16.165866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1b65f-eaf1-4600-b4a6-efc94d66fbde None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:16.166205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1b65f-eaf1-4600-b4a6-efc94d66fbde None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:16.166517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e1b65f-eaf1-4600-b4a6-efc94d66fbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:16.198007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-60e1b65f-eaf1-4600-b4a6-efc94d66fbde None None] MFA Rules not processed for user `abab220deb8943f0b9c7e5559d8e4440`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:16.199290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-60e1b65f-eaf1-4600-b4a6-efc94d66fbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:16.288797 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2249/4474] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:16.300196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f793bb-bcca-4f83-89c7-e35bdabbcc14 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:16.306665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-65f793bb-bcca-4f83-89c7-e35bdabbcc14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:16.429367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f793bb-bcca-4f83-89c7-e35bdabbcc14 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:16.434468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806e06e-0702-4f23-beef-4dd4a858d56f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:16.435325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806e06e-0702-4f23-beef-4dd4a858d56f service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:16.435927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806e06e-0702-4f23-beef-4dd4a858d56f service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:16.436364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e806e06e-0702-4f23-beef-4dd4a858d56f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:16.446406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806e06e-0702-4f23-beef-4dd4a858d56f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:16.448982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e806e06e-0702-4f23-beef-4dd4a858d56f service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:16.466762 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2226/4475] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:17:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:17:17.002450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1b65f-eaf1-4600-b4a6-efc94d66fbde None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:17.005625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-60e1b65f-eaf1-4600-b4a6-efc94d66fbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:17.030064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806e06e-0702-4f23-beef-4dd4a858d56f service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:17.032936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e806e06e-0702-4f23-beef-4dd4a858d56f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:17.089906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:17.119917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e1b65f-eaf1-4600-b4a6-efc94d66fbde None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:17.124006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e806e06e-0702-4f23-beef-4dd4a858d56f service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:17.126316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62184f-f437-4d5f-8711-d38df3446cf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:17.127178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62184f-f437-4d5f-8711-d38df3446cf4 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:17.127656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62184f-f437-4d5f-8711-d38df3446cf4 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:17.127943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62184f-f437-4d5f-8711-d38df3446cf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:17.130645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-af62184f-f437-4d5f-8711-d38df3446cf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:17.131417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25184b-e7f8-42c8-9041-1966d5d91b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:17.132442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25184b-e7f8-42c8-9041-1966d5d91b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:17.132792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25184b-e7f8-42c8-9041-1966d5d91b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:17.133167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25184b-e7f8-42c8-9041-1966d5d91b5c admin admin] PATH_INFO: `/v3/users/4c425cfffaea4e86ab29da7bb1f4c2ff` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:17.134513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25184b-e7f8-42c8-9041-1966d5d91b5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c425cfffaea4e86ab29da7bb1f4c2ff)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:17.136928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25184b-e7f8-42c8-9041-1966d5d91b5c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:17.203983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62184f-f437-4d5f-8711-d38df3446cf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:17.205804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62184f-f437-4d5f-8711-d38df3446cf4 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:17.229012 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2250/4476] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:17:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:17:17.409209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3f25184b-e7f8-42c8-9041-1966d5d91b5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c425cfffaea4e86ab29da7bb1f4c2ff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:17.410076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3f25184b-e7f8-42c8-9041-1966d5d91b5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c425cfffaea4e86ab29da7bb1f4c2ff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:17.420080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3f25184b-e7f8-42c8-9041-1966d5d91b5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c425cfffaea4e86ab29da7bb1f4c2ff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:17.580861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3f25184b-e7f8-42c8-9041-1966d5d91b5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c425cfffaea4e86ab29da7bb1f4c2ff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:17.582119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3f25184b-e7f8-42c8-9041-1966d5d91b5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c425cfffaea4e86ab29da7bb1f4c2ff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:17.587570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:17.618769 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2227/4477] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:17:17 2022] DELETE /identity/v3/users/4c425cfffaea4e86ab29da7bb1f4c2ff => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:17.992632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62184f-f437-4d5f-8711-d38df3446cf4 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:18.011557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62184f-f437-4d5f-8711-d38df3446cf4 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:18.111103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9672e04-6776-4121-8cb5-99bfc6ee7705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:18.114933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9672e04-6776-4121-8cb5-99bfc6ee7705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:18.114933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9672e04-6776-4121-8cb5-99bfc6ee7705 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:18.114933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9672e04-6776-4121-8cb5-99bfc6ee7705 admin admin] PATH_INFO: `/v3/projects/2be6782605ab4d8f8e4289aaa8b19bcd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:18.116002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9672e04-6776-4121-8cb5-99bfc6ee7705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2be6782605ab4d8f8e4289aaa8b19bcd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:18.120133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9672e04-6776-4121-8cb5-99bfc6ee7705 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:18.267331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d9672e04-6776-4121-8cb5-99bfc6ee7705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2be6782605ab4d8f8e4289aaa8b19bcd', 'request_id': 'req-d9672e04-6776-4121-8cb5-99bfc6ee7705'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:18.276418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d9672e04-6776-4121-8cb5-99bfc6ee7705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2be6782605ab4d8f8e4289aaa8b19bcd', 'request_id': 'req-d9672e04-6776-4121-8cb5-99bfc6ee7705'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:18.399100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d9672e04-6776-4121-8cb5-99bfc6ee7705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2be6782605ab4d8f8e4289aaa8b19bcd', 'request_id': 'req-d9672e04-6776-4121-8cb5-99bfc6ee7705'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:18.417373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d9672e04-6776-4121-8cb5-99bfc6ee7705 admin admin] The token cache is being invalidate because project 2be6782605ab4d8f8e4289aaa8b19bcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:17:18.417373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d9672e04-6776-4121-8cb5-99bfc6ee7705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:18.418204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d9672e04-6776-4121-8cb5-99bfc6ee7705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:18.423056 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2251/4478] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:17:17 2022] DELETE /identity/v3/projects/2be6782605ab4d8f8e4289aaa8b19bcd => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:17:18.429152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75d562-96a0-4f43-9332-40c412a8d73f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:18.429690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75d562-96a0-4f43-9332-40c412a8d73f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:18.434198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75d562-96a0-4f43-9332-40c412a8d73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:18.453150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6c75d562-96a0-4f43-9332-40c412a8d73f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:18.455087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6c75d562-96a0-4f43-9332-40c412a8d73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:18.548703 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2228/4479] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:18.557119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc69af-9c7c-4aae-adb3-0b03ccf831b9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:18.558004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc69af-9c7c-4aae-adb3-0b03ccf831b9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:18.558513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc69af-9c7c-4aae-adb3-0b03ccf831b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:18.591458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-eecc69af-9c7c-4aae-adb3-0b03ccf831b9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:18.592808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eecc69af-9c7c-4aae-adb3-0b03ccf831b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:18.604696 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2252/4480] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:18.609344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75d562-96a0-4f43-9332-40c412a8d73f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:18.621546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75d562-96a0-4f43-9332-40c412a8d73f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:18.627683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbad0a-b85a-43e9-a014-8a8763b84dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:18.628770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbad0a-b85a-43e9-a014-8a8763b84dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:18.629823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbad0a-b85a-43e9-a014-8a8763b84dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:18.630342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbad0a-b85a-43e9-a014-8a8763b84dd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:18.631395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbad0a-b85a-43e9-a014-8a8763b84dd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:17:18.631941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbad0a-b85a-43e9-a014-8a8763b84dd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:18.634112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbad0a-b85a-43e9-a014-8a8763b84dd9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:18.653373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc69af-9c7c-4aae-adb3-0b03ccf831b9 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:18.660774 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2229/4481] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:17:18.665274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:18.668699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc69af-9c7c-4aae-adb3-0b03ccf831b9 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:18.676557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817dcb9-ca23-4824-9edc-5f5f8bfd5ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:18.677917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817dcb9-ca23-4824-9edc-5f5f8bfd5ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:18.679387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817dcb9-ca23-4824-9edc-5f5f8bfd5ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:18.679923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817dcb9-ca23-4824-9edc-5f5f8bfd5ed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:18.680888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817dcb9-ca23-4824-9edc-5f5f8bfd5ed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:18.683479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5817dcb9-ca23-4824-9edc-5f5f8bfd5ed9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:18.789347 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2253/4482] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:17:18 2022] POST /identity/v3/projects => generated 399 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:17:18.793683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbad0a-b85a-43e9-a014-8a8763b84dd9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:18.807427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbad0a-b85a-43e9-a014-8a8763b84dd9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:18.887785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e68097-0b80-410e-af27-057ee7b4afd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:18.889737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e68097-0b80-410e-af27-057ee7b4afd8 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:18.889737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e68097-0b80-410e-af27-057ee7b4afd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:18.890107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e68097-0b80-410e-af27-057ee7b4afd8 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:18.890966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e68097-0b80-410e-af27-057ee7b4afd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:18.892684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e68097-0b80-410e-af27-057ee7b4afd8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:19.016151 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2230/4483] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:17:18 2022] POST /identity/v3/users => generated 300 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:17:19.020457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817dcb9-ca23-4824-9edc-5f5f8bfd5ed9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:19.031587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5817dcb9-ca23-4824-9edc-5f5f8bfd5ed9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:19.037604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da31586-6053-4230-b14b-3e9c9eb89d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:19.038486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da31586-6053-4230-b14b-3e9c9eb89d3c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:19.038876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da31586-6053-4230-b14b-3e9c9eb89d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:19.039254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da31586-6053-4230-b14b-3e9c9eb89d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:19.040108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da31586-6053-4230-b14b-3e9c9eb89d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:19.041886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da31586-6053-4230-b14b-3e9c9eb89d3c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:19.053879 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2254/4484] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:17:19.063651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e68097-0b80-410e-af27-057ee7b4afd8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:19.075494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e68097-0b80-410e-af27-057ee7b4afd8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:19.081663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b26f9-d6d9-42f0-bf05-10f81fe81685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:19.082469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b26f9-d6d9-42f0-bf05-10f81fe81685 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:19.082822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b26f9-d6d9-42f0-bf05-10f81fe81685 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:19.083165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b26f9-d6d9-42f0-bf05-10f81fe81685 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:19.083998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b26f9-d6d9-42f0-bf05-10f81fe81685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:19.085743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b26f9-d6d9-42f0-bf05-10f81fe81685 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:19.097288 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2231/4485] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:17:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:17:19.101774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da31586-6053-4230-b14b-3e9c9eb89d3c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:19.114475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da31586-6053-4230-b14b-3e9c9eb89d3c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:19.119338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5385cb0-30c5-4321-8bac-dc5d9f3dab98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:19.120246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5385cb0-30c5-4321-8bac-dc5d9f3dab98 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:19.120535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5385cb0-30c5-4321-8bac-dc5d9f3dab98 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:19.120814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5385cb0-30c5-4321-8bac-dc5d9f3dab98 admin admin] PATH_INFO: `/v3/projects/c9dd563271c04f6dae47402935636bb8/users/a952d7989d3945b6a3ed38c2e444a7ba/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:19.149446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5385cb0-30c5-4321-8bac-dc5d9f3dab98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9dd563271c04f6dae47402935636bb8, user_id=a952d7989d3945b6a3ed38c2e444a7ba, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:19.152835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5385cb0-30c5-4321-8bac-dc5d9f3dab98 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:19.287367 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2255/4486] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:17:19 2022] PUT /identity/v3/projects/c9dd563271c04f6dae47402935636bb8/users/a952d7989d3945b6a3ed38c2e444a7ba/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:19.293508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1311a2-0961-4a8a-8f12-251189adfc64 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:19.293815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1311a2-0961-4a8a-8f12-251189adfc64 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:19.294144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1311a2-0961-4a8a-8f12-251189adfc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:19.324110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ee1311a2-0961-4a8a-8f12-251189adfc64 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:19.325580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ee1311a2-0961-4a8a-8f12-251189adfc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:19.404081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:19.426506 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2232/4487] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:19.433506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5385cb0-30c5-4321-8bac-dc5d9f3dab98 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:19.435651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b5385cb0-30c5-4321-8bac-dc5d9f3dab98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:19.533541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5385cb0-30c5-4321-8bac-dc5d9f3dab98 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:19.541380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc9be4-50d0-4e26-afea-eff3b7052da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:19.542212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc9be4-50d0-4e26-afea-eff3b7052da7 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:19.542497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc9be4-50d0-4e26-afea-eff3b7052da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:19.542832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc9be4-50d0-4e26-afea-eff3b7052da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:19.554816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc9be4-50d0-4e26-afea-eff3b7052da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:19.556573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc9be4-50d0-4e26-afea-eff3b7052da7 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:19.570905 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2256/4488] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:17:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:17:21.794302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c2a8e-972c-42eb-b7f7-5a8042ac51bc None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:21.794839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c2a8e-972c-42eb-b7f7-5a8042ac51bc None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:21.794839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c2a8e-972c-42eb-b7f7-5a8042ac51bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:21.823982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-981c2a8e-972c-42eb-b7f7-5a8042ac51bc None None] MFA Rules not processed for user `a952d7989d3945b6a3ed38c2e444a7ba`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:21.825302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-981c2a8e-972c-42eb-b7f7-5a8042ac51bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:21.921307 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2233/4489] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:21.932827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16549e0e-b278-4786-9d60-7ed33efdbb9c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:21.933367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16549e0e-b278-4786-9d60-7ed33efdbb9c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:21.937397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16549e0e-b278-4786-9d60-7ed33efdbb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:21.959116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-16549e0e-b278-4786-9d60-7ed33efdbb9c None None] MFA Rules not processed for user `a952d7989d3945b6a3ed38c2e444a7ba`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:21.961319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-16549e0e-b278-4786-9d60-7ed33efdbb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:21.974603 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2257/4490] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:21.985810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c2a8e-972c-42eb-b7f7-5a8042ac51bc None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:21.989508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-981c2a8e-972c-42eb-b7f7-5a8042ac51bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:22.120680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c2a8e-972c-42eb-b7f7-5a8042ac51bc None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:22.127842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f7003-5bc2-4e62-a7ee-afdece859385 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:22.128728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f7003-5bc2-4e62-a7ee-afdece859385 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:22.129202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f7003-5bc2-4e62-a7ee-afdece859385 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:22.129661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f7003-5bc2-4e62-a7ee-afdece859385 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:22.142980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f7003-5bc2-4e62-a7ee-afdece859385 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:22.146649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f7003-5bc2-4e62-a7ee-afdece859385 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:22.158476 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2234/4491] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:17:21 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:17:31.947507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:32.153881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=71 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:32.516427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16549e0e-b278-4786-9d60-7ed33efdbb9c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:32.518053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-16549e0e-b278-4786-9d60-7ed33efdbb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:32.540576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16549e0e-b278-4786-9d60-7ed33efdbb9c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:32.550735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47804961-1797-434a-a3d7-f22eac593f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:32.550735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47804961-1797-434a-a3d7-f22eac593f6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:32.550735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47804961-1797-434a-a3d7-f22eac593f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:32.550735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47804961-1797-434a-a3d7-f22eac593f6f admin admin] PATH_INFO: `/v3/users/abab220deb8943f0b9c7e5559d8e4440` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:32.551266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47804961-1797-434a-a3d7-f22eac593f6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=abab220deb8943f0b9c7e5559d8e4440)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:32.552047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47804961-1797-434a-a3d7-f22eac593f6f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:32.801489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47804961-1797-434a-a3d7-f22eac593f6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abab220deb8943f0b9c7e5559d8e4440'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:32.811497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47804961-1797-434a-a3d7-f22eac593f6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abab220deb8943f0b9c7e5559d8e4440'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:32.812232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47804961-1797-434a-a3d7-f22eac593f6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abab220deb8943f0b9c7e5559d8e4440'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:32.813522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47804961-1797-434a-a3d7-f22eac593f6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abab220deb8943f0b9c7e5559d8e4440'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:32.903241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47804961-1797-434a-a3d7-f22eac593f6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abab220deb8943f0b9c7e5559d8e4440'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:32.941821 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2258/4492] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:17:32 2022] DELETE /identity/v3/users/abab220deb8943f0b9c7e5559d8e4440 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:32.943950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41a39a-dc35-4d66-96d3-25bbce4e3ece None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:32.944332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41a39a-dc35-4d66-96d3-25bbce4e3ece None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:32.944937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41a39a-dc35-4d66-96d3-25bbce4e3ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:32.964884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9e41a39a-dc35-4d66-96d3-25bbce4e3ece None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:32.966381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9e41a39a-dc35-4d66-96d3-25bbce4e3ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:33.027718 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2235/4493] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:33.038070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47804961-1797-434a-a3d7-f22eac593f6f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:33.039645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-47804961-1797-434a-a3d7-f22eac593f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:33.107691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47804961-1797-434a-a3d7-f22eac593f6f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:33.111640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebcdf24-f4ab-4ba8-95e3-d4a6030b64a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:33.112366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcdf24-f4ab-4ba8-95e3-d4a6030b64a9 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:33.112864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcdf24-f4ab-4ba8-95e3-d4a6030b64a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:33.113687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcdf24-f4ab-4ba8-95e3-d4a6030b64a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:33.123387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebcdf24-f4ab-4ba8-95e3-d4a6030b64a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:33.125064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebcdf24-f4ab-4ba8-95e3-d4a6030b64a9 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:33.135762 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2259/4494] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:17:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:17:33.662628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41a39a-dc35-4d66-96d3-25bbce4e3ece None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:33.665195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9e41a39a-dc35-4d66-96d3-25bbce4e3ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:33.678675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41a39a-dc35-4d66-96d3-25bbce4e3ece None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:33.684119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaef8ad-6aa0-4bac-9196-46ccc7abdb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:33.685275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaef8ad-6aa0-4bac-9196-46ccc7abdb5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:33.685889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaef8ad-6aa0-4bac-9196-46ccc7abdb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:33.686113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaef8ad-6aa0-4bac-9196-46ccc7abdb5b admin admin] PATH_INFO: `/v3/projects/161b7adf0b4a4d489d11c62ad0f03a3f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:33.687822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaef8ad-6aa0-4bac-9196-46ccc7abdb5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=161b7adf0b4a4d489d11c62ad0f03a3f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:33.690061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaef8ad-6aa0-4bac-9196-46ccc7abdb5b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:33.814970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ecaef8ad-6aa0-4bac-9196-46ccc7abdb5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '161b7adf0b4a4d489d11c62ad0f03a3f', 'request_id': 'req-ecaef8ad-6aa0-4bac-9196-46ccc7abdb5b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:33.822273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ecaef8ad-6aa0-4bac-9196-46ccc7abdb5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '161b7adf0b4a4d489d11c62ad0f03a3f', 'request_id': 'req-ecaef8ad-6aa0-4bac-9196-46ccc7abdb5b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:33.912539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ecaef8ad-6aa0-4bac-9196-46ccc7abdb5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '161b7adf0b4a4d489d11c62ad0f03a3f', 'request_id': 'req-ecaef8ad-6aa0-4bac-9196-46ccc7abdb5b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:33.924768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ecaef8ad-6aa0-4bac-9196-46ccc7abdb5b admin admin] The token cache is being invalidate because project 161b7adf0b4a4d489d11c62ad0f03a3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:17:33.925148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ecaef8ad-6aa0-4bac-9196-46ccc7abdb5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:33.926021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ecaef8ad-6aa0-4bac-9196-46ccc7abdb5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:33.927949 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2236/4495] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:17:33 2022] DELETE /identity/v3/projects/161b7adf0b4a4d489d11c62ad0f03a3f => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:33.932536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46381c5c-9841-48c9-aab1-052fb9b9861b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:33.932837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46381c5c-9841-48c9-aab1-052fb9b9861b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:33.933125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46381c5c-9841-48c9-aab1-052fb9b9861b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:33.953230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-46381c5c-9841-48c9-aab1-052fb9b9861b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:33.954737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-46381c5c-9841-48c9-aab1-052fb9b9861b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:34.030643 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2260/4496] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:17:34.039190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95431709-a558-4100-ab59-5c5b6c210f31 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:34.039190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95431709-a558-4100-ab59-5c5b6c210f31 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:34.039190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95431709-a558-4100-ab59-5c5b6c210f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:34.071584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-95431709-a558-4100-ab59-5c5b6c210f31 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:34.071584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-95431709-a558-4100-ab59-5c5b6c210f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:34.080715 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2237/4497] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:34.084863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46381c5c-9841-48c9-aab1-052fb9b9861b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:34.095302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46381c5c-9841-48c9-aab1-052fb9b9861b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:34.100721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43298b5d-b524-4726-8ca6-aa0647f56b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:34.101392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43298b5d-b524-4726-8ca6-aa0647f56b0d admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:34.101679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43298b5d-b524-4726-8ca6-aa0647f56b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:34.101997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43298b5d-b524-4726-8ca6-aa0647f56b0d admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:34.102741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43298b5d-b524-4726-8ca6-aa0647f56b0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:17:34.103053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43298b5d-b524-4726-8ca6-aa0647f56b0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:34.104759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43298b5d-b524-4726-8ca6-aa0647f56b0d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:34.120857 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2261/4498] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:17:34.121950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95431709-a558-4100-ab59-5c5b6c210f31 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:34.131510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95431709-a558-4100-ab59-5c5b6c210f31 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:34.137090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159cd0a-4776-40f0-bbe7-35b19b988839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:34.137747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159cd0a-4776-40f0-bbe7-35b19b988839 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:34.138212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159cd0a-4776-40f0-bbe7-35b19b988839 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:34.138466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159cd0a-4776-40f0-bbe7-35b19b988839 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:34.144975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159cd0a-4776-40f0-bbe7-35b19b988839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:34.144975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159cd0a-4776-40f0-bbe7-35b19b988839 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:34.257572 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2238/4499] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:17:34 2022] POST /identity/v3/projects => generated 361 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:17:34.261705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43298b5d-b524-4726-8ca6-aa0647f56b0d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:34.267116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:34.273034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43298b5d-b524-4726-8ca6-aa0647f56b0d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:34.333558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6acea-e4c4-48d5-981b-6b60a565706c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:34.334270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6acea-e4c4-48d5-981b-6b60a565706c admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:34.334554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6acea-e4c4-48d5-981b-6b60a565706c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:34.334830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa6acea-e4c4-48d5-981b-6b60a565706c admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:34.335532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6acea-e4c4-48d5-981b-6b60a565706c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:34.337456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa6acea-e4c4-48d5-981b-6b60a565706c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:34.473506 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2262/4500] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:17:34 2022] POST /identity/v3/users => generated 281 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:17:34.477541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159cd0a-4776-40f0-bbe7-35b19b988839 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:34.488887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159cd0a-4776-40f0-bbe7-35b19b988839 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:34.493644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1443cd-9c15-47e4-a7c6-bfd0efff88a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:34.494347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1443cd-9c15-47e4-a7c6-bfd0efff88a3 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:34.494637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1443cd-9c15-47e4-a7c6-bfd0efff88a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:34.494916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1443cd-9c15-47e4-a7c6-bfd0efff88a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:34.495720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1443cd-9c15-47e4-a7c6-bfd0efff88a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:34.497369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1443cd-9c15-47e4-a7c6-bfd0efff88a3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:34.508915 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2239/4501] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:17:34.513196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6acea-e4c4-48d5-981b-6b60a565706c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:34.525871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa6acea-e4c4-48d5-981b-6b60a565706c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:34.534050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e2452-f4c2-4e85-8254-7f798d05dbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:34.534754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e2452-f4c2-4e85-8254-7f798d05dbb3 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:34.535064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e2452-f4c2-4e85-8254-7f798d05dbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:34.535408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e2452-f4c2-4e85-8254-7f798d05dbb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:34.536141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e2452-f4c2-4e85-8254-7f798d05dbb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:34.537770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e2452-f4c2-4e85-8254-7f798d05dbb3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:34.553752 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2263/4502] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:17:34.554427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1443cd-9c15-47e4-a7c6-bfd0efff88a3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:34.571988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1443cd-9c15-47e4-a7c6-bfd0efff88a3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:34.577644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1ed91-9dc5-4ba4-8820-c0140c4818ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:34.585247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1ed91-9dc5-4ba4-8820-c0140c4818ee admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:34.585641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1ed91-9dc5-4ba4-8820-c0140c4818ee admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:34.591036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1ed91-9dc5-4ba4-8820-c0140c4818ee admin admin] PATH_INFO: `/v3/projects/eaee064440244f0da817830ab2d06f0a/users/631aa933ecb444c586481eac51f29677/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:34.616761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1ed91-9dc5-4ba4-8820-c0140c4818ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaee064440244f0da817830ab2d06f0a, user_id=631aa933ecb444c586481eac51f29677, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:34.621189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1ed91-9dc5-4ba4-8820-c0140c4818ee admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:34.718964 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2240/4503] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:17:34 2022] PUT /identity/v3/projects/eaee064440244f0da817830ab2d06f0a/users/631aa933ecb444c586481eac51f29677/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:34.730325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbab836-3cf6-4406-8d64-72c4ed07d77d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:34.732811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbab836-3cf6-4406-8d64-72c4ed07d77d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:34.733218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbab836-3cf6-4406-8d64-72c4ed07d77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:34.769623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8fbab836-3cf6-4406-8d64-72c4ed07d77d None None] MFA Rules not processed for user `631aa933ecb444c586481eac51f29677`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:34.770988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8fbab836-3cf6-4406-8d64-72c4ed07d77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:34.912113 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2264/4504] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:34.939393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1ed91-9dc5-4ba4-8820-c0140c4818ee admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:34.942603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-15f1ed91-9dc5-4ba4-8820-c0140c4818ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:34.977864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:35.046444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1ed91-9dc5-4ba4-8820-c0140c4818ee admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:35.052226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5931c1-774e-41cd-8556-813e08366d31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:35.052905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5931c1-774e-41cd-8556-813e08366d31 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:35.053194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5931c1-774e-41cd-8556-813e08366d31 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:35.053957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5931c1-774e-41cd-8556-813e08366d31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:35.066414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5931c1-774e-41cd-8556-813e08366d31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:35.068089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5931c1-774e-41cd-8556-813e08366d31 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:35.088524 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2241/4505] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:17:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:17:35.535941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069573f1-58d8-421a-bf6e-dd5879704448 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:35.536461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069573f1-58d8-421a-bf6e-dd5879704448 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:35.536559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069573f1-58d8-421a-bf6e-dd5879704448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:35.554734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-069573f1-58d8-421a-bf6e-dd5879704448 None None] MFA Rules not processed for user `af446b3d802a42e3bd72b08bace8ea8d`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:35.556006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-069573f1-58d8-421a-bf6e-dd5879704448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:35.623212 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2265/4506] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:44.537490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5931c1-774e-41cd-8556-813e08366d31 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:44.539623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4d5931c1-774e-41cd-8556-813e08366d31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:44.671223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5931c1-774e-41cd-8556-813e08366d31 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:44.676346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022359f8-692e-4a37-befe-c7caed9e9f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:44.679408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022359f8-692e-4a37-befe-c7caed9e9f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:44.679408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022359f8-692e-4a37-befe-c7caed9e9f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:44.679408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022359f8-692e-4a37-befe-c7caed9e9f64 admin admin] PATH_INFO: `/v3/users/a952d7989d3945b6a3ed38c2e444a7ba` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:44.679408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022359f8-692e-4a37-befe-c7caed9e9f64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a952d7989d3945b6a3ed38c2e444a7ba)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:44.681523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022359f8-692e-4a37-befe-c7caed9e9f64 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:44.925207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-022359f8-692e-4a37-befe-c7caed9e9f64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a952d7989d3945b6a3ed38c2e444a7ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:44.926603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-022359f8-692e-4a37-befe-c7caed9e9f64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a952d7989d3945b6a3ed38c2e444a7ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:44.935573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-022359f8-692e-4a37-befe-c7caed9e9f64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a952d7989d3945b6a3ed38c2e444a7ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:44.986478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-022359f8-692e-4a37-befe-c7caed9e9f64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a952d7989d3945b6a3ed38c2e444a7ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:44.987114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-022359f8-692e-4a37-befe-c7caed9e9f64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a952d7989d3945b6a3ed38c2e444a7ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:45.011787 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2242/4507] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:17:44 2022] DELETE /identity/v3/users/a952d7989d3945b6a3ed38c2e444a7ba => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:45.442750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069573f1-58d8-421a-bf6e-dd5879704448 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:45.456421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069573f1-58d8-421a-bf6e-dd5879704448 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:45.499927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:45.641705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02656f-2f4d-492c-b124-788cc30fd0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:45.643050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02656f-2f4d-492c-b124-788cc30fd0da admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:45.643583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02656f-2f4d-492c-b124-788cc30fd0da admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:45.644147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02656f-2f4d-492c-b124-788cc30fd0da admin admin] PATH_INFO: `/v3/projects/c9dd563271c04f6dae47402935636bb8` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:45.646616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02656f-2f4d-492c-b124-788cc30fd0da admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9dd563271c04f6dae47402935636bb8)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:45.650583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02656f-2f4d-492c-b124-788cc30fd0da admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:45.817174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cd02656f-2f4d-492c-b124-788cc30fd0da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9dd563271c04f6dae47402935636bb8', 'request_id': 'req-cd02656f-2f4d-492c-b124-788cc30fd0da'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:45.849815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cd02656f-2f4d-492c-b124-788cc30fd0da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9dd563271c04f6dae47402935636bb8', 'request_id': 'req-cd02656f-2f4d-492c-b124-788cc30fd0da'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:45.928778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cd02656f-2f4d-492c-b124-788cc30fd0da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9dd563271c04f6dae47402935636bb8', 'request_id': 'req-cd02656f-2f4d-492c-b124-788cc30fd0da'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:45.944820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cd02656f-2f4d-492c-b124-788cc30fd0da admin admin] The token cache is being invalidate because project c9dd563271c04f6dae47402935636bb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:17:45.945117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cd02656f-2f4d-492c-b124-788cc30fd0da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:45.952156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cd02656f-2f4d-492c-b124-788cc30fd0da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:45.952156 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2266/4508] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:17:45 2022] DELETE /identity/v3/projects/c9dd563271c04f6dae47402935636bb8 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:45.953654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4cd64-6a00-458a-a2fc-00ebaf9e577e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:45.954179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4cd64-6a00-458a-a2fc-00ebaf9e577e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:45.954469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b4cd64-6a00-458a-a2fc-00ebaf9e577e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:45.992627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-95b4cd64-6a00-458a-a2fc-00ebaf9e577e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:45.997022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-95b4cd64-6a00-458a-a2fc-00ebaf9e577e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:46.103677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:46.198500 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2243/4509] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:46.206446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f298949-7bdb-4e7d-9d7f-a6ae877af64c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:46.206823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f298949-7bdb-4e7d-9d7f-a6ae877af64c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:46.207404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f298949-7bdb-4e7d-9d7f-a6ae877af64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:46.240692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0f298949-7bdb-4e7d-9d7f-a6ae877af64c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:46.243638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0f298949-7bdb-4e7d-9d7f-a6ae877af64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:46.256813 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2267/4510] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:46.258323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4cd64-6a00-458a-a2fc-00ebaf9e577e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:46.272404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b4cd64-6a00-458a-a2fc-00ebaf9e577e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:46.278601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a5a13-67b3-4c89-8bda-64831dae0436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:46.279587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a5a13-67b3-4c89-8bda-64831dae0436 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:46.280117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a5a13-67b3-4c89-8bda-64831dae0436 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:46.280548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a5a13-67b3-4c89-8bda-64831dae0436 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:46.282199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a5a13-67b3-4c89-8bda-64831dae0436 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:17:46.282521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a5a13-67b3-4c89-8bda-64831dae0436 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:46.284198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a5a13-67b3-4c89-8bda-64831dae0436 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:46.306622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f298949-7bdb-4e7d-9d7f-a6ae877af64c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:46.314114 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2244/4511] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:17:46.321340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f298949-7bdb-4e7d-9d7f-a6ae877af64c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:46.351828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85bfa4-29e4-4650-b9b2-5c97d0a8e1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:46.351828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85bfa4-29e4-4650-b9b2-5c97d0a8e1da admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:46.351828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85bfa4-29e4-4650-b9b2-5c97d0a8e1da admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:46.352091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85bfa4-29e4-4650-b9b2-5c97d0a8e1da admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:46.355396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85bfa4-29e4-4650-b9b2-5c97d0a8e1da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:46.361115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85bfa4-29e4-4650-b9b2-5c97d0a8e1da admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:46.471967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a5a13-67b3-4c89-8bda-64831dae0436 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:46.478342 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2268/4512] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:17:46 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:17:46.486319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a5a13-67b3-4c89-8bda-64831dae0436 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:46.586377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2cb5f-8a9a-41a6-8fce-8e53c5adb487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:46.587027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2cb5f-8a9a-41a6-8fce-8e53c5adb487 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:46.587561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2cb5f-8a9a-41a6-8fce-8e53c5adb487 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:46.588366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f2cb5f-8a9a-41a6-8fce-8e53c5adb487 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:46.590393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2cb5f-8a9a-41a6-8fce-8e53c5adb487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:46.593177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f2cb5f-8a9a-41a6-8fce-8e53c5adb487 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:46.703421 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2245/4513] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:17:46 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:17:46.707766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85bfa4-29e4-4650-b9b2-5c97d0a8e1da admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:46.720729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85bfa4-29e4-4650-b9b2-5c97d0a8e1da admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:46.728591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4822bc4-3ddd-4b06-b26a-27e216c18a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:46.729588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4822bc4-3ddd-4b06-b26a-27e216c18a90 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:46.730102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4822bc4-3ddd-4b06-b26a-27e216c18a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:46.730564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4822bc4-3ddd-4b06-b26a-27e216c18a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:46.731745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4822bc4-3ddd-4b06-b26a-27e216c18a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:46.734567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4822bc4-3ddd-4b06-b26a-27e216c18a90 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:46.762668 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2269/4514] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:17:46.766810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2cb5f-8a9a-41a6-8fce-8e53c5adb487 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:46.805426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f2cb5f-8a9a-41a6-8fce-8e53c5adb487 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:46.824879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99487b37-ba95-4723-b5a1-353532f1b237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:46.825977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99487b37-ba95-4723-b5a1-353532f1b237 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:46.830610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99487b37-ba95-4723-b5a1-353532f1b237 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:46.830610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99487b37-ba95-4723-b5a1-353532f1b237 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:46.830610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99487b37-ba95-4723-b5a1-353532f1b237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:46.832848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99487b37-ba95-4723-b5a1-353532f1b237 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:46.867304 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2246/4515] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:17:46.871170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4822bc4-3ddd-4b06-b26a-27e216c18a90 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:46.886562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4822bc4-3ddd-4b06-b26a-27e216c18a90 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:46.905630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4c2ba-a9f3-4e1a-b345-9ad792ec61d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:46.906589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4c2ba-a9f3-4e1a-b345-9ad792ec61d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:46.907501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4c2ba-a9f3-4e1a-b345-9ad792ec61d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:46.907786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4c2ba-a9f3-4e1a-b345-9ad792ec61d6 admin admin] PATH_INFO: `/v3/projects/12b9ffa69df641558691a33ba072f78d/users/a01fd3c4a0184b41864b9184c507a8ff/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:46.936688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4c2ba-a9f3-4e1a-b345-9ad792ec61d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b9ffa69df641558691a33ba072f78d, user_id=a01fd3c4a0184b41864b9184c507a8ff, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:46.943042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4c2ba-a9f3-4e1a-b345-9ad792ec61d6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:47.032903 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2270/4516] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:17:46 2022] PUT /identity/v3/projects/12b9ffa69df641558691a33ba072f78d/users/a01fd3c4a0184b41864b9184c507a8ff/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:47.038871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a6790-9fa1-4627-99a7-6e1d2db6e952 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:47.039666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a6790-9fa1-4627-99a7-6e1d2db6e952 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:47.039948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a6790-9fa1-4627-99a7-6e1d2db6e952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:47.069656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-cc6a6790-9fa1-4627-99a7-6e1d2db6e952 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:47.071119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cc6a6790-9fa1-4627-99a7-6e1d2db6e952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:47.164668 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2247/4517] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:47.173877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4c2ba-a9f3-4e1a-b345-9ad792ec61d6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:47.178546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8bb4c2ba-a9f3-4e1a-b345-9ad792ec61d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:47.322656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:47.341440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4c2ba-a9f3-4e1a-b345-9ad792ec61d6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:47.348335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb975c2-da09-4dab-9ed1-4e3d4fab137e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:47.348601 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb975c2-da09-4dab-9ed1-4e3d4fab137e service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:47.349201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb975c2-da09-4dab-9ed1-4e3d4fab137e service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:47.349595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb975c2-da09-4dab-9ed1-4e3d4fab137e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:47.367682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb975c2-da09-4dab-9ed1-4e3d4fab137e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:47.368694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb975c2-da09-4dab-9ed1-4e3d4fab137e service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:47.392327 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2271/4518] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:17:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:17:49.677511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a6790-9fa1-4627-99a7-6e1d2db6e952 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:49.679784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cc6a6790-9fa1-4627-99a7-6e1d2db6e952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:49.697680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a6790-9fa1-4627-99a7-6e1d2db6e952 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:49.704798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553cca75-4a9b-402d-a493-e13cd706b160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:49.706155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cca75-4a9b-402d-a493-e13cd706b160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:49.706673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cca75-4a9b-402d-a493-e13cd706b160 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:49.707283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553cca75-4a9b-402d-a493-e13cd706b160 admin admin] PATH_INFO: `/v3/users/572170ccd67e4facbf10dad7cd1491c1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:49.708747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553cca75-4a9b-402d-a493-e13cd706b160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=572170ccd67e4facbf10dad7cd1491c1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:49.711829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553cca75-4a9b-402d-a493-e13cd706b160 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:50.005942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:50.026279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-553cca75-4a9b-402d-a493-e13cd706b160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '572170ccd67e4facbf10dad7cd1491c1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:50.027263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-553cca75-4a9b-402d-a493-e13cd706b160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '572170ccd67e4facbf10dad7cd1491c1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:50.036658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-553cca75-4a9b-402d-a493-e13cd706b160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '572170ccd67e4facbf10dad7cd1491c1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:50.109058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-553cca75-4a9b-402d-a493-e13cd706b160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '572170ccd67e4facbf10dad7cd1491c1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:50.109766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-553cca75-4a9b-402d-a493-e13cd706b160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '572170ccd67e4facbf10dad7cd1491c1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:50.130075 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2248/4519] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:17:49 2022] DELETE /identity/v3/users/572170ccd67e4facbf10dad7cd1491c1 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:50.411565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6249903-768b-4947-9ddd-bb4732f94481 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:50.412484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6249903-768b-4947-9ddd-bb4732f94481 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:50.413952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6249903-768b-4947-9ddd-bb4732f94481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:50.454332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c6249903-768b-4947-9ddd-bb4732f94481 None None] MFA Rules not processed for user `a01fd3c4a0184b41864b9184c507a8ff`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:50.461658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c6249903-768b-4947-9ddd-bb4732f94481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:50.461985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553cca75-4a9b-402d-a493-e13cd706b160 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:50.461985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-553cca75-4a9b-402d-a493-e13cd706b160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:50.562125 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2272/4520] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:50.564546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b782a-cdae-4257-a81b-a3be404df592 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:50.565038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b782a-cdae-4257-a81b-a3be404df592 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:50.567265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b782a-cdae-4257-a81b-a3be404df592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:50.585162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-051b782a-cdae-4257-a81b-a3be404df592 None None] MFA Rules not processed for user `a01fd3c4a0184b41864b9184c507a8ff`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:50.586756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-051b782a-cdae-4257-a81b-a3be404df592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:50.598147 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2273/4521] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:17:50.601918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553cca75-4a9b-402d-a493-e13cd706b160 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:50.611021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a674f-4d2f-4263-8d04-ac4e544fdd6d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:50.611313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a674f-4d2f-4263-8d04-ac4e544fdd6d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:50.611844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a674f-4d2f-4263-8d04-ac4e544fdd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:50.612916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676aa931-9838-43d4-99b2-ee39e5ac1f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:50.614147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676aa931-9838-43d4-99b2-ee39e5ac1f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:50.614440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676aa931-9838-43d4-99b2-ee39e5ac1f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:50.614722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676aa931-9838-43d4-99b2-ee39e5ac1f86 admin admin] PATH_INFO: `/v3/projects/5905c4b42ecd40298340a8a2ce4d5e24` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:50.618923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676aa931-9838-43d4-99b2-ee39e5ac1f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5905c4b42ecd40298340a8a2ce4d5e24)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:50.620908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676aa931-9838-43d4-99b2-ee39e5ac1f86 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:50.645657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-181a674f-4d2f-4263-8d04-ac4e544fdd6d None None] MFA Rules not processed for user `a01fd3c4a0184b41864b9184c507a8ff`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:50.649505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-181a674f-4d2f-4263-8d04-ac4e544fdd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:50.662098 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2274/4522] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:50.674391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a674f-4d2f-4263-8d04-ac4e544fdd6d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:50.675984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-181a674f-4d2f-4263-8d04-ac4e544fdd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:50.781595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-676aa931-9838-43d4-99b2-ee39e5ac1f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5905c4b42ecd40298340a8a2ce4d5e24', 'request_id': 'req-676aa931-9838-43d4-99b2-ee39e5ac1f86'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:50.791717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-676aa931-9838-43d4-99b2-ee39e5ac1f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5905c4b42ecd40298340a8a2ce4d5e24', 'request_id': 'req-676aa931-9838-43d4-99b2-ee39e5ac1f86'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:50.812270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a674f-4d2f-4263-8d04-ac4e544fdd6d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:50.872248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:50.896173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f73652d-b4a1-4de9-ae26-aedc21465010 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:50.896466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-676aa931-9838-43d4-99b2-ee39e5ac1f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5905c4b42ecd40298340a8a2ce4d5e24', 'request_id': 'req-676aa931-9838-43d4-99b2-ee39e5ac1f86'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:50.897471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73652d-b4a1-4de9-ae26-aedc21465010 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:50.898133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73652d-b4a1-4de9-ae26-aedc21465010 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:50.898566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f73652d-b4a1-4de9-ae26-aedc21465010 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:50.914360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-676aa931-9838-43d4-99b2-ee39e5ac1f86 admin admin] The token cache is being invalidate because project 5905c4b42ecd40298340a8a2ce4d5e24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:17:50.915057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-676aa931-9838-43d4-99b2-ee39e5ac1f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:50.918228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-676aa931-9838-43d4-99b2-ee39e5ac1f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:50.922609 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2249/4523] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:17:50 2022] DELETE /identity/v3/projects/5905c4b42ecd40298340a8a2ce4d5e24 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:50.927386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fdc3f-8942-45fc-a9e3-64017971bd40 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:50.927681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fdc3f-8942-45fc-a9e3-64017971bd40 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:50.927970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1fdc3f-8942-45fc-a9e3-64017971bd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:50.972230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-af1fdc3f-8942-45fc-a9e3-64017971bd40 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:50.973532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-af1fdc3f-8942-45fc-a9e3-64017971bd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:50.997862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f73652d-b4a1-4de9-ae26-aedc21465010 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:50.999612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f73652d-b4a1-4de9-ae26-aedc21465010 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:51.023160 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2275/4524] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:17:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:17:51.099102 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2250/4525] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:17:51.107905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133618c-b884-4317-a3b3-3602d508e12c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:51.108240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133618c-b884-4317-a3b3-3602d508e12c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:51.108538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133618c-b884-4317-a3b3-3602d508e12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:51.152280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c133618c-b884-4317-a3b3-3602d508e12c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:51.154190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c133618c-b884-4317-a3b3-3602d508e12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:51.158255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:51.171447 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2276/4526] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:51.175726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fdc3f-8942-45fc-a9e3-64017971bd40 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:51.189867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1fdc3f-8942-45fc-a9e3-64017971bd40 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:51.196217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2174a7-fb7e-4bff-8c20-106db6d72e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:51.196980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2174a7-fb7e-4bff-8c20-106db6d72e96 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:51.197764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2174a7-fb7e-4bff-8c20-106db6d72e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:51.198228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2174a7-fb7e-4bff-8c20-106db6d72e96 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:51.199186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2174a7-fb7e-4bff-8c20-106db6d72e96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:17:51.199572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2174a7-fb7e-4bff-8c20-106db6d72e96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:51.204048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2174a7-fb7e-4bff-8c20-106db6d72e96 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:51.222966 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2251/4527] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:17:51.227245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133618c-b884-4317-a3b3-3602d508e12c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:51.240273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c133618c-b884-4317-a3b3-3602d508e12c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:51.245527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe94300-578f-4346-94e0-5e30e4398387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:51.246414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe94300-578f-4346-94e0-5e30e4398387 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:51.246778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe94300-578f-4346-94e0-5e30e4398387 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:51.247170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe94300-578f-4346-94e0-5e30e4398387 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:51.247983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe94300-578f-4346-94e0-5e30e4398387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:51.250218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe94300-578f-4346-94e0-5e30e4398387 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:51.372791 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2277/4528] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:17:51 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:17:51.377175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2174a7-fb7e-4bff-8c20-106db6d72e96 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:51.390490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2174a7-fb7e-4bff-8c20-106db6d72e96 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:51.466871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e61c79-ee40-4e35-8ccf-87205b402949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:51.467661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e61c79-ee40-4e35-8ccf-87205b402949 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:51.468180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e61c79-ee40-4e35-8ccf-87205b402949 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:51.468556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e61c79-ee40-4e35-8ccf-87205b402949 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:51.469335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e61c79-ee40-4e35-8ccf-87205b402949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:51.471659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e61c79-ee40-4e35-8ccf-87205b402949 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:51.601517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe94300-578f-4346-94e0-5e30e4398387 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:51.604457 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2252/4529] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:17:51 2022] POST /identity/v3/users => generated 294 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:17:51.617779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe94300-578f-4346-94e0-5e30e4398387 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:51.633272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a468ca6-8920-42cc-87b5-01c6fbc0eccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:51.633272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a468ca6-8920-42cc-87b5-01c6fbc0eccd admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:51.633272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a468ca6-8920-42cc-87b5-01c6fbc0eccd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:51.633272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a468ca6-8920-42cc-87b5-01c6fbc0eccd admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:51.634044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a468ca6-8920-42cc-87b5-01c6fbc0eccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:51.636453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a468ca6-8920-42cc-87b5-01c6fbc0eccd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:51.658798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e61c79-ee40-4e35-8ccf-87205b402949 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:51.659692 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2278/4530] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:17:51.671671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e61c79-ee40-4e35-8ccf-87205b402949 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:51.689546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64d807-cb7b-46f6-8366-ac12a2f41c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:51.690744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64d807-cb7b-46f6-8366-ac12a2f41c34 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:51.692413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64d807-cb7b-46f6-8366-ac12a2f41c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:51.692714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64d807-cb7b-46f6-8366-ac12a2f41c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:51.693486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64d807-cb7b-46f6-8366-ac12a2f41c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:51.712204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64d807-cb7b-46f6-8366-ac12a2f41c34 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:51.725889 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2253/4531] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:17:51.730225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a468ca6-8920-42cc-87b5-01c6fbc0eccd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:51.741517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a468ca6-8920-42cc-87b5-01c6fbc0eccd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:51.746938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddc037-c99d-462a-bdda-feca196a5929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:51.747867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddc037-c99d-462a-bdda-feca196a5929 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:51.748149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddc037-c99d-462a-bdda-feca196a5929 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:51.748434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddc037-c99d-462a-bdda-feca196a5929 admin admin] PATH_INFO: `/v3/projects/4994e3fdabb2482396d3da7fe0fa94ff/users/4d0dc3f3b97c4ac88a89896c382f8b03/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:51.766882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cddc037-c99d-462a-bdda-feca196a5929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4994e3fdabb2482396d3da7fe0fa94ff, user_id=4d0dc3f3b97c4ac88a89896c382f8b03, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:51.770189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cddc037-c99d-462a-bdda-feca196a5929 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:51.892355 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2279/4532] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:17:51 2022] PUT /identity/v3/projects/4994e3fdabb2482396d3da7fe0fa94ff/users/4d0dc3f3b97c4ac88a89896c382f8b03/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:51.897539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac04819-9d45-4af8-affc-c444f7f9e3bb None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:51.898148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac04819-9d45-4af8-affc-c444f7f9e3bb None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:51.898148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac04819-9d45-4af8-affc-c444f7f9e3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:51.923828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4ac04819-9d45-4af8-affc-c444f7f9e3bb None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:51.927194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4ac04819-9d45-4af8-affc-c444f7f9e3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:52.014312 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2254/4533] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:17:52.023884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddc037-c99d-462a-bdda-feca196a5929 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:52.026937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0cddc037-c99d-462a-bdda-feca196a5929 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:52.162152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddc037-c99d-462a-bdda-feca196a5929 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:52.175875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c084e1b-770c-423b-8df8-c788dd5ee43f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:52.176747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c084e1b-770c-423b-8df8-c788dd5ee43f service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:52.177086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c084e1b-770c-423b-8df8-c788dd5ee43f service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:52.177453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c084e1b-770c-423b-8df8-c788dd5ee43f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:52.188782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c084e1b-770c-423b-8df8-c788dd5ee43f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:52.190399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c084e1b-770c-423b-8df8-c788dd5ee43f service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:52.201316 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2280/4534] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:17:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:17:52.238759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462ed8d-4899-49e7-b7d6-c2cb0bd58a3e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:52.249291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462ed8d-4899-49e7-b7d6-c2cb0bd58a3e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:52.251340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462ed8d-4899-49e7-b7d6-c2cb0bd58a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:52.283233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d462ed8d-4899-49e7-b7d6-c2cb0bd58a3e None None] MFA Rules not processed for user `a01fd3c4a0184b41864b9184c507a8ff`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:52.285222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d462ed8d-4899-49e7-b7d6-c2cb0bd58a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:52.359205 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2255/4535] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:52.808796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:53.271194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd85113-5707-4145-ae71-5bfd44c96d0d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:53.272095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd85113-5707-4145-ae71-5bfd44c96d0d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:53.273296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd85113-5707-4145-ae71-5bfd44c96d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:53.319554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-dcd85113-5707-4145-ae71-5bfd44c96d0d None None] MFA Rules not processed for user `a01fd3c4a0184b41864b9184c507a8ff`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:53.321292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dcd85113-5707-4145-ae71-5bfd44c96d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:53.331905 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2281/4536] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:54.387733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966c37b-ba33-4c10-85be-30ede94aa7b4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:54.388693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966c37b-ba33-4c10-85be-30ede94aa7b4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:54.389227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1966c37b-ba33-4c10-85be-30ede94aa7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:54.456885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1966c37b-ba33-4c10-85be-30ede94aa7b4 None None] MFA Rules not processed for user `4d0dc3f3b97c4ac88a89896c382f8b03`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:54.458435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1966c37b-ba33-4c10-85be-30ede94aa7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:54.565124 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2256/4537] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:56.088800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd85113-5707-4145-ae71-5bfd44c96d0d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:56.103072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd85113-5707-4145-ae71-5bfd44c96d0d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:56.119143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0b47d-c207-4720-b001-f206841bee1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:56.121132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0b47d-c207-4720-b001-f206841bee1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:56.121632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0b47d-c207-4720-b001-f206841bee1a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:56.122233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0b47d-c207-4720-b001-f206841bee1a admin admin] PATH_INFO: `/v3/users/4d0dc3f3b97c4ac88a89896c382f8b03` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:56.123788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0b47d-c207-4720-b001-f206841bee1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d0dc3f3b97c4ac88a89896c382f8b03)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:56.125764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0b47d-c207-4720-b001-f206841bee1a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:56.180427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:56.362001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-def0b47d-c207-4720-b001-f206841bee1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d0dc3f3b97c4ac88a89896c382f8b03'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:56.378952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-def0b47d-c207-4720-b001-f206841bee1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d0dc3f3b97c4ac88a89896c382f8b03'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:56.380544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-def0b47d-c207-4720-b001-f206841bee1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d0dc3f3b97c4ac88a89896c382f8b03'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:56.381241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-def0b47d-c207-4720-b001-f206841bee1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d0dc3f3b97c4ac88a89896c382f8b03'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:56.461011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-def0b47d-c207-4720-b001-f206841bee1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d0dc3f3b97c4ac88a89896c382f8b03'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:56.490246 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2282/4538] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:17:56 2022] DELETE /identity/v3/users/4d0dc3f3b97c4ac88a89896c382f8b03 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:57.061568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966c37b-ba33-4c10-85be-30ede94aa7b4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:57.088464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1966c37b-ba33-4c10-85be-30ede94aa7b4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:57.175553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f466366-c5ff-4a48-aa1c-50157cfa2e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:57.176590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f466366-c5ff-4a48-aa1c-50157cfa2e74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:57.177436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f466366-c5ff-4a48-aa1c-50157cfa2e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:57.177820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f466366-c5ff-4a48-aa1c-50157cfa2e74 admin admin] PATH_INFO: `/v3/projects/4994e3fdabb2482396d3da7fe0fa94ff` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:57.181207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f466366-c5ff-4a48-aa1c-50157cfa2e74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4994e3fdabb2482396d3da7fe0fa94ff)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:57.183477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f466366-c5ff-4a48-aa1c-50157cfa2e74 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:57.325948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f466366-c5ff-4a48-aa1c-50157cfa2e74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4994e3fdabb2482396d3da7fe0fa94ff', 'request_id': 'req-9f466366-c5ff-4a48-aa1c-50157cfa2e74'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:57.332908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f466366-c5ff-4a48-aa1c-50157cfa2e74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4994e3fdabb2482396d3da7fe0fa94ff', 'request_id': 'req-9f466366-c5ff-4a48-aa1c-50157cfa2e74'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:57.442983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f466366-c5ff-4a48-aa1c-50157cfa2e74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4994e3fdabb2482396d3da7fe0fa94ff', 'request_id': 'req-9f466366-c5ff-4a48-aa1c-50157cfa2e74'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:57.455495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f466366-c5ff-4a48-aa1c-50157cfa2e74 admin admin] The token cache is being invalidate because project 4994e3fdabb2482396d3da7fe0fa94ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:17:57.455786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f466366-c5ff-4a48-aa1c-50157cfa2e74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:57.457093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f466366-c5ff-4a48-aa1c-50157cfa2e74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:57.462609 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2257/4539] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:17:57 2022] DELETE /identity/v3/projects/4994e3fdabb2482396d3da7fe0fa94ff => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:57.464409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019adb4-bb14-44c0-ad0e-858814254d7f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:57.464880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019adb4-bb14-44c0-ad0e-858814254d7f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:57.465013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019adb4-bb14-44c0-ad0e-858814254d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:57.486918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7019adb4-bb14-44c0-ad0e-858814254d7f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:57.489977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7019adb4-bb14-44c0-ad0e-858814254d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:57.557907 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2283/4540] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:57.565715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3477d-8099-4fdd-8377-d98b9fb75a56 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:57.566043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3477d-8099-4fdd-8377-d98b9fb75a56 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:57.568193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3477d-8099-4fdd-8377-d98b9fb75a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:57.586529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7eb3477d-8099-4fdd-8377-d98b9fb75a56 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:57.587752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7eb3477d-8099-4fdd-8377-d98b9fb75a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:57.596699 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2258/4541] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:57.601457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019adb4-bb14-44c0-ad0e-858814254d7f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:57.612311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7019adb4-bb14-44c0-ad0e-858814254d7f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:57.619681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d35b61-7943-4f3b-9ff1-0c2307fc69ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:57.620363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d35b61-7943-4f3b-9ff1-0c2307fc69ad admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:57.620787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d35b61-7943-4f3b-9ff1-0c2307fc69ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:57.621221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d35b61-7943-4f3b-9ff1-0c2307fc69ad admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:57.622151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d35b61-7943-4f3b-9ff1-0c2307fc69ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:17:57.622500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d35b61-7943-4f3b-9ff1-0c2307fc69ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:57.624478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d35b61-7943-4f3b-9ff1-0c2307fc69ad admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:57.642421 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2284/4542] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:17:57.644823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3477d-8099-4fdd-8377-d98b9fb75a56 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:57.660755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3477d-8099-4fdd-8377-d98b9fb75a56 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:57.667620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37962de-e664-4c63-b747-17875ac6cb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:57.671179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37962de-e664-4c63-b747-17875ac6cb81 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:57.671179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37962de-e664-4c63-b747-17875ac6cb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:57.671179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37962de-e664-4c63-b747-17875ac6cb81 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:57.671179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37962de-e664-4c63-b747-17875ac6cb81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:57.671688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37962de-e664-4c63-b747-17875ac6cb81 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:57.675963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:57.804026 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2259/4543] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:17:57 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:17:57.808596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d35b61-7943-4f3b-9ff1-0c2307fc69ad admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:57.821444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d35b61-7943-4f3b-9ff1-0c2307fc69ad admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:57.903533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b33b4-84ea-496e-aac6-dd095d647883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:57.904913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b33b4-84ea-496e-aac6-dd095d647883 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:57.905406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b33b4-84ea-496e-aac6-dd095d647883 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:57.905910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b33b4-84ea-496e-aac6-dd095d647883 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:57.906928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b33b4-84ea-496e-aac6-dd095d647883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:57.910469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b33b4-84ea-496e-aac6-dd095d647883 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:58.011346 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2285/4544] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:17:57 2022] POST /identity/v3/users => generated 281 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:17:58.015624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37962de-e664-4c63-b747-17875ac6cb81 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:58.034248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37962de-e664-4c63-b747-17875ac6cb81 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:58.050534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e640196-76c7-41c5-9694-dc31c77614a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:58.051224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e640196-76c7-41c5-9694-dc31c77614a4 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:58.051507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e640196-76c7-41c5-9694-dc31c77614a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:58.051947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e640196-76c7-41c5-9694-dc31c77614a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:58.052704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e640196-76c7-41c5-9694-dc31c77614a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:58.054375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e640196-76c7-41c5-9694-dc31c77614a4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:58.079286 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2260/4545] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:17:58.083967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b33b4-84ea-496e-aac6-dd095d647883 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:58.097751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b33b4-84ea-496e-aac6-dd095d647883 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:58.110103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6e618-da48-4d8e-a4a4-197868c5eb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:58.112035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6e618-da48-4d8e-a4a4-197868c5eb0d admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:58.112406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6e618-da48-4d8e-a4a4-197868c5eb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:58.114037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6e618-da48-4d8e-a4a4-197868c5eb0d admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:58.118463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6e618-da48-4d8e-a4a4-197868c5eb0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:58.120667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6e618-da48-4d8e-a4a4-197868c5eb0d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:58.137148 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2286/4546] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:17:58.141033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e640196-76c7-41c5-9694-dc31c77614a4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:58.155880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e640196-76c7-41c5-9694-dc31c77614a4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:58.164365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390af91-15a7-4bf0-a0f5-cc3284caebc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:58.166075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390af91-15a7-4bf0-a0f5-cc3284caebc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:58.166532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390af91-15a7-4bf0-a0f5-cc3284caebc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:58.169767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b390af91-15a7-4bf0-a0f5-cc3284caebc6 admin admin] PATH_INFO: `/v3/projects/4e8feb29be1c40919845f2b37184cd15/users/c47dfc5d531849f78d888c2d65df0c5d/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:58.191715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390af91-15a7-4bf0-a0f5-cc3284caebc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8feb29be1c40919845f2b37184cd15, user_id=c47dfc5d531849f78d888c2d65df0c5d, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:58.194529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b390af91-15a7-4bf0-a0f5-cc3284caebc6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:58.290804 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2261/4547] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:17:58 2022] PUT /identity/v3/projects/4e8feb29be1c40919845f2b37184cd15/users/c47dfc5d531849f78d888c2d65df0c5d/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:58.301882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb4c8e-b232-44fd-a968-4455f3f02bca None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:58.302222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb4c8e-b232-44fd-a968-4455f3f02bca None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:58.302445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb4c8e-b232-44fd-a968-4455f3f02bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:58.323691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:58.345611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3feb4c8e-b232-44fd-a968-4455f3f02bca None None] MFA Rules not processed for user `c47dfc5d531849f78d888c2d65df0c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:58.347012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3feb4c8e-b232-44fd-a968-4455f3f02bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:58.458376 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2287/4548] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:17:58.472516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390af91-15a7-4bf0-a0f5-cc3284caebc6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:58.474595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b390af91-15a7-4bf0-a0f5-cc3284caebc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:58.618104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b390af91-15a7-4bf0-a0f5-cc3284caebc6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:58.628788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa8464-559c-4451-8963-2cfba1fe9dc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:58.629726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa8464-559c-4451-8963-2cfba1fe9dc5 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:58.630287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa8464-559c-4451-8963-2cfba1fe9dc5 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:58.630657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efa8464-559c-4451-8963-2cfba1fe9dc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:58.642816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa8464-559c-4451-8963-2cfba1fe9dc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:58.645562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efa8464-559c-4451-8963-2cfba1fe9dc5 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:58.657940 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2262/4549] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:17:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:17:58.670808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f4888-e161-484d-bbb4-bb991314a722 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:58.671346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f4888-e161-484d-bbb4-bb991314a722 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:58.671346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f4888-e161-484d-bbb4-bb991314a722 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:58.672987 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2288/4550] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:17:58.678623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa8464-559c-4451-8963-2cfba1fe9dc5 service nova] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:17:58.692990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa8464-559c-4451-8963-2cfba1fe9dc5 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:58.701634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efa8464-559c-4451-8963-2cfba1fe9dc5 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:58.706309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6d335-9238-4258-9eeb-0c0c4a3c4e6a tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47dfc5d531849f78d888c2d65df0c5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e8feb29be1c40919845f2b37184cd15', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:58.707350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6d335-9238-4258-9eeb-0c0c4a3c4e6a tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:58.707620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6d335-9238-4258-9eeb-0c0c4a3c4e6a tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:58.707960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6d335-9238-4258-9eeb-0c0c4a3c4e6a tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] PATH_INFO: `/v3/projects/4e8feb29be1c40919845f2b37184cd15` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:58.709768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6d335-9238-4258-9eeb-0c0c4a3c4e6a tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] RBAC: Authorizing `identity:get_project(project_id=4e8feb29be1c40919845f2b37184cd15)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:58.711059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa6d335-9238-4258-9eeb-0c0c4a3c4e6a tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:58.738616 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2263/4551] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:17:58 2022] GET /identity/v3/projects/4e8feb29be1c40919845f2b37184cd15 => generated 361 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:17:58.751410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad913ce0-7a1b-40e3-b6e3-7bf7abef4c90 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:58.751410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad913ce0-7a1b-40e3-b6e3-7bf7abef4c90 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:58.751410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad913ce0-7a1b-40e3-b6e3-7bf7abef4c90 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:58.751410 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2289/4552] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:17:58.751785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6d335-9238-4258-9eeb-0c0c4a3c4e6a tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:17:58.751908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2fa6d335-9238-4258-9eeb-0c0c4a3c4e6a tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:58.774636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6d335-9238-4258-9eeb-0c0c4a3c4e6a tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:58.784723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6d335-9238-4258-9eeb-0c0c4a3c4e6a tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:58.789040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316105c2-a96f-4f70-b5a8-f26f54ad87ff tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47dfc5d531849f78d888c2d65df0c5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e8feb29be1c40919845f2b37184cd15', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:58.790613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316105c2-a96f-4f70-b5a8-f26f54ad87ff tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:58.791061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316105c2-a96f-4f70-b5a8-f26f54ad87ff tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:58.791341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316105c2-a96f-4f70-b5a8-f26f54ad87ff tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] PATH_INFO: `/v3/projects/4e8feb29be1c40919845f2b37184cd15` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:58.792890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316105c2-a96f-4f70-b5a8-f26f54ad87ff tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] RBAC: Authorizing `identity:get_project(project_id=4e8feb29be1c40919845f2b37184cd15)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:58.794783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316105c2-a96f-4f70-b5a8-f26f54ad87ff tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:58.797694 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2264/4553] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:17:58 2022] GET /identity/v3/projects/4e8feb29be1c40919845f2b37184cd15 => generated 361 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:17:58.846025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cb1c7-70c6-4537-baa2-d9fc87f4b5bc None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:58.846314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cb1c7-70c6-4537-baa2-d9fc87f4b5bc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:58.846709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cb1c7-70c6-4537-baa2-d9fc87f4b5bc None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:58.848537 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2290/4554] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:17:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:17:58.855963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316105c2-a96f-4f70-b5a8-f26f54ad87ff tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:17:58.882596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316105c2-a96f-4f70-b5a8-f26f54ad87ff tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:58.892040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316105c2-a96f-4f70-b5a8-f26f54ad87ff tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:58.897430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691fdf3-606d-42ec-84ec-fd95079c3f2c tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47dfc5d531849f78d888c2d65df0c5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e8feb29be1c40919845f2b37184cd15', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:58.898488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691fdf3-606d-42ec-84ec-fd95079c3f2c tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:58.901454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691fdf3-606d-42ec-84ec-fd95079c3f2c tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:58.901454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2691fdf3-606d-42ec-84ec-fd95079c3f2c tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] PATH_INFO: `/v3/projects/4e8feb29be1c40919845f2b37184cd15` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:58.901454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691fdf3-606d-42ec-84ec-fd95079c3f2c tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] RBAC: Authorizing `identity:get_project(project_id=4e8feb29be1c40919845f2b37184cd15)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:58.915329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2691fdf3-606d-42ec-84ec-fd95079c3f2c tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:58.920743 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2265/4555] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:17:58 2022] GET /identity/v3/projects/4e8feb29be1c40919845f2b37184cd15 => generated 361 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:17:58.958126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8f22f-0136-4713-a976-7e91939cbdee None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:58.958619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8f22f-0136-4713-a976-7e91939cbdee None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:58.959125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8f22f-0136-4713-a976-7e91939cbdee None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:58.960914 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2291/4556] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:17:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:17:58.966588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691fdf3-606d-42ec-84ec-fd95079c3f2c tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:17:58.997423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691fdf3-606d-42ec-84ec-fd95079c3f2c tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:59.012892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2691fdf3-606d-42ec-84ec-fd95079c3f2c tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:59.028515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af82a9-501b-4403-a447-6452828ea261 tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47dfc5d531849f78d888c2d65df0c5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e8feb29be1c40919845f2b37184cd15', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:59.036245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af82a9-501b-4403-a447-6452828ea261 tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:59.036245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af82a9-501b-4403-a447-6452828ea261 tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:59.036245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01af82a9-501b-4403-a447-6452828ea261 tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] PATH_INFO: `/v3/projects/4e8feb29be1c40919845f2b37184cd15` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:59.036245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af82a9-501b-4403-a447-6452828ea261 tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] RBAC: Authorizing `identity:get_project(project_id=4e8feb29be1c40919845f2b37184cd15)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:59.046569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01af82a9-501b-4403-a447-6452828ea261 tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:59.051409 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2266/4557] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:17:58 2022] GET /identity/v3/projects/4e8feb29be1c40919845f2b37184cd15 => generated 361 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:17:59.100975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912a3e8-3cd9-4f05-8209-b0efa1ab50e7 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:59.101933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912a3e8-3cd9-4f05-8209-b0efa1ab50e7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:59.101933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912a3e8-3cd9-4f05-8209-b0efa1ab50e7 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:59.103856 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2292/4558] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:17:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:17:59.110706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af82a9-501b-4403-a447-6452828ea261 tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] Authenticating service token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:17:59.133253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af82a9-501b-4403-a447-6452828ea261 tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:59.147569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01af82a9-501b-4403-a447-6452828ea261 tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:59.153507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a65b05-bf59-44a7-803e-ab3afd1ee094 tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c47dfc5d531849f78d888c2d65df0c5d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e8feb29be1c40919845f2b37184cd15', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:59.154638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a65b05-bf59-44a7-803e-ab3afd1ee094 tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:59.155084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a65b05-bf59-44a7-803e-ab3afd1ee094 tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:59.155519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a65b05-bf59-44a7-803e-ab3afd1ee094 tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] PATH_INFO: `/v3/projects/4e8feb29be1c40919845f2b37184cd15` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:59.156919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a65b05-bf59-44a7-803e-ab3afd1ee094 tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] RBAC: Authorizing `identity:get_project(project_id=4e8feb29be1c40919845f2b37184cd15)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:59.161828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a65b05-bf59-44a7-803e-ab3afd1ee094 tempest-QuotasTestJSON-91403286 tempest-QuotasTestJSON-91403286-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:59.164103 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2267/4559] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:17:59 2022] GET /identity/v3/projects/4e8feb29be1c40919845f2b37184cd15 => generated 361 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:17:59.203700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912a3e8-3cd9-4f05-8209-b0efa1ab50e7 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:59.308265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912a3e8-3cd9-4f05-8209-b0efa1ab50e7 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:17:59.315352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6870af-cb9b-4ae9-8a12-cd71e88c64ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:17:59.316383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6870af-cb9b-4ae9-8a12-cd71e88c64ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:59.318607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6870af-cb9b-4ae9-8a12-cd71e88c64ec admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:59.320927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6870af-cb9b-4ae9-8a12-cd71e88c64ec admin admin] PATH_INFO: `/v3/users/c47dfc5d531849f78d888c2d65df0c5d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:59.323145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6870af-cb9b-4ae9-8a12-cd71e88c64ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=c47dfc5d531849f78d888c2d65df0c5d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:17:59.326880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6870af-cb9b-4ae9-8a12-cd71e88c64ec admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:17:59.546539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dc6870af-cb9b-4ae9-8a12-cd71e88c64ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c47dfc5d531849f78d888c2d65df0c5d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:59.559477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dc6870af-cb9b-4ae9-8a12-cd71e88c64ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c47dfc5d531849f78d888c2d65df0c5d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:59.560603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dc6870af-cb9b-4ae9-8a12-cd71e88c64ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c47dfc5d531849f78d888c2d65df0c5d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:59.561726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dc6870af-cb9b-4ae9-8a12-cd71e88c64ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c47dfc5d531849f78d888c2d65df0c5d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:59.649686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dc6870af-cb9b-4ae9-8a12-cd71e88c64ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c47dfc5d531849f78d888c2d65df0c5d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:17:59.683246 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2293/4560] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:17:59 2022] DELETE /identity/v3/users/c47dfc5d531849f78d888c2d65df0c5d => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:17:59.688455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5400914-839e-4191-bddb-864bdd30d419 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:17:59.688797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5400914-839e-4191-bddb-864bdd30d419 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:17:59.689159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5400914-839e-4191-bddb-864bdd30d419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:17:59.715815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a5400914-839e-4191-bddb-864bdd30d419 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:17:59.717257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a5400914-839e-4191-bddb-864bdd30d419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:59.812576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:59.839424 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2268/4561] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:17:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:17:59.880262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6870af-cb9b-4ae9-8a12-cd71e88c64ec admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:17:59.883464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dc6870af-cb9b-4ae9-8a12-cd71e88c64ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:17:59.914098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:00.001103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6870af-cb9b-4ae9-8a12-cd71e88c64ec admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:00.012609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419dd934-a528-44e5-9022-d75b9f95c5b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:00.012888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dd934-a528-44e5-9022-d75b9f95c5b8 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:00.013199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dd934-a528-44e5-9022-d75b9f95c5b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:00.013466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dd934-a528-44e5-9022-d75b9f95c5b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:00.034664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419dd934-a528-44e5-9022-d75b9f95c5b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:00.036170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419dd934-a528-44e5-9022-d75b9f95c5b8 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:00.054769 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2294/4562] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:17:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:01.015576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5400914-839e-4191-bddb-864bdd30d419 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:01.017648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a5400914-839e-4191-bddb-864bdd30d419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:01.079025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5400914-839e-4191-bddb-864bdd30d419 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:01.092071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5f558-a94a-484a-84ee-a7cd3e69e87b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:01.094421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5f558-a94a-484a-84ee-a7cd3e69e87b service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:01.099785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5f558-a94a-484a-84ee-a7cd3e69e87b service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:01.099785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5f558-a94a-484a-84ee-a7cd3e69e87b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:01.103649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c8c5f558-a94a-484a-84ee-a7cd3e69e87b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:01.267113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5f558-a94a-484a-84ee-a7cd3e69e87b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:01.274827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5f558-a94a-484a-84ee-a7cd3e69e87b service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:01.304402 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2269/4563] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:01.714769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419dd934-a528-44e5-9022-d75b9f95c5b8 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:01.716321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-419dd934-a528-44e5-9022-d75b9f95c5b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:01.744068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419dd934-a528-44e5-9022-d75b9f95c5b8 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:01.749266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0360ad7-e9de-4682-b4ca-065c67698fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:01.750761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0360ad7-e9de-4682-b4ca-065c67698fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:01.751360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0360ad7-e9de-4682-b4ca-065c67698fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:01.752478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0360ad7-e9de-4682-b4ca-065c67698fcf admin admin] PATH_INFO: `/v3/projects/4e8feb29be1c40919845f2b37184cd15` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:01.756201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0360ad7-e9de-4682-b4ca-065c67698fcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e8feb29be1c40919845f2b37184cd15)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:01.759316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0360ad7-e9de-4682-b4ca-065c67698fcf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:01.955744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b0360ad7-e9de-4682-b4ca-065c67698fcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e8feb29be1c40919845f2b37184cd15', 'request_id': 'req-b0360ad7-e9de-4682-b4ca-065c67698fcf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:01.962241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b0360ad7-e9de-4682-b4ca-065c67698fcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e8feb29be1c40919845f2b37184cd15', 'request_id': 'req-b0360ad7-e9de-4682-b4ca-065c67698fcf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:02.038371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b0360ad7-e9de-4682-b4ca-065c67698fcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e8feb29be1c40919845f2b37184cd15', 'request_id': 'req-b0360ad7-e9de-4682-b4ca-065c67698fcf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:02.057786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b0360ad7-e9de-4682-b4ca-065c67698fcf admin admin] The token cache is being invalidate because project 4e8feb29be1c40919845f2b37184cd15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:02.058353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b0360ad7-e9de-4682-b4ca-065c67698fcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:02.059228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b0360ad7-e9de-4682-b4ca-065c67698fcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:02.061456 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2295/4564] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:01 2022] DELETE /identity/v3/projects/4e8feb29be1c40919845f2b37184cd15 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:02.068635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b1812-a5ff-4139-96c8-9617256e3d56 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:02.069515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b1812-a5ff-4139-96c8-9617256e3d56 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:02.070074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b1812-a5ff-4139-96c8-9617256e3d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:02.094575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bf9b1812-a5ff-4139-96c8-9617256e3d56 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:02.096194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bf9b1812-a5ff-4139-96c8-9617256e3d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:02.167588 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2270/4565] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:02.176782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a659c-1651-4aba-b37f-ed38c3d14280 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:02.177141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a659c-1651-4aba-b37f-ed38c3d14280 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:02.177424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a659c-1651-4aba-b37f-ed38c3d14280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:02.203983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b80a659c-1651-4aba-b37f-ed38c3d14280 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:02.207930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b80a659c-1651-4aba-b37f-ed38c3d14280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:02.222918 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2296/4566] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:02.226609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b1812-a5ff-4139-96c8-9617256e3d56 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:02.241187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b1812-a5ff-4139-96c8-9617256e3d56 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:02.246491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5144ca-a059-4e04-84ba-5ef79ce5bf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:02.247532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5144ca-a059-4e04-84ba-5ef79ce5bf56 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:02.247933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5144ca-a059-4e04-84ba-5ef79ce5bf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:02.248315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5144ca-a059-4e04-84ba-5ef79ce5bf56 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:02.249153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5144ca-a059-4e04-84ba-5ef79ce5bf56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:18:02.251080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5144ca-a059-4e04-84ba-5ef79ce5bf56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:02.253088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5144ca-a059-4e04-84ba-5ef79ce5bf56 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:02.277156 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2271/4567] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:18:02.282317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a659c-1651-4aba-b37f-ed38c3d14280 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:02.294111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a659c-1651-4aba-b37f-ed38c3d14280 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:02.300750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e3812-ca26-4ab2-83e3-ae12f84e519f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:02.301669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e3812-ca26-4ab2-83e3-ae12f84e519f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:02.301669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e3812-ca26-4ab2-83e3-ae12f84e519f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:02.302006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e3812-ca26-4ab2-83e3-ae12f84e519f admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:02.302731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e3812-ca26-4ab2-83e3-ae12f84e519f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:02.304710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e3812-ca26-4ab2-83e3-ae12f84e519f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:02.422861 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2297/4568] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:02 2022] POST /identity/v3/projects => generated 377 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:02.427352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5144ca-a059-4e04-84ba-5ef79ce5bf56 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:02.438796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5144ca-a059-4e04-84ba-5ef79ce5bf56 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:02.528169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f7bf7-63a7-425b-84b0-b3ac2e665dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:02.529001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f7bf7-63a7-425b-84b0-b3ac2e665dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:02.529297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f7bf7-63a7-425b-84b0-b3ac2e665dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:02.529590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f7bf7-63a7-425b-84b0-b3ac2e665dc4 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:02.530331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f7bf7-63a7-425b-84b0-b3ac2e665dc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:02.532295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f7bf7-63a7-425b-84b0-b3ac2e665dc4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:02.542397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:02.652554 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2272/4569] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:02 2022] POST /identity/v3/users => generated 289 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:02.653820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e3812-ca26-4ab2-83e3-ae12f84e519f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:02.667451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e3812-ca26-4ab2-83e3-ae12f84e519f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:02.673094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81f500-649f-4e9c-89e8-2bad778502ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:02.673781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81f500-649f-4e9c-89e8-2bad778502ad admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:02.674555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81f500-649f-4e9c-89e8-2bad778502ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:02.674555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b81f500-649f-4e9c-89e8-2bad778502ad admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:02.675152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b81f500-649f-4e9c-89e8-2bad778502ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:02.676872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b81f500-649f-4e9c-89e8-2bad778502ad admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:02.692730 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2298/4570] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:02.693987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f7bf7-63a7-425b-84b0-b3ac2e665dc4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:02.705229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f7bf7-63a7-425b-84b0-b3ac2e665dc4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:02.713213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb0ca6-c46a-47ab-bd51-418c5d8803cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:02.714391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb0ca6-c46a-47ab-bd51-418c5d8803cb admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:02.714812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb0ca6-c46a-47ab-bd51-418c5d8803cb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:02.715163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb0ca6-c46a-47ab-bd51-418c5d8803cb admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:02.716011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb0ca6-c46a-47ab-bd51-418c5d8803cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:02.718500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb0ca6-c46a-47ab-bd51-418c5d8803cb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:02.737100 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2273/4571] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:18:02.739916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81f500-649f-4e9c-89e8-2bad778502ad admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:02.755113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b81f500-649f-4e9c-89e8-2bad778502ad admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:02.761525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3317cc2-a5b7-4433-87be-de69408c748e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:02.769391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3317cc2-a5b7-4433-87be-de69408c748e admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:02.769391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3317cc2-a5b7-4433-87be-de69408c748e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:02.769391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3317cc2-a5b7-4433-87be-de69408c748e admin admin] PATH_INFO: `/v3/projects/6b958a05310a4b749b2a03f7cc22b26c/users/61ec01e1794143eeb2c0c1a6410aa80c/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:02.783615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3317cc2-a5b7-4433-87be-de69408c748e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b958a05310a4b749b2a03f7cc22b26c, user_id=61ec01e1794143eeb2c0c1a6410aa80c, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:02.786807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3317cc2-a5b7-4433-87be-de69408c748e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:02.897273 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2299/4572] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:02 2022] PUT /identity/v3/projects/6b958a05310a4b749b2a03f7cc22b26c/users/61ec01e1794143eeb2c0c1a6410aa80c/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:02.899065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd5433-ce22-4d3d-ad54-15bfc7541d2a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:02.900473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd5433-ce22-4d3d-ad54-15bfc7541d2a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:02.900473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd5433-ce22-4d3d-ad54-15bfc7541d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:02.930673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5ddd5433-ce22-4d3d-ad54-15bfc7541d2a None None] MFA Rules not processed for user `61ec01e1794143eeb2c0c1a6410aa80c`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:02.932219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5ddd5433-ce22-4d3d-ad54-15bfc7541d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:03.078828 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2274/4573] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:03.083691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3317cc2-a5b7-4433-87be-de69408c748e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:03.144902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:03.168011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3317cc2-a5b7-4433-87be-de69408c748e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:03.173047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf529f-d052-42c5-aca7-85806a0d8bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:03.173737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf529f-d052-42c5-aca7-85806a0d8bfd admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:03.174244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf529f-d052-42c5-aca7-85806a0d8bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:03.174535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf529f-d052-42c5-aca7-85806a0d8bfd admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:03.175295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf529f-d052-42c5-aca7-85806a0d8bfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:03.177424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf529f-d052-42c5-aca7-85806a0d8bfd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:03.276341 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2300/4574] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:03 2022] POST /identity/v3/projects => generated 375 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:18:03.278990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd5433-ce22-4d3d-ad54-15bfc7541d2a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:03.300237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddd5433-ce22-4d3d-ad54-15bfc7541d2a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:03.327985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf529f-d052-42c5-aca7-85806a0d8bfd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:03.329869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-88cf529f-d052-42c5-aca7-85806a0d8bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:03.465525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65836786-ca1b-460d-b3f0-cdd7c064d983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:03.467248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65836786-ca1b-460d-b3f0-cdd7c064d983 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:03.467248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65836786-ca1b-460d-b3f0-cdd7c064d983 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:03.467248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65836786-ca1b-460d-b3f0-cdd7c064d983 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:03.468400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65836786-ca1b-460d-b3f0-cdd7c064d983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:03.472641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65836786-ca1b-460d-b3f0-cdd7c064d983 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:03.475344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf529f-d052-42c5-aca7-85806a0d8bfd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:03.483720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cdba3-f4c6-4d63-b7a4-e250397ad0d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:03.484640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cdba3-f4c6-4d63-b7a4-e250397ad0d2 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:03.485219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cdba3-f4c6-4d63-b7a4-e250397ad0d2 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:03.485684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cdba3-f4c6-4d63-b7a4-e250397ad0d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:03.488947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-732cdba3-f4c6-4d63-b7a4-e250397ad0d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:03.593202 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2275/4575] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:03 2022] POST /identity/v3/users => generated 294 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:18:03.595479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65836786-ca1b-460d-b3f0-cdd7c064d983 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:03.622241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65836786-ca1b-460d-b3f0-cdd7c064d983 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:03.637262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa9f20-effc-4ca8-98f0-3f85dd5ad4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:03.641564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa9f20-effc-4ca8-98f0-3f85dd5ad4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:03.641564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa9f20-effc-4ca8-98f0-3f85dd5ad4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:03.641564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa9f20-effc-4ca8-98f0-3f85dd5ad4d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:03.647468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa9f20-effc-4ca8-98f0-3f85dd5ad4d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:03.653985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa9f20-effc-4ca8-98f0-3f85dd5ad4d2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:03.680474 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2276/4576] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:03.685700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa9f20-effc-4ca8-98f0-3f85dd5ad4d2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:03.708128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa9f20-effc-4ca8-98f0-3f85dd5ad4d2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:03.725538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f042181-e0fa-4547-bc77-fe5ea383b5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:03.726696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f042181-e0fa-4547-bc77-fe5ea383b5cf admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:03.726985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f042181-e0fa-4547-bc77-fe5ea383b5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:03.727449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f042181-e0fa-4547-bc77-fe5ea383b5cf admin admin] PATH_INFO: `/v3/projects/5c7688ef853041beab5f217e2339dcaf/users/1463d2bda498447691e18aa4309310b0/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:03.771593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f042181-e0fa-4547-bc77-fe5ea383b5cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c7688ef853041beab5f217e2339dcaf, user_id=1463d2bda498447691e18aa4309310b0, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:03.777063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f042181-e0fa-4547-bc77-fe5ea383b5cf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:03.801675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732cdba3-f4c6-4d63-b7a4-e250397ad0d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:03.804066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732cdba3-f4c6-4d63-b7a4-e250397ad0d2 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:03.852959 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2301/4577] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:03.857676 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2277/4578] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:03 2022] PUT /identity/v3/projects/5c7688ef853041beab5f217e2339dcaf/users/1463d2bda498447691e18aa4309310b0/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:03.861745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cdba3-f4c6-4d63-b7a4-e250397ad0d2 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:03.881039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cdba3-f4c6-4d63-b7a4-e250397ad0d2 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:03.989812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f98882-af4f-41d5-82f2-b3e8fea9ff8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:03.990894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f98882-af4f-41d5-82f2-b3e8fea9ff8b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:03.991280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f98882-af4f-41d5-82f2-b3e8fea9ff8b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:03.991657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f98882-af4f-41d5-82f2-b3e8fea9ff8b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:03.992551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f98882-af4f-41d5-82f2-b3e8fea9ff8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:03.994306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f98882-af4f-41d5-82f2-b3e8fea9ff8b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:04.010568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f042181-e0fa-4547-bc77-fe5ea383b5cf admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:04.010785 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2302/4579] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:03 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:04.028294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f042181-e0fa-4547-bc77-fe5ea383b5cf admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:04.041687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b2c9b-7879-4b86-a9ea-5da7ebefb22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:04.043090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b2c9b-7879-4b86-a9ea-5da7ebefb22e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:04.044611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b2c9b-7879-4b86-a9ea-5da7ebefb22e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:04.047487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b2c9b-7879-4b86-a9ea-5da7ebefb22e admin admin] PATH_INFO: `/v3/projects/5c7688ef853041beab5f217e2339dcaf/users/1463d2bda498447691e18aa4309310b0/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:04.051160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b2c9b-7879-4b86-a9ea-5da7ebefb22e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c7688ef853041beab5f217e2339dcaf, user_id=1463d2bda498447691e18aa4309310b0, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:04.055602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b2c9b-7879-4b86-a9ea-5da7ebefb22e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:04.093124 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2278/4580] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:04 2022] PUT /identity/v3/projects/5c7688ef853041beab5f217e2339dcaf/users/1463d2bda498447691e18aa4309310b0/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:18:04.106457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa3530-947a-48af-b8b8-685ebb25180d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:04.107098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa3530-947a-48af-b8b8-685ebb25180d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:04.107594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa3530-947a-48af-b8b8-685ebb25180d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:04.183624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b0aa3530-947a-48af-b8b8-685ebb25180d None None] MFA Rules not processed for user `1463d2bda498447691e18aa4309310b0`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:04.185337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b0aa3530-947a-48af-b8b8-685ebb25180d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:04.306619 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2303/4581] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:04.311767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b2c9b-7879-4b86-a9ea-5da7ebefb22e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:04.336962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b2c9b-7879-4b86-a9ea-5da7ebefb22e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:04.344489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3f183-96b7-4ace-9c26-3e9f7942e8fa tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1463d2bda498447691e18aa4309310b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7688ef853041beab5f217e2339dcaf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:04.345460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3f183-96b7-4ace-9c26-3e9f7942e8fa tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:04.346307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3f183-96b7-4ace-9c26-3e9f7942e8fa tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:04.348070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3f183-96b7-4ace-9c26-3e9f7942e8fa tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:04.349910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3f183-96b7-4ace-9c26-3e9f7942e8fa tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:04.352135 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-d6a3f183-96b7-4ace-9c26-3e9f7942e8fa tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:18:04.352135 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:18:04.352135 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:18:04.355732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3f183-96b7-4ace-9c26-3e9f7942e8fa tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:04.449912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa3530-947a-48af-b8b8-685ebb25180d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:04.452421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b0aa3530-947a-48af-b8b8-685ebb25180d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:04.478477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:04.498962 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2279/4582] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:18:04 2022] POST /identity/v3/domains => generated 246 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:18:04.518303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3f183-96b7-4ace-9c26-3e9f7942e8fa tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:04.602824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa3530-947a-48af-b8b8-685ebb25180d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:04.648343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3f183-96b7-4ace-9c26-3e9f7942e8fa tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:04.660119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc853be5-4f26-41aa-a52c-87f9219bff93 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1463d2bda498447691e18aa4309310b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7688ef853041beab5f217e2339dcaf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:04.660956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc853be5-4f26-41aa-a52c-87f9219bff93 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:04.661441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc853be5-4f26-41aa-a52c-87f9219bff93 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:04.661715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc853be5-4f26-41aa-a52c-87f9219bff93 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:04.662549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc853be5-4f26-41aa-a52c-87f9219bff93 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:04.666466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc853be5-4f26-41aa-a52c-87f9219bff93 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:04.673051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680fe2e-5c94-4611-8d89-ef1b355e2517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:04.673725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680fe2e-5c94-4611-8d89-ef1b355e2517 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:04.674147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680fe2e-5c94-4611-8d89-ef1b355e2517 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:04.674435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680fe2e-5c94-4611-8d89-ef1b355e2517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:04.679121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8680fe2e-5c94-4611-8d89-ef1b355e2517 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:04.783146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680fe2e-5c94-4611-8d89-ef1b355e2517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:04.785387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680fe2e-5c94-4611-8d89-ef1b355e2517 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:04.799578 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2304/4583] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:18:04.824880 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2280/4584] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:04 2022] POST /identity/v3/projects => generated 409 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:04.829121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680fe2e-5c94-4611-8d89-ef1b355e2517 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:04.841919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680fe2e-5c94-4611-8d89-ef1b355e2517 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:04.930815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646eb2db-a8f5-4a83-a9a0-8580d79dae9b tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1463d2bda498447691e18aa4309310b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7688ef853041beab5f217e2339dcaf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:04.931648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646eb2db-a8f5-4a83-a9a0-8580d79dae9b tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:04.932000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646eb2db-a8f5-4a83-a9a0-8580d79dae9b tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:04.932404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646eb2db-a8f5-4a83-a9a0-8580d79dae9b tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:04.933203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646eb2db-a8f5-4a83-a9a0-8580d79dae9b tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:04.935246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646eb2db-a8f5-4a83-a9a0-8580d79dae9b tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:05.040137 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2305/4585] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:04 2022] POST /identity/v3/users => generated 347 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:05.042147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc853be5-4f26-41aa-a52c-87f9219bff93 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:05.060267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc853be5-4f26-41aa-a52c-87f9219bff93 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:05.067885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4d90d-a0c2-42df-9b77-7a3c31566cf8 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1463d2bda498447691e18aa4309310b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7688ef853041beab5f217e2339dcaf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:05.068691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4d90d-a0c2-42df-9b77-7a3c31566cf8 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:05.069086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4d90d-a0c2-42df-9b77-7a3c31566cf8 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:05.070310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4d90d-a0c2-42df-9b77-7a3c31566cf8 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:05.071602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4d90d-a0c2-42df-9b77-7a3c31566cf8 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:05.073580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4d90d-a0c2-42df-9b77-7a3c31566cf8 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:05.093606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646eb2db-a8f5-4a83-a9a0-8580d79dae9b tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:05.100821 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2281/4586] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:18:05.106766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646eb2db-a8f5-4a83-a9a0-8580d79dae9b tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:05.114986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87da2f-cf9c-4c5c-a63d-c9d45d423966 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1463d2bda498447691e18aa4309310b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7688ef853041beab5f217e2339dcaf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:05.119828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87da2f-cf9c-4c5c-a63d-c9d45d423966 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:05.119828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87da2f-cf9c-4c5c-a63d-c9d45d423966 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:05.119828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87da2f-cf9c-4c5c-a63d-c9d45d423966 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] PATH_INFO: `/v3/projects/f0cdfb7b6ecc4093bc2a02007d53426c/users/957ff95d3ee64f4db859b8509071f06e/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:05.148830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87da2f-cf9c-4c5c-a63d-c9d45d423966 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorizing `identity:create_grant(project_id=f0cdfb7b6ecc4093bc2a02007d53426c, user_id=957ff95d3ee64f4db859b8509071f06e, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:05.151651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87da2f-cf9c-4c5c-a63d-c9d45d423966 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:05.250065 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2306/4587] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:05 2022] PUT /identity/v3/projects/f0cdfb7b6ecc4093bc2a02007d53426c/users/957ff95d3ee64f4db859b8509071f06e/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:05.255240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24cc4c-cfb4-4e82-9d61-557a2df36378 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:05.255616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24cc4c-cfb4-4e82-9d61-557a2df36378 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:05.255978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24cc4c-cfb4-4e82-9d61-557a2df36378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:05.427596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ff24cc4c-cfb4-4e82-9d61-557a2df36378 None None] MFA Rules not processed for user `957ff95d3ee64f4db859b8509071f06e`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:05.429282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ff24cc4c-cfb4-4e82-9d61-557a2df36378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:05.452106 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2282/4588] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:05 2022] POST /identity/v3/auth/tokens => generated 3352 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:05.476054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e464c7d-08cd-4d38-a8d3-2959a117b478 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:05.476252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e464c7d-08cd-4d38-a8d3-2959a117b478 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:05.476539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e464c7d-08cd-4d38-a8d3-2959a117b478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:05.497649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3e464c7d-08cd-4d38-a8d3-2959a117b478 None None] MFA Rules not processed for user `957ff95d3ee64f4db859b8509071f06e`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:05.501709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3e464c7d-08cd-4d38-a8d3-2959a117b478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:05.516562 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2307/4589] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:05 2022] POST /identity/v3/auth/tokens => generated 3352 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:05.518008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24cc4c-cfb4-4e82-9d61-557a2df36378 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:05.528693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24cc4c-cfb4-4e82-9d61-557a2df36378 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:05.538162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b034304-00d5-479a-b5d2-6eac39ab9b33 tempest-test_project-2088398639 tempest-user-893810744] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '957ff95d3ee64f4db859b8509071f06e', 'user_domain_id': '55fe5f18c6d14b598b0d36346c3fe7c2', 'system_scope': None, 'project_id': 'f0cdfb7b6ecc4093bc2a02007d53426c', 'project_domain_id': '55fe5f18c6d14b598b0d36346c3fe7c2', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:05.539249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b034304-00d5-479a-b5d2-6eac39ab9b33 tempest-test_project-2088398639 tempest-user-893810744] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:05.539748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b034304-00d5-479a-b5d2-6eac39ab9b33 tempest-test_project-2088398639 tempest-user-893810744] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:05.540112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b034304-00d5-479a-b5d2-6eac39ab9b33 tempest-test_project-2088398639 tempest-user-893810744] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:05.551142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b034304-00d5-479a-b5d2-6eac39ab9b33 tempest-test_project-2088398639 tempest-user-893810744] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:05.553190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b034304-00d5-479a-b5d2-6eac39ab9b33 tempest-test_project-2088398639 tempest-user-893810744] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:05.564476 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2283/4590] 10.222.0.33 () {56 vars in 1283 bytes} [Sat May 28 21:18:05 2022] GET /identity/v3/auth/tokens => generated 3352 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:05.569398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e464c7d-08cd-4d38-a8d3-2959a117b478 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:05.653596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e464c7d-08cd-4d38-a8d3-2959a117b478 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:05.659806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d79ee5-c4f9-429c-a95f-c5d8df60274a tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1463d2bda498447691e18aa4309310b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7688ef853041beab5f217e2339dcaf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:05.660691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d79ee5-c4f9-429c-a95f-c5d8df60274a tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:05.660983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d79ee5-c4f9-429c-a95f-c5d8df60274a tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:05.661409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d79ee5-c4f9-429c-a95f-c5d8df60274a tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] PATH_INFO: `/v3/users/957ff95d3ee64f4db859b8509071f06e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:05.662735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d79ee5-c4f9-429c-a95f-c5d8df60274a tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorizing `identity:delete_user(user_id=957ff95d3ee64f4db859b8509071f06e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:05.664753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d79ee5-c4f9-429c-a95f-c5d8df60274a tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:05.752930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:05.968568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-33d79ee5-c4f9-429c-a95f-c5d8df60274a tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '957ff95d3ee64f4db859b8509071f06e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:05.986789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-33d79ee5-c4f9-429c-a95f-c5d8df60274a tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '957ff95d3ee64f4db859b8509071f06e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:05.988526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-33d79ee5-c4f9-429c-a95f-c5d8df60274a tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '957ff95d3ee64f4db859b8509071f06e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:05.989676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-33d79ee5-c4f9-429c-a95f-c5d8df60274a tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '957ff95d3ee64f4db859b8509071f06e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:06.091536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-33d79ee5-c4f9-429c-a95f-c5d8df60274a tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '957ff95d3ee64f4db859b8509071f06e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:06.146595 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2308/4591] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:05 2022] DELETE /identity/v3/users/957ff95d3ee64f4db859b8509071f06e => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:06.150676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b034304-00d5-479a-b5d2-6eac39ab9b33 tempest-test_project-2088398639 tempest-user-893810744] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:06.164294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b034304-00d5-479a-b5d2-6eac39ab9b33 tempest-test_project-2088398639 tempest-user-893810744] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:06.256376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079d16c-54da-4510-877e-271ec35369b6 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1463d2bda498447691e18aa4309310b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7688ef853041beab5f217e2339dcaf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:06.257610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079d16c-54da-4510-877e-271ec35369b6 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:06.258176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079d16c-54da-4510-877e-271ec35369b6 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:06.258709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079d16c-54da-4510-877e-271ec35369b6 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] PATH_INFO: `/v3/projects/f0cdfb7b6ecc4093bc2a02007d53426c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:06.260320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079d16c-54da-4510-877e-271ec35369b6 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f0cdfb7b6ecc4093bc2a02007d53426c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:06.262656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079d16c-54da-4510-877e-271ec35369b6 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:06.330091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:06.403172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d079d16c-54da-4510-877e-271ec35369b6 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0cdfb7b6ecc4093bc2a02007d53426c', 'request_id': 'req-d079d16c-54da-4510-877e-271ec35369b6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:06.411284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d079d16c-54da-4510-877e-271ec35369b6 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0cdfb7b6ecc4093bc2a02007d53426c', 'request_id': 'req-d079d16c-54da-4510-877e-271ec35369b6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:06.519328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d079d16c-54da-4510-877e-271ec35369b6 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0cdfb7b6ecc4093bc2a02007d53426c', 'request_id': 'req-d079d16c-54da-4510-877e-271ec35369b6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:06.543035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d079d16c-54da-4510-877e-271ec35369b6 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] The token cache is being invalidate because project f0cdfb7b6ecc4093bc2a02007d53426c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:06.543373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d079d16c-54da-4510-877e-271ec35369b6 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:06.546682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d079d16c-54da-4510-877e-271ec35369b6 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:06.547245 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2284/4592] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:06 2022] DELETE /identity/v3/projects/f0cdfb7b6ecc4093bc2a02007d53426c => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:06.552125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d79ee5-c4f9-429c-a95f-c5d8df60274a tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:06.554847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-33d79ee5-c4f9-429c-a95f-c5d8df60274a tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:06.588560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d79ee5-c4f9-429c-a95f-c5d8df60274a tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:06.715813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68c998-b42d-478b-b696-eb32e7fd4144 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1463d2bda498447691e18aa4309310b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7688ef853041beab5f217e2339dcaf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:06.718579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68c998-b42d-478b-b696-eb32e7fd4144 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:06.718970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68c998-b42d-478b-b696-eb32e7fd4144 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:06.719408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a68c998-b42d-478b-b696-eb32e7fd4144 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] PATH_INFO: `/v3/domains/55fe5f18c6d14b598b0d36346c3fe7c2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:06.722507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68c998-b42d-478b-b696-eb32e7fd4144 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=55fe5f18c6d14b598b0d36346c3fe7c2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:06.723861 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-1a68c998-b42d-478b-b696-eb32e7fd4144 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:18:06.723861 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:18:06.723861 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:18:06.724923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a68c998-b42d-478b-b696-eb32e7fd4144 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:06.750780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1a68c998-b42d-478b-b696-eb32e7fd4144 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '55fe5f18c6d14b598b0d36346c3fe7c2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:06.752037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1a68c998-b42d-478b-b696-eb32e7fd4144 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '55fe5f18c6d14b598b0d36346c3fe7c2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:06.865696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1a68c998-b42d-478b-b696-eb32e7fd4144 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '55fe5f18c6d14b598b0d36346c3fe7c2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:06.867207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1a68c998-b42d-478b-b696-eb32e7fd4144 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '55fe5f18c6d14b598b0d36346c3fe7c2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:06.874270 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2309/4593] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:18:06 2022] PATCH /identity/v3/domains/55fe5f18c6d14b598b0d36346c3fe7c2 => generated 247 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:18:06.881901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079d16c-54da-4510-877e-271ec35369b6 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:06.883346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d079d16c-54da-4510-877e-271ec35369b6 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:06.909125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079d16c-54da-4510-877e-271ec35369b6 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:07.090653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1463d2bda498447691e18aa4309310b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5c7688ef853041beab5f217e2339dcaf', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:07.092116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:07.092644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:07.093132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] PATH_INFO: `/v3/domains/55fe5f18c6d14b598b0d36346c3fe7c2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:07.116018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=55fe5f18c6d14b598b0d36346c3fe7c2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:07.119936 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:18:07.119936 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:18:07.119936 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:18:07.121324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:07.168847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '55fe5f18c6d14b598b0d36346c3fe7c2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:07.181442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '55fe5f18c6d14b598b0d36346c3fe7c2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:07.390163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55fe5f18c6d14b598b0d36346c3fe7c2', 'request_id': 'req-848303a5-9e84-40e0-b49f-e2d0df2b2284'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:07.405682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55fe5f18c6d14b598b0d36346c3fe7c2', 'request_id': 'req-848303a5-9e84-40e0-b49f-e2d0df2b2284'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:07.506673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55fe5f18c6d14b598b0d36346c3fe7c2', 'request_id': 'req-848303a5-9e84-40e0-b49f-e2d0df2b2284'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:07.528631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] The token cache is being invalidate because project 55fe5f18c6d14b598b0d36346c3fe7c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:07.529129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:07.530030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:07.552825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '55fe5f18c6d14b598b0d36346c3fe7c2', 'request_id': 'req-848303a5-9e84-40e0-b49f-e2d0df2b2284'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:07.553593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '55fe5f18c6d14b598b0d36346c3fe7c2', 'request_id': 'req-848303a5-9e84-40e0-b49f-e2d0df2b2284'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:07.565553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-848303a5-9e84-40e0-b49f-e2d0df2b2284 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '55fe5f18c6d14b598b0d36346c3fe7c2', 'request_id': 'req-848303a5-9e84-40e0-b49f-e2d0df2b2284'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:07.580083 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2285/4594] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:18:06 2022] DELETE /identity/v3/domains/55fe5f18c6d14b598b0d36346c3fe7c2 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:07.583368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68c998-b42d-478b-b696-eb32e7fd4144 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:07.593011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1a68c998-b42d-478b-b696-eb32e7fd4144 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:07.756221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a68c998-b42d-478b-b696-eb32e7fd4144 tempest-TestDefaultProjectId-684699713 tempest-TestDefaultProjectId-684699713-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:07.871076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2882ac2-befd-43b4-a231-f4c829ce0af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:07.874309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2882ac2-befd-43b4-a231-f4c829ce0af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:07.876039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2882ac2-befd-43b4-a231-f4c829ce0af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:07.876990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2882ac2-befd-43b4-a231-f4c829ce0af4 admin admin] PATH_INFO: `/v3/users/61ec01e1794143eeb2c0c1a6410aa80c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:07.884968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2882ac2-befd-43b4-a231-f4c829ce0af4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61ec01e1794143eeb2c0c1a6410aa80c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:07.895761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2882ac2-befd-43b4-a231-f4c829ce0af4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:08.171796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f2882ac2-befd-43b4-a231-f4c829ce0af4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61ec01e1794143eeb2c0c1a6410aa80c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:08.180477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f2882ac2-befd-43b4-a231-f4c829ce0af4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61ec01e1794143eeb2c0c1a6410aa80c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:08.181964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f2882ac2-befd-43b4-a231-f4c829ce0af4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61ec01e1794143eeb2c0c1a6410aa80c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:08.182714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f2882ac2-befd-43b4-a231-f4c829ce0af4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61ec01e1794143eeb2c0c1a6410aa80c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:08.190722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:08.283391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f2882ac2-befd-43b4-a231-f4c829ce0af4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61ec01e1794143eeb2c0c1a6410aa80c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:08.319107 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2310/4595] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:07 2022] DELETE /identity/v3/users/61ec01e1794143eeb2c0c1a6410aa80c => generated 0 bytes in 736 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:08.320921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040c4c7-1729-463c-8ef7-796d3951f59f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:08.321419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040c4c7-1729-463c-8ef7-796d3951f59f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:08.321515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040c4c7-1729-463c-8ef7-796d3951f59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:08.342109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8040c4c7-1729-463c-8ef7-796d3951f59f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:08.343782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8040c4c7-1729-463c-8ef7-796d3951f59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:08.438177 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2286/4596] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:18:08.448858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2882ac2-befd-43b4-a231-f4c829ce0af4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:08.452962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f2882ac2-befd-43b4-a231-f4c829ce0af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:08.526891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2882ac2-befd-43b4-a231-f4c829ce0af4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:08.531370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a7e64-ca63-4cf5-a0c0-cd99c4769d5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:08.532159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a7e64-ca63-4cf5-a0c0-cd99c4769d5e service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:08.536650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a7e64-ca63-4cf5-a0c0-cd99c4769d5e service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:08.537219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a7e64-ca63-4cf5-a0c0-cd99c4769d5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:08.551627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a7e64-ca63-4cf5-a0c0-cd99c4769d5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:08.553984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629a7e64-ca63-4cf5-a0c0-cd99c4769d5e service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:08.569757 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2311/4597] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:08.737390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:09.186906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040c4c7-1729-463c-8ef7-796d3951f59f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:09.188405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8040c4c7-1729-463c-8ef7-796d3951f59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:09.205768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040c4c7-1729-463c-8ef7-796d3951f59f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:09.217175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:09.220852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:09.221288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:09.221288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:09.221288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01 admin admin] PATH_INFO: `/v3/projects/6b958a05310a4b749b2a03f7cc22b26c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:09.224112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b958a05310a4b749b2a03f7cc22b26c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:09.226144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:09.391228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b958a05310a4b749b2a03f7cc22b26c', 'request_id': 'req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:09.408492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b958a05310a4b749b2a03f7cc22b26c', 'request_id': 'req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:09.477118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b958a05310a4b749b2a03f7cc22b26c', 'request_id': 'req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:09.490773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01 admin admin] The token cache is being invalidate because project 6b958a05310a4b749b2a03f7cc22b26c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:09.491068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:09.491727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:09.493282 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2287/4598] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:09 2022] DELETE /identity/v3/projects/6b958a05310a4b749b2a03f7cc22b26c => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:09.498833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a7e64-ca63-4cf5-a0c0-cd99c4769d5e service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:09.513095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a7e64-ca63-4cf5-a0c0-cd99c4769d5e service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:09.654548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b0f5f-7501-491a-b15c-cb7b16c48629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:09.655452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b0f5f-7501-491a-b15c-cb7b16c48629 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:09.655737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b0f5f-7501-491a-b15c-cb7b16c48629 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:09.656017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b0f5f-7501-491a-b15c-cb7b16c48629 admin admin] PATH_INFO: `/v3/users/1463d2bda498447691e18aa4309310b0` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:09.658712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b0f5f-7501-491a-b15c-cb7b16c48629 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1463d2bda498447691e18aa4309310b0)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:09.661227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332b0f5f-7501-491a-b15c-cb7b16c48629 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:09.913564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-332b0f5f-7501-491a-b15c-cb7b16c48629 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1463d2bda498447691e18aa4309310b0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:09.923402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-332b0f5f-7501-491a-b15c-cb7b16c48629 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1463d2bda498447691e18aa4309310b0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:09.924446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-332b0f5f-7501-491a-b15c-cb7b16c48629 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1463d2bda498447691e18aa4309310b0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:09.925120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-332b0f5f-7501-491a-b15c-cb7b16c48629 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1463d2bda498447691e18aa4309310b0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:10.022753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-332b0f5f-7501-491a-b15c-cb7b16c48629 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1463d2bda498447691e18aa4309310b0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:10.080705 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2312/4599] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:09 2022] DELETE /identity/v3/users/1463d2bda498447691e18aa4309310b0 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:11.783092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:11.784774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:11.805864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5e94d3-e390-4b6a-8f83-b8f72e701a01 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:11.820568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:11.891904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4f37e-58f1-4f8d-b7d1-27a9fd720d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:11.895415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4f37e-58f1-4f8d-b7d1-27a9fd720d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:11.895865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4f37e-58f1-4f8d-b7d1-27a9fd720d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:11.896635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4f37e-58f1-4f8d-b7d1-27a9fd720d48 admin admin] PATH_INFO: `/v3/projects/5c7688ef853041beab5f217e2339dcaf` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:11.902038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4f37e-58f1-4f8d-b7d1-27a9fd720d48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c7688ef853041beab5f217e2339dcaf)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:11.903034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc4f37e-58f1-4f8d-b7d1-27a9fd720d48 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:11.980300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:12.043962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fbc4f37e-58f1-4f8d-b7d1-27a9fd720d48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c7688ef853041beab5f217e2339dcaf', 'request_id': 'req-fbc4f37e-58f1-4f8d-b7d1-27a9fd720d48'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:12.058932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fbc4f37e-58f1-4f8d-b7d1-27a9fd720d48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c7688ef853041beab5f217e2339dcaf', 'request_id': 'req-fbc4f37e-58f1-4f8d-b7d1-27a9fd720d48'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:12.125043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b0f5f-7501-491a-b15c-cb7b16c48629 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:12.127589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-332b0f5f-7501-491a-b15c-cb7b16c48629 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:12.158688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fbc4f37e-58f1-4f8d-b7d1-27a9fd720d48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c7688ef853041beab5f217e2339dcaf', 'request_id': 'req-fbc4f37e-58f1-4f8d-b7d1-27a9fd720d48'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:12.193489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fbc4f37e-58f1-4f8d-b7d1-27a9fd720d48 admin admin] The token cache is being invalidate because project 5c7688ef853041beab5f217e2339dcaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:12.196925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fbc4f37e-58f1-4f8d-b7d1-27a9fd720d48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:12.199077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fbc4f37e-58f1-4f8d-b7d1-27a9fd720d48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:12.204247 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2288/4600] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:11 2022] DELETE /identity/v3/projects/5c7688ef853041beab5f217e2339dcaf => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:18:12.227917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d5157-f441-4b34-9c9f-44246f7f82ee None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:12.229668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d5157-f441-4b34-9c9f-44246f7f82ee None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:12.230025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d5157-f441-4b34-9c9f-44246f7f82ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:12.287171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d02d5157-f441-4b34-9c9f-44246f7f82ee None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:12.293527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d02d5157-f441-4b34-9c9f-44246f7f82ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:12.330784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b0f5f-7501-491a-b15c-cb7b16c48629 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:12.438928 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2289/4601] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:12.440936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284866b-f42a-4162-9b7c-f02a2abd6d7f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:12.441235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284866b-f42a-4162-9b7c-f02a2abd6d7f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:12.441520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3284866b-f42a-4162-9b7c-f02a2abd6d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:12.476096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df014a-20cf-4c88-b3e5-058d9fc658f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:12.476789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df014a-20cf-4c88-b3e5-058d9fc658f8 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:12.477081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df014a-20cf-4c88-b3e5-058d9fc658f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:12.485372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df014a-20cf-4c88-b3e5-058d9fc658f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:12.488011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b7df014a-20cf-4c88-b3e5-058d9fc658f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:12.497713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3284866b-f42a-4162-9b7c-f02a2abd6d7f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:12.499267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3284866b-f42a-4162-9b7c-f02a2abd6d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:12.520926 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2290/4602] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:12.522788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df014a-20cf-4c88-b3e5-058d9fc658f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:12.524267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df014a-20cf-4c88-b3e5-058d9fc658f8 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:12.525248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284866b-f42a-4162-9b7c-f02a2abd6d7f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:12.541081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3284866b-f42a-4162-9b7c-f02a2abd6d7f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:12.557541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d013b3-d0e4-4288-8494-0feeb0b6cb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:12.557541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d013b3-d0e4-4288-8494-0feeb0b6cb5c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:12.557541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d013b3-d0e4-4288-8494-0feeb0b6cb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:12.557541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d013b3-d0e4-4288-8494-0feeb0b6cb5c admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:12.558932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d013b3-d0e4-4288-8494-0feeb0b6cb5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:18:12.558932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d013b3-d0e4-4288-8494-0feeb0b6cb5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:12.560267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d013b3-d0e4-4288-8494-0feeb0b6cb5c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:12.561587 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2313/4603] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:18:12.593405 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2291/4604] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:18:12.594682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df014a-20cf-4c88-b3e5-058d9fc658f8 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:12.609585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df014a-20cf-4c88-b3e5-058d9fc658f8 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:12.616156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e5a45-ba40-4037-9bb9-ab60adbf4c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:12.617156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e5a45-ba40-4037-9bb9-ab60adbf4c62 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:12.617631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e5a45-ba40-4037-9bb9-ab60adbf4c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:12.618120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02e5a45-ba40-4037-9bb9-ab60adbf4c62 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:12.619066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e5a45-ba40-4037-9bb9-ab60adbf4c62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:12.621369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02e5a45-ba40-4037-9bb9-ab60adbf4c62 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:12.724628 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2314/4605] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:12 2022] POST /identity/v3/projects => generated 389 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:12.725961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d013b3-d0e4-4288-8494-0feeb0b6cb5c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:12.741298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d013b3-d0e4-4288-8494-0feeb0b6cb5c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:12.880806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaad92d-455a-4dab-9590-57979dd73e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:12.882641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaad92d-455a-4dab-9590-57979dd73e97 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:12.883328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaad92d-455a-4dab-9590-57979dd73e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:12.883830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaad92d-455a-4dab-9590-57979dd73e97 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:12.884821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaad92d-455a-4dab-9590-57979dd73e97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:12.896932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaad92d-455a-4dab-9590-57979dd73e97 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:13.054409 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2292/4606] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:12 2022] POST /identity/v3/users => generated 295 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:13.058037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e5a45-ba40-4037-9bb9-ab60adbf4c62 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:13.088979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02e5a45-ba40-4037-9bb9-ab60adbf4c62 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:13.108307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bbbbc-b115-4d7f-a8db-261f1c445f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:13.110272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bbbbc-b115-4d7f-a8db-261f1c445f8e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:13.110272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bbbbc-b115-4d7f-a8db-261f1c445f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:13.110272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bbbbc-b115-4d7f-a8db-261f1c445f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:13.113441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bbbbc-b115-4d7f-a8db-261f1c445f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:13.123619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bbbbc-b115-4d7f-a8db-261f1c445f8e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:13.161758 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2315/4607] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:13.163054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaad92d-455a-4dab-9590-57979dd73e97 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:13.180617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaad92d-455a-4dab-9590-57979dd73e97 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:13.196230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757638ff-8ab4-42e8-aa96-f84ee1f76e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:13.198794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757638ff-8ab4-42e8-aa96-f84ee1f76e02 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:13.198794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757638ff-8ab4-42e8-aa96-f84ee1f76e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:13.199939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757638ff-8ab4-42e8-aa96-f84ee1f76e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:13.202183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757638ff-8ab4-42e8-aa96-f84ee1f76e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:13.206691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757638ff-8ab4-42e8-aa96-f84ee1f76e02 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:13.239958 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2293/4608] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:13.246565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bbbbc-b115-4d7f-a8db-261f1c445f8e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:13.252205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bbbbc-b115-4d7f-a8db-261f1c445f8e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:13.269423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e2e8c-8d99-4eff-998d-903b249a2060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:13.270386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e2e8c-8d99-4eff-998d-903b249a2060 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:13.270709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e2e8c-8d99-4eff-998d-903b249a2060 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:13.274633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e2e8c-8d99-4eff-998d-903b249a2060 admin admin] PATH_INFO: `/v3/projects/7b5ffd4a084b4a06afba28fa52ed1bfd/users/0d804740fcf9430d89608ea1fe60dd16/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:13.312637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e2e8c-8d99-4eff-998d-903b249a2060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b5ffd4a084b4a06afba28fa52ed1bfd, user_id=0d804740fcf9430d89608ea1fe60dd16, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:13.316016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e2e8c-8d99-4eff-998d-903b249a2060 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:13.442115 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2316/4609] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:13 2022] PUT /identity/v3/projects/7b5ffd4a084b4a06afba28fa52ed1bfd/users/0d804740fcf9430d89608ea1fe60dd16/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:13.453112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a8af3-029c-48d6-b2e4-a84e400f9aa8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:13.453701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a8af3-029c-48d6-b2e4-a84e400f9aa8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:13.454195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a8af3-029c-48d6-b2e4-a84e400f9aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:13.501060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c67a8af3-029c-48d6-b2e4-a84e400f9aa8 None None] MFA Rules not processed for user `0d804740fcf9430d89608ea1fe60dd16`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:13.505867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c67a8af3-029c-48d6-b2e4-a84e400f9aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:13.617713 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2294/4610] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:13.625092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e2e8c-8d99-4eff-998d-903b249a2060 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:13.774897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:13.808724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e2e8c-8d99-4eff-998d-903b249a2060 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:13.829304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9d38c-29b4-4b48-afac-31cf7417f272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:13.830175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9d38c-29b4-4b48-afac-31cf7417f272 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:13.830724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9d38c-29b4-4b48-afac-31cf7417f272 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:13.831550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a9d38c-29b4-4b48-afac-31cf7417f272 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:13.832281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9d38c-29b4-4b48-afac-31cf7417f272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:13.837465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a9d38c-29b4-4b48-afac-31cf7417f272 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:13.997776 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2317/4611] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:13 2022] POST /identity/v3/projects => generated 385 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:14.001798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a8af3-029c-48d6-b2e4-a84e400f9aa8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:14.019189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67a8af3-029c-48d6-b2e4-a84e400f9aa8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:14.167068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02880862-ee0e-4f9c-9dfe-1d24551ffd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:14.167765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02880862-ee0e-4f9c-9dfe-1d24551ffd4f admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:14.168061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02880862-ee0e-4f9c-9dfe-1d24551ffd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:14.168358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02880862-ee0e-4f9c-9dfe-1d24551ffd4f admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:14.172679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02880862-ee0e-4f9c-9dfe-1d24551ffd4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:14.177578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02880862-ee0e-4f9c-9dfe-1d24551ffd4f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:14.303621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9d38c-29b4-4b48-afac-31cf7417f272 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:14.316858 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2295/4612] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:14 2022] POST /identity/v3/users => generated 299 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:14.321550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a9d38c-29b4-4b48-afac-31cf7417f272 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:14.334911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bef24-480c-4ca8-97c7-d2480fea075e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:14.336549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bef24-480c-4ca8-97c7-d2480fea075e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:14.336899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bef24-480c-4ca8-97c7-d2480fea075e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:14.337244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bef24-480c-4ca8-97c7-d2480fea075e admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:14.338185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bef24-480c-4ca8-97c7-d2480fea075e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:14.339907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bef24-480c-4ca8-97c7-d2480fea075e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:14.376526 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2318/4613] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:14.377968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02880862-ee0e-4f9c-9dfe-1d24551ffd4f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:14.394201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02880862-ee0e-4f9c-9dfe-1d24551ffd4f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:14.407424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b14f6-ee69-4e5a-bef1-280e276e0b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:14.411050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b14f6-ee69-4e5a-bef1-280e276e0b20 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:14.411737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b14f6-ee69-4e5a-bef1-280e276e0b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:14.412259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707b14f6-ee69-4e5a-bef1-280e276e0b20 admin admin] PATH_INFO: `/v3/projects/d0841ad330f7484295fbe1003e08dfa2/users/0a4c448b288644d8bf739bd003360e9a/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:14.457336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b14f6-ee69-4e5a-bef1-280e276e0b20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0841ad330f7484295fbe1003e08dfa2, user_id=0a4c448b288644d8bf739bd003360e9a, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:14.464972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707b14f6-ee69-4e5a-bef1-280e276e0b20 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:14.557219 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2296/4614] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:14 2022] PUT /identity/v3/projects/d0841ad330f7484295fbe1003e08dfa2/users/0a4c448b288644d8bf739bd003360e9a/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:14.559643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bef24-480c-4ca8-97c7-d2480fea075e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:14.572990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bef24-480c-4ca8-97c7-d2480fea075e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:14.745554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe200d4-a41a-4949-993a-20b68ddec862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:14.746334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe200d4-a41a-4949-993a-20b68ddec862 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:14.746695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe200d4-a41a-4949-993a-20b68ddec862 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:14.747049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe200d4-a41a-4949-993a-20b68ddec862 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:14.747893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe200d4-a41a-4949-993a-20b68ddec862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:14.749601 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe200d4-a41a-4949-993a-20b68ddec862 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:14.797870 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2319/4615] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:14 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:14.798214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b14f6-ee69-4e5a-bef1-280e276e0b20 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:14.812112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707b14f6-ee69-4e5a-bef1-280e276e0b20 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:14.823888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9aaa2d-d62b-4489-9f57-571d76b62e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:14.825250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9aaa2d-d62b-4489-9f57-571d76b62e47 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:14.825250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9aaa2d-d62b-4489-9f57-571d76b62e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:14.825545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9aaa2d-d62b-4489-9f57-571d76b62e47 admin admin] PATH_INFO: `/v3/projects/d0841ad330f7484295fbe1003e08dfa2/users/0a4c448b288644d8bf739bd003360e9a/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:14.827875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9aaa2d-d62b-4489-9f57-571d76b62e47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0841ad330f7484295fbe1003e08dfa2, user_id=0a4c448b288644d8bf739bd003360e9a, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:14.831667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9aaa2d-d62b-4489-9f57-571d76b62e47 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:14.863144 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2297/4616] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:14 2022] PUT /identity/v3/projects/d0841ad330f7484295fbe1003e08dfa2/users/0a4c448b288644d8bf739bd003360e9a/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:18:14.873787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221436d-5325-40db-99f7-6c594e2d8209 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:14.873981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221436d-5325-40db-99f7-6c594e2d8209 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:14.874480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5221436d-5325-40db-99f7-6c594e2d8209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:14.949015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5221436d-5325-40db-99f7-6c594e2d8209 None None] MFA Rules not processed for user `0a4c448b288644d8bf739bd003360e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:14.950405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5221436d-5325-40db-99f7-6c594e2d8209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:15.066051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:15.137477 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2320/4617] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:15.137584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9aaa2d-d62b-4489-9f57-571d76b62e47 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:15.158476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9aaa2d-d62b-4489-9f57-571d76b62e47 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:15.164239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d38732-5e8b-4488-930f-84683b69eca5 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4c448b288644d8bf739bd003360e9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0841ad330f7484295fbe1003e08dfa2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:15.169998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d38732-5e8b-4488-930f-84683b69eca5 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:15.170498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d38732-5e8b-4488-930f-84683b69eca5 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:15.171506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d38732-5e8b-4488-930f-84683b69eca5 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] PATH_INFO: `/v3/projects/7b5ffd4a084b4a06afba28fa52ed1bfd` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:15.172924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d38732-5e8b-4488-930f-84683b69eca5 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorizing `identity:get_project(project_id=7b5ffd4a084b4a06afba28fa52ed1bfd)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:15.176410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d38732-5e8b-4488-930f-84683b69eca5 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:15.182645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221436d-5325-40db-99f7-6c594e2d8209 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:15.196624 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2298/4618] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:18:15 2022] GET /identity/v3/projects/7b5ffd4a084b4a06afba28fa52ed1bfd => generated 389 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:18:15.199209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5221436d-5325-40db-99f7-6c594e2d8209 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:15.206046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ac458-aacc-4635-9dfd-25c52962ee63 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4c448b288644d8bf739bd003360e9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0841ad330f7484295fbe1003e08dfa2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:15.206046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ac458-aacc-4635-9dfd-25c52962ee63 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:15.206046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ac458-aacc-4635-9dfd-25c52962ee63 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:15.206732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ac458-aacc-4635-9dfd-25c52962ee63 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:15.207643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ac458-aacc-4635-9dfd-25c52962ee63 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:15.210092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05ac458-aacc-4635-9dfd-25c52962ee63 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:15.341379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d38732-5e8b-4488-930f-84683b69eca5 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:15.350018 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2321/4619] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:18:15 2022] POST /identity/v3/projects => generated 338 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:18:15.377604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d38732-5e8b-4488-930f-84683b69eca5 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:15.602457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67513d-e751-4025-93fe-f28660d80db0 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4c448b288644d8bf739bd003360e9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0841ad330f7484295fbe1003e08dfa2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:15.613072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67513d-e751-4025-93fe-f28660d80db0 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:15.613072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:15.613072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67513d-e751-4025-93fe-f28660d80db0 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:15.613072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a67513d-e751-4025-93fe-f28660d80db0 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:15.613865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67513d-e751-4025-93fe-f28660d80db0 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:15.615829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a67513d-e751-4025-93fe-f28660d80db0 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:15.758209 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2299/4620] 10.222.0.33 () {58 vars in 1130 bytes} [Sat May 28 21:18:15 2022] GET /identity/v3/projects => generated 6934 bytes in 417 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:15.762570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ac458-aacc-4635-9dfd-25c52962ee63 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:15.794706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05ac458-aacc-4635-9dfd-25c52962ee63 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:15.806515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02b5b9-f51b-4076-b67c-effaf5fbce85 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4c448b288644d8bf739bd003360e9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0841ad330f7484295fbe1003e08dfa2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:15.807523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02b5b9-f51b-4076-b67c-effaf5fbce85 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:15.808031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02b5b9-f51b-4076-b67c-effaf5fbce85 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:15.808381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02b5b9-f51b-4076-b67c-effaf5fbce85 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] PATH_INFO: `/v3/projects/7b5ffd4a084b4a06afba28fa52ed1bfd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:15.809714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02b5b9-f51b-4076-b67c-effaf5fbce85 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorizing `identity:get_project(project_id=7b5ffd4a084b4a06afba28fa52ed1bfd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:15.819181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02b5b9-f51b-4076-b67c-effaf5fbce85 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:15.830006 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2322/4621] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:18:15 2022] GET /identity/v3/projects/7b5ffd4a084b4a06afba28fa52ed1bfd => generated 389 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:18:15.832744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67513d-e751-4025-93fe-f28660d80db0 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:15.847500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a67513d-e751-4025-93fe-f28660d80db0 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:15.858928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b672f3-1b5c-4bca-9bd6-fc4f7cc21f72 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4c448b288644d8bf739bd003360e9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0841ad330f7484295fbe1003e08dfa2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:15.859831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b672f3-1b5c-4bca-9bd6-fc4f7cc21f72 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:15.860122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b672f3-1b5c-4bca-9bd6-fc4f7cc21f72 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:15.860403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b672f3-1b5c-4bca-9bd6-fc4f7cc21f72 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] PATH_INFO: `/v3/projects/030632a29c8647eeb06ba179d64d72a7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:15.864670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b672f3-1b5c-4bca-9bd6-fc4f7cc21f72 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorizing `identity:get_project(project_id=030632a29c8647eeb06ba179d64d72a7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:15.878631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b672f3-1b5c-4bca-9bd6-fc4f7cc21f72 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:15.888300 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2300/4622] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:18:15 2022] GET /identity/v3/projects/030632a29c8647eeb06ba179d64d72a7 => generated 338 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:18:15.893647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02b5b9-f51b-4076-b67c-effaf5fbce85 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:15.910608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02b5b9-f51b-4076-b67c-effaf5fbce85 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:15.919451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f10371-830d-45e4-847e-df204967af74 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4c448b288644d8bf739bd003360e9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0841ad330f7484295fbe1003e08dfa2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:15.920400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f10371-830d-45e4-847e-df204967af74 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:15.920846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f10371-830d-45e4-847e-df204967af74 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:15.921399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f10371-830d-45e4-847e-df204967af74 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:15.922373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f10371-830d-45e4-847e-df204967af74 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:15.923592 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-c7f10371-830d-45e4-847e-df204967af74 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:18:15.923592 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:18:15.923592 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:18:15.926038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f10371-830d-45e4-847e-df204967af74 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:15.963272 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2323/4623] 10.222.0.33 () {58 vars in 1130 bytes} [Sat May 28 21:18:15 2022] GET /identity/v3/projects => generated 6934 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:18:15.967956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b672f3-1b5c-4bca-9bd6-fc4f7cc21f72 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:15.982171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b672f3-1b5c-4bca-9bd6-fc4f7cc21f72 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:16.016568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8a5ec-14b0-4c10-b51f-8154f4be157a tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4c448b288644d8bf739bd003360e9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0841ad330f7484295fbe1003e08dfa2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:16.017280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8a5ec-14b0-4c10-b51f-8154f4be157a tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:16.017573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8a5ec-14b0-4c10-b51f-8154f4be157a tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:16.017881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b8a5ec-14b0-4c10-b51f-8154f4be157a tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:16.018730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8a5ec-14b0-4c10-b51f-8154f4be157a tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:18:16.019053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8a5ec-14b0-4c10-b51f-8154f4be157a tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:16.020985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b8a5ec-14b0-4c10-b51f-8154f4be157a tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:16.080455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f10371-830d-45e4-847e-df204967af74 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:16.081076 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2301/4624] 10.222.0.33 () {58 vars in 1165 bytes} [Sat May 28 21:18:15 2022] GET /identity/v3/projects?domain_id=default => generated 6603 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:16.094511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f10371-830d-45e4-847e-df204967af74 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:16.116810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47d7d7-1ab2-43d2-97a5-c3b6c51a3ce6 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4c448b288644d8bf739bd003360e9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0841ad330f7484295fbe1003e08dfa2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:16.117476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47d7d7-1ab2-43d2-97a5-c3b6c51a3ce6 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:16.117771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47d7d7-1ab2-43d2-97a5-c3b6c51a3ce6 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:16.118101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47d7d7-1ab2-43d2-97a5-c3b6c51a3ce6 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:16.118849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47d7d7-1ab2-43d2-97a5-c3b6c51a3ce6 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:16.120953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47d7d7-1ab2-43d2-97a5-c3b6c51a3ce6 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:16.159979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8a5ec-14b0-4c10-b51f-8154f4be157a tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:16.178128 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2324/4625] 10.222.0.33 () {58 vars in 1130 bytes} [Sat May 28 21:18:16 2022] GET /identity/v3/projects => generated 6934 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:18:16.183286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b8a5ec-14b0-4c10-b51f-8154f4be157a tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:16.206702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e672a-d946-43c0-82bc-5e0a17d25f03 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4c448b288644d8bf739bd003360e9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0841ad330f7484295fbe1003e08dfa2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:16.207395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e672a-d946-43c0-82bc-5e0a17d25f03 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:16.207678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e672a-d946-43c0-82bc-5e0a17d25f03 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:16.207965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e672a-d946-43c0-82bc-5e0a17d25f03 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:16.208742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e672a-d946-43c0-82bc-5e0a17d25f03 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:18:16.209307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e672a-d946-43c0-82bc-5e0a17d25f03 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:16.211420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e672a-d946-43c0-82bc-5e0a17d25f03 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:16.273398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47d7d7-1ab2-43d2-97a5-c3b6c51a3ce6 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:16.287007 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2302/4626] 10.222.0.33 () {58 vars in 1165 bytes} [Sat May 28 21:18:16 2022] GET /identity/v3/projects?domain_id=default => generated 6603 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:18:16.292433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47d7d7-1ab2-43d2-97a5-c3b6c51a3ce6 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:16.303587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff184ed4-0c46-4dbd-a8d9-a1364409da36 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4c448b288644d8bf739bd003360e9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0841ad330f7484295fbe1003e08dfa2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:16.305256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff184ed4-0c46-4dbd-a8d9-a1364409da36 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:16.305714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff184ed4-0c46-4dbd-a8d9-a1364409da36 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:16.310238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff184ed4-0c46-4dbd-a8d9-a1364409da36 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:16.311231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff184ed4-0c46-4dbd-a8d9-a1364409da36 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:16.317241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff184ed4-0c46-4dbd-a8d9-a1364409da36 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:16.366324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e672a-d946-43c0-82bc-5e0a17d25f03 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:16.373234 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2325/4627] 10.222.0.33 () {58 vars in 1130 bytes} [Sat May 28 21:18:16 2022] GET /identity/v3/projects => generated 6934 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:16.378246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e672a-d946-43c0-82bc-5e0a17d25f03 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:16.393951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc74cb3-7743-475d-a07f-941317878ae1 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4c448b288644d8bf739bd003360e9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0841ad330f7484295fbe1003e08dfa2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:16.395002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc74cb3-7743-475d-a07f-941317878ae1 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:16.395457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc74cb3-7743-475d-a07f-941317878ae1 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:16.395913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc74cb3-7743-475d-a07f-941317878ae1 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:16.397514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc74cb3-7743-475d-a07f-941317878ae1 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Adding query filter params (name=tempest-ListProjectsStaticTestJSON-1455867090) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:18:16.398039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc74cb3-7743-475d-a07f-941317878ae1 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:16.400158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc74cb3-7743-475d-a07f-941317878ae1 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:16.440730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff184ed4-0c46-4dbd-a8d9-a1364409da36 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:16.451526 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2303/4628] 10.222.0.33 () {58 vars in 1231 bytes} [Sat May 28 21:18:16 2022] GET /identity/v3/projects?name=tempest-ListProjectsStaticTestJSON-1455867090 => generated 537 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:18:16.453663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff184ed4-0c46-4dbd-a8d9-a1364409da36 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:16.461084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf3b70-504f-4e1c-96f7-b4604702c0b7 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a4c448b288644d8bf739bd003360e9a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0841ad330f7484295fbe1003e08dfa2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:16.462061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf3b70-504f-4e1c-96f7-b4604702c0b7 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:16.462355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf3b70-504f-4e1c-96f7-b4604702c0b7 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:16.463304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf3b70-504f-4e1c-96f7-b4604702c0b7 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] PATH_INFO: `/v3/projects/030632a29c8647eeb06ba179d64d72a7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:16.464660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf3b70-504f-4e1c-96f7-b4604702c0b7 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorizing `identity:delete_project(project_id=030632a29c8647eeb06ba179d64d72a7)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:16.466810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf3b70-504f-4e1c-96f7-b4604702c0b7 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:16.611237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:16.627740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a6cf3b70-504f-4e1c-96f7-b4604702c0b7 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '030632a29c8647eeb06ba179d64d72a7', 'request_id': 'req-a6cf3b70-504f-4e1c-96f7-b4604702c0b7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:16.651956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a6cf3b70-504f-4e1c-96f7-b4604702c0b7 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '030632a29c8647eeb06ba179d64d72a7', 'request_id': 'req-a6cf3b70-504f-4e1c-96f7-b4604702c0b7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:16.745880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a6cf3b70-504f-4e1c-96f7-b4604702c0b7 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '030632a29c8647eeb06ba179d64d72a7', 'request_id': 'req-a6cf3b70-504f-4e1c-96f7-b4604702c0b7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:16.775071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a6cf3b70-504f-4e1c-96f7-b4604702c0b7 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] The token cache is being invalidate because project 030632a29c8647eeb06ba179d64d72a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:16.780030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a6cf3b70-504f-4e1c-96f7-b4604702c0b7 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:16.782218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a6cf3b70-504f-4e1c-96f7-b4604702c0b7 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:16.786304 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2326/4629] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:16 2022] DELETE /identity/v3/projects/030632a29c8647eeb06ba179d64d72a7 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:16.790247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc74cb3-7743-475d-a07f-941317878ae1 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:16.796412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6dc74cb3-7743-475d-a07f-941317878ae1 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:16.967913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc74cb3-7743-475d-a07f-941317878ae1 tempest-ListProjectsStaticTestJSON-88803209 tempest-ListProjectsStaticTestJSON-88803209-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:17.096364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:17.151181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605c8f1-1ce2-447c-9a67-fbdce0a55529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:17.152695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605c8f1-1ce2-447c-9a67-fbdce0a55529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:17.155163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605c8f1-1ce2-447c-9a67-fbdce0a55529 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:17.155534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605c8f1-1ce2-447c-9a67-fbdce0a55529 admin admin] PATH_INFO: `/v3/users/0d804740fcf9430d89608ea1fe60dd16` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:17.157727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605c8f1-1ce2-447c-9a67-fbdce0a55529 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d804740fcf9430d89608ea1fe60dd16)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:17.164209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605c8f1-1ce2-447c-9a67-fbdce0a55529 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:17.461935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a605c8f1-1ce2-447c-9a67-fbdce0a55529 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d804740fcf9430d89608ea1fe60dd16'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:17.466140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a605c8f1-1ce2-447c-9a67-fbdce0a55529 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d804740fcf9430d89608ea1fe60dd16'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:17.505285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a605c8f1-1ce2-447c-9a67-fbdce0a55529 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d804740fcf9430d89608ea1fe60dd16'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:17.583449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a605c8f1-1ce2-447c-9a67-fbdce0a55529 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d804740fcf9430d89608ea1fe60dd16'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:17.584203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a605c8f1-1ce2-447c-9a67-fbdce0a55529 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d804740fcf9430d89608ea1fe60dd16'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:17.626805 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2304/4630] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:16 2022] DELETE /identity/v3/users/0d804740fcf9430d89608ea1fe60dd16 => generated 0 bytes in 836 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:17.635029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d9799-2c00-47dc-948e-7db01bae3ee2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:17.635594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d9799-2c00-47dc-948e-7db01bae3ee2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:17.635594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d9799-2c00-47dc-948e-7db01bae3ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:17.683848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ea6d9799-2c00-47dc-948e-7db01bae3ee2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:17.686456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ea6d9799-2c00-47dc-948e-7db01bae3ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:17.822077 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2327/4631] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:17.866795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605c8f1-1ce2-447c-9a67-fbdce0a55529 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:17.869490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a605c8f1-1ce2-447c-9a67-fbdce0a55529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:18.247160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605c8f1-1ce2-447c-9a67-fbdce0a55529 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:18.251135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26831a42-7c9f-4814-85bd-7a12be186971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:18.252763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26831a42-7c9f-4814-85bd-7a12be186971 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:18.253226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26831a42-7c9f-4814-85bd-7a12be186971 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:18.253665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26831a42-7c9f-4814-85bd-7a12be186971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:18.266545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26831a42-7c9f-4814-85bd-7a12be186971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:18.271569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26831a42-7c9f-4814-85bd-7a12be186971 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:18.281972 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2305/4632] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:18.494433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:20.248926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:20.265983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d9799-2c00-47dc-948e-7db01bae3ee2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:20.267825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ea6d9799-2c00-47dc-948e-7db01bae3ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:20.289928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d9799-2c00-47dc-948e-7db01bae3ee2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:20.300384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:20.301276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:20.301562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:20.301862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139 admin admin] PATH_INFO: `/v3/projects/7b5ffd4a084b4a06afba28fa52ed1bfd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:20.306798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b5ffd4a084b4a06afba28fa52ed1bfd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:20.310092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:20.520842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b5ffd4a084b4a06afba28fa52ed1bfd', 'request_id': 'req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:20.527945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b5ffd4a084b4a06afba28fa52ed1bfd', 'request_id': 'req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:20.637182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b5ffd4a084b4a06afba28fa52ed1bfd', 'request_id': 'req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:20.651469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139 admin admin] The token cache is being invalidate because project 7b5ffd4a084b4a06afba28fa52ed1bfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:20.652221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:20.653230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:20.655714 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2328/4633] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:20 2022] DELETE /identity/v3/projects/7b5ffd4a084b4a06afba28fa52ed1bfd => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:20.660349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26831a42-7c9f-4814-85bd-7a12be186971 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:20.673265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26831a42-7c9f-4814-85bd-7a12be186971 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:20.741523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3670de-b782-4cf0-9e1b-caed72101752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:20.742471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3670de-b782-4cf0-9e1b-caed72101752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:20.742888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3670de-b782-4cf0-9e1b-caed72101752 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:20.743167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3670de-b782-4cf0-9e1b-caed72101752 admin admin] PATH_INFO: `/v3/users/0a4c448b288644d8bf739bd003360e9a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:20.745279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3670de-b782-4cf0-9e1b-caed72101752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a4c448b288644d8bf739bd003360e9a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:20.747334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3670de-b782-4cf0-9e1b-caed72101752 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:20.749485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:20.930048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ba3670de-b782-4cf0-9e1b-caed72101752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a4c448b288644d8bf739bd003360e9a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:20.931111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ba3670de-b782-4cf0-9e1b-caed72101752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a4c448b288644d8bf739bd003360e9a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:20.942675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ba3670de-b782-4cf0-9e1b-caed72101752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a4c448b288644d8bf739bd003360e9a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:21.054023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ba3670de-b782-4cf0-9e1b-caed72101752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a4c448b288644d8bf739bd003360e9a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:21.054910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ba3670de-b782-4cf0-9e1b-caed72101752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a4c448b288644d8bf739bd003360e9a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:21.089950 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2306/4634] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:20 2022] DELETE /identity/v3/users/0a4c448b288644d8bf739bd003360e9a => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:22.191372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:22.193087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:22.225443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b91ba6-8d3d-49a2-abf2-4ed99a75d139 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:22.417380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275b69c-ab9f-4acf-a47b-84c1ee270038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:22.418616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275b69c-ab9f-4acf-a47b-84c1ee270038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:22.419089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275b69c-ab9f-4acf-a47b-84c1ee270038 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:22.419534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275b69c-ab9f-4acf-a47b-84c1ee270038 admin admin] PATH_INFO: `/v3/projects/d0841ad330f7484295fbe1003e08dfa2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:22.421226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275b69c-ab9f-4acf-a47b-84c1ee270038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0841ad330f7484295fbe1003e08dfa2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:22.427427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275b69c-ab9f-4acf-a47b-84c1ee270038 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:22.595794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4275b69c-ab9f-4acf-a47b-84c1ee270038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0841ad330f7484295fbe1003e08dfa2', 'request_id': 'req-4275b69c-ab9f-4acf-a47b-84c1ee270038'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:22.610950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4275b69c-ab9f-4acf-a47b-84c1ee270038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0841ad330f7484295fbe1003e08dfa2', 'request_id': 'req-4275b69c-ab9f-4acf-a47b-84c1ee270038'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:22.776368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4275b69c-ab9f-4acf-a47b-84c1ee270038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0841ad330f7484295fbe1003e08dfa2', 'request_id': 'req-4275b69c-ab9f-4acf-a47b-84c1ee270038'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:22.807700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4275b69c-ab9f-4acf-a47b-84c1ee270038 admin admin] The token cache is being invalidate because project d0841ad330f7484295fbe1003e08dfa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:22.808237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4275b69c-ab9f-4acf-a47b-84c1ee270038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:22.808961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4275b69c-ab9f-4acf-a47b-84c1ee270038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:22.819074 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2329/4635] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:22 2022] DELETE /identity/v3/projects/d0841ad330f7484295fbe1003e08dfa2 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:22.820545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd241b-09f4-48d1-8acd-b23df9b9be5a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:22.821024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd241b-09f4-48d1-8acd-b23df9b9be5a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:22.821115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd241b-09f4-48d1-8acd-b23df9b9be5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:22.876719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-77cd241b-09f4-48d1-8acd-b23df9b9be5a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:22.884211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-77cd241b-09f4-48d1-8acd-b23df9b9be5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:22.959911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:23.074111 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2307/4636] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:18:23.147844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665027a-473b-4cb8-8a73-bdd75abafdd2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:23.149703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665027a-473b-4cb8-8a73-bdd75abafdd2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:23.150175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665027a-473b-4cb8-8a73-bdd75abafdd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:23.187531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0665027a-473b-4cb8-8a73-bdd75abafdd2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:23.196630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0665027a-473b-4cb8-8a73-bdd75abafdd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:23.219396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd241b-09f4-48d1-8acd-b23df9b9be5a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:23.220239 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2330/4637] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:23.238591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd241b-09f4-48d1-8acd-b23df9b9be5a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:23.247696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53b6e5-8e54-49b0-b843-5e2e591af454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:23.248391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53b6e5-8e54-49b0-b843-5e2e591af454 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:23.248680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53b6e5-8e54-49b0-b843-5e2e591af454 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:23.248961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53b6e5-8e54-49b0-b843-5e2e591af454 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:23.249703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f53b6e5-8e54-49b0-b843-5e2e591af454 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:18:23.250053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f53b6e5-8e54-49b0-b843-5e2e591af454 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:23.253681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f53b6e5-8e54-49b0-b843-5e2e591af454 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:23.289308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665027a-473b-4cb8-8a73-bdd75abafdd2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:23.293644 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2308/4638] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:18:23.299041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665027a-473b-4cb8-8a73-bdd75abafdd2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:23.318431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9f9cd-dda8-4bc5-8459-c7c76981442a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:23.319167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9f9cd-dda8-4bc5-8459-c7c76981442a admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:23.319459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9f9cd-dda8-4bc5-8459-c7c76981442a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:23.319739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9f9cd-dda8-4bc5-8459-c7c76981442a admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:23.320449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9f9cd-dda8-4bc5-8459-c7c76981442a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:23.322800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9f9cd-dda8-4bc5-8459-c7c76981442a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:23.449151 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2331/4639] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:23 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:23.453473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53b6e5-8e54-49b0-b843-5e2e591af454 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:23.469315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53b6e5-8e54-49b0-b843-5e2e591af454 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:23.615591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d36d4-504d-4973-ac7c-1855ea993ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:23.616460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d36d4-504d-4973-ac7c-1855ea993ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:23.619775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d36d4-504d-4973-ac7c-1855ea993ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:23.620130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d36d4-504d-4973-ac7c-1855ea993ca2 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:23.621018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d36d4-504d-4973-ac7c-1855ea993ca2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:23.626273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752d36d4-504d-4973-ac7c-1855ea993ca2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:23.657561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:23.746564 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2309/4640] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:23 2022] POST /identity/v3/users => generated 289 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:23.750499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9f9cd-dda8-4bc5-8459-c7c76981442a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:23.775842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9f9cd-dda8-4bc5-8459-c7c76981442a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:23.791941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa07126-5b20-45fc-893f-220cfa99d5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:23.792974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa07126-5b20-45fc-893f-220cfa99d5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:23.793422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa07126-5b20-45fc-893f-220cfa99d5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:23.793880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa07126-5b20-45fc-893f-220cfa99d5b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:23.794799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa07126-5b20-45fc-893f-220cfa99d5b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:23.796593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa07126-5b20-45fc-893f-220cfa99d5b7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:23.830407 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2332/4641] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:23.831483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d36d4-504d-4973-ac7c-1855ea993ca2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:23.846110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d36d4-504d-4973-ac7c-1855ea993ca2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:23.866277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4548d2-fa82-46f2-a7e2-e5bcdd2e4975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:23.867298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4548d2-fa82-46f2-a7e2-e5bcdd2e4975 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:23.868188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4548d2-fa82-46f2-a7e2-e5bcdd2e4975 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:23.869037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4548d2-fa82-46f2-a7e2-e5bcdd2e4975 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:23.870569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4548d2-fa82-46f2-a7e2-e5bcdd2e4975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:23.873427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4548d2-fa82-46f2-a7e2-e5bcdd2e4975 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:23.906388 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2310/4642] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:23 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:23.910795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa07126-5b20-45fc-893f-220cfa99d5b7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:23.921913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa07126-5b20-45fc-893f-220cfa99d5b7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:23.942223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc022ef3-2e79-45d6-afa9-57e4e6ce0508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:23.943272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc022ef3-2e79-45d6-afa9-57e4e6ce0508 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:23.943562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc022ef3-2e79-45d6-afa9-57e4e6ce0508 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:23.943927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc022ef3-2e79-45d6-afa9-57e4e6ce0508 admin admin] PATH_INFO: `/v3/projects/a6e390d382eb4bf0ab42115523c4069e/users/4764d45193c5439982d77416273cfbca/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:23.979516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc022ef3-2e79-45d6-afa9-57e4e6ce0508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e390d382eb4bf0ab42115523c4069e, user_id=4764d45193c5439982d77416273cfbca, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:23.983940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc022ef3-2e79-45d6-afa9-57e4e6ce0508 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:24.099189 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2333/4643] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:23 2022] PUT /identity/v3/projects/a6e390d382eb4bf0ab42115523c4069e/users/4764d45193c5439982d77416273cfbca/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:24.110406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3904c5-989a-44a6-9631-ec5207dcc673 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:24.119992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3904c5-989a-44a6-9631-ec5207dcc673 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:24.120621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3904c5-989a-44a6-9631-ec5207dcc673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:24.182869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0f3904c5-989a-44a6-9631-ec5207dcc673 None None] MFA Rules not processed for user `4764d45193c5439982d77416273cfbca`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:24.184499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0f3904c5-989a-44a6-9631-ec5207dcc673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:24.299478 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2311/4644] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:24.300617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc022ef3-2e79-45d6-afa9-57e4e6ce0508 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:24.413369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3904c5-989a-44a6-9631-ec5207dcc673 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:24.419525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0f3904c5-989a-44a6-9631-ec5207dcc673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:24.467844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc022ef3-2e79-45d6-afa9-57e4e6ce0508 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:24.481887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edc05b-1bb9-4249-9117-4256772cf820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:24.483630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edc05b-1bb9-4249-9117-4256772cf820 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:24.483928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edc05b-1bb9-4249-9117-4256772cf820 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:24.484214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2edc05b-1bb9-4249-9117-4256772cf820 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:24.488450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2edc05b-1bb9-4249-9117-4256772cf820 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:24.488450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2edc05b-1bb9-4249-9117-4256772cf820 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:24.544059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3904c5-989a-44a6-9631-ec5207dcc673 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:24.559639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693cf76-058e-4262-8e69-25be7e5d7f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:24.560695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693cf76-058e-4262-8e69-25be7e5d7f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:24.561288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693cf76-058e-4262-8e69-25be7e5d7f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:24.561943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f693cf76-058e-4262-8e69-25be7e5d7f2c admin admin] PATH_INFO: `/v3/users/94f224e3fd2142d0991cb173e2a2c880` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:24.563643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693cf76-058e-4262-8e69-25be7e5d7f2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=94f224e3fd2142d0991cb173e2a2c880)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:24.582933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f693cf76-058e-4262-8e69-25be7e5d7f2c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:24.607137 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2334/4645] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:24 2022] POST /identity/v3/projects => generated 377 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:24.612787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edc05b-1bb9-4249-9117-4256772cf820 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:24.748135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2edc05b-1bb9-4249-9117-4256772cf820 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:24.756276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89511826-fddb-49fd-b9e5-ce5c9b94eba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:24.758591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89511826-fddb-49fd-b9e5-ce5c9b94eba3 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:24.760335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89511826-fddb-49fd-b9e5-ce5c9b94eba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:24.760335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89511826-fddb-49fd-b9e5-ce5c9b94eba3 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:24.760335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89511826-fddb-49fd-b9e5-ce5c9b94eba3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:24.763947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89511826-fddb-49fd-b9e5-ce5c9b94eba3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:24.828956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f693cf76-058e-4262-8e69-25be7e5d7f2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94f224e3fd2142d0991cb173e2a2c880'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:24.831161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f693cf76-058e-4262-8e69-25be7e5d7f2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94f224e3fd2142d0991cb173e2a2c880'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:24.846503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f693cf76-058e-4262-8e69-25be7e5d7f2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94f224e3fd2142d0991cb173e2a2c880'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:24.925893 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2335/4646] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:24 2022] POST /identity/v3/users => generated 295 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:24.927072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89511826-fddb-49fd-b9e5-ce5c9b94eba3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:24.954246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89511826-fddb-49fd-b9e5-ce5c9b94eba3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:24.969137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f693cf76-058e-4262-8e69-25be7e5d7f2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94f224e3fd2142d0991cb173e2a2c880'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:24.969903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f693cf76-058e-4262-8e69-25be7e5d7f2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94f224e3fd2142d0991cb173e2a2c880'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:24.976337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa23f17-ecfc-4d49-84f5-7c92d54ed6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:24.977718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa23f17-ecfc-4d49-84f5-7c92d54ed6ce admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:24.978792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa23f17-ecfc-4d49-84f5-7c92d54ed6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:24.979131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa23f17-ecfc-4d49-84f5-7c92d54ed6ce admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:24.980097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa23f17-ecfc-4d49-84f5-7c92d54ed6ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:24.982964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa23f17-ecfc-4d49-84f5-7c92d54ed6ce admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:25.004576 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2336/4647] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:25.005982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa23f17-ecfc-4d49-84f5-7c92d54ed6ce admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:25.009020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8aa23f17-ecfc-4d49-84f5-7c92d54ed6ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:25.016303 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2312/4648] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:24 2022] DELETE /identity/v3/users/94f224e3fd2142d0991cb173e2a2c880 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:25.040167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa23f17-ecfc-4d49-84f5-7c92d54ed6ce admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:25.175511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f8eb1-8dc4-4e4d-b5fa-11244d45877c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:25.177865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f8eb1-8dc4-4e4d-b5fa-11244d45877c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:25.178379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f8eb1-8dc4-4e4d-b5fa-11244d45877c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:25.178966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f8eb1-8dc4-4e4d-b5fa-11244d45877c admin admin] PATH_INFO: `/v3/projects/074257b478dc498b9a5835210f0a5511/users/29f7f6e2acb5458096efb6586c7a3de2/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:25.222948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8f8eb1-8dc4-4e4d-b5fa-11244d45877c admin admin] RBAC: Authorizing `identity:create_grant(project_id=074257b478dc498b9a5835210f0a5511, user_id=29f7f6e2acb5458096efb6586c7a3de2, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:25.238796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8f8eb1-8dc4-4e4d-b5fa-11244d45877c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:25.358549 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2337/4649] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:25 2022] PUT /identity/v3/projects/074257b478dc498b9a5835210f0a5511/users/29f7f6e2acb5458096efb6586c7a3de2/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:25.364377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693cf76-058e-4262-8e69-25be7e5d7f2c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:25.384718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f693cf76-058e-4262-8e69-25be7e5d7f2c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:25.387986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f8eb1-8dc4-4e4d-b5fa-11244d45877c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:25.390287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0e8f8eb1-8dc4-4e4d-b5fa-11244d45877c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:25.502217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=1,keystone:SELECT=93,keystone:INSERT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:25.513732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c3ed7-e2b5-41ac-876d-e4e0d0c7f6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:25.514961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c3ed7-e2b5-41ac-876d-e4e0d0c7f6cc admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:25.515423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c3ed7-e2b5-41ac-876d-e4e0d0c7f6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:25.515701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c3ed7-e2b5-41ac-876d-e4e0d0c7f6cc admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:25.516444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c3ed7-e2b5-41ac-876d-e4e0d0c7f6cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:25.518549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c3ed7-e2b5-41ac-876d-e4e0d0c7f6cc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:25.533552 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2313/4650] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:25 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:25.535244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c3ed7-e2b5-41ac-876d-e4e0d0c7f6cc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:25.546445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c3ed7-e2b5-41ac-876d-e4e0d0c7f6cc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:25.552736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575fa54-96a6-47c5-a877-04c4e89cb034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:25.553634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575fa54-96a6-47c5-a877-04c4e89cb034 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:25.553957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575fa54-96a6-47c5-a877-04c4e89cb034 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:25.554237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575fa54-96a6-47c5-a877-04c4e89cb034 admin admin] PATH_INFO: `/v3/projects/074257b478dc498b9a5835210f0a5511/users/29f7f6e2acb5458096efb6586c7a3de2/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:25.556355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575fa54-96a6-47c5-a877-04c4e89cb034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=074257b478dc498b9a5835210f0a5511, user_id=29f7f6e2acb5458096efb6586c7a3de2, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:25.559365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575fa54-96a6-47c5-a877-04c4e89cb034 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:25.571212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f8eb1-8dc4-4e4d-b5fa-11244d45877c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:25.577999 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2314/4651] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:25 2022] PUT /identity/v3/projects/074257b478dc498b9a5835210f0a5511/users/29f7f6e2acb5458096efb6586c7a3de2/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:18:25.587804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817509b7-911c-41d9-8902-6260275e2236 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:25.588129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817509b7-911c-41d9-8902-6260275e2236 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:25.588521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817509b7-911c-41d9-8902-6260275e2236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:25.645733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-817509b7-911c-41d9-8902-6260275e2236 None None] MFA Rules not processed for user `29f7f6e2acb5458096efb6586c7a3de2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:25.647883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-817509b7-911c-41d9-8902-6260275e2236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:25.658191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65de9c4-fd2b-48d6-88a1-4275b289dd57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:25.659542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65de9c4-fd2b-48d6-88a1-4275b289dd57 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:25.659925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65de9c4-fd2b-48d6-88a1-4275b289dd57 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:25.661238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65de9c4-fd2b-48d6-88a1-4275b289dd57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:25.663298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d65de9c4-fd2b-48d6-88a1-4275b289dd57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:25.682876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65de9c4-fd2b-48d6-88a1-4275b289dd57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:25.684718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65de9c4-fd2b-48d6-88a1-4275b289dd57 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:25.707947 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2338/4652] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:25.773418 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2315/4653] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:25 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:25.778599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65de9c4-fd2b-48d6-88a1-4275b289dd57 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:25.796668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65de9c4-fd2b-48d6-88a1-4275b289dd57 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:25.810313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1ed8f-5f92-4ece-a9b7-4e92d4239d03 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29f7f6e2acb5458096efb6586c7a3de2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '074257b478dc498b9a5835210f0a5511', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:25.811542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1ed8f-5f92-4ece-a9b7-4e92d4239d03 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:25.812003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1ed8f-5f92-4ece-a9b7-4e92d4239d03 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:25.812637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a1ed8f-5f92-4ece-a9b7-4e92d4239d03 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:25.813558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1ed8f-5f92-4ece-a9b7-4e92d4239d03 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:25.817751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a1ed8f-5f92-4ece-a9b7-4e92d4239d03 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:25.984767 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2339/4654] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:25 2022] POST /identity/v3/projects => generated 334 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:25.988670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817509b7-911c-41d9-8902-6260275e2236 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:25.999969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:26.007487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817509b7-911c-41d9-8902-6260275e2236 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:26.015270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1ed8f-5f92-4ece-a9b7-4e92d4239d03 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:26.017872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-26a1ed8f-5f92-4ece-a9b7-4e92d4239d03 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:26.182132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbaf1c-aec1-42ef-8a50-50b345fa109b tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29f7f6e2acb5458096efb6586c7a3de2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '074257b478dc498b9a5835210f0a5511', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:26.183140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbaf1c-aec1-42ef-8a50-50b345fa109b tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:26.183674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbaf1c-aec1-42ef-8a50-50b345fa109b tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:26.184163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbaf1c-aec1-42ef-8a50-50b345fa109b tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:26.184916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbaf1c-aec1-42ef-8a50-50b345fa109b tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:26.187521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbaf1c-aec1-42ef-8a50-50b345fa109b tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:26.195549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a1ed8f-5f92-4ece-a9b7-4e92d4239d03 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:26.203733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8958c-7dff-42d5-905c-0be29ec4ad08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:26.207077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8958c-7dff-42d5-905c-0be29ec4ad08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:26.207077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8958c-7dff-42d5-905c-0be29ec4ad08 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:26.207077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8958c-7dff-42d5-905c-0be29ec4ad08 admin admin] PATH_INFO: `/v3/projects/b4be72b7f8c54528b1c0a2a1505416c1` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:26.207664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8958c-7dff-42d5-905c-0be29ec4ad08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4be72b7f8c54528b1c0a2a1505416c1)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:26.210542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8958c-7dff-42d5-905c-0be29ec4ad08 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:26.398151 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2316/4655] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:18:25 2022] POST /identity/v3/projects => generated 332 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:26.399602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbaf1c-aec1-42ef-8a50-50b345fa109b tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:26.405899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-aafbaf1c-aec1-42ef-8a50-50b345fa109b tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:26.455762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e7b8958c-7dff-42d5-905c-0be29ec4ad08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4be72b7f8c54528b1c0a2a1505416c1', 'request_id': 'req-e7b8958c-7dff-42d5-905c-0be29ec4ad08'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:26.474510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e7b8958c-7dff-42d5-905c-0be29ec4ad08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4be72b7f8c54528b1c0a2a1505416c1', 'request_id': 'req-e7b8958c-7dff-42d5-905c-0be29ec4ad08'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:26.576005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbaf1c-aec1-42ef-8a50-50b345fa109b tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:26.592087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e7b8958c-7dff-42d5-905c-0be29ec4ad08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4be72b7f8c54528b1c0a2a1505416c1', 'request_id': 'req-e7b8958c-7dff-42d5-905c-0be29ec4ad08'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:26.616694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e7b8958c-7dff-42d5-905c-0be29ec4ad08 admin admin] The token cache is being invalidate because project b4be72b7f8c54528b1c0a2a1505416c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:26.617057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e7b8958c-7dff-42d5-905c-0be29ec4ad08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:26.619344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e7b8958c-7dff-42d5-905c-0be29ec4ad08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:26.620885 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2340/4656] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:26 2022] DELETE /identity/v3/projects/b4be72b7f8c54528b1c0a2a1505416c1 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:26.621794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8958c-7dff-42d5-905c-0be29ec4ad08 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:26.626237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e7b8958c-7dff-42d5-905c-0be29ec4ad08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:26.693438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c065856-cebe-4ea3-b751-e556bd260345 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:26.694152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c065856-cebe-4ea3-b751-e556bd260345 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:26.694439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c065856-cebe-4ea3-b751-e556bd260345 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:26.694719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c065856-cebe-4ea3-b751-e556bd260345 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:26.697170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4c065856-cebe-4ea3-b751-e556bd260345 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:26.804593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8958c-7dff-42d5-905c-0be29ec4ad08 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:26.809650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335035f-98a1-4dbd-af4c-4650b740cc32 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29f7f6e2acb5458096efb6586c7a3de2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '074257b478dc498b9a5835210f0a5511', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:26.810679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335035f-98a1-4dbd-af4c-4650b740cc32 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:26.810947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335035f-98a1-4dbd-af4c-4650b740cc32 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:26.811243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335035f-98a1-4dbd-af4c-4650b740cc32 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:26.812021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335035f-98a1-4dbd-af4c-4650b740cc32 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:26.815950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335035f-98a1-4dbd-af4c-4650b740cc32 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:26.826444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c065856-cebe-4ea3-b751-e556bd260345 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:26.828613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c065856-cebe-4ea3-b751-e556bd260345 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:26.863850 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2317/4657] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:26.865669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1e576-72b0-407c-afda-bcf416e1584e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:26.866062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1e576-72b0-407c-afda-bcf416e1584e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:26.866358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1e576-72b0-407c-afda-bcf416e1584e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:26.889469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-abf1e576-72b0-407c-afda-bcf416e1584e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:26.891236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-abf1e576-72b0-407c-afda-bcf416e1584e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:26.925475 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2341/4658] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:26 2022] POST /identity/v3/projects => generated 358 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:26.931544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335035f-98a1-4dbd-af4c-4650b740cc32 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:27.056162 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2318/4659] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:27.058920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc2a7e-1ca4-4e79-b4eb-d4300a58a0d9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:27.058920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc2a7e-1ca4-4e79-b4eb-d4300a58a0d9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:27.058920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc2a7e-1ca4-4e79-b4eb-d4300a58a0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:27.108890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e1bc2a7e-1ca4-4e79-b4eb-d4300a58a0d9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:27.110238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e1bc2a7e-1ca4-4e79-b4eb-d4300a58a0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:27.119155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335035f-98a1-4dbd-af4c-4650b740cc32 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:27.129853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666d855-27cb-4ddb-a9b1-bd8d326cfcba tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29f7f6e2acb5458096efb6586c7a3de2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '074257b478dc498b9a5835210f0a5511', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:27.130743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666d855-27cb-4ddb-a9b1-bd8d326cfcba tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:27.132265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666d855-27cb-4ddb-a9b1-bd8d326cfcba tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:27.133785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666d855-27cb-4ddb-a9b1-bd8d326cfcba tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:27.135162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666d855-27cb-4ddb-a9b1-bd8d326cfcba tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:27.138424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666d855-27cb-4ddb-a9b1-bd8d326cfcba tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:27.167197 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2342/4660] 10.222.0.33 () {58 vars in 1130 bytes} [Sat May 28 21:18:26 2022] GET /identity/v3/projects => generated 7205 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:27.172060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666d855-27cb-4ddb-a9b1-bd8d326cfcba tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:27.192346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666d855-27cb-4ddb-a9b1-bd8d326cfcba tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:27.198941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc91a0a-a275-4f38-9ec5-79df4eb331cb tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29f7f6e2acb5458096efb6586c7a3de2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '074257b478dc498b9a5835210f0a5511', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:27.200181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc91a0a-a275-4f38-9ec5-79df4eb331cb tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:27.200524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc91a0a-a275-4f38-9ec5-79df4eb331cb tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:27.201817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc91a0a-a275-4f38-9ec5-79df4eb331cb tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:27.201817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc91a0a-a275-4f38-9ec5-79df4eb331cb tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:18:27.202067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc91a0a-a275-4f38-9ec5-79df4eb331cb tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:27.204186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc91a0a-a275-4f38-9ec5-79df4eb331cb tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:27.205829 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2319/4661] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:27.210017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc2a7e-1ca4-4e79-b4eb-d4300a58a0d9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:27.220891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc2a7e-1ca4-4e79-b4eb-d4300a58a0d9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:27.228814 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2343/4662] 10.222.0.33 () {58 vars in 1157 bytes} [Sat May 28 21:18:27 2022] GET /identity/v3/projects?enabled=False => generated 445 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:18:27.236141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc91a0a-a275-4f38-9ec5-79df4eb331cb tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:27.237322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0485d-447b-4ff7-aee5-6366bf10df78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:27.238360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0485d-447b-4ff7-aee5-6366bf10df78 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:27.238864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0485d-447b-4ff7-aee5-6366bf10df78 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:27.239309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0485d-447b-4ff7-aee5-6366bf10df78 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:27.240966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0485d-447b-4ff7-aee5-6366bf10df78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:18:27.241466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0485d-447b-4ff7-aee5-6366bf10df78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:27.243719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0485d-447b-4ff7-aee5-6366bf10df78 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:27.251345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc91a0a-a275-4f38-9ec5-79df4eb331cb tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:27.260333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015902b-87e8-45af-b8bd-5b71a18329b9 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29f7f6e2acb5458096efb6586c7a3de2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '074257b478dc498b9a5835210f0a5511', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:27.261016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015902b-87e8-45af-b8bd-5b71a18329b9 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:27.261302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015902b-87e8-45af-b8bd-5b71a18329b9 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:27.261581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015902b-87e8-45af-b8bd-5b71a18329b9 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:27.262394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015902b-87e8-45af-b8bd-5b71a18329b9 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Adding query filter params (parent_id=7818b1ad566343a484acedc2fbc4ef05) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:18:27.262707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015902b-87e8-45af-b8bd-5b71a18329b9 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:27.263663 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2320/4663] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:18:27.265892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015902b-87e8-45af-b8bd-5b71a18329b9 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:27.268511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0485d-447b-4ff7-aee5-6366bf10df78 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:27.282457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0485d-447b-4ff7-aee5-6366bf10df78 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:27.302466 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2344/4664] 10.222.0.33 () {58 vars in 1215 bytes} [Sat May 28 21:18:27 2022] GET /identity/v3/projects?parent_id=7818b1ad566343a484acedc2fbc4ef05 => generated 498 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:18:27.308257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015902b-87e8-45af-b8bd-5b71a18329b9 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:27.319264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7dde7b-c175-49c6-a257-6b1664512a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:27.320150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7dde7b-c175-49c6-a257-6b1664512a49 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:27.320495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7dde7b-c175-49c6-a257-6b1664512a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:27.320785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7dde7b-c175-49c6-a257-6b1664512a49 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:27.322396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7dde7b-c175-49c6-a257-6b1664512a49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:27.326735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015902b-87e8-45af-b8bd-5b71a18329b9 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:27.327635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7dde7b-c175-49c6-a257-6b1664512a49 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:27.338850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15a691-5cb9-41a9-abc5-212d874c2d33 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29f7f6e2acb5458096efb6586c7a3de2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '074257b478dc498b9a5835210f0a5511', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:27.340669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15a691-5cb9-41a9-abc5-212d874c2d33 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:27.341134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15a691-5cb9-41a9-abc5-212d874c2d33 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:27.342493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15a691-5cb9-41a9-abc5-212d874c2d33 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] PATH_INFO: `/v3/projects/a37fb9157b9f4883b22bdabbdf0117c1` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:27.344095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15a691-5cb9-41a9-abc5-212d874c2d33 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a37fb9157b9f4883b22bdabbdf0117c1)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:27.347679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15a691-5cb9-41a9-abc5-212d874c2d33 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:27.502202 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2321/4665] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:27 2022] POST /identity/v3/projects => generated 383 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:27.503298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7dde7b-c175-49c6-a257-6b1664512a49 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:27.514735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4e15a691-5cb9-41a9-abc5-212d874c2d33 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a37fb9157b9f4883b22bdabbdf0117c1', 'request_id': 'req-4e15a691-5cb9-41a9-abc5-212d874c2d33'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:27.523116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4e15a691-5cb9-41a9-abc5-212d874c2d33 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a37fb9157b9f4883b22bdabbdf0117c1', 'request_id': 'req-4e15a691-5cb9-41a9-abc5-212d874c2d33'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:27.591192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4e15a691-5cb9-41a9-abc5-212d874c2d33 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a37fb9157b9f4883b22bdabbdf0117c1', 'request_id': 'req-4e15a691-5cb9-41a9-abc5-212d874c2d33'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:27.610189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4e15a691-5cb9-41a9-abc5-212d874c2d33 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] The token cache is being invalidate because project a37fb9157b9f4883b22bdabbdf0117c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:27.610358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4e15a691-5cb9-41a9-abc5-212d874c2d33 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:27.613492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4e15a691-5cb9-41a9-abc5-212d874c2d33 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:27.620105 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2345/4666] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:27 2022] DELETE /identity/v3/projects/a37fb9157b9f4883b22bdabbdf0117c1 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:27.621251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15a691-5cb9-41a9-abc5-212d874c2d33 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:27.623456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4e15a691-5cb9-41a9-abc5-212d874c2d33 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:27.647853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7dde7b-c175-49c6-a257-6b1664512a49 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:27.655714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7332f4-82b2-4c39-80f6-619b6a6fdd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:27.657169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7332f4-82b2-4c39-80f6-619b6a6fdd0e admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:27.657473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7332f4-82b2-4c39-80f6-619b6a6fdd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:27.657760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7332f4-82b2-4c39-80f6-619b6a6fdd0e admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:27.658512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7332f4-82b2-4c39-80f6-619b6a6fdd0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:27.663123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7332f4-82b2-4c39-80f6-619b6a6fdd0e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:27.742086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15a691-5cb9-41a9-abc5-212d874c2d33 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:27.790709 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2322/4667] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:27 2022] POST /identity/v3/users => generated 292 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:27.794797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7332f4-82b2-4c39-80f6-619b6a6fdd0e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:27.798514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bc7332f4-82b2-4c39-80f6-619b6a6fdd0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:27.823667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7332f4-82b2-4c39-80f6-619b6a6fdd0e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:27.840532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e0dcc-00bf-4db2-a2f6-b336ff678454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:27.850174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e0dcc-00bf-4db2-a2f6-b336ff678454 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:27.850174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e0dcc-00bf-4db2-a2f6-b336ff678454 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:27.850174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e0dcc-00bf-4db2-a2f6-b336ff678454 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:27.850174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e0dcc-00bf-4db2-a2f6-b336ff678454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:27.865638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e0dcc-00bf-4db2-a2f6-b336ff678454 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:27.890804 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2323/4668] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:18:27.892450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547fd48-a933-4502-9183-79308cd5c515 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29f7f6e2acb5458096efb6586c7a3de2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '074257b478dc498b9a5835210f0a5511', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:27.895172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e0dcc-00bf-4db2-a2f6-b336ff678454 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:27.895715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547fd48-a933-4502-9183-79308cd5c515 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:27.896833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547fd48-a933-4502-9183-79308cd5c515 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:27.898192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547fd48-a933-4502-9183-79308cd5c515 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] PATH_INFO: `/v3/projects/7818b1ad566343a484acedc2fbc4ef05` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:27.901031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547fd48-a933-4502-9183-79308cd5c515 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7818b1ad566343a484acedc2fbc4ef05)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:27.906809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547fd48-a933-4502-9183-79308cd5c515 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:27.937314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e0dcc-00bf-4db2-a2f6-b336ff678454 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:27.946066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:27.956326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9b048-cea0-4522-addf-bc9d492a6102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:27.958637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9b048-cea0-4522-addf-bc9d492a6102 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:27.958637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9b048-cea0-4522-addf-bc9d492a6102 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:27.958637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9b048-cea0-4522-addf-bc9d492a6102 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:27.958637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9b048-cea0-4522-addf-bc9d492a6102 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:27.960412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9b048-cea0-4522-addf-bc9d492a6102 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:28.000180 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2324/4669] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:27 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:28.004103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9b048-cea0-4522-addf-bc9d492a6102 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:28.014738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9b048-cea0-4522-addf-bc9d492a6102 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:28.032722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f89149-44b9-4291-9e1a-5ee35c8f1fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:28.034867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f89149-44b9-4291-9e1a-5ee35c8f1fa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:28.035156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f89149-44b9-4291-9e1a-5ee35c8f1fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:28.035434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f89149-44b9-4291-9e1a-5ee35c8f1fa2 admin admin] PATH_INFO: `/v3/projects/84e22c442b0241f18af6ee9408d47223/users/cb285a5e922745bbbd61f4c6ca663c71/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:28.064886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f89149-44b9-4291-9e1a-5ee35c8f1fa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e22c442b0241f18af6ee9408d47223, user_id=cb285a5e922745bbbd61f4c6ca663c71, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:28.069713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f89149-44b9-4291-9e1a-5ee35c8f1fa2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:28.070928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3547fd48-a933-4502-9183-79308cd5c515 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7818b1ad566343a484acedc2fbc4ef05', 'request_id': 'req-3547fd48-a933-4502-9183-79308cd5c515'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:28.085938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3547fd48-a933-4502-9183-79308cd5c515 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7818b1ad566343a484acedc2fbc4ef05', 'request_id': 'req-3547fd48-a933-4502-9183-79308cd5c515'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:28.230372 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2325/4670] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:28 2022] PUT /identity/v3/projects/84e22c442b0241f18af6ee9408d47223/users/cb285a5e922745bbbd61f4c6ca663c71/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:28.232502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb675396-9e16-495e-8a72-f7d67f758eba None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:28.232961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb675396-9e16-495e-8a72-f7d67f758eba None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:28.233402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb675396-9e16-495e-8a72-f7d67f758eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:28.260742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-eb675396-9e16-495e-8a72-f7d67f758eba None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:28.262462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-eb675396-9e16-495e-8a72-f7d67f758eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:28.272137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3547fd48-a933-4502-9183-79308cd5c515 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7818b1ad566343a484acedc2fbc4ef05', 'request_id': 'req-3547fd48-a933-4502-9183-79308cd5c515'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:28.320403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3547fd48-a933-4502-9183-79308cd5c515 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] The token cache is being invalidate because project 7818b1ad566343a484acedc2fbc4ef05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:28.320745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3547fd48-a933-4502-9183-79308cd5c515 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:28.322868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3547fd48-a933-4502-9183-79308cd5c515 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:28.327335 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2346/4671] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:27 2022] DELETE /identity/v3/projects/7818b1ad566343a484acedc2fbc4ef05 => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:28.332299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547fd48-a933-4502-9183-79308cd5c515 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:28.340745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3547fd48-a933-4502-9183-79308cd5c515 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:28.412144 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2326/4672] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:28.435255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb675396-9e16-495e-8a72-f7d67f758eba None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:28.436947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-eb675396-9e16-495e-8a72-f7d67f758eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:28.495410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547fd48-a933-4502-9183-79308cd5c515 tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:28.556834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb675396-9e16-495e-8a72-f7d67f758eba None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:28.559596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:28.566385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02118972-0117-45cc-9fd5-295cda01bfba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:28.568432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02118972-0117-45cc-9fd5-295cda01bfba service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:28.568828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02118972-0117-45cc-9fd5-295cda01bfba service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:28.569524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02118972-0117-45cc-9fd5-295cda01bfba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:28.595960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02118972-0117-45cc-9fd5-295cda01bfba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:28.600264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02118972-0117-45cc-9fd5-295cda01bfba service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:28.615397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422ef4b-0f9b-43b2-bf72-ecfd8b142f8d tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '29f7f6e2acb5458096efb6586c7a3de2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '074257b478dc498b9a5835210f0a5511', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:28.616688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422ef4b-0f9b-43b2-bf72-ecfd8b142f8d tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:28.616964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422ef4b-0f9b-43b2-bf72-ecfd8b142f8d tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:28.617343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422ef4b-0f9b-43b2-bf72-ecfd8b142f8d tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] PATH_INFO: `/v3/projects/c424bea90e1848968094413c9bb95b0b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:28.620277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422ef4b-0f9b-43b2-bf72-ecfd8b142f8d tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c424bea90e1848968094413c9bb95b0b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:28.624041 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2327/4673] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:28.624518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422ef4b-0f9b-43b2-bf72-ecfd8b142f8d tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:28.790695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b422ef4b-0f9b-43b2-bf72-ecfd8b142f8d tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c424bea90e1848968094413c9bb95b0b', 'request_id': 'req-b422ef4b-0f9b-43b2-bf72-ecfd8b142f8d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:28.809216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b422ef4b-0f9b-43b2-bf72-ecfd8b142f8d tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c424bea90e1848968094413c9bb95b0b', 'request_id': 'req-b422ef4b-0f9b-43b2-bf72-ecfd8b142f8d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:28.918422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b422ef4b-0f9b-43b2-bf72-ecfd8b142f8d tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c424bea90e1848968094413c9bb95b0b', 'request_id': 'req-b422ef4b-0f9b-43b2-bf72-ecfd8b142f8d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:28.941243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b422ef4b-0f9b-43b2-bf72-ecfd8b142f8d tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] The token cache is being invalidate because project c424bea90e1848968094413c9bb95b0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:28.941521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b422ef4b-0f9b-43b2-bf72-ecfd8b142f8d tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:28.942241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b422ef4b-0f9b-43b2-bf72-ecfd8b142f8d tempest-ListProjectsTestJSON-1178504868 tempest-ListProjectsTestJSON-1178504868-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:28.944889 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2347/4674] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:28 2022] DELETE /identity/v3/projects/c424bea90e1848968094413c9bb95b0b => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:28.948934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02118972-0117-45cc-9fd5-295cda01bfba service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:28.951546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-02118972-0117-45cc-9fd5-295cda01bfba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:28.991235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02118972-0117-45cc-9fd5-295cda01bfba service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:29.117909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a4f39-3b85-4dd6-895f-28d3fb01232b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:29.119715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a4f39-3b85-4dd6-895f-28d3fb01232b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:29.120484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a4f39-3b85-4dd6-895f-28d3fb01232b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:29.120940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a4f39-3b85-4dd6-895f-28d3fb01232b admin admin] PATH_INFO: `/v3/users/4764d45193c5439982d77416273cfbca` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:29.122547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a4f39-3b85-4dd6-895f-28d3fb01232b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4764d45193c5439982d77416273cfbca)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:29.125243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a4f39-3b85-4dd6-895f-28d3fb01232b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:29.353004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1c0a4f39-3b85-4dd6-895f-28d3fb01232b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4764d45193c5439982d77416273cfbca'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:29.354614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1c0a4f39-3b85-4dd6-895f-28d3fb01232b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4764d45193c5439982d77416273cfbca'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:29.368385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1c0a4f39-3b85-4dd6-895f-28d3fb01232b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4764d45193c5439982d77416273cfbca'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:29.455269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1c0a4f39-3b85-4dd6-895f-28d3fb01232b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4764d45193c5439982d77416273cfbca'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:29.462283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1c0a4f39-3b85-4dd6-895f-28d3fb01232b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4764d45193c5439982d77416273cfbca'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:29.496131 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2328/4675] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:28 2022] DELETE /identity/v3/users/4764d45193c5439982d77416273cfbca => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:18:29.498376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e934c8f-9c7e-4319-9985-b50e93a55efb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:29.498672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e934c8f-9c7e-4319-9985-b50e93a55efb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:29.498954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e934c8f-9c7e-4319-9985-b50e93a55efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:29.529473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4e934c8f-9c7e-4319-9985-b50e93a55efb None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:29.530769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4e934c8f-9c7e-4319-9985-b50e93a55efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:29.649123 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2348/4676] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:29.660937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a4f39-3b85-4dd6-895f-28d3fb01232b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:29.662822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1c0a4f39-3b85-4dd6-895f-28d3fb01232b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:29.733882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a4f39-3b85-4dd6-895f-28d3fb01232b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:29.738566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215cf9ce-ae18-4aa3-b659-75d6bcb46e97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:29.739239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cf9ce-ae18-4aa3-b659-75d6bcb46e97 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:29.739527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cf9ce-ae18-4aa3-b659-75d6bcb46e97 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:29.739884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215cf9ce-ae18-4aa3-b659-75d6bcb46e97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:29.752766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215cf9ce-ae18-4aa3-b659-75d6bcb46e97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:29.752766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215cf9ce-ae18-4aa3-b659-75d6bcb46e97 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:29.774276 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2329/4677] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:29.815311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:30.774203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e934c8f-9c7e-4319-9985-b50e93a55efb None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:30.774203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4e934c8f-9c7e-4319-9985-b50e93a55efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:30.793140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e934c8f-9c7e-4319-9985-b50e93a55efb None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:30.806296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f05e0-1bac-47af-9cc6-fd00be75c605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:30.807601 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f05e0-1bac-47af-9cc6-fd00be75c605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:30.808095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f05e0-1bac-47af-9cc6-fd00be75c605 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:30.809001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f05e0-1bac-47af-9cc6-fd00be75c605 admin admin] PATH_INFO: `/v3/projects/a6e390d382eb4bf0ab42115523c4069e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:30.810775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f05e0-1bac-47af-9cc6-fd00be75c605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6e390d382eb4bf0ab42115523c4069e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:30.813019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f05e0-1bac-47af-9cc6-fd00be75c605 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:31.004738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7f7f05e0-1bac-47af-9cc6-fd00be75c605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e390d382eb4bf0ab42115523c4069e', 'request_id': 'req-7f7f05e0-1bac-47af-9cc6-fd00be75c605'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:31.007248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3af941-30ab-47f3-a253-f7dcc74d672b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:31.007753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3af941-30ab-47f3-a253-f7dcc74d672b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:31.007993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3af941-30ab-47f3-a253-f7dcc74d672b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:31.025055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7f7f05e0-1bac-47af-9cc6-fd00be75c605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e390d382eb4bf0ab42115523c4069e', 'request_id': 'req-7f7f05e0-1bac-47af-9cc6-fd00be75c605'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:31.041700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ac3af941-30ab-47f3-a253-f7dcc74d672b None None] MFA Rules not processed for user `cb285a5e922745bbbd61f4c6ca663c71`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:31.042985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ac3af941-30ab-47f3-a253-f7dcc74d672b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:31.128177 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2330/4678] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:31.137957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fbd7e-f38d-4491-9e37-0f559885d11b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:31.138219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fbd7e-f38d-4491-9e37-0f559885d11b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:31.138564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063fbd7e-f38d-4491-9e37-0f559885d11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:31.139224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7f7f05e0-1bac-47af-9cc6-fd00be75c605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e390d382eb4bf0ab42115523c4069e', 'request_id': 'req-7f7f05e0-1bac-47af-9cc6-fd00be75c605'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:31.159336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7f7f05e0-1bac-47af-9cc6-fd00be75c605 admin admin] The token cache is being invalidate because project a6e390d382eb4bf0ab42115523c4069e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:31.160177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7f7f05e0-1bac-47af-9cc6-fd00be75c605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:31.161483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7f7f05e0-1bac-47af-9cc6-fd00be75c605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:31.162253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-063fbd7e-f38d-4491-9e37-0f559885d11b None None] MFA Rules not processed for user `cb285a5e922745bbbd61f4c6ca663c71`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:31.163782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-063fbd7e-f38d-4491-9e37-0f559885d11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:31.168508 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2349/4679] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:30 2022] DELETE /identity/v3/projects/a6e390d382eb4bf0ab42115523c4069e => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:18:31.171667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f05e0-1bac-47af-9cc6-fd00be75c605 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:31.173916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7f7f05e0-1bac-47af-9cc6-fd00be75c605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:31.184990 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2331/4680] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:31.200234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fbd7e-f38d-4491-9e37-0f559885d11b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:31.201887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-063fbd7e-f38d-4491-9e37-0f559885d11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:31.210485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:31.268050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f05e0-1bac-47af-9cc6-fd00be75c605 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:31.273357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e4aea-d710-49e4-b68d-0d8b9a24e36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:31.274400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e4aea-d710-49e4-b68d-0d8b9a24e36c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:31.274813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e4aea-d710-49e4-b68d-0d8b9a24e36c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:31.275152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e4aea-d710-49e4-b68d-0d8b9a24e36c admin admin] PATH_INFO: `/v3/users/29f7f6e2acb5458096efb6586c7a3de2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:31.276730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e4aea-d710-49e4-b68d-0d8b9a24e36c admin admin] RBAC: Authorizing `identity:delete_user(user_id=29f7f6e2acb5458096efb6586c7a3de2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:31.279391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e4aea-d710-49e4-b68d-0d8b9a24e36c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:31.330709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063fbd7e-f38d-4491-9e37-0f559885d11b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:31.339017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ab7da-bd42-417b-be7d-255c62faba9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:31.340277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ab7da-bd42-417b-be7d-255c62faba9c service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:31.340277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ab7da-bd42-417b-be7d-255c62faba9c service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:31.340833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ab7da-bd42-417b-be7d-255c62faba9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:31.343646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7b1ab7da-bd42-417b-be7d-255c62faba9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:31.356345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ab7da-bd42-417b-be7d-255c62faba9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:31.360918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ab7da-bd42-417b-be7d-255c62faba9c service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:31.369425 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2332/4681] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:18:31.492939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7b1e4aea-d710-49e4-b68d-0d8b9a24e36c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29f7f6e2acb5458096efb6586c7a3de2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:31.504103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7b1e4aea-d710-49e4-b68d-0d8b9a24e36c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29f7f6e2acb5458096efb6586c7a3de2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:31.505742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7b1e4aea-d710-49e4-b68d-0d8b9a24e36c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29f7f6e2acb5458096efb6586c7a3de2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:31.506484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7b1e4aea-d710-49e4-b68d-0d8b9a24e36c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29f7f6e2acb5458096efb6586c7a3de2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:31.585162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7b1e4aea-d710-49e4-b68d-0d8b9a24e36c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29f7f6e2acb5458096efb6586c7a3de2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:31.611310 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2350/4682] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:31 2022] DELETE /identity/v3/users/29f7f6e2acb5458096efb6586c7a3de2 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:31.912486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:32.220889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ab7da-bd42-417b-be7d-255c62faba9c service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:32.240550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ab7da-bd42-417b-be7d-255c62faba9c service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:32.290923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:32.389174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fb4daf-d4e0-48b6-b707-deb34122e20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:32.392598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb4daf-d4e0-48b6-b707-deb34122e20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:32.392942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb4daf-d4e0-48b6-b707-deb34122e20c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:32.393232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb4daf-d4e0-48b6-b707-deb34122e20c admin admin] PATH_INFO: `/v3/projects/074257b478dc498b9a5835210f0a5511` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:32.399290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb4daf-d4e0-48b6-b707-deb34122e20c admin admin] RBAC: Authorizing `identity:delete_project(project_id=074257b478dc498b9a5835210f0a5511)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:32.401283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fb4daf-d4e0-48b6-b707-deb34122e20c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:32.654291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-18fb4daf-d4e0-48b6-b707-deb34122e20c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '074257b478dc498b9a5835210f0a5511', 'request_id': 'req-18fb4daf-d4e0-48b6-b707-deb34122e20c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:32.670197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-18fb4daf-d4e0-48b6-b707-deb34122e20c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '074257b478dc498b9a5835210f0a5511', 'request_id': 'req-18fb4daf-d4e0-48b6-b707-deb34122e20c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:32.819844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-18fb4daf-d4e0-48b6-b707-deb34122e20c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '074257b478dc498b9a5835210f0a5511', 'request_id': 'req-18fb4daf-d4e0-48b6-b707-deb34122e20c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:32.842890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-18fb4daf-d4e0-48b6-b707-deb34122e20c admin admin] The token cache is being invalidate because project 074257b478dc498b9a5835210f0a5511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:32.846665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-18fb4daf-d4e0-48b6-b707-deb34122e20c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:32.846665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-18fb4daf-d4e0-48b6-b707-deb34122e20c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:32.849176 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2333/4683] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:32 2022] DELETE /identity/v3/projects/074257b478dc498b9a5835210f0a5511 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:32.854148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15b099-bd0a-439e-afd2-4c32f6fb234b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:32.854948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15b099-bd0a-439e-afd2-4c32f6fb234b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:32.855738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15b099-bd0a-439e-afd2-4c32f6fb234b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:32.881419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ea15b099-bd0a-439e-afd2-4c32f6fb234b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:32.883025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ea15b099-bd0a-439e-afd2-4c32f6fb234b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:32.995598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a359c7-a605-4c7b-9dd7-f939ab73bad8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:32.995893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a359c7-a605-4c7b-9dd7-f939ab73bad8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:32.996174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a359c7-a605-4c7b-9dd7-f939ab73bad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:32.997519 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2351/4684] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:33.012133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15b099-bd0a-439e-afd2-4c32f6fb234b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:33.015643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ea15b099-bd0a-439e-afd2-4c32f6fb234b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:33.027721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-02a359c7-a605-4c7b-9dd7-f939ab73bad8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:33.028941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-02a359c7-a605-4c7b-9dd7-f939ab73bad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:33.061982 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2334/4685] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:33.063466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a359c7-a605-4c7b-9dd7-f939ab73bad8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:33.088744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a359c7-a605-4c7b-9dd7-f939ab73bad8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:33.100847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ca19d-7b99-4939-8358-b4f5baea23ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:33.101523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ca19d-7b99-4939-8358-b4f5baea23ce admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:33.101824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ca19d-7b99-4939-8358-b4f5baea23ce admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:33.102104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ca19d-7b99-4939-8358-b4f5baea23ce admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:33.102821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ca19d-7b99-4939-8358-b4f5baea23ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:18:33.103122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ca19d-7b99-4939-8358-b4f5baea23ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:33.105697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ca19d-7b99-4939-8358-b4f5baea23ce admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:33.139360 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2335/4686] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:18:33.142271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ca19d-7b99-4939-8358-b4f5baea23ce admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:33.172740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ca19d-7b99-4939-8358-b4f5baea23ce admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:33.185199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767001d-92a5-4f86-9be5-6db0a14b4c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:33.187424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767001d-92a5-4f86-9be5-6db0a14b4c25 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:33.189026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767001d-92a5-4f86-9be5-6db0a14b4c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:33.189334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767001d-92a5-4f86-9be5-6db0a14b4c25 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:33.192596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767001d-92a5-4f86-9be5-6db0a14b4c25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:33.192596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767001d-92a5-4f86-9be5-6db0a14b4c25 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:33.199828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15b099-bd0a-439e-afd2-4c32f6fb234b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:33.212626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016560df-524a-47a6-879d-5cc2ad2ad38e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:33.213607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016560df-524a-47a6-879d-5cc2ad2ad38e service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:33.213924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016560df-524a-47a6-879d-5cc2ad2ad38e service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:33.214192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016560df-524a-47a6-879d-5cc2ad2ad38e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:33.219535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-016560df-524a-47a6-879d-5cc2ad2ad38e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:33.245233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016560df-524a-47a6-879d-5cc2ad2ad38e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:33.247769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016560df-524a-47a6-879d-5cc2ad2ad38e service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:33.278722 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2352/4687] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:33.298858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:33.337455 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2336/4688] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:33 2022] POST /identity/v3/projects => generated 397 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:33.340837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016560df-524a-47a6-879d-5cc2ad2ad38e service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:33.363280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016560df-524a-47a6-879d-5cc2ad2ad38e service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:33.364965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:33.449938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe12c01-8549-49c5-8e32-8caca7df9173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:33.451631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe12c01-8549-49c5-8e32-8caca7df9173 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:33.452607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe12c01-8549-49c5-8e32-8caca7df9173 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:33.453053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe12c01-8549-49c5-8e32-8caca7df9173 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:33.454706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe12c01-8549-49c5-8e32-8caca7df9173 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:33.462573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe12c01-8549-49c5-8e32-8caca7df9173 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:33.603760 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2353/4689] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:33 2022] POST /identity/v3/users => generated 299 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:33.608014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767001d-92a5-4f86-9be5-6db0a14b4c25 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:33.630357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767001d-92a5-4f86-9be5-6db0a14b4c25 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:33.639384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8a9e6-5a9a-4a42-ba7c-795312126b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:33.640080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8a9e6-5a9a-4a42-ba7c-795312126b25 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:33.640368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8a9e6-5a9a-4a42-ba7c-795312126b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:33.640650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8a9e6-5a9a-4a42-ba7c-795312126b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:33.641394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8a9e6-5a9a-4a42-ba7c-795312126b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:33.643726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8a9e6-5a9a-4a42-ba7c-795312126b25 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:33.672237 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2337/4690] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:33.679017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe12c01-8549-49c5-8e32-8caca7df9173 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:33.694672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe12c01-8549-49c5-8e32-8caca7df9173 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:33.710270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d6e709-2637-4fb2-afbd-047f5d784658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:33.711196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6e709-2637-4fb2-afbd-047f5d784658 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:33.711651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6e709-2637-4fb2-afbd-047f5d784658 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:33.712159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d6e709-2637-4fb2-afbd-047f5d784658 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:33.713074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6e709-2637-4fb2-afbd-047f5d784658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:33.717014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d6e709-2637-4fb2-afbd-047f5d784658 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:33.743020 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2354/4691] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:33.747613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8a9e6-5a9a-4a42-ba7c-795312126b25 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:33.767780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8a9e6-5a9a-4a42-ba7c-795312126b25 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:33.770465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89997343-bd63-4901-98e8-11e7c73fbebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:33.774130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89997343-bd63-4901-98e8-11e7c73fbebf admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:33.774499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89997343-bd63-4901-98e8-11e7c73fbebf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:33.778515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89997343-bd63-4901-98e8-11e7c73fbebf admin admin] PATH_INFO: `/v3/projects/bfe9913cf8464578b6730e8bc095600e/users/13de2c7fb6f14cd6b42598254743eed3/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:33.800069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89997343-bd63-4901-98e8-11e7c73fbebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfe9913cf8464578b6730e8bc095600e, user_id=13de2c7fb6f14cd6b42598254743eed3, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:33.803647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89997343-bd63-4901-98e8-11e7c73fbebf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:33.926875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb268ab-480e-41f5-bd52-8217ca6a17bc None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:33.927177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb268ab-480e-41f5-bd52-8217ca6a17bc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:33.927535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb268ab-480e-41f5-bd52-8217ca6a17bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:33.930984 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2338/4692] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:33 2022] PUT /identity/v3/projects/bfe9913cf8464578b6730e8bc095600e/users/13de2c7fb6f14cd6b42598254743eed3/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:18:33.961153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6eb268ab-480e-41f5-bd52-8217ca6a17bc None None] MFA Rules not processed for user `13de2c7fb6f14cd6b42598254743eed3`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:33.962522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6eb268ab-480e-41f5-bd52-8217ca6a17bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:34.093373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89997343-bd63-4901-98e8-11e7c73fbebf admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:34.097447 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2355/4693] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:33 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:34.101433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb268ab-480e-41f5-bd52-8217ca6a17bc None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:34.105795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-89997343-bd63-4901-98e8-11e7c73fbebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:34.265170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89997343-bd63-4901-98e8-11e7c73fbebf admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:34.279174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f3375-84fb-4c28-96a5-4ed924901c34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:34.280225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f3375-84fb-4c28-96a5-4ed924901c34 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:34.280769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f3375-84fb-4c28-96a5-4ed924901c34 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:34.281261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1f3375-84fb-4c28-96a5-4ed924901c34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:34.287950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0f1f3375-84fb-4c28-96a5-4ed924901c34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:34.297821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb268ab-480e-41f5-bd52-8217ca6a17bc None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:34.311104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac455e4-0ede-4cf4-a962-d14a6a68bb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:34.311104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac455e4-0ede-4cf4-a962-d14a6a68bb7b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:34.311339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac455e4-0ede-4cf4-a962-d14a6a68bb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:34.311499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f3375-84fb-4c28-96a5-4ed924901c34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:34.312238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac455e4-0ede-4cf4-a962-d14a6a68bb7b admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:34.312984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac455e4-0ede-4cf4-a962-d14a6a68bb7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:34.314681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1f3375-84fb-4c28-96a5-4ed924901c34 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:34.319508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac455e4-0ede-4cf4-a962-d14a6a68bb7b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:34.335911 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2339/4694] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:34.437304 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2356/4695] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:34 2022] POST /identity/v3/projects => generated 395 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:18:34.441213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f3375-84fb-4c28-96a5-4ed924901c34 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:34.449982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:34.458665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1f3375-84fb-4c28-96a5-4ed924901c34 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:34.583770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9528c0-31a0-43d6-a4ee-bc92fcfd935d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:34.584588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9528c0-31a0-43d6-a4ee-bc92fcfd935d admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:34.585223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9528c0-31a0-43d6-a4ee-bc92fcfd935d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:34.585719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9528c0-31a0-43d6-a4ee-bc92fcfd935d admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:34.586870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9528c0-31a0-43d6-a4ee-bc92fcfd935d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:34.589120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9528c0-31a0-43d6-a4ee-bc92fcfd935d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:34.695158 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2340/4696] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:34 2022] POST /identity/v3/users => generated 304 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:34.700002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac455e4-0ede-4cf4-a962-d14a6a68bb7b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:34.716460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac455e4-0ede-4cf4-a962-d14a6a68bb7b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:34.734305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69cbd7-539f-4189-9591-b5fc9e2d5dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:34.734923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69cbd7-539f-4189-9591-b5fc9e2d5dfd admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:34.735212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69cbd7-539f-4189-9591-b5fc9e2d5dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:34.735494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69cbd7-539f-4189-9591-b5fc9e2d5dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:34.736240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69cbd7-539f-4189-9591-b5fc9e2d5dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:34.739509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69cbd7-539f-4189-9591-b5fc9e2d5dfd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:34.774723 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2357/4697] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:34.792068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9528c0-31a0-43d6-a4ee-bc92fcfd935d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:34.804842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9528c0-31a0-43d6-a4ee-bc92fcfd935d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:34.828959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa385d-f80a-42e7-8729-60bb1eff24bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:34.830082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa385d-f80a-42e7-8729-60bb1eff24bf admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:34.830965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa385d-f80a-42e7-8729-60bb1eff24bf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:34.831314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa385d-f80a-42e7-8729-60bb1eff24bf admin admin] PATH_INFO: `/v3/projects/511a3b8d37eb47dabf04f0a974b7f047/users/80e6860d2efd47e68a2df5ca4e7f2048/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:34.849897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa385d-f80a-42e7-8729-60bb1eff24bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=511a3b8d37eb47dabf04f0a974b7f047, user_id=80e6860d2efd47e68a2df5ca4e7f2048, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:34.853376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa385d-f80a-42e7-8729-60bb1eff24bf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:34.966237 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2341/4698] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:34 2022] PUT /identity/v3/projects/511a3b8d37eb47dabf04f0a974b7f047/users/80e6860d2efd47e68a2df5ca4e7f2048/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:34.970145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69cbd7-539f-4189-9591-b5fc9e2d5dfd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:34.987795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69cbd7-539f-4189-9591-b5fc9e2d5dfd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:35.079121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565c64e-1dcb-4b2b-a26c-202872e76d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:35.079817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565c64e-1dcb-4b2b-a26c-202872e76d7e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:35.080106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565c64e-1dcb-4b2b-a26c-202872e76d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:35.080416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565c64e-1dcb-4b2b-a26c-202872e76d7e admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:35.081377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565c64e-1dcb-4b2b-a26c-202872e76d7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:35.083031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565c64e-1dcb-4b2b-a26c-202872e76d7e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:35.097926 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2358/4699] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:35.099237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa385d-f80a-42e7-8729-60bb1eff24bf admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:35.117103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa385d-f80a-42e7-8729-60bb1eff24bf admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:35.123274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2e923-adc2-4c1f-98b1-c1fbf30496ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:35.124858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2e923-adc2-4c1f-98b1-c1fbf30496ce admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:35.124858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2e923-adc2-4c1f-98b1-c1fbf30496ce admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:35.125066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c2e923-adc2-4c1f-98b1-c1fbf30496ce admin admin] PATH_INFO: `/v3/projects/511a3b8d37eb47dabf04f0a974b7f047/users/80e6860d2efd47e68a2df5ca4e7f2048/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:35.129203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c2e923-adc2-4c1f-98b1-c1fbf30496ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=511a3b8d37eb47dabf04f0a974b7f047, user_id=80e6860d2efd47e68a2df5ca4e7f2048, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:35.141437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c2e923-adc2-4c1f-98b1-c1fbf30496ce admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:35.161971 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2342/4700] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:35 2022] PUT /identity/v3/projects/511a3b8d37eb47dabf04f0a974b7f047/users/80e6860d2efd47e68a2df5ca4e7f2048/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:35.173157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057956e-6147-4cd2-80bc-974694d1bbaa None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:35.173440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057956e-6147-4cd2-80bc-974694d1bbaa None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:35.177577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057956e-6147-4cd2-80bc-974694d1bbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:35.201294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:35.215307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f057956e-6147-4cd2-80bc-974694d1bbaa None None] MFA Rules not processed for user `80e6860d2efd47e68a2df5ca4e7f2048`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:35.217242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f057956e-6147-4cd2-80bc-974694d1bbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:35.433329 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2359/4701] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:35.438067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2e923-adc2-4c1f-98b1-c1fbf30496ce admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:35.538744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c2e923-adc2-4c1f-98b1-c1fbf30496ce admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:35.543787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d11f20-e53b-46fc-a6ca-9ab20806e0fd tempest-ProjectsNegativeStaticTestJSON-1139453450 tempest-ProjectsNegativeStaticTestJSON-1139453450-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13de2c7fb6f14cd6b42598254743eed3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe9913cf8464578b6730e8bc095600e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:35.544961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d11f20-e53b-46fc-a6ca-9ab20806e0fd tempest-ProjectsNegativeStaticTestJSON-1139453450 tempest-ProjectsNegativeStaticTestJSON-1139453450-project] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:35.545576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d11f20-e53b-46fc-a6ca-9ab20806e0fd tempest-ProjectsNegativeStaticTestJSON-1139453450 tempest-ProjectsNegativeStaticTestJSON-1139453450-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:35.546411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d11f20-e53b-46fc-a6ca-9ab20806e0fd tempest-ProjectsNegativeStaticTestJSON-1139453450 tempest-ProjectsNegativeStaticTestJSON-1139453450-project] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:35.548009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d11f20-e53b-46fc-a6ca-9ab20806e0fd tempest-ProjectsNegativeStaticTestJSON-1139453450 tempest-ProjectsNegativeStaticTestJSON-1139453450-project] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:35.556553 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-15d11f20-e53b-46fc-a6ca-9ab20806e0fd tempest-ProjectsNegativeStaticTestJSON-1139453450 tempest-ProjectsNegativeStaticTestJSON-1139453450-project] You are not authorized to perform the requested action: identity:create_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. May 28 21:18:35.556553 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:18:35.556553 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:18:35.556553 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:18:35.556553 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:18:35.556553 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:18:35.556553 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:18:35.556553 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:18:35.556553 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:18:35.556553 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:18:35.556553 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:18:35.556553 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:18:35.556553 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 162, in post May 28 21:18:35.556553 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call( May 28 21:18:35.558593 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call May 28 21:18:35.558593 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application enforcer_obj._enforce( May 28 21:18:35.558593 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce May 28 21:18:35.558593 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application result = self._enforcer.enforce( May 28 21:18:35.558593 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce May 28 21:18:35.558593 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) May 28 21:18:35.558593 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:create_project. May 28 21:18:35.558593 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:18:35.586224 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2343/4702] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:18:35 2022] POST /identity/v3/projects => generated 136 bytes in 145 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) May 28 21:18:35.586839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057956e-6147-4cd2-80bc-974694d1bbaa None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:35.599878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057956e-6147-4cd2-80bc-974694d1bbaa None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:35.609457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eadc16-3de0-4f52-ad10-6e7c5f2a3b83 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80e6860d2efd47e68a2df5ca4e7f2048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '511a3b8d37eb47dabf04f0a974b7f047', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:35.610215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eadc16-3de0-4f52-ad10-6e7c5f2a3b83 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:35.610525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eadc16-3de0-4f52-ad10-6e7c5f2a3b83 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:35.610814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eadc16-3de0-4f52-ad10-6e7c5f2a3b83 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:35.612022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eadc16-3de0-4f52-ad10-6e7c5f2a3b83 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:35.614221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eadc16-3de0-4f52-ad10-6e7c5f2a3b83 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-d6eadc16-3de0-4f52-ad10-6e7c5f2a3b83 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Invalid input for field 'name': '' is too short May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: Failed validating 'minLength' in schema['properties']['name']: May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: On instance['name']: May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: Failed validating 'minLength' in schema['properties']['name']: May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: On instance['name']: May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: '' May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise error May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application On instance['name']: May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application '' May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:18:35.627817 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:18:35.629909 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:18:35.629909 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:18:35.629909 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:18:35.629909 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:18:35.629909 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:18:35.629909 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:18:35.629909 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:18:35.629909 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:18:35.629909 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:18:35.629909 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:18:35.629909 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:18:35.629909 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:18:35.629909 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post May 28 21:18:35.629909 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) May 28 21:18:35.629909 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate May 28 21:18:35.629909 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) May 28 21:18:35.629909 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate May 28 21:18:35.631786 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) May 28 21:18:35.631786 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short May 28 21:18:35.631786 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:18:35.631786 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: May 28 21:18:35.631786 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} May 28 21:18:35.631786 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:18:35.631786 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application On instance['name']: May 28 21:18:35.631786 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application '' May 28 21:18:35.631786 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:18:35.631786 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2360/4703] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:18:35 2022] POST /identity/v3/projects => generated 283 bytes in 37 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) May 28 21:18:35.632612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d11f20-e53b-46fc-a6ca-9ab20806e0fd tempest-ProjectsNegativeStaticTestJSON-1139453450 tempest-ProjectsNegativeStaticTestJSON-1139453450-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:35.646846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d11f20-e53b-46fc-a6ca-9ab20806e0fd tempest-ProjectsNegativeStaticTestJSON-1139453450 tempest-ProjectsNegativeStaticTestJSON-1139453450-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:35.655049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f0b00-d1a2-44b2-91ce-97def894e889 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80e6860d2efd47e68a2df5ca4e7f2048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '511a3b8d37eb47dabf04f0a974b7f047', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:35.655049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f0b00-d1a2-44b2-91ce-97def894e889 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:35.655049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f0b00-d1a2-44b2-91ce-97def894e889 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:35.655922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f0b00-d1a2-44b2-91ce-97def894e889 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:35.657281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f0b00-d1a2-44b2-91ce-97def894e889 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:35.660051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f0b00-d1a2-44b2-91ce-97def894e889 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-0a3f0b00-d1a2-44b2-91ce-97def894e889 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: Failed validating 'maxLength' in schema['properties']['name']: May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: On instance['name']: May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: Failed validating 'maxLength' in schema['properties']['name']: May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: On instance['name']: May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise error May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: May 28 21:18:35.668842 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application On instance['name']: May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:18:35.670174 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 165, in post May 28 21:18:35.672147 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application validation.lazy_validate(schema.project_create, project) May 28 21:18:35.672147 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate May 28 21:18:35.672147 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) May 28 21:18:35.672147 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate May 28 21:18:35.672147 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) May 28 21:18:35.672147 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long May 28 21:18:35.672147 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:18:35.672147 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: May 28 21:18:35.672147 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} May 28 21:18:35.672147 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:18:35.672147 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application On instance['name']: May 28 21:18:35.672147 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' May 28 21:18:35.672147 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:18:35.681266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eadc16-3de0-4f52-ad10-6e7c5f2a3b83 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:35.690362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eadc16-3de0-4f52-ad10-6e7c5f2a3b83 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:35.691422 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2344/4704] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:18:35 2022] POST /identity/v3/projects => generated 412 bytes in 59 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) May 28 21:18:35.697322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c72ca-d329-41ed-b04b-92ab9c99c7f4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80e6860d2efd47e68a2df5ca4e7f2048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '511a3b8d37eb47dabf04f0a974b7f047', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:35.698941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c72ca-d329-41ed-b04b-92ab9c99c7f4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:35.699193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c72ca-d329-41ed-b04b-92ab9c99c7f4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:35.699496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c72ca-d329-41ed-b04b-92ab9c99c7f4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] PATH_INFO: `/v3/projects/105c7b6b9c27409089b509559f7e07e6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:35.709886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c72ca-d329-41ed-b04b-92ab9c99c7f4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: Authorizing `identity:delete_project(project_id=105c7b6b9c27409089b509559f7e07e6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:35.711631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c72ca-d329-41ed-b04b-92ab9c99c7f4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:35.732756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-2a5c72ca-d329-41ed-b04b-92ab9c99c7f4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Could not find project: 105c7b6b9c27409089b509559f7e07e6.: keystone.exception.ProjectNotFound: Could not find project: 105c7b6b9c27409089b509559f7e07e6. May 28 21:18:35.732756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:18:35.732756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:18:35.732756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:18:35.732756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:18:35.732756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:18:35.732756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:18:35.732756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:18:35.732756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:18:35.732756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:18:35.732756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:18:35.732756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:18:35.732756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete May 28 21:18:35.732756 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( May 28 21:18:35.733793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:18:35.733793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:18:35.733793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project May 28 21:18:35.733793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) May 28 21:18:35.733793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:18:35.733793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:18:35.733793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:18:35.733793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:18:35.733793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 105c7b6b9c27409089b509559f7e07e6. May 28 21:18:35.733793 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:18:35.740411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f0b00-d1a2-44b2-91ce-97def894e889 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:35.740959 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2361/4705] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:35 2022] DELETE /identity/v3/projects/105c7b6b9c27409089b509559f7e07e6 => generated 113 bytes in 65 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:18:35.758658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f0b00-d1a2-44b2-91ce-97def894e889 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:35.765262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e6773-6a6b-402e-9f14-f77651045eab tempest-ProjectsNegativeStaticTestJSON-1139453450 tempest-ProjectsNegativeStaticTestJSON-1139453450-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '13de2c7fb6f14cd6b42598254743eed3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bfe9913cf8464578b6730e8bc095600e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:35.766371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e6773-6a6b-402e-9f14-f77651045eab tempest-ProjectsNegativeStaticTestJSON-1139453450 tempest-ProjectsNegativeStaticTestJSON-1139453450-project] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:35.766729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e6773-6a6b-402e-9f14-f77651045eab tempest-ProjectsNegativeStaticTestJSON-1139453450 tempest-ProjectsNegativeStaticTestJSON-1139453450-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:35.766953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68e6773-6a6b-402e-9f14-f77651045eab tempest-ProjectsNegativeStaticTestJSON-1139453450 tempest-ProjectsNegativeStaticTestJSON-1139453450-project] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:35.767807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68e6773-6a6b-402e-9f14-f77651045eab tempest-ProjectsNegativeStaticTestJSON-1139453450 tempest-ProjectsNegativeStaticTestJSON-1139453450-project] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:35.776610 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-a68e6773-6a6b-402e-9f14-f77651045eab tempest-ProjectsNegativeStaticTestJSON-1139453450 tempest-ProjectsNegativeStaticTestJSON-1139453450-project] You are not authorized to perform the requested action: identity:list_projects.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. May 28 21:18:35.776610 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:18:35.776610 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:18:35.776610 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:18:35.776610 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:18:35.776610 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:18:35.776610 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:18:35.776610 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:18:35.776610 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:18:35.776610 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:18:35.776610 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:18:35.776610 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:18:35.776610 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 153, in get May 28 21:18:35.776610 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._list_projects() May 28 21:18:35.777902 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 120, in _list_projects May 28 21:18:35.777902 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:list_projects', May 28 21:18:35.777902 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call May 28 21:18:35.777902 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application enforcer_obj._enforce( May 28 21:18:35.777902 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce May 28 21:18:35.777902 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application result = self._enforcer.enforce( May 28 21:18:35.777902 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce May 28 21:18:35.777902 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) May 28 21:18:35.777902 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:list_projects. May 28 21:18:35.777902 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:18:35.782360 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2345/4706] 10.222.0.33 () {58 vars in 1130 bytes} [Sat May 28 21:18:35 2022] GET /identity/v3/projects => generated 135 bytes in 42 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) May 28 21:18:35.784609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c72ca-d329-41ed-b04b-92ab9c99c7f4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:35.814142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c72ca-d329-41ed-b04b-92ab9c99c7f4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:35.819863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8d1fc-b9f6-439f-b591-81a4e7c9db4d tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80e6860d2efd47e68a2df5ca4e7f2048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '511a3b8d37eb47dabf04f0a974b7f047', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:35.820806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8d1fc-b9f6-439f-b591-81a4e7c9db4d tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:35.821652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8d1fc-b9f6-439f-b591-81a4e7c9db4d tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:35.822266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb8d1fc-b9f6-439f-b591-81a4e7c9db4d tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:35.823243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8d1fc-b9f6-439f-b591-81a4e7c9db4d tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:35.830623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb8d1fc-b9f6-439f-b591-81a4e7c9db4d tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:35.953750 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2362/4707] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:35 2022] POST /identity/v3/projects => generated 359 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:18:35.957796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e6773-6a6b-402e-9f14-f77651045eab tempest-ProjectsNegativeStaticTestJSON-1139453450 tempest-ProjectsNegativeStaticTestJSON-1139453450-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:35.971888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68e6773-6a6b-402e-9f14-f77651045eab tempest-ProjectsNegativeStaticTestJSON-1139453450 tempest-ProjectsNegativeStaticTestJSON-1139453450-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:36.098124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da047b-f0de-4ef5-8540-3d70d1bd05c4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80e6860d2efd47e68a2df5ca4e7f2048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '511a3b8d37eb47dabf04f0a974b7f047', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:36.099259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da047b-f0de-4ef5-8540-3d70d1bd05c4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:36.099653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da047b-f0de-4ef5-8540-3d70d1bd05c4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:36.100043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da047b-f0de-4ef5-8540-3d70d1bd05c4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:36.100942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da047b-f0de-4ef5-8540-3d70d1bd05c4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:36.102885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68da047b-f0de-4ef5-8540-3d70d1bd05c4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:36.150868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.sql.core [None req-68da047b-f0de-4ef5-8540-3d70d1bd05c4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-1249192959' for key 'project.ixu_project_name_domain_id'") May 28 21:18:36.150868 ncdv-master-1 devstack@keystone.service[103104]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 28 21:18:36.150868 ncdv-master-1 devstack@keystone.service[103104]: [parameters: {'id': '27f222bee7ea4184873470261b182d42', 'name': 'tempest-project-dup-1249192959', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 28 21:18:36.150868 ncdv-master-1 devstack@keystone.service[103104]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=103104) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 28 21:18:36.160743 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-68da047b-f0de-4ef5-8540-3d70d1bd05c4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-1249192959, project id 27f222bee7ea4184873470261b182d42.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-1249192959, project id 27f222bee7ea4184873470261b182d42. May 28 21:18:36.160743 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:18:36.160743 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 21:18:36.160743 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 21:18:36.160743 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 21:18:36.160743 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 21:18:36.160743 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 21:18:36.160743 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application result = self._query(query) May 28 21:18:36.160743 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 21:18:36.160743 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application conn.query(q) May 28 21:18:36.160743 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 21:18:36.160743 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 21:18:36.162073 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 21:18:36.162073 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application result.read() May 28 21:18:36.162073 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 21:18:36.162073 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 21:18:36.162073 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 21:18:36.162073 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 21:18:36.162073 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 21:18:36.162073 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 21:18:36.162073 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 21:18:36.162073 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 21:18:36.162073 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-tempest-project-dup-1249192959' for key 'project.ixu_project_name_domain_id'") May 28 21:18:36.162073 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:18:36.162073 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 28 21:18:36.162073 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:18:36.162073 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:18:36.162073 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 28 21:18:36.162073 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return project_ref.to_dict() May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application next(self.gen) May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application yield resource May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application next(self.gen) May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.session.rollback() May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.force_reraise() May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise self.value May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 28 21:18:36.164071 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application session.commit() May 28 21:18:36.165605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 28 21:18:36.165605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 28 21:18:36.165605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 28 21:18:36.165605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._prepare_impl() May 28 21:18:36.165605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 28 21:18:36.165605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.session.flush() May 28 21:18:36.165605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 28 21:18:36.165605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._flush(objects) May 28 21:18:36.165605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 28 21:18:36.165605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 28 21:18:36.165605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 28 21:18:36.165605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application compat.raise_( May 28 21:18:36.165605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 21:18:36.165605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception May 28 21:18:36.165605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 28 21:18:36.165605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application flush_context.execute() May 28 21:18:36.165605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 28 21:18:36.166984 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rec.execute(self) May 28 21:18:36.166984 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 28 21:18:36.166984 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 28 21:18:36.166984 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 28 21:18:36.166984 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application _emit_insert_statements( May 28 21:18:36.166984 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 28 21:18:36.166984 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application c = connection._execute_20( May 28 21:18:36.166984 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 28 21:18:36.166984 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 28 21:18:36.166984 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 28 21:18:36.166984 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 28 21:18:36.166984 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 28 21:18:36.166984 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ret = self._execute_context( May 28 21:18:36.166984 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 28 21:18:36.166984 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 28 21:18:36.166984 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 28 21:18:36.168241 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 28 21:18:36.168241 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 21:18:36.168241 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception May 28 21:18:36.168241 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 21:18:36.168241 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 21:18:36.168241 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 21:18:36.168241 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 21:18:36.168241 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 21:18:36.168241 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application result = self._query(query) May 28 21:18:36.168241 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 21:18:36.168241 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application conn.query(q) May 28 21:18:36.168241 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 21:18:36.168241 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 21:18:36.168241 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 21:18:36.168241 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application result.read() May 28 21:18:36.168241 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 21:18:36.168241 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 21:18:36.169601 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 21:18:36.169601 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 21:18:36.169601 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 21:18:36.169601 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 21:18:36.169601 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 21:18:36.169601 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 21:18:36.169601 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-tempest-project-dup-1249192959' for key 'project.ixu_project_name_domain_id'") May 28 21:18:36.169601 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 28 21:18:36.169601 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [parameters: {'id': '27f222bee7ea4184873470261b182d42', 'name': 'tempest-project-dup-1249192959', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 28 21:18:36.169601 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 28 21:18:36.169601 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:18:36.169601 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:18:36.169601 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:18:36.169601 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:18:36.169601 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 28 21:18:36.170941 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 28 21:18:36.170941 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 28 21:18:36.170941 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 28 21:18:36.170941 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-project-dup-1249192959 at domain ID default. May 28 21:18:36.170941 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:18:36.170941 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:18:36.170941 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:18:36.170941 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:18:36.170941 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:18:36.170941 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:18:36.170941 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:18:36.170941 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:18:36.170941 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:18:36.170941 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:18:36.170941 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:18:36.170941 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:18:36.170941 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:18:36.173038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:18:36.173038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 28 21:18:36.173038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 28 21:18:36.173038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:18:36.173038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:18:36.173038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 28 21:18:36.173038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.Conflict( May 28 21:18:36.173038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is tempest-project-dup-1249192959, project id 27f222bee7ea4184873470261b182d42. May 28 21:18:36.173038 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:18:36.173038 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2346/4708] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:18:35 2022] POST /identity/v3/projects => generated 285 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 28 21:18:36.173895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8d1fc-b9f6-439f-b591-81a4e7c9db4d tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:36.186076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb8d1fc-b9f6-439f-b591-81a4e7c9db4d tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:36.204489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '80e6860d2efd47e68a2df5ca4e7f2048', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '511a3b8d37eb47dabf04f0a974b7f047', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:36.207244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:36.207562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:36.207858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] PATH_INFO: `/v3/projects/c1e60845a7d54e22a86b45ccf63bf144` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:36.212862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c1e60845a7d54e22a86b45ccf63bf144)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:36.220181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:36.381000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1e60845a7d54e22a86b45ccf63bf144', 'request_id': 'req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:36.389459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1e60845a7d54e22a86b45ccf63bf144', 'request_id': 'req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:36.483592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1e60845a7d54e22a86b45ccf63bf144', 'request_id': 'req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:36.500872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] The token cache is being invalidate because project c1e60845a7d54e22a86b45ccf63bf144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:36.503351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:36.506321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:36.509830 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2363/4709] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:36 2022] DELETE /identity/v3/projects/c1e60845a7d54e22a86b45ccf63bf144 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:36.515568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da047b-f0de-4ef5-8540-3d70d1bd05c4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:36.519573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-68da047b-f0de-4ef5-8540-3d70d1bd05c4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:36.628709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da047b-f0de-4ef5-8540-3d70d1bd05c4 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:36.716855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545e265-d34c-4caf-9940-bbb92a64d616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:36.718231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545e265-d34c-4caf-9940-bbb92a64d616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:36.718520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545e265-d34c-4caf-9940-bbb92a64d616 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:36.719436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545e265-d34c-4caf-9940-bbb92a64d616 admin admin] PATH_INFO: `/v3/users/13de2c7fb6f14cd6b42598254743eed3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:36.721465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545e265-d34c-4caf-9940-bbb92a64d616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13de2c7fb6f14cd6b42598254743eed3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:36.724543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545e265-d34c-4caf-9940-bbb92a64d616 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:36.785863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:36.843082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:36.845189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:36.881830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0eb4b1-23bf-4295-a21b-e74c5ed637a6 tempest-ProjectsNegativeStaticTestJSON-641580403 tempest-ProjectsNegativeStaticTestJSON-641580403-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:36.889650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a9c7b6-d01d-4de9-9d4a-396b04c75085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:36.890612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9c7b6-d01d-4de9-9d4a-396b04c75085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:36.890902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9c7b6-d01d-4de9-9d4a-396b04c75085 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:36.891326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a9c7b6-d01d-4de9-9d4a-396b04c75085 admin admin] PATH_INFO: `/v3/users/631aa933ecb444c586481eac51f29677` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:36.892640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9c7b6-d01d-4de9-9d4a-396b04c75085 admin admin] RBAC: Authorizing `identity:delete_user(user_id=631aa933ecb444c586481eac51f29677)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:36.895646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a9c7b6-d01d-4de9-9d4a-396b04c75085 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:36.954290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b545e265-d34c-4caf-9940-bbb92a64d616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13de2c7fb6f14cd6b42598254743eed3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:36.955126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b545e265-d34c-4caf-9940-bbb92a64d616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13de2c7fb6f14cd6b42598254743eed3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:36.972640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b545e265-d34c-4caf-9940-bbb92a64d616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13de2c7fb6f14cd6b42598254743eed3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:37.095086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b545e265-d34c-4caf-9940-bbb92a64d616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13de2c7fb6f14cd6b42598254743eed3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:37.096853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b545e265-d34c-4caf-9940-bbb92a64d616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13de2c7fb6f14cd6b42598254743eed3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:37.131766 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2347/4710] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:36 2022] DELETE /identity/v3/users/13de2c7fb6f14cd6b42598254743eed3 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:18:37.138387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac9ee5-7cb5-43d9-bfa5-6e39c95c280a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:37.138913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac9ee5-7cb5-43d9-bfa5-6e39c95c280a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:37.139428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac9ee5-7cb5-43d9-bfa5-6e39c95c280a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:37.173988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-51a9c7b6-d01d-4de9-9d4a-396b04c75085 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '631aa933ecb444c586481eac51f29677'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:37.187005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-51a9c7b6-d01d-4de9-9d4a-396b04c75085 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '631aa933ecb444c586481eac51f29677'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:37.188160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-51a9c7b6-d01d-4de9-9d4a-396b04c75085 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '631aa933ecb444c586481eac51f29677'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:37.190844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-51a9c7b6-d01d-4de9-9d4a-396b04c75085 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '631aa933ecb444c586481eac51f29677'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:37.197044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-63ac9ee5-7cb5-43d9-bfa5-6e39c95c280a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:37.200713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-63ac9ee5-7cb5-43d9-bfa5-6e39c95c280a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:37.286631 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2348/4711] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:37.293356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-51a9c7b6-d01d-4de9-9d4a-396b04c75085 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '631aa933ecb444c586481eac51f29677'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:37.300963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac9ee5-7cb5-43d9-bfa5-6e39c95c280a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:37.304197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-63ac9ee5-7cb5-43d9-bfa5-6e39c95c280a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:37.342870 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2364/4712] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:36 2022] DELETE /identity/v3/users/631aa933ecb444c586481eac51f29677 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:37.348162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80a56a-6780-4bc5-8018-2d07a8393eea None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:37.348510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80a56a-6780-4bc5-8018-2d07a8393eea None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:37.349053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80a56a-6780-4bc5-8018-2d07a8393eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:37.374604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac9ee5-7cb5-43d9-bfa5-6e39c95c280a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:37.385372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cc80a56a-6780-4bc5-8018-2d07a8393eea None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:37.387607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cc80a56a-6780-4bc5-8018-2d07a8393eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:37.433059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316d2c7-c966-4f7c-90c3-0cda4eb7c2d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:37.433819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316d2c7-c966-4f7c-90c3-0cda4eb7c2d5 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:37.434224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316d2c7-c966-4f7c-90c3-0cda4eb7c2d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:37.434569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316d2c7-c966-4f7c-90c3-0cda4eb7c2d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:37.534873 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2365/4713] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:37.552447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:37.554022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316d2c7-c966-4f7c-90c3-0cda4eb7c2d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:37.555586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316d2c7-c966-4f7c-90c3-0cda4eb7c2d5 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:37.563199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80a56a-6780-4bc5-8018-2d07a8393eea None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:37.565568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cc80a56a-6780-4bc5-8018-2d07a8393eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:37.567132 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2349/4714] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:18:37.582877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80a56a-6780-4bc5-8018-2d07a8393eea None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:37.587744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2af3a6-d17c-4132-a7bb-023968316465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:37.588585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2af3a6-d17c-4132-a7bb-023968316465 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:37.588879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2af3a6-d17c-4132-a7bb-023968316465 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:37.589452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2af3a6-d17c-4132-a7bb-023968316465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:37.627061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2af3a6-d17c-4132-a7bb-023968316465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:37.629930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2af3a6-d17c-4132-a7bb-023968316465 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:37.641263 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2366/4715] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:18:38.519031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316d2c7-c966-4f7c-90c3-0cda4eb7c2d5 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:38.520753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6316d2c7-c966-4f7c-90c3-0cda4eb7c2d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:38.542568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316d2c7-c966-4f7c-90c3-0cda4eb7c2d5 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:38.554939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cf5474-fa31-4e8e-8f23-eb27838ae6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:38.557159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf5474-fa31-4e8e-8f23-eb27838ae6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:38.558269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf5474-fa31-4e8e-8f23-eb27838ae6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:38.558795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cf5474-fa31-4e8e-8f23-eb27838ae6d3 admin admin] PATH_INFO: `/v3/projects/eaee064440244f0da817830ab2d06f0a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:38.560575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf5474-fa31-4e8e-8f23-eb27838ae6d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaee064440244f0da817830ab2d06f0a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:38.565531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cf5474-fa31-4e8e-8f23-eb27838ae6d3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:38.702036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b7cf5474-fa31-4e8e-8f23-eb27838ae6d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaee064440244f0da817830ab2d06f0a', 'request_id': 'req-b7cf5474-fa31-4e8e-8f23-eb27838ae6d3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:38.708844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b7cf5474-fa31-4e8e-8f23-eb27838ae6d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaee064440244f0da817830ab2d06f0a', 'request_id': 'req-b7cf5474-fa31-4e8e-8f23-eb27838ae6d3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:38.790377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b7cf5474-fa31-4e8e-8f23-eb27838ae6d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaee064440244f0da817830ab2d06f0a', 'request_id': 'req-b7cf5474-fa31-4e8e-8f23-eb27838ae6d3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:38.810515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b7cf5474-fa31-4e8e-8f23-eb27838ae6d3 admin admin] The token cache is being invalidate because project eaee064440244f0da817830ab2d06f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:38.810788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b7cf5474-fa31-4e8e-8f23-eb27838ae6d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:38.811489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b7cf5474-fa31-4e8e-8f23-eb27838ae6d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:38.814417 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2350/4716] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:38 2022] DELETE /identity/v3/projects/eaee064440244f0da817830ab2d06f0a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:38.825598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58bd04-e30d-40b8-831e-636746d6ffc4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:38.825598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58bd04-e30d-40b8-831e-636746d6ffc4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:38.825598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58bd04-e30d-40b8-831e-636746d6ffc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:38.845397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8a58bd04-e30d-40b8-831e-636746d6ffc4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:38.847237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8a58bd04-e30d-40b8-831e-636746d6ffc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:38.897054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:38.941457 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2367/4717] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:38.943451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b00f7b-702b-445e-bd9a-97f43c191c29 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:38.944125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b00f7b-702b-445e-bd9a-97f43c191c29 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:38.944409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b00f7b-702b-445e-bd9a-97f43c191c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:38.974230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-59b00f7b-702b-445e-bd9a-97f43c191c29 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:38.974230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-59b00f7b-702b-445e-bd9a-97f43c191c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:38.979571 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2351/4718] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:38.984566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58bd04-e30d-40b8-831e-636746d6ffc4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:38.997768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58bd04-e30d-40b8-831e-636746d6ffc4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:39.004713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040a357-a27f-4dab-ab69-c8672d52ecb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:39.005651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040a357-a27f-4dab-ab69-c8672d52ecb9 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:39.006213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040a357-a27f-4dab-ab69-c8672d52ecb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:39.006678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040a357-a27f-4dab-ab69-c8672d52ecb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:39.007813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040a357-a27f-4dab-ab69-c8672d52ecb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:18:39.008506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040a357-a27f-4dab-ab69-c8672d52ecb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:39.010521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040a357-a27f-4dab-ab69-c8672d52ecb9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:39.030568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b00f7b-702b-445e-bd9a-97f43c191c29 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:39.030928 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2368/4719] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:18:39.047479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b00f7b-702b-445e-bd9a-97f43c191c29 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:39.056243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff9a4a-b61f-4e2f-83ee-24f7b764469a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:39.057315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff9a4a-b61f-4e2f-83ee-24f7b764469a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:39.057790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff9a4a-b61f-4e2f-83ee-24f7b764469a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:39.058603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff9a4a-b61f-4e2f-83ee-24f7b764469a admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:39.059721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff9a4a-b61f-4e2f-83ee-24f7b764469a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:39.062287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ff9a4a-b61f-4e2f-83ee-24f7b764469a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:39.224396 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2352/4720] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:39 2022] POST /identity/v3/projects => generated 361 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:18:39.228658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040a357-a27f-4dab-ab69-c8672d52ecb9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:39.249024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040a357-a27f-4dab-ab69-c8672d52ecb9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:39.269381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff9a4a-b61f-4e2f-83ee-24f7b764469a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:39.274235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-36ff9a4a-b61f-4e2f-83ee-24f7b764469a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:39.382390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a7cbc-11e2-417f-abd4-783a38610f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:39.383098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a7cbc-11e2-417f-abd4-783a38610f09 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:39.383327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a7cbc-11e2-417f-abd4-783a38610f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:39.383609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a7cbc-11e2-417f-abd4-783a38610f09 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:39.386246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a7cbc-11e2-417f-abd4-783a38610f09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:39.388830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a7cbc-11e2-417f-abd4-783a38610f09 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:39.408908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff9a4a-b61f-4e2f-83ee-24f7b764469a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:39.414735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:39.425975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8930763-a993-460b-a041-81241b9695b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:39.427619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8930763-a993-460b-a041-81241b9695b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:39.427911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8930763-a993-460b-a041-81241b9695b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:39.428195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8930763-a993-460b-a041-81241b9695b6 admin admin] PATH_INFO: `/v3/projects/bfe9913cf8464578b6730e8bc095600e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:39.430597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8930763-a993-460b-a041-81241b9695b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfe9913cf8464578b6730e8bc095600e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:39.436206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8930763-a993-460b-a041-81241b9695b6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:39.533935 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2369/4721] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:39 2022] POST /identity/v3/users => generated 281 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:39.538417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a7cbc-11e2-417f-abd4-783a38610f09 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:39.567138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a7cbc-11e2-417f-abd4-783a38610f09 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:39.577075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc902523-6e82-41e7-bd8f-d6c0bb1e5a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:39.578142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc902523-6e82-41e7-bd8f-d6c0bb1e5a09 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:39.578696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc902523-6e82-41e7-bd8f-d6c0bb1e5a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:39.579708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc902523-6e82-41e7-bd8f-d6c0bb1e5a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:39.580843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc902523-6e82-41e7-bd8f-d6c0bb1e5a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:39.582637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc902523-6e82-41e7-bd8f-d6c0bb1e5a09 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:39.594134 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2370/4722] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:39.599824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc902523-6e82-41e7-bd8f-d6c0bb1e5a09 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:39.611894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc902523-6e82-41e7-bd8f-d6c0bb1e5a09 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:39.617578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d02e8-fd8b-46bd-8e99-9ac0ec074b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:39.623541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d02e8-fd8b-46bd-8e99-9ac0ec074b9e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:39.623541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d02e8-fd8b-46bd-8e99-9ac0ec074b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:39.623541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d02e8-fd8b-46bd-8e99-9ac0ec074b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:39.623541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d02e8-fd8b-46bd-8e99-9ac0ec074b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:39.623541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d02e8-fd8b-46bd-8e99-9ac0ec074b9e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:39.640874 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2371/4723] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:18:39.642017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d02e8-fd8b-46bd-8e99-9ac0ec074b9e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:39.654249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d02e8-fd8b-46bd-8e99-9ac0ec074b9e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:39.668367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f8930763-a993-460b-a041-81241b9695b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfe9913cf8464578b6730e8bc095600e', 'request_id': 'req-f8930763-a993-460b-a041-81241b9695b6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:39.676138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f8930763-a993-460b-a041-81241b9695b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfe9913cf8464578b6730e8bc095600e', 'request_id': 'req-f8930763-a993-460b-a041-81241b9695b6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:39.716405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96911e7-16c3-4c89-8731-62652bb03fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:39.717915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96911e7-16c3-4c89-8731-62652bb03fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:39.718103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96911e7-16c3-4c89-8731-62652bb03fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:39.718988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96911e7-16c3-4c89-8731-62652bb03fc9 admin admin] PATH_INFO: `/v3/projects/e50d742e41464ad99c3dfd94005509ef/users/120a571243304f3f925fc79d60317bc1/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:39.737747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96911e7-16c3-4c89-8731-62652bb03fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e50d742e41464ad99c3dfd94005509ef, user_id=120a571243304f3f925fc79d60317bc1, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:39.743618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96911e7-16c3-4c89-8731-62652bb03fc9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:39.785250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f8930763-a993-460b-a041-81241b9695b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfe9913cf8464578b6730e8bc095600e', 'request_id': 'req-f8930763-a993-460b-a041-81241b9695b6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:39.805210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f8930763-a993-460b-a041-81241b9695b6 admin admin] The token cache is being invalidate because project bfe9913cf8464578b6730e8bc095600e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:39.805730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f8930763-a993-460b-a041-81241b9695b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:39.807603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f8930763-a993-460b-a041-81241b9695b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:39.814034 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2353/4724] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:39 2022] DELETE /identity/v3/projects/bfe9913cf8464578b6730e8bc095600e => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:39.816765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8930763-a993-460b-a041-81241b9695b6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:39.818757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f8930763-a993-460b-a041-81241b9695b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:39.838108 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2372/4725] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:39 2022] PUT /identity/v3/projects/e50d742e41464ad99c3dfd94005509ef/users/120a571243304f3f925fc79d60317bc1/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:18:39.850566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfd9a9-2ddc-4cbc-aeab-db3d890abe44 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:39.851626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfd9a9-2ddc-4cbc-aeab-db3d890abe44 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:39.852810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dfd9a9-2ddc-4cbc-aeab-db3d890abe44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:39.866396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8930763-a993-460b-a041-81241b9695b6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:39.929082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-99dfd9a9-2ddc-4cbc-aeab-db3d890abe44 None None] MFA Rules not processed for user `120a571243304f3f925fc79d60317bc1`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:39.931863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-99dfd9a9-2ddc-4cbc-aeab-db3d890abe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:39.943109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11dbf59-3b1a-42b4-86b0-0d5de7cafa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:39.944988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11dbf59-3b1a-42b4-86b0-0d5de7cafa20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:39.944988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11dbf59-3b1a-42b4-86b0-0d5de7cafa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:39.944988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11dbf59-3b1a-42b4-86b0-0d5de7cafa20 admin admin] PATH_INFO: `/v3/users/80e6860d2efd47e68a2df5ca4e7f2048` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:39.946684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11dbf59-3b1a-42b4-86b0-0d5de7cafa20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80e6860d2efd47e68a2df5ca4e7f2048)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:39.948800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11dbf59-3b1a-42b4-86b0-0d5de7cafa20 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:39.968302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:40.028738 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2373/4726] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:39 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:40.062316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dfd9a9-2ddc-4cbc-aeab-db3d890abe44 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:40.064100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-99dfd9a9-2ddc-4cbc-aeab-db3d890abe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:40.146993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dfd9a9-2ddc-4cbc-aeab-db3d890abe44 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:40.160954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e1889-8fa0-41e3-b187-b830569854f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:40.160954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e1889-8fa0-41e3-b187-b830569854f8 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:40.160954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e1889-8fa0-41e3-b187-b830569854f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:40.160954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5e1889-8fa0-41e3-b187-b830569854f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:40.168871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e1889-8fa0-41e3-b187-b830569854f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:40.170260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5e1889-8fa0-41e3-b187-b830569854f8 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:40.195427 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2374/4727] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:40 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:18:40.197278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e11dbf59-3b1a-42b4-86b0-0d5de7cafa20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80e6860d2efd47e68a2df5ca4e7f2048'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:40.198112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e11dbf59-3b1a-42b4-86b0-0d5de7cafa20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80e6860d2efd47e68a2df5ca4e7f2048'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:40.205556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e11dbf59-3b1a-42b4-86b0-0d5de7cafa20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80e6860d2efd47e68a2df5ca4e7f2048'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:40.316969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e11dbf59-3b1a-42b4-86b0-0d5de7cafa20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80e6860d2efd47e68a2df5ca4e7f2048'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:40.317130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e11dbf59-3b1a-42b4-86b0-0d5de7cafa20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80e6860d2efd47e68a2df5ca4e7f2048'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:40.336901 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2354/4728] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:39 2022] DELETE /identity/v3/users/80e6860d2efd47e68a2df5ca4e7f2048 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:40.754899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:41.132650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:41.328278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e1889-8fa0-41e3-b187-b830569854f8 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:41.348693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5e1889-8fa0-41e3-b187-b830569854f8 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:41.473916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe77c5-8752-4cf5-90da-b79eb3b51977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:41.474921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe77c5-8752-4cf5-90da-b79eb3b51977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:41.475206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe77c5-8752-4cf5-90da-b79eb3b51977 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:41.475519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe77c5-8752-4cf5-90da-b79eb3b51977 admin admin] PATH_INFO: `/v3/projects/511a3b8d37eb47dabf04f0a974b7f047` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:41.477139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe77c5-8752-4cf5-90da-b79eb3b51977 admin admin] RBAC: Authorizing `identity:delete_project(project_id=511a3b8d37eb47dabf04f0a974b7f047)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:41.479268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe77c5-8752-4cf5-90da-b79eb3b51977 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:41.637973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-96fe77c5-8752-4cf5-90da-b79eb3b51977 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '511a3b8d37eb47dabf04f0a974b7f047', 'request_id': 'req-96fe77c5-8752-4cf5-90da-b79eb3b51977'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:41.644013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-96fe77c5-8752-4cf5-90da-b79eb3b51977 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '511a3b8d37eb47dabf04f0a974b7f047', 'request_id': 'req-96fe77c5-8752-4cf5-90da-b79eb3b51977'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:41.706805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-96fe77c5-8752-4cf5-90da-b79eb3b51977 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '511a3b8d37eb47dabf04f0a974b7f047', 'request_id': 'req-96fe77c5-8752-4cf5-90da-b79eb3b51977'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:41.722867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-96fe77c5-8752-4cf5-90da-b79eb3b51977 admin admin] The token cache is being invalidate because project 511a3b8d37eb47dabf04f0a974b7f047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:41.723358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-96fe77c5-8752-4cf5-90da-b79eb3b51977 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:41.724183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-96fe77c5-8752-4cf5-90da-b79eb3b51977 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:41.727337 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2375/4729] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:41 2022] DELETE /identity/v3/projects/511a3b8d37eb47dabf04f0a974b7f047 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:41.732806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac5162-d0b4-4179-bde7-1d070b6a9399 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:41.733529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac5162-d0b4-4179-bde7-1d070b6a9399 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:41.733529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ac5162-d0b4-4179-bde7-1d070b6a9399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:41.755260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a6ac5162-d0b4-4179-bde7-1d070b6a9399 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:41.756567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a6ac5162-d0b4-4179-bde7-1d070b6a9399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:41.826290 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2355/4730] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:41.835261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835646a1-0f1a-4ffe-beea-8a3e12ccfb20 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:41.836050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835646a1-0f1a-4ffe-beea-8a3e12ccfb20 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:41.844637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835646a1-0f1a-4ffe-beea-8a3e12ccfb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:41.844637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:41.857203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-835646a1-0f1a-4ffe-beea-8a3e12ccfb20 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:41.858557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-835646a1-0f1a-4ffe-beea-8a3e12ccfb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:41.867191 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2376/4731] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:41.871429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac5162-d0b4-4179-bde7-1d070b6a9399 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:41.881723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ac5162-d0b4-4179-bde7-1d070b6a9399 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:41.890801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df126d0-2a97-492a-a695-8e62c52c306b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:41.891498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df126d0-2a97-492a-a695-8e62c52c306b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:41.892331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df126d0-2a97-492a-a695-8e62c52c306b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:41.892331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df126d0-2a97-492a-a695-8e62c52c306b admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:41.893431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df126d0-2a97-492a-a695-8e62c52c306b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:18:41.893746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df126d0-2a97-492a-a695-8e62c52c306b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:41.895422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df126d0-2a97-492a-a695-8e62c52c306b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:41.909352 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2356/4732] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:18:41.913795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835646a1-0f1a-4ffe-beea-8a3e12ccfb20 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:41.956184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835646a1-0f1a-4ffe-beea-8a3e12ccfb20 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:41.965543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273ad70-4754-49d1-bbdf-5100a31adc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:41.967888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273ad70-4754-49d1-bbdf-5100a31adc06 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:41.968183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273ad70-4754-49d1-bbdf-5100a31adc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:41.968466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273ad70-4754-49d1-bbdf-5100a31adc06 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:41.969194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273ad70-4754-49d1-bbdf-5100a31adc06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:41.971517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273ad70-4754-49d1-bbdf-5100a31adc06 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:42.098755 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2377/4733] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:41 2022] POST /identity/v3/projects => generated 385 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:42.099555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df126d0-2a97-492a-a695-8e62c52c306b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:42.120668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df126d0-2a97-492a-a695-8e62c52c306b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:42.206917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71113c4-f8a8-4e1c-bb01-bc0680ad9d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:42.208691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71113c4-f8a8-4e1c-bb01-bc0680ad9d31 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:42.209230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71113c4-f8a8-4e1c-bb01-bc0680ad9d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:42.213018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71113c4-f8a8-4e1c-bb01-bc0680ad9d31 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:42.214241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71113c4-f8a8-4e1c-bb01-bc0680ad9d31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:42.217260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71113c4-f8a8-4e1c-bb01-bc0680ad9d31 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:42.339593 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2357/4734] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:42 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:42.344204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273ad70-4754-49d1-bbdf-5100a31adc06 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:42.363510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273ad70-4754-49d1-bbdf-5100a31adc06 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:42.374839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e5f57-4796-4c22-911a-aced9f4563ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:42.375691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e5f57-4796-4c22-911a-aced9f4563ca admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:42.376146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e5f57-4796-4c22-911a-aced9f4563ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:42.378062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7e5f57-4796-4c22-911a-aced9f4563ca admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:42.378907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e5f57-4796-4c22-911a-aced9f4563ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:42.380633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7e5f57-4796-4c22-911a-aced9f4563ca admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:42.391949 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2378/4735] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:18:42.399217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71113c4-f8a8-4e1c-bb01-bc0680ad9d31 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:42.418550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71113c4-f8a8-4e1c-bb01-bc0680ad9d31 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:42.426552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690d62c-78e5-484c-a962-ee535a701e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:42.427416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690d62c-78e5-484c-a962-ee535a701e98 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:42.428345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690d62c-78e5-484c-a962-ee535a701e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:42.428345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690d62c-78e5-484c-a962-ee535a701e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:42.429126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690d62c-78e5-484c-a962-ee535a701e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:42.431131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690d62c-78e5-484c-a962-ee535a701e98 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:42.469922 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2358/4736] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:42.470361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e5f57-4796-4c22-911a-aced9f4563ca admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:42.489819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7e5f57-4796-4c22-911a-aced9f4563ca admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:42.495271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545c658-a972-431d-9dae-6ca1aec148f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:42.496370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545c658-a972-431d-9dae-6ca1aec148f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:42.496732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545c658-a972-431d-9dae-6ca1aec148f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:42.497176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c545c658-a972-431d-9dae-6ca1aec148f2 admin admin] PATH_INFO: `/v3/projects/340440caf7e54bcdbea3a7fa768c4dad/users/1a606dec4722416c8ab27b1aca243430/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:42.514449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545c658-a972-431d-9dae-6ca1aec148f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=340440caf7e54bcdbea3a7fa768c4dad, user_id=1a606dec4722416c8ab27b1aca243430, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:42.517130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c545c658-a972-431d-9dae-6ca1aec148f2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:42.660045 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2379/4737] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:42 2022] PUT /identity/v3/projects/340440caf7e54bcdbea3a7fa768c4dad/users/1a606dec4722416c8ab27b1aca243430/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:42.670567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b2a08-ab2e-40cb-99c2-8402c7ed774d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:42.670929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b2a08-ab2e-40cb-99c2-8402c7ed774d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:42.671281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b2a08-ab2e-40cb-99c2-8402c7ed774d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:42.702128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d07b2a08-ab2e-40cb-99c2-8402c7ed774d None None] MFA Rules not processed for user `1a606dec4722416c8ab27b1aca243430`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:42.703784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d07b2a08-ab2e-40cb-99c2-8402c7ed774d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:42.781559 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2359/4738] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:42.787571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545c658-a972-431d-9dae-6ca1aec148f2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:42.864444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c545c658-a972-431d-9dae-6ca1aec148f2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:42.869276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2298da5-798a-44a0-9406-64d063747a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:42.870017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2298da5-798a-44a0-9406-64d063747a1e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:42.870307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2298da5-798a-44a0-9406-64d063747a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:42.870585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2298da5-798a-44a0-9406-64d063747a1e admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:42.871534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2298da5-798a-44a0-9406-64d063747a1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:42.873465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2298da5-798a-44a0-9406-64d063747a1e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:42.992745 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2380/4739] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:42 2022] POST /identity/v3/projects => generated 383 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:42.994207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b2a08-ab2e-40cb-99c2-8402c7ed774d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:43.011896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b2a08-ab2e-40cb-99c2-8402c7ed774d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:43.107199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10d432-70ed-414e-ac54-09d96e591329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:43.108447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10d432-70ed-414e-ac54-09d96e591329 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:43.109022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10d432-70ed-414e-ac54-09d96e591329 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:43.109525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10d432-70ed-414e-ac54-09d96e591329 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:43.110710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10d432-70ed-414e-ac54-09d96e591329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:43.114638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e10d432-70ed-414e-ac54-09d96e591329 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:43.150966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:43.241624 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2360/4740] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:42 2022] POST /identity/v3/users => generated 298 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:43.245945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2298da5-798a-44a0-9406-64d063747a1e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:43.261646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2298da5-798a-44a0-9406-64d063747a1e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:43.277309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6a8a3-2212-46f3-9209-be6eaae1790d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:43.278294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6a8a3-2212-46f3-9209-be6eaae1790d admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:43.278795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6a8a3-2212-46f3-9209-be6eaae1790d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:43.285600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6a8a3-2212-46f3-9209-be6eaae1790d admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:43.285600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c6a8a3-2212-46f3-9209-be6eaae1790d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:43.285600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c6a8a3-2212-46f3-9209-be6eaae1790d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:43.301827 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2381/4741] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:18:43.305530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10d432-70ed-414e-ac54-09d96e591329 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:43.325145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e10d432-70ed-414e-ac54-09d96e591329 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:43.330844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd75486-ece9-43e8-821b-0ac4a9bc232a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:43.374660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd75486-ece9-43e8-821b-0ac4a9bc232a admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:43.374660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd75486-ece9-43e8-821b-0ac4a9bc232a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:43.374660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd75486-ece9-43e8-821b-0ac4a9bc232a admin admin] PATH_INFO: `/v3/projects/cfecc9ade4ac4a2a9f2aa98b72464dbc/users/b788ac1b97cf4a99a1fdc800bb3b7dd9/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:43.374660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd75486-ece9-43e8-821b-0ac4a9bc232a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfecc9ade4ac4a2a9f2aa98b72464dbc, user_id=b788ac1b97cf4a99a1fdc800bb3b7dd9, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:43.374660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd75486-ece9-43e8-821b-0ac4a9bc232a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:43.436013 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2361/4742] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:43 2022] PUT /identity/v3/projects/cfecc9ade4ac4a2a9f2aa98b72464dbc/users/b788ac1b97cf4a99a1fdc800bb3b7dd9/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:43.441280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6a8a3-2212-46f3-9209-be6eaae1790d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:43.455825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6a8a3-2212-46f3-9209-be6eaae1790d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:43.519684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e09196-f3a7-4fc8-ad73-86e3ff4cd96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:43.520637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e09196-f3a7-4fc8-ad73-86e3ff4cd96b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:43.526025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e09196-f3a7-4fc8-ad73-86e3ff4cd96b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:43.526025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e09196-f3a7-4fc8-ad73-86e3ff4cd96b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:43.526025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e09196-f3a7-4fc8-ad73-86e3ff4cd96b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:43.526025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e09196-f3a7-4fc8-ad73-86e3ff4cd96b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:43.538019 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2382/4743] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:43 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:43.541296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd75486-ece9-43e8-821b-0ac4a9bc232a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:43.568842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd75486-ece9-43e8-821b-0ac4a9bc232a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:43.575150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d33f3-7cdb-48cf-b26e-208bc1aefe51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:43.576193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d33f3-7cdb-48cf-b26e-208bc1aefe51 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:43.576429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d33f3-7cdb-48cf-b26e-208bc1aefe51 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:43.576832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d33f3-7cdb-48cf-b26e-208bc1aefe51 admin admin] PATH_INFO: `/v3/projects/cfecc9ade4ac4a2a9f2aa98b72464dbc/users/b788ac1b97cf4a99a1fdc800bb3b7dd9/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:43.579133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d33f3-7cdb-48cf-b26e-208bc1aefe51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfecc9ade4ac4a2a9f2aa98b72464dbc, user_id=b788ac1b97cf4a99a1fdc800bb3b7dd9, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:43.583298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d33f3-7cdb-48cf-b26e-208bc1aefe51 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:43.597749 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2362/4744] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:43 2022] PUT /identity/v3/projects/cfecc9ade4ac4a2a9f2aa98b72464dbc/users/b788ac1b97cf4a99a1fdc800bb3b7dd9/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:43.609972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c6530-a9b0-4694-be0d-82e2dbe8c99b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:43.612942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c6530-a9b0-4694-be0d-82e2dbe8c99b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:43.612942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c6530-a9b0-4694-be0d-82e2dbe8c99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:43.640144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-998c6530-a9b0-4694-be0d-82e2dbe8c99b None None] MFA Rules not processed for user `b788ac1b97cf4a99a1fdc800bb3b7dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:43.641598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-998c6530-a9b0-4694-be0d-82e2dbe8c99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:43.655740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:43.740497 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2383/4745] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:43.745810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d33f3-7cdb-48cf-b26e-208bc1aefe51 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:43.757315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d33f3-7cdb-48cf-b26e-208bc1aefe51 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:43.765016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e5a1f-8a28-4bba-a7af-fbc3742a356d tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b788ac1b97cf4a99a1fdc800bb3b7dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfecc9ade4ac4a2a9f2aa98b72464dbc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:43.765687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e5a1f-8a28-4bba-a7af-fbc3742a356d tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:43.766009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e5a1f-8a28-4bba-a7af-fbc3742a356d tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:43.766288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582e5a1f-8a28-4bba-a7af-fbc3742a356d tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:43.766981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e5a1f-8a28-4bba-a7af-fbc3742a356d tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:43.769392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582e5a1f-8a28-4bba-a7af-fbc3742a356d tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:43.909756 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2363/4746] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:43 2022] POST /identity/v3/projects => generated 358 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:43.913267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c6530-a9b0-4694-be0d-82e2dbe8c99b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:43.981268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c6530-a9b0-4694-be0d-82e2dbe8c99b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:44.038016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79fc97-7a90-470a-80dc-e83b48c0b373 tempest-ProjectsNegativeTestJSON-1520535430 tempest-ProjectsNegativeTestJSON-1520535430-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1a606dec4722416c8ab27b1aca243430', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '340440caf7e54bcdbea3a7fa768c4dad', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:44.038887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79fc97-7a90-470a-80dc-e83b48c0b373 tempest-ProjectsNegativeTestJSON-1520535430 tempest-ProjectsNegativeTestJSON-1520535430-project] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:44.039176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79fc97-7a90-470a-80dc-e83b48c0b373 tempest-ProjectsNegativeTestJSON-1520535430 tempest-ProjectsNegativeTestJSON-1520535430-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:44.039457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79fc97-7a90-470a-80dc-e83b48c0b373 tempest-ProjectsNegativeTestJSON-1520535430 tempest-ProjectsNegativeTestJSON-1520535430-project] PATH_INFO: `/v3/projects/7f31bd760a0b4ac48b2f62f449ea7328` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:44.040673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79fc97-7a90-470a-80dc-e83b48c0b373 tempest-ProjectsNegativeTestJSON-1520535430 tempest-ProjectsNegativeTestJSON-1520535430-project] RBAC: Authorizing `identity:delete_project(project_id=7f31bd760a0b4ac48b2f62f449ea7328)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:44.051714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-be79fc97-7a90-470a-80dc-e83b48c0b373 tempest-ProjectsNegativeTestJSON-1520535430 tempest-ProjectsNegativeTestJSON-1520535430-project] You are not authorized to perform the requested action: identity:delete_project.: keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. May 28 21:18:44.051714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:18:44.051714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:18:44.051714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:18:44.051714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:18:44.051714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:18:44.051714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:18:44.051714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:18:44.051714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:18:44.051714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:18:44.051714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:18:44.051714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:18:44.051714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 207, in delete May 28 21:18:44.051714 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call( May 28 21:18:44.058348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 455, in enforce_call May 28 21:18:44.058348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application enforcer_obj._enforce( May 28 21:18:44.058348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 125, in _enforce May 28 21:18:44.058348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application result = self._enforcer.enforce( May 28 21:18:44.058348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py", line 1088, in enforce May 28 21:18:44.058348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exc(*args, **kwargs) May 28 21:18:44.058348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ForbiddenAction: You are not authorized to perform the requested action: identity:delete_project. May 28 21:18:44.058348 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:18:44.058348 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2384/4747] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:43 2022] DELETE /identity/v3/projects/7f31bd760a0b4ac48b2f62f449ea7328 => generated 136 bytes in 140 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) May 28 21:18:44.059012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e5a1f-8a28-4bba-a7af-fbc3742a356d tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:44.144769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582e5a1f-8a28-4bba-a7af-fbc3742a356d tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:44.150434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6ea47-19f8-4a3b-8212-8f61fe7737ee tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b788ac1b97cf4a99a1fdc800bb3b7dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cfecc9ade4ac4a2a9f2aa98b72464dbc', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:44.151352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6ea47-19f8-4a3b-8212-8f61fe7737ee tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:44.151642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6ea47-19f8-4a3b-8212-8f61fe7737ee tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:44.151940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6ea47-19f8-4a3b-8212-8f61fe7737ee tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] PATH_INFO: `/v3/projects/7f31bd760a0b4ac48b2f62f449ea7328` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:44.153219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6ea47-19f8-4a3b-8212-8f61fe7737ee tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7f31bd760a0b4ac48b2f62f449ea7328)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:44.155614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6ea47-19f8-4a3b-8212-8f61fe7737ee tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:44.263749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cda6ea47-19f8-4a3b-8212-8f61fe7737ee tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f31bd760a0b4ac48b2f62f449ea7328', 'request_id': 'req-cda6ea47-19f8-4a3b-8212-8f61fe7737ee'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:44.271294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cda6ea47-19f8-4a3b-8212-8f61fe7737ee tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f31bd760a0b4ac48b2f62f449ea7328', 'request_id': 'req-cda6ea47-19f8-4a3b-8212-8f61fe7737ee'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:44.390906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cda6ea47-19f8-4a3b-8212-8f61fe7737ee tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f31bd760a0b4ac48b2f62f449ea7328', 'request_id': 'req-cda6ea47-19f8-4a3b-8212-8f61fe7737ee'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:44.403341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cda6ea47-19f8-4a3b-8212-8f61fe7737ee tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] The token cache is being invalidate because project 7f31bd760a0b4ac48b2f62f449ea7328 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:44.403675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cda6ea47-19f8-4a3b-8212-8f61fe7737ee tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:44.404412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cda6ea47-19f8-4a3b-8212-8f61fe7737ee tempest-ProjectsNegativeTestJSON-853740941 tempest-ProjectsNegativeTestJSON-853740941-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:44.407153 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2364/4748] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:44 2022] DELETE /identity/v3/projects/7f31bd760a0b4ac48b2f62f449ea7328 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:44.413321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79fc97-7a90-470a-80dc-e83b48c0b373 tempest-ProjectsNegativeTestJSON-1520535430 tempest-ProjectsNegativeTestJSON-1520535430-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:44.415640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-be79fc97-7a90-470a-80dc-e83b48c0b373 tempest-ProjectsNegativeTestJSON-1520535430 tempest-ProjectsNegativeTestJSON-1520535430-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:44.491065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79fc97-7a90-470a-80dc-e83b48c0b373 tempest-ProjectsNegativeTestJSON-1520535430 tempest-ProjectsNegativeTestJSON-1520535430-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:44.565606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb218f0-d82f-475f-9b43-d91200f8c1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:44.568199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb218f0-d82f-475f-9b43-d91200f8c1c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:44.568796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb218f0-d82f-475f-9b43-d91200f8c1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:44.569185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb218f0-d82f-475f-9b43-d91200f8c1c0 admin admin] PATH_INFO: `/v3/users/1a606dec4722416c8ab27b1aca243430` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:44.571625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb218f0-d82f-475f-9b43-d91200f8c1c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a606dec4722416c8ab27b1aca243430)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:44.574170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb218f0-d82f-475f-9b43-d91200f8c1c0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:44.807584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-aeb218f0-d82f-475f-9b43-d91200f8c1c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a606dec4722416c8ab27b1aca243430'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:44.816971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-aeb218f0-d82f-475f-9b43-d91200f8c1c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a606dec4722416c8ab27b1aca243430'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:44.818430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-aeb218f0-d82f-475f-9b43-d91200f8c1c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a606dec4722416c8ab27b1aca243430'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:44.819245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-aeb218f0-d82f-475f-9b43-d91200f8c1c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a606dec4722416c8ab27b1aca243430'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:44.894227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-aeb218f0-d82f-475f-9b43-d91200f8c1c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a606dec4722416c8ab27b1aca243430'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:44.912736 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2385/4749] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:44 2022] DELETE /identity/v3/users/1a606dec4722416c8ab27b1aca243430 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:44.918233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c638533-0115-47ee-a978-76184f20361f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:44.918521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c638533-0115-47ee-a978-76184f20361f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:44.918797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c638533-0115-47ee-a978-76184f20361f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:44.937527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4c638533-0115-47ee-a978-76184f20361f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:44.949497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4c638533-0115-47ee-a978-76184f20361f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:45.056523 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2365/4750] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:45.064237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb218f0-d82f-475f-9b43-d91200f8c1c0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:45.073381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-aeb218f0-d82f-475f-9b43-d91200f8c1c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:45.141356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb218f0-d82f-475f-9b43-d91200f8c1c0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:45.147798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b49bc-212f-43a2-8df4-46f682ba60c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:45.148767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b49bc-212f-43a2-8df4-46f682ba60c8 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:45.149277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b49bc-212f-43a2-8df4-46f682ba60c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:45.149773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b49bc-212f-43a2-8df4-46f682ba60c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:45.161829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b49bc-212f-43a2-8df4-46f682ba60c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:45.165234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881b49bc-212f-43a2-8df4-46f682ba60c8 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:45.175513 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2386/4751] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:45.192613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:45.264399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:45.480075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c638533-0115-47ee-a978-76184f20361f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:45.482244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4c638533-0115-47ee-a978-76184f20361f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:45.576050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c638533-0115-47ee-a978-76184f20361f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:45.580012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00504ce-a8eb-49f8-8d59-2ee806666749 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:45.581269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00504ce-a8eb-49f8-8d59-2ee806666749 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:45.581556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00504ce-a8eb-49f8-8d59-2ee806666749 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:45.581866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00504ce-a8eb-49f8-8d59-2ee806666749 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:45.583686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b00504ce-a8eb-49f8-8d59-2ee806666749 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:45.599005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00504ce-a8eb-49f8-8d59-2ee806666749 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:45.600725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00504ce-a8eb-49f8-8d59-2ee806666749 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:45.617874 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2366/4752] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:46.099380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b49bc-212f-43a2-8df4-46f682ba60c8 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:46.103316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-881b49bc-212f-43a2-8df4-46f682ba60c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:46.137632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881b49bc-212f-43a2-8df4-46f682ba60c8 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:46.149183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b1931-ef57-4230-9b82-1949e9a67d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:46.149183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b1931-ef57-4230-9b82-1949e9a67d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:46.149183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b1931-ef57-4230-9b82-1949e9a67d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:46.149183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b1931-ef57-4230-9b82-1949e9a67d83 admin admin] PATH_INFO: `/v3/projects/340440caf7e54bcdbea3a7fa768c4dad` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:46.149927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b1931-ef57-4230-9b82-1949e9a67d83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=340440caf7e54bcdbea3a7fa768c4dad)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:46.155940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b1931-ef57-4230-9b82-1949e9a67d83 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:46.295381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d34b1931-ef57-4230-9b82-1949e9a67d83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '340440caf7e54bcdbea3a7fa768c4dad', 'request_id': 'req-d34b1931-ef57-4230-9b82-1949e9a67d83'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:46.308662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d34b1931-ef57-4230-9b82-1949e9a67d83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '340440caf7e54bcdbea3a7fa768c4dad', 'request_id': 'req-d34b1931-ef57-4230-9b82-1949e9a67d83'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:46.413360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d34b1931-ef57-4230-9b82-1949e9a67d83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '340440caf7e54bcdbea3a7fa768c4dad', 'request_id': 'req-d34b1931-ef57-4230-9b82-1949e9a67d83'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:46.435965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d34b1931-ef57-4230-9b82-1949e9a67d83 admin admin] The token cache is being invalidate because project 340440caf7e54bcdbea3a7fa768c4dad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:46.436508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d34b1931-ef57-4230-9b82-1949e9a67d83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:46.440681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d34b1931-ef57-4230-9b82-1949e9a67d83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:46.440681 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2387/4753] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:46 2022] DELETE /identity/v3/projects/340440caf7e54bcdbea3a7fa768c4dad => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:46.443417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00504ce-a8eb-49f8-8d59-2ee806666749 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:46.459724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00504ce-a8eb-49f8-8d59-2ee806666749 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:46.521492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e2ea7-d29b-437e-8993-82ab09f92bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:46.522470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e2ea7-d29b-437e-8993-82ab09f92bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:46.522860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e2ea7-d29b-437e-8993-82ab09f92bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:46.523242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e2ea7-d29b-437e-8993-82ab09f92bfb admin admin] PATH_INFO: `/v3/users/b788ac1b97cf4a99a1fdc800bb3b7dd9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:46.524665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e2ea7-d29b-437e-8993-82ab09f92bfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b788ac1b97cf4a99a1fdc800bb3b7dd9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:46.526972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e2ea7-d29b-437e-8993-82ab09f92bfb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:46.799429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f31e2ea7-d29b-437e-8993-82ab09f92bfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b788ac1b97cf4a99a1fdc800bb3b7dd9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:46.800182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f31e2ea7-d29b-437e-8993-82ab09f92bfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b788ac1b97cf4a99a1fdc800bb3b7dd9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:46.807713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f31e2ea7-d29b-437e-8993-82ab09f92bfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b788ac1b97cf4a99a1fdc800bb3b7dd9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:46.864635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f31e2ea7-d29b-437e-8993-82ab09f92bfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b788ac1b97cf4a99a1fdc800bb3b7dd9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:46.865371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f31e2ea7-d29b-437e-8993-82ab09f92bfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b788ac1b97cf4a99a1fdc800bb3b7dd9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:46.887532 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2367/4754] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:46 2022] DELETE /identity/v3/users/b788ac1b97cf4a99a1fdc800bb3b7dd9 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:47.058556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:47.553801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b1931-ef57-4230-9b82-1949e9a67d83 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:47.555520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d34b1931-ef57-4230-9b82-1949e9a67d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:47.571876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b1931-ef57-4230-9b82-1949e9a67d83 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:47.636397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:47.656887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e34920-0e0d-4bd7-a91f-8492c80be88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:47.657768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e34920-0e0d-4bd7-a91f-8492c80be88a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:47.658642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e34920-0e0d-4bd7-a91f-8492c80be88a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:47.659133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e34920-0e0d-4bd7-a91f-8492c80be88a admin admin] PATH_INFO: `/v3/projects/cfecc9ade4ac4a2a9f2aa98b72464dbc` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:47.660377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e34920-0e0d-4bd7-a91f-8492c80be88a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfecc9ade4ac4a2a9f2aa98b72464dbc)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:47.662358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e34920-0e0d-4bd7-a91f-8492c80be88a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:47.814513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f4e34920-0e0d-4bd7-a91f-8492c80be88a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfecc9ade4ac4a2a9f2aa98b72464dbc', 'request_id': 'req-f4e34920-0e0d-4bd7-a91f-8492c80be88a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:47.828166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f4e34920-0e0d-4bd7-a91f-8492c80be88a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfecc9ade4ac4a2a9f2aa98b72464dbc', 'request_id': 'req-f4e34920-0e0d-4bd7-a91f-8492c80be88a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:47.916090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f4e34920-0e0d-4bd7-a91f-8492c80be88a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfecc9ade4ac4a2a9f2aa98b72464dbc', 'request_id': 'req-f4e34920-0e0d-4bd7-a91f-8492c80be88a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:47.930559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f4e34920-0e0d-4bd7-a91f-8492c80be88a admin admin] The token cache is being invalidate because project cfecc9ade4ac4a2a9f2aa98b72464dbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:47.930861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f4e34920-0e0d-4bd7-a91f-8492c80be88a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:47.931541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f4e34920-0e0d-4bd7-a91f-8492c80be88a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:47.933312 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2388/4755] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:47 2022] DELETE /identity/v3/projects/cfecc9ade4ac4a2a9f2aa98b72464dbc => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:18:47.940957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646b085-9f71-469e-b547-bcff674e6352 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:47.941461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646b085-9f71-469e-b547-bcff674e6352 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:47.941943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646b085-9f71-469e-b547-bcff674e6352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:47.967678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7646b085-9f71-469e-b547-bcff674e6352 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:47.969364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7646b085-9f71-469e-b547-bcff674e6352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:48.042937 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2368/4756] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:48.051620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b43f4a-b753-47b4-b338-6e9689c5718e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:48.051971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b43f4a-b753-47b4-b338-6e9689c5718e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:48.052257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b43f4a-b753-47b4-b338-6e9689c5718e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:48.075398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-25b43f4a-b753-47b4-b338-6e9689c5718e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:48.076717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-25b43f4a-b753-47b4-b338-6e9689c5718e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:48.086547 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2389/4757] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:48.091212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646b085-9f71-469e-b547-bcff674e6352 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:48.102282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646b085-9f71-469e-b547-bcff674e6352 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:48.107325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f617aff-d709-4b26-94f6-e2a31bbc5939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:48.107991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f617aff-d709-4b26-94f6-e2a31bbc5939 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:48.108316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f617aff-d709-4b26-94f6-e2a31bbc5939 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:48.108601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f617aff-d709-4b26-94f6-e2a31bbc5939 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:48.109286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f617aff-d709-4b26-94f6-e2a31bbc5939 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:18:48.109918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f617aff-d709-4b26-94f6-e2a31bbc5939 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:48.111554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f617aff-d709-4b26-94f6-e2a31bbc5939 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:48.123869 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2369/4758] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:18:48.128551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b43f4a-b753-47b4-b338-6e9689c5718e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:48.138918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b43f4a-b753-47b4-b338-6e9689c5718e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:48.145080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886d34a-206b-4472-a92b-c245e503a4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:48.145816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886d34a-206b-4472-a92b-c245e503a4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:48.146217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886d34a-206b-4472-a92b-c245e503a4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:48.146489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886d34a-206b-4472-a92b-c245e503a4c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:48.147233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886d34a-206b-4472-a92b-c245e503a4c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:48.149279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886d34a-206b-4472-a92b-c245e503a4c6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:48.226943 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2390/4759] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:48 2022] POST /identity/v3/projects => generated 391 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:18:48.227193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f617aff-d709-4b26-94f6-e2a31bbc5939 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:48.246208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f617aff-d709-4b26-94f6-e2a31bbc5939 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:48.322648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a63a9-f86e-49cd-a4bf-f8c4e30d654e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:48.324290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a63a9-f86e-49cd-a4bf-f8c4e30d654e admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:48.324597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a63a9-f86e-49cd-a4bf-f8c4e30d654e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:48.324903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a63a9-f86e-49cd-a4bf-f8c4e30d654e admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:48.325602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a63a9-f86e-49cd-a4bf-f8c4e30d654e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:48.328705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a63a9-f86e-49cd-a4bf-f8c4e30d654e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:48.440364 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2370/4760] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:48 2022] POST /identity/v3/users => generated 296 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:48.443018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886d34a-206b-4472-a92b-c245e503a4c6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:48.458750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886d34a-206b-4472-a92b-c245e503a4c6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:48.464811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b27960-bfb0-49b1-84fe-d696ec81a7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:48.465981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b27960-bfb0-49b1-84fe-d696ec81a7d5 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:48.466461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b27960-bfb0-49b1-84fe-d696ec81a7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:48.466910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b27960-bfb0-49b1-84fe-d696ec81a7d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:48.467847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b27960-bfb0-49b1-84fe-d696ec81a7d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:48.469928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b27960-bfb0-49b1-84fe-d696ec81a7d5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:48.483327 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2391/4761] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:48.487385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a63a9-f86e-49cd-a4bf-f8c4e30d654e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:48.503109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a63a9-f86e-49cd-a4bf-f8c4e30d654e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:48.520390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca53378-1f21-4136-bd09-cfb241d4c686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:48.521398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca53378-1f21-4136-bd09-cfb241d4c686 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:48.521860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca53378-1f21-4136-bd09-cfb241d4c686 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:48.522299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca53378-1f21-4136-bd09-cfb241d4c686 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:48.523233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca53378-1f21-4136-bd09-cfb241d4c686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:48.525213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca53378-1f21-4136-bd09-cfb241d4c686 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:48.547116 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2371/4762] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:48.554409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b27960-bfb0-49b1-84fe-d696ec81a7d5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:48.569432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b27960-bfb0-49b1-84fe-d696ec81a7d5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:48.576543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab6dcd-2531-4b51-a376-13e7c5935195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:48.577543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab6dcd-2531-4b51-a376-13e7c5935195 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:48.577946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab6dcd-2531-4b51-a376-13e7c5935195 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:48.578447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab6dcd-2531-4b51-a376-13e7c5935195 admin admin] PATH_INFO: `/v3/projects/42a896a87d1f41a3957c36e300dbe7eb/users/230ad340590c476793931c4f9a620f43/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:48.603681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab6dcd-2531-4b51-a376-13e7c5935195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42a896a87d1f41a3957c36e300dbe7eb, user_id=230ad340590c476793931c4f9a620f43, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:48.607787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab6dcd-2531-4b51-a376-13e7c5935195 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:48.707284 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2392/4763] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:48 2022] PUT /identity/v3/projects/42a896a87d1f41a3957c36e300dbe7eb/users/230ad340590c476793931c4f9a620f43/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:48.718155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea2a87-7015-409c-9fa9-1973b9abbd82 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:48.719306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea2a87-7015-409c-9fa9-1973b9abbd82 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:48.719674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea2a87-7015-409c-9fa9-1973b9abbd82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:48.758009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-33ea2a87-7015-409c-9fa9-1973b9abbd82 None None] MFA Rules not processed for user `230ad340590c476793931c4f9a620f43`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:48.759842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-33ea2a87-7015-409c-9fa9-1973b9abbd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:48.796475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:48.860819 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2372/4764] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:18:48.865804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab6dcd-2531-4b51-a376-13e7c5935195 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:48.883609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab6dcd-2531-4b51-a376-13e7c5935195 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:48.896207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd12e9-0758-40d2-92bc-e326ca16c03b tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '230ad340590c476793931c4f9a620f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42a896a87d1f41a3957c36e300dbe7eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:48.897755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd12e9-0758-40d2-92bc-e326ca16c03b tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:48.897755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd12e9-0758-40d2-92bc-e326ca16c03b tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:48.910627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd12e9-0758-40d2-92bc-e326ca16c03b tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43/application_credentials` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:48.914314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd12e9-0758-40d2-92bc-e326ca16c03b tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorizing `identity:create_application_credential(user_id=230ad340590c476793931c4f9a620f43)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:48.920438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd12e9-0758-40d2-92bc-e326ca16c03b tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:49.211694 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2393/4765] 10.222.0.33 () {60 vars in 1313 bytes} [Sat May 28 21:18:48 2022] POST /identity/v3/users/230ad340590c476793931c4f9a620f43/application_credentials => generated 789 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:18:49.215506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea2a87-7015-409c-9fa9-1973b9abbd82 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:49.240900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea2a87-7015-409c-9fa9-1973b9abbd82 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:49.261207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2981ed1-0da6-4b3d-8ba9-74db82fadb51 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '230ad340590c476793931c4f9a620f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42a896a87d1f41a3957c36e300dbe7eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:49.265163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2981ed1-0da6-4b3d-8ba9-74db82fadb51 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:49.267530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2981ed1-0da6-4b3d-8ba9-74db82fadb51 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:49.267954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2981ed1-0da6-4b3d-8ba9-74db82fadb51 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43/application_credentials/4ccc2371f32f43819310165dea91e0e8` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:49.306469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:49.310072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2981ed1-0da6-4b3d-8ba9-74db82fadb51 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorizing `identity:get_application_credential(user_id=230ad340590c476793931c4f9a620f43, application_credential_id=4ccc2371f32f43819310165dea91e0e8)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:49.312317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2981ed1-0da6-4b3d-8ba9-74db82fadb51 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:49.318256 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2373/4766] 10.222.0.33 () {58 vars in 1391 bytes} [Sat May 28 21:18:49 2022] GET /identity/v3/users/230ad340590c476793931c4f9a620f43/application_credentials/4ccc2371f32f43819310165dea91e0e8 => generated 689 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:18:49.323606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fa315-a110-4266-8751-813d0a56cebb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:49.323909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fa315-a110-4266-8751-813d0a56cebb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:49.324261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27fa315-a110-4266-8751-813d0a56cebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:49.362826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e27fa315-a110-4266-8751-813d0a56cebb None None] MFA Rules not processed for user `230ad340590c476793931c4f9a620f43`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:49.364086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e27fa315-a110-4266-8751-813d0a56cebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:49.548482 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2394/4767] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:49 2022] POST /identity/v3/auth/tokens => generated 3415 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) May 28 21:18:49.553086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2981ed1-0da6-4b3d-8ba9-74db82fadb51 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:49.571924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2981ed1-0da6-4b3d-8ba9-74db82fadb51 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:49.577491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df397c16-98a9-4192-a2b9-95f010cab652 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '230ad340590c476793931c4f9a620f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42a896a87d1f41a3957c36e300dbe7eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:49.579999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df397c16-98a9-4192-a2b9-95f010cab652 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:49.579999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df397c16-98a9-4192-a2b9-95f010cab652 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:49.579999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df397c16-98a9-4192-a2b9-95f010cab652 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43/application_credentials/4ccc2371f32f43819310165dea91e0e8` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:49.580634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df397c16-98a9-4192-a2b9-95f010cab652 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorizing `identity:delete_application_credential(user_id=230ad340590c476793931c4f9a620f43, application_credential_id=4ccc2371f32f43819310165dea91e0e8)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:49.582450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df397c16-98a9-4192-a2b9-95f010cab652 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:49.663865 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2374/4768] 10.222.0.33 () {58 vars in 1394 bytes} [Sat May 28 21:18:49 2022] DELETE /identity/v3/users/230ad340590c476793931c4f9a620f43/application_credentials/4ccc2371f32f43819310165dea91e0e8 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:49.669588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fa315-a110-4266-8751-813d0a56cebb None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:49.688628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27fa315-a110-4266-8751-813d0a56cebb None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:49.695511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a064c-2092-45da-9d82-71ba7599d7bc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '230ad340590c476793931c4f9a620f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42a896a87d1f41a3957c36e300dbe7eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:49.701155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a064c-2092-45da-9d82-71ba7599d7bc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:49.701155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a064c-2092-45da-9d82-71ba7599d7bc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:49.701155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a064c-2092-45da-9d82-71ba7599d7bc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43/application_credentials` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:49.701155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a064c-2092-45da-9d82-71ba7599d7bc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorizing `identity:create_application_credential(user_id=230ad340590c476793931c4f9a620f43)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:49.702896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a064c-2092-45da-9d82-71ba7599d7bc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:49.880511 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2395/4769] 10.222.0.33 () {60 vars in 1314 bytes} [Sat May 28 21:18:49 2022] POST /identity/v3/users/230ad340590c476793931c4f9a620f43/application_credentials => generated 921 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:49.885161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df397c16-98a9-4192-a2b9-95f010cab652 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:49.895029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df397c16-98a9-4192-a2b9-95f010cab652 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:49.899466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61ee97-d88d-4aa4-9659-486d1deb469f tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '230ad340590c476793931c4f9a620f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42a896a87d1f41a3957c36e300dbe7eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:49.900404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61ee97-d88d-4aa4-9659-486d1deb469f tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:49.900692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61ee97-d88d-4aa4-9659-486d1deb469f tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:49.900969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db61ee97-d88d-4aa4-9659-486d1deb469f tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43/application_credentials/82555f50eefe4eceb026c3c8e0a13373` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:49.913281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61ee97-d88d-4aa4-9659-486d1deb469f tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorizing `identity:delete_application_credential(user_id=230ad340590c476793931c4f9a620f43, application_credential_id=82555f50eefe4eceb026c3c8e0a13373)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:49.915004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db61ee97-d88d-4aa4-9659-486d1deb469f tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:49.985792 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2375/4770] 10.222.0.33 () {58 vars in 1394 bytes} [Sat May 28 21:18:49 2022] DELETE /identity/v3/users/230ad340590c476793931c4f9a620f43/application_credentials/82555f50eefe4eceb026c3c8e0a13373 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:49.992126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a064c-2092-45da-9d82-71ba7599d7bc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:50.002859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a064c-2092-45da-9d82-71ba7599d7bc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:50.009044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0e515-fe29-420f-9ae3-e589fc39995d tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '230ad340590c476793931c4f9a620f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42a896a87d1f41a3957c36e300dbe7eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:50.012449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0e515-fe29-420f-9ae3-e589fc39995d tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:50.012449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0e515-fe29-420f-9ae3-e589fc39995d tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:50.012449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd0e515-fe29-420f-9ae3-e589fc39995d tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43/application_credentials` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:50.012449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0e515-fe29-420f-9ae3-e589fc39995d tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorizing `identity:create_application_credential(user_id=230ad340590c476793931c4f9a620f43)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:50.012946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd0e515-fe29-420f-9ae3-e589fc39995d tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:50.222562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:50.234516 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2396/4771] 10.222.0.33 () {60 vars in 1314 bytes} [Sat May 28 21:18:49 2022] POST /identity/v3/users/230ad340590c476793931c4f9a620f43/application_credentials => generated 813 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:50.234782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61ee97-d88d-4aa4-9659-486d1deb469f tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:50.244759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db61ee97-d88d-4aa4-9659-486d1deb469f tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:50.249381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a1b3e-81c1-482a-b65a-0b56057cd17e tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '230ad340590c476793931c4f9a620f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42a896a87d1f41a3957c36e300dbe7eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:50.250637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a1b3e-81c1-482a-b65a-0b56057cd17e tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:50.250925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a1b3e-81c1-482a-b65a-0b56057cd17e tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:50.251254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a1b3e-81c1-482a-b65a-0b56057cd17e tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43/application_credentials/b8d8e0b32dfb41b287e85ef5c102c439` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:50.261755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a1b3e-81c1-482a-b65a-0b56057cd17e tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorizing `identity:delete_application_credential(user_id=230ad340590c476793931c4f9a620f43, application_credential_id=b8d8e0b32dfb41b287e85ef5c102c439)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:50.263399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004a1b3e-81c1-482a-b65a-0b56057cd17e tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:50.419811 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2376/4772] 10.222.0.33 () {58 vars in 1394 bytes} [Sat May 28 21:18:50 2022] DELETE /identity/v3/users/230ad340590c476793931c4f9a620f43/application_credentials/b8d8e0b32dfb41b287e85ef5c102c439 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:50.425043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0e515-fe29-420f-9ae3-e589fc39995d tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:50.435559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd0e515-fe29-420f-9ae3-e589fc39995d tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:50.443362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e85ae-c19f-4613-9645-0c264ffa5335 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '230ad340590c476793931c4f9a620f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42a896a87d1f41a3957c36e300dbe7eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:50.443856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e85ae-c19f-4613-9645-0c264ffa5335 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:50.444198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e85ae-c19f-4613-9645-0c264ffa5335 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:50.444552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e85ae-c19f-4613-9645-0c264ffa5335 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43/application_credentials` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:50.445392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e85ae-c19f-4613-9645-0c264ffa5335 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorizing `identity:create_application_credential(user_id=230ad340590c476793931c4f9a620f43)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:50.446707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735e85ae-c19f-4613-9645-0c264ffa5335 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:50.683626 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2397/4773] 10.222.0.33 () {60 vars in 1313 bytes} [Sat May 28 21:18:50 2022] POST /identity/v3/users/230ad340590c476793931c4f9a620f43/application_credentials => generated 788 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:50.689481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a1b3e-81c1-482a-b65a-0b56057cd17e tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:50.709872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004a1b3e-81c1-482a-b65a-0b56057cd17e tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:50.716272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e4519-79f8-46f1-8767-3e144cc7c0df tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '230ad340590c476793931c4f9a620f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42a896a87d1f41a3957c36e300dbe7eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:50.720150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e4519-79f8-46f1-8767-3e144cc7c0df tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:50.722609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e4519-79f8-46f1-8767-3e144cc7c0df tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:50.723379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e4519-79f8-46f1-8767-3e144cc7c0df tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43/application_credentials` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:50.724636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e4519-79f8-46f1-8767-3e144cc7c0df tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorizing `identity:create_application_credential(user_id=230ad340590c476793931c4f9a620f43)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:50.727400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7e4519-79f8-46f1-8767-3e144cc7c0df tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:50.912795 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2377/4774] 10.222.0.33 () {60 vars in 1313 bytes} [Sat May 28 21:18:50 2022] POST /identity/v3/users/230ad340590c476793931c4f9a620f43/application_credentials => generated 789 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:50.917796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e85ae-c19f-4613-9645-0c264ffa5335 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:50.932926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735e85ae-c19f-4613-9645-0c264ffa5335 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:50.943975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc929d23-419f-4d78-aacc-b129eb84ce39 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '230ad340590c476793931c4f9a620f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42a896a87d1f41a3957c36e300dbe7eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:50.949821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc929d23-419f-4d78-aacc-b129eb84ce39 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:50.949821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc929d23-419f-4d78-aacc-b129eb84ce39 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:50.949821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc929d23-419f-4d78-aacc-b129eb84ce39 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43/application_credentials` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:50.949821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc929d23-419f-4d78-aacc-b129eb84ce39 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorizing `identity:list_application_credentials(user_id=230ad340590c476793931c4f9a620f43)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:50.950272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc929d23-419f-4d78-aacc-b129eb84ce39 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:50.962390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:50.967656 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2398/4775] 10.222.0.33 () {58 vars in 1292 bytes} [Sat May 28 21:18:50 2022] GET /identity/v3/users/230ad340590c476793931c4f9a620f43/application_credentials => generated 1501 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:50.969059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e4519-79f8-46f1-8767-3e144cc7c0df tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:50.979265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e4519-79f8-46f1-8767-3e144cc7c0df tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:50.984044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb3d67-52c5-4c83-a587-9efa83bbc408 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '230ad340590c476793931c4f9a620f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42a896a87d1f41a3957c36e300dbe7eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:50.985014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb3d67-52c5-4c83-a587-9efa83bbc408 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:50.985305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb3d67-52c5-4c83-a587-9efa83bbc408 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:50.985664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb3d67-52c5-4c83-a587-9efa83bbc408 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43/application_credentials/40c6cf04835c466cb054ab67ca2a1749` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:50.996383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb3d67-52c5-4c83-a587-9efa83bbc408 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorizing `identity:delete_application_credential(user_id=230ad340590c476793931c4f9a620f43, application_credential_id=40c6cf04835c466cb054ab67ca2a1749)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:50.997964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb3d67-52c5-4c83-a587-9efa83bbc408 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:51.103449 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2378/4776] 10.222.0.33 () {58 vars in 1394 bytes} [Sat May 28 21:18:50 2022] DELETE /identity/v3/users/230ad340590c476793931c4f9a620f43/application_credentials/40c6cf04835c466cb054ab67ca2a1749 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:18:51.106068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc929d23-419f-4d78-aacc-b129eb84ce39 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:51.115432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc929d23-419f-4d78-aacc-b129eb84ce39 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:51.120311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d5dd2-5bcb-4fd7-896a-0e075e327226 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '230ad340590c476793931c4f9a620f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42a896a87d1f41a3957c36e300dbe7eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:51.121297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d5dd2-5bcb-4fd7-896a-0e075e327226 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:51.121535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d5dd2-5bcb-4fd7-896a-0e075e327226 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:51.121969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5d5dd2-5bcb-4fd7-896a-0e075e327226 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43/application_credentials/28ac27b28258498aa86e9d8d69a01629` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:51.132934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d5dd2-5bcb-4fd7-896a-0e075e327226 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorizing `identity:delete_application_credential(user_id=230ad340590c476793931c4f9a620f43, application_credential_id=28ac27b28258498aa86e9d8d69a01629)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:51.134823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5d5dd2-5bcb-4fd7-896a-0e075e327226 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:51.218425 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2399/4777] 10.222.0.33 () {58 vars in 1394 bytes} [Sat May 28 21:18:51 2022] DELETE /identity/v3/users/230ad340590c476793931c4f9a620f43/application_credentials/28ac27b28258498aa86e9d8d69a01629 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:51.224156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb3d67-52c5-4c83-a587-9efa83bbc408 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:51.234341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb3d67-52c5-4c83-a587-9efa83bbc408 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:51.244578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616f3e0-f6db-41f5-a58a-92031af0cdcc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '230ad340590c476793931c4f9a620f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42a896a87d1f41a3957c36e300dbe7eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:51.245484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616f3e0-f6db-41f5-a58a-92031af0cdcc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:51.245930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616f3e0-f6db-41f5-a58a-92031af0cdcc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:51.246229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616f3e0-f6db-41f5-a58a-92031af0cdcc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43/application_credentials` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:51.247047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616f3e0-f6db-41f5-a58a-92031af0cdcc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorizing `identity:create_application_credential(user_id=230ad340590c476793931c4f9a620f43)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:51.248273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a616f3e0-f6db-41f5-a58a-92031af0cdcc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:51.412310 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2379/4778] 10.222.0.33 () {60 vars in 1313 bytes} [Sat May 28 21:18:51 2022] POST /identity/v3/users/230ad340590c476793931c4f9a620f43/application_credentials => generated 788 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:51.415546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d5dd2-5bcb-4fd7-896a-0e075e327226 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:51.430480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5d5dd2-5bcb-4fd7-896a-0e075e327226 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:51.434868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003bb74-5b08-43b3-b597-8c707a81b69b tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '230ad340590c476793931c4f9a620f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42a896a87d1f41a3957c36e300dbe7eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:51.440010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003bb74-5b08-43b3-b597-8c707a81b69b tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:51.440010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003bb74-5b08-43b3-b597-8c707a81b69b tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:51.440010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003bb74-5b08-43b3-b597-8c707a81b69b tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43/application_credentials` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:51.440010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003bb74-5b08-43b3-b597-8c707a81b69b tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorizing `identity:create_application_credential(user_id=230ad340590c476793931c4f9a620f43)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:51.440635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003bb74-5b08-43b3-b597-8c707a81b69b tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:51.572069 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2400/4779] 10.222.0.33 () {60 vars in 1313 bytes} [Sat May 28 21:18:51 2022] POST /identity/v3/users/230ad340590c476793931c4f9a620f43/application_credentials => generated 788 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:18:51.574609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616f3e0-f6db-41f5-a58a-92031af0cdcc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:51.585039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616f3e0-f6db-41f5-a58a-92031af0cdcc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:51.589790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d2af8-1d06-4cb8-9c13-86ab7cc5d295 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '230ad340590c476793931c4f9a620f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42a896a87d1f41a3957c36e300dbe7eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:51.590707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d2af8-1d06-4cb8-9c13-86ab7cc5d295 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:51.590998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d2af8-1d06-4cb8-9c13-86ab7cc5d295 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:51.591280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d2af8-1d06-4cb8-9c13-86ab7cc5d295 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43/application_credentials` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:51.592010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d2af8-1d06-4cb8-9c13-86ab7cc5d295 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Adding query filter params (name=tempest-application_credential-993496101) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:18:51.592339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d2af8-1d06-4cb8-9c13-86ab7cc5d295 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorizing `identity:list_application_credentials(user_id=230ad340590c476793931c4f9a620f43)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:51.594016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d2af8-1d06-4cb8-9c13-86ab7cc5d295 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:51.604802 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2380/4780] 10.222.0.33 () {58 vars in 1383 bytes} [Sat May 28 21:18:51 2022] GET /identity/v3/users/230ad340590c476793931c4f9a620f43/application_credentials?name=tempest-application_credential-993496101 => generated 885 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:18:51.609494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003bb74-5b08-43b3-b597-8c707a81b69b tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:51.619430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003bb74-5b08-43b3-b597-8c707a81b69b tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:51.623813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4239b-5f85-4a47-acf6-f34c65ded954 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '230ad340590c476793931c4f9a620f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42a896a87d1f41a3957c36e300dbe7eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:51.624771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4239b-5f85-4a47-acf6-f34c65ded954 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:51.625075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4239b-5f85-4a47-acf6-f34c65ded954 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:51.625350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4239b-5f85-4a47-acf6-f34c65ded954 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43/application_credentials/9d957843e7464692a6c6480938302a33` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:51.646774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4239b-5f85-4a47-acf6-f34c65ded954 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorizing `identity:delete_application_credential(user_id=230ad340590c476793931c4f9a620f43, application_credential_id=9d957843e7464692a6c6480938302a33)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:51.650023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4239b-5f85-4a47-acf6-f34c65ded954 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:51.756291 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2401/4781] 10.222.0.33 () {58 vars in 1394 bytes} [Sat May 28 21:18:51 2022] DELETE /identity/v3/users/230ad340590c476793931c4f9a620f43/application_credentials/9d957843e7464692a6c6480938302a33 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:51.763831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d2af8-1d06-4cb8-9c13-86ab7cc5d295 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:51.764608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:51.775523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d2af8-1d06-4cb8-9c13-86ab7cc5d295 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:51.781059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b1960-7a2c-4f57-b892-b9162a4da3bc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '230ad340590c476793931c4f9a620f43', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '42a896a87d1f41a3957c36e300dbe7eb', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:51.782970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b1960-7a2c-4f57-b892-b9162a4da3bc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:51.783315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b1960-7a2c-4f57-b892-b9162a4da3bc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:51.783700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b1960-7a2c-4f57-b892-b9162a4da3bc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43/application_credentials/46cebf7d248b482b97384e2b7b9095b3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:51.797060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b1960-7a2c-4f57-b892-b9162a4da3bc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorizing `identity:delete_application_credential(user_id=230ad340590c476793931c4f9a620f43, application_credential_id=46cebf7d248b482b97384e2b7b9095b3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:51.799674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403b1960-7a2c-4f57-b892-b9162a4da3bc tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:51.892655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4239b-5f85-4a47-acf6-f34c65ded954 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:51.893746 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2381/4782] 10.222.0.33 () {58 vars in 1394 bytes} [Sat May 28 21:18:51 2022] DELETE /identity/v3/users/230ad340590c476793931c4f9a620f43/application_credentials/46cebf7d248b482b97384e2b7b9095b3 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:51.998604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4239b-5f85-4a47-acf6-f34c65ded954 tempest-ApplicationCredentialsV3Test-520970632 tempest-ApplicationCredentialsV3Test-520970632-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:52.012077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd8e40-dbc4-42ba-92fe-684a5f0ee610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:52.012986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd8e40-dbc4-42ba-92fe-684a5f0ee610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:52.013277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd8e40-dbc4-42ba-92fe-684a5f0ee610 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:52.013552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd8e40-dbc4-42ba-92fe-684a5f0ee610 admin admin] PATH_INFO: `/v3/users/230ad340590c476793931c4f9a620f43` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:52.018249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd8e40-dbc4-42ba-92fe-684a5f0ee610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=230ad340590c476793931c4f9a620f43)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:52.020210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd8e40-dbc4-42ba-92fe-684a5f0ee610 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:52.318673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6ccd8e40-dbc4-42ba-92fe-684a5f0ee610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '230ad340590c476793931c4f9a620f43'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:52.325901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6ccd8e40-dbc4-42ba-92fe-684a5f0ee610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '230ad340590c476793931c4f9a620f43'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:52.326753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6ccd8e40-dbc4-42ba-92fe-684a5f0ee610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '230ad340590c476793931c4f9a620f43'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:52.327378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6ccd8e40-dbc4-42ba-92fe-684a5f0ee610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '230ad340590c476793931c4f9a620f43'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:52.413783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6ccd8e40-dbc4-42ba-92fe-684a5f0ee610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '230ad340590c476793931c4f9a620f43'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:52.520053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64863d2-3066-4e15-9d53-58b9bd12d847 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:52.521750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64863d2-3066-4e15-9d53-58b9bd12d847 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:52.523101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64863d2-3066-4e15-9d53-58b9bd12d847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:52.524876 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2402/4783] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:51 2022] DELETE /identity/v3/users/230ad340590c476793931c4f9a620f43 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:52.546803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd8e40-dbc4-42ba-92fe-684a5f0ee610 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:52.549300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6ccd8e40-dbc4-42ba-92fe-684a5f0ee610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:52.555817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b64863d2-3066-4e15-9d53-58b9bd12d847 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:52.558334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b64863d2-3066-4e15-9d53-58b9bd12d847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:52.680897 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2382/4784] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:52.686941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd8e40-dbc4-42ba-92fe-684a5f0ee610 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:52.697194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46697ce3-fd32-40b5-9f68-0ec823fa5b2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:52.699630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46697ce3-fd32-40b5-9f68-0ec823fa5b2a service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:52.700158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46697ce3-fd32-40b5-9f68-0ec823fa5b2a service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:52.701273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46697ce3-fd32-40b5-9f68-0ec823fa5b2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:52.704285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64863d2-3066-4e15-9d53-58b9bd12d847 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:52.711087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-46697ce3-fd32-40b5-9f68-0ec823fa5b2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:52.728235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46697ce3-fd32-40b5-9f68-0ec823fa5b2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:52.730313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46697ce3-fd32-40b5-9f68-0ec823fa5b2a service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:52.734102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64863d2-3066-4e15-9d53-58b9bd12d847 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:52.737791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:52.743186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ff59f-00c1-4702-9365-fdc43ddbae8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:52.744300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ff59f-00c1-4702-9365-fdc43ddbae8d service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:52.744692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ff59f-00c1-4702-9365-fdc43ddbae8d service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:52.745079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ff59f-00c1-4702-9365-fdc43ddbae8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:52.753965 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2403/4785] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:18:52.756352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ff59f-00c1-4702-9365-fdc43ddbae8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:52.758336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9ff59f-00c1-4702-9365-fdc43ddbae8d service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:52.776498 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2383/4786] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:53.521307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:54.453175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46697ce3-fd32-40b5-9f68-0ec823fa5b2a service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:54.458275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-46697ce3-fd32-40b5-9f68-0ec823fa5b2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:54.490170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ff59f-00c1-4702-9365-fdc43ddbae8d service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:54.493154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2e9ff59f-00c1-4702-9365-fdc43ddbae8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:54.502101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46697ce3-fd32-40b5-9f68-0ec823fa5b2a service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:54.510780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a20d42-795c-4b76-8dcb-1e6893ba01d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:54.511988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a20d42-795c-4b76-8dcb-1e6893ba01d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:54.512610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a20d42-795c-4b76-8dcb-1e6893ba01d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:54.514498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a20d42-795c-4b76-8dcb-1e6893ba01d1 admin admin] PATH_INFO: `/v3/projects/42a896a87d1f41a3957c36e300dbe7eb` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:54.523167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a20d42-795c-4b76-8dcb-1e6893ba01d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42a896a87d1f41a3957c36e300dbe7eb)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:54.526095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a20d42-795c-4b76-8dcb-1e6893ba01d1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:54.567489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:54.609484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ff59f-00c1-4702-9365-fdc43ddbae8d service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:54.615651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d618a6e-ccfe-4e42-ad6f-a2ead849f8ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:54.616396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d618a6e-ccfe-4e42-ad6f-a2ead849f8ba service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:54.616748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d618a6e-ccfe-4e42-ad6f-a2ead849f8ba service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:54.617038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d618a6e-ccfe-4e42-ad6f-a2ead849f8ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:54.618962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6d618a6e-ccfe-4e42-ad6f-a2ead849f8ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:54.633887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d618a6e-ccfe-4e42-ad6f-a2ead849f8ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:54.635526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d618a6e-ccfe-4e42-ad6f-a2ead849f8ba service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:54.648585 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2384/4787] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:54.649400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d618a6e-ccfe-4e42-ad6f-a2ead849f8ba service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:54.650799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6d618a6e-ccfe-4e42-ad6f-a2ead849f8ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:54.670935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d618a6e-ccfe-4e42-ad6f-a2ead849f8ba service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:54.683548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-81a20d42-795c-4b76-8dcb-1e6893ba01d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42a896a87d1f41a3957c36e300dbe7eb', 'request_id': 'req-81a20d42-795c-4b76-8dcb-1e6893ba01d1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:54.690509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-81a20d42-795c-4b76-8dcb-1e6893ba01d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42a896a87d1f41a3957c36e300dbe7eb', 'request_id': 'req-81a20d42-795c-4b76-8dcb-1e6893ba01d1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:54.774115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fd5f7-8f0b-4613-80fb-83e2bb68b9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:54.775019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fd5f7-8f0b-4613-80fb-83e2bb68b9ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:54.775312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fd5f7-8f0b-4613-80fb-83e2bb68b9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:54.775682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fd5f7-8f0b-4613-80fb-83e2bb68b9ad admin admin] PATH_INFO: `/v3/users/14b9e4bebdd949319785e03cda251f1c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:54.777698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fd5f7-8f0b-4613-80fb-83e2bb68b9ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=14b9e4bebdd949319785e03cda251f1c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:54.779847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fd5f7-8f0b-4613-80fb-83e2bb68b9ad admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:54.807519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-81a20d42-795c-4b76-8dcb-1e6893ba01d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42a896a87d1f41a3957c36e300dbe7eb', 'request_id': 'req-81a20d42-795c-4b76-8dcb-1e6893ba01d1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:54.822589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-81a20d42-795c-4b76-8dcb-1e6893ba01d1 admin admin] The token cache is being invalidate because project 42a896a87d1f41a3957c36e300dbe7eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:54.822996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-81a20d42-795c-4b76-8dcb-1e6893ba01d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:54.823935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-81a20d42-795c-4b76-8dcb-1e6893ba01d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:54.826987 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2404/4788] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:54 2022] DELETE /identity/v3/projects/42a896a87d1f41a3957c36e300dbe7eb => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:54.833600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e2f37-3d22-4a32-8b79-7297212b0f27 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:54.834732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e2f37-3d22-4a32-8b79-7297212b0f27 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:54.835523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e2f37-3d22-4a32-8b79-7297212b0f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:54.874321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f90e2f37-3d22-4a32-8b79-7297212b0f27 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:54.875840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f90e2f37-3d22-4a32-8b79-7297212b0f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:54.886513 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2405/4789] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:54.896625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88eee9-47f7-4b99-b9bb-529043ece89f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:54.902268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88eee9-47f7-4b99-b9bb-529043ece89f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:54.902268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88eee9-47f7-4b99-b9bb-529043ece89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:54.916473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0b88eee9-47f7-4b99-b9bb-529043ece89f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:54.917704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0b88eee9-47f7-4b99-b9bb-529043ece89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:54.935729 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2406/4790] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:54.941118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88eee9-47f7-4b99-b9bb-529043ece89f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:54.953953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88eee9-47f7-4b99-b9bb-529043ece89f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:54.970696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70eaf7-3d82-4d04-8e0e-351901e58315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:54.971441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70eaf7-3d82-4d04-8e0e-351901e58315 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:54.971936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70eaf7-3d82-4d04-8e0e-351901e58315 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:54.972382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70eaf7-3d82-4d04-8e0e-351901e58315 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:54.973426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70eaf7-3d82-4d04-8e0e-351901e58315 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:18:54.974664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70eaf7-3d82-4d04-8e0e-351901e58315 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:54.978750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70eaf7-3d82-4d04-8e0e-351901e58315 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:54.994761 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2407/4791] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:18:55.002582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70eaf7-3d82-4d04-8e0e-351901e58315 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:55.013784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70eaf7-3d82-4d04-8e0e-351901e58315 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:55.022828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ee7fd5f7-8f0b-4613-80fb-83e2bb68b9ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14b9e4bebdd949319785e03cda251f1c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:55.023913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ee7fd5f7-8f0b-4613-80fb-83e2bb68b9ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14b9e4bebdd949319785e03cda251f1c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:55.028505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c8d2d-8d33-4817-b68e-45132c351759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:55.029449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c8d2d-8d33-4817-b68e-45132c351759 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:55.029777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c8d2d-8d33-4817-b68e-45132c351759 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:55.030156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c8d2d-8d33-4817-b68e-45132c351759 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:55.030951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c8d2d-8d33-4817-b68e-45132c351759 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:55.033035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265c8d2d-8d33-4817-b68e-45132c351759 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:55.036715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ee7fd5f7-8f0b-4613-80fb-83e2bb68b9ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14b9e4bebdd949319785e03cda251f1c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:55.150037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ee7fd5f7-8f0b-4613-80fb-83e2bb68b9ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14b9e4bebdd949319785e03cda251f1c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:55.150742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ee7fd5f7-8f0b-4613-80fb-83e2bb68b9ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14b9e4bebdd949319785e03cda251f1c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:55.181435 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2385/4792] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:54 2022] DELETE /identity/v3/users/14b9e4bebdd949319785e03cda251f1c => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:55.186814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200fb27-a9e0-4b44-91e4-b640d5b7d272 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:55.186814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200fb27-a9e0-4b44-91e4-b640d5b7d272 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:55.186814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200fb27-a9e0-4b44-91e4-b640d5b7d272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:55.189165 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2408/4793] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:55 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:18:55.190250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c8d2d-8d33-4817-b68e-45132c351759 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:55.191940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-265c8d2d-8d33-4817-b68e-45132c351759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:55.206916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1200fb27-a9e0-4b44-91e4-b640d5b7d272 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:55.208466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1200fb27-a9e0-4b44-91e4-b640d5b7d272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:55.281891 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2386/4794] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:55.291049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200fb27-a9e0-4b44-91e4-b640d5b7d272 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:55.295908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:55.296744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1200fb27-a9e0-4b44-91e4-b640d5b7d272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:55.298077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c8d2d-8d33-4817-b68e-45132c351759 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:55.305534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002079f6-c5f5-441a-8980-56d0fb075b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:55.306091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002079f6-c5f5-441a-8980-56d0fb075b1b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:55.306576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002079f6-c5f5-441a-8980-56d0fb075b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:55.307021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002079f6-c5f5-441a-8980-56d0fb075b1b admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:55.307732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002079f6-c5f5-441a-8980-56d0fb075b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:55.310413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002079f6-c5f5-441a-8980-56d0fb075b1b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:55.370586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200fb27-a9e0-4b44-91e4-b640d5b7d272 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:55.374495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2c5e7-5224-46e3-a4e6-e8d980d2c7ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:55.375146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2c5e7-5224-46e3-a4e6-e8d980d2c7ec service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:55.375436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2c5e7-5224-46e3-a4e6-e8d980d2c7ec service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:55.375716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2c5e7-5224-46e3-a4e6-e8d980d2c7ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:55.385084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2c5e7-5224-46e3-a4e6-e8d980d2c7ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:55.386579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e2c5e7-5224-46e3-a4e6-e8d980d2c7ec service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:55.400913 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2387/4795] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:55.401885 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2409/4796] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:55 2022] POST /identity/v3/users => generated 299 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:55.411640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2c5e7-5224-46e3-a4e6-e8d980d2c7ec service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:55.422902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2c5e7-5224-46e3-a4e6-e8d980d2c7ec service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:55.430268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9583b-ce37-4ac4-9852-e2dfef69c7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:55.430959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9583b-ce37-4ac4-9852-e2dfef69c7dd admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:55.431252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9583b-ce37-4ac4-9852-e2dfef69c7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:55.431684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f9583b-ce37-4ac4-9852-e2dfef69c7dd admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:55.432631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9583b-ce37-4ac4-9852-e2dfef69c7dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:55.434314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f9583b-ce37-4ac4-9852-e2dfef69c7dd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:55.449747 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2388/4797] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:55.450825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002079f6-c5f5-441a-8980-56d0fb075b1b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:55.469545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002079f6-c5f5-441a-8980-56d0fb075b1b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:55.477399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5f23f-56ed-412b-9bc6-0f4a8251e28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:55.479670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5f23f-56ed-412b-9bc6-0f4a8251e28a admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:55.480402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5f23f-56ed-412b-9bc6-0f4a8251e28a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:55.480864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5f23f-56ed-412b-9bc6-0f4a8251e28a admin admin] PATH_INFO: `/v3/projects/ac2f9c5b90db4fd4b63d0c2e151e0a58/users/eee60bc4cd2c4e9abcc48270bc2dca2f/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:55.501129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5f23f-56ed-412b-9bc6-0f4a8251e28a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac2f9c5b90db4fd4b63d0c2e151e0a58, user_id=eee60bc4cd2c4e9abcc48270bc2dca2f, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:55.503810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5f23f-56ed-412b-9bc6-0f4a8251e28a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:55.660176 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2410/4798] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:55 2022] PUT /identity/v3/projects/ac2f9c5b90db4fd4b63d0c2e151e0a58/users/eee60bc4cd2c4e9abcc48270bc2dca2f/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:18:55.664494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9583b-ce37-4ac4-9852-e2dfef69c7dd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:55.674574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f9583b-ce37-4ac4-9852-e2dfef69c7dd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:55.731169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5f23f-56ed-412b-9bc6-0f4a8251e28a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:55.733927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4fe5f23f-56ed-412b-9bc6-0f4a8251e28a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:55.741369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09e4fa-5803-4a28-865e-695bb367ba13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:55.741999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09e4fa-5803-4a28-865e-695bb367ba13 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:55.742299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09e4fa-5803-4a28-865e-695bb367ba13 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:55.742692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09e4fa-5803-4a28-865e-695bb367ba13 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:55.743524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09e4fa-5803-4a28-865e-695bb367ba13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:55.748366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a09e4fa-5803-4a28-865e-695bb367ba13 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:55.759538 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2389/4799] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:55.760850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09e4fa-5803-4a28-865e-695bb367ba13 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:55.778460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a09e4fa-5803-4a28-865e-695bb367ba13 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:55.778581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5f23f-56ed-412b-9bc6-0f4a8251e28a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:55.780453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d63dcfe-0f7f-4ff3-99ac-c945ed52934e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:55.781683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63dcfe-0f7f-4ff3-99ac-c945ed52934e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:55.782021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63dcfe-0f7f-4ff3-99ac-c945ed52934e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:55.782314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d63dcfe-0f7f-4ff3-99ac-c945ed52934e admin admin] PATH_INFO: `/v3/projects/ac2f9c5b90db4fd4b63d0c2e151e0a58/users/eee60bc4cd2c4e9abcc48270bc2dca2f/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:55.783939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d15378-93ca-422e-ae32-eaf472bcd3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:55.784800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d15378-93ca-422e-ae32-eaf472bcd3f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:55.785003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63dcfe-0f7f-4ff3-99ac-c945ed52934e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac2f9c5b90db4fd4b63d0c2e151e0a58, user_id=eee60bc4cd2c4e9abcc48270bc2dca2f, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:55.785133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d15378-93ca-422e-ae32-eaf472bcd3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:55.785355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d15378-93ca-422e-ae32-eaf472bcd3f2 admin admin] PATH_INFO: `/v3/projects/6c438a847524489a8c273b283bceafbe` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:55.786612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d15378-93ca-422e-ae32-eaf472bcd3f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c438a847524489a8c273b283bceafbe)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:55.787723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d63dcfe-0f7f-4ff3-99ac-c945ed52934e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:55.788524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d15378-93ca-422e-ae32-eaf472bcd3f2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:55.808794 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2390/4800] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:55 2022] PUT /identity/v3/projects/ac2f9c5b90db4fd4b63d0c2e151e0a58/users/eee60bc4cd2c4e9abcc48270bc2dca2f/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:18:55.810798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b61b3-3db5-4f5a-bb2f-19fe8084450b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:55.811610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b61b3-3db5-4f5a-bb2f-19fe8084450b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:55.811951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b61b3-3db5-4f5a-bb2f-19fe8084450b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:55.842048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1c3b61b3-3db5-4f5a-bb2f-19fe8084450b None None] MFA Rules not processed for user `eee60bc4cd2c4e9abcc48270bc2dca2f`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:55.843376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1c3b61b3-3db5-4f5a-bb2f-19fe8084450b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:55.917633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10d15378-93ca-422e-ae32-eaf472bcd3f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c438a847524489a8c273b283bceafbe', 'request_id': 'req-10d15378-93ca-422e-ae32-eaf472bcd3f2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:55.923959 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2391/4801] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:55 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:55.925013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10d15378-93ca-422e-ae32-eaf472bcd3f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c438a847524489a8c273b283bceafbe', 'request_id': 'req-10d15378-93ca-422e-ae32-eaf472bcd3f2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:55.927788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b61b3-3db5-4f5a-bb2f-19fe8084450b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:55.990697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10d15378-93ca-422e-ae32-eaf472bcd3f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c438a847524489a8c273b283bceafbe', 'request_id': 'req-10d15378-93ca-422e-ae32-eaf472bcd3f2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:56.001082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b61b3-3db5-4f5a-bb2f-19fe8084450b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:56.005958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10d15378-93ca-422e-ae32-eaf472bcd3f2 admin admin] The token cache is being invalidate because project 6c438a847524489a8c273b283bceafbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:56.006258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10d15378-93ca-422e-ae32-eaf472bcd3f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:56.006926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10d15378-93ca-422e-ae32-eaf472bcd3f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:56.008502 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2411/4802] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:55 2022] DELETE /identity/v3/projects/6c438a847524489a8c273b283bceafbe => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:56.013443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cf13b-ff6a-49a9-a95a-42c6cde5439c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:56.013670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cf13b-ff6a-49a9-a95a-42c6cde5439c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:56.014027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cf13b-ff6a-49a9-a95a-42c6cde5439c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:56.034417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-645cf13b-ff6a-49a9-a95a-42c6cde5439c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:56.035636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-645cf13b-ff6a-49a9-a95a-42c6cde5439c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:56.070091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aee7b0-ae43-4fe0-a510-98b784b94879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:56.070091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aee7b0-ae43-4fe0-a510-98b784b94879 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:56.070091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aee7b0-ae43-4fe0-a510-98b784b94879 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:56.070091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aee7b0-ae43-4fe0-a510-98b784b94879 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:56.070561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aee7b0-ae43-4fe0-a510-98b784b94879 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:56.071698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aee7b0-ae43-4fe0-a510-98b784b94879 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:56.113360 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2412/4803] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:18:56.120932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2368e5-2eba-48fc-be20-56f735791ca3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:56.121226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2368e5-2eba-48fc-be20-56f735791ca3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:56.121605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2368e5-2eba-48fc-be20-56f735791ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:56.140008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7b2368e5-2eba-48fc-be20-56f735791ca3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:56.141222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7b2368e5-2eba-48fc-be20-56f735791ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:56.150011 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2413/4804] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:18:56.154134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2368e5-2eba-48fc-be20-56f735791ca3 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:56.163347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2368e5-2eba-48fc-be20-56f735791ca3 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:56.168238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834a2af-3e73-4250-afe9-4f830e77df2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:56.168890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834a2af-3e73-4250-afe9-4f830e77df2c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:56.169176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834a2af-3e73-4250-afe9-4f830e77df2c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:56.169454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834a2af-3e73-4250-afe9-4f830e77df2c admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:56.170220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834a2af-3e73-4250-afe9-4f830e77df2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:18:56.170532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834a2af-3e73-4250-afe9-4f830e77df2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:56.172128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834a2af-3e73-4250-afe9-4f830e77df2c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:56.189664 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2414/4805] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:18:56.192865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834a2af-3e73-4250-afe9-4f830e77df2c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:56.195240 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2392/4806] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:55 2022] POST /identity/v3/projects => generated 385 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:56.200610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aee7b0-ae43-4fe0-a510-98b784b94879 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:56.202299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c3aee7b0-ae43-4fe0-a510-98b784b94879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:56.215836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834a2af-3e73-4250-afe9-4f830e77df2c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:56.289767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3c3ac-0773-41fc-8c98-a7cad968e787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:56.290751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3c3ac-0773-41fc-8c98-a7cad968e787 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:56.291112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3c3ac-0773-41fc-8c98-a7cad968e787 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:56.291522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3c3ac-0773-41fc-8c98-a7cad968e787 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:56.292987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3c3ac-0773-41fc-8c98-a7cad968e787 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:56.295979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3c3ac-0773-41fc-8c98-a7cad968e787 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:56.296135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aee7b0-ae43-4fe0-a510-98b784b94879 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:56.319100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc68379-f949-42b7-93b7-a4a1dbf1d26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:56.320125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc68379-f949-42b7-93b7-a4a1dbf1d26a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:56.320428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc68379-f949-42b7-93b7-a4a1dbf1d26a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:56.320736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc68379-f949-42b7-93b7-a4a1dbf1d26a admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:56.321366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc68379-f949-42b7-93b7-a4a1dbf1d26a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:56.323431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc68379-f949-42b7-93b7-a4a1dbf1d26a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:56.427107 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2415/4807] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:56 2022] POST /identity/v3/projects => generated 381 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:56.433793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3c3ac-0773-41fc-8c98-a7cad968e787 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:56.438886 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2393/4808] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:56 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:56.445196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc68379-f949-42b7-93b7-a4a1dbf1d26a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:56.466141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc68379-f949-42b7-93b7-a4a1dbf1d26a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:56.510770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:56.528781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3c3ac-0773-41fc-8c98-a7cad968e787 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:56.537465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03a338-5c03-47ce-b64e-4fec5eebdf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:56.538853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03a338-5c03-47ce-b64e-4fec5eebdf3a admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:56.539400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03a338-5c03-47ce-b64e-4fec5eebdf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:56.539690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03a338-5c03-47ce-b64e-4fec5eebdf3a admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:56.540540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03a338-5c03-47ce-b64e-4fec5eebdf3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:56.542657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03a338-5c03-47ce-b64e-4fec5eebdf3a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:56.567946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1df1a-e759-48ca-8b52-5cd717eb3f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:56.568599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1df1a-e759-48ca-8b52-5cd717eb3f50 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:56.568883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1df1a-e759-48ca-8b52-5cd717eb3f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:56.569159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1df1a-e759-48ca-8b52-5cd717eb3f50 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:56.569914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1df1a-e759-48ca-8b52-5cd717eb3f50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:56.574275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1df1a-e759-48ca-8b52-5cd717eb3f50 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:56.585869 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2394/4809] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:56.589063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1df1a-e759-48ca-8b52-5cd717eb3f50 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:56.598969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1df1a-e759-48ca-8b52-5cd717eb3f50 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:56.605676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03fba8-1f85-485f-84eb-e81ba994b87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:56.606773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03fba8-1f85-485f-84eb-e81ba994b87f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:56.606912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03fba8-1f85-485f-84eb-e81ba994b87f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:56.607128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa03fba8-1f85-485f-84eb-e81ba994b87f admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:56.614032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03fba8-1f85-485f-84eb-e81ba994b87f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:56.614032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa03fba8-1f85-485f-84eb-e81ba994b87f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:56.622737 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2395/4810] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:18:56.627459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03fba8-1f85-485f-84eb-e81ba994b87f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:56.638286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa03fba8-1f85-485f-84eb-e81ba994b87f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:56.643967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bce1c-dc8a-4dab-9a81-0547dcda9bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:56.644929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bce1c-dc8a-4dab-9a81-0547dcda9bb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:56.645467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bce1c-dc8a-4dab-9a81-0547dcda9bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:56.645761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bce1c-dc8a-4dab-9a81-0547dcda9bb1 admin admin] PATH_INFO: `/v3/projects/ff7ff5aada6e49dfa049f842cb0bb410/users/dfe7da21c536483ea8fa896872d1642c/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:56.651307 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2416/4811] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:56 2022] POST /identity/v3/users => generated 291 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:56.668362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bce1c-dc8a-4dab-9a81-0547dcda9bb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff7ff5aada6e49dfa049f842cb0bb410, user_id=dfe7da21c536483ea8fa896872d1642c, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:56.668362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bce1c-dc8a-4dab-9a81-0547dcda9bb1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:56.671502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03a338-5c03-47ce-b64e-4fec5eebdf3a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:56.682957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03a338-5c03-47ce-b64e-4fec5eebdf3a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:56.689408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c866e-1b48-4288-900e-6c12b4bd486a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:56.690296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c866e-1b48-4288-900e-6c12b4bd486a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:56.690604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c866e-1b48-4288-900e-6c12b4bd486a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:56.690898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2c866e-1b48-4288-900e-6c12b4bd486a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:56.692426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c866e-1b48-4288-900e-6c12b4bd486a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:56.694302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2c866e-1b48-4288-900e-6c12b4bd486a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:56.706214 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2417/4812] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:56.710093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c866e-1b48-4288-900e-6c12b4bd486a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:56.721493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2c866e-1b48-4288-900e-6c12b4bd486a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:56.726836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfad850-33be-4392-b409-17f931f76ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:56.727750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfad850-33be-4392-b409-17f931f76ecf admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:56.728139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfad850-33be-4392-b409-17f931f76ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:56.728476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfad850-33be-4392-b409-17f931f76ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:56.729229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfad850-33be-4392-b409-17f931f76ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:56.731140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfad850-33be-4392-b409-17f931f76ecf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:56.748307 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2418/4813] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:56.750836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfad850-33be-4392-b409-17f931f76ecf admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:56.758803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:56.765885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfad850-33be-4392-b409-17f931f76ecf admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:56.771078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c55aa-3bcd-473f-9a3e-8f638c324af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:56.772045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c55aa-3bcd-473f-9a3e-8f638c324af5 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:56.772401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c55aa-3bcd-473f-9a3e-8f638c324af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:56.772749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c55aa-3bcd-473f-9a3e-8f638c324af5 admin admin] PATH_INFO: `/v3/projects/f621ae2edc2143ebb303827c74b294a3/users/b6c9351dfb6c46f2aba4c2ad54814561/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:56.797528 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2396/4814] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:56 2022] PUT /identity/v3/projects/ff7ff5aada6e49dfa049f842cb0bb410/users/dfe7da21c536483ea8fa896872d1642c/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:18:56.803126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c55aa-3bcd-473f-9a3e-8f638c324af5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f621ae2edc2143ebb303827c74b294a3, user_id=b6c9351dfb6c46f2aba4c2ad54814561, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:56.806207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c55aa-3bcd-473f-9a3e-8f638c324af5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:56.812115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b366dd8-bb65-4a9b-8b60-87ad81f79d77 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:56.812408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b366dd8-bb65-4a9b-8b60-87ad81f79d77 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:56.812691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b366dd8-bb65-4a9b-8b60-87ad81f79d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:56.847458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8b366dd8-bb65-4a9b-8b60-87ad81f79d77 None None] MFA Rules not processed for user `dfe7da21c536483ea8fa896872d1642c`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:56.848701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8b366dd8-bb65-4a9b-8b60-87ad81f79d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:56.889678 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2419/4815] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:56 2022] PUT /identity/v3/projects/f621ae2edc2143ebb303827c74b294a3/users/b6c9351dfb6c46f2aba4c2ad54814561/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:56.911739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d73b7f-ed94-4a6c-94db-f59aa8fbd8b3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:56.912041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d73b7f-ed94-4a6c-94db-f59aa8fbd8b3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:56.912374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d73b7f-ed94-4a6c-94db-f59aa8fbd8b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:56.922284 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2397/4816] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:18:56.942858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b366dd8-bb65-4a9b-8b60-87ad81f79d77 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:56.944782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8b366dd8-bb65-4a9b-8b60-87ad81f79d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:56.955677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b5d73b7f-ed94-4a6c-94db-f59aa8fbd8b3 None None] MFA Rules not processed for user `b6c9351dfb6c46f2aba4c2ad54814561`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:56.958527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b5d73b7f-ed94-4a6c-94db-f59aa8fbd8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:57.066559 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2420/4817] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:18:57.069041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b366dd8-bb65-4a9b-8b60-87ad81f79d77 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:57.078235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d73b7f-ed94-4a6c-94db-f59aa8fbd8b3 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:57.081050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b5d73b7f-ed94-4a6c-94db-f59aa8fbd8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:57.131566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc809ff2-716e-403b-a35f-82a6f76e827d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:57.132317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc809ff2-716e-403b-a35f-82a6f76e827d service glance] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:57.132667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc809ff2-716e-403b-a35f-82a6f76e827d service glance] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:57.133016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc809ff2-716e-403b-a35f-82a6f76e827d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:57.134997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bc809ff2-716e-403b-a35f-82a6f76e827d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:57.152158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d73b7f-ed94-4a6c-94db-f59aa8fbd8b3 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:57.156262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125162c2-3f09-43e2-865e-e20a84d926ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:57.156932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125162c2-3f09-43e2-865e-e20a84d926ea service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:57.157222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125162c2-3f09-43e2-865e-e20a84d926ea service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:57.157504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125162c2-3f09-43e2-865e-e20a84d926ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:57.167688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125162c2-3f09-43e2-865e-e20a84d926ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:57.169153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125162c2-3f09-43e2-865e-e20a84d926ea service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:57.179368 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2421/4818] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:57.218212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc809ff2-716e-403b-a35f-82a6f76e827d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:57.219965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc809ff2-716e-403b-a35f-82a6f76e827d service glance] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:57.221770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125162c2-3f09-43e2-865e-e20a84d926ea service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:57.238909 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2398/4819] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:18:57.304157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125162c2-3f09-43e2-865e-e20a84d926ea service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:57.309211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3fdf8-4314-4cce-87d1-7d9aa8c1fd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:57.310120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3fdf8-4314-4cce-87d1-7d9aa8c1fd03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:57.310405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3fdf8-4314-4cce-87d1-7d9aa8c1fd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:57.310687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3fdf8-4314-4cce-87d1-7d9aa8c1fd03 admin admin] PATH_INFO: `/v3/users/b6c9351dfb6c46f2aba4c2ad54814561` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:57.311910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3fdf8-4314-4cce-87d1-7d9aa8c1fd03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6c9351dfb6c46f2aba4c2ad54814561)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:57.313871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d3fdf8-4314-4cce-87d1-7d9aa8c1fd03 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:57.598466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-74d3fdf8-4314-4cce-87d1-7d9aa8c1fd03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6c9351dfb6c46f2aba4c2ad54814561'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:57.618219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-74d3fdf8-4314-4cce-87d1-7d9aa8c1fd03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6c9351dfb6c46f2aba4c2ad54814561'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:57.618950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-74d3fdf8-4314-4cce-87d1-7d9aa8c1fd03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6c9351dfb6c46f2aba4c2ad54814561'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:57.619572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-74d3fdf8-4314-4cce-87d1-7d9aa8c1fd03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6c9351dfb6c46f2aba4c2ad54814561'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:57.745583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-74d3fdf8-4314-4cce-87d1-7d9aa8c1fd03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6c9351dfb6c46f2aba4c2ad54814561'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:57.789678 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2422/4820] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:57 2022] DELETE /identity/v3/users/b6c9351dfb6c46f2aba4c2ad54814561 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:18:57.809358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a3891-8900-4a1e-9f40-2e157b59daaa None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:57.809958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a3891-8900-4a1e-9f40-2e157b59daaa None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:57.810068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539a3891-8900-4a1e-9f40-2e157b59daaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:57.833070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-539a3891-8900-4a1e-9f40-2e157b59daaa None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:57.834527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-539a3891-8900-4a1e-9f40-2e157b59daaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:57.901368 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2399/4821] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:57.905877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3fdf8-4314-4cce-87d1-7d9aa8c1fd03 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:57.907312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-74d3fdf8-4314-4cce-87d1-7d9aa8c1fd03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:57.917194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a3891-8900-4a1e-9f40-2e157b59daaa None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:57.921056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-539a3891-8900-4a1e-9f40-2e157b59daaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:57.924419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3fdf8-4314-4cce-87d1-7d9aa8c1fd03 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:57.929632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36ce45-0a9d-4082-a623-dcc6fe76d193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:57.930566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36ce45-0a9d-4082-a623-dcc6fe76d193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:57.931038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36ce45-0a9d-4082-a623-dcc6fe76d193 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:57.931741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f36ce45-0a9d-4082-a623-dcc6fe76d193 admin admin] PATH_INFO: `/v3/users/eee60bc4cd2c4e9abcc48270bc2dca2f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:57.933418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36ce45-0a9d-4082-a623-dcc6fe76d193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eee60bc4cd2c4e9abcc48270bc2dca2f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:57.935383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f36ce45-0a9d-4082-a623-dcc6fe76d193 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:57.984194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539a3891-8900-4a1e-9f40-2e157b59daaa None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:57.988345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e4357-20ff-468f-be21-5e19694ccd56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:57.988989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e4357-20ff-468f-be21-5e19694ccd56 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:57.989269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e4357-20ff-468f-be21-5e19694ccd56 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:57.989543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e4357-20ff-468f-be21-5e19694ccd56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:57.998823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e4357-20ff-468f-be21-5e19694ccd56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:58.000271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e4357-20ff-468f-be21-5e19694ccd56 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:58.010448 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2400/4822] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:58.222576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7f36ce45-0a9d-4082-a623-dcc6fe76d193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eee60bc4cd2c4e9abcc48270bc2dca2f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:58.237354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7f36ce45-0a9d-4082-a623-dcc6fe76d193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eee60bc4cd2c4e9abcc48270bc2dca2f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:58.238212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7f36ce45-0a9d-4082-a623-dcc6fe76d193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eee60bc4cd2c4e9abcc48270bc2dca2f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:58.238840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7f36ce45-0a9d-4082-a623-dcc6fe76d193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eee60bc4cd2c4e9abcc48270bc2dca2f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:58.362209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7f36ce45-0a9d-4082-a623-dcc6fe76d193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eee60bc4cd2c4e9abcc48270bc2dca2f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:58.385114 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2423/4823] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:57 2022] DELETE /identity/v3/users/eee60bc4cd2c4e9abcc48270bc2dca2f => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:58.389719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122516b6-7426-4244-a587-f831bf872b6a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:58.390030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122516b6-7426-4244-a587-f831bf872b6a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:58.390307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122516b6-7426-4244-a587-f831bf872b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:58.408662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-122516b6-7426-4244-a587-f831bf872b6a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:58.411346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-122516b6-7426-4244-a587-f831bf872b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:58.477109 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2401/4824] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:58.488258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36ce45-0a9d-4082-a623-dcc6fe76d193 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:58.489764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7f36ce45-0a9d-4082-a623-dcc6fe76d193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:58.568760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f36ce45-0a9d-4082-a623-dcc6fe76d193 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:58.572430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda9c77-9162-4eac-9cb2-39409d81031e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:58.573082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda9c77-9162-4eac-9cb2-39409d81031e service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:58.573365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda9c77-9162-4eac-9cb2-39409d81031e service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:58.576351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda9c77-9162-4eac-9cb2-39409d81031e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:58.583902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda9c77-9162-4eac-9cb2-39409d81031e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:58.585446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda9c77-9162-4eac-9cb2-39409d81031e service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:58.597283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:58.598624 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2424/4825] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:18:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:18:58.679797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:58.734377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122516b6-7426-4244-a587-f831bf872b6a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:58.736118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-122516b6-7426-4244-a587-f831bf872b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:58.748370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122516b6-7426-4244-a587-f831bf872b6a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:58.754233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e8d30-1e19-4c3d-91b2-73d398fa8458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:58.755511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e8d30-1e19-4c3d-91b2-73d398fa8458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:58.755960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e8d30-1e19-4c3d-91b2-73d398fa8458 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:58.756389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e8d30-1e19-4c3d-91b2-73d398fa8458 admin admin] PATH_INFO: `/v3/projects/f621ae2edc2143ebb303827c74b294a3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:58.757855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e8d30-1e19-4c3d-91b2-73d398fa8458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f621ae2edc2143ebb303827c74b294a3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:58.760009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77e8d30-1e19-4c3d-91b2-73d398fa8458 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:58.901455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d77e8d30-1e19-4c3d-91b2-73d398fa8458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f621ae2edc2143ebb303827c74b294a3', 'request_id': 'req-d77e8d30-1e19-4c3d-91b2-73d398fa8458'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:58.909724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d77e8d30-1e19-4c3d-91b2-73d398fa8458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f621ae2edc2143ebb303827c74b294a3', 'request_id': 'req-d77e8d30-1e19-4c3d-91b2-73d398fa8458'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:59.025507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d77e8d30-1e19-4c3d-91b2-73d398fa8458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f621ae2edc2143ebb303827c74b294a3', 'request_id': 'req-d77e8d30-1e19-4c3d-91b2-73d398fa8458'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:59.047758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d77e8d30-1e19-4c3d-91b2-73d398fa8458 admin admin] The token cache is being invalidate because project f621ae2edc2143ebb303827c74b294a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:59.048559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d77e8d30-1e19-4c3d-91b2-73d398fa8458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:59.049575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d77e8d30-1e19-4c3d-91b2-73d398fa8458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:59.054685 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2402/4826] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:58 2022] DELETE /identity/v3/projects/f621ae2edc2143ebb303827c74b294a3 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:59.062711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71ffda-8c5b-4e42-984a-2b15b3630631 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:59.062711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71ffda-8c5b-4e42-984a-2b15b3630631 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:59.062711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e71ffda-8c5b-4e42-984a-2b15b3630631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:59.096579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9e71ffda-8c5b-4e42-984a-2b15b3630631 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:59.098324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9e71ffda-8c5b-4e42-984a-2b15b3630631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:59.174448 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2425/4827] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:18:59.183519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71a570-8808-4e3e-9008-e80c371fc751 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:59.183919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71a570-8808-4e3e-9008-e80c371fc751 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:59.184904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71a570-8808-4e3e-9008-e80c371fc751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:59.208035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5e71a570-8808-4e3e-9008-e80c371fc751 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:18:59.209363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5e71a570-8808-4e3e-9008-e80c371fc751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:59.223000 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2403/4828] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:18:59.224426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71ffda-8c5b-4e42-984a-2b15b3630631 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:59.245089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e71ffda-8c5b-4e42-984a-2b15b3630631 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:59.245089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fc723-0b3c-4af5-b003-f3245ff13e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:59.245089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fc723-0b3c-4af5-b003-f3245ff13e92 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:59.245089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fc723-0b3c-4af5-b003-f3245ff13e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:59.246027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8fc723-0b3c-4af5-b003-f3245ff13e92 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:59.246027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8fc723-0b3c-4af5-b003-f3245ff13e92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:18:59.246027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8fc723-0b3c-4af5-b003-f3245ff13e92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:59.246027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8fc723-0b3c-4af5-b003-f3245ff13e92 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:59.257870 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2426/4829] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:18:59.261889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71a570-8808-4e3e-9008-e80c371fc751 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:59.272145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71a570-8808-4e3e-9008-e80c371fc751 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:59.276853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fc723-0b3c-4af5-b003-f3245ff13e92 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:59.279290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fb8fc723-0b3c-4af5-b003-f3245ff13e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:59.289082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3949e6a-26e4-484b-a119-42fe7fed844f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:59.291193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3949e6a-26e4-484b-a119-42fe7fed844f admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:59.291193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3949e6a-26e4-484b-a119-42fe7fed844f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:59.291193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3949e6a-26e4-484b-a119-42fe7fed844f admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:59.292706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3949e6a-26e4-484b-a119-42fe7fed844f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:59.295874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3949e6a-26e4-484b-a119-42fe7fed844f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:59.311708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8fc723-0b3c-4af5-b003-f3245ff13e92 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:59.321001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450967b-c963-40cc-ab3b-ebf3ad1b268d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:59.321954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450967b-c963-40cc-ab3b-ebf3ad1b268d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:59.322252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450967b-c963-40cc-ab3b-ebf3ad1b268d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:59.322586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450967b-c963-40cc-ab3b-ebf3ad1b268d admin admin] PATH_INFO: `/v3/projects/ac2f9c5b90db4fd4b63d0c2e151e0a58` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:59.324268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450967b-c963-40cc-ab3b-ebf3ad1b268d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac2f9c5b90db4fd4b63d0c2e151e0a58)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:59.326169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450967b-c963-40cc-ab3b-ebf3ad1b268d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:59.466349 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2404/4830] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:18:59 2022] POST /identity/v3/projects => generated 369 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:18:59.470168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3949e6a-26e4-484b-a119-42fe7fed844f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:59.536496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b450967b-c963-40cc-ab3b-ebf3ad1b268d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac2f9c5b90db4fd4b63d0c2e151e0a58', 'request_id': 'req-b450967b-c963-40cc-ab3b-ebf3ad1b268d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:59.536756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3949e6a-26e4-484b-a119-42fe7fed844f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:59.543312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b450967b-c963-40cc-ab3b-ebf3ad1b268d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac2f9c5b90db4fd4b63d0c2e151e0a58', 'request_id': 'req-b450967b-c963-40cc-ab3b-ebf3ad1b268d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:59.599540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c681b-5994-4148-b12f-7f354f555616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:59.600234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c681b-5994-4148-b12f-7f354f555616 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:59.600529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c681b-5994-4148-b12f-7f354f555616 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:59.600815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c681b-5994-4148-b12f-7f354f555616 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:59.601648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c681b-5994-4148-b12f-7f354f555616 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:59.603837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c681b-5994-4148-b12f-7f354f555616 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:59.635743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b450967b-c963-40cc-ab3b-ebf3ad1b268d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac2f9c5b90db4fd4b63d0c2e151e0a58', 'request_id': 'req-b450967b-c963-40cc-ab3b-ebf3ad1b268d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:59.650865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b450967b-c963-40cc-ab3b-ebf3ad1b268d admin admin] The token cache is being invalidate because project ac2f9c5b90db4fd4b63d0c2e151e0a58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:18:59.651182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b450967b-c963-40cc-ab3b-ebf3ad1b268d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:59.651888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b450967b-c963-40cc-ab3b-ebf3ad1b268d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:59.657381 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2427/4831] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:18:59 2022] DELETE /identity/v3/projects/ac2f9c5b90db4fd4b63d0c2e151e0a58 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:18:59.657550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450967b-c963-40cc-ab3b-ebf3ad1b268d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:59.659455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b450967b-c963-40cc-ab3b-ebf3ad1b268d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:59.679301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450967b-c963-40cc-ab3b-ebf3ad1b268d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:59.688977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3939a-4a03-4ade-8c7c-6710ad2bb78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:59.690308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3939a-4a03-4ade-8c7c-6710ad2bb78a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:59.693997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3939a-4a03-4ade-8c7c-6710ad2bb78a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:59.693997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3939a-4a03-4ade-8c7c-6710ad2bb78a admin admin] PATH_INFO: `/v3/users/dfe7da21c536483ea8fa896872d1642c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:59.693997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3939a-4a03-4ade-8c7c-6710ad2bb78a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfe7da21c536483ea8fa896872d1642c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:59.695365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3939a-4a03-4ade-8c7c-6710ad2bb78a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:59.786270 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2405/4832] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:18:59 2022] POST /identity/v3/users => generated 285 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:18:59.790154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c681b-5994-4148-b12f-7f354f555616 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:59.791997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-145c681b-5994-4148-b12f-7f354f555616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:18:59.804560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:59.810783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c681b-5994-4148-b12f-7f354f555616 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:59.818936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9168f9c-e165-4f35-9f00-8ef643dbceb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:59.818936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9168f9c-e165-4f35-9f00-8ef643dbceb5 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:59.818936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9168f9c-e165-4f35-9f00-8ef643dbceb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:59.818936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9168f9c-e165-4f35-9f00-8ef643dbceb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:59.819291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9168f9c-e165-4f35-9f00-8ef643dbceb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:59.821960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9168f9c-e165-4f35-9f00-8ef643dbceb5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:59.834849 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2406/4833] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:59.835879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9168f9c-e165-4f35-9f00-8ef643dbceb5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:59.849193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9168f9c-e165-4f35-9f00-8ef643dbceb5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:59.855279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfa7d0-a1e5-42e1-9f8e-71d7022cf519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:59.856010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfa7d0-a1e5-42e1-9f8e-71d7022cf519 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:59.856308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfa7d0-a1e5-42e1-9f8e-71d7022cf519 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:59.856699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfa7d0-a1e5-42e1-9f8e-71d7022cf519 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:59.857565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfa7d0-a1e5-42e1-9f8e-71d7022cf519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:59.859420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfa7d0-a1e5-42e1-9f8e-71d7022cf519 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:59.876631 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2407/4834] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:18:59.877810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfa7d0-a1e5-42e1-9f8e-71d7022cf519 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:18:59.889709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfa7d0-a1e5-42e1-9f8e-71d7022cf519 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:18:59.894704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e61e2c8-a9f4-4488-8061-f50e6b238ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:18:59.895593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61e2c8-a9f4-4488-8061-f50e6b238ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:18:59.895917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61e2c8-a9f4-4488-8061-f50e6b238ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:18:59.896184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61e2c8-a9f4-4488-8061-f50e6b238ed2 admin admin] PATH_INFO: `/v3/projects/2e5d8fcff8014060a847cb4ae0738057/users/935b49aadd30450bb08df41f8e859007/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:18:59.912045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61e2c8-a9f4-4488-8061-f50e6b238ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e5d8fcff8014060a847cb4ae0738057, user_id=935b49aadd30450bb08df41f8e859007, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:18:59.914916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e61e2c8-a9f4-4488-8061-f50e6b238ed2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:18:59.915117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3db3939a-4a03-4ade-8c7c-6710ad2bb78a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfe7da21c536483ea8fa896872d1642c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:59.922718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3db3939a-4a03-4ade-8c7c-6710ad2bb78a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfe7da21c536483ea8fa896872d1642c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:59.924077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3db3939a-4a03-4ade-8c7c-6710ad2bb78a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfe7da21c536483ea8fa896872d1642c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:18:59.924801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3db3939a-4a03-4ade-8c7c-6710ad2bb78a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfe7da21c536483ea8fa896872d1642c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:00.049702 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2408/4835] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:18:59 2022] PUT /identity/v3/projects/2e5d8fcff8014060a847cb4ae0738057/users/935b49aadd30450bb08df41f8e859007/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:00.064253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3db3939a-4a03-4ade-8c7c-6710ad2bb78a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfe7da21c536483ea8fa896872d1642c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:00.067652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f492d-1e2b-4f52-be07-a078d97c6418 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:00.069214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f492d-1e2b-4f52-be07-a078d97c6418 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:00.071569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44f492d-1e2b-4f52-be07-a078d97c6418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:00.096909 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2428/4836] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:18:59 2022] DELETE /identity/v3/users/dfe7da21c536483ea8fa896872d1642c => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:00.123348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c44f492d-1e2b-4f52-be07-a078d97c6418 None None] MFA Rules not processed for user `935b49aadd30450bb08df41f8e859007`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:00.125064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c44f492d-1e2b-4f52-be07-a078d97c6418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:00.222220 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2409/4837] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:00 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:00.226816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3939a-4a03-4ade-8c7c-6710ad2bb78a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:00.228215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3db3939a-4a03-4ade-8c7c-6710ad2bb78a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:00.302422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3939a-4a03-4ade-8c7c-6710ad2bb78a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:00.309429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa14b4-271d-46f3-b21e-6a100fd4e581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:00.310087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa14b4-271d-46f3-b21e-6a100fd4e581 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:00.310425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa14b4-271d-46f3-b21e-6a100fd4e581 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:00.311943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa14b4-271d-46f3-b21e-6a100fd4e581 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:00.311943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa14b4-271d-46f3-b21e-6a100fd4e581 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:00.313532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa14b4-271d-46f3-b21e-6a100fd4e581 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:00.379045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f492d-1e2b-4f52-be07-a078d97c6418 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:00.379045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c44f492d-1e2b-4f52-be07-a078d97c6418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:00.394485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44f492d-1e2b-4f52-be07-a078d97c6418 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:00.403339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c804d052-d1cc-4f96-a29d-9fb54088e1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:00.403339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804d052-d1cc-4f96-a29d-9fb54088e1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:00.403339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804d052-d1cc-4f96-a29d-9fb54088e1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:00.403339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804d052-d1cc-4f96-a29d-9fb54088e1f8 admin admin] PATH_INFO: `/v3/users/120a571243304f3f925fc79d60317bc1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:00.404162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c804d052-d1cc-4f96-a29d-9fb54088e1f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=120a571243304f3f925fc79d60317bc1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:00.406637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c804d052-d1cc-4f96-a29d-9fb54088e1f8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:00.434637 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2429/4838] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:00 2022] POST /identity/v3/projects => generated 365 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:00.437575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa14b4-271d-46f3-b21e-6a100fd4e581 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:00.439950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:00.535368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa14b4-271d-46f3-b21e-6a100fd4e581 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:00.543705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b8d33-a05d-4c80-ad61-b3753d9577c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:00.544504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b8d33-a05d-4c80-ad61-b3753d9577c1 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:00.544936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b8d33-a05d-4c80-ad61-b3753d9577c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:00.545283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b8d33-a05d-4c80-ad61-b3753d9577c1 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:00.546721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b8d33-a05d-4c80-ad61-b3753d9577c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:00.551334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b8d33-a05d-4c80-ad61-b3753d9577c1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:00.669278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c804d052-d1cc-4f96-a29d-9fb54088e1f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '120a571243304f3f925fc79d60317bc1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:00.670260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c804d052-d1cc-4f96-a29d-9fb54088e1f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '120a571243304f3f925fc79d60317bc1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:00.675766 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2430/4839] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:00 2022] POST /identity/v3/users => generated 289 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:00.676864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b8d33-a05d-4c80-ad61-b3753d9577c1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:00.683324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c804d052-d1cc-4f96-a29d-9fb54088e1f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '120a571243304f3f925fc79d60317bc1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:00.699469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b8d33-a05d-4c80-ad61-b3753d9577c1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:00.705387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d841a16-f1b5-4d84-9ccb-90ae3c71cb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:00.709662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d841a16-f1b5-4d84-9ccb-90ae3c71cb5c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:00.709662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d841a16-f1b5-4d84-9ccb-90ae3c71cb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:00.709662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d841a16-f1b5-4d84-9ccb-90ae3c71cb5c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:00.709662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d841a16-f1b5-4d84-9ccb-90ae3c71cb5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:00.709662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d841a16-f1b5-4d84-9ccb-90ae3c71cb5c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:00.719781 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2431/4840] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:00.724148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d841a16-f1b5-4d84-9ccb-90ae3c71cb5c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:00.735162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d841a16-f1b5-4d84-9ccb-90ae3c71cb5c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:00.740100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f655e-04ef-4c47-b71b-93d0c0f73442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:00.741019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f655e-04ef-4c47-b71b-93d0c0f73442 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:00.741308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f655e-04ef-4c47-b71b-93d0c0f73442 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:00.741737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f655e-04ef-4c47-b71b-93d0c0f73442 admin admin] PATH_INFO: `/v3/projects/6e8ed7a1f60a4a608afabf8ab5e3c4cd/users/cf566d5f67964b21bc35934a74671fea/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:00.760480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f655e-04ef-4c47-b71b-93d0c0f73442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8ed7a1f60a4a608afabf8ab5e3c4cd, user_id=cf566d5f67964b21bc35934a74671fea, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:00.763209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f655e-04ef-4c47-b71b-93d0c0f73442 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:00.777895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c804d052-d1cc-4f96-a29d-9fb54088e1f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '120a571243304f3f925fc79d60317bc1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:00.778779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c804d052-d1cc-4f96-a29d-9fb54088e1f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '120a571243304f3f925fc79d60317bc1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:00.807366 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2410/4841] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:00 2022] DELETE /identity/v3/users/120a571243304f3f925fc79d60317bc1 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:00.808476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c804d052-d1cc-4f96-a29d-9fb54088e1f8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:00.816900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c804d052-d1cc-4f96-a29d-9fb54088e1f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:00.874372 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2432/4842] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:00 2022] PUT /identity/v3/projects/6e8ed7a1f60a4a608afabf8ab5e3c4cd/users/cf566d5f67964b21bc35934a74671fea/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:00.876163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50587b-9cfd-448b-9b95-d4d8c212e48d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:00.876307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50587b-9cfd-448b-9b95-d4d8c212e48d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:00.879105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50587b-9cfd-448b-9b95-d4d8c212e48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:00.916942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9e50587b-9cfd-448b-9b95-d4d8c212e48d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:00.918307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9e50587b-9cfd-448b-9b95-d4d8c212e48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:00.974046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c804d052-d1cc-4f96-a29d-9fb54088e1f8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:01.025119 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2433/4843] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:01.026329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50587b-9cfd-448b-9b95-d4d8c212e48d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:01.028935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9e50587b-9cfd-448b-9b95-d4d8c212e48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:01.047169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e50587b-9cfd-448b-9b95-d4d8c212e48d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:01.054872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecb585-5a13-4ee6-80a3-befd1952e6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:01.055491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecb585-5a13-4ee6-80a3-befd1952e6aa admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:01.055791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecb585-5a13-4ee6-80a3-befd1952e6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:01.056172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecb585-5a13-4ee6-80a3-befd1952e6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:01.057049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecb585-5a13-4ee6-80a3-befd1952e6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:01.058683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecb585-5a13-4ee6-80a3-befd1952e6aa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:01.073661 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2434/4844] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:01.075172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e4806c-8757-415b-9b68-07368be8526e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:01.077944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecb585-5a13-4ee6-80a3-befd1952e6aa admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:01.078078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4806c-8757-415b-9b68-07368be8526e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:01.078078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4806c-8757-415b-9b68-07368be8526e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:01.078078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e4806c-8757-415b-9b68-07368be8526e admin admin] PATH_INFO: `/v3/projects/ff7ff5aada6e49dfa049f842cb0bb410` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:01.078375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-02ecb585-5a13-4ee6-80a3-befd1952e6aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:01.079017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4806c-8757-415b-9b68-07368be8526e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff7ff5aada6e49dfa049f842cb0bb410)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:01.081416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e4806c-8757-415b-9b68-07368be8526e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:01.140922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecb585-5a13-4ee6-80a3-befd1952e6aa admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:01.144696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75fc77-ae21-4818-8e69-2cf62c93deb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:01.145338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75fc77-ae21-4818-8e69-2cf62c93deb9 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:01.145621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75fc77-ae21-4818-8e69-2cf62c93deb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:01.145939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75fc77-ae21-4818-8e69-2cf62c93deb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:01.155314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75fc77-ae21-4818-8e69-2cf62c93deb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:01.156988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75fc77-ae21-4818-8e69-2cf62c93deb9 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:01.168145 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2435/4845] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:01.169044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75fc77-ae21-4818-8e69-2cf62c93deb9 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:01.181488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75fc77-ae21-4818-8e69-2cf62c93deb9 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:01.186853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0360ffcb-b134-44ee-ade5-bb144aca3d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:01.187810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360ffcb-b134-44ee-ade5-bb144aca3d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:01.188120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360ffcb-b134-44ee-ade5-bb144aca3d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:01.188396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0360ffcb-b134-44ee-ade5-bb144aca3d0d admin admin] PATH_INFO: `/v3/projects/6e8ed7a1f60a4a608afabf8ab5e3c4cd/users/cf566d5f67964b21bc35934a74671fea/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:01.190667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0360ffcb-b134-44ee-ade5-bb144aca3d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8ed7a1f60a4a608afabf8ab5e3c4cd, user_id=cf566d5f67964b21bc35934a74671fea, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:01.194561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0360ffcb-b134-44ee-ade5-bb144aca3d0d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:01.215715 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2436/4846] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:01 2022] PUT /identity/v3/projects/6e8ed7a1f60a4a608afabf8ab5e3c4cd/users/cf566d5f67964b21bc35934a74671fea/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:01.226155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5385013-7a57-4ee8-93c6-2e381876405c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:01.229348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5385013-7a57-4ee8-93c6-2e381876405c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:01.229348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5385013-7a57-4ee8-93c6-2e381876405c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:01.238341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-17e4806c-8757-415b-9b68-07368be8526e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff7ff5aada6e49dfa049f842cb0bb410', 'request_id': 'req-17e4806c-8757-415b-9b68-07368be8526e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:01.244115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-17e4806c-8757-415b-9b68-07368be8526e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff7ff5aada6e49dfa049f842cb0bb410', 'request_id': 'req-17e4806c-8757-415b-9b68-07368be8526e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:01.265734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e5385013-7a57-4ee8-93c6-2e381876405c None None] MFA Rules not processed for user `cf566d5f67964b21bc35934a74671fea`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:01.267196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e5385013-7a57-4ee8-93c6-2e381876405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:01.362083 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2437/4847] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:01 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:01.368368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-17e4806c-8757-415b-9b68-07368be8526e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff7ff5aada6e49dfa049f842cb0bb410', 'request_id': 'req-17e4806c-8757-415b-9b68-07368be8526e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:01.369650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5385013-7a57-4ee8-93c6-2e381876405c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:01.372196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e5385013-7a57-4ee8-93c6-2e381876405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:01.391740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-17e4806c-8757-415b-9b68-07368be8526e admin admin] The token cache is being invalidate because project ff7ff5aada6e49dfa049f842cb0bb410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:01.392178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-17e4806c-8757-415b-9b68-07368be8526e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:01.392867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-17e4806c-8757-415b-9b68-07368be8526e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:01.394471 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2411/4848] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:00 2022] DELETE /identity/v3/projects/ff7ff5aada6e49dfa049f842cb0bb410 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:01.399237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696ac6f-9ca9-4eaa-baee-21d75ab41728 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:01.400689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696ac6f-9ca9-4eaa-baee-21d75ab41728 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:01.400689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696ac6f-9ca9-4eaa-baee-21d75ab41728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:01.418245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7696ac6f-9ca9-4eaa-baee-21d75ab41728 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:01.419372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7696ac6f-9ca9-4eaa-baee-21d75ab41728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:01.443383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:01.447979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5385013-7a57-4ee8-93c6-2e381876405c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:01.455237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fca4d-6464-43f3-b017-4358155e252c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:01.456172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fca4d-6464-43f3-b017-4358155e252c service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:01.456527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fca4d-6464-43f3-b017-4358155e252c service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:01.457186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fca4d-6464-43f3-b017-4358155e252c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:01.460417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fb0fca4d-6464-43f3-b017-4358155e252c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:01.475756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fca4d-6464-43f3-b017-4358155e252c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:01.481516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fca4d-6464-43f3-b017-4358155e252c service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:01.492331 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2438/4849] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:01 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:19:01.494954 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2412/4850] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:01.496112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696ac6f-9ca9-4eaa-baee-21d75ab41728 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:01.498921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7696ac6f-9ca9-4eaa-baee-21d75ab41728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:01.507212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd4aa4-6fd9-46e8-aaae-f34d524122fd None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:01.509582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd4aa4-6fd9-46e8-aaae-f34d524122fd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:01.516053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcd4aa4-6fd9-46e8-aaae-f34d524122fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:01.516821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696ac6f-9ca9-4eaa-baee-21d75ab41728 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:01.527318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f578c66-72bb-48af-88ea-18249f699f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:01.528429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f578c66-72bb-48af-88ea-18249f699f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:01.528790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f578c66-72bb-48af-88ea-18249f699f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:01.529263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f578c66-72bb-48af-88ea-18249f699f03 admin admin] PATH_INFO: `/v3/projects/e50d742e41464ad99c3dfd94005509ef` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:01.531087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f578c66-72bb-48af-88ea-18249f699f03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e50d742e41464ad99c3dfd94005509ef)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:01.533737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f578c66-72bb-48af-88ea-18249f699f03 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:01.539521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8fcd4aa4-6fd9-46e8-aaae-f34d524122fd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:01.541106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8fcd4aa4-6fd9-46e8-aaae-f34d524122fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:01.551328 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2439/4851] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:01.555876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd4aa4-6fd9-46e8-aaae-f34d524122fd None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:01.565912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcd4aa4-6fd9-46e8-aaae-f34d524122fd None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:01.571183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3542ab5-0ba4-4356-a728-3d9795fd0088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:01.573188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3542ab5-0ba4-4356-a728-3d9795fd0088 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:01.573188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3542ab5-0ba4-4356-a728-3d9795fd0088 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:01.573188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3542ab5-0ba4-4356-a728-3d9795fd0088 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:01.573691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3542ab5-0ba4-4356-a728-3d9795fd0088 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:01.573930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3542ab5-0ba4-4356-a728-3d9795fd0088 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:01.575615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3542ab5-0ba4-4356-a728-3d9795fd0088 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:01.589889 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2440/4852] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:01.594412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3542ab5-0ba4-4356-a728-3d9795fd0088 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:01.606731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3542ab5-0ba4-4356-a728-3d9795fd0088 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:01.611683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcaaa1e-66f3-4ab0-9180-a5669d8b541e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:01.612365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcaaa1e-66f3-4ab0-9180-a5669d8b541e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:01.612652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcaaa1e-66f3-4ab0-9180-a5669d8b541e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:01.612932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcaaa1e-66f3-4ab0-9180-a5669d8b541e admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:01.613635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcaaa1e-66f3-4ab0-9180-a5669d8b541e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:01.615617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcaaa1e-66f3-4ab0-9180-a5669d8b541e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:01.671970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f578c66-72bb-48af-88ea-18249f699f03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e50d742e41464ad99c3dfd94005509ef', 'request_id': 'req-9f578c66-72bb-48af-88ea-18249f699f03'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:01.678149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f578c66-72bb-48af-88ea-18249f699f03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e50d742e41464ad99c3dfd94005509ef', 'request_id': 'req-9f578c66-72bb-48af-88ea-18249f699f03'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:01.734836 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2441/4853] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:01 2022] POST /identity/v3/projects => generated 383 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:01.739480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcaaa1e-66f3-4ab0-9180-a5669d8b541e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:01.773293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f578c66-72bb-48af-88ea-18249f699f03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e50d742e41464ad99c3dfd94005509ef', 'request_id': 'req-9f578c66-72bb-48af-88ea-18249f699f03'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:01.787520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f578c66-72bb-48af-88ea-18249f699f03 admin admin] The token cache is being invalidate because project e50d742e41464ad99c3dfd94005509ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:01.787812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f578c66-72bb-48af-88ea-18249f699f03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:01.788508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f578c66-72bb-48af-88ea-18249f699f03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:01.790519 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2413/4854] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:01 2022] DELETE /identity/v3/projects/e50d742e41464ad99c3dfd94005509ef => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:01.795262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515b6bb-0ea2-40e4-ac4d-b186c526875f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:01.797304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515b6bb-0ea2-40e4-ac4d-b186c526875f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:01.797304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515b6bb-0ea2-40e4-ac4d-b186c526875f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:01.815291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4515b6bb-0ea2-40e4-ac4d-b186c526875f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:01.817136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4515b6bb-0ea2-40e4-ac4d-b186c526875f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:01.823613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcaaa1e-66f3-4ab0-9180-a5669d8b541e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:01.826273 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2414/4855] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:01.830441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce0b83-4f3a-405d-b8c9-6bb182d540de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:01.832114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce0b83-4f3a-405d-b8c9-6bb182d540de admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:01.832749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce0b83-4f3a-405d-b8c9-6bb182d540de admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:01.833229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce0b83-4f3a-405d-b8c9-6bb182d540de admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:01.834154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce0b83-4f3a-405d-b8c9-6bb182d540de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:01.837233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8862b-7d58-4d01-a9af-63fafe9d1db1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:01.837233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8862b-7d58-4d01-a9af-63fafe9d1db1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:01.837233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8862b-7d58-4d01-a9af-63fafe9d1db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:01.837527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce0b83-4f3a-405d-b8c9-6bb182d540de admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:01.883965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-84e8862b-7d58-4d01-a9af-63fafe9d1db1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:01.885549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-84e8862b-7d58-4d01-a9af-63fafe9d1db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:01.896300 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2415/4856] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:01.905534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8862b-7d58-4d01-a9af-63fafe9d1db1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:01.915931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8862b-7d58-4d01-a9af-63fafe9d1db1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:01.921061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462eeb0-0931-408a-9882-023df86231f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:01.925634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462eeb0-0931-408a-9882-023df86231f8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:01.925634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462eeb0-0931-408a-9882-023df86231f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:01.925634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462eeb0-0931-408a-9882-023df86231f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:01.925634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462eeb0-0931-408a-9882-023df86231f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:01.925634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462eeb0-0931-408a-9882-023df86231f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:01.925634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462eeb0-0931-408a-9882-023df86231f8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:01.936446 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2416/4857] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:19:01.941161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462eeb0-0931-408a-9882-023df86231f8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:01.950939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462eeb0-0931-408a-9882-023df86231f8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:01.957580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c842b28-771d-4133-9882-92216cd0d739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:01.958275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c842b28-771d-4133-9882-92216cd0d739 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:01.958568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c842b28-771d-4133-9882-92216cd0d739 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:01.958845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c842b28-771d-4133-9882-92216cd0d739 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:01.959539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c842b28-771d-4133-9882-92216cd0d739 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:01.961882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c842b28-771d-4133-9882-92216cd0d739 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:02.018184 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2442/4858] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:01 2022] POST /identity/v3/users => generated 292 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:02.019494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce0b83-4f3a-405d-b8c9-6bb182d540de admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:02.021614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e6ce0b83-4f3a-405d-b8c9-6bb182d540de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:02.036674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce0b83-4f3a-405d-b8c9-6bb182d540de admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:02.042117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945c5c2-b60d-4b02-8f0f-1f8b6424e336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:02.043068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945c5c2-b60d-4b02-8f0f-1f8b6424e336 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:02.044228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945c5c2-b60d-4b02-8f0f-1f8b6424e336 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:02.044628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945c5c2-b60d-4b02-8f0f-1f8b6424e336 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:02.045626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945c5c2-b60d-4b02-8f0f-1f8b6424e336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:02.047983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1945c5c2-b60d-4b02-8f0f-1f8b6424e336 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:02.073381 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2417/4859] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:01 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:02.075377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c842b28-771d-4133-9882-92216cd0d739 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:02.085759 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2443/4860] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:19:02.087897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:02.096335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945c5c2-b60d-4b02-8f0f-1f8b6424e336 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:02.114434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945c5c2-b60d-4b02-8f0f-1f8b6424e336 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:02.202729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c842b28-771d-4133-9882-92216cd0d739 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:02.204235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b66366-b0f2-4d43-ba8e-ec2300a94e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:02.205930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b66366-b0f2-4d43-ba8e-ec2300a94e31 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:02.206226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b66366-b0f2-4d43-ba8e-ec2300a94e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:02.206494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b66366-b0f2-4d43-ba8e-ec2300a94e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:02.207599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b66366-b0f2-4d43-ba8e-ec2300a94e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:02.211463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b66366-b0f2-4d43-ba8e-ec2300a94e31 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:02.217171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03fd3b-2d14-4dba-8718-5fe5e6afe404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:02.217171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03fd3b-2d14-4dba-8718-5fe5e6afe404 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:02.217461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03fd3b-2d14-4dba-8718-5fe5e6afe404 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:02.218207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03fd3b-2d14-4dba-8718-5fe5e6afe404 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:02.218921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03fd3b-2d14-4dba-8718-5fe5e6afe404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:02.221882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03fd3b-2d14-4dba-8718-5fe5e6afe404 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:02.226927 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2444/4861] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:02.229432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b66366-b0f2-4d43-ba8e-ec2300a94e31 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:02.245571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b66366-b0f2-4d43-ba8e-ec2300a94e31 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:02.257117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2174a8f2-7450-449d-bab5-ba06dc7c5544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:02.261420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174a8f2-7450-449d-bab5-ba06dc7c5544 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:02.261720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174a8f2-7450-449d-bab5-ba06dc7c5544 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:02.262042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174a8f2-7450-449d-bab5-ba06dc7c5544 admin admin] PATH_INFO: `/v3/projects/15e47e30726f4510b16bbf4fafeb5acf/users/8db0a84cc3674a7cb0cb1d2da4cb0afb/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:02.299338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174a8f2-7450-449d-bab5-ba06dc7c5544 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15e47e30726f4510b16bbf4fafeb5acf, user_id=8db0a84cc3674a7cb0cb1d2da4cb0afb, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:02.302067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2174a8f2-7450-449d-bab5-ba06dc7c5544 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:02.342689 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2418/4862] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:02 2022] POST /identity/v3/users => generated 288 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:02.344024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03fd3b-2d14-4dba-8718-5fe5e6afe404 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:02.358271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03fd3b-2d14-4dba-8718-5fe5e6afe404 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:02.360573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77081ee3-e031-4917-9e85-b88fb8fc0614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:02.361404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77081ee3-e031-4917-9e85-b88fb8fc0614 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:02.361697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77081ee3-e031-4917-9e85-b88fb8fc0614 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:02.362021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77081ee3-e031-4917-9e85-b88fb8fc0614 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:02.362785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77081ee3-e031-4917-9e85-b88fb8fc0614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:02.364457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77081ee3-e031-4917-9e85-b88fb8fc0614 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:02.375772 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2419/4863] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:02.398865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77081ee3-e031-4917-9e85-b88fb8fc0614 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:02.411978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77081ee3-e031-4917-9e85-b88fb8fc0614 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:02.419576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ffcf9-fd9f-4bca-995b-e6d02f3e80cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:02.420342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ffcf9-fd9f-4bca-995b-e6d02f3e80cd admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:02.420700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ffcf9-fd9f-4bca-995b-e6d02f3e80cd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:02.421057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ffcf9-fd9f-4bca-995b-e6d02f3e80cd admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:02.422566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ffcf9-fd9f-4bca-995b-e6d02f3e80cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:02.424760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ffcf9-fd9f-4bca-995b-e6d02f3e80cd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:02.437869 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2420/4864] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:02.442836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ffcf9-fd9f-4bca-995b-e6d02f3e80cd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:02.452816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ffcf9-fd9f-4bca-995b-e6d02f3e80cd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:02.457668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042b430-a993-46a9-8ca7-0d35b2521550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:02.458577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042b430-a993-46a9-8ca7-0d35b2521550 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:02.462820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042b430-a993-46a9-8ca7-0d35b2521550 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:02.462820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042b430-a993-46a9-8ca7-0d35b2521550 admin admin] PATH_INFO: `/v3/projects/a425d9daab7741c2bb877ebb0bfa8ec4/users/670e42b65f0b484b82a0b2c8139b8559/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:02.473578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042b430-a993-46a9-8ca7-0d35b2521550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a425d9daab7741c2bb877ebb0bfa8ec4, user_id=670e42b65f0b484b82a0b2c8139b8559, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:02.476342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042b430-a993-46a9-8ca7-0d35b2521550 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:02.488229 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2445/4865] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:02 2022] PUT /identity/v3/projects/15e47e30726f4510b16bbf4fafeb5acf/users/8db0a84cc3674a7cb0cb1d2da4cb0afb/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:02.498520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeaa46-2ea9-40a2-b4a2-af25bfb8ac85 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:02.498815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeaa46-2ea9-40a2-b4a2-af25bfb8ac85 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:02.499100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eeaa46-2ea9-40a2-b4a2-af25bfb8ac85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:02.525067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-57eeaa46-2ea9-40a2-b4a2-af25bfb8ac85 None None] MFA Rules not processed for user `8db0a84cc3674a7cb0cb1d2da4cb0afb`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:02.526392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-57eeaa46-2ea9-40a2-b4a2-af25bfb8ac85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:02.605721 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2446/4866] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:02.609821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeaa46-2ea9-40a2-b4a2-af25bfb8ac85 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:02.654754 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2421/4867] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:02 2022] PUT /identity/v3/projects/a425d9daab7741c2bb877ebb0bfa8ec4/users/670e42b65f0b484b82a0b2c8139b8559/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:02.667181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a5938-106f-4bc7-8aa8-849b5813025d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:02.667181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a5938-106f-4bc7-8aa8-849b5813025d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:02.667635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7a5938-106f-4bc7-8aa8-849b5813025d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:02.698771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6f7a5938-106f-4bc7-8aa8-849b5813025d None None] MFA Rules not processed for user `670e42b65f0b484b82a0b2c8139b8559`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:02.700776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6f7a5938-106f-4bc7-8aa8-849b5813025d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:02.717737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eeaa46-2ea9-40a2-b4a2-af25bfb8ac85 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:02.796868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03c53d-77c6-469a-9045-edbba987c433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:02.797611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03c53d-77c6-469a-9045-edbba987c433 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:02.797930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03c53d-77c6-469a-9045-edbba987c433 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:02.798234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03c53d-77c6-469a-9045-edbba987c433 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:02.799682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03c53d-77c6-469a-9045-edbba987c433 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:02.800850 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2422/4868] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:02.801699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03c53d-77c6-469a-9045-edbba987c433 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:02.811603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a5938-106f-4bc7-8aa8-849b5813025d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:02.815885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6f7a5938-106f-4bc7-8aa8-849b5813025d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:02.873600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7a5938-106f-4bc7-8aa8-849b5813025d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:02.877495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fd0d4-7c05-4264-aea9-9faa4e6a1f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:02.878270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fd0d4-7c05-4264-aea9-9faa4e6a1f88 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:02.878558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fd0d4-7c05-4264-aea9-9faa4e6a1f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:02.878840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fd0d4-7c05-4264-aea9-9faa4e6a1f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:02.888566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:02.898763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004fd0d4-7c05-4264-aea9-9faa4e6a1f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:02.901511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004fd0d4-7c05-4264-aea9-9faa4e6a1f88 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:02.913140 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2423/4869] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:02.938111 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2447/4870] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:02 2022] POST /identity/v3/projects => generated 381 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:02.940844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fd0d4-7c05-4264-aea9-9faa4e6a1f88 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:02.955202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03c53d-77c6-469a-9045-edbba987c433 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:02.965420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fd0d4-7c05-4264-aea9-9faa4e6a1f88 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:03.118259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03c53d-77c6-469a-9045-edbba987c433 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:03.126493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96e91b1-b719-45d1-bbff-443bb1ddd048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:03.128798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e91b1-b719-45d1-bbff-443bb1ddd048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:03.129090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e91b1-b719-45d1-bbff-443bb1ddd048 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:03.130796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f0e49-8751-453a-8371-03f35c22e3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:03.131123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f0e49-8751-453a-8371-03f35c22e3fc admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:03.131216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f0e49-8751-453a-8371-03f35c22e3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:03.131428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0f0e49-8751-453a-8371-03f35c22e3fc admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:03.131972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96e91b1-b719-45d1-bbff-443bb1ddd048 admin admin] PATH_INFO: `/v3/users/670e42b65f0b484b82a0b2c8139b8559` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:03.132142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f0e49-8751-453a-8371-03f35c22e3fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:03.133673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e91b1-b719-45d1-bbff-443bb1ddd048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=670e42b65f0b484b82a0b2c8139b8559)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:03.138190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0f0e49-8751-453a-8371-03f35c22e3fc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:03.146293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96e91b1-b719-45d1-bbff-443bb1ddd048 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:03.247027 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2448/4871] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:02 2022] POST /identity/v3/users => generated 291 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:03.248515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f0e49-8751-453a-8371-03f35c22e3fc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:03.261950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0f0e49-8751-453a-8371-03f35c22e3fc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:03.268397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764ce13-8979-47e2-9432-3c5d8c7c44f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:03.269200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764ce13-8979-47e2-9432-3c5d8c7c44f5 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:03.273718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764ce13-8979-47e2-9432-3c5d8c7c44f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:03.277471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764ce13-8979-47e2-9432-3c5d8c7c44f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:03.277471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764ce13-8979-47e2-9432-3c5d8c7c44f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:03.284713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c764ce13-8979-47e2-9432-3c5d8c7c44f5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:03.303048 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2449/4872] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:03.303941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764ce13-8979-47e2-9432-3c5d8c7c44f5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:03.316096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764ce13-8979-47e2-9432-3c5d8c7c44f5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:03.321004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df904cd-6257-452b-a68f-a8744d0137e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:03.321681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df904cd-6257-452b-a68f-a8744d0137e6 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:03.322016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df904cd-6257-452b-a68f-a8744d0137e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:03.322434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df904cd-6257-452b-a68f-a8744d0137e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:03.323330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df904cd-6257-452b-a68f-a8744d0137e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:03.325153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df904cd-6257-452b-a68f-a8744d0137e6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:03.346379 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2450/4873] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:03.347972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df904cd-6257-452b-a68f-a8744d0137e6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:03.359911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df904cd-6257-452b-a68f-a8744d0137e6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:03.365620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862f8e5-faf9-4e16-92f5-353c26781231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:03.367610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862f8e5-faf9-4e16-92f5-353c26781231 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:03.367751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862f8e5-faf9-4e16-92f5-353c26781231 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:03.368131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862f8e5-faf9-4e16-92f5-353c26781231 admin admin] PATH_INFO: `/v3/projects/89ca1e2b61c04e7ba89c61d7cf5f5d5b/users/849fd7c01fa34d25aedf00a7842e1344/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:03.387779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8862f8e5-faf9-4e16-92f5-353c26781231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ca1e2b61c04e7ba89c61d7cf5f5d5b, user_id=849fd7c01fa34d25aedf00a7842e1344, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:03.391003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8862f8e5-faf9-4e16-92f5-353c26781231 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:03.417680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e96e91b1-b719-45d1-bbff-443bb1ddd048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '670e42b65f0b484b82a0b2c8139b8559'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:03.419693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e96e91b1-b719-45d1-bbff-443bb1ddd048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '670e42b65f0b484b82a0b2c8139b8559'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:03.428133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e96e91b1-b719-45d1-bbff-443bb1ddd048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '670e42b65f0b484b82a0b2c8139b8559'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:03.471611 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2451/4874] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:03 2022] PUT /identity/v3/projects/89ca1e2b61c04e7ba89c61d7cf5f5d5b/users/849fd7c01fa34d25aedf00a7842e1344/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:03.482027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e96e91b1-b719-45d1-bbff-443bb1ddd048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '670e42b65f0b484b82a0b2c8139b8559'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:03.490146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbf63d-e073-46e0-9419-22c8cbbc0d9e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:03.490146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbf63d-e073-46e0-9419-22c8cbbc0d9e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:03.490146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbf63d-e073-46e0-9419-22c8cbbc0d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:03.490466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e96e91b1-b719-45d1-bbff-443bb1ddd048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '670e42b65f0b484b82a0b2c8139b8559'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:03.507309 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2424/4875] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:02 2022] DELETE /identity/v3/users/670e42b65f0b484b82a0b2c8139b8559 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:03.511665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362daa7-dbce-4f31-a783-04a466b11bc6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:03.511963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362daa7-dbce-4f31-a783-04a466b11bc6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:03.512245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362daa7-dbce-4f31-a783-04a466b11bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:03.514347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-06dbf63d-e073-46e0-9419-22c8cbbc0d9e None None] MFA Rules not processed for user `849fd7c01fa34d25aedf00a7842e1344`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:03.515618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-06dbf63d-e073-46e0-9419-22c8cbbc0d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:03.530442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e362daa7-dbce-4f31-a783-04a466b11bc6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:03.531755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e362daa7-dbce-4f31-a783-04a466b11bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:03.591655 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2452/4876] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:03.604325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbf63d-e073-46e0-9419-22c8cbbc0d9e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:03.606859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-06dbf63d-e073-46e0-9419-22c8cbbc0d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:03.622771 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2425/4877] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:03.632630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362daa7-dbce-4f31-a783-04a466b11bc6 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:03.634202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e362daa7-dbce-4f31-a783-04a466b11bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:03.667657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbf63d-e073-46e0-9419-22c8cbbc0d9e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:03.671305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918778b-206d-4279-b2a0-2c790ffc2a0c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:03.671953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918778b-206d-4279-b2a0-2c790ffc2a0c service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:03.672238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918778b-206d-4279-b2a0-2c790ffc2a0c service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:03.672516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918778b-206d-4279-b2a0-2c790ffc2a0c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:03.674948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1918778b-206d-4279-b2a0-2c790ffc2a0c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:03.696049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362daa7-dbce-4f31-a783-04a466b11bc6 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:03.700558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1baa9e-902b-4f76-b50a-db8f0a20c466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:03.701224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1baa9e-902b-4f76-b50a-db8f0a20c466 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:03.701510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1baa9e-902b-4f76-b50a-db8f0a20c466 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:03.702092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1baa9e-902b-4f76-b50a-db8f0a20c466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:03.712950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1baa9e-902b-4f76-b50a-db8f0a20c466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:03.715309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1baa9e-902b-4f76-b50a-db8f0a20c466 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:03.736765 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2426/4878] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:19:03.755685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918778b-206d-4279-b2a0-2c790ffc2a0c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:03.757423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918778b-206d-4279-b2a0-2c790ffc2a0c service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:03.769276 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2453/4879] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:19:04.063345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1baa9e-902b-4f76-b50a-db8f0a20c466 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:04.076321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1baa9e-902b-4f76-b50a-db8f0a20c466 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:04.080799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e3cf2-19aa-4246-a2a9-462a60440090 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:04.081502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e3cf2-19aa-4246-a2a9-462a60440090 service glance] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:04.081785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e3cf2-19aa-4246-a2a9-462a60440090 service glance] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:04.087750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e3cf2-19aa-4246-a2a9-462a60440090 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:04.092043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e3cf2-19aa-4246-a2a9-462a60440090 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:04.093773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e3cf2-19aa-4246-a2a9-462a60440090 service glance] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:04.122794 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2427/4880] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:04.286708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918778b-206d-4279-b2a0-2c790ffc2a0c service glance] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:04.288154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1918778b-206d-4279-b2a0-2c790ffc2a0c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:04.302715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918778b-206d-4279-b2a0-2c790ffc2a0c service glance] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:04.307424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d71c6-2961-419c-9f0d-d7cef584969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:04.308378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d71c6-2961-419c-9f0d-d7cef584969c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:04.308681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d71c6-2961-419c-9f0d-d7cef584969c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:04.308949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d71c6-2961-419c-9f0d-d7cef584969c admin admin] PATH_INFO: `/v3/users/cb285a5e922745bbbd61f4c6ca663c71` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:04.310235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d71c6-2961-419c-9f0d-d7cef584969c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb285a5e922745bbbd61f4c6ca663c71)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:04.312177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d71c6-2961-419c-9f0d-d7cef584969c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:04.384401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:04.540005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-802d71c6-2961-419c-9f0d-d7cef584969c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb285a5e922745bbbd61f4c6ca663c71'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:04.553051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-802d71c6-2961-419c-9f0d-d7cef584969c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb285a5e922745bbbd61f4c6ca663c71'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:04.554062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-802d71c6-2961-419c-9f0d-d7cef584969c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb285a5e922745bbbd61f4c6ca663c71'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:04.555061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-802d71c6-2961-419c-9f0d-d7cef584969c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb285a5e922745bbbd61f4c6ca663c71'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:04.591842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e3cf2-19aa-4246-a2a9-462a60440090 service glance] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:04.593811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a00e3cf2-19aa-4246-a2a9-462a60440090 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:04.622072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e3cf2-19aa-4246-a2a9-462a60440090 service glance] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:04.635222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51965551-47a4-4574-b53f-0d4d81177cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:04.637674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51965551-47a4-4574-b53f-0d4d81177cbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:04.640548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51965551-47a4-4574-b53f-0d4d81177cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:04.640996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51965551-47a4-4574-b53f-0d4d81177cbc admin admin] PATH_INFO: `/v3/projects/a425d9daab7741c2bb877ebb0bfa8ec4` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:04.644209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51965551-47a4-4574-b53f-0d4d81177cbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a425d9daab7741c2bb877ebb0bfa8ec4)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:04.647145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51965551-47a4-4574-b53f-0d4d81177cbc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:04.699192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-802d71c6-2961-419c-9f0d-d7cef584969c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb285a5e922745bbbd61f4c6ca663c71'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:04.718672 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2454/4881] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:04 2022] DELETE /identity/v3/users/cb285a5e922745bbbd61f4c6ca663c71 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:04.814709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:04.862615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51965551-47a4-4574-b53f-0d4d81177cbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a425d9daab7741c2bb877ebb0bfa8ec4', 'request_id': 'req-51965551-47a4-4574-b53f-0d4d81177cbc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:04.870113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51965551-47a4-4574-b53f-0d4d81177cbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a425d9daab7741c2bb877ebb0bfa8ec4', 'request_id': 'req-51965551-47a4-4574-b53f-0d4d81177cbc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:04.990572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51965551-47a4-4574-b53f-0d4d81177cbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a425d9daab7741c2bb877ebb0bfa8ec4', 'request_id': 'req-51965551-47a4-4574-b53f-0d4d81177cbc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:05.011914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51965551-47a4-4574-b53f-0d4d81177cbc admin admin] The token cache is being invalidate because project a425d9daab7741c2bb877ebb0bfa8ec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:05.012335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51965551-47a4-4574-b53f-0d4d81177cbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:05.013089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51965551-47a4-4574-b53f-0d4d81177cbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:05.014760 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2428/4882] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:04 2022] DELETE /identity/v3/projects/a425d9daab7741c2bb877ebb0bfa8ec4 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:05.019936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a01033-53c7-403c-8456-6866a75e958e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:05.020239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a01033-53c7-403c-8456-6866a75e958e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:05.020527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a01033-53c7-403c-8456-6866a75e958e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:05.039809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-76a01033-53c7-403c-8456-6866a75e958e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:05.041042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-76a01033-53c7-403c-8456-6866a75e958e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:05.107673 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2455/4883] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:05.121956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38ddfa-2446-4f02-a187-60b8b66c6a83 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:05.123586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38ddfa-2446-4f02-a187-60b8b66c6a83 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:05.125296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38ddfa-2446-4f02-a187-60b8b66c6a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:05.134135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a01033-53c7-403c-8456-6866a75e958e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:05.135834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-76a01033-53c7-403c-8456-6866a75e958e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:05.172960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2a38ddfa-2446-4f02-a187-60b8b66c6a83 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:05.174714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2a38ddfa-2446-4f02-a187-60b8b66c6a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:05.192835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a01033-53c7-403c-8456-6866a75e958e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:05.197413 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2429/4884] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:05.201518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38ddfa-2446-4f02-a187-60b8b66c6a83 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:05.203316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e0274-121d-4987-88c9-3b8eca410e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:05.204946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e0274-121d-4987-88c9-3b8eca410e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:05.206316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e0274-121d-4987-88c9-3b8eca410e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:05.206756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e0274-121d-4987-88c9-3b8eca410e41 admin admin] PATH_INFO: `/v3/projects/84e22c442b0241f18af6ee9408d47223` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:05.210084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e0274-121d-4987-88c9-3b8eca410e41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84e22c442b0241f18af6ee9408d47223)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:05.213944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e0274-121d-4987-88c9-3b8eca410e41 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:05.221184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38ddfa-2446-4f02-a187-60b8b66c6a83 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:05.231603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3ad49-6f3a-4b75-b609-7f25fb3fe969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:05.233332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3ad49-6f3a-4b75-b609-7f25fb3fe969 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:05.233810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3ad49-6f3a-4b75-b609-7f25fb3fe969 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:05.234302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3ad49-6f3a-4b75-b609-7f25fb3fe969 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:05.235574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3ad49-6f3a-4b75-b609-7f25fb3fe969 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:05.236730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3ad49-6f3a-4b75-b609-7f25fb3fe969 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:05.240721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3ad49-6f3a-4b75-b609-7f25fb3fe969 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:05.270941 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2430/4885] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:05.275027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3ad49-6f3a-4b75-b609-7f25fb3fe969 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:05.290067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3ad49-6f3a-4b75-b609-7f25fb3fe969 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:05.296604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181d211-bd82-44fb-b31e-452d73b1fed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:05.297364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181d211-bd82-44fb-b31e-452d73b1fed8 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:05.297710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181d211-bd82-44fb-b31e-452d73b1fed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:05.298089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181d211-bd82-44fb-b31e-452d73b1fed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:05.299805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181d211-bd82-44fb-b31e-452d73b1fed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:05.302153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181d211-bd82-44fb-b31e-452d73b1fed8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:05.339141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:05.409086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ff3e0274-121d-4987-88c9-3b8eca410e41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84e22c442b0241f18af6ee9408d47223', 'request_id': 'req-ff3e0274-121d-4987-88c9-3b8eca410e41'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:05.414800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ff3e0274-121d-4987-88c9-3b8eca410e41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84e22c442b0241f18af6ee9408d47223', 'request_id': 'req-ff3e0274-121d-4987-88c9-3b8eca410e41'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:05.435760 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2431/4886] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:05 2022] POST /identity/v3/projects => generated 357 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:05.435760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181d211-bd82-44fb-b31e-452d73b1fed8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:05.435760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b181d211-bd82-44fb-b31e-452d73b1fed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:05.519764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ff3e0274-121d-4987-88c9-3b8eca410e41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84e22c442b0241f18af6ee9408d47223', 'request_id': 'req-ff3e0274-121d-4987-88c9-3b8eca410e41'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:05.522660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181d211-bd82-44fb-b31e-452d73b1fed8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:05.528799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902d1b3-4523-4caa-a56e-3bc1f6f3eb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:05.530026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902d1b3-4523-4caa-a56e-3bc1f6f3eb36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:05.530485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902d1b3-4523-4caa-a56e-3bc1f6f3eb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:05.530935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e902d1b3-4523-4caa-a56e-3bc1f6f3eb36 admin admin] PATH_INFO: `/v3/users/8db0a84cc3674a7cb0cb1d2da4cb0afb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:05.532363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e902d1b3-4523-4caa-a56e-3bc1f6f3eb36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8db0a84cc3674a7cb0cb1d2da4cb0afb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:05.535384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e902d1b3-4523-4caa-a56e-3bc1f6f3eb36 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:05.540586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ff3e0274-121d-4987-88c9-3b8eca410e41 admin admin] The token cache is being invalidate because project 84e22c442b0241f18af6ee9408d47223 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:05.540832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ff3e0274-121d-4987-88c9-3b8eca410e41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:05.541600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ff3e0274-121d-4987-88c9-3b8eca410e41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:05.544907 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2456/4887] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:05 2022] DELETE /identity/v3/projects/84e22c442b0241f18af6ee9408d47223 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:05.550416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e0274-121d-4987-88c9-3b8eca410e41 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:05.551925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ff3e0274-121d-4987-88c9-3b8eca410e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:05.685454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e0274-121d-4987-88c9-3b8eca410e41 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:05.691954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa99e9cf-3336-4452-a837-df8091afaf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:05.692720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99e9cf-3336-4452-a837-df8091afaf6f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:05.692950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99e9cf-3336-4452-a837-df8091afaf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:05.693230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa99e9cf-3336-4452-a837-df8091afaf6f admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:05.694030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99e9cf-3336-4452-a837-df8091afaf6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:05.696073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa99e9cf-3336-4452-a837-df8091afaf6f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:05.807779 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2457/4888] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:05 2022] POST /identity/v3/users => generated 279 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:05.809611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0806d-0f90-487f-a4e8-618228e4bbf0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:05.809945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0806d-0f90-487f-a4e8-618228e4bbf0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:05.810373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c0806d-0f90-487f-a4e8-618228e4bbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:05.829560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-74c0806d-0f90-487f-a4e8-618228e4bbf0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:05.831345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-74c0806d-0f90-487f-a4e8-618228e4bbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:05.840126 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2458/4889] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:05.846618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0806d-0f90-487f-a4e8-618228e4bbf0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:05.846618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-74c0806d-0f90-487f-a4e8-618228e4bbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:05.853366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e902d1b3-4523-4caa-a56e-3bc1f6f3eb36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8db0a84cc3674a7cb0cb1d2da4cb0afb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:05.854108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e902d1b3-4523-4caa-a56e-3bc1f6f3eb36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8db0a84cc3674a7cb0cb1d2da4cb0afb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:05.855199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c0806d-0f90-487f-a4e8-618228e4bbf0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:05.862333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbee27c-4f21-4ccb-9dd3-80b309eecf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:05.863768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e902d1b3-4523-4caa-a56e-3bc1f6f3eb36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8db0a84cc3674a7cb0cb1d2da4cb0afb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:05.863887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbee27c-4f21-4ccb-9dd3-80b309eecf38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:05.863887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbee27c-4f21-4ccb-9dd3-80b309eecf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:05.863887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbee27c-4f21-4ccb-9dd3-80b309eecf38 admin admin] PATH_INFO: `/v3/users/935b49aadd30450bb08df41f8e859007` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:05.864975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbee27c-4f21-4ccb-9dd3-80b309eecf38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=935b49aadd30450bb08df41f8e859007)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:05.867050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbee27c-4f21-4ccb-9dd3-80b309eecf38 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:05.927105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e902d1b3-4523-4caa-a56e-3bc1f6f3eb36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8db0a84cc3674a7cb0cb1d2da4cb0afb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:05.927741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e902d1b3-4523-4caa-a56e-3bc1f6f3eb36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8db0a84cc3674a7cb0cb1d2da4cb0afb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:05.953763 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2432/4890] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:05 2022] DELETE /identity/v3/users/8db0a84cc3674a7cb0cb1d2da4cb0afb => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:05.953763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902d1b3-4523-4caa-a56e-3bc1f6f3eb36 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:05.955511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e902d1b3-4523-4caa-a56e-3bc1f6f3eb36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:06.038489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e902d1b3-4523-4caa-a56e-3bc1f6f3eb36 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:06.050315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab7a53d-2021-42f9-8908-c239aff261e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:06.051178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7a53d-2021-42f9-8908-c239aff261e0 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:06.051606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7a53d-2021-42f9-8908-c239aff261e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:06.051751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab7a53d-2021-42f9-8908-c239aff261e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:06.052651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7a53d-2021-42f9-8908-c239aff261e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:06.054396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab7a53d-2021-42f9-8908-c239aff261e0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:06.057044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9fbee27c-4f21-4ccb-9dd3-80b309eecf38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '935b49aadd30450bb08df41f8e859007'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:06.071810 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2433/4891] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:06.073281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2ee4c-1ae0-4f73-b25c-e02576bf6670 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:06.073576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2ee4c-1ae0-4f73-b25c-e02576bf6670 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:06.073936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2ee4c-1ae0-4f73-b25c-e02576bf6670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:06.084522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9fbee27c-4f21-4ccb-9dd3-80b309eecf38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '935b49aadd30450bb08df41f8e859007'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:06.085374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9fbee27c-4f21-4ccb-9dd3-80b309eecf38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '935b49aadd30450bb08df41f8e859007'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:06.086330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9fbee27c-4f21-4ccb-9dd3-80b309eecf38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '935b49aadd30450bb08df41f8e859007'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:06.094395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-83f2ee4c-1ae0-4f73-b25c-e02576bf6670 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:06.095691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-83f2ee4c-1ae0-4f73-b25c-e02576bf6670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:06.109447 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2434/4892] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:06.110548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48205af-0784-47d7-9b18-083a58a7f51a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:06.111372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48205af-0784-47d7-9b18-083a58a7f51a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:06.111652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48205af-0784-47d7-9b18-083a58a7f51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:06.130311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c48205af-0784-47d7-9b18-083a58a7f51a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:06.131524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c48205af-0784-47d7-9b18-083a58a7f51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:06.140550 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2435/4893] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:06.143003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48205af-0784-47d7-9b18-083a58a7f51a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:06.145071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c48205af-0784-47d7-9b18-083a58a7f51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:06.159191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48205af-0784-47d7-9b18-083a58a7f51a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:06.164863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649ae4a-29d8-4021-b2e3-b647d5efd041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:06.165632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649ae4a-29d8-4021-b2e3-b647d5efd041 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:06.165953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649ae4a-29d8-4021-b2e3-b647d5efd041 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:06.166234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649ae4a-29d8-4021-b2e3-b647d5efd041 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:06.166968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649ae4a-29d8-4021-b2e3-b647d5efd041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:06.168816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649ae4a-29d8-4021-b2e3-b647d5efd041 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:06.172838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9fbee27c-4f21-4ccb-9dd3-80b309eecf38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '935b49aadd30450bb08df41f8e859007'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:06.186023 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2436/4894] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:06.186023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649ae4a-29d8-4021-b2e3-b647d5efd041 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:06.194986 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2459/4895] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:05 2022] DELETE /identity/v3/users/935b49aadd30450bb08df41f8e859007 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:06.195926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbee27c-4f21-4ccb-9dd3-80b309eecf38 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:06.197942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9fbee27c-4f21-4ccb-9dd3-80b309eecf38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:06.204307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649ae4a-29d8-4021-b2e3-b647d5efd041 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:06.266390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbee27c-4f21-4ccb-9dd3-80b309eecf38 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:06.271156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf9664-0b1f-4f19-9d2f-9d2333469568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:06.271828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf9664-0b1f-4f19-9d2f-9d2333469568 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:06.272118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf9664-0b1f-4f19-9d2f-9d2333469568 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:06.272483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf9664-0b1f-4f19-9d2f-9d2333469568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:06.304505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c93b4e-7990-4481-96da-2e358e27dff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:06.305578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c93b4e-7990-4481-96da-2e358e27dff0 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:06.306126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c93b4e-7990-4481-96da-2e358e27dff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:06.306617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c93b4e-7990-4481-96da-2e358e27dff0 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:06.307590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c93b4e-7990-4481-96da-2e358e27dff0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:06.308099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c93b4e-7990-4481-96da-2e358e27dff0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:06.310363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c93b4e-7990-4481-96da-2e358e27dff0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:06.325771 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2437/4896] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:06.327342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c93b4e-7990-4481-96da-2e358e27dff0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:06.339146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c93b4e-7990-4481-96da-2e358e27dff0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:06.348304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44999c20-774f-4252-9604-eca304829202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:06.349086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44999c20-774f-4252-9604-eca304829202 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:06.349370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44999c20-774f-4252-9604-eca304829202 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:06.349648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44999c20-774f-4252-9604-eca304829202 admin admin] PATH_INFO: `/v3/projects/56c1d2ac8d054765968184cfa5494f67/users/db5d4b639d094820a7cdcfe8d7688c53/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:06.369005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44999c20-774f-4252-9604-eca304829202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56c1d2ac8d054765968184cfa5494f67, user_id=db5d4b639d094820a7cdcfe8d7688c53, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:06.371252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf9664-0b1f-4f19-9d2f-9d2333469568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:06.371821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44999c20-774f-4252-9604-eca304829202 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:06.373043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf9664-0b1f-4f19-9d2f-9d2333469568 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:06.386954 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2460/4897] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:06.388319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311284f-7017-47bb-a4ad-7879efe06c73 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:06.388612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311284f-7017-47bb-a4ad-7879efe06c73 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:06.388898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311284f-7017-47bb-a4ad-7879efe06c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:06.412564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9311284f-7017-47bb-a4ad-7879efe06c73 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:06.413484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9311284f-7017-47bb-a4ad-7879efe06c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:06.426689 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2461/4898] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:06.428377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311284f-7017-47bb-a4ad-7879efe06c73 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:06.444066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311284f-7017-47bb-a4ad-7879efe06c73 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:06.449281 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2438/4899] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:06 2022] PUT /identity/v3/projects/56c1d2ac8d054765968184cfa5494f67/users/db5d4b639d094820a7cdcfe8d7688c53/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:06.459443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44999c20-774f-4252-9604-eca304829202 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:06.461568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-44999c20-774f-4252-9604-eca304829202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:06.465064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:06.567176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44999c20-774f-4252-9604-eca304829202 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:06.571327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c96ba6-388c-4209-b39a-df72f520c3ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:06.572112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c96ba6-388c-4209-b39a-df72f520c3ad service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:06.572403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c96ba6-388c-4209-b39a-df72f520c3ad service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:06.572679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c96ba6-388c-4209-b39a-df72f520c3ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:06.577789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d317c7b-c5b0-48b6-8867-49e64347fdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:06.578570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d317c7b-c5b0-48b6-8867-49e64347fdd3 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:06.590871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d317c7b-c5b0-48b6-8867-49e64347fdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:06.590871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d317c7b-c5b0-48b6-8867-49e64347fdd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:06.590871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d317c7b-c5b0-48b6-8867-49e64347fdd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:06.590871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d317c7b-c5b0-48b6-8867-49e64347fdd3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:06.591309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c96ba6-388c-4209-b39a-df72f520c3ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:06.591309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c96ba6-388c-4209-b39a-df72f520c3ad service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:06.601274 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2439/4900] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:06.602861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69582951-8868-4ee1-b9cc-6749e5474de8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:06.603211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69582951-8868-4ee1-b9cc-6749e5474de8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:06.603501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69582951-8868-4ee1-b9cc-6749e5474de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:06.636877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-69582951-8868-4ee1-b9cc-6749e5474de8 None None] MFA Rules not processed for user `db5d4b639d094820a7cdcfe8d7688c53`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:06.638408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-69582951-8868-4ee1-b9cc-6749e5474de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:06.719005 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2440/4901] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:06 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:06.733285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69582951-8868-4ee1-b9cc-6749e5474de8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:06.745983 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2462/4902] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:06 2022] POST /identity/v3/projects => generated 389 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:06.752203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d317c7b-c5b0-48b6-8867-49e64347fdd3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:06.756171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69582951-8868-4ee1-b9cc-6749e5474de8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:06.858869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d317c7b-c5b0-48b6-8867-49e64347fdd3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:06.861910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfdafe-617d-4bbb-919f-5596959b0e90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:06.862773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfdafe-617d-4bbb-919f-5596959b0e90 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:06.863135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfdafe-617d-4bbb-919f-5596959b0e90 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:06.863624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dfdafe-617d-4bbb-919f-5596959b0e90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:06.866241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0c321-a2a3-4dbb-be29-509438edcc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:06.867043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0c321-a2a3-4dbb-be29-509438edcc18 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:06.867388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0c321-a2a3-4dbb-be29-509438edcc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:06.867939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0c321-a2a3-4dbb-be29-509438edcc18 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:06.868933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0c321-a2a3-4dbb-be29-509438edcc18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:06.872275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0c321-a2a3-4dbb-be29-509438edcc18 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:06.937083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:06.945954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfdafe-617d-4bbb-919f-5596959b0e90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:06.945954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dfdafe-617d-4bbb-919f-5596959b0e90 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:06.960318 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2441/4903] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:06 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:19:06.976472 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2463/4904] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:06 2022] POST /identity/v3/users => generated 295 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:06.980543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfdafe-617d-4bbb-919f-5596959b0e90 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:06.994075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dfdafe-617d-4bbb-919f-5596959b0e90 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:06.999229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0e298-cea7-4d9a-9595-212a9b7548ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:07.000062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0e298-cea7-4d9a-9595-212a9b7548ea admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:07.000359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0e298-cea7-4d9a-9595-212a9b7548ea admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:07.000671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0e298-cea7-4d9a-9595-212a9b7548ea admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:07.001574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0e298-cea7-4d9a-9595-212a9b7548ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:07.003253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0e298-cea7-4d9a-9595-212a9b7548ea admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:07.016414 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2442/4905] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:07.020825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0c321-a2a3-4dbb-be29-509438edcc18 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:07.035741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0c321-a2a3-4dbb-be29-509438edcc18 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:07.041114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ae66c-0e51-4368-8af8-5560b6231d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:07.042104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae66c-0e51-4368-8af8-5560b6231d0c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:07.042410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae66c-0e51-4368-8af8-5560b6231d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:07.042763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ae66c-0e51-4368-8af8-5560b6231d0c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:07.043585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ae66c-0e51-4368-8af8-5560b6231d0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:07.045337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479ae66c-0e51-4368-8af8-5560b6231d0c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:07.058792 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2464/4906] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:07.065052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0e298-cea7-4d9a-9595-212a9b7548ea admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:07.075947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0e298-cea7-4d9a-9595-212a9b7548ea admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:07.085118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a4c76-ae51-4f6c-bb13-1255b8680afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:07.085118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a4c76-ae51-4f6c-bb13-1255b8680afd admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:07.085118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a4c76-ae51-4f6c-bb13-1255b8680afd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:07.085419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a4c76-ae51-4f6c-bb13-1255b8680afd admin admin] PATH_INFO: `/v3/projects/2f1ab0fa428c4239ae77e49feabf77f3/users/e567fd5a2683491281fab7551a485a4d/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:07.114479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a4c76-ae51-4f6c-bb13-1255b8680afd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f1ab0fa428c4239ae77e49feabf77f3, user_id=e567fd5a2683491281fab7551a485a4d, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:07.117326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403a4c76-ae51-4f6c-bb13-1255b8680afd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:07.221895 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2443/4907] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:07 2022] PUT /identity/v3/projects/2f1ab0fa428c4239ae77e49feabf77f3/users/e567fd5a2683491281fab7551a485a4d/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:07.225775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794870a-7576-4d0a-82e6-6fd377266643 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:07.226180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794870a-7576-4d0a-82e6-6fd377266643 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:07.226643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794870a-7576-4d0a-82e6-6fd377266643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:07.256535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d794870a-7576-4d0a-82e6-6fd377266643 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:07.257822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d794870a-7576-4d0a-82e6-6fd377266643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:07.322335 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2465/4908] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:07.339576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a4c76-ae51-4f6c-bb13-1255b8680afd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:07.412151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a4c76-ae51-4f6c-bb13-1255b8680afd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:07.418826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef2006-a9f9-4338-a3e2-23e378d52209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:07.419596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef2006-a9f9-4338-a3e2-23e378d52209 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:07.419887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef2006-a9f9-4338-a3e2-23e378d52209 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:07.420171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ef2006-a9f9-4338-a3e2-23e378d52209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:07.430802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef2006-a9f9-4338-a3e2-23e378d52209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:07.432936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ef2006-a9f9-4338-a3e2-23e378d52209 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:07.433407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794870a-7576-4d0a-82e6-6fd377266643 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:07.435639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d794870a-7576-4d0a-82e6-6fd377266643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:07.446930 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2444/4909] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:07.447876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef2006-a9f9-4338-a3e2-23e378d52209 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:07.449675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-33ef2006-a9f9-4338-a3e2-23e378d52209 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:07.460421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794870a-7576-4d0a-82e6-6fd377266643 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:07.465430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ef2006-a9f9-4338-a3e2-23e378d52209 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:07.470732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b58a1-2ef6-4a60-9c5e-adcdb130751a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:07.471681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b58a1-2ef6-4a60-9c5e-adcdb130751a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:07.471972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b58a1-2ef6-4a60-9c5e-adcdb130751a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:07.472252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b58a1-2ef6-4a60-9c5e-adcdb130751a admin admin] PATH_INFO: `/v3/projects/2e5d8fcff8014060a847cb4ae0738057` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:07.473497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b58a1-2ef6-4a60-9c5e-adcdb130751a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e5d8fcff8014060a847cb4ae0738057)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:07.475768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b58a1-2ef6-4a60-9c5e-adcdb130751a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:07.476515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3d034-95a7-42c4-a8e7-677596fd6d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:07.477751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3d034-95a7-42c4-a8e7-677596fd6d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:07.478256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3d034-95a7-42c4-a8e7-677596fd6d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:07.478740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3d034-95a7-42c4-a8e7-677596fd6d77 admin admin] PATH_INFO: `/v3/projects/15e47e30726f4510b16bbf4fafeb5acf` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:07.481814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3d034-95a7-42c4-a8e7-677596fd6d77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15e47e30726f4510b16bbf4fafeb5acf)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:07.484358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3d034-95a7-42c4-a8e7-677596fd6d77 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:07.673064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-124b58a1-2ef6-4a60-9c5e-adcdb130751a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e5d8fcff8014060a847cb4ae0738057', 'request_id': 'req-124b58a1-2ef6-4a60-9c5e-adcdb130751a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:07.685042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-124b58a1-2ef6-4a60-9c5e-adcdb130751a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e5d8fcff8014060a847cb4ae0738057', 'request_id': 'req-124b58a1-2ef6-4a60-9c5e-adcdb130751a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:07.690692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d2d3d034-95a7-42c4-a8e7-677596fd6d77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15e47e30726f4510b16bbf4fafeb5acf', 'request_id': 'req-d2d3d034-95a7-42c4-a8e7-677596fd6d77'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:07.708775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d2d3d034-95a7-42c4-a8e7-677596fd6d77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15e47e30726f4510b16bbf4fafeb5acf', 'request_id': 'req-d2d3d034-95a7-42c4-a8e7-677596fd6d77'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:07.806140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-124b58a1-2ef6-4a60-9c5e-adcdb130751a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e5d8fcff8014060a847cb4ae0738057', 'request_id': 'req-124b58a1-2ef6-4a60-9c5e-adcdb130751a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:07.818818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-124b58a1-2ef6-4a60-9c5e-adcdb130751a admin admin] The token cache is being invalidate because project 2e5d8fcff8014060a847cb4ae0738057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:07.819395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-124b58a1-2ef6-4a60-9c5e-adcdb130751a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:07.820365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-124b58a1-2ef6-4a60-9c5e-adcdb130751a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:07.822994 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2445/4910] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:07 2022] DELETE /identity/v3/projects/2e5d8fcff8014060a847cb4ae0738057 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:07.826819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b58a1-2ef6-4a60-9c5e-adcdb130751a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:07.828890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-124b58a1-2ef6-4a60-9c5e-adcdb130751a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:07.925453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b58a1-2ef6-4a60-9c5e-adcdb130751a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:07.926979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d2d3d034-95a7-42c4-a8e7-677596fd6d77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15e47e30726f4510b16bbf4fafeb5acf', 'request_id': 'req-d2d3d034-95a7-42c4-a8e7-677596fd6d77'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:07.943721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d2d3d034-95a7-42c4-a8e7-677596fd6d77 admin admin] The token cache is being invalidate because project 15e47e30726f4510b16bbf4fafeb5acf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:07.944090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d2d3d034-95a7-42c4-a8e7-677596fd6d77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:07.944931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d2d3d034-95a7-42c4-a8e7-677596fd6d77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:07.947666 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2466/4911] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:07 2022] DELETE /identity/v3/projects/15e47e30726f4510b16bbf4fafeb5acf => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:07.951689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3d034-95a7-42c4-a8e7-677596fd6d77 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:07.955132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d2d3d034-95a7-42c4-a8e7-677596fd6d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:08.026740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b66bb-b501-4e51-9100-ab211393b8e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:08.028593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b66bb-b501-4e51-9100-ab211393b8e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:08.029996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b66bb-b501-4e51-9100-ab211393b8e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:08.030695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b66bb-b501-4e51-9100-ab211393b8e1 admin admin] PATH_INFO: `/v3/users/cf566d5f67964b21bc35934a74671fea` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:08.033622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b66bb-b501-4e51-9100-ab211393b8e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf566d5f67964b21bc35934a74671fea)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:08.055541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29b66bb-b501-4e51-9100-ab211393b8e1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:08.083763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3d034-95a7-42c4-a8e7-677596fd6d77 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:08.088835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9c78e-89d2-4dc7-84ff-50ca1b2de330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:08.093005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9c78e-89d2-4dc7-84ff-50ca1b2de330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:08.093295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9c78e-89d2-4dc7-84ff-50ca1b2de330 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:08.093776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9c78e-89d2-4dc7-84ff-50ca1b2de330 admin admin] PATH_INFO: `/v3/users/849fd7c01fa34d25aedf00a7842e1344` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:08.095882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9c78e-89d2-4dc7-84ff-50ca1b2de330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=849fd7c01fa34d25aedf00a7842e1344)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:08.099526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9c78e-89d2-4dc7-84ff-50ca1b2de330 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:08.275200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b29b66bb-b501-4e51-9100-ab211393b8e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf566d5f67964b21bc35934a74671fea'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:08.275905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b29b66bb-b501-4e51-9100-ab211393b8e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf566d5f67964b21bc35934a74671fea'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:08.285401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b29b66bb-b501-4e51-9100-ab211393b8e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf566d5f67964b21bc35934a74671fea'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:08.324285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-65e9c78e-89d2-4dc7-84ff-50ca1b2de330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '849fd7c01fa34d25aedf00a7842e1344'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:08.340218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-65e9c78e-89d2-4dc7-84ff-50ca1b2de330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '849fd7c01fa34d25aedf00a7842e1344'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:08.340218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-65e9c78e-89d2-4dc7-84ff-50ca1b2de330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '849fd7c01fa34d25aedf00a7842e1344'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:08.340627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-65e9c78e-89d2-4dc7-84ff-50ca1b2de330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '849fd7c01fa34d25aedf00a7842e1344'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:08.378279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b29b66bb-b501-4e51-9100-ab211393b8e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf566d5f67964b21bc35934a74671fea'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:08.378936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b29b66bb-b501-4e51-9100-ab211393b8e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf566d5f67964b21bc35934a74671fea'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:08.398429 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2446/4912] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:07 2022] DELETE /identity/v3/users/cf566d5f67964b21bc35934a74671fea => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:08.436344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-65e9c78e-89d2-4dc7-84ff-50ca1b2de330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '849fd7c01fa34d25aedf00a7842e1344'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:08.469626 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2467/4913] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:07 2022] DELETE /identity/v3/users/849fd7c01fa34d25aedf00a7842e1344 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:08.528339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:09.363622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:09.382350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b66bb-b501-4e51-9100-ab211393b8e1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:09.382350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b29b66bb-b501-4e51-9100-ab211393b8e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:09.398037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9c78e-89d2-4dc7-84ff-50ca1b2de330 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:09.400524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-65e9c78e-89d2-4dc7-84ff-50ca1b2de330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:09.517052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9c78e-89d2-4dc7-84ff-50ca1b2de330 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:09.525700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b66bb-b501-4e51-9100-ab211393b8e1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:09.528791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a038147-3828-4452-a2b2-066c903224d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:09.529681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a038147-3828-4452-a2b2-066c903224d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:09.530085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a038147-3828-4452-a2b2-066c903224d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:09.530414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a038147-3828-4452-a2b2-066c903224d5 admin admin] PATH_INFO: `/v3/projects/6e8ed7a1f60a4a608afabf8ab5e3c4cd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:09.531899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a038147-3828-4452-a2b2-066c903224d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e8ed7a1f60a4a608afabf8ab5e3c4cd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:09.534416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a038147-3828-4452-a2b2-066c903224d5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:09.538740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5da711-e584-4b7a-a159-ec6b62db05dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:09.539866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5da711-e584-4b7a-a159-ec6b62db05dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:09.540331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5da711-e584-4b7a-a159-ec6b62db05dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:09.540793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5da711-e584-4b7a-a159-ec6b62db05dc admin admin] PATH_INFO: `/v3/projects/89ca1e2b61c04e7ba89c61d7cf5f5d5b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:09.542225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5da711-e584-4b7a-a159-ec6b62db05dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ca1e2b61c04e7ba89c61d7cf5f5d5b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:09.544732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5da711-e584-4b7a-a159-ec6b62db05dc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:09.686890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1a038147-3828-4452-a2b2-066c903224d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e8ed7a1f60a4a608afabf8ab5e3c4cd', 'request_id': 'req-1a038147-3828-4452-a2b2-066c903224d5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:09.696781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1a038147-3828-4452-a2b2-066c903224d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e8ed7a1f60a4a608afabf8ab5e3c4cd', 'request_id': 'req-1a038147-3828-4452-a2b2-066c903224d5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:09.733819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6f5da711-e584-4b7a-a159-ec6b62db05dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ca1e2b61c04e7ba89c61d7cf5f5d5b', 'request_id': 'req-6f5da711-e584-4b7a-a159-ec6b62db05dc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:09.740420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6f5da711-e584-4b7a-a159-ec6b62db05dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ca1e2b61c04e7ba89c61d7cf5f5d5b', 'request_id': 'req-6f5da711-e584-4b7a-a159-ec6b62db05dc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:09.829256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1a038147-3828-4452-a2b2-066c903224d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e8ed7a1f60a4a608afabf8ab5e3c4cd', 'request_id': 'req-1a038147-3828-4452-a2b2-066c903224d5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:09.842001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1a038147-3828-4452-a2b2-066c903224d5 admin admin] The token cache is being invalidate because project 6e8ed7a1f60a4a608afabf8ab5e3c4cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:09.842304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1a038147-3828-4452-a2b2-066c903224d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:09.842980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1a038147-3828-4452-a2b2-066c903224d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:09.844544 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2468/4914] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:09 2022] DELETE /identity/v3/projects/6e8ed7a1f60a4a608afabf8ab5e3c4cd => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:09.845622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a038147-3828-4452-a2b2-066c903224d5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:09.848057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1a038147-3828-4452-a2b2-066c903224d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:09.915987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6f5da711-e584-4b7a-a159-ec6b62db05dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ca1e2b61c04e7ba89c61d7cf5f5d5b', 'request_id': 'req-6f5da711-e584-4b7a-a159-ec6b62db05dc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:09.916747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a038147-3828-4452-a2b2-066c903224d5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:09.927801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6f5da711-e584-4b7a-a159-ec6b62db05dc admin admin] The token cache is being invalidate because project 89ca1e2b61c04e7ba89c61d7cf5f5d5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:09.928097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6f5da711-e584-4b7a-a159-ec6b62db05dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:09.928753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6f5da711-e584-4b7a-a159-ec6b62db05dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:09.930352 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2447/4915] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:09 2022] DELETE /identity/v3/projects/89ca1e2b61c04e7ba89c61d7cf5f5d5b => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:09.931892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574107a-6a11-4b07-a7f2-a073772dc62a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:09.932172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574107a-6a11-4b07-a7f2-a073772dc62a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:09.932453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574107a-6a11-4b07-a7f2-a073772dc62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:09.950551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d574107a-6a11-4b07-a7f2-a073772dc62a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:09.952073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d574107a-6a11-4b07-a7f2-a073772dc62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:09.985049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580e86c-6030-4b42-8993-3337703c816b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:09.986019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580e86c-6030-4b42-8993-3337703c816b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:09.986334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580e86c-6030-4b42-8993-3337703c816b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:09.986620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6580e86c-6030-4b42-8993-3337703c816b admin admin] PATH_INFO: `/v3/users/af446b3d802a42e3bd72b08bace8ea8d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:09.987909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580e86c-6030-4b42-8993-3337703c816b admin admin] RBAC: Authorizing `identity:delete_user(user_id=af446b3d802a42e3bd72b08bace8ea8d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:09.990030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6580e86c-6030-4b42-8993-3337703c816b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:10.019629 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2448/4916] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:10.020959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae744e3-8fde-4151-9a75-6e92f6943546 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:10.021258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae744e3-8fde-4151-9a75-6e92f6943546 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:10.021543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae744e3-8fde-4151-9a75-6e92f6943546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:10.042008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-dae744e3-8fde-4151-9a75-6e92f6943546 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:10.043671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-dae744e3-8fde-4151-9a75-6e92f6943546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:10.052192 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2449/4917] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:10.053769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929be8f-f6e3-4522-bf87-0acda1b2aba6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:10.054097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929be8f-f6e3-4522-bf87-0acda1b2aba6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:10.054383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929be8f-f6e3-4522-bf87-0acda1b2aba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:10.084168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3929be8f-f6e3-4522-bf87-0acda1b2aba6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:10.085870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3929be8f-f6e3-4522-bf87-0acda1b2aba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:10.100681 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2450/4918] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:10.100681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78577fca-3765-4c5d-915c-f49163a0885d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:10.100681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78577fca-3765-4c5d-915c-f49163a0885d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:10.100681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78577fca-3765-4c5d-915c-f49163a0885d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:10.118065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-78577fca-3765-4c5d-915c-f49163a0885d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:10.119382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-78577fca-3765-4c5d-915c-f49163a0885d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:10.128036 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2451/4919] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:10.129095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78577fca-3765-4c5d-915c-f49163a0885d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:10.140892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78577fca-3765-4c5d-915c-f49163a0885d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:10.149869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2acc5-9f05-4087-9277-a56d15b6ef49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:10.150603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2acc5-9f05-4087-9277-a56d15b6ef49 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:10.150899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2acc5-9f05-4087-9277-a56d15b6ef49 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:10.151305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2acc5-9f05-4087-9277-a56d15b6ef49 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:10.152124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2acc5-9f05-4087-9277-a56d15b6ef49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:10.152463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2acc5-9f05-4087-9277-a56d15b6ef49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:10.165929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2acc5-9f05-4087-9277-a56d15b6ef49 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:10.187329 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2452/4920] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:10.192135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6580e86c-6030-4b42-8993-3337703c816b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af446b3d802a42e3bd72b08bace8ea8d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:10.192284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2acc5-9f05-4087-9277-a56d15b6ef49 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:10.200709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6580e86c-6030-4b42-8993-3337703c816b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af446b3d802a42e3bd72b08bace8ea8d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:10.201386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6580e86c-6030-4b42-8993-3337703c816b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af446b3d802a42e3bd72b08bace8ea8d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:10.202035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6580e86c-6030-4b42-8993-3337703c816b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af446b3d802a42e3bd72b08bace8ea8d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:10.213018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2acc5-9f05-4087-9277-a56d15b6ef49 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:10.220223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25217c16-8b85-4c24-a791-50b0fb1c235b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:10.221467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25217c16-8b85-4c24-a791-50b0fb1c235b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:10.223033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25217c16-8b85-4c24-a791-50b0fb1c235b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:10.223556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25217c16-8b85-4c24-a791-50b0fb1c235b admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:10.224516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25217c16-8b85-4c24-a791-50b0fb1c235b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:10.225156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25217c16-8b85-4c24-a791-50b0fb1c235b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:10.227027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25217c16-8b85-4c24-a791-50b0fb1c235b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:10.242127 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2453/4921] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:10.243770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79c50e-f7f5-4a97-9f02-abdb5c4fabe0 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:10.244119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79c50e-f7f5-4a97-9f02-abdb5c4fabe0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:10.244465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db79c50e-f7f5-4a97-9f02-abdb5c4fabe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:10.291664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-db79c50e-f7f5-4a97-9f02-abdb5c4fabe0 None None] MFA Rules not processed for user `e567fd5a2683491281fab7551a485a4d`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:10.294549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-db79c50e-f7f5-4a97-9f02-abdb5c4fabe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:10.304308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6580e86c-6030-4b42-8993-3337703c816b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af446b3d802a42e3bd72b08bace8ea8d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:10.329041 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2469/4922] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:09 2022] DELETE /identity/v3/users/af446b3d802a42e3bd72b08bace8ea8d => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:10.330649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580e86c-6030-4b42-8993-3337703c816b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:10.332116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6580e86c-6030-4b42-8993-3337703c816b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:10.400846 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2454/4923] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:10.403693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79c50e-f7f5-4a97-9f02-abdb5c4fabe0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:10.405538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-db79c50e-f7f5-4a97-9f02-abdb5c4fabe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:10.426245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db79c50e-f7f5-4a97-9f02-abdb5c4fabe0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:10.435585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c3fc3-3c07-422e-99ff-1245dff551d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:10.435585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c3fc3-3c07-422e-99ff-1245dff551d4 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:10.435585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c3fc3-3c07-422e-99ff-1245dff551d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:10.437101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6580e86c-6030-4b42-8993-3337703c816b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:10.437259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c3fc3-3c07-422e-99ff-1245dff551d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:10.438196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c3fc3-3c07-422e-99ff-1245dff551d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:10.441139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c3fc3-3c07-422e-99ff-1245dff551d4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:10.443285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba97fb4-0c4c-40b0-b49b-5541761aadf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:10.444082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba97fb4-0c4c-40b0-b49b-5541761aadf0 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:10.444454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba97fb4-0c4c-40b0-b49b-5541761aadf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:10.444847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba97fb4-0c4c-40b0-b49b-5541761aadf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:10.445671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba97fb4-0c4c-40b0-b49b-5541761aadf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:10.448585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba97fb4-0c4c-40b0-b49b-5541761aadf0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:10.586234 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2455/4924] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:10 2022] POST /identity/v3/projects => generated 369 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:10.586234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a3d50-601d-4cec-a6bd-4f424a225a64 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:10.586234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a3d50-601d-4cec-a6bd-4f424a225a64 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:10.586234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a3d50-601d-4cec-a6bd-4f424a225a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:10.592926 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2470/4925] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:10 2022] POST /identity/v3/projects => generated 379 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:10.594707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba97fb4-0c4c-40b0-b49b-5541761aadf0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:10.625341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-485a3d50-601d-4cec-a6bd-4f424a225a64 None None] MFA Rules not processed for user `e567fd5a2683491281fab7551a485a4d`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:10.628532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-485a3d50-601d-4cec-a6bd-4f424a225a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:10.688431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba97fb4-0c4c-40b0-b49b-5541761aadf0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:10.694287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276451e-781d-42ce-bbd1-e5f8e3c7a3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:10.695046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276451e-781d-42ce-bbd1-e5f8e3c7a3aa admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:10.695406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276451e-781d-42ce-bbd1-e5f8e3c7a3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:10.695751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d276451e-781d-42ce-bbd1-e5f8e3c7a3aa admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:10.696764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276451e-781d-42ce-bbd1-e5f8e3c7a3aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:10.699074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d276451e-781d-42ce-bbd1-e5f8e3c7a3aa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:10.699230 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2456/4926] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:10.700424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a3d50-601d-4cec-a6bd-4f424a225a64 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:10.712754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a3d50-601d-4cec-a6bd-4f424a225a64 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:10.719006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070c67b-d066-4b91-8e1e-a603e28612ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:10.719693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070c67b-d066-4b91-8e1e-a603e28612ea admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:10.720187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070c67b-d066-4b91-8e1e-a603e28612ea admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:10.720559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070c67b-d066-4b91-8e1e-a603e28612ea admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:10.721268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070c67b-d066-4b91-8e1e-a603e28612ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:10.723556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070c67b-d066-4b91-8e1e-a603e28612ea admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:10.813025 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2471/4927] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:10 2022] POST /identity/v3/users => generated 285 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:10.814492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276451e-781d-42ce-bbd1-e5f8e3c7a3aa admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:10.816061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d276451e-781d-42ce-bbd1-e5f8e3c7a3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:10.845826 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2457/4928] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:10 2022] POST /identity/v3/users => generated 290 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:10.846622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070c67b-d066-4b91-8e1e-a603e28612ea admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:10.851069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4070c67b-d066-4b91-8e1e-a603e28612ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:10.872448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070c67b-d066-4b91-8e1e-a603e28612ea admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:10.884880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:10.886078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:10.887110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:10.887636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7 admin admin] PATH_INFO: `/v3/projects/53b9cfacdbb74a058e52bed331a6f241` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:10.889150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53b9cfacdbb74a058e52bed331a6f241)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:10.892475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:10.927072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d276451e-781d-42ce-bbd1-e5f8e3c7a3aa admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:10.940353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18340ca8-fd7d-44be-9be9-5faf34934260 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:10.943159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18340ca8-fd7d-44be-9be9-5faf34934260 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:10.943159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18340ca8-fd7d-44be-9be9-5faf34934260 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:10.943159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18340ca8-fd7d-44be-9be9-5faf34934260 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:10.951995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18340ca8-fd7d-44be-9be9-5faf34934260 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:10.953764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18340ca8-fd7d-44be-9be9-5faf34934260 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:10.966710 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2472/4929] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:10.973735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18340ca8-fd7d-44be-9be9-5faf34934260 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:10.989460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18340ca8-fd7d-44be-9be9-5faf34934260 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:10.995396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8f1c3-a973-462b-8ae9-5a650fd59075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:10.996066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8f1c3-a973-462b-8ae9-5a650fd59075 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:10.996349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8f1c3-a973-462b-8ae9-5a650fd59075 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:10.996628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8f1c3-a973-462b-8ae9-5a650fd59075 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:10.997367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8f1c3-a973-462b-8ae9-5a650fd59075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:10.998997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8f1c3-a973-462b-8ae9-5a650fd59075 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:11.022781 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2473/4930] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:11.024957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8f1c3-a973-462b-8ae9-5a650fd59075 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:11.033318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53b9cfacdbb74a058e52bed331a6f241', 'request_id': 'req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:11.042407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53b9cfacdbb74a058e52bed331a6f241', 'request_id': 'req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:11.052073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8f1c3-a973-462b-8ae9-5a650fd59075 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:11.079693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:11.125788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3385a-9ba0-4f65-9863-ad16cfb73272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:11.126573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3385a-9ba0-4f65-9863-ad16cfb73272 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:11.126856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3385a-9ba0-4f65-9863-ad16cfb73272 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:11.127661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3385a-9ba0-4f65-9863-ad16cfb73272 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:11.128014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3385a-9ba0-4f65-9863-ad16cfb73272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:11.129772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3385a-9ba0-4f65-9863-ad16cfb73272 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:11.137300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53b9cfacdbb74a058e52bed331a6f241', 'request_id': 'req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:11.143538 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2474/4931] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:11.144551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3385a-9ba0-4f65-9863-ad16cfb73272 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:11.150835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7 admin admin] The token cache is being invalidate because project 53b9cfacdbb74a058e52bed331a6f241 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:11.151268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:11.152097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:11.154187 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2458/4932] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:10 2022] DELETE /identity/v3/projects/53b9cfacdbb74a058e52bed331a6f241 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:11.154961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:11.156298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:11.158452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3385a-9ba0-4f65-9863-ad16cfb73272 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:11.165730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9798c-6ee0-4778-b944-05a59f775381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:11.169903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9798c-6ee0-4778-b944-05a59f775381 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:11.169903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9798c-6ee0-4778-b944-05a59f775381 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:11.169903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9798c-6ee0-4778-b944-05a59f775381 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:11.169903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9798c-6ee0-4778-b944-05a59f775381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:11.170286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9798c-6ee0-4778-b944-05a59f775381 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:11.177805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b4732-66a9-42a5-a81e-da73ac6c96e7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:11.184729 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2475/4933] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:11.184729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b0e80-0e5e-4bd7-8f6d-1aad12065921 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:11.184729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b0e80-0e5e-4bd7-8f6d-1aad12065921 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:11.184729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974b0e80-0e5e-4bd7-8f6d-1aad12065921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:11.189648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dcadd-0542-484f-adcd-cf702b525cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:11.199710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dcadd-0542-484f-adcd-cf702b525cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:11.199710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dcadd-0542-484f-adcd-cf702b525cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:11.199710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dcadd-0542-484f-adcd-cf702b525cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:11.199710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dcadd-0542-484f-adcd-cf702b525cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:11.199710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dcadd-0542-484f-adcd-cf702b525cd7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:11.217977 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2459/4934] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:11.221391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dcadd-0542-484f-adcd-cf702b525cd7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:11.222915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c23dcadd-0542-484f-adcd-cf702b525cd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:11.231660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-974b0e80-0e5e-4bd7-8f6d-1aad12065921 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:11.234704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-974b0e80-0e5e-4bd7-8f6d-1aad12065921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:11.249037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dcadd-0542-484f-adcd-cf702b525cd7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:11.254379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4ab45-b69c-487c-b7ad-f3f4e539230b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:11.255491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4ab45-b69c-487c-b7ad-f3f4e539230b admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:11.255778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4ab45-b69c-487c-b7ad-f3f4e539230b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:11.256113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4ab45-b69c-487c-b7ad-f3f4e539230b admin admin] PATH_INFO: `/v3/projects/8e4641f49e584ec0ab8e21f2fb572cc0/users/1c340ea34bdd4a8d993f383d9feb496d/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:11.278573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4ab45-b69c-487c-b7ad-f3f4e539230b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e4641f49e584ec0ab8e21f2fb572cc0, user_id=1c340ea34bdd4a8d993f383d9feb496d, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:11.279943 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2476/4935] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:11.280669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b0e80-0e5e-4bd7-8f6d-1aad12065921 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:11.282325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4ab45-b69c-487c-b7ad-f3f4e539230b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:11.298828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974b0e80-0e5e-4bd7-8f6d-1aad12065921 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:11.304142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a88d8e-d6ce-4bb7-8b84-9bab8d038e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:11.305238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a88d8e-d6ce-4bb7-8b84-9bab8d038e14 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:11.305631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a88d8e-d6ce-4bb7-8b84-9bab8d038e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:11.306084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a88d8e-d6ce-4bb7-8b84-9bab8d038e14 admin admin] PATH_INFO: `/v3/projects/bef32a9958744263854bd63f7922cda9/users/843f3d747f604fe5845d4401cb570429/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:11.325925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a88d8e-d6ce-4bb7-8b84-9bab8d038e14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef32a9958744263854bd63f7922cda9, user_id=843f3d747f604fe5845d4401cb570429, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:11.330470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a88d8e-d6ce-4bb7-8b84-9bab8d038e14 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:11.419145 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2460/4936] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:11 2022] PUT /identity/v3/projects/8e4641f49e584ec0ab8e21f2fb572cc0/users/1c340ea34bdd4a8d993f383d9feb496d/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:11.421228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bc655-6966-473c-9999-f383b5263f44 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:11.421744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bc655-6966-473c-9999-f383b5263f44 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:11.422222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5bc655-6966-473c-9999-f383b5263f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:11.455472 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2477/4937] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:11 2022] PUT /identity/v3/projects/bef32a9958744263854bd63f7922cda9/users/843f3d747f604fe5845d4401cb570429/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:11.455581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4585c66-5f7f-4b8d-87df-3f58299b5b2e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:11.456008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4585c66-5f7f-4b8d-87df-3f58299b5b2e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:11.456205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4585c66-5f7f-4b8d-87df-3f58299b5b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:11.483501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-de5bc655-6966-473c-9999-f383b5263f44 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:11.487474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-de5bc655-6966-473c-9999-f383b5263f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:11.522839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c4585c66-5f7f-4b8d-87df-3f58299b5b2e None None] MFA Rules not processed for user `1c340ea34bdd4a8d993f383d9feb496d`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:11.524126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c4585c66-5f7f-4b8d-87df-3f58299b5b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:11.551345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:11.615237 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2461/4938] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:11.618701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930553c-204d-4bf8-9bdb-d0c25f401978 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:11.619927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930553c-204d-4bf8-9bdb-d0c25f401978 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:11.620213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9930553c-204d-4bf8-9bdb-d0c25f401978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:11.633639 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2478/4939] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:11.638765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4585c66-5f7f-4b8d-87df-3f58299b5b2e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:11.655628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4585c66-5f7f-4b8d-87df-3f58299b5b2e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:11.667555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb453e7-40e9-44b1-92e0-55575c288bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:11.668250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb453e7-40e9-44b1-92e0-55575c288bdb admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:11.668572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb453e7-40e9-44b1-92e0-55575c288bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:11.668818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb453e7-40e9-44b1-92e0-55575c288bdb admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:11.669660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb453e7-40e9-44b1-92e0-55575c288bdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:11.670008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb453e7-40e9-44b1-92e0-55575c288bdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:11.673272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9930553c-204d-4bf8-9bdb-d0c25f401978 None None] MFA Rules not processed for user `843f3d747f604fe5845d4401cb570429`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:11.673797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb453e7-40e9-44b1-92e0-55575c288bdb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:11.675268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9930553c-204d-4bf8-9bdb-d0c25f401978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:11.698261 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2479/4940] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:11.698261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb453e7-40e9-44b1-92e0-55575c288bdb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:11.710135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb453e7-40e9-44b1-92e0-55575c288bdb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:11.715488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4766da-6168-4333-8238-0636dd8a6447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:11.716702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4766da-6168-4333-8238-0636dd8a6447 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:11.716702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4766da-6168-4333-8238-0636dd8a6447 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:11.717068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4766da-6168-4333-8238-0636dd8a6447 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:11.717987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4766da-6168-4333-8238-0636dd8a6447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:11.719977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4766da-6168-4333-8238-0636dd8a6447 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:11.773128 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2462/4941] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:11.774204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930553c-204d-4bf8-9bdb-d0c25f401978 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:11.787789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9930553c-204d-4bf8-9bdb-d0c25f401978 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:11.793455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cf125-0ac6-47d5-b62d-e8925229532c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:11.794443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cf125-0ac6-47d5-b62d-e8925229532c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:11.794832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cf125-0ac6-47d5-b62d-e8925229532c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:11.795379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cf125-0ac6-47d5-b62d-e8925229532c admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:11.796206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941cf125-0ac6-47d5-b62d-e8925229532c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:11.798370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941cf125-0ac6-47d5-b62d-e8925229532c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:11.838629 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2480/4942] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:11 2022] POST /identity/v3/projects => generated 369 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:11.839474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4766da-6168-4333-8238-0636dd8a6447 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:11.907743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4766da-6168-4333-8238-0636dd8a6447 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:11.912784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffe857-36ee-488a-a8cc-865a8e45cad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:11.913454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffe857-36ee-488a-a8cc-865a8e45cad3 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:11.913769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffe857-36ee-488a-a8cc-865a8e45cad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:11.914141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffe857-36ee-488a-a8cc-865a8e45cad3 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:11.914927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffe857-36ee-488a-a8cc-865a8e45cad3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:11.916952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ffe857-36ee-488a-a8cc-865a8e45cad3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:11.937493 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2463/4943] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:11 2022] POST /identity/v3/projects => generated 363 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:11.938430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cf125-0ac6-47d5-b62d-e8925229532c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:12.019901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cf125-0ac6-47d5-b62d-e8925229532c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:12.025960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb504c-049e-4008-993a-29b2546a4d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:12.026802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb504c-049e-4008-993a-29b2546a4d5c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:12.027098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb504c-049e-4008-993a-29b2546a4d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:12.027383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb504c-049e-4008-993a-29b2546a4d5c admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:12.028329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb504c-049e-4008-993a-29b2546a4d5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:12.030510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fb504c-049e-4008-993a-29b2546a4d5c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:12.064711 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2481/4944] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:11 2022] POST /identity/v3/projects => generated 379 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:12.065687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffe857-36ee-488a-a8cc-865a8e45cad3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:12.167681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffe857-36ee-488a-a8cc-865a8e45cad3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:12.175609 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2464/4945] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:11 2022] POST /identity/v3/users => generated 291 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:12.176556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb504c-049e-4008-993a-29b2546a4d5c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:12.179100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d0c7c-2521-4ff8-ae74-ebeee12fb954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:12.179804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d0c7c-2521-4ff8-ae74-ebeee12fb954 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:12.180095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d0c7c-2521-4ff8-ae74-ebeee12fb954 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:12.180383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d0c7c-2521-4ff8-ae74-ebeee12fb954 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:12.181122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d0c7c-2521-4ff8-ae74-ebeee12fb954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:12.183194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d0c7c-2521-4ff8-ae74-ebeee12fb954 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:12.190555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb504c-049e-4008-993a-29b2546a4d5c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:12.197986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28f535-f5cf-488d-a15f-123be6787c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:12.198680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28f535-f5cf-488d-a15f-123be6787c8a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:12.198966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28f535-f5cf-488d-a15f-123be6787c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:12.199245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28f535-f5cf-488d-a15f-123be6787c8a admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:12.199945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28f535-f5cf-488d-a15f-123be6787c8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:12.202868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28f535-f5cf-488d-a15f-123be6787c8a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:12.310857 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2482/4946] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:12 2022] POST /identity/v3/users => generated 282 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:12.311831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d0c7c-2521-4ff8-ae74-ebeee12fb954 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:12.324678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d0c7c-2521-4ff8-ae74-ebeee12fb954 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:12.329941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c586e1-bc6c-4ccd-8c84-175a241233f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:12.330634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c586e1-bc6c-4ccd-8c84-175a241233f7 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:12.331148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c586e1-bc6c-4ccd-8c84-175a241233f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:12.331297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c586e1-bc6c-4ccd-8c84-175a241233f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:12.332021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c586e1-bc6c-4ccd-8c84-175a241233f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:12.333666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c586e1-bc6c-4ccd-8c84-175a241233f7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:12.361723 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2483/4947] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:12.362570 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2465/4948] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:12 2022] POST /identity/v3/users => generated 296 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:12.363887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28f535-f5cf-488d-a15f-123be6787c8a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:12.365057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c586e1-bc6c-4ccd-8c84-175a241233f7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:12.378528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28f535-f5cf-488d-a15f-123be6787c8a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:12.386645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1afd21-1d06-4ce5-bcbd-e2989afc809b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:12.388300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1afd21-1d06-4ce5-bcbd-e2989afc809b admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:12.388702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1afd21-1d06-4ce5-bcbd-e2989afc809b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:12.389053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1afd21-1d06-4ce5-bcbd-e2989afc809b admin admin] PATH_INFO: `/v3/projects/6c40d2ad8d084a3d832e3de6ea7aff74/users/13920683869e4a35970c5d1f380f3060/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:12.389828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c586e1-bc6c-4ccd-8c84-175a241233f7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:12.403467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6b02a-0577-492f-b527-0facf20d3e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:12.404359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6b02a-0577-492f-b527-0facf20d3e97 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:12.404496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6b02a-0577-492f-b527-0facf20d3e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:12.404729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6b02a-0577-492f-b527-0facf20d3e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:12.405572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd6b02a-0577-492f-b527-0facf20d3e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:12.410576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd6b02a-0577-492f-b527-0facf20d3e97 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:12.416260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1afd21-1d06-4ce5-bcbd-e2989afc809b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c40d2ad8d084a3d832e3de6ea7aff74, user_id=13920683869e4a35970c5d1f380f3060, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:12.421171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1afd21-1d06-4ce5-bcbd-e2989afc809b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:12.430192 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2484/4949] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:12.436526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6b02a-0577-492f-b527-0facf20d3e97 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:12.448723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6b02a-0577-492f-b527-0facf20d3e97 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:12.456155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b548d-ee9a-42e7-83d9-f71759ea5f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:12.456948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b548d-ee9a-42e7-83d9-f71759ea5f1c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:12.457309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b548d-ee9a-42e7-83d9-f71759ea5f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:12.457791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b548d-ee9a-42e7-83d9-f71759ea5f1c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:12.458706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b548d-ee9a-42e7-83d9-f71759ea5f1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:12.460900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b548d-ee9a-42e7-83d9-f71759ea5f1c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:12.474293 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2485/4950] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:12.475404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b548d-ee9a-42e7-83d9-f71759ea5f1c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:12.499233 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2466/4951] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:12 2022] PUT /identity/v3/projects/6c40d2ad8d084a3d832e3de6ea7aff74/users/13920683869e4a35970c5d1f380f3060/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:12.502877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1afd21-1d06-4ce5-bcbd-e2989afc809b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:12.503054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b548d-ee9a-42e7-83d9-f71759ea5f1c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:12.581969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1afd21-1d06-4ce5-bcbd-e2989afc809b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:12.592137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c2dcf-68ee-4c4a-b799-aae65abd6e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:12.592137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c2dcf-68ee-4c4a-b799-aae65abd6e57 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:12.592137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c2dcf-68ee-4c4a-b799-aae65abd6e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:12.592137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c2dcf-68ee-4c4a-b799-aae65abd6e57 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:12.593453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c2dcf-68ee-4c4a-b799-aae65abd6e57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:12.593453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48c2dcf-68ee-4c4a-b799-aae65abd6e57 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:12.596047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb2e8e-8e8f-48aa-9818-1d040ca33cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:12.598106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb2e8e-8e8f-48aa-9818-1d040ca33cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:12.598803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb2e8e-8e8f-48aa-9818-1d040ca33cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:12.599572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eb2e8e-8e8f-48aa-9818-1d040ca33cb2 admin admin] PATH_INFO: `/v3/projects/65630772e5d64ab190615b189537d59c/users/19a8ec3aee1842a78442c916aa8d5b2b/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:12.620654 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2486/4952] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:12.620654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c2dcf-68ee-4c4a-b799-aae65abd6e57 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:12.623804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eb2e8e-8e8f-48aa-9818-1d040ca33cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65630772e5d64ab190615b189537d59c, user_id=19a8ec3aee1842a78442c916aa8d5b2b, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:12.626826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eb2e8e-8e8f-48aa-9818-1d040ca33cb2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:12.631647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c2dcf-68ee-4c4a-b799-aae65abd6e57 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:12.661141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7e8c9-21c8-4d0e-8ce9-fe298b90288b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:12.662577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7e8c9-21c8-4d0e-8ce9-fe298b90288b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:12.666394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7e8c9-21c8-4d0e-8ce9-fe298b90288b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:12.666394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca7e8c9-21c8-4d0e-8ce9-fe298b90288b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:12.666394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7e8c9-21c8-4d0e-8ce9-fe298b90288b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:12.667217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca7e8c9-21c8-4d0e-8ce9-fe298b90288b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:12.687202 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2487/4953] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:12.687202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7e8c9-21c8-4d0e-8ce9-fe298b90288b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:12.706435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca7e8c9-21c8-4d0e-8ce9-fe298b90288b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:12.728821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a60c4-b404-468b-9225-8b65dd30ab01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:12.732750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a60c4-b404-468b-9225-8b65dd30ab01 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:12.733077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a60c4-b404-468b-9225-8b65dd30ab01 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:12.733375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a60c4-b404-468b-9225-8b65dd30ab01 admin admin] PATH_INFO: `/v3/projects/9ea17476e95b4b58826749c0c6082ba1/users/1fd40c9e6adc42d68e5b6b9265325bef/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:12.749526 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2467/4954] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:12 2022] PUT /identity/v3/projects/65630772e5d64ab190615b189537d59c/users/19a8ec3aee1842a78442c916aa8d5b2b/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:12.755926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb2e8e-8e8f-48aa-9818-1d040ca33cb2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:12.777133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a60c4-b404-468b-9225-8b65dd30ab01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ea17476e95b4b58826749c0c6082ba1, user_id=1fd40c9e6adc42d68e5b6b9265325bef, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:12.784240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a60c4-b404-468b-9225-8b65dd30ab01 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:12.886180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eb2e8e-8e8f-48aa-9818-1d040ca33cb2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:12.893001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79397f46-3bc5-494e-9440-dca809c144b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:12.894410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79397f46-3bc5-494e-9440-dca809c144b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:12.894889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79397f46-3bc5-494e-9440-dca809c144b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:12.895339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79397f46-3bc5-494e-9440-dca809c144b6 admin admin] PATH_INFO: `/v3/projects/6c40d2ad8d084a3d832e3de6ea7aff74/users/13920683869e4a35970c5d1f380f3060/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:12.897774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79397f46-3bc5-494e-9440-dca809c144b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c40d2ad8d084a3d832e3de6ea7aff74, user_id=13920683869e4a35970c5d1f380f3060, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:12.900658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79397f46-3bc5-494e-9440-dca809c144b6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:12.907414 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2488/4955] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:12 2022] PUT /identity/v3/projects/9ea17476e95b4b58826749c0c6082ba1/users/1fd40c9e6adc42d68e5b6b9265325bef/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:12.909523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a60c4-b404-468b-9225-8b65dd30ab01 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:12.926941 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2468/4956] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:12 2022] PUT /identity/v3/projects/6c40d2ad8d084a3d832e3de6ea7aff74/users/13920683869e4a35970c5d1f380f3060/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:12.928283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eddadb-0f73-46f4-9795-0a7e4b989b68 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:12.928589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eddadb-0f73-46f4-9795-0a7e4b989b68 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:12.928879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eddadb-0f73-46f4-9795-0a7e4b989b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:13.000453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e0eddadb-0f73-46f4-9795-0a7e4b989b68 None None] MFA Rules not processed for user `1fd40c9e6adc42d68e5b6b9265325bef`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:13.001777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e0eddadb-0f73-46f4-9795-0a7e4b989b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:13.083016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a60c4-b404-468b-9225-8b65dd30ab01 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:13.172477 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2469/4957] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:12 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:13.174018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817c874-2009-49d2-bc72-0adb34a463fa None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:13.174983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817c874-2009-49d2-bc72-0adb34a463fa None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:13.174983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817c874-2009-49d2-bc72-0adb34a463fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:13.185149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17b44d-7945-478c-a3b7-54fc31c30191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:13.189290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17b44d-7945-478c-a3b7-54fc31c30191 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:13.189290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17b44d-7945-478c-a3b7-54fc31c30191 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:13.189290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17b44d-7945-478c-a3b7-54fc31c30191 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:13.191406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17b44d-7945-478c-a3b7-54fc31c30191 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:13.193110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17b44d-7945-478c-a3b7-54fc31c30191 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:13.231475 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2489/4958] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:13.239479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17b44d-7945-478c-a3b7-54fc31c30191 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:13.244837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9817c874-2009-49d2-bc72-0adb34a463fa None None] MFA Rules not processed for user `13920683869e4a35970c5d1f380f3060`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:13.247376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9817c874-2009-49d2-bc72-0adb34a463fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:13.261852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17b44d-7945-478c-a3b7-54fc31c30191 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:13.267496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc9ce4-6614-45a7-bab6-512a2b3262ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:13.268945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc9ce4-6614-45a7-bab6-512a2b3262ff admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:13.269467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc9ce4-6614-45a7-bab6-512a2b3262ff admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:13.270023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc9ce4-6614-45a7-bab6-512a2b3262ff admin admin] PATH_INFO: `/v3/projects/65630772e5d64ab190615b189537d59c/users/19a8ec3aee1842a78442c916aa8d5b2b/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:13.273463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc9ce4-6614-45a7-bab6-512a2b3262ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=65630772e5d64ab190615b189537d59c, user_id=19a8ec3aee1842a78442c916aa8d5b2b, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:13.276730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc9ce4-6614-45a7-bab6-512a2b3262ff admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:13.298815 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2490/4959] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:13 2022] PUT /identity/v3/projects/65630772e5d64ab190615b189537d59c/users/19a8ec3aee1842a78442c916aa8d5b2b/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:13.299906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc9ce4-6614-45a7-bab6-512a2b3262ff admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:13.301730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a8fc9ce4-6614-45a7-bab6-512a2b3262ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:13.417230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc9ce4-6614-45a7-bab6-512a2b3262ff admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:13.424454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb3c3e-8cbd-472f-8bb3-9eeef0a3273a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:13.425197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb3c3e-8cbd-472f-8bb3-9eeef0a3273a service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:13.425484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb3c3e-8cbd-472f-8bb3-9eeef0a3273a service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:13.425764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfb3c3e-8cbd-472f-8bb3-9eeef0a3273a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:13.451389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:13.465897 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2470/4960] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:13 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:13.465897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af52b0-1e5f-4859-a8c1-f470261f7528 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:13.465897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af52b0-1e5f-4859-a8c1-f470261f7528 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:13.465897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af52b0-1e5f-4859-a8c1-f470261f7528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:13.496668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb3c3e-8cbd-472f-8bb3-9eeef0a3273a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:13.498677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfb3c3e-8cbd-472f-8bb3-9eeef0a3273a service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:13.503394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b0af52b0-1e5f-4859-a8c1-f470261f7528 None None] MFA Rules not processed for user `19a8ec3aee1842a78442c916aa8d5b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:13.506621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b0af52b0-1e5f-4859-a8c1-f470261f7528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:13.515215 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2491/4961] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:13 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:13.515215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb3c3e-8cbd-472f-8bb3-9eeef0a3273a service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:13.515472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4cfb3c3e-8cbd-472f-8bb3-9eeef0a3273a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:13.609781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfb3c3e-8cbd-472f-8bb3-9eeef0a3273a service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:13.613958 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2471/4962] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:13 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:13.613958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af52b0-1e5f-4859-a8c1-f470261f7528 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:13.620848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c932496-905f-477a-bea4-912f1f8e0674 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:13.621534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c932496-905f-477a-bea4-912f1f8e0674 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:13.621809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c932496-905f-477a-bea4-912f1f8e0674 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:13.622126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c932496-905f-477a-bea4-912f1f8e0674 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:13.710247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af52b0-1e5f-4859-a8c1-f470261f7528 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:13.714034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c932496-905f-477a-bea4-912f1f8e0674 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:13.715635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c932496-905f-477a-bea4-912f1f8e0674 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:13.715758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e558eb3-98f0-440e-a430-c13eaf0b9a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:13.716914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e558eb3-98f0-440e-a430-c13eaf0b9a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:13.717207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e558eb3-98f0-440e-a430-c13eaf0b9a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:13.717639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e558eb3-98f0-440e-a430-c13eaf0b9a27 admin admin] PATH_INFO: `/v3/users/1fd40c9e6adc42d68e5b6b9265325bef` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:13.719084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e558eb3-98f0-440e-a430-c13eaf0b9a27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fd40c9e6adc42d68e5b6b9265325bef)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:13.721103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e558eb3-98f0-440e-a430-c13eaf0b9a27 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:13.732411 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2492/4963] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:13 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:13.738123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c932496-905f-477a-bea4-912f1f8e0674 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:13.738123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4c932496-905f-477a-bea4-912f1f8e0674 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:13.860380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c932496-905f-477a-bea4-912f1f8e0674 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:13.864703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebf244-126a-4f16-ab75-b311b6500186 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:13.865361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebf244-126a-4f16-ab75-b311b6500186 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:13.865644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebf244-126a-4f16-ab75-b311b6500186 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:13.865953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebf244-126a-4f16-ab75-b311b6500186 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:13.876786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebf244-126a-4f16-ab75-b311b6500186 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:13.878408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebf244-126a-4f16-ab75-b311b6500186 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:13.888605 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2493/4964] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:13 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:14.009897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2e558eb3-98f0-440e-a430-c13eaf0b9a27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fd40c9e6adc42d68e5b6b9265325bef'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:14.014338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2e558eb3-98f0-440e-a430-c13eaf0b9a27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fd40c9e6adc42d68e5b6b9265325bef'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:14.022787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2e558eb3-98f0-440e-a430-c13eaf0b9a27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fd40c9e6adc42d68e5b6b9265325bef'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:14.110367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2e558eb3-98f0-440e-a430-c13eaf0b9a27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fd40c9e6adc42d68e5b6b9265325bef'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:14.118149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2e558eb3-98f0-440e-a430-c13eaf0b9a27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fd40c9e6adc42d68e5b6b9265325bef'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:14.163469 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2472/4965] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:13 2022] DELETE /identity/v3/users/1fd40c9e6adc42d68e5b6b9265325bef => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:14.167813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c844a-2a51-4abb-96b4-7427d7c0b890 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:14.168122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c844a-2a51-4abb-96b4-7427d7c0b890 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:14.168460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c844a-2a51-4abb-96b4-7427d7c0b890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:14.172788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:14.189537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-010c844a-2a51-4abb-96b4-7427d7c0b890 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:14.191046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-010c844a-2a51-4abb-96b4-7427d7c0b890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:14.261982 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2494/4966] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:14.269909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e558eb3-98f0-440e-a430-c13eaf0b9a27 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:14.277105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2e558eb3-98f0-440e-a430-c13eaf0b9a27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:14.350984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e558eb3-98f0-440e-a430-c13eaf0b9a27 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:14.360229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93589e-31a8-4633-a515-6f69eda340c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:14.360978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93589e-31a8-4633-a515-6f69eda340c0 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:14.361271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93589e-31a8-4633-a515-6f69eda340c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:14.361579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93589e-31a8-4633-a515-6f69eda340c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:14.378468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93589e-31a8-4633-a515-6f69eda340c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:14.378778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93589e-31a8-4633-a515-6f69eda340c0 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:14.390841 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2473/4967] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:15.045507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c844a-2a51-4abb-96b4-7427d7c0b890 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:15.063663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-010c844a-2a51-4abb-96b4-7427d7c0b890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:15.097237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c844a-2a51-4abb-96b4-7427d7c0b890 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:15.104465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf90979-2fc8-4e4d-85f5-4694f85c3a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:15.106003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf90979-2fc8-4e4d-85f5-4694f85c3a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:15.106542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf90979-2fc8-4e4d-85f5-4694f85c3a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:15.106926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf90979-2fc8-4e4d-85f5-4694f85c3a5f admin admin] PATH_INFO: `/v3/projects/9ea17476e95b4b58826749c0c6082ba1` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:15.109257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf90979-2fc8-4e4d-85f5-4694f85c3a5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ea17476e95b4b58826749c0c6082ba1)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:15.112241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf90979-2fc8-4e4d-85f5-4694f85c3a5f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:15.291058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4cf90979-2fc8-4e4d-85f5-4694f85c3a5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ea17476e95b4b58826749c0c6082ba1', 'request_id': 'req-4cf90979-2fc8-4e4d-85f5-4694f85c3a5f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:15.302869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4cf90979-2fc8-4e4d-85f5-4694f85c3a5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ea17476e95b4b58826749c0c6082ba1', 'request_id': 'req-4cf90979-2fc8-4e4d-85f5-4694f85c3a5f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:15.363031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93589e-31a8-4633-a515-6f69eda340c0 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:15.364792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-dd93589e-31a8-4633-a515-6f69eda340c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:15.407344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4cf90979-2fc8-4e4d-85f5-4694f85c3a5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ea17476e95b4b58826749c0c6082ba1', 'request_id': 'req-4cf90979-2fc8-4e4d-85f5-4694f85c3a5f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:15.423727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4cf90979-2fc8-4e4d-85f5-4694f85c3a5f admin admin] The token cache is being invalidate because project 9ea17476e95b4b58826749c0c6082ba1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:15.424129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4cf90979-2fc8-4e4d-85f5-4694f85c3a5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:15.425008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4cf90979-2fc8-4e4d-85f5-4694f85c3a5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:15.427642 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2495/4968] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:15 2022] DELETE /identity/v3/projects/9ea17476e95b4b58826749c0c6082ba1 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:15.432909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ec111-d3be-4e81-865c-0dffe6d829a5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:15.433204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ec111-d3be-4e81-865c-0dffe6d829a5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:15.433783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ec111-d3be-4e81-865c-0dffe6d829a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:15.499244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6d8ec111-d3be-4e81-865c-0dffe6d829a5 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:15.511340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6d8ec111-d3be-4e81-865c-0dffe6d829a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:15.555736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93589e-31a8-4633-a515-6f69eda340c0 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:15.628093 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2496/4969] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:15.637405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12322af-0588-48b3-8f5d-beb52a4f84f8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:15.645808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12322af-0588-48b3-8f5d-beb52a4f84f8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:15.645808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12322af-0588-48b3-8f5d-beb52a4f84f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:15.659850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b12322af-0588-48b3-8f5d-beb52a4f84f8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:15.661214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b12322af-0588-48b3-8f5d-beb52a4f84f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:15.677233 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2497/4970] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:15.678283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12322af-0588-48b3-8f5d-beb52a4f84f8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:15.706595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa107a4-1676-4eb2-81c8-d332a841231b tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a8ec3aee1842a78442c916aa8d5b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65630772e5d64ab190615b189537d59c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:15.714854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12322af-0588-48b3-8f5d-beb52a4f84f8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:15.715203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa107a4-1676-4eb2-81c8-d332a841231b tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:15.715203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa107a4-1676-4eb2-81c8-d332a841231b tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:15.715203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa107a4-1676-4eb2-81c8-d332a841231b tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:15.715203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa107a4-1676-4eb2-81c8-d332a841231b tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:15.715203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa107a4-1676-4eb2-81c8-d332a841231b tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:15.715656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa107a4-1676-4eb2-81c8-d332a841231b tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:15.721911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3db1d-f9c5-4fe9-be87-3c800dd30b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:15.722717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3db1d-f9c5-4fe9-be87-3c800dd30b8d admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:15.723065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3db1d-f9c5-4fe9-be87-3c800dd30b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:15.723533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3db1d-f9c5-4fe9-be87-3c800dd30b8d admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:15.725029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3db1d-f9c5-4fe9-be87-3c800dd30b8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:15.726739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3db1d-f9c5-4fe9-be87-3c800dd30b8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:15.730963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff3db1d-f9c5-4fe9-be87-3c800dd30b8d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:15.759477 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2474/4971] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 394 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:19:15.759477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa107a4-1676-4eb2-81c8-d332a841231b tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:15.761229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0fa107a4-1676-4eb2-81c8-d332a841231b tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:15.770849 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2498/4972] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:15.782360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3db1d-f9c5-4fe9-be87-3c800dd30b8d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:15.792990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa107a4-1676-4eb2-81c8-d332a841231b tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:15.800516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:15.803487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ebb43-a687-4de1-be4f-b1a82a7c9672 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a8ec3aee1842a78442c916aa8d5b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65630772e5d64ab190615b189537d59c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:15.817571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3db1d-f9c5-4fe9-be87-3c800dd30b8d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:15.818184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ebb43-a687-4de1-be4f-b1a82a7c9672 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:15.818184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ebb43-a687-4de1-be4f-b1a82a7c9672 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:15.818184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ebb43-a687-4de1-be4f-b1a82a7c9672 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:15.818184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ebb43-a687-4de1-be4f-b1a82a7c9672 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:15.818184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ebb43-a687-4de1-be4f-b1a82a7c9672 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:15.818611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4d066-8982-40a3-acf1-25bae9eaa872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:15.818611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4d066-8982-40a3-acf1-25bae9eaa872 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:15.818611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4d066-8982-40a3-acf1-25bae9eaa872 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:15.818611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4d066-8982-40a3-acf1-25bae9eaa872 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:15.818956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4d066-8982-40a3-acf1-25bae9eaa872 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:15.818956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4d066-8982-40a3-acf1-25bae9eaa872 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:15.898665 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2475/4973] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:15 2022] POST /identity/v3/projects => generated 354 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:15.902956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ebb43-a687-4de1-be4f-b1a82a7c9672 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:15.951229 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2499/4974] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:15 2022] POST /identity/v3/projects => generated 375 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:15.951229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4d066-8982-40a3-acf1-25bae9eaa872 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:15.951830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-30d4d066-8982-40a3-acf1-25bae9eaa872 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:16.008846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ebb43-a687-4de1-be4f-b1a82a7c9672 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:16.065412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4d066-8982-40a3-acf1-25bae9eaa872 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:16.074199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b704c-17d6-4ab2-8240-383579b1c7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:16.077039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b704c-17d6-4ab2-8240-383579b1c7e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:16.078006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b704c-17d6-4ab2-8240-383579b1c7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:16.078302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b704c-17d6-4ab2-8240-383579b1c7e1 admin admin] PATH_INFO: `/v3/users/a01fd3c4a0184b41864b9184c507a8ff` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:16.082056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b704c-17d6-4ab2-8240-383579b1c7e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a01fd3c4a0184b41864b9184c507a8ff)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:16.082495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b704c-17d6-4ab2-8240-383579b1c7e1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:16.108643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe847fde-4bd9-4807-bcde-89db489a3a95 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a8ec3aee1842a78442c916aa8d5b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65630772e5d64ab190615b189537d59c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:16.109394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe847fde-4bd9-4807-bcde-89db489a3a95 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:16.109743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe847fde-4bd9-4807-bcde-89db489a3a95 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:16.110115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe847fde-4bd9-4807-bcde-89db489a3a95 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:16.111481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe847fde-4bd9-4807-bcde-89db489a3a95 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:16.111856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe847fde-4bd9-4807-bcde-89db489a3a95 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:16.113615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe847fde-4bd9-4807-bcde-89db489a3a95 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:16.127399 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2476/4975] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:19:16.128503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe847fde-4bd9-4807-bcde-89db489a3a95 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:16.141035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe847fde-4bd9-4807-bcde-89db489a3a95 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:16.146473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b25cab-c472-4381-8fac-bc2c697db4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:16.147449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b25cab-c472-4381-8fac-bc2c697db4d8 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:16.147828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b25cab-c472-4381-8fac-bc2c697db4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:16.148345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b25cab-c472-4381-8fac-bc2c697db4d8 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:16.149248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b25cab-c472-4381-8fac-bc2c697db4d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:16.151617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b25cab-c472-4381-8fac-bc2c697db4d8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:16.273049 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2477/4976] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:16 2022] POST /identity/v3/users => generated 288 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:16.273049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b25cab-c472-4381-8fac-bc2c697db4d8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:16.288715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b25cab-c472-4381-8fac-bc2c697db4d8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:16.297883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19a8ec3aee1842a78442c916aa8d5b2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65630772e5d64ab190615b189537d59c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:16.304521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:16.304521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:16.304521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] PATH_INFO: `/v3/projects/cc89fceddc8b4f94b2312d5e740ed760` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:16.304521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cc89fceddc8b4f94b2312d5e740ed760)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:16.304883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:16.312571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-298b704c-17d6-4ab2-8240-383579b1c7e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a01fd3c4a0184b41864b9184c507a8ff'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:16.323030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-298b704c-17d6-4ab2-8240-383579b1c7e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a01fd3c4a0184b41864b9184c507a8ff'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:16.328054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-298b704c-17d6-4ab2-8240-383579b1c7e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a01fd3c4a0184b41864b9184c507a8ff'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:16.329930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-298b704c-17d6-4ab2-8240-383579b1c7e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a01fd3c4a0184b41864b9184c507a8ff'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:16.390561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-298b704c-17d6-4ab2-8240-383579b1c7e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a01fd3c4a0184b41864b9184c507a8ff'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:16.422347 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2500/4977] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:15 2022] DELETE /identity/v3/users/a01fd3c4a0184b41864b9184c507a8ff => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:16.424335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b704c-17d6-4ab2-8240-383579b1c7e1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:16.426156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-298b704c-17d6-4ab2-8240-383579b1c7e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:16.446470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc89fceddc8b4f94b2312d5e740ed760', 'request_id': 'req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:16.455278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc89fceddc8b4f94b2312d5e740ed760', 'request_id': 'req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:16.516136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc89fceddc8b4f94b2312d5e740ed760', 'request_id': 'req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:16.524854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b704c-17d6-4ab2-8240-383579b1c7e1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:16.534592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca96190-1561-4987-bd11-71449a937e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:16.535697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] The token cache is being invalidate because project cc89fceddc8b4f94b2312d5e740ed760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:16.536078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:16.536640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca96190-1561-4987-bd11-71449a937e17 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:16.536928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:16.537434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca96190-1561-4987-bd11-71449a937e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:16.538405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca96190-1561-4987-bd11-71449a937e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:16.538957 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2478/4978] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:16 2022] DELETE /identity/v3/projects/cc89fceddc8b4f94b2312d5e740ed760 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:16.539774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca96190-1561-4987-bd11-71449a937e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:16.544006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca96190-1561-4987-bd11-71449a937e17 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:16.561249 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2501/4979] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:16.569575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:16.569575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:16.652794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8b4e3-b2b9-41aa-8edf-5d22062d20e4 tempest-VolumeQuotaClassesTest-734426253 tempest-VolumeQuotaClassesTest-734426253-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:16.658560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1577797-0a5a-41b9-8004-b9c70f4dff88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:16.659342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1577797-0a5a-41b9-8004-b9c70f4dff88 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:16.659697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1577797-0a5a-41b9-8004-b9c70f4dff88 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:16.660071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1577797-0a5a-41b9-8004-b9c70f4dff88 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:16.660892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1577797-0a5a-41b9-8004-b9c70f4dff88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:16.662770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1577797-0a5a-41b9-8004-b9c70f4dff88 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:16.680811 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2479/4980] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:16.684018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca96190-1561-4987-bd11-71449a937e17 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:16.695576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca96190-1561-4987-bd11-71449a937e17 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:16.706861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7024c-5de5-4554-986e-6bb55fa9b8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:16.707309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7024c-5de5-4554-986e-6bb55fa9b8e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:16.707869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7024c-5de5-4554-986e-6bb55fa9b8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:16.708257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd7024c-5de5-4554-986e-6bb55fa9b8e3 admin admin] PATH_INFO: `/v3/projects/f4d533da7bb7424e9239a0fa9cac4818/users/e0ce748086004c5d89c7c29c1655cf09/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:16.732052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7024c-5de5-4554-986e-6bb55fa9b8e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d533da7bb7424e9239a0fa9cac4818, user_id=e0ce748086004c5d89c7c29c1655cf09, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:16.742353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd7024c-5de5-4554-986e-6bb55fa9b8e3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:16.816855 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2502/4981] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:16 2022] PUT /identity/v3/projects/f4d533da7bb7424e9239a0fa9cac4818/users/e0ce748086004c5d89c7c29c1655cf09/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:16.827700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653166d-c06e-45a2-bca3-8c7eb01bf243 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:16.830657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653166d-c06e-45a2-bca3-8c7eb01bf243 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:16.830657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b653166d-c06e-45a2-bca3-8c7eb01bf243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:16.869515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b653166d-c06e-45a2-bca3-8c7eb01bf243 None None] MFA Rules not processed for user `e0ce748086004c5d89c7c29c1655cf09`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:16.869515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b653166d-c06e-45a2-bca3-8c7eb01bf243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:16.944895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7024c-5de5-4554-986e-6bb55fa9b8e3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:16.947416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ffd7024c-5de5-4554-986e-6bb55fa9b8e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:16.958628 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2480/4982] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:16.972625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653166d-c06e-45a2-bca3-8c7eb01bf243 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:16.975018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b653166d-c06e-45a2-bca3-8c7eb01bf243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:17.095420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd7024c-5de5-4554-986e-6bb55fa9b8e3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:17.107210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b653166d-c06e-45a2-bca3-8c7eb01bf243 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:17.109894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:17.111063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:17.111583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:17.112028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b admin admin] PATH_INFO: `/v3/projects/12b9ffa69df641558691a33ba072f78d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:17.113457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b9ffa69df641558691a33ba072f78d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:17.115980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3de5a-0bf5-4429-9383-55f7198bc1ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:17.116968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3de5a-0bf5-4429-9383-55f7198bc1ea service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:17.117344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:17.118425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3de5a-0bf5-4429-9383-55f7198bc1ea service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:17.118824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3de5a-0bf5-4429-9383-55f7198bc1ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:17.133538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3de5a-0bf5-4429-9383-55f7198bc1ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:17.135660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3de5a-0bf5-4429-9383-55f7198bc1ea service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:17.150905 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2481/4983] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:17.345427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b9ffa69df641558691a33ba072f78d', 'request_id': 'req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:17.352700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b9ffa69df641558691a33ba072f78d', 'request_id': 'req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:17.407355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:17.446528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3de5a-0bf5-4429-9383-55f7198bc1ea service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:17.448243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-07d3de5a-0bf5-4429-9383-55f7198bc1ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:17.464948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b9ffa69df641558691a33ba072f78d', 'request_id': 'req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:17.467115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3de5a-0bf5-4429-9383-55f7198bc1ea service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:17.493287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b admin admin] The token cache is being invalidate because project 12b9ffa69df641558691a33ba072f78d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:17.501999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:17.501999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:17.501999 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2503/4984] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:16 2022] DELETE /identity/v3/projects/12b9ffa69df641558691a33ba072f78d => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:17.501999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:17.501999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:17.543454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47613942-8cac-4b23-8013-f34372668be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:17.544433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47613942-8cac-4b23-8013-f34372668be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:17.544683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47613942-8cac-4b23-8013-f34372668be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:17.544967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47613942-8cac-4b23-8013-f34372668be2 admin admin] PATH_INFO: `/v3/users/843f3d747f604fe5845d4401cb570429` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:17.546809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47613942-8cac-4b23-8013-f34372668be2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=843f3d747f604fe5845d4401cb570429)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:17.549664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47613942-8cac-4b23-8013-f34372668be2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:17.589446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:17.624459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8d4f0-3eb4-4a1b-89d5-3cbc0bec7b0b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:17.630385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177d845-77cf-4775-8d68-5dfe0e115046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:17.631268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177d845-77cf-4775-8d68-5dfe0e115046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:17.631268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177d845-77cf-4775-8d68-5dfe0e115046 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:17.631733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177d845-77cf-4775-8d68-5dfe0e115046 admin admin] PATH_INFO: `/v3/users/e0ce748086004c5d89c7c29c1655cf09` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:17.634014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177d845-77cf-4775-8d68-5dfe0e115046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0ce748086004c5d89c7c29c1655cf09)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:17.635533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177d845-77cf-4775-8d68-5dfe0e115046 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:17.707020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-47613942-8cac-4b23-8013-f34372668be2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '843f3d747f604fe5845d4401cb570429'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:17.707779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-47613942-8cac-4b23-8013-f34372668be2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '843f3d747f604fe5845d4401cb570429'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:17.718416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-47613942-8cac-4b23-8013-f34372668be2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '843f3d747f604fe5845d4401cb570429'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:17.801466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-47613942-8cac-4b23-8013-f34372668be2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '843f3d747f604fe5845d4401cb570429'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:17.802385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-47613942-8cac-4b23-8013-f34372668be2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '843f3d747f604fe5845d4401cb570429'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:17.817110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a177d845-77cf-4775-8d68-5dfe0e115046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0ce748086004c5d89c7c29c1655cf09'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:17.822741 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2482/4985] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:17 2022] DELETE /identity/v3/users/843f3d747f604fe5845d4401cb570429 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:17.825865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a177d845-77cf-4775-8d68-5dfe0e115046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0ce748086004c5d89c7c29c1655cf09'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:17.826910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a177d845-77cf-4775-8d68-5dfe0e115046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0ce748086004c5d89c7c29c1655cf09'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:17.827045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc93d86-c5d5-4c12-9bbf-91faa2163146 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:17.827220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc93d86-c5d5-4c12-9bbf-91faa2163146 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:17.827507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc93d86-c5d5-4c12-9bbf-91faa2163146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:17.828459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a177d845-77cf-4775-8d68-5dfe0e115046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0ce748086004c5d89c7c29c1655cf09'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:17.848526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4fc93d86-c5d5-4c12-9bbf-91faa2163146 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:17.849668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4fc93d86-c5d5-4c12-9bbf-91faa2163146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:17.918647 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2483/4986] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:17.920093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e66fb5-db31-4ab2-94f1-a02577cd30d9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:17.920384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e66fb5-db31-4ab2-94f1-a02577cd30d9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:17.920662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e66fb5-db31-4ab2-94f1-a02577cd30d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:17.926772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a177d845-77cf-4775-8d68-5dfe0e115046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0ce748086004c5d89c7c29c1655cf09'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:17.943231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-23e66fb5-db31-4ab2-94f1-a02577cd30d9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:17.944608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-23e66fb5-db31-4ab2-94f1-a02577cd30d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:17.952017 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2504/4987] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:17 2022] DELETE /identity/v3/users/e0ce748086004c5d89c7c29c1655cf09 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:17.953579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b030578-fb0d-4db1-ba60-e0dc3064c713 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:17.953895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b030578-fb0d-4db1-ba60-e0dc3064c713 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:17.954888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b030578-fb0d-4db1-ba60-e0dc3064c713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:17.955790 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2484/4988] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:17.961585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cc489-fb32-405e-87d7-7a093b8d4a7d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:17.963704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cc489-fb32-405e-87d7-7a093b8d4a7d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:17.963983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570cc489-fb32-405e-87d7-7a093b8d4a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:17.977556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2b030578-fb0d-4db1-ba60-e0dc3064c713 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:17.978841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2b030578-fb0d-4db1-ba60-e0dc3064c713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:17.990081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-570cc489-fb32-405e-87d7-7a093b8d4a7d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:17.991391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-570cc489-fb32-405e-87d7-7a093b8d4a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:18.049759 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2505/4989] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:18.050655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b030578-fb0d-4db1-ba60-e0dc3064c713 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:18.052183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2b030578-fb0d-4db1-ba60-e0dc3064c713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:18.087477 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2485/4990] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:18.088340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570cc489-fb32-405e-87d7-7a093b8d4a7d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:18.089930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-570cc489-fb32-405e-87d7-7a093b8d4a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:18.103224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570cc489-fb32-405e-87d7-7a093b8d4a7d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:18.109362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f34e4c-a03c-4215-a679-e592f591dd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:18.113764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f34e4c-a03c-4215-a679-e592f591dd37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:18.113764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f34e4c-a03c-4215-a679-e592f591dd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:18.113764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f34e4c-a03c-4215-a679-e592f591dd37 admin admin] PATH_INFO: `/v3/users/1c340ea34bdd4a8d993f383d9feb496d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:18.113764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f34e4c-a03c-4215-a679-e592f591dd37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c340ea34bdd4a8d993f383d9feb496d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:18.115609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f34e4c-a03c-4215-a679-e592f591dd37 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:18.121218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b030578-fb0d-4db1-ba60-e0dc3064c713 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:18.125164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3430a-bbce-4789-bd63-091a31bd8125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:18.125816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3430a-bbce-4789-bd63-091a31bd8125 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:18.126709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3430a-bbce-4789-bd63-091a31bd8125 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:18.126990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3430a-bbce-4789-bd63-091a31bd8125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:18.138775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a3430a-bbce-4789-bd63-091a31bd8125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:18.140312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a3430a-bbce-4789-bd63-091a31bd8125 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:18.151938 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2506/4991] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:18.152444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3430a-bbce-4789-bd63-091a31bd8125 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:18.162037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3430a-bbce-4789-bd63-091a31bd8125 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:18.166848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03696650-a398-4c64-b45c-f28234b788d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:18.167512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03696650-a398-4c64-b45c-f28234b788d1 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:18.167822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03696650-a398-4c64-b45c-f28234b788d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:18.168102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03696650-a398-4c64-b45c-f28234b788d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:18.170725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03696650-a398-4c64-b45c-f28234b788d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:18.170725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03696650-a398-4c64-b45c-f28234b788d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:18.170939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03696650-a398-4c64-b45c-f28234b788d1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:18.183355 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2507/4992] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:18.184182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03696650-a398-4c64-b45c-f28234b788d1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:18.189372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-03696650-a398-4c64-b45c-f28234b788d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:18.202887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03696650-a398-4c64-b45c-f28234b788d1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:18.206713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e163c-335a-4864-ac58-e746c617ddfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:18.207355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e163c-335a-4864-ac58-e746c617ddfe service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:18.207642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e163c-335a-4864-ac58-e746c617ddfe service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:18.207951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e163c-335a-4864-ac58-e746c617ddfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:18.218227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e163c-335a-4864-ac58-e746c617ddfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:18.219839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e163c-335a-4864-ac58-e746c617ddfe service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:18.234427 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2508/4993] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:18.236076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e163c-335a-4864-ac58-e746c617ddfe service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:18.246071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e163c-335a-4864-ac58-e746c617ddfe service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:18.254135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e3a4c-5fd3-43c0-ac8b-635cd60d25f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:18.255413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e3a4c-5fd3-43c0-ac8b-635cd60d25f1 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:18.255911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e3a4c-5fd3-43c0-ac8b-635cd60d25f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:18.256535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e3a4c-5fd3-43c0-ac8b-635cd60d25f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:18.258480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e3a4c-5fd3-43c0-ac8b-635cd60d25f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:18.262521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e3a4c-5fd3-43c0-ac8b-635cd60d25f1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:18.343054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-11f34e4c-a03c-4215-a679-e592f591dd37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c340ea34bdd4a8d993f383d9feb496d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:18.343790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-11f34e4c-a03c-4215-a679-e592f591dd37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c340ea34bdd4a8d993f383d9feb496d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:18.351362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-11f34e4c-a03c-4215-a679-e592f591dd37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c340ea34bdd4a8d993f383d9feb496d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:18.377047 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2509/4994] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:18 2022] POST /identity/v3/projects => generated 389 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:18.379398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e3a4c-5fd3-43c0-ac8b-635cd60d25f1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:18.449002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-11f34e4c-a03c-4215-a679-e592f591dd37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c340ea34bdd4a8d993f383d9feb496d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:18.451895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-11f34e4c-a03c-4215-a679-e592f591dd37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c340ea34bdd4a8d993f383d9feb496d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:18.470927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e3a4c-5fd3-43c0-ac8b-635cd60d25f1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:18.477360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71dfdd-ed3c-4e5b-88b4-9fe3493b4a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:18.478351 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2486/4995] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:18 2022] DELETE /identity/v3/users/1c340ea34bdd4a8d993f383d9feb496d => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:18.478648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71dfdd-ed3c-4e5b-88b4-9fe3493b4a4b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:18.479032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71dfdd-ed3c-4e5b-88b4-9fe3493b4a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:18.479987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71dfdd-ed3c-4e5b-88b4-9fe3493b4a4b admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:18.480770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71dfdd-ed3c-4e5b-88b4-9fe3493b4a4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:18.483013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a1e36-31cf-41ec-bf5b-36c5de338fdd None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:18.483496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e71dfdd-ed3c-4e5b-88b4-9fe3493b4a4b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:18.483827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a1e36-31cf-41ec-bf5b-36c5de338fdd None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:18.484321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a1e36-31cf-41ec-bf5b-36c5de338fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:18.515349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-110a1e36-31cf-41ec-bf5b-36c5de338fdd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:18.523177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-110a1e36-31cf-41ec-bf5b-36c5de338fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:18.580353 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2487/4996] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:18.590709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a1e36-31cf-41ec-bf5b-36c5de338fdd None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:18.592293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-110a1e36-31cf-41ec-bf5b-36c5de338fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:18.671527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a1e36-31cf-41ec-bf5b-36c5de338fdd None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:18.677260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cbd76-4c58-45ba-b77d-63045d42e45e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:18.679423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cbd76-4c58-45ba-b77d-63045d42e45e service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:18.679896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cbd76-4c58-45ba-b77d-63045d42e45e service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:18.680536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cbd76-4c58-45ba-b77d-63045d42e45e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:18.684985 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2510/4997] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:18 2022] POST /identity/v3/users => generated 295 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:18.689514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71dfdd-ed3c-4e5b-88b4-9fe3493b4a4b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:18.691368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5e71dfdd-ed3c-4e5b-88b4-9fe3493b4a4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:18.697556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879cbd76-4c58-45ba-b77d-63045d42e45e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:18.700798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879cbd76-4c58-45ba-b77d-63045d42e45e service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:18.706025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71dfdd-ed3c-4e5b-88b4-9fe3493b4a4b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:18.712548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efc03f-ee72-47a5-9072-ea21d706b19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:18.713499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efc03f-ee72-47a5-9072-ea21d706b19d admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:18.713931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efc03f-ee72-47a5-9072-ea21d706b19d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:18.714413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efc03f-ee72-47a5-9072-ea21d706b19d admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:18.715339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67efc03f-ee72-47a5-9072-ea21d706b19d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:18.717145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67efc03f-ee72-47a5-9072-ea21d706b19d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:18.723399 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2488/4998] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:19:18.738385 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2511/4999] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:18.741983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cbd76-4c58-45ba-b77d-63045d42e45e service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:18.753944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cbd76-4c58-45ba-b77d-63045d42e45e service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:18.762430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04548c3-8b96-4ba1-a932-17e3d4474bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:18.763937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04548c3-8b96-4ba1-a932-17e3d4474bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:18.764384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04548c3-8b96-4ba1-a932-17e3d4474bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:18.764796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04548c3-8b96-4ba1-a932-17e3d4474bf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:18.766324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04548c3-8b96-4ba1-a932-17e3d4474bf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:18.771791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04548c3-8b96-4ba1-a932-17e3d4474bf2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:18.799914 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2489/5000] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:18.804553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efc03f-ee72-47a5-9072-ea21d706b19d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:18.818417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efc03f-ee72-47a5-9072-ea21d706b19d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:18.823456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7bd8f0-7fc1-47ce-8d9e-a171c1d67a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:18.824352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bd8f0-7fc1-47ce-8d9e-a171c1d67a4f admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:18.824641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bd8f0-7fc1-47ce-8d9e-a171c1d67a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:18.824924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7bd8f0-7fc1-47ce-8d9e-a171c1d67a4f admin admin] PATH_INFO: `/v3/projects/5a8841fa441d413a92a15510a9d082c6/users/857a887b6c7c41588afb2845eed6f309/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:18.841741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7bd8f0-7fc1-47ce-8d9e-a171c1d67a4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a8841fa441d413a92a15510a9d082c6, user_id=857a887b6c7c41588afb2845eed6f309, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:18.844507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7bd8f0-7fc1-47ce-8d9e-a171c1d67a4f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:18.933260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04548c3-8b96-4ba1-a932-17e3d4474bf2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:18.935341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a04548c3-8b96-4ba1-a932-17e3d4474bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:18.949489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04548c3-8b96-4ba1-a932-17e3d4474bf2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:18.954656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a819ee-a39b-4783-89fe-675218db70ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:18.955527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a819ee-a39b-4783-89fe-675218db70ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:18.955813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a819ee-a39b-4783-89fe-675218db70ef admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:18.956090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a819ee-a39b-4783-89fe-675218db70ef admin admin] PATH_INFO: `/v3/projects/f4d533da7bb7424e9239a0fa9cac4818` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:18.957990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a819ee-a39b-4783-89fe-675218db70ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4d533da7bb7424e9239a0fa9cac4818)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:18.959319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a819ee-a39b-4783-89fe-675218db70ef admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:19.037094 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2512/5001] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:18 2022] PUT /identity/v3/projects/5a8841fa441d413a92a15510a9d082c6/users/857a887b6c7c41588afb2845eed6f309/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:19.039165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7bd8f0-7fc1-47ce-8d9e-a171c1d67a4f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:19.041035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-be7bd8f0-7fc1-47ce-8d9e-a171c1d67a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:19.069823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:19.131604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7bd8f0-7fc1-47ce-8d9e-a171c1d67a4f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:19.137123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53064082-609c-472f-aaf6-dc12cadddaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:19.138033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53064082-609c-472f-aaf6-dc12cadddaed admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:19.138319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53064082-609c-472f-aaf6-dc12cadddaed admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:19.138598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53064082-609c-472f-aaf6-dc12cadddaed admin admin] PATH_INFO: `/v3/projects/bef32a9958744263854bd63f7922cda9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:19.142289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53064082-609c-472f-aaf6-dc12cadddaed admin admin] RBAC: Authorizing `identity:delete_project(project_id=bef32a9958744263854bd63f7922cda9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:19.142289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53064082-609c-472f-aaf6-dc12cadddaed admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:19.183841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-16a819ee-a39b-4783-89fe-675218db70ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4d533da7bb7424e9239a0fa9cac4818', 'request_id': 'req-16a819ee-a39b-4783-89fe-675218db70ef'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.190381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-16a819ee-a39b-4783-89fe-675218db70ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4d533da7bb7424e9239a0fa9cac4818', 'request_id': 'req-16a819ee-a39b-4783-89fe-675218db70ef'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.275379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-16a819ee-a39b-4783-89fe-675218db70ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4d533da7bb7424e9239a0fa9cac4818', 'request_id': 'req-16a819ee-a39b-4783-89fe-675218db70ef'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.285016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53064082-609c-472f-aaf6-dc12cadddaed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bef32a9958744263854bd63f7922cda9', 'request_id': 'req-53064082-609c-472f-aaf6-dc12cadddaed'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.293359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53064082-609c-472f-aaf6-dc12cadddaed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bef32a9958744263854bd63f7922cda9', 'request_id': 'req-53064082-609c-472f-aaf6-dc12cadddaed'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.300086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-16a819ee-a39b-4783-89fe-675218db70ef admin admin] The token cache is being invalidate because project f4d533da7bb7424e9239a0fa9cac4818 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:19.300921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-16a819ee-a39b-4783-89fe-675218db70ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.301671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-16a819ee-a39b-4783-89fe-675218db70ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.308004 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2490/5002] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:18 2022] DELETE /identity/v3/projects/f4d533da7bb7424e9239a0fa9cac4818 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:19.309777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec98645-2394-4f75-9d68-a6489ca2fe43 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:19.310286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec98645-2394-4f75-9d68-a6489ca2fe43 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:19.310654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec98645-2394-4f75-9d68-a6489ca2fe43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:19.334806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9ec98645-2394-4f75-9d68-a6489ca2fe43 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:19.336139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9ec98645-2394-4f75-9d68-a6489ca2fe43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:19.350594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:19.388685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53064082-609c-472f-aaf6-dc12cadddaed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bef32a9958744263854bd63f7922cda9', 'request_id': 'req-53064082-609c-472f-aaf6-dc12cadddaed'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.400708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53064082-609c-472f-aaf6-dc12cadddaed admin admin] The token cache is being invalidate because project bef32a9958744263854bd63f7922cda9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:19.400903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53064082-609c-472f-aaf6-dc12cadddaed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.401564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53064082-609c-472f-aaf6-dc12cadddaed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.402410 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2491/5003] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:19.403283 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2513/5004] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:19 2022] DELETE /identity/v3/projects/bef32a9958744263854bd63f7922cda9 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:19.403734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031e6d4-fade-437b-af41-6857b6e1278c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:19.404021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031e6d4-fade-437b-af41-6857b6e1278c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:19.404302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031e6d4-fade-437b-af41-6857b6e1278c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:19.407888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53064082-609c-472f-aaf6-dc12cadddaed admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:19.409255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-53064082-609c-472f-aaf6-dc12cadddaed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:19.425180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53064082-609c-472f-aaf6-dc12cadddaed admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:19.430358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12018e9e-fd30-4ef5-aebb-a5eba6322b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:19.430942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4031e6d4-fade-437b-af41-6857b6e1278c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:19.431616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12018e9e-fd30-4ef5-aebb-a5eba6322b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:19.431976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12018e9e-fd30-4ef5-aebb-a5eba6322b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:19.432331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12018e9e-fd30-4ef5-aebb-a5eba6322b15 admin admin] PATH_INFO: `/v3/users/19a8ec3aee1842a78442c916aa8d5b2b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:19.433450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4031e6d4-fade-437b-af41-6857b6e1278c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:19.434431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12018e9e-fd30-4ef5-aebb-a5eba6322b15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19a8ec3aee1842a78442c916aa8d5b2b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:19.442405 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2492/5005] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:19.442586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12018e9e-fd30-4ef5-aebb-a5eba6322b15 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:19.443106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031e6d4-fade-437b-af41-6857b6e1278c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:19.451964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4031e6d4-fade-437b-af41-6857b6e1278c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:19.550043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031e6d4-fade-437b-af41-6857b6e1278c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:19.555573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5f548-0ff1-4b73-a7a5-99f7bf52ffd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:19.559160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5f548-0ff1-4b73-a7a5-99f7bf52ffd7 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:19.559160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5f548-0ff1-4b73-a7a5-99f7bf52ffd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:19.559160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5f548-0ff1-4b73-a7a5-99f7bf52ffd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:19.560311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fda5f548-0ff1-4b73-a7a5-99f7bf52ffd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:19.573684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda5f548-0ff1-4b73-a7a5-99f7bf52ffd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:19.575192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda5f548-0ff1-4b73-a7a5-99f7bf52ffd7 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:19.586835 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2493/5006] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:19.588296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41a786-3362-426f-8280-27ab4f9cc1d6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:19.588596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41a786-3362-426f-8280-27ab4f9cc1d6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:19.588880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac41a786-3362-426f-8280-27ab4f9cc1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:19.610991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ac41a786-3362-426f-8280-27ab4f9cc1d6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:19.613120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ac41a786-3362-426f-8280-27ab4f9cc1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:19.624402 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2494/5007] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:19.625655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41a786-3362-426f-8280-27ab4f9cc1d6 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:19.627001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ac41a786-3362-426f-8280-27ab4f9cc1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:19.640875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac41a786-3362-426f-8280-27ab4f9cc1d6 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:19.647852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48149e57-f9e1-43da-ab72-23ed82dca245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:19.648720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48149e57-f9e1-43da-ab72-23ed82dca245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:19.649006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48149e57-f9e1-43da-ab72-23ed82dca245 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:19.649284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48149e57-f9e1-43da-ab72-23ed82dca245 admin admin] PATH_INFO: `/v3/projects/8e4641f49e584ec0ab8e21f2fb572cc0` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:19.650571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48149e57-f9e1-43da-ab72-23ed82dca245 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e4641f49e584ec0ab8e21f2fb572cc0)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:19.655151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48149e57-f9e1-43da-ab72-23ed82dca245 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:19.792648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-12018e9e-fd30-4ef5-aebb-a5eba6322b15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19a8ec3aee1842a78442c916aa8d5b2b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.800993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-12018e9e-fd30-4ef5-aebb-a5eba6322b15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19a8ec3aee1842a78442c916aa8d5b2b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.801407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-12018e9e-fd30-4ef5-aebb-a5eba6322b15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19a8ec3aee1842a78442c916aa8d5b2b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.802185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-12018e9e-fd30-4ef5-aebb-a5eba6322b15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19a8ec3aee1842a78442c916aa8d5b2b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.875822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-12018e9e-fd30-4ef5-aebb-a5eba6322b15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19a8ec3aee1842a78442c916aa8d5b2b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.881786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-48149e57-f9e1-43da-ab72-23ed82dca245 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e4641f49e584ec0ab8e21f2fb572cc0', 'request_id': 'req-48149e57-f9e1-43da-ab72-23ed82dca245'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.889532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-48149e57-f9e1-43da-ab72-23ed82dca245 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e4641f49e584ec0ab8e21f2fb572cc0', 'request_id': 'req-48149e57-f9e1-43da-ab72-23ed82dca245'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.896512 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2514/5008] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:19 2022] DELETE /identity/v3/users/19a8ec3aee1842a78442c916aa8d5b2b => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:19.897374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12018e9e-fd30-4ef5-aebb-a5eba6322b15 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:19.899117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-12018e9e-fd30-4ef5-aebb-a5eba6322b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:19.973472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12018e9e-fd30-4ef5-aebb-a5eba6322b15 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:19.979941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff218764-4690-4d09-9ea4-9e8a1ef88081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:19.980739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff218764-4690-4d09-9ea4-9e8a1ef88081 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:19.981160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff218764-4690-4d09-9ea4-9e8a1ef88081 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:19.982049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff218764-4690-4d09-9ea4-9e8a1ef88081 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:19.983335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff218764-4690-4d09-9ea4-9e8a1ef88081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:19.984262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff218764-4690-4d09-9ea4-9e8a1ef88081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:19.986002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-48149e57-f9e1-43da-ab72-23ed82dca245 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e4641f49e584ec0ab8e21f2fb572cc0', 'request_id': 'req-48149e57-f9e1-43da-ab72-23ed82dca245'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:19.990901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff218764-4690-4d09-9ea4-9e8a1ef88081 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:20.003422 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2515/5009] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:20.006169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-48149e57-f9e1-43da-ab72-23ed82dca245 admin admin] The token cache is being invalidate because project 8e4641f49e584ec0ab8e21f2fb572cc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:20.006169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-48149e57-f9e1-43da-ab72-23ed82dca245 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:20.006169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-48149e57-f9e1-43da-ab72-23ed82dca245 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:20.008338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff218764-4690-4d09-9ea4-9e8a1ef88081 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:20.010160 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2495/5010] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:19 2022] DELETE /identity/v3/projects/8e4641f49e584ec0ab8e21f2fb572cc0 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:20.011041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ff218764-4690-4d09-9ea4-9e8a1ef88081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:20.014200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48149e57-f9e1-43da-ab72-23ed82dca245 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:20.017231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-48149e57-f9e1-43da-ab72-23ed82dca245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:20.025231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff218764-4690-4d09-9ea4-9e8a1ef88081 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:20.031741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b2761-796d-48ab-b45a-22c2414b0850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:20.032721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b2761-796d-48ab-b45a-22c2414b0850 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:20.033348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b2761-796d-48ab-b45a-22c2414b0850 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:20.033348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b2761-796d-48ab-b45a-22c2414b0850 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:20.034760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b2761-796d-48ab-b45a-22c2414b0850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:20.037153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b2761-796d-48ab-b45a-22c2414b0850 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:20.096005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48149e57-f9e1-43da-ab72-23ed82dca245 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:20.101140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6b798-2f12-4bdd-a71c-d26daebda74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:20.102271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6b798-2f12-4bdd-a71c-d26daebda74d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:20.102583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6b798-2f12-4bdd-a71c-d26daebda74d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:20.102872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6b798-2f12-4bdd-a71c-d26daebda74d admin admin] PATH_INFO: `/v3/users/13920683869e4a35970c5d1f380f3060` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:20.104098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6b798-2f12-4bdd-a71c-d26daebda74d admin admin] RBAC: Authorizing `identity:delete_user(user_id=13920683869e4a35970c5d1f380f3060)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:20.106434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6b798-2f12-4bdd-a71c-d26daebda74d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:20.127540 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2516/5011] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:20 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:20.128773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b2761-796d-48ab-b45a-22c2414b0850 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:20.207970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b2761-796d-48ab-b45a-22c2414b0850 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:20.222308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29a107-d19c-4232-b2b2-b8ad0728beeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:20.223483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29a107-d19c-4232-b2b2-b8ad0728beeb admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:20.224191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29a107-d19c-4232-b2b2-b8ad0728beeb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:20.224678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29a107-d19c-4232-b2b2-b8ad0728beeb admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:20.225784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed29a107-d19c-4232-b2b2-b8ad0728beeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:20.228237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed29a107-d19c-4232-b2b2-b8ad0728beeb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:20.296957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7ca6b798-2f12-4bdd-a71c-d26daebda74d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13920683869e4a35970c5d1f380f3060'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:20.298593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7ca6b798-2f12-4bdd-a71c-d26daebda74d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13920683869e4a35970c5d1f380f3060'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:20.323058 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2517/5012] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:20 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:20.328072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29a107-d19c-4232-b2b2-b8ad0728beeb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:20.336604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7ca6b798-2f12-4bdd-a71c-d26daebda74d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13920683869e4a35970c5d1f380f3060'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:20.348746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29a107-d19c-4232-b2b2-b8ad0728beeb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:20.365912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b10a4-bbf5-4e80-b640-81fa04e2a5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:20.366624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b10a4-bbf5-4e80-b640-81fa04e2a5ce admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:20.366915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b10a4-bbf5-4e80-b640-81fa04e2a5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:20.367197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b10a4-bbf5-4e80-b640-81fa04e2a5ce admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:20.368217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b10a4-bbf5-4e80-b640-81fa04e2a5ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:20.369706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b10a4-bbf5-4e80-b640-81fa04e2a5ce admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:20.389643 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2518/5013] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:20.390537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b10a4-bbf5-4e80-b640-81fa04e2a5ce admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:20.400576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b10a4-bbf5-4e80-b640-81fa04e2a5ce admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:20.408498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45f6ad-836d-4d20-ab23-b729b71e2ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:20.409074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45f6ad-836d-4d20-ab23-b729b71e2ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:20.409384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45f6ad-836d-4d20-ab23-b729b71e2ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:20.409656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45f6ad-836d-4d20-ab23-b729b71e2ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:20.410589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c45f6ad-836d-4d20-ab23-b729b71e2ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:20.412244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c45f6ad-836d-4d20-ab23-b729b71e2ff6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:20.428339 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2519/5014] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:20.431384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45f6ad-836d-4d20-ab23-b729b71e2ff6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:20.442169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45f6ad-836d-4d20-ab23-b729b71e2ff6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:20.447013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c3909-f4a4-42cf-8758-9adeedd859f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:20.447901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c3909-f4a4-42cf-8758-9adeedd859f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:20.448185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c3909-f4a4-42cf-8758-9adeedd859f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:20.448460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c3909-f4a4-42cf-8758-9adeedd859f7 admin admin] PATH_INFO: `/v3/projects/eff632e61c8c4e469ba91057f99d6b8d/users/c2ae7238114b43ae9b9d1790894ef9a7/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:20.464673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c3909-f4a4-42cf-8758-9adeedd859f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eff632e61c8c4e469ba91057f99d6b8d, user_id=c2ae7238114b43ae9b9d1790894ef9a7, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:20.467396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c3909-f4a4-42cf-8758-9adeedd859f7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:20.509921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7ca6b798-2f12-4bdd-a71c-d26daebda74d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13920683869e4a35970c5d1f380f3060'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:20.511264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7ca6b798-2f12-4bdd-a71c-d26daebda74d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13920683869e4a35970c5d1f380f3060'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:20.529740 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2496/5015] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:20 2022] DELETE /identity/v3/users/13920683869e4a35970c5d1f380f3060 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:20.552855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6b798-2f12-4bdd-a71c-d26daebda74d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:20.554574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7ca6b798-2f12-4bdd-a71c-d26daebda74d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:20.579327 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2520/5016] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:20 2022] PUT /identity/v3/projects/eff632e61c8c4e469ba91057f99d6b8d/users/c2ae7238114b43ae9b9d1790894ef9a7/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:20.590330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b933c9c-5baf-4909-83fb-0ca40f2ebd0c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:20.590646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b933c9c-5baf-4909-83fb-0ca40f2ebd0c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:20.590897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b933c9c-5baf-4909-83fb-0ca40f2ebd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:20.622564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9b933c9c-5baf-4909-83fb-0ca40f2ebd0c None None] MFA Rules not processed for user `c2ae7238114b43ae9b9d1790894ef9a7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:20.624271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9b933c9c-5baf-4909-83fb-0ca40f2ebd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:20.632875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6b798-2f12-4bdd-a71c-d26daebda74d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:20.649054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:INSERT=1,keystone:SELECT=94,keystone:DELETE=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:20.712591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc20404-018b-489b-b7f3-985b7762e496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:20.713467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc20404-018b-489b-b7f3-985b7762e496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:20.713748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc20404-018b-489b-b7f3-985b7762e496 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:20.714365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc20404-018b-489b-b7f3-985b7762e496 admin admin] PATH_INFO: `/v3/projects/65630772e5d64ab190615b189537d59c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:20.715854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc20404-018b-489b-b7f3-985b7762e496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65630772e5d64ab190615b189537d59c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:20.717787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc20404-018b-489b-b7f3-985b7762e496 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:20.718884 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2521/5017] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:20.723772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b933c9c-5baf-4909-83fb-0ca40f2ebd0c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:20.725869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9b933c9c-5baf-4909-83fb-0ca40f2ebd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:20.741716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b933c9c-5baf-4909-83fb-0ca40f2ebd0c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:20.746704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a11b23-3a65-4bee-8193-a69ff36d0f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:20.747605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a11b23-3a65-4bee-8193-a69ff36d0f85 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:20.747962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a11b23-3a65-4bee-8193-a69ff36d0f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:20.748253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a11b23-3a65-4bee-8193-a69ff36d0f85 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:20.749036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a11b23-3a65-4bee-8193-a69ff36d0f85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:20.751109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a11b23-3a65-4bee-8193-a69ff36d0f85 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:20.854326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ddc20404-018b-489b-b7f3-985b7762e496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65630772e5d64ab190615b189537d59c', 'request_id': 'req-ddc20404-018b-489b-b7f3-985b7762e496'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:20.862608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ddc20404-018b-489b-b7f3-985b7762e496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65630772e5d64ab190615b189537d59c', 'request_id': 'req-ddc20404-018b-489b-b7f3-985b7762e496'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:20.890635 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2522/5018] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:20 2022] POST /identity/v3/projects => generated 371 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:20.895064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a11b23-3a65-4bee-8193-a69ff36d0f85 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:20.972443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ddc20404-018b-489b-b7f3-985b7762e496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65630772e5d64ab190615b189537d59c', 'request_id': 'req-ddc20404-018b-489b-b7f3-985b7762e496'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:20.986594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a11b23-3a65-4bee-8193-a69ff36d0f85 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:20.987085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ddc20404-018b-489b-b7f3-985b7762e496 admin admin] The token cache is being invalidate because project 65630772e5d64ab190615b189537d59c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:20.987536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ddc20404-018b-489b-b7f3-985b7762e496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:20.988398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ddc20404-018b-489b-b7f3-985b7762e496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:20.994033 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2497/5019] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:20 2022] DELETE /identity/v3/projects/65630772e5d64ab190615b189537d59c => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:20.994217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b075b4f-de1f-4ba8-b32d-27a5dcd0e3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:20.994217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b075b4f-de1f-4ba8-b32d-27a5dcd0e3e3 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:20.994664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b075b4f-de1f-4ba8-b32d-27a5dcd0e3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:20.998982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b075b4f-de1f-4ba8-b32d-27a5dcd0e3e3 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:20.999135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6cceab-167c-49a8-b22e-f516c48978e4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:20.999135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6cceab-167c-49a8-b22e-f516c48978e4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:20.999135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6cceab-167c-49a8-b22e-f516c48978e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:20.999432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b075b4f-de1f-4ba8-b32d-27a5dcd0e3e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:21.000139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b075b4f-de1f-4ba8-b32d-27a5dcd0e3e3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:21.001958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:21.029580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ba6cceab-167c-49a8-b22e-f516c48978e4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:21.030980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ba6cceab-167c-49a8-b22e-f516c48978e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:21.041380 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2498/5020] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:21.050144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626e4e7-eb53-4d12-9aa2-2e8ad5643c1c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:21.050434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626e4e7-eb53-4d12-9aa2-2e8ad5643c1c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:21.050718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c626e4e7-eb53-4d12-9aa2-2e8ad5643c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:21.070112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c626e4e7-eb53-4d12-9aa2-2e8ad5643c1c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:21.071331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c626e4e7-eb53-4d12-9aa2-2e8ad5643c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:21.081552 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2499/5021] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:21.085774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626e4e7-eb53-4d12-9aa2-2e8ad5643c1c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:21.095368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c626e4e7-eb53-4d12-9aa2-2e8ad5643c1c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:21.100129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ec043-c723-4eaf-b290-3d51ea6b358f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:21.100781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ec043-c723-4eaf-b290-3d51ea6b358f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:21.101064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ec043-c723-4eaf-b290-3d51ea6b358f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:21.101340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8ec043-c723-4eaf-b290-3d51ea6b358f admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:21.102185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ec043-c723-4eaf-b290-3d51ea6b358f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:21.102422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ec043-c723-4eaf-b290-3d51ea6b358f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:21.104033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8ec043-c723-4eaf-b290-3d51ea6b358f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:21.118308 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2500/5022] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:19:21.123291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ec043-c723-4eaf-b290-3d51ea6b358f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:21.133300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8ec043-c723-4eaf-b290-3d51ea6b358f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:21.138230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c6a3e-4f2f-4a37-9c57-efb0c2a6f3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:21.140820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c6a3e-4f2f-4a37-9c57-efb0c2a6f3f6 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:21.140820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c6a3e-4f2f-4a37-9c57-efb0c2a6f3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:21.140820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00c6a3e-4f2f-4a37-9c57-efb0c2a6f3f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:21.140820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c6a3e-4f2f-4a37-9c57-efb0c2a6f3f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:21.142606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00c6a3e-4f2f-4a37-9c57-efb0c2a6f3f6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:21.169893 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2523/5023] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:20 2022] POST /identity/v3/users => generated 292 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:21.172584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b075b4f-de1f-4ba8-b32d-27a5dcd0e3e3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:21.174634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5b075b4f-de1f-4ba8-b32d-27a5dcd0e3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:21.193088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b075b4f-de1f-4ba8-b32d-27a5dcd0e3e3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:21.198913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f0e4b-45fb-4e68-a43e-f154d7364a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:21.200760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f0e4b-45fb-4e68-a43e-f154d7364a38 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:21.200760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f0e4b-45fb-4e68-a43e-f154d7364a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:21.200760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f0e4b-45fb-4e68-a43e-f154d7364a38 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:21.201120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f0e4b-45fb-4e68-a43e-f154d7364a38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:21.203078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32f0e4b-45fb-4e68-a43e-f154d7364a38 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:21.215021 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2524/5024] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:21.218961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f0e4b-45fb-4e68-a43e-f154d7364a38 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:21.233570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32f0e4b-45fb-4e68-a43e-f154d7364a38 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:21.238903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba85da-1a16-44e4-93d4-5c8ee9102373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:21.239836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba85da-1a16-44e4-93d4-5c8ee9102373 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:21.240123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba85da-1a16-44e4-93d4-5c8ee9102373 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:21.240688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba85da-1a16-44e4-93d4-5c8ee9102373 admin admin] PATH_INFO: `/v3/projects/539a73abcc1040c19db7f9252ea8c2b0/users/bb3fb0936c554fa882e46ba190d3b362/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:21.260917 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2501/5025] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:21 2022] POST /identity/v3/projects => generated 401 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:21.270198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba85da-1a16-44e4-93d4-5c8ee9102373 admin admin] RBAC: Authorizing `identity:create_grant(project_id=539a73abcc1040c19db7f9252ea8c2b0, user_id=bb3fb0936c554fa882e46ba190d3b362, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:21.270494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c6a3e-4f2f-4a37-9c57-efb0c2a6f3f6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:21.270936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba85da-1a16-44e4-93d4-5c8ee9102373 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:21.333565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00c6a3e-4f2f-4a37-9c57-efb0c2a6f3f6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:21.338452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a2133-8c17-4d07-9097-3d0933c1b8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:21.339130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a2133-8c17-4d07-9097-3d0933c1b8d8 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:21.339419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a2133-8c17-4d07-9097-3d0933c1b8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:21.339703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083a2133-8c17-4d07-9097-3d0933c1b8d8 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:21.340438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a2133-8c17-4d07-9097-3d0933c1b8d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:21.343003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083a2133-8c17-4d07-9097-3d0933c1b8d8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:21.383248 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2525/5026] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:21 2022] PUT /identity/v3/projects/539a73abcc1040c19db7f9252ea8c2b0/users/bb3fb0936c554fa882e46ba190d3b362/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:21.385033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba85da-1a16-44e4-93d4-5c8ee9102373 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:21.389379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a2ba85da-1a16-44e4-93d4-5c8ee9102373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:21.472948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba85da-1a16-44e4-93d4-5c8ee9102373 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:21.479754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e3673-64d9-43d7-9113-ee7dd7ae200c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:21.480793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e3673-64d9-43d7-9113-ee7dd7ae200c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:21.481082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e3673-64d9-43d7-9113-ee7dd7ae200c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:21.481372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e3673-64d9-43d7-9113-ee7dd7ae200c admin admin] PATH_INFO: `/v3/projects/6c40d2ad8d084a3d832e3de6ea7aff74` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:21.482843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e3673-64d9-43d7-9113-ee7dd7ae200c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c40d2ad8d084a3d832e3de6ea7aff74)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:21.484949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e3673-64d9-43d7-9113-ee7dd7ae200c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:21.489322 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2502/5027] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:21 2022] POST /identity/v3/users => generated 301 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:21.490292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a2133-8c17-4d07-9097-3d0933c1b8d8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:21.500801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083a2133-8c17-4d07-9097-3d0933c1b8d8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:21.522094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ed163-5806-4a15-85b0-7d45bbee9a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:21.523092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ed163-5806-4a15-85b0-7d45bbee9a3e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:21.523904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ed163-5806-4a15-85b0-7d45bbee9a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:21.523904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ed163-5806-4a15-85b0-7d45bbee9a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:21.524683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ed163-5806-4a15-85b0-7d45bbee9a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:21.526952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ed163-5806-4a15-85b0-7d45bbee9a3e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:21.540665 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2503/5028] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:21.541533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ed163-5806-4a15-85b0-7d45bbee9a3e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:21.557703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ed163-5806-4a15-85b0-7d45bbee9a3e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:21.562818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d0c49-5849-4f1f-9ad2-b80e133de723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:21.564099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d0c49-5849-4f1f-9ad2-b80e133de723 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:21.564389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d0c49-5849-4f1f-9ad2-b80e133de723 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:21.564781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d0c49-5849-4f1f-9ad2-b80e133de723 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:21.565390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d0c49-5849-4f1f-9ad2-b80e133de723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:21.567027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d0c49-5849-4f1f-9ad2-b80e133de723 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:21.578504 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2504/5029] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:21.580033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d0c49-5849-4f1f-9ad2-b80e133de723 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:21.591193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d0c49-5849-4f1f-9ad2-b80e133de723 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:21.596198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94cef4-3f8a-44b9-9af1-a5b992a64e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:21.597089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94cef4-3f8a-44b9-9af1-a5b992a64e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:21.597371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94cef4-3f8a-44b9-9af1-a5b992a64e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:21.597945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94cef4-3f8a-44b9-9af1-a5b992a64e6c admin admin] PATH_INFO: `/v3/projects/539a73abcc1040c19db7f9252ea8c2b0/users/bb3fb0936c554fa882e46ba190d3b362/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:21.600414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94cef4-3f8a-44b9-9af1-a5b992a64e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=539a73abcc1040c19db7f9252ea8c2b0, user_id=bb3fb0936c554fa882e46ba190d3b362, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:21.603135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94cef4-3f8a-44b9-9af1-a5b992a64e6c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:21.613997 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2505/5030] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:21 2022] PUT /identity/v3/projects/539a73abcc1040c19db7f9252ea8c2b0/users/bb3fb0936c554fa882e46ba190d3b362/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:21.615813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94cef4-3f8a-44b9-9af1-a5b992a64e6c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:21.643786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e51e3673-64d9-43d7-9113-ee7dd7ae200c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c40d2ad8d084a3d832e3de6ea7aff74', 'request_id': 'req-e51e3673-64d9-43d7-9113-ee7dd7ae200c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:21.652371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e51e3673-64d9-43d7-9113-ee7dd7ae200c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c40d2ad8d084a3d832e3de6ea7aff74', 'request_id': 'req-e51e3673-64d9-43d7-9113-ee7dd7ae200c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:21.685877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94cef4-3f8a-44b9-9af1-a5b992a64e6c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:21.755079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b28bb8-4b30-4e57-963c-11dd2018fe69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:21.756108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b28bb8-4b30-4e57-963c-11dd2018fe69 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:21.756353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b28bb8-4b30-4e57-963c-11dd2018fe69 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:21.756632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b28bb8-4b30-4e57-963c-11dd2018fe69 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:21.757930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b28bb8-4b30-4e57-963c-11dd2018fe69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:21.759335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b28bb8-4b30-4e57-963c-11dd2018fe69 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:21.772376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e51e3673-64d9-43d7-9113-ee7dd7ae200c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c40d2ad8d084a3d832e3de6ea7aff74', 'request_id': 'req-e51e3673-64d9-43d7-9113-ee7dd7ae200c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:21.778371 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2506/5031] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:21.781141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee8430-3a3e-4c74-860b-1849cd7bc16b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:21.781628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee8430-3a3e-4c74-860b-1849cd7bc16b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:21.782136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee8430-3a3e-4c74-860b-1849cd7bc16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:21.792934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e51e3673-64d9-43d7-9113-ee7dd7ae200c admin admin] The token cache is being invalidate because project 6c40d2ad8d084a3d832e3de6ea7aff74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:21.792934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e51e3673-64d9-43d7-9113-ee7dd7ae200c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:21.793716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e51e3673-64d9-43d7-9113-ee7dd7ae200c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:21.796052 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2526/5032] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:21 2022] DELETE /identity/v3/projects/6c40d2ad8d084a3d832e3de6ea7aff74 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:21.801048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e3673-64d9-43d7-9113-ee7dd7ae200c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:21.807282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e51e3673-64d9-43d7-9113-ee7dd7ae200c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:21.821649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7bee8430-3a3e-4c74-860b-1849cd7bc16b None None] MFA Rules not processed for user `bb3fb0936c554fa882e46ba190d3b362`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:21.823061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7bee8430-3a3e-4c74-860b-1849cd7bc16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:21.827264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e3673-64d9-43d7-9113-ee7dd7ae200c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:21.836629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a780d59-4a38-4a7d-89f6-760c2ab2f645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:21.837689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a780d59-4a38-4a7d-89f6-760c2ab2f645 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:21.838113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a780d59-4a38-4a7d-89f6-760c2ab2f645 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:21.838508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a780d59-4a38-4a7d-89f6-760c2ab2f645 admin admin] PATH_INFO: `/v3/projects/0f6041dc987549cfb48003c2b424ada5/users/661333ccdbc74a96ac12d051ad3c2279/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:21.859185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a780d59-4a38-4a7d-89f6-760c2ab2f645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f6041dc987549cfb48003c2b424ada5, user_id=661333ccdbc74a96ac12d051ad3c2279, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:21.863017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a780d59-4a38-4a7d-89f6-760c2ab2f645 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:21.915228 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2507/5033] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:21 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:21.915228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edc74e-505e-4288-adeb-32f188e37b17 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:21.915228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edc74e-505e-4288-adeb-32f188e37b17 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:21.915228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edc74e-505e-4288-adeb-32f188e37b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:21.942791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-65edc74e-505e-4288-adeb-32f188e37b17 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:21.944043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-65edc74e-505e-4288-adeb-32f188e37b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:21.959344 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2508/5034] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:21.959344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edc74e-505e-4288-adeb-32f188e37b17 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:21.973976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edc74e-505e-4288-adeb-32f188e37b17 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:21.984451 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2527/5035] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:21 2022] PUT /identity/v3/projects/0f6041dc987549cfb48003c2b424ada5/users/661333ccdbc74a96ac12d051ad3c2279/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:21.986739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ae21b-6965-4283-ae8f-effd5585cfba None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:21.987258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ae21b-6965-4283-ae8f-effd5585cfba None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:21.987759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ae21b-6965-4283-ae8f-effd5585cfba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:22.027679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-254ae21b-6965-4283-ae8f-effd5585cfba None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:22.030976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-254ae21b-6965-4283-ae8f-effd5585cfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:22.077094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18f4e7-430e-4634-af6a-c329156872c4 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:22.079104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18f4e7-430e-4634-af6a-c329156872c4 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:22.079104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18f4e7-430e-4634-af6a-c329156872c4 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:22.079104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18f4e7-430e-4634-af6a-c329156872c4 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:22.080009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18f4e7-430e-4634-af6a-c329156872c4 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:22.080998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a18f4e7-430e-4634-af6a-c329156872c4 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:22.187371 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2528/5036] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:22.187755 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2509/5037] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:21 2022] POST /identity/v3/services => generated 288 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:22.187860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829d7a0-5973-4105-9604-79ee8beaf1ed None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:22.187860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829d7a0-5973-4105-9604-79ee8beaf1ed None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:22.187860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9829d7a0-5973-4105-9604-79ee8beaf1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:22.189166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18f4e7-430e-4634-af6a-c329156872c4 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:22.239736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a18f4e7-430e-4634-af6a-c329156872c4 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:22.245220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9829d7a0-5973-4105-9604-79ee8beaf1ed None None] MFA Rules not processed for user `661333ccdbc74a96ac12d051ad3c2279`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:22.246585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9829d7a0-5973-4105-9604-79ee8beaf1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:22.255066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8417f1-dd6f-40c1-8646-0f716b4f0d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:22.255856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8417f1-dd6f-40c1-8646-0f716b4f0d7d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:22.256174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8417f1-dd6f-40c1-8646-0f716b4f0d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:22.256463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8417f1-dd6f-40c1-8646-0f716b4f0d7d admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:22.257211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8417f1-dd6f-40c1-8646-0f716b4f0d7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:22.257527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8417f1-dd6f-40c1-8646-0f716b4f0d7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:22.259205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8417f1-dd6f-40c1-8646-0f716b4f0d7d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:22.294302 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2510/5038] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:22.295307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8417f1-dd6f-40c1-8646-0f716b4f0d7d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:22.334260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8417f1-dd6f-40c1-8646-0f716b4f0d7d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:22.340505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6266e794-9103-4047-92b1-8188a1f45544 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:22.341179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266e794-9103-4047-92b1-8188a1f45544 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:22.341471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266e794-9103-4047-92b1-8188a1f45544 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:22.341759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6266e794-9103-4047-92b1-8188a1f45544 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:22.342678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6266e794-9103-4047-92b1-8188a1f45544 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:22.345866 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-6266e794-9103-4047-92b1-8188a1f45544 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:22.345866 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:19:22.345866 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:22.345866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6266e794-9103-4047-92b1-8188a1f45544 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:22.357169 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2529/5039] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:22 2022] POST /identity/v3/auth/tokens => generated 3401 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:22.358331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829d7a0-5973-4105-9604-79ee8beaf1ed None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:22.372264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9829d7a0-5973-4105-9604-79ee8beaf1ed None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:22.378370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87814a5-3097-4e32-9ee7-28040137495a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:22.379149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87814a5-3097-4e32-9ee7-28040137495a admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:22.379419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87814a5-3097-4e32-9ee7-28040137495a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:22.379686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87814a5-3097-4e32-9ee7-28040137495a admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:22.380401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87814a5-3097-4e32-9ee7-28040137495a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:22.382537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87814a5-3097-4e32-9ee7-28040137495a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:22.477553 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2511/5040] 10.222.0.33 () {60 vars in 1212 bytes} [Sat May 28 21:19:22 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 333 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:22.478903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6266e794-9103-4047-92b1-8188a1f45544 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:22.495810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6266e794-9103-4047-92b1-8188a1f45544 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:22.502735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7587492-1fbd-4284-8ba9-594d1b30c185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:22.503422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7587492-1fbd-4284-8ba9-594d1b30c185 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:22.506183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7587492-1fbd-4284-8ba9-594d1b30c185 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:22.506183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7587492-1fbd-4284-8ba9-594d1b30c185 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:22.506183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7587492-1fbd-4284-8ba9-594d1b30c185 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:22.506709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7587492-1fbd-4284-8ba9-594d1b30c185 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:22.534308 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2530/5041] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:22 2022] POST /identity/v3/projects => generated 393 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:22.534308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87814a5-3097-4e32-9ee7-28040137495a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:22.640083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87814a5-3097-4e32-9ee7-28040137495a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:22.650690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21123e9-6491-4715-bf4f-c871d55c3387 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:22.652468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21123e9-6491-4715-bf4f-c871d55c3387 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:22.653960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21123e9-6491-4715-bf4f-c871d55c3387 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:22.654348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21123e9-6491-4715-bf4f-c871d55c3387 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:22.655264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21123e9-6491-4715-bf4f-c871d55c3387 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:22.658619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21123e9-6491-4715-bf4f-c871d55c3387 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:22.671356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:22.680412 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2512/5042] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:22 2022] POST /identity/v3/projects => generated 401 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:22.687531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7587492-1fbd-4284-8ba9-594d1b30c185 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:22.841098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7587492-1fbd-4284-8ba9-594d1b30c185 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:22.847350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d19d15-9a99-4d42-9261-a4df7f6ef30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:22.851605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d19d15-9a99-4d42-9261-a4df7f6ef30e admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:22.851605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d19d15-9a99-4d42-9261-a4df7f6ef30e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:22.851605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d19d15-9a99-4d42-9261-a4df7f6ef30e admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:22.851605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d19d15-9a99-4d42-9261-a4df7f6ef30e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:22.852707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d19d15-9a99-4d42-9261-a4df7f6ef30e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:22.856679 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2531/5043] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:22 2022] POST /identity/v3/services => generated 287 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:22.865884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21123e9-6491-4715-bf4f-c871d55c3387 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:22.986193 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2513/5044] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:22 2022] POST /identity/v3/users => generated 297 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:22.990890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d19d15-9a99-4d42-9261-a4df7f6ef30e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:23.030943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21123e9-6491-4715-bf4f-c871d55c3387 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:23.045087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd4ec7-4bd7-40d2-8fe7-89cea790c197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:23.045936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd4ec7-4bd7-40d2-8fe7-89cea790c197 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:23.046304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd4ec7-4bd7-40d2-8fe7-89cea790c197 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:23.046655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd4ec7-4bd7-40d2-8fe7-89cea790c197 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:23.047605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd4ec7-4bd7-40d2-8fe7-89cea790c197 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:23.051712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd4ec7-4bd7-40d2-8fe7-89cea790c197 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:23.107418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:23.145871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d19d15-9a99-4d42-9261-a4df7f6ef30e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:23.177883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58caec-92b4-4df7-9e1a-22a02cd37b2d tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:23.178832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58caec-92b4-4df7-9e1a-22a02cd37b2d tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:23.179228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58caec-92b4-4df7-9e1a-22a02cd37b2d tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:23.179622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58caec-92b4-4df7-9e1a-22a02cd37b2d tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:23.180471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58caec-92b4-4df7-9e1a-22a02cd37b2d tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:23.183226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58caec-92b4-4df7-9e1a-22a02cd37b2d tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:23.197711 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2532/5045] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:22 2022] POST /identity/v3/users => generated 307 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:23.199153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd4ec7-4bd7-40d2-8fe7-89cea790c197 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:23.212708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd4ec7-4bd7-40d2-8fe7-89cea790c197 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:23.219740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06940bd-cf6c-45c0-a7dd-5f478b05f6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:23.220750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06940bd-cf6c-45c0-a7dd-5f478b05f6a1 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:23.221211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06940bd-cf6c-45c0-a7dd-5f478b05f6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:23.222652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06940bd-cf6c-45c0-a7dd-5f478b05f6a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:23.223584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06940bd-cf6c-45c0-a7dd-5f478b05f6a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:23.226080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06940bd-cf6c-45c0-a7dd-5f478b05f6a1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:23.239823 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2533/5046] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:23.240980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06940bd-cf6c-45c0-a7dd-5f478b05f6a1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:23.258006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06940bd-cf6c-45c0-a7dd-5f478b05f6a1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:23.267687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52c02c-62f8-4981-9472-ce7a405466e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:23.269207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52c02c-62f8-4981-9472-ce7a405466e2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:23.270569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52c02c-62f8-4981-9472-ce7a405466e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:23.271353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea52c02c-62f8-4981-9472-ce7a405466e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:23.272578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52c02c-62f8-4981-9472-ce7a405466e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:23.275019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea52c02c-62f8-4981-9472-ce7a405466e2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:23.294533 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2534/5047] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:23.297124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52c02c-62f8-4981-9472-ce7a405466e2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:23.308962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea52c02c-62f8-4981-9472-ce7a405466e2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:23.314824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b43743-7592-4c35-a209-4724ba03b37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:23.315779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b43743-7592-4c35-a209-4724ba03b37c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:23.316077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b43743-7592-4c35-a209-4724ba03b37c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:23.316367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b43743-7592-4c35-a209-4724ba03b37c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:23.320245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b43743-7592-4c35-a209-4724ba03b37c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:23.320385 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2514/5048] 10.222.0.33 () {60 vars in 1212 bytes} [Sat May 28 21:19:22 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 334 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:23.320385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58caec-92b4-4df7-9e1a-22a02cd37b2d tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:23.323574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b43743-7592-4c35-a209-4724ba03b37c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:23.337684 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2535/5049] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:23.338982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b43743-7592-4c35-a209-4724ba03b37c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:23.343212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58caec-92b4-4df7-9e1a-22a02cd37b2d tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:23.353243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290cbbc-5596-4a2f-8033-bcbb28de684e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:23.354400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290cbbc-5596-4a2f-8033-bcbb28de684e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:23.354962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290cbbc-5596-4a2f-8033-bcbb28de684e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:23.355245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290cbbc-5596-4a2f-8033-bcbb28de684e admin admin] PATH_INFO: `/v3/projects/116aaa32b380456498ac7043d5999da8/users/aa6e6c982a344993aa21d353775fefb6/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:23.360017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b43743-7592-4c35-a209-4724ba03b37c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:23.366398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcc54b-a643-409a-adb3-0604d06d61b2 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:23.367164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcc54b-a643-409a-adb3-0604d06d61b2 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:23.367567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcc54b-a643-409a-adb3-0604d06d61b2 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:23.367926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcc54b-a643-409a-adb3-0604d06d61b2 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:23.368722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcc54b-a643-409a-adb3-0604d06d61b2 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:23.369825 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-b3bcc54b-a643-409a-adb3-0604d06d61b2 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoint_groups": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:23.369825 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:19:23.369825 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:19:23.370866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcc54b-a643-409a-adb3-0604d06d61b2 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:23.378546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c290cbbc-5596-4a2f-8033-bcbb28de684e admin admin] RBAC: Authorizing `identity:create_grant(project_id=116aaa32b380456498ac7043d5999da8, user_id=aa6e6c982a344993aa21d353775fefb6, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:23.386034 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2536/5050] 10.222.0.33 () {58 vars in 1190 bytes} [Sat May 28 21:19:23 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 765 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:23.386034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcc54b-a643-409a-adb3-0604d06d61b2 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:23.390528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c290cbbc-5596-4a2f-8033-bcbb28de684e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:23.398218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcc54b-a643-409a-adb3-0604d06d61b2 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:23.403310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1ad45-c741-41f8-b433-38b7e051079c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:23.404335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1ad45-c741-41f8-b433-38b7e051079c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:23.404621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1ad45-c741-41f8-b433-38b7e051079c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:23.405018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1ad45-c741-41f8-b433-38b7e051079c admin admin] PATH_INFO: `/v3/projects/59224c8f992543a5ae5f4a4d27946bc2/users/f2326ded365e4f549f582704b412223a/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:23.423719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1ad45-c741-41f8-b433-38b7e051079c admin admin] RBAC: Authorizing `identity:create_grant(project_id=59224c8f992543a5ae5f4a4d27946bc2, user_id=f2326ded365e4f549f582704b412223a, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:23.426533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1ad45-c741-41f8-b433-38b7e051079c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:23.473659 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2515/5051] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:23 2022] PUT /identity/v3/projects/116aaa32b380456498ac7043d5999da8/users/aa6e6c982a344993aa21d353775fefb6/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:23.474722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290cbbc-5596-4a2f-8033-bcbb28de684e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:23.519689 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2537/5052] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:23 2022] PUT /identity/v3/projects/59224c8f992543a5ae5f4a4d27946bc2/users/f2326ded365e4f549f582704b412223a/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:23.520657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1ad45-c741-41f8-b433-38b7e051079c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:23.612470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290cbbc-5596-4a2f-8033-bcbb28de684e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:23.694274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1ad45-c741-41f8-b433-38b7e051079c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:23.703214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d84def-8bc4-4bb8-a342-4899a1e36c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:23.704187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d84def-8bc4-4bb8-a342-4899a1e36c74 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:23.704624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d84def-8bc4-4bb8-a342-4899a1e36c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:23.705045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d84def-8bc4-4bb8-a342-4899a1e36c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:23.705797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d84def-8bc4-4bb8-a342-4899a1e36c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:23.707969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d84def-8bc4-4bb8-a342-4899a1e36c74 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:23.719134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6010a0-e0b5-43f3-85f5-a502230d8a7b tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:23.725336 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2538/5053] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:23.733358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bac6c-2ed6-41f8-a8a2-c3d2aefa2a11 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:23.733358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bac6c-2ed6-41f8-a8a2-c3d2aefa2a11 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:23.733358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bac6c-2ed6-41f8-a8a2-c3d2aefa2a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:23.739743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6010a0-e0b5-43f3-85f5-a502230d8a7b tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:23.740428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6010a0-e0b5-43f3-85f5-a502230d8a7b tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:23.740728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6010a0-e0b5-43f3-85f5-a502230d8a7b tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/dd51c773a55a43b080d3c922b4a1fd3b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:23.744335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6010a0-e0b5-43f3-85f5-a502230d8a7b tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=dd51c773a55a43b080d3c922b4a1fd3b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:23.745459 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-8d6010a0-e0b5-43f3-85f5-a502230d8a7b tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:23.745459 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:19:23.745459 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:23.749070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6010a0-e0b5-43f3-85f5-a502230d8a7b tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:23.769420 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2516/5054] 10.222.0.33 () {58 vars in 1289 bytes} [Sat May 28 21:19:23 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups/dd51c773a55a43b080d3c922b4a1fd3b => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:23.770594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6010a0-e0b5-43f3-85f5-a502230d8a7b tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:23.772760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8c7bac6c-2ed6-41f8-a8a2-c3d2aefa2a11 None None] MFA Rules not processed for user `f2326ded365e4f549f582704b412223a`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:23.775852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8c7bac6c-2ed6-41f8-a8a2-c3d2aefa2a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:23.796022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6010a0-e0b5-43f3-85f5-a502230d8a7b tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:23.810157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5b241-3ade-423a-a261-894b158e1253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:23.811090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5b241-3ade-423a-a261-894b158e1253 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:23.816083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5b241-3ade-423a-a261-894b158e1253 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:23.816083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5b241-3ade-423a-a261-894b158e1253 admin admin] PATH_INFO: `/v3/projects/116aaa32b380456498ac7043d5999da8/users/aa6e6c982a344993aa21d353775fefb6/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:23.820893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5b241-3ade-423a-a261-894b158e1253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=116aaa32b380456498ac7043d5999da8, user_id=aa6e6c982a344993aa21d353775fefb6, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:23.823728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5b241-3ade-423a-a261-894b158e1253 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:23.862926 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2517/5055] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:23 2022] PUT /identity/v3/projects/116aaa32b380456498ac7043d5999da8/users/aa6e6c982a344993aa21d353775fefb6/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:23.864114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5b241-3ade-423a-a261-894b158e1253 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:23.963804 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2539/5056] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:23 2022] POST /identity/v3/auth/tokens => generated 3524 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:23.965281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccefaf-fb54-43fb-a101-2d5f31d99d91 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:23.965582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccefaf-fb54-43fb-a101-2d5f31d99d91 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:23.965883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccefaf-fb54-43fb-a101-2d5f31d99d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:24.029927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-99ccefaf-fb54-43fb-a101-2d5f31d99d91 None None] MFA Rules not processed for user `aa6e6c982a344993aa21d353775fefb6`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:24.031182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-99ccefaf-fb54-43fb-a101-2d5f31d99d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:24.041136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5b241-3ade-423a-a261-894b158e1253 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:24.053933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72bc60-54ec-49df-b60b-3b0736709d10 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:24.054955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72bc60-54ec-49df-b60b-3b0736709d10 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `HEAD` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:24.055326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72bc60-54ec-49df-b60b-3b0736709d10 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:24.055665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72bc60-54ec-49df-b60b-3b0736709d10 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/dd51c773a55a43b080d3c922b4a1fd3b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:24.060651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72bc60-54ec-49df-b60b-3b0736709d10 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:get_endpoint_group(endpoint_group_id=dd51c773a55a43b080d3c922b4a1fd3b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:24.065582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac72bc60-54ec-49df-b60b-3b0736709d10 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:24.082878 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2518/5057] 10.222.0.33 () {58 vars in 1290 bytes} [Sat May 28 21:19:23 2022] HEAD /identity/v3/OS-EP-FILTER/endpoint_groups/dd51c773a55a43b080d3c922b4a1fd3b => generated 0 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (0 switches on core 0) May 28 21:19:24.084382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72bc60-54ec-49df-b60b-3b0736709d10 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:24.085956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ac72bc60-54ec-49df-b60b-3b0736709d10 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:24.156691 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2540/5058] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:23 2022] POST /identity/v3/auth/tokens => generated 3599 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:24.159249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccefaf-fb54-43fb-a101-2d5f31d99d91 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:24.160207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-99ccefaf-fb54-43fb-a101-2d5f31d99d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:24.174721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72bc60-54ec-49df-b60b-3b0736709d10 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:24.179607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac322a-fe4a-47cc-ae70-df641b0a7ae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:24.180251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac322a-fe4a-47cc-ae70-df641b0a7ae5 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:24.182153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac322a-fe4a-47cc-ae70-df641b0a7ae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:24.182153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac322a-fe4a-47cc-ae70-df641b0a7ae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:24.247426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccefaf-fb54-43fb-a101-2d5f31d99d91 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:24.250463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac322a-fe4a-47cc-ae70-df641b0a7ae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:24.250463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac322a-fe4a-47cc-ae70-df641b0a7ae5 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:24.252764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728b4fc-acb2-4dd8-999f-1be0b8ff45f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:24.253747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728b4fc-acb2-4dd8-999f-1be0b8ff45f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:24.254158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728b4fc-acb2-4dd8-999f-1be0b8ff45f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:24.254559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728b4fc-acb2-4dd8-999f-1be0b8ff45f1 admin admin] PATH_INFO: `/v3/users/db5d4b639d094820a7cdcfe8d7688c53` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:24.255861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728b4fc-acb2-4dd8-999f-1be0b8ff45f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db5d4b639d094820a7cdcfe8d7688c53)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:24.257934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728b4fc-acb2-4dd8-999f-1be0b8ff45f1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:24.259609 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2519/5059] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:24 2022] GET /identity/v3/auth/tokens => generated 3524 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:24.260282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac322a-fe4a-47cc-ae70-df641b0a7ae5 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:24.272020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac322a-fe4a-47cc-ae70-df641b0a7ae5 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:24.279051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac12464-996c-449c-bfb6-59bc29274eb9 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:24.279051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac12464-996c-449c-bfb6-59bc29274eb9 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:24.279051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac12464-996c-449c-bfb6-59bc29274eb9 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:24.279473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac12464-996c-449c-bfb6-59bc29274eb9 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/dd51c773a55a43b080d3c922b4a1fd3b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:24.280352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac12464-996c-449c-bfb6-59bc29274eb9 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=dd51c773a55a43b080d3c922b4a1fd3b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:24.281519 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-cac12464-996c-449c-bfb6-59bc29274eb9 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:24.281519 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:19:24.281519 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:24.283188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac12464-996c-449c-bfb6-59bc29274eb9 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:24.403241 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2520/5060] 10.222.0.33 () {58 vars in 1292 bytes} [Sat May 28 21:19:24 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/dd51c773a55a43b080d3c922b4a1fd3b => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:24.406882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac12464-996c-449c-bfb6-59bc29274eb9 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:24.409870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cac12464-996c-449c-bfb6-59bc29274eb9 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:24.502739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac12464-996c-449c-bfb6-59bc29274eb9 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:24.507525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc75e53-a4d4-4881-8718-2a3f9a93eef8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:24.509998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc75e53-a4d4-4881-8718-2a3f9a93eef8 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:24.509998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc75e53-a4d4-4881-8718-2a3f9a93eef8 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:24.509998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc75e53-a4d4-4881-8718-2a3f9a93eef8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:24.528668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc75e53-a4d4-4881-8718-2a3f9a93eef8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:24.530633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc75e53-a4d4-4881-8718-2a3f9a93eef8 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:24.543375 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2521/5061] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:24 2022] GET /identity/v3/auth/tokens => generated 3599 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:24.544171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc75e53-a4d4-4881-8718-2a3f9a93eef8 service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:24.572515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc75e53-a4d4-4881-8718-2a3f9a93eef8 service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:24.580675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a728b4fc-acb2-4dd8-999f-1be0b8ff45f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db5d4b639d094820a7cdcfe8d7688c53'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:24.592552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a728b4fc-acb2-4dd8-999f-1be0b8ff45f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db5d4b639d094820a7cdcfe8d7688c53'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:24.593659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a7adc-9cd9-41db-9c07-71d664ab7989 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:24.594351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a7adc-9cd9-41db-9c07-71d664ab7989 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:24.594703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a7adc-9cd9-41db-9c07-71d664ab7989 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:24.594960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1a7adc-9cd9-41db-9c07-71d664ab7989 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:24.595644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a728b4fc-acb2-4dd8-999f-1be0b8ff45f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db5d4b639d094820a7cdcfe8d7688c53'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:24.597296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a728b4fc-acb2-4dd8-999f-1be0b8ff45f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db5d4b639d094820a7cdcfe8d7688c53'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:24.598520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a7adc-9cd9-41db-9c07-71d664ab7989 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:list_endpoint_groups()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:24.600985 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-2c1a7adc-9cd9-41db-9c07-71d664ab7989 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoint_groups": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:24.600985 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:19:24.600985 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:24.603552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1a7adc-9cd9-41db-9c07-71d664ab7989 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:24.611716 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2522/5062] 10.222.0.33 () {58 vars in 1190 bytes} [Sat May 28 21:19:24 2022] GET /identity/v3/OS-EP-FILTER/endpoint_groups => generated 450 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:24.615769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a7adc-9cd9-41db-9c07-71d664ab7989 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:24.619606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2c1a7adc-9cd9-41db-9c07-71d664ab7989 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:24.643502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1a7adc-9cd9-41db-9c07-71d664ab7989 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:24.649697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ce069-1b8e-40c8-abe7-b17844da678d tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:24.651229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ce069-1b8e-40c8-abe7-b17844da678d tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:24.651419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ce069-1b8e-40c8-abe7-b17844da678d tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:24.651807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ce069-1b8e-40c8-abe7-b17844da678d tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/dd51c773a55a43b080d3c922b4a1fd3b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:24.652790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ce069-1b8e-40c8-abe7-b17844da678d tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=dd51c773a55a43b080d3c922b4a1fd3b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:24.654687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ce069-1b8e-40c8-abe7-b17844da678d tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:24.687722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a728b4fc-acb2-4dd8-999f-1be0b8ff45f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db5d4b639d094820a7cdcfe8d7688c53'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:24.698907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-991ce069-1b8e-40c8-abe7-b17844da678d tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Could not find Endpoint Group: dd51c773a55a43b080d3c922b4a1fd3b.: keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: dd51c773a55a43b080d3c922b4a1fd3b. May 28 21:19:24.698907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:19:24.698907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:19:24.698907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:19:24.698907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:19:24.698907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:19:24.698907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:19:24.698907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:19:24.698907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:19:24.698907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:19:24.698907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:19:24.698907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:19:24.698907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_ep_filter.py", line 107, in delete May 28 21:19:24.698907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return (PROVIDERS.catalog_api.delete_endpoint_group(endpoint_group_id), May 28 21:19:24.700249 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 519, in delete_endpoint_group May 28 21:19:24.700249 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application endpoint_group_ref = self._get_endpoint_group(session, May 28 21:19:24.700249 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 495, in _get_endpoint_group May 28 21:19:24.700249 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.EndpointGroupNotFound( May 28 21:19:24.700249 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.EndpointGroupNotFound: Could not find Endpoint Group: dd51c773a55a43b080d3c922b4a1fd3b. May 28 21:19:24.700249 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:19:24.710272 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2523/5063] 10.222.0.33 () {58 vars in 1292 bytes} [Sat May 28 21:19:24 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/dd51c773a55a43b080d3c922b4a1fd3b => generated 120 bytes in 93 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:19:24.710684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ce069-1b8e-40c8-abe7-b17844da678d tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:24.732955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ce069-1b8e-40c8-abe7-b17844da678d tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:24.740919 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2541/5064] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:24 2022] DELETE /identity/v3/users/db5d4b639d094820a7cdcfe8d7688c53 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:24.743926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30cfd5-7e43-40ee-beb3-0e62bf578cf2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:24.744801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30cfd5-7e43-40ee-beb3-0e62bf578cf2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:24.745508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30cfd5-7e43-40ee-beb3-0e62bf578cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:24.748771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bd897-1777-4b87-a7b1-09337ee614b0 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:24.751326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bd897-1777-4b87-a7b1-09337ee614b0 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:24.752700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bd897-1777-4b87-a7b1-09337ee614b0 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:24.753865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5bd897-1777-4b87-a7b1-09337ee614b0 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/services/0f35eac01f554ec29a8c2786acb44da3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:24.754670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5bd897-1777-4b87-a7b1-09337ee614b0 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:delete_service(service_id=0f35eac01f554ec29a8c2786acb44da3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:24.767240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5bd897-1777-4b87-a7b1-09337ee614b0 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:24.781009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7e30cfd5-7e43-40ee-beb3-0e62bf578cf2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:24.784477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7e30cfd5-7e43-40ee-beb3-0e62bf578cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:24.882556 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2542/5065] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:24 2022] POST /identity/v3/auth/tokens => generated 3493 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:24.884082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30cfd5-7e43-40ee-beb3-0e62bf578cf2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:24.885670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7e30cfd5-7e43-40ee-beb3-0e62bf578cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:24.925662 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2524/5066] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:24 2022] DELETE /identity/v3/services/0f35eac01f554ec29a8c2786acb44da3 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:24.926452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bd897-1777-4b87-a7b1-09337ee614b0 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:24.928426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bb5bd897-1777-4b87-a7b1-09337ee614b0 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:24.991700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5bd897-1777-4b87-a7b1-09337ee614b0 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:24.994178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30cfd5-7e43-40ee-beb3-0e62bf578cf2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:25.017291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d7366-9840-4628-9596-bdd492ca6eaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:25.022811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d7366-9840-4628-9596-bdd492ca6eaf service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:25.022811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d7366-9840-4628-9596-bdd492ca6eaf service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:25.022811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d7366-9840-4628-9596-bdd492ca6eaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:25.022811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b83d7366-9840-4628-9596-bdd492ca6eaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:25.075242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:25.086059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c2916-eb8f-48f6-bd84-9edb9a64cb6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:25.088253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c2916-eb8f-48f6-bd84-9edb9a64cb6d service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:25.088649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c2916-eb8f-48f6-bd84-9edb9a64cb6d service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:25.089060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c2916-eb8f-48f6-bd84-9edb9a64cb6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:25.108416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c2916-eb8f-48f6-bd84-9edb9a64cb6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:25.111712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695c2916-eb8f-48f6-bd84-9edb9a64cb6d service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:25.139579 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2525/5067] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:25.140443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c2916-eb8f-48f6-bd84-9edb9a64cb6d service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:25.154084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d7366-9840-4628-9596-bdd492ca6eaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:25.156063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d7366-9840-4628-9596-bdd492ca6eaf service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:25.179148 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2543/5068] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:24 2022] GET /identity/v3/auth/tokens => generated 3401 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:25.292402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c2916-eb8f-48f6-bd84-9edb9a64cb6d service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:25.299417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5e006-0156-4de9-892c-047f35f3b749 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:25.300727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5e006-0156-4de9-892c-047f35f3b749 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:25.301036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5e006-0156-4de9-892c-047f35f3b749 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:25.301383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f5e006-0156-4de9-892c-047f35f3b749 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:25.302098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5e006-0156-4de9-892c-047f35f3b749 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:25.303793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f5e006-0156-4de9-892c-047f35f3b749 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:25.374121 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2526/5069] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:25 2022] POST /identity/v3/services => generated 285 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:25.375327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d7366-9840-4628-9596-bdd492ca6eaf service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:25.393953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d7366-9840-4628-9596-bdd492ca6eaf service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:25.434583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01251e-2e3e-4a91-8ed9-3090765ea837 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:25.435880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01251e-2e3e-4a91-8ed9-3090765ea837 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:25.436179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01251e-2e3e-4a91-8ed9-3090765ea837 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:25.436466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f01251e-2e3e-4a91-8ed9-3090765ea837 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:25.437493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01251e-2e3e-4a91-8ed9-3090765ea837 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:create_endpoint_group()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:25.438604 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-7f01251e-2e3e-4a91-8ed9-3090765ea837 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:25.438604 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:19:25.438604 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:19:25.439520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f01251e-2e3e-4a91-8ed9-3090765ea837 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:25.516838 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2544/5070] 10.222.0.33 () {60 vars in 1212 bytes} [Sat May 28 21:19:25 2022] POST /identity/v3/OS-EP-FILTER/endpoint_groups => generated 333 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:25.517812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5e006-0156-4de9-892c-047f35f3b749 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:25.529126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f5e006-0156-4de9-892c-047f35f3b749 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:25.534909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51106312-07a9-4a66-bc8c-823da538fd3c tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:25.535584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51106312-07a9-4a66-bc8c-823da538fd3c tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:25.535876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51106312-07a9-4a66-bc8c-823da538fd3c tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:25.536166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51106312-07a9-4a66-bc8c-823da538fd3c tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:25.536946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51106312-07a9-4a66-bc8c-823da538fd3c tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:25.538650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51106312-07a9-4a66-bc8c-823da538fd3c tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:25.644571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01251e-2e3e-4a91-8ed9-3090765ea837 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:25.647948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7f01251e-2e3e-4a91-8ed9-3090765ea837 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:25.688198 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2527/5071] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:25 2022] POST /identity/v3/services => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:25.692462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51106312-07a9-4a66-bc8c-823da538fd3c tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:25.721509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51106312-07a9-4a66-bc8c-823da538fd3c tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:25.723461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f01251e-2e3e-4a91-8ed9-3090765ea837 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:25.726624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a06be5-0f4c-42a6-9985-da40b101964c tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:25.727496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a06be5-0f4c-42a6-9985-da40b101964c tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:25.727836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a06be5-0f4c-42a6-9985-da40b101964c tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:25.728061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a06be5-0f4c-42a6-9985-da40b101964c tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/b6ebd1944db84990b6ef11119fdc98c2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:25.728880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a06be5-0f4c-42a6-9985-da40b101964c tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:update_endpoint_group(endpoint_group_id=b6ebd1944db84990b6ef11119fdc98c2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:25.730227 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-97a06be5-0f4c-42a6-9985-da40b101964c tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_endpoint_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:25.730227 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:19:25.730227 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:25.731094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a06be5-0f4c-42a6-9985-da40b101964c tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:25.743936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36878a-371b-412b-86df-ffd2164d9a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:25.745001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36878a-371b-412b-86df-ffd2164d9a20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:25.746380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36878a-371b-412b-86df-ffd2164d9a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:25.746380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36878a-371b-412b-86df-ffd2164d9a20 admin admin] PATH_INFO: `/v3/projects/56c1d2ac8d054765968184cfa5494f67` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:25.747176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36878a-371b-412b-86df-ffd2164d9a20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56c1d2ac8d054765968184cfa5494f67)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:25.749123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb36878a-371b-412b-86df-ffd2164d9a20 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:25.966616 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2528/5072] 10.222.0.33 () {60 vars in 1312 bytes} [Sat May 28 21:19:25 2022] PATCH /identity/v3/OS-EP-FILTER/endpoint_groups/b6ebd1944db84990b6ef11119fdc98c2 => generated 335 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:25.968450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a06be5-0f4c-42a6-9985-da40b101964c tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:25.996064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a06be5-0f4c-42a6-9985-da40b101964c tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:26.006419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bb36878a-371b-412b-86df-ffd2164d9a20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56c1d2ac8d054765968184cfa5494f67', 'request_id': 'req-bb36878a-371b-412b-86df-ffd2164d9a20'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:26.012693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bb36878a-371b-412b-86df-ffd2164d9a20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56c1d2ac8d054765968184cfa5494f67', 'request_id': 'req-bb36878a-371b-412b-86df-ffd2164d9a20'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:26.060018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e53c7f-0e77-4e7f-8703-9334ac042108 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:26.060889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e53c7f-0e77-4e7f-8703-9334ac042108 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:26.061192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e53c7f-0e77-4e7f-8703-9334ac042108 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:26.063969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e53c7f-0e77-4e7f-8703-9334ac042108 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/services/98a92ce14295446d898ce54022415c52` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:26.064712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e53c7f-0e77-4e7f-8703-9334ac042108 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:delete_service(service_id=98a92ce14295446d898ce54022415c52)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:26.066519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e53c7f-0e77-4e7f-8703-9334ac042108 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:26.184788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bb36878a-371b-412b-86df-ffd2164d9a20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56c1d2ac8d054765968184cfa5494f67', 'request_id': 'req-bb36878a-371b-412b-86df-ffd2164d9a20'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:26.194668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:26.207028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bb36878a-371b-412b-86df-ffd2164d9a20 admin admin] The token cache is being invalidate because project 56c1d2ac8d054765968184cfa5494f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:26.207329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bb36878a-371b-412b-86df-ffd2164d9a20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:26.208733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bb36878a-371b-412b-86df-ffd2164d9a20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:26.212131 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2545/5073] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:25 2022] DELETE /identity/v3/projects/56c1d2ac8d054765968184cfa5494f67 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:26.222340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01796833-c237-4a8c-85ed-93c97cf65298 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:26.222633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01796833-c237-4a8c-85ed-93c97cf65298 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:26.222912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01796833-c237-4a8c-85ed-93c97cf65298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:26.245873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-01796833-c237-4a8c-85ed-93c97cf65298 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:26.247139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-01796833-c237-4a8c-85ed-93c97cf65298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:26.288330 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2529/5074] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:25 2022] DELETE /identity/v3/services/98a92ce14295446d898ce54022415c52 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:26.293106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e53c7f-0e77-4e7f-8703-9334ac042108 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:26.295281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a4e53c7f-0e77-4e7f-8703-9334ac042108 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:26.329251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e53c7f-0e77-4e7f-8703-9334ac042108 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:26.336454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d1a50-8ec4-41d4-9610-5487c93a0b91 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:26.337715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d1a50-8ec4-41d4-9610-5487c93a0b91 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:26.338555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d1a50-8ec4-41d4-9610-5487c93a0b91 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:26.338947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d1a50-8ec4-41d4-9610-5487c93a0b91 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/b6ebd1944db84990b6ef11119fdc98c2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:26.340990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d1a50-8ec4-41d4-9610-5487c93a0b91 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=b6ebd1944db84990b6ef11119fdc98c2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:26.343185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d1a50-8ec4-41d4-9610-5487c93a0b91 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:26.370062 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2546/5075] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:26 2022] POST /identity/v3/auth/tokens => generated 3494 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:26.450665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e45a15-c63d-44ce-97e7-ba7286d06c44 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:26.452130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e45a15-c63d-44ce-97e7-ba7286d06c44 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:26.452791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e45a15-c63d-44ce-97e7-ba7286d06c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:26.479345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-10e45a15-c63d-44ce-97e7-ba7286d06c44 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:26.481777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-10e45a15-c63d-44ce-97e7-ba7286d06c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:26.506184 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2530/5076] 10.222.0.33 () {58 vars in 1292 bytes} [Sat May 28 21:19:26 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/b6ebd1944db84990b6ef11119fdc98c2 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:26.506673 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2547/5077] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:26 2022] POST /identity/v3/auth/tokens => generated 3494 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:26.511153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d1a50-8ec4-41d4-9610-5487c93a0b91 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:26.527265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e45a15-c63d-44ce-97e7-ba7286d06c44 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:26.538703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d1a50-8ec4-41d4-9610-5487c93a0b91 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:26.548095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a4fa9-4c8e-403e-ac1d-1c99d3c721b2 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:26.548768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e45a15-c63d-44ce-97e7-ba7286d06c44 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:26.549260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a4fa9-4c8e-403e-ac1d-1c99d3c721b2 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:26.549692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a4fa9-4c8e-403e-ac1d-1c99d3c721b2 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:26.550140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a4fa9-4c8e-403e-ac1d-1c99d3c721b2 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/services/8d528b4cff0142098ae51345442b6aec` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:26.551129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a4fa9-4c8e-403e-ac1d-1c99d3c721b2 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:delete_service(service_id=8d528b4cff0142098ae51345442b6aec)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:26.553058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a4fa9-4c8e-403e-ac1d-1c99d3c721b2 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:26.572396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb360c65-0ca6-4b82-8392-83f3a34396e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:26.573081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb360c65-0ca6-4b82-8392-83f3a34396e7 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:26.573382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb360c65-0ca6-4b82-8392-83f3a34396e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:26.573635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb360c65-0ca6-4b82-8392-83f3a34396e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:26.574475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb360c65-0ca6-4b82-8392-83f3a34396e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:26.574782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb360c65-0ca6-4b82-8392-83f3a34396e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:26.576401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb360c65-0ca6-4b82-8392-83f3a34396e7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:26.622640 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2548/5078] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:26.623984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb360c65-0ca6-4b82-8392-83f3a34396e7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:26.645018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb360c65-0ca6-4b82-8392-83f3a34396e7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:26.653156 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2531/5079] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:26 2022] DELETE /identity/v3/services/8d528b4cff0142098ae51345442b6aec => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:26.654719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a4fa9-4c8e-403e-ac1d-1c99d3c721b2 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:26.675945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa7871-f9f2-429d-9371-9b57762f815e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:26.677076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa7871-f9f2-429d-9371-9b57762f815e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:26.677556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa7871-f9f2-429d-9371-9b57762f815e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:26.683726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa7871-f9f2-429d-9371-9b57762f815e admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:26.683726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa7871-f9f2-429d-9371-9b57762f815e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:26.683726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa7871-f9f2-429d-9371-9b57762f815e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:26.690617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a4fa9-4c8e-403e-ac1d-1c99d3c721b2 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:26.702509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de980ff2-c9cf-44fd-8f03-5365740e4d15 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:26.703635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de980ff2-c9cf-44fd-8f03-5365740e4d15 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:26.704011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de980ff2-c9cf-44fd-8f03-5365740e4d15 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:26.704489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de980ff2-c9cf-44fd-8f03-5365740e4d15 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/OS-EP-FILTER/endpoint_groups/5b3d3b80c8ff45bfb8b38e6310ec44bf` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:26.705396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de980ff2-c9cf-44fd-8f03-5365740e4d15 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:delete_endpoint_group(endpoint_group_id=5b3d3b80c8ff45bfb8b38e6310ec44bf)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:26.709078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de980ff2-c9cf-44fd-8f03-5365740e4d15 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:26.921914 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2532/5080] 10.222.0.33 () {58 vars in 1292 bytes} [Sat May 28 21:19:26 2022] DELETE /identity/v3/OS-EP-FILTER/endpoint_groups/5b3d3b80c8ff45bfb8b38e6310ec44bf => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:26.926289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de980ff2-c9cf-44fd-8f03-5365740e4d15 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:26.930717 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2549/5081] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:26 2022] POST /identity/v3/projects => generated 365 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:26.934344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa7871-f9f2-429d-9371-9b57762f815e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:26.937452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de980ff2-c9cf-44fd-8f03-5365740e4d15 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:27.007264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1043a63-ebe3-4060-9382-58c480c9c413 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb3fb0936c554fa882e46ba190d3b362', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '539a73abcc1040c19db7f9252ea8c2b0', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:27.008920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1043a63-ebe3-4060-9382-58c480c9c413 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:27.009367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1043a63-ebe3-4060-9382-58c480c9c413 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:27.009802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1043a63-ebe3-4060-9382-58c480c9c413 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] PATH_INFO: `/v3/services/7c52f7d14c5b41ee93fabf9f4fb5c70e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:27.010704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1043a63-ebe3-4060-9382-58c480c9c413 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorizing `identity:delete_service(service_id=7c52f7d14c5b41ee93fabf9f4fb5c70e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:27.012938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1043a63-ebe3-4060-9382-58c480c9c413 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:27.028736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa7871-f9f2-429d-9371-9b57762f815e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:27.035586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346f97e-b7bc-40c6-a1fb-e11398155c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:27.036602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346f97e-b7bc-40c6-a1fb-e11398155c08 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:27.036770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346f97e-b7bc-40c6-a1fb-e11398155c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:27.037025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346f97e-b7bc-40c6-a1fb-e11398155c08 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:27.038122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346f97e-b7bc-40c6-a1fb-e11398155c08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:27.040112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346f97e-b7bc-40c6-a1fb-e11398155c08 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:27.191944 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2533/5082] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:26 2022] DELETE /identity/v3/services/7c52f7d14c5b41ee93fabf9f4fb5c70e => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:27.192878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1043a63-ebe3-4060-9382-58c480c9c413 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:27.201049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c1043a63-ebe3-4060-9382-58c480c9c413 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:27.206307 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2550/5083] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:26 2022] POST /identity/v3/users => generated 283 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:27.208864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346f97e-b7bc-40c6-a1fb-e11398155c08 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:27.231111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346f97e-b7bc-40c6-a1fb-e11398155c08 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:27.278268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1043a63-ebe3-4060-9382-58c480c9c413 tempest-EndPointGroupsTest-446187330 tempest-EndPointGroupsTest-446187330-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:27.299633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d65ee-a8d5-4124-9802-9b26c9c6c75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:27.300338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d65ee-a8d5-4124-9802-9b26c9c6c75f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:27.300649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d65ee-a8d5-4124-9802-9b26c9c6c75f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:27.300920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d65ee-a8d5-4124-9802-9b26c9c6c75f admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:27.302448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d65ee-a8d5-4124-9802-9b26c9c6c75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:27.306940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c679c6-35c9-40ea-a995-2170f1e52b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:27.308870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c679c6-35c9-40ea-a995-2170f1e52b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:27.309163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c679c6-35c9-40ea-a995-2170f1e52b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:27.312678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c679c6-35c9-40ea-a995-2170f1e52b48 admin admin] PATH_INFO: `/v3/users/c2ae7238114b43ae9b9d1790894ef9a7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:27.315396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d65ee-a8d5-4124-9802-9b26c9c6c75f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:27.329583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c679c6-35c9-40ea-a995-2170f1e52b48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2ae7238114b43ae9b9d1790894ef9a7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:27.329583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c679c6-35c9-40ea-a995-2170f1e52b48 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:27.367822 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2551/5084] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:27.369109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d65ee-a8d5-4124-9802-9b26c9c6c75f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:27.390430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d65ee-a8d5-4124-9802-9b26c9c6c75f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:27.399363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d5f5d-640c-4370-b635-a0b7355e90a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:27.400133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d5f5d-640c-4370-b635-a0b7355e90a7 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:27.400424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d5f5d-640c-4370-b635-a0b7355e90a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:27.400701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d5f5d-640c-4370-b635-a0b7355e90a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:27.401542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d5f5d-640c-4370-b635-a0b7355e90a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:27.403523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86d5f5d-640c-4370-b635-a0b7355e90a7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:27.444075 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2552/5085] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:27 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:19:27.448803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d5f5d-640c-4370-b635-a0b7355e90a7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:27.472312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d5f5d-640c-4370-b635-a0b7355e90a7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:27.497161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55478e0d-955e-4b24-8056-40104aebbde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:27.498131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55478e0d-955e-4b24-8056-40104aebbde2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:27.498611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55478e0d-955e-4b24-8056-40104aebbde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:27.498891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55478e0d-955e-4b24-8056-40104aebbde2 admin admin] PATH_INFO: `/v3/projects/530a8242ae014464b176c523ae99294e/users/cfa39677553343bcaf62286df0ec6379/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:27.524188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55478e0d-955e-4b24-8056-40104aebbde2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=530a8242ae014464b176c523ae99294e, user_id=cfa39677553343bcaf62286df0ec6379, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:27.528503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55478e0d-955e-4b24-8056-40104aebbde2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:27.651960 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2553/5086] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:27 2022] PUT /identity/v3/projects/530a8242ae014464b176c523ae99294e/users/cfa39677553343bcaf62286df0ec6379/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:27.654409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc966f3e-d9cf-4816-a296-1319244ac32e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:27.654771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc966f3e-d9cf-4816-a296-1319244ac32e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:27.655518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc966f3e-d9cf-4816-a296-1319244ac32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:27.655656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-03c679c6-35c9-40ea-a995-2170f1e52b48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2ae7238114b43ae9b9d1790894ef9a7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:27.658260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-03c679c6-35c9-40ea-a995-2170f1e52b48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2ae7238114b43ae9b9d1790894ef9a7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:27.675812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-03c679c6-35c9-40ea-a995-2170f1e52b48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2ae7238114b43ae9b9d1790894ef9a7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:27.692796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-dc966f3e-d9cf-4816-a296-1319244ac32e None None] MFA Rules not processed for user `cfa39677553343bcaf62286df0ec6379`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:27.696854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dc966f3e-d9cf-4816-a296-1319244ac32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:27.793535 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2554/5087] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:27.796259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc966f3e-d9cf-4816-a296-1319244ac32e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:27.823020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-03c679c6-35c9-40ea-a995-2170f1e52b48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2ae7238114b43ae9b9d1790894ef9a7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:27.823691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-03c679c6-35c9-40ea-a995-2170f1e52b48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2ae7238114b43ae9b9d1790894ef9a7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:27.852868 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2534/5088] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:27 2022] DELETE /identity/v3/users/c2ae7238114b43ae9b9d1790894ef9a7 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:27.857400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33773f9-5d85-4d31-bf93-8d578759b617 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:27.857690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33773f9-5d85-4d31-bf93-8d578759b617 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:27.858015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33773f9-5d85-4d31-bf93-8d578759b617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:27.886437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a33773f9-5d85-4d31-bf93-8d578759b617 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:27.887960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a33773f9-5d85-4d31-bf93-8d578759b617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:27.893804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc966f3e-d9cf-4816-a296-1319244ac32e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:27.955489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31908919-1cc7-4249-9b4c-39d36a172d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:27.956071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31908919-1cc7-4249-9b4c-39d36a172d9e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:27.956531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31908919-1cc7-4249-9b4c-39d36a172d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:27.956653 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2535/5089] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:27.956771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31908919-1cc7-4249-9b4c-39d36a172d9e admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:27.957394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31908919-1cc7-4249-9b4c-39d36a172d9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:27.964912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31908919-1cc7-4249-9b4c-39d36a172d9e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:27.967022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33773f9-5d85-4d31-bf93-8d578759b617 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:27.969387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a33773f9-5d85-4d31-bf93-8d578759b617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:28.092762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:28.138806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33773f9-5d85-4d31-bf93-8d578759b617 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:28.146804 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2555/5090] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:27 2022] POST /identity/v3/projects => generated 365 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:28.150580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31908919-1cc7-4249-9b4c-39d36a172d9e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:28.155465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-31908919-1cc7-4249-9b4c-39d36a172d9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:28.156293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d176df-2bc6-4cc6-b26c-a931f19b62d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:28.157674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d176df-2bc6-4cc6-b26c-a931f19b62d2 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:28.158009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d176df-2bc6-4cc6-b26c-a931f19b62d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:28.158284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d176df-2bc6-4cc6-b26c-a931f19b62d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:28.198260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d176df-2bc6-4cc6-b26c-a931f19b62d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:28.198260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d176df-2bc6-4cc6-b26c-a931f19b62d2 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:28.215063 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2536/5091] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:28.316285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31908919-1cc7-4249-9b4c-39d36a172d9e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:28.330849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfeb4b-9e58-4b0a-bbf0-6e06d1586682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:28.331920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfeb4b-9e58-4b0a-bbf0-6e06d1586682 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:28.332209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfeb4b-9e58-4b0a-bbf0-6e06d1586682 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:28.333631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cfeb4b-9e58-4b0a-bbf0-6e06d1586682 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:28.334440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfeb4b-9e58-4b0a-bbf0-6e06d1586682 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:28.336895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cfeb4b-9e58-4b0a-bbf0-6e06d1586682 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:28.440702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d176df-2bc6-4cc6-b26c-a931f19b62d2 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:28.453058 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2556/5092] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:28 2022] POST /identity/v3/users => generated 289 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:28.456308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d176df-2bc6-4cc6-b26c-a931f19b62d2 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:28.468116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646ca7e-4615-421f-ac30-ad212b58b6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:28.468910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646ca7e-4615-421f-ac30-ad212b58b6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:28.469359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646ca7e-4615-421f-ac30-ad212b58b6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:28.474930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646ca7e-4615-421f-ac30-ad212b58b6c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:28.474930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646ca7e-4615-421f-ac30-ad212b58b6c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:28.476088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646ca7e-4615-421f-ac30-ad212b58b6c1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:28.494380 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2537/5093] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:28.499880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfeb4b-9e58-4b0a-bbf0-6e06d1586682 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:28.515463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cfeb4b-9e58-4b0a-bbf0-6e06d1586682 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:28.522008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693d659e-aecb-4abb-8241-7d64a9790d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:28.523060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d659e-aecb-4abb-8241-7d64a9790d6e admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:28.523547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d659e-aecb-4abb-8241-7d64a9790d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:28.523899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693d659e-aecb-4abb-8241-7d64a9790d6e admin admin] PATH_INFO: `/v3/projects/629225485fc244deab41c69a42a71ee6/users/89e228766caf482abaee867cc932587f/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:28.582445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d659e-aecb-4abb-8241-7d64a9790d6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=629225485fc244deab41c69a42a71ee6, user_id=89e228766caf482abaee867cc932587f, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:28.586480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693d659e-aecb-4abb-8241-7d64a9790d6e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:28.709164 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2557/5094] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:28 2022] PUT /identity/v3/projects/629225485fc244deab41c69a42a71ee6/users/89e228766caf482abaee867cc932587f/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:28.713142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646ca7e-4615-421f-ac30-ad212b58b6c1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:28.731218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646ca7e-4615-421f-ac30-ad212b58b6c1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:28.836295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2cf6f-5616-4689-9377-9320a20c2cec None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:28.836656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2cf6f-5616-4689-9377-9320a20c2cec None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:28.836944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df2cf6f-5616-4689-9377-9320a20c2cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:28.882957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0df2cf6f-5616-4689-9377-9320a20c2cec None None] MFA Rules not processed for user `857a887b6c7c41588afb2845eed6f309`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:28.885006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0df2cf6f-5616-4689-9377-9320a20c2cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:28.982746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27cb1f-b608-47a2-a8a8-26a3ea5a6510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:28.985596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27cb1f-b608-47a2-a8a8-26a3ea5a6510 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:28.986049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27cb1f-b608-47a2-a8a8-26a3ea5a6510 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:28.987372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27cb1f-b608-47a2-a8a8-26a3ea5a6510 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:28.988334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27cb1f-b608-47a2-a8a8-26a3ea5a6510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:28.990350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27cb1f-b608-47a2-a8a8-26a3ea5a6510 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:29.002893 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2538/5095] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:29.006844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27cb1f-b608-47a2-a8a8-26a3ea5a6510 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:29.011903 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2558/5096] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:29.020829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27cb1f-b608-47a2-a8a8-26a3ea5a6510 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:29.029070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5cbc6f-f092-42d8-8723-280d6430964f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:29.030464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cbc6f-f092-42d8-8723-280d6430964f admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:29.031023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cbc6f-f092-42d8-8723-280d6430964f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:29.031429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2cf6f-5616-4689-9377-9320a20c2cec None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:29.031806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5cbc6f-f092-42d8-8723-280d6430964f admin admin] PATH_INFO: `/v3/projects/629225485fc244deab41c69a42a71ee6/users/89e228766caf482abaee867cc932587f/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:29.034605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5cbc6f-f092-42d8-8723-280d6430964f admin admin] RBAC: Authorizing `identity:create_grant(project_id=629225485fc244deab41c69a42a71ee6, user_id=89e228766caf482abaee867cc932587f, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:29.035509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0df2cf6f-5616-4689-9377-9320a20c2cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:29.039376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5cbc6f-f092-42d8-8723-280d6430964f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:29.064204 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2539/5097] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:29 2022] PUT /identity/v3/projects/629225485fc244deab41c69a42a71ee6/users/89e228766caf482abaee867cc932587f/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:29.075946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291506d3-cfb9-4b04-804f-2b93d32db854 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:29.076492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291506d3-cfb9-4b04-804f-2b93d32db854 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:29.076926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291506d3-cfb9-4b04-804f-2b93d32db854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:29.162157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-291506d3-cfb9-4b04-804f-2b93d32db854 None None] MFA Rules not processed for user `89e228766caf482abaee867cc932587f`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:29.163409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-291506d3-cfb9-4b04-804f-2b93d32db854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:29.197949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df2cf6f-5616-4689-9377-9320a20c2cec None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:29.302424 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2540/5098] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:29 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:29.303482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291506d3-cfb9-4b04-804f-2b93d32db854 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:29.307382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-291506d3-cfb9-4b04-804f-2b93d32db854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:29.314054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900112d-5994-4259-b1a8-033524b3d90b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:29.315906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900112d-5994-4259-b1a8-033524b3d90b service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:29.316453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900112d-5994-4259-b1a8-033524b3d90b service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:29.317764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900112d-5994-4259-b1a8-033524b3d90b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:29.399102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291506d3-cfb9-4b04-804f-2b93d32db854 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:29.401920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900112d-5994-4259-b1a8-033524b3d90b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:29.403676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900112d-5994-4259-b1a8-033524b3d90b service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:29.413980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf538c1f-317b-4ef8-b24a-29529462a9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:29.415134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf538c1f-317b-4ef8-b24a-29529462a9c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:29.416156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf538c1f-317b-4ef8-b24a-29529462a9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:29.416156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf538c1f-317b-4ef8-b24a-29529462a9c8 admin admin] PATH_INFO: `/v3/projects/eff632e61c8c4e469ba91057f99d6b8d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:29.417360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf538c1f-317b-4ef8-b24a-29529462a9c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eff632e61c8c4e469ba91057f99d6b8d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:29.418365 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2559/5099] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:29.419222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900112d-5994-4259-b1a8-033524b3d90b service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:29.420659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3900112d-5994-4259-b1a8-033524b3d90b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:29.434348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf538c1f-317b-4ef8-b24a-29529462a9c8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:29.578113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900112d-5994-4259-b1a8-033524b3d90b service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:29.583922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe980c-8980-41db-a875-56feddde6405 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:29.584655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe980c-8980-41db-a875-56feddde6405 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:29.584950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe980c-8980-41db-a875-56feddde6405 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:29.585276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe980c-8980-41db-a875-56feddde6405 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:29.606374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe980c-8980-41db-a875-56feddde6405 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:29.607855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe980c-8980-41db-a875-56feddde6405 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:29.638750 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2560/5100] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:29 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:29.662076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cf538c1f-317b-4ef8-b24a-29529462a9c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eff632e61c8c4e469ba91057f99d6b8d', 'request_id': 'req-cf538c1f-317b-4ef8-b24a-29529462a9c8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:29.677809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cf538c1f-317b-4ef8-b24a-29529462a9c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eff632e61c8c4e469ba91057f99d6b8d', 'request_id': 'req-cf538c1f-317b-4ef8-b24a-29529462a9c8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:29.778012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cf538c1f-317b-4ef8-b24a-29529462a9c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eff632e61c8c4e469ba91057f99d6b8d', 'request_id': 'req-cf538c1f-317b-4ef8-b24a-29529462a9c8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:29.799570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cf538c1f-317b-4ef8-b24a-29529462a9c8 admin admin] The token cache is being invalidate because project eff632e61c8c4e469ba91057f99d6b8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:29.800138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cf538c1f-317b-4ef8-b24a-29529462a9c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:29.801923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cf538c1f-317b-4ef8-b24a-29529462a9c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:29.804615 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2541/5101] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:29 2022] DELETE /identity/v3/projects/eff632e61c8c4e469ba91057f99d6b8d => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:29.807025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:29.808857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe980c-8980-41db-a875-56feddde6405 service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:29.825167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe980c-8980-41db-a875-56feddde6405 service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:29.937161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e2d23-2b96-4024-b5f9-2a844fc32160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:29.939444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e2d23-2b96-4024-b5f9-2a844fc32160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:29.939444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e2d23-2b96-4024-b5f9-2a844fc32160 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:29.939444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e2d23-2b96-4024-b5f9-2a844fc32160 admin admin] PATH_INFO: `/v3/users/bb3fb0936c554fa882e46ba190d3b362` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:29.940758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e2d23-2b96-4024-b5f9-2a844fc32160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb3fb0936c554fa882e46ba190d3b362)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:29.943391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e2d23-2b96-4024-b5f9-2a844fc32160 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:29.980828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf538c1f-317b-4ef8-b24a-29529462a9c8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:29.982647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cf538c1f-317b-4ef8-b24a-29529462a9c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:30.109699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf538c1f-317b-4ef8-b24a-29529462a9c8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:30.114553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd2988-7d6c-4b58-a40d-b028e78b1c8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:30.115209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd2988-7d6c-4b58-a40d-b028e78b1c8f service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:30.115492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd2988-7d6c-4b58-a40d-b028e78b1c8f service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:30.115770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dd2988-7d6c-4b58-a40d-b028e78b1c8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:30.118106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b3dd2988-7d6c-4b58-a40d-b028e78b1c8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:30.190595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd2988-7d6c-4b58-a40d-b028e78b1c8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:30.192556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dd2988-7d6c-4b58-a40d-b028e78b1c8f service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:30.202984 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2542/5102] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:30.328353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b0e2d23-2b96-4024-b5f9-2a844fc32160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb3fb0936c554fa882e46ba190d3b362'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:30.345023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b0e2d23-2b96-4024-b5f9-2a844fc32160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb3fb0936c554fa882e46ba190d3b362'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:30.346726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b0e2d23-2b96-4024-b5f9-2a844fc32160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb3fb0936c554fa882e46ba190d3b362'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:30.349141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b0e2d23-2b96-4024-b5f9-2a844fc32160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb3fb0936c554fa882e46ba190d3b362'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:30.503103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b0e2d23-2b96-4024-b5f9-2a844fc32160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb3fb0936c554fa882e46ba190d3b362'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:30.595131 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2561/5103] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:29 2022] DELETE /identity/v3/users/bb3fb0936c554fa882e46ba190d3b362 => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:31.303194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:31.526927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd2988-7d6c-4b58-a40d-b028e78b1c8f service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:31.529039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b3dd2988-7d6c-4b58-a40d-b028e78b1c8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:31.603040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dd2988-7d6c-4b58-a40d-b028e78b1c8f service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:31.754976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972b6834-84c6-4a0a-aac5-cbdaf33a1b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:31.756035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b6834-84c6-4a0a-aac5-cbdaf33a1b44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:31.762103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b6834-84c6-4a0a-aac5-cbdaf33a1b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:31.762460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972b6834-84c6-4a0a-aac5-cbdaf33a1b44 admin admin] PATH_INFO: `/v3/projects/539a73abcc1040c19db7f9252ea8c2b0` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:31.767878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b6834-84c6-4a0a-aac5-cbdaf33a1b44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=539a73abcc1040c19db7f9252ea8c2b0)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:31.775376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:31.775991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972b6834-84c6-4a0a-aac5-cbdaf33a1b44 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:31.947369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-972b6834-84c6-4a0a-aac5-cbdaf33a1b44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '539a73abcc1040c19db7f9252ea8c2b0', 'request_id': 'req-972b6834-84c6-4a0a-aac5-cbdaf33a1b44'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:31.964170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-972b6834-84c6-4a0a-aac5-cbdaf33a1b44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '539a73abcc1040c19db7f9252ea8c2b0', 'request_id': 'req-972b6834-84c6-4a0a-aac5-cbdaf33a1b44'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:32.055875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-972b6834-84c6-4a0a-aac5-cbdaf33a1b44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '539a73abcc1040c19db7f9252ea8c2b0', 'request_id': 'req-972b6834-84c6-4a0a-aac5-cbdaf33a1b44'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:32.087362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-972b6834-84c6-4a0a-aac5-cbdaf33a1b44 admin admin] The token cache is being invalidate because project 539a73abcc1040c19db7f9252ea8c2b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:32.088032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-972b6834-84c6-4a0a-aac5-cbdaf33a1b44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:32.090205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-972b6834-84c6-4a0a-aac5-cbdaf33a1b44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:32.092585 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2543/5104] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:31 2022] DELETE /identity/v3/projects/539a73abcc1040c19db7f9252ea8c2b0 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:32.097786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ec576-01de-4ca1-a40f-b9ae04ed5493 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:32.098098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ec576-01de-4ca1-a40f-b9ae04ed5493 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:32.102280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ec576-01de-4ca1-a40f-b9ae04ed5493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:32.147102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e79ec576-01de-4ca1-a40f-b9ae04ed5493 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:32.148375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e79ec576-01de-4ca1-a40f-b9ae04ed5493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:32.293001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14194b79-91f9-43f0-aa43-e043c06d84ce None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:32.293625 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2562/5105] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:32.294072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14194b79-91f9-43f0-aa43-e043c06d84ce None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:32.294606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14194b79-91f9-43f0-aa43-e043c06d84ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:32.328217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-14194b79-91f9-43f0-aa43-e043c06d84ce None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:32.330953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-14194b79-91f9-43f0-aa43-e043c06d84ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:32.354191 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2544/5106] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:32.366586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ec576-01de-4ca1-a40f-b9ae04ed5493 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:32.376772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ec576-01de-4ca1-a40f-b9ae04ed5493 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:32.392622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89276e63-a76c-48fe-b6fa-92a6b84f53f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:32.393292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89276e63-a76c-48fe-b6fa-92a6b84f53f8 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:32.393889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89276e63-a76c-48fe-b6fa-92a6b84f53f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:32.397950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89276e63-a76c-48fe-b6fa-92a6b84f53f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:32.399108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89276e63-a76c-48fe-b6fa-92a6b84f53f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:32.399427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89276e63-a76c-48fe-b6fa-92a6b84f53f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:32.401078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89276e63-a76c-48fe-b6fa-92a6b84f53f8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:32.429264 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2563/5107] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:32.433908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14194b79-91f9-43f0-aa43-e043c06d84ce None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:32.446772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14194b79-91f9-43f0-aa43-e043c06d84ce None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:32.459053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10a454-79d8-4668-b69b-01a3314b7206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:32.460820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10a454-79d8-4668-b69b-01a3314b7206 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:32.461213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10a454-79d8-4668-b69b-01a3314b7206 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:32.461534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10a454-79d8-4668-b69b-01a3314b7206 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:32.466147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10a454-79d8-4668-b69b-01a3314b7206 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:32.470998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10a454-79d8-4668-b69b-01a3314b7206 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:32.570498 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2545/5108] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:32 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:32.574870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89276e63-a76c-48fe-b6fa-92a6b84f53f8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:32.588227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89276e63-a76c-48fe-b6fa-92a6b84f53f8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:32.670377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33166a1f-872d-4a2b-b1f7-c6d40d07becc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:32.670377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33166a1f-872d-4a2b-b1f7-c6d40d07becc admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:32.670377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33166a1f-872d-4a2b-b1f7-c6d40d07becc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:32.670377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33166a1f-872d-4a2b-b1f7-c6d40d07becc admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:32.671113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33166a1f-872d-4a2b-b1f7-c6d40d07becc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:32.681024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33166a1f-872d-4a2b-b1f7-c6d40d07becc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:32.828731 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2564/5109] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:32 2022] POST /identity/v3/users => generated 286 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:32.830543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10a454-79d8-4668-b69b-01a3314b7206 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:32.846221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10a454-79d8-4668-b69b-01a3314b7206 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:32.869036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb400d3-4ae4-4680-9f93-4a12d7f79430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:32.869734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb400d3-4ae4-4680-9f93-4a12d7f79430 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:32.870049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb400d3-4ae4-4680-9f93-4a12d7f79430 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:32.871942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb400d3-4ae4-4680-9f93-4a12d7f79430 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:32.874739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb400d3-4ae4-4680-9f93-4a12d7f79430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:32.879786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb400d3-4ae4-4680-9f93-4a12d7f79430 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:32.914485 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2546/5110] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:19:32.916510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33166a1f-872d-4a2b-b1f7-c6d40d07becc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:32.934191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33166a1f-872d-4a2b-b1f7-c6d40d07becc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:32.944794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09594079-4949-4f27-b33e-3ba9b9dfdc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:32.949403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09594079-4949-4f27-b33e-3ba9b9dfdc2f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:32.949403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09594079-4949-4f27-b33e-3ba9b9dfdc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:32.949403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09594079-4949-4f27-b33e-3ba9b9dfdc2f admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:32.949403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09594079-4949-4f27-b33e-3ba9b9dfdc2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:32.949403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09594079-4949-4f27-b33e-3ba9b9dfdc2f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:32.979583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb400d3-4ae4-4680-9f93-4a12d7f79430 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:32.994540 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2565/5111] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:32 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:32.995122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb400d3-4ae4-4680-9f93-4a12d7f79430 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:33.004150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a194fa-e8e8-48b7-9200-1c6eb2cf79be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:33.005480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a194fa-e8e8-48b7-9200-1c6eb2cf79be admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:33.006255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a194fa-e8e8-48b7-9200-1c6eb2cf79be admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:33.006757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a194fa-e8e8-48b7-9200-1c6eb2cf79be admin admin] PATH_INFO: `/v3/projects/c5b843f32a774121a9f610bcd91db4cf/users/76931a71a5cf4a07b9ac46d18bf35799/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:33.028369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a194fa-e8e8-48b7-9200-1c6eb2cf79be admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5b843f32a774121a9f610bcd91db4cf, user_id=76931a71a5cf4a07b9ac46d18bf35799, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:33.031932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a194fa-e8e8-48b7-9200-1c6eb2cf79be admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:33.181437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c5195-9c47-4055-be95-778f4e8f0481 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:33.181699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c5195-9c47-4055-be95-778f4e8f0481 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:33.182517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c5195-9c47-4055-be95-778f4e8f0481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:33.183998 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2547/5112] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:32 2022] PUT /identity/v3/projects/c5b843f32a774121a9f610bcd91db4cf/users/76931a71a5cf4a07b9ac46d18bf35799/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:33.222589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c54c5195-9c47-4055-be95-778f4e8f0481 None None] MFA Rules not processed for user `76931a71a5cf4a07b9ac46d18bf35799`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:33.226167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c54c5195-9c47-4055-be95-778f4e8f0481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:33.249443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a194fa-e8e8-48b7-9200-1c6eb2cf79be admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:33.252620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-39a194fa-e8e8-48b7-9200-1c6eb2cf79be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:33.397526 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2566/5113] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:33.399247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c5195-9c47-4055-be95-778f4e8f0481 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:33.420209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c5195-9c47-4055-be95-778f4e8f0481 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:33.434838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe468f3-3e16-4a03-9f61-13eeae21d5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:33.435847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe468f3-3e16-4a03-9f61-13eeae21d5c2 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:33.436311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe468f3-3e16-4a03-9f61-13eeae21d5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:33.436842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe468f3-3e16-4a03-9f61-13eeae21d5c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:33.437641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a194fa-e8e8-48b7-9200-1c6eb2cf79be admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:33.438465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe468f3-3e16-4a03-9f61-13eeae21d5c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:33.441127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe468f3-3e16-4a03-9f61-13eeae21d5c2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:33.456242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0bb42-a18c-44cd-9bfa-6b8ca28f80a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:33.457716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0bb42-a18c-44cd-9bfa-6b8ca28f80a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:33.459265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0bb42-a18c-44cd-9bfa-6b8ca28f80a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:33.459643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0bb42-a18c-44cd-9bfa-6b8ca28f80a5 admin admin] PATH_INFO: `/v3/users/e567fd5a2683491281fab7551a485a4d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:33.461197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0bb42-a18c-44cd-9bfa-6b8ca28f80a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e567fd5a2683491281fab7551a485a4d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:33.465116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0bb42-a18c-44cd-9bfa-6b8ca28f80a5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:33.606847 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2567/5114] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:33 2022] POST /identity/v3/projects => generated 371 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:33.612958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe468f3-3e16-4a03-9f61-13eeae21d5c2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:33.738584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe468f3-3e16-4a03-9f61-13eeae21d5c2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:33.748074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5b9a6-7d2f-40ed-a679-d9c1965f77d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:33.748787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5b9a6-7d2f-40ed-a679-d9c1965f77d5 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:33.749177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5b9a6-7d2f-40ed-a679-d9c1965f77d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:33.749462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc5b9a6-7d2f-40ed-a679-d9c1965f77d5 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:33.750210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5b9a6-7d2f-40ed-a679-d9c1965f77d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:33.753436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc5b9a6-7d2f-40ed-a679-d9c1965f77d5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:33.896503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-faa0bb42-a18c-44cd-9bfa-6b8ca28f80a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e567fd5a2683491281fab7551a485a4d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:33.906143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-faa0bb42-a18c-44cd-9bfa-6b8ca28f80a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e567fd5a2683491281fab7551a485a4d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:33.930172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-faa0bb42-a18c-44cd-9bfa-6b8ca28f80a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e567fd5a2683491281fab7551a485a4d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:33.931612 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2568/5115] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:33 2022] POST /identity/v3/users => generated 292 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:33.935034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5b9a6-7d2f-40ed-a679-d9c1965f77d5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:33.950319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc5b9a6-7d2f-40ed-a679-d9c1965f77d5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:33.965352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9dc2f2-902c-45ce-8427-4016f4bc7723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:33.978045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dc2f2-902c-45ce-8427-4016f4bc7723 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:33.978045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dc2f2-902c-45ce-8427-4016f4bc7723 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:33.978045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9dc2f2-902c-45ce-8427-4016f4bc7723 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:33.978045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9dc2f2-902c-45ce-8427-4016f4bc7723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:33.978045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9dc2f2-902c-45ce-8427-4016f4bc7723 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:33.988411 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2569/5116] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:33.992388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9dc2f2-902c-45ce-8427-4016f4bc7723 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:34.004378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9dc2f2-902c-45ce-8427-4016f4bc7723 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:34.009979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25366b-e04a-4891-85fd-1d4be73d0060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:34.011191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25366b-e04a-4891-85fd-1d4be73d0060 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:34.011646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25366b-e04a-4891-85fd-1d4be73d0060 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:34.011931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25366b-e04a-4891-85fd-1d4be73d0060 admin admin] PATH_INFO: `/v3/projects/7eeba8b81e6f48a18494a2a00de6ca0b/users/3de4a1b055244bc18b60408b65c44548/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:34.035231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25366b-e04a-4891-85fd-1d4be73d0060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eeba8b81e6f48a18494a2a00de6ca0b, user_id=3de4a1b055244bc18b60408b65c44548, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:34.038589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e25366b-e04a-4891-85fd-1d4be73d0060 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:34.065656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-faa0bb42-a18c-44cd-9bfa-6b8ca28f80a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e567fd5a2683491281fab7551a485a4d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:34.066955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-faa0bb42-a18c-44cd-9bfa-6b8ca28f80a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e567fd5a2683491281fab7551a485a4d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:34.107524 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2548/5117] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:33 2022] DELETE /identity/v3/users/e567fd5a2683491281fab7551a485a4d => generated 0 bytes in 855 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:34.135346 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2570/5118] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:33 2022] PUT /identity/v3/projects/7eeba8b81e6f48a18494a2a00de6ca0b/users/3de4a1b055244bc18b60408b65c44548/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:34.145070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0bb42-a18c-44cd-9bfa-6b8ca28f80a5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:34.145070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-faa0bb42-a18c-44cd-9bfa-6b8ca28f80a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:34.197673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:34.233502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0bb42-a18c-44cd-9bfa-6b8ca28f80a5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:34.345992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b450e88c-aed5-40b1-82e9-5ce7ddbc4c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:34.347068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450e88c-aed5-40b1-82e9-5ce7ddbc4c6a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:34.348205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450e88c-aed5-40b1-82e9-5ce7ddbc4c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:34.348688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b450e88c-aed5-40b1-82e9-5ce7ddbc4c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:34.353801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450e88c-aed5-40b1-82e9-5ce7ddbc4c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:34.356246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b450e88c-aed5-40b1-82e9-5ce7ddbc4c6a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:34.384509 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2549/5119] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:34.385819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25366b-e04a-4891-85fd-1d4be73d0060 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:34.397293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e25366b-e04a-4891-85fd-1d4be73d0060 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:34.414381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51427e-2706-4a9f-b8e4-75efaeaa4fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:34.415654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51427e-2706-4a9f-b8e4-75efaeaa4fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:34.416117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51427e-2706-4a9f-b8e4-75efaeaa4fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:34.416557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c51427e-2706-4a9f-b8e4-75efaeaa4fb4 admin admin] PATH_INFO: `/v3/projects/7eeba8b81e6f48a18494a2a00de6ca0b/users/3de4a1b055244bc18b60408b65c44548/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:34.423329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51427e-2706-4a9f-b8e4-75efaeaa4fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eeba8b81e6f48a18494a2a00de6ca0b, user_id=3de4a1b055244bc18b60408b65c44548, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:34.433699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c51427e-2706-4a9f-b8e4-75efaeaa4fb4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:34.472349 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2571/5120] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:34 2022] PUT /identity/v3/projects/7eeba8b81e6f48a18494a2a00de6ca0b/users/3de4a1b055244bc18b60408b65c44548/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:34.474725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8355b-7934-4bf8-ab96-28d2a7ed18d3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:34.475173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8355b-7934-4bf8-ab96-28d2a7ed18d3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:34.475305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad8355b-7934-4bf8-ab96-28d2a7ed18d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:34.533083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6ad8355b-7934-4bf8-ab96-28d2a7ed18d3 None None] MFA Rules not processed for user `3de4a1b055244bc18b60408b65c44548`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:34.534588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6ad8355b-7934-4bf8-ab96-28d2a7ed18d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:34.663961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:34.677535 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2550/5121] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:34 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:34.681611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51427e-2706-4a9f-b8e4-75efaeaa4fb4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:34.694830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c51427e-2706-4a9f-b8e4-75efaeaa4fb4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:34.700284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298837b-51fe-40d5-b0a8-9c558c44c247 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:34.701245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298837b-51fe-40d5-b0a8-9c558c44c247 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:34.702940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298837b-51fe-40d5-b0a8-9c558c44c247 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:34.702940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5298837b-51fe-40d5-b0a8-9c558c44c247 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:34.703630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298837b-51fe-40d5-b0a8-9c558c44c247 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:34.705525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5298837b-51fe-40d5-b0a8-9c558c44c247 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:34.843396 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2572/5122] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:34 2022] POST /identity/v3/services => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:34.848084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8355b-7934-4bf8-ab96-28d2a7ed18d3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:34.870746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad8355b-7934-4bf8-ab96-28d2a7ed18d3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:34.917019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24970ae0-bf1c-417b-ab11-3ff066ca78d3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:34.918239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24970ae0-bf1c-417b-ab11-3ff066ca78d3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:34.918529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24970ae0-bf1c-417b-ab11-3ff066ca78d3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:34.918811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24970ae0-bf1c-417b-ab11-3ff066ca78d3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:34.919583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24970ae0-bf1c-417b-ab11-3ff066ca78d3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:34.924527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24970ae0-bf1c-417b-ab11-3ff066ca78d3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:35.016881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298837b-51fe-40d5-b0a8-9c558c44c247 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:35.019951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5298837b-51fe-40d5-b0a8-9c558c44c247 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:35.035010 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-24970ae0-bf1c-417b-ab11-3ff066ca78d3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1558968175. To ensure interoperability, identifiers should be a valid uuid. May 28 21:19:35.035010 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' May 28 21:19:35.035010 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:35.134556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5298837b-51fe-40d5-b0a8-9c558c44c247 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:35.161515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b017569a-0a2b-45e6-8489-e1506d3418f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:35.162709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017569a-0a2b-45e6-8489-e1506d3418f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:35.163237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017569a-0a2b-45e6-8489-e1506d3418f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:35.163749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b017569a-0a2b-45e6-8489-e1506d3418f3 admin admin] PATH_INFO: `/v3/projects/2f1ab0fa428c4239ae77e49feabf77f3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:35.165720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b017569a-0a2b-45e6-8489-e1506d3418f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f1ab0fa428c4239ae77e49feabf77f3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:35.168495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b017569a-0a2b-45e6-8489-e1506d3418f3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:35.189705 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2551/5123] 10.222.0.33 () {60 vars in 1155 bytes} [Sat May 28 21:19:34 2022] POST /identity/v3/endpoints => generated 361 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:35.191185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24970ae0-bf1c-417b-ab11-3ff066ca78d3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:35.250887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24970ae0-bf1c-417b-ab11-3ff066ca78d3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:35.262596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243026c-d865-4539-8a0a-f0d5c2f7cdad tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:35.262596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243026c-d865-4539-8a0a-f0d5c2f7cdad tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:35.262596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243026c-d865-4539-8a0a-f0d5c2f7cdad tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:35.262995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243026c-d865-4539-8a0a-f0d5c2f7cdad tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/regions/tempest-region-1558968175` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:35.262995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243026c-d865-4539-8a0a-f0d5c2f7cdad tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1558968175)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:35.262995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243026c-d865-4539-8a0a-f0d5c2f7cdad tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:35.262995 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2552/5124] 10.222.0.33 () {58 vars in 1205 bytes} [Sat May 28 21:19:35 2022] GET /identity/v3/regions/tempest-region-1558968175 => generated 180 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:35.264860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243026c-d865-4539-8a0a-f0d5c2f7cdad tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:35.274754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243026c-d865-4539-8a0a-f0d5c2f7cdad tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:35.279841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0f6e5-c2b3-4ace-ab77-0a3e13303b9b tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:35.285148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f6e5-c2b3-4ace-ab77-0a3e13303b9b tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:35.285148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f6e5-c2b3-4ace-ab77-0a3e13303b9b tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:35.285148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0f6e5-c2b3-4ace-ab77-0a3e13303b9b tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/services` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:35.285148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0f6e5-c2b3-4ace-ab77-0a3e13303b9b tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:35.285148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0f6e5-c2b3-4ace-ab77-0a3e13303b9b tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:35.356288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b017569a-0a2b-45e6-8489-e1506d3418f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f1ab0fa428c4239ae77e49feabf77f3', 'request_id': 'req-b017569a-0a2b-45e6-8489-e1506d3418f3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:35.364475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b017569a-0a2b-45e6-8489-e1506d3418f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f1ab0fa428c4239ae77e49feabf77f3', 'request_id': 'req-b017569a-0a2b-45e6-8489-e1506d3418f3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:35.439113 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2553/5125] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:35 2022] POST /identity/v3/services => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:35.439113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0f6e5-c2b3-4ace-ab77-0a3e13303b9b tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:35.440413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c6f0f6e5-c2b3-4ace-ab77-0a3e13303b9b tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:35.465103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0f6e5-c2b3-4ace-ab77-0a3e13303b9b tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:35.515408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b017569a-0a2b-45e6-8489-e1506d3418f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f1ab0fa428c4239ae77e49feabf77f3', 'request_id': 'req-b017569a-0a2b-45e6-8489-e1506d3418f3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:35.532473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c201f-35c7-489b-bdab-9cd95de82206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:35.533197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c201f-35c7-489b-bdab-9cd95de82206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:35.533375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c201f-35c7-489b-bdab-9cd95de82206 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:35.533608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b017569a-0a2b-45e6-8489-e1506d3418f3 admin admin] The token cache is being invalidate because project 2f1ab0fa428c4239ae77e49feabf77f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:35.533752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c201f-35c7-489b-bdab-9cd95de82206 admin admin] PATH_INFO: `/v3/users/f2326ded365e4f549f582704b412223a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:35.534439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b017569a-0a2b-45e6-8489-e1506d3418f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:35.534938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c201f-35c7-489b-bdab-9cd95de82206 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2326ded365e4f549f582704b412223a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:35.535284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b017569a-0a2b-45e6-8489-e1506d3418f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:35.536873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c201f-35c7-489b-bdab-9cd95de82206 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:35.537745 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2573/5126] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:35 2022] DELETE /identity/v3/projects/2f1ab0fa428c4239ae77e49feabf77f3 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:35.538290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b017569a-0a2b-45e6-8489-e1506d3418f3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:35.541272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b017569a-0a2b-45e6-8489-e1506d3418f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:35.637266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b017569a-0a2b-45e6-8489-e1506d3418f3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:35.659124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3dcf2-6636-471c-a6c4-43a344e354a6 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:35.660664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3dcf2-6636-471c-a6c4-43a344e354a6 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:35.660664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3dcf2-6636-471c-a6c4-43a344e354a6 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:35.660664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3dcf2-6636-471c-a6c4-43a344e354a6 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:35.661136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3dcf2-6636-471c-a6c4-43a344e354a6 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:35.663038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3dcf2-6636-471c-a6c4-43a344e354a6 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:35.755349 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-31b3dcf2-6636-471c-a6c4-43a344e354a6 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1138982789. To ensure interoperability, identifiers should be a valid uuid. May 28 21:19:35.755349 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' May 28 21:19:35.755349 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:19:35.769533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-640c201f-35c7-489b-bdab-9cd95de82206 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2326ded365e4f549f582704b412223a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:35.770229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-640c201f-35c7-489b-bdab-9cd95de82206 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2326ded365e4f549f582704b412223a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:35.777237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-640c201f-35c7-489b-bdab-9cd95de82206 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2326ded365e4f549f582704b412223a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:35.840512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-640c201f-35c7-489b-bdab-9cd95de82206 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2326ded365e4f549f582704b412223a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:35.841303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-640c201f-35c7-489b-bdab-9cd95de82206 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2326ded365e4f549f582704b412223a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:35.849762 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2574/5127] 10.222.0.33 () {60 vars in 1155 bytes} [Sat May 28 21:19:35 2022] POST /identity/v3/endpoints => generated 363 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:35.849762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081d26f-969a-4b34-b253-bcd3c8984819 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:35.849762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081d26f-969a-4b34-b253-bcd3c8984819 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:35.849762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081d26f-969a-4b34-b253-bcd3c8984819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:35.861134 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2554/5128] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:35 2022] DELETE /identity/v3/users/f2326ded365e4f549f582704b412223a => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:35.862470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b79a5-7866-412a-b464-be9cee7ac576 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:35.862761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b79a5-7866-412a-b464-be9cee7ac576 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:35.863039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b79a5-7866-412a-b464-be9cee7ac576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:35.868916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4081d26f-969a-4b34-b253-bcd3c8984819 None None] MFA Rules not processed for user `857a887b6c7c41588afb2845eed6f309`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:35.868916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4081d26f-969a-4b34-b253-bcd3c8984819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:35.884779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b02b79a5-7866-412a-b464-be9cee7ac576 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:35.886078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b02b79a5-7866-412a-b464-be9cee7ac576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:35.939672 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2575/5129] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:35 2022] POST /identity/v3/auth/tokens => generated 3891 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:35.940824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081d26f-969a-4b34-b253-bcd3c8984819 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:35.942229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4081d26f-969a-4b34-b253-bcd3c8984819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:35.959186 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2555/5130] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:35 2022] POST /identity/v3/auth/tokens => generated 3864 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:35.960656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89997ab5-3111-4730-94f9-e6aa039984d2 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:35.960956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89997ab5-3111-4730-94f9-e6aa039984d2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:35.961241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89997ab5-3111-4730-94f9-e6aa039984d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:35.980380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-89997ab5-3111-4730-94f9-e6aa039984d2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:35.981635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-89997ab5-3111-4730-94f9-e6aa039984d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:35.990529 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2556/5131] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:35 2022] POST /identity/v3/auth/tokens => generated 3864 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:35.995736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780607b-e605-4ffc-95a9-38893ae22007 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:35.995736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780607b-e605-4ffc-95a9-38893ae22007 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:35.995736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780607b-e605-4ffc-95a9-38893ae22007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:36.017938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2780607b-e605-4ffc-95a9-38893ae22007 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:36.019439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2780607b-e605-4ffc-95a9-38893ae22007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:36.038221 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2557/5132] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:35 2022] POST /identity/v3/auth/tokens => generated 3864 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:36.043059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780607b-e605-4ffc-95a9-38893ae22007 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:36.045398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2780607b-e605-4ffc-95a9-38893ae22007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:36.050708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081d26f-969a-4b34-b253-bcd3c8984819 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:36.064543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f7397-9dc9-4b01-a9cf-c03fd7e5405e tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:36.065765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f7397-9dc9-4b01-a9cf-c03fd7e5405e tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:36.066437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f7397-9dc9-4b01-a9cf-c03fd7e5405e tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:36.067057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f7397-9dc9-4b01-a9cf-c03fd7e5405e tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/regions/tempest-region-1138982789` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:36.068506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f7397-9dc9-4b01-a9cf-c03fd7e5405e tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1138982789)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:36.070439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46f7397-9dc9-4b01-a9cf-c03fd7e5405e tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:36.083307 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2576/5133] 10.222.0.33 () {58 vars in 1205 bytes} [Sat May 28 21:19:35 2022] GET /identity/v3/regions/tempest-region-1138982789 => generated 180 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:36.083307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f7397-9dc9-4b01-a9cf-c03fd7e5405e tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:36.094543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f7397-9dc9-4b01-a9cf-c03fd7e5405e tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:36.106925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65398b7a-d900-4f79-9996-874b0fd4e0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:36.108101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65398b7a-d900-4f79-9996-874b0fd4e0c7 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:36.108390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65398b7a-d900-4f79-9996-874b0fd4e0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:36.108671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65398b7a-d900-4f79-9996-874b0fd4e0c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:36.109409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65398b7a-d900-4f79-9996-874b0fd4e0c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:36.109724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65398b7a-d900-4f79-9996-874b0fd4e0c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:36.112906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65398b7a-d900-4f79-9996-874b0fd4e0c7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:36.126578 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2577/5134] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:36.127470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65398b7a-d900-4f79-9996-874b0fd4e0c7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:36.150669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65398b7a-d900-4f79-9996-874b0fd4e0c7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:36.163501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951179b6-e4ff-42c5-abc1-8411e285a013 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:36.165574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951179b6-e4ff-42c5-abc1-8411e285a013 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:36.166077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780607b-e605-4ffc-95a9-38893ae22007 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:36.166457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951179b6-e4ff-42c5-abc1-8411e285a013 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:36.166949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951179b6-e4ff-42c5-abc1-8411e285a013 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:36.168236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951179b6-e4ff-42c5-abc1-8411e285a013 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:36.171755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951179b6-e4ff-42c5-abc1-8411e285a013 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:36.172097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abcb1d-0810-472b-bb08-a31fb34f3ca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:36.176228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abcb1d-0810-472b-bb08-a31fb34f3ca1 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:36.176228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abcb1d-0810-472b-bb08-a31fb34f3ca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:36.176228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28abcb1d-0810-472b-bb08-a31fb34f3ca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:36.185062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28abcb1d-0810-472b-bb08-a31fb34f3ca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:36.186594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28abcb1d-0810-472b-bb08-a31fb34f3ca1 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:36.200754 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2558/5135] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:36 2022] GET /identity/v3/auth/tokens => generated 3864 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:36.201982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abcb1d-0810-472b-bb08-a31fb34f3ca1 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:36.212095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28abcb1d-0810-472b-bb08-a31fb34f3ca1 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:36.219231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340041b-4b8b-4020-95ec-de57fc057ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:36.220795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340041b-4b8b-4020-95ec-de57fc057ed7 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:36.221097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340041b-4b8b-4020-95ec-de57fc057ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:36.221385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3340041b-4b8b-4020-95ec-de57fc057ed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:36.222303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3340041b-4b8b-4020-95ec-de57fc057ed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:36.227986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3340041b-4b8b-4020-95ec-de57fc057ed7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:36.246068 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-951179b6-e4ff-42c5-abc1-8411e285a013 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-241726170. To ensure interoperability, identifiers should be a valid uuid. May 28 21:19:36.246068 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' May 28 21:19:36.246068 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:19:36.312342 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2559/5136] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:36 2022] POST /identity/v3/projects => generated 389 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:36.317817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340041b-4b8b-4020-95ec-de57fc057ed7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:36.373248 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2578/5137] 10.222.0.33 () {60 vars in 1155 bytes} [Sat May 28 21:19:36 2022] POST /identity/v3/endpoints => generated 359 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:36.374437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951179b6-e4ff-42c5-abc1-8411e285a013 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:36.391574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3340041b-4b8b-4020-95ec-de57fc057ed7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:36.410771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951179b6-e4ff-42c5-abc1-8411e285a013 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:36.416257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9432f-e33c-4cd3-a40b-9fc96939a268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:36.417391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9432f-e33c-4cd3-a40b-9fc96939a268 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:36.417391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9432f-e33c-4cd3-a40b-9fc96939a268 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:36.417660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9432f-e33c-4cd3-a40b-9fc96939a268 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:36.418436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c9432f-e33c-4cd3-a40b-9fc96939a268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:36.420095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c9432f-e33c-4cd3-a40b-9fc96939a268 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:36.463644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387b4ef-40a8-4801-8697-6114af025d2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:36.464189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387b4ef-40a8-4801-8697-6114af025d2b service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:36.464500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387b4ef-40a8-4801-8697-6114af025d2b service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:36.464775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387b4ef-40a8-4801-8697-6114af025d2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:36.466573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9387b4ef-40a8-4801-8697-6114af025d2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:36.477439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387b4ef-40a8-4801-8697-6114af025d2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:36.478925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387b4ef-40a8-4801-8697-6114af025d2b service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:36.488833 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2579/5138] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:36 2022] GET /identity/v3/auth/tokens => generated 3929 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:36.489611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387b4ef-40a8-4801-8697-6114af025d2b service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:36.524513 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2560/5139] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:36 2022] POST /identity/v3/users => generated 295 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:36.524513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9432f-e33c-4cd3-a40b-9fc96939a268 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:36.526730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-52c9432f-e33c-4cd3-a40b-9fc96939a268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:36.542759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9432f-e33c-4cd3-a40b-9fc96939a268 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:36.559232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ef049-eab8-4410-9d11-419e92ddeed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:36.560597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ef049-eab8-4410-9d11-419e92ddeed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:36.560597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ef049-eab8-4410-9d11-419e92ddeed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:36.560882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ef049-eab8-4410-9d11-419e92ddeed9 admin admin] PATH_INFO: `/v3/projects/59224c8f992543a5ae5f4a4d27946bc2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:36.562046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ef049-eab8-4410-9d11-419e92ddeed9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59224c8f992543a5ae5f4a4d27946bc2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:36.564308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ef049-eab8-4410-9d11-419e92ddeed9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:36.590644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387b4ef-40a8-4801-8697-6114af025d2b service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:36.598254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f389d3-dd14-4037-8070-d4c26048b2c7 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:36.599117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f389d3-dd14-4037-8070-d4c26048b2c7 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:36.600187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f389d3-dd14-4037-8070-d4c26048b2c7 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:36.600478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f389d3-dd14-4037-8070-d4c26048b2c7 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/regions/tempest-region-241726170` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:36.601149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f389d3-dd14-4037-8070-d4c26048b2c7 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-241726170)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:36.602511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f389d3-dd14-4037-8070-d4c26048b2c7 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:36.604696 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2580/5140] 10.222.0.33 () {58 vars in 1202 bytes} [Sat May 28 21:19:36 2022] GET /identity/v3/regions/tempest-region-241726170 => generated 178 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:36.605572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f389d3-dd14-4037-8070-d4c26048b2c7 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:36.620792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f389d3-dd14-4037-8070-d4c26048b2c7 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:36.626468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07567e-f4ee-4dad-be6e-2306701f8919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:36.627146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07567e-f4ee-4dad-be6e-2306701f8919 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:36.627434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07567e-f4ee-4dad-be6e-2306701f8919 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:36.627705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07567e-f4ee-4dad-be6e-2306701f8919 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:36.629066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07567e-f4ee-4dad-be6e-2306701f8919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:36.630726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07567e-f4ee-4dad-be6e-2306701f8919 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:36.646468 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2581/5141] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:36.646468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07567e-f4ee-4dad-be6e-2306701f8919 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:36.657542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07567e-f4ee-4dad-be6e-2306701f8919 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:36.662968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89daef4b-83b5-47e3-8730-27c2d0cccac6 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:36.663657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daef4b-83b5-47e3-8730-27c2d0cccac6 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:36.664018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daef4b-83b5-47e3-8730-27c2d0cccac6 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:36.664372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daef4b-83b5-47e3-8730-27c2d0cccac6 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:36.665166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89daef4b-83b5-47e3-8730-27c2d0cccac6 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:36.666285 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-89daef4b-83b5-47e3-8730-27c2d0cccac6 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:36.666285 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:19:36.666285 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:19:36.667340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89daef4b-83b5-47e3-8730-27c2d0cccac6 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:36.676818 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2582/5142] 10.222.0.33 () {58 vars in 1133 bytes} [Sat May 28 21:19:36 2022] GET /identity/v3/endpoints => generated 4387 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:36.678214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89daef4b-83b5-47e3-8730-27c2d0cccac6 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:36.691816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89daef4b-83b5-47e3-8730-27c2d0cccac6 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:36.705367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-920ef049-eab8-4410-9d11-419e92ddeed9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59224c8f992543a5ae5f4a4d27946bc2', 'request_id': 'req-920ef049-eab8-4410-9d11-419e92ddeed9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:36.740212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-920ef049-eab8-4410-9d11-419e92ddeed9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59224c8f992543a5ae5f4a4d27946bc2', 'request_id': 'req-920ef049-eab8-4410-9d11-419e92ddeed9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:36.775344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a345b9-f61d-43d9-b00f-a3b56dff1fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:36.776097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a345b9-f61d-43d9-b00f-a3b56dff1fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:36.776386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a345b9-f61d-43d9-b00f-a3b56dff1fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:36.776918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a345b9-f61d-43d9-b00f-a3b56dff1fc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:36.777402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a345b9-f61d-43d9-b00f-a3b56dff1fc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:36.779096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a345b9-f61d-43d9-b00f-a3b56dff1fc9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:36.791275 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2583/5143] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:36.792391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a345b9-f61d-43d9-b00f-a3b56dff1fc9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:36.807045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-920ef049-eab8-4410-9d11-419e92ddeed9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59224c8f992543a5ae5f4a4d27946bc2', 'request_id': 'req-920ef049-eab8-4410-9d11-419e92ddeed9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:36.810189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:36.827733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-920ef049-eab8-4410-9d11-419e92ddeed9 admin admin] The token cache is being invalidate because project 59224c8f992543a5ae5f4a4d27946bc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:36.828030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-920ef049-eab8-4410-9d11-419e92ddeed9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:36.828764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-920ef049-eab8-4410-9d11-419e92ddeed9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:36.830706 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2561/5144] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:36 2022] DELETE /identity/v3/projects/59224c8f992543a5ae5f4a4d27946bc2 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:36.831755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ef049-eab8-4410-9d11-419e92ddeed9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:36.834691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-920ef049-eab8-4410-9d11-419e92ddeed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:36.849151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ef049-eab8-4410-9d11-419e92ddeed9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:36.858540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b112f08-0b42-46c7-9a96-759082fd9b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:36.859442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b112f08-0b42-46c7-9a96-759082fd9b11 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:36.859723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b112f08-0b42-46c7-9a96-759082fd9b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:36.860540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b112f08-0b42-46c7-9a96-759082fd9b11 admin admin] PATH_INFO: `/v3/projects/e398fd1bbf124b01b5a8a155d040f21c/users/7ea0fccc16364b7a92e1ee7308268c09/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:36.878333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b112f08-0b42-46c7-9a96-759082fd9b11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e398fd1bbf124b01b5a8a155d040f21c, user_id=7ea0fccc16364b7a92e1ee7308268c09, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:36.881778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a345b9-f61d-43d9-b00f-a3b56dff1fc9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:36.883045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b112f08-0b42-46c7-9a96-759082fd9b11 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:36.893660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd16240-0eec-4b41-baf5-5698dbb0eea8 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:36.894706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd16240-0eec-4b41-baf5-5698dbb0eea8 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:36.895004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd16240-0eec-4b41-baf5-5698dbb0eea8 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:36.895286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd16240-0eec-4b41-baf5-5698dbb0eea8 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints/168d4031e93047b79aed8c880a67e670` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:36.901172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd16240-0eec-4b41-baf5-5698dbb0eea8 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:get_endpoint(endpoint_id=168d4031e93047b79aed8c880a67e670)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:36.902236 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-6bd16240-0eec-4b41-baf5-5698dbb0eea8 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_endpoint": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:36.902236 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:19:36.902236 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:19:36.903142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd16240-0eec-4b41-baf5-5698dbb0eea8 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:36.905878 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2584/5145] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:36 2022] GET /identity/v3/endpoints/168d4031e93047b79aed8c880a67e670 => generated 359 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:36.906114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f40b24-56cb-4353-9a9b-0a22bda34d71 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:36.906409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f40b24-56cb-4353-9a9b-0a22bda34d71 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:36.906728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f40b24-56cb-4353-9a9b-0a22bda34d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:36.925300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f8f40b24-56cb-4353-9a9b-0a22bda34d71 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:36.926572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f8f40b24-56cb-4353-9a9b-0a22bda34d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:36.934847 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2585/5146] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:36 2022] POST /identity/v3/auth/tokens => generated 4047 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:36.935650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f40b24-56cb-4353-9a9b-0a22bda34d71 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:36.937416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f8f40b24-56cb-4353-9a9b-0a22bda34d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:36.951281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f40b24-56cb-4353-9a9b-0a22bda34d71 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:36.956262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ea3e6-5de9-4541-b48f-d37f86f5196b tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:36.957190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ea3e6-5de9-4541-b48f-d37f86f5196b tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:36.957570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ea3e6-5de9-4541-b48f-d37f86f5196b tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:36.957977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ea3e6-5de9-4541-b48f-d37f86f5196b tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints/168d4031e93047b79aed8c880a67e670` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:36.963393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ea3e6-5de9-4541-b48f-d37f86f5196b tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=168d4031e93047b79aed8c880a67e670)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:36.963393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ea3e6-5de9-4541-b48f-d37f86f5196b tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:36.989263 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2562/5147] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:36 2022] PUT /identity/v3/projects/e398fd1bbf124b01b5a8a155d040f21c/users/7ea0fccc16364b7a92e1ee7308268c09/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:36.990804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5610824-b082-4ea6-b9d8-e570f38b893e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:36.991098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5610824-b082-4ea6-b9d8-e570f38b893e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:36.991384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5610824-b082-4ea6-b9d8-e570f38b893e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.015551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c5610824-b082-4ea6-b9d8-e570f38b893e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:37.017716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c5610824-b082-4ea6-b9d8-e570f38b893e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:37.039272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-346ea3e6-5de9-4541-b48f-d37f86f5196b tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '168d4031e93047b79aed8c880a67e670', 'request_id': 'req-346ea3e6-5de9-4541-b48f-d37f86f5196b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:37.053679 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2586/5148] 10.222.0.33 () {58 vars in 1235 bytes} [Sat May 28 21:19:36 2022] DELETE /identity/v3/endpoints/168d4031e93047b79aed8c880a67e670 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:37.055192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64755ea7-5677-4d2b-80d8-93f6f8449d65 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.055482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64755ea7-5677-4d2b-80d8-93f6f8449d65 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.055761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64755ea7-5677-4d2b-80d8-93f6f8449d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.094121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-64755ea7-5677-4d2b-80d8-93f6f8449d65 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:37.096385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-64755ea7-5677-4d2b-80d8-93f6f8449d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:37.121898 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2563/5149] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:36 2022] POST /identity/v3/auth/tokens => generated 3864 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:37.122800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5610824-b082-4ea6-b9d8-e570f38b893e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.167404 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2587/5150] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:37 2022] POST /identity/v3/auth/tokens => generated 3864 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:37.173421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64755ea7-5677-4d2b-80d8-93f6f8449d65 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.184053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64755ea7-5677-4d2b-80d8-93f6f8449d65 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:37.189297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61ab97-e058-481b-a6c3-05653f75f9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:37.190157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61ab97-e058-481b-a6c3-05653f75f9d2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.190526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61ab97-e058-481b-a6c3-05653f75f9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.190791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61ab97-e058-481b-a6c3-05653f75f9d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.191535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61ab97-e058-481b-a6c3-05653f75f9d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:37.191842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61ab97-e058-481b-a6c3-05653f75f9d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:37.193511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61ab97-e058-481b-a6c3-05653f75f9d2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:37.206965 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2588/5151] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:37.207879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61ab97-e058-481b-a6c3-05653f75f9d2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.212327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8e61ab97-e058-481b-a6c3-05653f75f9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:37.213484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5610824-b082-4ea6-b9d8-e570f38b893e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:37.218582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0ba4b-8d2a-45b0-813a-df3bc7ef62ec tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:37.224842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0ba4b-8d2a-45b0-813a-df3bc7ef62ec tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.224842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0ba4b-8d2a-45b0-813a-df3bc7ef62ec tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.224842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a0ba4b-8d2a-45b0-813a-df3bc7ef62ec tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.224842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a0ba4b-8d2a-45b0-813a-df3bc7ef62ec tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:37.225265 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-05a0ba4b-8d2a-45b0-813a-df3bc7ef62ec tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_endpoints": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:37.225265 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:19:37.225265 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:37.225265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a0ba4b-8d2a-45b0-813a-df3bc7ef62ec tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:37.233823 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2564/5152] 10.222.0.33 () {58 vars in 1133 bytes} [Sat May 28 21:19:37 2022] GET /identity/v3/endpoints => generated 4041 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:37.233823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0ba4b-8d2a-45b0-813a-df3bc7ef62ec tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.238480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:37.247758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a0ba4b-8d2a-45b0-813a-df3bc7ef62ec tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:37.250578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ea793-421c-4bff-aa0d-159ad1071dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:37.251250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ea793-421c-4bff-aa0d-159ad1071dfe admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.251570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ea793-421c-4bff-aa0d-159ad1071dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.251854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ea793-421c-4bff-aa0d-159ad1071dfe admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.252552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ea793-421c-4bff-aa0d-159ad1071dfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:37.254722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ea793-421c-4bff-aa0d-159ad1071dfe admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:37.288611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61ab97-e058-481b-a6c3-05653f75f9d2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:37.292506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d133167-59e2-44d6-9fa9-56bd95f944ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:37.293158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d133167-59e2-44d6-9fa9-56bd95f944ac service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.293446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d133167-59e2-44d6-9fa9-56bd95f944ac service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.293729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d133167-59e2-44d6-9fa9-56bd95f944ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.303618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d133167-59e2-44d6-9fa9-56bd95f944ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:37.305057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d133167-59e2-44d6-9fa9-56bd95f944ac service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:37.314789 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2589/5153] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:37 2022] GET /identity/v3/auth/tokens => generated 3864 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:37.315582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d133167-59e2-44d6-9fa9-56bd95f944ac service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.325552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d133167-59e2-44d6-9fa9-56bd95f944ac service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:37.330323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50622cda-7f41-4de7-965d-994f4e38d6fc tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:37.331202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50622cda-7f41-4de7-965d-994f4e38d6fc tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.331493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50622cda-7f41-4de7-965d-994f4e38d6fc tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.331770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50622cda-7f41-4de7-965d-994f4e38d6fc tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints/168d4031e93047b79aed8c880a67e670` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.336868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50622cda-7f41-4de7-965d-994f4e38d6fc tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:19:37.349188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-50622cda-7f41-4de7-965d-994f4e38d6fc tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Could not find endpoint: 168d4031e93047b79aed8c880a67e670.: keystone.exception.EndpointNotFound: Could not find endpoint: 168d4031e93047b79aed8c880a67e670. May 28 21:19:37.349188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:19:37.349188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 236, in _get_endpoint May 28 21:19:37.349188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return session.query(Endpoint).filter_by(id=endpoint_id).one() May 28 21:19:37.349188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2865, in one May 28 21:19:37.349188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._iter().one() May 28 21:19:37.349188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1417, in one May 28 21:19:37.349188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._only_one_row( May 28 21:19:37.349188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row May 28 21:19:37.349188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exc.NoResultFound( May 28 21:19:37.349188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required May 28 21:19:37.349188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:19:37.349188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:19:37.349188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:19:37.349188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:19:37.349188 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 228, in get_endpoint May 28 21:19:37.350726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.driver.get_endpoint(endpoint_id) May 28 21:19:37.350726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 242, in get_endpoint May 28 21:19:37.350726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_endpoint(session, endpoint_id).to_dict() May 28 21:19:37.350726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 238, in _get_endpoint May 28 21:19:37.350726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) May 28 21:19:37.350726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: 168d4031e93047b79aed8c880a67e670. May 28 21:19:37.350726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:19:37.350726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:19:37.350726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:19:37.350726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:19:37.350726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:19:37.350726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:19:37.350726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:19:37.350726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:19:37.350726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:19:37.350726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:19:37.350726 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:19:37.352117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:19:37.352117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:19:37.352117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:19:37.352117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/endpoints.py", line 113, in delete May 28 21:19:37.352117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_endpoint') May 28 21:19:37.352117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:19:37.352117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:19:37.352117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:19:37.352117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:19:37.352117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:19:37.352117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:19:37.352117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:19:37.352117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:19:37.352117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:19:37.352117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:19:37.352117 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:19:37.353511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 21:19:37.353511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:19:37.353511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 21:19:37.353511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:19:37.353511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:19:37.353511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:19:37.353511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 21:19:37.353511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:19:37.353511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 21:19:37.353511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 230, in get_endpoint May 28 21:19:37.353511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.EndpointNotFound(endpoint_id=endpoint_id) May 28 21:19:37.353511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.EndpointNotFound: Could not find endpoint: 168d4031e93047b79aed8c880a67e670. May 28 21:19:37.353511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:19:37.353511 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2590/5154] 10.222.0.33 () {58 vars in 1235 bytes} [Sat May 28 21:19:37 2022] DELETE /identity/v3/endpoints/168d4031e93047b79aed8c880a67e670 => generated 114 bytes in 35 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:19:37.354929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50622cda-7f41-4de7-965d-994f4e38d6fc tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.366639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50622cda-7f41-4de7-965d-994f4e38d6fc tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:37.371509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac0d17-584d-429e-af08-f6b6f383d036 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:37.375143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac0d17-584d-429e-af08-f6b6f383d036 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.375143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac0d17-584d-429e-af08-f6b6f383d036 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.375143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac0d17-584d-429e-af08-f6b6f383d036 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/regions/tempest-region-241726170` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.375525 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2565/5155] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:37 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:37.375672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac0d17-584d-429e-af08-f6b6f383d036 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-241726170)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:37.375672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac0d17-584d-429e-af08-f6b6f383d036 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:37.378015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ea793-421c-4bff-aa0d-159ad1071dfe admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.444112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ea793-421c-4bff-aa0d-159ad1071dfe admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:37.449354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34db70-493c-4142-9656-f5e7707f1033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:37.450089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34db70-493c-4142-9656-f5e7707f1033 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.450372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34db70-493c-4142-9656-f5e7707f1033 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.450651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d34db70-493c-4142-9656-f5e7707f1033 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.451343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34db70-493c-4142-9656-f5e7707f1033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:37.453321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d34db70-493c-4142-9656-f5e7707f1033 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:37.505346 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2591/5156] 10.222.0.33 () {58 vars in 1205 bytes} [Sat May 28 21:19:37 2022] DELETE /identity/v3/regions/tempest-region-241726170 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:37.510182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac0d17-584d-429e-af08-f6b6f383d036 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.529558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac0d17-584d-429e-af08-f6b6f383d036 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:37.585654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d739c-e7c3-47dd-88c5-9a9140aa0cb5 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:37.586352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d739c-e7c3-47dd-88c5-9a9140aa0cb5 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.586650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d739c-e7c3-47dd-88c5-9a9140aa0cb5 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.586931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d739c-e7c3-47dd-88c5-9a9140aa0cb5 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.587649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d739c-e7c3-47dd-88c5-9a9140aa0cb5 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:37.589392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d739c-e7c3-47dd-88c5-9a9140aa0cb5 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:37.589536 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2566/5157] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:37 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:37.594083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34db70-493c-4142-9656-f5e7707f1033 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.601659 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2592/5158] 10.222.0.33 () {58 vars in 1133 bytes} [Sat May 28 21:19:37 2022] GET /identity/v3/endpoints => generated 4041 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:19:37.602660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d739c-e7c3-47dd-88c5-9a9140aa0cb5 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.605906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d34db70-493c-4142-9656-f5e7707f1033 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:37.616409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d739c-e7c3-47dd-88c5-9a9140aa0cb5 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:37.623747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478dbd60-40d5-4b08-b9f3-047c32a7d01f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:37.624701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dbd60-40d5-4b08-b9f3-047c32a7d01f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.624799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dbd60-40d5-4b08-b9f3-047c32a7d01f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.625059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dbd60-40d5-4b08-b9f3-047c32a7d01f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.626039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478dbd60-40d5-4b08-b9f3-047c32a7d01f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Adding query filter params (service_id=1def0678a77143af88d41c8415e02bdb) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:37.626360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478dbd60-40d5-4b08-b9f3-047c32a7d01f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:37.626755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22d3df-32cc-4b7e-95f8-e9404dea0ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:37.627706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22d3df-32cc-4b7e-95f8-e9404dea0ecb admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.628313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22d3df-32cc-4b7e-95f8-e9404dea0ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.628813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b22d3df-32cc-4b7e-95f8-e9404dea0ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.628936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478dbd60-40d5-4b08-b9f3-047c32a7d01f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:37.630064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b22d3df-32cc-4b7e-95f8-e9404dea0ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:37.632036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b22d3df-32cc-4b7e-95f8-e9404dea0ecb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:37.637148 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2593/5159] 10.222.0.33 () {58 vars in 1220 bytes} [Sat May 28 21:19:37 2022] GET /identity/v3/endpoints?service_id=1def0678a77143af88d41c8415e02bdb => generated 503 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:37.642450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478dbd60-40d5-4b08-b9f3-047c32a7d01f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.645073 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2567/5160] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:37.649561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22d3df-32cc-4b7e-95f8-e9404dea0ecb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.659137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478dbd60-40d5-4b08-b9f3-047c32a7d01f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:37.664206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b22d3df-32cc-4b7e-95f8-e9404dea0ecb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:37.665275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491542d-cd6e-43cd-9283-45351fa9f1b3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:37.666073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491542d-cd6e-43cd-9283-45351fa9f1b3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.666543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491542d-cd6e-43cd-9283-45351fa9f1b3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.666901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491542d-cd6e-43cd-9283-45351fa9f1b3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.667803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491542d-cd6e-43cd-9283-45351fa9f1b3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Adding query filter params (service_id=c1b0d0635f324400b160ef1018b063e7) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:37.668128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491542d-cd6e-43cd-9283-45351fa9f1b3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:37.671603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491542d-cd6e-43cd-9283-45351fa9f1b3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:37.672212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c0d62-ec99-4fb6-8c6c-c8b7684dea5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:37.674100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c0d62-ec99-4fb6-8c6c-c8b7684dea5b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.675163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c0d62-ec99-4fb6-8c6c-c8b7684dea5b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.682007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c0d62-ec99-4fb6-8c6c-c8b7684dea5b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.682007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c0d62-ec99-4fb6-8c6c-c8b7684dea5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:37.682007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c0d62-ec99-4fb6-8c6c-c8b7684dea5b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:37.682376 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2594/5161] 10.222.0.33 () {58 vars in 1220 bytes} [Sat May 28 21:19:37 2022] GET /identity/v3/endpoints?service_id=c1b0d0635f324400b160ef1018b063e7 => generated 505 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:37.686176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491542d-cd6e-43cd-9283-45351fa9f1b3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.698003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491542d-cd6e-43cd-9283-45351fa9f1b3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:37.701564 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2568/5162] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:37.702717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c0d62-ec99-4fb6-8c6c-c8b7684dea5b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.704130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8068d-2627-4ac3-9d79-e0d0f93acdbf tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:37.704792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8068d-2627-4ac3-9d79-e0d0f93acdbf tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.705084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8068d-2627-4ac3-9d79-e0d0f93acdbf tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.711061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8068d-2627-4ac3-9d79-e0d0f93acdbf tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.711061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8068d-2627-4ac3-9d79-e0d0f93acdbf tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Adding query filter params (interface=public) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:37.711061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8068d-2627-4ac3-9d79-e0d0f93acdbf tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:37.711061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8068d-2627-4ac3-9d79-e0d0f93acdbf tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:37.716645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c0d62-ec99-4fb6-8c6c-c8b7684dea5b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:37.717705 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2595/5163] 10.222.0.33 () {58 vars in 1166 bytes} [Sat May 28 21:19:37 2022] GET /identity/v3/endpoints?interface=public => generated 3396 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:37.722518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8068d-2627-4ac3-9d79-e0d0f93acdbf tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.723090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23792e4a-bab5-4e2d-bdc3-7223608dd2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:37.724061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23792e4a-bab5-4e2d-bdc3-7223608dd2b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.724431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23792e4a-bab5-4e2d-bdc3-7223608dd2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.724784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23792e4a-bab5-4e2d-bdc3-7223608dd2b9 admin admin] PATH_INFO: `/v3/projects/d8bdf88ad0a24ba8822f9775db92fe69/users/78a5490b0a5e4393975a026e0c2e27bf/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.737199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8068d-2627-4ac3-9d79-e0d0f93acdbf tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:37.741793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23792e4a-bab5-4e2d-bdc3-7223608dd2b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8bdf88ad0a24ba8822f9775db92fe69, user_id=78a5490b0a5e4393975a026e0c2e27bf, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:37.742640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895ae1f-b62c-4f34-b6b7-3f91653590d4 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:37.743471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895ae1f-b62c-4f34-b6b7-3f91653590d4 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.743954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895ae1f-b62c-4f34-b6b7-3f91653590d4 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.744316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895ae1f-b62c-4f34-b6b7-3f91653590d4 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.744762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23792e4a-bab5-4e2d-bdc3-7223608dd2b9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:37.745195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895ae1f-b62c-4f34-b6b7-3f91653590d4 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Adding query filter params (interface=internal) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:37.745582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895ae1f-b62c-4f34-b6b7-3f91653590d4 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:37.747743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895ae1f-b62c-4f34-b6b7-3f91653590d4 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:37.754827 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2596/5164] 10.222.0.33 () {58 vars in 1170 bytes} [Sat May 28 21:19:37 2022] GET /identity/v3/endpoints?interface=internal => generated 480 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:19:37.764953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895ae1f-b62c-4f34-b6b7-3f91653590d4 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.775896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895ae1f-b62c-4f34-b6b7-3f91653590d4 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:37.781469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a9ee9-d288-4603-b52c-9f1a0800c9cb tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:37.782757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a9ee9-d288-4603-b52c-9f1a0800c9cb tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.783049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a9ee9-d288-4603-b52c-9f1a0800c9cb tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.783337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a9ee9-d288-4603-b52c-9f1a0800c9cb tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.784105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a9ee9-d288-4603-b52c-9f1a0800c9cb tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:create_service()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:37.785928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a9ee9-d288-4603-b52c-9f1a0800c9cb tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:37.835489 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2569/5165] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:37 2022] PUT /identity/v3/projects/d8bdf88ad0a24ba8822f9775db92fe69/users/78a5490b0a5e4393975a026e0c2e27bf/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:37.846048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37d3fe-f04c-465a-af30-1011d2acaee0 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:37.846341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37d3fe-f04c-465a-af30-1011d2acaee0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:37.846619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37d3fe-f04c-465a-af30-1011d2acaee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:37.873087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9f37d3fe-f04c-465a-af30-1011d2acaee0 None None] MFA Rules not processed for user `78a5490b0a5e4393975a026e0c2e27bf`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:37.875147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9f37d3fe-f04c-465a-af30-1011d2acaee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:37.913689 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2597/5166] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:37 2022] POST /identity/v3/services => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:37.917775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a9ee9-d288-4603-b52c-9f1a0800c9cb tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.943837 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2570/5167] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:37 2022] POST /identity/v3/auth/tokens => generated 4018 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:37.954474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37d3fe-f04c-465a-af30-1011d2acaee0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:37.955971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9f37d3fe-f04c-465a-af30-1011d2acaee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:37.977635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a9ee9-d288-4603-b52c-9f1a0800c9cb tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:38.066412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6733f75-c8b9-4664-8b50-3c10aabd4285 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:38.067334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6733f75-c8b9-4664-8b50-3c10aabd4285 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:38.067721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6733f75-c8b9-4664-8b50-3c10aabd4285 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:38.068337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6733f75-c8b9-4664-8b50-3c10aabd4285 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:38.068941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6733f75-c8b9-4664-8b50-3c10aabd4285 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:38.070899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6733f75-c8b9-4664-8b50-3c10aabd4285 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:38.084474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37d3fe-f04c-465a-af30-1011d2acaee0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:38.088266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c9e00-ec2f-489d-9de0-0b3a7744076d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:38.088912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c9e00-ec2f-489d-9de0-0b3a7744076d service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:38.089196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c9e00-ec2f-489d-9de0-0b3a7744076d service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:38.089609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c9e00-ec2f-489d-9de0-0b3a7744076d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:38.098227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c9e00-ec2f-489d-9de0-0b3a7744076d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:38.099656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863c9e00-ec2f-489d-9de0-0b3a7744076d service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:38.109460 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2571/5168] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:37 2022] GET /identity/v3/auth/tokens => generated 4018 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:19:38.217148 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-d6733f75-c8b9-4664-8b50-3c10aabd4285 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1122403936. To ensure interoperability, identifiers should be a valid uuid. May 28 21:19:38.217148 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' May 28 21:19:38.217148 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:19:38.336102 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2598/5169] 10.222.0.33 () {60 vars in 1155 bytes} [Sat May 28 21:19:37 2022] POST /identity/v3/endpoints => generated 361 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:38.337308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c9e00-ec2f-489d-9de0-0b3a7744076d service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:38.349110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c9e00-ec2f-489d-9de0-0b3a7744076d service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:38.365307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79a949-3c8c-4d9b-9e19-33c6ca0513cc tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:38.366569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79a949-3c8c-4d9b-9e19-33c6ca0513cc tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:38.367100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79a949-3c8c-4d9b-9e19-33c6ca0513cc tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:38.367840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc79a949-3c8c-4d9b-9e19-33c6ca0513cc tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/regions/tempest-region-1122403936` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:38.368693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79a949-3c8c-4d9b-9e19-33c6ca0513cc tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1122403936)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:38.369899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc79a949-3c8c-4d9b-9e19-33c6ca0513cc tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:38.374902 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2572/5170] 10.222.0.33 () {58 vars in 1205 bytes} [Sat May 28 21:19:38 2022] GET /identity/v3/regions/tempest-region-1122403936 => generated 180 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:19:38.377810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6733f75-c8b9-4664-8b50-3c10aabd4285 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:38.388395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6733f75-c8b9-4664-8b50-3c10aabd4285 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:38.393934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d3f73-d197-4bb3-bf2d-490fadd9b08e tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:38.394788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d3f73-d197-4bb3-bf2d-490fadd9b08e tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:38.395072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d3f73-d197-4bb3-bf2d-490fadd9b08e tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:38.395346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d3f73-d197-4bb3-bf2d-490fadd9b08e tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints/4b96158c8b70496ba1ee37c37440e8cd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:38.400882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d3f73-d197-4bb3-bf2d-490fadd9b08e tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:update_endpoint(endpoint_id=4b96158c8b70496ba1ee37c37440e8cd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:38.403642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d3f73-d197-4bb3-bf2d-490fadd9b08e tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:38.515188 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-a43d3f73-d197-4bb3-bf2d-490fadd9b08e tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1999842843. To ensure interoperability, identifiers should be a valid uuid. May 28 21:19:38.515188 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' May 28 21:19:38.515188 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:19:38.676208 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2599/5171] 10.222.0.33 () {60 vars in 1255 bytes} [Sat May 28 21:19:38 2022] PATCH /identity/v3/endpoints/4b96158c8b70496ba1ee37c37440e8cd => generated 364 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:19:38.680373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79a949-3c8c-4d9b-9e19-33c6ca0513cc tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:38.691592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc79a949-3c8c-4d9b-9e19-33c6ca0513cc tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:38.707535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15195066-ef19-41b0-9417-0b98de13b95a tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:38.708388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15195066-ef19-41b0-9417-0b98de13b95a tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:38.708683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15195066-ef19-41b0-9417-0b98de13b95a tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:38.708994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15195066-ef19-41b0-9417-0b98de13b95a tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/regions/tempest-region-1999842843` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:38.709657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15195066-ef19-41b0-9417-0b98de13b95a tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:get_region(region_id=tempest-region-1999842843)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:38.710798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15195066-ef19-41b0-9417-0b98de13b95a tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:38.715839 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2573/5172] 10.222.0.33 () {58 vars in 1205 bytes} [Sat May 28 21:19:38 2022] GET /identity/v3/regions/tempest-region-1999842843 => generated 180 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:38.717026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d3f73-d197-4bb3-bf2d-490fadd9b08e tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:38.729012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d3f73-d197-4bb3-bf2d-490fadd9b08e tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:38.734239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991d603-8ebb-4866-a4ff-f64be75245af tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:38.735189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991d603-8ebb-4866-a4ff-f64be75245af tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:38.735477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991d603-8ebb-4866-a4ff-f64be75245af tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:38.735756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991d603-8ebb-4866-a4ff-f64be75245af tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints/4b96158c8b70496ba1ee37c37440e8cd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:38.741256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991d603-8ebb-4866-a4ff-f64be75245af tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=4b96158c8b70496ba1ee37c37440e8cd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:38.743121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991d603-8ebb-4866-a4ff-f64be75245af tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:38.814528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4991d603-8ebb-4866-a4ff-f64be75245af tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '4b96158c8b70496ba1ee37c37440e8cd', 'request_id': 'req-4991d603-8ebb-4866-a4ff-f64be75245af'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:38.829413 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2600/5173] 10.222.0.33 () {58 vars in 1235 bytes} [Sat May 28 21:19:38 2022] DELETE /identity/v3/endpoints/4b96158c8b70496ba1ee37c37440e8cd => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:38.833647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15195066-ef19-41b0-9417-0b98de13b95a tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:38.857192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15195066-ef19-41b0-9417-0b98de13b95a tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:38.879328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b419712-13ff-4ccd-9e65-1abdc30fd2b1 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:38.880731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b419712-13ff-4ccd-9e65-1abdc30fd2b1 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:38.881013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b419712-13ff-4ccd-9e65-1abdc30fd2b1 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:38.881299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b419712-13ff-4ccd-9e65-1abdc30fd2b1 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/regions/tempest-region-1999842843` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:38.882001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b419712-13ff-4ccd-9e65-1abdc30fd2b1 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1999842843)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:38.883625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b419712-13ff-4ccd-9e65-1abdc30fd2b1 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:39.011269 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-9b419712-13ff-4ccd-9e65-1abdc30fd2b1 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1999842843. To ensure interoperability, identifiers should be a valid uuid. May 28 21:19:39.011269 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' May 28 21:19:39.011269 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:39.019798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d2e7f-c6a8-401b-928d-7f0b6e6a3509 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:39.020422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d2e7f-c6a8-401b-928d-7f0b6e6a3509 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:39.020704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d2e7f-c6a8-401b-928d-7f0b6e6a3509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:39.027756 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2574/5174] 10.222.0.33 () {58 vars in 1208 bytes} [Sat May 28 21:19:38 2022] DELETE /identity/v3/regions/tempest-region-1999842843 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:39.046996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b419712-13ff-4ccd-9e65-1abdc30fd2b1 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:39.060920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-240d2e7f-c6a8-401b-928d-7f0b6e6a3509 None None] MFA Rules not processed for user `7ea0fccc16364b7a92e1ee7308268c09`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:39.060920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-240d2e7f-c6a8-401b-928d-7f0b6e6a3509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:39.095307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b419712-13ff-4ccd-9e65-1abdc30fd2b1 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:39.107354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fce163-8889-4b53-8c70-163327c40e9f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:39.109514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fce163-8889-4b53-8c70-163327c40e9f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:39.110831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fce163-8889-4b53-8c70-163327c40e9f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:39.111309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fce163-8889-4b53-8c70-163327c40e9f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/regions/tempest-region-1122403936` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:39.112149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fce163-8889-4b53-8c70-163327c40e9f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1122403936)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:39.115593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fce163-8889-4b53-8c70-163327c40e9f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:39.170285 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2601/5175] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:39 2022] POST /identity/v3/auth/tokens => generated 4024 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:39.183017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cf3db-9f61-45f0-a69b-e2d7ac7a9be0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:39.188638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cf3db-9f61-45f0-a69b-e2d7ac7a9be0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:39.188638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cf3db-9f61-45f0-a69b-e2d7ac7a9be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:39.202658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-083cf3db-9f61-45f0-a69b-e2d7ac7a9be0 None None] MFA Rules not processed for user `7ea0fccc16364b7a92e1ee7308268c09`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:39.203884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-083cf3db-9f61-45f0-a69b-e2d7ac7a9be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:39.205598 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-01fce163-8889-4b53-8c70-163327c40e9f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] /usr/local/lib/python3.8/dist-packages/pycadf/identifier.py:71: UserWarning: Invalid uuid: tempest-region-1122403936. To ensure interoperability, identifiers should be a valid uuid. May 28 21:19:39.205598 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(('Invalid uuid: %s. To ensure interoperability, ' May 28 21:19:39.205598 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:39.213647 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2575/5176] 10.222.0.33 () {58 vars in 1208 bytes} [Sat May 28 21:19:39 2022] DELETE /identity/v3/regions/tempest-region-1122403936 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:39.217588 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2602/5177] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:39 2022] POST /identity/v3/auth/tokens => generated 4024 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:39.217946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fce163-8889-4b53-8c70-163327c40e9f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:39.223927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cf3db-9f61-45f0-a69b-e2d7ac7a9be0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:39.225473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-083cf3db-9f61-45f0-a69b-e2d7ac7a9be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:39.260064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fce163-8889-4b53-8c70-163327c40e9f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:39.266210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b81a9b-4a90-48e9-b9fa-800d08fb13d3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:39.269124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b81a9b-4a90-48e9-b9fa-800d08fb13d3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:39.269124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b81a9b-4a90-48e9-b9fa-800d08fb13d3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:39.269124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b81a9b-4a90-48e9-b9fa-800d08fb13d3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/services/f8b12bf829384a7785b55725363045c8` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:39.269124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b81a9b-4a90-48e9-b9fa-800d08fb13d3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:delete_service(service_id=f8b12bf829384a7785b55725363045c8)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:39.270339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b81a9b-4a90-48e9-b9fa-800d08fb13d3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:39.309867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:39.335612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cf3db-9f61-45f0-a69b-e2d7ac7a9be0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:39.354915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a6d60-368e-4b3d-9980-7fe21d00b79c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:39.355673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a6d60-368e-4b3d-9980-7fe21d00b79c service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:39.356022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a6d60-368e-4b3d-9980-7fe21d00b79c service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:39.356555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a6d60-368e-4b3d-9980-7fe21d00b79c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:39.366257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a6d60-368e-4b3d-9980-7fe21d00b79c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:39.368126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a6d60-368e-4b3d-9980-7fe21d00b79c service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:39.374024 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2576/5178] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:39 2022] DELETE /identity/v3/services/f8b12bf829384a7785b55725363045c8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:39.379320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b81a9b-4a90-48e9-b9fa-800d08fb13d3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:39.395679 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2603/5179] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:39 2022] GET /identity/v3/auth/tokens => generated 3891 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:19:39.404717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b81a9b-4a90-48e9-b9fa-800d08fb13d3 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:39.410683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca09d02-a225-4d8a-884a-be8e8f8fb73d tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:39.411546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca09d02-a225-4d8a-884a-be8e8f8fb73d tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:39.411840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca09d02-a225-4d8a-884a-be8e8f8fb73d tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:39.412121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca09d02-a225-4d8a-884a-be8e8f8fb73d tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints/f8c5dd595c0d415db5cf048a671689f4` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:39.418571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca09d02-a225-4d8a-884a-be8e8f8fb73d tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=f8c5dd595c0d415db5cf048a671689f4)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:39.421595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca09d02-a225-4d8a-884a-be8e8f8fb73d tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:39.492050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4ca09d02-a225-4d8a-884a-be8e8f8fb73d tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': 'f8c5dd595c0d415db5cf048a671689f4', 'request_id': 'req-4ca09d02-a225-4d8a-884a-be8e8f8fb73d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:39.502734 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2577/5180] 10.222.0.33 () {58 vars in 1235 bytes} [Sat May 28 21:19:39 2022] DELETE /identity/v3/endpoints/f8c5dd595c0d415db5cf048a671689f4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:39.506692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a6d60-368e-4b3d-9980-7fe21d00b79c service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:39.517317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a6d60-368e-4b3d-9980-7fe21d00b79c service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:39.533323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fce63-47a3-4c2b-80fd-3d2a9b06867f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:39.534473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fce63-47a3-4c2b-80fd-3d2a9b06867f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:39.534890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fce63-47a3-4c2b-80fd-3d2a9b06867f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:39.535205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601fce63-47a3-4c2b-80fd-3d2a9b06867f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/regions/tempest-region-1138982789` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:39.535873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fce63-47a3-4c2b-80fd-3d2a9b06867f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1138982789)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:39.537604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601fce63-47a3-4c2b-80fd-3d2a9b06867f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:39.622343 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2604/5181] 10.222.0.33 () {58 vars in 1208 bytes} [Sat May 28 21:19:39 2022] DELETE /identity/v3/regions/tempest-region-1138982789 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:39.626811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca09d02-a225-4d8a-884a-be8e8f8fb73d tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:39.638816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca09d02-a225-4d8a-884a-be8e8f8fb73d tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:39.659429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516e449-1655-4260-898d-10a79d21e16a tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:39.660334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516e449-1655-4260-898d-10a79d21e16a tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:39.660723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516e449-1655-4260-898d-10a79d21e16a tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:39.661100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516e449-1655-4260-898d-10a79d21e16a tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/services/c1b0d0635f324400b160ef1018b063e7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:39.662656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516e449-1655-4260-898d-10a79d21e16a tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:delete_service(service_id=c1b0d0635f324400b160ef1018b063e7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:39.664424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516e449-1655-4260-898d-10a79d21e16a tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:39.754107 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2578/5182] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:39 2022] DELETE /identity/v3/services/c1b0d0635f324400b160ef1018b063e7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:39.755195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fce63-47a3-4c2b-80fd-3d2a9b06867f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:39.766673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601fce63-47a3-4c2b-80fd-3d2a9b06867f tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:39.784887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f49ff9-6396-4593-82f0-f45e64ba2139 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:39.785969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f49ff9-6396-4593-82f0-f45e64ba2139 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:39.786247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f49ff9-6396-4593-82f0-f45e64ba2139 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:39.786518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f49ff9-6396-4593-82f0-f45e64ba2139 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/endpoints/957ea887c6b944a79c1b9121a2328616` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:39.791775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f49ff9-6396-4593-82f0-f45e64ba2139 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:delete_endpoint(endpoint_id=957ea887c6b944a79c1b9121a2328616)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:39.793390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f49ff9-6396-4593-82f0-f45e64ba2139 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:39.920250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-30f49ff9-6396-4593-82f0-f45e64ba2139 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity endpoint deleted for {'resource_info': '957ea887c6b944a79c1b9121a2328616', 'request_id': 'req-30f49ff9-6396-4593-82f0-f45e64ba2139'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:39.944244 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2605/5183] 10.222.0.33 () {58 vars in 1235 bytes} [Sat May 28 21:19:39 2022] DELETE /identity/v3/endpoints/957ea887c6b944a79c1b9121a2328616 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:39.944554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516e449-1655-4260-898d-10a79d21e16a tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:39.955082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516e449-1655-4260-898d-10a79d21e16a tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:39.974097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45bfc89-6946-4b3d-91e3-12e2e29079a7 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:39.974968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bfc89-6946-4b3d-91e3-12e2e29079a7 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:39.975292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bfc89-6946-4b3d-91e3-12e2e29079a7 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:39.975855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45bfc89-6946-4b3d-91e3-12e2e29079a7 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/regions/tempest-region-1558968175` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:39.976245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45bfc89-6946-4b3d-91e3-12e2e29079a7 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:delete_region(region_id=tempest-region-1558968175)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:39.979692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45bfc89-6946-4b3d-91e3-12e2e29079a7 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:40.309003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:40.319397 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2579/5184] 10.222.0.33 () {58 vars in 1208 bytes} [Sat May 28 21:19:39 2022] DELETE /identity/v3/regions/tempest-region-1558968175 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:40.323387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f49ff9-6396-4593-82f0-f45e64ba2139 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:40.340173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f49ff9-6396-4593-82f0-f45e64ba2139 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:40.356633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fb547-aa53-444c-a3a4-398ca572596c tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3de4a1b055244bc18b60408b65c44548', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7eeba8b81e6f48a18494a2a00de6ca0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:40.357511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fb547-aa53-444c-a3a4-398ca572596c tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:40.357881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fb547-aa53-444c-a3a4-398ca572596c tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:40.358164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fb547-aa53-444c-a3a4-398ca572596c tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] PATH_INFO: `/v3/services/1def0678a77143af88d41c8415e02bdb` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:40.358980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5fb547-aa53-444c-a3a4-398ca572596c tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorizing `identity:delete_service(service_id=1def0678a77143af88d41c8415e02bdb)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:40.361452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5fb547-aa53-444c-a3a4-398ca572596c tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:40.494915 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2606/5185] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:40 2022] DELETE /identity/v3/services/1def0678a77143af88d41c8415e02bdb => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:40.495077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45bfc89-6946-4b3d-91e3-12e2e29079a7 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:40.495609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b45bfc89-6946-4b3d-91e3-12e2e29079a7 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:40.628825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45bfc89-6946-4b3d-91e3-12e2e29079a7 tempest-EndPointsTestJSON-1719238252 tempest-EndPointsTestJSON-1719238252-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:40.647389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc29cc-3612-4172-a7b8-c901c8ac0d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:40.648916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc29cc-3612-4172-a7b8-c901c8ac0d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:40.649408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc29cc-3612-4172-a7b8-c901c8ac0d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:40.650454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc29cc-3612-4172-a7b8-c901c8ac0d67 admin admin] PATH_INFO: `/v3/users/76931a71a5cf4a07b9ac46d18bf35799` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:40.652030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc29cc-3612-4172-a7b8-c901c8ac0d67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76931a71a5cf4a07b9ac46d18bf35799)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:40.655076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc29cc-3612-4172-a7b8-c901c8ac0d67 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:40.913445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-11fc29cc-3612-4172-a7b8-c901c8ac0d67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76931a71a5cf4a07b9ac46d18bf35799'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:40.914751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-11fc29cc-3612-4172-a7b8-c901c8ac0d67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76931a71a5cf4a07b9ac46d18bf35799'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:40.946183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-11fc29cc-3612-4172-a7b8-c901c8ac0d67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76931a71a5cf4a07b9ac46d18bf35799'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:41.058614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-11fc29cc-3612-4172-a7b8-c901c8ac0d67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76931a71a5cf4a07b9ac46d18bf35799'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:41.059544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-11fc29cc-3612-4172-a7b8-c901c8ac0d67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76931a71a5cf4a07b9ac46d18bf35799'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:41.084136 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2580/5186] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:40 2022] DELETE /identity/v3/users/76931a71a5cf4a07b9ac46d18bf35799 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:41.087442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae5c31-7b18-4502-8358-cb3020071ce4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:41.087736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae5c31-7b18-4502-8358-cb3020071ce4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:41.088021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae5c31-7b18-4502-8358-cb3020071ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:41.108920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-51ae5c31-7b18-4502-8358-cb3020071ce4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:41.110802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-51ae5c31-7b18-4502-8358-cb3020071ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:41.186438 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2607/5187] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:41.207021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc29cc-3612-4172-a7b8-c901c8ac0d67 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:41.209979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-11fc29cc-3612-4172-a7b8-c901c8ac0d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:41.344748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae5c31-7b18-4502-8358-cb3020071ce4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:41.344748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-51ae5c31-7b18-4502-8358-cb3020071ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:41.364197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae5c31-7b18-4502-8358-cb3020071ce4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:41.372835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc29cc-3612-4172-a7b8-c901c8ac0d67 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:41.377970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316e99a-3d53-468c-8c29-81ed53266e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:41.377970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316e99a-3d53-468c-8c29-81ed53266e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:41.377970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316e99a-3d53-468c-8c29-81ed53266e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:41.377970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316e99a-3d53-468c-8c29-81ed53266e3e admin admin] PATH_INFO: `/v3/users/661333ccdbc74a96ac12d051ad3c2279` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:41.378849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316e99a-3d53-468c-8c29-81ed53266e3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=661333ccdbc74a96ac12d051ad3c2279)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:41.381981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316e99a-3d53-468c-8c29-81ed53266e3e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:41.383961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d7452-d09f-4c09-ac80-7b32538fffe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:41.384686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d7452-d09f-4c09-ac80-7b32538fffe9 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:41.384973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d7452-d09f-4c09-ac80-7b32538fffe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:41.385294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d7452-d09f-4c09-ac80-7b32538fffe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:41.413337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d7452-d09f-4c09-ac80-7b32538fffe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:41.416515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d7452-d09f-4c09-ac80-7b32538fffe9 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:41.439623 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2581/5188] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:19:41.480987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:41.800866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f316e99a-3d53-468c-8c29-81ed53266e3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '661333ccdbc74a96ac12d051ad3c2279'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:41.814648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f316e99a-3d53-468c-8c29-81ed53266e3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '661333ccdbc74a96ac12d051ad3c2279'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:41.815574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f316e99a-3d53-468c-8c29-81ed53266e3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '661333ccdbc74a96ac12d051ad3c2279'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:41.816577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f316e99a-3d53-468c-8c29-81ed53266e3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '661333ccdbc74a96ac12d051ad3c2279'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:41.906874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f316e99a-3d53-468c-8c29-81ed53266e3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '661333ccdbc74a96ac12d051ad3c2279'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:41.968001 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2608/5189] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:41 2022] DELETE /identity/v3/users/661333ccdbc74a96ac12d051ad3c2279 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:41.972800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc05889-06c9-42ac-800e-cb6a969d9ba4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:41.973340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc05889-06c9-42ac-800e-cb6a969d9ba4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:41.973583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc05889-06c9-42ac-800e-cb6a969d9ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:42.007970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bfc05889-06c9-42ac-800e-cb6a969d9ba4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:42.009971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bfc05889-06c9-42ac-800e-cb6a969d9ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:42.138237 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2582/5190] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:42.198790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316e99a-3d53-468c-8c29-81ed53266e3e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:42.206995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:42.206995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f316e99a-3d53-468c-8c29-81ed53266e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:42.303345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316e99a-3d53-468c-8c29-81ed53266e3e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:42.308371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410be9f-3619-4cb8-a37a-f7024124df39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:42.315503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410be9f-3619-4cb8-a37a-f7024124df39 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:42.315503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410be9f-3619-4cb8-a37a-f7024124df39 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:42.315503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410be9f-3619-4cb8-a37a-f7024124df39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:42.323907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410be9f-3619-4cb8-a37a-f7024124df39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:42.325848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410be9f-3619-4cb8-a37a-f7024124df39 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:42.338479 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2609/5191] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:42.769667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc05889-06c9-42ac-800e-cb6a969d9ba4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:42.771827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bfc05889-06c9-42ac-800e-cb6a969d9ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:42.789724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc05889-06c9-42ac-800e-cb6a969d9ba4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:42.795794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4a4ca-f37d-498a-8537-e0a8c855b048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:42.796939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4a4ca-f37d-498a-8537-e0a8c855b048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:42.798053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4a4ca-f37d-498a-8537-e0a8c855b048 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:42.798402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4a4ca-f37d-498a-8537-e0a8c855b048 admin admin] PATH_INFO: `/v3/projects/c5b843f32a774121a9f610bcd91db4cf` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:42.800088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4a4ca-f37d-498a-8537-e0a8c855b048 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5b843f32a774121a9f610bcd91db4cf)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:42.802651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4a4ca-f37d-498a-8537-e0a8c855b048 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:43.038651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-81f4a4ca-f37d-498a-8537-e0a8c855b048 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5b843f32a774121a9f610bcd91db4cf', 'request_id': 'req-81f4a4ca-f37d-498a-8537-e0a8c855b048'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:43.064613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-81f4a4ca-f37d-498a-8537-e0a8c855b048 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5b843f32a774121a9f610bcd91db4cf', 'request_id': 'req-81f4a4ca-f37d-498a-8537-e0a8c855b048'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:43.154166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-81f4a4ca-f37d-498a-8537-e0a8c855b048 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5b843f32a774121a9f610bcd91db4cf', 'request_id': 'req-81f4a4ca-f37d-498a-8537-e0a8c855b048'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:43.169202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-81f4a4ca-f37d-498a-8537-e0a8c855b048 admin admin] The token cache is being invalidate because project c5b843f32a774121a9f610bcd91db4cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:43.169622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-81f4a4ca-f37d-498a-8537-e0a8c855b048 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:43.170896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-81f4a4ca-f37d-498a-8537-e0a8c855b048 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:43.173430 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2583/5192] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:42 2022] DELETE /identity/v3/projects/c5b843f32a774121a9f610bcd91db4cf => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:43.177641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410be9f-3619-4cb8-a37a-f7024124df39 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:43.188060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410be9f-3619-4cb8-a37a-f7024124df39 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:43.283246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe88355-b77b-4e43-9435-d7e03529c9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:43.284320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe88355-b77b-4e43-9435-d7e03529c9d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:43.284919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe88355-b77b-4e43-9435-d7e03529c9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:43.285212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe88355-b77b-4e43-9435-d7e03529c9d1 admin admin] PATH_INFO: `/v3/users/3de4a1b055244bc18b60408b65c44548` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:43.286495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe88355-b77b-4e43-9435-d7e03529c9d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3de4a1b055244bc18b60408b65c44548)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:43.288497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe88355-b77b-4e43-9435-d7e03529c9d1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:43.313295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4a4ca-f37d-498a-8537-e0a8c855b048 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:43.316101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-81f4a4ca-f37d-498a-8537-e0a8c855b048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:43.386061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4a4ca-f37d-498a-8537-e0a8c855b048 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:43.397812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d90e06-e464-44c2-9bb4-e355b2457d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:43.404283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d90e06-e464-44c2-9bb4-e355b2457d2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:43.405876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d90e06-e464-44c2-9bb4-e355b2457d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:43.410295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d90e06-e464-44c2-9bb4-e355b2457d2d admin admin] PATH_INFO: `/v3/projects/0f6041dc987549cfb48003c2b424ada5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:43.418719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d90e06-e464-44c2-9bb4-e355b2457d2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f6041dc987549cfb48003c2b424ada5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:43.418719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d90e06-e464-44c2-9bb4-e355b2457d2d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:43.543771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:43.554201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0fe88355-b77b-4e43-9435-d7e03529c9d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3de4a1b055244bc18b60408b65c44548'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:43.564165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0fe88355-b77b-4e43-9435-d7e03529c9d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3de4a1b055244bc18b60408b65c44548'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:43.565725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0fe88355-b77b-4e43-9435-d7e03529c9d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3de4a1b055244bc18b60408b65c44548'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:43.568653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0fe88355-b77b-4e43-9435-d7e03529c9d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3de4a1b055244bc18b60408b65c44548'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:43.638871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-30d90e06-e464-44c2-9bb4-e355b2457d2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f6041dc987549cfb48003c2b424ada5', 'request_id': 'req-30d90e06-e464-44c2-9bb4-e355b2457d2d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:43.649144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-30d90e06-e464-44c2-9bb4-e355b2457d2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f6041dc987549cfb48003c2b424ada5', 'request_id': 'req-30d90e06-e464-44c2-9bb4-e355b2457d2d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:43.754900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0fe88355-b77b-4e43-9435-d7e03529c9d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3de4a1b055244bc18b60408b65c44548'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:43.792377 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2610/5193] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:43 2022] DELETE /identity/v3/users/3de4a1b055244bc18b60408b65c44548 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:43.864322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-30d90e06-e464-44c2-9bb4-e355b2457d2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f6041dc987549cfb48003c2b424ada5', 'request_id': 'req-30d90e06-e464-44c2-9bb4-e355b2457d2d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:43.893960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-30d90e06-e464-44c2-9bb4-e355b2457d2d admin admin] The token cache is being invalidate because project 0f6041dc987549cfb48003c2b424ada5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:43.896640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-30d90e06-e464-44c2-9bb4-e355b2457d2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:43.898335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-30d90e06-e464-44c2-9bb4-e355b2457d2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:43.905627 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2584/5194] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:43 2022] DELETE /identity/v3/projects/0f6041dc987549cfb48003c2b424ada5 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:43.909992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe88355-b77b-4e43-9435-d7e03529c9d1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:43.911770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0fe88355-b77b-4e43-9435-d7e03529c9d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:44.020165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe88355-b77b-4e43-9435-d7e03529c9d1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:44.028545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c0805-6001-48dd-9def-64d6b62dbc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:44.030227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c0805-6001-48dd-9def-64d6b62dbc91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:44.030743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c0805-6001-48dd-9def-64d6b62dbc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:44.031245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c0805-6001-48dd-9def-64d6b62dbc91 admin admin] PATH_INFO: `/v3/users/aa6e6c982a344993aa21d353775fefb6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:44.033112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c0805-6001-48dd-9def-64d6b62dbc91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa6e6c982a344993aa21d353775fefb6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:44.035807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c0805-6001-48dd-9def-64d6b62dbc91 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:44.384792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7c6c0805-6001-48dd-9def-64d6b62dbc91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa6e6c982a344993aa21d353775fefb6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:44.398626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7c6c0805-6001-48dd-9def-64d6b62dbc91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa6e6c982a344993aa21d353775fefb6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:44.400452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7c6c0805-6001-48dd-9def-64d6b62dbc91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa6e6c982a344993aa21d353775fefb6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:44.401383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7c6c0805-6001-48dd-9def-64d6b62dbc91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa6e6c982a344993aa21d353775fefb6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:44.527939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7c6c0805-6001-48dd-9def-64d6b62dbc91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa6e6c982a344993aa21d353775fefb6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:44.560668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:44.583545 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2611/5195] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:43 2022] DELETE /identity/v3/users/aa6e6c982a344993aa21d353775fefb6 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:44.877597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d90e06-e464-44c2-9bb4-e355b2457d2d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:44.879400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-30d90e06-e464-44c2-9bb4-e355b2457d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:45.038186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d90e06-e464-44c2-9bb4-e355b2457d2d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:45.138030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1add7a93-5425-4c76-bb5d-3e837ba50a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:45.139095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add7a93-5425-4c76-bb5d-3e837ba50a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:45.139391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add7a93-5425-4c76-bb5d-3e837ba50a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:45.139670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1add7a93-5425-4c76-bb5d-3e837ba50a26 admin admin] PATH_INFO: `/v3/projects/7eeba8b81e6f48a18494a2a00de6ca0b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:45.141726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add7a93-5425-4c76-bb5d-3e837ba50a26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eeba8b81e6f48a18494a2a00de6ca0b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:45.144363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1add7a93-5425-4c76-bb5d-3e837ba50a26 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:45.372238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1add7a93-5425-4c76-bb5d-3e837ba50a26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eeba8b81e6f48a18494a2a00de6ca0b', 'request_id': 'req-1add7a93-5425-4c76-bb5d-3e837ba50a26'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:45.388124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1add7a93-5425-4c76-bb5d-3e837ba50a26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eeba8b81e6f48a18494a2a00de6ca0b', 'request_id': 'req-1add7a93-5425-4c76-bb5d-3e837ba50a26'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:45.463563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1add7a93-5425-4c76-bb5d-3e837ba50a26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eeba8b81e6f48a18494a2a00de6ca0b', 'request_id': 'req-1add7a93-5425-4c76-bb5d-3e837ba50a26'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:45.498798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1add7a93-5425-4c76-bb5d-3e837ba50a26 admin admin] The token cache is being invalidate because project 7eeba8b81e6f48a18494a2a00de6ca0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:45.499223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1add7a93-5425-4c76-bb5d-3e837ba50a26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:45.502516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1add7a93-5425-4c76-bb5d-3e837ba50a26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:45.509370 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2585/5196] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:44 2022] DELETE /identity/v3/projects/7eeba8b81e6f48a18494a2a00de6ca0b => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:45.510962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266c503-c26e-4c1f-ba23-d02974144445 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:45.511430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266c503-c26e-4c1f-ba23-d02974144445 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:45.512741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266c503-c26e-4c1f-ba23-d02974144445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:45.540665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d266c503-c26e-4c1f-ba23-d02974144445 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:45.542439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d266c503-c26e-4c1f-ba23-d02974144445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:45.657317 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2612/5197] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:45.665443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866fcf1-e611-4402-a887-0422c423c76c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:45.665735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866fcf1-e611-4402-a887-0422c423c76c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:45.666051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866fcf1-e611-4402-a887-0422c423c76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:45.688818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a866fcf1-e611-4402-a887-0422c423c76c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:45.690109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a866fcf1-e611-4402-a887-0422c423c76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:45.714257 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2586/5198] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:45.719558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266c503-c26e-4c1f-ba23-d02974144445 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:45.730230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266c503-c26e-4c1f-ba23-d02974144445 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:45.736491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0b407-6a6b-44a4-94c4-33d014f42477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:45.737169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0b407-6a6b-44a4-94c4-33d014f42477 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:45.737457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0b407-6a6b-44a4-94c4-33d014f42477 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:45.737734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f0b407-6a6b-44a4-94c4-33d014f42477 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:45.738560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0b407-6a6b-44a4-94c4-33d014f42477 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:45.739300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0b407-6a6b-44a4-94c4-33d014f42477 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:45.740929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f0b407-6a6b-44a4-94c4-33d014f42477 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:45.755689 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2613/5199] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:45.762334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866fcf1-e611-4402-a887-0422c423c76c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:45.778760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866fcf1-e611-4402-a887-0422c423c76c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:45.784877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9776908-3d08-4b47-9aaa-4e220803f583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:45.785760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9776908-3d08-4b47-9aaa-4e220803f583 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:45.787092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9776908-3d08-4b47-9aaa-4e220803f583 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:45.787481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9776908-3d08-4b47-9aaa-4e220803f583 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:45.788205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9776908-3d08-4b47-9aaa-4e220803f583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:45.790422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9776908-3d08-4b47-9aaa-4e220803f583 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:45.927534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:45.936591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0b407-6a6b-44a4-94c4-33d014f42477 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:45.939881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f7f0b407-6a6b-44a4-94c4-33d014f42477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:45.959421 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2587/5200] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:45 2022] POST /identity/v3/projects => generated 367 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:45.968591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f0b407-6a6b-44a4-94c4-33d014f42477 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:45.973540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9776908-3d08-4b47-9aaa-4e220803f583 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:46.111973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9776908-3d08-4b47-9aaa-4e220803f583 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:46.118063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef33e1-8987-44b6-9849-9e48527c027a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:46.118814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef33e1-8987-44b6-9849-9e48527c027a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:46.120057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef33e1-8987-44b6-9849-9e48527c027a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:46.120353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ef33e1-8987-44b6-9849-9e48527c027a admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:46.121092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef33e1-8987-44b6-9849-9e48527c027a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:46.123121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ef33e1-8987-44b6-9849-9e48527c027a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:46.131449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc6a48-3f93-4d89-bb27-db8f64e2ac09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:46.132718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc6a48-3f93-4d89-bb27-db8f64e2ac09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:46.133177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc6a48-3f93-4d89-bb27-db8f64e2ac09 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:46.133629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc6a48-3f93-4d89-bb27-db8f64e2ac09 admin admin] PATH_INFO: `/v3/projects/116aaa32b380456498ac7043d5999da8` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:46.136648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc6a48-3f93-4d89-bb27-db8f64e2ac09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=116aaa32b380456498ac7043d5999da8)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:46.143128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc6a48-3f93-4d89-bb27-db8f64e2ac09 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:46.249782 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2588/5201] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:45 2022] POST /identity/v3/users => generated 284 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:46.255175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef33e1-8987-44b6-9849-9e48527c027a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:46.298150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ef33e1-8987-44b6-9849-9e48527c027a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:46.302296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8fdc6a48-3f93-4d89-bb27-db8f64e2ac09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '116aaa32b380456498ac7043d5999da8', 'request_id': 'req-8fdc6a48-3f93-4d89-bb27-db8f64e2ac09'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:46.308504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8fdc6a48-3f93-4d89-bb27-db8f64e2ac09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '116aaa32b380456498ac7043d5999da8', 'request_id': 'req-8fdc6a48-3f93-4d89-bb27-db8f64e2ac09'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:46.399755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8fdc6a48-3f93-4d89-bb27-db8f64e2ac09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '116aaa32b380456498ac7043d5999da8', 'request_id': 'req-8fdc6a48-3f93-4d89-bb27-db8f64e2ac09'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:46.414024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1675242-803c-40fc-af0a-a43b70b7b979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:46.414946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1675242-803c-40fc-af0a-a43b70b7b979 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:46.415249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1675242-803c-40fc-af0a-a43b70b7b979 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:46.415531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1675242-803c-40fc-af0a-a43b70b7b979 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:46.416294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1675242-803c-40fc-af0a-a43b70b7b979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:46.420831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1675242-803c-40fc-af0a-a43b70b7b979 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:46.431055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8fdc6a48-3f93-4d89-bb27-db8f64e2ac09 admin admin] The token cache is being invalidate because project 116aaa32b380456498ac7043d5999da8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:46.433277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8fdc6a48-3f93-4d89-bb27-db8f64e2ac09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:46.437682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8fdc6a48-3f93-4d89-bb27-db8f64e2ac09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:46.442897 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2589/5202] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:19:46.447354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1675242-803c-40fc-af0a-a43b70b7b979 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:46.449007 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2614/5203] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:45 2022] DELETE /identity/v3/projects/116aaa32b380456498ac7043d5999da8 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:46.450584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b1675242-803c-40fc-af0a-a43b70b7b979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:46.457874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bcc1d-4a99-40c1-ac2a-8f1fe8ac9248 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:46.459504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bcc1d-4a99-40c1-ac2a-8f1fe8ac9248 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:46.460063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bcc1d-4a99-40c1-ac2a-8f1fe8ac9248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:46.473203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1675242-803c-40fc-af0a-a43b70b7b979 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:46.492680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2343fd9-1fce-45eb-8ce5-d008cf3ccc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:46.493399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2343fd9-1fce-45eb-8ce5-d008cf3ccc47 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:46.493741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2343fd9-1fce-45eb-8ce5-d008cf3ccc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:46.494136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2343fd9-1fce-45eb-8ce5-d008cf3ccc47 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:46.495241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2343fd9-1fce-45eb-8ce5-d008cf3ccc47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:46.497114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2343fd9-1fce-45eb-8ce5-d008cf3ccc47 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:46.511886 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2590/5204] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:46.515828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2343fd9-1fce-45eb-8ce5-d008cf3ccc47 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:46.524942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-429bcc1d-4a99-40c1-ac2a-8f1fe8ac9248 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:46.528248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-429bcc1d-4a99-40c1-ac2a-8f1fe8ac9248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:46.529391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2343fd9-1fce-45eb-8ce5-d008cf3ccc47 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:46.536219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12484b80-511c-47fa-b7f9-5e18f8885acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:46.537137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12484b80-511c-47fa-b7f9-5e18f8885acd admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:46.537613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12484b80-511c-47fa-b7f9-5e18f8885acd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:46.537931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12484b80-511c-47fa-b7f9-5e18f8885acd admin admin] PATH_INFO: `/v3/projects/57e7b3cb2f1d48d8a3af74c6692096a3/users/2c4dffb8ed07419e9440524acf50c98b/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:46.543739 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2615/5205] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:46.552378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fd1fb-8309-42de-b767-051074330fee None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:46.552666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fd1fb-8309-42de-b767-051074330fee None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:46.553143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2fd1fb-8309-42de-b767-051074330fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:46.564742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12484b80-511c-47fa-b7f9-5e18f8885acd admin admin] RBAC: Authorizing `identity:create_grant(project_id=57e7b3cb2f1d48d8a3af74c6692096a3, user_id=2c4dffb8ed07419e9440524acf50c98b, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:46.571280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12484b80-511c-47fa-b7f9-5e18f8885acd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:46.577053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4d2fd1fb-8309-42de-b767-051074330fee None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:46.578865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4d2fd1fb-8309-42de-b767-051074330fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:46.592204 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2616/5206] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:46.593425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2fd1fb-8309-42de-b767-051074330fee None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:46.604505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2fd1fb-8309-42de-b767-051074330fee None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:46.610738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ffaf1-f631-4226-b30d-d125124aa92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:46.611425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ffaf1-f631-4226-b30d-d125124aa92b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:46.611687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ffaf1-f631-4226-b30d-d125124aa92b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:46.611962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3ffaf1-f631-4226-b30d-d125124aa92b admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:46.613371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ffaf1-f631-4226-b30d-d125124aa92b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:46.613371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ffaf1-f631-4226-b30d-d125124aa92b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:46.614834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3ffaf1-f631-4226-b30d-d125124aa92b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:46.618890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:46.632767 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2617/5207] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:46.637546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ffaf1-f631-4226-b30d-d125124aa92b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:46.649879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3ffaf1-f631-4226-b30d-d125124aa92b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:46.659356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a9a87-9966-46c2-951f-5601e404435d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:46.660228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a9a87-9966-46c2-951f-5601e404435d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:46.660863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a9a87-9966-46c2-951f-5601e404435d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:46.661340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a9a87-9966-46c2-951f-5601e404435d admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:46.667293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a9a87-9966-46c2-951f-5601e404435d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:46.667293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6a9a87-9966-46c2-951f-5601e404435d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:46.783743 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2591/5208] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:46 2022] PUT /identity/v3/projects/57e7b3cb2f1d48d8a3af74c6692096a3/users/2c4dffb8ed07419e9440524acf50c98b/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:46.796803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f35e7b-9fcc-4661-be77-3ee1d1c4275e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:46.797178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f35e7b-9fcc-4661-be77-3ee1d1c4275e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:46.799257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f35e7b-9fcc-4661-be77-3ee1d1c4275e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:46.807103 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2618/5209] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:46 2022] POST /identity/v3/projects => generated 407 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:46.814232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a9a87-9966-46c2-951f-5601e404435d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:46.853530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b8f35e7b-9fcc-4661-be77-3ee1d1c4275e None None] MFA Rules not processed for user `2c4dffb8ed07419e9440524acf50c98b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:46.855435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b8f35e7b-9fcc-4661-be77-3ee1d1c4275e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:46.975780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a9a87-9966-46c2-951f-5601e404435d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:46.995541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f87e5-ca40-47f8-90b1-8bf946ed7533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:46.997506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f87e5-ca40-47f8-90b1-8bf946ed7533 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:46.999181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f87e5-ca40-47f8-90b1-8bf946ed7533 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:47.004043 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2592/5210] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:47.008465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250f87e5-ca40-47f8-90b1-8bf946ed7533 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:47.009105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f35e7b-9fcc-4661-be77-3ee1d1c4275e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:47.010270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f87e5-ca40-47f8-90b1-8bf946ed7533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:47.013538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250f87e5-ca40-47f8-90b1-8bf946ed7533 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:47.040631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f35e7b-9fcc-4661-be77-3ee1d1c4275e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:47.049099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ba428-3791-4f90-83a0-1d9e77d9b1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:47.049955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ba428-3791-4f90-83a0-1d9e77d9b1ed admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:47.050294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ba428-3791-4f90-83a0-1d9e77d9b1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:47.050579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ba428-3791-4f90-83a0-1d9e77d9b1ed admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:47.051310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ba428-3791-4f90-83a0-1d9e77d9b1ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:47.053582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ba428-3791-4f90-83a0-1d9e77d9b1ed admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:47.152248 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2619/5211] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:46 2022] POST /identity/v3/users => generated 304 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:47.152683 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2593/5212] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:47 2022] POST /identity/v3/projects => generated 369 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:47.152811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f87e5-ca40-47f8-90b1-8bf946ed7533 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:47.154936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ba428-3791-4f90-83a0-1d9e77d9b1ed admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:47.164220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250f87e5-ca40-47f8-90b1-8bf946ed7533 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:47.235899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f542c-8af7-422a-abf5-6098a2eb210a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:47.236584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f542c-8af7-422a-abf5-6098a2eb210a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:47.236872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f542c-8af7-422a-abf5-6098a2eb210a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:47.237157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f542c-8af7-422a-abf5-6098a2eb210a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:47.238013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f542c-8af7-422a-abf5-6098a2eb210a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:47.242726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f542c-8af7-422a-abf5-6098a2eb210a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:47.242854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ba428-3791-4f90-83a0-1d9e77d9b1ed admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:47.251494 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2620/5213] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:47.257224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f542c-8af7-422a-abf5-6098a2eb210a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:47.262859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff04786-30ba-4f70-aeb0-58cb29e082a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:47.263844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff04786-30ba-4f70-aeb0-58cb29e082a1 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:47.265931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff04786-30ba-4f70-aeb0-58cb29e082a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:47.267340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff04786-30ba-4f70-aeb0-58cb29e082a1 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:47.268285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff04786-30ba-4f70-aeb0-58cb29e082a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:47.268823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f542c-8af7-422a-abf5-6098a2eb210a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:47.275428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff04786-30ba-4f70-aeb0-58cb29e082a1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:47.289263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf6a41-25bf-4358-8b16-cd9529b041ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:47.289995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf6a41-25bf-4358-8b16-cd9529b041ee admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:47.290279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf6a41-25bf-4358-8b16-cd9529b041ee admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:47.290551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacf6a41-25bf-4358-8b16-cd9529b041ee admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:47.291288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacf6a41-25bf-4358-8b16-cd9529b041ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:47.293035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacf6a41-25bf-4358-8b16-cd9529b041ee admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:47.305545 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2621/5214] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:47.310834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf6a41-25bf-4358-8b16-cd9529b041ee admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:47.322868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacf6a41-25bf-4358-8b16-cd9529b041ee admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:47.329794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251ee85-9bde-4aa6-9f5b-7706c8351a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:47.334155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251ee85-9bde-4aa6-9f5b-7706c8351a0f admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:47.334155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251ee85-9bde-4aa6-9f5b-7706c8351a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:47.334155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251ee85-9bde-4aa6-9f5b-7706c8351a0f admin admin] PATH_INFO: `/v3/projects/c76d04ad120d41f6a3bfabb87bf6968e/users/d44fb59970fb442da24187a6ec827b75/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:47.356673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251ee85-9bde-4aa6-9f5b-7706c8351a0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c76d04ad120d41f6a3bfabb87bf6968e, user_id=d44fb59970fb442da24187a6ec827b75, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:47.361797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251ee85-9bde-4aa6-9f5b-7706c8351a0f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:47.483391 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2594/5215] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:47 2022] POST /identity/v3/users => generated 291 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:47.494364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff04786-30ba-4f70-aeb0-58cb29e082a1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:47.510812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff04786-30ba-4f70-aeb0-58cb29e082a1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:47.516642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9c95d-7554-49ea-a299-5058c899859b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:47.524149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9c95d-7554-49ea-a299-5058c899859b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:47.526454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9c95d-7554-49ea-a299-5058c899859b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:47.526454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9c95d-7554-49ea-a299-5058c899859b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:47.526454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9c95d-7554-49ea-a299-5058c899859b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:47.528235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9c95d-7554-49ea-a299-5058c899859b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:47.541370 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2622/5216] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:47 2022] PUT /identity/v3/projects/c76d04ad120d41f6a3bfabb87bf6968e/users/d44fb59970fb442da24187a6ec827b75/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:47.555413 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2595/5217] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:47.559665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e81d7-3da3-46bd-b6d8-0487961f2bcd None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:47.562971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e81d7-3da3-46bd-b6d8-0487961f2bcd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:47.564430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9c95d-7554-49ea-a299-5058c899859b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:47.566443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e81d7-3da3-46bd-b6d8-0487961f2bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:47.583362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9c95d-7554-49ea-a299-5058c899859b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:47.632301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5a8e81d7-3da3-46bd-b6d8-0487961f2bcd None None] MFA Rules not processed for user `d44fb59970fb442da24187a6ec827b75`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:47.638637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5a8e81d7-3da3-46bd-b6d8-0487961f2bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:47.687101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1c0bf-edc3-4b1d-a460-53447d8803b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:47.688556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1c0bf-edc3-4b1d-a460-53447d8803b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:47.688934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1c0bf-edc3-4b1d-a460-53447d8803b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:47.689223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a1c0bf-edc3-4b1d-a460-53447d8803b6 admin admin] PATH_INFO: `/v3/projects/6ed9816eca4442d0975d8b6eddef7ebe/users/7250636420f84a11a2b01026f4623aa5/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:47.711326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1c0bf-edc3-4b1d-a460-53447d8803b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed9816eca4442d0975d8b6eddef7ebe, user_id=7250636420f84a11a2b01026f4623aa5, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:47.715492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a1c0bf-edc3-4b1d-a460-53447d8803b6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:47.732281 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2623/5218] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:47 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:47.737031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e81d7-3da3-46bd-b6d8-0487961f2bcd None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:47.749675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e81d7-3da3-46bd-b6d8-0487961f2bcd None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:47.757093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f35df3-f5f7-48d0-86c1-881d196c308e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:47.757702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f35df3-f5f7-48d0-86c1-881d196c308e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:47.758028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f35df3-f5f7-48d0-86c1-881d196c308e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:47.758822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f35df3-f5f7-48d0-86c1-881d196c308e admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:47.759080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f35df3-f5f7-48d0-86c1-881d196c308e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:47.761169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f35df3-f5f7-48d0-86c1-881d196c308e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:47.793240 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2596/5219] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:47 2022] PUT /identity/v3/projects/6ed9816eca4442d0975d8b6eddef7ebe/users/7250636420f84a11a2b01026f4623aa5/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:47.794504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1c0bf-edc3-4b1d-a460-53447d8803b6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:47.865484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a1c0bf-edc3-4b1d-a460-53447d8803b6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:47.871448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf81ac-bc6a-4092-a041-de67ef79f692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:47.872139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf81ac-bc6a-4092-a041-de67ef79f692 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:47.872439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf81ac-bc6a-4092-a041-de67ef79f692 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:47.872751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf81ac-bc6a-4092-a041-de67ef79f692 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:47.873511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf81ac-bc6a-4092-a041-de67ef79f692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:47.873800 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2624/5220] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:47 2022] POST /identity/v3/projects => generated 407 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:47.878108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf81ac-bc6a-4092-a041-de67ef79f692 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:47.878947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f35df3-f5f7-48d0-86c1-881d196c308e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:47.897306 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2597/5221] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:47.898120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf81ac-bc6a-4092-a041-de67ef79f692 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:47.924233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf81ac-bc6a-4092-a041-de67ef79f692 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:47.980855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f35df3-f5f7-48d0-86c1-881d196c308e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:47.989096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dec7bb-8fb5-4cf6-b9c9-81d0a859239e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:47.990043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dec7bb-8fb5-4cf6-b9c9-81d0a859239e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:47.990470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dec7bb-8fb5-4cf6-b9c9-81d0a859239e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:47.990684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dec7bb-8fb5-4cf6-b9c9-81d0a859239e admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:47.991594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dec7bb-8fb5-4cf6-b9c9-81d0a859239e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:47.996200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dec7bb-8fb5-4cf6-b9c9-81d0a859239e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:48.016708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d316f-804f-49ae-b813-c1c443c37047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:48.017829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d316f-804f-49ae-b813-c1c443c37047 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:48.018685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d316f-804f-49ae-b813-c1c443c37047 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:48.019070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d316f-804f-49ae-b813-c1c443c37047 admin admin] PATH_INFO: `/v3/projects/6ed9816eca4442d0975d8b6eddef7ebe/users/7250636420f84a11a2b01026f4623aa5/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:48.021639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d316f-804f-49ae-b813-c1c443c37047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed9816eca4442d0975d8b6eddef7ebe, user_id=7250636420f84a11a2b01026f4623aa5, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:48.027830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d316f-804f-49ae-b813-c1c443c37047 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:48.051580 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2598/5222] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:47 2022] PUT /identity/v3/projects/6ed9816eca4442d0975d8b6eddef7ebe/users/7250636420f84a11a2b01026f4623aa5/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:48.054691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b246e-dc96-4093-8f78-54e4a8958458 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:48.055115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b246e-dc96-4093-8f78-54e4a8958458 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:48.055499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b246e-dc96-4093-8f78-54e4a8958458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:48.086955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f97b246e-dc96-4093-8f78-54e4a8958458 None None] MFA Rules not processed for user `7250636420f84a11a2b01026f4623aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:48.088321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f97b246e-dc96-4093-8f78-54e4a8958458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:48.147289 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2625/5223] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:47 2022] POST /identity/v3/users => generated 310 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:48.151460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dec7bb-8fb5-4cf6-b9c9-81d0a859239e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:48.170277 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2599/5224] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:48 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:48.174527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b246e-dc96-4093-8f78-54e4a8958458 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:48.184479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b246e-dc96-4093-8f78-54e4a8958458 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:48.186943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dec7bb-8fb5-4cf6-b9c9-81d0a859239e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:48.189773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6bf97-1ce7-4de8-a056-9da6a619f657 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:48.190907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6bf97-1ce7-4de8-a056-9da6a619f657 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:48.191263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6bf97-1ce7-4de8-a056-9da6a619f657 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:48.191627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6bf97-1ce7-4de8-a056-9da6a619f657 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:48.192432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6bf97-1ce7-4de8-a056-9da6a619f657 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:48.194239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6bf97-1ce7-4de8-a056-9da6a619f657 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:48.267109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6283772-62b6-4be1-8cbd-9ccc26cf3b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:48.268628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6283772-62b6-4be1-8cbd-9ccc26cf3b0b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:48.268628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6283772-62b6-4be1-8cbd-9ccc26cf3b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:48.268628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6283772-62b6-4be1-8cbd-9ccc26cf3b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:48.269323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6283772-62b6-4be1-8cbd-9ccc26cf3b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:48.271035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6283772-62b6-4be1-8cbd-9ccc26cf3b0b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:48.277340 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2600/5225] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:19:48 2022] POST /identity/v3/domains => generated 277 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:48.279138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6bf97-1ce7-4de8-a056-9da6a619f657 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:48.282629 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2626/5226] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:48.287004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6283772-62b6-4be1-8cbd-9ccc26cf3b0b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:48.298354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6283772-62b6-4be1-8cbd-9ccc26cf3b0b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:48.379502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6bf97-1ce7-4de8-a056-9da6a619f657 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:48.391339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f8bb8-6c6b-41bf-ab33-6c6ad53891b1 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:48.392002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f8bb8-6c6b-41bf-ab33-6c6ad53891b1 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:48.393048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f8bb8-6c6b-41bf-ab33-6c6ad53891b1 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:48.393048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f8bb8-6c6b-41bf-ab33-6c6ad53891b1 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:48.393404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f8bb8-6c6b-41bf-ab33-6c6ad53891b1 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:48.398316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccfa0cd-d131-4d38-a65d-886f82dea561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:48.402898 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-f97f8bb8-6c6b-41bf-ab33-6c6ad53891b1 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:48.402898 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:19:48.402898 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:48.403218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfa0cd-d131-4d38-a65d-886f82dea561 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:48.403218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfa0cd-d131-4d38-a65d-886f82dea561 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:48.403218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccfa0cd-d131-4d38-a65d-886f82dea561 admin admin] PATH_INFO: `/v3/projects/a9f02d5cb3c24032bcb2697292f31638/users/71ff0e897321473684786ae57014b9b4/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:48.403463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f8bb8-6c6b-41bf-ab33-6c6ad53891b1 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:48.442006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccfa0cd-d131-4d38-a65d-886f82dea561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9f02d5cb3c24032bcb2697292f31638, user_id=71ff0e897321473684786ae57014b9b4, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:48.445077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccfa0cd-d131-4d38-a65d-886f82dea561 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:48.545961 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2627/5227] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:48 2022] PUT /identity/v3/projects/a9f02d5cb3c24032bcb2697292f31638/users/71ff0e897321473684786ae57014b9b4/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:48.559144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccfa0cd-d131-4d38-a65d-886f82dea561 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:48.559741 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2601/5228] 10.222.0.33 () {60 vars in 1146 bytes} [Sat May 28 21:19:48 2022] POST /identity/v3/groups => generated 280 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:48.564154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f8bb8-6c6b-41bf-ab33-6c6ad53891b1 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:48.628870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f8bb8-6c6b-41bf-ab33-6c6ad53891b1 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:48.645543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccfa0cd-d131-4d38-a65d-886f82dea561 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:48.668890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74edb6-cca2-42ca-8407-dfefef7d82cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:48.670320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74edb6-cca2-42ca-8407-dfefef7d82cf admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:48.670320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74edb6-cca2-42ca-8407-dfefef7d82cf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:48.670509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a74edb6-cca2-42ca-8407-dfefef7d82cf admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:48.671416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a74edb6-cca2-42ca-8407-dfefef7d82cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:48.674287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a74edb6-cca2-42ca-8407-dfefef7d82cf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:48.691777 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2628/5229] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:48.697323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74edb6-cca2-42ca-8407-dfefef7d82cf admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:48.712395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a74edb6-cca2-42ca-8407-dfefef7d82cf admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:48.714630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:48.722493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc00d6b-9822-41fd-a933-f4898d1faf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:48.724285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc00d6b-9822-41fd-a933-f4898d1faf8f admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:48.724660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc00d6b-9822-41fd-a933-f4898d1faf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:48.725126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc00d6b-9822-41fd-a933-f4898d1faf8f admin admin] PATH_INFO: `/v3/projects/a9f02d5cb3c24032bcb2697292f31638/users/71ff0e897321473684786ae57014b9b4/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:48.727931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc00d6b-9822-41fd-a933-f4898d1faf8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9f02d5cb3c24032bcb2697292f31638, user_id=71ff0e897321473684786ae57014b9b4, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:48.734021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc00d6b-9822-41fd-a933-f4898d1faf8f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:48.752043 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2629/5230] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:48 2022] PUT /identity/v3/projects/a9f02d5cb3c24032bcb2697292f31638/users/71ff0e897321473684786ae57014b9b4/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:48.753228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd4fb8-f1b0-45a3-869f-5ee093cf5063 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:48.754404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd4fb8-f1b0-45a3-869f-5ee093cf5063 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:48.755000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd4fb8-f1b0-45a3-869f-5ee093cf5063 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:48.755760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd4fb8-f1b0-45a3-869f-5ee093cf5063 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/245a915789674d438e5195b8d1cfb45e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:48.762038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467141eb-9ab3-4b90-9319-734e8ed8a7e0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:48.762461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467141eb-9ab3-4b90-9319-734e8ed8a7e0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:48.763129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467141eb-9ab3-4b90-9319-734e8ed8a7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:48.765494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd4fb8-f1b0-45a3-869f-5ee093cf5063 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:update_group(group_id=245a915789674d438e5195b8d1cfb45e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:48.769828 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-13dd4fb8-f1b0-45a3-869f-5ee093cf5063 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:48.769828 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:19:48.769828 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:48.774378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd4fb8-f1b0-45a3-869f-5ee093cf5063 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:48.802963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-467141eb-9ab3-4b90-9319-734e8ed8a7e0 None None] MFA Rules not processed for user `71ff0e897321473684786ae57014b9b4`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:48.806342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-467141eb-9ab3-4b90-9319-734e8ed8a7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:48.806685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:48.893862 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2602/5231] 10.222.0.33 () {60 vars in 1246 bytes} [Sat May 28 21:19:48 2022] PATCH /identity/v3/groups/245a915789674d438e5195b8d1cfb45e => generated 293 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:48.898076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd4fb8-f1b0-45a3-869f-5ee093cf5063 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:48.912079 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2630/5232] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:48 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:48.920273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd4fb8-f1b0-45a3-869f-5ee093cf5063 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:48.923555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467141eb-9ab3-4b90-9319-734e8ed8a7e0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:48.925242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-467141eb-9ab3-4b90-9319-734e8ed8a7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:49.101562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467141eb-9ab3-4b90-9319-734e8ed8a7e0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:49.103250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62af9c9-91e3-4e9a-bd43-b0866b324f16 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:49.104353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62af9c9-91e3-4e9a-bd43-b0866b324f16 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:49.104607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62af9c9-91e3-4e9a-bd43-b0866b324f16 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:49.104930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62af9c9-91e3-4e9a-bd43-b0866b324f16 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/245a915789674d438e5195b8d1cfb45e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:49.106969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d302d8-bbcf-4ea5-87df-e493400a0c50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:49.108048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d302d8-bbcf-4ea5-87df-e493400a0c50 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:49.108332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d302d8-bbcf-4ea5-87df-e493400a0c50 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:49.108609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d302d8-bbcf-4ea5-87df-e493400a0c50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:49.121121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62af9c9-91e3-4e9a-bd43-b0866b324f16 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:get_group(group_id=245a915789674d438e5195b8d1cfb45e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:49.122884 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-a62af9c9-91e3-4e9a-bd43-b0866b324f16 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:49.122884 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:19:49.122884 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:49.124413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62af9c9-91e3-4e9a-bd43-b0866b324f16 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:49.130326 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2603/5233] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:48 2022] GET /identity/v3/groups/245a915789674d438e5195b8d1cfb45e => generated 293 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:49.131241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62af9c9-91e3-4e9a-bd43-b0866b324f16 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:49.134682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d302d8-bbcf-4ea5-87df-e493400a0c50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:49.136966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d302d8-bbcf-4ea5-87df-e493400a0c50 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:49.150695 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2631/5234] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:48 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:49.158172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62af9c9-91e3-4e9a-bd43-b0866b324f16 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:49.172623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da8819-2de4-4d50-bfa4-a4a974adf4e4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:49.173497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da8819-2de4-4d50-bfa4-a4a974adf4e4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:49.173788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da8819-2de4-4d50-bfa4-a4a974adf4e4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:49.174097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da8819-2de4-4d50-bfa4-a4a974adf4e4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/245a915789674d438e5195b8d1cfb45e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:49.175404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da8819-2de4-4d50-bfa4-a4a974adf4e4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:update_group(group_id=245a915789674d438e5195b8d1cfb45e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:49.178124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da8819-2de4-4d50-bfa4-a4a974adf4e4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:49.286785 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2604/5235] 10.222.0.33 () {60 vars in 1245 bytes} [Sat May 28 21:19:49 2022] PATCH /identity/v3/groups/245a915789674d438e5195b8d1cfb45e => generated 309 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:49.291487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d302d8-bbcf-4ea5-87df-e493400a0c50 service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:49.310005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d302d8-bbcf-4ea5-87df-e493400a0c50 service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:49.320301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86c9a9-0d1f-41dc-907b-fd726b096b9c tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:49.321568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86c9a9-0d1f-41dc-907b-fd726b096b9c tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:49.323266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86c9a9-0d1f-41dc-907b-fd726b096b9c tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:49.323797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86c9a9-0d1f-41dc-907b-fd726b096b9c tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/245a915789674d438e5195b8d1cfb45e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:49.343249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86c9a9-0d1f-41dc-907b-fd726b096b9c tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:delete_group(group_id=245a915789674d438e5195b8d1cfb45e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:49.344376 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-ae86c9a9-0d1f-41dc-907b-fd726b096b9c tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:49.344376 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:19:49.344376 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:19:49.345660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae86c9a9-0d1f-41dc-907b-fd726b096b9c tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:49.606099 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2632/5236] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:19:49 2022] DELETE /identity/v3/groups/245a915789674d438e5195b8d1cfb45e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:49.607167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da8819-2de4-4d50-bfa4-a4a974adf4e4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:49.615765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da8819-2de4-4d50-bfa4-a4a974adf4e4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:49.685620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd5dd4-19ea-4296-a8d1-9e6950534d64 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:49.686070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd5dd4-19ea-4296-a8d1-9e6950534d64 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:49.689733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd5dd4-19ea-4296-a8d1-9e6950534d64 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:49.689733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd5dd4-19ea-4296-a8d1-9e6950534d64 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:49.689733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd5dd4-19ea-4296-a8d1-9e6950534d64 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:49.690202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd5dd4-19ea-4296-a8d1-9e6950534d64 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:49.774811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86c9a9-0d1f-41dc-907b-fd726b096b9c tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:49.784571 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2605/5237] 10.222.0.33 () {60 vars in 1146 bytes} [Sat May 28 21:19:49 2022] POST /identity/v3/groups => generated 325 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:49.788176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86c9a9-0d1f-41dc-907b-fd726b096b9c tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:49.794580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8838b8-ce9b-4cac-a61a-a8ecb2cf83e4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:49.795450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8838b8-ce9b-4cac-a61a-a8ecb2cf83e4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:49.795822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8838b8-ce9b-4cac-a61a-a8ecb2cf83e4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:49.796206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8838b8-ce9b-4cac-a61a-a8ecb2cf83e4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:49.797527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8838b8-ce9b-4cac-a61a-a8ecb2cf83e4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:49.799648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8838b8-ce9b-4cac-a61a-a8ecb2cf83e4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:49.823859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd5dd4-19ea-4296-a8d1-9e6950534d64 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:49.847930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:49.889735 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2633/5238] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:49 2022] POST /identity/v3/users => generated 321 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:49.891543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8838b8-ce9b-4cac-a61a-a8ecb2cf83e4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:49.893792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2e8838b8-ce9b-4cac-a61a-a8ecb2cf83e4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:49.908131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd5dd4-19ea-4296-a8d1-9e6950534d64 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:49.915512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d2f83-e8ef-435a-956c-f23b0dc39970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:49.915512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d2f83-e8ef-435a-956c-f23b0dc39970 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:49.915512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d2f83-e8ef-435a-956c-f23b0dc39970 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:49.915512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d2f83-e8ef-435a-956c-f23b0dc39970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:49.990449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d2f83-e8ef-435a-956c-f23b0dc39970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:49.992253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d2f83-e8ef-435a-956c-f23b0dc39970 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:50.002300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8838b8-ce9b-4cac-a61a-a8ecb2cf83e4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:50.003793 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2606/5239] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:49 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:19:50.004628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d2f83-e8ef-435a-956c-f23b0dc39970 service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:50.010898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440427d-f2d6-4dac-a814-044e7c0b6113 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:50.011888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440427d-f2d6-4dac-a814-044e7c0b6113 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:50.013014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440427d-f2d6-4dac-a814-044e7c0b6113 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:50.013380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440427d-f2d6-4dac-a814-044e7c0b6113 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:50.015958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c440427d-f2d6-4dac-a814-044e7c0b6113 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:50.018030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d2f83-e8ef-435a-956c-f23b0dc39970 service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:50.026195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3a05f-902a-47c9-ab45-2b4614035023 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:50.027415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3a05f-902a-47c9-ab45-2b4614035023 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:50.027810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3a05f-902a-47c9-ab45-2b4614035023 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:50.030496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3a05f-902a-47c9-ab45-2b4614035023 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/42c1fe1544244266ba7139cd95023650/users/f74d21d8cc19471ab361bf8ed9937582` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:50.034396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440427d-f2d6-4dac-a814-044e7c0b6113 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:50.036887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440427d-f2d6-4dac-a814-044e7c0b6113 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:50.050991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3a05f-902a-47c9-ab45-2b4614035023 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=42c1fe1544244266ba7139cd95023650, user_id=f74d21d8cc19471ab361bf8ed9937582)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:50.051927 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2634/5240] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:50.052323 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-5dc3a05f-902a-47c9-ab45-2b4614035023 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:50.052323 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:19:50.052323 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:50.054507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3a05f-902a-47c9-ab45-2b4614035023 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:50.107602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440427d-f2d6-4dac-a814-044e7c0b6113 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:50.109127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c440427d-f2d6-4dac-a814-044e7c0b6113 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:50.180667 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2607/5241] 10.222.0.33 () {60 vars in 1359 bytes} [Sat May 28 21:19:50 2022] PUT /identity/v3/groups/42c1fe1544244266ba7139cd95023650/users/f74d21d8cc19471ab361bf8ed9937582 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:50.191284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3a05f-902a-47c9-ab45-2b4614035023 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:50.236928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440427d-f2d6-4dac-a814-044e7c0b6113 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:50.320582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3a05f-902a-47c9-ab45-2b4614035023 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:50.326741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a0f00-3b5a-4dff-b280-eec0deba28a2 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:50.328867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a0f00-3b5a-4dff-b280-eec0deba28a2 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:50.329216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a0f00-3b5a-4dff-b280-eec0deba28a2 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:50.329728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a0f00-3b5a-4dff-b280-eec0deba28a2 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:50.330463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a0f00-3b5a-4dff-b280-eec0deba28a2 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:50.332033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a525e82-ae86-4c1c-b511-481e98a9ed53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:50.332582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a0f00-3b5a-4dff-b280-eec0deba28a2 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:50.333296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a525e82-ae86-4c1c-b511-481e98a9ed53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:50.333929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a525e82-ae86-4c1c-b511-481e98a9ed53 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:50.334720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a525e82-ae86-4c1c-b511-481e98a9ed53 admin admin] PATH_INFO: `/v3/users/78a5490b0a5e4393975a026e0c2e27bf` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:50.335698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a525e82-ae86-4c1c-b511-481e98a9ed53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a5490b0a5e4393975a026e0c2e27bf)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:50.339314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a525e82-ae86-4c1c-b511-481e98a9ed53 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:50.517931 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2608/5242] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:50 2022] POST /identity/v3/users => generated 319 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:50.519033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a0f00-3b5a-4dff-b280-eec0deba28a2 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:50.530642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a0f00-3b5a-4dff-b280-eec0deba28a2 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:50.537445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c596f9-4ec2-4992-8756-0300808626f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:50.538396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c596f9-4ec2-4992-8756-0300808626f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:50.538688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c596f9-4ec2-4992-8756-0300808626f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:50.538973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c596f9-4ec2-4992-8756-0300808626f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/42c1fe1544244266ba7139cd95023650/users/f4be1d1766644de3a3fef9ccfffa7ce1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:50.554547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c596f9-4ec2-4992-8756-0300808626f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=42c1fe1544244266ba7139cd95023650, user_id=f4be1d1766644de3a3fef9ccfffa7ce1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:50.565107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c596f9-4ec2-4992-8756-0300808626f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:50.620766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9a525e82-ae86-4c1c-b511-481e98a9ed53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a5490b0a5e4393975a026e0c2e27bf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:50.639076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9a525e82-ae86-4c1c-b511-481e98a9ed53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a5490b0a5e4393975a026e0c2e27bf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:50.640427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9a525e82-ae86-4c1c-b511-481e98a9ed53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a5490b0a5e4393975a026e0c2e27bf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:50.643697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9a525e82-ae86-4c1c-b511-481e98a9ed53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a5490b0a5e4393975a026e0c2e27bf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:50.740545 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2609/5243] 10.222.0.33 () {60 vars in 1359 bytes} [Sat May 28 21:19:50 2022] PUT /identity/v3/groups/42c1fe1544244266ba7139cd95023650/users/f4be1d1766644de3a3fef9ccfffa7ce1 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:50.744381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c596f9-4ec2-4992-8756-0300808626f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:50.746654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-69c596f9-4ec2-4992-8756-0300808626f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:50.801808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9a525e82-ae86-4c1c-b511-481e98a9ed53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a5490b0a5e4393975a026e0c2e27bf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:50.841949 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2635/5244] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:50 2022] DELETE /identity/v3/users/78a5490b0a5e4393975a026e0c2e27bf => generated 0 bytes in 732 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:50.843344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1da297-e826-4a6d-ac58-702bfd9c78dc None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:50.843564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1da297-e826-4a6d-ac58-702bfd9c78dc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:50.844628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1da297-e826-4a6d-ac58-702bfd9c78dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:50.887877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c596f9-4ec2-4992-8756-0300808626f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:50.891326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3b1da297-e826-4a6d-ac58-702bfd9c78dc None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:50.892924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3b1da297-e826-4a6d-ac58-702bfd9c78dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:50.971566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841e3eb-5e45-464f-b9fa-521ebd402adb tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:50.972652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841e3eb-5e45-464f-b9fa-521ebd402adb tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:50.973160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841e3eb-5e45-464f-b9fa-521ebd402adb tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:50.973677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2841e3eb-5e45-464f-b9fa-521ebd402adb tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:50.974798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841e3eb-5e45-464f-b9fa-521ebd402adb tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:50.977340 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2636/5245] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:50.979563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2841e3eb-5e45-464f-b9fa-521ebd402adb tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:50.994503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1da297-e826-4a6d-ac58-702bfd9c78dc None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:50.996388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3b1da297-e826-4a6d-ac58-702bfd9c78dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:51.081186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1da297-e826-4a6d-ac58-702bfd9c78dc None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:51.094620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747e03d-a5b5-489b-95ca-486f287b750d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:51.095306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747e03d-a5b5-489b-95ca-486f287b750d service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:51.096050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747e03d-a5b5-489b-95ca-486f287b750d service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:51.096347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747e03d-a5b5-489b-95ca-486f287b750d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:51.111603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747e03d-a5b5-489b-95ca-486f287b750d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:51.113568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747e03d-a5b5-489b-95ca-486f287b750d service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:51.118021 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2610/5246] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:50 2022] POST /identity/v3/users => generated 323 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:51.127981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841e3eb-5e45-464f-b9fa-521ebd402adb tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:51.146214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2841e3eb-5e45-464f-b9fa-521ebd402adb tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:51.149736 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2637/5247] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:19:51.161761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc1b19-5b54-45e5-b8eb-22f14fba8f17 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:51.164730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc1b19-5b54-45e5-b8eb-22f14fba8f17 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:51.165112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc1b19-5b54-45e5-b8eb-22f14fba8f17 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:51.165483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc1b19-5b54-45e5-b8eb-22f14fba8f17 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/42c1fe1544244266ba7139cd95023650/users/800876e92c474795b10c7871e9aee935` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:51.188602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc1b19-5b54-45e5-b8eb-22f14fba8f17 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=42c1fe1544244266ba7139cd95023650, user_id=800876e92c474795b10c7871e9aee935)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:51.202624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc1b19-5b54-45e5-b8eb-22f14fba8f17 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:51.347360 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2611/5248] 10.222.0.33 () {60 vars in 1359 bytes} [Sat May 28 21:19:51 2022] PUT /identity/v3/groups/42c1fe1544244266ba7139cd95023650/users/800876e92c474795b10c7871e9aee935 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:51.352304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747e03d-a5b5-489b-95ca-486f287b750d service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:51.366931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747e03d-a5b5-489b-95ca-486f287b750d service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:51.430307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0446435-7503-4d3c-80c1-b6a4ae596b94 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:51.431155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0446435-7503-4d3c-80c1-b6a4ae596b94 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:51.431448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0446435-7503-4d3c-80c1-b6a4ae596b94 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:51.431745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0446435-7503-4d3c-80c1-b6a4ae596b94 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/42c1fe1544244266ba7139cd95023650/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:51.433152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0446435-7503-4d3c-80c1-b6a4ae596b94 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=42c1fe1544244266ba7139cd95023650)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:51.434201 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-e0446435-7503-4d3c-80c1-b6a4ae596b94 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:51.434201 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:19:51.434201 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:19:51.435423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0446435-7503-4d3c-80c1-b6a4ae596b94 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:51.458226 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2638/5249] 10.222.0.33 () {58 vars in 1241 bytes} [Sat May 28 21:19:51 2022] GET /identity/v3/groups/42c1fe1544244266ba7139cd95023650/users => generated 1079 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:51.462117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc1b19-5b54-45e5-b8eb-22f14fba8f17 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:51.473357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc1b19-5b54-45e5-b8eb-22f14fba8f17 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:51.478049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64a77d-085c-4f97-9008-8a602757bfe7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:51.479307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64a77d-085c-4f97-9008-8a602757bfe7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `HEAD` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:51.479595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64a77d-085c-4f97-9008-8a602757bfe7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:51.479876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64a77d-085c-4f97-9008-8a602757bfe7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/42c1fe1544244266ba7139cd95023650/users/f74d21d8cc19471ab361bf8ed9937582` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:51.481625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64a77d-085c-4f97-9008-8a602757bfe7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=42c1fe1544244266ba7139cd95023650, user_id=f74d21d8cc19471ab361bf8ed9937582)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:51.482706 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-8b64a77d-085c-4f97-9008-8a602757bfe7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_user_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:51.482706 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:19:51.482706 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:51.483958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b64a77d-085c-4f97-9008-8a602757bfe7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:51.500796 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2612/5250] 10.222.0.33 () {58 vars in 1341 bytes} [Sat May 28 21:19:51 2022] HEAD /identity/v3/groups/42c1fe1544244266ba7139cd95023650/users/f74d21d8cc19471ab361bf8ed9937582 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:51.504881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0446435-7503-4d3c-80c1-b6a4ae596b94 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:51.521232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0446435-7503-4d3c-80c1-b6a4ae596b94 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:51.528303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03c464-0c70-472e-83ab-17758dd09404 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:51.529451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03c464-0c70-472e-83ab-17758dd09404 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:51.530018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03c464-0c70-472e-83ab-17758dd09404 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:51.530486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab03c464-0c70-472e-83ab-17758dd09404 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/42c1fe1544244266ba7139cd95023650/users/f74d21d8cc19471ab361bf8ed9937582` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:51.533229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03c464-0c70-472e-83ab-17758dd09404 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=42c1fe1544244266ba7139cd95023650, user_id=f74d21d8cc19471ab361bf8ed9937582)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:51.534394 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-ab03c464-0c70-472e-83ab-17758dd09404 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:51.534394 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:19:51.534394 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:19:51.535797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab03c464-0c70-472e-83ab-17758dd09404 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:51.647284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ab03c464-0c70-472e-83ab-17758dd09404 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'f74d21d8cc19471ab361bf8ed9937582'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:51.812114 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2639/5251] 10.222.0.33 () {58 vars in 1343 bytes} [Sat May 28 21:19:51 2022] DELETE /identity/v3/groups/42c1fe1544244266ba7139cd95023650/users/f74d21d8cc19471ab361bf8ed9937582 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:51.816920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64a77d-085c-4f97-9008-8a602757bfe7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:51.827029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64a77d-085c-4f97-9008-8a602757bfe7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:51.890584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e978095-6295-443a-bf02-a6e4b7e7b587 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:51.890584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e978095-6295-443a-bf02-a6e4b7e7b587 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `HEAD` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:51.890584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e978095-6295-443a-bf02-a6e4b7e7b587 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:51.890936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e978095-6295-443a-bf02-a6e4b7e7b587 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/42c1fe1544244266ba7139cd95023650/users/f4be1d1766644de3a3fef9ccfffa7ce1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:51.894525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e978095-6295-443a-bf02-a6e4b7e7b587 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=42c1fe1544244266ba7139cd95023650, user_id=f4be1d1766644de3a3fef9ccfffa7ce1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:51.896529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e978095-6295-443a-bf02-a6e4b7e7b587 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:51.914318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03c464-0c70-472e-83ab-17758dd09404 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:51.919210 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2613/5252] 10.222.0.33 () {58 vars in 1341 bytes} [Sat May 28 21:19:51 2022] HEAD /identity/v3/groups/42c1fe1544244266ba7139cd95023650/users/f4be1d1766644de3a3fef9ccfffa7ce1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:51.920365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e978095-6295-443a-bf02-a6e4b7e7b587 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:51.936351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e978095-6295-443a-bf02-a6e4b7e7b587 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:51.938441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ab03c464-0c70-472e-83ab-17758dd09404 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:51.947628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8432422-b23a-4533-b66a-f4c39d8c9c09 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:51.949773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8432422-b23a-4533-b66a-f4c39d8c9c09 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:51.953412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8432422-b23a-4533-b66a-f4c39d8c9c09 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:51.953412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8432422-b23a-4533-b66a-f4c39d8c9c09 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/42c1fe1544244266ba7139cd95023650/users/f4be1d1766644de3a3fef9ccfffa7ce1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:51.954668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8432422-b23a-4533-b66a-f4c39d8c9c09 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=42c1fe1544244266ba7139cd95023650, user_id=f4be1d1766644de3a3fef9ccfffa7ce1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:51.955701 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-c8432422-b23a-4533-b66a-f4c39d8c9c09 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:remove_user_from_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:51.955701 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:19:51.955701 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:51.956903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8432422-b23a-4533-b66a-f4c39d8c9c09 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:52.026709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab03c464-0c70-472e-83ab-17758dd09404 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:52.030957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c8432422-b23a-4533-b66a-f4c39d8c9c09 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'f4be1d1766644de3a3fef9ccfffa7ce1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:52.036978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78db8b-e577-4d58-8931-d757b8512168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:52.038978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78db8b-e577-4d58-8931-d757b8512168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:52.039276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78db8b-e577-4d58-8931-d757b8512168 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:52.039614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78db8b-e577-4d58-8931-d757b8512168 admin admin] PATH_INFO: `/v3/projects/d8bdf88ad0a24ba8822f9775db92fe69` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:52.040899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78db8b-e577-4d58-8931-d757b8512168 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8bdf88ad0a24ba8822f9775db92fe69)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:52.042917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78db8b-e577-4d58-8931-d757b8512168 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:52.150426 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2614/5253] 10.222.0.33 () {58 vars in 1343 bytes} [Sat May 28 21:19:51 2022] DELETE /identity/v3/groups/42c1fe1544244266ba7139cd95023650/users/f4be1d1766644de3a3fef9ccfffa7ce1 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:52.153814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8432422-b23a-4533-b66a-f4c39d8c9c09 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:52.181627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5c78db8b-e577-4d58-8931-d757b8512168 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8bdf88ad0a24ba8822f9775db92fe69', 'request_id': 'req-5c78db8b-e577-4d58-8931-d757b8512168'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:52.189389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5c78db8b-e577-4d58-8931-d757b8512168 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8bdf88ad0a24ba8822f9775db92fe69', 'request_id': 'req-5c78db8b-e577-4d58-8931-d757b8512168'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:52.240052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8432422-b23a-4533-b66a-f4c39d8c9c09 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:52.285875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5c78db8b-e577-4d58-8931-d757b8512168 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8bdf88ad0a24ba8822f9775db92fe69', 'request_id': 'req-5c78db8b-e577-4d58-8931-d757b8512168'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:52.302177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5c78db8b-e577-4d58-8931-d757b8512168 admin admin] The token cache is being invalidate because project d8bdf88ad0a24ba8822f9775db92fe69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:52.302791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5c78db8b-e577-4d58-8931-d757b8512168 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:52.303758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5c78db8b-e577-4d58-8931-d757b8512168 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:52.306462 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2640/5254] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:51 2022] DELETE /identity/v3/projects/d8bdf88ad0a24ba8822f9775db92fe69 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:52.315748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244291a-3dd5-4c77-b9af-6eb2cc768162 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:52.316358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244291a-3dd5-4c77-b9af-6eb2cc768162 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:52.316830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0244291a-3dd5-4c77-b9af-6eb2cc768162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:52.347860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f688c8-680b-47c9-b7c4-aa388564dca5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:52.349668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f688c8-680b-47c9-b7c4-aa388564dca5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `HEAD` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:52.350164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f688c8-680b-47c9-b7c4-aa388564dca5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:52.350619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f688c8-680b-47c9-b7c4-aa388564dca5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/42c1fe1544244266ba7139cd95023650/users/800876e92c474795b10c7871e9aee935` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:52.351648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0244291a-3dd5-4c77-b9af-6eb2cc768162 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:52.353225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0244291a-3dd5-4c77-b9af-6eb2cc768162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:52.354424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f688c8-680b-47c9-b7c4-aa388564dca5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:check_user_in_group(group_id=42c1fe1544244266ba7139cd95023650, user_id=800876e92c474795b10c7871e9aee935)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:52.367829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f688c8-680b-47c9-b7c4-aa388564dca5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:52.386356 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2615/5255] 10.222.0.33 () {58 vars in 1341 bytes} [Sat May 28 21:19:52 2022] HEAD /identity/v3/groups/42c1fe1544244266ba7139cd95023650/users/800876e92c474795b10c7871e9aee935 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:52.391503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f688c8-680b-47c9-b7c4-aa388564dca5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:52.394387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-58f688c8-680b-47c9-b7c4-aa388564dca5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:52.412525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f688c8-680b-47c9-b7c4-aa388564dca5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:52.418477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f19e1e-0da5-4f17-82d9-a732cf111fd3 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:52.426204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f19e1e-0da5-4f17-82d9-a732cf111fd3 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:52.427232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f19e1e-0da5-4f17-82d9-a732cf111fd3 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:52.430692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f19e1e-0da5-4f17-82d9-a732cf111fd3 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/42c1fe1544244266ba7139cd95023650/users/800876e92c474795b10c7871e9aee935` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:52.436009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f19e1e-0da5-4f17-82d9-a732cf111fd3 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=42c1fe1544244266ba7139cd95023650, user_id=800876e92c474795b10c7871e9aee935)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:52.441014 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2641/5256] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:52.442614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f19e1e-0da5-4f17-82d9-a732cf111fd3 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:52.450026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef9ab4-29ae-45ed-a205-b237d42696a2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:52.450331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef9ab4-29ae-45ed-a205-b237d42696a2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:52.450612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef9ab4-29ae-45ed-a205-b237d42696a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:52.478698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-bcef9ab4-29ae-45ed-a205-b237d42696a2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:52.480206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bcef9ab4-29ae-45ed-a205-b237d42696a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:52.502413 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2642/5257] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:52.502413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef9ab4-29ae-45ed-a205-b237d42696a2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:52.507584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef9ab4-29ae-45ed-a205-b237d42696a2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:52.512523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34239f16-9f9c-4e67-85ef-1a94c21787c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:52.513450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34239f16-9f9c-4e67-85ef-1a94c21787c5 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:52.513804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34239f16-9f9c-4e67-85ef-1a94c21787c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:52.514190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34239f16-9f9c-4e67-85ef-1a94c21787c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:52.515117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34239f16-9f9c-4e67-85ef-1a94c21787c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:19:52.515548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34239f16-9f9c-4e67-85ef-1a94c21787c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:52.517287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34239f16-9f9c-4e67-85ef-1a94c21787c5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:52.530222 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2643/5258] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:52.536192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34239f16-9f9c-4e67-85ef-1a94c21787c5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:52.543228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-76f19e1e-0da5-4f17-82d9-a732cf111fd3 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '800876e92c474795b10c7871e9aee935'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:52.548828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34239f16-9f9c-4e67-85ef-1a94c21787c5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:52.555843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b1dae-5a0d-4700-91de-2af65510f3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:52.556684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b1dae-5a0d-4700-91de-2af65510f3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:52.556982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b1dae-5a0d-4700-91de-2af65510f3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:52.557266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b1dae-5a0d-4700-91de-2af65510f3b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:52.558012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b1dae-5a0d-4700-91de-2af65510f3b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:52.560493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b1dae-5a0d-4700-91de-2af65510f3b6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:52.595795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:52.635469 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2616/5259] 10.222.0.33 () {58 vars in 1343 bytes} [Sat May 28 21:19:52 2022] DELETE /identity/v3/groups/42c1fe1544244266ba7139cd95023650/users/800876e92c474795b10c7871e9aee935 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:52.639819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f19e1e-0da5-4f17-82d9-a732cf111fd3 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:52.668045 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2644/5260] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:19:52 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:52.669502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b1dae-5a0d-4700-91de-2af65510f3b6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:52.734911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f19e1e-0da5-4f17-82d9-a732cf111fd3 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:52.761559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b1dae-5a0d-4700-91de-2af65510f3b6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:52.769261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e12071-6152-4ca8-9b5a-b69398169274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:52.769979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e12071-6152-4ca8-9b5a-b69398169274 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:52.770460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e12071-6152-4ca8-9b5a-b69398169274 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:52.770796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e12071-6152-4ca8-9b5a-b69398169274 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:52.771497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e12071-6152-4ca8-9b5a-b69398169274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:52.773467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e12071-6152-4ca8-9b5a-b69398169274 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:52.812650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a1245-7e82-46c6-8bbf-a1e4c0677e66 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:52.813547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a1245-7e82-46c6-8bbf-a1e4c0677e66 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:52.813793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a1245-7e82-46c6-8bbf-a1e4c0677e66 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:52.814123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a1245-7e82-46c6-8bbf-a1e4c0677e66 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/42c1fe1544244266ba7139cd95023650/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:52.815395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a1245-7e82-46c6-8bbf-a1e4c0677e66 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:list_users_in_group(group_id=42c1fe1544244266ba7139cd95023650)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:52.816487 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-ca1a1245-7e82-46c6-8bbf-a1e4c0677e66 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_users_in_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:52.816487 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:19:52.816487 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:52.817864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a1245-7e82-46c6-8bbf-a1e4c0677e66 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:52.843529 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2617/5261] 10.222.0.33 () {58 vars in 1241 bytes} [Sat May 28 21:19:52 2022] GET /identity/v3/groups/42c1fe1544244266ba7139cd95023650/users => generated 145 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:52.845880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a1245-7e82-46c6-8bbf-a1e4c0677e66 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:52.856308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a1245-7e82-46c6-8bbf-a1e4c0677e66 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:52.861191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e8ad9-d775-4dd3-9fb0-fecaca2f8950 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:52.862107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e8ad9-d775-4dd3-9fb0-fecaca2f8950 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:52.862402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e8ad9-d775-4dd3-9fb0-fecaca2f8950 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:52.862687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e8ad9-d775-4dd3-9fb0-fecaca2f8950 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/users/800876e92c474795b10c7871e9aee935` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:52.864451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e8ad9-d775-4dd3-9fb0-fecaca2f8950 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:delete_user(user_id=800876e92c474795b10c7871e9aee935)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:52.866444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e8ad9-d775-4dd3-9fb0-fecaca2f8950 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:52.923565 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2645/5262] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:52 2022] POST /identity/v3/users => generated 289 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:52.924736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e12071-6152-4ca8-9b5a-b69398169274 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:52.934862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e12071-6152-4ca8-9b5a-b69398169274 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:52.940258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0c223-48ea-4b45-a081-f5a9a3e8f535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:52.940953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0c223-48ea-4b45-a081-f5a9a3e8f535 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:52.946695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0c223-48ea-4b45-a081-f5a9a3e8f535 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:52.946695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0c223-48ea-4b45-a081-f5a9a3e8f535 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:52.946695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0c223-48ea-4b45-a081-f5a9a3e8f535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:52.946695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0c223-48ea-4b45-a081-f5a9a3e8f535 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:52.955109 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2646/5263] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:52.959282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0c223-48ea-4b45-a081-f5a9a3e8f535 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:52.969318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0c223-48ea-4b45-a081-f5a9a3e8f535 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:52.974110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c774f2-1e85-400c-8184-5cf8bc0f1b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:52.979166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c774f2-1e85-400c-8184-5cf8bc0f1b59 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:52.979166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c774f2-1e85-400c-8184-5cf8bc0f1b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:52.979166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c774f2-1e85-400c-8184-5cf8bc0f1b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:52.979166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c774f2-1e85-400c-8184-5cf8bc0f1b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:52.979166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c774f2-1e85-400c-8184-5cf8bc0f1b59 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:52.992945 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2647/5264] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:52.994018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c774f2-1e85-400c-8184-5cf8bc0f1b59 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:53.007759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c774f2-1e85-400c-8184-5cf8bc0f1b59 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:53.023195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b18287a-a054-4ef2-b33b-efe4270e1df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:53.024108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18287a-a054-4ef2-b33b-efe4270e1df5 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:53.024399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18287a-a054-4ef2-b33b-efe4270e1df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:53.024687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b18287a-a054-4ef2-b33b-efe4270e1df5 admin admin] PATH_INFO: `/v3/projects/84b2e3d985bd4e2fa098ffd8d7422fcb/users/96957ea63b5b47f4a2beb806064b1b63/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:53.030091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-eb8e8ad9-d775-4dd3-9fb0-fecaca2f8950 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '800876e92c474795b10c7871e9aee935'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:53.031052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-eb8e8ad9-d775-4dd3-9fb0-fecaca2f8950 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '800876e92c474795b10c7871e9aee935'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:53.044957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18287a-a054-4ef2-b33b-efe4270e1df5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b2e3d985bd4e2fa098ffd8d7422fcb, user_id=96957ea63b5b47f4a2beb806064b1b63, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:53.045650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-eb8e8ad9-d775-4dd3-9fb0-fecaca2f8950 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '800876e92c474795b10c7871e9aee935'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:53.047716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b18287a-a054-4ef2-b33b-efe4270e1df5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:53.120767 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2648/5265] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:19:52 2022] PUT /identity/v3/projects/84b2e3d985bd4e2fa098ffd8d7422fcb/users/96957ea63b5b47f4a2beb806064b1b63/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:53.132765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee9d2f-63a0-4f4c-9918-ed64fb73b91f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:53.133066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee9d2f-63a0-4f4c-9918-ed64fb73b91f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:53.133352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee9d2f-63a0-4f4c-9918-ed64fb73b91f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:53.147462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-eb8e8ad9-d775-4dd3-9fb0-fecaca2f8950 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '800876e92c474795b10c7871e9aee935'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:53.148345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-eb8e8ad9-d775-4dd3-9fb0-fecaca2f8950 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '800876e92c474795b10c7871e9aee935'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:53.162992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-48ee9d2f-63a0-4f4c-9918-ed64fb73b91f None None] MFA Rules not processed for user `96957ea63b5b47f4a2beb806064b1b63`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:53.164887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-48ee9d2f-63a0-4f4c-9918-ed64fb73b91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:53.168045 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2618/5266] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:52 2022] DELETE /identity/v3/users/800876e92c474795b10c7871e9aee935 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:53.172437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e8ad9-d775-4dd3-9fb0-fecaca2f8950 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:53.174052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-eb8e8ad9-d775-4dd3-9fb0-fecaca2f8950 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:53.237653 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2649/5267] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:19:53.247502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee9d2f-63a0-4f4c-9918-ed64fb73b91f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:53.249336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-48ee9d2f-63a0-4f4c-9918-ed64fb73b91f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:53.249480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e8ad9-d775-4dd3-9fb0-fecaca2f8950 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:53.257153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4860f-dc39-4c3b-ba7d-ce8d3cbba4f5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:53.258189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4860f-dc39-4c3b-ba7d-ce8d3cbba4f5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:53.258473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4860f-dc39-4c3b-ba7d-ce8d3cbba4f5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:53.258892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4860f-dc39-4c3b-ba7d-ce8d3cbba4f5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/users/f4be1d1766644de3a3fef9ccfffa7ce1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:53.260259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4860f-dc39-4c3b-ba7d-ce8d3cbba4f5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:delete_user(user_id=f4be1d1766644de3a3fef9ccfffa7ce1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:53.262387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4860f-dc39-4c3b-ba7d-ce8d3cbba4f5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:53.282914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:53.317086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee9d2f-63a0-4f4c-9918-ed64fb73b91f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:53.385890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a1081-5663-4a79-87c4-659aa45bf813 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:53.386711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a1081-5663-4a79-87c4-659aa45bf813 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:53.387007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a1081-5663-4a79-87c4-659aa45bf813 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:53.387492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a1081-5663-4a79-87c4-659aa45bf813 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:53.412497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-99b4860f-dc39-4c3b-ba7d-ce8d3cbba4f5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4be1d1766644de3a3fef9ccfffa7ce1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:53.414200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-99b4860f-dc39-4c3b-ba7d-ce8d3cbba4f5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4be1d1766644de3a3fef9ccfffa7ce1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:53.422437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-99b4860f-dc39-4c3b-ba7d-ce8d3cbba4f5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4be1d1766644de3a3fef9ccfffa7ce1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:53.469002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a1081-5663-4a79-87c4-659aa45bf813 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:53.469002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a1081-5663-4a79-87c4-659aa45bf813 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:53.482557 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2650/5268] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:19:53.526055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-99b4860f-dc39-4c3b-ba7d-ce8d3cbba4f5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4be1d1766644de3a3fef9ccfffa7ce1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:53.534709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-99b4860f-dc39-4c3b-ba7d-ce8d3cbba4f5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4be1d1766644de3a3fef9ccfffa7ce1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:53.557903 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2619/5269] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:53 2022] DELETE /identity/v3/users/f4be1d1766644de3a3fef9ccfffa7ce1 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:53.561264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a1081-5663-4a79-87c4-659aa45bf813 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:53.575424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a1081-5663-4a79-87c4-659aa45bf813 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:53.654987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5304d-4278-4af5-beaa-4c5a2ac46ef0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:53.656167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5304d-4278-4af5-beaa-4c5a2ac46ef0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:53.657385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5304d-4278-4af5-beaa-4c5a2ac46ef0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:53.658021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5304d-4278-4af5-beaa-4c5a2ac46ef0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/users/f74d21d8cc19471ab361bf8ed9937582` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:53.659977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5304d-4278-4af5-beaa-4c5a2ac46ef0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:delete_user(user_id=f74d21d8cc19471ab361bf8ed9937582)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:53.662618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5304d-4278-4af5-beaa-4c5a2ac46ef0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:53.857878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-afc5304d-4278-4af5-beaa-4c5a2ac46ef0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f74d21d8cc19471ab361bf8ed9937582'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:53.869616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-afc5304d-4278-4af5-beaa-4c5a2ac46ef0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f74d21d8cc19471ab361bf8ed9937582'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:53.871335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-afc5304d-4278-4af5-beaa-4c5a2ac46ef0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f74d21d8cc19471ab361bf8ed9937582'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:53.872100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-afc5304d-4278-4af5-beaa-4c5a2ac46ef0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f74d21d8cc19471ab361bf8ed9937582'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:53.956198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-afc5304d-4278-4af5-beaa-4c5a2ac46ef0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f74d21d8cc19471ab361bf8ed9937582'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:53.981341 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2651/5270] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:53 2022] DELETE /identity/v3/users/f74d21d8cc19471ab361bf8ed9937582 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:53.986026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4860f-dc39-4c3b-ba7d-ce8d3cbba4f5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:53.988149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-99b4860f-dc39-4c3b-ba7d-ce8d3cbba4f5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:54.005047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4860f-dc39-4c3b-ba7d-ce8d3cbba4f5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:54.123032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5988614-a6c1-497e-a2cd-7467f0d73091 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:54.125755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5988614-a6c1-497e-a2cd-7467f0d73091 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:54.127049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5988614-a6c1-497e-a2cd-7467f0d73091 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:54.127786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5988614-a6c1-497e-a2cd-7467f0d73091 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/42c1fe1544244266ba7139cd95023650` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:54.132307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5988614-a6c1-497e-a2cd-7467f0d73091 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:delete_group(group_id=42c1fe1544244266ba7139cd95023650)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:54.134298 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-e5988614-a6c1-497e-a2cd-7467f0d73091 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:54.134298 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:19:54.134298 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:54.136449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5988614-a6c1-497e-a2cd-7467f0d73091 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:54.288734 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2620/5271] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:19:53 2022] DELETE /identity/v3/groups/42c1fe1544244266ba7139cd95023650 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:54.294229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5304d-4278-4af5-beaa-4c5a2ac46ef0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:54.296217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-afc5304d-4278-4af5-beaa-4c5a2ac46ef0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:54.315979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5304d-4278-4af5-beaa-4c5a2ac46ef0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:54.446181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e188eb-3de8-430d-8a63-b7922816a4e5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:54.447217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e188eb-3de8-430d-8a63-b7922816a4e5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:54.447571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e188eb-3de8-430d-8a63-b7922816a4e5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:54.447926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e188eb-3de8-430d-8a63-b7922816a4e5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:54.449213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e188eb-3de8-430d-8a63-b7922816a4e5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:54.450374 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-60e188eb-3de8-430d-8a63-b7922816a4e5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:54.450374 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:19:54.450374 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:19:54.451808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e188eb-3de8-430d-8a63-b7922816a4e5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:54.631211 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2652/5272] 10.222.0.33 () {60 vars in 1146 bytes} [Sat May 28 21:19:54 2022] POST /identity/v3/groups => generated 327 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:54.635122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5988614-a6c1-497e-a2cd-7467f0d73091 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:54.648348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:54.652871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5988614-a6c1-497e-a2cd-7467f0d73091 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:54.666589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e380516-8cef-409e-aa6d-f792d71a65f6 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:54.667267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e380516-8cef-409e-aa6d-f792d71a65f6 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:54.667696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e380516-8cef-409e-aa6d-f792d71a65f6 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:54.667822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e380516-8cef-409e-aa6d-f792d71a65f6 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:54.668544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e380516-8cef-409e-aa6d-f792d71a65f6 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:54.670526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e380516-8cef-409e-aa6d-f792d71a65f6 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:54.790570 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2621/5273] 10.222.0.33 () {60 vars in 1146 bytes} [Sat May 28 21:19:54 2022] POST /identity/v3/groups => generated 324 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:54.794879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e188eb-3de8-430d-8a63-b7922816a4e5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:54.808976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e188eb-3de8-430d-8a63-b7922816a4e5 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:54.817069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796e759-c88a-4482-965c-1ff7b417c40d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:54.818174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796e759-c88a-4482-965c-1ff7b417c40d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:54.818469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796e759-c88a-4482-965c-1ff7b417c40d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:54.818756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2796e759-c88a-4482-965c-1ff7b417c40d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:54.819469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796e759-c88a-4482-965c-1ff7b417c40d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:54.821589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2796e759-c88a-4482-965c-1ff7b417c40d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:54.908923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e380516-8cef-409e-aa6d-f792d71a65f6 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:54.916672 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2653/5274] 10.222.0.33 () {60 vars in 1146 bytes} [Sat May 28 21:19:54 2022] POST /identity/v3/groups => generated 326 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:54.921589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e380516-8cef-409e-aa6d-f792d71a65f6 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:54.927981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2d951-4098-43c4-82ed-2910ff584a91 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:54.929573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2d951-4098-43c4-82ed-2910ff584a91 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:54.930167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2d951-4098-43c4-82ed-2910ff584a91 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:54.930849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e2d951-4098-43c4-82ed-2910ff584a91 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:54.931880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2d951-4098-43c4-82ed-2910ff584a91 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:list_groups()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:54.933160 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-82e2d951-4098-43c4-82ed-2910ff584a91 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_groups": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:19:54.933160 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:19:54.933160 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:19:54.934868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e2d951-4098-43c4-82ed-2910ff584a91 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:54.951885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796e759-c88a-4482-965c-1ff7b417c40d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:54.969393 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2622/5275] 10.222.0.33 () {58 vars in 1124 bytes} [Sat May 28 21:19:54 2022] GET /identity/v3/groups => generated 1487 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:19:54.974134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2796e759-c88a-4482-965c-1ff7b417c40d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:54.985724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ec05d-4a5e-4fe4-839c-16b82074297e tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:54.987162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ec05d-4a5e-4fe4-839c-16b82074297e tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:54.988733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ec05d-4a5e-4fe4-839c-16b82074297e tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:54.989460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ec05d-4a5e-4fe4-839c-16b82074297e tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/1e4690281dc94e0fb0d482f13b9eb604` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:54.995867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ec05d-4a5e-4fe4-839c-16b82074297e tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:delete_group(group_id=1e4690281dc94e0fb0d482f13b9eb604)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:54.998011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ec05d-4a5e-4fe4-839c-16b82074297e tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:55.104983 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2654/5276] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:19:54 2022] DELETE /identity/v3/groups/1e4690281dc94e0fb0d482f13b9eb604 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:55.106436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2d951-4098-43c4-82ed-2910ff584a91 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:55.122212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e2d951-4098-43c4-82ed-2910ff584a91 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:55.194593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2be682-5dcc-46a3-9ef5-3b6bdeb1a937 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:55.195520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2be682-5dcc-46a3-9ef5-3b6bdeb1a937 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:55.196175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2be682-5dcc-46a3-9ef5-3b6bdeb1a937 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:55.196870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2be682-5dcc-46a3-9ef5-3b6bdeb1a937 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/b8f0ebe7262843808bdcc53d88cdfd17` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:55.203046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2be682-5dcc-46a3-9ef5-3b6bdeb1a937 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:delete_group(group_id=b8f0ebe7262843808bdcc53d88cdfd17)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:55.205242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2be682-5dcc-46a3-9ef5-3b6bdeb1a937 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:55.224914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:55.336314 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2623/5277] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:19:55 2022] DELETE /identity/v3/groups/b8f0ebe7262843808bdcc53d88cdfd17 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:55.340166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ec05d-4a5e-4fe4-839c-16b82074297e tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:55.352251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ec05d-4a5e-4fe4-839c-16b82074297e tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:55.415923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe5a74-4758-4933-afbd-38cbfd09bfdf tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:55.416938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe5a74-4758-4933-afbd-38cbfd09bfdf tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:55.417294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe5a74-4758-4933-afbd-38cbfd09bfdf tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:55.418028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effe5a74-4758-4933-afbd-38cbfd09bfdf tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/7314cefd1d4a411a9848d681acebe939` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:55.424173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe5a74-4758-4933-afbd-38cbfd09bfdf tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:delete_group(group_id=7314cefd1d4a411a9848d681acebe939)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:55.426286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effe5a74-4758-4933-afbd-38cbfd09bfdf tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:55.606255 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2655/5278] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:19:55 2022] DELETE /identity/v3/groups/7314cefd1d4a411a9848d681acebe939 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:55.607350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2be682-5dcc-46a3-9ef5-3b6bdeb1a937 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:55.618677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2be682-5dcc-46a3-9ef5-3b6bdeb1a937 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:55.675748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1dc59-22c4-405e-9c24-a33adfb7b388 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:55.676434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1dc59-22c4-405e-9c24-a33adfb7b388 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:55.676724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1dc59-22c4-405e-9c24-a33adfb7b388 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:55.677004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1dc59-22c4-405e-9c24-a33adfb7b388 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:55.677756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1dc59-22c4-405e-9c24-a33adfb7b388 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:55.679775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1dc59-22c4-405e-9c24-a33adfb7b388 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:55.895590 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2624/5279] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:19:55 2022] POST /identity/v3/users => generated 323 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:55.899835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe5a74-4758-4933-afbd-38cbfd09bfdf tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:55.913573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effe5a74-4758-4933-afbd-38cbfd09bfdf tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:55.919102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48857f1-7196-46f5-b84f-bbfeb5b42945 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:55.920412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48857f1-7196-46f5-b84f-bbfeb5b42945 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:55.920702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48857f1-7196-46f5-b84f-bbfeb5b42945 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:55.920979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48857f1-7196-46f5-b84f-bbfeb5b42945 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:55.921712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48857f1-7196-46f5-b84f-bbfeb5b42945 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:55.923868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48857f1-7196-46f5-b84f-bbfeb5b42945 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:56.123487 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2656/5280] 10.222.0.33 () {60 vars in 1146 bytes} [Sat May 28 21:19:55 2022] POST /identity/v3/groups => generated 325 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:19:56.129374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1dc59-22c4-405e-9c24-a33adfb7b388 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:56.144042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1dc59-22c4-405e-9c24-a33adfb7b388 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:56.159788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39e584-7efe-4ba0-a4e9-b5df19effdc4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:56.161458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39e584-7efe-4ba0-a4e9-b5df19effdc4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:56.161801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39e584-7efe-4ba0-a4e9-b5df19effdc4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:56.162306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39e584-7efe-4ba0-a4e9-b5df19effdc4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/bd871288023441b7a1a096403906bbe6/users/927ce39bda1d46bb932fc38c6b54924c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:56.230521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39e584-7efe-4ba0-a4e9-b5df19effdc4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=bd871288023441b7a1a096403906bbe6, user_id=927ce39bda1d46bb932fc38c6b54924c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:56.236257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39e584-7efe-4ba0-a4e9-b5df19effdc4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:56.362910 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2625/5281] 10.222.0.33 () {60 vars in 1359 bytes} [Sat May 28 21:19:56 2022] PUT /identity/v3/groups/bd871288023441b7a1a096403906bbe6/users/927ce39bda1d46bb932fc38c6b54924c => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:56.367026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48857f1-7196-46f5-b84f-bbfeb5b42945 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:56.393182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48857f1-7196-46f5-b84f-bbfeb5b42945 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:56.455475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf97be9-5645-4b65-a4e4-781d1ee8b19a tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:56.456322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf97be9-5645-4b65-a4e4-781d1ee8b19a tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:56.456723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf97be9-5645-4b65-a4e4-781d1ee8b19a tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:56.457111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf97be9-5645-4b65-a4e4-781d1ee8b19a tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:56.458007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf97be9-5645-4b65-a4e4-781d1ee8b19a tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:56.460073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf97be9-5645-4b65-a4e4-781d1ee8b19a tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:56.595169 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2657/5282] 10.222.0.33 () {60 vars in 1146 bytes} [Sat May 28 21:19:56 2022] POST /identity/v3/groups => generated 327 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:19:56.599353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39e584-7efe-4ba0-a4e9-b5df19effdc4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:56.614993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39e584-7efe-4ba0-a4e9-b5df19effdc4 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:56.629891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c28367-b2c0-484c-85a1-efa96205fe4e tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:56.631349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c28367-b2c0-484c-85a1-efa96205fe4e tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:56.631844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c28367-b2c0-484c-85a1-efa96205fe4e tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:56.639097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c28367-b2c0-484c-85a1-efa96205fe4e tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/95e610d6b7634136b4e9d6da2aabf79e/users/927ce39bda1d46bb932fc38c6b54924c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:56.645628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c28367-b2c0-484c-85a1-efa96205fe4e tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=95e610d6b7634136b4e9d6da2aabf79e, user_id=927ce39bda1d46bb932fc38c6b54924c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:56.647862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c28367-b2c0-484c-85a1-efa96205fe4e tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:56.766670 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2626/5283] 10.222.0.33 () {60 vars in 1359 bytes} [Sat May 28 21:19:56 2022] PUT /identity/v3/groups/95e610d6b7634136b4e9d6da2aabf79e/users/927ce39bda1d46bb932fc38c6b54924c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:19:56.770755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf97be9-5645-4b65-a4e4-781d1ee8b19a tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:56.780776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf97be9-5645-4b65-a4e4-781d1ee8b19a tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:56.840645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62057643-75d2-448d-a152-752a4e739764 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:56.841573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62057643-75d2-448d-a152-752a4e739764 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:56.841925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62057643-75d2-448d-a152-752a4e739764 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:56.842244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62057643-75d2-448d-a152-752a4e739764 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/users/927ce39bda1d46bb932fc38c6b54924c/groups` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:56.844134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62057643-75d2-448d-a152-752a4e739764 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:list_groups_for_user(user_id=927ce39bda1d46bb932fc38c6b54924c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:56.846447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62057643-75d2-448d-a152-752a4e739764 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:56.865583 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2658/5284] 10.222.0.33 () {58 vars in 1241 bytes} [Sat May 28 21:19:56 2022] GET /identity/v3/users/927ce39bda1d46bb932fc38c6b54924c/groups => generated 838 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:56.866885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c28367-b2c0-484c-85a1-efa96205fe4e tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:56.877318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c28367-b2c0-484c-85a1-efa96205fe4e tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:56.882168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8654573-26b0-4e97-9035-d9250018937c tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:56.883025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8654573-26b0-4e97-9035-d9250018937c tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:56.883317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8654573-26b0-4e97-9035-d9250018937c tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:56.883601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8654573-26b0-4e97-9035-d9250018937c tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/95e610d6b7634136b4e9d6da2aabf79e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:56.885114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8654573-26b0-4e97-9035-d9250018937c tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:delete_group(group_id=95e610d6b7634136b4e9d6da2aabf79e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:56.887057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8654573-26b0-4e97-9035-d9250018937c tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:57.011937 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2627/5285] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:19:56 2022] DELETE /identity/v3/groups/95e610d6b7634136b4e9d6da2aabf79e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:57.016460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62057643-75d2-448d-a152-752a4e739764 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:57.027215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62057643-75d2-448d-a152-752a4e739764 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:57.089684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90ad36-ab9e-4eea-8866-df213379ccc9 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:57.090638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90ad36-ab9e-4eea-8866-df213379ccc9 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:57.090934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90ad36-ab9e-4eea-8866-df213379ccc9 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:57.091248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90ad36-ab9e-4eea-8866-df213379ccc9 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/groups/bd871288023441b7a1a096403906bbe6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:57.092605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90ad36-ab9e-4eea-8866-df213379ccc9 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:delete_group(group_id=bd871288023441b7a1a096403906bbe6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:57.095194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90ad36-ab9e-4eea-8866-df213379ccc9 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:57.097264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:57.238669 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2659/5286] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:19:57 2022] DELETE /identity/v3/groups/bd871288023441b7a1a096403906bbe6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:57.241253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8654573-26b0-4e97-9035-d9250018937c tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:57.256086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8654573-26b0-4e97-9035-d9250018937c tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:57.351157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d97479-0b0b-40de-a27e-696f5effe46d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:57.352451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d97479-0b0b-40de-a27e-696f5effe46d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:57.352849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d97479-0b0b-40de-a27e-696f5effe46d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:57.353379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d97479-0b0b-40de-a27e-696f5effe46d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/users/927ce39bda1d46bb932fc38c6b54924c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:57.356442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d97479-0b0b-40de-a27e-696f5effe46d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:delete_user(user_id=927ce39bda1d46bb932fc38c6b54924c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:57.365295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d97479-0b0b-40de-a27e-696f5effe46d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:57.580501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c3d97479-0b0b-40de-a27e-696f5effe46d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '927ce39bda1d46bb932fc38c6b54924c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:57.581266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c3d97479-0b0b-40de-a27e-696f5effe46d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '927ce39bda1d46bb932fc38c6b54924c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:57.588355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c3d97479-0b0b-40de-a27e-696f5effe46d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '927ce39bda1d46bb932fc38c6b54924c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:57.606450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:57.782978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c3d97479-0b0b-40de-a27e-696f5effe46d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '927ce39bda1d46bb932fc38c6b54924c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:57.783724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c3d97479-0b0b-40de-a27e-696f5effe46d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '927ce39bda1d46bb932fc38c6b54924c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:57.804486 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2628/5287] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:57 2022] DELETE /identity/v3/users/927ce39bda1d46bb932fc38c6b54924c => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:57.809522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90ad36-ab9e-4eea-8866-df213379ccc9 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:57.811678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ba90ad36-ab9e-4eea-8866-df213379ccc9 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:57.826036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90ad36-ab9e-4eea-8866-df213379ccc9 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:57.892275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5778a-afca-4b52-b8b5-2d5e4836d0e7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:57.893258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5778a-afca-4b52-b8b5-2d5e4836d0e7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:57.893628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5778a-afca-4b52-b8b5-2d5e4836d0e7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:57.894013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5778a-afca-4b52-b8b5-2d5e4836d0e7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/domains/2788c54ec5d6470fa8cb3821d34d91c8` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:57.895436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5778a-afca-4b52-b8b5-2d5e4836d0e7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=2788c54ec5d6470fa8cb3821d34d91c8)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:57.897319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5778a-afca-4b52-b8b5-2d5e4836d0e7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:57.927678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e9b5778a-afca-4b52-b8b5-2d5e4836d0e7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '2788c54ec5d6470fa8cb3821d34d91c8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:57.928622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e9b5778a-afca-4b52-b8b5-2d5e4836d0e7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '2788c54ec5d6470fa8cb3821d34d91c8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.090903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e9b5778a-afca-4b52-b8b5-2d5e4836d0e7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '2788c54ec5d6470fa8cb3821d34d91c8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.091673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e9b5778a-afca-4b52-b8b5-2d5e4836d0e7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '2788c54ec5d6470fa8cb3821d34d91c8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.095617 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2660/5288] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:19:57 2022] PATCH /identity/v3/domains/2788c54ec5d6470fa8cb3821d34d91c8 => generated 278 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:19:58.099441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d97479-0b0b-40de-a27e-696f5effe46d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:58.129440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d97479-0b0b-40de-a27e-696f5effe46d tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:58.200104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744d5fd-7459-41c9-accc-070515bee4f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7250636420f84a11a2b01026f4623aa5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6ed9816eca4442d0975d8b6eddef7ebe', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:58.200983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744d5fd-7459-41c9-accc-070515bee4f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:58.201268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744d5fd-7459-41c9-accc-070515bee4f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:58.201743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744d5fd-7459-41c9-accc-070515bee4f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] PATH_INFO: `/v3/domains/2788c54ec5d6470fa8cb3821d34d91c8` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:58.213578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744d5fd-7459-41c9-accc-070515bee4f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=2788c54ec5d6470fa8cb3821d34d91c8)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:58.215266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744d5fd-7459-41c9-accc-070515bee4f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:58.238550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0744d5fd-7459-41c9-accc-070515bee4f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '2788c54ec5d6470fa8cb3821d34d91c8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.241726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0744d5fd-7459-41c9-accc-070515bee4f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '2788c54ec5d6470fa8cb3821d34d91c8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.382734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0744d5fd-7459-41c9-accc-070515bee4f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2788c54ec5d6470fa8cb3821d34d91c8', 'request_id': 'req-0744d5fd-7459-41c9-accc-070515bee4f0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.388074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0744d5fd-7459-41c9-accc-070515bee4f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2788c54ec5d6470fa8cb3821d34d91c8', 'request_id': 'req-0744d5fd-7459-41c9-accc-070515bee4f0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.469356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0744d5fd-7459-41c9-accc-070515bee4f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2788c54ec5d6470fa8cb3821d34d91c8', 'request_id': 'req-0744d5fd-7459-41c9-accc-070515bee4f0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.482906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0744d5fd-7459-41c9-accc-070515bee4f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] The token cache is being invalidate because project 2788c54ec5d6470fa8cb3821d34d91c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:19:58.483287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0744d5fd-7459-41c9-accc-070515bee4f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.484185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0744d5fd-7459-41c9-accc-070515bee4f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.492805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0744d5fd-7459-41c9-accc-070515bee4f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '2788c54ec5d6470fa8cb3821d34d91c8', 'request_id': 'req-0744d5fd-7459-41c9-accc-070515bee4f0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.493490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0744d5fd-7459-41c9-accc-070515bee4f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '2788c54ec5d6470fa8cb3821d34d91c8', 'request_id': 'req-0744d5fd-7459-41c9-accc-070515bee4f0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.497771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0744d5fd-7459-41c9-accc-070515bee4f0 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '2788c54ec5d6470fa8cb3821d34d91c8', 'request_id': 'req-0744d5fd-7459-41c9-accc-070515bee4f0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.507048 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2629/5289] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:19:58 2022] DELETE /identity/v3/domains/2788c54ec5d6470fa8cb3821d34d91c8 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:58.508145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5778a-afca-4b52-b8b5-2d5e4836d0e7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:58.509633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e9b5778a-afca-4b52-b8b5-2d5e4836d0e7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:58.588134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5778a-afca-4b52-b8b5-2d5e4836d0e7 tempest-GroupsV3TestJSON-1581727476 tempest-GroupsV3TestJSON-1581727476-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:58.667643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33c256-1f1a-47e2-84c7-66ed62ce8f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:58.673325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33c256-1f1a-47e2-84c7-66ed62ce8f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:58.673626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33c256-1f1a-47e2-84c7-66ed62ce8f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:58.673944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb33c256-1f1a-47e2-84c7-66ed62ce8f21 admin admin] PATH_INFO: `/v3/users/2c4dffb8ed07419e9440524acf50c98b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:58.675794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33c256-1f1a-47e2-84c7-66ed62ce8f21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c4dffb8ed07419e9440524acf50c98b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:58.680219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb33c256-1f1a-47e2-84c7-66ed62ce8f21 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:58.854495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fb33c256-1f1a-47e2-84c7-66ed62ce8f21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c4dffb8ed07419e9440524acf50c98b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.861754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fb33c256-1f1a-47e2-84c7-66ed62ce8f21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c4dffb8ed07419e9440524acf50c98b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.866505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fb33c256-1f1a-47e2-84c7-66ed62ce8f21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c4dffb8ed07419e9440524acf50c98b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.871435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fb33c256-1f1a-47e2-84c7-66ed62ce8f21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c4dffb8ed07419e9440524acf50c98b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.965677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fb33c256-1f1a-47e2-84c7-66ed62ce8f21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c4dffb8ed07419e9440524acf50c98b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:58.985184 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2661/5290] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:19:58 2022] DELETE /identity/v3/users/2c4dffb8ed07419e9440524acf50c98b => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:19:58.989995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5b003-ebb4-401e-9b46-3b95284a92e0 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:58.990292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5b003-ebb4-401e-9b46-3b95284a92e0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:58.990597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5b003-ebb4-401e-9b46-3b95284a92e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:59.013405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-23d5b003-ebb4-401e-9b46-3b95284a92e0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:19:59.014936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-23d5b003-ebb4-401e-9b46-3b95284a92e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:59.086982 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2630/5291] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:19:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:19:59.097154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33c256-1f1a-47e2-84c7-66ed62ce8f21 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:59.098959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fb33c256-1f1a-47e2-84c7-66ed62ce8f21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:59.165332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb33c256-1f1a-47e2-84c7-66ed62ce8f21 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:59.169115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4880163-50bb-480c-bb82-f917b59cbca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:59.169922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4880163-50bb-480c-bb82-f917b59cbca1 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:59.170312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4880163-50bb-480c-bb82-f917b59cbca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:59.170688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4880163-50bb-480c-bb82-f917b59cbca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:59.185098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4880163-50bb-480c-bb82-f917b59cbca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:59.187149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4880163-50bb-480c-bb82-f917b59cbca1 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:59.197740 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2662/5292] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:19:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:19:59.699119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5b003-ebb4-401e-9b46-3b95284a92e0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:19:59.700678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-23d5b003-ebb4-401e-9b46-3b95284a92e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:19:59.715694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5b003-ebb4-401e-9b46-3b95284a92e0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:19:59.721208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d4c25-7f91-4b08-8380-43e10e191c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:19:59.722134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d4c25-7f91-4b08-8380-43e10e191c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:19:59.722476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d4c25-7f91-4b08-8380-43e10e191c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:19:59.722757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d4c25-7f91-4b08-8380-43e10e191c19 admin admin] PATH_INFO: `/v3/projects/57e7b3cb2f1d48d8a3af74c6692096a3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:19:59.728529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d4c25-7f91-4b08-8380-43e10e191c19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57e7b3cb2f1d48d8a3af74c6692096a3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:19:59.734905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d4c25-7f91-4b08-8380-43e10e191c19 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:19:59.737932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:59.937509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-141d4c25-7f91-4b08-8380-43e10e191c19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57e7b3cb2f1d48d8a3af74c6692096a3', 'request_id': 'req-141d4c25-7f91-4b08-8380-43e10e191c19'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:19:59.948239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-141d4c25-7f91-4b08-8380-43e10e191c19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57e7b3cb2f1d48d8a3af74c6692096a3', 'request_id': 'req-141d4c25-7f91-4b08-8380-43e10e191c19'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:00.051259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-141d4c25-7f91-4b08-8380-43e10e191c19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57e7b3cb2f1d48d8a3af74c6692096a3', 'request_id': 'req-141d4c25-7f91-4b08-8380-43e10e191c19'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:00.063848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-141d4c25-7f91-4b08-8380-43e10e191c19 admin admin] The token cache is being invalidate because project 57e7b3cb2f1d48d8a3af74c6692096a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:00.064255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-141d4c25-7f91-4b08-8380-43e10e191c19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:00.065093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-141d4c25-7f91-4b08-8380-43e10e191c19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:00.069653 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2631/5293] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:19:59 2022] DELETE /identity/v3/projects/57e7b3cb2f1d48d8a3af74c6692096a3 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:00.070945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4880163-50bb-480c-bb82-f917b59cbca1 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:00.081658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4880163-50bb-480c-bb82-f917b59cbca1 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:00.151046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06beacee-b597-4101-b22d-a9830850f6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:00.151923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06beacee-b597-4101-b22d-a9830850f6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:00.152206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06beacee-b597-4101-b22d-a9830850f6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:00.152480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06beacee-b597-4101-b22d-a9830850f6c0 admin admin] PATH_INFO: `/v3/users/7250636420f84a11a2b01026f4623aa5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:00.153669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06beacee-b597-4101-b22d-a9830850f6c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7250636420f84a11a2b01026f4623aa5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:00.155745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06beacee-b597-4101-b22d-a9830850f6c0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:00.469947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-06beacee-b597-4101-b22d-a9830850f6c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7250636420f84a11a2b01026f4623aa5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:00.485035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:00.492243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-06beacee-b597-4101-b22d-a9830850f6c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7250636420f84a11a2b01026f4623aa5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:00.493332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-06beacee-b597-4101-b22d-a9830850f6c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7250636420f84a11a2b01026f4623aa5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:00.494297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-06beacee-b597-4101-b22d-a9830850f6c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7250636420f84a11a2b01026f4623aa5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:00.622412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-06beacee-b597-4101-b22d-a9830850f6c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7250636420f84a11a2b01026f4623aa5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:00.654209 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2663/5294] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:00 2022] DELETE /identity/v3/users/7250636420f84a11a2b01026f4623aa5 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:01.476284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d4c25-7f91-4b08-8380-43e10e191c19 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:01.479068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-141d4c25-7f91-4b08-8380-43e10e191c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:01.495234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d4c25-7f91-4b08-8380-43e10e191c19 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:01.579187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc29e3-3a44-4955-bfd4-5141aab6d725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:01.580153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc29e3-3a44-4955-bfd4-5141aab6d725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:01.580678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc29e3-3a44-4955-bfd4-5141aab6d725 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:01.582445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc29e3-3a44-4955-bfd4-5141aab6d725 admin admin] PATH_INFO: `/v3/projects/6ed9816eca4442d0975d8b6eddef7ebe` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:01.584105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc29e3-3a44-4955-bfd4-5141aab6d725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ed9816eca4442d0975d8b6eddef7ebe)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:01.598506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc29e3-3a44-4955-bfd4-5141aab6d725 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:01.818267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-69fc29e3-3a44-4955-bfd4-5141aab6d725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ed9816eca4442d0975d8b6eddef7ebe', 'request_id': 'req-69fc29e3-3a44-4955-bfd4-5141aab6d725'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:01.824919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-69fc29e3-3a44-4955-bfd4-5141aab6d725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ed9816eca4442d0975d8b6eddef7ebe', 'request_id': 'req-69fc29e3-3a44-4955-bfd4-5141aab6d725'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:01.914515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-69fc29e3-3a44-4955-bfd4-5141aab6d725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ed9816eca4442d0975d8b6eddef7ebe', 'request_id': 'req-69fc29e3-3a44-4955-bfd4-5141aab6d725'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:01.935100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-69fc29e3-3a44-4955-bfd4-5141aab6d725 admin admin] The token cache is being invalidate because project 6ed9816eca4442d0975d8b6eddef7ebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:01.935399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-69fc29e3-3a44-4955-bfd4-5141aab6d725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:01.936178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-69fc29e3-3a44-4955-bfd4-5141aab6d725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:01.938851 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2632/5295] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:01 2022] DELETE /identity/v3/projects/6ed9816eca4442d0975d8b6eddef7ebe => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:01.943589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96054e4f-68d3-4c44-9b64-e2ce5dd2dc0a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:01.944525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96054e4f-68d3-4c44-9b64-e2ce5dd2dc0a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:01.945004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96054e4f-68d3-4c44-9b64-e2ce5dd2dc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:01.969286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-96054e4f-68d3-4c44-9b64-e2ce5dd2dc0a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:01.970886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-96054e4f-68d3-4c44-9b64-e2ce5dd2dc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:02.038631 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2664/5296] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:02.046414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31730f67-6b6c-4b2d-8f17-312a6a0286a3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:02.047058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31730f67-6b6c-4b2d-8f17-312a6a0286a3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:02.047563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31730f67-6b6c-4b2d-8f17-312a6a0286a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:02.077327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-31730f67-6b6c-4b2d-8f17-312a6a0286a3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:02.079151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-31730f67-6b6c-4b2d-8f17-312a6a0286a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:02.087727 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2633/5297] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:20:02.091748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96054e4f-68d3-4c44-9b64-e2ce5dd2dc0a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:02.102705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96054e4f-68d3-4c44-9b64-e2ce5dd2dc0a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:02.108463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c56d05-3409-4459-9780-a9acef5fbdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:02.109213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c56d05-3409-4459-9780-a9acef5fbdc2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:02.109506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c56d05-3409-4459-9780-a9acef5fbdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:02.109790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c56d05-3409-4459-9780-a9acef5fbdc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:02.110603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c56d05-3409-4459-9780-a9acef5fbdc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:20:02.110916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c56d05-3409-4459-9780-a9acef5fbdc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:02.112663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c56d05-3409-4459-9780-a9acef5fbdc2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:02.126272 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2665/5298] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:20:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:02.130820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31730f67-6b6c-4b2d-8f17-312a6a0286a3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:02.141765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31730f67-6b6c-4b2d-8f17-312a6a0286a3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:02.147192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d68fbe-5b48-4643-ba12-5b1ebfd3c452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:02.147862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d68fbe-5b48-4643-ba12-5b1ebfd3c452 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:02.148158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d68fbe-5b48-4643-ba12-5b1ebfd3c452 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:02.148425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d68fbe-5b48-4643-ba12-5b1ebfd3c452 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:02.149130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d68fbe-5b48-4643-ba12-5b1ebfd3c452 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:02.151107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d68fbe-5b48-4643-ba12-5b1ebfd3c452 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:02.237965 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2634/5299] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:02 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:02.243737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c56d05-3409-4459-9780-a9acef5fbdc2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:02.260517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c56d05-3409-4459-9780-a9acef5fbdc2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:02.329008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93c246-759a-42f3-8ff2-ec9d670eab51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:02.329692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93c246-759a-42f3-8ff2-ec9d670eab51 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:02.330210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93c246-759a-42f3-8ff2-ec9d670eab51 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:02.330492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d93c246-759a-42f3-8ff2-ec9d670eab51 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:02.331209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d93c246-759a-42f3-8ff2-ec9d670eab51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:02.333157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d93c246-759a-42f3-8ff2-ec9d670eab51 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:02.492697 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2666/5300] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:02 2022] POST /identity/v3/users => generated 289 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:20:02.497552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d68fbe-5b48-4643-ba12-5b1ebfd3c452 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:02.509181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d68fbe-5b48-4643-ba12-5b1ebfd3c452 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:02.515709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273a391-478e-4c2c-81b7-9fb47bb5c278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:02.518628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273a391-478e-4c2c-81b7-9fb47bb5c278 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:02.518628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273a391-478e-4c2c-81b7-9fb47bb5c278 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:02.518628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9273a391-478e-4c2c-81b7-9fb47bb5c278 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:02.518628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273a391-478e-4c2c-81b7-9fb47bb5c278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:02.520056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9273a391-478e-4c2c-81b7-9fb47bb5c278 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:02.532065 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2635/5301] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:02.536282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93c246-759a-42f3-8ff2-ec9d670eab51 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:02.548780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d93c246-759a-42f3-8ff2-ec9d670eab51 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:02.555992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269660fc-a72b-4a43-ad4a-add39addaa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:02.556780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269660fc-a72b-4a43-ad4a-add39addaa05 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:02.557136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269660fc-a72b-4a43-ad4a-add39addaa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:02.557504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269660fc-a72b-4a43-ad4a-add39addaa05 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:02.558383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269660fc-a72b-4a43-ad4a-add39addaa05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:02.560151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269660fc-a72b-4a43-ad4a-add39addaa05 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:02.578070 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2667/5302] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:02 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:02.578939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273a391-478e-4c2c-81b7-9fb47bb5c278 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:02.589646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9273a391-478e-4c2c-81b7-9fb47bb5c278 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:02.598079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b43a2b-a24e-483f-862b-bd5bc5140fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:02.598079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b43a2b-a24e-483f-862b-bd5bc5140fae admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:02.598079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b43a2b-a24e-483f-862b-bd5bc5140fae admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:02.598571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b43a2b-a24e-483f-862b-bd5bc5140fae admin admin] PATH_INFO: `/v3/projects/19167cf5c43d4308a278f7761729a621/users/04430a128abd41a1b9f3f1cdd205c6d2/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:02.614327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b43a2b-a24e-483f-862b-bd5bc5140fae admin admin] RBAC: Authorizing `identity:create_grant(project_id=19167cf5c43d4308a278f7761729a621, user_id=04430a128abd41a1b9f3f1cdd205c6d2, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:02.618031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b43a2b-a24e-483f-862b-bd5bc5140fae admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:02.733115 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2636/5303] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:02 2022] PUT /identity/v3/projects/19167cf5c43d4308a278f7761729a621/users/04430a128abd41a1b9f3f1cdd205c6d2/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:02.749174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f149c-754c-4cad-9912-a072c99a4998 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:02.749174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f149c-754c-4cad-9912-a072c99a4998 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:02.749174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f149c-754c-4cad-9912-a072c99a4998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:02.778607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-819f149c-754c-4cad-9912-a072c99a4998 None None] MFA Rules not processed for user `04430a128abd41a1b9f3f1cdd205c6d2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:02.780586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-819f149c-754c-4cad-9912-a072c99a4998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:02.880218 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2668/5304] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:02.884773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b43a2b-a24e-483f-862b-bd5bc5140fae admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:03.008961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b43a2b-a24e-483f-862b-bd5bc5140fae admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:03.022290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c104f-91ce-4446-8d20-d7b9bde13484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:03.023360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c104f-91ce-4446-8d20-d7b9bde13484 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:03.024160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c104f-91ce-4446-8d20-d7b9bde13484 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:03.024841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c104f-91ce-4446-8d20-d7b9bde13484 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:03.026288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c104f-91ce-4446-8d20-d7b9bde13484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:03.030001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c104f-91ce-4446-8d20-d7b9bde13484 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:03.173932 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2637/5305] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:02 2022] POST /identity/v3/projects => generated 375 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:03.177885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f149c-754c-4cad-9912-a072c99a4998 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:03.192368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819f149c-754c-4cad-9912-a072c99a4998 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:03.256137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d979d-71af-48fe-b9b6-0cc629f4a1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:03.256737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d979d-71af-48fe-b9b6-0cc629f4a1ab admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:03.257022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d979d-71af-48fe-b9b6-0cc629f4a1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:03.257300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d979d-71af-48fe-b9b6-0cc629f4a1ab admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:03.258054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d979d-71af-48fe-b9b6-0cc629f4a1ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:03.260195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d979d-71af-48fe-b9b6-0cc629f4a1ab admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:03.396344 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2669/5306] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:03 2022] POST /identity/v3/users => generated 294 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:03.400203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c104f-91ce-4446-8d20-d7b9bde13484 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:03.411032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c104f-91ce-4446-8d20-d7b9bde13484 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:03.416093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc9f2e-3d77-433e-bcf8-b2e4d509e42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:03.417469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc9f2e-3d77-433e-bcf8-b2e4d509e42d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:03.417469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc9f2e-3d77-433e-bcf8-b2e4d509e42d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:03.417696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bc9f2e-3d77-433e-bcf8-b2e4d509e42d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:03.420624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc9f2e-3d77-433e-bcf8-b2e4d509e42d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:03.420624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bc9f2e-3d77-433e-bcf8-b2e4d509e42d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:03.435566 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2638/5307] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:03.440323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:03.443476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d979d-71af-48fe-b9b6-0cc629f4a1ab admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:03.451135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d979d-71af-48fe-b9b6-0cc629f4a1ab admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:03.458732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d8b0e-c7b1-4c6a-b92b-9d47b057f984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:03.459643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d8b0e-c7b1-4c6a-b92b-9d47b057f984 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:03.460128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d8b0e-c7b1-4c6a-b92b-9d47b057f984 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:03.460582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d8b0e-c7b1-4c6a-b92b-9d47b057f984 admin admin] PATH_INFO: `/v3/projects/151aef2810e34a33829a1a6f2607a127/users/44e263506ca44804b6c2f05f9628589e/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:03.477556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d8b0e-c7b1-4c6a-b92b-9d47b057f984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=151aef2810e34a33829a1a6f2607a127, user_id=44e263506ca44804b6c2f05f9628589e, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:03.480591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d8b0e-c7b1-4c6a-b92b-9d47b057f984 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:03.664710 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2670/5308] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:03 2022] PUT /identity/v3/projects/151aef2810e34a33829a1a6f2607a127/users/44e263506ca44804b6c2f05f9628589e/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:03.668533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc9f2e-3d77-433e-bcf8-b2e4d509e42d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:03.680401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bc9f2e-3d77-433e-bcf8-b2e4d509e42d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:03.746896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987e6c1a-64d9-4843-b56e-09b1e84f8817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:03.747909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e6c1a-64d9-4843-b56e-09b1e84f8817 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:03.748226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e6c1a-64d9-4843-b56e-09b1e84f8817 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:03.748515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987e6c1a-64d9-4843-b56e-09b1e84f8817 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:03.749456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e6c1a-64d9-4843-b56e-09b1e84f8817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:03.751783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987e6c1a-64d9-4843-b56e-09b1e84f8817 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:03.762884 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2639/5309] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:03 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:20:03.767424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d8b0e-c7b1-4c6a-b92b-9d47b057f984 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:03.779169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d8b0e-c7b1-4c6a-b92b-9d47b057f984 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:03.785112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c27182-6645-4c32-8985-5c230b2f7b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:03.786110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c27182-6645-4c32-8985-5c230b2f7b82 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:03.786405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c27182-6645-4c32-8985-5c230b2f7b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:03.786688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c27182-6645-4c32-8985-5c230b2f7b82 admin admin] PATH_INFO: `/v3/projects/151aef2810e34a33829a1a6f2607a127/users/44e263506ca44804b6c2f05f9628589e/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:03.788955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c27182-6645-4c32-8985-5c230b2f7b82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=151aef2810e34a33829a1a6f2607a127, user_id=44e263506ca44804b6c2f05f9628589e, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:03.793829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c27182-6645-4c32-8985-5c230b2f7b82 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:03.806278 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2671/5310] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:03 2022] PUT /identity/v3/projects/151aef2810e34a33829a1a6f2607a127/users/44e263506ca44804b6c2f05f9628589e/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:03.818871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470ba32-3fce-4c3e-924d-0caafe454d20 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:03.825497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470ba32-3fce-4c3e-924d-0caafe454d20 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:03.825497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4470ba32-3fce-4c3e-924d-0caafe454d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:03.850086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4470ba32-3fce-4c3e-924d-0caafe454d20 None None] MFA Rules not processed for user `44e263506ca44804b6c2f05f9628589e`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:03.851235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4470ba32-3fce-4c3e-924d-0caafe454d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:03.932498 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2640/5311] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:03 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:03.933684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c27182-6645-4c32-8985-5c230b2f7b82 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:03.946432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c27182-6645-4c32-8985-5c230b2f7b82 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:03.951733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005c69c-713f-48fe-af63-f56731b9e0e5 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44e263506ca44804b6c2f05f9628589e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '151aef2810e34a33829a1a6f2607a127', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:03.952535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005c69c-713f-48fe-af63-f56731b9e0e5 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:03.952955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005c69c-713f-48fe-af63-f56731b9e0e5 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:03.953253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005c69c-713f-48fe-af63-f56731b9e0e5 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:03.954077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005c69c-713f-48fe-af63-f56731b9e0e5 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:03.955114 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-0005c69c-713f-48fe-af63-f56731b9e0e5 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:20:03.955114 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:20:03.955114 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:20:03.956016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0005c69c-713f-48fe-af63-f56731b9e0e5 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:04.056081 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2672/5312] 10.222.0.33 () {60 vars in 1184 bytes} [Sat May 28 21:20:03 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:20:04.057927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4470ba32-3fce-4c3e-924d-0caafe454d20 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:04.070660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4470ba32-3fce-4c3e-924d-0caafe454d20 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:04.078840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6748b-0e18-4768-a9cf-22065e63b7a7 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44e263506ca44804b6c2f05f9628589e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '151aef2810e34a33829a1a6f2607a127', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:04.079762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6748b-0e18-4768-a9cf-22065e63b7a7 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:04.080030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6748b-0e18-4768-a9cf-22065e63b7a7 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:04.080315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee6748b-0e18-4768-a9cf-22065e63b7a7 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/bd4ef296edaa487bae663981ce96a933` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:04.080993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6748b-0e18-4768-a9cf-22065e63b7a7 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=bd4ef296edaa487bae663981ce96a933)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:04.082158 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-bee6748b-0e18-4768-a9cf-22065e63b7a7 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_consumer": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:20:04.082158 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:20:04.082158 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:20:04.083069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee6748b-0e18-4768-a9cf-22065e63b7a7 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:04.093194 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2641/5313] 10.222.0.33 () {58 vars in 1262 bytes} [Sat May 28 21:20:04 2022] GET /identity/v3/OS-OAUTH1/consumers/bd4ef296edaa487bae663981ce96a933 => generated 220 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:04.097425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005c69c-713f-48fe-af63-f56731b9e0e5 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:04.111353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0005c69c-713f-48fe-af63-f56731b9e0e5 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:04.116457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b4ebf-ea8f-4dc3-8505-fe11ac5534f4 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44e263506ca44804b6c2f05f9628589e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '151aef2810e34a33829a1a6f2607a127', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:04.117371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b4ebf-ea8f-4dc3-8505-fe11ac5534f4 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:04.117775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b4ebf-ea8f-4dc3-8505-fe11ac5534f4 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:04.118230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844b4ebf-ea8f-4dc3-8505-fe11ac5534f4 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/bd4ef296edaa487bae663981ce96a933` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:04.119140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b4ebf-ea8f-4dc3-8505-fe11ac5534f4 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=bd4ef296edaa487bae663981ce96a933)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:04.120812 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-844b4ebf-ea8f-4dc3-8505-fe11ac5534f4 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:20:04.120812 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:20:04.120812 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:20:04.121489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844b4ebf-ea8f-4dc3-8505-fe11ac5534f4 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:04.121874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-844b4ebf-ea8f-4dc3-8505-fe11ac5534f4 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invalidating token cache because consumer bd4ef296edaa487bae663981ce96a933 has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:04.122267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-844b4ebf-ea8f-4dc3-8505-fe11ac5534f4 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:04.123646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-844b4ebf-ea8f-4dc3-8505-fe11ac5534f4 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:04.239512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-844b4ebf-ea8f-4dc3-8505-fe11ac5534f4 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'bd4ef296edaa487bae663981ce96a933', 'request_id': 'req-844b4ebf-ea8f-4dc3-8505-fe11ac5534f4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:04.360566 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2673/5314] 10.222.0.33 () {58 vars in 1265 bytes} [Sat May 28 21:20:04 2022] DELETE /identity/v3/OS-OAUTH1/consumers/bd4ef296edaa487bae663981ce96a933 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:04.365459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6748b-0e18-4768-a9cf-22065e63b7a7 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:04.367233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bee6748b-0e18-4768-a9cf-22065e63b7a7 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:04.379923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee6748b-0e18-4768-a9cf-22065e63b7a7 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:04.386756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c37c89-cc35-4f85-8c32-65b63c111912 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44e263506ca44804b6c2f05f9628589e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '151aef2810e34a33829a1a6f2607a127', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:04.386756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c37c89-cc35-4f85-8c32-65b63c111912 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:04.386756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c37c89-cc35-4f85-8c32-65b63c111912 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:04.387086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c37c89-cc35-4f85-8c32-65b63c111912 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:04.387086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c37c89-cc35-4f85-8c32-65b63c111912 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:04.387288 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-72c37c89-cc35-4f85-8c32-65b63c111912 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:20:04.387288 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:20:04.387288 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:20:04.388130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c37c89-cc35-4f85-8c32-65b63c111912 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:04.452424 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2642/5315] 10.222.0.33 () {60 vars in 1184 bytes} [Sat May 28 21:20:04 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 267 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:04.456617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b4ebf-ea8f-4dc3-8505-fe11ac5534f4 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:04.467263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844b4ebf-ea8f-4dc3-8505-fe11ac5534f4 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:04.472789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746d1fe-85ea-4c38-92b3-a482400abf97 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44e263506ca44804b6c2f05f9628589e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '151aef2810e34a33829a1a6f2607a127', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:04.473625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746d1fe-85ea-4c38-92b3-a482400abf97 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:04.473947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746d1fe-85ea-4c38-92b3-a482400abf97 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:04.474228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746d1fe-85ea-4c38-92b3-a482400abf97 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/caf35177a3ac41799478a8e9ae64bb8b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:04.474889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746d1fe-85ea-4c38-92b3-a482400abf97 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=caf35177a3ac41799478a8e9ae64bb8b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:04.475895 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-e746d1fe-85ea-4c38-92b3-a482400abf97 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_consumer": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:20:04.475895 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:20:04.475895 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:20:04.476805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e746d1fe-85ea-4c38-92b3-a482400abf97 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:04.482752 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2674/5316] 10.222.0.33 () {58 vars in 1262 bytes} [Sat May 28 21:20:04 2022] GET /identity/v3/OS-OAUTH1/consumers/caf35177a3ac41799478a8e9ae64bb8b => generated 221 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:04.487736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c37c89-cc35-4f85-8c32-65b63c111912 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:04.498593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c37c89-cc35-4f85-8c32-65b63c111912 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:04.503545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee6dd5-cc58-4780-b758-625ca069812c tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44e263506ca44804b6c2f05f9628589e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '151aef2810e34a33829a1a6f2607a127', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:04.504408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee6dd5-cc58-4780-b758-625ca069812c tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:04.504694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee6dd5-cc58-4780-b758-625ca069812c tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:04.504998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee6dd5-cc58-4780-b758-625ca069812c tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/caf35177a3ac41799478a8e9ae64bb8b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:04.505642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee6dd5-cc58-4780-b758-625ca069812c tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=caf35177a3ac41799478a8e9ae64bb8b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:04.506883 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-83ee6dd5-cc58-4780-b758-625ca069812c tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:20:04.506883 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:20:04.506883 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:20:04.508330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ee6dd5-cc58-4780-b758-625ca069812c tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:04.509100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-83ee6dd5-cc58-4780-b758-625ca069812c tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invalidating token cache because consumer caf35177a3ac41799478a8e9ae64bb8b has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:04.509100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-83ee6dd5-cc58-4780-b758-625ca069812c tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:04.509593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-83ee6dd5-cc58-4780-b758-625ca069812c tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:04.595005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:04.609673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-83ee6dd5-cc58-4780-b758-625ca069812c tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'caf35177a3ac41799478a8e9ae64bb8b', 'request_id': 'req-83ee6dd5-cc58-4780-b758-625ca069812c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:04.776880 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2643/5317] 10.222.0.33 () {58 vars in 1265 bytes} [Sat May 28 21:20:04 2022] DELETE /identity/v3/OS-OAUTH1/consumers/caf35177a3ac41799478a8e9ae64bb8b => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:04.782461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746d1fe-85ea-4c38-92b3-a482400abf97 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:04.786437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e746d1fe-85ea-4c38-92b3-a482400abf97 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:04.801782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e746d1fe-85ea-4c38-92b3-a482400abf97 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:04.806960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b0693-5917-4b48-abe3-5f82c309ba69 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44e263506ca44804b6c2f05f9628589e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '151aef2810e34a33829a1a6f2607a127', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:04.810200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b0693-5917-4b48-abe3-5f82c309ba69 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:04.810200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b0693-5917-4b48-abe3-5f82c309ba69 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:04.810200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b0693-5917-4b48-abe3-5f82c309ba69 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/caf35177a3ac41799478a8e9ae64bb8b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:04.810200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b0693-5917-4b48-abe3-5f82c309ba69 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=caf35177a3ac41799478a8e9ae64bb8b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:04.810840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b0693-5917-4b48-abe3-5f82c309ba69 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:04.830842 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-3c3b0693-5917-4b48-abe3-5f82c309ba69 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found May 28 21:20:04.830842 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:20:04.830842 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:20:04.830842 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:20:04.830842 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:20:04.830842 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:20:04.830842 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:20:04.830842 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:20:04.830842 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:20:04.830842 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:20:04.830842 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:20:04.830842 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:20:04.830842 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 100, in get May 28 21:20:04.830842 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_consumer(consumer_id) May 28 21:20:04.830842 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 95, in _get_consumer May 28 21:20:04.835493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.oauth_api.get_consumer(consumer_id)) May 28 21:20:04.835493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 101, in get_consumer May 28 21:20:04.835493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.get_consumer_with_secret(consumer_id)) May 28 21:20:04.835493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 96, in get_consumer_with_secret May 28 21:20:04.835493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) May 28 21:20:04.835493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer May 28 21:20:04.835493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) May 28 21:20:04.835493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found May 28 21:20:04.835493 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:20:04.835493 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2675/5318] 10.222.0.33 () {58 vars in 1262 bytes} [Sat May 28 21:20:04 2022] GET /identity/v3/OS-OAUTH1/consumers/caf35177a3ac41799478a8e9ae64bb8b => generated 74 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 21:20:04.837441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee6dd5-cc58-4780-b758-625ca069812c tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:04.848470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee6dd5-cc58-4780-b758-625ca069812c tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:04.853499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9e1e9-9d8f-48d9-8e80-fe80b269f8ec tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44e263506ca44804b6c2f05f9628589e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '151aef2810e34a33829a1a6f2607a127', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:04.854849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9e1e9-9d8f-48d9-8e80-fe80b269f8ec tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:04.855498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9e1e9-9d8f-48d9-8e80-fe80b269f8ec tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:04.855787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9e1e9-9d8f-48d9-8e80-fe80b269f8ec tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/caf35177a3ac41799478a8e9ae64bb8b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:04.858304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9e1e9-9d8f-48d9-8e80-fe80b269f8ec tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=caf35177a3ac41799478a8e9ae64bb8b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:04.858304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9e1e9-9d8f-48d9-8e80-fe80b269f8ec tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:04.858514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dbe9e1e9-9d8f-48d9-8e80-fe80b269f8ec tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invalidating token cache because consumer caf35177a3ac41799478a8e9ae64bb8b has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:04.858642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dbe9e1e9-9d8f-48d9-8e80-fe80b269f8ec tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:04.859353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dbe9e1e9-9d8f-48d9-8e80-fe80b269f8ec tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:04.883714 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-dbe9e1e9-9d8f-48d9-8e80-fe80b269f8ec tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Consumer not found: keystone.exception.NotFound: Consumer not found May 28 21:20:04.883714 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:20:04.883714 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:20:04.883714 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:20:04.883714 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:20:04.883714 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:20:04.883714 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:20:04.883714 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:20:04.883714 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:20:04.883714 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:20:04.883714 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:20:04.883714 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:20:04.883714 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/os_oauth1.py", line 123, in delete May 28 21:20:04.883714 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application PROVIDERS.oauth_api.delete_consumer( May 28 21:20:04.883714 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:20:04.885179 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:20:04.885179 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/core.py", line 139, in delete_consumer May 28 21:20:04.885179 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ret = self.driver.delete_consumer(consumer_id) May 28 21:20:04.885179 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 133, in delete_consumer May 28 21:20:04.885179 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._delete_consumer(session, consumer_id) May 28 21:20:04.885179 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 110, in _delete_consumer May 28 21:20:04.885179 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application consumer_ref = self._get_consumer(session, consumer_id) May 28 21:20:04.885179 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/oauth1/backends/sql.py", line 91, in _get_consumer May 28 21:20:04.885179 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.NotFound(_('Consumer not found')) May 28 21:20:04.885179 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.NotFound: Consumer not found May 28 21:20:04.885179 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:20:04.885179 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2644/5319] 10.222.0.33 () {58 vars in 1265 bytes} [Sat May 28 21:20:04 2022] DELETE /identity/v3/OS-OAUTH1/consumers/caf35177a3ac41799478a8e9ae64bb8b => generated 74 bytes in 48 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 21:20:04.890990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b0693-5917-4b48-abe3-5f82c309ba69 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:04.892768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3c3b0693-5917-4b48-abe3-5f82c309ba69 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:04.907285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b0693-5917-4b48-abe3-5f82c309ba69 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:04.912082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f8295-6011-4457-9c99-7c7cf649b02a tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44e263506ca44804b6c2f05f9628589e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '151aef2810e34a33829a1a6f2607a127', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:04.912828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f8295-6011-4457-9c99-7c7cf649b02a tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:04.913121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f8295-6011-4457-9c99-7c7cf649b02a tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:04.913407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f8295-6011-4457-9c99-7c7cf649b02a tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:04.914184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f8295-6011-4457-9c99-7c7cf649b02a tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:04.915786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f8295-6011-4457-9c99-7c7cf649b02a tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:05.017371 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2676/5320] 10.222.0.33 () {60 vars in 1184 bytes} [Sat May 28 21:20:04 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:05.018816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9e1e9-9d8f-48d9-8e80-fe80b269f8ec tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:05.035364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9e1e9-9d8f-48d9-8e80-fe80b269f8ec tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:05.041721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef45f1-d398-48a6-b562-94cfca5a66b3 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44e263506ca44804b6c2f05f9628589e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '151aef2810e34a33829a1a6f2607a127', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:05.043987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef45f1-d398-48a6-b562-94cfca5a66b3 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:05.044377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef45f1-d398-48a6-b562-94cfca5a66b3 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:05.044767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef45f1-d398-48a6-b562-94cfca5a66b3 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:05.046273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef45f1-d398-48a6-b562-94cfca5a66b3 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:05.048752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef45f1-d398-48a6-b562-94cfca5a66b3 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:05.129211 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2645/5321] 10.222.0.33 () {60 vars in 1184 bytes} [Sat May 28 21:20:05 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:20:05.133402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f8295-6011-4457-9c99-7c7cf649b02a tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:05.162984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f8295-6011-4457-9c99-7c7cf649b02a tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:05.176565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb258797-ef2f-4e97-99ea-bfb8ae194e8f tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44e263506ca44804b6c2f05f9628589e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '151aef2810e34a33829a1a6f2607a127', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:05.177310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb258797-ef2f-4e97-99ea-bfb8ae194e8f tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:05.178426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb258797-ef2f-4e97-99ea-bfb8ae194e8f tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:05.178534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb258797-ef2f-4e97-99ea-bfb8ae194e8f tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:05.179143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb258797-ef2f-4e97-99ea-bfb8ae194e8f tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorizing `identity:list_consumers()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:05.180174 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-eb258797-ef2f-4e97-99ea-bfb8ae194e8f tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_consumers": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:20:05.180174 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:20:05.180174 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:20:05.181084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb258797-ef2f-4e97-99ea-bfb8ae194e8f tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:05.194159 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2677/5322] 10.222.0.33 () {58 vars in 1163 bytes} [Sat May 28 21:20:05 2022] GET /identity/v3/OS-OAUTH1/consumers => generated 535 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:05.197116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef45f1-d398-48a6-b562-94cfca5a66b3 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:05.210354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef45f1-d398-48a6-b562-94cfca5a66b3 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:05.215574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6958096-f0fc-4c45-907c-eb27fc415a35 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44e263506ca44804b6c2f05f9628589e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '151aef2810e34a33829a1a6f2607a127', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:05.216920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6958096-f0fc-4c45-907c-eb27fc415a35 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:05.217222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6958096-f0fc-4c45-907c-eb27fc415a35 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:05.217510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6958096-f0fc-4c45-907c-eb27fc415a35 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/b7ef6fb922ad44809394f48010ca96fc` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:05.218287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6958096-f0fc-4c45-907c-eb27fc415a35 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=b7ef6fb922ad44809394f48010ca96fc)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:05.219950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6958096-f0fc-4c45-907c-eb27fc415a35 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:05.220226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d6958096-f0fc-4c45-907c-eb27fc415a35 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invalidating token cache because consumer b7ef6fb922ad44809394f48010ca96fc has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:05.220513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d6958096-f0fc-4c45-907c-eb27fc415a35 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:05.221216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d6958096-f0fc-4c45-907c-eb27fc415a35 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:05.310385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d6958096-f0fc-4c45-907c-eb27fc415a35 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'b7ef6fb922ad44809394f48010ca96fc', 'request_id': 'req-d6958096-f0fc-4c45-907c-eb27fc415a35'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:05.534010 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2646/5323] 10.222.0.33 () {58 vars in 1265 bytes} [Sat May 28 21:20:05 2022] DELETE /identity/v3/OS-OAUTH1/consumers/b7ef6fb922ad44809394f48010ca96fc => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:05.536772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb258797-ef2f-4e97-99ea-bfb8ae194e8f tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:05.538570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eb258797-ef2f-4e97-99ea-bfb8ae194e8f tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:05.572802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb258797-ef2f-4e97-99ea-bfb8ae194e8f tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:05.600071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ad309-c364-4a0b-9cd3-0a88290edf34 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44e263506ca44804b6c2f05f9628589e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '151aef2810e34a33829a1a6f2607a127', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:05.601671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ad309-c364-4a0b-9cd3-0a88290edf34 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:05.602009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ad309-c364-4a0b-9cd3-0a88290edf34 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:05.605513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183ad309-c364-4a0b-9cd3-0a88290edf34 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/cf2db17399244c62aef19fb0839c034a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:05.606568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ad309-c364-4a0b-9cd3-0a88290edf34 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=cf2db17399244c62aef19fb0839c034a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:05.610250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183ad309-c364-4a0b-9cd3-0a88290edf34 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:05.610588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-183ad309-c364-4a0b-9cd3-0a88290edf34 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invalidating token cache because consumer cf2db17399244c62aef19fb0839c034a has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:05.610964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-183ad309-c364-4a0b-9cd3-0a88290edf34 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:05.617286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-183ad309-c364-4a0b-9cd3-0a88290edf34 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:05.620150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6958096-f0fc-4c45-907c-eb27fc415a35 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:05.622802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d6958096-f0fc-4c45-907c-eb27fc415a35 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:05.761265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6958096-f0fc-4c45-907c-eb27fc415a35 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:05.770691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23e07e-5851-4e0c-ac14-060d901a5b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:05.774872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-183ad309-c364-4a0b-9cd3-0a88290edf34 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'cf2db17399244c62aef19fb0839c034a', 'request_id': 'req-183ad309-c364-4a0b-9cd3-0a88290edf34'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:05.775026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23e07e-5851-4e0c-ac14-060d901a5b65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:05.775026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23e07e-5851-4e0c-ac14-060d901a5b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:05.775026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23e07e-5851-4e0c-ac14-060d901a5b65 admin admin] PATH_INFO: `/v3/users/cfa39677553343bcaf62286df0ec6379` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:05.776352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23e07e-5851-4e0c-ac14-060d901a5b65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfa39677553343bcaf62286df0ec6379)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:05.778627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23e07e-5851-4e0c-ac14-060d901a5b65 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:05.871087 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2678/5324] 10.222.0.33 () {58 vars in 1265 bytes} [Sat May 28 21:20:05 2022] DELETE /identity/v3/OS-OAUTH1/consumers/cf2db17399244c62aef19fb0839c034a => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:05.872497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ad309-c364-4a0b-9cd3-0a88290edf34 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:05.874794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-183ad309-c364-4a0b-9cd3-0a88290edf34 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:05.890585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183ad309-c364-4a0b-9cd3-0a88290edf34 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:05.895690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b54f71-96e4-478f-9516-7b0554feb52e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44e263506ca44804b6c2f05f9628589e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '151aef2810e34a33829a1a6f2607a127', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:05.896699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b54f71-96e4-478f-9516-7b0554feb52e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:05.897195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b54f71-96e4-478f-9516-7b0554feb52e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:05.897486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b54f71-96e4-478f-9516-7b0554feb52e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:05.898287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b54f71-96e4-478f-9516-7b0554feb52e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorizing `identity:create_consumer()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:05.899889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b54f71-96e4-478f-9516-7b0554feb52e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:05.997736 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2679/5325] 10.222.0.33 () {60 vars in 1184 bytes} [Sat May 28 21:20:05 2022] POST /identity/v3/OS-OAUTH1/consumers => generated 266 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:05.999004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b54f71-96e4-478f-9516-7b0554feb52e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:06.013345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b54f71-96e4-478f-9516-7b0554feb52e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:06.019109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c68476-7f6d-4190-a40c-54e7966d453b tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44e263506ca44804b6c2f05f9628589e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '151aef2810e34a33829a1a6f2607a127', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:06.019953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c68476-7f6d-4190-a40c-54e7966d453b tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:06.020242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c68476-7f6d-4190-a40c-54e7966d453b tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:06.020532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c68476-7f6d-4190-a40c-54e7966d453b tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/caf6a45513af46b18275fe35b2f0219a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:06.021323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c68476-7f6d-4190-a40c-54e7966d453b tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorizing `identity:update_consumer(consumer_id=caf6a45513af46b18275fe35b2f0219a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:06.022392 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-a2c68476-7f6d-4190-a40c-54e7966d453b tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_consumer": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:20:06.022392 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:20:06.022392 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:20:06.023300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c68476-7f6d-4190-a40c-54e7966d453b tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:06.047897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6b23e07e-5851-4e0c-ac14-060d901a5b65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfa39677553343bcaf62286df0ec6379'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:06.048557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6b23e07e-5851-4e0c-ac14-060d901a5b65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfa39677553343bcaf62286df0ec6379'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:06.056062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6b23e07e-5851-4e0c-ac14-060d901a5b65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfa39677553343bcaf62286df0ec6379'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:06.157188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6b23e07e-5851-4e0c-ac14-060d901a5b65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfa39677553343bcaf62286df0ec6379'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:06.157896 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2680/5326] 10.222.0.33 () {60 vars in 1284 bytes} [Sat May 28 21:20:05 2022] PATCH /identity/v3/OS-OAUTH1/consumers/caf6a45513af46b18275fe35b2f0219a => generated 220 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:20:06.159364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6b23e07e-5851-4e0c-ac14-060d901a5b65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfa39677553343bcaf62286df0ec6379'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:06.162697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c68476-7f6d-4190-a40c-54e7966d453b tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:06.167110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a2c68476-7f6d-4190-a40c-54e7966d453b tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:06.212081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c68476-7f6d-4190-a40c-54e7966d453b tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:06.217768 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2647/5327] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:05 2022] DELETE /identity/v3/users/cfa39677553343bcaf62286df0ec6379 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:06.220169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45de5f-41fe-49d3-b940-3a6f0fd0181f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:06.220568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45de5f-41fe-49d3-b940-3a6f0fd0181f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:06.220952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45de5f-41fe-49d3-b940-3a6f0fd0181f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:06.246526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3b45de5f-41fe-49d3-b940-3a6f0fd0181f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:06.248497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3b45de5f-41fe-49d3-b940-3a6f0fd0181f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:06.312260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde7dc3-d2af-4f48-89f7-3f0b5b772f41 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44e263506ca44804b6c2f05f9628589e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '151aef2810e34a33829a1a6f2607a127', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:06.313218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde7dc3-d2af-4f48-89f7-3f0b5b772f41 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:06.313614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde7dc3-d2af-4f48-89f7-3f0b5b772f41 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:06.314010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde7dc3-d2af-4f48-89f7-3f0b5b772f41 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/caf6a45513af46b18275fe35b2f0219a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:06.315345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde7dc3-d2af-4f48-89f7-3f0b5b772f41 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorizing `identity:get_consumer(consumer_id=caf6a45513af46b18275fe35b2f0219a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:06.319798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cde7dc3-d2af-4f48-89f7-3f0b5b772f41 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:06.332985 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2681/5328] 10.222.0.33 () {58 vars in 1262 bytes} [Sat May 28 21:20:06 2022] GET /identity/v3/OS-OAUTH1/consumers/caf6a45513af46b18275fe35b2f0219a => generated 220 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:06.338456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde7dc3-d2af-4f48-89f7-3f0b5b772f41 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:06.342826 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2648/5329] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:20:06.351796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45de5f-41fe-49d3-b940-3a6f0fd0181f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:06.352714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde7dc3-d2af-4f48-89f7-3f0b5b772f41 tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:06.354404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3b45de5f-41fe-49d3-b940-3a6f0fd0181f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:06.359398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271d27c-ec2d-4e62-8e9d-3cf1bfc2301e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44e263506ca44804b6c2f05f9628589e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '151aef2810e34a33829a1a6f2607a127', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:06.360257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271d27c-ec2d-4e62-8e9d-3cf1bfc2301e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:06.360544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271d27c-ec2d-4e62-8e9d-3cf1bfc2301e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:06.360828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271d27c-ec2d-4e62-8e9d-3cf1bfc2301e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] PATH_INFO: `/v3/OS-OAUTH1/consumers/caf6a45513af46b18275fe35b2f0219a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:06.361539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271d27c-ec2d-4e62-8e9d-3cf1bfc2301e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorizing `identity:delete_consumer(consumer_id=caf6a45513af46b18275fe35b2f0219a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:06.365071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271d27c-ec2d-4e62-8e9d-3cf1bfc2301e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:06.365357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e271d27c-ec2d-4e62-8e9d-3cf1bfc2301e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invalidating token cache because consumer caf6a45513af46b18275fe35b2f0219a has been deleted. Authorization for users with OAuth tokens will be recalculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:06.365668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e271d27c-ec2d-4e62-8e9d-3cf1bfc2301e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:06.366671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e271d27c-ec2d-4e62-8e9d-3cf1bfc2301e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:06.443704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45de5f-41fe-49d3-b940-3a6f0fd0181f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:06.447912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fcad3-4244-4876-9121-b2c1fdc61e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:06.448698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fcad3-4244-4876-9121-b2c1fdc61e95 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:06.449309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fcad3-4244-4876-9121-b2c1fdc61e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:06.449515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fcad3-4244-4876-9121-b2c1fdc61e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:06.451659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-163fcad3-4244-4876-9121-b2c1fdc61e95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:06.465804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163fcad3-4244-4876-9121-b2c1fdc61e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:06.467371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163fcad3-4244-4876-9121-b2c1fdc61e95 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:06.477937 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2649/5330] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:20:06.502909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e271d27c-ec2d-4e62-8e9d-3cf1bfc2301e tempest-OAUTHConsumersV3Test-103692750 tempest-OAUTHConsumersV3Test-103692750-project-admin] Invoking callback _consumer_callback for event identity OS-OAUTH1:consumer deleted for {'resource_info': 'caf6a45513af46b18275fe35b2f0219a', 'request_id': 'req-e271d27c-ec2d-4e62-8e9d-3cf1bfc2301e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:06.689143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fcad3-4244-4876-9121-b2c1fdc61e95 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:06.690014 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2682/5331] 10.222.0.33 () {58 vars in 1265 bytes} [Sat May 28 21:20:06 2022] DELETE /identity/v3/OS-OAUTH1/consumers/caf6a45513af46b18275fe35b2f0219a => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:06.690782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-163fcad3-4244-4876-9121-b2c1fdc61e95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:06.706289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fcad3-4244-4876-9121-b2c1fdc61e95 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:06.714511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:06.722835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2fdbf-284a-47ea-873f-9586985d8dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:06.723736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2fdbf-284a-47ea-873f-9586985d8dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:06.724026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2fdbf-284a-47ea-873f-9586985d8dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:06.724339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2fdbf-284a-47ea-873f-9586985d8dd7 admin admin] PATH_INFO: `/v3/users/04430a128abd41a1b9f3f1cdd205c6d2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:06.725573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2fdbf-284a-47ea-873f-9586985d8dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04430a128abd41a1b9f3f1cdd205c6d2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:06.727593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2fdbf-284a-47ea-873f-9586985d8dd7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:06.955524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c9f2fdbf-284a-47ea-873f-9586985d8dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04430a128abd41a1b9f3f1cdd205c6d2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:06.956577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c9f2fdbf-284a-47ea-873f-9586985d8dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04430a128abd41a1b9f3f1cdd205c6d2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:06.966635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c9f2fdbf-284a-47ea-873f-9586985d8dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04430a128abd41a1b9f3f1cdd205c6d2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:07.092377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c9f2fdbf-284a-47ea-873f-9586985d8dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04430a128abd41a1b9f3f1cdd205c6d2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:07.096317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c9f2fdbf-284a-47ea-873f-9586985d8dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04430a128abd41a1b9f3f1cdd205c6d2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:07.115217 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2650/5332] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:06 2022] DELETE /identity/v3/users/04430a128abd41a1b9f3f1cdd205c6d2 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:07.120039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f3e63-4411-44b7-9fce-cacc187aabc8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:07.120334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f3e63-4411-44b7-9fce-cacc187aabc8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:07.120621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f3e63-4411-44b7-9fce-cacc187aabc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:07.151937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-780f3e63-4411-44b7-9fce-cacc187aabc8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:07.153128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-780f3e63-4411-44b7-9fce-cacc187aabc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:07.222039 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2683/5333] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:07.224932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2fdbf-284a-47ea-873f-9586985d8dd7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:07.226495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c9f2fdbf-284a-47ea-873f-9586985d8dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:07.240657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:07.250396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2fdbf-284a-47ea-873f-9586985d8dd7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:07.257498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae441f1-f3a8-45da-8d39-c79d35f980d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:07.258590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae441f1-f3a8-45da-8d39-c79d35f980d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:07.259002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae441f1-f3a8-45da-8d39-c79d35f980d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:07.259453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae441f1-f3a8-45da-8d39-c79d35f980d5 admin admin] PATH_INFO: `/v3/projects/530a8242ae014464b176c523ae99294e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:07.260925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae441f1-f3a8-45da-8d39-c79d35f980d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=530a8242ae014464b176c523ae99294e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:07.264043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f3e63-4411-44b7-9fce-cacc187aabc8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:07.264362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae441f1-f3a8-45da-8d39-c79d35f980d5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:07.267168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-780f3e63-4411-44b7-9fce-cacc187aabc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:07.341346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f3e63-4411-44b7-9fce-cacc187aabc8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:07.347167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0478ed7-5756-43d7-b420-aa972627071e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:07.348918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0478ed7-5756-43d7-b420-aa972627071e service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:07.349403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0478ed7-5756-43d7-b420-aa972627071e service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:07.350059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0478ed7-5756-43d7-b420-aa972627071e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:07.362581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0478ed7-5756-43d7-b420-aa972627071e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:07.364049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0478ed7-5756-43d7-b420-aa972627071e service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:07.374565 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2684/5334] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:07.399128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8ae441f1-f3a8-45da-8d39-c79d35f980d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '530a8242ae014464b176c523ae99294e', 'request_id': 'req-8ae441f1-f3a8-45da-8d39-c79d35f980d5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:07.410789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8ae441f1-f3a8-45da-8d39-c79d35f980d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '530a8242ae014464b176c523ae99294e', 'request_id': 'req-8ae441f1-f3a8-45da-8d39-c79d35f980d5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:07.611430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8ae441f1-f3a8-45da-8d39-c79d35f980d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '530a8242ae014464b176c523ae99294e', 'request_id': 'req-8ae441f1-f3a8-45da-8d39-c79d35f980d5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:07.625268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8ae441f1-f3a8-45da-8d39-c79d35f980d5 admin admin] The token cache is being invalidate because project 530a8242ae014464b176c523ae99294e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:07.625807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8ae441f1-f3a8-45da-8d39-c79d35f980d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:07.627006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8ae441f1-f3a8-45da-8d39-c79d35f980d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:07.628760 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2651/5335] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:07 2022] DELETE /identity/v3/projects/530a8242ae014464b176c523ae99294e => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:07.634390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0478ed7-5756-43d7-b420-aa972627071e service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:07.648296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0478ed7-5756-43d7-b420-aa972627071e service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:07.733884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc41ff-1caa-4d6c-a9df-65fdfb826393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:07.733884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc41ff-1caa-4d6c-a9df-65fdfb826393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:07.733884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc41ff-1caa-4d6c-a9df-65fdfb826393 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:07.733884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc41ff-1caa-4d6c-a9df-65fdfb826393 admin admin] PATH_INFO: `/v3/users/89e228766caf482abaee867cc932587f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:07.734738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc41ff-1caa-4d6c-a9df-65fdfb826393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89e228766caf482abaee867cc932587f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:07.736296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85cc41ff-1caa-4d6c-a9df-65fdfb826393 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:07.907745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-85cc41ff-1caa-4d6c-a9df-65fdfb826393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89e228766caf482abaee867cc932587f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:07.915587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-85cc41ff-1caa-4d6c-a9df-65fdfb826393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89e228766caf482abaee867cc932587f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:07.916250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-85cc41ff-1caa-4d6c-a9df-65fdfb826393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89e228766caf482abaee867cc932587f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:07.916882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-85cc41ff-1caa-4d6c-a9df-65fdfb826393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89e228766caf482abaee867cc932587f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:07.931194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae441f1-f3a8-45da-8d39-c79d35f980d5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:07.934860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8ae441f1-f3a8-45da-8d39-c79d35f980d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:07.980266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae441f1-f3a8-45da-8d39-c79d35f980d5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:07.986997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c774d-629f-4105-929a-98fcd46b7362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:07.988069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c774d-629f-4105-929a-98fcd46b7362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:07.988615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c774d-629f-4105-929a-98fcd46b7362 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:07.988902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c774d-629f-4105-929a-98fcd46b7362 admin admin] PATH_INFO: `/v3/projects/19167cf5c43d4308a278f7761729a621` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:07.990593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c774d-629f-4105-929a-98fcd46b7362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19167cf5c43d4308a278f7761729a621)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:07.993074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c774d-629f-4105-929a-98fcd46b7362 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:08.023075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-85cc41ff-1caa-4d6c-a9df-65fdfb826393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89e228766caf482abaee867cc932587f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:08.053820 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2685/5336] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:07 2022] DELETE /identity/v3/users/89e228766caf482abaee867cc932587f => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:08.162574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b79c774d-629f-4105-929a-98fcd46b7362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19167cf5c43d4308a278f7761729a621', 'request_id': 'req-b79c774d-629f-4105-929a-98fcd46b7362'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:08.169409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b79c774d-629f-4105-929a-98fcd46b7362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19167cf5c43d4308a278f7761729a621', 'request_id': 'req-b79c774d-629f-4105-929a-98fcd46b7362'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:08.308711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b79c774d-629f-4105-929a-98fcd46b7362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19167cf5c43d4308a278f7761729a621', 'request_id': 'req-b79c774d-629f-4105-929a-98fcd46b7362'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:08.321917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b79c774d-629f-4105-929a-98fcd46b7362 admin admin] The token cache is being invalidate because project 19167cf5c43d4308a278f7761729a621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:08.323873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b79c774d-629f-4105-929a-98fcd46b7362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:08.324692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b79c774d-629f-4105-929a-98fcd46b7362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:08.326327 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2652/5337] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:07 2022] DELETE /identity/v3/projects/19167cf5c43d4308a278f7761729a621 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:08.331451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc41ff-1caa-4d6c-a9df-65fdfb826393 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:08.333004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-85cc41ff-1caa-4d6c-a9df-65fdfb826393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:08.411344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc41ff-1caa-4d6c-a9df-65fdfb826393 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:08.487456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e78383-b29b-4669-933e-b9fc401d7155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:08.488355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e78383-b29b-4669-933e-b9fc401d7155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:08.488648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e78383-b29b-4669-933e-b9fc401d7155 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:08.488931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e78383-b29b-4669-933e-b9fc401d7155 admin admin] PATH_INFO: `/v3/users/44e263506ca44804b6c2f05f9628589e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:08.490273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e78383-b29b-4669-933e-b9fc401d7155 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44e263506ca44804b6c2f05f9628589e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:08.495730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e78383-b29b-4669-933e-b9fc401d7155 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:08.515795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:08.743890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c774d-629f-4105-929a-98fcd46b7362 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:08.745509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b79c774d-629f-4105-929a-98fcd46b7362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:08.762331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:08.778498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a8e78383-b29b-4669-933e-b9fc401d7155 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44e263506ca44804b6c2f05f9628589e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:08.778955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c774d-629f-4105-929a-98fcd46b7362 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:08.783721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f92ae-c4cd-4b59-8d02-fb8f18564d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:08.784609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f92ae-c4cd-4b59-8d02-fb8f18564d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:08.784945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f92ae-c4cd-4b59-8d02-fb8f18564d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:08.785232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f92ae-c4cd-4b59-8d02-fb8f18564d02 admin admin] PATH_INFO: `/v3/projects/629225485fc244deab41c69a42a71ee6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:08.786799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f92ae-c4cd-4b59-8d02-fb8f18564d02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=629225485fc244deab41c69a42a71ee6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:08.788811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f92ae-c4cd-4b59-8d02-fb8f18564d02 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:08.792436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a8e78383-b29b-4669-933e-b9fc401d7155 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44e263506ca44804b6c2f05f9628589e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:08.793409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a8e78383-b29b-4669-933e-b9fc401d7155 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44e263506ca44804b6c2f05f9628589e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:08.794234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a8e78383-b29b-4669-933e-b9fc401d7155 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44e263506ca44804b6c2f05f9628589e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:08.881935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a8e78383-b29b-4669-933e-b9fc401d7155 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44e263506ca44804b6c2f05f9628589e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:08.917256 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2686/5338] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:08 2022] DELETE /identity/v3/users/44e263506ca44804b6c2f05f9628589e => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:08.936769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e78383-b29b-4669-933e-b9fc401d7155 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:08.938277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a8e78383-b29b-4669-933e-b9fc401d7155 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:08.967084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f0f92ae-c4cd-4b59-8d02-fb8f18564d02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '629225485fc244deab41c69a42a71ee6', 'request_id': 'req-0f0f92ae-c4cd-4b59-8d02-fb8f18564d02'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:08.975087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f0f92ae-c4cd-4b59-8d02-fb8f18564d02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '629225485fc244deab41c69a42a71ee6', 'request_id': 'req-0f0f92ae-c4cd-4b59-8d02-fb8f18564d02'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:09.030623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e78383-b29b-4669-933e-b9fc401d7155 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:09.091449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f0f92ae-c4cd-4b59-8d02-fb8f18564d02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '629225485fc244deab41c69a42a71ee6', 'request_id': 'req-0f0f92ae-c4cd-4b59-8d02-fb8f18564d02'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:09.093096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461df2ad-b577-4432-a6a1-d91bedb7a343 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:09.094370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461df2ad-b577-4432-a6a1-d91bedb7a343 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:09.094654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461df2ad-b577-4432-a6a1-d91bedb7a343 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:09.094923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461df2ad-b577-4432-a6a1-d91bedb7a343 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:09.096998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-461df2ad-b577-4432-a6a1-d91bedb7a343 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:09.109144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f0f92ae-c4cd-4b59-8d02-fb8f18564d02 admin admin] The token cache is being invalidate because project 629225485fc244deab41c69a42a71ee6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:09.109701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f0f92ae-c4cd-4b59-8d02-fb8f18564d02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:09.110775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f0f92ae-c4cd-4b59-8d02-fb8f18564d02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:09.113245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461df2ad-b577-4432-a6a1-d91bedb7a343 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:09.113615 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2653/5339] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:08 2022] DELETE /identity/v3/projects/629225485fc244deab41c69a42a71ee6 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:09.115070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461df2ad-b577-4432-a6a1-d91bedb7a343 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:09.118301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88355ce5-e90f-4e72-9c33-d627d06a0d89 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:09.118604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88355ce5-e90f-4e72-9c33-d627d06a0d89 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:09.119027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88355ce5-e90f-4e72-9c33-d627d06a0d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:09.131072 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2687/5340] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:09.177716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-88355ce5-e90f-4e72-9c33-d627d06a0d89 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:09.180020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-88355ce5-e90f-4e72-9c33-d627d06a0d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:09.254313 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2654/5341] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:09.266645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997d2ad-b6d1-4754-b33b-8ce31bbcc619 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:09.267017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997d2ad-b6d1-4754-b33b-8ce31bbcc619 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:09.267369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997d2ad-b6d1-4754-b33b-8ce31bbcc619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:09.293360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2997d2ad-b6d1-4754-b33b-8ce31bbcc619 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:09.295273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2997d2ad-b6d1-4754-b33b-8ce31bbcc619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:09.307683 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2688/5342] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:09.309077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88355ce5-e90f-4e72-9c33-d627d06a0d89 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:09.320572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88355ce5-e90f-4e72-9c33-d627d06a0d89 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:09.325779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80a34d-f538-4d56-921a-ce7a522725ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:09.326559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80a34d-f538-4d56-921a-ce7a522725ee admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:09.327063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80a34d-f538-4d56-921a-ce7a522725ee admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:09.327340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80a34d-f538-4d56-921a-ce7a522725ee admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:09.328839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80a34d-f538-4d56-921a-ce7a522725ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:20:09.329185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80a34d-f538-4d56-921a-ce7a522725ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:09.330831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80a34d-f538-4d56-921a-ce7a522725ee admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:09.344910 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2655/5343] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:09.349336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997d2ad-b6d1-4754-b33b-8ce31bbcc619 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:09.360372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997d2ad-b6d1-4754-b33b-8ce31bbcc619 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:09.372250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed2b2c-9d94-4fa8-bbc8-3103c4817ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:09.374403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed2b2c-9d94-4fa8-bbc8-3103c4817ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:09.376277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed2b2c-9d94-4fa8-bbc8-3103c4817ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:09.376784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed2b2c-9d94-4fa8-bbc8-3103c4817ff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:09.377585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed2b2c-9d94-4fa8-bbc8-3103c4817ff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:09.379667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed2b2c-9d94-4fa8-bbc8-3103c4817ff2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:09.523216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80a34d-f538-4d56-921a-ce7a522725ee admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:09.525150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8a80a34d-f538-4d56-921a-ce7a522725ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:09.542187 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2689/5344] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:09 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:09.543143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed2b2c-9d94-4fa8-bbc8-3103c4817ff2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:09.552564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80a34d-f538-4d56-921a-ce7a522725ee admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:09.642134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82562521-6195-4769-9b8f-fc7c27f13c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:09.643030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82562521-6195-4769-9b8f-fc7c27f13c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:09.643329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82562521-6195-4769-9b8f-fc7c27f13c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:09.643905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82562521-6195-4769-9b8f-fc7c27f13c79 admin admin] PATH_INFO: `/v3/projects/151aef2810e34a33829a1a6f2607a127` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:09.645178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82562521-6195-4769-9b8f-fc7c27f13c79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=151aef2810e34a33829a1a6f2607a127)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:09.647231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82562521-6195-4769-9b8f-fc7c27f13c79 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:09.672480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed2b2c-9d94-4fa8-bbc8-3103c4817ff2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:09.680322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d599de-aa20-437c-aa4c-1c44791b735c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:09.680849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d599de-aa20-437c-aa4c-1c44791b735c admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:09.681201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d599de-aa20-437c-aa4c-1c44791b735c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:09.681501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d599de-aa20-437c-aa4c-1c44791b735c admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:09.682873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d599de-aa20-437c-aa4c-1c44791b735c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:09.685320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d599de-aa20-437c-aa4c-1c44791b735c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:09.781684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-82562521-6195-4769-9b8f-fc7c27f13c79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '151aef2810e34a33829a1a6f2607a127', 'request_id': 'req-82562521-6195-4769-9b8f-fc7c27f13c79'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:09.788571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-82562521-6195-4769-9b8f-fc7c27f13c79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '151aef2810e34a33829a1a6f2607a127', 'request_id': 'req-82562521-6195-4769-9b8f-fc7c27f13c79'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:09.792728 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2690/5345] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:09 2022] POST /identity/v3/users => generated 289 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:09.796996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d599de-aa20-437c-aa4c-1c44791b735c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:09.811933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d599de-aa20-437c-aa4c-1c44791b735c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:09.878381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157c07d-bf31-4d68-ad42-167862f0cbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:09.879096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157c07d-bf31-4d68-ad42-167862f0cbad admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:09.879386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157c07d-bf31-4d68-ad42-167862f0cbad admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:09.879666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157c07d-bf31-4d68-ad42-167862f0cbad admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:09.880412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157c07d-bf31-4d68-ad42-167862f0cbad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:09.882164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157c07d-bf31-4d68-ad42-167862f0cbad admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:09.885932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-82562521-6195-4769-9b8f-fc7c27f13c79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '151aef2810e34a33829a1a6f2607a127', 'request_id': 'req-82562521-6195-4769-9b8f-fc7c27f13c79'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:09.894248 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2691/5346] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:09.898212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157c07d-bf31-4d68-ad42-167862f0cbad admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:09.900722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-82562521-6195-4769-9b8f-fc7c27f13c79 admin admin] The token cache is being invalidate because project 151aef2810e34a33829a1a6f2607a127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:09.900722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-82562521-6195-4769-9b8f-fc7c27f13c79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:09.902022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-82562521-6195-4769-9b8f-fc7c27f13c79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:09.903726 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2656/5347] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:09 2022] DELETE /identity/v3/projects/151aef2810e34a33829a1a6f2607a127 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:09.908602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc3576-4903-451d-a424-0c2fe12b8649 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:09.913514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc3576-4903-451d-a424-0c2fe12b8649 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:09.913514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc3576-4903-451d-a424-0c2fe12b8649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:09.913987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157c07d-bf31-4d68-ad42-167862f0cbad admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:09.915179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875949e8-cdcd-45dd-8bcc-44fabc093039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:09.915920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875949e8-cdcd-45dd-8bcc-44fabc093039 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:09.916294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875949e8-cdcd-45dd-8bcc-44fabc093039 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:09.916669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875949e8-cdcd-45dd-8bcc-44fabc093039 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:09.917470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875949e8-cdcd-45dd-8bcc-44fabc093039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:09.919199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875949e8-cdcd-45dd-8bcc-44fabc093039 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:09.930349 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2692/5348] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:20:09.931179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-40bc3576-4903-451d-a424-0c2fe12b8649 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:09.932449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-40bc3576-4903-451d-a424-0c2fe12b8649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:09.934092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875949e8-cdcd-45dd-8bcc-44fabc093039 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:09.935907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-875949e8-cdcd-45dd-8bcc-44fabc093039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:09.942051 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2657/5349] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:20:09.955499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513bc50-2441-413e-8bb2-39995a26a332 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:09.956060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513bc50-2441-413e-8bb2-39995a26a332 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:09.956586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a513bc50-2441-413e-8bb2-39995a26a332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:09.962085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875949e8-cdcd-45dd-8bcc-44fabc093039 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:09.987494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8eddb8-ff4c-40b2-b989-48346bd6b8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:09.987494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8eddb8-ff4c-40b2-b989-48346bd6b8ed admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:09.987494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8eddb8-ff4c-40b2-b989-48346bd6b8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:09.987836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8eddb8-ff4c-40b2-b989-48346bd6b8ed admin admin] PATH_INFO: `/v3/projects/05724d255c4d44eebb233af520a321f3/users/c6817a6607aa48218bea76d56b0eb1f0/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:09.987942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a513bc50-2441-413e-8bb2-39995a26a332 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:09.987942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a513bc50-2441-413e-8bb2-39995a26a332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:09.988109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8eddb8-ff4c-40b2-b989-48346bd6b8ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=05724d255c4d44eebb233af520a321f3, user_id=c6817a6607aa48218bea76d56b0eb1f0, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:09.989196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8eddb8-ff4c-40b2-b989-48346bd6b8ed admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:09.994236 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2658/5350] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:09.998670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513bc50-2441-413e-8bb2-39995a26a332 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:10.008483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a513bc50-2441-413e-8bb2-39995a26a332 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:10.013225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495451c-ffd0-4763-902f-d8c9044dca53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:10.013904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495451c-ffd0-4763-902f-d8c9044dca53 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:10.014188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495451c-ffd0-4763-902f-d8c9044dca53 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:10.016726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495451c-ffd0-4763-902f-d8c9044dca53 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:10.016726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495451c-ffd0-4763-902f-d8c9044dca53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:20:10.016726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495451c-ffd0-4763-902f-d8c9044dca53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:10.017122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495451c-ffd0-4763-902f-d8c9044dca53 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:10.029307 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2659/5351] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:10.033742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495451c-ffd0-4763-902f-d8c9044dca53 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:10.043730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495451c-ffd0-4763-902f-d8c9044dca53 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:10.048865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987de98-9235-47dc-8f7b-4cffe7fb8393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:10.049549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987de98-9235-47dc-8f7b-4cffe7fb8393 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:10.049865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987de98-9235-47dc-8f7b-4cffe7fb8393 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:10.050138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1987de98-9235-47dc-8f7b-4cffe7fb8393 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:10.050842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987de98-9235-47dc-8f7b-4cffe7fb8393 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:10.052738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1987de98-9235-47dc-8f7b-4cffe7fb8393 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:10.103047 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2693/5352] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:09 2022] PUT /identity/v3/projects/05724d255c4d44eebb233af520a321f3/users/c6817a6607aa48218bea76d56b0eb1f0/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:10.114628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae45ae-b118-418f-9c71-9a9afe4cfecc None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:10.115069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae45ae-b118-418f-9c71-9a9afe4cfecc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:10.115370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae45ae-b118-418f-9c71-9a9afe4cfecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:10.144257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-afae45ae-b118-418f-9c71-9a9afe4cfecc None None] MFA Rules not processed for user `c6817a6607aa48218bea76d56b0eb1f0`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:10.145496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-afae45ae-b118-418f-9c71-9a9afe4cfecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:10.180107 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2660/5353] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:10 2022] POST /identity/v3/projects => generated 367 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:20:10.184168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987de98-9235-47dc-8f7b-4cffe7fb8393 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:10.230671 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2694/5354] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:20:10.242089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae45ae-b118-418f-9c71-9a9afe4cfecc None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:10.262067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1987de98-9235-47dc-8f7b-4cffe7fb8393 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:10.267353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30abfc2e-e542-4536-b253-893177ae875d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:10.268035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abfc2e-e542-4536-b253-893177ae875d admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:10.268324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abfc2e-e542-4536-b253-893177ae875d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:10.269024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30abfc2e-e542-4536-b253-893177ae875d admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:10.269435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30abfc2e-e542-4536-b253-893177ae875d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:10.271573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30abfc2e-e542-4536-b253-893177ae875d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:10.308191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae45ae-b118-418f-9c71-9a9afe4cfecc None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:10.312749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc3d04-c0e0-44fa-8ecf-fdf836b82471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:10.313408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc3d04-c0e0-44fa-8ecf-fdf836b82471 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:10.313693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc3d04-c0e0-44fa-8ecf-fdf836b82471 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:10.314016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc3d04-c0e0-44fa-8ecf-fdf836b82471 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:10.314708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc3d04-c0e0-44fa-8ecf-fdf836b82471 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:10.316615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cc3d04-c0e0-44fa-8ecf-fdf836b82471 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:10.359462 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2661/5355] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:10 2022] POST /identity/v3/users => generated 284 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:20:10.363496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30abfc2e-e542-4536-b253-893177ae875d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:10.373507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30abfc2e-e542-4536-b253-893177ae875d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:10.379086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0362f07-50de-4d6b-b093-8f7f4e36bf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:10.379753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0362f07-50de-4d6b-b093-8f7f4e36bf1f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:10.380869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0362f07-50de-4d6b-b093-8f7f4e36bf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:10.380869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0362f07-50de-4d6b-b093-8f7f4e36bf1f admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:10.381168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0362f07-50de-4d6b-b093-8f7f4e36bf1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:10.382826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0362f07-50de-4d6b-b093-8f7f4e36bf1f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:10.394329 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2662/5356] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:20:10.398191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0362f07-50de-4d6b-b093-8f7f4e36bf1f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:10.408288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0362f07-50de-4d6b-b093-8f7f4e36bf1f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:10.413128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be61fe-1eb7-40d7-b7dd-a69c98de178d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:10.413778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be61fe-1eb7-40d7-b7dd-a69c98de178d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:10.414094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be61fe-1eb7-40d7-b7dd-a69c98de178d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:10.414374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be61fe-1eb7-40d7-b7dd-a69c98de178d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:10.415091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be61fe-1eb7-40d7-b7dd-a69c98de178d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:10.416734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be61fe-1eb7-40d7-b7dd-a69c98de178d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:10.427619 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2663/5357] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:20:10.431837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be61fe-1eb7-40d7-b7dd-a69c98de178d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:10.442163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be61fe-1eb7-40d7-b7dd-a69c98de178d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:10.447948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4522a8-2cb9-4d79-9ca2-0428d8192667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:10.449954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4522a8-2cb9-4d79-9ca2-0428d8192667 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:10.449954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4522a8-2cb9-4d79-9ca2-0428d8192667 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:10.449954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4522a8-2cb9-4d79-9ca2-0428d8192667 admin admin] PATH_INFO: `/v3/projects/3bdf644b9665483a81c078e005116fcf/users/ab4fb84afcdd4782ac34747e666e8c13/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:10.465422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4522a8-2cb9-4d79-9ca2-0428d8192667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bdf644b9665483a81c078e005116fcf, user_id=ab4fb84afcdd4782ac34747e666e8c13, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:10.468272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4522a8-2cb9-4d79-9ca2-0428d8192667 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:10.476062 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2695/5358] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:10 2022] POST /identity/v3/projects => generated 377 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:20:10.480092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc3d04-c0e0-44fa-8ecf-fdf836b82471 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:10.544180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc3d04-c0e0-44fa-8ecf-fdf836b82471 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:10.552060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:10.555185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021f1cb-fb69-46c2-97e9-833ef3746d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:10.556002 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2664/5359] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:10 2022] PUT /identity/v3/projects/3bdf644b9665483a81c078e005116fcf/users/ab4fb84afcdd4782ac34747e666e8c13/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:10.556158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021f1cb-fb69-46c2-97e9-833ef3746d08 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:10.556158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021f1cb-fb69-46c2-97e9-833ef3746d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:10.556428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021f1cb-fb69-46c2-97e9-833ef3746d08 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:10.557136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021f1cb-fb69-46c2-97e9-833ef3746d08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:10.559268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4021f1cb-fb69-46c2-97e9-833ef3746d08 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:10.568339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a690f-e63b-4264-af5b-957cb0aeff1c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:10.568959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a690f-e63b-4264-af5b-957cb0aeff1c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:10.569906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a690f-e63b-4264-af5b-957cb0aeff1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:10.602456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-575a690f-e63b-4264-af5b-957cb0aeff1c None None] MFA Rules not processed for user `ab4fb84afcdd4782ac34747e666e8c13`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:10.604035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-575a690f-e63b-4264-af5b-957cb0aeff1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:10.676362 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2665/5360] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:10 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:20:10.680783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a690f-e63b-4264-af5b-957cb0aeff1c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:10.698039 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2696/5361] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:10 2022] POST /identity/v3/users => generated 295 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:20:10.700699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021f1cb-fb69-46c2-97e9-833ef3746d08 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:10.711066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021f1cb-fb69-46c2-97e9-833ef3746d08 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:10.763645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a690f-e63b-4264-af5b-957cb0aeff1c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:10.769355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63eeed5-2285-4bf4-9088-c2131348f8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:10.770059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63eeed5-2285-4bf4-9088-c2131348f8c4 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:10.770348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63eeed5-2285-4bf4-9088-c2131348f8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:10.770626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63eeed5-2285-4bf4-9088-c2131348f8c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:10.771336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63eeed5-2285-4bf4-9088-c2131348f8c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:10.773382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63eeed5-2285-4bf4-9088-c2131348f8c4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:10.779257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019c7ac-28f4-421a-bd0d-8e8bc6167d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:10.780278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019c7ac-28f4-421a-bd0d-8e8bc6167d60 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:10.780764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019c7ac-28f4-421a-bd0d-8e8bc6167d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:10.781215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019c7ac-28f4-421a-bd0d-8e8bc6167d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:10.782607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019c7ac-28f4-421a-bd0d-8e8bc6167d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:10.785041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019c7ac-28f4-421a-bd0d-8e8bc6167d60 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:10.797526 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2697/5362] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:20:10.801632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019c7ac-28f4-421a-bd0d-8e8bc6167d60 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:10.811807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:10.812075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019c7ac-28f4-421a-bd0d-8e8bc6167d60 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:10.817223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705e213-32cf-4fc6-9905-10ed11c2c5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:10.818347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705e213-32cf-4fc6-9905-10ed11c2c5aa admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:10.818638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705e213-32cf-4fc6-9905-10ed11c2c5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:10.818925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705e213-32cf-4fc6-9905-10ed11c2c5aa admin admin] PATH_INFO: `/v3/projects/a1aba7ca20234e6081a4e25fb5d5e15d/users/4e291b619e1f4ea3a6e3fa491761f01a/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:10.834904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705e213-32cf-4fc6-9905-10ed11c2c5aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1aba7ca20234e6081a4e25fb5d5e15d, user_id=4e291b619e1f4ea3a6e3fa491761f01a, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:10.837795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705e213-32cf-4fc6-9905-10ed11c2c5aa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:10.899839 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2666/5363] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:10 2022] POST /identity/v3/projects => generated 369 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:20:10.903782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63eeed5-2285-4bf4-9088-c2131348f8c4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:10.967633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63eeed5-2285-4bf4-9088-c2131348f8c4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:10.973459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97d3e4-f1ca-4247-9db3-366b2a8b6ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:10.974107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97d3e4-f1ca-4247-9db3-366b2a8b6ee4 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:10.974400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97d3e4-f1ca-4247-9db3-366b2a8b6ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:10.974711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97d3e4-f1ca-4247-9db3-366b2a8b6ee4 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:10.975466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97d3e4-f1ca-4247-9db3-366b2a8b6ee4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:10.977625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97d3e4-f1ca-4247-9db3-366b2a8b6ee4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:11.139622 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2698/5364] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:10 2022] PUT /identity/v3/projects/a1aba7ca20234e6081a4e25fb5d5e15d/users/4e291b619e1f4ea3a6e3fa491761f01a/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:11.139622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705e213-32cf-4fc6-9905-10ed11c2c5aa admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:11.160062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705e213-32cf-4fc6-9905-10ed11c2c5aa admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:11.165961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f996c45-ec42-4fde-804d-79a332029240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:11.166724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f996c45-ec42-4fde-804d-79a332029240 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:11.167092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f996c45-ec42-4fde-804d-79a332029240 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:11.167434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f996c45-ec42-4fde-804d-79a332029240 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:11.168241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f996c45-ec42-4fde-804d-79a332029240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:11.170707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f996c45-ec42-4fde-804d-79a332029240 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:11.182078 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2699/5365] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:20:11.185716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f996c45-ec42-4fde-804d-79a332029240 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:11.201228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f996c45-ec42-4fde-804d-79a332029240 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:11.206765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa7659-6da2-4db1-acbd-17c743c8a624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:11.208285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa7659-6da2-4db1-acbd-17c743c8a624 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:11.208676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa7659-6da2-4db1-acbd-17c743c8a624 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:11.209050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa7659-6da2-4db1-acbd-17c743c8a624 admin admin] PATH_INFO: `/v3/projects/a1aba7ca20234e6081a4e25fb5d5e15d/users/4e291b619e1f4ea3a6e3fa491761f01a/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:11.211331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa7659-6da2-4db1-acbd-17c743c8a624 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1aba7ca20234e6081a4e25fb5d5e15d, user_id=4e291b619e1f4ea3a6e3fa491761f01a, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:11.214067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa7659-6da2-4db1-acbd-17c743c8a624 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:11.226119 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2700/5366] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:11 2022] PUT /identity/v3/projects/a1aba7ca20234e6081a4e25fb5d5e15d/users/4e291b619e1f4ea3a6e3fa491761f01a/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:11.239307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa2de0-48c5-46d2-ab4d-21e8ff370d60 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:11.239980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa2de0-48c5-46d2-ab4d-21e8ff370d60 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:11.240697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa2de0-48c5-46d2-ab4d-21e8ff370d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:11.271159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7ffa2de0-48c5-46d2-ab4d-21e8ff370d60 None None] MFA Rules not processed for user `4e291b619e1f4ea3a6e3fa491761f01a`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:11.272430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7ffa2de0-48c5-46d2-ab4d-21e8ff370d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:11.317731 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2667/5367] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:10 2022] POST /identity/v3/users => generated 291 bytes in 414 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:20:11.321676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97d3e4-f1ca-4247-9db3-366b2a8b6ee4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:11.340432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97d3e4-f1ca-4247-9db3-366b2a8b6ee4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:11.401596 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2701/5368] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:11 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:11.413761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa2de0-48c5-46d2-ab4d-21e8ff370d60 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:11.423422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7ffa2de0-48c5-46d2-ab4d-21e8ff370d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:11.426519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72c038-1377-47ee-915f-149b308d6a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:11.427631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72c038-1377-47ee-915f-149b308d6a7b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:11.428565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72c038-1377-47ee-915f-149b308d6a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:11.429028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72c038-1377-47ee-915f-149b308d6a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:11.430332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72c038-1377-47ee-915f-149b308d6a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:11.432926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72c038-1377-47ee-915f-149b308d6a7b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:11.450975 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2668/5369] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:11.456194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72c038-1377-47ee-915f-149b308d6a7b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:11.478487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72c038-1377-47ee-915f-149b308d6a7b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:11.489933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a1438-6aeb-441b-bcf9-37030af7adfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:11.493594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a1438-6aeb-441b-bcf9-37030af7adfb admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:11.494120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a1438-6aeb-441b-bcf9-37030af7adfb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:11.494593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a1438-6aeb-441b-bcf9-37030af7adfb admin admin] PATH_INFO: `/v3/projects/cc02ab74778f4ebc92da025a71966779/users/a0f56a465d0b4776a4fc3b81e4f382c1/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:11.522104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a1438-6aeb-441b-bcf9-37030af7adfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc02ab74778f4ebc92da025a71966779, user_id=a0f56a465d0b4776a4fc3b81e4f382c1, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:11.526307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a1438-6aeb-441b-bcf9-37030af7adfb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:11.559201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa2de0-48c5-46d2-ab4d-21e8ff370d60 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:11.563855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fd79f-6475-4c0e-8348-799516892bdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:11.564783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fd79f-6475-4c0e-8348-799516892bdd service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:11.565201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fd79f-6475-4c0e-8348-799516892bdd service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:11.565668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fd79f-6475-4c0e-8348-799516892bdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:11.620373 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2669/5370] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:11 2022] PUT /identity/v3/projects/cc02ab74778f4ebc92da025a71966779/users/a0f56a465d0b4776a4fc3b81e4f382c1/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:11.624332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a1438-6aeb-441b-bcf9-37030af7adfb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:11.655672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25fd79f-6475-4c0e-8348-799516892bdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:11.658145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25fd79f-6475-4c0e-8348-799516892bdd service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:11.669260 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2702/5371] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:20:11.755356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a1438-6aeb-441b-bcf9-37030af7adfb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:11.761760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9476261-cee6-45e3-af61-ad0eea5f2726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:11.762617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9476261-cee6-45e3-af61-ad0eea5f2726 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:11.762907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9476261-cee6-45e3-af61-ad0eea5f2726 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:11.763312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9476261-cee6-45e3-af61-ad0eea5f2726 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:11.764890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9476261-cee6-45e3-af61-ad0eea5f2726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:11.778119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9476261-cee6-45e3-af61-ad0eea5f2726 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:11.797975 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2670/5372] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:11.800315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fd79f-6475-4c0e-8348-799516892bdd service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:11.817990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fd79f-6475-4c0e-8348-799516892bdd service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:11.826075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685645fb-d8fb-4db3-9fcc-04fb60d95af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:11.827108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685645fb-d8fb-4db3-9fcc-04fb60d95af9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:11.827401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685645fb-d8fb-4db3-9fcc-04fb60d95af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:11.828287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685645fb-d8fb-4db3-9fcc-04fb60d95af9 admin admin] PATH_INFO: `/v3/projects/cc02ab74778f4ebc92da025a71966779/users/a0f56a465d0b4776a4fc3b81e4f382c1/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:11.831937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685645fb-d8fb-4db3-9fcc-04fb60d95af9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc02ab74778f4ebc92da025a71966779, user_id=a0f56a465d0b4776a4fc3b81e4f382c1, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:11.834950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685645fb-d8fb-4db3-9fcc-04fb60d95af9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:11.851046 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2703/5373] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:11 2022] PUT /identity/v3/projects/cc02ab74778f4ebc92da025a71966779/users/a0f56a465d0b4776a4fc3b81e4f382c1/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:11.861664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca882307-812e-4efd-8929-046f6b06f02b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:11.867470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca882307-812e-4efd-8929-046f6b06f02b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:11.867470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca882307-812e-4efd-8929-046f6b06f02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:11.895012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ca882307-812e-4efd-8929-046f6b06f02b None None] MFA Rules not processed for user `a0f56a465d0b4776a4fc3b81e4f382c1`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:11.896730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ca882307-812e-4efd-8929-046f6b06f02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:11.976895 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2671/5374] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:11 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:11.982277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685645fb-d8fb-4db3-9fcc-04fb60d95af9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:11.993749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685645fb-d8fb-4db3-9fcc-04fb60d95af9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:11.998599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f12b18-5c06-45fa-86ce-29719c9881a7 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f56a465d0b4776a4fc3b81e4f382c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc02ab74778f4ebc92da025a71966779', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:11.999268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f12b18-5c06-45fa-86ce-29719c9881a7 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:12.000088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f12b18-5c06-45fa-86ce-29719c9881a7 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:12.000360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f12b18-5c06-45fa-86ce-29719c9881a7 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] PATH_INFO: `/v3/policies` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:12.001413 ncdv-master-1 devstack@keystone.service[103103]: WARNING oslo_log.versionutils [None req-f9f12b18-5c06-45fa-86ce-29719c9881a7 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Deprecated: identity:create_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. May 28 21:20:12.001991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f12b18-5c06-45fa-86ce-29719c9881a7 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:12.003245 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-f9f12b18-5c06-45fa-86ce-29719c9881a7 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:20:12.003245 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:20:12.003245 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:20:12.004177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f12b18-5c06-45fa-86ce-29719c9881a7 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:12.151514 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2704/5375] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:20:11 2022] POST /identity/v3/policies => generated 228 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:12.155500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca882307-812e-4efd-8929-046f6b06f02b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:12.167749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca882307-812e-4efd-8929-046f6b06f02b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:12.172688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffc5b6-0dc0-415e-9f86-9a478e1a0856 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f56a465d0b4776a4fc3b81e4f382c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc02ab74778f4ebc92da025a71966779', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:12.173661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffc5b6-0dc0-415e-9f86-9a478e1a0856 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:12.174051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffc5b6-0dc0-415e-9f86-9a478e1a0856 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:12.174418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ffc5b6-0dc0-415e-9f86-9a478e1a0856 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] PATH_INFO: `/v3/policies/c40fc1488c304fabb3024272c41419d3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:12.175661 ncdv-master-1 devstack@keystone.service[103104]: WARNING oslo_log.versionutils [None req-21ffc5b6-0dc0-415e-9f86-9a478e1a0856 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Deprecated: identity:update_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. May 28 21:20:12.176260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ffc5b6-0dc0-415e-9f86-9a478e1a0856 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorizing `identity:update_policy(policy_id=c40fc1488c304fabb3024272c41419d3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:12.177558 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-21ffc5b6-0dc0-415e-9f86-9a478e1a0856 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:20:12.177558 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:20:12.177558 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:20:12.178715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ffc5b6-0dc0-415e-9f86-9a478e1a0856 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:12.298145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f12b18-5c06-45fa-86ce-29719c9881a7 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:12.304685 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2672/5376] 10.222.0.33 () {60 vars in 1251 bytes} [Sat May 28 21:20:12 2022] PATCH /identity/v3/policies/c40fc1488c304fabb3024272c41419d3 => generated 235 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:20:12.326790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f12b18-5c06-45fa-86ce-29719c9881a7 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:12.353939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbf549-6b66-4ff5-a067-def8a026328f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f56a465d0b4776a4fc3b81e4f382c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc02ab74778f4ebc92da025a71966779', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:12.358599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:12.359592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbf549-6b66-4ff5-a067-def8a026328f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:12.359900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbf549-6b66-4ff5-a067-def8a026328f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:12.360257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfbf549-6b66-4ff5-a067-def8a026328f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] PATH_INFO: `/v3/policies/c40fc1488c304fabb3024272c41419d3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:12.361481 ncdv-master-1 devstack@keystone.service[103103]: WARNING oslo_log.versionutils [None req-dbfbf549-6b66-4ff5-a067-def8a026328f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Deprecated: identity:get_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. May 28 21:20:12.361994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfbf549-6b66-4ff5-a067-def8a026328f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorizing `identity:get_policy(policy_id=c40fc1488c304fabb3024272c41419d3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:12.365401 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-dbfbf549-6b66-4ff5-a067-def8a026328f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_policy": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:20:12.365401 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:20:12.365401 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:20:12.368183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfbf549-6b66-4ff5-a067-def8a026328f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:12.406516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffc5b6-0dc0-415e-9f86-9a478e1a0856 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:12.412618 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2705/5377] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:20:12 2022] GET /identity/v3/policies/c40fc1488c304fabb3024272c41419d3 => generated 235 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:12.425631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ffc5b6-0dc0-415e-9f86-9a478e1a0856 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:12.436462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3bcc5-42a9-485c-ac78-80da8f93ccd1 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f56a465d0b4776a4fc3b81e4f382c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc02ab74778f4ebc92da025a71966779', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:12.439227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3bcc5-42a9-485c-ac78-80da8f93ccd1 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:12.441393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3bcc5-42a9-485c-ac78-80da8f93ccd1 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:12.441393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3bcc5-42a9-485c-ac78-80da8f93ccd1 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] PATH_INFO: `/v3/policies/c40fc1488c304fabb3024272c41419d3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:12.443719 ncdv-master-1 devstack@keystone.service[103104]: WARNING oslo_log.versionutils [None req-42f3bcc5-42a9-485c-ac78-80da8f93ccd1 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Deprecated: identity:delete_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. May 28 21:20:12.444702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3bcc5-42a9-485c-ac78-80da8f93ccd1 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=c40fc1488c304fabb3024272c41419d3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:12.448880 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-42f3bcc5-42a9-485c-ac78-80da8f93ccd1 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:20:12.448880 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:20:12.448880 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:20:12.449813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3bcc5-42a9-485c-ac78-80da8f93ccd1 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:12.569917 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2673/5378] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:12 2022] DELETE /identity/v3/policies/c40fc1488c304fabb3024272c41419d3 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:12.576640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbf549-6b66-4ff5-a067-def8a026328f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:12.588207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfbf549-6b66-4ff5-a067-def8a026328f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:12.593522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767a057-fbbb-4c39-915a-12a1b99b335f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f56a465d0b4776a4fc3b81e4f382c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc02ab74778f4ebc92da025a71966779', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:12.594259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767a057-fbbb-4c39-915a-12a1b99b335f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:12.594582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767a057-fbbb-4c39-915a-12a1b99b335f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:12.594879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767a057-fbbb-4c39-915a-12a1b99b335f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] PATH_INFO: `/v3/policies` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:12.596127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767a057-fbbb-4c39-915a-12a1b99b335f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:12.598034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1767a057-fbbb-4c39-915a-12a1b99b335f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:12.721274 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2706/5379] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:20:12 2022] POST /identity/v3/policies => generated 230 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:12.725432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3bcc5-42a9-485c-ac78-80da8f93ccd1 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:12.736185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3bcc5-42a9-485c-ac78-80da8f93ccd1 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:12.740769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8150f98-2ccc-4d19-ac29-4a3fed28c7f9 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f56a465d0b4776a4fc3b81e4f382c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc02ab74778f4ebc92da025a71966779', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:12.741422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8150f98-2ccc-4d19-ac29-4a3fed28c7f9 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:12.741711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8150f98-2ccc-4d19-ac29-4a3fed28c7f9 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:12.742041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8150f98-2ccc-4d19-ac29-4a3fed28c7f9 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] PATH_INFO: `/v3/policies` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:12.742982 ncdv-master-1 devstack@keystone.service[103104]: WARNING oslo_log.versionutils [None req-d8150f98-2ccc-4d19-ac29-4a3fed28c7f9 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Deprecated: identity:create_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. May 28 21:20:12.743515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8150f98-2ccc-4d19-ac29-4a3fed28c7f9 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:12.744757 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-d8150f98-2ccc-4d19-ac29-4a3fed28c7f9 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:20:12.744757 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:20:12.744757 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:20:12.745651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8150f98-2ccc-4d19-ac29-4a3fed28c7f9 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:12.896520 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2674/5380] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:20:12 2022] POST /identity/v3/policies => generated 230 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:12.898077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767a057-fbbb-4c39-915a-12a1b99b335f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:12.913273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767a057-fbbb-4c39-915a-12a1b99b335f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:12.916255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74723e10-14c7-4565-9e57-0f66a64bb1dd tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f56a465d0b4776a4fc3b81e4f382c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc02ab74778f4ebc92da025a71966779', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:12.916992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74723e10-14c7-4565-9e57-0f66a64bb1dd tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:12.917655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74723e10-14c7-4565-9e57-0f66a64bb1dd tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:12.918222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74723e10-14c7-4565-9e57-0f66a64bb1dd tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] PATH_INFO: `/v3/policies` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:12.919468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74723e10-14c7-4565-9e57-0f66a64bb1dd tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorizing `identity:create_policy()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:12.921283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74723e10-14c7-4565-9e57-0f66a64bb1dd tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:12.931763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:13.048223 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2707/5381] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:20:12 2022] POST /identity/v3/policies => generated 228 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:13.050190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8150f98-2ccc-4d19-ac29-4a3fed28c7f9 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:13.063828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8150f98-2ccc-4d19-ac29-4a3fed28c7f9 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:13.070223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f6433-3737-4ebc-b907-30e07115c93c tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f56a465d0b4776a4fc3b81e4f382c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc02ab74778f4ebc92da025a71966779', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:13.071061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f6433-3737-4ebc-b907-30e07115c93c tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:13.071341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f6433-3737-4ebc-b907-30e07115c93c tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:13.071634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f6433-3737-4ebc-b907-30e07115c93c tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] PATH_INFO: `/v3/policies` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:13.072473 ncdv-master-1 devstack@keystone.service[103104]: WARNING oslo_log.versionutils [None req-258f6433-3737-4ebc-b907-30e07115c93c tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Deprecated: identity:list_policies of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. May 28 21:20:13.073157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f6433-3737-4ebc-b907-30e07115c93c tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorizing `identity:list_policies()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:13.074457 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-258f6433-3737-4ebc-b907-30e07115c93c tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_policies": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:20:13.074457 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:20:13.074457 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:20:13.075355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258f6433-3737-4ebc-b907-30e07115c93c tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:13.088104 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2675/5382] 10.222.0.33 () {58 vars in 1130 bytes} [Sat May 28 21:20:13 2022] GET /identity/v3/policies => generated 764 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:13.096373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74723e10-14c7-4565-9e57-0f66a64bb1dd tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:13.100187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74723e10-14c7-4565-9e57-0f66a64bb1dd tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:13.105122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11d0cd-882d-4931-9d2c-d85860fe184f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f56a465d0b4776a4fc3b81e4f382c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc02ab74778f4ebc92da025a71966779', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:13.106023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11d0cd-882d-4931-9d2c-d85860fe184f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:13.106350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11d0cd-882d-4931-9d2c-d85860fe184f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:13.106644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11d0cd-882d-4931-9d2c-d85860fe184f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] PATH_INFO: `/v3/policies/5da498c4f1af4ccebab76e0cf57a236e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:13.107918 ncdv-master-1 devstack@keystone.service[103103]: WARNING oslo_log.versionutils [None req-7c11d0cd-882d-4931-9d2c-d85860fe184f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Deprecated: identity:delete_policy of the v3 Policy APIs is deprecated as of Queens and may be removed in Stein. It will not be superseded. May 28 21:20:13.108564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11d0cd-882d-4931-9d2c-d85860fe184f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=5da498c4f1af4ccebab76e0cf57a236e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:13.110413 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-7c11d0cd-882d-4931-9d2c-d85860fe184f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_policy": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:20:13.110413 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:20:13.110413 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:20:13.111296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c11d0cd-882d-4931-9d2c-d85860fe184f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:13.223490 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2708/5383] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:13 2022] DELETE /identity/v3/policies/5da498c4f1af4ccebab76e0cf57a236e => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:13.227894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f6433-3737-4ebc-b907-30e07115c93c tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:13.240622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f6433-3737-4ebc-b907-30e07115c93c tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:13.245510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826c005-d3d9-4f91-a169-193bc9148cea tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f56a465d0b4776a4fc3b81e4f382c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc02ab74778f4ebc92da025a71966779', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:13.246459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826c005-d3d9-4f91-a169-193bc9148cea tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:13.247825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826c005-d3d9-4f91-a169-193bc9148cea tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:13.247825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826c005-d3d9-4f91-a169-193bc9148cea tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] PATH_INFO: `/v3/policies/1bf2ebd1fdbc4661be25f2847bb83f5c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:13.248182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826c005-d3d9-4f91-a169-193bc9148cea tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=1bf2ebd1fdbc4661be25f2847bb83f5c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:13.250082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826c005-d3d9-4f91-a169-193bc9148cea tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:13.335472 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2676/5384] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:13 2022] DELETE /identity/v3/policies/1bf2ebd1fdbc4661be25f2847bb83f5c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:13.337337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11d0cd-882d-4931-9d2c-d85860fe184f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:13.366473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11d0cd-882d-4931-9d2c-d85860fe184f tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:13.371778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc11d1bd-b977-46dd-b8d9-d2da42d9e434 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0f56a465d0b4776a4fc3b81e4f382c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cc02ab74778f4ebc92da025a71966779', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:13.372656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11d1bd-b977-46dd-b8d9-d2da42d9e434 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:13.372953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11d1bd-b977-46dd-b8d9-d2da42d9e434 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:13.373238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc11d1bd-b977-46dd-b8d9-d2da42d9e434 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] PATH_INFO: `/v3/policies/80e1312d200e4fbc9282b3309bb1d99b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:13.374489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11d1bd-b977-46dd-b8d9-d2da42d9e434 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorizing `identity:delete_policy(policy_id=80e1312d200e4fbc9282b3309bb1d99b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:13.376336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc11d1bd-b977-46dd-b8d9-d2da42d9e434 tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:13.480813 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2709/5385] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:13 2022] DELETE /identity/v3/policies/80e1312d200e4fbc9282b3309bb1d99b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:13.487186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826c005-d3d9-4f91-a169-193bc9148cea tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:13.557609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826c005-d3d9-4f91-a169-193bc9148cea tempest-PoliciesTestJSON-1570384811 tempest-PoliciesTestJSON-1570384811-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:13.562678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27109c-be54-4bbe-9e84-4d8b7fcd14cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:13.563833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27109c-be54-4bbe-9e84-4d8b7fcd14cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:13.569402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27109c-be54-4bbe-9e84-4d8b7fcd14cc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:13.569402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27109c-be54-4bbe-9e84-4d8b7fcd14cc admin admin] PATH_INFO: `/v3/users/ab4fb84afcdd4782ac34747e666e8c13` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:13.569402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27109c-be54-4bbe-9e84-4d8b7fcd14cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab4fb84afcdd4782ac34747e666e8c13)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:13.569402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27109c-be54-4bbe-9e84-4d8b7fcd14cc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:13.778684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ac27109c-be54-4bbe-9e84-4d8b7fcd14cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab4fb84afcdd4782ac34747e666e8c13'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:13.779767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ac27109c-be54-4bbe-9e84-4d8b7fcd14cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab4fb84afcdd4782ac34747e666e8c13'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:13.788411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ac27109c-be54-4bbe-9e84-4d8b7fcd14cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab4fb84afcdd4782ac34747e666e8c13'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:13.879005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ac27109c-be54-4bbe-9e84-4d8b7fcd14cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab4fb84afcdd4782ac34747e666e8c13'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:13.879005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ac27109c-be54-4bbe-9e84-4d8b7fcd14cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab4fb84afcdd4782ac34747e666e8c13'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:13.898890 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2677/5386] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:13 2022] DELETE /identity/v3/users/ab4fb84afcdd4782ac34747e666e8c13 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:13.905563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90a9b7-1059-40ff-93e5-a80b58eba3a2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:13.906857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90a9b7-1059-40ff-93e5-a80b58eba3a2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:13.906857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a90a9b7-1059-40ff-93e5-a80b58eba3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:13.925344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6a90a9b7-1059-40ff-93e5-a80b58eba3a2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:13.926949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6a90a9b7-1059-40ff-93e5-a80b58eba3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:13.996920 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2710/5387] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:14.007342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27109c-be54-4bbe-9e84-4d8b7fcd14cc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:14.009124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ac27109c-be54-4bbe-9e84-4d8b7fcd14cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:14.074431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27109c-be54-4bbe-9e84-4d8b7fcd14cc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:14.078940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2aa972-c57b-4dd9-b6c7-1f353d737cdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:14.079808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2aa972-c57b-4dd9-b6c7-1f353d737cdc service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:14.080093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2aa972-c57b-4dd9-b6c7-1f353d737cdc service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:14.080370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2aa972-c57b-4dd9-b6c7-1f353d737cdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:14.090467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2aa972-c57b-4dd9-b6c7-1f353d737cdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:14.091936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2aa972-c57b-4dd9-b6c7-1f353d737cdc service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:14.101819 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2678/5388] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:14.178587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:14.579036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90a9b7-1059-40ff-93e5-a80b58eba3a2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:14.580609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6a90a9b7-1059-40ff-93e5-a80b58eba3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:14.596982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a90a9b7-1059-40ff-93e5-a80b58eba3a2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:14.602267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0846fc-4163-4cf5-b53e-617e4534720c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:14.603788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0846fc-4163-4cf5-b53e-617e4534720c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:14.603908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0846fc-4163-4cf5-b53e-617e4534720c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:14.604177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0846fc-4163-4cf5-b53e-617e4534720c admin admin] PATH_INFO: `/v3/users/7ea0fccc16364b7a92e1ee7308268c09` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:14.605623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0846fc-4163-4cf5-b53e-617e4534720c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ea0fccc16364b7a92e1ee7308268c09)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:14.607850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0846fc-4163-4cf5-b53e-617e4534720c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:14.652003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:14.761465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2aa972-c57b-4dd9-b6c7-1f353d737cdc service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:14.763059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5f2aa972-c57b-4dd9-b6c7-1f353d737cdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:14.777509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2aa972-c57b-4dd9-b6c7-1f353d737cdc service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:14.783311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117df07-0c15-47f1-bf62-445377d5bcc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:14.785918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117df07-0c15-47f1-bf62-445377d5bcc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:14.785918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117df07-0c15-47f1-bf62-445377d5bcc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:14.785918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8117df07-0c15-47f1-bf62-445377d5bcc1 admin admin] PATH_INFO: `/v3/projects/3bdf644b9665483a81c078e005116fcf` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:14.786267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117df07-0c15-47f1-bf62-445377d5bcc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bdf644b9665483a81c078e005116fcf)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:14.787942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8117df07-0c15-47f1-bf62-445377d5bcc1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:14.923597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ff0846fc-4163-4cf5-b53e-617e4534720c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ea0fccc16364b7a92e1ee7308268c09'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:14.931401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ff0846fc-4163-4cf5-b53e-617e4534720c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ea0fccc16364b7a92e1ee7308268c09'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:14.931579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8117df07-0c15-47f1-bf62-445377d5bcc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bdf644b9665483a81c078e005116fcf', 'request_id': 'req-8117df07-0c15-47f1-bf62-445377d5bcc1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:14.932846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ff0846fc-4163-4cf5-b53e-617e4534720c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ea0fccc16364b7a92e1ee7308268c09'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:14.933568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ff0846fc-4163-4cf5-b53e-617e4534720c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ea0fccc16364b7a92e1ee7308268c09'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:14.938975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8117df07-0c15-47f1-bf62-445377d5bcc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bdf644b9665483a81c078e005116fcf', 'request_id': 'req-8117df07-0c15-47f1-bf62-445377d5bcc1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.011310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ff0846fc-4163-4cf5-b53e-617e4534720c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ea0fccc16364b7a92e1ee7308268c09'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.050977 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2711/5389] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:14 2022] DELETE /identity/v3/users/7ea0fccc16364b7a92e1ee7308268c09 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:15.098182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8117df07-0c15-47f1-bf62-445377d5bcc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bdf644b9665483a81c078e005116fcf', 'request_id': 'req-8117df07-0c15-47f1-bf62-445377d5bcc1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.112591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8117df07-0c15-47f1-bf62-445377d5bcc1 admin admin] The token cache is being invalidate because project 3bdf644b9665483a81c078e005116fcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:15.112891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8117df07-0c15-47f1-bf62-445377d5bcc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.113772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8117df07-0c15-47f1-bf62-445377d5bcc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.115527 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2679/5390] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:14 2022] DELETE /identity/v3/projects/3bdf644b9665483a81c078e005116fcf => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:15.119682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0846fc-4163-4cf5-b53e-617e4534720c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:15.121740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ff0846fc-4163-4cf5-b53e-617e4534720c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:15.196070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0846fc-4163-4cf5-b53e-617e4534720c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:15.201393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bafa71-f3e0-4bf1-aa29-b58fdcd1ecf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:15.210120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bafa71-f3e0-4bf1-aa29-b58fdcd1ecf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:15.210120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bafa71-f3e0-4bf1-aa29-b58fdcd1ecf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:15.210120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bafa71-f3e0-4bf1-aa29-b58fdcd1ecf0 admin admin] PATH_INFO: `/v3/users/a0f56a465d0b4776a4fc3b81e4f382c1` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:15.210120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bafa71-f3e0-4bf1-aa29-b58fdcd1ecf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0f56a465d0b4776a4fc3b81e4f382c1)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:15.210120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bafa71-f3e0-4bf1-aa29-b58fdcd1ecf0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:15.274732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117df07-0c15-47f1-bf62-445377d5bcc1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:15.276229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8117df07-0c15-47f1-bf62-445377d5bcc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:15.379118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8117df07-0c15-47f1-bf62-445377d5bcc1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:15.384458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2bafa71-f3e0-4bf1-aa29-b58fdcd1ecf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0f56a465d0b4776a4fc3b81e4f382c1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.384860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180c0fc-553b-4a2e-8912-b85c7d309527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:15.386302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180c0fc-553b-4a2e-8912-b85c7d309527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:15.386755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180c0fc-553b-4a2e-8912-b85c7d309527 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:15.386909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180c0fc-553b-4a2e-8912-b85c7d309527 admin admin] PATH_INFO: `/v3/users/d44fb59970fb442da24187a6ec827b75` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:15.388503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8180c0fc-553b-4a2e-8912-b85c7d309527 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d44fb59970fb442da24187a6ec827b75)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:15.391564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8180c0fc-553b-4a2e-8912-b85c7d309527 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:15.401088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2bafa71-f3e0-4bf1-aa29-b58fdcd1ecf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0f56a465d0b4776a4fc3b81e4f382c1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.414578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2bafa71-f3e0-4bf1-aa29-b58fdcd1ecf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0f56a465d0b4776a4fc3b81e4f382c1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.414578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2bafa71-f3e0-4bf1-aa29-b58fdcd1ecf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0f56a465d0b4776a4fc3b81e4f382c1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.495754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2bafa71-f3e0-4bf1-aa29-b58fdcd1ecf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0f56a465d0b4776a4fc3b81e4f382c1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.514277 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2712/5391] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:15 2022] DELETE /identity/v3/users/a0f56a465d0b4776a4fc3b81e4f382c1 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:15.516306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bafa71-f3e0-4bf1-aa29-b58fdcd1ecf0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:15.517685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e2bafa71-f3e0-4bf1-aa29-b58fdcd1ecf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:15.609220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bafa71-f3e0-4bf1-aa29-b58fdcd1ecf0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:15.616099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc254f-2499-4c59-b8cb-9a65521b2f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:15.617054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc254f-2499-4c59-b8cb-9a65521b2f65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:15.617401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc254f-2499-4c59-b8cb-9a65521b2f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:15.617742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc254f-2499-4c59-b8cb-9a65521b2f65 admin admin] PATH_INFO: `/v3/projects/e398fd1bbf124b01b5a8a155d040f21c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:15.619265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc254f-2499-4c59-b8cb-9a65521b2f65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e398fd1bbf124b01b5a8a155d040f21c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:15.621974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc254f-2499-4c59-b8cb-9a65521b2f65 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:15.694781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8180c0fc-553b-4a2e-8912-b85c7d309527 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd44fb59970fb442da24187a6ec827b75'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.695778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8180c0fc-553b-4a2e-8912-b85c7d309527 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd44fb59970fb442da24187a6ec827b75'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.703766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8180c0fc-553b-4a2e-8912-b85c7d309527 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd44fb59970fb442da24187a6ec827b75'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.772889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dacc254f-2499-4c59-b8cb-9a65521b2f65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e398fd1bbf124b01b5a8a155d040f21c', 'request_id': 'req-dacc254f-2499-4c59-b8cb-9a65521b2f65'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.779025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dacc254f-2499-4c59-b8cb-9a65521b2f65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e398fd1bbf124b01b5a8a155d040f21c', 'request_id': 'req-dacc254f-2499-4c59-b8cb-9a65521b2f65'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.805210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8180c0fc-553b-4a2e-8912-b85c7d309527 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd44fb59970fb442da24187a6ec827b75'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.808858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8180c0fc-553b-4a2e-8912-b85c7d309527 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd44fb59970fb442da24187a6ec827b75'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.824801 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2680/5392] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:15 2022] DELETE /identity/v3/users/d44fb59970fb442da24187a6ec827b75 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:15.835068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24495208-d714-4b4d-a9bc-7e2a87ae2bfa None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:15.835311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24495208-d714-4b4d-a9bc-7e2a87ae2bfa None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:15.835593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24495208-d714-4b4d-a9bc-7e2a87ae2bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:15.852893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-24495208-d714-4b4d-a9bc-7e2a87ae2bfa None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:15.854913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-24495208-d714-4b4d-a9bc-7e2a87ae2bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:15.896430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dacc254f-2499-4c59-b8cb-9a65521b2f65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e398fd1bbf124b01b5a8a155d040f21c', 'request_id': 'req-dacc254f-2499-4c59-b8cb-9a65521b2f65'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.910660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dacc254f-2499-4c59-b8cb-9a65521b2f65 admin admin] The token cache is being invalidate because project e398fd1bbf124b01b5a8a155d040f21c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:15.911486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dacc254f-2499-4c59-b8cb-9a65521b2f65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.912277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dacc254f-2499-4c59-b8cb-9a65521b2f65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:15.913986 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2713/5393] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:15 2022] DELETE /identity/v3/projects/e398fd1bbf124b01b5a8a155d040f21c => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:15.919038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656e231-24db-417a-b171-00f7e5131e59 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:15.919484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656e231-24db-417a-b171-00f7e5131e59 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:15.919864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656e231-24db-417a-b171-00f7e5131e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:15.920620 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2681/5394] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:20:15.931475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24495208-d714-4b4d-a9bc-7e2a87ae2bfa None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:15.932985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-24495208-d714-4b4d-a9bc-7e2a87ae2bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:15.942074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d656e231-24db-417a-b171-00f7e5131e59 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:15.943288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d656e231-24db-417a-b171-00f7e5131e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:15.951247 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2714/5395] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:15.959782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ba783-afb2-470e-80f4-dbf61132563b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:15.960065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ba783-afb2-470e-80f4-dbf61132563b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:15.964089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70ba783-afb2-470e-80f4-dbf61132563b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:15.979205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b70ba783-afb2-470e-80f4-dbf61132563b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:15.980396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b70ba783-afb2-470e-80f4-dbf61132563b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:15.989255 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2715/5396] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:20:15.993522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ba783-afb2-470e-80f4-dbf61132563b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:15.995588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24495208-d714-4b4d-a9bc-7e2a87ae2bfa None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:15.999283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f38126-ab5b-4522-b864-49327e86d1f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:16.000517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f38126-ab5b-4522-b864-49327e86d1f5 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:16.000804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f38126-ab5b-4522-b864-49327e86d1f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:16.001081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f38126-ab5b-4522-b864-49327e86d1f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:16.003347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70ba783-afb2-470e-80f4-dbf61132563b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:16.010144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2edd4-4c8e-4381-a3e0-c7066a3ce53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:16.010144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2edd4-4c8e-4381-a3e0-c7066a3ce53f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:16.010144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2edd4-4c8e-4381-a3e0-c7066a3ce53f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:16.010144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e2edd4-4c8e-4381-a3e0-c7066a3ce53f admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:16.011256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2edd4-4c8e-4381-a3e0-c7066a3ce53f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:20:16.011256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2edd4-4c8e-4381-a3e0-c7066a3ce53f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:16.012135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e2edd4-4c8e-4381-a3e0-c7066a3ce53f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:16.026092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f38126-ab5b-4522-b864-49327e86d1f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:16.028600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f38126-ab5b-4522-b864-49327e86d1f5 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:16.038763 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2716/5397] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:20:16.041553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2edd4-4c8e-4381-a3e0-c7066a3ce53f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:16.043395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-91e2edd4-4c8e-4381-a3e0-c7066a3ce53f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:16.054432 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2682/5398] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:20:16.055846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f38126-ab5b-4522-b864-49327e86d1f5 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:16.058929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e2edd4-4c8e-4381-a3e0-c7066a3ce53f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:16.064954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb3541-2607-4a07-bccf-25e7131b5f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:16.066973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb3541-2607-4a07-bccf-25e7131b5f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:16.067340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb3541-2607-4a07-bccf-25e7131b5f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:16.067659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb3541-2607-4a07-bccf-25e7131b5f78 admin admin] PATH_INFO: `/v3/projects/cc02ab74778f4ebc92da025a71966779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:16.069200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb3541-2607-4a07-bccf-25e7131b5f78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc02ab74778f4ebc92da025a71966779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:16.070799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f38126-ab5b-4522-b864-49327e86d1f5 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:16.071221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb3541-2607-4a07-bccf-25e7131b5f78 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:16.076872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1537e-d1b0-472e-aa36-d46dde469d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:16.083606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1537e-d1b0-472e-aa36-d46dde469d79 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:16.083606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1537e-d1b0-472e-aa36-d46dde469d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:16.083606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1537e-d1b0-472e-aa36-d46dde469d79 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:16.083606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1537e-d1b0-472e-aa36-d46dde469d79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:16.083606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1537e-d1b0-472e-aa36-d46dde469d79 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:16.211725 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2683/5399] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:16 2022] POST /identity/v3/projects => generated 395 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:16.215189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1537e-d1b0-472e-aa36-d46dde469d79 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:16.267987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-32bb3541-2607-4a07-bccf-25e7131b5f78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc02ab74778f4ebc92da025a71966779', 'request_id': 'req-32bb3541-2607-4a07-bccf-25e7131b5f78'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:16.278384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-32bb3541-2607-4a07-bccf-25e7131b5f78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc02ab74778f4ebc92da025a71966779', 'request_id': 'req-32bb3541-2607-4a07-bccf-25e7131b5f78'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:16.307035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1537e-d1b0-472e-aa36-d46dde469d79 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:16.369491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-32bb3541-2607-4a07-bccf-25e7131b5f78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc02ab74778f4ebc92da025a71966779', 'request_id': 'req-32bb3541-2607-4a07-bccf-25e7131b5f78'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:16.384036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e25399-6abb-4051-bd51-2c7b549d9558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:16.384793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e25399-6abb-4051-bd51-2c7b549d9558 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:16.385083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e25399-6abb-4051-bd51-2c7b549d9558 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:16.385362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e25399-6abb-4051-bd51-2c7b549d9558 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:16.388793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e25399-6abb-4051-bd51-2c7b549d9558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:16.392097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-32bb3541-2607-4a07-bccf-25e7131b5f78 admin admin] The token cache is being invalidate because project cc02ab74778f4ebc92da025a71966779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:16.392992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-32bb3541-2607-4a07-bccf-25e7131b5f78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:16.394459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e25399-6abb-4051-bd51-2c7b549d9558 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:16.395345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-32bb3541-2607-4a07-bccf-25e7131b5f78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:16.403675 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2717/5400] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:16 2022] DELETE /identity/v3/projects/cc02ab74778f4ebc92da025a71966779 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:16.406857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446db158-034c-4e9f-93e5-6d2849014dff None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:16.407151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446db158-034c-4e9f-93e5-6d2849014dff None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:16.407471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446db158-034c-4e9f-93e5-6d2849014dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:16.433995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-446db158-034c-4e9f-93e5-6d2849014dff None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:16.433995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-446db158-034c-4e9f-93e5-6d2849014dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:16.446150 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2718/5401] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:20:16.456033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d26f2-83e2-49da-ae62-2ccc7f728b87 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:16.456716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d26f2-83e2-49da-ae62-2ccc7f728b87 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:16.457409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d26f2-83e2-49da-ae62-2ccc7f728b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:16.483686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-955d26f2-83e2-49da-ae62-2ccc7f728b87 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:16.491138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-955d26f2-83e2-49da-ae62-2ccc7f728b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:16.494321 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2719/5402] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:20:16.498505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d26f2-83e2-49da-ae62-2ccc7f728b87 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:16.511891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d26f2-83e2-49da-ae62-2ccc7f728b87 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:16.520412 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2684/5403] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:16 2022] POST /identity/v3/users => generated 298 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:16.521510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e25399-6abb-4051-bd51-2c7b549d9558 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:16.522820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41869031-ebb1-4fd0-ab87-53378642e0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:16.523748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e4e25399-6abb-4051-bd51-2c7b549d9558 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:16.524177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41869031-ebb1-4fd0-ab87-53378642e0dc admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:16.525095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41869031-ebb1-4fd0-ab87-53378642e0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:16.525566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41869031-ebb1-4fd0-ab87-53378642e0dc admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:16.526824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41869031-ebb1-4fd0-ab87-53378642e0dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:20:16.528044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41869031-ebb1-4fd0-ab87-53378642e0dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:16.531716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41869031-ebb1-4fd0-ab87-53378642e0dc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:16.538720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e25399-6abb-4051-bd51-2c7b549d9558 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:16.544360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a94253-1fd6-4123-a9e4-b0bc87aa99a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:16.553376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a94253-1fd6-4123-a9e4-b0bc87aa99a4 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:16.553376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a94253-1fd6-4123-a9e4-b0bc87aa99a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:16.553376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a94253-1fd6-4123-a9e4-b0bc87aa99a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:16.553376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a94253-1fd6-4123-a9e4-b0bc87aa99a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:16.553376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a94253-1fd6-4123-a9e4-b0bc87aa99a4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:16.553897 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2720/5404] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:20:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:20:16.556330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41869031-ebb1-4fd0-ab87-53378642e0dc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:16.569199 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2685/5405] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:16.573730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a94253-1fd6-4123-a9e4-b0bc87aa99a4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:16.577014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41869031-ebb1-4fd0-ab87-53378642e0dc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:16.582478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b4252-5fa5-456d-94c1-f16a7d068409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:16.583147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b4252-5fa5-456d-94c1-f16a7d068409 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:16.583492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b4252-5fa5-456d-94c1-f16a7d068409 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:16.583775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b4252-5fa5-456d-94c1-f16a7d068409 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:16.584529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b4252-5fa5-456d-94c1-f16a7d068409 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:16.586504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b4252-5fa5-456d-94c1-f16a7d068409 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:16.586954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a94253-1fd6-4123-a9e4-b0bc87aa99a4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:16.595845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c7314-e141-4535-bcf4-6839acd12069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:16.597171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c7314-e141-4535-bcf4-6839acd12069 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:16.597466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c7314-e141-4535-bcf4-6839acd12069 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:16.597748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c7314-e141-4535-bcf4-6839acd12069 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:16.598553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c7314-e141-4535-bcf4-6839acd12069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:16.600179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944c7314-e141-4535-bcf4-6839acd12069 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:16.612997 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2686/5406] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:16.617313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c7314-e141-4535-bcf4-6839acd12069 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:16.627845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c7314-e141-4535-bcf4-6839acd12069 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:16.632813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7ee7f-a700-4ac8-b79f-38e006ce49d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:16.633699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7ee7f-a700-4ac8-b79f-38e006ce49d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:16.634012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7ee7f-a700-4ac8-b79f-38e006ce49d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:16.634288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7ee7f-a700-4ac8-b79f-38e006ce49d3 admin admin] PATH_INFO: `/v3/projects/f03a2cfe8fd94e4e8581358beadaf177/users/2fb3ef26871e4b30a9bc4a5045fe62ad/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:16.657698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7ee7f-a700-4ac8-b79f-38e006ce49d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f03a2cfe8fd94e4e8581358beadaf177, user_id=2fb3ef26871e4b30a9bc4a5045fe62ad, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:16.660660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7ee7f-a700-4ac8-b79f-38e006ce49d3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:16.681564 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2721/5407] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:16 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:20:16.682536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b4252-5fa5-456d-94c1-f16a7d068409 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:16.685245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e54b4252-5fa5-456d-94c1-f16a7d068409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:16.752252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b4252-5fa5-456d-94c1-f16a7d068409 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:16.759474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94953c01-2087-4fe0-92ee-c75f91334be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:16.760389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94953c01-2087-4fe0-92ee-c75f91334be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:16.760680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94953c01-2087-4fe0-92ee-c75f91334be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:16.760982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94953c01-2087-4fe0-92ee-c75f91334be5 admin admin] PATH_INFO: `/v3/projects/c76d04ad120d41f6a3bfabb87bf6968e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:16.762224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94953c01-2087-4fe0-92ee-c75f91334be5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c76d04ad120d41f6a3bfabb87bf6968e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:16.764168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94953c01-2087-4fe0-92ee-c75f91334be5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:16.775019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:16.785172 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2687/5408] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:16 2022] PUT /identity/v3/projects/f03a2cfe8fd94e4e8581358beadaf177/users/2fb3ef26871e4b30a9bc4a5045fe62ad/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:16.786219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7ee7f-a700-4ac8-b79f-38e006ce49d3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:16.855720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7ee7f-a700-4ac8-b79f-38e006ce49d3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:16.860649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985598de-27c0-43ba-80ac-4c52c5b45585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:16.861394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985598de-27c0-43ba-80ac-4c52c5b45585 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:16.861681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985598de-27c0-43ba-80ac-4c52c5b45585 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:16.862069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985598de-27c0-43ba-80ac-4c52c5b45585 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:16.862771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985598de-27c0-43ba-80ac-4c52c5b45585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:16.864718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985598de-27c0-43ba-80ac-4c52c5b45585 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:16.942969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-94953c01-2087-4fe0-92ee-c75f91334be5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c76d04ad120d41f6a3bfabb87bf6968e', 'request_id': 'req-94953c01-2087-4fe0-92ee-c75f91334be5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:16.948714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-94953c01-2087-4fe0-92ee-c75f91334be5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c76d04ad120d41f6a3bfabb87bf6968e', 'request_id': 'req-94953c01-2087-4fe0-92ee-c75f91334be5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:16.984284 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2688/5409] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:16 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:16.985592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940f8b8-b32b-49ca-a7dc-f1f43ab8a222 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:16.985931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940f8b8-b32b-49ca-a7dc-f1f43ab8a222 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:16.986224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940f8b8-b32b-49ca-a7dc-f1f43ab8a222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:17.004460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7940f8b8-b32b-49ca-a7dc-f1f43ab8a222 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:17.005612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7940f8b8-b32b-49ca-a7dc-f1f43ab8a222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:17.077203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-94953c01-2087-4fe0-92ee-c75f91334be5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c76d04ad120d41f6a3bfabb87bf6968e', 'request_id': 'req-94953c01-2087-4fe0-92ee-c75f91334be5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:17.085899 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2689/5410] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:17.086816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940f8b8-b32b-49ca-a7dc-f1f43ab8a222 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:17.102295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-94953c01-2087-4fe0-92ee-c75f91334be5 admin admin] The token cache is being invalidate because project c76d04ad120d41f6a3bfabb87bf6968e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:17.102687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-94953c01-2087-4fe0-92ee-c75f91334be5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:17.103726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-94953c01-2087-4fe0-92ee-c75f91334be5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:17.104090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940f8b8-b32b-49ca-a7dc-f1f43ab8a222 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:17.105610 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2722/5411] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:16 2022] DELETE /identity/v3/projects/c76d04ad120d41f6a3bfabb87bf6968e => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:17.109742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c600e-3df8-48cd-9d6f-a319ce20feda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:17.110469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c600e-3df8-48cd-9d6f-a319ce20feda admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:17.110754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c600e-3df8-48cd-9d6f-a319ce20feda admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:17.111031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c600e-3df8-48cd-9d6f-a319ce20feda admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:17.111773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c600e-3df8-48cd-9d6f-a319ce20feda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:17.112461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94953c01-2087-4fe0-92ee-c75f91334be5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:17.113755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c600e-3df8-48cd-9d6f-a319ce20feda admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:17.116205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-94953c01-2087-4fe0-92ee-c75f91334be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:17.126079 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2690/5412] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:17.128026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c600e-3df8-48cd-9d6f-a319ce20feda admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:17.130505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3a1c600e-3df8-48cd-9d6f-a319ce20feda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:17.149722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c600e-3df8-48cd-9d6f-a319ce20feda admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:17.159199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafee087-9e9e-4b26-bc93-04c4dfdc6e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:17.160198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafee087-9e9e-4b26-bc93-04c4dfdc6e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:17.160486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafee087-9e9e-4b26-bc93-04c4dfdc6e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:17.160768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafee087-9e9e-4b26-bc93-04c4dfdc6e98 admin admin] PATH_INFO: `/v3/users/71ff0e897321473684786ae57014b9b4` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:17.164602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafee087-9e9e-4b26-bc93-04c4dfdc6e98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71ff0e897321473684786ae57014b9b4)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:17.164602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafee087-9e9e-4b26-bc93-04c4dfdc6e98 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:17.195277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94953c01-2087-4fe0-92ee-c75f91334be5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:17.199239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af204fc0-8abe-4e77-a203-293024bb7776 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:17.199991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af204fc0-8abe-4e77-a203-293024bb7776 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:17.200278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af204fc0-8abe-4e77-a203-293024bb7776 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:17.200556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af204fc0-8abe-4e77-a203-293024bb7776 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:17.202401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-af204fc0-8abe-4e77-a203-293024bb7776 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:17.216744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af204fc0-8abe-4e77-a203-293024bb7776 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:17.218790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af204fc0-8abe-4e77-a203-293024bb7776 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:17.224367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:17.234498 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2723/5413] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:17.235737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af204fc0-8abe-4e77-a203-293024bb7776 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:17.237214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-af204fc0-8abe-4e77-a203-293024bb7776 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:17.251783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af204fc0-8abe-4e77-a203-293024bb7776 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:17.256961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e63474-4a28-47a6-b8da-8a5ccd650822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:17.257576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e63474-4a28-47a6-b8da-8a5ccd650822 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:17.257884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e63474-4a28-47a6-b8da-8a5ccd650822 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:17.258169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e63474-4a28-47a6-b8da-8a5ccd650822 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:17.258907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e63474-4a28-47a6-b8da-8a5ccd650822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:17.260697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e63474-4a28-47a6-b8da-8a5ccd650822 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:17.272319 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2724/5414] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:17.277384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e63474-4a28-47a6-b8da-8a5ccd650822 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:17.291271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e63474-4a28-47a6-b8da-8a5ccd650822 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:17.308392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7402f1-ed47-4894-b80b-5a00675a58d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:17.309317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7402f1-ed47-4894-b80b-5a00675a58d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:17.309607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7402f1-ed47-4894-b80b-5a00675a58d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:17.311102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7402f1-ed47-4894-b80b-5a00675a58d1 admin admin] PATH_INFO: `/v3/projects/4c50b56053e24a5daa5f5885b773a80e/users/ee55d5fc3a234d4d8f0da4a227a92064/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:17.335965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7402f1-ed47-4894-b80b-5a00675a58d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c50b56053e24a5daa5f5885b773a80e, user_id=ee55d5fc3a234d4d8f0da4a227a92064, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:17.338800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7402f1-ed47-4894-b80b-5a00675a58d1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:17.396577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cafee087-9e9e-4b26-bc93-04c4dfdc6e98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71ff0e897321473684786ae57014b9b4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:17.398271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cafee087-9e9e-4b26-bc93-04c4dfdc6e98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71ff0e897321473684786ae57014b9b4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:17.420020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cafee087-9e9e-4b26-bc93-04c4dfdc6e98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71ff0e897321473684786ae57014b9b4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:17.454140 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2725/5415] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:17 2022] PUT /identity/v3/projects/4c50b56053e24a5daa5f5885b773a80e/users/ee55d5fc3a234d4d8f0da4a227a92064/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:17.455714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065ac43-be3f-4e9c-b970-2a601ae57ae5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:17.456011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065ac43-be3f-4e9c-b970-2a601ae57ae5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:17.456297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065ac43-be3f-4e9c-b970-2a601ae57ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:17.496255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2065ac43-be3f-4e9c-b970-2a601ae57ae5 None None] MFA Rules not processed for user `ee55d5fc3a234d4d8f0da4a227a92064`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:17.497658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2065ac43-be3f-4e9c-b970-2a601ae57ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:17.530316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cafee087-9e9e-4b26-bc93-04c4dfdc6e98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71ff0e897321473684786ae57014b9b4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:17.531508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cafee087-9e9e-4b26-bc93-04c4dfdc6e98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71ff0e897321473684786ae57014b9b4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:17.551231 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2691/5416] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:17 2022] DELETE /identity/v3/users/71ff0e897321473684786ae57014b9b4 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:17.579654 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2726/5417] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:17.584660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafee087-9e9e-4b26-bc93-04c4dfdc6e98 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:17.712311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafee087-9e9e-4b26-bc93-04c4dfdc6e98 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:17.716634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e84f3-a0b5-48de-9e9f-8dc59d6e7ca1 tempest-IdentityCatalogTest-1867421426 tempest-IdentityCatalogTest-1867421426-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee55d5fc3a234d4d8f0da4a227a92064', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4c50b56053e24a5daa5f5885b773a80e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:17.717302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e84f3-a0b5-48de-9e9f-8dc59d6e7ca1 tempest-IdentityCatalogTest-1867421426 tempest-IdentityCatalogTest-1867421426-project] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:17.717595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e84f3-a0b5-48de-9e9f-8dc59d6e7ca1 tempest-IdentityCatalogTest-1867421426 tempest-IdentityCatalogTest-1867421426-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:17.724346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e84f3-a0b5-48de-9e9f-8dc59d6e7ca1 tempest-IdentityCatalogTest-1867421426 tempest-IdentityCatalogTest-1867421426-project] PATH_INFO: `/v3/auth/catalog` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:17.724346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e84f3-a0b5-48de-9e9f-8dc59d6e7ca1 tempest-IdentityCatalogTest-1867421426 tempest-IdentityCatalogTest-1867421426-project] RBAC: Authorizing `identity:get_auth_catalog()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:17.724346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e84f3-a0b5-48de-9e9f-8dc59d6e7ca1 tempest-IdentityCatalogTest-1867421426 tempest-IdentityCatalogTest-1867421426-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:17.724346 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2692/5418] 10.222.0.33 () {58 vars in 1142 bytes} [Sat May 28 21:20:17 2022] GET /identity/v3/auth/catalog => generated 2648 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:17.730828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065ac43-be3f-4e9c-b970-2a601ae57ae5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:17.732714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2065ac43-be3f-4e9c-b970-2a601ae57ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:17.844275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065ac43-be3f-4e9c-b970-2a601ae57ae5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:17.975253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837b024-0239-4375-87ae-f36d99d0cdc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:17.984387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837b024-0239-4375-87ae-f36d99d0cdc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:17.984805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837b024-0239-4375-87ae-f36d99d0cdc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:17.985104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837b024-0239-4375-87ae-f36d99d0cdc4 admin admin] PATH_INFO: `/v3/users/ee55d5fc3a234d4d8f0da4a227a92064` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:17.987658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837b024-0239-4375-87ae-f36d99d0cdc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee55d5fc3a234d4d8f0da4a227a92064)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:17.990334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837b024-0239-4375-87ae-f36d99d0cdc4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:18.215246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c837b024-0239-4375-87ae-f36d99d0cdc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee55d5fc3a234d4d8f0da4a227a92064'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:18.224689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c837b024-0239-4375-87ae-f36d99d0cdc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee55d5fc3a234d4d8f0da4a227a92064'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:18.225512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c837b024-0239-4375-87ae-f36d99d0cdc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee55d5fc3a234d4d8f0da4a227a92064'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:18.230870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c837b024-0239-4375-87ae-f36d99d0cdc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee55d5fc3a234d4d8f0da4a227a92064'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:18.344153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c837b024-0239-4375-87ae-f36d99d0cdc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee55d5fc3a234d4d8f0da4a227a92064'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:18.345708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:18.360933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e84f3-a0b5-48de-9e9f-8dc59d6e7ca1 tempest-IdentityCatalogTest-1867421426 tempest-IdentityCatalogTest-1867421426-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:18.363292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e84e84f3-a0b5-48de-9e9f-8dc59d6e7ca1 tempest-IdentityCatalogTest-1867421426 tempest-IdentityCatalogTest-1867421426-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:18.375985 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2727/5419] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:17 2022] DELETE /identity/v3/users/ee55d5fc3a234d4d8f0da4a227a92064 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:18.387230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e84f3-a0b5-48de-9e9f-8dc59d6e7ca1 tempest-IdentityCatalogTest-1867421426 tempest-IdentityCatalogTest-1867421426-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:18.387558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4a6c3-4790-4c9c-aef9-1b87371fcffd None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:18.391732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4a6c3-4790-4c9c-aef9-1b87371fcffd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:18.391732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc4a6c3-4790-4c9c-aef9-1b87371fcffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:18.415270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-adc4a6c3-4790-4c9c-aef9-1b87371fcffd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:18.415270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-adc4a6c3-4790-4c9c-aef9-1b87371fcffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:18.493028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee6d022-5b26-4f09-901f-7e0d9cbb60cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:18.494952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6d022-5b26-4f09-901f-7e0d9cbb60cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:18.496499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6d022-5b26-4f09-901f-7e0d9cbb60cd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:18.497322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee6d022-5b26-4f09-901f-7e0d9cbb60cd admin admin] PATH_INFO: `/v3/projects/a9f02d5cb3c24032bcb2697292f31638` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:18.501542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6d022-5b26-4f09-901f-7e0d9cbb60cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9f02d5cb3c24032bcb2697292f31638)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:18.504809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee6d022-5b26-4f09-901f-7e0d9cbb60cd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:18.560275 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2728/5420] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:18.570785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4a6c3-4790-4c9c-aef9-1b87371fcffd None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:18.572059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-adc4a6c3-4790-4c9c-aef9-1b87371fcffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:18.636460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc4a6c3-4790-4c9c-aef9-1b87371fcffd None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:18.640800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ea9a1-ff73-40a7-a216-d43d70c878e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:18.641490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ea9a1-ff73-40a7-a216-d43d70c878e9 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:18.641776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ea9a1-ff73-40a7-a216-d43d70c878e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:18.642086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ea9a1-ff73-40a7-a216-d43d70c878e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:18.658857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ea9a1-ff73-40a7-a216-d43d70c878e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:18.660410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ea9a1-ff73-40a7-a216-d43d70c878e9 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:18.674899 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2729/5421] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:20:18.684130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1ee6d022-5b26-4f09-901f-7e0d9cbb60cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9f02d5cb3c24032bcb2697292f31638', 'request_id': 'req-1ee6d022-5b26-4f09-901f-7e0d9cbb60cd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:18.691165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1ee6d022-5b26-4f09-901f-7e0d9cbb60cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9f02d5cb3c24032bcb2697292f31638', 'request_id': 'req-1ee6d022-5b26-4f09-901f-7e0d9cbb60cd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:18.816094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1ee6d022-5b26-4f09-901f-7e0d9cbb60cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9f02d5cb3c24032bcb2697292f31638', 'request_id': 'req-1ee6d022-5b26-4f09-901f-7e0d9cbb60cd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:18.828668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1ee6d022-5b26-4f09-901f-7e0d9cbb60cd admin admin] The token cache is being invalidate because project a9f02d5cb3c24032bcb2697292f31638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:18.828965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1ee6d022-5b26-4f09-901f-7e0d9cbb60cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:18.829626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1ee6d022-5b26-4f09-901f-7e0d9cbb60cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:18.831482 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2693/5422] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:18 2022] DELETE /identity/v3/projects/a9f02d5cb3c24032bcb2697292f31638 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:18.836689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61896f-54a6-43b1-88d1-e8f33932a6ee None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:18.837414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61896f-54a6-43b1-88d1-e8f33932a6ee None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:18.837727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61896f-54a6-43b1-88d1-e8f33932a6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:18.856398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5b61896f-54a6-43b1-88d1-e8f33932a6ee None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:18.857614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5b61896f-54a6-43b1-88d1-e8f33932a6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:18.933566 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2730/5423] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:18.939948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fb728-e24f-4e73-9d4b-825d49f152f5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:18.939948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fb728-e24f-4e73-9d4b-825d49f152f5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:18.939948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94fb728-e24f-4e73-9d4b-825d49f152f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:18.959660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b94fb728-e24f-4e73-9d4b-825d49f152f5 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:18.960977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b94fb728-e24f-4e73-9d4b-825d49f152f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:18.969911 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2694/5424] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:18.974020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61896f-54a6-43b1-88d1-e8f33932a6ee None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:18.984985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61896f-54a6-43b1-88d1-e8f33932a6ee None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:18.990284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a41091-ba69-446e-a290-bd67b9b67304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:18.990954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a41091-ba69-446e-a290-bd67b9b67304 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:18.991239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a41091-ba69-446e-a290-bd67b9b67304 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:18.991521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a41091-ba69-446e-a290-bd67b9b67304 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:18.992265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a41091-ba69-446e-a290-bd67b9b67304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:20:18.992615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a41091-ba69-446e-a290-bd67b9b67304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:18.994277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a41091-ba69-446e-a290-bd67b9b67304 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:19.017076 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2731/5425] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:20:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:19.018577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fb728-e24f-4e73-9d4b-825d49f152f5 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:19.032508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94fb728-e24f-4e73-9d4b-825d49f152f5 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:19.036854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d30558-fb95-427f-87bd-aa17f5a2dba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:19.037773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d30558-fb95-427f-87bd-aa17f5a2dba2 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:19.038101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d30558-fb95-427f-87bd-aa17f5a2dba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:19.038361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d30558-fb95-427f-87bd-aa17f5a2dba2 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:19.039078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d30558-fb95-427f-87bd-aa17f5a2dba2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:19.041139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d30558-fb95-427f-87bd-aa17f5a2dba2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:19.165331 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2695/5426] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:19 2022] POST /identity/v3/projects => generated 375 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:19.166628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a41091-ba69-446e-a290-bd67b9b67304 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:19.179157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a41091-ba69-446e-a290-bd67b9b67304 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:19.258324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90400e9-19d7-4d4b-9686-fbc539ecae89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:19.259002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90400e9-19d7-4d4b-9686-fbc539ecae89 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:19.259291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90400e9-19d7-4d4b-9686-fbc539ecae89 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:19.259560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90400e9-19d7-4d4b-9686-fbc539ecae89 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:19.260273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90400e9-19d7-4d4b-9686-fbc539ecae89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:19.262475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90400e9-19d7-4d4b-9686-fbc539ecae89 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:19.304340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d30558-fb95-427f-87bd-aa17f5a2dba2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:19.304901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b2d30558-fb95-427f-87bd-aa17f5a2dba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:19.365023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:19.375541 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2732/5427] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:19 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:20:19.375541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90400e9-19d7-4d4b-9686-fbc539ecae89 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:19.378307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b90400e9-19d7-4d4b-9686-fbc539ecae89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:19.379757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d30558-fb95-427f-87bd-aa17f5a2dba2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:19.384223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630b87d-bdfe-4938-bb60-5bc51f70a256 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:19.385020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630b87d-bdfe-4938-bb60-5bc51f70a256 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:19.385425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630b87d-bdfe-4938-bb60-5bc51f70a256 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:19.385784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1630b87d-bdfe-4938-bb60-5bc51f70a256 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:19.388618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1630b87d-bdfe-4938-bb60-5bc51f70a256 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:19.392782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90400e9-19d7-4d4b-9686-fbc539ecae89 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:19.397696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b4ad4-15af-4023-a072-2183bb97e23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:19.398787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b4ad4-15af-4023-a072-2183bb97e23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:19.399079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b4ad4-15af-4023-a072-2183bb97e23b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:19.399361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939b4ad4-15af-4023-a072-2183bb97e23b admin admin] PATH_INFO: `/v3/projects/4c50b56053e24a5daa5f5885b773a80e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:19.400626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b4ad4-15af-4023-a072-2183bb97e23b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c50b56053e24a5daa5f5885b773a80e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:19.413860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939b4ad4-15af-4023-a072-2183bb97e23b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:19.508641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630b87d-bdfe-4938-bb60-5bc51f70a256 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:19.510254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1630b87d-bdfe-4938-bb60-5bc51f70a256 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:19.524164 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2696/5428] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:19.525341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630b87d-bdfe-4938-bb60-5bc51f70a256 service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:19.554554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1630b87d-bdfe-4938-bb60-5bc51f70a256 service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:19.582765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-939b4ad4-15af-4023-a072-2183bb97e23b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c50b56053e24a5daa5f5885b773a80e', 'request_id': 'req-939b4ad4-15af-4023-a072-2183bb97e23b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:19.597685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-939b4ad4-15af-4023-a072-2183bb97e23b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c50b56053e24a5daa5f5885b773a80e', 'request_id': 'req-939b4ad4-15af-4023-a072-2183bb97e23b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:19.652776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e6585-3e09-49eb-a5b5-f7ab6fd59450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:19.653620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e6585-3e09-49eb-a5b5-f7ab6fd59450 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:19.654040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e6585-3e09-49eb-a5b5-f7ab6fd59450 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:19.654414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e6585-3e09-49eb-a5b5-f7ab6fd59450 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:19.655295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e6585-3e09-49eb-a5b5-f7ab6fd59450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:19.657702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e6585-3e09-49eb-a5b5-f7ab6fd59450 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:19.670916 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2697/5429] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:19.672428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad17fc-b9d2-467d-b2cd-9dbedbcd4b7b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:19.672894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad17fc-b9d2-467d-b2cd-9dbedbcd4b7b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:19.673094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad17fc-b9d2-467d-b2cd-9dbedbcd4b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:19.703300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-80ad17fc-b9d2-467d-b2cd-9dbedbcd4b7b None None] MFA Rules not processed for user `2fb3ef26871e4b30a9bc4a5045fe62ad`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:19.704524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-80ad17fc-b9d2-467d-b2cd-9dbedbcd4b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:19.713297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-939b4ad4-15af-4023-a072-2183bb97e23b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c50b56053e24a5daa5f5885b773a80e', 'request_id': 'req-939b4ad4-15af-4023-a072-2183bb97e23b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:19.727947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-939b4ad4-15af-4023-a072-2183bb97e23b admin admin] The token cache is being invalidate because project 4c50b56053e24a5daa5f5885b773a80e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:19.728813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-939b4ad4-15af-4023-a072-2183bb97e23b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:19.729465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-939b4ad4-15af-4023-a072-2183bb97e23b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:19.731393 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2733/5430] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:19 2022] DELETE /identity/v3/projects/4c50b56053e24a5daa5f5885b773a80e => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:19.732350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b4ad4-15af-4023-a072-2183bb97e23b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:19.733760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-939b4ad4-15af-4023-a072-2183bb97e23b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:19.752180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939b4ad4-15af-4023-a072-2183bb97e23b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:19.759306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337bd59-0030-4f81-be67-a0caf5467432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:19.759995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337bd59-0030-4f81-be67-a0caf5467432 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:19.760287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337bd59-0030-4f81-be67-a0caf5467432 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:19.760570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337bd59-0030-4f81-be67-a0caf5467432 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:19.761821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337bd59-0030-4f81-be67-a0caf5467432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:19.764003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337bd59-0030-4f81-be67-a0caf5467432 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:19.778628 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2734/5431] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:19.780453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097a7bd-225d-42e5-a619-53ae981410e8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:19.781316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097a7bd-225d-42e5-a619-53ae981410e8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:19.781657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097a7bd-225d-42e5-a619-53ae981410e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:19.802384 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2698/5432] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:19.802384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad17fc-b9d2-467d-b2cd-9dbedbcd4b7b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:19.803194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b097a7bd-225d-42e5-a619-53ae981410e8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:19.804568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b097a7bd-225d-42e5-a619-53ae981410e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:19.813526 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2735/5433] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:19.815287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a15f53-b24b-47b5-960e-a8c1bc68a8c1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:19.815685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a15f53-b24b-47b5-960e-a8c1bc68a8c1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:19.816059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a15f53-b24b-47b5-960e-a8c1bc68a8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:19.825172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad17fc-b9d2-467d-b2cd-9dbedbcd4b7b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:19.828608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652efef-0414-4494-82b5-15191bc1158e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:19.829542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652efef-0414-4494-82b5-15191bc1158e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:19.829830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652efef-0414-4494-82b5-15191bc1158e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:19.830143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652efef-0414-4494-82b5-15191bc1158e admin admin] PATH_INFO: `/v3/projects/72d4e610a6d641ae9c2c94be6de7de36/users/7bfaf45cd74a4e148f2661c249869977/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:19.838932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c3a15f53-b24b-47b5-960e-a8c1bc68a8c1 None None] MFA Rules not processed for user `2fb3ef26871e4b30a9bc4a5045fe62ad`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:19.840279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c3a15f53-b24b-47b5-960e-a8c1bc68a8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:19.847536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652efef-0414-4494-82b5-15191bc1158e admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d4e610a6d641ae9c2c94be6de7de36, user_id=7bfaf45cd74a4e148f2661c249869977, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:19.849202 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2736/5434] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:19.850336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5aa3b5-5f06-4286-a028-293fa091858a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:19.850604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5aa3b5-5f06-4286-a028-293fa091858a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:19.850890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5aa3b5-5f06-4286-a028-293fa091858a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:19.852365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652efef-0414-4494-82b5-15191bc1158e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:19.880830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ca5aa3b5-5f06-4286-a028-293fa091858a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:19.882209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ca5aa3b5-5f06-4286-a028-293fa091858a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:19.893994 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2737/5435] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:19.893994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5aa3b5-5f06-4286-a028-293fa091858a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:19.893994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ca5aa3b5-5f06-4286-a028-293fa091858a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:19.976545 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2699/5436] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:19 2022] PUT /identity/v3/projects/72d4e610a6d641ae9c2c94be6de7de36/users/7bfaf45cd74a4e148f2661c249869977/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:19.977537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652efef-0414-4494-82b5-15191bc1158e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:19.991790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5aa3b5-5f06-4286-a028-293fa091858a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:20.056312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652efef-0414-4494-82b5-15191bc1158e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:20.069499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3452e-706a-48e7-b650-b919633d504d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:20.070473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6019a0-c6de-4608-90ae-09e23c57e1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:20.070741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3452e-706a-48e7-b650-b919633d504d service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:20.071277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3452e-706a-48e7-b650-b919633d504d service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:20.071773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6019a0-c6de-4608-90ae-09e23c57e1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:20.072104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6019a0-c6de-4608-90ae-09e23c57e1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:20.072387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6019a0-c6de-4608-90ae-09e23c57e1c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:20.072962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3452e-706a-48e7-b650-b919633d504d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:20.073240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6019a0-c6de-4608-90ae-09e23c57e1c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:20:20.073558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6019a0-c6de-4608-90ae-09e23c57e1c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:20.075509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6019a0-c6de-4608-90ae-09e23c57e1c9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:20.089928 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2700/5437] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:20:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:20.091543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7932f-6c5a-410c-8a38-613760ac9991 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:20.091834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7932f-6c5a-410c-8a38-613760ac9991 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:20.092115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7932f-6c5a-410c-8a38-613760ac9991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:20.134704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-34b7932f-6c5a-410c-8a38-613760ac9991 None None] MFA Rules not processed for user `7bfaf45cd74a4e148f2661c249869977`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:20.136718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-34b7932f-6c5a-410c-8a38-613760ac9991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:20.150091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3452e-706a-48e7-b650-b919633d504d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:20.151647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3452e-706a-48e7-b650-b919633d504d service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:20.161116 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2738/5438] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:20.161863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3452e-706a-48e7-b650-b919633d504d service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:20.173255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3452e-706a-48e7-b650-b919633d504d service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:20.178576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c7627-61cd-4e08-8dc9-adeb94f98d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:20.179318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c7627-61cd-4e08-8dc9-adeb94f98d6e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:20.179608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c7627-61cd-4e08-8dc9-adeb94f98d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:20.179874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c7627-61cd-4e08-8dc9-adeb94f98d6e admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:20.180578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c7627-61cd-4e08-8dc9-adeb94f98d6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:20.182597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c7627-61cd-4e08-8dc9-adeb94f98d6e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:20.221283 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2701/5439] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:20.239251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7932f-6c5a-410c-8a38-613760ac9991 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:20.240647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-34b7932f-6c5a-410c-8a38-613760ac9991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:20.314566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7932f-6c5a-410c-8a38-613760ac9991 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:20.318370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6ae12-9aba-4834-a1de-acc6459cc297 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:20.319026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6ae12-9aba-4834-a1de-acc6459cc297 service glance] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:20.319307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6ae12-9aba-4834-a1de-acc6459cc297 service glance] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:20.319575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6ae12-9aba-4834-a1de-acc6459cc297 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:20.332010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6ae12-9aba-4834-a1de-acc6459cc297 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:20.333662 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2739/5440] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:20 2022] POST /identity/v3/projects => generated 379 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:20.333801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6ae12-9aba-4834-a1de-acc6459cc297 service glance] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:20.337592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c7627-61cd-4e08-8dc9-adeb94f98d6e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:20.344962 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2702/5441] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:20:20.441047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c7627-61cd-4e08-8dc9-adeb94f98d6e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:20.448820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee51f5c-d8de-4c3b-90de-3dc7dd230063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:20.448820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee51f5c-d8de-4c3b-90de-3dc7dd230063 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:20.448820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee51f5c-d8de-4c3b-90de-3dc7dd230063 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:20.448820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee51f5c-d8de-4c3b-90de-3dc7dd230063 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:20.449224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee51f5c-d8de-4c3b-90de-3dc7dd230063 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:20.451133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee51f5c-d8de-4c3b-90de-3dc7dd230063 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:20.555957 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2740/5442] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:20 2022] POST /identity/v3/users => generated 290 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:20.559797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6ae12-9aba-4834-a1de-acc6459cc297 service glance] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:20.573188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6ae12-9aba-4834-a1de-acc6459cc297 service glance] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:20.578404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=84,keystone:DELETE=16 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:20.584488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3f4f7-d192-46aa-b2cf-e3bb132c6ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:20.585441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3f4f7-d192-46aa-b2cf-e3bb132c6ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:20.586140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3f4f7-d192-46aa-b2cf-e3bb132c6ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:20.586684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3f4f7-d192-46aa-b2cf-e3bb132c6ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:20.587698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3f4f7-d192-46aa-b2cf-e3bb132c6ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:20.589584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3f4f7-d192-46aa-b2cf-e3bb132c6ae8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:20.602917 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2703/5443] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:20.607290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee51f5c-d8de-4c3b-90de-3dc7dd230063 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:20.618172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee51f5c-d8de-4c3b-90de-3dc7dd230063 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:20.623268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f6dd7-1d49-4300-9eef-f38b18f1b8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:20.623954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f6dd7-1d49-4300-9eef-f38b18f1b8ee admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:20.624415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f6dd7-1d49-4300-9eef-f38b18f1b8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:20.624705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f6dd7-1d49-4300-9eef-f38b18f1b8ee admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:20.625576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f6dd7-1d49-4300-9eef-f38b18f1b8ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:20.627354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f6dd7-1d49-4300-9eef-f38b18f1b8ee admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:20.643444 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2741/5444] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:20.651672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3f4f7-d192-46aa-b2cf-e3bb132c6ae8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:20.662817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3f4f7-d192-46aa-b2cf-e3bb132c6ae8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:20.668397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3add02c-b255-49a7-a61d-75d729828404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:20.671285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3add02c-b255-49a7-a61d-75d729828404 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:20.671285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3add02c-b255-49a7-a61d-75d729828404 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:20.671285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3add02c-b255-49a7-a61d-75d729828404 admin admin] PATH_INFO: `/v3/projects/04338f70dc804beb8e70684a90cc7c2e/users/242787be03c24b47adf817a3fc99e71a/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:20.690553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3add02c-b255-49a7-a61d-75d729828404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04338f70dc804beb8e70684a90cc7c2e, user_id=242787be03c24b47adf817a3fc99e71a, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:20.694193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3add02c-b255-49a7-a61d-75d729828404 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:20.838635 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2704/5445] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:20 2022] PUT /identity/v3/projects/04338f70dc804beb8e70684a90cc7c2e/users/242787be03c24b47adf817a3fc99e71a/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:20.840420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35042f2-9a3c-476c-83de-7ff1fb69f23f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:20.840773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35042f2-9a3c-476c-83de-7ff1fb69f23f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:20.841117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35042f2-9a3c-476c-83de-7ff1fb69f23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:20.873875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e35042f2-9a3c-476c-83de-7ff1fb69f23f None None] MFA Rules not processed for user `242787be03c24b47adf817a3fc99e71a`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:20.875662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e35042f2-9a3c-476c-83de-7ff1fb69f23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:20.947866 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2742/5446] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:20.950430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3add02c-b255-49a7-a61d-75d729828404 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:21.024892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3add02c-b255-49a7-a61d-75d729828404 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:21.031342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164be8ac-635c-4f37-833f-84a8a7a1626c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:21.032135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164be8ac-635c-4f37-833f-84a8a7a1626c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:21.032421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164be8ac-635c-4f37-833f-84a8a7a1626c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:21.032697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164be8ac-635c-4f37-833f-84a8a7a1626c admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:21.033409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164be8ac-635c-4f37-833f-84a8a7a1626c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:21.035361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164be8ac-635c-4f37-833f-84a8a7a1626c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:21.133890 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2705/5447] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:20 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:21.138272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35042f2-9a3c-476c-83de-7ff1fb69f23f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:21.149846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35042f2-9a3c-476c-83de-7ff1fb69f23f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:21.211388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f4534-7815-4655-8b41-3c334c935fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:21.212073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f4534-7815-4655-8b41-3c334c935fcd admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:21.212366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f4534-7815-4655-8b41-3c334c935fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:21.212649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f4534-7815-4655-8b41-3c334c935fcd admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:21.213347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f4534-7815-4655-8b41-3c334c935fcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:21.215349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f4534-7815-4655-8b41-3c334c935fcd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:21.366203 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2743/5448] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:21 2022] POST /identity/v3/users => generated 291 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:21.370130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164be8ac-635c-4f37-833f-84a8a7a1626c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:21.381104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164be8ac-635c-4f37-833f-84a8a7a1626c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:21.385724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b8f03-09ad-476f-8bf0-29cb20956f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:21.386687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b8f03-09ad-476f-8bf0-29cb20956f0b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:21.386954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b8f03-09ad-476f-8bf0-29cb20956f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:21.387240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b8f03-09ad-476f-8bf0-29cb20956f0b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:21.387979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b8f03-09ad-476f-8bf0-29cb20956f0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:21.389593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b8f03-09ad-476f-8bf0-29cb20956f0b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:21.403631 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2706/5449] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:21.409602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f4534-7815-4655-8b41-3c334c935fcd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:21.419977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f4534-7815-4655-8b41-3c334c935fcd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:21.428518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ee57a-e42f-4ba0-8b1b-6514ddd54457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:21.431779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ee57a-e42f-4ba0-8b1b-6514ddd54457 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:21.431779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ee57a-e42f-4ba0-8b1b-6514ddd54457 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:21.431779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ee57a-e42f-4ba0-8b1b-6514ddd54457 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:21.431779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ee57a-e42f-4ba0-8b1b-6514ddd54457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:21.433784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ee57a-e42f-4ba0-8b1b-6514ddd54457 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:21.467474 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2744/5450] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:21.474337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b8f03-09ad-476f-8bf0-29cb20956f0b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:21.508484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b8f03-09ad-476f-8bf0-29cb20956f0b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:21.521279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602668ed-0453-4b8c-a417-c34c525fc76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:21.534040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602668ed-0453-4b8c-a417-c34c525fc76e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:21.534537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602668ed-0453-4b8c-a417-c34c525fc76e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:21.534744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602668ed-0453-4b8c-a417-c34c525fc76e admin admin] PATH_INFO: `/v3/projects/71b9d1a53b954a68a961993fdcdb5cc9/users/f43e75de1fc64de8b7f47f87ffa1c8fb/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:21.553449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602668ed-0453-4b8c-a417-c34c525fc76e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b9d1a53b954a68a961993fdcdb5cc9, user_id=f43e75de1fc64de8b7f47f87ffa1c8fb, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:21.556573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602668ed-0453-4b8c-a417-c34c525fc76e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:21.663896 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2707/5451] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:21 2022] PUT /identity/v3/projects/71b9d1a53b954a68a961993fdcdb5cc9/users/f43e75de1fc64de8b7f47f87ffa1c8fb/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:21.759984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028d636-cf24-4c2e-bc39-ac93675e5140 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:21.759984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028d636-cf24-4c2e-bc39-ac93675e5140 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:21.759984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028d636-cf24-4c2e-bc39-ac93675e5140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:21.789808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5028d636-cf24-4c2e-bc39-ac93675e5140 None None] MFA Rules not processed for user `f43e75de1fc64de8b7f47f87ffa1c8fb`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:21.791301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5028d636-cf24-4c2e-bc39-ac93675e5140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:21.861300 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2745/5452] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:21.866313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602668ed-0453-4b8c-a417-c34c525fc76e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:21.936525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602668ed-0453-4b8c-a417-c34c525fc76e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:21.943251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:21.945524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92ddb1-d789-41f3-89a8-af213a454f59 tempest-IdentityV3ProjectsTest-415396022 tempest-IdentityV3ProjectsTest-415396022-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '242787be03c24b47adf817a3fc99e71a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04338f70dc804beb8e70684a90cc7c2e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:21.946609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92ddb1-d789-41f3-89a8-af213a454f59 tempest-IdentityV3ProjectsTest-415396022 tempest-IdentityV3ProjectsTest-415396022-project] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:21.953135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92ddb1-d789-41f3-89a8-af213a454f59 tempest-IdentityV3ProjectsTest-415396022 tempest-IdentityV3ProjectsTest-415396022-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:21.953135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92ddb1-d789-41f3-89a8-af213a454f59 tempest-IdentityV3ProjectsTest-415396022 tempest-IdentityV3ProjectsTest-415396022-project] PATH_INFO: `/v3/users/242787be03c24b47adf817a3fc99e71a/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:21.953135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92ddb1-d789-41f3-89a8-af213a454f59 tempest-IdentityV3ProjectsTest-415396022 tempest-IdentityV3ProjectsTest-415396022-project] RBAC: Authorizing `identity:list_user_projects(user_id=242787be03c24b47adf817a3fc99e71a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:21.953135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c92ddb1-d789-41f3-89a8-af213a454f59 tempest-IdentityV3ProjectsTest-415396022 tempest-IdentityV3ProjectsTest-415396022-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:22.009728 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2708/5453] 10.222.0.33 () {58 vars in 1247 bytes} [Sat May 28 21:20:21 2022] GET /identity/v3/users/242787be03c24b47adf817a3fc99e71a/projects => generated 515 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:22.014508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83664134-88b9-475f-bcd8-cfc41a54fbae None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:22.015824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83664134-88b9-475f-bcd8-cfc41a54fbae None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:22.021563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83664134-88b9-475f-bcd8-cfc41a54fbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:22.050347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-83664134-88b9-475f-bcd8-cfc41a54fbae None None] MFA Rules not processed for user `242787be03c24b47adf817a3fc99e71a`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:22.050347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-83664134-88b9-475f-bcd8-cfc41a54fbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:22.059695 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2746/5454] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:22.064711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef6102-c4f2-4671-9dd2-8d4adf3582a2 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:22.065195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef6102-c4f2-4671-9dd2-8d4adf3582a2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:22.065583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef6102-c4f2-4671-9dd2-8d4adf3582a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:22.085710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-38ef6102-c4f2-4671-9dd2-8d4adf3582a2 None None] MFA Rules not processed for user `242787be03c24b47adf817a3fc99e71a`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:22.087068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-38ef6102-c4f2-4671-9dd2-8d4adf3582a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:22.154957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.models.token_model [None req-38ef6102-c4f2-4671-9dd2-8d4adf3582a2 None None] User 242787be03c24b47adf817a3fc99e71a has no access to project 71b9d1a53b954a68a961993fdcdb5cc9 {{(pid=103104) _validate_project_scope /opt/stack/keystone/keystone/models/token_model.py:539}} May 28 21:20:22.160302 ncdv-master-1 devstack@keystone.service[103104]: WARNING keystone.server.flask.application [None req-38ef6102-c4f2-4671-9dd2-8d4adf3582a2 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.33: keystone.exception.Unauthorized: The request you have made requires authentication. May 28 21:20:22.161822 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2709/5455] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:22 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 98 msecs (HTTP/1.1 401) 6 headers in 252 bytes (2 switches on core 0) May 28 21:20:22.167114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83664134-88b9-475f-bcd8-cfc41a54fbae None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:22.246765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83664134-88b9-475f-bcd8-cfc41a54fbae None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:22.252984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d87d45-f343-42f0-aef1-5af29c2a72eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:22.253925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d87d45-f343-42f0-aef1-5af29c2a72eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:22.254469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d87d45-f343-42f0-aef1-5af29c2a72eb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:22.254830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d87d45-f343-42f0-aef1-5af29c2a72eb admin admin] PATH_INFO: `/v3/users/242787be03c24b47adf817a3fc99e71a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:22.256148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d87d45-f343-42f0-aef1-5af29c2a72eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=242787be03c24b47adf817a3fc99e71a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:22.258209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d87d45-f343-42f0-aef1-5af29c2a72eb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:22.462965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:22.579468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-24d87d45-f343-42f0-aef1-5af29c2a72eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '242787be03c24b47adf817a3fc99e71a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:22.586945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-24d87d45-f343-42f0-aef1-5af29c2a72eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '242787be03c24b47adf817a3fc99e71a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:22.587824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-24d87d45-f343-42f0-aef1-5af29c2a72eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '242787be03c24b47adf817a3fc99e71a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:22.588979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-24d87d45-f343-42f0-aef1-5af29c2a72eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '242787be03c24b47adf817a3fc99e71a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:22.752984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-24d87d45-f343-42f0-aef1-5af29c2a72eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '242787be03c24b47adf817a3fc99e71a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:22.771311 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2747/5456] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:22 2022] DELETE /identity/v3/users/242787be03c24b47adf817a3fc99e71a => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:22.775980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a135f37-fb76-4c44-b419-cad7156a74b9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:22.776483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a135f37-fb76-4c44-b419-cad7156a74b9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:22.776593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a135f37-fb76-4c44-b419-cad7156a74b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:22.794026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7a135f37-fb76-4c44-b419-cad7156a74b9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:22.795263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7a135f37-fb76-4c44-b419-cad7156a74b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:22.848592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d87d45-f343-42f0-aef1-5af29c2a72eb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:22.850217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-24d87d45-f343-42f0-aef1-5af29c2a72eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:22.870912 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2710/5457] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:22.882558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a135f37-fb76-4c44-b419-cad7156a74b9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:22.884123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7a135f37-fb76-4c44-b419-cad7156a74b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:22.938663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d87d45-f343-42f0-aef1-5af29c2a72eb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:22.944578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99879c-9a02-4410-b650-bada1ae69ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:22.945494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99879c-9a02-4410-b650-bada1ae69ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:22.945781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99879c-9a02-4410-b650-bada1ae69ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:22.946127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99879c-9a02-4410-b650-bada1ae69ef1 admin admin] PATH_INFO: `/v3/users/96957ea63b5b47f4a2beb806064b1b63` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:22.947868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99879c-9a02-4410-b650-bada1ae69ef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96957ea63b5b47f4a2beb806064b1b63)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:22.950133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a99879c-9a02-4410-b650-bada1ae69ef1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:22.960037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a135f37-fb76-4c44-b419-cad7156a74b9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:22.964460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76863583-608a-4c23-b387-b067a25203d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:22.965296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76863583-608a-4c23-b387-b067a25203d1 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:22.965775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76863583-608a-4c23-b387-b067a25203d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:22.966297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76863583-608a-4c23-b387-b067a25203d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:22.977665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76863583-608a-4c23-b387-b067a25203d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:22.979531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76863583-608a-4c23-b387-b067a25203d1 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:22.991494 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2711/5458] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:20:23.173242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7a99879c-9a02-4410-b650-bada1ae69ef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96957ea63b5b47f4a2beb806064b1b63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:23.180424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7a99879c-9a02-4410-b650-bada1ae69ef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96957ea63b5b47f4a2beb806064b1b63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:23.181098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7a99879c-9a02-4410-b650-bada1ae69ef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96957ea63b5b47f4a2beb806064b1b63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:23.181763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7a99879c-9a02-4410-b650-bada1ae69ef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96957ea63b5b47f4a2beb806064b1b63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:23.262315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7a99879c-9a02-4410-b650-bada1ae69ef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96957ea63b5b47f4a2beb806064b1b63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:23.286703 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2748/5459] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:22 2022] DELETE /identity/v3/users/96957ea63b5b47f4a2beb806064b1b63 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:23.291291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71350fb0-dc7c-4c90-9894-e811a123c3d1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:23.291579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71350fb0-dc7c-4c90-9894-e811a123c3d1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:23.291854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71350fb0-dc7c-4c90-9894-e811a123c3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:23.311544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-71350fb0-dc7c-4c90-9894-e811a123c3d1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:23.312844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-71350fb0-dc7c-4c90-9894-e811a123c3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:23.381179 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2712/5460] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:23.392256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99879c-9a02-4410-b650-bada1ae69ef1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:23.393885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7a99879c-9a02-4410-b650-bada1ae69ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:23.460228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99879c-9a02-4410-b650-bada1ae69ef1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:23.467004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdc110-edc0-458e-97f2-6581a3baf0e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:23.467004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdc110-edc0-458e-97f2-6581a3baf0e9 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:23.467004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdc110-edc0-458e-97f2-6581a3baf0e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:23.467004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdc110-edc0-458e-97f2-6581a3baf0e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:23.470269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71350fb0-dc7c-4c90-9894-e811a123c3d1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:23.471752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-71350fb0-dc7c-4c90-9894-e811a123c3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:23.479150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffdc110-edc0-458e-97f2-6581a3baf0e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:23.480640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffdc110-edc0-458e-97f2-6581a3baf0e9 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:23.487722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71350fb0-dc7c-4c90-9894-e811a123c3d1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:23.491306 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2749/5461] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:23.493906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936fc74-9bb2-4c29-9d20-d9d4ea945814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:23.494960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936fc74-9bb2-4c29-9d20-d9d4ea945814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:23.495251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936fc74-9bb2-4c29-9d20-d9d4ea945814 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:23.495551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936fc74-9bb2-4c29-9d20-d9d4ea945814 admin admin] PATH_INFO: `/v3/projects/04338f70dc804beb8e70684a90cc7c2e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:23.496801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936fc74-9bb2-4c29-9d20-d9d4ea945814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04338f70dc804beb8e70684a90cc7c2e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:23.499729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936fc74-9bb2-4c29-9d20-d9d4ea945814 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:23.623455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6936fc74-9bb2-4c29-9d20-d9d4ea945814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04338f70dc804beb8e70684a90cc7c2e', 'request_id': 'req-6936fc74-9bb2-4c29-9d20-d9d4ea945814'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:23.629597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6936fc74-9bb2-4c29-9d20-d9d4ea945814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04338f70dc804beb8e70684a90cc7c2e', 'request_id': 'req-6936fc74-9bb2-4c29-9d20-d9d4ea945814'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:23.691006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6936fc74-9bb2-4c29-9d20-d9d4ea945814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04338f70dc804beb8e70684a90cc7c2e', 'request_id': 'req-6936fc74-9bb2-4c29-9d20-d9d4ea945814'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:23.703403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6936fc74-9bb2-4c29-9d20-d9d4ea945814 admin admin] The token cache is being invalidate because project 04338f70dc804beb8e70684a90cc7c2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:23.703695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6936fc74-9bb2-4c29-9d20-d9d4ea945814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:23.704345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6936fc74-9bb2-4c29-9d20-d9d4ea945814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:23.706000 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2713/5462] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:23 2022] DELETE /identity/v3/projects/04338f70dc804beb8e70684a90cc7c2e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:23.710046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdc110-edc0-458e-97f2-6581a3baf0e9 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:23.720060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdc110-edc0-458e-97f2-6581a3baf0e9 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:23.759866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936fc74-9bb2-4c29-9d20-d9d4ea945814 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:23.761426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6936fc74-9bb2-4c29-9d20-d9d4ea945814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:23.783561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:23.797506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a5abe-120b-41c8-8fa6-5313280a363d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:23.797506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a5abe-120b-41c8-8fa6-5313280a363d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:23.797506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a5abe-120b-41c8-8fa6-5313280a363d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:23.797506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a5abe-120b-41c8-8fa6-5313280a363d admin admin] PATH_INFO: `/v3/users/f43e75de1fc64de8b7f47f87ffa1c8fb` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:23.798432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a5abe-120b-41c8-8fa6-5313280a363d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f43e75de1fc64de8b7f47f87ffa1c8fb)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:23.799022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a5abe-120b-41c8-8fa6-5313280a363d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:23.850476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936fc74-9bb2-4c29-9d20-d9d4ea945814 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:23.856221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca33cf9-f5e9-4b2e-85eb-3dde5887ba1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:23.857303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca33cf9-f5e9-4b2e-85eb-3dde5887ba1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:23.857589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca33cf9-f5e9-4b2e-85eb-3dde5887ba1a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:23.857898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca33cf9-f5e9-4b2e-85eb-3dde5887ba1a admin admin] PATH_INFO: `/v3/projects/84b2e3d985bd4e2fa098ffd8d7422fcb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:23.859240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca33cf9-f5e9-4b2e-85eb-3dde5887ba1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=84b2e3d985bd4e2fa098ffd8d7422fcb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:23.861289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca33cf9-f5e9-4b2e-85eb-3dde5887ba1a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:24.071805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b37a5abe-120b-41c8-8fa6-5313280a363d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f43e75de1fc64de8b7f47f87ffa1c8fb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:24.080250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b37a5abe-120b-41c8-8fa6-5313280a363d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f43e75de1fc64de8b7f47f87ffa1c8fb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:24.080876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b37a5abe-120b-41c8-8fa6-5313280a363d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f43e75de1fc64de8b7f47f87ffa1c8fb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:24.081243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b37a5abe-120b-41c8-8fa6-5313280a363d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f43e75de1fc64de8b7f47f87ffa1c8fb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:24.088028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7ca33cf9-f5e9-4b2e-85eb-3dde5887ba1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84b2e3d985bd4e2fa098ffd8d7422fcb', 'request_id': 'req-7ca33cf9-f5e9-4b2e-85eb-3dde5887ba1a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:24.094291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7ca33cf9-f5e9-4b2e-85eb-3dde5887ba1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84b2e3d985bd4e2fa098ffd8d7422fcb', 'request_id': 'req-7ca33cf9-f5e9-4b2e-85eb-3dde5887ba1a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:24.194168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b37a5abe-120b-41c8-8fa6-5313280a363d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f43e75de1fc64de8b7f47f87ffa1c8fb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:24.213905 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2750/5463] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:23 2022] DELETE /identity/v3/users/f43e75de1fc64de8b7f47f87ffa1c8fb => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:24.274413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7ca33cf9-f5e9-4b2e-85eb-3dde5887ba1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84b2e3d985bd4e2fa098ffd8d7422fcb', 'request_id': 'req-7ca33cf9-f5e9-4b2e-85eb-3dde5887ba1a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:24.293454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7ca33cf9-f5e9-4b2e-85eb-3dde5887ba1a admin admin] The token cache is being invalidate because project 84b2e3d985bd4e2fa098ffd8d7422fcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:24.294412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7ca33cf9-f5e9-4b2e-85eb-3dde5887ba1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:24.295658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7ca33cf9-f5e9-4b2e-85eb-3dde5887ba1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:24.298408 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2714/5464] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:23 2022] DELETE /identity/v3/projects/84b2e3d985bd4e2fa098ffd8d7422fcb => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:24.303349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1c4a2-1e08-4e39-ba12-f5ced7a8b8a8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:24.303809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1c4a2-1e08-4e39-ba12-f5ced7a8b8a8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:24.304108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d1c4a2-1e08-4e39-ba12-f5ced7a8b8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:24.329079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c6d1c4a2-1e08-4e39-ba12-f5ced7a8b8a8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:24.330813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c6d1c4a2-1e08-4e39-ba12-f5ced7a8b8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:24.392588 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2751/5465] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:24.400761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae631d-566f-4a1d-9da1-c0b98437a4c3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:24.401061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae631d-566f-4a1d-9da1-c0b98437a4c3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:24.401418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae631d-566f-4a1d-9da1-c0b98437a4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:24.420504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-84ae631d-566f-4a1d-9da1-c0b98437a4c3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:24.421884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-84ae631d-566f-4a1d-9da1-c0b98437a4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:24.430038 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2715/5466] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:24.435411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1c4a2-1e08-4e39-ba12-f5ced7a8b8a8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:24.456548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d1c4a2-1e08-4e39-ba12-f5ced7a8b8a8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:24.456548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797bf60-6a2e-40c1-bfd6-56bbef5118c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:24.456791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797bf60-6a2e-40c1-bfd6-56bbef5118c3 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:24.457031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797bf60-6a2e-40c1-bfd6-56bbef5118c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:24.457309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797bf60-6a2e-40c1-bfd6-56bbef5118c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:24.458097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797bf60-6a2e-40c1-bfd6-56bbef5118c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:20:24.458414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797bf60-6a2e-40c1-bfd6-56bbef5118c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:24.460027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797bf60-6a2e-40c1-bfd6-56bbef5118c3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:24.476412 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2752/5467] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:20:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:24.479480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae631d-566f-4a1d-9da1-c0b98437a4c3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:24.488090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae631d-566f-4a1d-9da1-c0b98437a4c3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:24.493137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c767d-7a95-4139-9a0b-937be6a0fddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:24.493798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c767d-7a95-4139-9a0b-937be6a0fddb admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:24.494163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c767d-7a95-4139-9a0b-937be6a0fddb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:24.494429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c767d-7a95-4139-9a0b-937be6a0fddb admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:24.495260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c767d-7a95-4139-9a0b-937be6a0fddb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:24.497204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c767d-7a95-4139-9a0b-937be6a0fddb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:24.643518 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2716/5468] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:24 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:24.654181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797bf60-6a2e-40c1-bfd6-56bbef5118c3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:24.665276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797bf60-6a2e-40c1-bfd6-56bbef5118c3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:24.732170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307283a-351b-4e89-aac6-66144e1ac648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:24.732954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307283a-351b-4e89-aac6-66144e1ac648 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:24.733245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307283a-351b-4e89-aac6-66144e1ac648 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:24.733551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307283a-351b-4e89-aac6-66144e1ac648 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:24.734303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307283a-351b-4e89-aac6-66144e1ac648 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:24.736278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307283a-351b-4e89-aac6-66144e1ac648 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:24.875316 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2753/5469] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:24 2022] POST /identity/v3/users => generated 286 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:20:24.877562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c767d-7a95-4139-9a0b-937be6a0fddb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:24.879663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-512c767d-7a95-4139-9a0b-937be6a0fddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:24.881900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307283a-351b-4e89-aac6-66144e1ac648 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:24.901300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c767d-7a95-4139-9a0b-937be6a0fddb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:24.906524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5579c9-4306-422a-87d6-d4ab42a1d964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:24.907871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5579c9-4306-422a-87d6-d4ab42a1d964 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:24.908078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307283a-351b-4e89-aac6-66144e1ac648 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:24.908597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5579c9-4306-422a-87d6-d4ab42a1d964 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:24.909096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5579c9-4306-422a-87d6-d4ab42a1d964 admin admin] PATH_INFO: `/v3/projects/71b9d1a53b954a68a961993fdcdb5cc9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:24.910853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5579c9-4306-422a-87d6-d4ab42a1d964 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b9d1a53b954a68a961993fdcdb5cc9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:24.912913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5579c9-4306-422a-87d6-d4ab42a1d964 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:24.927769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12856e-1fec-409f-aa97-866a695c7958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:24.928464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12856e-1fec-409f-aa97-866a695c7958 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:24.928900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12856e-1fec-409f-aa97-866a695c7958 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:24.929321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12856e-1fec-409f-aa97-866a695c7958 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:24.929959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12856e-1fec-409f-aa97-866a695c7958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:24.931777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12856e-1fec-409f-aa97-866a695c7958 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:24.944323 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2754/5470] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:24.948252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12856e-1fec-409f-aa97-866a695c7958 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:24.959587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12856e-1fec-409f-aa97-866a695c7958 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:24.964512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d692002-1d49-499a-873e-860e0fa6dfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:24.965180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d692002-1d49-499a-873e-860e0fa6dfa3 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:24.965462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d692002-1d49-499a-873e-860e0fa6dfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:24.965740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d692002-1d49-499a-873e-860e0fa6dfa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:24.966510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d692002-1d49-499a-873e-860e0fa6dfa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:24.968121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d692002-1d49-499a-873e-860e0fa6dfa3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:24.981657 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2755/5471] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:20:24.985952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d692002-1d49-499a-873e-860e0fa6dfa3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:24.996110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d692002-1d49-499a-873e-860e0fa6dfa3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:25.001603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e7000-c635-4134-90ba-990b311740d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:25.002553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e7000-c635-4134-90ba-990b311740d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:25.002835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e7000-c635-4134-90ba-990b311740d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:25.003198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e7000-c635-4134-90ba-990b311740d2 admin admin] PATH_INFO: `/v3/projects/5ecd1770c98247f596a01300a73d3ae0/users/23cbce472f3147f385b3f952369bf1df/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:25.022039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e7000-c635-4134-90ba-990b311740d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ecd1770c98247f596a01300a73d3ae0, user_id=23cbce472f3147f385b3f952369bf1df, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:25.027607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e7000-c635-4134-90ba-990b311740d2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:25.103685 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2756/5472] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:24 2022] PUT /identity/v3/projects/5ecd1770c98247f596a01300a73d3ae0/users/23cbce472f3147f385b3f952369bf1df/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:25.114092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4ff3e-b57e-4c32-a30d-600ba146f478 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:25.114441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4ff3e-b57e-4c32-a30d-600ba146f478 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:25.114735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4ff3e-b57e-4c32-a30d-600ba146f478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:25.115440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bd5579c9-4306-422a-87d6-d4ab42a1d964 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b9d1a53b954a68a961993fdcdb5cc9', 'request_id': 'req-bd5579c9-4306-422a-87d6-d4ab42a1d964'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:25.121397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bd5579c9-4306-422a-87d6-d4ab42a1d964 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b9d1a53b954a68a961993fdcdb5cc9', 'request_id': 'req-bd5579c9-4306-422a-87d6-d4ab42a1d964'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:25.142405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8ad4ff3e-b57e-4c32-a30d-600ba146f478 None None] MFA Rules not processed for user `23cbce472f3147f385b3f952369bf1df`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:25.143639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8ad4ff3e-b57e-4c32-a30d-600ba146f478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:25.177193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bd5579c9-4306-422a-87d6-d4ab42a1d964 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b9d1a53b954a68a961993fdcdb5cc9', 'request_id': 'req-bd5579c9-4306-422a-87d6-d4ab42a1d964'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:25.189464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:25.192168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bd5579c9-4306-422a-87d6-d4ab42a1d964 admin admin] The token cache is being invalidate because project 71b9d1a53b954a68a961993fdcdb5cc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:25.192438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bd5579c9-4306-422a-87d6-d4ab42a1d964 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:25.193393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bd5579c9-4306-422a-87d6-d4ab42a1d964 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:25.195127 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2717/5473] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:24 2022] DELETE /identity/v3/projects/71b9d1a53b954a68a961993fdcdb5cc9 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:25.199887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7cd64-253e-4145-bf81-34f8324e62cc None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:25.200214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7cd64-253e-4145-bf81-34f8324e62cc None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:25.200461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7cd64-253e-4145-bf81-34f8324e62cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:25.216081 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2757/5474] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:25.221315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-76d7cd64-253e-4145-bf81-34f8324e62cc None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:25.222584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-76d7cd64-253e-4145-bf81-34f8324e62cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:25.229378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4ff3e-b57e-4c32-a30d-600ba146f478 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:25.231163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8ad4ff3e-b57e-4c32-a30d-600ba146f478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:25.305213 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2718/5475] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:25.308144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4ff3e-b57e-4c32-a30d-600ba146f478 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:25.312738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9dda1-1180-410d-8de2-42a49053c298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:25.313409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9dda1-1180-410d-8de2-42a49053c298 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:25.314344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9dda1-1180-410d-8de2-42a49053c298 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:25.314344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d9dda1-1180-410d-8de2-42a49053c298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:25.314605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e4398-f9e2-4307-a670-e74d9fe91ae9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:25.314605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e4398-f9e2-4307-a670-e74d9fe91ae9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:25.314830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727e4398-f9e2-4307-a670-e74d9fe91ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:25.325349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d9dda1-1180-410d-8de2-42a49053c298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:25.326970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d9dda1-1180-410d-8de2-42a49053c298 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:25.336698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-727e4398-f9e2-4307-a670-e74d9fe91ae9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:25.338056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-727e4398-f9e2-4307-a670-e74d9fe91ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:25.342362 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2758/5476] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:20:25.347790 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2719/5477] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:20:25.354222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9dda1-1180-410d-8de2-42a49053c298 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:25.367020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d9dda1-1180-410d-8de2-42a49053c298 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:25.372662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ab0f7-781c-403b-8e0c-8ca8e563eed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:25.378511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ab0f7-781c-403b-8e0c-8ca8e563eed8 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:25.378511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ab0f7-781c-403b-8e0c-8ca8e563eed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:25.378511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ab0f7-781c-403b-8e0c-8ca8e563eed8 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:25.378511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ab0f7-781c-403b-8e0c-8ca8e563eed8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:20:25.378511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ab0f7-781c-403b-8e0c-8ca8e563eed8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:25.378511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ab0f7-781c-403b-8e0c-8ca8e563eed8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:25.392967 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2759/5478] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:25.394549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e4398-f9e2-4307-a670-e74d9fe91ae9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:25.405304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727e4398-f9e2-4307-a670-e74d9fe91ae9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:25.410713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e4547-6d2c-4043-95b9-26aed3395e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:25.411377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e4547-6d2c-4043-95b9-26aed3395e84 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:25.411694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e4547-6d2c-4043-95b9-26aed3395e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:25.411990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e4547-6d2c-4043-95b9-26aed3395e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:25.412686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e4547-6d2c-4043-95b9-26aed3395e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:25.414703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e4547-6d2c-4043-95b9-26aed3395e84 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:25.515180 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2720/5479] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:25 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:20:25.519343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ab0f7-781c-403b-8e0c-8ca8e563eed8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:25.529564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ab0f7-781c-403b-8e0c-8ca8e563eed8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:25.536805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:25.595247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd01d2-36c2-4963-8cda-b79de6781116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:25.596032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd01d2-36c2-4963-8cda-b79de6781116 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:25.596432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd01d2-36c2-4963-8cda-b79de6781116 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:25.596878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd01d2-36c2-4963-8cda-b79de6781116 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:25.597684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd01d2-36c2-4963-8cda-b79de6781116 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:25.599824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd01d2-36c2-4963-8cda-b79de6781116 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:25.721357 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2760/5480] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:25 2022] POST /identity/v3/users => generated 299 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:25.725534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e4547-6d2c-4043-95b9-26aed3395e84 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:25.737291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e4547-6d2c-4043-95b9-26aed3395e84 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:25.742953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0dbea3-6fb8-488d-819f-7f2a367c6804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:25.743636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0dbea3-6fb8-488d-819f-7f2a367c6804 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:25.743927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0dbea3-6fb8-488d-819f-7f2a367c6804 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:25.744216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0dbea3-6fb8-488d-819f-7f2a367c6804 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:25.744959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0dbea3-6fb8-488d-819f-7f2a367c6804 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:25.746681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0dbea3-6fb8-488d-819f-7f2a367c6804 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:25.758565 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2721/5481] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:20:25.762978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd01d2-36c2-4963-8cda-b79de6781116 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:25.773583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd01d2-36c2-4963-8cda-b79de6781116 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:25.778330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0d07c-2fcc-45ff-bb3e-987db3342722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:25.779207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0d07c-2fcc-45ff-bb3e-987db3342722 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:25.779494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0d07c-2fcc-45ff-bb3e-987db3342722 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:25.779778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb0d07c-2fcc-45ff-bb3e-987db3342722 admin admin] PATH_INFO: `/v3/projects/bf6b9b15150e475d8eeb2890759db5fd/users/ade5792307084494ae28775266b1ed47/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:25.795612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0d07c-2fcc-45ff-bb3e-987db3342722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf6b9b15150e475d8eeb2890759db5fd, user_id=ade5792307084494ae28775266b1ed47, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:25.798351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb0d07c-2fcc-45ff-bb3e-987db3342722 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:25.890772 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2761/5482] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:25 2022] PUT /identity/v3/projects/bf6b9b15150e475d8eeb2890759db5fd/users/ade5792307084494ae28775266b1ed47/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:25.894488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0dbea3-6fb8-488d-819f-7f2a367c6804 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:25.915348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0dbea3-6fb8-488d-819f-7f2a367c6804 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:25.990566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4e5613-7f88-4cbf-bdd3-3235d78a6b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:25.991351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e5613-7f88-4cbf-bdd3-3235d78a6b55 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:25.991733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e5613-7f88-4cbf-bdd3-3235d78a6b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:25.992089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4e5613-7f88-4cbf-bdd3-3235d78a6b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:25.992905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e5613-7f88-4cbf-bdd3-3235d78a6b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:25.994667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4e5613-7f88-4cbf-bdd3-3235d78a6b55 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:26.006310 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2722/5483] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:26.010643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0d07c-2fcc-45ff-bb3e-987db3342722 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:26.023132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb0d07c-2fcc-45ff-bb3e-987db3342722 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:26.034802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70efcac-b2d8-4fa3-800a-15a76ad18822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:26.036090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70efcac-b2d8-4fa3-800a-15a76ad18822 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:26.037372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70efcac-b2d8-4fa3-800a-15a76ad18822 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:26.037700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70efcac-b2d8-4fa3-800a-15a76ad18822 admin admin] PATH_INFO: `/v3/projects/bf6b9b15150e475d8eeb2890759db5fd/users/ade5792307084494ae28775266b1ed47/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:26.040006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70efcac-b2d8-4fa3-800a-15a76ad18822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf6b9b15150e475d8eeb2890759db5fd, user_id=ade5792307084494ae28775266b1ed47, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:26.042761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70efcac-b2d8-4fa3-800a-15a76ad18822 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:26.056912 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2762/5484] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:26 2022] PUT /identity/v3/projects/bf6b9b15150e475d8eeb2890759db5fd/users/ade5792307084494ae28775266b1ed47/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:26.067234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59baa9-9037-476f-bb2e-35063358f5c1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:26.067539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59baa9-9037-476f-bb2e-35063358f5c1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:26.067826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c59baa9-9037-476f-bb2e-35063358f5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:26.103597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0c59baa9-9037-476f-bb2e-35063358f5c1 None None] MFA Rules not processed for user `ade5792307084494ae28775266b1ed47`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:26.103597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0c59baa9-9037-476f-bb2e-35063358f5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:26.165880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:26.178838 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2723/5485] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:26.183314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70efcac-b2d8-4fa3-800a-15a76ad18822 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:26.250024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70efcac-b2d8-4fa3-800a-15a76ad18822 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:26.255468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627074f-8939-42d3-b8be-1c299b0dabcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:26.260396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627074f-8939-42d3-b8be-1c299b0dabcd admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:26.260396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627074f-8939-42d3-b8be-1c299b0dabcd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:26.260396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2627074f-8939-42d3-b8be-1c299b0dabcd admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:26.260396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627074f-8939-42d3-b8be-1c299b0dabcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:26.260396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2627074f-8939-42d3-b8be-1c299b0dabcd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:26.396605 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2763/5486] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:26 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:26.400781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59baa9-9037-476f-bb2e-35063358f5c1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:26.412173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c59baa9-9037-476f-bb2e-35063358f5c1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:26.480725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7e568-e5ce-4626-a046-aa64c6cdeb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:26.481554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7e568-e5ce-4626-a046-aa64c6cdeb69 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:26.481874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7e568-e5ce-4626-a046-aa64c6cdeb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:26.482148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7e568-e5ce-4626-a046-aa64c6cdeb69 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:26.482915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7e568-e5ce-4626-a046-aa64c6cdeb69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:26.484984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7e568-e5ce-4626-a046-aa64c6cdeb69 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:26.595442 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2724/5487] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:26 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:26.599105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627074f-8939-42d3-b8be-1c299b0dabcd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:26.610181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2627074f-8939-42d3-b8be-1c299b0dabcd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:26.616371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13f8c0-5442-4d20-80a3-bbf962f5e877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:26.623183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13f8c0-5442-4d20-80a3-bbf962f5e877 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:26.623183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13f8c0-5442-4d20-80a3-bbf962f5e877 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:26.623183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13f8c0-5442-4d20-80a3-bbf962f5e877 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:26.623183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13f8c0-5442-4d20-80a3-bbf962f5e877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:26.623183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13f8c0-5442-4d20-80a3-bbf962f5e877 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:26.631116 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2764/5488] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:26.635006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7e568-e5ce-4626-a046-aa64c6cdeb69 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:26.646177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7e568-e5ce-4626-a046-aa64c6cdeb69 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:26.651887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93d191-e331-49d1-b566-a3e0b63e8cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:26.651887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93d191-e331-49d1-b566-a3e0b63e8cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:26.651887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93d191-e331-49d1-b566-a3e0b63e8cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:26.652242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93d191-e331-49d1-b566-a3e0b63e8cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:26.652878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93d191-e331-49d1-b566-a3e0b63e8cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:26.654583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93d191-e331-49d1-b566-a3e0b63e8cd0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:26.672138 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2725/5489] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:26.680603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13f8c0-5442-4d20-80a3-bbf962f5e877 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:26.694614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13f8c0-5442-4d20-80a3-bbf962f5e877 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:26.701932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b3d87-1adb-46aa-8f90-54a897298e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:26.703892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b3d87-1adb-46aa-8f90-54a897298e55 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:26.703892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b3d87-1adb-46aa-8f90-54a897298e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:26.709562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b3d87-1adb-46aa-8f90-54a897298e55 admin admin] PATH_INFO: `/v3/projects/2dc479c9bb994e7e83a9672d1445d938/users/66ad6e26465f4be1a38a67d611acd159/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:26.720339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b3d87-1adb-46aa-8f90-54a897298e55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dc479c9bb994e7e83a9672d1445d938, user_id=66ad6e26465f4be1a38a67d611acd159, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:26.725249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b3d87-1adb-46aa-8f90-54a897298e55 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:26.858911 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2765/5490] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:26 2022] PUT /identity/v3/projects/2dc479c9bb994e7e83a9672d1445d938/users/66ad6e26465f4be1a38a67d611acd159/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:26.869467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30272775-c60a-4b80-b328-cf3dbcf5b95d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:26.869894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30272775-c60a-4b80-b328-cf3dbcf5b95d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:26.870477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30272775-c60a-4b80-b328-cf3dbcf5b95d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:26.901085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-30272775-c60a-4b80-b328-cf3dbcf5b95d None None] MFA Rules not processed for user `66ad6e26465f4be1a38a67d611acd159`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:26.902523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-30272775-c60a-4b80-b328-cf3dbcf5b95d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:26.977364 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2726/5491] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:20:26.986315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b3d87-1adb-46aa-8f90-54a897298e55 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:26.989967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-084b3d87-1adb-46aa-8f90-54a897298e55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:27.073640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b3d87-1adb-46aa-8f90-54a897298e55 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:27.078687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171d563-15e1-4d46-a9b9-8ad18de71223 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:27.079338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171d563-15e1-4d46-a9b9-8ad18de71223 service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:27.083788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171d563-15e1-4d46-a9b9-8ad18de71223 service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:27.083788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171d563-15e1-4d46-a9b9-8ad18de71223 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:27.143920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171d563-15e1-4d46-a9b9-8ad18de71223 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:27.145391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171d563-15e1-4d46-a9b9-8ad18de71223 service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:27.164105 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2766/5492] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:27.377373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:28.406662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:28.735294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30272775-c60a-4b80-b328-cf3dbcf5b95d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:28.802577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30272775-c60a-4b80-b328-cf3dbcf5b95d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:28.807911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb39e71-0a40-4064-848e-5444c5a96d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:28.809579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb39e71-0a40-4064-848e-5444c5a96d0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:28.809579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb39e71-0a40-4064-848e-5444c5a96d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:28.809579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb39e71-0a40-4064-848e-5444c5a96d0f admin admin] PATH_INFO: `/v3/users/ade5792307084494ae28775266b1ed47` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:28.810765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb39e71-0a40-4064-848e-5444c5a96d0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ade5792307084494ae28775266b1ed47)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:28.820002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb39e71-0a40-4064-848e-5444c5a96d0f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:29.166130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fcb39e71-0a40-4064-848e-5444c5a96d0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ade5792307084494ae28775266b1ed47'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:29.166832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fcb39e71-0a40-4064-848e-5444c5a96d0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ade5792307084494ae28775266b1ed47'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:29.175902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fcb39e71-0a40-4064-848e-5444c5a96d0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ade5792307084494ae28775266b1ed47'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:29.262323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fcb39e71-0a40-4064-848e-5444c5a96d0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ade5792307084494ae28775266b1ed47'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:29.264812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fcb39e71-0a40-4064-848e-5444c5a96d0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ade5792307084494ae28775266b1ed47'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:29.295161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:29.301409 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2727/5493] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:28 2022] DELETE /identity/v3/users/ade5792307084494ae28775266b1ed47 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:29.306477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d8031-7f53-4a90-b053-f7352fee9995 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:29.309603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d8031-7f53-4a90-b053-f7352fee9995 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:29.309603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846d8031-7f53-4a90-b053-f7352fee9995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:29.331095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-846d8031-7f53-4a90-b053-f7352fee9995 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:29.332671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-846d8031-7f53-4a90-b053-f7352fee9995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:29.418290 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2767/5494] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:29.428164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb39e71-0a40-4064-848e-5444c5a96d0f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:29.429626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fcb39e71-0a40-4064-848e-5444c5a96d0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:29.497682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb39e71-0a40-4064-848e-5444c5a96d0f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:29.501563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fcc8f-5cdd-4c4b-8d25-3e2a6ee3b1d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:29.502307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fcc8f-5cdd-4c4b-8d25-3e2a6ee3b1d7 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:29.502652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fcc8f-5cdd-4c4b-8d25-3e2a6ee3b1d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:29.503292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fcc8f-5cdd-4c4b-8d25-3e2a6ee3b1d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:29.512931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fcc8f-5cdd-4c4b-8d25-3e2a6ee3b1d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:29.514528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fcc8f-5cdd-4c4b-8d25-3e2a6ee3b1d7 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:29.524779 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2728/5495] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:29.797060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:30.269744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d8031-7f53-4a90-b053-f7352fee9995 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:30.271189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-846d8031-7f53-4a90-b053-f7352fee9995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:30.283142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846d8031-7f53-4a90-b053-f7352fee9995 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:30.289539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c85b16-b79f-446f-b13e-977a5f5b654a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:30.289539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c85b16-b79f-446f-b13e-977a5f5b654a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:30.289539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c85b16-b79f-446f-b13e-977a5f5b654a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:30.289539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c85b16-b79f-446f-b13e-977a5f5b654a admin admin] PATH_INFO: `/v3/projects/bf6b9b15150e475d8eeb2890759db5fd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:30.290843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c85b16-b79f-446f-b13e-977a5f5b654a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf6b9b15150e475d8eeb2890759db5fd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:30.292778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c85b16-b79f-446f-b13e-977a5f5b654a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:30.423209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2c85b16-b79f-446f-b13e-977a5f5b654a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf6b9b15150e475d8eeb2890759db5fd', 'request_id': 'req-e2c85b16-b79f-446f-b13e-977a5f5b654a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:30.430793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2c85b16-b79f-446f-b13e-977a5f5b654a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf6b9b15150e475d8eeb2890759db5fd', 'request_id': 'req-e2c85b16-b79f-446f-b13e-977a5f5b654a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:30.512437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2c85b16-b79f-446f-b13e-977a5f5b654a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf6b9b15150e475d8eeb2890759db5fd', 'request_id': 'req-e2c85b16-b79f-446f-b13e-977a5f5b654a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:30.524915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2c85b16-b79f-446f-b13e-977a5f5b654a admin admin] The token cache is being invalidate because project bf6b9b15150e475d8eeb2890759db5fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:30.525210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2c85b16-b79f-446f-b13e-977a5f5b654a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:30.525887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2c85b16-b79f-446f-b13e-977a5f5b654a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:30.527450 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2768/5496] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:30 2022] DELETE /identity/v3/projects/bf6b9b15150e475d8eeb2890759db5fd => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:30.532112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fcc8f-5cdd-4c4b-8d25-3e2a6ee3b1d7 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:30.542480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fcc8f-5cdd-4c4b-8d25-3e2a6ee3b1d7 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:30.607997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f716cc-a2e8-498a-9370-24bfbd30cb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:30.609620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f716cc-a2e8-498a-9370-24bfbd30cb46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:30.610182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f716cc-a2e8-498a-9370-24bfbd30cb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:30.610468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f716cc-a2e8-498a-9370-24bfbd30cb46 admin admin] PATH_INFO: `/v3/users/66ad6e26465f4be1a38a67d611acd159` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:30.611705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f716cc-a2e8-498a-9370-24bfbd30cb46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66ad6e26465f4be1a38a67d611acd159)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:30.613686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f716cc-a2e8-498a-9370-24bfbd30cb46 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:30.784731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94f716cc-a2e8-498a-9370-24bfbd30cb46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66ad6e26465f4be1a38a67d611acd159'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:30.787010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94f716cc-a2e8-498a-9370-24bfbd30cb46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66ad6e26465f4be1a38a67d611acd159'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:30.792773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94f716cc-a2e8-498a-9370-24bfbd30cb46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66ad6e26465f4be1a38a67d611acd159'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:30.874899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94f716cc-a2e8-498a-9370-24bfbd30cb46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66ad6e26465f4be1a38a67d611acd159'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:30.876148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94f716cc-a2e8-498a-9370-24bfbd30cb46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66ad6e26465f4be1a38a67d611acd159'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:30.876281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:30.898010 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2729/5497] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:30 2022] DELETE /identity/v3/users/66ad6e26465f4be1a38a67d611acd159 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:31.080040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:31.549044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c85b16-b79f-446f-b13e-977a5f5b654a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:31.552871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e2c85b16-b79f-446f-b13e-977a5f5b654a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:31.573279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c85b16-b79f-446f-b13e-977a5f5b654a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:31.642074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1673ac7-b1dd-41b3-a516-e45e05e26919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:31.643056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1673ac7-b1dd-41b3-a516-e45e05e26919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:31.643655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1673ac7-b1dd-41b3-a516-e45e05e26919 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:31.644243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1673ac7-b1dd-41b3-a516-e45e05e26919 admin admin] PATH_INFO: `/v3/projects/2dc479c9bb994e7e83a9672d1445d938` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:31.645509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1673ac7-b1dd-41b3-a516-e45e05e26919 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dc479c9bb994e7e83a9672d1445d938)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:31.647492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1673ac7-b1dd-41b3-a516-e45e05e26919 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:31.803963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e1673ac7-b1dd-41b3-a516-e45e05e26919 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dc479c9bb994e7e83a9672d1445d938', 'request_id': 'req-e1673ac7-b1dd-41b3-a516-e45e05e26919'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:31.811860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e1673ac7-b1dd-41b3-a516-e45e05e26919 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dc479c9bb994e7e83a9672d1445d938', 'request_id': 'req-e1673ac7-b1dd-41b3-a516-e45e05e26919'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:31.908001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e1673ac7-b1dd-41b3-a516-e45e05e26919 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dc479c9bb994e7e83a9672d1445d938', 'request_id': 'req-e1673ac7-b1dd-41b3-a516-e45e05e26919'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:31.921200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e1673ac7-b1dd-41b3-a516-e45e05e26919 admin admin] The token cache is being invalidate because project 2dc479c9bb994e7e83a9672d1445d938 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:31.921506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e1673ac7-b1dd-41b3-a516-e45e05e26919 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:31.922202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e1673ac7-b1dd-41b3-a516-e45e05e26919 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:31.923784 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2769/5498] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:31 2022] DELETE /identity/v3/projects/2dc479c9bb994e7e83a9672d1445d938 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:31.928861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeac85a-1389-46ac-89a4-d564435b9d0b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:31.929360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeac85a-1389-46ac-89a4-d564435b9d0b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:31.929494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeac85a-1389-46ac-89a4-d564435b9d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:31.951208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ddeac85a-1389-46ac-89a4-d564435b9d0b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:31.953189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ddeac85a-1389-46ac-89a4-d564435b9d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:32.017229 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2730/5499] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:32.025036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67c7b5-d8d9-4594-92cb-f4df3e21574e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:32.025337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67c7b5-d8d9-4594-92cb-f4df3e21574e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:32.025627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67c7b5-d8d9-4594-92cb-f4df3e21574e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:32.044375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8e67c7b5-d8d9-4594-92cb-f4df3e21574e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:32.045601 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8e67c7b5-d8d9-4594-92cb-f4df3e21574e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:32.053962 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2770/5500] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:32.058283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeac85a-1389-46ac-89a4-d564435b9d0b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:32.069160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeac85a-1389-46ac-89a4-d564435b9d0b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:32.075074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042581e-76f6-4180-bc64-3a711be65664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:32.075984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042581e-76f6-4180-bc64-3a711be65664 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:32.076277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042581e-76f6-4180-bc64-3a711be65664 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:32.076554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a042581e-76f6-4180-bc64-3a711be65664 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:32.077305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042581e-76f6-4180-bc64-3a711be65664 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:20:32.078205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042581e-76f6-4180-bc64-3a711be65664 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:32.079836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a042581e-76f6-4180-bc64-3a711be65664 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:32.095486 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2731/5501] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:32.096802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67c7b5-d8d9-4594-92cb-f4df3e21574e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:32.106714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67c7b5-d8d9-4594-92cb-f4df3e21574e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:32.111561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d02c5-69a8-43fa-bf52-2ece2676426d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:32.112303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d02c5-69a8-43fa-bf52-2ece2676426d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:32.112596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d02c5-69a8-43fa-bf52-2ece2676426d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:32.112881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d02c5-69a8-43fa-bf52-2ece2676426d admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:32.113596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d02c5-69a8-43fa-bf52-2ece2676426d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:32.115580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d02c5-69a8-43fa-bf52-2ece2676426d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:32.215270 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2771/5502] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:32 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:32.218130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042581e-76f6-4180-bc64-3a711be65664 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:32.230316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a042581e-76f6-4180-bc64-3a711be65664 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:32.370812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:32.376040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480a924-cc20-4e59-824d-a9f687945055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:32.376040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480a924-cc20-4e59-824d-a9f687945055 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:32.376040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480a924-cc20-4e59-824d-a9f687945055 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:32.376040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480a924-cc20-4e59-824d-a9f687945055 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:32.376508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480a924-cc20-4e59-824d-a9f687945055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:32.377065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480a924-cc20-4e59-824d-a9f687945055 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:32.486975 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2732/5503] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:32 2022] POST /identity/v3/users => generated 294 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:32.491007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d02c5-69a8-43fa-bf52-2ece2676426d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:32.511412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d02c5-69a8-43fa-bf52-2ece2676426d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:32.520309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b751771-bf50-4169-8247-b4bad91c6a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:32.522106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b751771-bf50-4169-8247-b4bad91c6a3b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:32.523048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b751771-bf50-4169-8247-b4bad91c6a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:32.524174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b751771-bf50-4169-8247-b4bad91c6a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:32.526213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b751771-bf50-4169-8247-b4bad91c6a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:32.528651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b751771-bf50-4169-8247-b4bad91c6a3b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:32.541794 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2772/5504] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:32.545900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480a924-cc20-4e59-824d-a9f687945055 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:32.557262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480a924-cc20-4e59-824d-a9f687945055 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:32.562810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b595d-37b0-4cea-b682-5359c1b14288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:32.563507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b595d-37b0-4cea-b682-5359c1b14288 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:32.563791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b595d-37b0-4cea-b682-5359c1b14288 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:32.564073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b595d-37b0-4cea-b682-5359c1b14288 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:32.565714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b595d-37b0-4cea-b682-5359c1b14288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:32.567571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b595d-37b0-4cea-b682-5359c1b14288 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:32.579010 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2733/5505] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:32.583282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b751771-bf50-4169-8247-b4bad91c6a3b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:32.595502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b751771-bf50-4169-8247-b4bad91c6a3b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:32.600584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd214001-7520-4203-acde-2767e7f47084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:32.601664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd214001-7520-4203-acde-2767e7f47084 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:32.602130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd214001-7520-4203-acde-2767e7f47084 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:32.602475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd214001-7520-4203-acde-2767e7f47084 admin admin] PATH_INFO: `/v3/projects/2573d11ff7c84af7bfdca0777c61b158/users/73c61f350d444706adada72a33056647/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:32.620063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd214001-7520-4203-acde-2767e7f47084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2573d11ff7c84af7bfdca0777c61b158, user_id=73c61f350d444706adada72a33056647, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:32.622904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd214001-7520-4203-acde-2767e7f47084 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:32.708210 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2773/5506] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:32 2022] PUT /identity/v3/projects/2573d11ff7c84af7bfdca0777c61b158/users/73c61f350d444706adada72a33056647/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:32.719256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca1dfc-1df1-49ad-af6c-30c6635f7cdd None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:32.719546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca1dfc-1df1-49ad-af6c-30c6635f7cdd None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:32.719870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca1dfc-1df1-49ad-af6c-30c6635f7cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:32.748978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:32.751663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-beca1dfc-1df1-49ad-af6c-30c6635f7cdd None None] MFA Rules not processed for user `73c61f350d444706adada72a33056647`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:32.752917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-beca1dfc-1df1-49ad-af6c-30c6635f7cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:32.823008 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2734/5507] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:20:32.838326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd214001-7520-4203-acde-2767e7f47084 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:32.840088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fd214001-7520-4203-acde-2767e7f47084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:32.902430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd214001-7520-4203-acde-2767e7f47084 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:32.906607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa3c62-6a0e-48dc-b723-e3c782970b2d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:32.907264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa3c62-6a0e-48dc-b723-e3c782970b2d service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:32.907554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa3c62-6a0e-48dc-b723-e3c782970b2d service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:32.907836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa3c62-6a0e-48dc-b723-e3c782970b2d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:32.916507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa3c62-6a0e-48dc-b723-e3c782970b2d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:32.918054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa3c62-6a0e-48dc-b723-e3c782970b2d service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:32.928257 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2774/5508] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:33.214917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635377e-514b-446f-b8b0-c2cf49cde367 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:33.215418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635377e-514b-446f-b8b0-c2cf49cde367 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:33.215536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3635377e-514b-446f-b8b0-c2cf49cde367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:33.236350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3635377e-514b-446f-b8b0-c2cf49cde367 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:33.237583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3635377e-514b-446f-b8b0-c2cf49cde367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:33.323341 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2735/5509] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:20:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:33.471238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa3c62-6a0e-48dc-b723-e3c782970b2d service glance] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:33.472824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d4aa3c62-6a0e-48dc-b723-e3c782970b2d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:33.837695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa3c62-6a0e-48dc-b723-e3c782970b2d service glance] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:33.837695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c4a8c-f2d9-4679-b67b-64b7b95ee49e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:33.837695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c4a8c-f2d9-4679-b67b-64b7b95ee49e service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:33.837695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c4a8c-f2d9-4679-b67b-64b7b95ee49e service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:33.840163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c4a8c-f2d9-4679-b67b-64b7b95ee49e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:33.840163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c4a8c-f2d9-4679-b67b-64b7b95ee49e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:33.840163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c4a8c-f2d9-4679-b67b-64b7b95ee49e service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:33.840163 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2775/5510] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:20:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 21:20:34.634226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3355bdd-9d86-4143-bb9f-3acaee1b8a21 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:34.636018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3355bdd-9d86-4143-bb9f-3acaee1b8a21 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:34.636453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3355bdd-9d86-4143-bb9f-3acaee1b8a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:34.639342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:34.661796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f3355bdd-9d86-4143-bb9f-3acaee1b8a21 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:34.663065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f3355bdd-9d86-4143-bb9f-3acaee1b8a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:34.672117 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2736/5511] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:20:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:34.684150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c4a8c-f2d9-4679-b67b-64b7b95ee49e service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:34.698000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c4a8c-f2d9-4679-b67b-64b7b95ee49e service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:34.704468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b642e1-5135-4dc0-a1a3-38a93155e4cd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:34.705171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b642e1-5135-4dc0-a1a3-38a93155e4cd service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:34.705462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b642e1-5135-4dc0-a1a3-38a93155e4cd service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:34.705787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b642e1-5135-4dc0-a1a3-38a93155e4cd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:34.719760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b642e1-5135-4dc0-a1a3-38a93155e4cd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:34.721496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b642e1-5135-4dc0-a1a3-38a93155e4cd service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:34.732260 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2776/5512] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:20:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:20:35.434790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:35.724901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:35.765433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d391e-751a-407f-b44d-db36f9480e0d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:35.765925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d391e-751a-407f-b44d-db36f9480e0d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:35.766055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d391e-751a-407f-b44d-db36f9480e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:35.783680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3b4d391e-751a-407f-b44d-db36f9480e0d None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:35.784898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3b4d391e-751a-407f-b44d-db36f9480e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:35.792184 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2737/5513] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:20:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:20:35.804715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b642e1-5135-4dc0-a1a3-38a93155e4cd service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:35.818616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b642e1-5135-4dc0-a1a3-38a93155e4cd service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:35.823939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bb3dd-f279-40f5-ac72-04023b299324 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:35.824650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bb3dd-f279-40f5-ac72-04023b299324 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:35.824965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bb3dd-f279-40f5-ac72-04023b299324 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:35.825246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bb3dd-f279-40f5-ac72-04023b299324 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:35.834565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bb3dd-f279-40f5-ac72-04023b299324 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:35.836164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bb3dd-f279-40f5-ac72-04023b299324 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:35.844644 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2777/5514] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:20:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:20:36.441551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000a174-00da-4827-b437-f491d569acb4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:36.442086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000a174-00da-4827-b437-f491d569acb4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:36.442542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000a174-00da-4827-b437-f491d569acb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:36.465050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e000a174-00da-4827-b437-f491d569acb4 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:36.466691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e000a174-00da-4827-b437-f491d569acb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:36.490173 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2738/5515] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:20:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:36.504110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bb3dd-f279-40f5-ac72-04023b299324 service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:36.533888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bb3dd-f279-40f5-ac72-04023b299324 service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:36.543961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f88ae1-d44b-4d77-bb81-af8c78587aa2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:36.544944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f88ae1-d44b-4d77-bb81-af8c78587aa2 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:36.545289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f88ae1-d44b-4d77-bb81-af8c78587aa2 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:36.545580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f88ae1-d44b-4d77-bb81-af8c78587aa2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:36.557438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f88ae1-d44b-4d77-bb81-af8c78587aa2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:36.559295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f88ae1-d44b-4d77-bb81-af8c78587aa2 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:36.569310 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2778/5516] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:20:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:20:36.951047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:37.251019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aece5a7-777e-4659-9f05-7acee467e541 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:37.251134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aece5a7-777e-4659-9f05-7acee467e541 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:37.251646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aece5a7-777e-4659-9f05-7acee467e541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:37.295627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d366e-cdee-4056-aefc-cbe4b43ffa30 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:37.295627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d366e-cdee-4056-aefc-cbe4b43ffa30 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:37.295627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d366e-cdee-4056-aefc-cbe4b43ffa30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:37.317503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:37.320881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7aece5a7-777e-4659-9f05-7acee467e541 None None] MFA Rules not processed for user `2fb3ef26871e4b30a9bc4a5045fe62ad`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:37.322429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7aece5a7-777e-4659-9f05-7acee467e541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:37.331872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-349d366e-cdee-4056-aefc-cbe4b43ffa30 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:37.333679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-349d366e-cdee-4056-aefc-cbe4b43ffa30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:37.343653 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2779/5517] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:20:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:37.362211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d366e-cdee-4056-aefc-cbe4b43ffa30 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:37.384202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d366e-cdee-4056-aefc-cbe4b43ffa30 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:37.399129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02af7d8a-211c-4fec-9065-fa42fd8904a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:37.399810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af7d8a-211c-4fec-9065-fa42fd8904a2 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:37.400098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af7d8a-211c-4fec-9065-fa42fd8904a2 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:37.400383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02af7d8a-211c-4fec-9065-fa42fd8904a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:37.411760 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2739/5518] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:37.447447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af7d8a-211c-4fec-9065-fa42fd8904a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:37.465363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02af7d8a-211c-4fec-9065-fa42fd8904a2 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:37.490533 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2780/5519] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:20:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:20:37.536303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268183e-ea88-46ee-a9d2-351f8bd028bd None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:37.536686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268183e-ea88-46ee-a9d2-351f8bd028bd None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:37.537141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268183e-ea88-46ee-a9d2-351f8bd028bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:37.560512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4268183e-ea88-46ee-a9d2-351f8bd028bd None None] MFA Rules not processed for user `2fb3ef26871e4b30a9bc4a5045fe62ad`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:37.561934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4268183e-ea88-46ee-a9d2-351f8bd028bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:37.573114 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2740/5520] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:37.606762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0396d1b-4dc7-44d2-94a2-541657f11802 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:37.608510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0396d1b-4dc7-44d2-94a2-541657f11802 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:37.609004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0396d1b-4dc7-44d2-94a2-541657f11802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:37.644956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e0396d1b-4dc7-44d2-94a2-541657f11802 None None] MFA Rules not processed for user `2fb3ef26871e4b30a9bc4a5045fe62ad`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:37.647920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e0396d1b-4dc7-44d2-94a2-541657f11802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:37.677695 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2781/5521] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:37.715200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5021ce-a8de-4035-8dfe-90efa0e752a1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:37.716563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5021ce-a8de-4035-8dfe-90efa0e752a1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:37.718140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5021ce-a8de-4035-8dfe-90efa0e752a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:37.764886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5e5021ce-a8de-4035-8dfe-90efa0e752a1 None None] MFA Rules not processed for user `2fb3ef26871e4b30a9bc4a5045fe62ad`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:37.767277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5e5021ce-a8de-4035-8dfe-90efa0e752a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:37.778526 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2741/5522] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:37.803023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cfecb-361f-4ce5-a845-0f60038f937a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:37.803778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cfecb-361f-4ce5-a845-0f60038f937a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:37.804232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52cfecb-361f-4ce5-a845-0f60038f937a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:37.828037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b52cfecb-361f-4ce5-a845-0f60038f937a None None] MFA Rules not processed for user `2fb3ef26871e4b30a9bc4a5045fe62ad`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:37.830368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b52cfecb-361f-4ce5-a845-0f60038f937a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:37.843701 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2782/5523] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:38.245114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406f936-2e22-461d-ad8a-99523bfe0485 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:38.245646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406f936-2e22-461d-ad8a-99523bfe0485 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:38.245949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d406f936-2e22-461d-ad8a-99523bfe0485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:38.279098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d406f936-2e22-461d-ad8a-99523bfe0485 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:38.280466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d406f936-2e22-461d-ad8a-99523bfe0485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:38.297267 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2742/5524] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:20:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:38.305011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cfecb-361f-4ce5-a845-0f60038f937a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:38.324117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52cfecb-361f-4ce5-a845-0f60038f937a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:38.332114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690d686-662f-43cd-8614-8834d5ee8224 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:38.333087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690d686-662f-43cd-8614-8834d5ee8224 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:38.334191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690d686-662f-43cd-8614-8834d5ee8224 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:38.334613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690d686-662f-43cd-8614-8834d5ee8224 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:38.344248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690d686-662f-43cd-8614-8834d5ee8224 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:38.346858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690d686-662f-43cd-8614-8834d5ee8224 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:38.363610 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2783/5525] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:20:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:20:38.895746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406f936-2e22-461d-ad8a-99523bfe0485 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:38.902187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:UPDATE=1,keystone:DELETE=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:38.963863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d406f936-2e22-461d-ad8a-99523bfe0485 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:38.970827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9fc44-bdcd-49f8-a654-7f8d458a8fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:38.971638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9fc44-bdcd-49f8-a654-7f8d458a8fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:38.971889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9fc44-bdcd-49f8-a654-7f8d458a8fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:38.972169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9fc44-bdcd-49f8-a654-7f8d458a8fb3 admin admin] PATH_INFO: `/v3/users/73c61f350d444706adada72a33056647` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:38.973455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9fc44-bdcd-49f8-a654-7f8d458a8fb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c61f350d444706adada72a33056647)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:38.975495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9fc44-bdcd-49f8-a654-7f8d458a8fb3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:39.226678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d1b9fc44-bdcd-49f8-a654-7f8d458a8fb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c61f350d444706adada72a33056647'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:39.229694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d1b9fc44-bdcd-49f8-a654-7f8d458a8fb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c61f350d444706adada72a33056647'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:39.244915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d1b9fc44-bdcd-49f8-a654-7f8d458a8fb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c61f350d444706adada72a33056647'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:39.347267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d1b9fc44-bdcd-49f8-a654-7f8d458a8fb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c61f350d444706adada72a33056647'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:39.351716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d1b9fc44-bdcd-49f8-a654-7f8d458a8fb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c61f350d444706adada72a33056647'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:39.377007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:39.413267 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2743/5526] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:38 2022] DELETE /identity/v3/users/73c61f350d444706adada72a33056647 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:39.418299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f4d39-27c2-4c65-86dd-53b252379395 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:39.418821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f4d39-27c2-4c65-86dd-53b252379395 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:39.418907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f4d39-27c2-4c65-86dd-53b252379395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:39.463799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-860f4d39-27c2-4c65-86dd-53b252379395 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:39.467666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-860f4d39-27c2-4c65-86dd-53b252379395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:39.607272 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2784/5527] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:39.616419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9fc44-bdcd-49f8-a654-7f8d458a8fb3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:39.617883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d1b9fc44-bdcd-49f8-a654-7f8d458a8fb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:39.782024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9fc44-bdcd-49f8-a654-7f8d458a8fb3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:39.796332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b896f-4a6e-4fbe-ad33-98d057c85669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:39.796332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b896f-4a6e-4fbe-ad33-98d057c85669 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:39.796332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b896f-4a6e-4fbe-ad33-98d057c85669 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:39.796332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b896f-4a6e-4fbe-ad33-98d057c85669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:39.821963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b896f-4a6e-4fbe-ad33-98d057c85669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:39.822679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b896f-4a6e-4fbe-ad33-98d057c85669 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:39.837601 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2744/5528] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:40.247097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f4d39-27c2-4c65-86dd-53b252379395 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:40.249755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-860f4d39-27c2-4c65-86dd-53b252379395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:40.273260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f4d39-27c2-4c65-86dd-53b252379395 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:40.315947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c3439-d0cb-496c-9345-bab8181e80ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:40.317053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c3439-d0cb-496c-9345-bab8181e80ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:40.318191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c3439-d0cb-496c-9345-bab8181e80ae admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:40.318614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1c3439-d0cb-496c-9345-bab8181e80ae admin admin] PATH_INFO: `/v3/users/7bfaf45cd74a4e148f2661c249869977` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:40.323230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c3439-d0cb-496c-9345-bab8181e80ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bfaf45cd74a4e148f2661c249869977)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:40.327257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1c3439-d0cb-496c-9345-bab8181e80ae admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:40.641130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2d1c3439-d0cb-496c-9345-bab8181e80ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bfaf45cd74a4e148f2661c249869977'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:40.652788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2d1c3439-d0cb-496c-9345-bab8181e80ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bfaf45cd74a4e148f2661c249869977'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:40.653594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2d1c3439-d0cb-496c-9345-bab8181e80ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bfaf45cd74a4e148f2661c249869977'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:40.655242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2d1c3439-d0cb-496c-9345-bab8181e80ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bfaf45cd74a4e148f2661c249869977'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:40.767390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2d1c3439-d0cb-496c-9345-bab8181e80ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bfaf45cd74a4e148f2661c249869977'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:40.782269 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2785/5529] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:40 2022] DELETE /identity/v3/users/7bfaf45cd74a4e148f2661c249869977 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:40.787040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c1288-4341-4c14-af59-bab8f5b5a94c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:40.787531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c1288-4341-4c14-af59-bab8f5b5a94c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:40.787682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c1288-4341-4c14-af59-bab8f5b5a94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:40.819598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d83c1288-4341-4c14-af59-bab8f5b5a94c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:40.820999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d83c1288-4341-4c14-af59-bab8f5b5a94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:40.955503 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2745/5530] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:40.976656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c3439-d0cb-496c-9345-bab8181e80ae admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:40.979058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2d1c3439-d0cb-496c-9345-bab8181e80ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:41.093424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1c3439-d0cb-496c-9345-bab8181e80ae admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:41.099111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a23444-8cfb-436a-b2aa-2f28b8cfca0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:41.099468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a23444-8cfb-436a-b2aa-2f28b8cfca0d service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:41.099757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a23444-8cfb-436a-b2aa-2f28b8cfca0d service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:41.100061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a23444-8cfb-436a-b2aa-2f28b8cfca0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:41.118419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a23444-8cfb-436a-b2aa-2f28b8cfca0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:41.120550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a23444-8cfb-436a-b2aa-2f28b8cfca0d service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:41.136000 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2786/5531] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:41.211730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c1288-4341-4c14-af59-bab8f5b5a94c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:41.215146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d83c1288-4341-4c14-af59-bab8f5b5a94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:41.228834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c1288-4341-4c14-af59-bab8f5b5a94c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:41.235688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f83592-82ed-40a5-b44e-b470ab589e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:41.236842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f83592-82ed-40a5-b44e-b470ab589e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:41.237364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f83592-82ed-40a5-b44e-b470ab589e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:41.237738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f83592-82ed-40a5-b44e-b470ab589e5f admin admin] PATH_INFO: `/v3/projects/2573d11ff7c84af7bfdca0777c61b158` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:41.239379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f83592-82ed-40a5-b44e-b470ab589e5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2573d11ff7c84af7bfdca0777c61b158)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:41.241712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f83592-82ed-40a5-b44e-b470ab589e5f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:41.376885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:41.446139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-58f83592-82ed-40a5-b44e-b470ab589e5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2573d11ff7c84af7bfdca0777c61b158', 'request_id': 'req-58f83592-82ed-40a5-b44e-b470ab589e5f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:41.453403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-58f83592-82ed-40a5-b44e-b470ab589e5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2573d11ff7c84af7bfdca0777c61b158', 'request_id': 'req-58f83592-82ed-40a5-b44e-b470ab589e5f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:41.584253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-58f83592-82ed-40a5-b44e-b470ab589e5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2573d11ff7c84af7bfdca0777c61b158', 'request_id': 'req-58f83592-82ed-40a5-b44e-b470ab589e5f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:41.609811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-58f83592-82ed-40a5-b44e-b470ab589e5f admin admin] The token cache is being invalidate because project 2573d11ff7c84af7bfdca0777c61b158 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:41.609811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-58f83592-82ed-40a5-b44e-b470ab589e5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:41.609811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-58f83592-82ed-40a5-b44e-b470ab589e5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:41.612191 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2746/5532] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:41 2022] DELETE /identity/v3/projects/2573d11ff7c84af7bfdca0777c61b158 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:41.617234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01f343-6306-431f-a090-621bf3394e6b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:41.617690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01f343-6306-431f-a090-621bf3394e6b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:41.618144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01f343-6306-431f-a090-621bf3394e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:41.638357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ac01f343-6306-431f-a090-621bf3394e6b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:41.639846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ac01f343-6306-431f-a090-621bf3394e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:41.680515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:41.730391 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2787/5533] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:41.749350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc3737-aa93-4094-9d0f-71cc90b14947 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:41.749806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc3737-aa93-4094-9d0f-71cc90b14947 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:41.750240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc3737-aa93-4094-9d0f-71cc90b14947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:41.793744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9fdc3737-aa93-4094-9d0f-71cc90b14947 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:41.796717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9fdc3737-aa93-4094-9d0f-71cc90b14947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:41.819697 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2747/5534] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:41.825232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01f343-6306-431f-a090-621bf3394e6b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:41.836618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01f343-6306-431f-a090-621bf3394e6b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:41.845777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748d593-5c59-4faf-a9fe-48157816dc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:41.863983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748d593-5c59-4faf-a9fe-48157816dc04 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:41.863983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748d593-5c59-4faf-a9fe-48157816dc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:41.863983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748d593-5c59-4faf-a9fe-48157816dc04 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:41.863983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e748d593-5c59-4faf-a9fe-48157816dc04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:20:41.863983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e748d593-5c59-4faf-a9fe-48157816dc04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:41.863983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e748d593-5c59-4faf-a9fe-48157816dc04 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:41.876074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc3737-aa93-4094-9d0f-71cc90b14947 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:41.886078 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2788/5535] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:20:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:41.889303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc3737-aa93-4094-9d0f-71cc90b14947 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:41.896335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f06db-e2bf-4f65-94fd-414c5b52a25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:41.898055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f06db-e2bf-4f65-94fd-414c5b52a25b admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:41.898663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f06db-e2bf-4f65-94fd-414c5b52a25b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:41.899439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f06db-e2bf-4f65-94fd-414c5b52a25b admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:41.900441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f06db-e2bf-4f65-94fd-414c5b52a25b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:41.902611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449f06db-e2bf-4f65-94fd-414c5b52a25b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:42.085747 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2748/5536] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:41 2022] POST /identity/v3/projects => generated 365 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:42.091066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748d593-5c59-4faf-a9fe-48157816dc04 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:42.110412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748d593-5c59-4faf-a9fe-48157816dc04 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:42.168679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f06db-e2bf-4f65-94fd-414c5b52a25b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:42.171650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-449f06db-e2bf-4f65-94fd-414c5b52a25b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:42.194288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6f22e-eb9d-4f93-a9ba-14d423486b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:42.194874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6f22e-eb9d-4f93-a9ba-14d423486b88 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:42.195341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6f22e-eb9d-4f93-a9ba-14d423486b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:42.196326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6f22e-eb9d-4f93-a9ba-14d423486b88 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:42.197267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6f22e-eb9d-4f93-a9ba-14d423486b88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:42.199396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6f22e-eb9d-4f93-a9ba-14d423486b88 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:42.268607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f06db-e2bf-4f65-94fd-414c5b52a25b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:42.274164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e07b7c-1902-42dc-8a32-babf906a89e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:42.275053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e07b7c-1902-42dc-8a32-babf906a89e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:42.275399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e07b7c-1902-42dc-8a32-babf906a89e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:42.275685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e07b7c-1902-42dc-8a32-babf906a89e6 admin admin] PATH_INFO: `/v3/projects/72d4e610a6d641ae9c2c94be6de7de36` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:42.277333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e07b7c-1902-42dc-8a32-babf906a89e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72d4e610a6d641ae9c2c94be6de7de36)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:42.279562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e07b7c-1902-42dc-8a32-babf906a89e6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:42.325599 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2789/5537] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:42 2022] POST /identity/v3/users => generated 283 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:20:42.335557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6f22e-eb9d-4f93-a9ba-14d423486b88 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:42.357001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6f22e-eb9d-4f93-a9ba-14d423486b88 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:42.368877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a7a39-771a-449a-addf-957c03f54a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:42.369821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a7a39-771a-449a-addf-957c03f54a14 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:42.376134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a7a39-771a-449a-addf-957c03f54a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:42.376134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a7a39-771a-449a-addf-957c03f54a14 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:42.376134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a7a39-771a-449a-addf-957c03f54a14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:42.377754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a7a39-771a-449a-addf-957c03f54a14 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:42.396902 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2790/5538] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:42.400151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a7a39-771a-449a-addf-957c03f54a14 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:42.413273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a7a39-771a-449a-addf-957c03f54a14 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:42.419377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ada00-7526-438f-889d-55013caf96e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:42.421290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ada00-7526-438f-889d-55013caf96e8 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:42.421889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ada00-7526-438f-889d-55013caf96e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:42.422949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7ada00-7526-438f-889d-55013caf96e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:42.423948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ada00-7526-438f-889d-55013caf96e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:42.425898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7ada00-7526-438f-889d-55013caf96e8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:42.459913 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2791/5539] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:20:42.459913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ada00-7526-438f-889d-55013caf96e8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:42.471692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-76e07b7c-1902-42dc-8a32-babf906a89e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72d4e610a6d641ae9c2c94be6de7de36', 'request_id': 'req-76e07b7c-1902-42dc-8a32-babf906a89e6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:42.476766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7ada00-7526-438f-889d-55013caf96e8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:42.503351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-76e07b7c-1902-42dc-8a32-babf906a89e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72d4e610a6d641ae9c2c94be6de7de36', 'request_id': 'req-76e07b7c-1902-42dc-8a32-babf906a89e6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:42.593623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-76e07b7c-1902-42dc-8a32-babf906a89e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72d4e610a6d641ae9c2c94be6de7de36', 'request_id': 'req-76e07b7c-1902-42dc-8a32-babf906a89e6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:42.615015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-76e07b7c-1902-42dc-8a32-babf906a89e6 admin admin] The token cache is being invalidate because project 72d4e610a6d641ae9c2c94be6de7de36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:42.615239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-76e07b7c-1902-42dc-8a32-babf906a89e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:42.615927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-76e07b7c-1902-42dc-8a32-babf906a89e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:42.617798 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2749/5540] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:42 2022] DELETE /identity/v3/projects/72d4e610a6d641ae9c2c94be6de7de36 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:42.622090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd773e4-ce7d-41fd-966c-ee20e5ed0f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:42.626830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd773e4-ce7d-41fd-966c-ee20e5ed0f68 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:42.626830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd773e4-ce7d-41fd-966c-ee20e5ed0f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:42.626830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd773e4-ce7d-41fd-966c-ee20e5ed0f68 admin admin] PATH_INFO: `/v3/projects/63829e0935e74e9a8390a6beecd1fc74/users/e676eee7877b40249a5377db8e5efe4f/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:42.634189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8f579-83ae-4ced-a470-87c8c90035ed None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:42.634479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8f579-83ae-4ced-a470-87c8c90035ed None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:42.634760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8f579-83ae-4ced-a470-87c8c90035ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:42.671261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd773e4-ce7d-41fd-966c-ee20e5ed0f68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63829e0935e74e9a8390a6beecd1fc74, user_id=e676eee7877b40249a5377db8e5efe4f, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:42.676971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd773e4-ce7d-41fd-966c-ee20e5ed0f68 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:42.691636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7fe8f579-83ae-4ced-a470-87c8c90035ed None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:42.692857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7fe8f579-83ae-4ced-a470-87c8c90035ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:42.702591 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2750/5541] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:42.710971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af2512-6b78-4e03-83ab-3d087c0da751 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:42.711568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af2512-6b78-4e03-83ab-3d087c0da751 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:42.711944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8af2512-6b78-4e03-83ab-3d087c0da751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:42.736716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a8af2512-6b78-4e03-83ab-3d087c0da751 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:42.738675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a8af2512-6b78-4e03-83ab-3d087c0da751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:42.749998 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2751/5542] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:42.754014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af2512-6b78-4e03-83ab-3d087c0da751 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:42.766654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8af2512-6b78-4e03-83ab-3d087c0da751 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:42.772863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a3db5-30ff-4337-86c7-2d466e5b8505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:42.773815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a3db5-30ff-4337-86c7-2d466e5b8505 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:42.774228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a3db5-30ff-4337-86c7-2d466e5b8505 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:42.774713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a3db5-30ff-4337-86c7-2d466e5b8505 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:42.775553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a3db5-30ff-4337-86c7-2d466e5b8505 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:20:42.775984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a3db5-30ff-4337-86c7-2d466e5b8505 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:42.778189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a3db5-30ff-4337-86c7-2d466e5b8505 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:42.802768 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2752/5543] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:42.807744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a3db5-30ff-4337-86c7-2d466e5b8505 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:42.824411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a3db5-30ff-4337-86c7-2d466e5b8505 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:42.832979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89440bde-bf29-4579-aeb9-f12117f8db64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:42.834086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89440bde-bf29-4579-aeb9-f12117f8db64 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:42.834525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89440bde-bf29-4579-aeb9-f12117f8db64 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:42.834971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89440bde-bf29-4579-aeb9-f12117f8db64 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:42.835818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89440bde-bf29-4579-aeb9-f12117f8db64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:42.841584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89440bde-bf29-4579-aeb9-f12117f8db64 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:42.845063 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2792/5544] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:42 2022] PUT /identity/v3/projects/63829e0935e74e9a8390a6beecd1fc74/users/e676eee7877b40249a5377db8e5efe4f/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:42.858456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6f064-a405-4c8c-8188-04443ee45c38 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:42.858756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6f064-a405-4c8c-8188-04443ee45c38 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:42.859041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6f064-a405-4c8c-8188-04443ee45c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:42.918792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e7b6f064-a405-4c8c-8188-04443ee45c38 None None] MFA Rules not processed for user `e676eee7877b40249a5377db8e5efe4f`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:42.920360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e7b6f064-a405-4c8c-8188-04443ee45c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:42.961858 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2753/5545] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:42 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:20:42.967121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89440bde-bf29-4579-aeb9-f12117f8db64 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:43.058515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=11 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:43.075825 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2793/5546] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:20:43.087849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6f064-a405-4c8c-8188-04443ee45c38 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:43.090284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e7b6f064-a405-4c8c-8188-04443ee45c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:43.092704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89440bde-bf29-4579-aeb9-f12117f8db64 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:43.103543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573d2cd-8345-484e-b109-1d61ab3aa803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:43.105248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573d2cd-8345-484e-b109-1d61ab3aa803 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:43.105682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573d2cd-8345-484e-b109-1d61ab3aa803 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:43.106448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4573d2cd-8345-484e-b109-1d61ab3aa803 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:43.109133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573d2cd-8345-484e-b109-1d61ab3aa803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:43.115255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4573d2cd-8345-484e-b109-1d61ab3aa803 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:43.196837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6f064-a405-4c8c-8188-04443ee45c38 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:43.257742 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2754/5547] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:42 2022] POST /identity/v3/users => generated 286 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:43.262560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573d2cd-8345-484e-b109-1d61ab3aa803 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:43.285797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4573d2cd-8345-484e-b109-1d61ab3aa803 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:43.292279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e5ad2-c835-4a1f-a438-ab72eac6cee0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:43.294135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e5ad2-c835-4a1f-a438-ab72eac6cee0 service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:43.294857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e5ad2-c835-4a1f-a438-ab72eac6cee0 service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:43.295290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e5ad2-c835-4a1f-a438-ab72eac6cee0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:43.296490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89f2f0-92f0-44b6-900b-519ceb5570ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:43.300964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89f2f0-92f0-44b6-900b-519ceb5570ad admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:43.300964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89f2f0-92f0-44b6-900b-519ceb5570ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:43.301919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89f2f0-92f0-44b6-900b-519ceb5570ad admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:43.302726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89f2f0-92f0-44b6-900b-519ceb5570ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:43.305001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89f2f0-92f0-44b6-900b-519ceb5570ad admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:43.335245 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2755/5548] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:20:43.336311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89f2f0-92f0-44b6-900b-519ceb5570ad admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:43.357316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89f2f0-92f0-44b6-900b-519ceb5570ad admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:43.370504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afb658-85c8-4704-9705-f65a867288a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:43.371208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afb658-85c8-4704-9705-f65a867288a8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:43.371499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afb658-85c8-4704-9705-f65a867288a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:43.371785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afb658-85c8-4704-9705-f65a867288a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:43.372538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afb658-85c8-4704-9705-f65a867288a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:43.374219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afb658-85c8-4704-9705-f65a867288a8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:43.411664 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2756/5549] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:20:43.414129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afb658-85c8-4704-9705-f65a867288a8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:43.454906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e5ad2-c835-4a1f-a438-ab72eac6cee0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:43.457168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e5ad2-c835-4a1f-a438-ab72eac6cee0 service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:43.464905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afb658-85c8-4704-9705-f65a867288a8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:43.479420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2b716-df5d-4d07-a47c-ebda0c49f9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:43.480343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2b716-df5d-4d07-a47c-ebda0c49f9d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:43.481364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2b716-df5d-4d07-a47c-ebda0c49f9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:43.481751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2b716-df5d-4d07-a47c-ebda0c49f9d8 admin admin] PATH_INFO: `/v3/projects/877bd1cd8912437c8e4871df8a5f7260/users/84f18f11aad344c18bb6b3997370576b/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:43.483546 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2794/5550] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:43 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:43.536843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2b716-df5d-4d07-a47c-ebda0c49f9d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=877bd1cd8912437c8e4871df8a5f7260, user_id=84f18f11aad344c18bb6b3997370576b, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:43.541015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2b716-df5d-4d07-a47c-ebda0c49f9d8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:43.630481 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2757/5551] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:43 2022] PUT /identity/v3/projects/877bd1cd8912437c8e4871df8a5f7260/users/84f18f11aad344c18bb6b3997370576b/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:43.636062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32035e0c-6440-4196-8ef5-66f6ec99c385 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:43.636500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32035e0c-6440-4196-8ef5-66f6ec99c385 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:43.636916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32035e0c-6440-4196-8ef5-66f6ec99c385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:43.668102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-32035e0c-6440-4196-8ef5-66f6ec99c385 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:43.669655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-32035e0c-6440-4196-8ef5-66f6ec99c385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:43.761359 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2795/5552] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:43.787734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2b716-df5d-4d07-a47c-ebda0c49f9d8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:43.791337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-00b2b716-df5d-4d07-a47c-ebda0c49f9d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:43.957387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2b716-df5d-4d07-a47c-ebda0c49f9d8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:43.963114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a44fb63-03be-4fe6-896f-58c6c7b80f5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:43.964322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44fb63-03be-4fe6-896f-58c6c7b80f5d service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:43.965939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44fb63-03be-4fe6-896f-58c6c7b80f5d service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:43.966586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a44fb63-03be-4fe6-896f-58c6c7b80f5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:43.996726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44fb63-03be-4fe6-896f-58c6c7b80f5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:44.000468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a44fb63-03be-4fe6-896f-58c6c7b80f5d service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:44.025666 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2758/5553] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:44.189259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:44.497318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32035e0c-6440-4196-8ef5-66f6ec99c385 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:44.498766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-32035e0c-6440-4196-8ef5-66f6ec99c385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:44.512168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32035e0c-6440-4196-8ef5-66f6ec99c385 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:44.517544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bd3b3-de84-46d3-a4cd-6e1a0eb7bf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:44.518663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bd3b3-de84-46d3-a4cd-6e1a0eb7bf7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:44.518774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bd3b3-de84-46d3-a4cd-6e1a0eb7bf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:44.519266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88bd3b3-de84-46d3-a4cd-6e1a0eb7bf7f admin admin] PATH_INFO: `/v3/users/e676eee7877b40249a5377db8e5efe4f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:44.520541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bd3b3-de84-46d3-a4cd-6e1a0eb7bf7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e676eee7877b40249a5377db8e5efe4f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:44.522656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88bd3b3-de84-46d3-a4cd-6e1a0eb7bf7f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:44.536476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:44.788187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f88bd3b3-de84-46d3-a4cd-6e1a0eb7bf7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e676eee7877b40249a5377db8e5efe4f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:44.805830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f88bd3b3-de84-46d3-a4cd-6e1a0eb7bf7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e676eee7877b40249a5377db8e5efe4f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:44.806830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f88bd3b3-de84-46d3-a4cd-6e1a0eb7bf7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e676eee7877b40249a5377db8e5efe4f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:44.807583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f88bd3b3-de84-46d3-a4cd-6e1a0eb7bf7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e676eee7877b40249a5377db8e5efe4f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:44.919496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f88bd3b3-de84-46d3-a4cd-6e1a0eb7bf7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e676eee7877b40249a5377db8e5efe4f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:44.955988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71131a4-714b-4a6a-b743-3eaf41909115 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:44.956273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71131a4-714b-4a6a-b743-3eaf41909115 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:44.956974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71131a4-714b-4a6a-b743-3eaf41909115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:44.967754 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2796/5554] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:44 2022] DELETE /identity/v3/users/e676eee7877b40249a5377db8e5efe4f => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:44.981917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c71131a4-714b-4a6a-b743-3eaf41909115 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:44.983851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c71131a4-714b-4a6a-b743-3eaf41909115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:45.084289 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2759/5555] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:45.095411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bd3b3-de84-46d3-a4cd-6e1a0eb7bf7f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:45.097447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f88bd3b3-de84-46d3-a4cd-6e1a0eb7bf7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:45.171259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71131a4-714b-4a6a-b743-3eaf41909115 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:45.173487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c71131a4-714b-4a6a-b743-3eaf41909115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:45.203662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71131a4-714b-4a6a-b743-3eaf41909115 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:45.213064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95be05-32a5-4769-9fcd-90c23b7b63a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:45.214343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95be05-32a5-4769-9fcd-90c23b7b63a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:45.215362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95be05-32a5-4769-9fcd-90c23b7b63a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:45.217367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95be05-32a5-4769-9fcd-90c23b7b63a2 admin admin] PATH_INFO: `/v3/users/2fb3ef26871e4b30a9bc4a5045fe62ad` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:45.221954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95be05-32a5-4769-9fcd-90c23b7b63a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fb3ef26871e4b30a9bc4a5045fe62ad)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:45.227196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95be05-32a5-4769-9fcd-90c23b7b63a2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:45.254383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88bd3b3-de84-46d3-a4cd-6e1a0eb7bf7f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:45.258670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c99916-406b-40f1-8c1f-0b1bc31d6c90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:45.259905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c99916-406b-40f1-8c1f-0b1bc31d6c90 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:45.260456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c99916-406b-40f1-8c1f-0b1bc31d6c90 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:45.260940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c99916-406b-40f1-8c1f-0b1bc31d6c90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:45.283199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c99916-406b-40f1-8c1f-0b1bc31d6c90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:45.287549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c99916-406b-40f1-8c1f-0b1bc31d6c90 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:45.302718 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2797/5556] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:45.460114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7c95be05-32a5-4769-9fcd-90c23b7b63a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fb3ef26871e4b30a9bc4a5045fe62ad'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:45.460919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7c95be05-32a5-4769-9fcd-90c23b7b63a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fb3ef26871e4b30a9bc4a5045fe62ad'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:45.472945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7c95be05-32a5-4769-9fcd-90c23b7b63a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fb3ef26871e4b30a9bc4a5045fe62ad'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:45.547726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7c95be05-32a5-4769-9fcd-90c23b7b63a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fb3ef26871e4b30a9bc4a5045fe62ad'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:45.549391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7c95be05-32a5-4769-9fcd-90c23b7b63a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fb3ef26871e4b30a9bc4a5045fe62ad'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:45.583391 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2760/5557] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:20:45 2022] DELETE /identity/v3/users/2fb3ef26871e4b30a9bc4a5045fe62ad => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:45.795466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd386ca-0646-41da-b0e8-7ae0f014da16 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:45.795946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd386ca-0646-41da-b0e8-7ae0f014da16 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:45.796368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd386ca-0646-41da-b0e8-7ae0f014da16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:45.834064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8bd386ca-0646-41da-b0e8-7ae0f014da16 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:45.835921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8bd386ca-0646-41da-b0e8-7ae0f014da16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:45.925713 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2798/5558] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:20:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:45.941465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95be05-32a5-4769-9fcd-90c23b7b63a2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:45.943094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7c95be05-32a5-4769-9fcd-90c23b7b63a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:45.986711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd386ca-0646-41da-b0e8-7ae0f014da16 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:45.988559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8bd386ca-0646-41da-b0e8-7ae0f014da16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:46.089494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95be05-32a5-4769-9fcd-90c23b7b63a2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:46.101637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd59e8-a795-4f76-9ebb-563682b6e372 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:46.103716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd59e8-a795-4f76-9ebb-563682b6e372 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:46.103716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd59e8-a795-4f76-9ebb-563682b6e372 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:46.103716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd59e8-a795-4f76-9ebb-563682b6e372 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:46.115538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd59e8-a795-4f76-9ebb-563682b6e372 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:46.119809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd59e8-a795-4f76-9ebb-563682b6e372 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:46.142352 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2761/5559] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:20:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:20:46.159958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd386ca-0646-41da-b0e8-7ae0f014da16 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:46.166982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:46.168073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:46.168587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:46.169010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6 admin admin] PATH_INFO: `/v3/projects/f03a2cfe8fd94e4e8581358beadaf177` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:46.171399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f03a2cfe8fd94e4e8581358beadaf177)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:46.174441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:46.362236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f03a2cfe8fd94e4e8581358beadaf177', 'request_id': 'req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:46.373895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f03a2cfe8fd94e4e8581358beadaf177', 'request_id': 'req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:46.382364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd59e8-a795-4f76-9ebb-563682b6e372 service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:46.384971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1edd59e8-a795-4f76-9ebb-563682b6e372 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:46.517832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f03a2cfe8fd94e4e8581358beadaf177', 'request_id': 'req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:46.536557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6 admin admin] The token cache is being invalidate because project f03a2cfe8fd94e4e8581358beadaf177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:46.540395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:46.556308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:46.556308 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2799/5560] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:45 2022] DELETE /identity/v3/projects/f03a2cfe8fd94e4e8581358beadaf177 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:46.556308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:46.558782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:46.641327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd59e8-a795-4f76-9ebb-563682b6e372 service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:46.655545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:46.740628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a97b07-74e2-4ccd-b2dd-767963fbd3c6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:46.746516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d366e64d-4ec4-4bcd-a27f-c85d90c07b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:46.747403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366e64d-4ec4-4bcd-a27f-c85d90c07b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:46.747778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366e64d-4ec4-4bcd-a27f-c85d90c07b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:46.748098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366e64d-4ec4-4bcd-a27f-c85d90c07b90 admin admin] PATH_INFO: `/v3/projects/63829e0935e74e9a8390a6beecd1fc74` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:46.749334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366e64d-4ec4-4bcd-a27f-c85d90c07b90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63829e0935e74e9a8390a6beecd1fc74)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:46.752052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d366e64d-4ec4-4bcd-a27f-c85d90c07b90 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:46.781315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e086e6e-52ee-4707-b691-e50c55653d67 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:46.782721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e086e6e-52ee-4707-b691-e50c55653d67 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:46.783587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e086e6e-52ee-4707-b691-e50c55653d67 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:46.783881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e086e6e-52ee-4707-b691-e50c55653d67 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:46.787912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5e086e6e-52ee-4707-b691-e50c55653d67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:46.827394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e086e6e-52ee-4707-b691-e50c55653d67 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:46.829305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e086e6e-52ee-4707-b691-e50c55653d67 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:46.846939 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2762/5561] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:46.849204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98aa009-628d-4cac-8909-bc6dc71d762d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:46.849204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98aa009-628d-4cac-8909-bc6dc71d762d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:46.849682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98aa009-628d-4cac-8909-bc6dc71d762d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:46.889470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e98aa009-628d-4cac-8909-bc6dc71d762d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:46.891576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e98aa009-628d-4cac-8909-bc6dc71d762d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:46.917384 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2763/5562] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:46.926844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fb0ce-1ae4-40e0-98db-6b4f481c1551 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:46.927264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fb0ce-1ae4-40e0-98db-6b4f481c1551 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:46.927689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fb0ce-1ae4-40e0-98db-6b4f481c1551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:46.965936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d366e64d-4ec4-4bcd-a27f-c85d90c07b90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63829e0935e74e9a8390a6beecd1fc74', 'request_id': 'req-d366e64d-4ec4-4bcd-a27f-c85d90c07b90'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:46.970216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6d4fb0ce-1ae4-40e0-98db-6b4f481c1551 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:46.973444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6d4fb0ce-1ae4-40e0-98db-6b4f481c1551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:47.009594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d366e64d-4ec4-4bcd-a27f-c85d90c07b90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63829e0935e74e9a8390a6beecd1fc74', 'request_id': 'req-d366e64d-4ec4-4bcd-a27f-c85d90c07b90'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:47.101932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d366e64d-4ec4-4bcd-a27f-c85d90c07b90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63829e0935e74e9a8390a6beecd1fc74', 'request_id': 'req-d366e64d-4ec4-4bcd-a27f-c85d90c07b90'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:47.110703 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2764/5563] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:47.116639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fb0ce-1ae4-40e0-98db-6b4f481c1551 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:47.129063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d366e64d-4ec4-4bcd-a27f-c85d90c07b90 admin admin] The token cache is being invalidate because project 63829e0935e74e9a8390a6beecd1fc74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:20:47.129427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d366e64d-4ec4-4bcd-a27f-c85d90c07b90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:47.130399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d366e64d-4ec4-4bcd-a27f-c85d90c07b90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:20:47.131179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fb0ce-1ae4-40e0-98db-6b4f481c1551 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:47.132202 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2800/5564] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:20:46 2022] DELETE /identity/v3/projects/63829e0935e74e9a8390a6beecd1fc74 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:47.140902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db63f55-b882-47c5-92b2-793e65651127 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:47.141747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db63f55-b882-47c5-92b2-793e65651127 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:47.142251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db63f55-b882-47c5-92b2-793e65651127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:47.147446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9259a-105f-451a-8dd3-43429bcbeafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:47.148818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9259a-105f-451a-8dd3-43429bcbeafc admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:47.150185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9259a-105f-451a-8dd3-43429bcbeafc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:47.150710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9259a-105f-451a-8dd3-43429bcbeafc admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:47.152074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9259a-105f-451a-8dd3-43429bcbeafc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:20:47.152598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9259a-105f-451a-8dd3-43429bcbeafc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:47.155003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c9259a-105f-451a-8dd3-43429bcbeafc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:47.174620 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2765/5565] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:47.178449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9259a-105f-451a-8dd3-43429bcbeafc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:47.182285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-79c9259a-105f-451a-8dd3-43429bcbeafc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:47.187999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6db63f55-b882-47c5-92b2-793e65651127 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:47.191414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6db63f55-b882-47c5-92b2-793e65651127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:47.209073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9259a-105f-451a-8dd3-43429bcbeafc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:47.211760 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2801/5566] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:20:47.212344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:47.217743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bbb97-ac12-42fc-a640-12f12a111f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:47.219214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bbb97-ac12-42fc-a640-12f12a111f4d admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:47.219214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bbb97-ac12-42fc-a640-12f12a111f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:47.220245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68279e-dd95-498c-86ec-007a0c4943f8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:47.220612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211bbb97-ac12-42fc-a640-12f12a111f4d admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:47.220867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68279e-dd95-498c-86ec-007a0c4943f8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:47.221406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68279e-dd95-498c-86ec-007a0c4943f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:47.222044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211bbb97-ac12-42fc-a640-12f12a111f4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:47.225571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211bbb97-ac12-42fc-a640-12f12a111f4d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:47.295112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4c68279e-dd95-498c-86ec-007a0c4943f8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:47.300062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4c68279e-dd95-498c-86ec-007a0c4943f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:47.326209 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2802/5567] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:47.327588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68279e-dd95-498c-86ec-007a0c4943f8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:47.358600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68279e-dd95-498c-86ec-007a0c4943f8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:47.370778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67aa35-9d82-419a-8c8a-81d141b2d5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:47.371499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67aa35-9d82-419a-8c8a-81d141b2d5e4 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:47.371755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67aa35-9d82-419a-8c8a-81d141b2d5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:47.372197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb67aa35-9d82-419a-8c8a-81d141b2d5e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:47.373723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67aa35-9d82-419a-8c8a-81d141b2d5e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:20:47.374289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67aa35-9d82-419a-8c8a-81d141b2d5e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:47.378015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb67aa35-9d82-419a-8c8a-81d141b2d5e4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:47.396261 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2803/5568] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:20:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:20:47.400562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67aa35-9d82-419a-8c8a-81d141b2d5e4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:47.413320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb67aa35-9d82-419a-8c8a-81d141b2d5e4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:47.422835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf3e28-85f3-408e-957a-aa73b48b536a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:47.423826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf3e28-85f3-408e-957a-aa73b48b536a admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:47.424279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf3e28-85f3-408e-957a-aa73b48b536a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:47.424758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf3e28-85f3-408e-957a-aa73b48b536a admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:47.426974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf3e28-85f3-408e-957a-aa73b48b536a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:47.429791 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2766/5569] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:47 2022] POST /identity/v3/projects => generated 375 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:47.431190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf3e28-85f3-408e-957a-aa73b48b536a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:47.433678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bbb97-ac12-42fc-a640-12f12a111f4d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:47.572856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211bbb97-ac12-42fc-a640-12f12a111f4d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:47.581434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829bfe2-76a2-4194-8fae-c297c2864bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:47.581434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829bfe2-76a2-4194-8fae-c297c2864bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:47.581434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829bfe2-76a2-4194-8fae-c297c2864bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:47.581434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829bfe2-76a2-4194-8fae-c297c2864bc6 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:47.581952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829bfe2-76a2-4194-8fae-c297c2864bc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:47.583028 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2804/5570] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:47 2022] POST /identity/v3/projects => generated 369 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:47.583297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c829bfe2-76a2-4194-8fae-c297c2864bc6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:47.584276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf3e28-85f3-408e-957a-aa73b48b536a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:47.672812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf3e28-85f3-408e-957a-aa73b48b536a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:47.678723 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2767/5571] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:47 2022] POST /identity/v3/users => generated 288 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:47.683459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829bfe2-76a2-4194-8fae-c297c2864bc6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:47.697821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff818d1b-c262-4760-8515-7faedffea60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:47.698801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff818d1b-c262-4760-8515-7faedffea60a admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:47.699151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff818d1b-c262-4760-8515-7faedffea60a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:47.699493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff818d1b-c262-4760-8515-7faedffea60a admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:47.700213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff818d1b-c262-4760-8515-7faedffea60a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:47.702254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff818d1b-c262-4760-8515-7faedffea60a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:47.704202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c829bfe2-76a2-4194-8fae-c297c2864bc6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:47.712937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606e367-8ee7-493a-b325-c0af5dd99262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:47.714110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606e367-8ee7-493a-b325-c0af5dd99262 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:47.714575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606e367-8ee7-493a-b325-c0af5dd99262 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:47.714915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606e367-8ee7-493a-b325-c0af5dd99262 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:47.715812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606e367-8ee7-493a-b325-c0af5dd99262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:47.717876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606e367-8ee7-493a-b325-c0af5dd99262 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:47.738564 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2768/5572] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:47.743404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606e367-8ee7-493a-b325-c0af5dd99262 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:47.760814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606e367-8ee7-493a-b325-c0af5dd99262 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:47.780032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872b9c1-3ddd-4c0d-bfef-1a95bf56509c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:47.850155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872b9c1-3ddd-4c0d-bfef-1a95bf56509c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:47.850155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872b9c1-3ddd-4c0d-bfef-1a95bf56509c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:47.850155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872b9c1-3ddd-4c0d-bfef-1a95bf56509c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:47.850155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872b9c1-3ddd-4c0d-bfef-1a95bf56509c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:47.850155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872b9c1-3ddd-4c0d-bfef-1a95bf56509c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:47.850155 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2769/5573] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:47 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:47.855385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872b9c1-3ddd-4c0d-bfef-1a95bf56509c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:47.911336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872b9c1-3ddd-4c0d-bfef-1a95bf56509c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:47.927878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b023a14-950e-433b-8022-4c65ba65c4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:47.929758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b023a14-950e-433b-8022-4c65ba65c4ea admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:47.930236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b023a14-950e-433b-8022-4c65ba65c4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:47.930686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b023a14-950e-433b-8022-4c65ba65c4ea admin admin] PATH_INFO: `/v3/projects/3c6886d8b684413da24b02f5b5fb6ef8/users/f1acb2fffdc64ce29b71b2b40e2285f0/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:47.968147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b023a14-950e-433b-8022-4c65ba65c4ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c6886d8b684413da24b02f5b5fb6ef8, user_id=f1acb2fffdc64ce29b71b2b40e2285f0, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:47.973179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b023a14-950e-433b-8022-4c65ba65c4ea admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:48.283638 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2805/5574] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:47 2022] POST /identity/v3/users => generated 285 bytes in 699 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:48.285806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff818d1b-c262-4760-8515-7faedffea60a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:48.316028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff818d1b-c262-4760-8515-7faedffea60a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:48.340859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac90758-9889-42a1-96a1-93b59b783a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:48.344334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac90758-9889-42a1-96a1-93b59b783a43 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:48.344334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac90758-9889-42a1-96a1-93b59b783a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:48.344334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac90758-9889-42a1-96a1-93b59b783a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:48.344334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac90758-9889-42a1-96a1-93b59b783a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:48.346391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac90758-9889-42a1-96a1-93b59b783a43 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:48.369614 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2806/5575] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:48.372137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac90758-9889-42a1-96a1-93b59b783a43 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:48.377641 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2770/5576] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:47 2022] PUT /identity/v3/projects/3c6886d8b684413da24b02f5b5fb6ef8/users/f1acb2fffdc64ce29b71b2b40e2285f0/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:20:48.383751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9908e-8e2d-44d0-9bff-16149263cdb8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:48.384275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9908e-8e2d-44d0-9bff-16149263cdb8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:48.402666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9908e-8e2d-44d0-9bff-16149263cdb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:48.405091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac90758-9889-42a1-96a1-93b59b783a43 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:48.462094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-05a9908e-8e2d-44d0-9bff-16149263cdb8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:48.471872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-05a9908e-8e2d-44d0-9bff-16149263cdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:48.595284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c735ad9-baf2-4723-847b-815069dcc6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:48.597126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c735ad9-baf2-4723-847b-815069dcc6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:48.599850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c735ad9-baf2-4723-847b-815069dcc6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:48.601194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c735ad9-baf2-4723-847b-815069dcc6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:48.601319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c735ad9-baf2-4723-847b-815069dcc6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:48.611710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c735ad9-baf2-4723-847b-815069dcc6e0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:48.631943 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2807/5577] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:48 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:48.633685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da878eb6-7d3b-4200-b95e-27c3a889240c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:48.634082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da878eb6-7d3b-4200-b95e-27c3a889240c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:48.634452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da878eb6-7d3b-4200-b95e-27c3a889240c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:48.659160 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2771/5578] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:48.660750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9908e-8e2d-44d0-9bff-16149263cdb8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:48.679593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9908e-8e2d-44d0-9bff-16149263cdb8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:48.690004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-da878eb6-7d3b-4200-b95e-27c3a889240c None None] MFA Rules not processed for user `84f18f11aad344c18bb6b3997370576b`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:48.692076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-da878eb6-7d3b-4200-b95e-27c3a889240c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:48.692934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcf676-eaa3-402b-8ef7-b983ab304aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:48.694562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcf676-eaa3-402b-8ef7-b983ab304aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:48.695195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcf676-eaa3-402b-8ef7-b983ab304aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:48.695670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcf676-eaa3-402b-8ef7-b983ab304aa0 admin admin] PATH_INFO: `/v3/projects/b63aa53e28e44141b5f714a06743af19/users/561bfffea82b4b9ea2a93ef49bd49af6/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:48.727161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcf676-eaa3-402b-8ef7-b983ab304aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63aa53e28e44141b5f714a06743af19, user_id=561bfffea82b4b9ea2a93ef49bd49af6, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:48.731370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbcf676-eaa3-402b-8ef7-b983ab304aa0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:48.823423 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2808/5579] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:48 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:48.824476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da878eb6-7d3b-4200-b95e-27c3a889240c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:48.827376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-da878eb6-7d3b-4200-b95e-27c3a889240c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:48.857483 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2772/5580] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:48 2022] PUT /identity/v3/projects/b63aa53e28e44141b5f714a06743af19/users/561bfffea82b4b9ea2a93ef49bd49af6/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:48.859196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcf676-eaa3-402b-8ef7-b983ab304aa0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:48.865307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6bbcf676-eaa3-402b-8ef7-b983ab304aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:48.960501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da878eb6-7d3b-4200-b95e-27c3a889240c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:48.996534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcf676-eaa3-402b-8ef7-b983ab304aa0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:49.003232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01143829-8af5-4a97-9f4f-ef680d2c483b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:49.004540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01143829-8af5-4a97-9f4f-ef680d2c483b service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:49.004940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01143829-8af5-4a97-9f4f-ef680d2c483b service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:49.005332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01143829-8af5-4a97-9f4f-ef680d2c483b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:49.090362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01143829-8af5-4a97-9f4f-ef680d2c483b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:49.092353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01143829-8af5-4a97-9f4f-ef680d2c483b service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:49.099859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59638208-6443-4a4d-b335-1b64f535501e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:49.100848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59638208-6443-4a4d-b335-1b64f535501e service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:49.101617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59638208-6443-4a4d-b335-1b64f535501e service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:49.102579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59638208-6443-4a4d-b335-1b64f535501e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:49.118907 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2773/5581] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:49.122829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406f335-0100-4c19-bd05-0f74ef544e6a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:49.131834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406f335-0100-4c19-bd05-0f74ef544e6a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:49.131834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406f335-0100-4c19-bd05-0f74ef544e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:49.196633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1406f335-0100-4c19-bd05-0f74ef544e6a None None] MFA Rules not processed for user `561bfffea82b4b9ea2a93ef49bd49af6`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:49.211038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1406f335-0100-4c19-bd05-0f74ef544e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:49.238052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:49.346848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59638208-6443-4a4d-b335-1b64f535501e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:49.352040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59638208-6443-4a4d-b335-1b64f535501e service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:49.386576 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2774/5582] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:49.393702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406f335-0100-4c19-bd05-0f74ef544e6a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:49.401675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1406f335-0100-4c19-bd05-0f74ef544e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:49.408161 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2809/5583] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:49.410434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59638208-6443-4a4d-b335-1b64f535501e service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:49.436001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59638208-6443-4a4d-b335-1b64f535501e service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:49.447235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a32a0-c5a0-4cba-b38b-b95c5eae1ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:49.448532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a32a0-c5a0-4cba-b38b-b95c5eae1ffd admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:49.449533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a32a0-c5a0-4cba-b38b-b95c5eae1ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:49.450067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03a32a0-c5a0-4cba-b38b-b95c5eae1ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:49.451210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a32a0-c5a0-4cba-b38b-b95c5eae1ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:49.454991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03a32a0-c5a0-4cba-b38b-b95c5eae1ffd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:49.487079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:49.549407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406f335-0100-4c19-bd05-0f74ef544e6a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:49.555444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764c8ee-e452-46d1-abde-802beb1b1791 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:49.556236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764c8ee-e452-46d1-abde-802beb1b1791 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:49.556617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764c8ee-e452-46d1-abde-802beb1b1791 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:49.557036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0764c8ee-e452-46d1-abde-802beb1b1791 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:49.559176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0764c8ee-e452-46d1-abde-802beb1b1791 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:49.600616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0764c8ee-e452-46d1-abde-802beb1b1791 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:49.603668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0764c8ee-e452-46d1-abde-802beb1b1791 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:49.621360 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2810/5584] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:20:49 2022] POST /identity/v3/projects => generated 369 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:49.626801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a32a0-c5a0-4cba-b38b-b95c5eae1ffd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:49.665500 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2775/5585] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:49.792124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03a32a0-c5a0-4cba-b38b-b95c5eae1ffd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:49.797428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504daf71-c373-4a1b-90e6-714f03160a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:49.798033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504daf71-c373-4a1b-90e6-714f03160a9d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:49.798318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504daf71-c373-4a1b-90e6-714f03160a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:49.799106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504daf71-c373-4a1b-90e6-714f03160a9d admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:49.800025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504daf71-c373-4a1b-90e6-714f03160a9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:49.801982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504daf71-c373-4a1b-90e6-714f03160a9d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:49.953905 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2811/5586] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:20:49 2022] POST /identity/v3/users => generated 291 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:20:49.956565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764c8ee-e452-46d1-abde-802beb1b1791 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:49.972416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0764c8ee-e452-46d1-abde-802beb1b1791 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:49.992103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a943e8-4ed1-499f-b090-f336c8b4e6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:49.995738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a943e8-4ed1-499f-b090-f336c8b4e6a8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:49.996093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a943e8-4ed1-499f-b090-f336c8b4e6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:49.996716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a943e8-4ed1-499f-b090-f336c8b4e6a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:49.998952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a943e8-4ed1-499f-b090-f336c8b4e6a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:50.002846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a943e8-4ed1-499f-b090-f336c8b4e6a8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:50.028707 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2776/5587] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:50.033028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504daf71-c373-4a1b-90e6-714f03160a9d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:50.061265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504daf71-c373-4a1b-90e6-714f03160a9d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:50.079327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30b08b-efca-408b-b025-57c3113eb4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:50.080690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30b08b-efca-408b-b025-57c3113eb4ae admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:50.081919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30b08b-efca-408b-b025-57c3113eb4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:50.083257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30b08b-efca-408b-b025-57c3113eb4ae admin admin] PATH_INFO: `/v3/projects/7f402ea60d7c48a8b433695f43295461/users/b095b82889ce475bb15a11971f1c4a60/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:50.132097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30b08b-efca-408b-b025-57c3113eb4ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f402ea60d7c48a8b433695f43295461, user_id=b095b82889ce475bb15a11971f1c4a60, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:50.135456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30b08b-efca-408b-b025-57c3113eb4ae admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:50.315061 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2812/5588] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:50 2022] PUT /identity/v3/projects/7f402ea60d7c48a8b433695f43295461/users/b095b82889ce475bb15a11971f1c4a60/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:50.319473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a943e8-4ed1-499f-b090-f336c8b4e6a8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:50.349012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a943e8-4ed1-499f-b090-f336c8b4e6a8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:50.484842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab7d154-5f1f-4786-b301-9c190da69d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:50.485688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7d154-5f1f-4786-b301-9c190da69d9e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:50.486035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7d154-5f1f-4786-b301-9c190da69d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:50.486313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab7d154-5f1f-4786-b301-9c190da69d9e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:50.487091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab7d154-5f1f-4786-b301-9c190da69d9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:50.488858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab7d154-5f1f-4786-b301-9c190da69d9e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:50.505472 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2777/5589] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:20:50.511257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30b08b-efca-408b-b025-57c3113eb4ae admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:50.534151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30b08b-efca-408b-b025-57c3113eb4ae admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:50.540577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e52c4a-5fa6-4c03-8e63-7832babdabdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:50.541918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e52c4a-5fa6-4c03-8e63-7832babdabdd admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:50.542630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e52c4a-5fa6-4c03-8e63-7832babdabdd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:50.543135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e52c4a-5fa6-4c03-8e63-7832babdabdd admin admin] PATH_INFO: `/v3/projects/7f402ea60d7c48a8b433695f43295461/users/b095b82889ce475bb15a11971f1c4a60/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:50.545495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e52c4a-5fa6-4c03-8e63-7832babdabdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f402ea60d7c48a8b433695f43295461, user_id=b095b82889ce475bb15a11971f1c4a60, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:50.551431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e52c4a-5fa6-4c03-8e63-7832babdabdd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:50.574703 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2813/5590] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:20:50 2022] PUT /identity/v3/projects/7f402ea60d7c48a8b433695f43295461/users/b095b82889ce475bb15a11971f1c4a60/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:20:50.590203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01aeda-10cf-4a2e-b2de-59eef1668d27 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:50.590556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01aeda-10cf-4a2e-b2de-59eef1668d27 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:50.590776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01aeda-10cf-4a2e-b2de-59eef1668d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:50.678852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3f01aeda-10cf-4a2e-b2de-59eef1668d27 None None] MFA Rules not processed for user `b095b82889ce475bb15a11971f1c4a60`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:50.680254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3f01aeda-10cf-4a2e-b2de-59eef1668d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:50.892280 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2778/5591] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:50 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:50.907418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e52c4a-5fa6-4c03-8e63-7832babdabdd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:50.909878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-94e52c4a-5fa6-4c03-8e63-7832babdabdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:51.098970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e52c4a-5fa6-4c03-8e63-7832babdabdd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:51.103655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22b9a4-2033-4989-b0a1-a1ab153c183b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:51.104385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22b9a4-2033-4989-b0a1-a1ab153c183b service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:51.104673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22b9a4-2033-4989-b0a1-a1ab153c183b service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:51.104953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22b9a4-2033-4989-b0a1-a1ab153c183b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:51.135248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22b9a4-2033-4989-b0a1-a1ab153c183b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:51.137935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22b9a4-2033-4989-b0a1-a1ab153c183b service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:51.163909 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2814/5592] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:50 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:51.240253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:51.658297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:52.761079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:53.188471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:53.631657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe64cd-a64a-4cc5-acc0-5d140241c951 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:53.631952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe64cd-a64a-4cc5-acc0-5d140241c951 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:53.632235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe64cd-a64a-4cc5-acc0-5d140241c951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:53.670333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d1fe64cd-a64a-4cc5-acc0-5d140241c951 None None] MFA Rules not processed for user `84f18f11aad344c18bb6b3997370576b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:20:53.671634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d1fe64cd-a64a-4cc5-acc0-5d140241c951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:53.763954 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2779/5593] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:20:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:20:54.142890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:54.337696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:54.446980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22b9a4-2033-4989-b0a1-a1ab153c183b service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:20:54.491192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22b9a4-2033-4989-b0a1-a1ab153c183b service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:20:54.509329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea29d8d-3641-41ae-9904-90d9add0d95f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:20:54.514027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea29d8d-3641-41ae-9904-90d9add0d95f service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:20:54.514375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea29d8d-3641-41ae-9904-90d9add0d95f service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:20:54.515018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea29d8d-3641-41ae-9904-90d9add0d95f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:20:54.525685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9ea29d8d-3641-41ae-9904-90d9add0d95f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:20:54.544535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea29d8d-3641-41ae-9904-90d9add0d95f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:20:54.546596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea29d8d-3641-41ae-9904-90d9add0d95f service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:20:54.563157 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2815/5594] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:20:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:20:55.640344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:57.421990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:59.301921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:00.913310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:01.708003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:02.159018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:03.631700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c2008-0f93-4e3f-82d3-0833e0160eea None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:03.632724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c2008-0f93-4e3f-82d3-0833e0160eea None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:03.633159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c2008-0f93-4e3f-82d3-0833e0160eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:03.721498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-fa5c2008-0f93-4e3f-82d3-0833e0160eea None None] MFA Rules not processed for user `f1acb2fffdc64ce29b71b2b40e2285f0`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:03.723059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fa5c2008-0f93-4e3f-82d3-0833e0160eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:03.886845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:03.920283 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2780/5595] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:03.931394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd2c13-aed9-4b40-98b9-e2e1fb3ab0b1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:03.931943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd2c13-aed9-4b40-98b9-e2e1fb3ab0b1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:03.932370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd2c13-aed9-4b40-98b9-e2e1fb3ab0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:03.986872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-49fd2c13-aed9-4b40-98b9-e2e1fb3ab0b1 None None] MFA Rules not processed for user `f1acb2fffdc64ce29b71b2b40e2285f0`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:03.995438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-49fd2c13-aed9-4b40-98b9-e2e1fb3ab0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:04.014963 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2816/5596] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:04.108670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:04.356751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c2008-0f93-4e3f-82d3-0833e0160eea None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:04.358298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fa5c2008-0f93-4e3f-82d3-0833e0160eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:04.670879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c2008-0f93-4e3f-82d3-0833e0160eea None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:04.690626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb1acb-eb75-448e-b5e1-e8be1b83ed62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:04.693149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb1acb-eb75-448e-b5e1-e8be1b83ed62 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:04.693448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb1acb-eb75-448e-b5e1-e8be1b83ed62 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:04.697039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb1acb-eb75-448e-b5e1-e8be1b83ed62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:04.715343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb1acb-eb75-448e-b5e1-e8be1b83ed62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:04.717010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb1acb-eb75-448e-b5e1-e8be1b83ed62 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:04.763014 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2781/5597] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:21:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:21:04.783677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:05.669722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:06.057045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:09.210790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd2c13-aed9-4b40-98b9-e2e1fb3ab0b1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:09.224365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd2c13-aed9-4b40-98b9-e2e1fb3ab0b1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:09.230145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f8a58-726e-4c42-a4fe-9e125ae3fb34 tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b095b82889ce475bb15a11971f1c4a60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f402ea60d7c48a8b433695f43295461', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:09.230987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f8a58-726e-4c42-a4fe-9e125ae3fb34 tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:09.231585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f8a58-726e-4c42-a4fe-9e125ae3fb34 tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:09.232030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9f8a58-726e-4c42-a4fe-9e125ae3fb34 tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:09.232937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f8a58-726e-4c42-a4fe-9e125ae3fb34 tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:21:09.233410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f8a58-726e-4c42-a4fe-9e125ae3fb34 tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:09.235867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9f8a58-726e-4c42-a4fe-9e125ae3fb34 tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:09.249901 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2817/5598] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:21:09.254274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb1acb-eb75-448e-b5e1-e8be1b83ed62 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:09.264758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb1acb-eb75-448e-b5e1-e8be1b83ed62 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:09.269361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e30612-03ff-4d8d-b9e4-71ce90850e30 tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b095b82889ce475bb15a11971f1c4a60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f402ea60d7c48a8b433695f43295461', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:09.270041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e30612-03ff-4d8d-b9e4-71ce90850e30 tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:09.270331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e30612-03ff-4d8d-b9e4-71ce90850e30 tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:09.270610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e30612-03ff-4d8d-b9e4-71ce90850e30 tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:09.271338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e30612-03ff-4d8d-b9e4-71ce90850e30 tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:09.273398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e30612-03ff-4d8d-b9e4-71ce90850e30 tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:09.279052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:09.477104 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2782/5599] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:21:09 2022] POST /identity/v3/projects => generated 349 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:21:09.481495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f8a58-726e-4c42-a4fe-9e125ae3fb34 tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:09.500938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9f8a58-726e-4c42-a4fe-9e125ae3fb34 tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:09.638015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4fd97-5778-47df-9236-718a3730bade tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b095b82889ce475bb15a11971f1c4a60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f402ea60d7c48a8b433695f43295461', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:09.641105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4fd97-5778-47df-9236-718a3730bade tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:09.641413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4fd97-5778-47df-9236-718a3730bade tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:09.641913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da4fd97-5778-47df-9236-718a3730bade tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:09.642512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4fd97-5778-47df-9236-718a3730bade tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:21:09.644748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4fd97-5778-47df-9236-718a3730bade tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:09.646444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da4fd97-5778-47df-9236-718a3730bade tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:09.665825 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2818/5600] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:21:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:21:10.063152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:10.642481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e30612-03ff-4d8d-b9e4-71ce90850e30 tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:10.656666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e30612-03ff-4d8d-b9e4-71ce90850e30 tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:10.664058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954896fa-0cb6-4ff8-b002-5567e43af8cf tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b095b82889ce475bb15a11971f1c4a60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f402ea60d7c48a8b433695f43295461', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:10.664987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954896fa-0cb6-4ff8-b002-5567e43af8cf tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:10.665360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954896fa-0cb6-4ff8-b002-5567e43af8cf tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:10.665764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954896fa-0cb6-4ff8-b002-5567e43af8cf tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] PATH_INFO: `/v3/projects/ec5491966dcc4773a609d0cb0d8aa5a9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:10.667940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954896fa-0cb6-4ff8-b002-5567e43af8cf tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ec5491966dcc4773a609d0cb0d8aa5a9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:10.670280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954896fa-0cb6-4ff8-b002-5567e43af8cf tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:10.819475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-954896fa-0cb6-4ff8-b002-5567e43af8cf tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec5491966dcc4773a609d0cb0d8aa5a9', 'request_id': 'req-954896fa-0cb6-4ff8-b002-5567e43af8cf'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:10.826928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-954896fa-0cb6-4ff8-b002-5567e43af8cf tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec5491966dcc4773a609d0cb0d8aa5a9', 'request_id': 'req-954896fa-0cb6-4ff8-b002-5567e43af8cf'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:10.887074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-954896fa-0cb6-4ff8-b002-5567e43af8cf tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec5491966dcc4773a609d0cb0d8aa5a9', 'request_id': 'req-954896fa-0cb6-4ff8-b002-5567e43af8cf'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:10.903312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-954896fa-0cb6-4ff8-b002-5567e43af8cf tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] The token cache is being invalidate because project ec5491966dcc4773a609d0cb0d8aa5a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:21:10.903805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-954896fa-0cb6-4ff8-b002-5567e43af8cf tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:10.904596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-954896fa-0cb6-4ff8-b002-5567e43af8cf tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:10.907345 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2783/5601] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:21:10 2022] DELETE /identity/v3/projects/ec5491966dcc4773a609d0cb0d8aa5a9 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:10.918895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4fd97-5778-47df-9236-718a3730bade tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:10.921936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8da4fd97-5778-47df-9236-718a3730bade tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:11.044861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da4fd97-5778-47df-9236-718a3730bade tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:11.111348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc7de9-249c-42f7-ae3c-632bde253d94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:11.116633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc7de9-249c-42f7-ae3c-632bde253d94 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:11.116633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc7de9-249c-42f7-ae3c-632bde253d94 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:11.116633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc7de9-249c-42f7-ae3c-632bde253d94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:11.117554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d2dc7de9-249c-42f7-ae3c-632bde253d94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:11.220882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc7de9-249c-42f7-ae3c-632bde253d94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:11.222708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc7de9-249c-42f7-ae3c-632bde253d94 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:11.234241 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2819/5602] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:21:10 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:21:12.595384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:12.861881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:14.888984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:UPDATE=1,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:15.381661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:15.452574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954896fa-0cb6-4ff8-b002-5567e43af8cf tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:15.455336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-954896fa-0cb6-4ff8-b002-5567e43af8cf tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:15.536380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954896fa-0cb6-4ff8-b002-5567e43af8cf tempest-RoutersAdminTest-1880349875 tempest-RoutersAdminTest-1880349875-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:15.540920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b584770-e3fa-4cb7-af69-33543213feae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:15.541690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b584770-e3fa-4cb7-af69-33543213feae service glance] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:15.542107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b584770-e3fa-4cb7-af69-33543213feae service glance] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:15.542463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b584770-e3fa-4cb7-af69-33543213feae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:15.544396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0b584770-e3fa-4cb7-af69-33543213feae service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:15.641100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b584770-e3fa-4cb7-af69-33543213feae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:15.642031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b584770-e3fa-4cb7-af69-33543213feae service glance] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:15.658655 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2784/5603] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:21:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:21:15.949642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc7de9-249c-42f7-ae3c-632bde253d94 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:15.952318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d2dc7de9-249c-42f7-ae3c-632bde253d94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:15.966733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc7de9-249c-42f7-ae3c-632bde253d94 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:15.974739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291766a-5e41-43a7-a6d8-fdc7fc93f500 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:15.986451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291766a-5e41-43a7-a6d8-fdc7fc93f500 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:15.986451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291766a-5e41-43a7-a6d8-fdc7fc93f500 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:15.986451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1291766a-5e41-43a7-a6d8-fdc7fc93f500 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:15.986451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1291766a-5e41-43a7-a6d8-fdc7fc93f500 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:15.990569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291766a-5e41-43a7-a6d8-fdc7fc93f500 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:15.992024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1291766a-5e41-43a7-a6d8-fdc7fc93f500 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:16.003102 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2820/5604] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:21:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:21:16.174207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b584770-e3fa-4cb7-af69-33543213feae service glance] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:16.189361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b584770-e3fa-4cb7-af69-33543213feae service glance] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:16.194761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e01c7-394a-441d-ad4a-dd68e18ee58a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:16.195715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e01c7-394a-441d-ad4a-dd68e18ee58a service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:16.196103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e01c7-394a-441d-ad4a-dd68e18ee58a service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:16.196487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e01c7-394a-441d-ad4a-dd68e18ee58a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:16.198558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-772e01c7-394a-441d-ad4a-dd68e18ee58a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:16.217159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e01c7-394a-441d-ad4a-dd68e18ee58a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:16.225729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e01c7-394a-441d-ad4a-dd68e18ee58a service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:16.234449 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2785/5605] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:21:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:21:16.947039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:17.238686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:19.487972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:20.053366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:22.206250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291766a-5e41-43a7-a6d8-fdc7fc93f500 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:22.211596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1291766a-5e41-43a7-a6d8-fdc7fc93f500 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:22.317656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1291766a-5e41-43a7-a6d8-fdc7fc93f500 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:22.323596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca319e-b1cc-4df7-b279-f0d6ac73f7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:22.324699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca319e-b1cc-4df7-b279-f0d6ac73f7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:22.325078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca319e-b1cc-4df7-b279-f0d6ac73f7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:22.325466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca319e-b1cc-4df7-b279-f0d6ac73f7a9 admin admin] PATH_INFO: `/v3/users/c6817a6607aa48218bea76d56b0eb1f0` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:22.326957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca319e-b1cc-4df7-b279-f0d6ac73f7a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6817a6607aa48218bea76d56b0eb1f0)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:22.329225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca319e-b1cc-4df7-b279-f0d6ac73f7a9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:22.555208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-52ca319e-b1cc-4df7-b279-f0d6ac73f7a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6817a6607aa48218bea76d56b0eb1f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:22.577904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-52ca319e-b1cc-4df7-b279-f0d6ac73f7a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6817a6607aa48218bea76d56b0eb1f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:22.581581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-52ca319e-b1cc-4df7-b279-f0d6ac73f7a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6817a6607aa48218bea76d56b0eb1f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:22.583741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-52ca319e-b1cc-4df7-b279-f0d6ac73f7a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6817a6607aa48218bea76d56b0eb1f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:22.678458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-52ca319e-b1cc-4df7-b279-f0d6ac73f7a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6817a6607aa48218bea76d56b0eb1f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:22.707904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a4ac5-04d6-4542-8a3f-d4c9ccd832f1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:22.712543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a4ac5-04d6-4542-8a3f-d4c9ccd832f1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:22.712543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a4ac5-04d6-4542-8a3f-d4c9ccd832f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:22.718675 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2821/5606] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:21:22 2022] DELETE /identity/v3/users/c6817a6607aa48218bea76d56b0eb1f0 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:22.735556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-670a4ac5-04d6-4542-8a3f-d4c9ccd832f1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:22.737318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-670a4ac5-04d6-4542-8a3f-d4c9ccd832f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:22.909100 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2786/5607] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:22.927160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca319e-b1cc-4df7-b279-f0d6ac73f7a9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:22.928572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-52ca319e-b1cc-4df7-b279-f0d6ac73f7a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:23.045894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca319e-b1cc-4df7-b279-f0d6ac73f7a9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:23.050366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f46276e-0133-4c45-8289-cc4e6c7d7fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:23.056534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46276e-0133-4c45-8289-cc4e6c7d7fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:23.056534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46276e-0133-4c45-8289-cc4e6c7d7fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:23.056534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f46276e-0133-4c45-8289-cc4e6c7d7fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:23.067187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f46276e-0133-4c45-8289-cc4e6c7d7fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:23.068693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f46276e-0133-4c45-8289-cc4e6c7d7fe5 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:23.087099 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2822/5608] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:21:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:21:23.214343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:INSERT=9,keystone:DELETE=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:23.559129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d316af5-08c6-4114-b780-9a9bd28c21e9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:23.559673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d316af5-08c6-4114-b780-9a9bd28c21e9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:23.560106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d316af5-08c6-4114-b780-9a9bd28c21e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:23.612504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1d316af5-08c6-4114-b780-9a9bd28c21e9 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:23.619524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1d316af5-08c6-4114-b780-9a9bd28c21e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:23.717262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe29f2-f070-49e5-a957-fd1224a21412 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:23.723660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe29f2-f070-49e5-a957-fd1224a21412 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:23.723660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe29f2-f070-49e5-a957-fd1224a21412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:23.744896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6dbe29f2-f070-49e5-a957-fd1224a21412 None None] MFA Rules not processed for user `84f18f11aad344c18bb6b3997370576b`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:23.746667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6dbe29f2-f070-49e5-a957-fd1224a21412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:23.764741 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2787/5609] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:21:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:23.776319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:23.785445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d316af5-08c6-4114-b780-9a9bd28c21e9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:23.787119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1d316af5-08c6-4114-b780-9a9bd28c21e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:23.849527 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2823/5610] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:21:23.938210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d316af5-08c6-4114-b780-9a9bd28c21e9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:23.946070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6d39f-5e48-4369-b82a-f4cf8c27e03d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:23.947028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6d39f-5e48-4369-b82a-f4cf8c27e03d service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:23.947629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6d39f-5e48-4369-b82a-f4cf8c27e03d service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:23.948092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6d39f-5e48-4369-b82a-f4cf8c27e03d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:23.960436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6d39f-5e48-4369-b82a-f4cf8c27e03d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:23.962829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6d39f-5e48-4369-b82a-f4cf8c27e03d service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:23.978408 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2788/5611] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:21:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 21:21:24.285380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe29f2-f070-49e5-a957-fd1224a21412 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:24.289007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6dbe29f2-f070-49e5-a957-fd1224a21412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:24.348332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe29f2-f070-49e5-a957-fd1224a21412 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:24.362795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee54840-166c-4d8d-bb64-84e3db801061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:24.363964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee54840-166c-4d8d-bb64-84e3db801061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:24.367680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee54840-166c-4d8d-bb64-84e3db801061 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:24.368159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee54840-166c-4d8d-bb64-84e3db801061 admin admin] PATH_INFO: `/v3/projects/05724d255c4d44eebb233af520a321f3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:24.371532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee54840-166c-4d8d-bb64-84e3db801061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05724d255c4d44eebb233af520a321f3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:24.373755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee54840-166c-4d8d-bb64-84e3db801061 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:24.585142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bee54840-166c-4d8d-bb64-84e3db801061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05724d255c4d44eebb233af520a321f3', 'request_id': 'req-bee54840-166c-4d8d-bb64-84e3db801061'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:24.610440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bee54840-166c-4d8d-bb64-84e3db801061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05724d255c4d44eebb233af520a321f3', 'request_id': 'req-bee54840-166c-4d8d-bb64-84e3db801061'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:24.695240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bee54840-166c-4d8d-bb64-84e3db801061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05724d255c4d44eebb233af520a321f3', 'request_id': 'req-bee54840-166c-4d8d-bb64-84e3db801061'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:24.717250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bee54840-166c-4d8d-bb64-84e3db801061 admin admin] The token cache is being invalidate because project 05724d255c4d44eebb233af520a321f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:21:24.721465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bee54840-166c-4d8d-bb64-84e3db801061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:24.732076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bee54840-166c-4d8d-bb64-84e3db801061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:24.736110 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2824/5612] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:21:24 2022] DELETE /identity/v3/projects/05724d255c4d44eebb233af520a321f3 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:24.741170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6d39f-5e48-4369-b82a-f4cf8c27e03d service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:24.791157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6d39f-5e48-4369-b82a-f4cf8c27e03d service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:24.919352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee2236b-8661-48af-9181-3c7db5558194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:24.920674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2236b-8661-48af-9181-3c7db5558194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:24.921425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2236b-8661-48af-9181-3c7db5558194 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:24.922291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2236b-8661-48af-9181-3c7db5558194 admin admin] PATH_INFO: `/v3/users/4e291b619e1f4ea3a6e3fa491761f01a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:24.934544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2236b-8661-48af-9181-3c7db5558194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e291b619e1f4ea3a6e3fa491761f01a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:24.934544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee2236b-8661-48af-9181-3c7db5558194 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:25.236688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6ee2236b-8661-48af-9181-3c7db5558194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e291b619e1f4ea3a6e3fa491761f01a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:25.238750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6ee2236b-8661-48af-9181-3c7db5558194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e291b619e1f4ea3a6e3fa491761f01a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:25.275130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6ee2236b-8661-48af-9181-3c7db5558194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e291b619e1f4ea3a6e3fa491761f01a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:25.414984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6ee2236b-8661-48af-9181-3c7db5558194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e291b619e1f4ea3a6e3fa491761f01a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:25.415761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6ee2236b-8661-48af-9181-3c7db5558194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e291b619e1f4ea3a6e3fa491761f01a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:25.477431 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2789/5613] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:21:24 2022] DELETE /identity/v3/users/4e291b619e1f4ea3a6e3fa491761f01a => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:25.758880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:26.130436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:26.876871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee54840-166c-4d8d-bb64-84e3db801061 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:26.878731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bee54840-166c-4d8d-bb64-84e3db801061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:26.907076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee54840-166c-4d8d-bb64-84e3db801061 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:26.989623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30600cf-09d4-4a0c-a147-146431e3377d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:26.990643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30600cf-09d4-4a0c-a147-146431e3377d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:26.990940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30600cf-09d4-4a0c-a147-146431e3377d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:26.991229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30600cf-09d4-4a0c-a147-146431e3377d admin admin] PATH_INFO: `/v3/projects/a1aba7ca20234e6081a4e25fb5d5e15d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:26.995830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30600cf-09d4-4a0c-a147-146431e3377d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1aba7ca20234e6081a4e25fb5d5e15d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:26.998371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30600cf-09d4-4a0c-a147-146431e3377d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:27.242562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f30600cf-09d4-4a0c-a147-146431e3377d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1aba7ca20234e6081a4e25fb5d5e15d', 'request_id': 'req-f30600cf-09d4-4a0c-a147-146431e3377d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:27.258932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f30600cf-09d4-4a0c-a147-146431e3377d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1aba7ca20234e6081a4e25fb5d5e15d', 'request_id': 'req-f30600cf-09d4-4a0c-a147-146431e3377d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:27.339908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f30600cf-09d4-4a0c-a147-146431e3377d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1aba7ca20234e6081a4e25fb5d5e15d', 'request_id': 'req-f30600cf-09d4-4a0c-a147-146431e3377d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:27.368612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f30600cf-09d4-4a0c-a147-146431e3377d admin admin] The token cache is being invalidate because project a1aba7ca20234e6081a4e25fb5d5e15d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:21:27.369263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f30600cf-09d4-4a0c-a147-146431e3377d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:27.371619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f30600cf-09d4-4a0c-a147-146431e3377d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:27.374583 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2825/5614] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:21:26 2022] DELETE /identity/v3/projects/a1aba7ca20234e6081a4e25fb5d5e15d => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:27.380814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cad455-80cf-4de9-a669-8c10e69c7ebc None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:27.381703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cad455-80cf-4de9-a669-8c10e69c7ebc None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:27.382211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cad455-80cf-4de9-a669-8c10e69c7ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:27.417812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e2cad455-80cf-4de9-a669-8c10e69c7ebc None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:27.419271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e2cad455-80cf-4de9-a669-8c10e69c7ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:27.550181 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2790/5615] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:27.551404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094c902-42fd-48f9-8ed8-777b36009a5b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:27.551830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094c902-42fd-48f9-8ed8-777b36009a5b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:27.552047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094c902-42fd-48f9-8ed8-777b36009a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:27.596942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1094c902-42fd-48f9-8ed8-777b36009a5b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:27.598370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1094c902-42fd-48f9-8ed8-777b36009a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:27.627153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cad455-80cf-4de9-a669-8c10e69c7ebc None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:27.637872 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2826/5616] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:27.663306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cad455-80cf-4de9-a669-8c10e69c7ebc None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:27.681620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51362809-1176-49a3-a353-f864315c8235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:27.682774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51362809-1176-49a3-a353-f864315c8235 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:27.683272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51362809-1176-49a3-a353-f864315c8235 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:27.683759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51362809-1176-49a3-a353-f864315c8235 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:27.684729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51362809-1176-49a3-a353-f864315c8235 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:21:27.685262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51362809-1176-49a3-a353-f864315c8235 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:27.687274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51362809-1176-49a3-a353-f864315c8235 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:27.737876 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2791/5617] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:21:27.747216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094c902-42fd-48f9-8ed8-777b36009a5b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:27.752272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094c902-42fd-48f9-8ed8-777b36009a5b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:27.816784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c0f85-f3d6-40b0-a4a8-4c462569654d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:27.817550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c0f85-f3d6-40b0-a4a8-4c462569654d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:27.817975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c0f85-f3d6-40b0-a4a8-4c462569654d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:27.818461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c0f85-f3d6-40b0-a4a8-4c462569654d admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:27.830695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c0f85-f3d6-40b0-a4a8-4c462569654d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:27.837177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c0f85-f3d6-40b0-a4a8-4c462569654d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:27.995921 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2827/5618] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:21:27 2022] POST /identity/v3/projects => generated 383 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:21:28.010002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51362809-1176-49a3-a353-f864315c8235 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:28.031508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51362809-1176-49a3-a353-f864315c8235 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:28.204386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5530a4-4ae9-4910-9a23-c74a07f56b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:28.205559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5530a4-4ae9-4910-9a23-c74a07f56b8f admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:28.206046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5530a4-4ae9-4910-9a23-c74a07f56b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:28.206500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5530a4-4ae9-4910-9a23-c74a07f56b8f admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:28.213302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5530a4-4ae9-4910-9a23-c74a07f56b8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:28.216374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5530a4-4ae9-4910-9a23-c74a07f56b8f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:28.235280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:28.310506 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2792/5619] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:21:28 2022] POST /identity/v3/users => generated 292 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:21:28.315543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c0f85-f3d6-40b0-a4a8-4c462569654d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:28.335765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c0f85-f3d6-40b0-a4a8-4c462569654d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:28.349743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48136508-fb44-4f82-b63f-edbe76f7bcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:28.351120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48136508-fb44-4f82-b63f-edbe76f7bcfc admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:28.353498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48136508-fb44-4f82-b63f-edbe76f7bcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:28.353796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48136508-fb44-4f82-b63f-edbe76f7bcfc admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:28.357706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48136508-fb44-4f82-b63f-edbe76f7bcfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:28.359654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48136508-fb44-4f82-b63f-edbe76f7bcfc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:28.390495 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2828/5620] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:21:28.394683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5530a4-4ae9-4910-9a23-c74a07f56b8f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:28.430962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5530a4-4ae9-4910-9a23-c74a07f56b8f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:28.445544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5fc93f-e5bf-4ee8-a1c2-3823fdfc687e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:28.449704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fc93f-e5bf-4ee8-a1c2-3823fdfc687e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:28.451307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fc93f-e5bf-4ee8-a1c2-3823fdfc687e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:28.454746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5fc93f-e5bf-4ee8-a1c2-3823fdfc687e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:28.455913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5fc93f-e5bf-4ee8-a1c2-3823fdfc687e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:28.457716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5fc93f-e5bf-4ee8-a1c2-3823fdfc687e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:28.488248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48136508-fb44-4f82-b63f-edbe76f7bcfc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:28.495057 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2793/5621] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:21:28.499932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48136508-fb44-4f82-b63f-edbe76f7bcfc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:28.517097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a4170-a7ab-4e6b-a5f1-c8f0da52e2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:28.518429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a4170-a7ab-4e6b-a5f1-c8f0da52e2e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:28.518953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a4170-a7ab-4e6b-a5f1-c8f0da52e2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:28.519461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a4170-a7ab-4e6b-a5f1-c8f0da52e2e9 admin admin] PATH_INFO: `/v3/projects/7a928180249f4221816af07f13ae903d/users/3130f81184394026a343f9c317f3240d/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:28.548125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a4170-a7ab-4e6b-a5f1-c8f0da52e2e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a928180249f4221816af07f13ae903d, user_id=3130f81184394026a343f9c317f3240d, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:28.551326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a4170-a7ab-4e6b-a5f1-c8f0da52e2e9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:28.667819 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2829/5622] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:21:28 2022] PUT /identity/v3/projects/7a928180249f4221816af07f13ae903d/users/3130f81184394026a343f9c317f3240d/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:28.679663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22252090-8113-47bf-a69f-cd4615787047 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:28.680433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22252090-8113-47bf-a69f-cd4615787047 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:28.680875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22252090-8113-47bf-a69f-cd4615787047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:28.747433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-22252090-8113-47bf-a69f-cd4615787047 None None] MFA Rules not processed for user `3130f81184394026a343f9c317f3240d`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:28.749037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-22252090-8113-47bf-a69f-cd4615787047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:28.907548 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2794/5623] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:28.911399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a4170-a7ab-4e6b-a5f1-c8f0da52e2e9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:29.115876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a4170-a7ab-4e6b-a5f1-c8f0da52e2e9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:29.133923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8f355-53ae-48c0-9bdd-4bdd81f5ee59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:29.140029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8f355-53ae-48c0-9bdd-4bdd81f5ee59 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:29.140029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8f355-53ae-48c0-9bdd-4bdd81f5ee59 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:29.140029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd8f355-53ae-48c0-9bdd-4bdd81f5ee59 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:29.140029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd8f355-53ae-48c0-9bdd-4bdd81f5ee59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:29.147702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd8f355-53ae-48c0-9bdd-4bdd81f5ee59 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:29.304776 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2830/5624] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:21:28 2022] POST /identity/v3/projects => generated 377 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:21:29.309128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22252090-8113-47bf-a69f-cd4615787047 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:29.327224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22252090-8113-47bf-a69f-cd4615787047 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:29.399027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:29.454307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc150c4d-c2bf-4c6f-bb01-a5dedb615f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:29.455172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc150c4d-c2bf-4c6f-bb01-a5dedb615f8b admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:29.455425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc150c4d-c2bf-4c6f-bb01-a5dedb615f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:29.455713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc150c4d-c2bf-4c6f-bb01-a5dedb615f8b admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:29.460276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc150c4d-c2bf-4c6f-bb01-a5dedb615f8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:29.460276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc150c4d-c2bf-4c6f-bb01-a5dedb615f8b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:29.593881 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2795/5625] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:21:29 2022] POST /identity/v3/users => generated 295 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:21:29.595130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8f355-53ae-48c0-9bdd-4bdd81f5ee59 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:29.618913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd8f355-53ae-48c0-9bdd-4bdd81f5ee59 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:29.630362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b939a3a-7518-4538-b7a8-fab0961f79be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:29.631305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b939a3a-7518-4538-b7a8-fab0961f79be admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:29.635711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b939a3a-7518-4538-b7a8-fab0961f79be admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:29.636654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b939a3a-7518-4538-b7a8-fab0961f79be admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:29.637935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b939a3a-7518-4538-b7a8-fab0961f79be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:29.640853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b939a3a-7518-4538-b7a8-fab0961f79be admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:29.657699 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2831/5626] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:21:29.662380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc150c4d-c2bf-4c6f-bb01-a5dedb615f8b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:29.683470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc150c4d-c2bf-4c6f-bb01-a5dedb615f8b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:29.691399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79c46f-d177-45e2-99e2-4eac73655dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:29.692789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79c46f-d177-45e2-99e2-4eac73655dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:29.694423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79c46f-d177-45e2-99e2-4eac73655dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:29.695023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79c46f-d177-45e2-99e2-4eac73655dc3 admin admin] PATH_INFO: `/v3/projects/4ee3a576c25641689037a19a60360e3e/users/99ad8f2521154190b986d6d65c2a2ab7/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:29.739202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79c46f-d177-45e2-99e2-4eac73655dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ee3a576c25641689037a19a60360e3e, user_id=99ad8f2521154190b986d6d65c2a2ab7, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:29.742990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79c46f-d177-45e2-99e2-4eac73655dc3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:29.799709 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2796/5627] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:21:29 2022] PUT /identity/v3/projects/4ee3a576c25641689037a19a60360e3e/users/99ad8f2521154190b986d6d65c2a2ab7/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:29.803718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b939a3a-7518-4538-b7a8-fab0961f79be admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:29.824761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b939a3a-7518-4538-b7a8-fab0961f79be admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:29.983169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee23f77-8b5c-4d4f-86c4-af1856952cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:29.990769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee23f77-8b5c-4d4f-86c4-af1856952cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:29.995965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee23f77-8b5c-4d4f-86c4-af1856952cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:29.995965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee23f77-8b5c-4d4f-86c4-af1856952cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:29.995965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee23f77-8b5c-4d4f-86c4-af1856952cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:29.996534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee23f77-8b5c-4d4f-86c4-af1856952cc0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:30.024443 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2832/5628] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:21:30.029903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79c46f-d177-45e2-99e2-4eac73655dc3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:30.045395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:30.054831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79c46f-d177-45e2-99e2-4eac73655dc3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:30.066962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c97a91-addc-41d5-8f98-8a0de246245e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:30.068477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c97a91-addc-41d5-8f98-8a0de246245e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:30.068477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c97a91-addc-41d5-8f98-8a0de246245e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:30.068691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c97a91-addc-41d5-8f98-8a0de246245e admin admin] PATH_INFO: `/v3/projects/4ee3a576c25641689037a19a60360e3e/users/99ad8f2521154190b986d6d65c2a2ab7/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:30.078371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c97a91-addc-41d5-8f98-8a0de246245e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ee3a576c25641689037a19a60360e3e, user_id=99ad8f2521154190b986d6d65c2a2ab7, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:30.081116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c97a91-addc-41d5-8f98-8a0de246245e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:30.125083 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2797/5629] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:21:30 2022] PUT /identity/v3/projects/4ee3a576c25641689037a19a60360e3e/users/99ad8f2521154190b986d6d65c2a2ab7/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:30.127277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c9322-e700-4a66-9372-829aa8b5fcf5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:30.127575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c9322-e700-4a66-9372-829aa8b5fcf5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:30.127861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c9322-e700-4a66-9372-829aa8b5fcf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:30.196203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ef7c9322-e700-4a66-9372-829aa8b5fcf5 None None] MFA Rules not processed for user `99ad8f2521154190b986d6d65c2a2ab7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:30.198140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ef7c9322-e700-4a66-9372-829aa8b5fcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:30.329887 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2833/5630] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:21:30.343434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c97a91-addc-41d5-8f98-8a0de246245e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:30.347765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-49c97a91-addc-41d5-8f98-8a0de246245e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:30.446965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c97a91-addc-41d5-8f98-8a0de246245e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:30.463948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf18f6-820f-40c8-ae9a-572f43e12f68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:30.464728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf18f6-820f-40c8-ae9a-572f43e12f68 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:30.465730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf18f6-820f-40c8-ae9a-572f43e12f68 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:30.465730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abf18f6-820f-40c8-ae9a-572f43e12f68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:30.479089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf18f6-820f-40c8-ae9a-572f43e12f68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:30.480320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abf18f6-820f-40c8-ae9a-572f43e12f68 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:30.506799 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2798/5631] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:21:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:21:30.518926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:30.752028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c9322-e700-4a66-9372-829aa8b5fcf5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:30.753516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ef7c9322-e700-4a66-9372-829aa8b5fcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:30.770696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c9322-e700-4a66-9372-829aa8b5fcf5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:30.775521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e2d40-4f70-4464-9b4b-160abdddd431 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:30.782932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e2d40-4f70-4464-9b4b-160abdddd431 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:30.782932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e2d40-4f70-4464-9b4b-160abdddd431 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:30.782932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e2d40-4f70-4464-9b4b-160abdddd431 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:30.894259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e2d40-4f70-4464-9b4b-160abdddd431 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:30.897404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e2d40-4f70-4464-9b4b-160abdddd431 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:30.920523 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2834/5632] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:21:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:21:31.698012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf18f6-820f-40c8-ae9a-572f43e12f68 service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:31.700102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2abf18f6-820f-40c8-ae9a-572f43e12f68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:31.959377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abf18f6-820f-40c8-ae9a-572f43e12f68 service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:31.988751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3722b05-a422-4b55-a3e1-3763dbe29f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:31.992009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3722b05-a422-4b55-a3e1-3763dbe29f0f service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:31.992299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3722b05-a422-4b55-a3e1-3763dbe29f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:31.994800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3722b05-a422-4b55-a3e1-3763dbe29f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:32.011716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f3722b05-a422-4b55-a3e1-3763dbe29f0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:32.234144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3722b05-a422-4b55-a3e1-3763dbe29f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:32.235412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3722b05-a422-4b55-a3e1-3763dbe29f0f service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:32.273635 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2799/5633] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:21:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 576 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:21:32.293634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e2d40-4f70-4464-9b4b-160abdddd431 service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:32.315931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e2d40-4f70-4464-9b4b-160abdddd431 service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:32.345371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec5db4-3f98-478f-9d08-a02de567a0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:32.350082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec5db4-3f98-478f-9d08-a02de567a0ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:32.350082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec5db4-3f98-478f-9d08-a02de567a0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:32.350082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ec5db4-3f98-478f-9d08-a02de567a0ba admin admin] PATH_INFO: `/v3/users/3130f81184394026a343f9c317f3240d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:32.350082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec5db4-3f98-478f-9d08-a02de567a0ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=3130f81184394026a343f9c317f3240d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:32.355106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ec5db4-3f98-478f-9d08-a02de567a0ba admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:32.422759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:32.688876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b6ec5db4-3f98-478f-9d08-a02de567a0ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3130f81184394026a343f9c317f3240d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:32.703501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b6ec5db4-3f98-478f-9d08-a02de567a0ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3130f81184394026a343f9c317f3240d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:32.704657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b6ec5db4-3f98-478f-9d08-a02de567a0ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3130f81184394026a343f9c317f3240d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:32.705786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b6ec5db4-3f98-478f-9d08-a02de567a0ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3130f81184394026a343f9c317f3240d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:32.860988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b6ec5db4-3f98-478f-9d08-a02de567a0ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3130f81184394026a343f9c317f3240d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:32.905176 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2835/5634] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:21:32 2022] DELETE /identity/v3/users/3130f81184394026a343f9c317f3240d => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:32.910659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8da86-fd8a-48b7-bfe7-2561386c1d03 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:32.910659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8da86-fd8a-48b7-bfe7-2561386c1d03 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:32.911234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8da86-fd8a-48b7-bfe7-2561386c1d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:32.952056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1be8da86-fd8a-48b7-bfe7-2561386c1d03 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:32.954502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1be8da86-fd8a-48b7-bfe7-2561386c1d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:33.034860 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2800/5635] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:33.054971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec5db4-3f98-478f-9d08-a02de567a0ba admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:33.070525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b6ec5db4-3f98-478f-9d08-a02de567a0ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:33.208865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:33.325043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ec5db4-3f98-478f-9d08-a02de567a0ba admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:33.345334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96470766-6359-4f3f-9ad6-fef2fabea483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:33.346104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96470766-6359-4f3f-9ad6-fef2fabea483 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:33.346414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96470766-6359-4f3f-9ad6-fef2fabea483 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:33.346697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96470766-6359-4f3f-9ad6-fef2fabea483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:33.383694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96470766-6359-4f3f-9ad6-fef2fabea483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:33.385462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96470766-6359-4f3f-9ad6-fef2fabea483 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:33.423966 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2836/5636] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:21:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:21:33.775463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:34.249889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:34.879029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:35.003672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:35.504184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8da86-fd8a-48b7-bfe7-2561386c1d03 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:35.505638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1be8da86-fd8a-48b7-bfe7-2561386c1d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:35.529146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8da86-fd8a-48b7-bfe7-2561386c1d03 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:35.547533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:35.548743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:35.549187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:35.549634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0 admin admin] PATH_INFO: `/v3/projects/7a928180249f4221816af07f13ae903d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:35.553472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a928180249f4221816af07f13ae903d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:35.562714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:35.746374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a928180249f4221816af07f13ae903d', 'request_id': 'req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:35.757705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a928180249f4221816af07f13ae903d', 'request_id': 'req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:35.798189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:35.851300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a928180249f4221816af07f13ae903d', 'request_id': 'req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:35.868467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0 admin admin] The token cache is being invalidate because project 7a928180249f4221816af07f13ae903d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:21:35.868876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:35.869812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:35.875315 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2801/5637] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:21:35 2022] DELETE /identity/v3/projects/7a928180249f4221816af07f13ae903d => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:35.876274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96470766-6359-4f3f-9ad6-fef2fabea483 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:35.888899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96470766-6359-4f3f-9ad6-fef2fabea483 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:36.134878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da306b1-f374-464a-a322-267a24f729f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:36.138970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da306b1-f374-464a-a322-267a24f729f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:36.139269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da306b1-f374-464a-a322-267a24f729f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:36.139672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da306b1-f374-464a-a322-267a24f729f0 admin admin] PATH_INFO: `/v3/users/99ad8f2521154190b986d6d65c2a2ab7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:36.147845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da306b1-f374-464a-a322-267a24f729f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99ad8f2521154190b986d6d65c2a2ab7)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:36.153319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da306b1-f374-464a-a322-267a24f729f0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:36.156489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:UPDATE=1,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:36.459322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3da306b1-f374-464a-a322-267a24f729f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99ad8f2521154190b986d6d65c2a2ab7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:36.528596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3da306b1-f374-464a-a322-267a24f729f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99ad8f2521154190b986d6d65c2a2ab7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:36.537340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3da306b1-f374-464a-a322-267a24f729f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99ad8f2521154190b986d6d65c2a2ab7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:36.539217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3da306b1-f374-464a-a322-267a24f729f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99ad8f2521154190b986d6d65c2a2ab7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:36.660753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3da306b1-f374-464a-a322-267a24f729f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99ad8f2521154190b986d6d65c2a2ab7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:36.706707 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2837/5638] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:21:35 2022] DELETE /identity/v3/users/99ad8f2521154190b986d6d65c2a2ab7 => generated 0 bytes in 831 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:37.975440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:38.502941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:38.505111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:38.528878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f4bfe-c3c0-49ba-99f4-93266f4e2ec0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:38.658293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f823eb-7f83-4a11-a203-7fd1e63b6dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:38.659416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f823eb-7f83-4a11-a203-7fd1e63b6dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:38.664610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f823eb-7f83-4a11-a203-7fd1e63b6dea admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:38.664610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f823eb-7f83-4a11-a203-7fd1e63b6dea admin admin] PATH_INFO: `/v3/projects/4ee3a576c25641689037a19a60360e3e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:38.664610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f823eb-7f83-4a11-a203-7fd1e63b6dea admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ee3a576c25641689037a19a60360e3e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:38.664610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f823eb-7f83-4a11-a203-7fd1e63b6dea admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:38.823148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-53f823eb-7f83-4a11-a203-7fd1e63b6dea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ee3a576c25641689037a19a60360e3e', 'request_id': 'req-53f823eb-7f83-4a11-a203-7fd1e63b6dea'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:38.831689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-53f823eb-7f83-4a11-a203-7fd1e63b6dea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ee3a576c25641689037a19a60360e3e', 'request_id': 'req-53f823eb-7f83-4a11-a203-7fd1e63b6dea'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:38.877398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-53f823eb-7f83-4a11-a203-7fd1e63b6dea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ee3a576c25641689037a19a60360e3e', 'request_id': 'req-53f823eb-7f83-4a11-a203-7fd1e63b6dea'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:38.895251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-53f823eb-7f83-4a11-a203-7fd1e63b6dea admin admin] The token cache is being invalidate because project 4ee3a576c25641689037a19a60360e3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:21:38.895833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-53f823eb-7f83-4a11-a203-7fd1e63b6dea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:38.896697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-53f823eb-7f83-4a11-a203-7fd1e63b6dea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:38.898499 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2802/5639] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:21:38 2022] DELETE /identity/v3/projects/4ee3a576c25641689037a19a60360e3e => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:38.903835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c9982-c400-40a8-92bb-339b9526c411 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:38.905804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c9982-c400-40a8-92bb-339b9526c411 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:38.905804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c9982-c400-40a8-92bb-339b9526c411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:38.925950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7e8c9982-c400-40a8-92bb-339b9526c411 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:38.927677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7e8c9982-c400-40a8-92bb-339b9526c411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:38.953150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:39.037033 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2838/5640] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:39.040243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf57b7-929b-4541-8b28-d31c323a1c33 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:39.040642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf57b7-929b-4541-8b28-d31c323a1c33 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:39.040999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf57b7-929b-4541-8b28-d31c323a1c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:39.062048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4bbf57b7-929b-4541-8b28-d31c323a1c33 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:39.063365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4bbf57b7-929b-4541-8b28-d31c323a1c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:39.078000 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2803/5641] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:39.082133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c9982-c400-40a8-92bb-339b9526c411 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:39.094907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c9982-c400-40a8-92bb-339b9526c411 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:39.102576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb5bd5-941f-4516-90d5-d93c62ca649b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:39.103428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb5bd5-941f-4516-90d5-d93c62ca649b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:39.104635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb5bd5-941f-4516-90d5-d93c62ca649b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:39.106254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beb5bd5-941f-4516-90d5-d93c62ca649b admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:39.107447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb5bd5-941f-4516-90d5-d93c62ca649b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:21:39.107982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb5bd5-941f-4516-90d5-d93c62ca649b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:39.109991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beb5bd5-941f-4516-90d5-d93c62ca649b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:39.123838 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2839/5642] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:21:39.129118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf57b7-929b-4541-8b28-d31c323a1c33 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:39.153488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf57b7-929b-4541-8b28-d31c323a1c33 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:39.158607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9e100-ea55-4c54-823d-3f11b9ad25cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:39.159275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9e100-ea55-4c54-823d-3f11b9ad25cc admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:39.159558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9e100-ea55-4c54-823d-3f11b9ad25cc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:39.159832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9e100-ea55-4c54-823d-3f11b9ad25cc admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:39.160609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9e100-ea55-4c54-823d-3f11b9ad25cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:39.163880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9e100-ea55-4c54-823d-3f11b9ad25cc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:39.274264 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2804/5643] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:21:39 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:21:39.278919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb5bd5-941f-4516-90d5-d93c62ca649b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:39.294244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beb5bd5-941f-4516-90d5-d93c62ca649b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:39.370495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe5da1-e1f9-4ea6-88a6-d9a35e73e813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:39.371376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe5da1-e1f9-4ea6-88a6-d9a35e73e813 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:39.371982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe5da1-e1f9-4ea6-88a6-d9a35e73e813 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:39.372395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe5da1-e1f9-4ea6-88a6-d9a35e73e813 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:39.373224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe5da1-e1f9-4ea6-88a6-d9a35e73e813 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:39.375865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe5da1-e1f9-4ea6-88a6-d9a35e73e813 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:39.502852 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2840/5644] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:21:39 2022] POST /identity/v3/users => generated 287 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:21:39.505420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9e100-ea55-4c54-823d-3f11b9ad25cc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:39.517009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9e100-ea55-4c54-823d-3f11b9ad25cc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:39.523042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9172f-b3f6-4a57-acb2-923164595b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:39.523726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9172f-b3f6-4a57-acb2-923164595b4e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:39.524016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9172f-b3f6-4a57-acb2-923164595b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:39.524298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d9172f-b3f6-4a57-acb2-923164595b4e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:39.525046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9172f-b3f6-4a57-acb2-923164595b4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:39.527230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d9172f-b3f6-4a57-acb2-923164595b4e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:39.538769 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2805/5645] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:21:39.542795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe5da1-e1f9-4ea6-88a6-d9a35e73e813 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:39.555451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe5da1-e1f9-4ea6-88a6-d9a35e73e813 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:39.561489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3455e-3b68-43c7-a9b5-a1e6770ae409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:39.562581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3455e-3b68-43c7-a9b5-a1e6770ae409 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:39.563445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3455e-3b68-43c7-a9b5-a1e6770ae409 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:39.563743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3455e-3b68-43c7-a9b5-a1e6770ae409 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:39.564498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3455e-3b68-43c7-a9b5-a1e6770ae409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:39.566162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3455e-3b68-43c7-a9b5-a1e6770ae409 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:39.577779 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2841/5646] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:21:39.582210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9172f-b3f6-4a57-acb2-923164595b4e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:39.595645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d9172f-b3f6-4a57-acb2-923164595b4e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:39.600988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241361bd-a61b-4db5-aa4f-20d1869a67af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:39.602569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241361bd-a61b-4db5-aa4f-20d1869a67af admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:39.603411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241361bd-a61b-4db5-aa4f-20d1869a67af admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:39.603851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241361bd-a61b-4db5-aa4f-20d1869a67af admin admin] PATH_INFO: `/v3/projects/4ee7d7d4ff4a456fb4223df1c109722f/users/254031c1b33a42c1b72cbfae52f10f24/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:39.627965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241361bd-a61b-4db5-aa4f-20d1869a67af admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ee7d7d4ff4a456fb4223df1c109722f, user_id=254031c1b33a42c1b72cbfae52f10f24, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:39.637439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241361bd-a61b-4db5-aa4f-20d1869a67af admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:39.706525 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2806/5647] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:21:39 2022] PUT /identity/v3/projects/4ee7d7d4ff4a456fb4223df1c109722f/users/254031c1b33a42c1b72cbfae52f10f24/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:39.717337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b9277-ef68-4ee0-8bd9-6fed65b91679 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:39.717878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b9277-ef68-4ee0-8bd9-6fed65b91679 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:39.718270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b9277-ef68-4ee0-8bd9-6fed65b91679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:39.748382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8d7b9277-ef68-4ee0-8bd9-6fed65b91679 None None] MFA Rules not processed for user `254031c1b33a42c1b72cbfae52f10f24`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:39.750458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8d7b9277-ef68-4ee0-8bd9-6fed65b91679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:39.776552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:39.909207 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2842/5648] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:39.916254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241361bd-a61b-4db5-aa4f-20d1869a67af admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:39.918905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-241361bd-a61b-4db5-aa4f-20d1869a67af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:40.040676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241361bd-a61b-4db5-aa4f-20d1869a67af admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:40.053483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b256b80-7c7e-4ebd-97bd-963a096db3f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:40.054452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b256b80-7c7e-4ebd-97bd-963a096db3f9 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:40.054882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b256b80-7c7e-4ebd-97bd-963a096db3f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:40.055292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b256b80-7c7e-4ebd-97bd-963a096db3f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:40.083823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b256b80-7c7e-4ebd-97bd-963a096db3f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:40.085321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b256b80-7c7e-4ebd-97bd-963a096db3f9 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:40.102109 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2807/5649] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:21:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:21:40.126966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b9277-ef68-4ee0-8bd9-6fed65b91679 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:40.227155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b9277-ef68-4ee0-8bd9-6fed65b91679 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:40.234977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c1b378-f51e-4ecd-8ac8-5081a2f457bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:40.236506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1b378-f51e-4ecd-8ac8-5081a2f457bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:40.236797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1b378-f51e-4ecd-8ac8-5081a2f457bc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:40.237080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c1b378-f51e-4ecd-8ac8-5081a2f457bc admin admin] PATH_INFO: `/v3/users/254031c1b33a42c1b72cbfae52f10f24` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:40.238646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c1b378-f51e-4ecd-8ac8-5081a2f457bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=254031c1b33a42c1b72cbfae52f10f24)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:40.240746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c1b378-f51e-4ecd-8ac8-5081a2f457bc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:40.575384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b6c1b378-f51e-4ecd-8ac8-5081a2f457bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '254031c1b33a42c1b72cbfae52f10f24'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:40.575484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:40.575960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b6c1b378-f51e-4ecd-8ac8-5081a2f457bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '254031c1b33a42c1b72cbfae52f10f24'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:40.576700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b6c1b378-f51e-4ecd-8ac8-5081a2f457bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '254031c1b33a42c1b72cbfae52f10f24'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:40.577961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b6c1b378-f51e-4ecd-8ac8-5081a2f457bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '254031c1b33a42c1b72cbfae52f10f24'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:40.700625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b6c1b378-f51e-4ecd-8ac8-5081a2f457bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '254031c1b33a42c1b72cbfae52f10f24'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:40.737820 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2843/5650] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:21:40 2022] DELETE /identity/v3/users/254031c1b33a42c1b72cbfae52f10f24 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:40.742113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c4781-998e-4a67-9fdb-fc528fb76821 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:40.742571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c4781-998e-4a67-9fdb-fc528fb76821 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:40.743180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c4781-998e-4a67-9fdb-fc528fb76821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:40.780004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2f1c4781-998e-4a67-9fdb-fc528fb76821 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:40.782061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2f1c4781-998e-4a67-9fdb-fc528fb76821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:40.878914 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2808/5651] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:40.902327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c1b378-f51e-4ecd-8ac8-5081a2f457bc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:40.906757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b6c1b378-f51e-4ecd-8ac8-5081a2f457bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:40.990373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c1b378-f51e-4ecd-8ac8-5081a2f457bc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:40.994350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddad9bab-cc95-4173-ac22-9307ef06b3b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:40.995117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad9bab-cc95-4173-ac22-9307ef06b3b1 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:40.995538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad9bab-cc95-4173-ac22-9307ef06b3b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:40.995936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad9bab-cc95-4173-ac22-9307ef06b3b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:41.008846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad9bab-cc95-4173-ac22-9307ef06b3b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:41.012516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddad9bab-cc95-4173-ac22-9307ef06b3b1 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:41.022479 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2844/5652] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:21:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:21:41.256103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:42.098339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c4781-998e-4a67-9fdb-fc528fb76821 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:42.099831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2f1c4781-998e-4a67-9fdb-fc528fb76821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:42.113624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c4781-998e-4a67-9fdb-fc528fb76821 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:42.118586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94347a38-d4ce-464e-8882-8b1ad5d59b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:42.119458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94347a38-d4ce-464e-8882-8b1ad5d59b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:42.119746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94347a38-d4ce-464e-8882-8b1ad5d59b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:42.120029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94347a38-d4ce-464e-8882-8b1ad5d59b04 admin admin] PATH_INFO: `/v3/projects/4ee7d7d4ff4a456fb4223df1c109722f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:42.121228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94347a38-d4ce-464e-8882-8b1ad5d59b04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ee7d7d4ff4a456fb4223df1c109722f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:42.123248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94347a38-d4ce-464e-8882-8b1ad5d59b04 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:42.230472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94347a38-d4ce-464e-8882-8b1ad5d59b04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ee7d7d4ff4a456fb4223df1c109722f', 'request_id': 'req-94347a38-d4ce-464e-8882-8b1ad5d59b04'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:42.237408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94347a38-d4ce-464e-8882-8b1ad5d59b04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ee7d7d4ff4a456fb4223df1c109722f', 'request_id': 'req-94347a38-d4ce-464e-8882-8b1ad5d59b04'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:42.306235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94347a38-d4ce-464e-8882-8b1ad5d59b04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ee7d7d4ff4a456fb4223df1c109722f', 'request_id': 'req-94347a38-d4ce-464e-8882-8b1ad5d59b04'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:42.321832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94347a38-d4ce-464e-8882-8b1ad5d59b04 admin admin] The token cache is being invalidate because project 4ee7d7d4ff4a456fb4223df1c109722f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:21:42.322499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94347a38-d4ce-464e-8882-8b1ad5d59b04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:42.323460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94347a38-d4ce-464e-8882-8b1ad5d59b04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:42.325114 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2809/5653] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:21:42 2022] DELETE /identity/v3/projects/4ee7d7d4ff4a456fb4223df1c109722f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:42.331497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46799512-dc4a-4934-8c5e-834146a0a635 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:42.331995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46799512-dc4a-4934-8c5e-834146a0a635 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:42.332312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46799512-dc4a-4934-8c5e-834146a0a635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:42.335101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:42.354050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-46799512-dc4a-4934-8c5e-834146a0a635 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:42.355318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-46799512-dc4a-4934-8c5e-834146a0a635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:42.429636 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2845/5654] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:42.431618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc8b35-ddc5-475a-855b-ec87730d85d4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:42.431909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc8b35-ddc5-475a-855b-ec87730d85d4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:42.432185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc8b35-ddc5-475a-855b-ec87730d85d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:42.455955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-12fc8b35-ddc5-475a-855b-ec87730d85d4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:42.456498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-12fc8b35-ddc5-475a-855b-ec87730d85d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:42.472678 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2810/5655] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:21:42.479049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46799512-dc4a-4934-8c5e-834146a0a635 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:42.486011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46799512-dc4a-4934-8c5e-834146a0a635 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:42.493366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11239f16-f324-41f2-b0d4-1ebb6cd18d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:42.494517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11239f16-f324-41f2-b0d4-1ebb6cd18d5b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:42.495015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11239f16-f324-41f2-b0d4-1ebb6cd18d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:42.495305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11239f16-f324-41f2-b0d4-1ebb6cd18d5b admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:42.496483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11239f16-f324-41f2-b0d4-1ebb6cd18d5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:21:42.496927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11239f16-f324-41f2-b0d4-1ebb6cd18d5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:42.498742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11239f16-f324-41f2-b0d4-1ebb6cd18d5b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:42.514024 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2846/5656] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:21:42.517440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc8b35-ddc5-475a-855b-ec87730d85d4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:42.527379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc8b35-ddc5-475a-855b-ec87730d85d4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:42.532178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59639c12-fee4-44fc-ba9b-0ad0155d566d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:42.532935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59639c12-fee4-44fc-ba9b-0ad0155d566d admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:42.533291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59639c12-fee4-44fc-ba9b-0ad0155d566d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:42.533572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59639c12-fee4-44fc-ba9b-0ad0155d566d admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:42.534314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59639c12-fee4-44fc-ba9b-0ad0155d566d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:42.538315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59639c12-fee4-44fc-ba9b-0ad0155d566d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:42.618970 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2811/5657] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:21:42 2022] POST /identity/v3/projects => generated 361 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:21:42.623142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11239f16-f324-41f2-b0d4-1ebb6cd18d5b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:42.647946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11239f16-f324-41f2-b0d4-1ebb6cd18d5b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:42.736457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fac9a-1a68-4c0a-bcbf-848a38889b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:42.738245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fac9a-1a68-4c0a-bcbf-848a38889b38 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:42.738714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fac9a-1a68-4c0a-bcbf-848a38889b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:42.739264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317fac9a-1a68-4c0a-bcbf-848a38889b38 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:42.740225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317fac9a-1a68-4c0a-bcbf-848a38889b38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:42.742803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317fac9a-1a68-4c0a-bcbf-848a38889b38 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:42.845755 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2847/5658] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:21:42 2022] POST /identity/v3/users => generated 281 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:21:42.848564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59639c12-fee4-44fc-ba9b-0ad0155d566d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:42.869017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59639c12-fee4-44fc-ba9b-0ad0155d566d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:42.892480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e57072-26e2-404c-be9a-6d580c7d8a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:42.895560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57072-26e2-404c-be9a-6d580c7d8a99 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:42.895866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57072-26e2-404c-be9a-6d580c7d8a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:42.896150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e57072-26e2-404c-be9a-6d580c7d8a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:42.902548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e57072-26e2-404c-be9a-6d580c7d8a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:42.919195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e57072-26e2-404c-be9a-6d580c7d8a99 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:42.954327 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2812/5659] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:21:42.956410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fac9a-1a68-4c0a-bcbf-848a38889b38 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:42.969694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317fac9a-1a68-4c0a-bcbf-848a38889b38 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:42.980321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46509c-91f7-4fcd-b364-c87cd6212865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:42.981369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46509c-91f7-4fcd-b364-c87cd6212865 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:42.981805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46509c-91f7-4fcd-b364-c87cd6212865 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:42.982344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46509c-91f7-4fcd-b364-c87cd6212865 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:42.983988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46509c-91f7-4fcd-b364-c87cd6212865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:42.985862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f46509c-91f7-4fcd-b364-c87cd6212865 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:42.998821 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2848/5660] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:21:43.005374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e57072-26e2-404c-be9a-6d580c7d8a99 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:43.023514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e57072-26e2-404c-be9a-6d580c7d8a99 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:43.030851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74f211-9915-43dc-853c-e6cab0dba654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:43.032049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74f211-9915-43dc-853c-e6cab0dba654 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:43.032345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74f211-9915-43dc-853c-e6cab0dba654 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:43.032677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d74f211-9915-43dc-853c-e6cab0dba654 admin admin] PATH_INFO: `/v3/projects/415da79d13fb4cf98c4e83c7969212cb/users/6f6252d140f54ce7871de73110b41fc9/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:43.054628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d74f211-9915-43dc-853c-e6cab0dba654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=415da79d13fb4cf98c4e83c7969212cb, user_id=6f6252d140f54ce7871de73110b41fc9, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:43.059081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d74f211-9915-43dc-853c-e6cab0dba654 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:43.147919 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2813/5661] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:21:43 2022] PUT /identity/v3/projects/415da79d13fb4cf98c4e83c7969212cb/users/6f6252d140f54ce7871de73110b41fc9/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:43.160988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08224c-3961-476a-b0b6-13e0b30fc4b0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:43.164966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08224c-3961-476a-b0b6-13e0b30fc4b0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:43.165358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08224c-3961-476a-b0b6-13e0b30fc4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:43.210455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8f08224c-3961-476a-b0b6-13e0b30fc4b0 None None] MFA Rules not processed for user `6f6252d140f54ce7871de73110b41fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:43.211723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8f08224c-3961-476a-b0b6-13e0b30fc4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:43.294780 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2849/5662] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:43.345359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74f211-9915-43dc-853c-e6cab0dba654 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:43.347191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6d74f211-9915-43dc-853c-e6cab0dba654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:43.438912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:43.442998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d74f211-9915-43dc-853c-e6cab0dba654 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:43.447298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3db96-3e81-4474-a5cc-53311ac94fa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:43.447924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3db96-3e81-4474-a5cc-53311ac94fa3 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:43.448267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3db96-3e81-4474-a5cc-53311ac94fa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:43.449437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3db96-3e81-4474-a5cc-53311ac94fa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:43.461909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3db96-3e81-4474-a5cc-53311ac94fa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:43.465607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3db96-3e81-4474-a5cc-53311ac94fa3 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:43.479348 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2814/5663] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:21:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:21:43.497551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08224c-3961-476a-b0b6-13e0b30fc4b0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:43.569818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08224c-3961-476a-b0b6-13e0b30fc4b0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:43.575517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d512a4-ed2a-4782-9353-21ac1aa427f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:43.576733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d512a4-ed2a-4782-9353-21ac1aa427f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:43.577761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d512a4-ed2a-4782-9353-21ac1aa427f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:43.578308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d512a4-ed2a-4782-9353-21ac1aa427f2 admin admin] PATH_INFO: `/v3/users/6f6252d140f54ce7871de73110b41fc9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:43.579866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d512a4-ed2a-4782-9353-21ac1aa427f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f6252d140f54ce7871de73110b41fc9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:43.582117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d512a4-ed2a-4782-9353-21ac1aa427f2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:43.823756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-55d512a4-ed2a-4782-9353-21ac1aa427f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f6252d140f54ce7871de73110b41fc9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:43.834120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-55d512a4-ed2a-4782-9353-21ac1aa427f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f6252d140f54ce7871de73110b41fc9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:43.835272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-55d512a4-ed2a-4782-9353-21ac1aa427f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f6252d140f54ce7871de73110b41fc9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:43.836145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-55d512a4-ed2a-4782-9353-21ac1aa427f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f6252d140f54ce7871de73110b41fc9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:43.872422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:43.901223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-55d512a4-ed2a-4782-9353-21ac1aa427f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f6252d140f54ce7871de73110b41fc9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:43.992932 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2850/5664] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:21:43 2022] DELETE /identity/v3/users/6f6252d140f54ce7871de73110b41fc9 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:43.997529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d69284-3b01-493d-8fa9-8c41e5c9db65 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:43.998783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d69284-3b01-493d-8fa9-8c41e5c9db65 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:43.998783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d69284-3b01-493d-8fa9-8c41e5c9db65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:44.031553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-88d69284-3b01-493d-8fa9-8c41e5c9db65 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:44.032815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-88d69284-3b01-493d-8fa9-8c41e5c9db65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:44.154682 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2815/5665] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:44.176624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d512a4-ed2a-4782-9353-21ac1aa427f2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:44.178811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-55d512a4-ed2a-4782-9353-21ac1aa427f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:44.273824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d512a4-ed2a-4782-9353-21ac1aa427f2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:44.278798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a64fa6-fc9b-4511-9c73-1789cd4c727a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:44.279476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a64fa6-fc9b-4511-9c73-1789cd4c727a service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:44.280407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a64fa6-fc9b-4511-9c73-1789cd4c727a service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:44.280407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a64fa6-fc9b-4511-9c73-1789cd4c727a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:44.304107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a64fa6-fc9b-4511-9c73-1789cd4c727a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:44.305741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a64fa6-fc9b-4511-9c73-1789cd4c727a service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:44.324094 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2851/5666] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:21:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:21:44.896089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:45.191565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:45.436823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d69284-3b01-493d-8fa9-8c41e5c9db65 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:45.439036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-88d69284-3b01-493d-8fa9-8c41e5c9db65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:45.462494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d69284-3b01-493d-8fa9-8c41e5c9db65 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:45.470439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cbfe04-fe90-46a2-9a95-f8ada16bcd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:45.471841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbfe04-fe90-46a2-9a95-f8ada16bcd0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:45.472277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbfe04-fe90-46a2-9a95-f8ada16bcd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:45.472563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cbfe04-fe90-46a2-9a95-f8ada16bcd0a admin admin] PATH_INFO: `/v3/projects/415da79d13fb4cf98c4e83c7969212cb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:45.476513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbfe04-fe90-46a2-9a95-f8ada16bcd0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=415da79d13fb4cf98c4e83c7969212cb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:45.480529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cbfe04-fe90-46a2-9a95-f8ada16bcd0a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:45.638572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-05cbfe04-fe90-46a2-9a95-f8ada16bcd0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '415da79d13fb4cf98c4e83c7969212cb', 'request_id': 'req-05cbfe04-fe90-46a2-9a95-f8ada16bcd0a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:45.660932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-05cbfe04-fe90-46a2-9a95-f8ada16bcd0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '415da79d13fb4cf98c4e83c7969212cb', 'request_id': 'req-05cbfe04-fe90-46a2-9a95-f8ada16bcd0a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:45.800143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-05cbfe04-fe90-46a2-9a95-f8ada16bcd0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '415da79d13fb4cf98c4e83c7969212cb', 'request_id': 'req-05cbfe04-fe90-46a2-9a95-f8ada16bcd0a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:45.819312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-05cbfe04-fe90-46a2-9a95-f8ada16bcd0a admin admin] The token cache is being invalidate because project 415da79d13fb4cf98c4e83c7969212cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:21:45.819608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-05cbfe04-fe90-46a2-9a95-f8ada16bcd0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:45.820368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-05cbfe04-fe90-46a2-9a95-f8ada16bcd0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:45.823298 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2816/5667] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:21:45 2022] DELETE /identity/v3/projects/415da79d13fb4cf98c4e83c7969212cb => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:45.831276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddf259-5d56-4683-a640-b1667f70ead4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:45.831625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddf259-5d56-4683-a640-b1667f70ead4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:45.831984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ddf259-5d56-4683-a640-b1667f70ead4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:45.874907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d7ddf259-5d56-4683-a640-b1667f70ead4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:45.876311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d7ddf259-5d56-4683-a640-b1667f70ead4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:45.957076 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2852/5668] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:45.965034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9843b-a0ee-4f67-b1ea-20f2d0218f1d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:45.965451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9843b-a0ee-4f67-b1ea-20f2d0218f1d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:45.965595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9843b-a0ee-4f67-b1ea-20f2d0218f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:45.998553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-17a9843b-a0ee-4f67-b1ea-20f2d0218f1d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:46.000089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-17a9843b-a0ee-4f67-b1ea-20f2d0218f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:46.016363 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2817/5669] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:46.016942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ddf259-5d56-4683-a640-b1667f70ead4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:46.031776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ddf259-5d56-4683-a640-b1667f70ead4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:46.047413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70932760-2857-4c41-9ccd-3bf20b30fc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:46.047994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70932760-2857-4c41-9ccd-3bf20b30fc6f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:46.048454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70932760-2857-4c41-9ccd-3bf20b30fc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:46.048902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70932760-2857-4c41-9ccd-3bf20b30fc6f admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:46.049893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70932760-2857-4c41-9ccd-3bf20b30fc6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:21:46.050403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70932760-2857-4c41-9ccd-3bf20b30fc6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:46.052389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70932760-2857-4c41-9ccd-3bf20b30fc6f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:46.081184 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2853/5670] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:21:46.085502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9843b-a0ee-4f67-b1ea-20f2d0218f1d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:46.102442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9843b-a0ee-4f67-b1ea-20f2d0218f1d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:46.116894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02579aa2-0ada-41a0-9dd2-9d4a5005ee4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:46.117857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02579aa2-0ada-41a0-9dd2-9d4a5005ee4f admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:46.118561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02579aa2-0ada-41a0-9dd2-9d4a5005ee4f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:46.119311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02579aa2-0ada-41a0-9dd2-9d4a5005ee4f admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:46.122062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02579aa2-0ada-41a0-9dd2-9d4a5005ee4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:46.127245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02579aa2-0ada-41a0-9dd2-9d4a5005ee4f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:46.272395 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2818/5671] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:21:46 2022] POST /identity/v3/projects => generated 375 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:21:46.277587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70932760-2857-4c41-9ccd-3bf20b30fc6f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:46.289637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70932760-2857-4c41-9ccd-3bf20b30fc6f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:46.458968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:46.479553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0684225-456d-4567-ab10-2ccb191664ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:46.480266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0684225-456d-4567-ab10-2ccb191664ba admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:46.480559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0684225-456d-4567-ab10-2ccb191664ba admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:46.480843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0684225-456d-4567-ab10-2ccb191664ba admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:46.481557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0684225-456d-4567-ab10-2ccb191664ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:46.484120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0684225-456d-4567-ab10-2ccb191664ba admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:46.653472 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2854/5672] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:21:46 2022] POST /identity/v3/users => generated 288 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:21:46.658515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02579aa2-0ada-41a0-9dd2-9d4a5005ee4f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:46.672753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02579aa2-0ada-41a0-9dd2-9d4a5005ee4f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:46.689530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696eb55-1107-427f-a931-a9ca31d1bacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:46.693095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696eb55-1107-427f-a931-a9ca31d1bacd admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:46.693504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696eb55-1107-427f-a931-a9ca31d1bacd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:46.693884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696eb55-1107-427f-a931-a9ca31d1bacd admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:46.694835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696eb55-1107-427f-a931-a9ca31d1bacd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:46.698830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696eb55-1107-427f-a931-a9ca31d1bacd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:46.718441 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2819/5673] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:21:46.722971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0684225-456d-4567-ab10-2ccb191664ba admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:46.736126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0684225-456d-4567-ab10-2ccb191664ba admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:46.750548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd41a5-c69f-480b-98aa-d631f546fb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:46.751283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd41a5-c69f-480b-98aa-d631f546fb9c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:46.751742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd41a5-c69f-480b-98aa-d631f546fb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:46.752029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd41a5-c69f-480b-98aa-d631f546fb9c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:46.752942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd41a5-c69f-480b-98aa-d631f546fb9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:46.755566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd41a5-c69f-480b-98aa-d631f546fb9c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:46.782978 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2855/5674] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:21:46.786803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696eb55-1107-427f-a931-a9ca31d1bacd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:46.818028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696eb55-1107-427f-a931-a9ca31d1bacd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:46.837701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29169889-7142-4c97-bce4-409f0ca36a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:46.848319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29169889-7142-4c97-bce4-409f0ca36a0e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:46.849002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29169889-7142-4c97-bce4-409f0ca36a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:46.849603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29169889-7142-4c97-bce4-409f0ca36a0e admin admin] PATH_INFO: `/v3/projects/1a1fd2902ed9488cb220da0fba115078/users/0d7218edd07846c2b99e993bfebebb74/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:46.905691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29169889-7142-4c97-bce4-409f0ca36a0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a1fd2902ed9488cb220da0fba115078, user_id=0d7218edd07846c2b99e993bfebebb74, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:46.913737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29169889-7142-4c97-bce4-409f0ca36a0e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:47.014210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:47.044166 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2820/5675] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:21:46 2022] PUT /identity/v3/projects/1a1fd2902ed9488cb220da0fba115078/users/0d7218edd07846c2b99e993bfebebb74/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:47.045741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7200acb-302a-4c76-a0ec-c3c0706d7c52 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:47.046253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7200acb-302a-4c76-a0ec-c3c0706d7c52 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:47.046852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7200acb-302a-4c76-a0ec-c3c0706d7c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:47.110023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e7200acb-302a-4c76-a0ec-c3c0706d7c52 None None] MFA Rules not processed for user `0d7218edd07846c2b99e993bfebebb74`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:47.111618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e7200acb-302a-4c76-a0ec-c3c0706d7c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:47.316363 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2856/5676] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:21:47.328060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29169889-7142-4c97-bce4-409f0ca36a0e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:47.331876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-29169889-7142-4c97-bce4-409f0ca36a0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:47.443548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29169889-7142-4c97-bce4-409f0ca36a0e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:47.448798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd432d-d68d-4da8-b44f-2ca57cfb6ec8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:47.449887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd432d-d68d-4da8-b44f-2ca57cfb6ec8 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:47.449887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd432d-d68d-4da8-b44f-2ca57cfb6ec8 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:47.450099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd432d-d68d-4da8-b44f-2ca57cfb6ec8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:47.461856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd432d-d68d-4da8-b44f-2ca57cfb6ec8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:47.463811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd432d-d68d-4da8-b44f-2ca57cfb6ec8 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:47.474789 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2821/5677] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:21:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:21:48.638009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:48.775124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:49.152719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cae76d-e90d-425c-ae45-17af37af2aac None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:49.153903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cae76d-e90d-425c-ae45-17af37af2aac None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:49.154472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cae76d-e90d-425c-ae45-17af37af2aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:49.178874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-30cae76d-e90d-425c-ae45-17af37af2aac None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:49.180251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-30cae76d-e90d-425c-ae45-17af37af2aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:49.289391 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2857/5678] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:21:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:49.332446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd432d-d68d-4da8-b44f-2ca57cfb6ec8 service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:49.335148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-84dd432d-d68d-4da8-b44f-2ca57cfb6ec8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:49.460822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd432d-d68d-4da8-b44f-2ca57cfb6ec8 service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:49.471571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1fe83-4e6b-4401-90e2-277624ab9139 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:49.471571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1fe83-4e6b-4401-90e2-277624ab9139 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:49.471571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1fe83-4e6b-4401-90e2-277624ab9139 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:49.471571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1fe83-4e6b-4401-90e2-277624ab9139 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:49.480670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1fe83-4e6b-4401-90e2-277624ab9139 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:49.483216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1fe83-4e6b-4401-90e2-277624ab9139 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:49.515638 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2822/5679] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:21:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:21:49.821179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:50.372229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:51.013567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cae76d-e90d-425c-ae45-17af37af2aac None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:51.019575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:51.020130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-30cae76d-e90d-425c-ae45-17af37af2aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:51.132489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cae76d-e90d-425c-ae45-17af37af2aac None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:51.134762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529032e-9e2b-4197-a979-b207a2b6e379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:51.135735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529032e-9e2b-4197-a979-b207a2b6e379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:51.136108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529032e-9e2b-4197-a979-b207a2b6e379 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:51.136460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529032e-9e2b-4197-a979-b207a2b6e379 admin admin] PATH_INFO: `/v3/users/cafc81a185c645e893a77b7fd297d950` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:51.156835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529032e-9e2b-4197-a979-b207a2b6e379 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cafc81a185c645e893a77b7fd297d950)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:51.160234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529032e-9e2b-4197-a979-b207a2b6e379 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:51.474991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a529032e-9e2b-4197-a979-b207a2b6e379 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cafc81a185c645e893a77b7fd297d950'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:51.486819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a529032e-9e2b-4197-a979-b207a2b6e379 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cafc81a185c645e893a77b7fd297d950'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:51.487621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a529032e-9e2b-4197-a979-b207a2b6e379 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cafc81a185c645e893a77b7fd297d950'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:51.488253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a529032e-9e2b-4197-a979-b207a2b6e379 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cafc81a185c645e893a77b7fd297d950'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:51.564933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a529032e-9e2b-4197-a979-b207a2b6e379 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cafc81a185c645e893a77b7fd297d950'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:51.588284 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2858/5680] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:21:51 2022] DELETE /identity/v3/users/cafc81a185c645e893a77b7fd297d950 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:51.967405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1fe83-4e6b-4401-90e2-277624ab9139 service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:51.989198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1fe83-4e6b-4401-90e2-277624ab9139 service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:52.074978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:52.083908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93a23e-7249-4146-96b2-305e62b7c317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:52.084794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93a23e-7249-4146-96b2-305e62b7c317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:52.085081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93a23e-7249-4146-96b2-305e62b7c317 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:52.085360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de93a23e-7249-4146-96b2-305e62b7c317 admin admin] PATH_INFO: `/v3/projects/8f20390f83d3407e93a3d8d2a196281b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:52.097991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93a23e-7249-4146-96b2-305e62b7c317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f20390f83d3407e93a3d8d2a196281b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:52.099976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de93a23e-7249-4146-96b2-305e62b7c317 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:52.238172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-de93a23e-7249-4146-96b2-305e62b7c317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f20390f83d3407e93a3d8d2a196281b', 'request_id': 'req-de93a23e-7249-4146-96b2-305e62b7c317'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:52.245032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-de93a23e-7249-4146-96b2-305e62b7c317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f20390f83d3407e93a3d8d2a196281b', 'request_id': 'req-de93a23e-7249-4146-96b2-305e62b7c317'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:52.357101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-de93a23e-7249-4146-96b2-305e62b7c317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f20390f83d3407e93a3d8d2a196281b', 'request_id': 'req-de93a23e-7249-4146-96b2-305e62b7c317'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:52.372199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-de93a23e-7249-4146-96b2-305e62b7c317 admin admin] The token cache is being invalidate because project 8f20390f83d3407e93a3d8d2a196281b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:21:52.372493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-de93a23e-7249-4146-96b2-305e62b7c317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:52.374494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-de93a23e-7249-4146-96b2-305e62b7c317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:52.379497 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2823/5681] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:21:51 2022] DELETE /identity/v3/projects/8f20390f83d3407e93a3d8d2a196281b => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:52.380622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529032e-9e2b-4197-a979-b207a2b6e379 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:52.382377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a529032e-9e2b-4197-a979-b207a2b6e379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:52.411197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529032e-9e2b-4197-a979-b207a2b6e379 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:52.565076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2b20e4-3721-477c-88c0-572ba0b81d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:52.566687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b20e4-3721-477c-88c0-572ba0b81d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:52.567354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b20e4-3721-477c-88c0-572ba0b81d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:52.567781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2b20e4-3721-477c-88c0-572ba0b81d9c admin admin] PATH_INFO: `/v3/users/2d473cc820a94134a6fbd72bf6b1a0a6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:52.587034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b20e4-3721-477c-88c0-572ba0b81d9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d473cc820a94134a6fbd72bf6b1a0a6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:52.589346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2b20e4-3721-477c-88c0-572ba0b81d9c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:52.872132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5a2b20e4-3721-477c-88c0-572ba0b81d9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d473cc820a94134a6fbd72bf6b1a0a6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:52.880881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5a2b20e4-3721-477c-88c0-572ba0b81d9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d473cc820a94134a6fbd72bf6b1a0a6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:52.881992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5a2b20e4-3721-477c-88c0-572ba0b81d9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d473cc820a94134a6fbd72bf6b1a0a6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:52.882782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5a2b20e4-3721-477c-88c0-572ba0b81d9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d473cc820a94134a6fbd72bf6b1a0a6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:52.972696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5a2b20e4-3721-477c-88c0-572ba0b81d9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d473cc820a94134a6fbd72bf6b1a0a6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:52.995458 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2859/5682] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:21:52 2022] DELETE /identity/v3/users/2d473cc820a94134a6fbd72bf6b1a0a6 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:53.045106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:53.443133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:53.569029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93a23e-7249-4146-96b2-305e62b7c317 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:53.571946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-de93a23e-7249-4146-96b2-305e62b7c317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:53.630465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de93a23e-7249-4146-96b2-305e62b7c317 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:53.724864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e550c4cd-e0b9-4df8-bf95-742edd207765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:53.725747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550c4cd-e0b9-4df8-bf95-742edd207765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:53.726117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550c4cd-e0b9-4df8-bf95-742edd207765 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:53.726408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e550c4cd-e0b9-4df8-bf95-742edd207765 admin admin] PATH_INFO: `/v3/projects/cf23d1694b31430f9c7315c7c3195937` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:53.742462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550c4cd-e0b9-4df8-bf95-742edd207765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf23d1694b31430f9c7315c7c3195937)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:53.744429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e550c4cd-e0b9-4df8-bf95-742edd207765 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:53.825499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef366d-2fdb-4411-8a6b-ee159e3f2730 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:53.825996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef366d-2fdb-4411-8a6b-ee159e3f2730 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:53.826196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef366d-2fdb-4411-8a6b-ee159e3f2730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:53.849336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a3ef366d-2fdb-4411-8a6b-ee159e3f2730 None None] MFA Rules not processed for user `f1acb2fffdc64ce29b71b2b40e2285f0`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:53.850635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a3ef366d-2fdb-4411-8a6b-ee159e3f2730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:53.895122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e550c4cd-e0b9-4df8-bf95-742edd207765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf23d1694b31430f9c7315c7c3195937', 'request_id': 'req-e550c4cd-e0b9-4df8-bf95-742edd207765'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:53.908380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e550c4cd-e0b9-4df8-bf95-742edd207765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf23d1694b31430f9c7315c7c3195937', 'request_id': 'req-e550c4cd-e0b9-4df8-bf95-742edd207765'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:53.931375 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2860/5683] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:54.015311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e550c4cd-e0b9-4df8-bf95-742edd207765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf23d1694b31430f9c7315c7c3195937', 'request_id': 'req-e550c4cd-e0b9-4df8-bf95-742edd207765'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:54.032096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e550c4cd-e0b9-4df8-bf95-742edd207765 admin admin] The token cache is being invalidate because project cf23d1694b31430f9c7315c7c3195937 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:21:54.032688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e550c4cd-e0b9-4df8-bf95-742edd207765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:54.033520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e550c4cd-e0b9-4df8-bf95-742edd207765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:54.035892 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2824/5684] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:21:53 2022] DELETE /identity/v3/projects/cf23d1694b31430f9c7315c7c3195937 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:54.040883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde6d95-7bf7-47b2-9ac8-1ceca1e3af2e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:54.041775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde6d95-7bf7-47b2-9ac8-1ceca1e3af2e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:54.042968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde6d95-7bf7-47b2-9ac8-1ceca1e3af2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:54.073194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3bde6d95-7bf7-47b2-9ac8-1ceca1e3af2e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:54.075094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3bde6d95-7bf7-47b2-9ac8-1ceca1e3af2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:54.154547 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2861/5685] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:54.168468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd44c2d-9ee4-4699-bace-b4c86f4f579e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:54.168962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd44c2d-9ee4-4699-bace-b4c86f4f579e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:54.169319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd44c2d-9ee4-4699-bace-b4c86f4f579e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:54.194283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6bd44c2d-9ee4-4699-bace-b4c86f4f579e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:54.196542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6bd44c2d-9ee4-4699-bace-b4c86f4f579e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:54.206366 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2825/5686] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:54.211844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde6d95-7bf7-47b2-9ac8-1ceca1e3af2e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:54.236670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde6d95-7bf7-47b2-9ac8-1ceca1e3af2e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:54.243916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5d0f3-9abb-4275-8d22-fab20d31ed17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:54.244644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5d0f3-9abb-4275-8d22-fab20d31ed17 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:54.244937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5d0f3-9abb-4275-8d22-fab20d31ed17 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:54.245220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5d0f3-9abb-4275-8d22-fab20d31ed17 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:54.246009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5d0f3-9abb-4275-8d22-fab20d31ed17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:21:54.246328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5d0f3-9abb-4275-8d22-fab20d31ed17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:54.248304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5d0f3-9abb-4275-8d22-fab20d31ed17 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:54.262290 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2862/5687] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:21:54.266811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd44c2d-9ee4-4699-bace-b4c86f4f579e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:54.276901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd44c2d-9ee4-4699-bace-b4c86f4f579e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:54.283128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d3d61-98b1-415e-b2de-82993c33acd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:54.284110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d3d61-98b1-415e-b2de-82993c33acd2 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:54.284646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d3d61-98b1-415e-b2de-82993c33acd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:54.285271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d3d61-98b1-415e-b2de-82993c33acd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:54.286146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d3d61-98b1-415e-b2de-82993c33acd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:54.288259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d3d61-98b1-415e-b2de-82993c33acd2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:54.399616 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2826/5688] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:21:54 2022] POST /identity/v3/projects => generated 379 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:21:54.401247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5d0f3-9abb-4275-8d22-fab20d31ed17 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:54.422461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5d0f3-9abb-4275-8d22-fab20d31ed17 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:54.633566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421eb4d-7f85-472b-b742-cd3f68efa4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:54.635398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421eb4d-7f85-472b-b742-cd3f68efa4ad admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:54.636248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421eb4d-7f85-472b-b742-cd3f68efa4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:54.636720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421eb4d-7f85-472b-b742-cd3f68efa4ad admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:54.638995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421eb4d-7f85-472b-b742-cd3f68efa4ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:54.642370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1421eb4d-7f85-472b-b742-cd3f68efa4ad admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:54.648987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:54.770753 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2863/5689] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:21:54 2022] POST /identity/v3/users => generated 290 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:21:54.772198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d3d61-98b1-415e-b2de-82993c33acd2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:54.791623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d3d61-98b1-415e-b2de-82993c33acd2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:54.806591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47523df-02f4-4689-97d1-c72f2724ba85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:54.807374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47523df-02f4-4689-97d1-c72f2724ba85 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:54.807666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47523df-02f4-4689-97d1-c72f2724ba85 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:54.808035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47523df-02f4-4689-97d1-c72f2724ba85 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:54.808866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47523df-02f4-4689-97d1-c72f2724ba85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:54.810727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47523df-02f4-4689-97d1-c72f2724ba85 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:54.845170 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2827/5690] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:21:54.847209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421eb4d-7f85-472b-b742-cd3f68efa4ad admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:54.864906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421eb4d-7f85-472b-b742-cd3f68efa4ad admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:54.879643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7e5ea2-3c2f-46a5-8842-e67cb4b4627b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:54.880408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e5ea2-3c2f-46a5-8842-e67cb4b4627b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:54.882897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e5ea2-3c2f-46a5-8842-e67cb4b4627b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:54.883288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7e5ea2-3c2f-46a5-8842-e67cb4b4627b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:54.884599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e5ea2-3c2f-46a5-8842-e67cb4b4627b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:54.891135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7e5ea2-3c2f-46a5-8842-e67cb4b4627b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:54.922795 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2864/5691] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:21:54.927637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47523df-02f4-4689-97d1-c72f2724ba85 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:54.948469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47523df-02f4-4689-97d1-c72f2724ba85 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:54.962639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c92d69-61ca-4860-b8d6-67f670179936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:54.963729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c92d69-61ca-4860-b8d6-67f670179936 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:54.964040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c92d69-61ca-4860-b8d6-67f670179936 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:54.964312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c92d69-61ca-4860-b8d6-67f670179936 admin admin] PATH_INFO: `/v3/projects/54447ea481574a128f06c24315f34e46/users/02d5fbb072814fd7923ecf12c7ecad7b/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:55.005912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c92d69-61ca-4860-b8d6-67f670179936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54447ea481574a128f06c24315f34e46, user_id=02d5fbb072814fd7923ecf12c7ecad7b, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:55.011257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c92d69-61ca-4860-b8d6-67f670179936 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:55.121423 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2828/5692] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:21:54 2022] PUT /identity/v3/projects/54447ea481574a128f06c24315f34e46/users/02d5fbb072814fd7923ecf12c7ecad7b/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:55.127314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855035cf-8913-47f1-bc8e-de651f1f0234 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:55.128523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855035cf-8913-47f1-bc8e-de651f1f0234 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:55.128623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855035cf-8913-47f1-bc8e-de651f1f0234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:55.171067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-855035cf-8913-47f1-bc8e-de651f1f0234 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:55.175394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-855035cf-8913-47f1-bc8e-de651f1f0234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:55.292365 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2865/5693] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:55.309688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c92d69-61ca-4860-b8d6-67f670179936 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:55.310752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-81c92d69-61ca-4860-b8d6-67f670179936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:55.329306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:55.409606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c92d69-61ca-4860-b8d6-67f670179936 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:55.413313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bcd21-394d-41e3-af61-003b58378aad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:55.414006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bcd21-394d-41e3-af61-003b58378aad service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:55.414295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bcd21-394d-41e3-af61-003b58378aad service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:55.414622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17bcd21-394d-41e3-af61-003b58378aad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:55.424013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17bcd21-394d-41e3-af61-003b58378aad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:55.425457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17bcd21-394d-41e3-af61-003b58378aad service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:55.450095 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2829/5694] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:21:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:21:55.578302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7de0c5-cbd8-4f2e-904c-60e3975a57f0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:55.579107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7de0c5-cbd8-4f2e-904c-60e3975a57f0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:55.579107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7de0c5-cbd8-4f2e-904c-60e3975a57f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:55.637347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5b7de0c5-cbd8-4f2e-904c-60e3975a57f0 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:55.638593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5b7de0c5-cbd8-4f2e-904c-60e3975a57f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:55.720763 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2866/5695] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:21:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:21:55.741339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bcd21-394d-41e3-af61-003b58378aad service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:55.742758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b17bcd21-394d-41e3-af61-003b58378aad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:55.841321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17bcd21-394d-41e3-af61-003b58378aad service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:55.849956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c1bc0d-68c5-4472-9d06-a02e1842224b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:55.850723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1bc0d-68c5-4472-9d06-a02e1842224b service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:55.851095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1bc0d-68c5-4472-9d06-a02e1842224b service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:55.851442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c1bc0d-68c5-4472-9d06-a02e1842224b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:55.862958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1bc0d-68c5-4472-9d06-a02e1842224b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:55.864738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c1bc0d-68c5-4472-9d06-a02e1842224b service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:55.873938 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2830/5696] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:21:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:21:57.243207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:57.740248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:58.554613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:58.757748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:59.075586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7de0c5-cbd8-4f2e-904c-60e3975a57f0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:59.077942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5b7de0c5-cbd8-4f2e-904c-60e3975a57f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:59.095753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7de0c5-cbd8-4f2e-904c-60e3975a57f0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:21:59.104086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf9961-be80-4279-935f-099d5771d065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:21:59.106092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf9961-be80-4279-935f-099d5771d065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:59.106092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf9961-be80-4279-935f-099d5771d065 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:59.106092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf9961-be80-4279-935f-099d5771d065 admin admin] PATH_INFO: `/v3/users/0d7218edd07846c2b99e993bfebebb74` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:59.109075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf9961-be80-4279-935f-099d5771d065 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d7218edd07846c2b99e993bfebebb74)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:21:59.117371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:59.121509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaf9961-be80-4279-935f-099d5771d065 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:21:59.237454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:59.393131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bcaf9961-be80-4279-935f-099d5771d065 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d7218edd07846c2b99e993bfebebb74'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:59.423538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bcaf9961-be80-4279-935f-099d5771d065 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d7218edd07846c2b99e993bfebebb74'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:59.424437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bcaf9961-be80-4279-935f-099d5771d065 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d7218edd07846c2b99e993bfebebb74'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:59.425159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bcaf9961-be80-4279-935f-099d5771d065 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d7218edd07846c2b99e993bfebebb74'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:59.490229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bcaf9961-be80-4279-935f-099d5771d065 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d7218edd07846c2b99e993bfebebb74'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:21:59.560676 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2867/5697] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:21:59 2022] DELETE /identity/v3/users/0d7218edd07846c2b99e993bfebebb74 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:21:59.561735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8900c-c917-4e5e-a7e7-8770782aec7d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:21:59.562216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8900c-c917-4e5e-a7e7-8770782aec7d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:21:59.562396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8900c-c917-4e5e-a7e7-8770782aec7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:21:59.603614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b9c8900c-c917-4e5e-a7e7-8770782aec7d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:21:59.605149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b9c8900c-c917-4e5e-a7e7-8770782aec7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:21:59.885058 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2831/5698] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:21:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:21:59.889995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf9961-be80-4279-935f-099d5771d065 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:21:59.891518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bcaf9961-be80-4279-935f-099d5771d065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:00.125255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf9961-be80-4279-935f-099d5771d065 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:00.135912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:00.136909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50046357-c083-4f94-8b5e-3a0c3af13a3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:00.137612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50046357-c083-4f94-8b5e-3a0c3af13a3c service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:00.137925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50046357-c083-4f94-8b5e-3a0c3af13a3c service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:00.138267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50046357-c083-4f94-8b5e-3a0c3af13a3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:00.158361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50046357-c083-4f94-8b5e-3a0c3af13a3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:00.162590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50046357-c083-4f94-8b5e-3a0c3af13a3c service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:00.188445 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2868/5699] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:21:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:22:00.300523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:01.208672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:01.650115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:02.374756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8900c-c917-4e5e-a7e7-8770782aec7d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:02.377586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b9c8900c-c917-4e5e-a7e7-8770782aec7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:02.391389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8900c-c917-4e5e-a7e7-8770782aec7d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:02.405278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:02.415520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd4a2a-78dc-4d36-a055-4108c994f426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:02.417506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd4a2a-78dc-4d36-a055-4108c994f426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:02.418316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd4a2a-78dc-4d36-a055-4108c994f426 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:02.419728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd4a2a-78dc-4d36-a055-4108c994f426 admin admin] PATH_INFO: `/v3/projects/1a1fd2902ed9488cb220da0fba115078` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:02.424798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd4a2a-78dc-4d36-a055-4108c994f426 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a1fd2902ed9488cb220da0fba115078)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:02.432975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd4a2a-78dc-4d36-a055-4108c994f426 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:02.993901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3bdd4a2a-78dc-4d36-a055-4108c994f426 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a1fd2902ed9488cb220da0fba115078', 'request_id': 'req-3bdd4a2a-78dc-4d36-a055-4108c994f426'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:03.013702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3bdd4a2a-78dc-4d36-a055-4108c994f426 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a1fd2902ed9488cb220da0fba115078', 'request_id': 'req-3bdd4a2a-78dc-4d36-a055-4108c994f426'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:03.174270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3bdd4a2a-78dc-4d36-a055-4108c994f426 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a1fd2902ed9488cb220da0fba115078', 'request_id': 'req-3bdd4a2a-78dc-4d36-a055-4108c994f426'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:03.203287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3bdd4a2a-78dc-4d36-a055-4108c994f426 admin admin] The token cache is being invalidate because project 1a1fd2902ed9488cb220da0fba115078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:22:03.208373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3bdd4a2a-78dc-4d36-a055-4108c994f426 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:03.208373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3bdd4a2a-78dc-4d36-a055-4108c994f426 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:03.212516 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2832/5700] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:22:02 2022] DELETE /identity/v3/projects/1a1fd2902ed9488cb220da0fba115078 => generated 0 bytes in 839 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:22:03.218138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a707af4-d12c-4c13-8eef-494a5022d308 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:03.218658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a707af4-d12c-4c13-8eef-494a5022d308 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:03.218750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a707af4-d12c-4c13-8eef-494a5022d308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:03.293822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1a707af4-d12c-4c13-8eef-494a5022d308 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:03.295893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1a707af4-d12c-4c13-8eef-494a5022d308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:03.436259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:03.453346 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2869/5701] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:03.461410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319bb6e-bcfa-4aaa-a845-4498310ce90f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:03.461790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319bb6e-bcfa-4aaa-a845-4498310ce90f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:03.462074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319bb6e-bcfa-4aaa-a845-4498310ce90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:03.507691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e319bb6e-bcfa-4aaa-a845-4498310ce90f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:03.509112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e319bb6e-bcfa-4aaa-a845-4498310ce90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:03.535358 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2833/5702] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:03.542356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a707af4-d12c-4c13-8eef-494a5022d308 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:03.547916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a707af4-d12c-4c13-8eef-494a5022d308 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:03.555146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bebc7d-2c3d-4371-9725-5736a2bb2677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:03.556167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bebc7d-2c3d-4371-9725-5736a2bb2677 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:03.556657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bebc7d-2c3d-4371-9725-5736a2bb2677 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:03.557132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bebc7d-2c3d-4371-9725-5736a2bb2677 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:03.558134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bebc7d-2c3d-4371-9725-5736a2bb2677 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:22:03.558657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bebc7d-2c3d-4371-9725-5736a2bb2677 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:03.562656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bebc7d-2c3d-4371-9725-5736a2bb2677 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:03.581445 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2870/5703] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:22:03.583258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319bb6e-bcfa-4aaa-a845-4498310ce90f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:03.604309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319bb6e-bcfa-4aaa-a845-4498310ce90f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:03.611686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f3ec4-fa53-4c4b-b850-dfedd538ff2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:03.612646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f3ec4-fa53-4c4b-b850-dfedd538ff2e admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:03.613268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f3ec4-fa53-4c4b-b850-dfedd538ff2e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:03.613746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f3ec4-fa53-4c4b-b850-dfedd538ff2e admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:03.614857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f3ec4-fa53-4c4b-b850-dfedd538ff2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:03.631335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f3ec4-fa53-4c4b-b850-dfedd538ff2e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:03.757200 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2834/5704] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:22:03 2022] POST /identity/v3/projects => generated 385 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:22:03.761048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bebc7d-2c3d-4371-9725-5736a2bb2677 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:03.774098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bebc7d-2c3d-4371-9725-5736a2bb2677 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:03.819552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:03.844895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:04.004226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5fec2-3417-41bf-8623-ce51cba9fb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:04.010327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5fec2-3417-41bf-8623-ce51cba9fb7f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:04.010327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5fec2-3417-41bf-8623-ce51cba9fb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:04.010327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5fec2-3417-41bf-8623-ce51cba9fb7f admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:04.010327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5fec2-3417-41bf-8623-ce51cba9fb7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:04.037536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5fec2-3417-41bf-8623-ce51cba9fb7f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:04.219362 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2871/5705] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:22:03 2022] POST /identity/v3/users => generated 293 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:22:04.223063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f3ec4-fa53-4c4b-b850-dfedd538ff2e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:04.244321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f3ec4-fa53-4c4b-b850-dfedd538ff2e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:04.265135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb22eae-0a8c-45f9-9b7e-83314ea59876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:04.268585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22eae-0a8c-45f9-9b7e-83314ea59876 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:04.268585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22eae-0a8c-45f9-9b7e-83314ea59876 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:04.268585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22eae-0a8c-45f9-9b7e-83314ea59876 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:04.268585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb22eae-0a8c-45f9-9b7e-83314ea59876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:04.271749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb22eae-0a8c-45f9-9b7e-83314ea59876 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:04.316139 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2835/5706] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:22:04.317769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5fec2-3417-41bf-8623-ce51cba9fb7f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:04.333733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5fec2-3417-41bf-8623-ce51cba9fb7f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:04.350113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d24e9-344c-4653-968e-f0b48d37a13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:04.350959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d24e9-344c-4653-968e-f0b48d37a13a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:04.351701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d24e9-344c-4653-968e-f0b48d37a13a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:04.352155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d24e9-344c-4653-968e-f0b48d37a13a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:04.353076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d24e9-344c-4653-968e-f0b48d37a13a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:04.354897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d24e9-344c-4653-968e-f0b48d37a13a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:04.374219 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2872/5707] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:22:04.378145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb22eae-0a8c-45f9-9b7e-83314ea59876 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:04.391118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb22eae-0a8c-45f9-9b7e-83314ea59876 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:04.407379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac29aa-ba0e-4fc0-a2a8-5ca4b92cef81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:04.408349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac29aa-ba0e-4fc0-a2a8-5ca4b92cef81 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:04.408781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac29aa-ba0e-4fc0-a2a8-5ca4b92cef81 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:04.409078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac29aa-ba0e-4fc0-a2a8-5ca4b92cef81 admin admin] PATH_INFO: `/v3/projects/32ea7778e193493fac70cb7da7c7ba5d/users/abc82744eb774701b36fcc350a50e509/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:04.447489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac29aa-ba0e-4fc0-a2a8-5ca4b92cef81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32ea7778e193493fac70cb7da7c7ba5d, user_id=abc82744eb774701b36fcc350a50e509, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:04.454463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac29aa-ba0e-4fc0-a2a8-5ca4b92cef81 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:04.567692 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2836/5708] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:22:04 2022] PUT /identity/v3/projects/32ea7778e193493fac70cb7da7c7ba5d/users/abc82744eb774701b36fcc350a50e509/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:22:04.570760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459270c-c04a-40c3-90fe-d0b2a8bd26a7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:04.571065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459270c-c04a-40c3-90fe-d0b2a8bd26a7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:04.571415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459270c-c04a-40c3-90fe-d0b2a8bd26a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:04.682003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e459270c-c04a-40c3-90fe-d0b2a8bd26a7 None None] MFA Rules not processed for user `abc82744eb774701b36fcc350a50e509`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:04.683362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e459270c-c04a-40c3-90fe-d0b2a8bd26a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:04.894617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:04.900452 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2873/5709] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:04.915368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac29aa-ba0e-4fc0-a2a8-5ca4b92cef81 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:04.918341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-57ac29aa-ba0e-4fc0-a2a8-5ca4b92cef81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:05.168100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac29aa-ba0e-4fc0-a2a8-5ca4b92cef81 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:05.178698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc126e-dfa4-4aec-a174-ef33a446adc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:05.179387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc126e-dfa4-4aec-a174-ef33a446adc3 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:05.179658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc126e-dfa4-4aec-a174-ef33a446adc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:05.179940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc126e-dfa4-4aec-a174-ef33a446adc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:05.207145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc126e-dfa4-4aec-a174-ef33a446adc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:05.211622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc126e-dfa4-4aec-a174-ef33a446adc3 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:05.238244 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2837/5710] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:22:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:22:05.346308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:06.963238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:06.991506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:07.991307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:09.954401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:10.488819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:10.608550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:11.392411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:11.583913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:12.045316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:12.345368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:12.392268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:12.986688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:13.088069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:13.599598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459270c-c04a-40c3-90fe-d0b2a8bd26a7 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:13.739540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:13.802812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:13.854403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459270c-c04a-40c3-90fe-d0b2a8bd26a7 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:13.875006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c80c4-5dbe-482e-812c-a14c61bd557f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:13.875006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c80c4-5dbe-482e-812c-a14c61bd557f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:13.881223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c80c4-5dbe-482e-812c-a14c61bd557f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:13.881730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c80c4-5dbe-482e-812c-a14c61bd557f admin admin] PATH_INFO: `/v3/users/abc82744eb774701b36fcc350a50e509` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:13.883416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c80c4-5dbe-482e-812c-a14c61bd557f admin admin] RBAC: Authorizing `identity:delete_user(user_id=abc82744eb774701b36fcc350a50e509)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:13.886757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c80c4-5dbe-482e-812c-a14c61bd557f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:13.932328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:14.095158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:14.235496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-548c80c4-5dbe-482e-812c-a14c61bd557f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abc82744eb774701b36fcc350a50e509'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:14.254667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-548c80c4-5dbe-482e-812c-a14c61bd557f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abc82744eb774701b36fcc350a50e509'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:14.256001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-548c80c4-5dbe-482e-812c-a14c61bd557f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abc82744eb774701b36fcc350a50e509'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:14.257693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-548c80c4-5dbe-482e-812c-a14c61bd557f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abc82744eb774701b36fcc350a50e509'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:14.350900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-548c80c4-5dbe-482e-812c-a14c61bd557f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abc82744eb774701b36fcc350a50e509'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:14.392414 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2874/5711] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:22:13 2022] DELETE /identity/v3/users/abc82744eb774701b36fcc350a50e509 => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:22:14.396773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4e3d8-bff9-48c2-ad59-f85dab2dfa74 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:14.397298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4e3d8-bff9-48c2-ad59-f85dab2dfa74 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:14.397958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4e3d8-bff9-48c2-ad59-f85dab2dfa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:14.435132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-88a4e3d8-bff9-48c2-ad59-f85dab2dfa74 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:14.436419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-88a4e3d8-bff9-48c2-ad59-f85dab2dfa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:14.589665 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2838/5712] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:14.795509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c80c4-5dbe-482e-812c-a14c61bd557f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:14.818926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-548c80c4-5dbe-482e-812c-a14c61bd557f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:15.035711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c80c4-5dbe-482e-812c-a14c61bd557f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:15.064564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09efe7-e5ee-4b42-b878-fab5c1389ccf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:15.065277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09efe7-e5ee-4b42-b878-fab5c1389ccf service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:15.066247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09efe7-e5ee-4b42-b878-fab5c1389ccf service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:15.075142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f09efe7-e5ee-4b42-b878-fab5c1389ccf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:15.098076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09efe7-e5ee-4b42-b878-fab5c1389ccf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:15.099585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f09efe7-e5ee-4b42-b878-fab5c1389ccf service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:15.154497 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2875/5713] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:22:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:22:15.397232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:15.430624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:15.718945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:15.848976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:16.745711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:16.782325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dc648-f0c0-4b39-addd-a6c805bfba61 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:16.782615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dc648-f0c0-4b39-addd-a6c805bfba61 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:16.782899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dc648-f0c0-4b39-addd-a6c805bfba61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:16.844045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8b2dc648-f0c0-4b39-addd-a6c805bfba61 None None] MFA Rules not processed for user `02d5fbb072814fd7923ecf12c7ecad7b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:16.845593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8b2dc648-f0c0-4b39-addd-a6c805bfba61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:17.052923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09efe7-e5ee-4b42-b878-fab5c1389ccf service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:17.054478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3f09efe7-e5ee-4b42-b878-fab5c1389ccf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:17.073915 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2839/5714] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:17.088533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f09efe7-e5ee-4b42-b878-fab5c1389ccf service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:17.096626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2dc648-f0c0-4b39-addd-a6c805bfba61 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:17.103443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8b2dc648-f0c0-4b39-addd-a6c805bfba61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:17.108704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e9fa4-e667-40fc-9d69-f6e6511723b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:17.109662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e9fa4-e667-40fc-9d69-f6e6511723b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:17.111154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e9fa4-e667-40fc-9d69-f6e6511723b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:17.111460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e9fa4-e667-40fc-9d69-f6e6511723b6 admin admin] PATH_INFO: `/v3/projects/32ea7778e193493fac70cb7da7c7ba5d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:17.114733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e9fa4-e667-40fc-9d69-f6e6511723b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32ea7778e193493fac70cb7da7c7ba5d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:17.121419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e9fa4-e667-40fc-9d69-f6e6511723b6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:17.156896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:17.319175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2dc648-f0c0-4b39-addd-a6c805bfba61 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:17.336818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-434e9fa4-e667-40fc-9d69-f6e6511723b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32ea7778e193493fac70cb7da7c7ba5d', 'request_id': 'req-434e9fa4-e667-40fc-9d69-f6e6511723b6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:17.351388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-434e9fa4-e667-40fc-9d69-f6e6511723b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32ea7778e193493fac70cb7da7c7ba5d', 'request_id': 'req-434e9fa4-e667-40fc-9d69-f6e6511723b6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:17.477061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-434e9fa4-e667-40fc-9d69-f6e6511723b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32ea7778e193493fac70cb7da7c7ba5d', 'request_id': 'req-434e9fa4-e667-40fc-9d69-f6e6511723b6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:17.479887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1149f0b1-d149-417a-8d67-19b103907f18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:17.488212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149f0b1-d149-417a-8d67-19b103907f18 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:17.488212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149f0b1-d149-417a-8d67-19b103907f18 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:17.488212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1149f0b1-d149-417a-8d67-19b103907f18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:17.508579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-434e9fa4-e667-40fc-9d69-f6e6511723b6 admin admin] The token cache is being invalidate because project 32ea7778e193493fac70cb7da7c7ba5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:22:17.508868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-434e9fa4-e667-40fc-9d69-f6e6511723b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:17.516643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-434e9fa4-e667-40fc-9d69-f6e6511723b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:17.529768 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2876/5715] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:22:17 2022] DELETE /identity/v3/projects/32ea7778e193493fac70cb7da7c7ba5d => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:22:17.536053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8401cf-e8fc-4ecf-a88e-49b23d8e9f05 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:17.536606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8401cf-e8fc-4ecf-a88e-49b23d8e9f05 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:17.539919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8401cf-e8fc-4ecf-a88e-49b23d8e9f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:17.604799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-bc8401cf-e8fc-4ecf-a88e-49b23d8e9f05 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:17.610395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bc8401cf-e8fc-4ecf-a88e-49b23d8e9f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:17.646176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149f0b1-d149-417a-8d67-19b103907f18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:17.647831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1149f0b1-d149-417a-8d67-19b103907f18 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:17.697473 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2840/5716] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:22:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:22:17.753685 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2877/5717] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:17.761687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292d702-bd92-4f8d-ae97-83ee277ec26d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:17.765283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292d702-bd92-4f8d-ae97-83ee277ec26d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:17.765283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b292d702-bd92-4f8d-ae97-83ee277ec26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:17.803276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b292d702-bd92-4f8d-ae97-83ee277ec26d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:17.810181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b292d702-bd92-4f8d-ae97-83ee277ec26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:17.824446 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2841/5718] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:17.829291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8401cf-e8fc-4ecf-a88e-49b23d8e9f05 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:17.847889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8401cf-e8fc-4ecf-a88e-49b23d8e9f05 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:17.872340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34e117-de9b-46ec-94ec-e0cd2d527853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:17.876559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34e117-de9b-46ec-94ec-e0cd2d527853 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:17.876854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34e117-de9b-46ec-94ec-e0cd2d527853 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:17.877144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd34e117-de9b-46ec-94ec-e0cd2d527853 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:17.877962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34e117-de9b-46ec-94ec-e0cd2d527853 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:22:17.879741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34e117-de9b-46ec-94ec-e0cd2d527853 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:17.882354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd34e117-de9b-46ec-94ec-e0cd2d527853 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:17.927518 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2878/5719] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:22:17.930340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292d702-bd92-4f8d-ae97-83ee277ec26d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:17.943880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b292d702-bd92-4f8d-ae97-83ee277ec26d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:17.956872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712252d-238d-4772-86c1-761382a521a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:17.961553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712252d-238d-4772-86c1-761382a521a6 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:17.963022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712252d-238d-4772-86c1-761382a521a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:17.963502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712252d-238d-4772-86c1-761382a521a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:17.965712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8712252d-238d-4772-86c1-761382a521a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:17.970416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8712252d-238d-4772-86c1-761382a521a6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:18.149730 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2842/5720] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:22:17 2022] POST /identity/v3/projects => generated 391 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:22:18.150083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34e117-de9b-46ec-94ec-e0cd2d527853 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:18.164920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd34e117-de9b-46ec-94ec-e0cd2d527853 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:18.202809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:18.290642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0e522-a2d7-4ca5-83b8-240237301d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:18.291891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0e522-a2d7-4ca5-83b8-240237301d15 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:18.292392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0e522-a2d7-4ca5-83b8-240237301d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:18.292877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0e522-a2d7-4ca5-83b8-240237301d15 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:18.293776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0e522-a2d7-4ca5-83b8-240237301d15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:18.296960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c0e522-a2d7-4ca5-83b8-240237301d15 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:18.414636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712252d-238d-4772-86c1-761382a521a6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:18.416028 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2879/5721] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:22:18 2022] POST /identity/v3/users => generated 296 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:22:18.482711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712252d-238d-4772-86c1-761382a521a6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:18.494061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eadd06-dd65-470f-9a1f-d664e0c417a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:18.495285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eadd06-dd65-470f-9a1f-d664e0c417a2 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:18.495737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eadd06-dd65-470f-9a1f-d664e0c417a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:18.498701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eadd06-dd65-470f-9a1f-d664e0c417a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:18.498701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eadd06-dd65-470f-9a1f-d664e0c417a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:18.502302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eadd06-dd65-470f-9a1f-d664e0c417a2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:18.550364 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2843/5722] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:22:18.574137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0e522-a2d7-4ca5-83b8-240237301d15 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:18.595146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c0e522-a2d7-4ca5-83b8-240237301d15 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:18.602350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191637b1-af92-4c62-a5ed-8d5d44ef7f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:18.603231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191637b1-af92-4c62-a5ed-8d5d44ef7f64 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:18.603517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191637b1-af92-4c62-a5ed-8d5d44ef7f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:18.603995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191637b1-af92-4c62-a5ed-8d5d44ef7f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:18.605014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191637b1-af92-4c62-a5ed-8d5d44ef7f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:18.609243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191637b1-af92-4c62-a5ed-8d5d44ef7f64 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:18.631086 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2880/5723] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:22:18.632679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eadd06-dd65-470f-9a1f-d664e0c417a2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:18.656124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eadd06-dd65-470f-9a1f-d664e0c417a2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:18.665693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601279cb-9470-46a5-bbe9-300e1425884a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:18.666737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601279cb-9470-46a5-bbe9-300e1425884a admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:18.667195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601279cb-9470-46a5-bbe9-300e1425884a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:18.671128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601279cb-9470-46a5-bbe9-300e1425884a admin admin] PATH_INFO: `/v3/projects/c3588eab36044e439158f876836ea514/users/f3468a0b2d324603ab6c3080993bc0c8/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:18.703772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601279cb-9470-46a5-bbe9-300e1425884a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3588eab36044e439158f876836ea514, user_id=f3468a0b2d324603ab6c3080993bc0c8, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:18.711565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601279cb-9470-46a5-bbe9-300e1425884a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:18.833798 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2844/5724] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:22:18 2022] PUT /identity/v3/projects/c3588eab36044e439158f876836ea514/users/f3468a0b2d324603ab6c3080993bc0c8/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:22:18.844461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a0d8a-d647-4bd7-ac3f-1ae1e5a4d673 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:18.846255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a0d8a-d647-4bd7-ac3f-1ae1e5a4d673 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:18.847322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5a0d8a-d647-4bd7-ac3f-1ae1e5a4d673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:18.920777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-bb5a0d8a-d647-4bd7-ac3f-1ae1e5a4d673 None None] MFA Rules not processed for user `f3468a0b2d324603ab6c3080993bc0c8`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:18.925547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bb5a0d8a-d647-4bd7-ac3f-1ae1e5a4d673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:19.111993 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2881/5725] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:19.114946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601279cb-9470-46a5-bbe9-300e1425884a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:19.242947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:19.284544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601279cb-9470-46a5-bbe9-300e1425884a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:19.294489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b36d8-1409-4997-81e4-c251e832ed76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:19.295747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b36d8-1409-4997-81e4-c251e832ed76 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:19.296576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b36d8-1409-4997-81e4-c251e832ed76 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:19.297148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b36d8-1409-4997-81e4-c251e832ed76 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:19.298413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b36d8-1409-4997-81e4-c251e832ed76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:19.302963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b36d8-1409-4997-81e4-c251e832ed76 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:19.439809 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2845/5726] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:22:19 2022] POST /identity/v3/projects => generated 393 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:22:19.445555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a0d8a-d647-4bd7-ac3f-1ae1e5a4d673 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:19.470958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5a0d8a-d647-4bd7-ac3f-1ae1e5a4d673 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:19.701604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8ff0b-eee8-4173-891e-2b4f387d8c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:19.701604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8ff0b-eee8-4173-891e-2b4f387d8c91 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:19.701604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8ff0b-eee8-4173-891e-2b4f387d8c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:19.701604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8ff0b-eee8-4173-891e-2b4f387d8c91 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:19.702102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8ff0b-eee8-4173-891e-2b4f387d8c91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:19.703273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8ff0b-eee8-4173-891e-2b4f387d8c91 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:19.863796 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2882/5727] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:22:19 2022] POST /identity/v3/users => generated 303 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:22:19.867504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b36d8-1409-4997-81e4-c251e832ed76 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:19.881883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b36d8-1409-4997-81e4-c251e832ed76 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:19.899297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbf798-6a4d-4253-ab9d-f12fa0f74c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:19.899297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbf798-6a4d-4253-ab9d-f12fa0f74c88 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:19.899297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbf798-6a4d-4253-ab9d-f12fa0f74c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:19.899297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffbf798-6a4d-4253-ab9d-f12fa0f74c88 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:19.901658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbf798-6a4d-4253-ab9d-f12fa0f74c88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:19.904214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffbf798-6a4d-4253-ab9d-f12fa0f74c88 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:19.923840 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2846/5728] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:22:19.927810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8ff0b-eee8-4173-891e-2b4f387d8c91 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:19.946596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8ff0b-eee8-4173-891e-2b4f387d8c91 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:19.961040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b6c15-1f01-44a7-9595-eef5ca96ed2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:19.962066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b6c15-1f01-44a7-9595-eef5ca96ed2b admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:19.962352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b6c15-1f01-44a7-9595-eef5ca96ed2b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:19.962712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496b6c15-1f01-44a7-9595-eef5ca96ed2b admin admin] PATH_INFO: `/v3/projects/7dae9dd6b0b54bf2954f3d8427e092ee/users/e2da25bb9fdc42fbbc3665d6618ecd18/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:19.995741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b6c15-1f01-44a7-9595-eef5ca96ed2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dae9dd6b0b54bf2954f3d8427e092ee, user_id=e2da25bb9fdc42fbbc3665d6618ecd18, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:20.002201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496b6c15-1f01-44a7-9595-eef5ca96ed2b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:20.089353 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2883/5729] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:22:19 2022] PUT /identity/v3/projects/7dae9dd6b0b54bf2954f3d8427e092ee/users/e2da25bb9fdc42fbbc3665d6618ecd18/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:22:20.093472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbf798-6a4d-4253-ab9d-f12fa0f74c88 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:20.108127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b6c15-1f01-44a7-9595-eef5ca96ed2b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:20.110023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-496b6c15-1f01-44a7-9595-eef5ca96ed2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:20.111924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffbf798-6a4d-4253-ab9d-f12fa0f74c88 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:20.312691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496b6c15-1f01-44a7-9595-eef5ca96ed2b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:20.328255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23466168-d9f3-43d3-95af-a0719fd5b1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:20.329767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23466168-d9f3-43d3-95af-a0719fd5b1c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:20.331747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23466168-d9f3-43d3-95af-a0719fd5b1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:20.334792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23466168-d9f3-43d3-95af-a0719fd5b1c9 admin admin] PATH_INFO: `/v3/users/561bfffea82b4b9ea2a93ef49bd49af6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:20.338487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23466168-d9f3-43d3-95af-a0719fd5b1c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=561bfffea82b4b9ea2a93ef49bd49af6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:20.339228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621aa427-940c-45b9-9895-b82128297a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:20.340895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23466168-d9f3-43d3-95af-a0719fd5b1c9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:20.341123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621aa427-940c-45b9-9895-b82128297a24 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:20.342138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621aa427-940c-45b9-9895-b82128297a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:20.342726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621aa427-940c-45b9-9895-b82128297a24 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:20.360475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621aa427-940c-45b9-9895-b82128297a24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:20.363575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621aa427-940c-45b9-9895-b82128297a24 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:20.379410 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2847/5730] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:22:20.380737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:20.389148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621aa427-940c-45b9-9895-b82128297a24 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:20.405868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621aa427-940c-45b9-9895-b82128297a24 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:20.433357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf66f0f-cc19-4b7e-b072-73a115d0b390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:20.434353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf66f0f-cc19-4b7e-b072-73a115d0b390 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:20.434736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf66f0f-cc19-4b7e-b072-73a115d0b390 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:20.435098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf66f0f-cc19-4b7e-b072-73a115d0b390 admin admin] PATH_INFO: `/v3/projects/7dae9dd6b0b54bf2954f3d8427e092ee/users/e2da25bb9fdc42fbbc3665d6618ecd18/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:20.438268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf66f0f-cc19-4b7e-b072-73a115d0b390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dae9dd6b0b54bf2954f3d8427e092ee, user_id=e2da25bb9fdc42fbbc3665d6618ecd18, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:20.444215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf66f0f-cc19-4b7e-b072-73a115d0b390 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:20.477052 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2848/5731] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:22:20 2022] PUT /identity/v3/projects/7dae9dd6b0b54bf2954f3d8427e092ee/users/e2da25bb9fdc42fbbc3665d6618ecd18/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:22:20.487743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403d8df-c636-4117-861c-b927eee3b35f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:20.488274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403d8df-c636-4117-861c-b927eee3b35f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:20.488706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403d8df-c636-4117-861c-b927eee3b35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:20.579246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-23466168-d9f3-43d3-95af-a0719fd5b1c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '561bfffea82b4b9ea2a93ef49bd49af6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:20.608190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-23466168-d9f3-43d3-95af-a0719fd5b1c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '561bfffea82b4b9ea2a93ef49bd49af6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:20.611468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-23466168-d9f3-43d3-95af-a0719fd5b1c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '561bfffea82b4b9ea2a93ef49bd49af6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:20.613032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-23466168-d9f3-43d3-95af-a0719fd5b1c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '561bfffea82b4b9ea2a93ef49bd49af6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:20.614212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5403d8df-c636-4117-861c-b927eee3b35f None None] MFA Rules not processed for user `e2da25bb9fdc42fbbc3665d6618ecd18`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:20.618206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5403d8df-c636-4117-861c-b927eee3b35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:20.687701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-23466168-d9f3-43d3-95af-a0719fd5b1c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '561bfffea82b4b9ea2a93ef49bd49af6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:20.731151 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2884/5732] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:22:20 2022] DELETE /identity/v3/users/561bfffea82b4b9ea2a93ef49bd49af6 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:22:20.735713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fca7b-1071-470b-8276-3971f9e038bc None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:20.736362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fca7b-1071-470b-8276-3971f9e038bc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:20.737081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9fca7b-1071-470b-8276-3971f9e038bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:20.806814 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2849/5733] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:20.816298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ca9fca7b-1071-470b-8276-3971f9e038bc None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:20.817561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ca9fca7b-1071-470b-8276-3971f9e038bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:20.823047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403d8df-c636-4117-861c-b927eee3b35f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:20.825240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5403d8df-c636-4117-861c-b927eee3b35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:21.018785 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2885/5734] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:21.019720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:21.025310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403d8df-c636-4117-861c-b927eee3b35f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:21.102405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fca7b-1071-470b-8276-3971f9e038bc None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:21.105015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ca9fca7b-1071-470b-8276-3971f9e038bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:21.189948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9fca7b-1071-470b-8276-3971f9e038bc None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:21.243820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf5e68-437e-467f-b92b-f2f1cd843beb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:21.244974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf5e68-437e-467f-b92b-f2f1cd843beb service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:21.246562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf5e68-437e-467f-b92b-f2f1cd843beb service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:21.249223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cf5e68-437e-467f-b92b-f2f1cd843beb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:21.261280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69592cc1-bf42-481d-a5e4-d97ad7c4aa7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:21.266771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69592cc1-bf42-481d-a5e4-d97ad7c4aa7c service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:21.267593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69592cc1-bf42-481d-a5e4-d97ad7c4aa7c service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:21.267972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69592cc1-bf42-481d-a5e4-d97ad7c4aa7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:21.269112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf5e68-437e-467f-b92b-f2f1cd843beb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:21.272334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-69592cc1-bf42-481d-a5e4-d97ad7c4aa7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:21.273716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cf5e68-437e-467f-b92b-f2f1cd843beb service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:21.310282 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2886/5735] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:22:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:22:21.412096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:21.457502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69592cc1-bf42-481d-a5e4-d97ad7c4aa7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:21.462403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69592cc1-bf42-481d-a5e4-d97ad7c4aa7c service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:21.488373 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2850/5736] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:22:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:22:21.956958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:22.865327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:22.973358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:23.484428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf5e68-437e-467f-b92b-f2f1cd843beb service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:23.486233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-97cf5e68-437e-467f-b92b-f2f1cd843beb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:23.519304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cf5e68-437e-467f-b92b-f2f1cd843beb service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:23.546479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df691be8-8856-423e-b620-ffd0523f889c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:23.551239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df691be8-8856-423e-b620-ffd0523f889c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:23.554234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df691be8-8856-423e-b620-ffd0523f889c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:23.554723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df691be8-8856-423e-b620-ffd0523f889c admin admin] PATH_INFO: `/v3/projects/b63aa53e28e44141b5f714a06743af19` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:23.562912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df691be8-8856-423e-b620-ffd0523f889c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b63aa53e28e44141b5f714a06743af19)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:23.577372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df691be8-8856-423e-b620-ffd0523f889c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:23.830989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-df691be8-8856-423e-b620-ffd0523f889c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b63aa53e28e44141b5f714a06743af19', 'request_id': 'req-df691be8-8856-423e-b620-ffd0523f889c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:23.876966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-df691be8-8856-423e-b620-ffd0523f889c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b63aa53e28e44141b5f714a06743af19', 'request_id': 'req-df691be8-8856-423e-b620-ffd0523f889c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:23.980119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-df691be8-8856-423e-b620-ffd0523f889c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b63aa53e28e44141b5f714a06743af19', 'request_id': 'req-df691be8-8856-423e-b620-ffd0523f889c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:24.019404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-df691be8-8856-423e-b620-ffd0523f889c admin admin] The token cache is being invalidate because project b63aa53e28e44141b5f714a06743af19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:22:24.019404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-df691be8-8856-423e-b620-ffd0523f889c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:24.021660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-df691be8-8856-423e-b620-ffd0523f889c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:24.032456 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2887/5737] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:22:23 2022] DELETE /identity/v3/projects/b63aa53e28e44141b5f714a06743af19 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:22:24.036657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69592cc1-bf42-481d-a5e4-d97ad7c4aa7c service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:24.085059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69592cc1-bf42-481d-a5e4-d97ad7c4aa7c service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:24.262030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a085f-1f14-45b3-9e74-78187d6d9625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:24.263094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a085f-1f14-45b3-9e74-78187d6d9625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:24.263720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a085f-1f14-45b3-9e74-78187d6d9625 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:24.264367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a085f-1f14-45b3-9e74-78187d6d9625 admin admin] PATH_INFO: `/v3/users/b095b82889ce475bb15a11971f1c4a60` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:24.265888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a085f-1f14-45b3-9e74-78187d6d9625 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b095b82889ce475bb15a11971f1c4a60)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:24.276030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a085f-1f14-45b3-9e74-78187d6d9625 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:24.464690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:24.577981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e05a085f-1f14-45b3-9e74-78187d6d9625 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b095b82889ce475bb15a11971f1c4a60'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:24.579095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e05a085f-1f14-45b3-9e74-78187d6d9625 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b095b82889ce475bb15a11971f1c4a60'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:24.592785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e05a085f-1f14-45b3-9e74-78187d6d9625 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b095b82889ce475bb15a11971f1c4a60'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:24.699543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e05a085f-1f14-45b3-9e74-78187d6d9625 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b095b82889ce475bb15a11971f1c4a60'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:24.699543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e05a085f-1f14-45b3-9e74-78187d6d9625 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b095b82889ce475bb15a11971f1c4a60'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:24.740929 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2851/5738] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:22:24 2022] DELETE /identity/v3/users/b095b82889ce475bb15a11971f1c4a60 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:22:25.007605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:26.166665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:26.359318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df691be8-8856-423e-b620-ffd0523f889c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:26.362562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-df691be8-8856-423e-b620-ffd0523f889c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:26.390785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df691be8-8856-423e-b620-ffd0523f889c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:26.540967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce3de2-fdda-4ca1-8377-b64453aa313a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:26.542490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce3de2-fdda-4ca1-8377-b64453aa313a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:26.544073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce3de2-fdda-4ca1-8377-b64453aa313a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:26.545303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce3de2-fdda-4ca1-8377-b64453aa313a admin admin] PATH_INFO: `/v3/projects/7f402ea60d7c48a8b433695f43295461` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:26.548638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce3de2-fdda-4ca1-8377-b64453aa313a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f402ea60d7c48a8b433695f43295461)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:26.556068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce3de2-fdda-4ca1-8377-b64453aa313a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:26.658096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6307fa-20e3-4553-8685-7aad6f8801c3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:26.658599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6307fa-20e3-4553-8685-7aad6f8801c3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:26.658692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6307fa-20e3-4553-8685-7aad6f8801c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:26.683935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ac6307fa-20e3-4553-8685-7aad6f8801c3 None None] MFA Rules not processed for user `02d5fbb072814fd7923ecf12c7ecad7b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:26.686348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ac6307fa-20e3-4553-8685-7aad6f8801c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:26.745123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-97ce3de2-fdda-4ca1-8377-b64453aa313a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f402ea60d7c48a8b433695f43295461', 'request_id': 'req-97ce3de2-fdda-4ca1-8377-b64453aa313a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:26.756060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-97ce3de2-fdda-4ca1-8377-b64453aa313a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f402ea60d7c48a8b433695f43295461', 'request_id': 'req-97ce3de2-fdda-4ca1-8377-b64453aa313a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:26.769725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:26.816726 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2852/5739] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:22:26.830915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-97ce3de2-fdda-4ca1-8377-b64453aa313a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f402ea60d7c48a8b433695f43295461', 'request_id': 'req-97ce3de2-fdda-4ca1-8377-b64453aa313a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:26.857526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-97ce3de2-fdda-4ca1-8377-b64453aa313a admin admin] The token cache is being invalidate because project 7f402ea60d7c48a8b433695f43295461 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:22:26.862176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-97ce3de2-fdda-4ca1-8377-b64453aa313a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:26.864702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-97ce3de2-fdda-4ca1-8377-b64453aa313a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:26.873760 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2888/5740] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:22:26 2022] DELETE /identity/v3/projects/7f402ea60d7c48a8b433695f43295461 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:22:26.878470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07747283-c554-4c25-81c0-05b0fdb96fad None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:26.879884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07747283-c554-4c25-81c0-05b0fdb96fad None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:26.880819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07747283-c554-4c25-81c0-05b0fdb96fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:26.931806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-07747283-c554-4c25-81c0-05b0fdb96fad None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:26.934721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-07747283-c554-4c25-81c0-05b0fdb96fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:27.114769 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2853/5741] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:27.116655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bca55-c867-4f58-819b-13b43cc8c973 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:27.116954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bca55-c867-4f58-819b-13b43cc8c973 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:27.117631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bca55-c867-4f58-819b-13b43cc8c973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:27.152807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-013bca55-c867-4f58-819b-13b43cc8c973 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:27.154173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-013bca55-c867-4f58-819b-13b43cc8c973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:27.170925 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2889/5742] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:27.175864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07747283-c554-4c25-81c0-05b0fdb96fad None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:27.189538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07747283-c554-4c25-81c0-05b0fdb96fad None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:27.196412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb751124-4cd1-41ea-81ce-99bb1fc56807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:27.197724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb751124-4cd1-41ea-81ce-99bb1fc56807 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:27.198653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb751124-4cd1-41ea-81ce-99bb1fc56807 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:27.199152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb751124-4cd1-41ea-81ce-99bb1fc56807 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:27.200053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb751124-4cd1-41ea-81ce-99bb1fc56807 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:22:27.200573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb751124-4cd1-41ea-81ce-99bb1fc56807 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:27.203229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb751124-4cd1-41ea-81ce-99bb1fc56807 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:27.224426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bca55-c867-4f58-819b-13b43cc8c973 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:27.232836 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2854/5743] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:22:27.237335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bca55-c867-4f58-819b-13b43cc8c973 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:27.267440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111477ad-4971-4d49-b9c4-a19d181bcee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:27.277527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111477ad-4971-4d49-b9c4-a19d181bcee0 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:27.277815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111477ad-4971-4d49-b9c4-a19d181bcee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:27.283782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111477ad-4971-4d49-b9c4-a19d181bcee0 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:27.283782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111477ad-4971-4d49-b9c4-a19d181bcee0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:27.286269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111477ad-4971-4d49-b9c4-a19d181bcee0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:27.361537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:27.408442 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2890/5744] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:22:27 2022] POST /identity/v3/projects => generated 373 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:22:27.412427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb751124-4cd1-41ea-81ce-99bb1fc56807 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:27.452229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb751124-4cd1-41ea-81ce-99bb1fc56807 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:27.623672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4018d-dc6e-4a7e-aa16-ee3c79f1d767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:27.624296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4018d-dc6e-4a7e-aa16-ee3c79f1d767 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:27.624569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4018d-dc6e-4a7e-aa16-ee3c79f1d767 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:27.624848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4018d-dc6e-4a7e-aa16-ee3c79f1d767 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:27.625555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4018d-dc6e-4a7e-aa16-ee3c79f1d767 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:27.627544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4018d-dc6e-4a7e-aa16-ee3c79f1d767 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:27.741297 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2855/5745] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:22:27 2022] POST /identity/v3/users => generated 287 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:22:27.741995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111477ad-4971-4d49-b9c4-a19d181bcee0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:27.768228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111477ad-4971-4d49-b9c4-a19d181bcee0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:27.784846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442f1ad-f3bf-4cab-9cc0-fa1dcee2279f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:27.785544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442f1ad-f3bf-4cab-9cc0-fa1dcee2279f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:27.786557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442f1ad-f3bf-4cab-9cc0-fa1dcee2279f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:27.788133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8442f1ad-f3bf-4cab-9cc0-fa1dcee2279f admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:27.789581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442f1ad-f3bf-4cab-9cc0-fa1dcee2279f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:27.792795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8442f1ad-f3bf-4cab-9cc0-fa1dcee2279f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:27.822973 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2891/5746] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:22:27.826829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4018d-dc6e-4a7e-aa16-ee3c79f1d767 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:27.850399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4018d-dc6e-4a7e-aa16-ee3c79f1d767 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:27.862171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307d4ff-3e5e-49a2-ab84-6bbee247eca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:27.863670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307d4ff-3e5e-49a2-ab84-6bbee247eca8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:27.864814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307d4ff-3e5e-49a2-ab84-6bbee247eca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:27.865269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307d4ff-3e5e-49a2-ab84-6bbee247eca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:27.866246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307d4ff-3e5e-49a2-ab84-6bbee247eca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:27.868352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307d4ff-3e5e-49a2-ab84-6bbee247eca8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:27.889625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:27.890527 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2856/5747] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:22:27 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:22:27.892432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442f1ad-f3bf-4cab-9cc0-fa1dcee2279f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:27.920544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8442f1ad-f3bf-4cab-9cc0-fa1dcee2279f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:27.926813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1e068-609b-42e5-b9d3-ddb007d04dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:27.927996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1e068-609b-42e5-b9d3-ddb007d04dec admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:27.928756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1e068-609b-42e5-b9d3-ddb007d04dec admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:27.929259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1e068-609b-42e5-b9d3-ddb007d04dec admin admin] PATH_INFO: `/v3/projects/322f969ab3e143fc8bb2447775841516/users/6c97894b4544419ea9f91ec0a4fe4c41/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:27.983169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1e068-609b-42e5-b9d3-ddb007d04dec admin admin] RBAC: Authorizing `identity:create_grant(project_id=322f969ab3e143fc8bb2447775841516, user_id=6c97894b4544419ea9f91ec0a4fe4c41, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:27.997354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1e068-609b-42e5-b9d3-ddb007d04dec admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:28.094794 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2892/5748] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:22:27 2022] PUT /identity/v3/projects/322f969ab3e143fc8bb2447775841516/users/6c97894b4544419ea9f91ec0a4fe4c41/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:22:28.105207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541942f-9901-40c1-9a98-b88d7ecc1351 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:28.105875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541942f-9901-40c1-9a98-b88d7ecc1351 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:28.106010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541942f-9901-40c1-9a98-b88d7ecc1351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:28.214024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d541942f-9901-40c1-9a98-b88d7ecc1351 None None] MFA Rules not processed for user `6c97894b4544419ea9f91ec0a4fe4c41`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:28.216030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d541942f-9901-40c1-9a98-b88d7ecc1351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:28.333541 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2857/5749] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:28.389086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1e068-609b-42e5-b9d3-ddb007d04dec admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:28.392503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-24a1e068-609b-42e5-b9d3-ddb007d04dec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:28.577136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1e068-609b-42e5-b9d3-ddb007d04dec admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:28.587953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b12a3a-476c-4e57-898c-c5a151d67d21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:28.589067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b12a3a-476c-4e57-898c-c5a151d67d21 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:28.589777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b12a3a-476c-4e57-898c-c5a151d67d21 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:28.590260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b12a3a-476c-4e57-898c-c5a151d67d21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:28.630430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b12a3a-476c-4e57-898c-c5a151d67d21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:28.634990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b12a3a-476c-4e57-898c-c5a151d67d21 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:28.661129 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2893/5750] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:22:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:22:28.722600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:28.939512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:29.573916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:31.094229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:31.230179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:32.854296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:33.168055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:34.350057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:34.745166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:36.751464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:37.175537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fb958-81ef-48d7-95e5-a79b5413ab8d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:37.176034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fb958-81ef-48d7-95e5-a79b5413ab8d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:37.176128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1fb958-81ef-48d7-95e5-a79b5413ab8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:37.227678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8a1fb958-81ef-48d7-95e5-a79b5413ab8d None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:37.237181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8a1fb958-81ef-48d7-95e5-a79b5413ab8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:37.404464 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2858/5751] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:22:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:22:37.428748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b12a3a-476c-4e57-898c-c5a151d67d21 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:37.435663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-48b12a3a-476c-4e57-898c-c5a151d67d21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:37.588310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b12a3a-476c-4e57-898c-c5a151d67d21 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:37.601428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd755fb-a855-45e8-b214-857824abc039 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:37.602491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd755fb-a855-45e8-b214-857824abc039 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:37.602945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd755fb-a855-45e8-b214-857824abc039 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:37.603377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd755fb-a855-45e8-b214-857824abc039 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:37.622645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd755fb-a855-45e8-b214-857824abc039 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:37.626919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd755fb-a855-45e8-b214-857824abc039 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:37.657282 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2894/5752] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:22:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 21:22:38.131437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:38.707475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1fb958-81ef-48d7-95e5-a79b5413ab8d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:38.726152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1fb958-81ef-48d7-95e5-a79b5413ab8d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:38.739600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deee593-596a-4d1d-85ee-976eb4d951f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:38.740624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deee593-596a-4d1d-85ee-976eb4d951f8 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:38.745667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deee593-596a-4d1d-85ee-976eb4d951f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:38.750007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deee593-596a-4d1d-85ee-976eb4d951f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:38.755327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0deee593-596a-4d1d-85ee-976eb4d951f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:38.885598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deee593-596a-4d1d-85ee-976eb4d951f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:38.887522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deee593-596a-4d1d-85ee-976eb4d951f8 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:38.900512 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2859/5753] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:22:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:22:39.161780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:40.010115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:40.744491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:41.104251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af5e98-d481-47d5-ba60-a79571ebd273 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:41.104725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af5e98-d481-47d5-ba60-a79571ebd273 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:41.105268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af5e98-d481-47d5-ba60-a79571ebd273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:41.140412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-64af5e98-d481-47d5-ba60-a79571ebd273 None None] MFA Rules not processed for user `857a887b6c7c41588afb2845eed6f309`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:41.142713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-64af5e98-d481-47d5-ba60-a79571ebd273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:41.257552 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2895/5754] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:42.101683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deee593-596a-4d1d-85ee-976eb4d951f8 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:42.107354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0deee593-596a-4d1d-85ee-976eb4d951f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:42.213173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deee593-596a-4d1d-85ee-976eb4d951f8 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:42.219628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adaeb2c-3940-49d1-8710-09e333821b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:42.220862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaeb2c-3940-49d1-8710-09e333821b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:42.221440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaeb2c-3940-49d1-8710-09e333821b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:42.221926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaeb2c-3940-49d1-8710-09e333821b59 admin admin] PATH_INFO: `/v3/users/23cbce472f3147f385b3f952369bf1df` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:42.223777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adaeb2c-3940-49d1-8710-09e333821b59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23cbce472f3147f385b3f952369bf1df)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:42.226269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adaeb2c-3940-49d1-8710-09e333821b59 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:42.242287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:42.522201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9adaeb2c-3940-49d1-8710-09e333821b59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23cbce472f3147f385b3f952369bf1df'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:42.523167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9adaeb2c-3940-49d1-8710-09e333821b59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23cbce472f3147f385b3f952369bf1df'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:42.565793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9adaeb2c-3940-49d1-8710-09e333821b59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23cbce472f3147f385b3f952369bf1df'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:42.658871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9adaeb2c-3940-49d1-8710-09e333821b59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23cbce472f3147f385b3f952369bf1df'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:42.659605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9adaeb2c-3940-49d1-8710-09e333821b59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23cbce472f3147f385b3f952369bf1df'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:42.709022 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2860/5755] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:22:42 2022] DELETE /identity/v3/users/23cbce472f3147f385b3f952369bf1df => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:22:42.713621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74da38b-207e-412a-85a7-95a22a28db27 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:42.714923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74da38b-207e-412a-85a7-95a22a28db27 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:42.715234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74da38b-207e-412a-85a7-95a22a28db27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:42.778171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f74da38b-207e-412a-85a7-95a22a28db27 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:42.779126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f74da38b-207e-412a-85a7-95a22a28db27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:42.918889 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2896/5756] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:42.941872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adaeb2c-3940-49d1-8710-09e333821b59 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:42.943583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9adaeb2c-3940-49d1-8710-09e333821b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:43.123753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adaeb2c-3940-49d1-8710-09e333821b59 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:43.138923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb22c62-bb82-4f2e-b1e6-f5914f3009b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:43.140572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb22c62-bb82-4f2e-b1e6-f5914f3009b2 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:43.141169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb22c62-bb82-4f2e-b1e6-f5914f3009b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:43.141946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb22c62-bb82-4f2e-b1e6-f5914f3009b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:43.154868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb22c62-bb82-4f2e-b1e6-f5914f3009b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:43.156412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb22c62-bb82-4f2e-b1e6-f5914f3009b2 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:43.174164 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2861/5757] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:22:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:22:44.171503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74da38b-207e-412a-85a7-95a22a28db27 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:44.173205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f74da38b-207e-412a-85a7-95a22a28db27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:44.192490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74da38b-207e-412a-85a7-95a22a28db27 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:44.209202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f78290-c486-44de-bf8b-8fb740ff6d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:44.210837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f78290-c486-44de-bf8b-8fb740ff6d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:44.210837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f78290-c486-44de-bf8b-8fb740ff6d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:44.210837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f78290-c486-44de-bf8b-8fb740ff6d3a admin admin] PATH_INFO: `/v3/projects/5ecd1770c98247f596a01300a73d3ae0` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:44.217568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f78290-c486-44de-bf8b-8fb740ff6d3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ecd1770c98247f596a01300a73d3ae0)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:44.219587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f78290-c486-44de-bf8b-8fb740ff6d3a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:44.441673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:44.470035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e9f78290-c486-44de-bf8b-8fb740ff6d3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ecd1770c98247f596a01300a73d3ae0', 'request_id': 'req-e9f78290-c486-44de-bf8b-8fb740ff6d3a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:44.484070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e9f78290-c486-44de-bf8b-8fb740ff6d3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ecd1770c98247f596a01300a73d3ae0', 'request_id': 'req-e9f78290-c486-44de-bf8b-8fb740ff6d3a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:44.626505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e9f78290-c486-44de-bf8b-8fb740ff6d3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ecd1770c98247f596a01300a73d3ae0', 'request_id': 'req-e9f78290-c486-44de-bf8b-8fb740ff6d3a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:44.647809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e9f78290-c486-44de-bf8b-8fb740ff6d3a admin admin] The token cache is being invalidate because project 5ecd1770c98247f596a01300a73d3ae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:22:44.648430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e9f78290-c486-44de-bf8b-8fb740ff6d3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:44.650380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e9f78290-c486-44de-bf8b-8fb740ff6d3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:44.651044 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2897/5758] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:22:44 2022] DELETE /identity/v3/projects/5ecd1770c98247f596a01300a73d3ae0 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:22:44.656723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9aa35-e74d-4bf3-82e5-a1978b6347a7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:44.657186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9aa35-e74d-4bf3-82e5-a1978b6347a7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:44.657287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9aa35-e74d-4bf3-82e5-a1978b6347a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:44.698064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-10f9aa35-e74d-4bf3-82e5-a1978b6347a7 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:44.702806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-10f9aa35-e74d-4bf3-82e5-a1978b6347a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:44.821338 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2862/5759] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:44.830720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e959a4-0a84-4a23-bdff-07dbf0054809 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:44.831871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e959a4-0a84-4a23-bdff-07dbf0054809 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:44.832367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e959a4-0a84-4a23-bdff-07dbf0054809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:44.940712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-67e959a4-0a84-4a23-bdff-07dbf0054809 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:44.945465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-67e959a4-0a84-4a23-bdff-07dbf0054809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:44.987734 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2898/5760] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:44.994027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9aa35-e74d-4bf3-82e5-a1978b6347a7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:45.016550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9aa35-e74d-4bf3-82e5-a1978b6347a7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:45.042526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8594b-f265-48cd-a84e-5ce9df313e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:45.046164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8594b-f265-48cd-a84e-5ce9df313e67 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:45.046164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8594b-f265-48cd-a84e-5ce9df313e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:45.046164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8594b-f265-48cd-a84e-5ce9df313e67 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:45.046164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8594b-f265-48cd-a84e-5ce9df313e67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:22:45.048786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8594b-f265-48cd-a84e-5ce9df313e67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:45.052979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8594b-f265-48cd-a84e-5ce9df313e67 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:45.089330 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2863/5761] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:22:45.093427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e959a4-0a84-4a23-bdff-07dbf0054809 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:45.124401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e959a4-0a84-4a23-bdff-07dbf0054809 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:45.151419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5af955-d5a4-44f3-9b52-4881cde2c117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:45.153783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5af955-d5a4-44f3-9b52-4881cde2c117 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:45.154995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5af955-d5a4-44f3-9b52-4881cde2c117 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:45.158317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5af955-d5a4-44f3-9b52-4881cde2c117 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:45.159670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5af955-d5a4-44f3-9b52-4881cde2c117 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:45.162065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5af955-d5a4-44f3-9b52-4881cde2c117 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:45.328596 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2899/5762] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:22:45 2022] POST /identity/v3/projects => generated 375 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:22:45.338816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8594b-f265-48cd-a84e-5ce9df313e67 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:45.356811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8594b-f265-48cd-a84e-5ce9df313e67 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:45.413640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:45.539916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f39edb-89ea-4200-8b43-195bfccbabe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:45.541700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f39edb-89ea-4200-8b43-195bfccbabe0 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:45.542072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f39edb-89ea-4200-8b43-195bfccbabe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:45.542311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f39edb-89ea-4200-8b43-195bfccbabe0 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:45.546850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f39edb-89ea-4200-8b43-195bfccbabe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:45.554391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f39edb-89ea-4200-8b43-195bfccbabe0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:45.704418 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2864/5763] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:22:45 2022] POST /identity/v3/users => generated 288 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:22:45.708677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5af955-d5a4-44f3-9b52-4881cde2c117 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:45.729727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5af955-d5a4-44f3-9b52-4881cde2c117 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:45.750617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e975f9-a80a-4ae1-9062-ab4032330c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:45.751451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e975f9-a80a-4ae1-9062-ab4032330c1f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:45.754479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e975f9-a80a-4ae1-9062-ab4032330c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:45.754479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e975f9-a80a-4ae1-9062-ab4032330c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:45.754479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e975f9-a80a-4ae1-9062-ab4032330c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:45.758341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e975f9-a80a-4ae1-9062-ab4032330c1f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:45.790587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f39edb-89ea-4200-8b43-195bfccbabe0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:45.792663 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2900/5764] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:22:45.799607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f39edb-89ea-4200-8b43-195bfccbabe0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:45.819880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f8eff-c7f4-4317-a019-e47b639740a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:45.820895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f8eff-c7f4-4317-a019-e47b639740a2 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:45.823739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f8eff-c7f4-4317-a019-e47b639740a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:45.824273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f8eff-c7f4-4317-a019-e47b639740a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:45.825309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f8eff-c7f4-4317-a019-e47b639740a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:45.830284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f8eff-c7f4-4317-a019-e47b639740a2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:45.870235 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2865/5765] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:22:45.873213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e975f9-a80a-4ae1-9062-ab4032330c1f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:45.889437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e975f9-a80a-4ae1-9062-ab4032330c1f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:45.901001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c073f54-fcb2-4539-aa8e-cf1f727778e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:45.902040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c073f54-fcb2-4539-aa8e-cf1f727778e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:45.902524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c073f54-fcb2-4539-aa8e-cf1f727778e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:45.902982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c073f54-fcb2-4539-aa8e-cf1f727778e7 admin admin] PATH_INFO: `/v3/projects/bc0217da40f2457595bbd982919cf7be/users/742bf95e5af740ae9a8e460c22e30928/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:45.938582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c073f54-fcb2-4539-aa8e-cf1f727778e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc0217da40f2457595bbd982919cf7be, user_id=742bf95e5af740ae9a8e460c22e30928, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:45.944215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c073f54-fcb2-4539-aa8e-cf1f727778e7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:46.009254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f8eff-c7f4-4317-a019-e47b639740a2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:46.012354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-086f8eff-c7f4-4317-a019-e47b639740a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:46.052963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f8eff-c7f4-4317-a019-e47b639740a2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:46.059824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e097b-4cf7-4581-871b-e905d62a2469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:46.060732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e097b-4cf7-4581-871b-e905d62a2469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:46.061017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e097b-4cf7-4581-871b-e905d62a2469 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:46.061389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e097b-4cf7-4581-871b-e905d62a2469 admin admin] PATH_INFO: `/v3/users/84f18f11aad344c18bb6b3997370576b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:46.063010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e097b-4cf7-4581-871b-e905d62a2469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f18f11aad344c18bb6b3997370576b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:46.070123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e097b-4cf7-4581-871b-e905d62a2469 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:46.081932 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2901/5766] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:22:45 2022] PUT /identity/v3/projects/bc0217da40f2457595bbd982919cf7be/users/742bf95e5af740ae9a8e460c22e30928/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:22:46.092876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc154630-b413-4169-816b-7f1bfb07bcac None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:46.093549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc154630-b413-4169-816b-7f1bfb07bcac None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:46.094326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc154630-b413-4169-816b-7f1bfb07bcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:46.165411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-bc154630-b413-4169-816b-7f1bfb07bcac None None] MFA Rules not processed for user `742bf95e5af740ae9a8e460c22e30928`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:46.165411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bc154630-b413-4169-816b-7f1bfb07bcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:46.293668 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2902/5767] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:22:46.341931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc154630-b413-4169-816b-7f1bfb07bcac None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:46.347137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bc154630-b413-4169-816b-7f1bfb07bcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:46.355449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-389e097b-4cf7-4581-871b-e905d62a2469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f18f11aad344c18bb6b3997370576b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:46.356706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-389e097b-4cf7-4581-871b-e905d62a2469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f18f11aad344c18bb6b3997370576b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:46.371294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-389e097b-4cf7-4581-871b-e905d62a2469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f18f11aad344c18bb6b3997370576b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:46.464519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-389e097b-4cf7-4581-871b-e905d62a2469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f18f11aad344c18bb6b3997370576b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:46.465501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-389e097b-4cf7-4581-871b-e905d62a2469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f18f11aad344c18bb6b3997370576b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:46.490575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:46.505284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc154630-b413-4169-816b-7f1bfb07bcac None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:46.513491 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2866/5768] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:22:46 2022] DELETE /identity/v3/users/84f18f11aad344c18bb6b3997370576b => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:22:46.616336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84441bc-ff3b-4264-8f52-707b1791298e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:46.617462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84441bc-ff3b-4264-8f52-707b1791298e service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:46.618260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84441bc-ff3b-4264-8f52-707b1791298e service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:46.618659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84441bc-ff3b-4264-8f52-707b1791298e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:46.620713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e84441bc-ff3b-4264-8f52-707b1791298e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:46.752303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84441bc-ff3b-4264-8f52-707b1791298e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:46.756193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84441bc-ff3b-4264-8f52-707b1791298e service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:46.771704 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2903/5769] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:22:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:22:47.603937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e097b-4cf7-4581-871b-e905d62a2469 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:47.606424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-389e097b-4cf7-4581-871b-e905d62a2469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:47.784422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:47.845306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e097b-4cf7-4581-871b-e905d62a2469 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:47.852432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76746f-a801-4926-a789-c2e3f7388727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:47.853700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76746f-a801-4926-a789-c2e3f7388727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:47.854255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76746f-a801-4926-a789-c2e3f7388727 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:47.854776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76746f-a801-4926-a789-c2e3f7388727 admin admin] PATH_INFO: `/v3/projects/877bd1cd8912437c8e4871df8a5f7260` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:47.856301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c76746f-a801-4926-a789-c2e3f7388727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=877bd1cd8912437c8e4871df8a5f7260)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:47.858581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c76746f-a801-4926-a789-c2e3f7388727 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:48.041946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6c76746f-a801-4926-a789-c2e3f7388727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '877bd1cd8912437c8e4871df8a5f7260', 'request_id': 'req-6c76746f-a801-4926-a789-c2e3f7388727'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:48.055728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6c76746f-a801-4926-a789-c2e3f7388727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '877bd1cd8912437c8e4871df8a5f7260', 'request_id': 'req-6c76746f-a801-4926-a789-c2e3f7388727'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:48.150261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6c76746f-a801-4926-a789-c2e3f7388727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '877bd1cd8912437c8e4871df8a5f7260', 'request_id': 'req-6c76746f-a801-4926-a789-c2e3f7388727'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:48.169593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6c76746f-a801-4926-a789-c2e3f7388727 admin admin] The token cache is being invalidate because project 877bd1cd8912437c8e4871df8a5f7260 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:22:48.169908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6c76746f-a801-4926-a789-c2e3f7388727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:48.171595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6c76746f-a801-4926-a789-c2e3f7388727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:22:48.182108 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2867/5770] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:22:47 2022] DELETE /identity/v3/projects/877bd1cd8912437c8e4871df8a5f7260 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:22:48.187723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044f27d-e3a4-4968-ba5b-9fa5a7a3327b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:48.188227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044f27d-e3a4-4968-ba5b-9fa5a7a3327b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:48.188317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044f27d-e3a4-4968-ba5b-9fa5a7a3327b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:48.324129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c044f27d-e3a4-4968-ba5b-9fa5a7a3327b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:48.326318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c044f27d-e3a4-4968-ba5b-9fa5a7a3327b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:48.452449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63d54e-0d69-4a84-81b6-3408a28b02d8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:48.452809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63d54e-0d69-4a84-81b6-3408a28b02d8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:48.453013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e63d54e-0d69-4a84-81b6-3408a28b02d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:48.488323 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2904/5771] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:48.490889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93eaa5-f5de-449a-b441-19da20f21db8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:48.493558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93eaa5-f5de-449a-b441-19da20f21db8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:48.493988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db93eaa5-f5de-449a-b441-19da20f21db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:48.507749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3e63d54e-0d69-4a84-81b6-3408a28b02d8 None None] MFA Rules not processed for user `f3468a0b2d324603ab6c3080993bc0c8`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:48.517063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3e63d54e-0d69-4a84-81b6-3408a28b02d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:48.533096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-db93eaa5-f5de-449a-b441-19da20f21db8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:48.535187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-db93eaa5-f5de-449a-b441-19da20f21db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:48.553929 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2905/5772] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:48.555301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93eaa5-f5de-449a-b441-19da20f21db8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:48.571042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db93eaa5-f5de-449a-b441-19da20f21db8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:48.576463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081653f-896a-4aac-a570-b6a8142d4c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:48.578407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081653f-896a-4aac-a570-b6a8142d4c09 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:48.578932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081653f-896a-4aac-a570-b6a8142d4c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:48.579286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081653f-896a-4aac-a570-b6a8142d4c09 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:48.580158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081653f-896a-4aac-a570-b6a8142d4c09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:22:48.580535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081653f-896a-4aac-a570-b6a8142d4c09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:48.582486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081653f-896a-4aac-a570-b6a8142d4c09 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:48.599290 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2906/5773] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:22:48.607125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081653f-896a-4aac-a570-b6a8142d4c09 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:48.649339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081653f-896a-4aac-a570-b6a8142d4c09 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:48.674151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41760d84-6155-4319-9363-95a495135e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:48.677217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41760d84-6155-4319-9363-95a495135e66 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:48.678396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41760d84-6155-4319-9363-95a495135e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:48.682438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41760d84-6155-4319-9363-95a495135e66 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:48.682438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41760d84-6155-4319-9363-95a495135e66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:48.690420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41760d84-6155-4319-9363-95a495135e66 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:48.692983 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2868/5774] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:48.861470 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2907/5775] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:22:48 2022] POST /identity/v3/projects => generated 371 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:22:48.866014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63d54e-0d69-4a84-81b6-3408a28b02d8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:48.898641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e63d54e-0d69-4a84-81b6-3408a28b02d8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:48.943722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:49.070553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86c02e-9dc8-4e9f-a700-4e181a448fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:49.074524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86c02e-9dc8-4e9f-a700-4e181a448fa6 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:49.075276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86c02e-9dc8-4e9f-a700-4e181a448fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:49.075646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86c02e-9dc8-4e9f-a700-4e181a448fa6 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:49.078827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d86c02e-9dc8-4e9f-a700-4e181a448fa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:49.082765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d86c02e-9dc8-4e9f-a700-4e181a448fa6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:49.230845 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2869/5776] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:22:48 2022] POST /identity/v3/users => generated 286 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:22:49.232073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41760d84-6155-4319-9363-95a495135e66 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:49.250844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41760d84-6155-4319-9363-95a495135e66 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:49.261258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e023a-5dd7-48bb-b15b-9358022e98bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:49.263743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e023a-5dd7-48bb-b15b-9358022e98bb admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:49.264212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e023a-5dd7-48bb-b15b-9358022e98bb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:49.264643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e023a-5dd7-48bb-b15b-9358022e98bb admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:49.269212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e023a-5dd7-48bb-b15b-9358022e98bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:49.269212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e023a-5dd7-48bb-b15b-9358022e98bb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:49.287530 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2908/5777] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:22:49.293076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86c02e-9dc8-4e9f-a700-4e181a448fa6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:49.311774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86c02e-9dc8-4e9f-a700-4e181a448fa6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:49.323362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b11ff80-f436-44e6-a939-b45720982d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:49.324544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11ff80-f436-44e6-a939-b45720982d63 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:49.325580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11ff80-f436-44e6-a939-b45720982d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:49.326155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b11ff80-f436-44e6-a939-b45720982d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:49.327356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11ff80-f436-44e6-a939-b45720982d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:49.331748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b11ff80-f436-44e6-a939-b45720982d63 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:49.354854 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2870/5778] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:22:49.358307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e023a-5dd7-48bb-b15b-9358022e98bb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:49.374513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e023a-5dd7-48bb-b15b-9358022e98bb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:49.382544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa56aa-ccab-4ca3-b17b-0be52382e8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:49.383181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa56aa-ccab-4ca3-b17b-0be52382e8ee admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:49.383793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa56aa-ccab-4ca3-b17b-0be52382e8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:49.384276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa56aa-ccab-4ca3-b17b-0be52382e8ee admin admin] PATH_INFO: `/v3/projects/753b86e7a74c40369a4697ba5e7920d7/users/d8a35655682e4ece8da56d74d6f535cd/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:49.416719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa56aa-ccab-4ca3-b17b-0be52382e8ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=753b86e7a74c40369a4697ba5e7920d7, user_id=d8a35655682e4ece8da56d74d6f535cd, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:49.423274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa56aa-ccab-4ca3-b17b-0be52382e8ee admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:49.567907 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2909/5779] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:22:49 2022] PUT /identity/v3/projects/753b86e7a74c40369a4697ba5e7920d7/users/d8a35655682e4ece8da56d74d6f535cd/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:22:49.578663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806275e0-7e3f-4ab6-867c-a86c85b1298e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:49.579050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806275e0-7e3f-4ab6-867c-a86c85b1298e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:49.579428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806275e0-7e3f-4ab6-867c-a86c85b1298e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:49.615465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-806275e0-7e3f-4ab6-867c-a86c85b1298e None None] MFA Rules not processed for user `d8a35655682e4ece8da56d74d6f535cd`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:22:49.616818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-806275e0-7e3f-4ab6-867c-a86c85b1298e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:49.727125 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2871/5780] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:22:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:22:49.740081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa56aa-ccab-4ca3-b17b-0be52382e8ee admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:49.742060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e8aa56aa-ccab-4ca3-b17b-0be52382e8ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:49.795985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806275e0-7e3f-4ab6-867c-a86c85b1298e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:22:49.798080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-806275e0-7e3f-4ab6-867c-a86c85b1298e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:49.909381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa56aa-ccab-4ca3-b17b-0be52382e8ee admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:49.920705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8f4bd-b0d0-49ad-b87b-53fb6803bb15 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:49.921494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8f4bd-b0d0-49ad-b87b-53fb6803bb15 service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:49.921873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8f4bd-b0d0-49ad-b87b-53fb6803bb15 service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:49.923229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8f4bd-b0d0-49ad-b87b-53fb6803bb15 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:49.951806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806275e0-7e3f-4ab6-867c-a86c85b1298e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:22:49.963332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a08f00-fd9a-4500-bb8c-424fe10f10ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:22:49.963594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8f4bd-b0d0-49ad-b87b-53fb6803bb15 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:49.964041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a08f00-fd9a-4500-bb8c-424fe10f10ff service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:22:49.964507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a08f00-fd9a-4500-bb8c-424fe10f10ff service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:22:49.964948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a08f00-fd9a-4500-bb8c-424fe10f10ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:22:49.968430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8f4bd-b0d0-49ad-b87b-53fb6803bb15 service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:49.969352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-38a08f00-fd9a-4500-bb8c-424fe10f10ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:22:50.018187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:50.024559 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2910/5781] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:22:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:22:50.181622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a08f00-fd9a-4500-bb8c-424fe10f10ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:22:50.182311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a08f00-fd9a-4500-bb8c-424fe10f10ff service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:22:50.198367 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2872/5782] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:22:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:22:59.988225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=46,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:01.323245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8f4bd-b0d0-49ad-b87b-53fb6803bb15 service glance] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:01.325937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bef8f4bd-b0d0-49ad-b87b-53fb6803bb15 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:01.413904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8f4bd-b0d0-49ad-b87b-53fb6803bb15 service glance] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:01.430521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfad9a4-73a4-454b-926d-f406e64e625d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:01.430521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfad9a4-73a4-454b-926d-f406e64e625d service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:01.430521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfad9a4-73a4-454b-926d-f406e64e625d service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:01.430521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfad9a4-73a4-454b-926d-f406e64e625d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:01.431255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7dfad9a4-73a4-454b-926d-f406e64e625d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:01.584541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfad9a4-73a4-454b-926d-f406e64e625d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:01.586905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfad9a4-73a4-454b-926d-f406e64e625d service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:01.608131 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2911/5783] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:23:02.444450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:04.386578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a08f00-fd9a-4500-bb8c-424fe10f10ff service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:04.416533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a08f00-fd9a-4500-bb8c-424fe10f10ff service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:04.427309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35eca57-ebc4-43ab-ade1-063628e6b1e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:04.428085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35eca57-ebc4-43ab-ade1-063628e6b1e9 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:04.428284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35eca57-ebc4-43ab-ade1-063628e6b1e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:04.428572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35eca57-ebc4-43ab-ade1-063628e6b1e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:04.433359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a35eca57-ebc4-43ab-ade1-063628e6b1e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:04.492811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35eca57-ebc4-43ab-ade1-063628e6b1e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:04.495334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35eca57-ebc4-43ab-ade1-063628e6b1e9 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:04.512576 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2873/5784] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:23:10.078717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfad9a4-73a4-454b-926d-f406e64e625d service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:10.080490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7dfad9a4-73a4-454b-926d-f406e64e625d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:10.105336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfad9a4-73a4-454b-926d-f406e64e625d service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:10.119487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af29631-a217-4184-addc-9ee805569fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:10.133614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af29631-a217-4184-addc-9ee805569fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:10.135093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af29631-a217-4184-addc-9ee805569fae admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:10.140183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af29631-a217-4184-addc-9ee805569fae admin admin] PATH_INFO: `/v3/users/742bf95e5af740ae9a8e460c22e30928` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:10.146066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af29631-a217-4184-addc-9ee805569fae admin admin] RBAC: Authorizing `identity:delete_user(user_id=742bf95e5af740ae9a8e460c22e30928)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:10.152389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af29631-a217-4184-addc-9ee805569fae admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:10.380523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca6c0e-b649-47e7-a3be-346d6eacf3c4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:10.381023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca6c0e-b649-47e7-a3be-346d6eacf3c4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:10.381119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ca6c0e-b649-47e7-a3be-346d6eacf3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:10.431134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0af29631-a217-4184-addc-9ee805569fae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '742bf95e5af740ae9a8e460c22e30928'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:10.451503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0af29631-a217-4184-addc-9ee805569fae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '742bf95e5af740ae9a8e460c22e30928'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:10.453668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0af29631-a217-4184-addc-9ee805569fae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '742bf95e5af740ae9a8e460c22e30928'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:10.455658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0af29631-a217-4184-addc-9ee805569fae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '742bf95e5af740ae9a8e460c22e30928'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:10.455930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d7ca6c0e-b649-47e7-a3be-346d6eacf3c4 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:10.457159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d7ca6c0e-b649-47e7-a3be-346d6eacf3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:10.554532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0af29631-a217-4184-addc-9ee805569fae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '742bf95e5af740ae9a8e460c22e30928'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:10.597949 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2912/5785] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:23:10 2022] DELETE /identity/v3/users/742bf95e5af740ae9a8e460c22e30928 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:10.602848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933e225-d747-40aa-92d4-ee413a1bb013 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:10.606429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933e225-d747-40aa-92d4-ee413a1bb013 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:10.606429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0933e225-d747-40aa-92d4-ee413a1bb013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:10.606760 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2874/5786] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:23:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:10.643639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca6c0e-b649-47e7-a3be-346d6eacf3c4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:10.650522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d7ca6c0e-b649-47e7-a3be-346d6eacf3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:10.667497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0933e225-d747-40aa-92d4-ee413a1bb013 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:10.669709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0933e225-d747-40aa-92d4-ee413a1bb013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:10.682314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:10.803535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ca6c0e-b649-47e7-a3be-346d6eacf3c4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:10.812684 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2913/5787] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:23:10.901390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933e225-d747-40aa-92d4-ee413a1bb013 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:10.904352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0933e225-d747-40aa-92d4-ee413a1bb013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:10.945190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf711b13-662f-4cd8-a509-696c2850f2cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:10.946058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf711b13-662f-4cd8-a509-696c2850f2cb service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:10.946345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf711b13-662f-4cd8-a509-696c2850f2cb service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:10.946626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf711b13-662f-4cd8-a509-696c2850f2cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:10.976774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:11.035652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0933e225-d747-40aa-92d4-ee413a1bb013 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:11.040419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e615a1-ebce-443f-bbc7-5d94e0334d0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:11.041082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e615a1-ebce-443f-bbc7-5d94e0334d0a service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:11.041365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e615a1-ebce-443f-bbc7-5d94e0334d0a service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:11.041643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e615a1-ebce-443f-bbc7-5d94e0334d0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:11.076773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e615a1-ebce-443f-bbc7-5d94e0334d0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:11.082663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e615a1-ebce-443f-bbc7-5d94e0334d0a service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:11.101692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf711b13-662f-4cd8-a509-696c2850f2cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:11.107274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf711b13-662f-4cd8-a509-696c2850f2cb service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:11.120999 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2914/5788] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:23:11.143067 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2875/5789] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:23:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 21:23:12.798613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e615a1-ebce-443f-bbc7-5d94e0334d0a service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:12.800252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e8e615a1-ebce-443f-bbc7-5d94e0334d0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:12.825619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e615a1-ebce-443f-bbc7-5d94e0334d0a service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:12.852617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ad86a-c432-4a5f-ba8d-1efea5873112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:12.853536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ad86a-c432-4a5f-ba8d-1efea5873112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:12.853872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ad86a-c432-4a5f-ba8d-1efea5873112 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:12.854156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ad86a-c432-4a5f-ba8d-1efea5873112 admin admin] PATH_INFO: `/v3/projects/bc0217da40f2457595bbd982919cf7be` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:12.867190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ad86a-c432-4a5f-ba8d-1efea5873112 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc0217da40f2457595bbd982919cf7be)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:12.872713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ad86a-c432-4a5f-ba8d-1efea5873112 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:13.113494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-282ad86a-c432-4a5f-ba8d-1efea5873112 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc0217da40f2457595bbd982919cf7be', 'request_id': 'req-282ad86a-c432-4a5f-ba8d-1efea5873112'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:13.113494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-282ad86a-c432-4a5f-ba8d-1efea5873112 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc0217da40f2457595bbd982919cf7be', 'request_id': 'req-282ad86a-c432-4a5f-ba8d-1efea5873112'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:13.191192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-282ad86a-c432-4a5f-ba8d-1efea5873112 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc0217da40f2457595bbd982919cf7be', 'request_id': 'req-282ad86a-c432-4a5f-ba8d-1efea5873112'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:13.240271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-282ad86a-c432-4a5f-ba8d-1efea5873112 admin admin] The token cache is being invalidate because project bc0217da40f2457595bbd982919cf7be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:23:13.240897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-282ad86a-c432-4a5f-ba8d-1efea5873112 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:13.241736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-282ad86a-c432-4a5f-ba8d-1efea5873112 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:13.244389 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2915/5790] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:23:12 2022] DELETE /identity/v3/projects/bc0217da40f2457595bbd982919cf7be => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:13.253077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8e30d-3e33-4225-8920-64e50fd13b3f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:13.253563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8e30d-3e33-4225-8920-64e50fd13b3f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:13.253807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8e30d-3e33-4225-8920-64e50fd13b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:13.297276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a7e8e30d-3e33-4225-8920-64e50fd13b3f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:13.298556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a7e8e30d-3e33-4225-8920-64e50fd13b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:13.316502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ad86a-c432-4a5f-ba8d-1efea5873112 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:13.318016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-282ad86a-c432-4a5f-ba8d-1efea5873112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:13.454696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:13.488041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ad86a-c432-4a5f-ba8d-1efea5873112 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:13.501737 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2876/5791] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:13.501737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d43f8e-56e7-4beb-b852-1d273350040f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:13.501737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d43f8e-56e7-4beb-b852-1d273350040f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:13.501737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d43f8e-56e7-4beb-b852-1d273350040f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:13.503648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77382be4-03d5-49e5-bc3b-b86647be2aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:13.504746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77382be4-03d5-49e5-bc3b-b86647be2aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:13.506025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77382be4-03d5-49e5-bc3b-b86647be2aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:13.506490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77382be4-03d5-49e5-bc3b-b86647be2aaf admin admin] PATH_INFO: `/v3/users/f1acb2fffdc64ce29b71b2b40e2285f0` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:13.508774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77382be4-03d5-49e5-bc3b-b86647be2aaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1acb2fffdc64ce29b71b2b40e2285f0)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:13.511036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77382be4-03d5-49e5-bc3b-b86647be2aaf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:13.556087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-58d43f8e-56e7-4beb-b852-1d273350040f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:13.558645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-58d43f8e-56e7-4beb-b852-1d273350040f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:13.594035 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2877/5792] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:13.597560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d43f8e-56e7-4beb-b852-1d273350040f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:13.610492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d43f8e-56e7-4beb-b852-1d273350040f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:13.623056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16e42b-b4a4-46da-aad8-dbe748a09b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:13.623965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16e42b-b4a4-46da-aad8-dbe748a09b67 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:13.624415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16e42b-b4a4-46da-aad8-dbe748a09b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:13.624924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e16e42b-b4a4-46da-aad8-dbe748a09b67 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:13.626150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16e42b-b4a4-46da-aad8-dbe748a09b67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:23:13.626780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16e42b-b4a4-46da-aad8-dbe748a09b67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:13.629364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e16e42b-b4a4-46da-aad8-dbe748a09b67 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:13.652618 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2878/5793] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:23:13.657046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16e42b-b4a4-46da-aad8-dbe748a09b67 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:13.713170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e16e42b-b4a4-46da-aad8-dbe748a09b67 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:13.723151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f4e41-900a-4acb-b7dc-05bbcbca44f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:13.730648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f4e41-900a-4acb-b7dc-05bbcbca44f1 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:13.737653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f4e41-900a-4acb-b7dc-05bbcbca44f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:13.737653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44f4e41-900a-4acb-b7dc-05bbcbca44f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:13.737653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f4e41-900a-4acb-b7dc-05bbcbca44f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:13.738710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44f4e41-900a-4acb-b7dc-05bbcbca44f1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:13.770533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-77382be4-03d5-49e5-bc3b-b86647be2aaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1acb2fffdc64ce29b71b2b40e2285f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:13.804944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-77382be4-03d5-49e5-bc3b-b86647be2aaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1acb2fffdc64ce29b71b2b40e2285f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:13.806898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-77382be4-03d5-49e5-bc3b-b86647be2aaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1acb2fffdc64ce29b71b2b40e2285f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:13.808303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-77382be4-03d5-49e5-bc3b-b86647be2aaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1acb2fffdc64ce29b71b2b40e2285f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:13.839184 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2879/5794] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:23:13 2022] POST /identity/v3/projects => generated 387 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:23:13.846909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f4e41-900a-4acb-b7dc-05bbcbca44f1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:13.866420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d44f4e41-900a-4acb-b7dc-05bbcbca44f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:13.886601 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-77382be4-03d5-49e5-bc3b-b86647be2aaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1acb2fffdc64ce29b71b2b40e2285f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:13.950805 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2916/5795] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:23:13 2022] DELETE /identity/v3/users/f1acb2fffdc64ce29b71b2b40e2285f0 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:14.165984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44f4e41-900a-4acb-b7dc-05bbcbca44f1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:14.228842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:14.304291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788ad01-d6ce-4484-83b7-8852ef1d26cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:14.305822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788ad01-d6ce-4484-83b7-8852ef1d26cf admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:14.315565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788ad01-d6ce-4484-83b7-8852ef1d26cf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:14.326278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788ad01-d6ce-4484-83b7-8852ef1d26cf admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:14.327119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2788ad01-d6ce-4484-83b7-8852ef1d26cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:14.332996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2788ad01-d6ce-4484-83b7-8852ef1d26cf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:14.491477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77382be4-03d5-49e5-bc3b-b86647be2aaf admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:14.502675 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2880/5796] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:23:13 2022] POST /identity/v3/users => generated 294 bytes in 642 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:23:14.505325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77382be4-03d5-49e5-bc3b-b86647be2aaf admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:14.529609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7a46d-48c3-474a-aff0-4c90eb88c428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:14.530758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7a46d-48c3-474a-aff0-4c90eb88c428 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:14.532135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7a46d-48c3-474a-aff0-4c90eb88c428 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:14.532677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7a46d-48c3-474a-aff0-4c90eb88c428 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:14.538349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7a46d-48c3-474a-aff0-4c90eb88c428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:14.539991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7a46d-48c3-474a-aff0-4c90eb88c428 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:14.583193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788ad01-d6ce-4484-83b7-8852ef1d26cf admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:14.593486 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2917/5797] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:14.597967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788ad01-d6ce-4484-83b7-8852ef1d26cf admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:14.622102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b31b9-a34c-47fd-8a73-504bfd5f83e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:14.622779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b31b9-a34c-47fd-8a73-504bfd5f83e5 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:14.623047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b31b9-a34c-47fd-8a73-504bfd5f83e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:14.623325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b31b9-a34c-47fd-8a73-504bfd5f83e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:14.624065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b31b9-a34c-47fd-8a73-504bfd5f83e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:14.625703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b31b9-a34c-47fd-8a73-504bfd5f83e5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:14.673111 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2881/5798] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:14.678513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7a46d-48c3-474a-aff0-4c90eb88c428 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:14.688420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7a46d-48c3-474a-aff0-4c90eb88c428 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:14.718519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b31b9-a34c-47fd-8a73-504bfd5f83e5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:14.720263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e61b31b9-a34c-47fd-8a73-504bfd5f83e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:14.726094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdd9c3-d863-453b-8c71-69838449d1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:14.727111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdd9c3-d863-453b-8c71-69838449d1b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:14.727398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdd9c3-d863-453b-8c71-69838449d1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:14.727687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdd9c3-d863-453b-8c71-69838449d1b6 admin admin] PATH_INFO: `/v3/projects/ce4ccb9f2fb945e19bfed037a38ea59d/users/f525c29419db48e684dd53c758abb74b/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:14.752819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b31b9-a34c-47fd-8a73-504bfd5f83e5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:14.767792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6c343-d23b-4045-af7b-b9c50127c8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:14.769985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c343-d23b-4045-af7b-b9c50127c8bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:14.772269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c343-d23b-4045-af7b-b9c50127c8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:14.776202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6c343-d23b-4045-af7b-b9c50127c8bd admin admin] PATH_INFO: `/v3/users/eb7c3a01b54f4db2841f4836635eff2e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:14.781795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdd9c3-d863-453b-8c71-69838449d1b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce4ccb9f2fb945e19bfed037a38ea59d, user_id=f525c29419db48e684dd53c758abb74b, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:14.784546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdd9c3-d863-453b-8c71-69838449d1b6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:14.818451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6c343-d23b-4045-af7b-b9c50127c8bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb7c3a01b54f4db2841f4836635eff2e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:14.822917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6c343-d23b-4045-af7b-b9c50127c8bd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:14.938024 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2918/5799] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:23:14 2022] PUT /identity/v3/projects/ce4ccb9f2fb945e19bfed037a38ea59d/users/f525c29419db48e684dd53c758abb74b/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:23:14.948883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea95314-7f17-4afc-b9dc-2761c0b2d5f2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:14.949211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea95314-7f17-4afc-b9dc-2761c0b2d5f2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:14.949974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea95314-7f17-4afc-b9dc-2761c0b2d5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:15.024669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5ea95314-7f17-4afc-b9dc-2761c0b2d5f2 None None] MFA Rules not processed for user `f525c29419db48e684dd53c758abb74b`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:15.027129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5ea95314-7f17-4afc-b9dc-2761c0b2d5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:15.129595 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2919/5800] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:15.130647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea95314-7f17-4afc-b9dc-2761c0b2d5f2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:15.132604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5ea95314-7f17-4afc-b9dc-2761c0b2d5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:15.149271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-abe6c343-d23b-4045-af7b-b9c50127c8bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb7c3a01b54f4db2841f4836635eff2e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:15.151063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-abe6c343-d23b-4045-af7b-b9c50127c8bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb7c3a01b54f4db2841f4836635eff2e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:15.153176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:15.190844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-abe6c343-d23b-4045-af7b-b9c50127c8bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb7c3a01b54f4db2841f4836635eff2e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:15.225487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea95314-7f17-4afc-b9dc-2761c0b2d5f2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:15.237537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef277a36-89bc-4e61-90d6-2d2f3044acac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:15.238926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef277a36-89bc-4e61-90d6-2d2f3044acac admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:15.239452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef277a36-89bc-4e61-90d6-2d2f3044acac admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:15.239836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef277a36-89bc-4e61-90d6-2d2f3044acac admin admin] PATH_INFO: `/v3/projects/3c6886d8b684413da24b02f5b5fb6ef8` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:15.242154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef277a36-89bc-4e61-90d6-2d2f3044acac admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c6886d8b684413da24b02f5b5fb6ef8)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:15.244545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef277a36-89bc-4e61-90d6-2d2f3044acac admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:15.287857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-abe6c343-d23b-4045-af7b-b9c50127c8bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb7c3a01b54f4db2841f4836635eff2e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:15.288848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-abe6c343-d23b-4045-af7b-b9c50127c8bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb7c3a01b54f4db2841f4836635eff2e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:15.325091 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2882/5801] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:23:14 2022] DELETE /identity/v3/users/eb7c3a01b54f4db2841f4836635eff2e => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:23:15.325091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6c343-d23b-4045-af7b-b9c50127c8bd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:15.325091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-abe6c343-d23b-4045-af7b-b9c50127c8bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:15.406513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ef277a36-89bc-4e61-90d6-2d2f3044acac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c6886d8b684413da24b02f5b5fb6ef8', 'request_id': 'req-ef277a36-89bc-4e61-90d6-2d2f3044acac'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:15.419031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ef277a36-89bc-4e61-90d6-2d2f3044acac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c6886d8b684413da24b02f5b5fb6ef8', 'request_id': 'req-ef277a36-89bc-4e61-90d6-2d2f3044acac'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:15.517765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6c343-d23b-4045-af7b-b9c50127c8bd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:15.526434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ef277a36-89bc-4e61-90d6-2d2f3044acac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c6886d8b684413da24b02f5b5fb6ef8', 'request_id': 'req-ef277a36-89bc-4e61-90d6-2d2f3044acac'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:15.550211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ef277a36-89bc-4e61-90d6-2d2f3044acac admin admin] The token cache is being invalidate because project 3c6886d8b684413da24b02f5b5fb6ef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:23:15.551066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:15.553583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ef277a36-89bc-4e61-90d6-2d2f3044acac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:15.556086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ef277a36-89bc-4e61-90d6-2d2f3044acac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:15.560873 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2920/5802] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:23:15 2022] DELETE /identity/v3/projects/3c6886d8b684413da24b02f5b5fb6ef8 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:15.566371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942f962-411f-45b6-b2f8-11f51775ef3f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:15.568730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942f962-411f-45b6-b2f8-11f51775ef3f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:15.571025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942f962-411f-45b6-b2f8-11f51775ef3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:15.631355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4942f962-411f-45b6-b2f8-11f51775ef3f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:15.632567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4942f962-411f-45b6-b2f8-11f51775ef3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:15.642274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ba03b-e462-405f-83f9-f3aded97d8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:15.643064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ba03b-e462-405f-83f9-f3aded97d8e5 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:15.643535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ba03b-e462-405f-83f9-f3aded97d8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:15.643908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ba03b-e462-405f-83f9-f3aded97d8e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:15.645035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ba03b-e462-405f-83f9-f3aded97d8e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:15.648695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ba03b-e462-405f-83f9-f3aded97d8e5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:15.657792 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2921/5803] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:15.668467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04e1cd-97f9-467a-90d8-7777384ee1d2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:15.669880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04e1cd-97f9-467a-90d8-7777384ee1d2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:15.669880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f04e1cd-97f9-467a-90d8-7777384ee1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:15.702452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4f04e1cd-97f9-467a-90d8-7777384ee1d2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:15.703719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4f04e1cd-97f9-467a-90d8-7777384ee1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:15.713823 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2922/5804] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:15.718614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04e1cd-97f9-467a-90d8-7777384ee1d2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:15.735689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f04e1cd-97f9-467a-90d8-7777384ee1d2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:15.740893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d34839-29be-454c-b82f-ff5aaa8f6a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:15.741569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d34839-29be-454c-b82f-ff5aaa8f6a04 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:15.741871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d34839-29be-454c-b82f-ff5aaa8f6a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:15.742165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d34839-29be-454c-b82f-ff5aaa8f6a04 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:15.742971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d34839-29be-454c-b82f-ff5aaa8f6a04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:23:15.743291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d34839-29be-454c-b82f-ff5aaa8f6a04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:15.744964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d34839-29be-454c-b82f-ff5aaa8f6a04 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:15.765892 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2923/5805] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:23:15.768532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d34839-29be-454c-b82f-ff5aaa8f6a04 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:15.786231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d34839-29be-454c-b82f-ff5aaa8f6a04 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:15.791365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2d778-9d15-4e9d-b273-38f050da1f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:15.792185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2d778-9d15-4e9d-b273-38f050da1f70 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:15.792695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2d778-9d15-4e9d-b273-38f050da1f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:15.793714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2d778-9d15-4e9d-b273-38f050da1f70 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:15.794787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2d778-9d15-4e9d-b273-38f050da1f70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:15.797070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e2d778-9d15-4e9d-b273-38f050da1f70 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:15.804451 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2883/5806] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:23:15 2022] POST /identity/v3/projects => generated 387 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:23:15.809727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ba03b-e462-405f-83f9-f3aded97d8e5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:15.815074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4b4ba03b-e462-405f-83f9-f3aded97d8e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:15.958037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ba03b-e462-405f-83f9-f3aded97d8e5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:15.958319 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2924/5807] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:23:15 2022] POST /identity/v3/projects => generated 371 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:23:15.960200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2d778-9d15-4e9d-b273-38f050da1f70 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:15.963540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a6e2d778-9d15-4e9d-b273-38f050da1f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:16.051575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b0afc-5c6f-4370-a479-c421f3ff8c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:16.053502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b0afc-5c6f-4370-a479-c421f3ff8c28 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:16.053502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b0afc-5c6f-4370-a479-c421f3ff8c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:16.053502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b0afc-5c6f-4370-a479-c421f3ff8c28 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:16.053982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b0afc-5c6f-4370-a479-c421f3ff8c28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:16.056862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b0afc-5c6f-4370-a479-c421f3ff8c28 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:16.070487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e2d778-9d15-4e9d-b273-38f050da1f70 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:16.113960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:16.118544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:16.123151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:16.123628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9 admin admin] PATH_INFO: `/v3/projects/ca680361a7b349e39f77a7f7c96934f8` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:16.145672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca680361a7b349e39f77a7f7c96934f8)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:16.148483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:16.261242 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2884/5808] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:23:15 2022] POST /identity/v3/users => generated 294 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:23:16.263364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b0afc-5c6f-4370-a479-c421f3ff8c28 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:16.293982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b0afc-5c6f-4370-a479-c421f3ff8c28 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:16.302892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36241dda-f117-45c4-bcd3-e7e7b5ca2544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:16.304216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36241dda-f117-45c4-bcd3-e7e7b5ca2544 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:16.304743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36241dda-f117-45c4-bcd3-e7e7b5ca2544 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:16.305383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36241dda-f117-45c4-bcd3-e7e7b5ca2544 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:16.306358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36241dda-f117-45c4-bcd3-e7e7b5ca2544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:16.308808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36241dda-f117-45c4-bcd3-e7e7b5ca2544 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:16.374542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca680361a7b349e39f77a7f7c96934f8', 'request_id': 'req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:16.381226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca680361a7b349e39f77a7f7c96934f8', 'request_id': 'req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:16.430878 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2885/5809] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:23:16 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:23:16.435087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36241dda-f117-45c4-bcd3-e7e7b5ca2544 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:16.470506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca680361a7b349e39f77a7f7c96934f8', 'request_id': 'req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:16.493721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9 admin admin] The token cache is being invalidate because project ca680361a7b349e39f77a7f7c96934f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:23:16.494199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:16.494325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36241dda-f117-45c4-bcd3-e7e7b5ca2544 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:16.495716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:16.501879 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2925/5810] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:23:15 2022] DELETE /identity/v3/projects/ca680361a7b349e39f77a7f7c96934f8 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:16.502674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:16.505524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:16.625246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23080686-ac7e-4fe3-9ae8-0114b1dd4da9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:16.638754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf1548-ad37-4108-a30b-95c170871d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:16.640732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf1548-ad37-4108-a30b-95c170871d6e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:16.642055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf1548-ad37-4108-a30b-95c170871d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:16.643864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf1548-ad37-4108-a30b-95c170871d6e admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:16.646421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf1548-ad37-4108-a30b-95c170871d6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:16.650330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf1548-ad37-4108-a30b-95c170871d6e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:16.668892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e18d5b-7897-47b7-beff-3d96ee0f48ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:16.669593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e18d5b-7897-47b7-beff-3d96ee0f48ff admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:16.670037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e18d5b-7897-47b7-beff-3d96ee0f48ff admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:16.670762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e18d5b-7897-47b7-beff-3d96ee0f48ff admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:16.671530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e18d5b-7897-47b7-beff-3d96ee0f48ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:16.674171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e18d5b-7897-47b7-beff-3d96ee0f48ff admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:16.680457 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2926/5811] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:16.683278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf1548-ad37-4108-a30b-95c170871d6e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:16.692419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9cdf1548-ad37-4108-a30b-95c170871d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:16.695087 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2886/5812] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:16.696075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e18d5b-7897-47b7-beff-3d96ee0f48ff admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:16.710468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e18d5b-7897-47b7-beff-3d96ee0f48ff admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:16.716975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf1548-ad37-4108-a30b-95c170871d6e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:16.717146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf21665-93e3-4650-b9ab-d16dcd0aef75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:16.717775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf21665-93e3-4650-b9ab-d16dcd0aef75 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:16.718114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf21665-93e3-4650-b9ab-d16dcd0aef75 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:16.718401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf21665-93e3-4650-b9ab-d16dcd0aef75 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:16.720265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf21665-93e3-4650-b9ab-d16dcd0aef75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:16.723186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf21665-93e3-4650-b9ab-d16dcd0aef75 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:16.729241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef22ac-0f64-49df-bbba-c9a0fc36208d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:16.732812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef22ac-0f64-49df-bbba-c9a0fc36208d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:16.732812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef22ac-0f64-49df-bbba-c9a0fc36208d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:16.732812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef22ac-0f64-49df-bbba-c9a0fc36208d admin admin] PATH_INFO: `/v3/users/a729b39321f642ae93c55019dbf07db9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:16.737505 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2887/5813] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:16.742977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf21665-93e3-4650-b9ab-d16dcd0aef75 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:16.745485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bcf21665-93e3-4650-b9ab-d16dcd0aef75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:16.766805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef22ac-0f64-49df-bbba-c9a0fc36208d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a729b39321f642ae93c55019dbf07db9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:16.767921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf21665-93e3-4650-b9ab-d16dcd0aef75 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:16.769271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef22ac-0f64-49df-bbba-c9a0fc36208d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:16.775334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac08805-01f1-4fa9-9c70-37f49b3c7084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:16.776254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac08805-01f1-4fa9-9c70-37f49b3c7084 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:16.776851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac08805-01f1-4fa9-9c70-37f49b3c7084 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:16.777264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac08805-01f1-4fa9-9c70-37f49b3c7084 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:16.778581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac08805-01f1-4fa9-9c70-37f49b3c7084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:16.780415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac08805-01f1-4fa9-9c70-37f49b3c7084 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:16.795310 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2888/5814] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:16 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:16.797227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac08805-01f1-4fa9-9c70-37f49b3c7084 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:16.820238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac08805-01f1-4fa9-9c70-37f49b3c7084 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:16.833261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9701fa-c414-4872-a7bf-842606adbbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:16.833261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9701fa-c414-4872-a7bf-842606adbbcc admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:16.833261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9701fa-c414-4872-a7bf-842606adbbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:16.833715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9701fa-c414-4872-a7bf-842606adbbcc admin admin] PATH_INFO: `/v3/projects/4c998d22b841432b828345edcf54af63/users/8149349295714b1791af1b88fb565f74/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:16.864339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9701fa-c414-4872-a7bf-842606adbbcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c998d22b841432b828345edcf54af63, user_id=8149349295714b1791af1b88fb565f74, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:16.867604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9701fa-c414-4872-a7bf-842606adbbcc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:16.926694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:16.975886 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2889/5815] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:23:16 2022] PUT /identity/v3/projects/4c998d22b841432b828345edcf54af63/users/8149349295714b1791af1b88fb565f74/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:16.976897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9701fa-c414-4872-a7bf-842606adbbcc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:17.042017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5bef22ac-0f64-49df-bbba-c9a0fc36208d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a729b39321f642ae93c55019dbf07db9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:17.073617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5bef22ac-0f64-49df-bbba-c9a0fc36208d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a729b39321f642ae93c55019dbf07db9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:17.074849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5bef22ac-0f64-49df-bbba-c9a0fc36208d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a729b39321f642ae93c55019dbf07db9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:17.078944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5bef22ac-0f64-49df-bbba-c9a0fc36208d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a729b39321f642ae93c55019dbf07db9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:17.156686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9701fa-c414-4872-a7bf-842606adbbcc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:17.163083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ee99fd-d3f9-461b-a666-c82fbd9ace2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:17.164516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee99fd-d3f9-461b-a666-c82fbd9ace2a admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:17.164516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee99fd-d3f9-461b-a666-c82fbd9ace2a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:17.165112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ee99fd-d3f9-461b-a666-c82fbd9ace2a admin admin] PATH_INFO: `/v3/projects/dbb4877d1a184ecd85b7a201ede5470d/users/1d17b32735de4648a03d6001e8e398b7/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:17.184287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5bef22ac-0f64-49df-bbba-c9a0fc36208d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a729b39321f642ae93c55019dbf07db9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:17.196326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee99fd-d3f9-461b-a666-c82fbd9ace2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbb4877d1a184ecd85b7a201ede5470d, user_id=1d17b32735de4648a03d6001e8e398b7, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:17.201218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ee99fd-d3f9-461b-a666-c82fbd9ace2a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:17.226280 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2927/5816] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:23:16 2022] DELETE /identity/v3/users/a729b39321f642ae93c55019dbf07db9 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:17.228139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334abdb-5410-4533-924b-ad427b24cfee None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:17.228837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334abdb-5410-4533-924b-ad427b24cfee None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:17.234307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334abdb-5410-4533-924b-ad427b24cfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:17.283794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0334abdb-5410-4533-924b-ad427b24cfee None None] MFA Rules not processed for user `8149349295714b1791af1b88fb565f74`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:17.285506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0334abdb-5410-4533-924b-ad427b24cfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:17.357977 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2890/5817] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:23:16 2022] PUT /identity/v3/projects/dbb4877d1a184ecd85b7a201ede5470d/users/1d17b32735de4648a03d6001e8e398b7/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:17.404690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac267a9-2705-465e-8c96-cd9a8f095a1d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:17.404881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac267a9-2705-465e-8c96-cd9a8f095a1d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:17.405176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac267a9-2705-465e-8c96-cd9a8f095a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:17.415755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:17.501129 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2928/5818] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:17.509819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8ad85-0a30-4f3e-88dc-57c5c983d659 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:17.509819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8ad85-0a30-4f3e-88dc-57c5c983d659 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:17.509819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b8ad85-0a30-4f3e-88dc-57c5c983d659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:17.510066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-cac267a9-2705-465e-8c96-cd9a8f095a1d None None] MFA Rules not processed for user `1d17b32735de4648a03d6001e8e398b7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:17.510066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cac267a9-2705-465e-8c96-cd9a8f095a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:17.570396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-24b8ad85-0a30-4f3e-88dc-57c5c983d659 None None] MFA Rules not processed for user `8149349295714b1791af1b88fb565f74`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:17.574106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-24b8ad85-0a30-4f3e-88dc-57c5c983d659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:17.733172 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2891/5819] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:23:17.735039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac267a9-2705-465e-8c96-cd9a8f095a1d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:17.738629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cac267a9-2705-465e-8c96-cd9a8f095a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:17.826796 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2929/5820] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:17.826796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8ad85-0a30-4f3e-88dc-57c5c983d659 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:17.850193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-24b8ad85-0a30-4f3e-88dc-57c5c983d659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:17.963521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac267a9-2705-465e-8c96-cd9a8f095a1d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:17.971001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163385ef-ad30-4e79-8098-30fff93dac28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:17.972647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163385ef-ad30-4e79-8098-30fff93dac28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:17.972945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163385ef-ad30-4e79-8098-30fff93dac28 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:17.973220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163385ef-ad30-4e79-8098-30fff93dac28 admin admin] PATH_INFO: `/v3/projects/2911813847a3410c893dff40021c3099` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:17.989138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163385ef-ad30-4e79-8098-30fff93dac28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2911813847a3410c893dff40021c3099)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:17.992138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163385ef-ad30-4e79-8098-30fff93dac28 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:17.997994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b8ad85-0a30-4f3e-88dc-57c5c983d659 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:18.013048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6140fa3-db01-4b46-a537-5116844f3716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:18.014579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6140fa3-db01-4b46-a537-5116844f3716 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:18.014705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6140fa3-db01-4b46-a537-5116844f3716 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:18.015433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6140fa3-db01-4b46-a537-5116844f3716 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:18.016209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6140fa3-db01-4b46-a537-5116844f3716 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:18.023479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6140fa3-db01-4b46-a537-5116844f3716 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:18.143472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-163385ef-ad30-4e79-8098-30fff93dac28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2911813847a3410c893dff40021c3099', 'request_id': 'req-163385ef-ad30-4e79-8098-30fff93dac28'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:18.151747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-163385ef-ad30-4e79-8098-30fff93dac28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2911813847a3410c893dff40021c3099', 'request_id': 'req-163385ef-ad30-4e79-8098-30fff93dac28'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:18.177693 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2930/5821] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:23:17 2022] POST /identity/v3/projects => generated 385 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:23:18.178835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6140fa3-db01-4b46-a537-5116844f3716 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:18.182265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f6140fa3-db01-4b46-a537-5116844f3716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:18.382403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6140fa3-db01-4b46-a537-5116844f3716 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:18.382403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca7469-4149-46f2-a855-9c09b6efb87c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:18.382403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca7469-4149-46f2-a855-9c09b6efb87c service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:18.382403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca7469-4149-46f2-a855-9c09b6efb87c service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:18.383129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca7469-4149-46f2-a855-9c09b6efb87c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:18.383129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca7469-4149-46f2-a855-9c09b6efb87c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:18.383129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ca7469-4149-46f2-a855-9c09b6efb87c service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:18.384923 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2931/5822] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:18 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:23:18.385624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca7469-4149-46f2-a855-9c09b6efb87c service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:18.464208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-163385ef-ad30-4e79-8098-30fff93dac28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2911813847a3410c893dff40021c3099', 'request_id': 'req-163385ef-ad30-4e79-8098-30fff93dac28'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:18.501291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-163385ef-ad30-4e79-8098-30fff93dac28 admin admin] The token cache is being invalidate because project 2911813847a3410c893dff40021c3099 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:23:18.501590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-163385ef-ad30-4e79-8098-30fff93dac28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:18.504297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-163385ef-ad30-4e79-8098-30fff93dac28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:18.511119 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2892/5823] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:23:17 2022] DELETE /identity/v3/projects/2911813847a3410c893dff40021c3099 => generated 0 bytes in 777 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:18.518560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57714c70-e8b9-4249-9853-c422412160cf None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:18.518798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57714c70-e8b9-4249-9853-c422412160cf None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:18.519075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57714c70-e8b9-4249-9853-c422412160cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:18.549792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca7469-4149-46f2-a855-9c09b6efb87c service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:18.566622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c0e9b-d152-412e-986b-322ddd56d45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:18.567342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c0e9b-d152-412e-986b-322ddd56d45d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:18.567610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c0e9b-d152-412e-986b-322ddd56d45d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:18.567896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c0e9b-d152-412e-986b-322ddd56d45d admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:18.568609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c0e9b-d152-412e-986b-322ddd56d45d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:18.570217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-57714c70-e8b9-4249-9853-c422412160cf None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:18.572389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-57714c70-e8b9-4249-9853-c422412160cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:18.575357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c0e9b-d152-412e-986b-322ddd56d45d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:18.601762 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2893/5824] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:18.603301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8367aae-7cc3-407d-8f49-9a616543457c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:18.603594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8367aae-7cc3-407d-8f49-9a616543457c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:18.603975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8367aae-7cc3-407d-8f49-9a616543457c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:18.627662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a8367aae-7cc3-407d-8f49-9a616543457c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:18.629692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a8367aae-7cc3-407d-8f49-9a616543457c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:18.656948 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2894/5825] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:18.657127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8367aae-7cc3-407d-8f49-9a616543457c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:18.678964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8367aae-7cc3-407d-8f49-9a616543457c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:18.690982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4960ee-4507-4759-b971-c33420daf447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:18.691673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4960ee-4507-4759-b971-c33420daf447 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:18.691966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4960ee-4507-4759-b971-c33420daf447 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:18.692244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4960ee-4507-4759-b971-c33420daf447 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:18.692995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4960ee-4507-4759-b971-c33420daf447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:23:18.693314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4960ee-4507-4759-b971-c33420daf447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:18.704621 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2932/5826] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:23:18 2022] POST /identity/v3/users => generated 299 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:23:18.707355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4960ee-4507-4759-b971-c33420daf447 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:18.708555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c0e9b-d152-412e-986b-322ddd56d45d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:18.711718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8e5c0e9b-d152-412e-986b-322ddd56d45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:18.745263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c0e9b-d152-412e-986b-322ddd56d45d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:18.746922 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2895/5827] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:23:18.752575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4960ee-4507-4759-b971-c33420daf447 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:18.753677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ef912-2384-4e43-8581-3328b9f0314b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:18.754610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ef912-2384-4e43-8581-3328b9f0314b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:18.755112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ef912-2384-4e43-8581-3328b9f0314b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:18.755802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ef912-2384-4e43-8581-3328b9f0314b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:18.756737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ef912-2384-4e43-8581-3328b9f0314b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:18.758720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ef912-2384-4e43-8581-3328b9f0314b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:18.769628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4960ee-4507-4759-b971-c33420daf447 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:18.777059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a44ad-a137-43da-ab95-c707065b10b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:18.781659 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2933/5828] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:18.782871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a44ad-a137-43da-ab95-c707065b10b5 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:18.783169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a44ad-a137-43da-ab95-c707065b10b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:18.783451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a44ad-a137-43da-ab95-c707065b10b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:18.783879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ef912-2384-4e43-8581-3328b9f0314b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:18.784171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a44ad-a137-43da-ab95-c707065b10b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:18.786321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a44ad-a137-43da-ab95-c707065b10b5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:18.801278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ef912-2384-4e43-8581-3328b9f0314b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:18.816140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970aba9a-73ab-44b6-a41f-7aab629b5684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:18.818730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aba9a-73ab-44b6-a41f-7aab629b5684 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:18.819145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aba9a-73ab-44b6-a41f-7aab629b5684 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:18.819529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970aba9a-73ab-44b6-a41f-7aab629b5684 admin admin] PATH_INFO: `/v3/projects/d642828272ff49bc9a0b6dbaa4e754e9/users/49533fb58b4742fc912ddc7147ab2006/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:18.877021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970aba9a-73ab-44b6-a41f-7aab629b5684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d642828272ff49bc9a0b6dbaa4e754e9, user_id=49533fb58b4742fc912ddc7147ab2006, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:18.880547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970aba9a-73ab-44b6-a41f-7aab629b5684 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:18.998989 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2896/5829] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:23:18 2022] POST /identity/v3/projects => generated 381 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:23:19.003007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a44ad-a137-43da-ab95-c707065b10b5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:19.023775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:19.025571 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2934/5830] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:23:18 2022] PUT /identity/v3/projects/d642828272ff49bc9a0b6dbaa4e754e9/users/49533fb58b4742fc912ddc7147ab2006/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:19.029707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970aba9a-73ab-44b6-a41f-7aab629b5684 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:19.182644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a44ad-a137-43da-ab95-c707065b10b5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:19.189254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:19.236672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970aba9a-73ab-44b6-a41f-7aab629b5684 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:19.270529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c3adc-525a-4e12-b891-57b6b87f69e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:19.270529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c3adc-525a-4e12-b891-57b6b87f69e7 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:19.270529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c3adc-525a-4e12-b891-57b6b87f69e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:19.270529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266c3adc-525a-4e12-b891-57b6b87f69e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:19.274203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c3adc-525a-4e12-b891-57b6b87f69e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:19.296203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266c3adc-525a-4e12-b891-57b6b87f69e7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:19.313513 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2935/5831] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:19.321257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c3adc-525a-4e12-b891-57b6b87f69e7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:19.336586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266c3adc-525a-4e12-b891-57b6b87f69e7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:19.346547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634dafe4-3410-4b71-a58e-6358644a556a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:19.347470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dafe4-3410-4b71-a58e-6358644a556a admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:19.347757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dafe4-3410-4b71-a58e-6358644a556a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:19.348039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634dafe4-3410-4b71-a58e-6358644a556a admin admin] PATH_INFO: `/v3/projects/d642828272ff49bc9a0b6dbaa4e754e9/users/49533fb58b4742fc912ddc7147ab2006/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:19.361676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634dafe4-3410-4b71-a58e-6358644a556a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d642828272ff49bc9a0b6dbaa4e754e9, user_id=49533fb58b4742fc912ddc7147ab2006, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:19.369207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634dafe4-3410-4b71-a58e-6358644a556a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:19.398393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b222e-49bc-4021-bb04-96e70bdb408d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:19.400549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b222e-49bc-4021-bb04-96e70bdb408d admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:19.401207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b222e-49bc-4021-bb04-96e70bdb408d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:19.401207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b222e-49bc-4021-bb04-96e70bdb408d admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:19.402018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b222e-49bc-4021-bb04-96e70bdb408d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:19.404405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b222e-49bc-4021-bb04-96e70bdb408d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:19.424941 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2936/5832] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:23:19 2022] PUT /identity/v3/projects/d642828272ff49bc9a0b6dbaa4e754e9/users/49533fb58b4742fc912ddc7147ab2006/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:19.438212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d327859-23c1-4f2d-9b8f-15415f259f95 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:19.438574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d327859-23c1-4f2d-9b8f-15415f259f95 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:19.438922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d327859-23c1-4f2d-9b8f-15415f259f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:19.539869 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2897/5833] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:23:19 2022] POST /identity/v3/users => generated 291 bytes in 538 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:23:19.541795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57ca8c-c04e-4496-890c-3d841de3d1a3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:19.544005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57ca8c-c04e-4496-890c-3d841de3d1a3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:19.544297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57ca8c-c04e-4496-890c-3d841de3d1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:19.552952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8d327859-23c1-4f2d-9b8f-15415f259f95 None None] MFA Rules not processed for user `49533fb58b4742fc912ddc7147ab2006`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:19.554323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8d327859-23c1-4f2d-9b8f-15415f259f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:19.624583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7e57ca8c-c04e-4496-890c-3d841de3d1a3 None None] MFA Rules not processed for user `8149349295714b1791af1b88fb565f74`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:19.629250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7e57ca8c-c04e-4496-890c-3d841de3d1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:19.800256 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2937/5834] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:19.801407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d327859-23c1-4f2d-9b8f-15415f259f95 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:19.872049 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2898/5835] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:19.873335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57ca8c-c04e-4496-890c-3d841de3d1a3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:19.880900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7e57ca8c-c04e-4496-890c-3d841de3d1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:19.922154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d327859-23c1-4f2d-9b8f-15415f259f95 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:19.928716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576cf64-ce91-4b68-882a-237210fc06dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:19.929533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576cf64-ce91-4b68-882a-237210fc06dc admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:19.929829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576cf64-ce91-4b68-882a-237210fc06dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:19.930146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576cf64-ce91-4b68-882a-237210fc06dc admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:19.931036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576cf64-ce91-4b68-882a-237210fc06dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:19.932816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576cf64-ce91-4b68-882a-237210fc06dc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:19.959533 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2938/5836] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:19.961223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576cf64-ce91-4b68-882a-237210fc06dc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:19.986154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576cf64-ce91-4b68-882a-237210fc06dc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:20.002518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fd602-3938-423e-8277-52e5aaaaad91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:20.012553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fd602-3938-423e-8277-52e5aaaaad91 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:20.012553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fd602-3938-423e-8277-52e5aaaaad91 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:20.012553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2fd602-3938-423e-8277-52e5aaaaad91 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:20.012553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2fd602-3938-423e-8277-52e5aaaaad91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:20.014614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2fd602-3938-423e-8277-52e5aaaaad91 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:20.053331 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2939/5837] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:20.054730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fd602-3938-423e-8277-52e5aaaaad91 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:20.078406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2fd602-3938-423e-8277-52e5aaaaad91 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:20.100730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7793f-c605-45f7-83e6-3b15e72994a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:20.102333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7793f-c605-45f7-83e6-3b15e72994a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:20.104738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7793f-c605-45f7-83e6-3b15e72994a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:20.104738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7793f-c605-45f7-83e6-3b15e72994a6 admin admin] PATH_INFO: `/v3/projects/d3112549c42f41cda4a54ec51c131023/users/f7c1a400e0dd462da3bb33b7e3fe57fc/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:20.139159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57ca8c-c04e-4496-890c-3d841de3d1a3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:20.145432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a83bb-9af3-4524-a860-487bd8fec268 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:20.146773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a83bb-9af3-4524-a860-487bd8fec268 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:20.147637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a83bb-9af3-4524-a860-487bd8fec268 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:20.148143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a83bb-9af3-4524-a860-487bd8fec268 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:20.151740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7793f-c605-45f7-83e6-3b15e72994a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3112549c42f41cda4a54ec51c131023, user_id=f7c1a400e0dd462da3bb33b7e3fe57fc, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:20.157670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7793f-c605-45f7-83e6-3b15e72994a6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:20.163056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a83bb-9af3-4524-a860-487bd8fec268 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:20.165047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18a83bb-9af3-4524-a860-487bd8fec268 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:20.179141 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2899/5838] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:23:20.276274 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2940/5839] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:23:20 2022] PUT /identity/v3/projects/d3112549c42f41cda4a54ec51c131023/users/f7c1a400e0dd462da3bb33b7e3fe57fc/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:20.279125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68ad86-0383-4af2-a0ff-792fde809f3f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:20.279362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68ad86-0383-4af2-a0ff-792fde809f3f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:20.279756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f68ad86-0383-4af2-a0ff-792fde809f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:20.335548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3f68ad86-0383-4af2-a0ff-792fde809f3f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:20.336747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3f68ad86-0383-4af2-a0ff-792fde809f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:20.430131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:20.517849 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2900/5840] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:20.524677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7793f-c605-45f7-83e6-3b15e72994a6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:20.529125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-75c7793f-c605-45f7-83e6-3b15e72994a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:20.746077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7793f-c605-45f7-83e6-3b15e72994a6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:20.751032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced56f1e-f4ff-494b-af88-af8745d1a070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:20.752411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced56f1e-f4ff-494b-af88-af8745d1a070 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:20.752602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced56f1e-f4ff-494b-af88-af8745d1a070 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:20.752868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced56f1e-f4ff-494b-af88-af8745d1a070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:20.775203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced56f1e-f4ff-494b-af88-af8745d1a070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:20.779973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced56f1e-f4ff-494b-af88-af8745d1a070 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:20.794203 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2941/5841] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:23:21.201376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68ad86-0383-4af2-a0ff-792fde809f3f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:21.204467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3f68ad86-0383-4af2-a0ff-792fde809f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:21.219534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:21.249010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f68ad86-0383-4af2-a0ff-792fde809f3f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:21.259682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dd658-05c5-4612-b5d3-7db07386c322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:21.261456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dd658-05c5-4612-b5d3-7db07386c322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:21.262802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dd658-05c5-4612-b5d3-7db07386c322 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:21.263310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dd658-05c5-4612-b5d3-7db07386c322 admin admin] PATH_INFO: `/v3/users/d8a35655682e4ece8da56d74d6f535cd` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:21.267435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03dd658-05c5-4612-b5d3-7db07386c322 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8a35655682e4ece8da56d74d6f535cd)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:21.270394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03dd658-05c5-4612-b5d3-7db07386c322 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:21.591409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a03dd658-05c5-4612-b5d3-7db07386c322 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8a35655682e4ece8da56d74d6f535cd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:21.592340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a03dd658-05c5-4612-b5d3-7db07386c322 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8a35655682e4ece8da56d74d6f535cd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:21.646822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a03dd658-05c5-4612-b5d3-7db07386c322 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8a35655682e4ece8da56d74d6f535cd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:21.766765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a03dd658-05c5-4612-b5d3-7db07386c322 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8a35655682e4ece8da56d74d6f535cd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:21.769015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a03dd658-05c5-4612-b5d3-7db07386c322 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8a35655682e4ece8da56d74d6f535cd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:21.807417 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2901/5842] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:23:21 2022] DELETE /identity/v3/users/d8a35655682e4ece8da56d74d6f535cd => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:21.812217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bf887-bd2b-4daf-acde-a4f13c84f825 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:21.812734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bf887-bd2b-4daf-acde-a4f13c84f825 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:21.812836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bf887-bd2b-4daf-acde-a4f13c84f825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:21.907841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0d2bf887-bd2b-4daf-acde-a4f13c84f825 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:21.909103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0d2bf887-bd2b-4daf-acde-a4f13c84f825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:22.243649 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2942/5843] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:22.268050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dd658-05c5-4612-b5d3-7db07386c322 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:22.270334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a03dd658-05c5-4612-b5d3-7db07386c322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:22.450453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dd658-05c5-4612-b5d3-7db07386c322 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:22.470171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a03ff6-206f-4eec-83a2-73b5f631d282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:22.470990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a03ff6-206f-4eec-83a2-73b5f631d282 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:22.471314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a03ff6-206f-4eec-83a2-73b5f631d282 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:22.471592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a03ff6-206f-4eec-83a2-73b5f631d282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:22.478521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bf887-bd2b-4daf-acde-a4f13c84f825 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:22.494451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0d2bf887-bd2b-4daf-acde-a4f13c84f825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:22.506256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a03ff6-206f-4eec-83a2-73b5f631d282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:22.507779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a03ff6-206f-4eec-83a2-73b5f631d282 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:22.532375 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2902/5844] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:23:22.697309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bf887-bd2b-4daf-acde-a4f13c84f825 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:22.713350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47e3414-392b-4a15-a1c5-7ee543a6427b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:22.714227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e3414-392b-4a15-a1c5-7ee543a6427b service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:22.714432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e3414-392b-4a15-a1c5-7ee543a6427b service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:22.716889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47e3414-392b-4a15-a1c5-7ee543a6427b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:22.726011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b47e3414-392b-4a15-a1c5-7ee543a6427b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:22.870287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e3414-392b-4a15-a1c5-7ee543a6427b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:22.871839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47e3414-392b-4a15-a1c5-7ee543a6427b service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:22.920977 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2943/5845] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:23:23.293627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:24.497815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:24.671384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a03ff6-206f-4eec-83a2-73b5f631d282 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:24.673666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f3a03ff6-206f-4eec-83a2-73b5f631d282 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:24.701130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a03ff6-206f-4eec-83a2-73b5f631d282 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:24.718216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcae5c3-d2ba-4eb6-a7f6-7837ffb9fc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:24.719355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcae5c3-d2ba-4eb6-a7f6-7837ffb9fc0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:24.724723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcae5c3-d2ba-4eb6-a7f6-7837ffb9fc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:24.725476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcae5c3-d2ba-4eb6-a7f6-7837ffb9fc0a admin admin] PATH_INFO: `/v3/projects/753b86e7a74c40369a4697ba5e7920d7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:24.728312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcae5c3-d2ba-4eb6-a7f6-7837ffb9fc0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=753b86e7a74c40369a4697ba5e7920d7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:24.730852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcae5c3-d2ba-4eb6-a7f6-7837ffb9fc0a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:24.785447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c84dca-e20b-461a-9aa4-0cf147e1611e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:24.785731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c84dca-e20b-461a-9aa4-0cf147e1611e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:24.786053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c84dca-e20b-461a-9aa4-0cf147e1611e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:24.818258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-51c84dca-e20b-461a-9aa4-0cf147e1611e None None] MFA Rules not processed for user `f7c1a400e0dd462da3bb33b7e3fe57fc`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:24.819785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-51c84dca-e20b-461a-9aa4-0cf147e1611e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:24.906844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-adcae5c3-d2ba-4eb6-a7f6-7837ffb9fc0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '753b86e7a74c40369a4697ba5e7920d7', 'request_id': 'req-adcae5c3-d2ba-4eb6-a7f6-7837ffb9fc0a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:24.922773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-adcae5c3-d2ba-4eb6-a7f6-7837ffb9fc0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '753b86e7a74c40369a4697ba5e7920d7', 'request_id': 'req-adcae5c3-d2ba-4eb6-a7f6-7837ffb9fc0a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:24.931805 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2944/5846] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:23:24.935399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50733b1-96ed-44f8-b63f-2f056ef68341 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:24.935949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50733b1-96ed-44f8-b63f-2f056ef68341 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:24.936422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50733b1-96ed-44f8-b63f-2f056ef68341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:24.979275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c50733b1-96ed-44f8-b63f-2f056ef68341 None None] MFA Rules not processed for user `f7c1a400e0dd462da3bb33b7e3fe57fc`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:24.982946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c50733b1-96ed-44f8-b63f-2f056ef68341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:25.017992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-adcae5c3-d2ba-4eb6-a7f6-7837ffb9fc0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '753b86e7a74c40369a4697ba5e7920d7', 'request_id': 'req-adcae5c3-d2ba-4eb6-a7f6-7837ffb9fc0a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:25.059353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-adcae5c3-d2ba-4eb6-a7f6-7837ffb9fc0a admin admin] The token cache is being invalidate because project 753b86e7a74c40369a4697ba5e7920d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:23:25.059768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-adcae5c3-d2ba-4eb6-a7f6-7837ffb9fc0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:25.060623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-adcae5c3-d2ba-4eb6-a7f6-7837ffb9fc0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:25.067248 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2903/5847] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:23:24 2022] DELETE /identity/v3/projects/753b86e7a74c40369a4697ba5e7920d7 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:25.069361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b15fd-26b4-4065-8290-f5dbb44606ee None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:25.069661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b15fd-26b4-4065-8290-f5dbb44606ee None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:25.069966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9b15fd-26b4-4065-8290-f5dbb44606ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:25.095052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5b9b15fd-26b4-4065-8290-f5dbb44606ee None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:25.095835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5b9b15fd-26b4-4065-8290-f5dbb44606ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:25.128959 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2945/5848] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:25.142731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50733b1-96ed-44f8-b63f-2f056ef68341 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:25.145785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c50733b1-96ed-44f8-b63f-2f056ef68341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:25.237118 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2904/5849] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:25.321435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76762721-306d-4fe5-91c8-1abeed286b0f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:25.321921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76762721-306d-4fe5-91c8-1abeed286b0f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:25.322292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76762721-306d-4fe5-91c8-1abeed286b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:25.326810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:25.345200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50733b1-96ed-44f8-b63f-2f056ef68341 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:25.350690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-76762721-306d-4fe5-91c8-1abeed286b0f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:25.352831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-76762721-306d-4fe5-91c8-1abeed286b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:25.359949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66118e8-6276-459f-b6d9-caa2e79fbc06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:25.360741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66118e8-6276-459f-b6d9-caa2e79fbc06 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:25.361175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66118e8-6276-459f-b6d9-caa2e79fbc06 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:25.361527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66118e8-6276-459f-b6d9-caa2e79fbc06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:25.364630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b66118e8-6276-459f-b6d9-caa2e79fbc06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:25.373857 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2905/5850] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:25.378716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76762721-306d-4fe5-91c8-1abeed286b0f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:25.396898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76762721-306d-4fe5-91c8-1abeed286b0f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:25.402960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a693feb-4df6-4d8e-8947-b719654d90a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:25.403870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a693feb-4df6-4d8e-8947-b719654d90a7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:25.404425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a693feb-4df6-4d8e-8947-b719654d90a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:25.404682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66118e8-6276-459f-b6d9-caa2e79fbc06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:25.405224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a693feb-4df6-4d8e-8947-b719654d90a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:25.406575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66118e8-6276-459f-b6d9-caa2e79fbc06 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:25.407213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a693feb-4df6-4d8e-8947-b719654d90a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:23:25.407836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a693feb-4df6-4d8e-8947-b719654d90a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:25.411329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a693feb-4df6-4d8e-8947-b719654d90a7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:25.429916 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2906/5851] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:23:25.435138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a693feb-4df6-4d8e-8947-b719654d90a7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:25.435346 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2946/5852] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:23:25.450509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a693feb-4df6-4d8e-8947-b719654d90a7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:25.457348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99063930-bcc9-4873-8dd8-d1dfd4503700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:25.458302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99063930-bcc9-4873-8dd8-d1dfd4503700 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:25.458740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99063930-bcc9-4873-8dd8-d1dfd4503700 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:25.460151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99063930-bcc9-4873-8dd8-d1dfd4503700 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:25.460315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99063930-bcc9-4873-8dd8-d1dfd4503700 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:25.462810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99063930-bcc9-4873-8dd8-d1dfd4503700 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:25.606950 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2907/5853] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:23:25 2022] POST /identity/v3/projects => generated 371 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:23:25.613331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66118e8-6276-459f-b6d9-caa2e79fbc06 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:25.650226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66118e8-6276-459f-b6d9-caa2e79fbc06 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:25.741500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff648de5-645e-4ad3-9a17-93515a6166b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:25.742728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff648de5-645e-4ad3-9a17-93515a6166b1 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:25.745685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff648de5-645e-4ad3-9a17-93515a6166b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:25.745964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff648de5-645e-4ad3-9a17-93515a6166b1 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:25.746956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff648de5-645e-4ad3-9a17-93515a6166b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:25.750318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff648de5-645e-4ad3-9a17-93515a6166b1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:25.885107 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2947/5854] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:23:25 2022] POST /identity/v3/users => generated 286 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:23:25.887100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99063930-bcc9-4873-8dd8-d1dfd4503700 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:25.901737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99063930-bcc9-4873-8dd8-d1dfd4503700 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:25.910101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9ba92-2f9b-43a2-9b15-eed8a9029109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:25.911428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9ba92-2f9b-43a2-9b15-eed8a9029109 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:25.917166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9ba92-2f9b-43a2-9b15-eed8a9029109 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:25.917166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d9ba92-2f9b-43a2-9b15-eed8a9029109 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:25.917166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d9ba92-2f9b-43a2-9b15-eed8a9029109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:25.917166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d9ba92-2f9b-43a2-9b15-eed8a9029109 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:25.936255 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2908/5855] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:25.942742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff648de5-645e-4ad3-9a17-93515a6166b1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:25.952812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff648de5-645e-4ad3-9a17-93515a6166b1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:25.962972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808c33f-25eb-466c-93be-d81d220ffd2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:25.963881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808c33f-25eb-466c-93be-d81d220ffd2b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:25.964376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808c33f-25eb-466c-93be-d81d220ffd2b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:25.965388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808c33f-25eb-466c-93be-d81d220ffd2b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:25.966255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808c33f-25eb-466c-93be-d81d220ffd2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:25.969084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808c33f-25eb-466c-93be-d81d220ffd2b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:25.998094 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2948/5856] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:26.002081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9ba92-2f9b-43a2-9b15-eed8a9029109 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:26.013647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d9ba92-2f9b-43a2-9b15-eed8a9029109 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:26.049227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0dfc6-a1b6-4400-81cf-27db77b0d140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:26.051835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0dfc6-a1b6-4400-81cf-27db77b0d140 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:26.057161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0dfc6-a1b6-4400-81cf-27db77b0d140 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:26.057712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0dfc6-a1b6-4400-81cf-27db77b0d140 admin admin] PATH_INFO: `/v3/projects/63af59b2d6d6416f996f9b08c5d49a98/users/3139efbd5aa6487c90b44c44bae79746/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:26.069892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:26.140889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0dfc6-a1b6-4400-81cf-27db77b0d140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63af59b2d6d6416f996f9b08c5d49a98, user_id=3139efbd5aa6487c90b44c44bae79746, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:26.151004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0dfc6-a1b6-4400-81cf-27db77b0d140 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:26.676096 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2909/5857] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:23:26 2022] PUT /identity/v3/projects/63af59b2d6d6416f996f9b08c5d49a98/users/3139efbd5aa6487c90b44c44bae79746/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:26.686862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea88397-a0f6-4c72-b21b-21baea281844 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:26.687326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea88397-a0f6-4c72-b21b-21baea281844 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:26.687448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea88397-a0f6-4c72-b21b-21baea281844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:26.718592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5ea88397-a0f6-4c72-b21b-21baea281844 None None] MFA Rules not processed for user `3139efbd5aa6487c90b44c44bae79746`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:26.720166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5ea88397-a0f6-4c72-b21b-21baea281844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:26.847487 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2949/5858] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:26.871611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0dfc6-a1b6-4400-81cf-27db77b0d140 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:26.875932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-19a0dfc6-a1b6-4400-81cf-27db77b0d140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:27.001544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0dfc6-a1b6-4400-81cf-27db77b0d140 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:27.006250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a3a82-5c24-4136-9a85-45d23b2c03a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:27.009946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a3a82-5c24-4136-9a85-45d23b2c03a5 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:27.010452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a3a82-5c24-4136-9a85-45d23b2c03a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:27.010947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a3a82-5c24-4136-9a85-45d23b2c03a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:27.054017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a3a82-5c24-4136-9a85-45d23b2c03a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:27.057352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a3a82-5c24-4136-9a85-45d23b2c03a5 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:27.109717 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2910/5859] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:23:27.171330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea88397-a0f6-4c72-b21b-21baea281844 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:27.335093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea88397-a0f6-4c72-b21b-21baea281844 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:27.344010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fe766-394c-4f5b-b704-348cb63072ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:27.345033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fe766-394c-4f5b-b704-348cb63072ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:27.345377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fe766-394c-4f5b-b704-348cb63072ef admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:27.345760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fe766-394c-4f5b-b704-348cb63072ef admin admin] PATH_INFO: `/v3/users/3139efbd5aa6487c90b44c44bae79746` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:27.348406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fe766-394c-4f5b-b704-348cb63072ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=3139efbd5aa6487c90b44c44bae79746)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:27.350886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fe766-394c-4f5b-b704-348cb63072ef admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:27.704700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6f5fe766-394c-4f5b-b704-348cb63072ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3139efbd5aa6487c90b44c44bae79746'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:27.712686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6f5fe766-394c-4f5b-b704-348cb63072ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3139efbd5aa6487c90b44c44bae79746'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:27.713524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6f5fe766-394c-4f5b-b704-348cb63072ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3139efbd5aa6487c90b44c44bae79746'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:27.714276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6f5fe766-394c-4f5b-b704-348cb63072ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3139efbd5aa6487c90b44c44bae79746'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:27.835563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6f5fe766-394c-4f5b-b704-348cb63072ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3139efbd5aa6487c90b44c44bae79746'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:27.883517 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2950/5860] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:23:27 2022] DELETE /identity/v3/users/3139efbd5aa6487c90b44c44bae79746 => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:27.885146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd89bab-314c-405f-9080-aa17fdd798cf None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:27.885430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd89bab-314c-405f-9080-aa17fdd798cf None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:27.885685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd89bab-314c-405f-9080-aa17fdd798cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:27.920568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9fd89bab-314c-405f-9080-aa17fdd798cf None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:27.920568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9fd89bab-314c-405f-9080-aa17fdd798cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:28.066623 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2911/5861] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:28.094843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fe766-394c-4f5b-b704-348cb63072ef admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:28.097046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6f5fe766-394c-4f5b-b704-348cb63072ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:28.144494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:28.207723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fe766-394c-4f5b-b704-348cb63072ef admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:28.215331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232c88c-ea9a-4337-9567-d510759eddd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:28.216197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232c88c-ea9a-4337-9567-d510759eddd5 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:28.216547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232c88c-ea9a-4337-9567-d510759eddd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:28.216832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232c88c-ea9a-4337-9567-d510759eddd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:28.232857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232c88c-ea9a-4337-9567-d510759eddd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:28.234912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232c88c-ea9a-4337-9567-d510759eddd5 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:28.247204 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2951/5862] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:23:28.304624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:29.051295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd89bab-314c-405f-9080-aa17fdd798cf None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:29.052834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9fd89bab-314c-405f-9080-aa17fdd798cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:29.079487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd89bab-314c-405f-9080-aa17fdd798cf None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:29.094377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f9fb26-4961-40d8-bf09-755fcabae648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:29.095252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9fb26-4961-40d8-bf09-755fcabae648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:29.095810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9fb26-4961-40d8-bf09-755fcabae648 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:29.095810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f9fb26-4961-40d8-bf09-755fcabae648 admin admin] PATH_INFO: `/v3/projects/63af59b2d6d6416f996f9b08c5d49a98` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:29.097014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f9fb26-4961-40d8-bf09-755fcabae648 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63af59b2d6d6416f996f9b08c5d49a98)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:29.099142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f9fb26-4961-40d8-bf09-755fcabae648 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:29.326882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51f9fb26-4961-40d8-bf09-755fcabae648 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63af59b2d6d6416f996f9b08c5d49a98', 'request_id': 'req-51f9fb26-4961-40d8-bf09-755fcabae648'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:29.336309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51f9fb26-4961-40d8-bf09-755fcabae648 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63af59b2d6d6416f996f9b08c5d49a98', 'request_id': 'req-51f9fb26-4961-40d8-bf09-755fcabae648'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:29.445467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51f9fb26-4961-40d8-bf09-755fcabae648 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63af59b2d6d6416f996f9b08c5d49a98', 'request_id': 'req-51f9fb26-4961-40d8-bf09-755fcabae648'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:29.464349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51f9fb26-4961-40d8-bf09-755fcabae648 admin admin] The token cache is being invalidate because project 63af59b2d6d6416f996f9b08c5d49a98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:23:29.465617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51f9fb26-4961-40d8-bf09-755fcabae648 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:29.468937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51f9fb26-4961-40d8-bf09-755fcabae648 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:29.477119 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2912/5863] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:23:29 2022] DELETE /identity/v3/projects/63af59b2d6d6416f996f9b08c5d49a98 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:29.480452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32687298-f049-47f9-ac9c-76acda035327 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:29.481764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32687298-f049-47f9-ac9c-76acda035327 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:29.482090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32687298-f049-47f9-ac9c-76acda035327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:29.522663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-32687298-f049-47f9-ac9c-76acda035327 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:29.524303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-32687298-f049-47f9-ac9c-76acda035327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:29.629605 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2952/5864] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:23:29.638372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17065ec-aa64-413c-afd2-7f5858e65b49 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:29.638681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17065ec-aa64-413c-afd2-7f5858e65b49 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:29.638949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17065ec-aa64-413c-afd2-7f5858e65b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:29.683956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c17065ec-aa64-413c-afd2-7f5858e65b49 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:29.688835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c17065ec-aa64-413c-afd2-7f5858e65b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:29.701283 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2913/5865] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:29.705778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32687298-f049-47f9-ac9c-76acda035327 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:29.721024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32687298-f049-47f9-ac9c-76acda035327 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:29.731384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1858d2-b8d5-4dd8-9996-b6b5327d2ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:29.732375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1858d2-b8d5-4dd8-9996-b6b5327d2ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:29.732816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1858d2-b8d5-4dd8-9996-b6b5327d2ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:29.733303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1858d2-b8d5-4dd8-9996-b6b5327d2ec5 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:29.735291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1858d2-b8d5-4dd8-9996-b6b5327d2ec5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:23:29.735764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1858d2-b8d5-4dd8-9996-b6b5327d2ec5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:29.737901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1858d2-b8d5-4dd8-9996-b6b5327d2ec5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:29.770880 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2953/5866] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:23:29.771495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17065ec-aa64-413c-afd2-7f5858e65b49 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:29.791223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17065ec-aa64-413c-afd2-7f5858e65b49 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:29.805850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bd098-a336-4146-b654-1e88de260907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:29.806572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bd098-a336-4146-b654-1e88de260907 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:29.806856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bd098-a336-4146-b654-1e88de260907 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:29.807119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4bd098-a336-4146-b654-1e88de260907 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:29.807829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4bd098-a336-4146-b654-1e88de260907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:29.820067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4bd098-a336-4146-b654-1e88de260907 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:30.011920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1858d2-b8d5-4dd8-9996-b6b5327d2ec5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:30.019889 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2914/5867] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:23:29 2022] POST /identity/v3/projects => generated 375 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:23:30.028811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1858d2-b8d5-4dd8-9996-b6b5327d2ec5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:30.107588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9e171-4254-4bfa-80b5-d8ef8dd879a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:30.108382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9e171-4254-4bfa-80b5-d8ef8dd879a3 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:30.108746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9e171-4254-4bfa-80b5-d8ef8dd879a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:30.109225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9e171-4254-4bfa-80b5-d8ef8dd879a3 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:30.110063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9e171-4254-4bfa-80b5-d8ef8dd879a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:30.112091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9e171-4254-4bfa-80b5-d8ef8dd879a3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:30.248686 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2954/5868] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:23:30 2022] POST /identity/v3/users => generated 288 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:23:30.253562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bd098-a336-4146-b654-1e88de260907 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:30.289270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4bd098-a336-4146-b654-1e88de260907 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:30.302675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f2dc9-efe1-4d8e-a757-30acc825977a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:30.304038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f2dc9-efe1-4d8e-a757-30acc825977a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:30.307564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f2dc9-efe1-4d8e-a757-30acc825977a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:30.309768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f2dc9-efe1-4d8e-a757-30acc825977a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:30.318632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f2dc9-efe1-4d8e-a757-30acc825977a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:30.320526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f2dc9-efe1-4d8e-a757-30acc825977a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:30.350831 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2915/5869] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:30.356628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9e171-4254-4bfa-80b5-d8ef8dd879a3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:30.372876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9e171-4254-4bfa-80b5-d8ef8dd879a3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:30.385538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b7161-d811-4e00-a545-b52fa2dd9949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:30.390430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b7161-d811-4e00-a545-b52fa2dd9949 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:30.391647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b7161-d811-4e00-a545-b52fa2dd9949 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:30.391984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b7161-d811-4e00-a545-b52fa2dd9949 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:30.397259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b7161-d811-4e00-a545-b52fa2dd9949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:30.398941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b7161-d811-4e00-a545-b52fa2dd9949 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:30.426593 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2955/5870] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:30.431325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f2dc9-efe1-4d8e-a757-30acc825977a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:30.446896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f2dc9-efe1-4d8e-a757-30acc825977a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:30.452706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07488d58-b89f-42b5-b3fb-18ae897fe1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:30.454408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07488d58-b89f-42b5-b3fb-18ae897fe1cf admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:30.455273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07488d58-b89f-42b5-b3fb-18ae897fe1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:30.455958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07488d58-b89f-42b5-b3fb-18ae897fe1cf admin admin] PATH_INFO: `/v3/projects/e0352bb38c2c4a87bed4a2ca8dbbf437/users/15841f513249448081439041b58d091c/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:30.478635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07488d58-b89f-42b5-b3fb-18ae897fe1cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0352bb38c2c4a87bed4a2ca8dbbf437, user_id=15841f513249448081439041b58d091c, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:30.482171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07488d58-b89f-42b5-b3fb-18ae897fe1cf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:30.594991 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2916/5871] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:23:30 2022] PUT /identity/v3/projects/e0352bb38c2c4a87bed4a2ca8dbbf437/users/15841f513249448081439041b58d091c/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:23:30.606997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d65181-646b-4a8e-bfeb-395d0ad2aa77 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:30.612917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d65181-646b-4a8e-bfeb-395d0ad2aa77 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:30.612917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d65181-646b-4a8e-bfeb-395d0ad2aa77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:30.678797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e9d65181-646b-4a8e-bfeb-395d0ad2aa77 None None] MFA Rules not processed for user `15841f513249448081439041b58d091c`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:30.680407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e9d65181-646b-4a8e-bfeb-395d0ad2aa77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:30.716471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:30.864918 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2956/5872] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:30.877958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07488d58-b89f-42b5-b3fb-18ae897fe1cf admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:30.879893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-07488d58-b89f-42b5-b3fb-18ae897fe1cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:31.086188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07488d58-b89f-42b5-b3fb-18ae897fe1cf admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:31.104716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eebc4-5bec-48eb-8fec-07b3de5cae88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:31.115662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eebc4-5bec-48eb-8fec-07b3de5cae88 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:31.122211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eebc4-5bec-48eb-8fec-07b3de5cae88 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:31.122557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eebc4-5bec-48eb-8fec-07b3de5cae88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:31.134669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7eebc4-5bec-48eb-8fec-07b3de5cae88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:31.137396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7eebc4-5bec-48eb-8fec-07b3de5cae88 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:31.150619 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2917/5873] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:23:31.285549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:31.774837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:33.966591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:34.038356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d65181-646b-4a8e-bfeb-395d0ad2aa77 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:34.039920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e9d65181-646b-4a8e-bfeb-395d0ad2aa77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:34.148773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d65181-646b-4a8e-bfeb-395d0ad2aa77 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:34.152746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e5aba-8ab3-418f-8cbc-4314a6b844aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:34.153475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e5aba-8ab3-418f-8cbc-4314a6b844aa service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:34.153765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e5aba-8ab3-418f-8cbc-4314a6b844aa service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:34.154278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e5aba-8ab3-418f-8cbc-4314a6b844aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:34.156742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f54e5aba-8ab3-418f-8cbc-4314a6b844aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:34.170741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e5aba-8ab3-418f-8cbc-4314a6b844aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:34.172992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54e5aba-8ab3-418f-8cbc-4314a6b844aa service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:34.185708 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2957/5874] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:23:35.478351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eebc4-5bec-48eb-8fec-07b3de5cae88 service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:35.492222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eebc4-5bec-48eb-8fec-07b3de5cae88 service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:35.503906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fee945-d9a2-4e80-817f-cd0c69fcd9bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:35.504723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fee945-d9a2-4e80-817f-cd0c69fcd9bc service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:35.505222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fee945-d9a2-4e80-817f-cd0c69fcd9bc service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:35.505685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fee945-d9a2-4e80-817f-cd0c69fcd9bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:35.509548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b0fee945-d9a2-4e80-817f-cd0c69fcd9bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:35.639879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fee945-d9a2-4e80-817f-cd0c69fcd9bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:35.641453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fee945-d9a2-4e80-817f-cd0c69fcd9bc service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:35.654037 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2918/5875] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:23:36.965022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:40.977174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e5203-2f88-44cb-aef7-9a0990040c58 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:40.979746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e5203-2f88-44cb-aef7-9a0990040c58 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:40.980834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64e5203-2f88-44cb-aef7-9a0990040c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:41.004002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c64e5203-2f88-44cb-aef7-9a0990040c58 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:41.006199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c64e5203-2f88-44cb-aef7-9a0990040c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:41.118351 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2958/5876] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:23:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:41.176554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fee945-d9a2-4e80-817f-cd0c69fcd9bc service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:41.176554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b0fee945-d9a2-4e80-817f-cd0c69fcd9bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:41.316609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fee945-d9a2-4e80-817f-cd0c69fcd9bc service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:41.324788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f982ceb-1ed0-4d1c-88a0-6319d6053a64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:41.325478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f982ceb-1ed0-4d1c-88a0-6319d6053a64 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:41.325772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f982ceb-1ed0-4d1c-88a0-6319d6053a64 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:41.326099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f982ceb-1ed0-4d1c-88a0-6319d6053a64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:41.341551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f982ceb-1ed0-4d1c-88a0-6319d6053a64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:41.345302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f982ceb-1ed0-4d1c-88a0-6319d6053a64 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:41.363707 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2919/5877] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:23:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:23:44.753286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e5203-2f88-44cb-aef7-9a0990040c58 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:44.755166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c64e5203-2f88-44cb-aef7-9a0990040c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:44.932275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64e5203-2f88-44cb-aef7-9a0990040c58 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:44.944910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bb474-c569-4f50-958b-e3ba18c0d786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:44.946365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bb474-c569-4f50-958b-e3ba18c0d786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:44.948017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bb474-c569-4f50-958b-e3ba18c0d786 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:44.950846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5bb474-c569-4f50-958b-e3ba18c0d786 admin admin] PATH_INFO: `/v3/users/f525c29419db48e684dd53c758abb74b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:44.953577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5bb474-c569-4f50-958b-e3ba18c0d786 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f525c29419db48e684dd53c758abb74b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:44.961876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5bb474-c569-4f50-958b-e3ba18c0d786 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:45.206664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7a5bb474-c569-4f50-958b-e3ba18c0d786 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f525c29419db48e684dd53c758abb74b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:45.220942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7a5bb474-c569-4f50-958b-e3ba18c0d786 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f525c29419db48e684dd53c758abb74b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:45.221994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7a5bb474-c569-4f50-958b-e3ba18c0d786 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f525c29419db48e684dd53c758abb74b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:45.222695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7a5bb474-c569-4f50-958b-e3ba18c0d786 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f525c29419db48e684dd53c758abb74b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:45.326452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7a5bb474-c569-4f50-958b-e3ba18c0d786 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f525c29419db48e684dd53c758abb74b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:45.369243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a97add-9b40-4150-b9b2-585eabe3fb35 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:45.370120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a97add-9b40-4150-b9b2-585eabe3fb35 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:45.371123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a97add-9b40-4150-b9b2-585eabe3fb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:45.373020 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2959/5878] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:23:44 2022] DELETE /identity/v3/users/f525c29419db48e684dd53c758abb74b => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:45.406965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e8a97add-9b40-4150-b9b2-585eabe3fb35 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:45.408202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e8a97add-9b40-4150-b9b2-585eabe3fb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:45.439854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:45.570685 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2920/5879] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:45.578262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bb474-c569-4f50-958b-e3ba18c0d786 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:45.580491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7a5bb474-c569-4f50-958b-e3ba18c0d786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:45.650976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:45.775545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5bb474-c569-4f50-958b-e3ba18c0d786 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:45.797062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ba347-c88d-4781-9bac-595b92b7a872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:45.797062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ba347-c88d-4781-9bac-595b92b7a872 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:45.797062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ba347-c88d-4781-9bac-595b92b7a872 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:45.797062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ba347-c88d-4781-9bac-595b92b7a872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:45.818219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ba347-c88d-4781-9bac-595b92b7a872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:45.819728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ba347-c88d-4781-9bac-595b92b7a872 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:45.887447 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2960/5880] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:23:46.652357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a97add-9b40-4150-b9b2-585eabe3fb35 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:46.654545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e8a97add-9b40-4150-b9b2-585eabe3fb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:46.678471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a97add-9b40-4150-b9b2-585eabe3fb35 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:46.686849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da9f302-1a7e-450b-afd2-f77688ccbeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:46.688207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9f302-1a7e-450b-afd2-f77688ccbeae admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:46.688677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9f302-1a7e-450b-afd2-f77688ccbeae admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:46.689758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da9f302-1a7e-450b-afd2-f77688ccbeae admin admin] PATH_INFO: `/v3/projects/ce4ccb9f2fb945e19bfed037a38ea59d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:46.691751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9f302-1a7e-450b-afd2-f77688ccbeae admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce4ccb9f2fb945e19bfed037a38ea59d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:46.694391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da9f302-1a7e-450b-afd2-f77688ccbeae admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:46.858381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0da9f302-1a7e-450b-afd2-f77688ccbeae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce4ccb9f2fb945e19bfed037a38ea59d', 'request_id': 'req-0da9f302-1a7e-450b-afd2-f77688ccbeae'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:46.875900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0da9f302-1a7e-450b-afd2-f77688ccbeae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce4ccb9f2fb945e19bfed037a38ea59d', 'request_id': 'req-0da9f302-1a7e-450b-afd2-f77688ccbeae'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:46.991875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0da9f302-1a7e-450b-afd2-f77688ccbeae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce4ccb9f2fb945e19bfed037a38ea59d', 'request_id': 'req-0da9f302-1a7e-450b-afd2-f77688ccbeae'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:47.022556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0da9f302-1a7e-450b-afd2-f77688ccbeae admin admin] The token cache is being invalidate because project ce4ccb9f2fb945e19bfed037a38ea59d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:23:47.025709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0da9f302-1a7e-450b-afd2-f77688ccbeae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:47.026926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0da9f302-1a7e-450b-afd2-f77688ccbeae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:47.029554 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2921/5881] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:23:46 2022] DELETE /identity/v3/projects/ce4ccb9f2fb945e19bfed037a38ea59d => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:47.034663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ba347-c88d-4781-9bac-595b92b7a872 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:47.052747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ba347-c88d-4781-9bac-595b92b7a872 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:47.228927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72fd84-066d-4da9-958f-37706049e0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:47.230813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72fd84-066d-4da9-958f-37706049e0c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:47.231270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72fd84-066d-4da9-958f-37706049e0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:47.231736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b72fd84-066d-4da9-958f-37706049e0c4 admin admin] PATH_INFO: `/v3/users/1d17b32735de4648a03d6001e8e398b7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:47.233266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72fd84-066d-4da9-958f-37706049e0c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d17b32735de4648a03d6001e8e398b7)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:47.235447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b72fd84-066d-4da9-958f-37706049e0c4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:47.563802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5b72fd84-066d-4da9-958f-37706049e0c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d17b32735de4648a03d6001e8e398b7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:47.577305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5b72fd84-066d-4da9-958f-37706049e0c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d17b32735de4648a03d6001e8e398b7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:47.579818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5b72fd84-066d-4da9-958f-37706049e0c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d17b32735de4648a03d6001e8e398b7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:47.582268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5b72fd84-066d-4da9-958f-37706049e0c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d17b32735de4648a03d6001e8e398b7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:47.681528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5b72fd84-066d-4da9-958f-37706049e0c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d17b32735de4648a03d6001e8e398b7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:47.719153 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2961/5882] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:23:47 2022] DELETE /identity/v3/users/1d17b32735de4648a03d6001e8e398b7 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:47.959637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00032730-8466-4132-9c59-abba0b9e6c78 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:47.960105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00032730-8466-4132-9c59-abba0b9e6c78 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:47.960407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00032730-8466-4132-9c59-abba0b9e6c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:47.991595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-00032730-8466-4132-9c59-abba0b9e6c78 None None] MFA Rules not processed for user `f7c1a400e0dd462da3bb33b7e3fe57fc`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:47.993179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-00032730-8466-4132-9c59-abba0b9e6c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:48.095104 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2922/5883] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:48.561210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72fd84-066d-4da9-958f-37706049e0c4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:48.562676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5b72fd84-066d-4da9-958f-37706049e0c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:48.635968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:48.675865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b72fd84-066d-4da9-958f-37706049e0c4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:48.682319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975020b2-cab1-4313-b03f-06edd6a877bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:48.683769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975020b2-cab1-4313-b03f-06edd6a877bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:48.684055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975020b2-cab1-4313-b03f-06edd6a877bc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:48.684330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975020b2-cab1-4313-b03f-06edd6a877bc admin admin] PATH_INFO: `/v3/projects/dbb4877d1a184ecd85b7a201ede5470d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:48.685548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975020b2-cab1-4313-b03f-06edd6a877bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbb4877d1a184ecd85b7a201ede5470d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:48.688916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975020b2-cab1-4313-b03f-06edd6a877bc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:48.835069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-975020b2-cab1-4313-b03f-06edd6a877bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbb4877d1a184ecd85b7a201ede5470d', 'request_id': 'req-975020b2-cab1-4313-b03f-06edd6a877bc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:48.845717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-975020b2-cab1-4313-b03f-06edd6a877bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbb4877d1a184ecd85b7a201ede5470d', 'request_id': 'req-975020b2-cab1-4313-b03f-06edd6a877bc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:48.948961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-975020b2-cab1-4313-b03f-06edd6a877bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbb4877d1a184ecd85b7a201ede5470d', 'request_id': 'req-975020b2-cab1-4313-b03f-06edd6a877bc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:48.975503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-975020b2-cab1-4313-b03f-06edd6a877bc admin admin] The token cache is being invalidate because project dbb4877d1a184ecd85b7a201ede5470d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:23:48.977657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-975020b2-cab1-4313-b03f-06edd6a877bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:48.981775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-975020b2-cab1-4313-b03f-06edd6a877bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:48.991149 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2962/5884] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:23:48 2022] DELETE /identity/v3/projects/dbb4877d1a184ecd85b7a201ede5470d => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:48.995820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00032730-8466-4132-9c59-abba0b9e6c78 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:49.012840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00032730-8466-4132-9c59-abba0b9e6c78 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:49.124749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:49.176081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb20f0-5adb-4e40-b72b-5d31a225240d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:49.178194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb20f0-5adb-4e40-b72b-5d31a225240d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:49.178738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb20f0-5adb-4e40-b72b-5d31a225240d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:49.181027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb20f0-5adb-4e40-b72b-5d31a225240d admin admin] PATH_INFO: `/v3/users/49533fb58b4742fc912ddc7147ab2006` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:49.182749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb20f0-5adb-4e40-b72b-5d31a225240d admin admin] RBAC: Authorizing `identity:delete_user(user_id=49533fb58b4742fc912ddc7147ab2006)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:49.188182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb20f0-5adb-4e40-b72b-5d31a225240d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:49.487394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-46cb20f0-5adb-4e40-b72b-5d31a225240d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49533fb58b4742fc912ddc7147ab2006'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:49.493067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-46cb20f0-5adb-4e40-b72b-5d31a225240d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49533fb58b4742fc912ddc7147ab2006'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:49.553893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-46cb20f0-5adb-4e40-b72b-5d31a225240d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49533fb58b4742fc912ddc7147ab2006'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:49.691666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-46cb20f0-5adb-4e40-b72b-5d31a225240d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49533fb58b4742fc912ddc7147ab2006'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:49.697981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-46cb20f0-5adb-4e40-b72b-5d31a225240d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49533fb58b4742fc912ddc7147ab2006'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:49.756591 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2923/5885] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:23:48 2022] DELETE /identity/v3/users/49533fb58b4742fc912ddc7147ab2006 => generated 0 bytes in 762 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:50.307003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975020b2-cab1-4313-b03f-06edd6a877bc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:50.312126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-975020b2-cab1-4313-b03f-06edd6a877bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:50.590138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975020b2-cab1-4313-b03f-06edd6a877bc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:50.720953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab34ac1-17f0-4630-935a-21ed3c898679 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:50.721934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab34ac1-17f0-4630-935a-21ed3c898679 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:50.722202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab34ac1-17f0-4630-935a-21ed3c898679 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:50.723025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab34ac1-17f0-4630-935a-21ed3c898679 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:50.725971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cab34ac1-17f0-4630-935a-21ed3c898679 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:50.753511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab34ac1-17f0-4630-935a-21ed3c898679 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:50.755463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab34ac1-17f0-4630-935a-21ed3c898679 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:50.764648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:50.784370 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2963/5886] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:23:51.397124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb20f0-5adb-4e40-b72b-5d31a225240d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:51.399114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-46cb20f0-5adb-4e40-b72b-5d31a225240d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:51.661826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb20f0-5adb-4e40-b72b-5d31a225240d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:51.679183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9da052-bbd6-40be-ba1c-bd22360d4dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:51.682305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9da052-bbd6-40be-ba1c-bd22360d4dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:51.682599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9da052-bbd6-40be-ba1c-bd22360d4dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:51.682917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9da052-bbd6-40be-ba1c-bd22360d4dd5 admin admin] PATH_INFO: `/v3/projects/d642828272ff49bc9a0b6dbaa4e754e9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:51.691463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9da052-bbd6-40be-ba1c-bd22360d4dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d642828272ff49bc9a0b6dbaa4e754e9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:51.701275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9da052-bbd6-40be-ba1c-bd22360d4dd5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:51.853703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab34ac1-17f0-4630-935a-21ed3c898679 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:51.856490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cab34ac1-17f0-4630-935a-21ed3c898679 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:51.905328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2d9da052-bbd6-40be-ba1c-bd22360d4dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd642828272ff49bc9a0b6dbaa4e754e9', 'request_id': 'req-2d9da052-bbd6-40be-ba1c-bd22360d4dd5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:51.986981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2d9da052-bbd6-40be-ba1c-bd22360d4dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd642828272ff49bc9a0b6dbaa4e754e9', 'request_id': 'req-2d9da052-bbd6-40be-ba1c-bd22360d4dd5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:52.004833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:52.060771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab34ac1-17f0-4630-935a-21ed3c898679 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:52.130672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2d9da052-bbd6-40be-ba1c-bd22360d4dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd642828272ff49bc9a0b6dbaa4e754e9', 'request_id': 'req-2d9da052-bbd6-40be-ba1c-bd22360d4dd5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:52.159098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13acfac-2329-47b7-a8a2-075c93f51c9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:52.159098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13acfac-2329-47b7-a8a2-075c93f51c9f service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:52.159098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13acfac-2329-47b7-a8a2-075c93f51c9f service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:52.159098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13acfac-2329-47b7-a8a2-075c93f51c9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:52.159781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2d9da052-bbd6-40be-ba1c-bd22360d4dd5 admin admin] The token cache is being invalidate because project d642828272ff49bc9a0b6dbaa4e754e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:23:52.159781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2d9da052-bbd6-40be-ba1c-bd22360d4dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:52.160678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2d9da052-bbd6-40be-ba1c-bd22360d4dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:23:52.162765 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2924/5887] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:23:51 2022] DELETE /identity/v3/projects/d642828272ff49bc9a0b6dbaa4e754e9 => generated 0 bytes in 766 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:52.169088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d13acfac-2329-47b7-a8a2-075c93f51c9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:52.171836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17f0eb-0dbb-408e-9557-7acab281dc8c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:52.172119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17f0eb-0dbb-408e-9557-7acab281dc8c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:52.172405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf17f0eb-0dbb-408e-9557-7acab281dc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:52.193175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13acfac-2329-47b7-a8a2-075c93f51c9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:52.195035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13acfac-2329-47b7-a8a2-075c93f51c9f service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:52.211427 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2964/5888] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:23:52.237193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bf17f0eb-0dbb-408e-9557-7acab281dc8c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:52.243775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bf17f0eb-0dbb-408e-9557-7acab281dc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:52.435659 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2925/5889] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:52.444570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc66cf-ceba-4706-b914-0d2d5020f602 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:52.445093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc66cf-ceba-4706-b914-0d2d5020f602 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:52.445490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dc66cf-ceba-4706-b914-0d2d5020f602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:52.480962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a5dc66cf-ceba-4706-b914-0d2d5020f602 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:52.483011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a5dc66cf-ceba-4706-b914-0d2d5020f602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:52.499450 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2965/5890] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:52.503335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17f0eb-0dbb-408e-9557-7acab281dc8c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:52.515828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf17f0eb-0dbb-408e-9557-7acab281dc8c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:52.545800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ad86b-9c3a-4851-bddd-429b5271f24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:52.546780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ad86b-9c3a-4851-bddd-429b5271f24a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:52.555132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ad86b-9c3a-4851-bddd-429b5271f24a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:52.555132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ad86b-9c3a-4851-bddd-429b5271f24a admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:52.555132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ad86b-9c3a-4851-bddd-429b5271f24a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:23:52.555132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ad86b-9c3a-4851-bddd-429b5271f24a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:52.555132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ad86b-9c3a-4851-bddd-429b5271f24a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:52.595436 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2926/5891] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:23:52.596127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc66cf-ceba-4706-b914-0d2d5020f602 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:52.612248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dc66cf-ceba-4706-b914-0d2d5020f602 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:52.618204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a089c41-e4ee-4cd5-916d-72ba44c06add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:52.619003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a089c41-e4ee-4cd5-916d-72ba44c06add admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:52.619448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a089c41-e4ee-4cd5-916d-72ba44c06add admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:52.619831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a089c41-e4ee-4cd5-916d-72ba44c06add admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:52.621350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a089c41-e4ee-4cd5-916d-72ba44c06add admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:52.625017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a089c41-e4ee-4cd5-916d-72ba44c06add admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:52.759394 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2966/5892] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:23:52 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:23:52.766340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ad86b-9c3a-4851-bddd-429b5271f24a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:52.784499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ad86b-9c3a-4851-bddd-429b5271f24a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:52.945345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70501b3b-0571-413a-bdb9-74ba76c4fa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:52.945345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70501b3b-0571-413a-bdb9-74ba76c4fa99 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:52.945345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70501b3b-0571-413a-bdb9-74ba76c4fa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:52.945345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70501b3b-0571-413a-bdb9-74ba76c4fa99 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:52.954832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70501b3b-0571-413a-bdb9-74ba76c4fa99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:52.958238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70501b3b-0571-413a-bdb9-74ba76c4fa99 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:53.125724 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2927/5893] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:23:52 2022] POST /identity/v3/users => generated 289 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:23:53.129932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a089c41-e4ee-4cd5-916d-72ba44c06add admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:53.141670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a089c41-e4ee-4cd5-916d-72ba44c06add admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:53.161322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae461c-bf13-4b4d-abf1-f015cf7bd816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:53.161322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae461c-bf13-4b4d-abf1-f015cf7bd816 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:53.161322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae461c-bf13-4b4d-abf1-f015cf7bd816 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:53.161322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae461c-bf13-4b4d-abf1-f015cf7bd816 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:53.161709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae461c-bf13-4b4d-abf1-f015cf7bd816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:53.162906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae461c-bf13-4b4d-abf1-f015cf7bd816 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:53.192988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70501b3b-0571-413a-bdb9-74ba76c4fa99 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:53.197398 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2967/5894] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:53.225240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70501b3b-0571-413a-bdb9-74ba76c4fa99 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:53.247542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b228ffa-7b40-4fbd-91bb-70f146ced979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:53.248520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b228ffa-7b40-4fbd-91bb-70f146ced979 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:53.249027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b228ffa-7b40-4fbd-91bb-70f146ced979 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:53.249572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b228ffa-7b40-4fbd-91bb-70f146ced979 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:53.250704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b228ffa-7b40-4fbd-91bb-70f146ced979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:53.252740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b228ffa-7b40-4fbd-91bb-70f146ced979 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:53.278900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae461c-bf13-4b4d-abf1-f015cf7bd816 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:53.292872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae461c-bf13-4b4d-abf1-f015cf7bd816 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:53.295050 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2928/5895] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:53 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:53.314522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4fbbb-da68-4342-b67f-6f09561019e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:53.315990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4fbbb-da68-4342-b67f-6f09561019e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:53.316680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4fbbb-da68-4342-b67f-6f09561019e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:53.317211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb4fbbb-da68-4342-b67f-6f09561019e7 admin admin] PATH_INFO: `/v3/projects/2169bc0792a74919bbe07258b5ae5cc4/users/a81fdaea686041ac96619055a9371731/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:53.358995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4fbbb-da68-4342-b67f-6f09561019e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2169bc0792a74919bbe07258b5ae5cc4, user_id=a81fdaea686041ac96619055a9371731, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:53.366358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb4fbbb-da68-4342-b67f-6f09561019e7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:53.551220 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2968/5896] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:23:53 2022] PUT /identity/v3/projects/2169bc0792a74919bbe07258b5ae5cc4/users/a81fdaea686041ac96619055a9371731/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:53.552288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60fe9b-2f4d-4444-88ab-f361beb0b6cc None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:53.553109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60fe9b-2f4d-4444-88ab-f361beb0b6cc None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:53.553573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60fe9b-2f4d-4444-88ab-f361beb0b6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:53.613383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ec60fe9b-2f4d-4444-88ab-f361beb0b6cc None None] MFA Rules not processed for user `a81fdaea686041ac96619055a9371731`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:53.619395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ec60fe9b-2f4d-4444-88ab-f361beb0b6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:53.745955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4fbbb-da68-4342-b67f-6f09561019e7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:53.748271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3bb4fbbb-da68-4342-b67f-6f09561019e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:53.847441 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2929/5897] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:53.852370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60fe9b-2f4d-4444-88ab-f361beb0b6cc None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:53.877333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:53.958462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb4fbbb-da68-4342-b67f-6f09561019e7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:53.974697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcdee2-b780-4293-a82f-cec83ad134d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:53.976489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcdee2-b780-4293-a82f-cec83ad134d7 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:53.992011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcdee2-b780-4293-a82f-cec83ad134d7 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:53.992327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcdee2-b780-4293-a82f-cec83ad134d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:53.997478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8cbcdee2-b780-4293-a82f-cec83ad134d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:54.018049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60fe9b-2f4d-4444-88ab-f361beb0b6cc None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:54.018295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbcdee2-b780-4293-a82f-cec83ad134d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:54.023978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbcdee2-b780-4293-a82f-cec83ad134d7 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:54.030091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7676b-5b37-4e7d-a32a-39b0192c9454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:54.034150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7676b-5b37-4e7d-a32a-39b0192c9454 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:54.034460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7676b-5b37-4e7d-a32a-39b0192c9454 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:54.034742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7676b-5b37-4e7d-a32a-39b0192c9454 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:54.035462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7676b-5b37-4e7d-a32a-39b0192c9454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:54.042967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7676b-5b37-4e7d-a32a-39b0192c9454 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:54.043571 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2969/5898] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:23:54.235636 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2930/5899] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:23:53 2022] POST /identity/v3/projects => generated 377 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:23:54.240050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcdee2-b780-4293-a82f-cec83ad134d7 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:54.282275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcdee2-b780-4293-a82f-cec83ad134d7 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:54.534971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4914b2f-929f-4355-8d3f-54a98b325132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:54.535956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4914b2f-929f-4355-8d3f-54a98b325132 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:54.536246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4914b2f-929f-4355-8d3f-54a98b325132 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:54.536528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4914b2f-929f-4355-8d3f-54a98b325132 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:54.537253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4914b2f-929f-4355-8d3f-54a98b325132 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:54.539806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4914b2f-929f-4355-8d3f-54a98b325132 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:55.022472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7676b-5b37-4e7d-a32a-39b0192c9454 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:55.024092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8ed7676b-5b37-4e7d-a32a-39b0192c9454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:55.062721 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2970/5900] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:23:54 2022] POST /identity/v3/users => generated 289 bytes in 820 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:23:55.064845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4914b2f-929f-4355-8d3f-54a98b325132 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:55.108969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4914b2f-929f-4355-8d3f-54a98b325132 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:55.115870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc21f8d-0e72-4ecd-93d8-cbcd59f4beab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:55.116690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc21f8d-0e72-4ecd-93d8-cbcd59f4beab admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:55.116980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc21f8d-0e72-4ecd-93d8-cbcd59f4beab admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:55.118082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc21f8d-0e72-4ecd-93d8-cbcd59f4beab admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:55.119044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc21f8d-0e72-4ecd-93d8-cbcd59f4beab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:55.124965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc21f8d-0e72-4ecd-93d8-cbcd59f4beab admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:55.148458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7676b-5b37-4e7d-a32a-39b0192c9454 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:55.156722 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2971/5901] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:55.160702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc21f8d-0e72-4ecd-93d8-cbcd59f4beab admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:55.161557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a4188-4ca5-436b-9cf1-3121192c4a81 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:55.162780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a4188-4ca5-436b-9cf1-3121192c4a81 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:55.163076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a4188-4ca5-436b-9cf1-3121192c4a81 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:55.163361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a4188-4ca5-436b-9cf1-3121192c4a81 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:55.165825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-452a4188-4ca5-436b-9cf1-3121192c4a81 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:55.178607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc21f8d-0e72-4ecd-93d8-cbcd59f4beab admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:55.180693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a4188-4ca5-436b-9cf1-3121192c4a81 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:55.182445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a4188-4ca5-436b-9cf1-3121192c4a81 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:55.185497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326101b8-e74a-40af-baa4-7dc27525ca6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:55.186193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326101b8-e74a-40af-baa4-7dc27525ca6c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:55.186580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326101b8-e74a-40af-baa4-7dc27525ca6c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:55.186870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326101b8-e74a-40af-baa4-7dc27525ca6c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:55.188403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326101b8-e74a-40af-baa4-7dc27525ca6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:55.190512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326101b8-e74a-40af-baa4-7dc27525ca6c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:55.197009 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2931/5902] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:23:55.205247 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2972/5903] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:55.209366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a4188-4ca5-436b-9cf1-3121192c4a81 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:55.212033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:55.230070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a4188-4ca5-436b-9cf1-3121192c4a81 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:55.235821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de20070-49cc-44ed-8e58-50cb2f55a56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:55.236939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de20070-49cc-44ed-8e58-50cb2f55a56a admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:55.237395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de20070-49cc-44ed-8e58-50cb2f55a56a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:55.237828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de20070-49cc-44ed-8e58-50cb2f55a56a admin admin] PATH_INFO: `/v3/projects/1f59f624c2e24eb291f0a741a8406864/users/526ea594b62848cb81fe32c560ede0b7/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:55.307102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de20070-49cc-44ed-8e58-50cb2f55a56a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f59f624c2e24eb291f0a741a8406864, user_id=526ea594b62848cb81fe32c560ede0b7, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:55.310022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de20070-49cc-44ed-8e58-50cb2f55a56a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:55.395082 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2932/5904] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:23:55 2022] PUT /identity/v3/projects/1f59f624c2e24eb291f0a741a8406864/users/526ea594b62848cb81fe32c560ede0b7/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:55.407416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002ab9f-d934-484e-a5d9-2ed01d9ba907 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:55.408291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002ab9f-d934-484e-a5d9-2ed01d9ba907 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:55.408783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002ab9f-d934-484e-a5d9-2ed01d9ba907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:55.472828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e002ab9f-d934-484e-a5d9-2ed01d9ba907 None None] MFA Rules not processed for user `526ea594b62848cb81fe32c560ede0b7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:55.477110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e002ab9f-d934-484e-a5d9-2ed01d9ba907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:55.769137 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2973/5905] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:55.769594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de20070-49cc-44ed-8e58-50cb2f55a56a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:55.913563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de20070-49cc-44ed-8e58-50cb2f55a56a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:55.920019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8e4bc-0f97-40d0-946d-ed2e60da5c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:55.920702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8e4bc-0f97-40d0-946d-ed2e60da5c2a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:55.921046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8e4bc-0f97-40d0-946d-ed2e60da5c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:55.921398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be8e4bc-0f97-40d0-946d-ed2e60da5c2a admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:55.922076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8e4bc-0f97-40d0-946d-ed2e60da5c2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:55.934195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be8e4bc-0f97-40d0-946d-ed2e60da5c2a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:56.067994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002ab9f-d934-484e-a5d9-2ed01d9ba907 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:56.068840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e002ab9f-d934-484e-a5d9-2ed01d9ba907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:56.082197 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2933/5906] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:23:55 2022] POST /identity/v3/projects => generated 379 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:23:56.083631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8e4bc-0f97-40d0-946d-ed2e60da5c2a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:56.268707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:56.316387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002ab9f-d934-484e-a5d9-2ed01d9ba907 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:56.405666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be8e4bc-0f97-40d0-946d-ed2e60da5c2a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:56.421658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a2c1c-faf7-4105-896a-60cbddd322cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:56.422249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a2c1c-faf7-4105-896a-60cbddd322cf admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:56.422558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a2c1c-faf7-4105-896a-60cbddd322cf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:56.422818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a2c1c-faf7-4105-896a-60cbddd322cf admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:56.423523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a2c1c-faf7-4105-896a-60cbddd322cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:56.425862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a2c1c-faf7-4105-896a-60cbddd322cf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:56.504255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c63ee6-f2f4-4272-9744-ab42a4e129ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:56.505118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c63ee6-f2f4-4272-9744-ab42a4e129ec service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:56.505679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c63ee6-f2f4-4272-9744-ab42a4e129ec service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:56.506208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c63ee6-f2f4-4272-9744-ab42a4e129ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:56.508692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d3c63ee6-f2f4-4272-9744-ab42a4e129ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:56.527969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c63ee6-f2f4-4272-9744-ab42a4e129ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:56.530249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c63ee6-f2f4-4272-9744-ab42a4e129ec service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:56.561011 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2974/5907] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:23:56.640437 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2934/5908] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:23:56 2022] POST /identity/v3/users => generated 296 bytes in 557 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:23:56.645592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c63ee6-f2f4-4272-9744-ab42a4e129ec service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:56.687331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c63ee6-f2f4-4272-9744-ab42a4e129ec service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:56.704373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9c477-0e3b-4d47-a009-9121484d2601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:56.704948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9c477-0e3b-4d47-a009-9121484d2601 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:56.705706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9c477-0e3b-4d47-a009-9121484d2601 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:56.705706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9c477-0e3b-4d47-a009-9121484d2601 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:56.706752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9c477-0e3b-4d47-a009-9121484d2601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:56.708593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9c477-0e3b-4d47-a009-9121484d2601 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:56.737008 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2975/5909] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:56.744038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a2c1c-faf7-4105-896a-60cbddd322cf admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:56.769686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a2c1c-faf7-4105-896a-60cbddd322cf admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:56.793118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616bf32-3268-44b9-82fd-7e144105e5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:56.793118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616bf32-3268-44b9-82fd-7e144105e5a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:56.797542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616bf32-3268-44b9-82fd-7e144105e5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:56.798651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6616bf32-3268-44b9-82fd-7e144105e5a6 admin admin] PATH_INFO: `/v3/projects/9f5f37eb21524c0db5d69a43e6623696/users/64f6b3652ce44ad091e59dc009ea83af/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:56.824867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33329f-5769-4155-bc9a-547f3c4d96df None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:56.825498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33329f-5769-4155-bc9a-547f3c4d96df None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:56.825788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33329f-5769-4155-bc9a-547f3c4d96df None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:56.827586 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2976/5910] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:23:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 21:23:56.833476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ac34b-4914-4e6c-83eb-32a21204f307 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:56.833682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ac34b-4914-4e6c-83eb-32a21204f307 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:56.833938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ac34b-4914-4e6c-83eb-32a21204f307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:56.836699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616bf32-3268-44b9-82fd-7e144105e5a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f5f37eb21524c0db5d69a43e6623696, user_id=64f6b3652ce44ad091e59dc009ea83af, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:56.843702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6616bf32-3268-44b9-82fd-7e144105e5a6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:56.903450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-707ac34b-4914-4e6c-83eb-32a21204f307 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:56.905155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-707ac34b-4914-4e6c-83eb-32a21204f307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:56.963436 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2935/5911] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:23:56 2022] PUT /identity/v3/projects/9f5f37eb21524c0db5d69a43e6623696/users/64f6b3652ce44ad091e59dc009ea83af/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:56.970756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616bf32-3268-44b9-82fd-7e144105e5a6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:57.181545 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2977/5912] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:23:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:57.186481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:57.195956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6616bf32-3268-44b9-82fd-7e144105e5a6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:57.205391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ad813-fab0-4b1e-9ccf-56fe29ac07e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:57.209507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ad813-fab0-4b1e-9ccf-56fe29ac07e1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:57.209507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ad813-fab0-4b1e-9ccf-56fe29ac07e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:57.209507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ad813-fab0-4b1e-9ccf-56fe29ac07e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:57.209507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ad813-fab0-4b1e-9ccf-56fe29ac07e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:57.211468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ad813-fab0-4b1e-9ccf-56fe29ac07e1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:57.219578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ac34b-4914-4e6c-83eb-32a21204f307 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:57.238162 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2936/5913] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:23:56 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:23:57.242301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ad813-fab0-4b1e-9ccf-56fe29ac07e1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:57.266239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ac34b-4914-4e6c-83eb-32a21204f307 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:57.268965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ad813-fab0-4b1e-9ccf-56fe29ac07e1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:57.290477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c6d29-9dbf-402f-ac33-07f912facf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:57.291467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c6d29-9dbf-402f-ac33-07f912facf6a admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:57.291907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c6d29-9dbf-402f-ac33-07f912facf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:57.293975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c6d29-9dbf-402f-ac33-07f912facf6a admin admin] PATH_INFO: `/v3/projects/9f5f37eb21524c0db5d69a43e6623696/users/64f6b3652ce44ad091e59dc009ea83af/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:57.303867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c6d29-9dbf-402f-ac33-07f912facf6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f5f37eb21524c0db5d69a43e6623696, user_id=64f6b3652ce44ad091e59dc009ea83af, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:57.318290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3c6d29-9dbf-402f-ac33-07f912facf6a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:57.349895 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2937/5914] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:23:57 2022] PUT /identity/v3/projects/9f5f37eb21524c0db5d69a43e6623696/users/64f6b3652ce44ad091e59dc009ea83af/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:23:57.365801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb671d7-7c1d-4b02-90d0-38008647614f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:57.366130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb671d7-7c1d-4b02-90d0-38008647614f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:57.366398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb671d7-7c1d-4b02-90d0-38008647614f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:57.376460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe182c9-a7e9-4d47-87ae-938e7809b12b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:57.378596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe182c9-a7e9-4d47-87ae-938e7809b12b service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:57.378957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe182c9-a7e9-4d47-87ae-938e7809b12b service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:57.379325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe182c9-a7e9-4d47-87ae-938e7809b12b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:57.393492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe182c9-a7e9-4d47-87ae-938e7809b12b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:57.398018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe182c9-a7e9-4d47-87ae-938e7809b12b service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:57.413910 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2978/5915] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:23:57.440251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ebb671d7-7c1d-4b02-90d0-38008647614f None None] MFA Rules not processed for user `64f6b3652ce44ad091e59dc009ea83af`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:23:57.457347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ebb671d7-7c1d-4b02-90d0-38008647614f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:57.643838 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2938/5916] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:23:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:23:57.656352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe182c9-a7e9-4d47-87ae-938e7809b12b service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:57.658990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cfe182c9-a7e9-4d47-87ae-938e7809b12b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:57.834520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe182c9-a7e9-4d47-87ae-938e7809b12b service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:57.893133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae593dd-d679-4e1b-925c-d348c9576c03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:57.893928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae593dd-d679-4e1b-925c-d348c9576c03 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:57.894278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae593dd-d679-4e1b-925c-d348c9576c03 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:57.894567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae593dd-d679-4e1b-925c-d348c9576c03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:57.906608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae593dd-d679-4e1b-925c-d348c9576c03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:57.907906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae593dd-d679-4e1b-925c-d348c9576c03 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:57.921197 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2979/5917] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:23:58.177904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb671d7-7c1d-4b02-90d0-38008647614f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:58.179404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ebb671d7-7c1d-4b02-90d0-38008647614f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:58.207972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb671d7-7c1d-4b02-90d0-38008647614f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:58.230787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f9d14c-e343-487e-a8da-b96a97c36790 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:58.231721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9d14c-e343-487e-a8da-b96a97c36790 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:58.231721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9d14c-e343-487e-a8da-b96a97c36790 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:58.231897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f9d14c-e343-487e-a8da-b96a97c36790 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:58.280048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:58.446502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9d14c-e343-487e-a8da-b96a97c36790 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:58.446973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f9d14c-e343-487e-a8da-b96a97c36790 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:58.463011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:58.484204 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2939/5918] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:23:59.674048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae593dd-d679-4e1b-925c-d348c9576c03 service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:23:59.912625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae593dd-d679-4e1b-925c-d348c9576c03 service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:23:59.921393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935599cb-93c2-4eab-89ee-a9b121ee7e61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:23:59.922418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935599cb-93c2-4eab-89ee-a9b121ee7e61 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:23:59.923068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935599cb-93c2-4eab-89ee-a9b121ee7e61 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:23:59.923526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935599cb-93c2-4eab-89ee-a9b121ee7e61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:23:59.925767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-935599cb-93c2-4eab-89ee-a9b121ee7e61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:23:59.951080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935599cb-93c2-4eab-89ee-a9b121ee7e61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:23:59.952723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935599cb-93c2-4eab-89ee-a9b121ee7e61 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:23:59.983138 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2980/5919] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:23:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:24:00.052229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefef2d8-4610-4f56-ba07-b8c069136514 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:00.053514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefef2d8-4610-4f56-ba07-b8c069136514 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:00.054099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefef2d8-4610-4f56-ba07-b8c069136514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:00.134982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-cefef2d8-4610-4f56-ba07-b8c069136514 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:00.147069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cefef2d8-4610-4f56-ba07-b8c069136514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:00.236034 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2940/5920] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:24:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:00.251106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935599cb-93c2-4eab-89ee-a9b121ee7e61 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:00.252954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-935599cb-93c2-4eab-89ee-a9b121ee7e61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:00.350944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:00.410076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935599cb-93c2-4eab-89ee-a9b121ee7e61 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:00.416533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9f4a0-2015-435d-ba8f-a8685b8da981 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:00.418000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9f4a0-2015-435d-ba8f-a8685b8da981 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:00.420059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9f4a0-2015-435d-ba8f-a8685b8da981 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:00.424401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9f4a0-2015-435d-ba8f-a8685b8da981 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:00.433291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9f4a0-2015-435d-ba8f-a8685b8da981 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:00.435492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d9f4a0-2015-435d-ba8f-a8685b8da981 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:00.447614 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2981/5921] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:24:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:24:01.396010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefef2d8-4610-4f56-ba07-b8c069136514 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:01.397486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cefef2d8-4610-4f56-ba07-b8c069136514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:01.530384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9f4a0-2015-435d-ba8f-a8685b8da981 service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:01.559234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9f4a0-2015-435d-ba8f-a8685b8da981 service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:01.569039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:01.571190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150554c-6a92-40c9-a106-83c124ad2fe6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:01.572686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150554c-6a92-40c9-a106-83c124ad2fe6 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:01.574773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150554c-6a92-40c9-a106-83c124ad2fe6 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:01.575214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150554c-6a92-40c9-a106-83c124ad2fe6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:01.579156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e150554c-6a92-40c9-a106-83c124ad2fe6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:01.611330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150554c-6a92-40c9-a106-83c124ad2fe6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:01.613300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150554c-6a92-40c9-a106-83c124ad2fe6 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:01.625059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefef2d8-4610-4f56-ba07-b8c069136514 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:01.640601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490ff65-c17d-497a-b7e5-38f83580b112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:01.646549 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2982/5922] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:24:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:24:01.650344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490ff65-c17d-497a-b7e5-38f83580b112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:01.650944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490ff65-c17d-497a-b7e5-38f83580b112 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:01.654949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490ff65-c17d-497a-b7e5-38f83580b112 admin admin] PATH_INFO: `/v3/users/15841f513249448081439041b58d091c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:01.654949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490ff65-c17d-497a-b7e5-38f83580b112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15841f513249448081439041b58d091c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:01.656319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490ff65-c17d-497a-b7e5-38f83580b112 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:01.986431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5490ff65-c17d-497a-b7e5-38f83580b112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15841f513249448081439041b58d091c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:01.988591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5490ff65-c17d-497a-b7e5-38f83580b112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15841f513249448081439041b58d091c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:02.015528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5490ff65-c17d-497a-b7e5-38f83580b112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15841f513249448081439041b58d091c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:02.091973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5490ff65-c17d-497a-b7e5-38f83580b112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15841f513249448081439041b58d091c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:02.092605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5490ff65-c17d-497a-b7e5-38f83580b112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15841f513249448081439041b58d091c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:02.152882 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2941/5923] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:24:01 2022] DELETE /identity/v3/users/15841f513249448081439041b58d091c => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:02.154397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1dd9f-48d4-46cb-82ac-7b83a85b5a48 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:02.154694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1dd9f-48d4-46cb-82ac-7b83a85b5a48 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:02.155023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e1dd9f-48d4-46cb-82ac-7b83a85b5a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:02.211985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d3e1dd9f-48d4-46cb-82ac-7b83a85b5a48 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:02.220031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d3e1dd9f-48d4-46cb-82ac-7b83a85b5a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:02.533458 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2983/5924] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:02.660855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490ff65-c17d-497a-b7e5-38f83580b112 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:02.662656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5490ff65-c17d-497a-b7e5-38f83580b112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:02.834383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490ff65-c17d-497a-b7e5-38f83580b112 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:02.838846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fceaeec-b804-4b7d-aa9e-2c777c9e2c85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:02.840324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fceaeec-b804-4b7d-aa9e-2c777c9e2c85 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:02.840814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fceaeec-b804-4b7d-aa9e-2c777c9e2c85 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:02.841425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fceaeec-b804-4b7d-aa9e-2c777c9e2c85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:02.865118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fceaeec-b804-4b7d-aa9e-2c777c9e2c85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:02.866751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fceaeec-b804-4b7d-aa9e-2c777c9e2c85 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:02.904767 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2942/5925] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:24:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:24:04.145105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1dd9f-48d4-46cb-82ac-7b83a85b5a48 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:04.148869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d3e1dd9f-48d4-46cb-82ac-7b83a85b5a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:04.219226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e1dd9f-48d4-46cb-82ac-7b83a85b5a48 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:04.295433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422b3c4-1a7b-4a44-a945-e3d2de33b204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:04.296905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422b3c4-1a7b-4a44-a945-e3d2de33b204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:04.307754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422b3c4-1a7b-4a44-a945-e3d2de33b204 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:04.320763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422b3c4-1a7b-4a44-a945-e3d2de33b204 admin admin] PATH_INFO: `/v3/projects/e0352bb38c2c4a87bed4a2ca8dbbf437` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:04.330012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422b3c4-1a7b-4a44-a945-e3d2de33b204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0352bb38c2c4a87bed4a2ca8dbbf437)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:04.348466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422b3c4-1a7b-4a44-a945-e3d2de33b204 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:04.749758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d422b3c4-1a7b-4a44-a945-e3d2de33b204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0352bb38c2c4a87bed4a2ca8dbbf437', 'request_id': 'req-d422b3c4-1a7b-4a44-a945-e3d2de33b204'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:04.790514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d422b3c4-1a7b-4a44-a945-e3d2de33b204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0352bb38c2c4a87bed4a2ca8dbbf437', 'request_id': 'req-d422b3c4-1a7b-4a44-a945-e3d2de33b204'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:04.900258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d422b3c4-1a7b-4a44-a945-e3d2de33b204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0352bb38c2c4a87bed4a2ca8dbbf437', 'request_id': 'req-d422b3c4-1a7b-4a44-a945-e3d2de33b204'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:04.966885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d422b3c4-1a7b-4a44-a945-e3d2de33b204 admin admin] The token cache is being invalidate because project e0352bb38c2c4a87bed4a2ca8dbbf437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:24:04.970346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d422b3c4-1a7b-4a44-a945-e3d2de33b204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:04.975162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d422b3c4-1a7b-4a44-a945-e3d2de33b204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:04.983165 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2984/5926] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:24:04 2022] DELETE /identity/v3/projects/e0352bb38c2c4a87bed4a2ca8dbbf437 => generated 0 bytes in 839 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:24:09.136450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fceaeec-b804-4b7d-aa9e-2c777c9e2c85 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:09.138521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4fceaeec-b804-4b7d-aa9e-2c777c9e2c85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:09.174651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fceaeec-b804-4b7d-aa9e-2c777c9e2c85 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:09.353916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081d370-bae7-4e5f-930f-78e26c69892f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:09.355337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081d370-bae7-4e5f-930f-78e26c69892f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:09.355634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081d370-bae7-4e5f-930f-78e26c69892f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:09.355894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081d370-bae7-4e5f-930f-78e26c69892f admin admin] PATH_INFO: `/v3/users/a81fdaea686041ac96619055a9371731` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:09.367257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081d370-bae7-4e5f-930f-78e26c69892f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a81fdaea686041ac96619055a9371731)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:09.369254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081d370-bae7-4e5f-930f-78e26c69892f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:09.373371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:09.675485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9081d370-bae7-4e5f-930f-78e26c69892f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a81fdaea686041ac96619055a9371731'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:09.676194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9081d370-bae7-4e5f-930f-78e26c69892f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a81fdaea686041ac96619055a9371731'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:09.685706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9081d370-bae7-4e5f-930f-78e26c69892f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a81fdaea686041ac96619055a9371731'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:09.765249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9081d370-bae7-4e5f-930f-78e26c69892f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a81fdaea686041ac96619055a9371731'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:09.766169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9081d370-bae7-4e5f-930f-78e26c69892f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a81fdaea686041ac96619055a9371731'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:09.794910 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2943/5927] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:24:09 2022] DELETE /identity/v3/users/a81fdaea686041ac96619055a9371731 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:09.795352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692ebe7-055f-430b-8b81-a0cb59f2ad24 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:09.795816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692ebe7-055f-430b-8b81-a0cb59f2ad24 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:09.796050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692ebe7-055f-430b-8b81-a0cb59f2ad24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:09.846870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5692ebe7-055f-430b-8b81-a0cb59f2ad24 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:09.846870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5692ebe7-055f-430b-8b81-a0cb59f2ad24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:09.913286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:09.987175 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2985/5928] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:24:10.002846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081d370-bae7-4e5f-930f-78e26c69892f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:10.005800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9081d370-bae7-4e5f-930f-78e26c69892f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:10.200281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081d370-bae7-4e5f-930f-78e26c69892f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:10.215538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c1f22-b773-4261-9725-81b344dd25e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:10.218657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c1f22-b773-4261-9725-81b344dd25e5 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:10.224338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c1f22-b773-4261-9725-81b344dd25e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:10.224769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333c1f22-b773-4261-9725-81b344dd25e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:10.265991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c1f22-b773-4261-9725-81b344dd25e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:10.267873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333c1f22-b773-4261-9725-81b344dd25e5 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:10.303552 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2944/5929] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:24:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:24:11.355002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692ebe7-055f-430b-8b81-a0cb59f2ad24 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:11.355535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5692ebe7-055f-430b-8b81-a0cb59f2ad24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:11.370460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692ebe7-055f-430b-8b81-a0cb59f2ad24 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:11.383332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:11.383332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:11.383332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:11.383332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d admin admin] PATH_INFO: `/v3/projects/2169bc0792a74919bbe07258b5ae5cc4` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:11.383675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2169bc0792a74919bbe07258b5ae5cc4)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:11.383675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:11.513403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2169bc0792a74919bbe07258b5ae5cc4', 'request_id': 'req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:11.523988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2169bc0792a74919bbe07258b5ae5cc4', 'request_id': 'req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:11.572166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2169bc0792a74919bbe07258b5ae5cc4', 'request_id': 'req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:11.593296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d admin admin] The token cache is being invalidate because project 2169bc0792a74919bbe07258b5ae5cc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:24:11.593609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:11.598291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:11.600393 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2986/5930] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:24:11 2022] DELETE /identity/v3/projects/2169bc0792a74919bbe07258b5ae5cc4 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:11.605191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c1f22-b773-4261-9725-81b344dd25e5 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:11.621513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333c1f22-b773-4261-9725-81b344dd25e5 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:11.793256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8cbd5-528e-4bff-99f4-37624bf68245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:11.794832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8cbd5-528e-4bff-99f4-37624bf68245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:11.795293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8cbd5-528e-4bff-99f4-37624bf68245 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:11.796206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8cbd5-528e-4bff-99f4-37624bf68245 admin admin] PATH_INFO: `/v3/users/526ea594b62848cb81fe32c560ede0b7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:11.798238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8cbd5-528e-4bff-99f4-37624bf68245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=526ea594b62848cb81fe32c560ede0b7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:11.800798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8cbd5-528e-4bff-99f4-37624bf68245 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:11.955937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:12.106132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-42c8cbd5-528e-4bff-99f4-37624bf68245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '526ea594b62848cb81fe32c560ede0b7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:12.109488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-42c8cbd5-528e-4bff-99f4-37624bf68245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '526ea594b62848cb81fe32c560ede0b7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:12.130628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-42c8cbd5-528e-4bff-99f4-37624bf68245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '526ea594b62848cb81fe32c560ede0b7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:12.226478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-42c8cbd5-528e-4bff-99f4-37624bf68245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '526ea594b62848cb81fe32c560ede0b7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:12.227744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-42c8cbd5-528e-4bff-99f4-37624bf68245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '526ea594b62848cb81fe32c560ede0b7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:12.273873 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2945/5931] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:24:11 2022] DELETE /identity/v3/users/526ea594b62848cb81fe32c560ede0b7 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:13.003201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:13.004179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:13.025495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec4dfe-2125-4fa5-9059-f9f95ebfbd6d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:13.156245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc88520-9f23-4a29-98bb-b9623ffa3eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:13.158432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc88520-9f23-4a29-98bb-b9623ffa3eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:13.164424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc88520-9f23-4a29-98bb-b9623ffa3eec admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:13.164424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc88520-9f23-4a29-98bb-b9623ffa3eec admin admin] PATH_INFO: `/v3/projects/1f59f624c2e24eb291f0a741a8406864` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:13.164424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc88520-9f23-4a29-98bb-b9623ffa3eec admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f59f624c2e24eb291f0a741a8406864)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:13.165087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc88520-9f23-4a29-98bb-b9623ffa3eec admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:13.406476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5fc88520-9f23-4a29-98bb-b9623ffa3eec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f59f624c2e24eb291f0a741a8406864', 'request_id': 'req-5fc88520-9f23-4a29-98bb-b9623ffa3eec'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:13.427132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5fc88520-9f23-4a29-98bb-b9623ffa3eec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f59f624c2e24eb291f0a741a8406864', 'request_id': 'req-5fc88520-9f23-4a29-98bb-b9623ffa3eec'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:13.519702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5fc88520-9f23-4a29-98bb-b9623ffa3eec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f59f624c2e24eb291f0a741a8406864', 'request_id': 'req-5fc88520-9f23-4a29-98bb-b9623ffa3eec'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:13.521831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:13.543204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5fc88520-9f23-4a29-98bb-b9623ffa3eec admin admin] The token cache is being invalidate because project 1f59f624c2e24eb291f0a741a8406864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:24:13.543527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5fc88520-9f23-4a29-98bb-b9623ffa3eec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:13.544256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5fc88520-9f23-4a29-98bb-b9623ffa3eec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:13.550219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8cbd5-528e-4bff-99f4-37624bf68245 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:13.551708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-42c8cbd5-528e-4bff-99f4-37624bf68245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:13.555281 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2987/5932] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:24:13 2022] DELETE /identity/v3/projects/1f59f624c2e24eb291f0a741a8406864 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:13.574073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8cbd5-528e-4bff-99f4-37624bf68245 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:13.705014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd72eba-f75c-4671-b14a-6ec1ef093305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:13.706608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd72eba-f75c-4671-b14a-6ec1ef093305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:13.706608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd72eba-f75c-4671-b14a-6ec1ef093305 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:13.706791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd72eba-f75c-4671-b14a-6ec1ef093305 admin admin] PATH_INFO: `/v3/users/64f6b3652ce44ad091e59dc009ea83af` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:13.714961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd72eba-f75c-4671-b14a-6ec1ef093305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64f6b3652ce44ad091e59dc009ea83af)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:13.716974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd72eba-f75c-4671-b14a-6ec1ef093305 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:14.253507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2fd72eba-f75c-4671-b14a-6ec1ef093305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64f6b3652ce44ad091e59dc009ea83af'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:14.255968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2fd72eba-f75c-4671-b14a-6ec1ef093305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64f6b3652ce44ad091e59dc009ea83af'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:14.265625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2fd72eba-f75c-4671-b14a-6ec1ef093305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64f6b3652ce44ad091e59dc009ea83af'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:14.364705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2fd72eba-f75c-4671-b14a-6ec1ef093305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64f6b3652ce44ad091e59dc009ea83af'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:14.365715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2fd72eba-f75c-4671-b14a-6ec1ef093305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64f6b3652ce44ad091e59dc009ea83af'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:14.423727 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2946/5933] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:24:13 2022] DELETE /identity/v3/users/64f6b3652ce44ad091e59dc009ea83af => generated 0 bytes in 868 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:15.558129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc88520-9f23-4a29-98bb-b9623ffa3eec admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:15.560296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5fc88520-9f23-4a29-98bb-b9623ffa3eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:15.587328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc88520-9f23-4a29-98bb-b9623ffa3eec admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:15.812799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc727c05-4f62-4f80-bf8d-2b6563d957a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:15.814079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc727c05-4f62-4f80-bf8d-2b6563d957a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:15.814756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc727c05-4f62-4f80-bf8d-2b6563d957a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:15.814903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc727c05-4f62-4f80-bf8d-2b6563d957a4 admin admin] PATH_INFO: `/v3/projects/9f5f37eb21524c0db5d69a43e6623696` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:15.817179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc727c05-4f62-4f80-bf8d-2b6563d957a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f5f37eb21524c0db5d69a43e6623696)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:15.819292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc727c05-4f62-4f80-bf8d-2b6563d957a4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:16.011509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc727c05-4f62-4f80-bf8d-2b6563d957a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f5f37eb21524c0db5d69a43e6623696', 'request_id': 'req-cc727c05-4f62-4f80-bf8d-2b6563d957a4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:16.039214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc727c05-4f62-4f80-bf8d-2b6563d957a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f5f37eb21524c0db5d69a43e6623696', 'request_id': 'req-cc727c05-4f62-4f80-bf8d-2b6563d957a4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:16.125224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc727c05-4f62-4f80-bf8d-2b6563d957a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f5f37eb21524c0db5d69a43e6623696', 'request_id': 'req-cc727c05-4f62-4f80-bf8d-2b6563d957a4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:16.151211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc727c05-4f62-4f80-bf8d-2b6563d957a4 admin admin] The token cache is being invalidate because project 9f5f37eb21524c0db5d69a43e6623696 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:24:16.151513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc727c05-4f62-4f80-bf8d-2b6563d957a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:16.152744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cc727c05-4f62-4f80-bf8d-2b6563d957a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:16.159362 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2988/5934] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:24:15 2022] DELETE /identity/v3/projects/9f5f37eb21524c0db5d69a43e6623696 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:16.165381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32966f45-17cf-4598-8eba-efb908ed8b32 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:16.166304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32966f45-17cf-4598-8eba-efb908ed8b32 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:16.168349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32966f45-17cf-4598-8eba-efb908ed8b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:16.216677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-32966f45-17cf-4598-8eba-efb908ed8b32 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:16.218279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-32966f45-17cf-4598-8eba-efb908ed8b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:16.303729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:16.443415 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2947/5935] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:16.451046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfcfa4-c957-4485-bea4-2cf5f0292002 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:16.451334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfcfa4-c957-4485-bea4-2cf5f0292002 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:16.451927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dfcfa4-c957-4485-bea4-2cf5f0292002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:16.537484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-52dfcfa4-c957-4485-bea4-2cf5f0292002 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:16.539769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-52dfcfa4-c957-4485-bea4-2cf5f0292002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:16.559039 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2989/5936] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:16.560334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32966f45-17cf-4598-8eba-efb908ed8b32 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:16.577902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32966f45-17cf-4598-8eba-efb908ed8b32 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:16.602056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01212eb-01d9-4873-b1d5-c6f73bae5a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:16.602877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01212eb-01d9-4873-b1d5-c6f73bae5a34 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:16.603000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01212eb-01d9-4873-b1d5-c6f73bae5a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:16.603285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01212eb-01d9-4873-b1d5-c6f73bae5a34 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:16.604624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01212eb-01d9-4873-b1d5-c6f73bae5a34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:24:16.604918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01212eb-01d9-4873-b1d5-c6f73bae5a34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:16.614128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01212eb-01d9-4873-b1d5-c6f73bae5a34 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:16.654471 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2948/5937] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:24:16.654700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfcfa4-c957-4485-bea4-2cf5f0292002 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:16.673299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dfcfa4-c957-4485-bea4-2cf5f0292002 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:16.693290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acca96-fc4a-4f7e-afe1-35a34a666f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:16.694022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acca96-fc4a-4f7e-afe1-35a34a666f9c admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:16.694377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acca96-fc4a-4f7e-afe1-35a34a666f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:16.694660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acca96-fc4a-4f7e-afe1-35a34a666f9c admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:16.695368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acca96-fc4a-4f7e-afe1-35a34a666f9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:16.703488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acca96-fc4a-4f7e-afe1-35a34a666f9c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:16.744412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e6b09-e349-427c-8ff1-2bfe3de95639 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:16.744979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e6b09-e349-427c-8ff1-2bfe3de95639 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:16.745452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e6b09-e349-427c-8ff1-2bfe3de95639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:16.782775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c13e6b09-e349-427c-8ff1-2bfe3de95639 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:16.789477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c13e6b09-e349-427c-8ff1-2bfe3de95639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:16.834031 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2990/5938] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:24:16 2022] POST /identity/v3/projects => generated 367 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:24:16.852285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acca96-fc4a-4f7e-afe1-35a34a666f9c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:17.019758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:17.020559 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2949/5939] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:24:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:24:17.038777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acca96-fc4a-4f7e-afe1-35a34a666f9c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:17.048094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e6b09-e349-427c-8ff1-2bfe3de95639 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:17.051131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c13e6b09-e349-427c-8ff1-2bfe3de95639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:17.063621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ab94e-1119-40ec-99ee-be27992778e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:17.063621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ab94e-1119-40ec-99ee-be27992778e4 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:17.063621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ab94e-1119-40ec-99ee-be27992778e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:17.063621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ab94e-1119-40ec-99ee-be27992778e4 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:17.064060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ab94e-1119-40ec-99ee-be27992778e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:17.064060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ab94e-1119-40ec-99ee-be27992778e4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:17.186511 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2991/5940] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:24:16 2022] POST /identity/v3/users => generated 284 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:24:17.191158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ab94e-1119-40ec-99ee-be27992778e4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:17.209208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ab94e-1119-40ec-99ee-be27992778e4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:17.220811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3230019-333b-45b6-8f12-1f82abe85c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:17.225995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3230019-333b-45b6-8f12-1f82abe85c1b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:17.225995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3230019-333b-45b6-8f12-1f82abe85c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:17.225995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3230019-333b-45b6-8f12-1f82abe85c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:17.226684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3230019-333b-45b6-8f12-1f82abe85c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:17.228352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3230019-333b-45b6-8f12-1f82abe85c1b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:17.252368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13e6b09-e349-427c-8ff1-2bfe3de95639 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:17.258331 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2992/5941] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:24:17.259410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3230019-333b-45b6-8f12-1f82abe85c1b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:17.285114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3230019-333b-45b6-8f12-1f82abe85c1b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:17.294079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ad4df-8a8d-4be2-936e-f3aa6b3853ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:17.294882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ad4df-8a8d-4be2-936e-f3aa6b3853ae admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:17.295171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ad4df-8a8d-4be2-936e-f3aa6b3853ae admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:17.295452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80ad4df-8a8d-4be2-936e-f3aa6b3853ae admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:17.312102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ad4df-8a8d-4be2-936e-f3aa6b3853ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:17.316119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80ad4df-8a8d-4be2-936e-f3aa6b3853ae admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:17.337286 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2993/5942] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:24:17.341275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ad4df-8a8d-4be2-936e-f3aa6b3853ae admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:17.360922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80ad4df-8a8d-4be2-936e-f3aa6b3853ae admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:17.369293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cfb76-321e-4168-b879-10d9635acb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:17.371308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cfb76-321e-4168-b879-10d9635acb51 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:17.372662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cfb76-321e-4168-b879-10d9635acb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:17.373125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cfb76-321e-4168-b879-10d9635acb51 admin admin] PATH_INFO: `/v3/projects/0fe7e4f1396c4cf7be16906e19cb9a19/users/53840cc1ce544712a9954e3d22d98cfe/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:17.383854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed833a-0893-4783-9734-d889e8bea0a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:17.392353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed833a-0893-4783-9734-d889e8bea0a4 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:17.392744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed833a-0893-4783-9734-d889e8bea0a4 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:17.393105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ed833a-0893-4783-9734-d889e8bea0a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:17.425223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cfb76-321e-4168-b879-10d9635acb51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe7e4f1396c4cf7be16906e19cb9a19, user_id=53840cc1ce544712a9954e3d22d98cfe, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:17.430029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cfb76-321e-4168-b879-10d9635acb51 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:17.504280 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2994/5943] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:24:17 2022] PUT /identity/v3/projects/0fe7e4f1396c4cf7be16906e19cb9a19/users/53840cc1ce544712a9954e3d22d98cfe/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:17.516238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b022f-f984-43b6-820c-7b95dc6e5cce None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:17.516751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b022f-f984-43b6-820c-7b95dc6e5cce None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:17.517213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b022f-f984-43b6-820c-7b95dc6e5cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:17.558902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed833a-0893-4783-9734-d889e8bea0a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:17.562841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ed833a-0893-4783-9734-d889e8bea0a4 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:17.579788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-624b022f-f984-43b6-820c-7b95dc6e5cce None None] MFA Rules not processed for user `53840cc1ce544712a9954e3d22d98cfe`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:17.580998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-624b022f-f984-43b6-820c-7b95dc6e5cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:17.584983 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2950/5944] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:24:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 21:24:17.716841 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2995/5945] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:17 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:17.721514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed833a-0893-4783-9734-d889e8bea0a4 service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:17.766177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ed833a-0893-4783-9734-d889e8bea0a4 service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:17.849552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:17.980037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65224658-5614-49a7-9578-7d9a63923f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:17.980883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65224658-5614-49a7-9578-7d9a63923f33 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:17.981173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65224658-5614-49a7-9578-7d9a63923f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:17.981450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65224658-5614-49a7-9578-7d9a63923f33 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:17.982368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65224658-5614-49a7-9578-7d9a63923f33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:17.989460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65224658-5614-49a7-9578-7d9a63923f33 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:18.106441 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2951/5946] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:24:17 2022] POST /identity/v3/projects => generated 361 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:24:18.110352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b022f-f984-43b6-820c-7b95dc6e5cce None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:18.130177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b022f-f984-43b6-820c-7b95dc6e5cce None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:18.418649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb345b0-a2ca-41de-b26c-7cf6a4b0e0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:18.419354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb345b0-a2ca-41de-b26c-7cf6a4b0e0fd admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:18.419638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb345b0-a2ca-41de-b26c-7cf6a4b0e0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:18.419930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb345b0-a2ca-41de-b26c-7cf6a4b0e0fd admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:18.426865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb345b0-a2ca-41de-b26c-7cf6a4b0e0fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:18.431628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb345b0-a2ca-41de-b26c-7cf6a4b0e0fd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:18.562101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65224658-5614-49a7-9578-7d9a63923f33 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:18.562651 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2996/5947] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:24:18 2022] POST /identity/v3/users => generated 287 bytes in 451 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:24:18.598640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65224658-5614-49a7-9578-7d9a63923f33 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:18.619934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0ca4e-f699-425b-bf6b-7ea48cb33453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:18.621687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0ca4e-f699-425b-bf6b-7ea48cb33453 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:18.623460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0ca4e-f699-425b-bf6b-7ea48cb33453 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:18.623989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0ca4e-f699-425b-bf6b-7ea48cb33453 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:18.627053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd0ca4e-f699-425b-bf6b-7ea48cb33453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:18.636623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd0ca4e-f699-425b-bf6b-7ea48cb33453 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:18.685412 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2952/5948] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:24:18.686874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb345b0-a2ca-41de-b26c-7cf6a4b0e0fd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:18.714217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb345b0-a2ca-41de-b26c-7cf6a4b0e0fd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:18.733431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab0658-2691-471d-b1e3-1aaaf92e52f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:18.739153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab0658-2691-471d-b1e3-1aaaf92e52f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:18.739485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab0658-2691-471d-b1e3-1aaaf92e52f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:18.741396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab0658-2691-471d-b1e3-1aaaf92e52f9 admin admin] PATH_INFO: `/v3/projects/6576d2aaba2a4508bef3c6f6e298c3a9/users/6524b3ba52d04e6a9bd60cc86ab8129e/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:18.778497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0ca4e-f699-425b-bf6b-7ea48cb33453 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:18.780504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-dcd0ca4e-f699-425b-bf6b-7ea48cb33453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:18.804296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab0658-2691-471d-b1e3-1aaaf92e52f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6576d2aaba2a4508bef3c6f6e298c3a9, user_id=6524b3ba52d04e6a9bd60cc86ab8129e, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:18.808731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd0ca4e-f699-425b-bf6b-7ea48cb33453 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:18.818014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab0658-2691-471d-b1e3-1aaaf92e52f9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:18.823209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aed8694-25a1-4b96-ae6b-001ab372d042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:18.823955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed8694-25a1-4b96-ae6b-001ab372d042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:18.824554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed8694-25a1-4b96-ae6b-001ab372d042 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:18.824554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed8694-25a1-4b96-ae6b-001ab372d042 admin admin] PATH_INFO: `/v3/users/8149349295714b1791af1b88fb565f74` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:18.826144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:18.827377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed8694-25a1-4b96-ae6b-001ab372d042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8149349295714b1791af1b88fb565f74)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:18.831231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aed8694-25a1-4b96-ae6b-001ab372d042 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:18.924594 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2997/5949] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:24:18 2022] PUT /identity/v3/projects/6576d2aaba2a4508bef3c6f6e298c3a9/users/6524b3ba52d04e6a9bd60cc86ab8129e/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:18.928965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab0658-2691-471d-b1e3-1aaaf92e52f9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:19.109861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1aed8694-25a1-4b96-ae6b-001ab372d042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8149349295714b1791af1b88fb565f74'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:19.110984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1aed8694-25a1-4b96-ae6b-001ab372d042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8149349295714b1791af1b88fb565f74'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:19.129553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1aed8694-25a1-4b96-ae6b-001ab372d042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8149349295714b1791af1b88fb565f74'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:19.168806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab0658-2691-471d-b1e3-1aaaf92e52f9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:19.186800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec62cf-c23e-4f02-b8a4-0b3722a39bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:19.187776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec62cf-c23e-4f02-b8a4-0b3722a39bac admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:19.188131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec62cf-c23e-4f02-b8a4-0b3722a39bac admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:19.188506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec62cf-c23e-4f02-b8a4-0b3722a39bac admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:19.194655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec62cf-c23e-4f02-b8a4-0b3722a39bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:19.194655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec62cf-c23e-4f02-b8a4-0b3722a39bac admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:19.218366 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2998/5950] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:24:19.224742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec62cf-c23e-4f02-b8a4-0b3722a39bac admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:19.245309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec62cf-c23e-4f02-b8a4-0b3722a39bac admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:19.251027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1aed8694-25a1-4b96-ae6b-001ab372d042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8149349295714b1791af1b88fb565f74'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:19.252070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1aed8694-25a1-4b96-ae6b-001ab372d042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8149349295714b1791af1b88fb565f74'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:19.267847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:19.273504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672605ff-076e-4c43-bd1e-4f8c4c8e94c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:19.276542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672605ff-076e-4c43-bd1e-4f8c4c8e94c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:19.280401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672605ff-076e-4c43-bd1e-4f8c4c8e94c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:19.280401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672605ff-076e-4c43-bd1e-4f8c4c8e94c3 admin admin] PATH_INFO: `/v3/projects/6576d2aaba2a4508bef3c6f6e298c3a9/users/6524b3ba52d04e6a9bd60cc86ab8129e/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:19.287793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672605ff-076e-4c43-bd1e-4f8c4c8e94c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6576d2aaba2a4508bef3c6f6e298c3a9, user_id=6524b3ba52d04e6a9bd60cc86ab8129e, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:19.292773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672605ff-076e-4c43-bd1e-4f8c4c8e94c3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:19.313100 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2953/5951] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:24:18 2022] DELETE /identity/v3/users/8149349295714b1791af1b88fb565f74 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:24:19.329818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c7670-0ace-4d2d-88b6-8013f729014a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:19.330347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c7670-0ace-4d2d-88b6-8013f729014a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:19.331918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4c7670-0ace-4d2d-88b6-8013f729014a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:19.352163 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 2999/5952] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:24:19 2022] PUT /identity/v3/projects/6576d2aaba2a4508bef3c6f6e298c3a9/users/6524b3ba52d04e6a9bd60cc86ab8129e/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:24:19.368811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a79bf-c4c0-4649-96ef-c876eb9351df None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:19.371863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a79bf-c4c0-4649-96ef-c876eb9351df None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:19.372611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a79bf-c4c0-4649-96ef-c876eb9351df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:19.426346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3b4c7670-0ace-4d2d-88b6-8013f729014a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:19.448850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3b4c7670-0ace-4d2d-88b6-8013f729014a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:19.477326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-070a79bf-c4c0-4649-96ef-c876eb9351df None None] MFA Rules not processed for user `6524b3ba52d04e6a9bd60cc86ab8129e`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:19.479675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-070a79bf-c4c0-4649-96ef-c876eb9351df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:19.598151 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2954/5953] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:19.608660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c7670-0ace-4d2d-88b6-8013f729014a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:19.610987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3b4c7670-0ace-4d2d-88b6-8013f729014a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:19.691017 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3000/5954] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:19 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:19.700344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a79bf-c4c0-4649-96ef-c876eb9351df None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:19.701933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-070a79bf-c4c0-4649-96ef-c876eb9351df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:19.853888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a79bf-c4c0-4649-96ef-c876eb9351df None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:19.869010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da6f5b-a8d0-496a-8438-6c9b5d84be3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:19.870344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da6f5b-a8d0-496a-8438-6c9b5d84be3c service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:19.870805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da6f5b-a8d0-496a-8438-6c9b5d84be3c service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:19.871092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da6f5b-a8d0-496a-8438-6c9b5d84be3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:19.911900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da6f5b-a8d0-496a-8438-6c9b5d84be3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:19.913425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da6f5b-a8d0-496a-8438-6c9b5d84be3c service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:19.957541 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3001/5955] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:24:19 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:24:19.988988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4c7670-0ace-4d2d-88b6-8013f729014a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:19.998173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123fa0b-e57a-4bef-87ae-ca47a490fc2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:19.998903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123fa0b-e57a-4bef-87ae-ca47a490fc2e service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:19.999194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123fa0b-e57a-4bef-87ae-ca47a490fc2e service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:19.999471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123fa0b-e57a-4bef-87ae-ca47a490fc2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:20.033578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123fa0b-e57a-4bef-87ae-ca47a490fc2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:20.044972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123fa0b-e57a-4bef-87ae-ca47a490fc2e service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:20.071354 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2955/5956] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:24:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:24:20.268805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da6f5b-a8d0-496a-8438-6c9b5d84be3c service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:20.277226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:20.296203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da6f5b-a8d0-496a-8438-6c9b5d84be3c service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:20.312443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af4afc-9be9-4676-9ad0-363f68275b4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:20.314759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af4afc-9be9-4676-9ad0-363f68275b4f service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:20.315222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af4afc-9be9-4676-9ad0-363f68275b4f service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:20.319442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af4afc-9be9-4676-9ad0-363f68275b4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:20.323853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e9af4afc-9be9-4676-9ad0-363f68275b4f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:20.565685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af4afc-9be9-4676-9ad0-363f68275b4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:20.565685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af4afc-9be9-4676-9ad0-363f68275b4f service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:20.608592 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3002/5957] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:24:20 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:24:21.934405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123fa0b-e57a-4bef-87ae-ca47a490fc2e service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:21.936612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2123fa0b-e57a-4bef-87ae-ca47a490fc2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:21.956975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123fa0b-e57a-4bef-87ae-ca47a490fc2e service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:21.973068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5509b-4cd4-4717-be06-155767770ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:21.974627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5509b-4cd4-4717-be06-155767770ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:21.975120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5509b-4cd4-4717-be06-155767770ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:21.975591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5509b-4cd4-4717-be06-155767770ed3 admin admin] PATH_INFO: `/v3/projects/4c998d22b841432b828345edcf54af63` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:21.977046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5509b-4cd4-4717-be06-155767770ed3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c998d22b841432b828345edcf54af63)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:21.984038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5509b-4cd4-4717-be06-155767770ed3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:22.227725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3ae5509b-4cd4-4717-be06-155767770ed3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c998d22b841432b828345edcf54af63', 'request_id': 'req-3ae5509b-4cd4-4717-be06-155767770ed3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:22.258181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3ae5509b-4cd4-4717-be06-155767770ed3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c998d22b841432b828345edcf54af63', 'request_id': 'req-3ae5509b-4cd4-4717-be06-155767770ed3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:22.354011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3ae5509b-4cd4-4717-be06-155767770ed3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c998d22b841432b828345edcf54af63', 'request_id': 'req-3ae5509b-4cd4-4717-be06-155767770ed3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:22.415309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3ae5509b-4cd4-4717-be06-155767770ed3 admin admin] The token cache is being invalidate because project 4c998d22b841432b828345edcf54af63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:24:22.417355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3ae5509b-4cd4-4717-be06-155767770ed3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:22.419592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3ae5509b-4cd4-4717-be06-155767770ed3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:22.429101 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2956/5958] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:24:21 2022] DELETE /identity/v3/projects/4c998d22b841432b828345edcf54af63 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:22.434299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007dd1-ee6e-4fa7-b791-71d4f0e9c9e4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:22.435334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007dd1-ee6e-4fa7-b791-71d4f0e9c9e4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:22.436022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007dd1-ee6e-4fa7-b791-71d4f0e9c9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:22.487110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b6007dd1-ee6e-4fa7-b791-71d4f0e9c9e4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:22.495830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b6007dd1-ee6e-4fa7-b791-71d4f0e9c9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:22.688455 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3003/5959] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:22.695912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbc373-1eed-4df4-a9f6-b552fb796bb7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:22.696583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbc373-1eed-4df4-a9f6-b552fb796bb7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:22.697027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbc373-1eed-4df4-a9f6-b552fb796bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:22.787032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4efbc373-1eed-4df4-a9f6-b552fb796bb7 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:22.788651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4efbc373-1eed-4df4-a9f6-b552fb796bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:22.838183 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2957/5960] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:22.843234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6007dd1-ee6e-4fa7-b791-71d4f0e9c9e4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:22.866054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6007dd1-ee6e-4fa7-b791-71d4f0e9c9e4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:22.881315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5080aaa-ad1e-4a91-b3f3-247e9fe7bbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:22.881964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080aaa-ad1e-4a91-b3f3-247e9fe7bbbd admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:22.886195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080aaa-ad1e-4a91-b3f3-247e9fe7bbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:22.886502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5080aaa-ad1e-4a91-b3f3-247e9fe7bbbd admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:22.887297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5080aaa-ad1e-4a91-b3f3-247e9fe7bbbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:24:22.887621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5080aaa-ad1e-4a91-b3f3-247e9fe7bbbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:22.889270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5080aaa-ad1e-4a91-b3f3-247e9fe7bbbd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:22.919396 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3004/5961] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:24:22.924004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbc373-1eed-4df4-a9f6-b552fb796bb7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:22.935454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:22.945594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbc373-1eed-4df4-a9f6-b552fb796bb7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:22.959648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da5d92-139d-4e54-876a-ff086159cd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:22.960666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da5d92-139d-4e54-876a-ff086159cd9f admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:22.962242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da5d92-139d-4e54-876a-ff086159cd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:22.964926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da5d92-139d-4e54-876a-ff086159cd9f admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:22.965946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da5d92-139d-4e54-876a-ff086159cd9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:22.969293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da5d92-139d-4e54-876a-ff086159cd9f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:23.187030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5080aaa-ad1e-4a91-b3f3-247e9fe7bbbd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:23.195120 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2958/5962] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:24:22 2022] POST /identity/v3/projects => generated 381 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:24:23.200461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5080aaa-ad1e-4a91-b3f3-247e9fe7bbbd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:23.291044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e644ad6-62d8-4599-b5a1-27cd0701740b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:23.292425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e644ad6-62d8-4599-b5a1-27cd0701740b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:23.292722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e644ad6-62d8-4599-b5a1-27cd0701740b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:23.293009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e644ad6-62d8-4599-b5a1-27cd0701740b admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:23.293724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e644ad6-62d8-4599-b5a1-27cd0701740b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:23.295951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e644ad6-62d8-4599-b5a1-27cd0701740b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:23.459591 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3005/5963] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:24:23 2022] POST /identity/v3/users => generated 291 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:24:23.460311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da5d92-139d-4e54-876a-ff086159cd9f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:23.473187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da5d92-139d-4e54-876a-ff086159cd9f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:23.480368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fc73c-9fe1-4d9c-abcb-909b39c4d6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:23.481063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fc73c-9fe1-4d9c-abcb-909b39c4d6ec admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:23.481355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fc73c-9fe1-4d9c-abcb-909b39c4d6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:23.481698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9fc73c-9fe1-4d9c-abcb-909b39c4d6ec admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:23.482493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9fc73c-9fe1-4d9c-abcb-909b39c4d6ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:23.484160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9fc73c-9fe1-4d9c-abcb-909b39c4d6ec admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:23.494959 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2959/5964] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:24:23.499014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e644ad6-62d8-4599-b5a1-27cd0701740b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:23.510762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e644ad6-62d8-4599-b5a1-27cd0701740b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:23.515997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70323016-59d5-4f14-a119-cc101ada031b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:23.516755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70323016-59d5-4f14-a119-cc101ada031b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:23.521073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70323016-59d5-4f14-a119-cc101ada031b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:23.521426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70323016-59d5-4f14-a119-cc101ada031b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:23.534061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70323016-59d5-4f14-a119-cc101ada031b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:23.534061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70323016-59d5-4f14-a119-cc101ada031b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:23.540167 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3006/5965] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:24:23.541146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fc73c-9fe1-4d9c-abcb-909b39c4d6ec admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:23.565447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9fc73c-9fe1-4d9c-abcb-909b39c4d6ec admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:23.572128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eca0b2-1495-457b-bdd6-afc06d646bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:23.573025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eca0b2-1495-457b-bdd6-afc06d646bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:23.573381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eca0b2-1495-457b-bdd6-afc06d646bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:23.573662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eca0b2-1495-457b-bdd6-afc06d646bcf admin admin] PATH_INFO: `/v3/projects/01fa7893b58741ce84a316632efe7882/users/ebc8e00965334bd1bb811a0eefd64b84/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:23.591795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eca0b2-1495-457b-bdd6-afc06d646bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=01fa7893b58741ce84a316632efe7882, user_id=ebc8e00965334bd1bb811a0eefd64b84, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:23.594816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eca0b2-1495-457b-bdd6-afc06d646bcf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:23.884760 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2960/5966] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:24:23 2022] PUT /identity/v3/projects/01fa7893b58741ce84a316632efe7882/users/ebc8e00965334bd1bb811a0eefd64b84/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:23.886843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24038fe-b75e-4aae-b346-3a087cb03a6b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:23.887266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24038fe-b75e-4aae-b346-3a087cb03a6b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:23.887612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24038fe-b75e-4aae-b346-3a087cb03a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:23.912991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e24038fe-b75e-4aae-b346-3a087cb03a6b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:23.914315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e24038fe-b75e-4aae-b346-3a087cb03a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:23.926100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:23.989555 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3007/5967] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:23.999705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eca0b2-1495-457b-bdd6-afc06d646bcf admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:24.001412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-33eca0b2-1495-457b-bdd6-afc06d646bcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:24.067341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eca0b2-1495-457b-bdd6-afc06d646bcf admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:24.071182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df66ded5-9e04-4e33-808c-ccaf05b0d440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:24.071862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66ded5-9e04-4e33-808c-ccaf05b0d440 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:24.072149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66ded5-9e04-4e33-808c-ccaf05b0d440 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:24.072571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df66ded5-9e04-4e33-808c-ccaf05b0d440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:24.083058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df66ded5-9e04-4e33-808c-ccaf05b0d440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:24.084581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df66ded5-9e04-4e33-808c-ccaf05b0d440 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:24.095055 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2961/5968] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:24:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:24:29.986980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24038fe-b75e-4aae-b346-3a087cb03a6b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:29.989170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e24038fe-b75e-4aae-b346-3a087cb03a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:30.185201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24038fe-b75e-4aae-b346-3a087cb03a6b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:30.203707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7d352-a0b8-46b8-b995-3933b7734691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:30.204679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7d352-a0b8-46b8-b995-3933b7734691 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:30.204975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7d352-a0b8-46b8-b995-3933b7734691 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:30.205256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7d352-a0b8-46b8-b995-3933b7734691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:30.211795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-48b7d352-a0b8-46b8-b995-3933b7734691 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:30.385606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7d352-a0b8-46b8-b995-3933b7734691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:30.390229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7d352-a0b8-46b8-b995-3933b7734691 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:30.424767 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3008/5969] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:24:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:24:30.608154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d936fe-0085-47a6-82cd-06d722699ff8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:30.609235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d936fe-0085-47a6-82cd-06d722699ff8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:30.609694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d936fe-0085-47a6-82cd-06d722699ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:30.679534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a1d936fe-0085-47a6-82cd-06d722699ff8 None None] MFA Rules not processed for user `ebc8e00965334bd1bb811a0eefd64b84`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:30.681342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a1d936fe-0085-47a6-82cd-06d722699ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:30.922059 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2962/5970] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:30.951876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fccd66-4020-42e3-b45e-dc139d90f8db None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:30.952175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fccd66-4020-42e3-b45e-dc139d90f8db None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:30.952470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fccd66-4020-42e3-b45e-dc139d90f8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:30.982358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b8fccd66-4020-42e3-b45e-dc139d90f8db None None] MFA Rules not processed for user `ebc8e00965334bd1bb811a0eefd64b84`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:30.983832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b8fccd66-4020-42e3-b45e-dc139d90f8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:31.003808 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3009/5971] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:31.694220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d936fe-0085-47a6-82cd-06d722699ff8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:31.716844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d936fe-0085-47a6-82cd-06d722699ff8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:31.723765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f295b-8ef7-4899-bafe-cb7aec580258 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:31.724690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f295b-8ef7-4899-bafe-cb7aec580258 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:31.725052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f295b-8ef7-4899-bafe-cb7aec580258 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:31.725405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f295b-8ef7-4899-bafe-cb7aec580258 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:31.766670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f295b-8ef7-4899-bafe-cb7aec580258 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:31.768382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f295b-8ef7-4899-bafe-cb7aec580258 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:31.797053 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2963/5972] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:24:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:24:34.144811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fccd66-4020-42e3-b45e-dc139d90f8db None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:34.146460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b8fccd66-4020-42e3-b45e-dc139d90f8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:34.173823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fccd66-4020-42e3-b45e-dc139d90f8db None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:34.182992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915c3ad-fa44-4079-90e3-07085ac1fc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:34.185986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915c3ad-fa44-4079-90e3-07085ac1fc39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:34.185986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915c3ad-fa44-4079-90e3-07085ac1fc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:34.186356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e915c3ad-fa44-4079-90e3-07085ac1fc39 admin admin] PATH_INFO: `/v3/users/f7c1a400e0dd462da3bb33b7e3fe57fc` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:34.188968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e915c3ad-fa44-4079-90e3-07085ac1fc39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7c1a400e0dd462da3bb33b7e3fe57fc)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:34.192009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e915c3ad-fa44-4079-90e3-07085ac1fc39 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:34.508015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:34.544791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e915c3ad-fa44-4079-90e3-07085ac1fc39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7c1a400e0dd462da3bb33b7e3fe57fc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:34.567336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e915c3ad-fa44-4079-90e3-07085ac1fc39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7c1a400e0dd462da3bb33b7e3fe57fc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:34.575866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e915c3ad-fa44-4079-90e3-07085ac1fc39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7c1a400e0dd462da3bb33b7e3fe57fc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:34.575866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e915c3ad-fa44-4079-90e3-07085ac1fc39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7c1a400e0dd462da3bb33b7e3fe57fc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:34.706185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e915c3ad-fa44-4079-90e3-07085ac1fc39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7c1a400e0dd462da3bb33b7e3fe57fc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:34.743038 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3010/5973] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:24:34 2022] DELETE /identity/v3/users/f7c1a400e0dd462da3bb33b7e3fe57fc => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:35.191786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f295b-8ef7-4899-bafe-cb7aec580258 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:35.206507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f295b-8ef7-4899-bafe-cb7aec580258 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:35.251938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:35.326955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e612ace-37cd-40b3-91a7-c545ebbbdeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:35.326955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e612ace-37cd-40b3-91a7-c545ebbbdeb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:35.326955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e612ace-37cd-40b3-91a7-c545ebbbdeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:35.326955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e612ace-37cd-40b3-91a7-c545ebbbdeb2 admin admin] PATH_INFO: `/v3/projects/d3112549c42f41cda4a54ec51c131023` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:35.330346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e612ace-37cd-40b3-91a7-c545ebbbdeb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3112549c42f41cda4a54ec51c131023)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:35.333370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e612ace-37cd-40b3-91a7-c545ebbbdeb2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:35.600043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1e612ace-37cd-40b3-91a7-c545ebbbdeb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3112549c42f41cda4a54ec51c131023', 'request_id': 'req-1e612ace-37cd-40b3-91a7-c545ebbbdeb2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:35.621129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1e612ace-37cd-40b3-91a7-c545ebbbdeb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3112549c42f41cda4a54ec51c131023', 'request_id': 'req-1e612ace-37cd-40b3-91a7-c545ebbbdeb2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:35.760937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1e612ace-37cd-40b3-91a7-c545ebbbdeb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3112549c42f41cda4a54ec51c131023', 'request_id': 'req-1e612ace-37cd-40b3-91a7-c545ebbbdeb2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:35.773415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915c3ad-fa44-4079-90e3-07085ac1fc39 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:35.775737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e915c3ad-fa44-4079-90e3-07085ac1fc39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:35.784097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1e612ace-37cd-40b3-91a7-c545ebbbdeb2 admin admin] The token cache is being invalidate because project d3112549c42f41cda4a54ec51c131023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:24:35.790316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e915c3ad-fa44-4079-90e3-07085ac1fc39 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:35.798512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1e612ace-37cd-40b3-91a7-c545ebbbdeb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:35.800575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1e612ace-37cd-40b3-91a7-c545ebbbdeb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:35.806180 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2964/5974] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:24:35 2022] DELETE /identity/v3/projects/d3112549c42f41cda4a54ec51c131023 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:35.812392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd935e2d-52d1-4767-9766-9c720a9082f6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:35.812918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd935e2d-52d1-4767-9766-9c720a9082f6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:35.813701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd935e2d-52d1-4767-9766-9c720a9082f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:35.843293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-cd935e2d-52d1-4767-9766-9c720a9082f6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:35.845797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cd935e2d-52d1-4767-9766-9c720a9082f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:35.874251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0274e2e-b296-43eb-b74c-98689d060585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:35.875666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0274e2e-b296-43eb-b74c-98689d060585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:35.876104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0274e2e-b296-43eb-b74c-98689d060585 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:35.876631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0274e2e-b296-43eb-b74c-98689d060585 admin admin] PATH_INFO: `/v3/users/53840cc1ce544712a9954e3d22d98cfe` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:35.878213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0274e2e-b296-43eb-b74c-98689d060585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53840cc1ce544712a9954e3d22d98cfe)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:35.880462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0274e2e-b296-43eb-b74c-98689d060585 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:35.950397 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2965/5975] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:24:36.021863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af644209-9400-4b51-89b6-031d1c7489c7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:36.022354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af644209-9400-4b51-89b6-031d1c7489c7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:36.022737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af644209-9400-4b51-89b6-031d1c7489c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:36.080881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-af644209-9400-4b51-89b6-031d1c7489c7 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:36.082418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-af644209-9400-4b51-89b6-031d1c7489c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:36.098964 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2966/5976] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:24:36.101423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af644209-9400-4b51-89b6-031d1c7489c7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:36.120782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af644209-9400-4b51-89b6-031d1c7489c7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:36.127282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3b8bc-83b6-4e71-aabc-22734b586921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:36.128066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3b8bc-83b6-4e71-aabc-22734b586921 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:36.128419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3b8bc-83b6-4e71-aabc-22734b586921 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:36.128779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3b8bc-83b6-4e71-aabc-22734b586921 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:36.129630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d3b8bc-83b6-4e71-aabc-22734b586921 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:24:36.130014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d3b8bc-83b6-4e71-aabc-22734b586921 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:36.131896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d3b8bc-83b6-4e71-aabc-22734b586921 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:36.149619 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2967/5977] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:24:36.151496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3b8bc-83b6-4e71-aabc-22734b586921 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:36.163869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3b8bc-83b6-4e71-aabc-22734b586921 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:36.169942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154f40a-8cea-498c-b2ae-050889b1c004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:36.170812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154f40a-8cea-498c-b2ae-050889b1c004 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:36.171358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154f40a-8cea-498c-b2ae-050889b1c004 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:36.171863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6154f40a-8cea-498c-b2ae-050889b1c004 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:36.172668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154f40a-8cea-498c-b2ae-050889b1c004 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:36.175381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6154f40a-8cea-498c-b2ae-050889b1c004 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:36.212694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d0274e2e-b296-43eb-b74c-98689d060585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53840cc1ce544712a9954e3d22d98cfe'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:36.223763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d0274e2e-b296-43eb-b74c-98689d060585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53840cc1ce544712a9954e3d22d98cfe'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:36.224704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d0274e2e-b296-43eb-b74c-98689d060585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53840cc1ce544712a9954e3d22d98cfe'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:36.225470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d0274e2e-b296-43eb-b74c-98689d060585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53840cc1ce544712a9954e3d22d98cfe'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:36.326215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d0274e2e-b296-43eb-b74c-98689d060585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53840cc1ce544712a9954e3d22d98cfe'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:36.336330 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2968/5978] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:24:36 2022] POST /identity/v3/projects => generated 357 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:24:36.339948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154f40a-8cea-498c-b2ae-050889b1c004 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:36.343359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6154f40a-8cea-498c-b2ae-050889b1c004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:36.364978 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3011/5979] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:24:35 2022] DELETE /identity/v3/users/53840cc1ce544712a9954e3d22d98cfe => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:36.376426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe02cd4-adc3-4d9f-9bc0-f283a21bfd6a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:36.376426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe02cd4-adc3-4d9f-9bc0-f283a21bfd6a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:36.376426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe02cd4-adc3-4d9f-9bc0-f283a21bfd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:36.411095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0fe02cd4-adc3-4d9f-9bc0-f283a21bfd6a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:36.411095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0fe02cd4-adc3-4d9f-9bc0-f283a21bfd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:36.464817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:36.567995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6154f40a-8cea-498c-b2ae-050889b1c004 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:36.597521 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3012/5980] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:36.612286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe02cd4-adc3-4d9f-9bc0-f283a21bfd6a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:36.614221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0fe02cd4-adc3-4d9f-9bc0-f283a21bfd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:36.674072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f3afd-5b4e-4f06-a631-4a536c54649d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:36.674783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f3afd-5b4e-4f06-a631-4a536c54649d admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:36.675070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f3afd-5b4e-4f06-a631-4a536c54649d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:36.675360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f3afd-5b4e-4f06-a631-4a536c54649d admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:36.676069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f3afd-5b4e-4f06-a631-4a536c54649d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:36.678350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f3afd-5b4e-4f06-a631-4a536c54649d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:36.760486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe02cd4-adc3-4d9f-9bc0-f283a21bfd6a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:36.785213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1740ee7-8f7e-42a7-893d-c7d9c8200167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:36.785213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1740ee7-8f7e-42a7-893d-c7d9c8200167 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:36.785213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1740ee7-8f7e-42a7-893d-c7d9c8200167 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:36.785213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1740ee7-8f7e-42a7-893d-c7d9c8200167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:36.786474 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2969/5981] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:24:36 2022] POST /identity/v3/users => generated 279 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:24:36.799636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f3afd-5b4e-4f06-a631-4a536c54649d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:36.801685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1740ee7-8f7e-42a7-893d-c7d9c8200167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:36.803176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1740ee7-8f7e-42a7-893d-c7d9c8200167 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:36.814574 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3013/5982] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:24:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:24:36.846834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f3afd-5b4e-4f06-a631-4a536c54649d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:36.862611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdb93b-57b2-4fa8-82f0-d1155e45244e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:36.863321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdb93b-57b2-4fa8-82f0-d1155e45244e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:36.863605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdb93b-57b2-4fa8-82f0-d1155e45244e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:36.863884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdb93b-57b2-4fa8-82f0-d1155e45244e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:36.864771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bdb93b-57b2-4fa8-82f0-d1155e45244e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:36.866797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bdb93b-57b2-4fa8-82f0-d1155e45244e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:36.882313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1740ee7-8f7e-42a7-893d-c7d9c8200167 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:36.890161 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2970/5983] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:24:36.898803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1740ee7-8f7e-42a7-893d-c7d9c8200167 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:36.921465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce74f44-a768-4dc7-99ba-a9fa9cfea0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:36.921465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce74f44-a768-4dc7-99ba-a9fa9cfea0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:36.921465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce74f44-a768-4dc7-99ba-a9fa9cfea0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:36.921465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce74f44-a768-4dc7-99ba-a9fa9cfea0e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:36.921949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce74f44-a768-4dc7-99ba-a9fa9cfea0e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:36.921949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce74f44-a768-4dc7-99ba-a9fa9cfea0e2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:36.921949 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3014/5984] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:24:36.929693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdb93b-57b2-4fa8-82f0-d1155e45244e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:36.944330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdb93b-57b2-4fa8-82f0-d1155e45244e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:36.949586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90051f74-6ee3-457f-91f3-118f91c99b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:36.950629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90051f74-6ee3-457f-91f3-118f91c99b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:36.950997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90051f74-6ee3-457f-91f3-118f91c99b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:36.951357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90051f74-6ee3-457f-91f3-118f91c99b05 admin admin] PATH_INFO: `/v3/projects/c4bba500689440a3b5fe07b8e1885bd3/users/728b87da2aa547ee92eb9f4316d31efd/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:36.969737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90051f74-6ee3-457f-91f3-118f91c99b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4bba500689440a3b5fe07b8e1885bd3, user_id=728b87da2aa547ee92eb9f4316d31efd, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:36.972607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90051f74-6ee3-457f-91f3-118f91c99b05 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:37.452590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0d349-46df-4700-bdf0-06b531730a72 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:37.452590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0d349-46df-4700-bdf0-06b531730a72 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:37.452590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0d349-46df-4700-bdf0-06b531730a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:37.463997 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2971/5985] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:24:36 2022] PUT /identity/v3/projects/c4bba500689440a3b5fe07b8e1885bd3/users/728b87da2aa547ee92eb9f4316d31efd/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:37.520066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-68a0d349-46df-4700-bdf0-06b531730a72 None None] MFA Rules not processed for user `728b87da2aa547ee92eb9f4316d31efd`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:37.525145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-68a0d349-46df-4700-bdf0-06b531730a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:37.646436 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3015/5986] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:37 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:24:37.700003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90051f74-6ee3-457f-91f3-118f91c99b05 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:37.706367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:37.751381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0d349-46df-4700-bdf0-06b531730a72 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:37.753123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-68a0d349-46df-4700-bdf0-06b531730a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:37.867636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90051f74-6ee3-457f-91f3-118f91c99b05 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:37.884491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f16dc-bb3c-4975-b353-59e9aaa8610d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:37.889507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f16dc-bb3c-4975-b353-59e9aaa8610d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:37.889859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f16dc-bb3c-4975-b353-59e9aaa8610d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:37.890107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f16dc-bb3c-4975-b353-59e9aaa8610d admin admin] PATH_INFO: `/v3/users/728b87da2aa547ee92eb9f4316d31efd` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:37.900428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f16dc-bb3c-4975-b353-59e9aaa8610d admin admin] RBAC: Authorizing `identity:delete_user(user_id=728b87da2aa547ee92eb9f4316d31efd)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:37.909326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f16dc-bb3c-4975-b353-59e9aaa8610d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:37.911141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a0d349-46df-4700-bdf0-06b531730a72 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:37.916515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684be2e7-b505-4043-957e-403e8fc80460 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:37.917191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684be2e7-b505-4043-957e-403e8fc80460 service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:37.917482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684be2e7-b505-4043-957e-403e8fc80460 service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:37.917771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684be2e7-b505-4043-957e-403e8fc80460 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:37.921155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-684be2e7-b505-4043-957e-403e8fc80460 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:38.065107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684be2e7-b505-4043-957e-403e8fc80460 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:38.070815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684be2e7-b505-4043-957e-403e8fc80460 service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:38.088549 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3016/5987] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:24:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:24:38.166588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684be2e7-b505-4043-957e-403e8fc80460 service glance] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:38.170062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-684be2e7-b505-4043-957e-403e8fc80460 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:38.175267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-884f16dc-bb3c-4975-b353-59e9aaa8610d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '728b87da2aa547ee92eb9f4316d31efd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:38.176031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-884f16dc-bb3c-4975-b353-59e9aaa8610d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '728b87da2aa547ee92eb9f4316d31efd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:38.197317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684be2e7-b505-4043-957e-403e8fc80460 service glance] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:38.200311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-884f16dc-bb3c-4975-b353-59e9aaa8610d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '728b87da2aa547ee92eb9f4316d31efd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:38.221008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b6776-3434-4492-8493-22672fc76830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:38.222405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b6776-3434-4492-8493-22672fc76830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:38.222759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b6776-3434-4492-8493-22672fc76830 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:38.223050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1b6776-3434-4492-8493-22672fc76830 admin admin] PATH_INFO: `/v3/projects/0fe7e4f1396c4cf7be16906e19cb9a19` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:38.224604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b6776-3434-4492-8493-22672fc76830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fe7e4f1396c4cf7be16906e19cb9a19)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:38.226644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1b6776-3434-4492-8493-22672fc76830 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:38.336448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-884f16dc-bb3c-4975-b353-59e9aaa8610d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '728b87da2aa547ee92eb9f4316d31efd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:38.337587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-884f16dc-bb3c-4975-b353-59e9aaa8610d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '728b87da2aa547ee92eb9f4316d31efd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:38.374966 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2972/5988] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:24:37 2022] DELETE /identity/v3/users/728b87da2aa547ee92eb9f4316d31efd => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:38.379846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b8938-6399-4240-9a03-495057a05838 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:38.380209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b8938-6399-4240-9a03-495057a05838 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:38.380620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b8938-6399-4240-9a03-495057a05838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:38.396397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d1b6776-3434-4492-8493-22672fc76830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fe7e4f1396c4cf7be16906e19cb9a19', 'request_id': 'req-5d1b6776-3434-4492-8493-22672fc76830'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:38.399010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c29b8938-6399-4240-9a03-495057a05838 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:38.400810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c29b8938-6399-4240-9a03-495057a05838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:38.405234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d1b6776-3434-4492-8493-22672fc76830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fe7e4f1396c4cf7be16906e19cb9a19', 'request_id': 'req-5d1b6776-3434-4492-8493-22672fc76830'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:38.490397 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2973/5989] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:24:38.500936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b8938-6399-4240-9a03-495057a05838 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:38.507599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c29b8938-6399-4240-9a03-495057a05838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:38.530690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d1b6776-3434-4492-8493-22672fc76830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fe7e4f1396c4cf7be16906e19cb9a19', 'request_id': 'req-5d1b6776-3434-4492-8493-22672fc76830'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:38.558288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d1b6776-3434-4492-8493-22672fc76830 admin admin] The token cache is being invalidate because project 0fe7e4f1396c4cf7be16906e19cb9a19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:24:38.558604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d1b6776-3434-4492-8493-22672fc76830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:38.559732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d1b6776-3434-4492-8493-22672fc76830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:38.561466 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3017/5990] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:24:38 2022] DELETE /identity/v3/projects/0fe7e4f1396c4cf7be16906e19cb9a19 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:24:38.565739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b6776-3434-4492-8493-22672fc76830 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:38.572717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5d1b6776-3434-4492-8493-22672fc76830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:38.611652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b8938-6399-4240-9a03-495057a05838 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:38.616036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f421ca-6e6d-4411-861d-8036d6c13ea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:38.618138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f421ca-6e6d-4411-861d-8036d6c13ea5 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:38.618138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f421ca-6e6d-4411-861d-8036d6c13ea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:38.618138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f421ca-6e6d-4411-861d-8036d6c13ea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:38.619555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-21f421ca-6e6d-4411-861d-8036d6c13ea5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:38.637148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f421ca-6e6d-4411-861d-8036d6c13ea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:38.639900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f421ca-6e6d-4411-861d-8036d6c13ea5 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:38.654384 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2974/5991] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:24:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:24:38.679971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1b6776-3434-4492-8493-22672fc76830 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:38.697747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38dc4e3-8e20-47b4-9e80-345c07ab9eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:38.698743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dc4e3-8e20-47b4-9e80-345c07ab9eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:38.699094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dc4e3-8e20-47b4-9e80-345c07ab9eea admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:38.699442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38dc4e3-8e20-47b4-9e80-345c07ab9eea admin admin] PATH_INFO: `/v3/users/6524b3ba52d04e6a9bd60cc86ab8129e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:38.707150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38dc4e3-8e20-47b4-9e80-345c07ab9eea admin admin] RBAC: Authorizing `identity:delete_user(user_id=6524b3ba52d04e6a9bd60cc86ab8129e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:38.709158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38dc4e3-8e20-47b4-9e80-345c07ab9eea admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:38.977357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a38dc4e3-8e20-47b4-9e80-345c07ab9eea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6524b3ba52d04e6a9bd60cc86ab8129e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:38.989703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a38dc4e3-8e20-47b4-9e80-345c07ab9eea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6524b3ba52d04e6a9bd60cc86ab8129e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:38.990548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a38dc4e3-8e20-47b4-9e80-345c07ab9eea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6524b3ba52d04e6a9bd60cc86ab8129e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:38.991202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a38dc4e3-8e20-47b4-9e80-345c07ab9eea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6524b3ba52d04e6a9bd60cc86ab8129e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:39.083507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a38dc4e3-8e20-47b4-9e80-345c07ab9eea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6524b3ba52d04e6a9bd60cc86ab8129e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:39.119898 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3018/5992] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:24:38 2022] DELETE /identity/v3/users/6524b3ba52d04e6a9bd60cc86ab8129e => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:24:39.501596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f421ca-6e6d-4411-861d-8036d6c13ea5 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:39.503870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-21f421ca-6e6d-4411-861d-8036d6c13ea5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:39.520356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f421ca-6e6d-4411-861d-8036d6c13ea5 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:39.633175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986dc05-1e1f-4001-9bf4-b8f8657d9bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:39.637372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986dc05-1e1f-4001-9bf4-b8f8657d9bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:39.637732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986dc05-1e1f-4001-9bf4-b8f8657d9bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:39.638050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986dc05-1e1f-4001-9bf4-b8f8657d9bd9 admin admin] PATH_INFO: `/v3/projects/c4bba500689440a3b5fe07b8e1885bd3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:39.641766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986dc05-1e1f-4001-9bf4-b8f8657d9bd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4bba500689440a3b5fe07b8e1885bd3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:39.649230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986dc05-1e1f-4001-9bf4-b8f8657d9bd9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:39.799342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1986dc05-1e1f-4001-9bf4-b8f8657d9bd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4bba500689440a3b5fe07b8e1885bd3', 'request_id': 'req-1986dc05-1e1f-4001-9bf4-b8f8657d9bd9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:39.805557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1986dc05-1e1f-4001-9bf4-b8f8657d9bd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4bba500689440a3b5fe07b8e1885bd3', 'request_id': 'req-1986dc05-1e1f-4001-9bf4-b8f8657d9bd9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:39.874985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c78cce-8be1-41cc-88b8-fa2fc24b7856 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:39.875485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c78cce-8be1-41cc-88b8-fa2fc24b7856 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:39.875928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c78cce-8be1-41cc-88b8-fa2fc24b7856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:39.900595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a0c78cce-8be1-41cc-88b8-fa2fc24b7856 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:39.904674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a0c78cce-8be1-41cc-88b8-fa2fc24b7856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:39.927395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1986dc05-1e1f-4001-9bf4-b8f8657d9bd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4bba500689440a3b5fe07b8e1885bd3', 'request_id': 'req-1986dc05-1e1f-4001-9bf4-b8f8657d9bd9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:39.955117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1986dc05-1e1f-4001-9bf4-b8f8657d9bd9 admin admin] The token cache is being invalidate because project c4bba500689440a3b5fe07b8e1885bd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:24:39.955419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1986dc05-1e1f-4001-9bf4-b8f8657d9bd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:39.956121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1986dc05-1e1f-4001-9bf4-b8f8657d9bd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:39.959456 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2975/5993] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:24:39 2022] DELETE /identity/v3/projects/c4bba500689440a3b5fe07b8e1885bd3 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:39.964405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1da6f-cde2-4eaf-955c-2fe61f4c9caa None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:39.965534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1da6f-cde2-4eaf-955c-2fe61f4c9caa None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:39.966245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1da6f-cde2-4eaf-955c-2fe61f4c9caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:39.995351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-85f1da6f-cde2-4eaf-955c-2fe61f4c9caa None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:39.997508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-85f1da6f-cde2-4eaf-955c-2fe61f4c9caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:40.008511 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3019/5994] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:24:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 28 21:24:40.019763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c78cce-8be1-41cc-88b8-fa2fc24b7856 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:40.025473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a0c78cce-8be1-41cc-88b8-fa2fc24b7856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:40.058335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:40.146274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c78cce-8be1-41cc-88b8-fa2fc24b7856 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:40.148911 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2976/5995] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:24:40.150594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1da6f-cde2-4eaf-955c-2fe61f4c9caa None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:40.154487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-85f1da6f-cde2-4eaf-955c-2fe61f4c9caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:40.161136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2cc1f-5b03-43df-a73e-933add2c2812 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:40.162384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2cc1f-5b03-43df-a73e-933add2c2812 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:40.162670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2cc1f-5b03-43df-a73e-933add2c2812 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:40.162947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2cc1f-5b03-43df-a73e-933add2c2812 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:40.173427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1da6f-cde2-4eaf-955c-2fe61f4c9caa None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:40.174867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2cc1f-5b03-43df-a73e-933add2c2812 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:40.177105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2cc1f-5b03-43df-a73e-933add2c2812 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:40.188269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0d549-64d3-41df-96c1-97dd3f5a6f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:40.189171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0d549-64d3-41df-96c1-97dd3f5a6f12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:40.189468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0d549-64d3-41df-96c1-97dd3f5a6f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:40.189751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0d549-64d3-41df-96c1-97dd3f5a6f12 admin admin] PATH_INFO: `/v3/projects/6576d2aaba2a4508bef3c6f6e298c3a9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:40.207017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0d549-64d3-41df-96c1-97dd3f5a6f12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6576d2aaba2a4508bef3c6f6e298c3a9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:40.207017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0d549-64d3-41df-96c1-97dd3f5a6f12 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:40.208318 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3020/5996] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:24:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:24:40.209328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed5150-7c5a-4a11-95b9-11d78144c52f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:40.209636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed5150-7c5a-4a11-95b9-11d78144c52f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:40.211697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aed5150-7c5a-4a11-95b9-11d78144c52f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:40.263338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6aed5150-7c5a-4a11-95b9-11d78144c52f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:40.266097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6aed5150-7c5a-4a11-95b9-11d78144c52f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:40.287914 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3021/5997] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:40.295149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed5150-7c5a-4a11-95b9-11d78144c52f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:40.305330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aed5150-7c5a-4a11-95b9-11d78144c52f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:40.311127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6232c4-8d4e-4783-9588-2e2a7a57aa73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:40.311799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6232c4-8d4e-4783-9588-2e2a7a57aa73 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:40.312084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6232c4-8d4e-4783-9588-2e2a7a57aa73 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:40.312368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6232c4-8d4e-4783-9588-2e2a7a57aa73 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:40.313088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6232c4-8d4e-4783-9588-2e2a7a57aa73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:24:40.313883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6232c4-8d4e-4783-9588-2e2a7a57aa73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:40.315070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6232c4-8d4e-4783-9588-2e2a7a57aa73 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:40.328422 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3022/5998] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:24:40.332747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6232c4-8d4e-4783-9588-2e2a7a57aa73 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:40.343755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6232c4-8d4e-4783-9588-2e2a7a57aa73 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:40.349153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5557dfe-18a1-4eb1-8609-5e442cd18e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:40.349830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5557dfe-18a1-4eb1-8609-5e442cd18e1b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:40.350198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5557dfe-18a1-4eb1-8609-5e442cd18e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:40.350434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5557dfe-18a1-4eb1-8609-5e442cd18e1b admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:40.351200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5557dfe-18a1-4eb1-8609-5e442cd18e1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:40.353139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5557dfe-18a1-4eb1-8609-5e442cd18e1b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:40.393127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:40.445346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3df0d549-64d3-41df-96c1-97dd3f5a6f12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6576d2aaba2a4508bef3c6f6e298c3a9', 'request_id': 'req-3df0d549-64d3-41df-96c1-97dd3f5a6f12'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:40.458515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3df0d549-64d3-41df-96c1-97dd3f5a6f12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6576d2aaba2a4508bef3c6f6e298c3a9', 'request_id': 'req-3df0d549-64d3-41df-96c1-97dd3f5a6f12'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:40.462868 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3023/5999] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:24:40 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:24:40.463244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5557dfe-18a1-4eb1-8609-5e442cd18e1b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:40.552746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3df0d549-64d3-41df-96c1-97dd3f5a6f12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6576d2aaba2a4508bef3c6f6e298c3a9', 'request_id': 'req-3df0d549-64d3-41df-96c1-97dd3f5a6f12'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:40.566672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3df0d549-64d3-41df-96c1-97dd3f5a6f12 admin admin] The token cache is being invalidate because project 6576d2aaba2a4508bef3c6f6e298c3a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:24:40.567576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3df0d549-64d3-41df-96c1-97dd3f5a6f12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:40.568841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3df0d549-64d3-41df-96c1-97dd3f5a6f12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:24:40.572629 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2977/6000] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:24:40 2022] DELETE /identity/v3/projects/6576d2aaba2a4508bef3c6f6e298c3a9 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:40.588493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a161b8-cdf9-4d90-8bb4-72b9056ecbb3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:40.588790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a161b8-cdf9-4d90-8bb4-72b9056ecbb3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:40.589088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a161b8-cdf9-4d90-8bb4-72b9056ecbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:40.594022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5557dfe-18a1-4eb1-8609-5e442cd18e1b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:40.599734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7265e2-497d-4f51-904d-7c5c73fd4772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:40.600807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7265e2-497d-4f51-904d-7c5c73fd4772 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:40.601213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7265e2-497d-4f51-904d-7c5c73fd4772 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:40.601625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7265e2-497d-4f51-904d-7c5c73fd4772 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:40.602578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7265e2-497d-4f51-904d-7c5c73fd4772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:40.605237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7265e2-497d-4f51-904d-7c5c73fd4772 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:40.612943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-39a161b8-cdf9-4d90-8bb4-72b9056ecbb3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:40.615311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-39a161b8-cdf9-4d90-8bb4-72b9056ecbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:40.633623 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2978/6001] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:40.635345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544cb96-1746-4c35-a8de-218cc85ad438 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:40.635697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544cb96-1746-4c35-a8de-218cc85ad438 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:40.635987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544cb96-1746-4c35-a8de-218cc85ad438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:40.668629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3544cb96-1746-4c35-a8de-218cc85ad438 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:40.670275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3544cb96-1746-4c35-a8de-218cc85ad438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:40.680357 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2979/6002] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:40.684808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544cb96-1746-4c35-a8de-218cc85ad438 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:40.697306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544cb96-1746-4c35-a8de-218cc85ad438 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:40.706856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35a874-ebd0-4e24-92e2-703e005e3253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:40.706856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35a874-ebd0-4e24-92e2-703e005e3253 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:40.706856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35a874-ebd0-4e24-92e2-703e005e3253 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:40.706856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35a874-ebd0-4e24-92e2-703e005e3253 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:40.708153 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3024/6003] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:24:40 2022] POST /identity/v3/users => generated 283 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:24:40.708605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35a874-ebd0-4e24-92e2-703e005e3253 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:24:40.709199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35a874-ebd0-4e24-92e2-703e005e3253 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:40.712332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:40.713650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35a874-ebd0-4e24-92e2-703e005e3253 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:40.715077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7265e2-497d-4f51-904d-7c5c73fd4772 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:40.717196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9e7265e2-497d-4f51-904d-7c5c73fd4772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:40.735223 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2980/6004] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:24:40.738427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7265e2-497d-4f51-904d-7c5c73fd4772 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:40.739706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35a874-ebd0-4e24-92e2-703e005e3253 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:40.749192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c8b59-ca0c-4263-84e5-749990b7f90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:40.749945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c8b59-ca0c-4263-84e5-749990b7f90a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:40.750236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c8b59-ca0c-4263-84e5-749990b7f90a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:40.750517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c8b59-ca0c-4263-84e5-749990b7f90a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:40.751491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c8b59-ca0c-4263-84e5-749990b7f90a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:40.757344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c8b59-ca0c-4263-84e5-749990b7f90a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:40.760197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35a874-ebd0-4e24-92e2-703e005e3253 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:40.771995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accb28c-98ae-48e4-a4b4-f4d7a9ea1592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:40.772945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accb28c-98ae-48e4-a4b4-f4d7a9ea1592 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:40.773384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accb28c-98ae-48e4-a4b4-f4d7a9ea1592 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:40.773824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0accb28c-98ae-48e4-a4b4-f4d7a9ea1592 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:40.776009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accb28c-98ae-48e4-a4b4-f4d7a9ea1592 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:40.780842 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3025/6005] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:24:40.780842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c8b59-ca0c-4263-84e5-749990b7f90a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:40.781061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0accb28c-98ae-48e4-a4b4-f4d7a9ea1592 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:40.793905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c8b59-ca0c-4263-84e5-749990b7f90a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:40.807975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306161b-7cdd-47c4-a91c-de827cc76ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:40.808688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306161b-7cdd-47c4-a91c-de827cc76ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:40.808987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306161b-7cdd-47c4-a91c-de827cc76ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:40.809261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306161b-7cdd-47c4-a91c-de827cc76ad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:40.810305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306161b-7cdd-47c4-a91c-de827cc76ad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:40.811940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306161b-7cdd-47c4-a91c-de827cc76ad3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:40.838392 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3026/6006] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:24:40.840159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306161b-7cdd-47c4-a91c-de827cc76ad3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:40.858045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306161b-7cdd-47c4-a91c-de827cc76ad3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:40.865241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7e992-2662-4dc5-8803-fea3259ddbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:40.866220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7e992-2662-4dc5-8803-fea3259ddbd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:40.867648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7e992-2662-4dc5-8803-fea3259ddbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:40.867648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7e992-2662-4dc5-8803-fea3259ddbd1 admin admin] PATH_INFO: `/v3/projects/2ce54a0434cb4333a5a7072b01079883/users/f6ea3fb90c2343ceaaeac6c7efcef26d/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:40.885579 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2981/6007] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:24:40 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:24:40.892487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accb28c-98ae-48e4-a4b4-f4d7a9ea1592 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:40.898590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7e992-2662-4dc5-8803-fea3259ddbd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ce54a0434cb4333a5a7072b01079883, user_id=f6ea3fb90c2343ceaaeac6c7efcef26d, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:40.904208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7e992-2662-4dc5-8803-fea3259ddbd1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:41.033769 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3027/6008] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:24:40 2022] PUT /identity/v3/projects/2ce54a0434cb4333a5a7072b01079883/users/f6ea3fb90c2343ceaaeac6c7efcef26d/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:24:41.040023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0accb28c-98ae-48e4-a4b4-f4d7a9ea1592 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:41.046781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc1821-1b06-4156-a75a-590e0b9494b5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:41.047198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc1821-1b06-4156-a75a-590e0b9494b5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:41.049026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc1821-1b06-4156-a75a-590e0b9494b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:41.137588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-52bc1821-1b06-4156-a75a-590e0b9494b5 None None] MFA Rules not processed for user `f6ea3fb90c2343ceaaeac6c7efcef26d`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:41.139259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-52bc1821-1b06-4156-a75a-590e0b9494b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:41.145542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac0ee4-203e-4302-af30-5d79377e8b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:41.146284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac0ee4-203e-4302-af30-5d79377e8b82 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:41.146563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac0ee4-203e-4302-af30-5d79377e8b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:41.146844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac0ee4-203e-4302-af30-5d79377e8b82 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:41.147548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac0ee4-203e-4302-af30-5d79377e8b82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:41.149553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac0ee4-203e-4302-af30-5d79377e8b82 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:41.263942 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2982/6009] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:24:40 2022] POST /identity/v3/users => generated 292 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:24:41.265415 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3028/6010] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:41 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:41.265534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac0ee4-203e-4302-af30-5d79377e8b82 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:41.276468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc1821-1b06-4156-a75a-590e0b9494b5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:41.277986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-52bc1821-1b06-4156-a75a-590e0b9494b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:41.308889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac0ee4-203e-4302-af30-5d79377e8b82 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:41.325056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2aa45-53a4-4eb8-b3d4-75c7d4d49eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:41.325805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2aa45-53a4-4eb8-b3d4-75c7d4d49eae admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:41.326134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2aa45-53a4-4eb8-b3d4-75c7d4d49eae admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:41.326476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2aa45-53a4-4eb8-b3d4-75c7d4d49eae admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:41.327222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2aa45-53a4-4eb8-b3d4-75c7d4d49eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:41.328984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2aa45-53a4-4eb8-b3d4-75c7d4d49eae admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:41.343026 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2983/6011] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:24:41.347455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2aa45-53a4-4eb8-b3d4-75c7d4d49eae admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:41.359250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2aa45-53a4-4eb8-b3d4-75c7d4d49eae admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:41.364509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bb931-714f-45da-a5b6-471fd386516e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:41.367866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bb931-714f-45da-a5b6-471fd386516e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:41.367866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bb931-714f-45da-a5b6-471fd386516e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:41.367866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7bb931-714f-45da-a5b6-471fd386516e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:41.368123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7bb931-714f-45da-a5b6-471fd386516e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:41.370349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7bb931-714f-45da-a5b6-471fd386516e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:41.379781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc1821-1b06-4156-a75a-590e0b9494b5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:41.386750 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2984/6012] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:24:41.387643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bb931-714f-45da-a5b6-471fd386516e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:41.390781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265cab52-d304-4893-9893-808e4ef3c9ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:41.392531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cab52-d304-4893-9893-808e4ef3c9ed service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:41.392853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cab52-d304-4893-9893-808e4ef3c9ed service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:41.393149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265cab52-d304-4893-9893-808e4ef3c9ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:41.399537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7bb931-714f-45da-a5b6-471fd386516e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:41.405280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4965b-46a1-4ddc-b8a3-c25e56964f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:41.406289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4965b-46a1-4ddc-b8a3-c25e56964f6e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:41.406432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265cab52-d304-4893-9893-808e4ef3c9ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:41.406571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4965b-46a1-4ddc-b8a3-c25e56964f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:41.406851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c4965b-46a1-4ddc-b8a3-c25e56964f6e admin admin] PATH_INFO: `/v3/projects/52102c097d7b4a81aa28fe0007377926/users/c6be46d2745c45729e2b105052747a02/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:41.408173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265cab52-d304-4893-9893-808e4ef3c9ed service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:41.420079 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3029/6013] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:24:41 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:24:41.437018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4965b-46a1-4ddc-b8a3-c25e56964f6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=52102c097d7b4a81aa28fe0007377926, user_id=c6be46d2745c45729e2b105052747a02, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:41.440070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c4965b-46a1-4ddc-b8a3-c25e56964f6e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:41.546474 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2985/6014] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:24:41 2022] PUT /identity/v3/projects/52102c097d7b4a81aa28fe0007377926/users/c6be46d2745c45729e2b105052747a02/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:24:41.557946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29872f-1269-488f-a3d0-b4f2bf1b559f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:41.558232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29872f-1269-488f-a3d0-b4f2bf1b559f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:41.558520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29872f-1269-488f-a3d0-b4f2bf1b559f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:41.602499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0d29872f-1269-488f-a3d0-b4f2bf1b559f None None] MFA Rules not processed for user `c6be46d2745c45729e2b105052747a02`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:41.603770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0d29872f-1269-488f-a3d0-b4f2bf1b559f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:41.704457 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3030/6015] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:41.709162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4965b-46a1-4ddc-b8a3-c25e56964f6e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:41.784003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c4965b-46a1-4ddc-b8a3-c25e56964f6e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:41.789050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650901e-1738-467b-97bb-a9db3a373283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:41.789749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650901e-1738-467b-97bb-a9db3a373283 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:41.790073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650901e-1738-467b-97bb-a9db3a373283 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:41.790356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4650901e-1738-467b-97bb-a9db3a373283 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:41.792329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650901e-1738-467b-97bb-a9db3a373283 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:41.794349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4650901e-1738-467b-97bb-a9db3a373283 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:41.976892 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2986/6016] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:24:41 2022] POST /identity/v3/projects => generated 383 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:24:41.981880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29872f-1269-488f-a3d0-b4f2bf1b559f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:42.017721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29872f-1269-488f-a3d0-b4f2bf1b559f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:42.124815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bbd3c8-107c-4dbd-9bcb-23eadf2f9760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:42.125535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbd3c8-107c-4dbd-9bcb-23eadf2f9760 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:42.125920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbd3c8-107c-4dbd-9bcb-23eadf2f9760 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:42.126099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbd3c8-107c-4dbd-9bcb-23eadf2f9760 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:42.126941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bbd3c8-107c-4dbd-9bcb-23eadf2f9760 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:42.149436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bbd3c8-107c-4dbd-9bcb-23eadf2f9760 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:42.295681 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3031/6017] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:24:41 2022] POST /identity/v3/users => generated 298 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:24:42.295931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650901e-1738-467b-97bb-a9db3a373283 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:42.329097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4650901e-1738-467b-97bb-a9db3a373283 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:42.335056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b2879-80e8-42b9-a621-5c78e719f940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:42.338257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b2879-80e8-42b9-a621-5c78e719f940 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:42.338257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b2879-80e8-42b9-a621-5c78e719f940 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:42.338257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b2879-80e8-42b9-a621-5c78e719f940 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:42.338257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b2879-80e8-42b9-a621-5c78e719f940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:42.340111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b2879-80e8-42b9-a621-5c78e719f940 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:42.362296 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2987/6018] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:24:42.371755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bbd3c8-107c-4dbd-9bcb-23eadf2f9760 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:42.388382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bbd3c8-107c-4dbd-9bcb-23eadf2f9760 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:42.417173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f98ebc-bf7b-4d07-b245-bd85593b46eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:42.418363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f98ebc-bf7b-4d07-b245-bd85593b46eb admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:42.421712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f98ebc-bf7b-4d07-b245-bd85593b46eb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:42.425608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f98ebc-bf7b-4d07-b245-bd85593b46eb admin admin] PATH_INFO: `/v3/projects/78b4e47aaec146f8be53d9b3a26b1b6b/users/fca31bf977974d349e713cd3b6c09f27/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:42.439740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:42.466087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f98ebc-bf7b-4d07-b245-bd85593b46eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=78b4e47aaec146f8be53d9b3a26b1b6b, user_id=fca31bf977974d349e713cd3b6c09f27, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:42.474753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f98ebc-bf7b-4d07-b245-bd85593b46eb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:42.566088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:42.595727 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3032/6019] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:24:42 2022] PUT /identity/v3/projects/78b4e47aaec146f8be53d9b3a26b1b6b/users/fca31bf977974d349e713cd3b6c09f27/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:42.599929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b2879-80e8-42b9-a621-5c78e719f940 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:42.612271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b2879-80e8-42b9-a621-5c78e719f940 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:42.708711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12683017-4f5f-4493-becd-81ce095cae54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:42.709517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12683017-4f5f-4493-becd-81ce095cae54 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:42.710088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12683017-4f5f-4493-becd-81ce095cae54 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:42.710205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12683017-4f5f-4493-becd-81ce095cae54 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:42.712207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12683017-4f5f-4493-becd-81ce095cae54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:42.716895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12683017-4f5f-4493-becd-81ce095cae54 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:42.730767 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2988/6020] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:24:42.732015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f98ebc-bf7b-4d07-b245-bd85593b46eb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:42.746572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f98ebc-bf7b-4d07-b245-bd85593b46eb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:42.764161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30004a-b6b5-4492-bfb7-447c2a3a27b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:42.768757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30004a-b6b5-4492-bfb7-447c2a3a27b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:42.769309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30004a-b6b5-4492-bfb7-447c2a3a27b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:42.771565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec30004a-b6b5-4492-bfb7-447c2a3a27b3 admin admin] PATH_INFO: `/v3/projects/78b4e47aaec146f8be53d9b3a26b1b6b/users/fca31bf977974d349e713cd3b6c09f27/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:42.777936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30004a-b6b5-4492-bfb7-447c2a3a27b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78b4e47aaec146f8be53d9b3a26b1b6b, user_id=fca31bf977974d349e713cd3b6c09f27, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:42.781864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec30004a-b6b5-4492-bfb7-447c2a3a27b3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:42.842724 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3033/6021] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:24:42 2022] PUT /identity/v3/projects/78b4e47aaec146f8be53d9b3a26b1b6b/users/fca31bf977974d349e713cd3b6c09f27/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:24:42.847007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a311af-5dfd-4582-957c-20a7bb64701b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:42.852046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a311af-5dfd-4582-957c-20a7bb64701b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:42.852046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a311af-5dfd-4582-957c-20a7bb64701b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:42.885679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-82a311af-5dfd-4582-957c-20a7bb64701b None None] MFA Rules not processed for user `fca31bf977974d349e713cd3b6c09f27`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:42.888040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-82a311af-5dfd-4582-957c-20a7bb64701b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:42.984106 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2989/6022] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:42.996747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30004a-b6b5-4492-bfb7-447c2a3a27b3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:43.082285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec30004a-b6b5-4492-bfb7-447c2a3a27b3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:43.100436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af274733-1775-4dc0-af79-e8be51faad1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:43.101126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af274733-1775-4dc0-af79-e8be51faad1e service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:43.101431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af274733-1775-4dc0-af79-e8be51faad1e service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:43.101718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af274733-1775-4dc0-af79-e8be51faad1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:43.199973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af274733-1775-4dc0-af79-e8be51faad1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:43.202746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af274733-1775-4dc0-af79-e8be51faad1e service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:43.216253 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3034/6023] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:24:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:24:43.399936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f86e27-30e6-4487-9ccc-0c07ba2b87ae None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:43.400261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f86e27-30e6-4487-9ccc-0c07ba2b87ae None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:43.400546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f86e27-30e6-4487-9ccc-0c07ba2b87ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:43.431198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b5f86e27-30e6-4487-9ccc-0c07ba2b87ae None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:43.433253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b5f86e27-30e6-4487-9ccc-0c07ba2b87ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:43.499150 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2990/6024] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:24:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:24:43.511822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af274733-1775-4dc0-af79-e8be51faad1e service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:43.513330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-af274733-1775-4dc0-af79-e8be51faad1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:43.591312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af274733-1775-4dc0-af79-e8be51faad1e service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:43.600972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72d473-9f4b-4535-aa4f-5993ace28ec6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:43.601965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72d473-9f4b-4535-aa4f-5993ace28ec6 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:43.602420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72d473-9f4b-4535-aa4f-5993ace28ec6 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:43.602960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72d473-9f4b-4535-aa4f-5993ace28ec6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:43.616807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72d473-9f4b-4535-aa4f-5993ace28ec6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:43.620214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72d473-9f4b-4535-aa4f-5993ace28ec6 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:43.633969 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3035/6025] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:24:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:24:44.863590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab4d0e-17f7-4f27-b14d-18456a5d22d1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:44.864069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab4d0e-17f7-4f27-b14d-18456a5d22d1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:44.864149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab4d0e-17f7-4f27-b14d-18456a5d22d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:44.902380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-17ab4d0e-17f7-4f27-b14d-18456a5d22d1 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:44.911663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-17ab4d0e-17f7-4f27-b14d-18456a5d22d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:44.925643 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2991/6026] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:24:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:44.943323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72d473-9f4b-4535-aa4f-5993ace28ec6 service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:44.965925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72d473-9f4b-4535-aa4f-5993ace28ec6 service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:44.985774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3e18e-3fc1-4ff5-9f42-6ddac428d0c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:44.989348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3e18e-3fc1-4ff5-9f42-6ddac428d0c3 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:44.989812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3e18e-3fc1-4ff5-9f42-6ddac428d0c3 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:44.990202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3e18e-3fc1-4ff5-9f42-6ddac428d0c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:45.001415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3e18e-3fc1-4ff5-9f42-6ddac428d0c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:45.003058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3e18e-3fc1-4ff5-9f42-6ddac428d0c3 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:45.016402 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3036/6027] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:24:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:24:45.186795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65fa98-c969-4535-a623-4236957f7747 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:45.187292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65fa98-c969-4535-a623-4236957f7747 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:45.187725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65fa98-c969-4535-a623-4236957f7747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:45.209720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2b65fa98-c969-4535-a623-4236957f7747 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:45.211381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2b65fa98-c969-4535-a623-4236957f7747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:45.228025 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2992/6028] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:24:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:24:45.255742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3e18e-3fc1-4ff5-9f42-6ddac428d0c3 service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:45.278986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3e18e-3fc1-4ff5-9f42-6ddac428d0c3 service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:45.305769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4c355-d0d3-43ab-a41d-645aa10790a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:45.306607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4c355-d0d3-43ab-a41d-645aa10790a2 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:45.306972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4c355-d0d3-43ab-a41d-645aa10790a2 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:45.311321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4c355-d0d3-43ab-a41d-645aa10790a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:45.342398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4c355-d0d3-43ab-a41d-645aa10790a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:45.344688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4c355-d0d3-43ab-a41d-645aa10790a2 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:45.375220 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3037/6029] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:24:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:24:45.453506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:45.714051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:48.250381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:49.061474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:49.258501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d517115-15ee-48fb-bd8b-68975e0d9ed8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:49.258774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d517115-15ee-48fb-bd8b-68975e0d9ed8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:49.259053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d517115-15ee-48fb-bd8b-68975e0d9ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:49.309394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4d517115-15ee-48fb-bd8b-68975e0d9ed8 None None] MFA Rules not processed for user `ebc8e00965334bd1bb811a0eefd64b84`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:49.316662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4d517115-15ee-48fb-bd8b-68975e0d9ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:49.490123 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2993/6030] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:50.947190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:53.101979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4c355-d0d3-43ab-a41d-645aa10790a2 service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:53.104061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6fc4c355-d0d3-43ab-a41d-645aa10790a2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:53.131248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4c355-d0d3-43ab-a41d-645aa10790a2 service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:53.146037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9eb097-b30f-46eb-a56c-71a606250c10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:53.146832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9eb097-b30f-46eb-a56c-71a606250c10 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:53.148169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9eb097-b30f-46eb-a56c-71a606250c10 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:53.149171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9eb097-b30f-46eb-a56c-71a606250c10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:53.172857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9eb097-b30f-46eb-a56c-71a606250c10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:53.175394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9eb097-b30f-46eb-a56c-71a606250c10 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:53.200521 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3038/6031] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:24:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:24:58.085424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2261a-607c-4649-9984-adf0f1bbc2d8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:58.085995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2261a-607c-4649-9984-adf0f1bbc2d8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:58.085995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2261a-607c-4649-9984-adf0f1bbc2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:58.106139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bce2261a-607c-4649-9984-adf0f1bbc2d8 None None] MFA Rules not processed for user `02d5fbb072814fd7923ecf12c7ecad7b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:24:58.107385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bce2261a-607c-4649-9984-adf0f1bbc2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:58.228726 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2994/6032] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:24:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:24:58.766281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9eb097-b30f-46eb-a56c-71a606250c10 service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:24:58.768538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bf9eb097-b30f-46eb-a56c-71a606250c10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:58.783365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:58.831087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9eb097-b30f-46eb-a56c-71a606250c10 service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:24:58.835853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f1d58-57cc-4d47-9510-b3ef5c72a190 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:24:58.836957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f1d58-57cc-4d47-9510-b3ef5c72a190 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:24:58.838040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f1d58-57cc-4d47-9510-b3ef5c72a190 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:24:58.838391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f1d58-57cc-4d47-9510-b3ef5c72a190 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:24:58.840343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cb6f1d58-57cc-4d47-9510-b3ef5c72a190 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:24:58.854802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f1d58-57cc-4d47-9510-b3ef5c72a190 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:24:58.858123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f1d58-57cc-4d47-9510-b3ef5c72a190 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:24:58.866826 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3039/6033] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:24:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:25:06.495957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b95c8e-455a-46f6-904e-a0fe8bd28321 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:06.496528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b95c8e-455a-46f6-904e-a0fe8bd28321 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:06.496528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b95c8e-455a-46f6-904e-a0fe8bd28321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:06.540581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-31b95c8e-455a-46f6-904e-a0fe8bd28321 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:06.547298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-31b95c8e-455a-46f6-904e-a0fe8bd28321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:06.584328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f1d58-57cc-4d47-9510-b3ef5c72a190 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:06.596280 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2995/6034] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:25:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:06.600993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f1d58-57cc-4d47-9510-b3ef5c72a190 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:06.608270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994eeb8-7875-44de-9733-fd9b2d07fa5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:06.610392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994eeb8-7875-44de-9733-fd9b2d07fa5a service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:06.610799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994eeb8-7875-44de-9733-fd9b2d07fa5a service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:06.613270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994eeb8-7875-44de-9733-fd9b2d07fa5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:06.635346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994eeb8-7875-44de-9733-fd9b2d07fa5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:06.637146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994eeb8-7875-44de-9733-fd9b2d07fa5a service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:06.647819 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3040/6035] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:25:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 21:25:12.010557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b95c8e-455a-46f6-904e-a0fe8bd28321 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:12.043551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:12.081366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b95c8e-455a-46f6-904e-a0fe8bd28321 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:12.086106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12745a5-cb93-41d2-81e0-eabd965b20da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:12.087079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12745a5-cb93-41d2-81e0-eabd965b20da admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:12.087390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12745a5-cb93-41d2-81e0-eabd965b20da admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:12.087854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12745a5-cb93-41d2-81e0-eabd965b20da admin admin] PATH_INFO: `/v3/users/f6ea3fb90c2343ceaaeac6c7efcef26d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:12.089352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12745a5-cb93-41d2-81e0-eabd965b20da admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6ea3fb90c2343ceaaeac6c7efcef26d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:12.091641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12745a5-cb93-41d2-81e0-eabd965b20da admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:12.328082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e12745a5-cb93-41d2-81e0-eabd965b20da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6ea3fb90c2343ceaaeac6c7efcef26d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:12.329443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e12745a5-cb93-41d2-81e0-eabd965b20da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6ea3fb90c2343ceaaeac6c7efcef26d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:12.338339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e12745a5-cb93-41d2-81e0-eabd965b20da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6ea3fb90c2343ceaaeac6c7efcef26d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:12.419631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e12745a5-cb93-41d2-81e0-eabd965b20da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6ea3fb90c2343ceaaeac6c7efcef26d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:12.420384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e12745a5-cb93-41d2-81e0-eabd965b20da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6ea3fb90c2343ceaaeac6c7efcef26d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:12.467478 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2996/6036] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:12 2022] DELETE /identity/v3/users/f6ea3fb90c2343ceaaeac6c7efcef26d => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:12.471974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d2fd2-c6f3-4e69-878c-8addadead9fe None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:12.473124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d2fd2-c6f3-4e69-878c-8addadead9fe None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:12.473618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d2fd2-c6f3-4e69-878c-8addadead9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:12.495230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b05d2fd2-c6f3-4e69-878c-8addadead9fe None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:12.496997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b05d2fd2-c6f3-4e69-878c-8addadead9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:12.633391 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3041/6037] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:12.646855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12745a5-cb93-41d2-81e0-eabd965b20da admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:12.648424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e12745a5-cb93-41d2-81e0-eabd965b20da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:12.751320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12745a5-cb93-41d2-81e0-eabd965b20da admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:12.767566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dd0f58-b302-4964-beba-c48222d4360a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:12.767566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd0f58-b302-4964-beba-c48222d4360a service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:12.767566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd0f58-b302-4964-beba-c48222d4360a service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:12.767566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dd0f58-b302-4964-beba-c48222d4360a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:12.768049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd0f58-b302-4964-beba-c48222d4360a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:12.768049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dd0f58-b302-4964-beba-c48222d4360a service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:12.779039 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2997/6038] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:25:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:25:13.253871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d2fd2-c6f3-4e69-878c-8addadead9fe None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:13.255734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b05d2fd2-c6f3-4e69-878c-8addadead9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:13.270931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d2fd2-c6f3-4e69-878c-8addadead9fe None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:13.277161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268e7c5-cb52-40dc-a6b3-3d4a25dbc7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:13.278244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268e7c5-cb52-40dc-a6b3-3d4a25dbc7c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:13.278364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268e7c5-cb52-40dc-a6b3-3d4a25dbc7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:13.278619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268e7c5-cb52-40dc-a6b3-3d4a25dbc7c3 admin admin] PATH_INFO: `/v3/projects/2ce54a0434cb4333a5a7072b01079883` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:13.280109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268e7c5-cb52-40dc-a6b3-3d4a25dbc7c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ce54a0434cb4333a5a7072b01079883)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:13.282233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5268e7c5-cb52-40dc-a6b3-3d4a25dbc7c3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:13.454459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5268e7c5-cb52-40dc-a6b3-3d4a25dbc7c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ce54a0434cb4333a5a7072b01079883', 'request_id': 'req-5268e7c5-cb52-40dc-a6b3-3d4a25dbc7c3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:13.462416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5268e7c5-cb52-40dc-a6b3-3d4a25dbc7c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ce54a0434cb4333a5a7072b01079883', 'request_id': 'req-5268e7c5-cb52-40dc-a6b3-3d4a25dbc7c3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:13.631961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5268e7c5-cb52-40dc-a6b3-3d4a25dbc7c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ce54a0434cb4333a5a7072b01079883', 'request_id': 'req-5268e7c5-cb52-40dc-a6b3-3d4a25dbc7c3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:13.648058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5268e7c5-cb52-40dc-a6b3-3d4a25dbc7c3 admin admin] The token cache is being invalidate because project 2ce54a0434cb4333a5a7072b01079883 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:13.648710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5268e7c5-cb52-40dc-a6b3-3d4a25dbc7c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:13.649637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5268e7c5-cb52-40dc-a6b3-3d4a25dbc7c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:13.651834 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3042/6039] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:13 2022] DELETE /identity/v3/projects/2ce54a0434cb4333a5a7072b01079883 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:13.658869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f528e19-5767-4dae-aedb-67a2a2422c0d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:13.659804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f528e19-5767-4dae-aedb-67a2a2422c0d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:13.660259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f528e19-5767-4dae-aedb-67a2a2422c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:13.695073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0f528e19-5767-4dae-aedb-67a2a2422c0d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:13.695073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0f528e19-5767-4dae-aedb-67a2a2422c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:13.826537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:13.852799 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2998/6040] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:13.854719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0bd9b-f3b5-48d6-b442-c3e767c6d9b1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:13.861465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0bd9b-f3b5-48d6-b442-c3e767c6d9b1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:13.861465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a0bd9b-f3b5-48d6-b442-c3e767c6d9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:13.882067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-76a0bd9b-f3b5-48d6-b442-c3e767c6d9b1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:13.888655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-76a0bd9b-f3b5-48d6-b442-c3e767c6d9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:13.908593 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3043/6041] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:25:13.910189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f528e19-5767-4dae-aedb-67a2a2422c0d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:13.922506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f528e19-5767-4dae-aedb-67a2a2422c0d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:13.930147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465943da-416d-4475-8127-f020675e168a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:13.931609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465943da-416d-4475-8127-f020675e168a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:13.932060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465943da-416d-4475-8127-f020675e168a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:13.932963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465943da-416d-4475-8127-f020675e168a admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:13.933941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465943da-416d-4475-8127-f020675e168a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:25:13.934416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465943da-416d-4475-8127-f020675e168a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:13.937936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465943da-416d-4475-8127-f020675e168a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:13.960022 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 2999/6042] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:13.964710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0bd9b-f3b5-48d6-b442-c3e767c6d9b1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:13.977131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a0bd9b-f3b5-48d6-b442-c3e767c6d9b1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:13.994202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973a92b-f286-4f75-8556-24b365375834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:13.994891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973a92b-f286-4f75-8556-24b365375834 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:13.995176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973a92b-f286-4f75-8556-24b365375834 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:13.995447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973a92b-f286-4f75-8556-24b365375834 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:13.996328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973a92b-f286-4f75-8556-24b365375834 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:14.004830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973a92b-f286-4f75-8556-24b365375834 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:14.025771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:14.125988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465943da-416d-4475-8127-f020675e168a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:14.127239 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3044/6043] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:13 2022] POST /identity/v3/projects => generated 375 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:14.137195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465943da-416d-4475-8127-f020675e168a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:14.235578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a8372-2577-42c5-aa88-af39bcfb069a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:14.236948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a8372-2577-42c5-aa88-af39bcfb069a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:14.236948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a8372-2577-42c5-aa88-af39bcfb069a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:14.237264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a8372-2577-42c5-aa88-af39bcfb069a admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:14.237987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a8372-2577-42c5-aa88-af39bcfb069a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:14.240039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a8372-2577-42c5-aa88-af39bcfb069a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:14.387230 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3000/6044] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:25:14 2022] POST /identity/v3/users => generated 288 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:25:14.391273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973a92b-f286-4f75-8556-24b365375834 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:14.421822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973a92b-f286-4f75-8556-24b365375834 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:14.432426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3c937-56ec-4410-9d99-0a96c57f0f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:14.433121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3c937-56ec-4410-9d99-0a96c57f0f0a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:14.433411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3c937-56ec-4410-9d99-0a96c57f0f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:14.433818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3c937-56ec-4410-9d99-0a96c57f0f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:14.434733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3c937-56ec-4410-9d99-0a96c57f0f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:14.437948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3c937-56ec-4410-9d99-0a96c57f0f0a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:14.459281 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3045/6045] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:25:14.463663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a8372-2577-42c5-aa88-af39bcfb069a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:14.478305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a8372-2577-42c5-aa88-af39bcfb069a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:14.491274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1613b6-245d-4896-a3f3-9b30838888a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:14.492571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1613b6-245d-4896-a3f3-9b30838888a2 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:14.499073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1613b6-245d-4896-a3f3-9b30838888a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:14.499746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1613b6-245d-4896-a3f3-9b30838888a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:14.501363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1613b6-245d-4896-a3f3-9b30838888a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:14.503396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1613b6-245d-4896-a3f3-9b30838888a2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:14.532334 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3001/6046] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:25:14.534686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3c937-56ec-4410-9d99-0a96c57f0f0a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:14.555519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3c937-56ec-4410-9d99-0a96c57f0f0a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:14.577138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f174ff-c5a2-414f-9584-da67384978b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:14.578095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f174ff-c5a2-414f-9584-da67384978b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:14.578377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f174ff-c5a2-414f-9584-da67384978b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:14.578659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f174ff-c5a2-414f-9584-da67384978b5 admin admin] PATH_INFO: `/v3/projects/d4c1fb1cc3aa4137af0e776e411c7663/users/ad41dd50aa5740ae8e3ff104c162c24b/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:14.628086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f174ff-c5a2-414f-9584-da67384978b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4c1fb1cc3aa4137af0e776e411c7663, user_id=ad41dd50aa5740ae8e3ff104c162c24b, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:14.631349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f174ff-c5a2-414f-9584-da67384978b5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:14.733937 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3046/6047] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:25:14 2022] PUT /identity/v3/projects/d4c1fb1cc3aa4137af0e776e411c7663/users/ad41dd50aa5740ae8e3ff104c162c24b/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:14.744758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46429e7-6ffd-4ad4-9cf5-255b61498cb7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:14.746158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46429e7-6ffd-4ad4-9cf5-255b61498cb7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:14.746917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46429e7-6ffd-4ad4-9cf5-255b61498cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:14.806707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a46429e7-6ffd-4ad4-9cf5-255b61498cb7 None None] MFA Rules not processed for user `ad41dd50aa5740ae8e3ff104c162c24b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:14.808190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a46429e7-6ffd-4ad4-9cf5-255b61498cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:15.000221 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3002/6048] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:15.001535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f174ff-c5a2-414f-9584-da67384978b5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:15.112864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f174ff-c5a2-414f-9584-da67384978b5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:15.125463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca9369-703a-4529-8714-d37ea8309104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:15.126629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca9369-703a-4529-8714-d37ea8309104 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:15.127508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca9369-703a-4529-8714-d37ea8309104 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:15.127738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca9369-703a-4529-8714-d37ea8309104 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:15.128531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca9369-703a-4529-8714-d37ea8309104 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:15.130926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca9369-703a-4529-8714-d37ea8309104 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:15.245255 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3047/6049] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:15 2022] POST /identity/v3/projects => generated 375 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:15.250058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46429e7-6ffd-4ad4-9cf5-255b61498cb7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:15.265284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46429e7-6ffd-4ad4-9cf5-255b61498cb7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:15.369687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fa6c1-8a51-4938-9b37-5b92c085df74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:15.373374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fa6c1-8a51-4938-9b37-5b92c085df74 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:15.373927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fa6c1-8a51-4938-9b37-5b92c085df74 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:15.374534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3fa6c1-8a51-4938-9b37-5b92c085df74 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:15.375537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3fa6c1-8a51-4938-9b37-5b92c085df74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:15.378042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3fa6c1-8a51-4938-9b37-5b92c085df74 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:15.485809 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3003/6050] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:25:15 2022] POST /identity/v3/users => generated 294 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:25:15.495553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca9369-703a-4529-8714-d37ea8309104 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:15.504407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca9369-703a-4529-8714-d37ea8309104 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:15.512182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66c55c-69e2-4e3c-b862-5e3905bf7142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:15.514461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66c55c-69e2-4e3c-b862-5e3905bf7142 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:15.515087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66c55c-69e2-4e3c-b862-5e3905bf7142 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:15.515710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66c55c-69e2-4e3c-b862-5e3905bf7142 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:15.516872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66c55c-69e2-4e3c-b862-5e3905bf7142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:15.523370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b66c55c-69e2-4e3c-b862-5e3905bf7142 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:15.535256 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3048/6051] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:25:15.540136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fa6c1-8a51-4938-9b37-5b92c085df74 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:15.555044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3fa6c1-8a51-4938-9b37-5b92c085df74 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:15.573217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30efde2-640f-4509-a062-f2b2404c6915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:15.575590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30efde2-640f-4509-a062-f2b2404c6915 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:15.575590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30efde2-640f-4509-a062-f2b2404c6915 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:15.575590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30efde2-640f-4509-a062-f2b2404c6915 admin admin] PATH_INFO: `/v3/projects/6568c92cbdaa43cfaafa98fdf660fa62/users/3d4404251e0c4c5b966e73a9431b84ee/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:15.637087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30efde2-640f-4509-a062-f2b2404c6915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6568c92cbdaa43cfaafa98fdf660fa62, user_id=3d4404251e0c4c5b966e73a9431b84ee, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:15.643440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30efde2-640f-4509-a062-f2b2404c6915 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:15.730404 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3004/6052] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:25:15 2022] PUT /identity/v3/projects/6568c92cbdaa43cfaafa98fdf660fa62/users/3d4404251e0c4c5b966e73a9431b84ee/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:15.734638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66c55c-69e2-4e3c-b862-5e3905bf7142 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:15.752497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b66c55c-69e2-4e3c-b862-5e3905bf7142 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:15.878257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa34d6-ea99-4995-9e31-c77a4e471375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:15.879220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa34d6-ea99-4995-9e31-c77a4e471375 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:15.879692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa34d6-ea99-4995-9e31-c77a4e471375 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:15.880148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa34d6-ea99-4995-9e31-c77a4e471375 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:15.881917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa34d6-ea99-4995-9e31-c77a4e471375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:15.884520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa34d6-ea99-4995-9e31-c77a4e471375 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:15.898932 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3049/6053] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:25:15.905915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30efde2-640f-4509-a062-f2b2404c6915 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:15.932389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30efde2-640f-4509-a062-f2b2404c6915 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:15.938796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1274330-fc24-417b-96b6-4c65ae93f3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:15.940006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1274330-fc24-417b-96b6-4c65ae93f3a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:15.940589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1274330-fc24-417b-96b6-4c65ae93f3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:15.941166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1274330-fc24-417b-96b6-4c65ae93f3a0 admin admin] PATH_INFO: `/v3/projects/6568c92cbdaa43cfaafa98fdf660fa62/users/3d4404251e0c4c5b966e73a9431b84ee/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:15.943848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1274330-fc24-417b-96b6-4c65ae93f3a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6568c92cbdaa43cfaafa98fdf660fa62, user_id=3d4404251e0c4c5b966e73a9431b84ee, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:15.947913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1274330-fc24-417b-96b6-4c65ae93f3a0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:15.975939 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3005/6054] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:25:15 2022] PUT /identity/v3/projects/6568c92cbdaa43cfaafa98fdf660fa62/users/3d4404251e0c4c5b966e73a9431b84ee/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:15.986352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a188e99-28fb-4235-bff9-4abd9e460635 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:15.987657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a188e99-28fb-4235-bff9-4abd9e460635 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:15.988440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a188e99-28fb-4235-bff9-4abd9e460635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:16.063139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8a188e99-28fb-4235-bff9-4abd9e460635 None None] MFA Rules not processed for user `3d4404251e0c4c5b966e73a9431b84ee`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:16.066898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8a188e99-28fb-4235-bff9-4abd9e460635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:16.215059 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3050/6055] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:16.219616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1274330-fc24-417b-96b6-4c65ae93f3a0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:16.232292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:16.237186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1274330-fc24-417b-96b6-4c65ae93f3a0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:16.246759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7002b-3294-49b5-8d77-743693401970 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4404251e0c4c5b966e73a9431b84ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6568c92cbdaa43cfaafa98fdf660fa62', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:16.247596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7002b-3294-49b5-8d77-743693401970 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:16.248021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7002b-3294-49b5-8d77-743693401970 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:16.248308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7002b-3294-49b5-8d77-743693401970 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:16.249012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7002b-3294-49b5-8d77-743693401970 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:16.250919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7002b-3294-49b5-8d77-743693401970 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:16.384186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a188e99-28fb-4235-bff9-4abd9e460635 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:16.390148 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3006/6056] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:16 2022] POST /identity/v3/projects => generated 350 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:25:16.399454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a188e99-28fb-4235-bff9-4abd9e460635 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:16.470008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:16.500790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b045fc-c5a7-46a7-bc6d-0e35522dea32 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:16.501306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b045fc-c5a7-46a7-bc6d-0e35522dea32 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:16.501854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b045fc-c5a7-46a7-bc6d-0e35522dea32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:16.523441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119ff1b-f7cf-47cb-a768-f6dd87d1459d tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4404251e0c4c5b966e73a9431b84ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6568c92cbdaa43cfaafa98fdf660fa62', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:16.523441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119ff1b-f7cf-47cb-a768-f6dd87d1459d tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:16.523441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119ff1b-f7cf-47cb-a768-f6dd87d1459d tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:16.523755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119ff1b-f7cf-47cb-a768-f6dd87d1459d tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:16.523755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119ff1b-f7cf-47cb-a768-f6dd87d1459d tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:16.525226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-60b045fc-c5a7-46a7-bc6d-0e35522dea32 None None] MFA Rules not processed for user `ebc8e00965334bd1bb811a0eefd64b84`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:16.527017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-60b045fc-c5a7-46a7-bc6d-0e35522dea32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:16.527167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119ff1b-f7cf-47cb-a768-f6dd87d1459d tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:16.612032 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3007/6057] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:16.613108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b045fc-c5a7-46a7-bc6d-0e35522dea32 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:16.614484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-60b045fc-c5a7-46a7-bc6d-0e35522dea32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:16.684712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b045fc-c5a7-46a7-bc6d-0e35522dea32 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:16.689876 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3051/6058] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:16 2022] POST /identity/v3/projects => generated 350 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:16.694103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119ff1b-f7cf-47cb-a768-f6dd87d1459d tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:16.806838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6f7483-34db-4b5d-9285-779ec77896dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:16.807776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f7483-34db-4b5d-9285-779ec77896dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:16.810340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f7483-34db-4b5d-9285-779ec77896dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:16.810816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6f7483-34db-4b5d-9285-779ec77896dd admin admin] PATH_INFO: `/v3/users/c6be46d2745c45729e2b105052747a02` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:16.816424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f7483-34db-4b5d-9285-779ec77896dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6be46d2745c45729e2b105052747a02)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:16.816424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6f7483-34db-4b5d-9285-779ec77896dd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:16.849063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119ff1b-f7cf-47cb-a768-f6dd87d1459d tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:16.875365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831a366-8a47-413d-89cb-66edd71c9c9a tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4404251e0c4c5b966e73a9431b84ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6568c92cbdaa43cfaafa98fdf660fa62', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:16.877110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831a366-8a47-413d-89cb-66edd71c9c9a tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:16.877738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831a366-8a47-413d-89cb-66edd71c9c9a tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:16.878375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e831a366-8a47-413d-89cb-66edd71c9c9a tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] PATH_INFO: `/v3/users/ad41dd50aa5740ae8e3ff104c162c24b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:16.880184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831a366-8a47-413d-89cb-66edd71c9c9a tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorizing `identity:get_user(user_id=ad41dd50aa5740ae8e3ff104c162c24b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:16.884804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e831a366-8a47-413d-89cb-66edd71c9c9a tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:16.889022 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3052/6059] 10.222.0.33 () {58 vars in 1220 bytes} [Sat May 28 21:25:16 2022] GET /identity/v3/users/ad41dd50aa5740ae8e3ff104c162c24b => generated 288 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:25:16.893344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831a366-8a47-413d-89cb-66edd71c9c9a tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:16.907976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e831a366-8a47-413d-89cb-66edd71c9c9a tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:16.913695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7b8f3-a601-49bf-a786-104dbd33fcf7 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4404251e0c4c5b966e73a9431b84ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6568c92cbdaa43cfaafa98fdf660fa62', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:16.915067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7b8f3-a601-49bf-a786-104dbd33fcf7 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:16.915437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7b8f3-a601-49bf-a786-104dbd33fcf7 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:16.915862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b7b8f3-a601-49bf-a786-104dbd33fcf7 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] PATH_INFO: `/v3/credentials` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:16.917671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7b8f3-a601-49bf-a786-104dbd33fcf7 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:16.920193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b7b8f3-a601-49bf-a786-104dbd33fcf7 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:16.999934 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3053/6060] 10.222.0.33 () {60 vars in 1161 bytes} [Sat May 28 21:25:16 2022] POST /identity/v3/credentials => generated 431 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:17.001242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7b8f3-a601-49bf-a786-104dbd33fcf7 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:17.015456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b7b8f3-a601-49bf-a786-104dbd33fcf7 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:17.025348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc191c0-21c0-44d7-a454-2157a1e67656 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4404251e0c4c5b966e73a9431b84ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6568c92cbdaa43cfaafa98fdf660fa62', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:17.026740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc191c0-21c0-44d7-a454-2157a1e67656 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:17.027419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc191c0-21c0-44d7-a454-2157a1e67656 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:17.027943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc191c0-21c0-44d7-a454-2157a1e67656 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] PATH_INFO: `/v3/credentials/5ae7eeb3fbdbc7e2053d651ae2bac9f457c8bd974332aea2acdc89578fad9b64` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:17.032318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc191c0-21c0-44d7-a454-2157a1e67656 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=5ae7eeb3fbdbc7e2053d651ae2bac9f457c8bd974332aea2acdc89578fad9b64)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:17.034459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc191c0-21c0-44d7-a454-2157a1e67656 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:17.037668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc191c0-21c0-44d7-a454-2157a1e67656 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorizing `identity:update_credential(credential_id=5ae7eeb3fbdbc7e2053d651ae2bac9f457c8bd974332aea2acdc89578fad9b64)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:17.039822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc191c0-21c0-44d7-a454-2157a1e67656 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:17.044661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb6f7483-34db-4b5d-9285-779ec77896dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6be46d2745c45729e2b105052747a02'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:17.045365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb6f7483-34db-4b5d-9285-779ec77896dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6be46d2745c45729e2b105052747a02'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:17.081258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb6f7483-34db-4b5d-9285-779ec77896dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6be46d2745c45729e2b105052747a02'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:17.131315 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3054/6061] 10.222.0.33 () {60 vars in 1357 bytes} [Sat May 28 21:25:17 2022] PATCH /identity/v3/credentials/5ae7eeb3fbdbc7e2053d651ae2bac9f457c8bd974332aea2acdc89578fad9b64 => generated 437 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:17.136227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc191c0-21c0-44d7-a454-2157a1e67656 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:17.147645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc191c0-21c0-44d7-a454-2157a1e67656 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:17.155212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff677a-3312-4607-8d1b-ed88032cb6c2 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4404251e0c4c5b966e73a9431b84ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6568c92cbdaa43cfaafa98fdf660fa62', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:17.156876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff677a-3312-4607-8d1b-ed88032cb6c2 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:17.157391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff677a-3312-4607-8d1b-ed88032cb6c2 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:17.157939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff677a-3312-4607-8d1b-ed88032cb6c2 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] PATH_INFO: `/v3/credentials/5ae7eeb3fbdbc7e2053d651ae2bac9f457c8bd974332aea2acdc89578fad9b64` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:17.160925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff677a-3312-4607-8d1b-ed88032cb6c2 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorizing `identity:get_credential(credential_id=5ae7eeb3fbdbc7e2053d651ae2bac9f457c8bd974332aea2acdc89578fad9b64)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:17.163681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb6f7483-34db-4b5d-9285-779ec77896dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6be46d2745c45729e2b105052747a02'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:17.164523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff677a-3312-4607-8d1b-ed88032cb6c2 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:17.165751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb6f7483-34db-4b5d-9285-779ec77896dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6be46d2745c45729e2b105052747a02'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:17.168618 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3055/6062] 10.222.0.33 () {58 vars in 1334 bytes} [Sat May 28 21:25:17 2022] GET /identity/v3/credentials/5ae7eeb3fbdbc7e2053d651ae2bac9f457c8bd974332aea2acdc89578fad9b64 => generated 437 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:17.174334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff677a-3312-4607-8d1b-ed88032cb6c2 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:17.177762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-12ff677a-3312-4607-8d1b-ed88032cb6c2 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:17.208257 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3008/6063] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:16 2022] DELETE /identity/v3/users/c6be46d2745c45729e2b105052747a02 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:17.211557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a91d7-054f-4df1-9b4a-6c039ee617db None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:17.213614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff677a-3312-4607-8d1b-ed88032cb6c2 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:17.214235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a91d7-054f-4df1-9b4a-6c039ee617db None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:17.215253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614a91d7-054f-4df1-9b4a-6c039ee617db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:17.261880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-614a91d7-054f-4df1-9b4a-6c039ee617db None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:17.265625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-614a91d7-054f-4df1-9b4a-6c039ee617db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:17.349666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84e862-8c41-43d5-a8a7-2f22414bf7d1 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4404251e0c4c5b966e73a9431b84ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6568c92cbdaa43cfaafa98fdf660fa62', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:17.350714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84e862-8c41-43d5-a8a7-2f22414bf7d1 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:17.352360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84e862-8c41-43d5-a8a7-2f22414bf7d1 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:17.352360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84e862-8c41-43d5-a8a7-2f22414bf7d1 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] PATH_INFO: `/v3/credentials/5ae7eeb3fbdbc7e2053d651ae2bac9f457c8bd974332aea2acdc89578fad9b64` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:17.354620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84e862-8c41-43d5-a8a7-2f22414bf7d1 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=5ae7eeb3fbdbc7e2053d651ae2bac9f457c8bd974332aea2acdc89578fad9b64)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:17.356693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84e862-8c41-43d5-a8a7-2f22414bf7d1 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:17.440949 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3009/6064] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:17.451088 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3056/6065] 10.222.0.33 () {58 vars in 1337 bytes} [Sat May 28 21:25:17 2022] DELETE /identity/v3/credentials/5ae7eeb3fbdbc7e2053d651ae2bac9f457c8bd974332aea2acdc89578fad9b64 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:25:17.457233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a91d7-054f-4df1-9b4a-6c039ee617db None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:17.464038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84e862-8c41-43d5-a8a7-2f22414bf7d1 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:17.469372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7f84e862-8c41-43d5-a8a7-2f22414bf7d1 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:17.480490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614a91d7-054f-4df1-9b4a-6c039ee617db None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:17.500625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a8ed1-7e59-4791-af25-ea683a917126 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4404251e0c4c5b966e73a9431b84ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6568c92cbdaa43cfaafa98fdf660fa62', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:17.501273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a8ed1-7e59-4791-af25-ea683a917126 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:17.501569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a8ed1-7e59-4791-af25-ea683a917126 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:17.503094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54a8ed1-7e59-4791-af25-ea683a917126 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] PATH_INFO: `/v3/credentials` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:17.504315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a8ed1-7e59-4791-af25-ea683a917126 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:17.506665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54a8ed1-7e59-4791-af25-ea683a917126 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:17.584059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84e862-8c41-43d5-a8a7-2f22414bf7d1 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:17.599424 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3010/6066] 10.222.0.33 () {60 vars in 1161 bytes} [Sat May 28 21:25:17 2022] POST /identity/v3/credentials => generated 430 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:17.603427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a8ed1-7e59-4791-af25-ea683a917126 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:17.610075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982f1bd-5324-43aa-8c76-ccf956cd0294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:17.610362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982f1bd-5324-43aa-8c76-ccf956cd0294 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:17.610654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982f1bd-5324-43aa-8c76-ccf956cd0294 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:17.610935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982f1bd-5324-43aa-8c76-ccf956cd0294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:17.614942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54a8ed1-7e59-4791-af25-ea683a917126 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:17.622720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c31f7-54dd-4019-90b3-142f88f68b84 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4404251e0c4c5b966e73a9431b84ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6568c92cbdaa43cfaafa98fdf660fa62', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:17.622720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c31f7-54dd-4019-90b3-142f88f68b84 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:17.623028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c31f7-54dd-4019-90b3-142f88f68b84 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:17.623234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c31f7-54dd-4019-90b3-142f88f68b84 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] PATH_INFO: `/v3/credentials` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:17.624229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c31f7-54dd-4019-90b3-142f88f68b84 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorizing `identity:create_credential()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:17.630295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3c31f7-54dd-4019-90b3-142f88f68b84 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:17.640381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982f1bd-5324-43aa-8c76-ccf956cd0294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:17.641952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982f1bd-5324-43aa-8c76-ccf956cd0294 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:17.659874 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3057/6067] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:25:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:25:17.731446 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3011/6068] 10.222.0.33 () {60 vars in 1161 bytes} [Sat May 28 21:25:17 2022] POST /identity/v3/credentials => generated 431 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:17.731663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982f1bd-5324-43aa-8c76-ccf956cd0294 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:17.744403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982f1bd-5324-43aa-8c76-ccf956cd0294 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:17.750731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659d212-e9e1-4272-bd38-0a2acdaf7c10 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4404251e0c4c5b966e73a9431b84ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6568c92cbdaa43cfaafa98fdf660fa62', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:17.751611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659d212-e9e1-4272-bd38-0a2acdaf7c10 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:17.752062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659d212-e9e1-4272-bd38-0a2acdaf7c10 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:17.752349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659d212-e9e1-4272-bd38-0a2acdaf7c10 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] PATH_INFO: `/v3/credentials` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:17.753063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659d212-e9e1-4272-bd38-0a2acdaf7c10 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorizing `identity:list_credentials()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:17.754672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659d212-e9e1-4272-bd38-0a2acdaf7c10 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:17.767538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659d212-e9e1-4272-bd38-0a2acdaf7c10 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:17.769335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659d212-e9e1-4272-bd38-0a2acdaf7c10 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:17.772897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659d212-e9e1-4272-bd38-0a2acdaf7c10 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorizing `identity:get_credential()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:17.775870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b659d212-e9e1-4272-bd38-0a2acdaf7c10 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:17.781929 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3058/6069] 10.222.0.33 () {58 vars in 1139 bytes} [Sat May 28 21:25:17 2022] GET /identity/v3/credentials => generated 946 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:25:17.782415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c31f7-54dd-4019-90b3-142f88f68b84 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:17.793986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:17.802422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3c31f7-54dd-4019-90b3-142f88f68b84 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:17.808260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba486bdc-266d-4661-b64c-a1d2f64e4711 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4404251e0c4c5b966e73a9431b84ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6568c92cbdaa43cfaafa98fdf660fa62', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:17.809227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba486bdc-266d-4661-b64c-a1d2f64e4711 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:17.809496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba486bdc-266d-4661-b64c-a1d2f64e4711 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:17.809779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba486bdc-266d-4661-b64c-a1d2f64e4711 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] PATH_INFO: `/v3/credentials/35d7739a2b05f54c204c522f1f555c7c583f2e257567a04b55f966cefd4744bc` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:17.815434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba486bdc-266d-4661-b64c-a1d2f64e4711 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=35d7739a2b05f54c204c522f1f555c7c583f2e257567a04b55f966cefd4744bc)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:17.817240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba486bdc-266d-4661-b64c-a1d2f64e4711 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:17.850214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:17.955712 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3012/6070] 10.222.0.33 () {58 vars in 1337 bytes} [Sat May 28 21:25:17 2022] DELETE /identity/v3/credentials/35d7739a2b05f54c204c522f1f555c7c583f2e257567a04b55f966cefd4744bc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:25:17.960867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659d212-e9e1-4272-bd38-0a2acdaf7c10 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:17.987443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659d212-e9e1-4272-bd38-0a2acdaf7c10 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:17.998191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0d96c-dfb5-446d-ba7f-67f352543a8f tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4404251e0c4c5b966e73a9431b84ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6568c92cbdaa43cfaafa98fdf660fa62', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:17.999647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0d96c-dfb5-446d-ba7f-67f352543a8f tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:18.000177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0d96c-dfb5-446d-ba7f-67f352543a8f tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:18.000687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d0d96c-dfb5-446d-ba7f-67f352543a8f tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] PATH_INFO: `/v3/credentials/7a5a945d7c1ce6bf7b89a9b55a555a78770beb1f0510e9759ff77760a1554246` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:18.003069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0d96c-dfb5-446d-ba7f-67f352543a8f tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorizing `identity:delete_credential(credential_id=7a5a945d7c1ce6bf7b89a9b55a555a78770beb1f0510e9759ff77760a1554246)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:18.005227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d0d96c-dfb5-446d-ba7f-67f352543a8f tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:18.143635 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3059/6071] 10.222.0.33 () {58 vars in 1337 bytes} [Sat May 28 21:25:17 2022] DELETE /identity/v3/credentials/7a5a945d7c1ce6bf7b89a9b55a555a78770beb1f0510e9759ff77760a1554246 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:18.144056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba486bdc-266d-4661-b64c-a1d2f64e4711 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:18.185790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba486bdc-266d-4661-b64c-a1d2f64e4711 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:18.203781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e627b02-f8c5-4c36-83ee-1a5c64d96150 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4404251e0c4c5b966e73a9431b84ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6568c92cbdaa43cfaafa98fdf660fa62', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:18.205225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e627b02-f8c5-4c36-83ee-1a5c64d96150 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:18.205521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e627b02-f8c5-4c36-83ee-1a5c64d96150 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:18.205808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e627b02-f8c5-4c36-83ee-1a5c64d96150 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] PATH_INFO: `/v3/projects/900ab4e2404d45f6925692dbf6ccef07` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:18.208070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e627b02-f8c5-4c36-83ee-1a5c64d96150 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorizing `identity:delete_project(project_id=900ab4e2404d45f6925692dbf6ccef07)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:18.210382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e627b02-f8c5-4c36-83ee-1a5c64d96150 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:18.396530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e627b02-f8c5-4c36-83ee-1a5c64d96150 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '900ab4e2404d45f6925692dbf6ccef07', 'request_id': 'req-7e627b02-f8c5-4c36-83ee-1a5c64d96150'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:18.429525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e627b02-f8c5-4c36-83ee-1a5c64d96150 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '900ab4e2404d45f6925692dbf6ccef07', 'request_id': 'req-7e627b02-f8c5-4c36-83ee-1a5c64d96150'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:18.500640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e627b02-f8c5-4c36-83ee-1a5c64d96150 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '900ab4e2404d45f6925692dbf6ccef07', 'request_id': 'req-7e627b02-f8c5-4c36-83ee-1a5c64d96150'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:18.518855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e627b02-f8c5-4c36-83ee-1a5c64d96150 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] The token cache is being invalidate because project 900ab4e2404d45f6925692dbf6ccef07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:18.519159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e627b02-f8c5-4c36-83ee-1a5c64d96150 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:18.520549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e627b02-f8c5-4c36-83ee-1a5c64d96150 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:18.523160 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3013/6072] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:18 2022] DELETE /identity/v3/projects/900ab4e2404d45f6925692dbf6ccef07 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:25:18.527356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0d96c-dfb5-446d-ba7f-67f352543a8f tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:18.529305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-29d0d96c-dfb5-446d-ba7f-67f352543a8f tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:18.550699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d0d96c-dfb5-446d-ba7f-67f352543a8f tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:18.654827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e3121-893b-4e48-a3cc-b7959e304fa6 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d4404251e0c4c5b966e73a9431b84ee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6568c92cbdaa43cfaafa98fdf660fa62', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:18.655813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e3121-893b-4e48-a3cc-b7959e304fa6 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:18.656250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e3121-893b-4e48-a3cc-b7959e304fa6 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:18.656476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282e3121-893b-4e48-a3cc-b7959e304fa6 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] PATH_INFO: `/v3/projects/9b4dcdc458874165924bcc47d8073efe` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:18.657829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e3121-893b-4e48-a3cc-b7959e304fa6 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9b4dcdc458874165924bcc47d8073efe)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:18.667860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282e3121-893b-4e48-a3cc-b7959e304fa6 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:18.702131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e627b02-f8c5-4c36-83ee-1a5c64d96150 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:18.703574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7e627b02-f8c5-4c36-83ee-1a5c64d96150 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:18.826535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-282e3121-893b-4e48-a3cc-b7959e304fa6 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b4dcdc458874165924bcc47d8073efe', 'request_id': 'req-282e3121-893b-4e48-a3cc-b7959e304fa6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:18.845093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-282e3121-893b-4e48-a3cc-b7959e304fa6 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b4dcdc458874165924bcc47d8073efe', 'request_id': 'req-282e3121-893b-4e48-a3cc-b7959e304fa6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:18.870706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e627b02-f8c5-4c36-83ee-1a5c64d96150 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:18.928788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-282e3121-893b-4e48-a3cc-b7959e304fa6 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b4dcdc458874165924bcc47d8073efe', 'request_id': 'req-282e3121-893b-4e48-a3cc-b7959e304fa6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:18.953387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-282e3121-893b-4e48-a3cc-b7959e304fa6 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] The token cache is being invalidate because project 9b4dcdc458874165924bcc47d8073efe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:18.953700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-282e3121-893b-4e48-a3cc-b7959e304fa6 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:18.954323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe2bc0-7604-426c-9dd7-8cbeb1e6c2a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:18.954520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-282e3121-893b-4e48-a3cc-b7959e304fa6 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:18.955451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe2bc0-7604-426c-9dd7-8cbeb1e6c2a5 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:18.955953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe2bc0-7604-426c-9dd7-8cbeb1e6c2a5 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:18.956424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe2bc0-7604-426c-9dd7-8cbeb1e6c2a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:18.959452 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3060/6073] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:18 2022] DELETE /identity/v3/projects/9b4dcdc458874165924bcc47d8073efe => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:18.961103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e3121-893b-4e48-a3cc-b7959e304fa6 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:18.961281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-73fe2bc0-7604-426c-9dd7-8cbeb1e6c2a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:18.964704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-282e3121-893b-4e48-a3cc-b7959e304fa6 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:18.989251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe2bc0-7604-426c-9dd7-8cbeb1e6c2a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:18.990914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fe2bc0-7604-426c-9dd7-8cbeb1e6c2a5 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:19.010924 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3014/6074] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:25:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:25:19.011879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe2bc0-7604-426c-9dd7-8cbeb1e6c2a5 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:19.022237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-73fe2bc0-7604-426c-9dd7-8cbeb1e6c2a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:19.069008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:19.124117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282e3121-893b-4e48-a3cc-b7959e304fa6 tempest-CredentialsTestJSON-1989897446 tempest-CredentialsTestJSON-1989897446-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:19.139723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f7127-1f4d-4f35-b331-caac040cc42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:19.145133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f7127-1f4d-4f35-b331-caac040cc42c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:19.145682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f7127-1f4d-4f35-b331-caac040cc42c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:19.146271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f7127-1f4d-4f35-b331-caac040cc42c admin admin] PATH_INFO: `/v3/projects/52102c097d7b4a81aa28fe0007377926` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:19.148740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f7127-1f4d-4f35-b331-caac040cc42c admin admin] RBAC: Authorizing `identity:delete_project(project_id=52102c097d7b4a81aa28fe0007377926)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:19.154426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f7127-1f4d-4f35-b331-caac040cc42c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:19.167415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe2bc0-7604-426c-9dd7-8cbeb1e6c2a5 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:19.174756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec83a2-80ea-4901-ae7d-f55543da758e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:19.175999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec83a2-80ea-4901-ae7d-f55543da758e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:19.176318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec83a2-80ea-4901-ae7d-f55543da758e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:19.176589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec83a2-80ea-4901-ae7d-f55543da758e admin admin] PATH_INFO: `/v3/users/ad41dd50aa5740ae8e3ff104c162c24b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:19.177980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec83a2-80ea-4901-ae7d-f55543da758e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad41dd50aa5740ae8e3ff104c162c24b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:19.180749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec83a2-80ea-4901-ae7d-f55543da758e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:19.303584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e14f7127-1f4d-4f35-b331-caac040cc42c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52102c097d7b4a81aa28fe0007377926', 'request_id': 'req-e14f7127-1f4d-4f35-b331-caac040cc42c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:19.310449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e14f7127-1f4d-4f35-b331-caac040cc42c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52102c097d7b4a81aa28fe0007377926', 'request_id': 'req-e14f7127-1f4d-4f35-b331-caac040cc42c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:19.391774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e14f7127-1f4d-4f35-b331-caac040cc42c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52102c097d7b4a81aa28fe0007377926', 'request_id': 'req-e14f7127-1f4d-4f35-b331-caac040cc42c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:19.409406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e14f7127-1f4d-4f35-b331-caac040cc42c admin admin] The token cache is being invalidate because project 52102c097d7b4a81aa28fe0007377926 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:19.409705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e14f7127-1f4d-4f35-b331-caac040cc42c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:19.410590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e14f7127-1f4d-4f35-b331-caac040cc42c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:19.412177 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3061/6075] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:18 2022] DELETE /identity/v3/projects/52102c097d7b4a81aa28fe0007377926 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:19.416016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f7127-1f4d-4f35-b331-caac040cc42c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:19.423167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0ec83a2-80ea-4901-ae7d-f55543da758e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad41dd50aa5740ae8e3ff104c162c24b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:19.424247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0ec83a2-80ea-4901-ae7d-f55543da758e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad41dd50aa5740ae8e3ff104c162c24b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:19.426283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e14f7127-1f4d-4f35-b331-caac040cc42c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:19.440892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0ec83a2-80ea-4901-ae7d-f55543da758e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad41dd50aa5740ae8e3ff104c162c24b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:19.493985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0ec83a2-80ea-4901-ae7d-f55543da758e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad41dd50aa5740ae8e3ff104c162c24b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:19.508294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0ec83a2-80ea-4901-ae7d-f55543da758e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad41dd50aa5740ae8e3ff104c162c24b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:19.540676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:19.549160 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3015/6076] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:19 2022] DELETE /identity/v3/users/ad41dd50aa5740ae8e3ff104c162c24b => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:19.558938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be0f46-678a-4365-9151-4534dcb502aa None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:19.558938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be0f46-678a-4365-9151-4534dcb502aa None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:19.558938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be0f46-678a-4365-9151-4534dcb502aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:19.578784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-94be0f46-678a-4365-9151-4534dcb502aa None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:19.580306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-94be0f46-678a-4365-9151-4534dcb502aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:19.582935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f7127-1f4d-4f35-b331-caac040cc42c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:19.671089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1727504f-071b-4d73-9599-fffe1a8a67d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:19.672207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727504f-071b-4d73-9599-fffe1a8a67d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:19.672758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727504f-071b-4d73-9599-fffe1a8a67d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:19.673278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1727504f-071b-4d73-9599-fffe1a8a67d5 admin admin] PATH_INFO: `/v3/users/fca31bf977974d349e713cd3b6c09f27` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:19.675422 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3016/6077] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:19.675723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727504f-071b-4d73-9599-fffe1a8a67d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fca31bf977974d349e713cd3b6c09f27)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:19.677910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1727504f-071b-4d73-9599-fffe1a8a67d5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:19.687632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be0f46-678a-4365-9151-4534dcb502aa None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:19.689901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-94be0f46-678a-4365-9151-4534dcb502aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:19.784902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be0f46-678a-4365-9151-4534dcb502aa None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:19.791790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411eca86-8247-4ac2-b2af-61f9de1b83f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:19.791964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411eca86-8247-4ac2-b2af-61f9de1b83f8 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:19.792263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411eca86-8247-4ac2-b2af-61f9de1b83f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:19.792709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411eca86-8247-4ac2-b2af-61f9de1b83f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:19.810497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411eca86-8247-4ac2-b2af-61f9de1b83f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:19.812697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411eca86-8247-4ac2-b2af-61f9de1b83f8 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:19.829785 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3017/6078] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:25:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:25:20.122418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1727504f-071b-4d73-9599-fffe1a8a67d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fca31bf977974d349e713cd3b6c09f27'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:20.141105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1727504f-071b-4d73-9599-fffe1a8a67d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fca31bf977974d349e713cd3b6c09f27'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:20.142051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1727504f-071b-4d73-9599-fffe1a8a67d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fca31bf977974d349e713cd3b6c09f27'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:20.143895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1727504f-071b-4d73-9599-fffe1a8a67d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fca31bf977974d349e713cd3b6c09f27'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:20.232385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1727504f-071b-4d73-9599-fffe1a8a67d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fca31bf977974d349e713cd3b6c09f27'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:20.256495 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3062/6079] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:19 2022] DELETE /identity/v3/users/fca31bf977974d349e713cd3b6c09f27 => generated 0 bytes in 838 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:25:20.395973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411eca86-8247-4ac2-b2af-61f9de1b83f8 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:20.397496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-411eca86-8247-4ac2-b2af-61f9de1b83f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:20.415561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411eca86-8247-4ac2-b2af-61f9de1b83f8 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:20.528271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e5229-83fd-44de-a06b-73f9d02db517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:20.529147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e5229-83fd-44de-a06b-73f9d02db517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:20.529441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e5229-83fd-44de-a06b-73f9d02db517 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:20.529725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e5229-83fd-44de-a06b-73f9d02db517 admin admin] PATH_INFO: `/v3/projects/d4c1fb1cc3aa4137af0e776e411c7663` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:20.534646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e5229-83fd-44de-a06b-73f9d02db517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4c1fb1cc3aa4137af0e776e411c7663)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:20.535142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919e5229-83fd-44de-a06b-73f9d02db517 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:20.586189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:20.691469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:20.788642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-919e5229-83fd-44de-a06b-73f9d02db517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4c1fb1cc3aa4137af0e776e411c7663', 'request_id': 'req-919e5229-83fd-44de-a06b-73f9d02db517'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:20.836345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-919e5229-83fd-44de-a06b-73f9d02db517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4c1fb1cc3aa4137af0e776e411c7663', 'request_id': 'req-919e5229-83fd-44de-a06b-73f9d02db517'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:20.852809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f0f0a-7a54-4fca-85c0-87f372b978dd None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:20.853417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f0f0a-7a54-4fca-85c0-87f372b978dd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:20.853903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f0f0a-7a54-4fca-85c0-87f372b978dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:20.873092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-808f0f0a-7a54-4fca-85c0-87f372b978dd None None] MFA Rules not processed for user `ebc8e00965334bd1bb811a0eefd64b84`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:20.875913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-808f0f0a-7a54-4fca-85c0-87f372b978dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:20.933978 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3063/6080] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:25:20.952061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-919e5229-83fd-44de-a06b-73f9d02db517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4c1fb1cc3aa4137af0e776e411c7663', 'request_id': 'req-919e5229-83fd-44de-a06b-73f9d02db517'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:20.969428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-919e5229-83fd-44de-a06b-73f9d02db517 admin admin] The token cache is being invalidate because project d4c1fb1cc3aa4137af0e776e411c7663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:20.969725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-919e5229-83fd-44de-a06b-73f9d02db517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:20.972336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-919e5229-83fd-44de-a06b-73f9d02db517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:20.983210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f0f0a-7a54-4fca-85c0-87f372b978dd None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:20.984863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-808f0f0a-7a54-4fca-85c0-87f372b978dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:20.988747 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3018/6081] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:20 2022] DELETE /identity/v3/projects/d4c1fb1cc3aa4137af0e776e411c7663 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:20.999161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e5229-83fd-44de-a06b-73f9d02db517 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:21.002477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-919e5229-83fd-44de-a06b-73f9d02db517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:21.106554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f0f0a-7a54-4fca-85c0-87f372b978dd None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:21.119672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2ebb1-46dc-4621-a632-4be72a72aa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:21.119672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2ebb1-46dc-4621-a632-4be72a72aa45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:21.119672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2ebb1-46dc-4621-a632-4be72a72aa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:21.119672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2ebb1-46dc-4621-a632-4be72a72aa45 admin admin] PATH_INFO: `/v3/projects/78b4e47aaec146f8be53d9b3a26b1b6b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:21.122999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2ebb1-46dc-4621-a632-4be72a72aa45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78b4e47aaec146f8be53d9b3a26b1b6b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:21.126083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2ebb1-46dc-4621-a632-4be72a72aa45 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:21.160340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e5229-83fd-44de-a06b-73f9d02db517 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:21.165482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43268a93-12d0-48a4-b28a-2fd2f71350c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:21.166454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43268a93-12d0-48a4-b28a-2fd2f71350c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:21.166883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43268a93-12d0-48a4-b28a-2fd2f71350c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:21.167171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43268a93-12d0-48a4-b28a-2fd2f71350c0 admin admin] PATH_INFO: `/v3/users/3d4404251e0c4c5b966e73a9431b84ee` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:21.168446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43268a93-12d0-48a4-b28a-2fd2f71350c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d4404251e0c4c5b966e73a9431b84ee)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:21.170473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43268a93-12d0-48a4-b28a-2fd2f71350c0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:21.268746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f5e2ebb1-46dc-4621-a632-4be72a72aa45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78b4e47aaec146f8be53d9b3a26b1b6b', 'request_id': 'req-f5e2ebb1-46dc-4621-a632-4be72a72aa45'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:21.280460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f5e2ebb1-46dc-4621-a632-4be72a72aa45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78b4e47aaec146f8be53d9b3a26b1b6b', 'request_id': 'req-f5e2ebb1-46dc-4621-a632-4be72a72aa45'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:21.410346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f5e2ebb1-46dc-4621-a632-4be72a72aa45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78b4e47aaec146f8be53d9b3a26b1b6b', 'request_id': 'req-f5e2ebb1-46dc-4621-a632-4be72a72aa45'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:21.432530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f5e2ebb1-46dc-4621-a632-4be72a72aa45 admin admin] The token cache is being invalidate because project 78b4e47aaec146f8be53d9b3a26b1b6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:21.432859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f5e2ebb1-46dc-4621-a632-4be72a72aa45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:21.433267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-43268a93-12d0-48a4-b28a-2fd2f71350c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d4404251e0c4c5b966e73a9431b84ee'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:21.434675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f5e2ebb1-46dc-4621-a632-4be72a72aa45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:21.435215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-43268a93-12d0-48a4-b28a-2fd2f71350c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d4404251e0c4c5b966e73a9431b84ee'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:21.439933 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3064/6082] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:20 2022] DELETE /identity/v3/projects/78b4e47aaec146f8be53d9b3a26b1b6b => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:25:21.446098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40dfc6d-1b78-441e-bf52-2fa6f19e0621 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:21.446392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40dfc6d-1b78-441e-bf52-2fa6f19e0621 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:21.446723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40dfc6d-1b78-441e-bf52-2fa6f19e0621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:21.449350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-43268a93-12d0-48a4-b28a-2fd2f71350c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d4404251e0c4c5b966e73a9431b84ee'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:21.468602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f40dfc6d-1b78-441e-bf52-2fa6f19e0621 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:21.471295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f40dfc6d-1b78-441e-bf52-2fa6f19e0621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:21.546508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-43268a93-12d0-48a4-b28a-2fd2f71350c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d4404251e0c4c5b966e73a9431b84ee'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:21.548457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-43268a93-12d0-48a4-b28a-2fd2f71350c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d4404251e0c4c5b966e73a9431b84ee'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:21.553952 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3065/6083] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:21.567595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fd456-e25f-4de6-a206-3410c4e4ab84 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:21.568202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fd456-e25f-4de6-a206-3410c4e4ab84 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:21.568685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594fd456-e25f-4de6-a206-3410c4e4ab84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:21.596526 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3019/6084] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:20 2022] DELETE /identity/v3/users/3d4404251e0c4c5b966e73a9431b84ee => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:21.603331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-594fd456-e25f-4de6-a206-3410c4e4ab84 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:21.606121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-594fd456-e25f-4de6-a206-3410c4e4ab84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:21.705735 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3066/6085] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:21.707395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43268a93-12d0-48a4-b28a-2fd2f71350c0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:21.721239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43268a93-12d0-48a4-b28a-2fd2f71350c0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:21.728594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c476f95-bda5-495e-bf19-b6016160465f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:21.729275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c476f95-bda5-495e-bf19-b6016160465f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:21.729562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c476f95-bda5-495e-bf19-b6016160465f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:21.729863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c476f95-bda5-495e-bf19-b6016160465f admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:21.731142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c476f95-bda5-495e-bf19-b6016160465f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:25:21.731459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c476f95-bda5-495e-bf19-b6016160465f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:21.734452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c476f95-bda5-495e-bf19-b6016160465f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:21.751046 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3020/6086] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:21.758834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fd456-e25f-4de6-a206-3410c4e4ab84 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:21.768435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594fd456-e25f-4de6-a206-3410c4e4ab84 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:21.774703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841801e-dddc-49af-b210-f66f448cda9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:21.775367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841801e-dddc-49af-b210-f66f448cda9d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:21.775653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841801e-dddc-49af-b210-f66f448cda9d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:21.775935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841801e-dddc-49af-b210-f66f448cda9d admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:21.776636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841801e-dddc-49af-b210-f66f448cda9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:21.778615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a841801e-dddc-49af-b210-f66f448cda9d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:21.856779 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3067/6087] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:21 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:21.859459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c476f95-bda5-495e-bf19-b6016160465f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:21.871853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c476f95-bda5-495e-bf19-b6016160465f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:22.017605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4706b25-b12d-4a62-9446-ea79bad8c0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:22.018648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4706b25-b12d-4a62-9446-ea79bad8c0dd admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:22.019161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4706b25-b12d-4a62-9446-ea79bad8c0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:22.019589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4706b25-b12d-4a62-9446-ea79bad8c0dd admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:22.021766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4706b25-b12d-4a62-9446-ea79bad8c0dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:22.025380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4706b25-b12d-4a62-9446-ea79bad8c0dd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:22.169466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841801e-dddc-49af-b210-f66f448cda9d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:22.171767 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3021/6088] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:25:21 2022] POST /identity/v3/users => generated 290 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:22.185482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a841801e-dddc-49af-b210-f66f448cda9d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:22.186250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367d8d7-188b-4697-aa61-c6514dc4a4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:22.186993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367d8d7-188b-4697-aa61-c6514dc4a4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:22.187536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367d8d7-188b-4697-aa61-c6514dc4a4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:22.187814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367d8d7-188b-4697-aa61-c6514dc4a4c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:22.188804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367d8d7-188b-4697-aa61-c6514dc4a4c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:22.190666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367d8d7-188b-4697-aa61-c6514dc4a4c1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:22.206024 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3068/6089] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:25:22.206938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4706b25-b12d-4a62-9446-ea79bad8c0dd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:22.219740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4706b25-b12d-4a62-9446-ea79bad8c0dd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:22.228374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7cd96-3da2-489b-955f-a764d13f648f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:22.229068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7cd96-3da2-489b-955f-a764d13f648f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:22.229355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7cd96-3da2-489b-955f-a764d13f648f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:22.229633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7cd96-3da2-489b-955f-a764d13f648f admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:22.234078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7cd96-3da2-489b-955f-a764d13f648f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:22.246496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7cd96-3da2-489b-955f-a764d13f648f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:22.260882 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3022/6090] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:25:22.264772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367d8d7-188b-4697-aa61-c6514dc4a4c1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:22.277213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367d8d7-188b-4697-aa61-c6514dc4a4c1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:22.290731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab1f0b-0c8f-436a-8dde-9daf336e0a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:22.291774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab1f0b-0c8f-436a-8dde-9daf336e0a38 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:22.292063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab1f0b-0c8f-436a-8dde-9daf336e0a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:22.292720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab1f0b-0c8f-436a-8dde-9daf336e0a38 admin admin] PATH_INFO: `/v3/projects/f791936ea1fc49cfb07986f26a47cabe/users/2267c55067d549e79963425bfc21f45c/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:22.313388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab1f0b-0c8f-436a-8dde-9daf336e0a38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f791936ea1fc49cfb07986f26a47cabe, user_id=2267c55067d549e79963425bfc21f45c, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:22.316324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab1f0b-0c8f-436a-8dde-9daf336e0a38 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:22.320663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7cd96-3da2-489b-955f-a764d13f648f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:22.323272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b5e7cd96-3da2-489b-955f-a764d13f648f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:22.338511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7cd96-3da2-489b-955f-a764d13f648f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:22.344490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b942c-0fd9-4a21-8092-1328f3f775db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:22.345023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b942c-0fd9-4a21-8092-1328f3f775db admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:22.345315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b942c-0fd9-4a21-8092-1328f3f775db admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:22.345574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b942c-0fd9-4a21-8092-1328f3f775db admin admin] PATH_INFO: `/v3/projects/6568c92cbdaa43cfaafa98fdf660fa62` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:22.346840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b942c-0fd9-4a21-8092-1328f3f775db admin admin] RBAC: Authorizing `identity:delete_project(project_id=6568c92cbdaa43cfaafa98fdf660fa62)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:22.349384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559b942c-0fd9-4a21-8092-1328f3f775db admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:22.422192 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3069/6091] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:25:22 2022] PUT /identity/v3/projects/f791936ea1fc49cfb07986f26a47cabe/users/2267c55067d549e79963425bfc21f45c/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:22.434796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb57338-3b48-4df8-9b9d-1ee0db29f698 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:22.435524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb57338-3b48-4df8-9b9d-1ee0db29f698 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:22.436160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb57338-3b48-4df8-9b9d-1ee0db29f698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:22.475010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-dfb57338-3b48-4df8-9b9d-1ee0db29f698 None None] MFA Rules not processed for user `2267c55067d549e79963425bfc21f45c`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:22.477144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dfb57338-3b48-4df8-9b9d-1ee0db29f698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:22.566445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-559b942c-0fd9-4a21-8092-1328f3f775db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6568c92cbdaa43cfaafa98fdf660fa62', 'request_id': 'req-559b942c-0fd9-4a21-8092-1328f3f775db'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:22.574481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-559b942c-0fd9-4a21-8092-1328f3f775db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6568c92cbdaa43cfaafa98fdf660fa62', 'request_id': 'req-559b942c-0fd9-4a21-8092-1328f3f775db'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:22.620508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:22.637142 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3070/6092] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:25:22.650608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb57338-3b48-4df8-9b9d-1ee0db29f698 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:22.652358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dfb57338-3b48-4df8-9b9d-1ee0db29f698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:22.694777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-559b942c-0fd9-4a21-8092-1328f3f775db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6568c92cbdaa43cfaafa98fdf660fa62', 'request_id': 'req-559b942c-0fd9-4a21-8092-1328f3f775db'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:22.709964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-559b942c-0fd9-4a21-8092-1328f3f775db admin admin] The token cache is being invalidate because project 6568c92cbdaa43cfaafa98fdf660fa62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:22.710618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-559b942c-0fd9-4a21-8092-1328f3f775db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:22.712326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-559b942c-0fd9-4a21-8092-1328f3f775db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:22.715217 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3023/6093] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:22 2022] DELETE /identity/v3/projects/6568c92cbdaa43cfaafa98fdf660fa62 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:22.725591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e08ec2-16e2-40c0-9d03-dee1c7123789 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:22.725773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e08ec2-16e2-40c0-9d03-dee1c7123789 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:22.726287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e08ec2-16e2-40c0-9d03-dee1c7123789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:22.727376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb57338-3b48-4df8-9b9d-1ee0db29f698 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:22.747838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-81e08ec2-16e2-40c0-9d03-dee1c7123789 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:22.754337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-81e08ec2-16e2-40c0-9d03-dee1c7123789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:22.813448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c48293-4f44-4bca-bb4d-23ba1cdc6dfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:22.814156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c48293-4f44-4bca-bb4d-23ba1cdc6dfd service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:22.814439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c48293-4f44-4bca-bb4d-23ba1cdc6dfd service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:22.814973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c48293-4f44-4bca-bb4d-23ba1cdc6dfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:22.816891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-59c48293-4f44-4bca-bb4d-23ba1cdc6dfd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:22.836818 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3024/6094] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:25:22.843727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573f792-4617-44bd-ab26-782f540f3da2 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:22.844649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573f792-4617-44bd-ab26-782f540f3da2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:22.844649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573f792-4617-44bd-ab26-782f540f3da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:22.866995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e573f792-4617-44bd-ab26-782f540f3da2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:22.868724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e573f792-4617-44bd-ab26-782f540f3da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:22.877280 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3025/6095] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:25:22.881629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573f792-4617-44bd-ab26-782f540f3da2 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:22.891024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573f792-4617-44bd-ab26-782f540f3da2 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:22.892153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c48293-4f44-4bca-bb4d-23ba1cdc6dfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:22.894439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c48293-4f44-4bca-bb4d-23ba1cdc6dfd service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:22.906966 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3071/6096] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:25:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:25:22.911950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d1e85-75b4-4bcc-ba16-11837de16fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:22.912641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d1e85-75b4-4bcc-ba16-11837de16fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:22.912928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d1e85-75b4-4bcc-ba16-11837de16fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:22.913378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d1e85-75b4-4bcc-ba16-11837de16fb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:22.915876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082d1e85-75b4-4bcc-ba16-11837de16fb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:25:22.917765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082d1e85-75b4-4bcc-ba16-11837de16fb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:22.919443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082d1e85-75b4-4bcc-ba16-11837de16fb7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:22.923496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:22.947189 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3026/6097] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:25:22.951979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c48293-4f44-4bca-bb4d-23ba1cdc6dfd service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:22.963545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c48293-4f44-4bca-bb4d-23ba1cdc6dfd service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:22.968942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78c28a-905d-4b5c-b3ac-b1e56cd78257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:22.969711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78c28a-905d-4b5c-b3ac-b1e56cd78257 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:22.970653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78c28a-905d-4b5c-b3ac-b1e56cd78257 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:22.971015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78c28a-905d-4b5c-b3ac-b1e56cd78257 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:22.971807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf78c28a-905d-4b5c-b3ac-b1e56cd78257 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:22.974340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf78c28a-905d-4b5c-b3ac-b1e56cd78257 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:23.078665 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3072/6098] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:22 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:23.082719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d1e85-75b4-4bcc-ba16-11837de16fb7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:23.097034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d1e85-75b4-4bcc-ba16-11837de16fb7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:23.162796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca05dc-fcc6-4ac7-a5a7-7dd175cd2821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:23.163720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca05dc-fcc6-4ac7-a5a7-7dd175cd2821 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:23.165478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca05dc-fcc6-4ac7-a5a7-7dd175cd2821 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:23.166439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dca05dc-fcc6-4ac7-a5a7-7dd175cd2821 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:23.168170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca05dc-fcc6-4ac7-a5a7-7dd175cd2821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:23.174544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dca05dc-fcc6-4ac7-a5a7-7dd175cd2821 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:23.323267 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3027/6099] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:25:23 2022] POST /identity/v3/users => generated 287 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:23.324488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78c28a-905d-4b5c-b3ac-b1e56cd78257 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:23.346950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78c28a-905d-4b5c-b3ac-b1e56cd78257 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:23.354919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d737c-2fad-42fd-b3f6-c3540a76fce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:23.355623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d737c-2fad-42fd-b3f6-c3540a76fce4 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:23.356083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d737c-2fad-42fd-b3f6-c3540a76fce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:23.356362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d737c-2fad-42fd-b3f6-c3540a76fce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:23.357104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d737c-2fad-42fd-b3f6-c3540a76fce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:23.358870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d737c-2fad-42fd-b3f6-c3540a76fce4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:23.374480 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3073/6100] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:25:23.381807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca05dc-fcc6-4ac7-a5a7-7dd175cd2821 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:23.393732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dca05dc-fcc6-4ac7-a5a7-7dd175cd2821 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:23.399575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83bd107-9df9-47e6-86e2-23725fe36368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:23.400290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bd107-9df9-47e6-86e2-23725fe36368 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:23.400576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bd107-9df9-47e6-86e2-23725fe36368 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:23.400991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83bd107-9df9-47e6-86e2-23725fe36368 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:23.407444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83bd107-9df9-47e6-86e2-23725fe36368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:23.407444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83bd107-9df9-47e6-86e2-23725fe36368 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:23.416807 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3028/6101] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:25:23.421024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d737c-2fad-42fd-b3f6-c3540a76fce4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:23.431798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d737c-2fad-42fd-b3f6-c3540a76fce4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:23.436798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b697149-4a12-4a51-abf1-125cd26f2351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:23.437690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b697149-4a12-4a51-abf1-125cd26f2351 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:23.438007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b697149-4a12-4a51-abf1-125cd26f2351 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:23.438282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b697149-4a12-4a51-abf1-125cd26f2351 admin admin] PATH_INFO: `/v3/projects/501a1f1a443b42f1b6b247de8ce20ae6/users/2ac8583f518a45b7a8ed7618ab4f8cf4/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:23.464049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b697149-4a12-4a51-abf1-125cd26f2351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=501a1f1a443b42f1b6b247de8ce20ae6, user_id=2ac8583f518a45b7a8ed7618ab4f8cf4, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:23.466929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b697149-4a12-4a51-abf1-125cd26f2351 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:23.656591 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3074/6102] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:25:23 2022] PUT /identity/v3/projects/501a1f1a443b42f1b6b247de8ce20ae6/users/2ac8583f518a45b7a8ed7618ab4f8cf4/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:23.659766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf954b-677c-45be-a5b7-368f21b4d63c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:23.660247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf954b-677c-45be-a5b7-368f21b4d63c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:23.660367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bf954b-677c-45be-a5b7-368f21b4d63c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:23.702627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-17bf954b-677c-45be-a5b7-368f21b4d63c None None] MFA Rules not processed for user `2ac8583f518a45b7a8ed7618ab4f8cf4`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:23.703860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-17bf954b-677c-45be-a5b7-368f21b4d63c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:23.806687 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3029/6103] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:23.811865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b697149-4a12-4a51-abf1-125cd26f2351 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:23.884723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b697149-4a12-4a51-abf1-125cd26f2351 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:23.889553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982471f-79fc-422a-8ceb-934bd3c425c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:23.890261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982471f-79fc-422a-8ceb-934bd3c425c4 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:23.890548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982471f-79fc-422a-8ceb-934bd3c425c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:23.890855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982471f-79fc-422a-8ceb-934bd3c425c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:23.891558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982471f-79fc-422a-8ceb-934bd3c425c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:23.894780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982471f-79fc-422a-8ceb-934bd3c425c4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:24.279521 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3075/6104] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:23 2022] POST /identity/v3/projects => generated 371 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:24.283505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf954b-677c-45be-a5b7-368f21b4d63c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:24.297610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bf954b-677c-45be-a5b7-368f21b4d63c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:24.406719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1435ece-cd6a-42f8-b9ee-e5917e7ca6c1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:24.407315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1435ece-cd6a-42f8-b9ee-e5917e7ca6c1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:24.407315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1435ece-cd6a-42f8-b9ee-e5917e7ca6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:24.444214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075790d-9ad0-4b7e-80e8-0c43e5f55e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:24.445398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075790d-9ad0-4b7e-80e8-0c43e5f55e86 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:24.445924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075790d-9ad0-4b7e-80e8-0c43e5f55e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:24.446307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075790d-9ad0-4b7e-80e8-0c43e5f55e86 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:24.447602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075790d-9ad0-4b7e-80e8-0c43e5f55e86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:24.450750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2075790d-9ad0-4b7e-80e8-0c43e5f55e86 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:24.473307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f1435ece-cd6a-42f8-b9ee-e5917e7ca6c1 None None] MFA Rules not processed for user `ebc8e00965334bd1bb811a0eefd64b84`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:24.478124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f1435ece-cd6a-42f8-b9ee-e5917e7ca6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:24.539264 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3030/6105] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:25:24 2022] POST /identity/v3/users => generated 292 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:24.541674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075790d-9ad0-4b7e-80e8-0c43e5f55e86 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:24.562367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075790d-9ad0-4b7e-80e8-0c43e5f55e86 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:24.567814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670113e3-3bb4-4200-957a-1142b3d9790e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:24.568816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670113e3-3bb4-4200-957a-1142b3d9790e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:24.569479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670113e3-3bb4-4200-957a-1142b3d9790e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:24.570020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670113e3-3bb4-4200-957a-1142b3d9790e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:24.571018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670113e3-3bb4-4200-957a-1142b3d9790e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:24.572894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670113e3-3bb4-4200-957a-1142b3d9790e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:24.588317 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3031/6106] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:25:24.589636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670113e3-3bb4-4200-957a-1142b3d9790e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:24.601765 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3076/6107] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:25:24.609245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670113e3-3bb4-4200-957a-1142b3d9790e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:24.616198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc810a84-7d3f-464c-ab9e-3957d54f4e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:24.616198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc810a84-7d3f-464c-ab9e-3957d54f4e1d admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:24.616474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc810a84-7d3f-464c-ab9e-3957d54f4e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:24.617538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc810a84-7d3f-464c-ab9e-3957d54f4e1d admin admin] PATH_INFO: `/v3/projects/9c4ce39d915d4ed3ae2e48befd81e882/users/7bc9f29063544fababe76c6d5fe9a671/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:24.653888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc810a84-7d3f-464c-ab9e-3957d54f4e1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4ce39d915d4ed3ae2e48befd81e882, user_id=7bc9f29063544fababe76c6d5fe9a671, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:24.658177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc810a84-7d3f-464c-ab9e-3957d54f4e1d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:24.718555 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3032/6108] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:25:24 2022] PUT /identity/v3/projects/9c4ce39d915d4ed3ae2e48befd81e882/users/7bc9f29063544fababe76c6d5fe9a671/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:24.720019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1435ece-cd6a-42f8-b9ee-e5917e7ca6c1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:24.735233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1435ece-cd6a-42f8-b9ee-e5917e7ca6c1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:24.807923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f83e33-a35e-4b46-91cf-afee9ebdaa83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:24.808714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f83e33-a35e-4b46-91cf-afee9ebdaa83 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:24.809099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f83e33-a35e-4b46-91cf-afee9ebdaa83 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:24.809569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f83e33-a35e-4b46-91cf-afee9ebdaa83 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:24.810485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f83e33-a35e-4b46-91cf-afee9ebdaa83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:24.812553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f83e33-a35e-4b46-91cf-afee9ebdaa83 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:24.816647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:24.827854 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3077/6109] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:25:24.828881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc810a84-7d3f-464c-ab9e-3957d54f4e1d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:24.843758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc810a84-7d3f-464c-ab9e-3957d54f4e1d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:24.854409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cd415-e7a6-4073-8b2c-f1362ff44444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:24.856055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cd415-e7a6-4073-8b2c-f1362ff44444 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:24.856582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cd415-e7a6-4073-8b2c-f1362ff44444 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:24.857069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480cd415-e7a6-4073-8b2c-f1362ff44444 admin admin] PATH_INFO: `/v3/projects/9c4ce39d915d4ed3ae2e48befd81e882/users/7bc9f29063544fababe76c6d5fe9a671/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:24.860987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480cd415-e7a6-4073-8b2c-f1362ff44444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4ce39d915d4ed3ae2e48befd81e882, user_id=7bc9f29063544fababe76c6d5fe9a671, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:24.864366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480cd415-e7a6-4073-8b2c-f1362ff44444 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:24.882411 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3033/6110] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:25:24 2022] PUT /identity/v3/projects/9c4ce39d915d4ed3ae2e48befd81e882/users/7bc9f29063544fababe76c6d5fe9a671/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:25:24.889992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:24.893148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf634f34-b4e9-41af-b1fe-6a5fe258f94e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:24.893442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf634f34-b4e9-41af-b1fe-6a5fe258f94e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:24.893727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf634f34-b4e9-41af-b1fe-6a5fe258f94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:24.933127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-bf634f34-b4e9-41af-b1fe-6a5fe258f94e None None] MFA Rules not processed for user `7bc9f29063544fababe76c6d5fe9a671`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:24.934693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bf634f34-b4e9-41af-b1fe-6a5fe258f94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:25.013492 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3078/6111] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:24 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:25.017876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cd415-e7a6-4073-8b2c-f1362ff44444 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:25.038265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480cd415-e7a6-4073-8b2c-f1362ff44444 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:25.059332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a49712-b736-4d4b-929e-9be0ba803333 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:25.060299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a49712-b736-4d4b-929e-9be0ba803333 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:25.060773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a49712-b736-4d4b-929e-9be0ba803333 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:25.061235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a49712-b736-4d4b-929e-9be0ba803333 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:25.062018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a49712-b736-4d4b-929e-9be0ba803333 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:25.063826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a49712-b736-4d4b-929e-9be0ba803333 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:25.131529 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3034/6112] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:25:25 2022] POST /identity/v3/domains => generated 276 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:25.136136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf634f34-b4e9-41af-b1fe-6a5fe258f94e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:25.145378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf634f34-b4e9-41af-b1fe-6a5fe258f94e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:25.204026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072d8e1-e406-45ed-8bfa-7973417f824f tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:25.204693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072d8e1-e406-45ed-8bfa-7973417f824f tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:25.204978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072d8e1-e406-45ed-8bfa-7973417f824f tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:25.205256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072d8e1-e406-45ed-8bfa-7973417f824f tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:25.205980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0072d8e1-e406-45ed-8bfa-7973417f824f tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:25.207894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0072d8e1-e406-45ed-8bfa-7973417f824f tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:25.348603 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3079/6113] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:25 2022] POST /identity/v3/projects => generated 401 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:25.352707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a49712-b736-4d4b-929e-9be0ba803333 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:25.366682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a49712-b736-4d4b-929e-9be0ba803333 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:25.424663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a58466-e841-4392-92e8-a23fbacb0b14 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:25.425502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a58466-e841-4392-92e8-a23fbacb0b14 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:25.425800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a58466-e841-4392-92e8-a23fbacb0b14 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:25.426133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a58466-e841-4392-92e8-a23fbacb0b14 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/groups` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:25.426837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a58466-e841-4392-92e8-a23fbacb0b14 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:25.428858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a58466-e841-4392-92e8-a23fbacb0b14 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:25.500633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0d85e-92dc-4274-a8a5-e0cead46182f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:25.510980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0d85e-92dc-4274-a8a5-e0cead46182f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:25.510980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa0d85e-92dc-4274-a8a5-e0cead46182f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:25.519996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2aa0d85e-92dc-4274-a8a5-e0cead46182f None None] MFA Rules not processed for user `ebc8e00965334bd1bb811a0eefd64b84`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:25.520812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2aa0d85e-92dc-4274-a8a5-e0cead46182f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:25.564749 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3035/6114] 10.222.0.33 () {60 vars in 1146 bytes} [Sat May 28 21:25:25 2022] POST /identity/v3/groups => generated 301 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:25:25.568893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a58466-e841-4392-92e8-a23fbacb0b14 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:25.580630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a58466-e841-4392-92e8-a23fbacb0b14 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:25.581499 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3080/6115] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:25.590222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3339272-77a9-41b7-acb8-76318c6b42da tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:25.591089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3339272-77a9-41b7-acb8-76318c6b42da tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:25.594754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3339272-77a9-41b7-acb8-76318c6b42da tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:25.594754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3339272-77a9-41b7-acb8-76318c6b42da tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:25.594754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3339272-77a9-41b7-acb8-76318c6b42da tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:25.594754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3339272-77a9-41b7-acb8-76318c6b42da tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:25.717936 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3036/6116] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:25:25 2022] POST /identity/v3/users => generated 445 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:25.723356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0d85e-92dc-4274-a8a5-e0cead46182f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:25.735379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa0d85e-92dc-4274-a8a5-e0cead46182f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:25.740433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5faa248-dfe1-493b-ab0c-891e4d8e54a9 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:25.741219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faa248-dfe1-493b-ab0c-891e4d8e54a9 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:25.741575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faa248-dfe1-493b-ab0c-891e4d8e54a9 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:25.741951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5faa248-dfe1-493b-ab0c-891e4d8e54a9 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:25.743746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5faa248-dfe1-493b-ab0c-891e4d8e54a9 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:25.747155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5faa248-dfe1-493b-ab0c-891e4d8e54a9 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:25.822871 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3081/6117] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:25:25 2022] POST /identity/v3/roles => generated 239 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:25.826726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3339272-77a9-41b7-acb8-76318c6b42da tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:25.837413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3339272-77a9-41b7-acb8-76318c6b42da tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:25.842409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10480621-d917-4527-8909-6f820d73ac3d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:25.843272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10480621-d917-4527-8909-6f820d73ac3d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:25.843560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10480621-d917-4527-8909-6f820d73ac3d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:25.843839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10480621-d917-4527-8909-6f820d73ac3d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/54da26a310c34bf4963f95566b80a440/groups/0b77d389cac94d88ade184abe9f137c9/roles/4330e86a92d34122bc73cc2734736081/inherited_to_projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:25.850989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10480621-d917-4527-8909-6f820d73ac3d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_grant(project_id=54da26a310c34bf4963f95566b80a440, group_id=0b77d389cac94d88ade184abe9f137c9, role_id=4330e86a92d34122bc73cc2734736081)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:25.853946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10480621-d917-4527-8909-6f820d73ac3d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:25.966757 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3037/6118] 10.222.0.33 () {60 vars in 1584 bytes} [Sat May 28 21:25:25 2022] PUT /identity/v3/OS-INHERIT/projects/54da26a310c34bf4963f95566b80a440/groups/0b77d389cac94d88ade184abe9f137c9/roles/4330e86a92d34122bc73cc2734736081/inherited_to_projects => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:25.967636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5faa248-dfe1-493b-ab0c-891e4d8e54a9 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:25.987217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5faa248-dfe1-493b-ab0c-891e4d8e54a9 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:26.081802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2542fd8-8571-4f28-9d59-cac620715973 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:26.082761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2542fd8-8571-4f28-9d59-cac620715973 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `HEAD` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:26.083055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2542fd8-8571-4f28-9d59-cac620715973 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:26.083344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2542fd8-8571-4f28-9d59-cac620715973 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/54da26a310c34bf4963f95566b80a440/groups/0b77d389cac94d88ade184abe9f137c9/roles/4330e86a92d34122bc73cc2734736081/inherited_to_projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:26.085818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2542fd8-8571-4f28-9d59-cac620715973 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:check_grant(project_id=54da26a310c34bf4963f95566b80a440, group_id=0b77d389cac94d88ade184abe9f137c9, role_id=4330e86a92d34122bc73cc2734736081)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:26.087004 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-e2542fd8-8571-4f28-9d59-cac620715973 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:25:26.087004 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:25:26.087004 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:25:26.088963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2542fd8-8571-4f28-9d59-cac620715973 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:26.097127 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3082/6119] 10.222.0.33 () {58 vars in 1566 bytes} [Sat May 28 21:25:25 2022] HEAD /identity/v3/OS-INHERIT/projects/54da26a310c34bf4963f95566b80a440/groups/0b77d389cac94d88ade184abe9f137c9/roles/4330e86a92d34122bc73cc2734736081/inherited_to_projects => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:26.101344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10480621-d917-4527-8909-6f820d73ac3d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:26.117192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10480621-d917-4527-8909-6f820d73ac3d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:26.129017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d81b0d-1708-45f3-b738-37953e99e27c tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:26.132429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d81b0d-1708-45f3-b738-37953e99e27c tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:26.132429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d81b0d-1708-45f3-b738-37953e99e27c tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:26.132429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d81b0d-1708-45f3-b738-37953e99e27c tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/54da26a310c34bf4963f95566b80a440/groups/0b77d389cac94d88ade184abe9f137c9/roles/4330e86a92d34122bc73cc2734736081/inherited_to_projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:26.137533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d81b0d-1708-45f3-b738-37953e99e27c tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=54da26a310c34bf4963f95566b80a440, group_id=0b77d389cac94d88ade184abe9f137c9, role_id=4330e86a92d34122bc73cc2734736081)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:26.138805 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-38d81b0d-1708-45f3-b738-37953e99e27c tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:25:26.138805 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:25:26.138805 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:25:26.149169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d81b0d-1708-45f3-b738-37953e99e27c tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:26.173663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-38d81b0d-1708-45f3-b738-37953e99e27c tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invalidating the token cache because role 4330e86a92d34122bc73cc2734736081 was removed from group 0b77d389cac94d88ade184abe9f137c9 on project 54da26a310c34bf4963f95566b80a440. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:26.175947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-38d81b0d-1708-45f3-b738-37953e99e27c tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:26.176960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-38d81b0d-1708-45f3-b738-37953e99e27c tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:26.260526 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3038/6120] 10.222.0.33 () {58 vars in 1568 bytes} [Sat May 28 21:25:26 2022] DELETE /identity/v3/OS-INHERIT/projects/54da26a310c34bf4963f95566b80a440/groups/0b77d389cac94d88ade184abe9f137c9/roles/4330e86a92d34122bc73cc2734736081/inherited_to_projects => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:25:26.261803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2542fd8-8571-4f28-9d59-cac620715973 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:26.264681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e2542fd8-8571-4f28-9d59-cac620715973 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:26.265299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37aaba-a45c-45e2-874b-da6a6a80c31c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:26.265698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37aaba-a45c-45e2-874b-da6a6a80c31c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:26.266908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37aaba-a45c-45e2-874b-da6a6a80c31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:26.279336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2542fd8-8571-4f28-9d59-cac620715973 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:26.317930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4c37aaba-a45c-45e2-874b-da6a6a80c31c None None] MFA Rules not processed for user `ebc8e00965334bd1bb811a0eefd64b84`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:26.319225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4c37aaba-a45c-45e2-874b-da6a6a80c31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:26.341528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:26.375411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1245423-d9de-46c4-9dff-94b885a54fa7 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:26.376931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1245423-d9de-46c4-9dff-94b885a54fa7 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:26.377247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1245423-d9de-46c4-9dff-94b885a54fa7 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:26.377536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1245423-d9de-46c4-9dff-94b885a54fa7 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/roles/4330e86a92d34122bc73cc2734736081` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:26.379256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1245423-d9de-46c4-9dff-94b885a54fa7 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:delete_role(role_id=4330e86a92d34122bc73cc2734736081)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:26.381167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1245423-d9de-46c4-9dff-94b885a54fa7 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:26.400716 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3039/6121] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:26.483318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:26.496824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d1245423-d9de-46c4-9dff-94b885a54fa7 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invalidating the token cache because role 4330e86a92d34122bc73cc2734736081 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:26.497176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d1245423-d9de-46c4-9dff-94b885a54fa7 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:26.497901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d1245423-d9de-46c4-9dff-94b885a54fa7 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:26.500267 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3083/6122] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:26 2022] DELETE /identity/v3/roles/4330e86a92d34122bc73cc2734736081 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:26.505266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37aaba-a45c-45e2-874b-da6a6a80c31c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:26.518213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37aaba-a45c-45e2-874b-da6a6a80c31c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:26.578737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cad514-f1ac-4009-a89b-f1119bab4799 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:26.579430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cad514-f1ac-4009-a89b-f1119bab4799 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:26.579721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cad514-f1ac-4009-a89b-f1119bab4799 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:26.580007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cad514-f1ac-4009-a89b-f1119bab4799 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:26.580790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cad514-f1ac-4009-a89b-f1119bab4799 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:26.582467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cad514-f1ac-4009-a89b-f1119bab4799 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:26.690414 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3040/6123] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:25:26 2022] POST /identity/v3/roles => generated 240 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:26.694788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1245423-d9de-46c4-9dff-94b885a54fa7 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:26.697414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d1245423-d9de-46c4-9dff-94b885a54fa7 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:26.719631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1245423-d9de-46c4-9dff-94b885a54fa7 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:26.735244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905c20a-7efa-40e3-a9b6-c60e877a141b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:26.736156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905c20a-7efa-40e3-a9b6-c60e877a141b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:26.736497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905c20a-7efa-40e3-a9b6-c60e877a141b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:26.736729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905c20a-7efa-40e3-a9b6-c60e877a141b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/54da26a310c34bf4963f95566b80a440/users/ca6cede109334c74928e103c372003e9/roles/4792298f15354bbbbcc4181dc3fb05bf/inherited_to_projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:26.757877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905c20a-7efa-40e3-a9b6-c60e877a141b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_grant(project_id=54da26a310c34bf4963f95566b80a440, user_id=ca6cede109334c74928e103c372003e9, role_id=4792298f15354bbbbcc4181dc3fb05bf)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:26.764298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905c20a-7efa-40e3-a9b6-c60e877a141b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:26.911550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cad514-f1ac-4009-a89b-f1119bab4799 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:26.917937 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3084/6124] 10.222.0.33 () {60 vars in 1581 bytes} [Sat May 28 21:25:26 2022] PUT /identity/v3/OS-INHERIT/projects/54da26a310c34bf4963f95566b80a440/users/ca6cede109334c74928e103c372003e9/roles/4792298f15354bbbbcc4181dc3fb05bf/inherited_to_projects => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:26.927763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cad514-f1ac-4009-a89b-f1119bab4799 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:27.001398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654eaba-1640-44b6-9796-9c994f3f5bbc tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:27.007842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654eaba-1640-44b6-9796-9c994f3f5bbc tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `HEAD` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:27.007842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654eaba-1640-44b6-9796-9c994f3f5bbc tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:27.007842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654eaba-1640-44b6-9796-9c994f3f5bbc tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/54da26a310c34bf4963f95566b80a440/users/ca6cede109334c74928e103c372003e9/roles/4792298f15354bbbbcc4181dc3fb05bf/inherited_to_projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:27.007842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654eaba-1640-44b6-9796-9c994f3f5bbc tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:check_grant(project_id=54da26a310c34bf4963f95566b80a440, user_id=ca6cede109334c74928e103c372003e9, role_id=4792298f15354bbbbcc4181dc3fb05bf)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:27.008990 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-2654eaba-1640-44b6-9796-9c994f3f5bbc tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_grant": "(role:reader and system_scope:all) or ((role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:25:27.008990 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:25:27.008990 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:25:27.009293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654eaba-1640-44b6-9796-9c994f3f5bbc tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:27.015845 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3041/6125] 10.222.0.33 () {58 vars in 1563 bytes} [Sat May 28 21:25:26 2022] HEAD /identity/v3/OS-INHERIT/projects/54da26a310c34bf4963f95566b80a440/users/ca6cede109334c74928e103c372003e9/roles/4792298f15354bbbbcc4181dc3fb05bf/inherited_to_projects => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:27.020235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905c20a-7efa-40e3-a9b6-c60e877a141b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:27.034060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905c20a-7efa-40e3-a9b6-c60e877a141b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:27.040569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb52233-bce0-45f4-bb5e-137ff75b4cfb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:27.043002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb52233-bce0-45f4-bb5e-137ff75b4cfb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:27.043002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb52233-bce0-45f4-bb5e-137ff75b4cfb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:27.043255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb52233-bce0-45f4-bb5e-137ff75b4cfb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/54da26a310c34bf4963f95566b80a440/users/ca6cede109334c74928e103c372003e9/roles/4792298f15354bbbbcc4181dc3fb05bf/inherited_to_projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:27.048129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb52233-bce0-45f4-bb5e-137ff75b4cfb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=54da26a310c34bf4963f95566b80a440, user_id=ca6cede109334c74928e103c372003e9, role_id=4792298f15354bbbbcc4181dc3fb05bf)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:27.050009 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-6bb52233-bce0-45f4-bb5e-137ff75b4cfb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:25:27.050009 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:25:27.050009 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:25:27.060676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb52233-bce0-45f4-bb5e-137ff75b4cfb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:27.089430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6bb52233-bce0-45f4-bb5e-137ff75b4cfb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invalidating the token cache because role 4792298f15354bbbbcc4181dc3fb05bf was removed from user ca6cede109334c74928e103c372003e9 on project 54da26a310c34bf4963f95566b80a440. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:27.089430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6bb52233-bce0-45f4-bb5e-137ff75b4cfb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:27.089430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6bb52233-bce0-45f4-bb5e-137ff75b4cfb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:27.161343 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3085/6126] 10.222.0.33 () {58 vars in 1565 bytes} [Sat May 28 21:25:27 2022] DELETE /identity/v3/OS-INHERIT/projects/54da26a310c34bf4963f95566b80a440/users/ca6cede109334c74928e103c372003e9/roles/4792298f15354bbbbcc4181dc3fb05bf/inherited_to_projects => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:27.162315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654eaba-1640-44b6-9796-9c994f3f5bbc tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:27.164072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2654eaba-1640-44b6-9796-9c994f3f5bbc tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:27.176875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654eaba-1640-44b6-9796-9c994f3f5bbc tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:27.265107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f804c43-56a3-4c44-b593-cd85d7908226 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:27.266010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f804c43-56a3-4c44-b593-cd85d7908226 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:27.267087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f804c43-56a3-4c44-b593-cd85d7908226 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:27.267374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f804c43-56a3-4c44-b593-cd85d7908226 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/roles/4792298f15354bbbbcc4181dc3fb05bf` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:27.269319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f804c43-56a3-4c44-b593-cd85d7908226 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:delete_role(role_id=4792298f15354bbbbcc4181dc3fb05bf)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:27.271183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f804c43-56a3-4c44-b593-cd85d7908226 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:27.370976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5f804c43-56a3-4c44-b593-cd85d7908226 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invalidating the token cache because role 4792298f15354bbbbcc4181dc3fb05bf has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:27.371757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5f804c43-56a3-4c44-b593-cd85d7908226 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:27.372957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5f804c43-56a3-4c44-b593-cd85d7908226 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:27.379020 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3042/6127] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:27 2022] DELETE /identity/v3/roles/4792298f15354bbbbcc4181dc3fb05bf => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:25:27.380428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb52233-bce0-45f4-bb5e-137ff75b4cfb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:27.392659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:27.400167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb52233-bce0-45f4-bb5e-137ff75b4cfb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:27.511089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54232ce4-0a4c-49e0-abab-77809a6e412b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:27.511965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54232ce4-0a4c-49e0-abab-77809a6e412b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:27.512263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54232ce4-0a4c-49e0-abab-77809a6e412b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:27.512551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54232ce4-0a4c-49e0-abab-77809a6e412b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:27.513761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54232ce4-0a4c-49e0-abab-77809a6e412b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:27.516379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54232ce4-0a4c-49e0-abab-77809a6e412b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:27.568524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd9f7e-6286-4510-a331-eaf5d5266934 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:27.568901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd9f7e-6286-4510-a331-eaf5d5266934 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:27.569243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd9f7e-6286-4510-a331-eaf5d5266934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:27.587595 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3086/6128] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:25:27 2022] POST /identity/v3/roles => generated 239 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:27.591826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54232ce4-0a4c-49e0-abab-77809a6e412b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:27.594199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-54232ce4-0a4c-49e0-abab-77809a6e412b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:27.597956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-eadd9f7e-6286-4510-a331-eaf5d5266934 None None] MFA Rules not processed for user `ebc8e00965334bd1bb811a0eefd64b84`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:27.599198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-eadd9f7e-6286-4510-a331-eaf5d5266934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:27.611478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54232ce4-0a4c-49e0-abab-77809a6e412b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:27.619153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eebdb8-2580-49b2-8e77-67b4cd38a47d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:27.620119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eebdb8-2580-49b2-8e77-67b4cd38a47d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:27.620410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eebdb8-2580-49b2-8e77-67b4cd38a47d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:27.620753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eebdb8-2580-49b2-8e77-67b4cd38a47d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/groups/0b77d389cac94d88ade184abe9f137c9/roles/9c3d24c25b34469b811e1689be226697/inherited_to_projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:27.623538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eebdb8-2580-49b2-8e77-67b4cd38a47d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=a720f7c59b8c4dd5ac1a55d1b511c737, group_id=0b77d389cac94d88ade184abe9f137c9, role_id=9c3d24c25b34469b811e1689be226697)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:27.627170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eebdb8-2580-49b2-8e77-67b4cd38a47d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:27.648716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:27.710902 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3087/6129] 10.222.0.33 () {60 vars in 1581 bytes} [Sat May 28 21:25:27 2022] PUT /identity/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/groups/0b77d389cac94d88ade184abe9f137c9/roles/9c3d24c25b34469b811e1689be226697/inherited_to_projects => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:27.715528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eebdb8-2580-49b2-8e77-67b4cd38a47d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:27.879492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eebdb8-2580-49b2-8e77-67b4cd38a47d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:27.887983 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3043/6130] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:25:27.902644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29e2d1-df52-4ff1-8732-23c8d22762d0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:27.903501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29e2d1-df52-4ff1-8732-23c8d22762d0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:27.903790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29e2d1-df52-4ff1-8732-23c8d22762d0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:27.908417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29e2d1-df52-4ff1-8732-23c8d22762d0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/groups/0b77d389cac94d88ade184abe9f137c9/roles/inherited_to_projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:27.916825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29e2d1-df52-4ff1-8732-23c8d22762d0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=a720f7c59b8c4dd5ac1a55d1b511c737, group_id=0b77d389cac94d88ade184abe9f137c9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:27.918116 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-0d29e2d1-df52-4ff1-8732-23c8d22762d0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_grants": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:25:27.918116 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:25:27.918116 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:25:27.922709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29e2d1-df52-4ff1-8732-23c8d22762d0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:27.956503 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3088/6131] 10.222.0.33 () {58 vars in 1463 bytes} [Sat May 28 21:25:27 2022] GET /identity/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/groups/0b77d389cac94d88ade184abe9f137c9/roles/inherited_to_projects => generated 447 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:27.958869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd9f7e-6286-4510-a331-eaf5d5266934 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:27.990748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd9f7e-6286-4510-a331-eaf5d5266934 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:27.999467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60abb9-56e2-454a-b318-bf05860ccffb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:28.001082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60abb9-56e2-454a-b318-bf05860ccffb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `HEAD` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:28.001662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60abb9-56e2-454a-b318-bf05860ccffb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:28.002223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf60abb9-56e2-454a-b318-bf05860ccffb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/groups/0b77d389cac94d88ade184abe9f137c9/roles/9c3d24c25b34469b811e1689be226697/inherited_to_projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:28.008980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60abb9-56e2-454a-b318-bf05860ccffb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=a720f7c59b8c4dd5ac1a55d1b511c737, group_id=0b77d389cac94d88ade184abe9f137c9, role_id=9c3d24c25b34469b811e1689be226697)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:28.011124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf60abb9-56e2-454a-b318-bf05860ccffb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:28.018806 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3044/6132] 10.222.0.33 () {58 vars in 1563 bytes} [Sat May 28 21:25:27 2022] HEAD /identity/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/groups/0b77d389cac94d88ade184abe9f137c9/roles/9c3d24c25b34469b811e1689be226697/inherited_to_projects => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:28.022767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29e2d1-df52-4ff1-8732-23c8d22762d0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:28.035541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29e2d1-df52-4ff1-8732-23c8d22762d0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:28.047707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624f464-2db0-406f-8220-2beb93d43528 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:28.048890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624f464-2db0-406f-8220-2beb93d43528 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:28.049354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624f464-2db0-406f-8220-2beb93d43528 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:28.049814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624f464-2db0-406f-8220-2beb93d43528 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/groups/0b77d389cac94d88ade184abe9f137c9/roles/9c3d24c25b34469b811e1689be226697/inherited_to_projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:28.054238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624f464-2db0-406f-8220-2beb93d43528 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=a720f7c59b8c4dd5ac1a55d1b511c737, group_id=0b77d389cac94d88ade184abe9f137c9, role_id=9c3d24c25b34469b811e1689be226697)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:28.060520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624f464-2db0-406f-8220-2beb93d43528 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:28.080077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:28.083550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c624f464-2db0-406f-8220-2beb93d43528 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invalidating the token cache because role 9c3d24c25b34469b811e1689be226697 was removed from group 0b77d389cac94d88ade184abe9f137c9 on domain a720f7c59b8c4dd5ac1a55d1b511c737. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:28.090190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c624f464-2db0-406f-8220-2beb93d43528 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:28.090190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c624f464-2db0-406f-8220-2beb93d43528 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:28.158475 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3089/6133] 10.222.0.33 () {58 vars in 1565 bytes} [Sat May 28 21:25:28 2022] DELETE /identity/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/groups/0b77d389cac94d88ade184abe9f137c9/roles/9c3d24c25b34469b811e1689be226697/inherited_to_projects => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:28.164112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60abb9-56e2-454a-b318-bf05860ccffb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:28.169264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cf60abb9-56e2-454a-b318-bf05860ccffb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:28.196882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf60abb9-56e2-454a-b318-bf05860ccffb tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:28.308163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d2754-6871-4514-9072-ad157bf4c080 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:28.309621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d2754-6871-4514-9072-ad157bf4c080 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:28.310058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d2754-6871-4514-9072-ad157bf4c080 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:28.310446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d2754-6871-4514-9072-ad157bf4c080 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/roles/9c3d24c25b34469b811e1689be226697` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:28.312516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d2754-6871-4514-9072-ad157bf4c080 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:delete_role(role_id=9c3d24c25b34469b811e1689be226697)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:28.316029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d2754-6871-4514-9072-ad157bf4c080 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:28.341541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:28.454401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-133d2754-6871-4514-9072-ad157bf4c080 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invalidating the token cache because role 9c3d24c25b34469b811e1689be226697 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:28.454606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-133d2754-6871-4514-9072-ad157bf4c080 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:28.455273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-133d2754-6871-4514-9072-ad157bf4c080 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:28.457817 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3045/6134] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:28 2022] DELETE /identity/v3/roles/9c3d24c25b34469b811e1689be226697 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:28.463194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624f464-2db0-406f-8220-2beb93d43528 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:28.486638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624f464-2db0-406f-8220-2beb93d43528 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:28.564979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34efb70-7f2d-4da5-808c-b438ca0da585 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:28.565702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34efb70-7f2d-4da5-808c-b438ca0da585 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:28.566047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34efb70-7f2d-4da5-808c-b438ca0da585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:28.592341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a4024-735f-4853-bb85-ba76266ebfef tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:28.593037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a4024-735f-4853-bb85-ba76266ebfef tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:28.593197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f34efb70-7f2d-4da5-808c-b438ca0da585 None None] MFA Rules not processed for user `ebc8e00965334bd1bb811a0eefd64b84`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:28.593385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a4024-735f-4853-bb85-ba76266ebfef tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:28.593608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a4024-735f-4853-bb85-ba76266ebfef tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:28.594286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f34efb70-7f2d-4da5-808c-b438ca0da585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:28.594419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a4024-735f-4853-bb85-ba76266ebfef tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:28.600071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a4024-735f-4853-bb85-ba76266ebfef tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:28.706103 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3090/6135] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:25:28 2022] POST /identity/v3/roles => generated 240 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:28.708253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a4024-735f-4853-bb85-ba76266ebfef tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:28.714386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-295a4024-735f-4853-bb85-ba76266ebfef tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:28.721312 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3046/6136] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:28.749136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a4024-735f-4853-bb85-ba76266ebfef tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:28.760899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce2d00-66e0-4855-bbc6-1a2e734418e0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:28.764812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce2d00-66e0-4855-bbc6-1a2e734418e0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:28.765334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce2d00-66e0-4855-bbc6-1a2e734418e0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:28.765801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce2d00-66e0-4855-bbc6-1a2e734418e0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/users/ca6cede109334c74928e103c372003e9/roles/d968b1ca81d345718849c07792fca75a/inherited_to_projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:28.769028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce2d00-66e0-4855-bbc6-1a2e734418e0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=a720f7c59b8c4dd5ac1a55d1b511c737, user_id=ca6cede109334c74928e103c372003e9, role_id=d968b1ca81d345718849c07792fca75a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:28.772461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ce2d00-66e0-4855-bbc6-1a2e734418e0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:28.785273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:28.872397 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3091/6137] 10.222.0.33 () {60 vars in 1578 bytes} [Sat May 28 21:25:28 2022] PUT /identity/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/users/ca6cede109334c74928e103c372003e9/roles/d968b1ca81d345718849c07792fca75a/inherited_to_projects => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:28.876656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34efb70-7f2d-4da5-808c-b438ca0da585 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:28.894156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34efb70-7f2d-4da5-808c-b438ca0da585 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:29.011832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b0afc-d497-4932-a3bc-488d4fbfc0b0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:29.019243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b0afc-d497-4932-a3bc-488d4fbfc0b0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:29.019243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b0afc-d497-4932-a3bc-488d4fbfc0b0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:29.019243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9b0afc-d497-4932-a3bc-488d4fbfc0b0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/users/ca6cede109334c74928e103c372003e9/roles/inherited_to_projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:29.019243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b0afc-d497-4932-a3bc-488d4fbfc0b0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=a720f7c59b8c4dd5ac1a55d1b511c737, user_id=ca6cede109334c74928e103c372003e9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:29.019895 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-9b9b0afc-d497-4932-a3bc-488d4fbfc0b0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_grants": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:reader and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:25:29.019895 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:25:29.019895 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:25:29.020404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9b0afc-d497-4932-a3bc-488d4fbfc0b0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:29.039282 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3047/6138] 10.222.0.33 () {58 vars in 1460 bytes} [Sat May 28 21:25:28 2022] GET /identity/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/users/ca6cede109334c74928e103c372003e9/roles/inherited_to_projects => generated 447 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:25:29.043671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce2d00-66e0-4855-bbc6-1a2e734418e0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:29.060202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce2d00-66e0-4855-bbc6-1a2e734418e0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:29.071576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97888a-86c7-417b-8a84-5e1e96d6a091 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:29.072728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97888a-86c7-417b-8a84-5e1e96d6a091 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `HEAD` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:29.073197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97888a-86c7-417b-8a84-5e1e96d6a091 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:29.073654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97888a-86c7-417b-8a84-5e1e96d6a091 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/users/ca6cede109334c74928e103c372003e9/roles/d968b1ca81d345718849c07792fca75a/inherited_to_projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:29.088945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97888a-86c7-417b-8a84-5e1e96d6a091 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=a720f7c59b8c4dd5ac1a55d1b511c737, user_id=ca6cede109334c74928e103c372003e9, role_id=d968b1ca81d345718849c07792fca75a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:29.092543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97888a-86c7-417b-8a84-5e1e96d6a091 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:29.107713 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3092/6139] 10.222.0.33 () {58 vars in 1560 bytes} [Sat May 28 21:25:29 2022] HEAD /identity/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/users/ca6cede109334c74928e103c372003e9/roles/d968b1ca81d345718849c07792fca75a/inherited_to_projects => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:29.111981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b0afc-d497-4932-a3bc-488d4fbfc0b0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:29.124259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9b0afc-d497-4932-a3bc-488d4fbfc0b0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:29.148246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62c0f6-ea11-41ac-9655-3394f05fb286 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:29.150107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62c0f6-ea11-41ac-9655-3394f05fb286 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:29.150800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62c0f6-ea11-41ac-9655-3394f05fb286 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:29.151254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62c0f6-ea11-41ac-9655-3394f05fb286 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/users/ca6cede109334c74928e103c372003e9/roles/d968b1ca81d345718849c07792fca75a/inherited_to_projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:29.155714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62c0f6-ea11-41ac-9655-3394f05fb286 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=a720f7c59b8c4dd5ac1a55d1b511c737, user_id=ca6cede109334c74928e103c372003e9, role_id=d968b1ca81d345718849c07792fca75a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:29.158816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b62c0f6-ea11-41ac-9655-3394f05fb286 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:29.171593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4b62c0f6-ea11-41ac-9655-3394f05fb286 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invalidating the token cache because role d968b1ca81d345718849c07792fca75a was removed from user ca6cede109334c74928e103c372003e9 on domain a720f7c59b8c4dd5ac1a55d1b511c737. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:29.171783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4b62c0f6-ea11-41ac-9655-3394f05fb286 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:29.175526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4b62c0f6-ea11-41ac-9655-3394f05fb286 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:29.225938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310038dd-a76c-4e32-b7c8-2e93333440bb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:29.226554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310038dd-a76c-4e32-b7c8-2e93333440bb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:29.227025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310038dd-a76c-4e32-b7c8-2e93333440bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:29.254563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-310038dd-a76c-4e32-b7c8-2e93333440bb None None] MFA Rules not processed for user `ebc8e00965334bd1bb811a0eefd64b84`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:29.256202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-310038dd-a76c-4e32-b7c8-2e93333440bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:29.297951 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3048/6140] 10.222.0.33 () {58 vars in 1562 bytes} [Sat May 28 21:25:29 2022] DELETE /identity/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/users/ca6cede109334c74928e103c372003e9/roles/d968b1ca81d345718849c07792fca75a/inherited_to_projects => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:29.299176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62c0f6-ea11-41ac-9655-3394f05fb286 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:29.301367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4b62c0f6-ea11-41ac-9655-3394f05fb286 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:29.301786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:29.359957 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3093/6141] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:29.440965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b62c0f6-ea11-41ac-9655-3394f05fb286 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:29.449239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3d855-ad10-4657-a7a2-312c7c35bcec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:29.450845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3d855-ad10-4657-a7a2-312c7c35bcec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:29.451292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3d855-ad10-4657-a7a2-312c7c35bcec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:29.452394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3d855-ad10-4657-a7a2-312c7c35bcec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/roles/d968b1ca81d345718849c07792fca75a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:29.454571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3d855-ad10-4657-a7a2-312c7c35bcec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:delete_role(role_id=d968b1ca81d345718849c07792fca75a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:29.456775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3d855-ad10-4657-a7a2-312c7c35bcec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:29.599672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-90c3d855-ad10-4657-a7a2-312c7c35bcec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invalidating the token cache because role d968b1ca81d345718849c07792fca75a has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:29.600036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-90c3d855-ad10-4657-a7a2-312c7c35bcec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:29.601697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-90c3d855-ad10-4657-a7a2-312c7c35bcec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:29.605757 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3049/6142] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:29 2022] DELETE /identity/v3/roles/d968b1ca81d345718849c07792fca75a => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:29.619857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310038dd-a76c-4e32-b7c8-2e93333440bb None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:29.655851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310038dd-a76c-4e32-b7c8-2e93333440bb None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:29.835669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b90be4-7ea3-4739-ab6d-3f9599d4eca3 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:29.839051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b90be4-7ea3-4739-ab6d-3f9599d4eca3 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:29.841112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b90be4-7ea3-4739-ab6d-3f9599d4eca3 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:29.841603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b90be4-7ea3-4739-ab6d-3f9599d4eca3 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:29.842625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b90be4-7ea3-4739-ab6d-3f9599d4eca3 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:29.844621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b90be4-7ea3-4739-ab6d-3f9599d4eca3 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:29.987215 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3094/6143] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:25:29 2022] POST /identity/v3/roles => generated 239 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:29.988518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3d855-ad10-4657-a7a2-312c7c35bcec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:29.990243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-90c3d855-ad10-4657-a7a2-312c7c35bcec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:30.006889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3d855-ad10-4657-a7a2-312c7c35bcec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:30.015146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9849a-7bf4-4add-83cc-f68c95da4d5a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:30.016005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9849a-7bf4-4add-83cc-f68c95da4d5a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:30.016401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9849a-7bf4-4add-83cc-f68c95da4d5a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:30.016791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d9849a-7bf4-4add-83cc-f68c95da4d5a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:30.017621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9849a-7bf4-4add-83cc-f68c95da4d5a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:30.019831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d9849a-7bf4-4add-83cc-f68c95da4d5a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:30.058231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4b2f2-f8ef-4381-9519-497dd3b68be0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:30.060492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:30.061503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4b2f2-f8ef-4381-9519-497dd3b68be0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:30.062059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4b2f2-f8ef-4381-9519-497dd3b68be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:30.099004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5cb4b2f2-f8ef-4381-9519-497dd3b68be0 None None] MFA Rules not processed for user `ebc8e00965334bd1bb811a0eefd64b84`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:30.100351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5cb4b2f2-f8ef-4381-9519-497dd3b68be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:30.137629 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3050/6144] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:29 2022] POST /identity/v3/projects => generated 409 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:30.144119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9849a-7bf4-4add-83cc-f68c95da4d5a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:30.184823 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3095/6145] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:30.266346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d9849a-7bf4-4add-83cc-f68c95da4d5a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:30.276462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3decc-a86e-48c3-9c72-c4a1384bb04d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:30.277611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3decc-a86e-48c3-9c72-c4a1384bb04d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:30.278110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3decc-a86e-48c3-9c72-c4a1384bb04d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:30.280009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff3decc-a86e-48c3-9c72-c4a1384bb04d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/users/ca6cede109334c74928e103c372003e9/roles/4bb455b2e55e4e45a6afb2b2c740cf83/inherited_to_projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:30.281497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3decc-a86e-48c3-9c72-c4a1384bb04d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=a720f7c59b8c4dd5ac1a55d1b511c737, user_id=ca6cede109334c74928e103c372003e9, role_id=4bb455b2e55e4e45a6afb2b2c740cf83)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:30.284891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff3decc-a86e-48c3-9c72-c4a1384bb04d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:30.363815 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3051/6146] 10.222.0.33 () {60 vars in 1578 bytes} [Sat May 28 21:25:30 2022] PUT /identity/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/users/ca6cede109334c74928e103c372003e9/roles/4bb455b2e55e4e45a6afb2b2c740cf83/inherited_to_projects => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:30.367957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4b2f2-f8ef-4381-9519-497dd3b68be0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:30.383641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4b2f2-f8ef-4381-9519-497dd3b68be0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:30.496368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c70efc-272f-4700-aeac-d6c505e808b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:30.497423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c70efc-272f-4700-aeac-d6c505e808b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:30.498381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c70efc-272f-4700-aeac-d6c505e808b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:30.500291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c70efc-272f-4700-aeac-d6c505e808b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:30.501481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c70efc-272f-4700-aeac-d6c505e808b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Adding query filter params (scope.project.id=54da26a310c34bf4963f95566b80a440, user.id=ca6cede109334c74928e103c372003e9) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:25:30.502128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c70efc-272f-4700-aeac-d6c505e808b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:30.504662 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-45c70efc-272f-4700-aeac-d6c505e808b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:25:30.504662 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:25:30.504662 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:25:30.508004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c70efc-272f-4700-aeac-d6c505e808b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:30.611838 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3096/6147] 10.222.0.33 () {58 vars in 1355 bytes} [Sat May 28 21:25:30 2022] GET /identity/v3/role_assignments?scope.project.id=54da26a310c34bf4963f95566b80a440&user.id=ca6cede109334c74928e103c372003e9&effective => generated 648 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:25:30.616169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3decc-a86e-48c3-9c72-c4a1384bb04d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:30.633686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff3decc-a86e-48c3-9c72-c4a1384bb04d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:30.647379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d2d05-dbef-4f18-89ae-313ed3d540b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:30.648165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d2d05-dbef-4f18-89ae-313ed3d540b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:30.648569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d2d05-dbef-4f18-89ae-313ed3d540b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:30.649534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d2d05-dbef-4f18-89ae-313ed3d540b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:30.651054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d2d05-dbef-4f18-89ae-313ed3d540b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Adding query filter params (scope.project.id=aaee5ab438bc4a4bb92b8a7963549db3, user.id=ca6cede109334c74928e103c372003e9) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:25:30.651254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d2d05-dbef-4f18-89ae-313ed3d540b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:30.653758 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-3f6d2d05-dbef-4f18-89ae-313ed3d540b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_assignments": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:25:30.653758 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:25:30.653758 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:25:30.660952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d2d05-dbef-4f18-89ae-313ed3d540b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:30.777435 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3052/6148] 10.222.0.33 () {58 vars in 1355 bytes} [Sat May 28 21:25:30 2022] GET /identity/v3/role_assignments?scope.project.id=aaee5ab438bc4a4bb92b8a7963549db3&user.id=ca6cede109334c74928e103c372003e9&effective => generated 648 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:30.783241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c70efc-272f-4700-aeac-d6c505e808b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:30.803489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c70efc-272f-4700-aeac-d6c505e808b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:30.812924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6c27e-1d39-4aab-b9b6-f4c301271225 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:30.814266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6c27e-1d39-4aab-b9b6-f4c301271225 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:30.814568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6c27e-1d39-4aab-b9b6-f4c301271225 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:30.815726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6c27e-1d39-4aab-b9b6-f4c301271225 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/users/ca6cede109334c74928e103c372003e9/roles/4bb455b2e55e4e45a6afb2b2c740cf83/inherited_to_projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:30.817300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6c27e-1d39-4aab-b9b6-f4c301271225 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=a720f7c59b8c4dd5ac1a55d1b511c737, user_id=ca6cede109334c74928e103c372003e9, role_id=4bb455b2e55e4e45a6afb2b2c740cf83)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:30.820647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6c27e-1d39-4aab-b9b6-f4c301271225 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:30.828228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-78f6c27e-1d39-4aab-b9b6-f4c301271225 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invalidating the token cache because role 4bb455b2e55e4e45a6afb2b2c740cf83 was removed from user ca6cede109334c74928e103c372003e9 on domain a720f7c59b8c4dd5ac1a55d1b511c737. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:30.828573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-78f6c27e-1d39-4aab-b9b6-f4c301271225 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:30.829340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-78f6c27e-1d39-4aab-b9b6-f4c301271225 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:30.951015 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3097/6149] 10.222.0.33 () {58 vars in 1562 bytes} [Sat May 28 21:25:30 2022] DELETE /identity/v3/OS-INHERIT/domains/a720f7c59b8c4dd5ac1a55d1b511c737/users/ca6cede109334c74928e103c372003e9/roles/4bb455b2e55e4e45a6afb2b2c740cf83/inherited_to_projects => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:30.955327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d2d05-dbef-4f18-89ae-313ed3d540b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:30.957627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3f6d2d05-dbef-4f18-89ae-313ed3d540b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:30.976316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d2d05-dbef-4f18-89ae-313ed3d540b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:31.122795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528ba4e-d509-49e6-ac8f-aa451b14e2ec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:31.124234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528ba4e-d509-49e6-ac8f-aa451b14e2ec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:31.124770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528ba4e-d509-49e6-ac8f-aa451b14e2ec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:31.125451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e528ba4e-d509-49e6-ac8f-aa451b14e2ec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:31.126777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528ba4e-d509-49e6-ac8f-aa451b14e2ec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Adding query filter params (scope.project.id=54da26a310c34bf4963f95566b80a440, user.id=ca6cede109334c74928e103c372003e9) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:25:31.127275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528ba4e-d509-49e6-ac8f-aa451b14e2ec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:31.130691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e528ba4e-d509-49e6-ac8f-aa451b14e2ec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:31.181133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:31.248109 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3053/6150] 10.222.0.33 () {58 vars in 1355 bytes} [Sat May 28 21:25:30 2022] GET /identity/v3/role_assignments?scope.project.id=54da26a310c34bf4963f95566b80a440&user.id=ca6cede109334c74928e103c372003e9&effective => generated 228 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:31.249020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6c27e-1d39-4aab-b9b6-f4c301271225 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:31.268711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6c27e-1d39-4aab-b9b6-f4c301271225 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:31.290846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2192fa-6566-420a-ac19-ec51c4c2288a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:31.293104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2192fa-6566-420a-ac19-ec51c4c2288a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:31.293407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2192fa-6566-420a-ac19-ec51c4c2288a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:31.293699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2192fa-6566-420a-ac19-ec51c4c2288a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:31.294642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2192fa-6566-420a-ac19-ec51c4c2288a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Adding query filter params (scope.project.id=aaee5ab438bc4a4bb92b8a7963549db3, user.id=ca6cede109334c74928e103c372003e9) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:25:31.294962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2192fa-6566-420a-ac19-ec51c4c2288a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:31.296891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2192fa-6566-420a-ac19-ec51c4c2288a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:31.415712 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3098/6151] 10.222.0.33 () {58 vars in 1355 bytes} [Sat May 28 21:25:31 2022] GET /identity/v3/role_assignments?scope.project.id=aaee5ab438bc4a4bb92b8a7963549db3&user.id=ca6cede109334c74928e103c372003e9&effective => generated 228 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:31.416834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528ba4e-d509-49e6-ac8f-aa451b14e2ec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:31.440170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e528ba4e-d509-49e6-ac8f-aa451b14e2ec tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:31.465939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a600db8-04f1-4566-9b2f-e65eaada8ea6 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:31.467231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a600db8-04f1-4566-9b2f-e65eaada8ea6 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:31.467661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a600db8-04f1-4566-9b2f-e65eaada8ea6 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:31.468100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a600db8-04f1-4566-9b2f-e65eaada8ea6 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/projects/aaee5ab438bc4a4bb92b8a7963549db3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:31.473814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a600db8-04f1-4566-9b2f-e65eaada8ea6 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:delete_project(project_id=aaee5ab438bc4a4bb92b8a7963549db3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:31.476059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a600db8-04f1-4566-9b2f-e65eaada8ea6 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:31.643207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:31.712330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9a600db8-04f1-4566-9b2f-e65eaada8ea6 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaee5ab438bc4a4bb92b8a7963549db3', 'request_id': 'req-9a600db8-04f1-4566-9b2f-e65eaada8ea6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:31.725711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9a600db8-04f1-4566-9b2f-e65eaada8ea6 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaee5ab438bc4a4bb92b8a7963549db3', 'request_id': 'req-9a600db8-04f1-4566-9b2f-e65eaada8ea6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:31.808795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9a600db8-04f1-4566-9b2f-e65eaada8ea6 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaee5ab438bc4a4bb92b8a7963549db3', 'request_id': 'req-9a600db8-04f1-4566-9b2f-e65eaada8ea6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:31.841613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9a600db8-04f1-4566-9b2f-e65eaada8ea6 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] The token cache is being invalidate because project aaee5ab438bc4a4bb92b8a7963549db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:31.841861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9a600db8-04f1-4566-9b2f-e65eaada8ea6 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:31.842617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9a600db8-04f1-4566-9b2f-e65eaada8ea6 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:31.850119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2192fa-6566-420a-ac19-ec51c4c2288a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:31.850584 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3054/6152] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:31 2022] DELETE /identity/v3/projects/aaee5ab438bc4a4bb92b8a7963549db3 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:31.852984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4c2192fa-6566-420a-ac19-ec51c4c2288a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:31.872388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2192fa-6566-420a-ac19-ec51c4c2288a tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:31.889827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcbfee-2b33-4104-bc1f-e0127ca8dae2 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:31.890162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcbfee-2b33-4104-bc1f-e0127ca8dae2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:31.891090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcbfee-2b33-4104-bc1f-e0127ca8dae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:31.918838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-aebcbfee-2b33-4104-bc1f-e0127ca8dae2 None None] MFA Rules not processed for user `ebc8e00965334bd1bb811a0eefd64b84`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:31.926050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-aebcbfee-2b33-4104-bc1f-e0127ca8dae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:31.994516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf110c-3231-4237-9f80-6e93a6eec244 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:31.995440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf110c-3231-4237-9f80-6e93a6eec244 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:31.995726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf110c-3231-4237-9f80-6e93a6eec244 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:31.996008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bf110c-3231-4237-9f80-6e93a6eec244 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/roles/4bb455b2e55e4e45a6afb2b2c740cf83` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:32.003007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bf110c-3231-4237-9f80-6e93a6eec244 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:delete_role(role_id=4bb455b2e55e4e45a6afb2b2c740cf83)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:32.047928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bf110c-3231-4237-9f80-6e93a6eec244 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:32.105429 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3055/6153] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:32.214303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f6bf110c-3231-4237-9f80-6e93a6eec244 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invalidating the token cache because role 4bb455b2e55e4e45a6afb2b2c740cf83 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:32.214947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f6bf110c-3231-4237-9f80-6e93a6eec244 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:32.216269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f6bf110c-3231-4237-9f80-6e93a6eec244 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:32.219101 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3099/6154] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:31 2022] DELETE /identity/v3/roles/4bb455b2e55e4e45a6afb2b2c740cf83 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:32.225338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcbfee-2b33-4104-bc1f-e0127ca8dae2 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:32.245636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebcbfee-2b33-4104-bc1f-e0127ca8dae2 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:32.393692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453da2b5-b4ce-4f76-9014-c93b57c4196e tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:32.393692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453da2b5-b4ce-4f76-9014-c93b57c4196e tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:32.394477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453da2b5-b4ce-4f76-9014-c93b57c4196e tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:32.394872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453da2b5-b4ce-4f76-9014-c93b57c4196e tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:32.405376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453da2b5-b4ce-4f76-9014-c93b57c4196e tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:32.405376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453da2b5-b4ce-4f76-9014-c93b57c4196e tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:32.518902 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3056/6155] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:25:32 2022] POST /identity/v3/roles => generated 239 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:32.519599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf110c-3231-4237-9f80-6e93a6eec244 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:32.521409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f6bf110c-3231-4237-9f80-6e93a6eec244 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:32.559241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bf110c-3231-4237-9f80-6e93a6eec244 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:32.568050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e46a1-7615-4821-92d5-b72940e227c9 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:32.568876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e46a1-7615-4821-92d5-b72940e227c9 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:32.569240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e46a1-7615-4821-92d5-b72940e227c9 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:32.569688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e46a1-7615-4821-92d5-b72940e227c9 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:32.570536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e46a1-7615-4821-92d5-b72940e227c9 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:32.573262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e46a1-7615-4821-92d5-b72940e227c9 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:32.705257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453da2b5-b4ce-4f76-9014-c93b57c4196e tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:32.711857 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3100/6156] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:32 2022] POST /identity/v3/projects => generated 407 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:32.717617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:32.720704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453da2b5-b4ce-4f76-9014-c93b57c4196e tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:32.769733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:32.877645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ebbdd-a68c-4fe4-afd6-f63747d43e35 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:32.878607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ebbdd-a68c-4fe4-afd6-f63747d43e35 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:32.878875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ebbdd-a68c-4fe4-afd6-f63747d43e35 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:32.879159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ebbdd-a68c-4fe4-afd6-f63747d43e35 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/54da26a310c34bf4963f95566b80a440/users/ca6cede109334c74928e103c372003e9/roles/3e93eb1ce2f84ae79bb1c662e4bc4120/inherited_to_projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:32.883525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ebbdd-a68c-4fe4-afd6-f63747d43e35 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:create_grant(project_id=54da26a310c34bf4963f95566b80a440, user_id=ca6cede109334c74928e103c372003e9, role_id=3e93eb1ce2f84ae79bb1c662e4bc4120)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:32.886875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ebbdd-a68c-4fe4-afd6-f63747d43e35 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:32.990578 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3057/6157] 10.222.0.33 () {60 vars in 1581 bytes} [Sat May 28 21:25:32 2022] PUT /identity/v3/OS-INHERIT/projects/54da26a310c34bf4963f95566b80a440/users/ca6cede109334c74928e103c372003e9/roles/3e93eb1ce2f84ae79bb1c662e4bc4120/inherited_to_projects => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:32.991803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e46a1-7615-4821-92d5-b72940e227c9 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:33.025868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e46a1-7615-4821-92d5-b72940e227c9 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:33.208640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c362747-39bb-4ca4-8c72-6bdb5df212ed tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:33.211779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c362747-39bb-4ca4-8c72-6bdb5df212ed tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:33.217803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c362747-39bb-4ca4-8c72-6bdb5df212ed tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:33.218556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c362747-39bb-4ca4-8c72-6bdb5df212ed tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:33.221335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c362747-39bb-4ca4-8c72-6bdb5df212ed tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Adding query filter params (scope.project.id=733e53257a6347a49007bd4fc57d2469, user.id=ca6cede109334c74928e103c372003e9) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:25:33.222870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c362747-39bb-4ca4-8c72-6bdb5df212ed tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:33.226331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c362747-39bb-4ca4-8c72-6bdb5df212ed tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:33.447337 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3101/6158] 10.222.0.33 () {58 vars in 1355 bytes} [Sat May 28 21:25:32 2022] GET /identity/v3/role_assignments?scope.project.id=733e53257a6347a49007bd4fc57d2469&user.id=ca6cede109334c74928e103c372003e9&effective => generated 649 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:33.448588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ebbdd-a68c-4fe4-afd6-f63747d43e35 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:33.478151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ebbdd-a68c-4fe4-afd6-f63747d43e35 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:33.499374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbcd32-7a22-4e70-a2e9-a6a9bf6dc738 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:33.500266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbcd32-7a22-4e70-a2e9-a6a9bf6dc738 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:33.500709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbcd32-7a22-4e70-a2e9-a6a9bf6dc738 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:33.501004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdbcd32-7a22-4e70-a2e9-a6a9bf6dc738 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/OS-INHERIT/projects/54da26a310c34bf4963f95566b80a440/users/ca6cede109334c74928e103c372003e9/roles/3e93eb1ce2f84ae79bb1c662e4bc4120/inherited_to_projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:33.512015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdbcd32-7a22-4e70-a2e9-a6a9bf6dc738 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=54da26a310c34bf4963f95566b80a440, user_id=ca6cede109334c74928e103c372003e9, role_id=3e93eb1ce2f84ae79bb1c662e4bc4120)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:33.518426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdbcd32-7a22-4e70-a2e9-a6a9bf6dc738 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:33.540735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bfdbcd32-7a22-4e70-a2e9-a6a9bf6dc738 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invalidating the token cache because role 3e93eb1ce2f84ae79bb1c662e4bc4120 was removed from user ca6cede109334c74928e103c372003e9 on project 54da26a310c34bf4963f95566b80a440. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:33.544390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bfdbcd32-7a22-4e70-a2e9-a6a9bf6dc738 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:33.552446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bfdbcd32-7a22-4e70-a2e9-a6a9bf6dc738 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:33.637094 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3058/6159] 10.222.0.33 () {58 vars in 1565 bytes} [Sat May 28 21:25:33 2022] DELETE /identity/v3/OS-INHERIT/projects/54da26a310c34bf4963f95566b80a440/users/ca6cede109334c74928e103c372003e9/roles/3e93eb1ce2f84ae79bb1c662e4bc4120/inherited_to_projects => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:33.639656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c362747-39bb-4ca4-8c72-6bdb5df212ed tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:33.641826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4c362747-39bb-4ca4-8c72-6bdb5df212ed tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:33.660358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c362747-39bb-4ca4-8c72-6bdb5df212ed tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:33.768565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d547b85-6309-45bb-9637-6b6355ac891b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:33.770245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d547b85-6309-45bb-9637-6b6355ac891b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:33.771526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d547b85-6309-45bb-9637-6b6355ac891b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:33.772303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d547b85-6309-45bb-9637-6b6355ac891b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:33.776262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d547b85-6309-45bb-9637-6b6355ac891b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Adding query filter params (scope.project.id=733e53257a6347a49007bd4fc57d2469, user.id=ca6cede109334c74928e103c372003e9) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:25:33.776660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d547b85-6309-45bb-9637-6b6355ac891b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:33.782308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d547b85-6309-45bb-9637-6b6355ac891b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:33.803072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:33.907032 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3102/6160] 10.222.0.33 () {58 vars in 1355 bytes} [Sat May 28 21:25:33 2022] GET /identity/v3/role_assignments?scope.project.id=733e53257a6347a49007bd4fc57d2469&user.id=ca6cede109334c74928e103c372003e9&effective => generated 228 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:25:33.910354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbcd32-7a22-4e70-a2e9-a6a9bf6dc738 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:33.940204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdbcd32-7a22-4e70-a2e9-a6a9bf6dc738 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:33.952814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39c5f3-1b47-4318-87f6-e4c0e3e917b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:33.954072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39c5f3-1b47-4318-87f6-e4c0e3e917b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:33.954517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39c5f3-1b47-4318-87f6-e4c0e3e917b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:33.954949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39c5f3-1b47-4318-87f6-e4c0e3e917b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/projects/733e53257a6347a49007bd4fc57d2469` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:33.959887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39c5f3-1b47-4318-87f6-e4c0e3e917b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:delete_project(project_id=733e53257a6347a49007bd4fc57d2469)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:33.968465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39c5f3-1b47-4318-87f6-e4c0e3e917b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:34.213381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6c39c5f3-1b47-4318-87f6-e4c0e3e917b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '733e53257a6347a49007bd4fc57d2469', 'request_id': 'req-6c39c5f3-1b47-4318-87f6-e4c0e3e917b2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:34.228276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6c39c5f3-1b47-4318-87f6-e4c0e3e917b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '733e53257a6347a49007bd4fc57d2469', 'request_id': 'req-6c39c5f3-1b47-4318-87f6-e4c0e3e917b2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:34.351557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6c39c5f3-1b47-4318-87f6-e4c0e3e917b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '733e53257a6347a49007bd4fc57d2469', 'request_id': 'req-6c39c5f3-1b47-4318-87f6-e4c0e3e917b2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:34.376422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6c39c5f3-1b47-4318-87f6-e4c0e3e917b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] The token cache is being invalidate because project 733e53257a6347a49007bd4fc57d2469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:34.376886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6c39c5f3-1b47-4318-87f6-e4c0e3e917b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:34.377687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6c39c5f3-1b47-4318-87f6-e4c0e3e917b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:34.380048 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3059/6161] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:33 2022] DELETE /identity/v3/projects/733e53257a6347a49007bd4fc57d2469 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:34.384313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d547b85-6309-45bb-9637-6b6355ac891b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:34.387499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8d547b85-6309-45bb-9637-6b6355ac891b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:34.419427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d547b85-6309-45bb-9637-6b6355ac891b tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:34.464024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:34.576460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6055a0d-bafd-4aeb-bc7e-39f6998abaf0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:34.587056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6055a0d-bafd-4aeb-bc7e-39f6998abaf0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:34.587491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6055a0d-bafd-4aeb-bc7e-39f6998abaf0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:34.587786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6055a0d-bafd-4aeb-bc7e-39f6998abaf0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/roles/3e93eb1ce2f84ae79bb1c662e4bc4120` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:34.589556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6055a0d-bafd-4aeb-bc7e-39f6998abaf0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:delete_role(role_id=3e93eb1ce2f84ae79bb1c662e4bc4120)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:34.591352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6055a0d-bafd-4aeb-bc7e-39f6998abaf0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:34.750984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d6055a0d-bafd-4aeb-bc7e-39f6998abaf0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invalidating the token cache because role 3e93eb1ce2f84ae79bb1c662e4bc4120 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:34.752747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d6055a0d-bafd-4aeb-bc7e-39f6998abaf0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:34.752747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d6055a0d-bafd-4aeb-bc7e-39f6998abaf0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:34.754022 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3103/6162] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:34 2022] DELETE /identity/v3/roles/3e93eb1ce2f84ae79bb1c662e4bc4120 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:34.759086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39c5f3-1b47-4318-87f6-e4c0e3e917b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:34.776062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39c5f3-1b47-4318-87f6-e4c0e3e917b2 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:34.879199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a87fb-464a-43fd-8a59-208264b4ab0d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:34.880457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a87fb-464a-43fd-8a59-208264b4ab0d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:34.881224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a87fb-464a-43fd-8a59-208264b4ab0d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:34.881809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a87fb-464a-43fd-8a59-208264b4ab0d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/users/ca6cede109334c74928e103c372003e9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:34.884436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a87fb-464a-43fd-8a59-208264b4ab0d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:delete_user(user_id=ca6cede109334c74928e103c372003e9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:34.887756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a87fb-464a-43fd-8a59-208264b4ab0d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:35.064820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ae9a87fb-464a-43fd-8a59-208264b4ab0d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca6cede109334c74928e103c372003e9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:35.065488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ae9a87fb-464a-43fd-8a59-208264b4ab0d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca6cede109334c74928e103c372003e9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:35.085149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ae9a87fb-464a-43fd-8a59-208264b4ab0d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca6cede109334c74928e103c372003e9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:35.151390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ae9a87fb-464a-43fd-8a59-208264b4ab0d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca6cede109334c74928e103c372003e9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:35.152080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ae9a87fb-464a-43fd-8a59-208264b4ab0d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca6cede109334c74928e103c372003e9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:35.191920 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3060/6163] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:34 2022] DELETE /identity/v3/users/ca6cede109334c74928e103c372003e9 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:35.195111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6055a0d-bafd-4aeb-bc7e-39f6998abaf0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:35.198867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d6055a0d-bafd-4aeb-bc7e-39f6998abaf0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:35.230543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6055a0d-bafd-4aeb-bc7e-39f6998abaf0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:35.328691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:35.385489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635e8095-3d13-40b2-b4a3-cd8dd8ec1bed tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:35.387813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e8095-3d13-40b2-b4a3-cd8dd8ec1bed tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:35.388115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e8095-3d13-40b2-b4a3-cd8dd8ec1bed tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:35.388647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635e8095-3d13-40b2-b4a3-cd8dd8ec1bed tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/groups/0b77d389cac94d88ade184abe9f137c9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:35.394223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e8095-3d13-40b2-b4a3-cd8dd8ec1bed tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:delete_group(group_id=0b77d389cac94d88ade184abe9f137c9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:35.396302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635e8095-3d13-40b2-b4a3-cd8dd8ec1bed tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:35.554736 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3104/6164] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:25:35 2022] DELETE /identity/v3/groups/0b77d389cac94d88ade184abe9f137c9 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:35.559476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a87fb-464a-43fd-8a59-208264b4ab0d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:35.560936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ae9a87fb-464a-43fd-8a59-208264b4ab0d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:35.578514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a87fb-464a-43fd-8a59-208264b4ab0d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:35.714297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67220283-9f80-4759-8676-976648c8e39d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:35.715196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67220283-9f80-4759-8676-976648c8e39d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:35.715487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67220283-9f80-4759-8676-976648c8e39d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:35.715769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67220283-9f80-4759-8676-976648c8e39d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/projects/54da26a310c34bf4963f95566b80a440` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:35.717054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67220283-9f80-4759-8676-976648c8e39d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:delete_project(project_id=54da26a310c34bf4963f95566b80a440)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:35.719047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67220283-9f80-4759-8676-976648c8e39d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:35.761577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba6a0a-45ae-411a-a1f4-ae4b2bd502f3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:35.762881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba6a0a-45ae-411a-a1f4-ae4b2bd502f3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:35.763398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ba6a0a-45ae-411a-a1f4-ae4b2bd502f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:35.798260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-40ba6a0a-45ae-411a-a1f4-ae4b2bd502f3 None None] MFA Rules not processed for user `f3468a0b2d324603ab6c3080993bc0c8`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:35.800316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-40ba6a0a-45ae-411a-a1f4-ae4b2bd502f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:35.921956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-67220283-9f80-4759-8676-976648c8e39d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54da26a310c34bf4963f95566b80a440', 'request_id': 'req-67220283-9f80-4759-8676-976648c8e39d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:35.949437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-67220283-9f80-4759-8676-976648c8e39d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54da26a310c34bf4963f95566b80a440', 'request_id': 'req-67220283-9f80-4759-8676-976648c8e39d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:35.973784 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3105/6165] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:25:35.998638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:36.026164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-67220283-9f80-4759-8676-976648c8e39d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54da26a310c34bf4963f95566b80a440', 'request_id': 'req-67220283-9f80-4759-8676-976648c8e39d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:36.047519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-67220283-9f80-4759-8676-976648c8e39d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] The token cache is being invalidate because project 54da26a310c34bf4963f95566b80a440 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:36.049081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-67220283-9f80-4759-8676-976648c8e39d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:36.051881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-67220283-9f80-4759-8676-976648c8e39d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:36.062223 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3061/6166] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:35 2022] DELETE /identity/v3/projects/54da26a310c34bf4963f95566b80a440 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:36.062766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba6a0a-45ae-411a-a1f4-ae4b2bd502f3 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:36.101042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ba6a0a-45ae-411a-a1f4-ae4b2bd502f3 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:36.312858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265709b-cf39-4193-9d61-a960935a1575 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:36.313942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265709b-cf39-4193-9d61-a960935a1575 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:36.314048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265709b-cf39-4193-9d61-a960935a1575 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:36.314412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265709b-cf39-4193-9d61-a960935a1575 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/domains/a720f7c59b8c4dd5ac1a55d1b511c737` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:36.329094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265709b-cf39-4193-9d61-a960935a1575 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=a720f7c59b8c4dd5ac1a55d1b511c737)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:36.330857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265709b-cf39-4193-9d61-a960935a1575 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:36.362443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:36.419161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1265709b-cf39-4193-9d61-a960935a1575 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'a720f7c59b8c4dd5ac1a55d1b511c737'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:36.421177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1265709b-cf39-4193-9d61-a960935a1575 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'a720f7c59b8c4dd5ac1a55d1b511c737'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:36.460311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:36.547154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1265709b-cf39-4193-9d61-a960935a1575 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'a720f7c59b8c4dd5ac1a55d1b511c737'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:36.548005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1265709b-cf39-4193-9d61-a960935a1575 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'a720f7c59b8c4dd5ac1a55d1b511c737'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:36.560617 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3106/6167] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:25:36 2022] PATCH /identity/v3/domains/a720f7c59b8c4dd5ac1a55d1b511c737 => generated 277 bytes in 498 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:36.564541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67220283-9f80-4759-8676-976648c8e39d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:36.566054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-67220283-9f80-4759-8676-976648c8e39d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:36.599388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67220283-9f80-4759-8676-976648c8e39d tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:36.729517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bc9f29063544fababe76c6d5fe9a671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9c4ce39d915d4ed3ae2e48befd81e882', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:36.730526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:36.731544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:36.731544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] PATH_INFO: `/v3/domains/a720f7c59b8c4dd5ac1a55d1b511c737` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:36.744825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=a720f7c59b8c4dd5ac1a55d1b511c737)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:36.746602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:36.777400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'a720f7c59b8c4dd5ac1a55d1b511c737'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:36.781474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'a720f7c59b8c4dd5ac1a55d1b511c737'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:36.916236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a720f7c59b8c4dd5ac1a55d1b511c737', 'request_id': 'req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:36.927453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a720f7c59b8c4dd5ac1a55d1b511c737', 'request_id': 'req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:37.082435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a720f7c59b8c4dd5ac1a55d1b511c737', 'request_id': 'req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:37.102768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] The token cache is being invalidate because project a720f7c59b8c4dd5ac1a55d1b511c737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:37.103344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:37.105632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:37.133901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'a720f7c59b8c4dd5ac1a55d1b511c737', 'request_id': 'req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:37.140196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'a720f7c59b8c4dd5ac1a55d1b511c737', 'request_id': 'req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:37.144774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'a720f7c59b8c4dd5ac1a55d1b511c737', 'request_id': 'req-802df6bf-2afb-47fa-b6c4-ac9af9633ef0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:37.152984 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3062/6168] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:25:36 2022] DELETE /identity/v3/domains/a720f7c59b8c4dd5ac1a55d1b511c737 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:25:37.157273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265709b-cf39-4193-9d61-a960935a1575 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:37.158892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1265709b-cf39-4193-9d61-a960935a1575 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:37.247366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265709b-cf39-4193-9d61-a960935a1575 tempest-InheritsV3TestJSON-955179633 tempest-InheritsV3TestJSON-955179633-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:37.320580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa4dcb-f9c1-4e44-8f15-2530926f5e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:37.320580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa4dcb-f9c1-4e44-8f15-2530926f5e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:37.320580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa4dcb-f9c1-4e44-8f15-2530926f5e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:37.320580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa4dcb-f9c1-4e44-8f15-2530926f5e8c admin admin] PATH_INFO: `/v3/users/2ac8583f518a45b7a8ed7618ab4f8cf4` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:37.320959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa4dcb-f9c1-4e44-8f15-2530926f5e8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ac8583f518a45b7a8ed7618ab4f8cf4)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:37.322064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa4dcb-f9c1-4e44-8f15-2530926f5e8c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:37.427116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:37.635806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b7fa4dcb-f9c1-4e44-8f15-2530926f5e8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ac8583f518a45b7a8ed7618ab4f8cf4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:37.671438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b7fa4dcb-f9c1-4e44-8f15-2530926f5e8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ac8583f518a45b7a8ed7618ab4f8cf4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:37.675543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b7fa4dcb-f9c1-4e44-8f15-2530926f5e8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ac8583f518a45b7a8ed7618ab4f8cf4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:37.677144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b7fa4dcb-f9c1-4e44-8f15-2530926f5e8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ac8583f518a45b7a8ed7618ab4f8cf4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:37.775153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b7fa4dcb-f9c1-4e44-8f15-2530926f5e8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ac8583f518a45b7a8ed7618ab4f8cf4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:37.790140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:UPDATE=1,keystone:DELETE=8,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:37.835862 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3107/6169] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:37 2022] DELETE /identity/v3/users/2ac8583f518a45b7a8ed7618ab4f8cf4 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:37.837420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacef536-1ab2-438d-a24d-b364ce75745a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:37.837613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacef536-1ab2-438d-a24d-b364ce75745a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:37.837830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacef536-1ab2-438d-a24d-b364ce75745a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:37.866878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bacef536-1ab2-438d-a24d-b364ce75745a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:37.869336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bacef536-1ab2-438d-a24d-b364ce75745a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:37.997699 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3063/6170] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:25:38.013422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa4dcb-f9c1-4e44-8f15-2530926f5e8c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:38.015258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b7fa4dcb-f9c1-4e44-8f15-2530926f5e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:38.236776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa4dcb-f9c1-4e44-8f15-2530926f5e8c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:38.273300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8464fa2-d08b-46b8-b0b1-8dda3b0e89e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:38.275509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8464fa2-d08b-46b8-b0b1-8dda3b0e89e3 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:38.281374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8464fa2-d08b-46b8-b0b1-8dda3b0e89e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:38.282027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8464fa2-d08b-46b8-b0b1-8dda3b0e89e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:38.329466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8464fa2-d08b-46b8-b0b1-8dda3b0e89e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:38.334800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8464fa2-d08b-46b8-b0b1-8dda3b0e89e3 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:38.375343 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3108/6171] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:25:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:25:38.802141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:39.488509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacef536-1ab2-438d-a24d-b364ce75745a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:39.490715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bacef536-1ab2-438d-a24d-b364ce75745a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:39.511715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacef536-1ab2-438d-a24d-b364ce75745a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:39.541138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286d305-2cd8-4f38-af53-43bfcab8289e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:39.542489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286d305-2cd8-4f38-af53-43bfcab8289e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:39.544639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286d305-2cd8-4f38-af53-43bfcab8289e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:39.545135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286d305-2cd8-4f38-af53-43bfcab8289e admin admin] PATH_INFO: `/v3/projects/501a1f1a443b42f1b6b247de8ce20ae6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:39.549022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286d305-2cd8-4f38-af53-43bfcab8289e admin admin] RBAC: Authorizing `identity:delete_project(project_id=501a1f1a443b42f1b6b247de8ce20ae6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:39.563809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286d305-2cd8-4f38-af53-43bfcab8289e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:39.782858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a286d305-2cd8-4f38-af53-43bfcab8289e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '501a1f1a443b42f1b6b247de8ce20ae6', 'request_id': 'req-a286d305-2cd8-4f38-af53-43bfcab8289e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:39.801077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a286d305-2cd8-4f38-af53-43bfcab8289e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '501a1f1a443b42f1b6b247de8ce20ae6', 'request_id': 'req-a286d305-2cd8-4f38-af53-43bfcab8289e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:39.891524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a286d305-2cd8-4f38-af53-43bfcab8289e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '501a1f1a443b42f1b6b247de8ce20ae6', 'request_id': 'req-a286d305-2cd8-4f38-af53-43bfcab8289e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:39.907184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a286d305-2cd8-4f38-af53-43bfcab8289e admin admin] The token cache is being invalidate because project 501a1f1a443b42f1b6b247de8ce20ae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:39.907853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a286d305-2cd8-4f38-af53-43bfcab8289e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:39.908797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a286d305-2cd8-4f38-af53-43bfcab8289e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:39.910627 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3064/6172] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:39 2022] DELETE /identity/v3/projects/501a1f1a443b42f1b6b247de8ce20ae6 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:39.915164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8464fa2-d08b-46b8-b0b1-8dda3b0e89e3 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:39.934952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8464fa2-d08b-46b8-b0b1-8dda3b0e89e3 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:40.019105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7f87f-abf6-4ef2-8c51-10279869dbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:40.020463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7f87f-abf6-4ef2-8c51-10279869dbc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:40.020906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7f87f-abf6-4ef2-8c51-10279869dbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:40.022149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7f87f-abf6-4ef2-8c51-10279869dbc9 admin admin] PATH_INFO: `/v3/users/7bc9f29063544fababe76c6d5fe9a671` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:40.023108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7f87f-abf6-4ef2-8c51-10279869dbc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bc9f29063544fababe76c6d5fe9a671)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:40.025805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa7f87f-abf6-4ef2-8c51-10279869dbc9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:40.333418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08285d08-02f7-4a79-84f0-109313db7428 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:40.333740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08285d08-02f7-4a79-84f0-109313db7428 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:40.334583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08285d08-02f7-4a79-84f0-109313db7428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:40.346377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8fa7f87f-abf6-4ef2-8c51-10279869dbc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bc9f29063544fababe76c6d5fe9a671'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:40.370034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8fa7f87f-abf6-4ef2-8c51-10279869dbc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bc9f29063544fababe76c6d5fe9a671'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:40.370034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8fa7f87f-abf6-4ef2-8c51-10279869dbc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bc9f29063544fababe76c6d5fe9a671'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:40.370034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8fa7f87f-abf6-4ef2-8c51-10279869dbc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bc9f29063544fababe76c6d5fe9a671'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:40.380268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-08285d08-02f7-4a79-84f0-109313db7428 None None] MFA Rules not processed for user `ebc8e00965334bd1bb811a0eefd64b84`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:40.391160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-08285d08-02f7-4a79-84f0-109313db7428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:40.450248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8fa7f87f-abf6-4ef2-8c51-10279869dbc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bc9f29063544fababe76c6d5fe9a671'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:40.511230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:40.542087 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3065/6173] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:40.552851 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3109/6174] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:39 2022] DELETE /identity/v3/users/7bc9f29063544fababe76c6d5fe9a671 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:25:41.225660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaacf7a-9180-4905-afe3-46e3726fdf64 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:41.226151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaacf7a-9180-4905-afe3-46e3726fdf64 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:41.228737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaacf7a-9180-4905-afe3-46e3726fdf64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:41.278833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7eaacf7a-9180-4905-afe3-46e3726fdf64 None None] MFA Rules not processed for user `ebc8e00965334bd1bb811a0eefd64b84`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:41.280186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7eaacf7a-9180-4905-afe3-46e3726fdf64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:41.378022 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3066/6175] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:42.200576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7f87f-abf6-4ef2-8c51-10279869dbc9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:42.201458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8fa7f87f-abf6-4ef2-8c51-10279869dbc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:42.213500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:42.294418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7f87f-abf6-4ef2-8c51-10279869dbc9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:42.304828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4458e-0167-43c8-b724-ff310f0c532e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:42.307240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4458e-0167-43c8-b724-ff310f0c532e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:42.315564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4458e-0167-43c8-b724-ff310f0c532e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:42.315564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4458e-0167-43c8-b724-ff310f0c532e admin admin] PATH_INFO: `/v3/projects/9c4ce39d915d4ed3ae2e48befd81e882` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:42.315564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4458e-0167-43c8-b724-ff310f0c532e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4ce39d915d4ed3ae2e48befd81e882)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:42.315564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c4458e-0167-43c8-b724-ff310f0c532e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:42.453689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-52c4458e-0167-43c8-b724-ff310f0c532e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4ce39d915d4ed3ae2e48befd81e882', 'request_id': 'req-52c4458e-0167-43c8-b724-ff310f0c532e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:42.475432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-52c4458e-0167-43c8-b724-ff310f0c532e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4ce39d915d4ed3ae2e48befd81e882', 'request_id': 'req-52c4458e-0167-43c8-b724-ff310f0c532e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:42.593958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-52c4458e-0167-43c8-b724-ff310f0c532e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4ce39d915d4ed3ae2e48befd81e882', 'request_id': 'req-52c4458e-0167-43c8-b724-ff310f0c532e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:42.606542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-52c4458e-0167-43c8-b724-ff310f0c532e admin admin] The token cache is being invalidate because project 9c4ce39d915d4ed3ae2e48befd81e882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:42.606833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-52c4458e-0167-43c8-b724-ff310f0c532e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:42.607672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-52c4458e-0167-43c8-b724-ff310f0c532e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:42.609346 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3110/6176] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:42 2022] DELETE /identity/v3/projects/9c4ce39d915d4ed3ae2e48befd81e882 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:42.614529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f8089-f48e-4815-9bcc-e7dc0d755e41 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:42.615327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f8089-f48e-4815-9bcc-e7dc0d755e41 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:42.615634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f8089-f48e-4815-9bcc-e7dc0d755e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:42.642490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2d4f8089-f48e-4815-9bcc-e7dc0d755e41 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:42.643780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2d4f8089-f48e-4815-9bcc-e7dc0d755e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:42.760301 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3067/6177] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:42.769176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966674dc-63e3-446a-b41e-28cd1019fbb6 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:42.770406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966674dc-63e3-446a-b41e-28cd1019fbb6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:42.770844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966674dc-63e3-446a-b41e-28cd1019fbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:42.806046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-966674dc-63e3-446a-b41e-28cd1019fbb6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:42.807842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-966674dc-63e3-446a-b41e-28cd1019fbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:42.845054 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3111/6178] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:42.852651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f8089-f48e-4815-9bcc-e7dc0d755e41 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:42.868122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f8089-f48e-4815-9bcc-e7dc0d755e41 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:42.899902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a6472-7684-4a4c-9e0b-c0f15226e789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:42.899902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a6472-7684-4a4c-9e0b-c0f15226e789 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:42.899902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a6472-7684-4a4c-9e0b-c0f15226e789 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:42.899902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a6472-7684-4a4c-9e0b-c0f15226e789 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:42.900816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a6472-7684-4a4c-9e0b-c0f15226e789 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:25:42.900816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a6472-7684-4a4c-9e0b-c0f15226e789 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:42.903996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a6472-7684-4a4c-9e0b-c0f15226e789 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:42.939005 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3068/6179] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:42.939520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966674dc-63e3-446a-b41e-28cd1019fbb6 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:42.956647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966674dc-63e3-446a-b41e-28cd1019fbb6 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:42.963994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df23f5a-87b2-4fc9-b5e7-a6735cd8cb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:42.964730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df23f5a-87b2-4fc9-b5e7-a6735cd8cb42 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:42.965013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df23f5a-87b2-4fc9-b5e7-a6735cd8cb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:42.965296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df23f5a-87b2-4fc9-b5e7-a6735cd8cb42 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:42.966223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df23f5a-87b2-4fc9-b5e7-a6735cd8cb42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:42.968352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df23f5a-87b2-4fc9-b5e7-a6735cd8cb42 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:43.117548 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3112/6180] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:42 2022] POST /identity/v3/projects => generated 369 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:43.120783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a6472-7684-4a4c-9e0b-c0f15226e789 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:43.135143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a6472-7684-4a4c-9e0b-c0f15226e789 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:43.196652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df23f5a-87b2-4fc9-b5e7-a6735cd8cb42 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:43.198798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6df23f5a-87b2-4fc9-b5e7-a6735cd8cb42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:43.216843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327da894-e792-45be-bc24-734ee9c76181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:43.217574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327da894-e792-45be-bc24-734ee9c76181 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:43.217931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327da894-e792-45be-bc24-734ee9c76181 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:43.218222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327da894-e792-45be-bc24-734ee9c76181 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:43.218931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327da894-e792-45be-bc24-734ee9c76181 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:43.220883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327da894-e792-45be-bc24-734ee9c76181 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:43.290839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df23f5a-87b2-4fc9-b5e7-a6735cd8cb42 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:43.301131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ab704-d899-406f-bc01-b3ed073d3ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:43.301131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ab704-d899-406f-bc01-b3ed073d3ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:43.301131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ab704-d899-406f-bc01-b3ed073d3ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:43.301131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ab704-d899-406f-bc01-b3ed073d3ca9 admin admin] PATH_INFO: `/v3/users/6c97894b4544419ea9f91ec0a4fe4c41` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:43.301449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ab704-d899-406f-bc01-b3ed073d3ca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c97894b4544419ea9f91ec0a4fe4c41)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:43.302052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ab704-d899-406f-bc01-b3ed073d3ca9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:43.317004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:43.327741 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3069/6181] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:25:43 2022] POST /identity/v3/users => generated 285 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:43.334685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327da894-e792-45be-bc24-734ee9c76181 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:43.346587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327da894-e792-45be-bc24-734ee9c76181 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:43.354275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5071edb-2f88-4386-b509-7629cda4ee6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:43.354964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5071edb-2f88-4386-b509-7629cda4ee6a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:43.355251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5071edb-2f88-4386-b509-7629cda4ee6a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:43.355534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5071edb-2f88-4386-b509-7629cda4ee6a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:43.356282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5071edb-2f88-4386-b509-7629cda4ee6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:43.357958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5071edb-2f88-4386-b509-7629cda4ee6a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:43.370408 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3070/6182] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:25:43.374319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5071edb-2f88-4386-b509-7629cda4ee6a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:43.385616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5071edb-2f88-4386-b509-7629cda4ee6a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:43.390692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b5e47-2233-4db4-bd5a-af004abdca08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:43.391361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b5e47-2233-4db4-bd5a-af004abdca08 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:43.391649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b5e47-2233-4db4-bd5a-af004abdca08 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:43.392015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b5e47-2233-4db4-bd5a-af004abdca08 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:43.392659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b5e47-2233-4db4-bd5a-af004abdca08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:43.394303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b5e47-2233-4db4-bd5a-af004abdca08 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:43.396767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:43.407108 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3071/6183] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:25:43.412353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b5e47-2233-4db4-bd5a-af004abdca08 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:43.423031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b5e47-2233-4db4-bd5a-af004abdca08 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:43.428517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc6591-18a4-44e9-9480-051f4b110651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:43.429444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc6591-18a4-44e9-9480-051f4b110651 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:43.429763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc6591-18a4-44e9-9480-051f4b110651 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:43.430091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc6591-18a4-44e9-9480-051f4b110651 admin admin] PATH_INFO: `/v3/projects/f851982a0cf6417cbde471869958da3a/users/499596c2b18248deb72e59b42859df04/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:43.448298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc6591-18a4-44e9-9480-051f4b110651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f851982a0cf6417cbde471869958da3a, user_id=499596c2b18248deb72e59b42859df04, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:43.451038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc6591-18a4-44e9-9480-051f4b110651 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:43.543385 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3072/6184] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:25:43 2022] PUT /identity/v3/projects/f851982a0cf6417cbde471869958da3a/users/499596c2b18248deb72e59b42859df04/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:43.554126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-780ab704-d899-406f-bc01-b3ed073d3ca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c97894b4544419ea9f91ec0a4fe4c41'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:43.557062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf3716-5cf1-45be-bd13-4e8d1cd97aea None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:43.557762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf3716-5cf1-45be-bd13-4e8d1cd97aea None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:43.558231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcf3716-5cf1-45be-bd13-4e8d1cd97aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:43.563685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-780ab704-d899-406f-bc01-b3ed073d3ca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c97894b4544419ea9f91ec0a4fe4c41'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:43.564142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-780ab704-d899-406f-bc01-b3ed073d3ca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c97894b4544419ea9f91ec0a4fe4c41'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:43.564788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-780ab704-d899-406f-bc01-b3ed073d3ca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c97894b4544419ea9f91ec0a4fe4c41'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:43.589010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-afcf3716-5cf1-45be-bd13-4e8d1cd97aea None None] MFA Rules not processed for user `499596c2b18248deb72e59b42859df04`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:43.590366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-afcf3716-5cf1-45be-bd13-4e8d1cd97aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:43.649061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-780ab704-d899-406f-bc01-b3ed073d3ca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c97894b4544419ea9f91ec0a4fe4c41'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:43.675241 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3113/6185] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:43 2022] DELETE /identity/v3/users/6c97894b4544419ea9f91ec0a4fe4c41 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:43.676783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c60ad3-12a1-4a29-b529-49356119516f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:43.677080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c60ad3-12a1-4a29-b529-49356119516f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:43.677369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c60ad3-12a1-4a29-b529-49356119516f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:43.706720 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3073/6186] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:25:43.708226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078bdce-b4a4-494a-b50b-1dbca5f02ef5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:43.708575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078bdce-b4a4-494a-b50b-1dbca5f02ef5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:43.708861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7078bdce-b4a4-494a-b50b-1dbca5f02ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:43.717358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c6c60ad3-12a1-4a29-b529-49356119516f None None] MFA Rules not processed for user `02d5fbb072814fd7923ecf12c7ecad7b`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:43.718710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c6c60ad3-12a1-4a29-b529-49356119516f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:43.729308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7078bdce-b4a4-494a-b50b-1dbca5f02ef5 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:43.731343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7078bdce-b4a4-494a-b50b-1dbca5f02ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:43.799337 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3114/6187] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:43.800381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c60ad3-12a1-4a29-b529-49356119516f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:43.801697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c6c60ad3-12a1-4a29-b529-49356119516f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:43.849910 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3074/6188] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:43.865811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078bdce-b4a4-494a-b50b-1dbca5f02ef5 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:43.874072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7078bdce-b4a4-494a-b50b-1dbca5f02ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:43.892094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c60ad3-12a1-4a29-b529-49356119516f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:43.898470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9265bd1-a628-431a-8121-46eac0fd6937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:43.899343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9265bd1-a628-431a-8121-46eac0fd6937 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:43.899722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9265bd1-a628-431a-8121-46eac0fd6937 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:43.900008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9265bd1-a628-431a-8121-46eac0fd6937 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:43.900706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9265bd1-a628-431a-8121-46eac0fd6937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:43.904063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9265bd1-a628-431a-8121-46eac0fd6937 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:43.964437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7078bdce-b4a4-494a-b50b-1dbca5f02ef5 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:43.968351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dc0fc-5b90-4ae4-856a-f9bee4c7c1f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:43.969024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dc0fc-5b90-4ae4-856a-f9bee4c7c1f0 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:43.969617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dc0fc-5b90-4ae4-856a-f9bee4c7c1f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:43.970095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710dc0fc-5b90-4ae4-856a-f9bee4c7c1f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:43.982976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710dc0fc-5b90-4ae4-856a-f9bee4c7c1f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:43.984819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710dc0fc-5b90-4ae4-856a-f9bee4c7c1f0 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:43.998639 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3115/6189] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:43 2022] POST /identity/v3/projects => generated 369 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:44.000261 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3075/6190] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:25:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:25:44.003500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9265bd1-a628-431a-8121-46eac0fd6937 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:44.105340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9265bd1-a628-431a-8121-46eac0fd6937 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:44.114903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a62543-bf4c-4389-be7b-b528b61fe7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:44.117992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a62543-bf4c-4389-be7b-b528b61fe7db admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:44.118288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a62543-bf4c-4389-be7b-b528b61fe7db admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:44.118555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a62543-bf4c-4389-be7b-b528b61fe7db admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:44.119614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a62543-bf4c-4389-be7b-b528b61fe7db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:44.121760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a62543-bf4c-4389-be7b-b528b61fe7db admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:44.220614 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3116/6191] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:25:44 2022] POST /identity/v3/users => generated 291 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:44.221825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dc0fc-5b90-4ae4-856a-f9bee4c7c1f0 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:44.234641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710dc0fc-5b90-4ae4-856a-f9bee4c7c1f0 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:44.240071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544f751-d5e1-40fd-b0ca-ca1b420bd100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:44.241567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544f751-d5e1-40fd-b0ca-ca1b420bd100 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:44.241567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544f751-d5e1-40fd-b0ca-ca1b420bd100 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:44.243073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c544f751-d5e1-40fd-b0ca-ca1b420bd100 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:44.244217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544f751-d5e1-40fd-b0ca-ca1b420bd100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:44.245967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c544f751-d5e1-40fd-b0ca-ca1b420bd100 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:44.247728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:44.261392 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3076/6192] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:25:44.263333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a62543-bf4c-4389-be7b-b528b61fe7db admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:44.274384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a62543-bf4c-4389-be7b-b528b61fe7db admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:44.278062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:44.281353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460f6bd-bba9-40cc-a01a-1789232833bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:44.288209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460f6bd-bba9-40cc-a01a-1789232833bd admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:44.288209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460f6bd-bba9-40cc-a01a-1789232833bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:44.288209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460f6bd-bba9-40cc-a01a-1789232833bd admin admin] PATH_INFO: `/v3/projects/31f42fca38a6448eb1d1cc64062a8ca7/users/9b4f328d2b2d476fa245063334995618/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:44.300736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460f6bd-bba9-40cc-a01a-1789232833bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=31f42fca38a6448eb1d1cc64062a8ca7, user_id=9b4f328d2b2d476fa245063334995618, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:44.303523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460f6bd-bba9-40cc-a01a-1789232833bd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:44.343380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544f751-d5e1-40fd-b0ca-ca1b420bd100 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:44.345217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c544f751-d5e1-40fd-b0ca-ca1b420bd100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:44.363970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c544f751-d5e1-40fd-b0ca-ca1b420bd100 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:44.369098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f18ba-f82f-4303-bbbb-ea44d4dcbf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:44.371464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f18ba-f82f-4303-bbbb-ea44d4dcbf3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:44.371464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f18ba-f82f-4303-bbbb-ea44d4dcbf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:44.374014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f18ba-f82f-4303-bbbb-ea44d4dcbf3e admin admin] PATH_INFO: `/v3/projects/322f969ab3e143fc8bb2447775841516` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:44.375898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f18ba-f82f-4303-bbbb-ea44d4dcbf3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=322f969ab3e143fc8bb2447775841516)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:44.378991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f18ba-f82f-4303-bbbb-ea44d4dcbf3e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:44.423084 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3117/6193] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:25:44 2022] PUT /identity/v3/projects/31f42fca38a6448eb1d1cc64062a8ca7/users/9b4f328d2b2d476fa245063334995618/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:44.427623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460f6bd-bba9-40cc-a01a-1789232833bd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:44.508333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460f6bd-bba9-40cc-a01a-1789232833bd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:44.518005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5f9f18ba-f82f-4303-bbbb-ea44d4dcbf3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '322f969ab3e143fc8bb2447775841516', 'request_id': 'req-5f9f18ba-f82f-4303-bbbb-ea44d4dcbf3e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:44.524601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5f9f18ba-f82f-4303-bbbb-ea44d4dcbf3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '322f969ab3e143fc8bb2447775841516', 'request_id': 'req-5f9f18ba-f82f-4303-bbbb-ea44d4dcbf3e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:44.571188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e20d6b-80dc-44a1-8b19-2d7a9077c040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:44.571873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e20d6b-80dc-44a1-8b19-2d7a9077c040 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:44.572400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e20d6b-80dc-44a1-8b19-2d7a9077c040 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:44.572767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e20d6b-80dc-44a1-8b19-2d7a9077c040 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:44.573653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e20d6b-80dc-44a1-8b19-2d7a9077c040 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:44.575326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e20d6b-80dc-44a1-8b19-2d7a9077c040 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:44.585755 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3118/6194] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:25:44.590804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e20d6b-80dc-44a1-8b19-2d7a9077c040 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:44.602135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e20d6b-80dc-44a1-8b19-2d7a9077c040 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:44.607292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaddaf55-2ecc-41ac-ac00-f2bc8e0b09a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:44.608191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddaf55-2ecc-41ac-ac00-f2bc8e0b09a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:44.608478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddaf55-2ecc-41ac-ac00-f2bc8e0b09a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:44.608757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaddaf55-2ecc-41ac-ac00-f2bc8e0b09a2 admin admin] PATH_INFO: `/v3/projects/31f42fca38a6448eb1d1cc64062a8ca7/users/9b4f328d2b2d476fa245063334995618/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:44.611026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaddaf55-2ecc-41ac-ac00-f2bc8e0b09a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31f42fca38a6448eb1d1cc64062a8ca7, user_id=9b4f328d2b2d476fa245063334995618, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:44.613890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaddaf55-2ecc-41ac-ac00-f2bc8e0b09a2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:44.625446 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3119/6195] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:25:44 2022] PUT /identity/v3/projects/31f42fca38a6448eb1d1cc64062a8ca7/users/9b4f328d2b2d476fa245063334995618/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:44.636541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598e6d2-b1ce-4ed0-8f00-0063891050e8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:44.636838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598e6d2-b1ce-4ed0-8f00-0063891050e8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:44.637120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598e6d2-b1ce-4ed0-8f00-0063891050e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:44.672778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c598e6d2-b1ce-4ed0-8f00-0063891050e8 None None] MFA Rules not processed for user `9b4f328d2b2d476fa245063334995618`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:44.674142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c598e6d2-b1ce-4ed0-8f00-0063891050e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:44.688305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5f9f18ba-f82f-4303-bbbb-ea44d4dcbf3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '322f969ab3e143fc8bb2447775841516', 'request_id': 'req-5f9f18ba-f82f-4303-bbbb-ea44d4dcbf3e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:44.704712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5f9f18ba-f82f-4303-bbbb-ea44d4dcbf3e admin admin] The token cache is being invalidate because project 322f969ab3e143fc8bb2447775841516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:44.705756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5f9f18ba-f82f-4303-bbbb-ea44d4dcbf3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:44.710153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5f9f18ba-f82f-4303-bbbb-ea44d4dcbf3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:44.714083 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3077/6196] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:44 2022] DELETE /identity/v3/projects/322f969ab3e143fc8bb2447775841516 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:25:44.719626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf78233-2108-433e-9efa-bf707359e21d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:44.720363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf78233-2108-433e-9efa-bf707359e21d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:44.720800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf78233-2108-433e-9efa-bf707359e21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:44.745188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-cdf78233-2108-433e-9efa-bf707359e21d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:44.746860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cdf78233-2108-433e-9efa-bf707359e21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:44.810708 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3120/6197] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:44 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:25:44.811681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598e6d2-b1ce-4ed0-8f00-0063891050e8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:44.823241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598e6d2-b1ce-4ed0-8f00-0063891050e8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:44.828344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a20f6-c76e-492b-9287-6296ffe88496 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:44.829020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a20f6-c76e-492b-9287-6296ffe88496 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:44.829312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a20f6-c76e-492b-9287-6296ffe88496 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:44.829597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a20f6-c76e-492b-9287-6296ffe88496 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:44.830411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a20f6-c76e-492b-9287-6296ffe88496 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:44.833347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a20f6-c76e-492b-9287-6296ffe88496 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:44.837421 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3078/6198] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:44.845169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb1df0-d71f-43f1-9714-8913cf575eab None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:44.846764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb1df0-d71f-43f1-9714-8913cf575eab None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:44.846927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb1df0-d71f-43f1-9714-8913cf575eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:44.880811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e7eb1df0-d71f-43f1-9714-8913cf575eab None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:44.886042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e7eb1df0-d71f-43f1-9714-8913cf575eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:44.899410 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3079/6199] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:44.900706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb1df0-d71f-43f1-9714-8913cf575eab None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:44.913304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb1df0-d71f-43f1-9714-8913cf575eab None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:44.919688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572abb4-e08d-4fc1-8879-e647c02278b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:44.920963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572abb4-e08d-4fc1-8879-e647c02278b4 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:44.921326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572abb4-e08d-4fc1-8879-e647c02278b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:44.921519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572abb4-e08d-4fc1-8879-e647c02278b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:44.922425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572abb4-e08d-4fc1-8879-e647c02278b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:25:44.923013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572abb4-e08d-4fc1-8879-e647c02278b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:44.924691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e572abb4-e08d-4fc1-8879-e647c02278b4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:44.940419 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3080/6200] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:25:44.942343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572abb4-e08d-4fc1-8879-e647c02278b4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:44.954338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572abb4-e08d-4fc1-8879-e647c02278b4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:44.959292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a66f8-6334-4732-9fd7-039aee2efa12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:44.959944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a66f8-6334-4732-9fd7-039aee2efa12 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:44.960227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a66f8-6334-4732-9fd7-039aee2efa12 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:44.960610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a66f8-6334-4732-9fd7-039aee2efa12 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:44.969332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a66f8-6334-4732-9fd7-039aee2efa12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:44.969332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a66f8-6334-4732-9fd7-039aee2efa12 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:44.980256 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3121/6201] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:44 2022] POST /identity/v3/projects => generated 360 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:44.982565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a20f6-c76e-492b-9287-6296ffe88496 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:45.061501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a20f6-c76e-492b-9287-6296ffe88496 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:45.074018 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3081/6202] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:44 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:25:45.074182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2aaa3c-1499-4646-9d0c-39672b6cfbc0 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:45.074182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2aaa3c-1499-4646-9d0c-39672b6cfbc0 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:45.074402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a66f8-6334-4732-9fd7-039aee2efa12 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:45.074511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2aaa3c-1499-4646-9d0c-39672b6cfbc0 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:45.074788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2aaa3c-1499-4646-9d0c-39672b6cfbc0 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:45.078749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2aaa3c-1499-4646-9d0c-39672b6cfbc0 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:45.081216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2aaa3c-1499-4646-9d0c-39672b6cfbc0 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:45.158605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a66f8-6334-4732-9fd7-039aee2efa12 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:45.166555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945706e-3d8f-4b30-9a75-8dd98c48e53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:45.167433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945706e-3d8f-4b30-9a75-8dd98c48e53c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:45.167823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945706e-3d8f-4b30-9a75-8dd98c48e53c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:45.168261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2945706e-3d8f-4b30-9a75-8dd98c48e53c admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:45.169060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2945706e-3d8f-4b30-9a75-8dd98c48e53c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:45.172173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2945706e-3d8f-4b30-9a75-8dd98c48e53c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:45.248511 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3122/6203] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:25:44 2022] POST /identity/v3/users => generated 416 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:25:45.252255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2aaa3c-1499-4646-9d0c-39672b6cfbc0 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:45.264662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2aaa3c-1499-4646-9d0c-39672b6cfbc0 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:45.295243 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3082/6204] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:25:45 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:45.295243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:45.295243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945706e-3d8f-4b30-9a75-8dd98c48e53c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:45.309996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2945706e-3d8f-4b30-9a75-8dd98c48e53c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:45.315627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c5007-6bdd-483c-8fdc-5ea9431e1d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:45.316992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c5007-6bdd-483c-8fdc-5ea9431e1d3b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:45.317352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c5007-6bdd-483c-8fdc-5ea9431e1d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:45.317577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92c5007-6bdd-483c-8fdc-5ea9431e1d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:45.318414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c5007-6bdd-483c-8fdc-5ea9431e1d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:45.331187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92c5007-6bdd-483c-8fdc-5ea9431e1d3b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:45.334300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebf795-0fef-4e2d-90a7-db800e813f91 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:45.335724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf795-0fef-4e2d-90a7-db800e813f91 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:45.336718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf795-0fef-4e2d-90a7-db800e813f91 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:45.337152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebf795-0fef-4e2d-90a7-db800e813f91 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/users/bce995a1e8444f40a07fbddd187af29e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:45.355745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebf795-0fef-4e2d-90a7-db800e813f91 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:get_user(user_id=bce995a1e8444f40a07fbddd187af29e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:45.358436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ebf795-0fef-4e2d-90a7-db800e813f91 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:45.359205 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3083/6205] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:25:45.360713 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3123/6206] 10.222.0.33 () {58 vars in 1220 bytes} [Sat May 28 21:25:45 2022] GET /identity/v3/users/bce995a1e8444f40a07fbddd187af29e => generated 416 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:25:45.372798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c5007-6bdd-483c-8fdc-5ea9431e1d3b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:45.373089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebf795-0fef-4e2d-90a7-db800e813f91 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:45.391017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebf795-0fef-4e2d-90a7-db800e813f91 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:45.392535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92c5007-6bdd-483c-8fdc-5ea9431e1d3b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:45.397148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59a815-5c7f-453a-8e31-f7608a1c36f8 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:45.400012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59a815-5c7f-453a-8e31-f7608a1c36f8 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:45.400123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6a466-8015-48dd-8a46-1a6904ab03c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:45.400337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59a815-5c7f-453a-8e31-f7608a1c36f8 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:45.400443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59a815-5c7f-453a-8e31-f7608a1c36f8 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/users/bce995a1e8444f40a07fbddd187af29e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:45.402256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6a466-8015-48dd-8a46-1a6904ab03c1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:45.402389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59a815-5c7f-453a-8e31-f7608a1c36f8 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:delete_user(user_id=bce995a1e8444f40a07fbddd187af29e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:45.402720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6a466-8015-48dd-8a46-1a6904ab03c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:45.403103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd6a466-8015-48dd-8a46-1a6904ab03c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:45.403982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6a466-8015-48dd-8a46-1a6904ab03c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:45.405694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd6a466-8015-48dd-8a46-1a6904ab03c1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:45.407316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59a815-5c7f-453a-8e31-f7608a1c36f8 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:45.423410 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3084/6207] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:25:45.427345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6a466-8015-48dd-8a46-1a6904ab03c1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:45.440453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd6a466-8015-48dd-8a46-1a6904ab03c1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:45.447985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14f2be7-1281-49fa-bff9-1bdf731f67dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:45.452977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f2be7-1281-49fa-bff9-1bdf731f67dc admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:45.452977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f2be7-1281-49fa-bff9-1bdf731f67dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:45.452977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14f2be7-1281-49fa-bff9-1bdf731f67dc admin admin] PATH_INFO: `/v3/projects/da0769045cc34c80ad5cd2e5d9089558/users/54b1cb3b227a48f39cae8ee9294576e7/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:45.470950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f2be7-1281-49fa-bff9-1bdf731f67dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=da0769045cc34c80ad5cd2e5d9089558, user_id=54b1cb3b227a48f39cae8ee9294576e7, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:45.475887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14f2be7-1281-49fa-bff9-1bdf731f67dc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:45.556443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fa59a815-5c7f-453a-8e31-f7608a1c36f8 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bce995a1e8444f40a07fbddd187af29e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:45.561117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fa59a815-5c7f-453a-8e31-f7608a1c36f8 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bce995a1e8444f40a07fbddd187af29e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:45.562229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fa59a815-5c7f-453a-8e31-f7608a1c36f8 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bce995a1e8444f40a07fbddd187af29e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:45.562957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fa59a815-5c7f-453a-8e31-f7608a1c36f8 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bce995a1e8444f40a07fbddd187af29e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:45.565129 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3085/6208] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:25:45 2022] PUT /identity/v3/projects/da0769045cc34c80ad5cd2e5d9089558/users/54b1cb3b227a48f39cae8ee9294576e7/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:45.579443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57600ba9-a3b3-47a2-9476-591fa265989a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:45.579636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57600ba9-a3b3-47a2-9476-591fa265989a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:45.579831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57600ba9-a3b3-47a2-9476-591fa265989a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:45.614066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-57600ba9-a3b3-47a2-9476-591fa265989a None None] MFA Rules not processed for user `54b1cb3b227a48f39cae8ee9294576e7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:45.616224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-57600ba9-a3b3-47a2-9476-591fa265989a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:45.659955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fa59a815-5c7f-453a-8e31-f7608a1c36f8 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bce995a1e8444f40a07fbddd187af29e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:45.685821 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3124/6209] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:45 2022] DELETE /identity/v3/users/bce995a1e8444f40a07fbddd187af29e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:45.692952 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3086/6210] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:45.700927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59a815-5c7f-453a-8e31-f7608a1c36f8 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:45.704865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57600ba9-a3b3-47a2-9476-591fa265989a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:45.706637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-57600ba9-a3b3-47a2-9476-591fa265989a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:45.707481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fa59a815-5c7f-453a-8e31-f7608a1c36f8 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:45.786503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57600ba9-a3b3-47a2-9476-591fa265989a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:45.790904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59a815-5c7f-453a-8e31-f7608a1c36f8 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:45.797011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379a5eb-d372-419c-a4ce-495008afb5f2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:45.797735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379a5eb-d372-419c-a4ce-495008afb5f2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:45.798618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379a5eb-d372-419c-a4ce-495008afb5f2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:45.798618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379a5eb-d372-419c-a4ce-495008afb5f2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/e9032f0816a842bb9a0bc78d0f47bd8b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:45.799750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379a5eb-d372-419c-a4ce-495008afb5f2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:delete_project(project_id=e9032f0816a842bb9a0bc78d0f47bd8b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:45.802138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379a5eb-d372-419c-a4ce-495008afb5f2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:45.856347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49903b6d-9494-453a-8c14-eada28b2d86f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:45.857087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49903b6d-9494-453a-8c14-eada28b2d86f service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:45.857382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49903b6d-9494-453a-8c14-eada28b2d86f service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:45.857662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49903b6d-9494-453a-8c14-eada28b2d86f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:45.882076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:45.923022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49903b6d-9494-453a-8c14-eada28b2d86f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:45.925059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49903b6d-9494-453a-8c14-eada28b2d86f service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:45.927213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8379a5eb-d372-419c-a4ce-495008afb5f2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9032f0816a842bb9a0bc78d0f47bd8b', 'request_id': 'req-8379a5eb-d372-419c-a4ce-495008afb5f2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:45.934513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8379a5eb-d372-419c-a4ce-495008afb5f2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9032f0816a842bb9a0bc78d0f47bd8b', 'request_id': 'req-8379a5eb-d372-419c-a4ce-495008afb5f2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:45.949094 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3087/6211] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:25:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:25:46.036528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8379a5eb-d372-419c-a4ce-495008afb5f2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9032f0816a842bb9a0bc78d0f47bd8b', 'request_id': 'req-8379a5eb-d372-419c-a4ce-495008afb5f2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:46.052286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8379a5eb-d372-419c-a4ce-495008afb5f2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] The token cache is being invalidate because project e9032f0816a842bb9a0bc78d0f47bd8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:46.052688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8379a5eb-d372-419c-a4ce-495008afb5f2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:46.053477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8379a5eb-d372-419c-a4ce-495008afb5f2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:46.055484 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3125/6212] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:45 2022] DELETE /identity/v3/projects/e9032f0816a842bb9a0bc78d0f47bd8b => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:46.065283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49903b6d-9494-453a-8c14-eada28b2d86f service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:46.099560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49903b6d-9494-453a-8c14-eada28b2d86f service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:46.192018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172d7b8-1b80-473e-98df-0b4ff0786f5a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:46.192705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172d7b8-1b80-473e-98df-0b4ff0786f5a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:46.192995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172d7b8-1b80-473e-98df-0b4ff0786f5a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:46.193279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172d7b8-1b80-473e-98df-0b4ff0786f5a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:46.194043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172d7b8-1b80-473e-98df-0b4ff0786f5a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:46.196399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172d7b8-1b80-473e-98df-0b4ff0786f5a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:46.313605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379a5eb-d372-419c-a4ce-495008afb5f2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:46.315116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8379a5eb-d372-419c-a4ce-495008afb5f2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:46.361823 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3088/6213] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:46 2022] POST /identity/v3/projects => generated 348 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:46.365170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172d7b8-1b80-473e-98df-0b4ff0786f5a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:46.371368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f172d7b8-1b80-473e-98df-0b4ff0786f5a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:46.399511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379a5eb-d372-419c-a4ce-495008afb5f2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:46.469427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172d7b8-1b80-473e-98df-0b4ff0786f5a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:46.469531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fa587f-809b-41f0-85cc-d87ebc5306c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:46.469531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa587f-809b-41f0-85cc-d87ebc5306c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:46.469713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a684a-0e6d-402a-ab10-51c9d2393220 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:46.469815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa587f-809b-41f0-85cc-d87ebc5306c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:46.469815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa587f-809b-41f0-85cc-d87ebc5306c2 admin admin] PATH_INFO: `/v3/users/54b1cb3b227a48f39cae8ee9294576e7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:46.470001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a684a-0e6d-402a-ab10-51c9d2393220 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:46.470001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a684a-0e6d-402a-ab10-51c9d2393220 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:46.470240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a684a-0e6d-402a-ab10-51c9d2393220 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:46.471007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a684a-0e6d-402a-ab10-51c9d2393220 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Adding query filter params (is_domain=True) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:25:46.471297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa587f-809b-41f0-85cc-d87ebc5306c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54b1cb3b227a48f39cae8ee9294576e7)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:46.471420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a684a-0e6d-402a-ab10-51c9d2393220 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:46.473531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a684a-0e6d-402a-ab10-51c9d2393220 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:46.473795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fa587f-809b-41f0-85cc-d87ebc5306c2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:46.491288 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3089/6214] 10.222.0.33 () {58 vars in 1159 bytes} [Sat May 28 21:25:46 2022] GET /identity/v3/projects?is_domain=True => generated 1020 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:25:46.498532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a684a-0e6d-402a-ab10-51c9d2393220 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:46.509432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a684a-0e6d-402a-ab10-51c9d2393220 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:46.514165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcfb43-3e6b-4868-9b91-0e510a531e3c tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:46.514829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcfb43-3e6b-4868-9b91-0e510a531e3c tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:46.515121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcfb43-3e6b-4868-9b91-0e510a531e3c tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:46.515401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fcfb43-3e6b-4868-9b91-0e510a531e3c tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:46.516107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fcfb43-3e6b-4868-9b91-0e510a531e3c tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:46.517716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fcfb43-3e6b-4868-9b91-0e510a531e3c tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:46.530764 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3090/6215] 10.222.0.33 () {58 vars in 1127 bytes} [Sat May 28 21:25:46 2022] GET /identity/v3/domains => generated 829 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:46.534850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcfb43-3e6b-4868-9b91-0e510a531e3c tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:46.544999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fcfb43-3e6b-4868-9b91-0e510a531e3c tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:46.550707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8899e-44fa-40f0-80cc-be2c8b39f900 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:46.551664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8899e-44fa-40f0-80cc-be2c8b39f900 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:46.552031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8899e-44fa-40f0-80cc-be2c8b39f900 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:46.552388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba8899e-44fa-40f0-80cc-be2c8b39f900 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/a0fb3a4f59974d75baeb83757a081b1a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:46.553752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8899e-44fa-40f0-80cc-be2c8b39f900 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:update_project(project_id=a0fb3a4f59974d75baeb83757a081b1a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:46.555451 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-fba8899e-44fa-40f0-80cc-be2c8b39f900 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:25:46.555451 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:25:46.555451 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:25:46.556615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba8899e-44fa-40f0-80cc-be2c8b39f900 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:46.569018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fba8899e-44fa-40f0-80cc-be2c8b39f900 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'a0fb3a4f59974d75baeb83757a081b1a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:46.569770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fba8899e-44fa-40f0-80cc-be2c8b39f900 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'a0fb3a4f59974d75baeb83757a081b1a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:46.630941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fba8899e-44fa-40f0-80cc-be2c8b39f900 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'a0fb3a4f59974d75baeb83757a081b1a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:46.631511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fba8899e-44fa-40f0-80cc-be2c8b39f900 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'a0fb3a4f59974d75baeb83757a081b1a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:46.634775 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3091/6216] 10.222.0.33 () {60 vars in 1251 bytes} [Sat May 28 21:25:46 2022] PATCH /identity/v3/projects/a0fb3a4f59974d75baeb83757a081b1a => generated 362 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:25:46.638560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8899e-44fa-40f0-80cc-be2c8b39f900 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:46.640477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fba8899e-44fa-40f0-80cc-be2c8b39f900 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:46.656829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f7fa587f-809b-41f0-85cc-d87ebc5306c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54b1cb3b227a48f39cae8ee9294576e7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:46.665988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f7fa587f-809b-41f0-85cc-d87ebc5306c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54b1cb3b227a48f39cae8ee9294576e7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:46.666817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f7fa587f-809b-41f0-85cc-d87ebc5306c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54b1cb3b227a48f39cae8ee9294576e7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:46.667443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f7fa587f-809b-41f0-85cc-d87ebc5306c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54b1cb3b227a48f39cae8ee9294576e7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:46.711900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba8899e-44fa-40f0-80cc-be2c8b39f900 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:46.717161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:46.718268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:46.718562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:46.718840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/a0fb3a4f59974d75baeb83757a081b1a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:46.731355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a0fb3a4f59974d75baeb83757a081b1a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:46.732994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:46.756886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'a0fb3a4f59974d75baeb83757a081b1a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:46.761122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'a0fb3a4f59974d75baeb83757a081b1a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:46.790414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f7fa587f-809b-41f0-85cc-d87ebc5306c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54b1cb3b227a48f39cae8ee9294576e7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:46.835961 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3126/6217] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:46 2022] DELETE /identity/v3/users/54b1cb3b227a48f39cae8ee9294576e7 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:25:46.841731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214e16c-8695-444c-9615-728f73e172c3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:46.845545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214e16c-8695-444c-9615-728f73e172c3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:46.845545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9214e16c-8695-444c-9615-728f73e172c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:46.861391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9214e16c-8695-444c-9615-728f73e172c3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:46.862651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9214e16c-8695-444c-9615-728f73e172c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:46.936476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0fb3a4f59974d75baeb83757a081b1a', 'request_id': 'req-fb361be8-cd23-493e-a4e6-c414060e6cff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:46.948298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0fb3a4f59974d75baeb83757a081b1a', 'request_id': 'req-fb361be8-cd23-493e-a4e6-c414060e6cff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:46.954512 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3127/6218] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:46.955558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214e16c-8695-444c-9615-728f73e172c3 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:46.957466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9214e16c-8695-444c-9615-728f73e172c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:47.025348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0fb3a4f59974d75baeb83757a081b1a', 'request_id': 'req-fb361be8-cd23-493e-a4e6-c414060e6cff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:47.038439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9214e16c-8695-444c-9615-728f73e172c3 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:47.045880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] The token cache is being invalidate because project a0fb3a4f59974d75baeb83757a081b1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:47.046232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:47.047026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:47.063212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'a0fb3a4f59974d75baeb83757a081b1a', 'request_id': 'req-fb361be8-cd23-493e-a4e6-c414060e6cff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:47.066093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'a0fb3a4f59974d75baeb83757a081b1a', 'request_id': 'req-fb361be8-cd23-493e-a4e6-c414060e6cff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:47.076567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'a0fb3a4f59974d75baeb83757a081b1a', 'request_id': 'req-fb361be8-cd23-493e-a4e6-c414060e6cff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:47.090364 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3092/6219] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:46 2022] DELETE /identity/v3/projects/a0fb3a4f59974d75baeb83757a081b1a => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:25:47.096000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:47.099751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:47.118509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fb6fc-369e-4593-ad9a-666364bf538c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:47.119179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fb6fc-369e-4593-ad9a-666364bf538c service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:47.119462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fb6fc-369e-4593-ad9a-666364bf538c service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:47.120090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fb6fc-369e-4593-ad9a-666364bf538c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:47.121638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a03fb6fc-369e-4593-ad9a-666364bf538c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:47.219622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb361be8-cd23-493e-a4e6-c414060e6cff tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:47.225346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fb6fc-369e-4593-ad9a-666364bf538c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:47.229319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fb6fc-369e-4593-ad9a-666364bf538c service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:47.235550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d75411-b3a5-4e1c-b3d5-12df177c5e24 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:47.236251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d75411-b3a5-4e1c-b3d5-12df177c5e24 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:47.236548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d75411-b3a5-4e1c-b3d5-12df177c5e24 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:47.236832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d75411-b3a5-4e1c-b3d5-12df177c5e24 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:47.237544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d75411-b3a5-4e1c-b3d5-12df177c5e24 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:47.239489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d75411-b3a5-4e1c-b3d5-12df177c5e24 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:47.248192 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3128/6220] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:25:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:25:47.280325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:47.410463 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3093/6221] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:47 2022] POST /identity/v3/projects => generated 360 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:25:47.416290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fb6fc-369e-4593-ad9a-666364bf538c service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:47.433681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fb6fc-369e-4593-ad9a-666364bf538c service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:47.462319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:47.535910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d75411-b3a5-4e1c-b3d5-12df177c5e24 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:47.536125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59180c6-2aa5-4b11-b819-7674598401ec tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:47.537430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59180c6-2aa5-4b11-b819-7674598401ec tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:47.537430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59180c6-2aa5-4b11-b819-7674598401ec tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:47.537430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59180c6-2aa5-4b11-b819-7674598401ec tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/485bfa1680cf4464b834375840a7a7d9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:47.538945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59180c6-2aa5-4b11-b819-7674598401ec tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:get_project(project_id=485bfa1680cf4464b834375840a7a7d9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:47.541787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59180c6-2aa5-4b11-b819-7674598401ec tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:47.543921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-86d75411-b3a5-4e1c-b3d5-12df177c5e24 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:47.544832 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3129/6222] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:25:47 2022] GET /identity/v3/projects/485bfa1680cf4464b834375840a7a7d9 => generated 360 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:47.549864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59180c6-2aa5-4b11-b819-7674598401ec tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:47.562762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59180c6-2aa5-4b11-b819-7674598401ec tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:47.568231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7589bc-020d-4188-8ed9-5d6528089537 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:47.569794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7589bc-020d-4188-8ed9-5d6528089537 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:47.583001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7589bc-020d-4188-8ed9-5d6528089537 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:47.583575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7589bc-020d-4188-8ed9-5d6528089537 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/485bfa1680cf4464b834375840a7a7d9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:47.587717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7589bc-020d-4188-8ed9-5d6528089537 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:delete_project(project_id=485bfa1680cf4464b834375840a7a7d9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:47.591900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7589bc-020d-4188-8ed9-5d6528089537 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:47.659743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d75411-b3a5-4e1c-b3d5-12df177c5e24 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:47.668862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702a991-fd42-407a-a95f-30cc99583038 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:47.676503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702a991-fd42-407a-a95f-30cc99583038 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:47.676503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702a991-fd42-407a-a95f-30cc99583038 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:47.676503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702a991-fd42-407a-a95f-30cc99583038 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:47.676503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b702a991-fd42-407a-a95f-30cc99583038 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:47.701889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702a991-fd42-407a-a95f-30cc99583038 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:47.711573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702a991-fd42-407a-a95f-30cc99583038 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:47.724969 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3094/6223] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:25:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:25:47.762853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-db7589bc-020d-4188-8ed9-5d6528089537 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '485bfa1680cf4464b834375840a7a7d9', 'request_id': 'req-db7589bc-020d-4188-8ed9-5d6528089537'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:47.785243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-db7589bc-020d-4188-8ed9-5d6528089537 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '485bfa1680cf4464b834375840a7a7d9', 'request_id': 'req-db7589bc-020d-4188-8ed9-5d6528089537'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:47.859800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-db7589bc-020d-4188-8ed9-5d6528089537 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '485bfa1680cf4464b834375840a7a7d9', 'request_id': 'req-db7589bc-020d-4188-8ed9-5d6528089537'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:47.877478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-db7589bc-020d-4188-8ed9-5d6528089537 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] The token cache is being invalidate because project 485bfa1680cf4464b834375840a7a7d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:47.878177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-db7589bc-020d-4188-8ed9-5d6528089537 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:47.879115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-db7589bc-020d-4188-8ed9-5d6528089537 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:47.883933 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3130/6224] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:47 2022] DELETE /identity/v3/projects/485bfa1680cf4464b834375840a7a7d9 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:47.889056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702a991-fd42-407a-a95f-30cc99583038 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:47.908245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702a991-fd42-407a-a95f-30cc99583038 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:48.054976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca407125-7268-4b85-a4fc-64b973e1e538 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:48.055906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca407125-7268-4b85-a4fc-64b973e1e538 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:48.056191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca407125-7268-4b85-a4fc-64b973e1e538 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:48.061988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca407125-7268-4b85-a4fc-64b973e1e538 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:48.061988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca407125-7268-4b85-a4fc-64b973e1e538 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:48.061988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca407125-7268-4b85-a4fc-64b973e1e538 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:48.202448 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3095/6225] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:47 2022] POST /identity/v3/projects => generated 360 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:48.206827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7589bc-020d-4188-8ed9-5d6528089537 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:48.208327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-db7589bc-020d-4188-8ed9-5d6528089537 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:48.238236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7589bc-020d-4188-8ed9-5d6528089537 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:48.362686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202e9bd-d6cc-44b9-a1dc-20e46c0a639a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:48.364111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202e9bd-d6cc-44b9-a1dc-20e46c0a639a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:48.364509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202e9bd-d6cc-44b9-a1dc-20e46c0a639a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:48.365150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4202e9bd-d6cc-44b9-a1dc-20e46c0a639a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/c2f834c8724e4e619a6b5dc832c27b9f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:48.366963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202e9bd-d6cc-44b9-a1dc-20e46c0a639a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:get_project(project_id=c2f834c8724e4e619a6b5dc832c27b9f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:48.372485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4202e9bd-d6cc-44b9-a1dc-20e46c0a639a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:48.373576 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3131/6226] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:25:48 2022] GET /identity/v3/projects/c2f834c8724e4e619a6b5dc832c27b9f => generated 360 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:48.377609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca407125-7268-4b85-a4fc-64b973e1e538 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:48.397156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca407125-7268-4b85-a4fc-64b973e1e538 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:48.413148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a659c-b077-41b1-a335-0f642db209e5 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:48.414571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a659c-b077-41b1-a335-0f642db209e5 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:48.415212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a659c-b077-41b1-a335-0f642db209e5 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:48.415773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a659c-b077-41b1-a335-0f642db209e5 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/c2f834c8724e4e619a6b5dc832c27b9f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:48.419602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a659c-b077-41b1-a335-0f642db209e5 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c2f834c8724e4e619a6b5dc832c27b9f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:48.422679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a659c-b077-41b1-a335-0f642db209e5 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:48.535143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202e9bd-d6cc-44b9-a1dc-20e46c0a639a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:48.537513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4202e9bd-d6cc-44b9-a1dc-20e46c0a639a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:48.600924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-124a659c-b077-41b1-a335-0f642db209e5 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2f834c8724e4e619a6b5dc832c27b9f', 'request_id': 'req-124a659c-b077-41b1-a335-0f642db209e5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:48.608654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-124a659c-b077-41b1-a335-0f642db209e5 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2f834c8724e4e619a6b5dc832c27b9f', 'request_id': 'req-124a659c-b077-41b1-a335-0f642db209e5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:48.653594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4202e9bd-d6cc-44b9-a1dc-20e46c0a639a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:48.730604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8e28b-9bef-4188-94f0-668e5d8df76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:48.731493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8e28b-9bef-4188-94f0-668e5d8df76c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:48.732241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8e28b-9bef-4188-94f0-668e5d8df76c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:48.732241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8e28b-9bef-4188-94f0-668e5d8df76c admin admin] PATH_INFO: `/v3/projects/da0769045cc34c80ad5cd2e5d9089558` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:48.733691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8e28b-9bef-4188-94f0-668e5d8df76c admin admin] RBAC: Authorizing `identity:delete_project(project_id=da0769045cc34c80ad5cd2e5d9089558)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:48.735582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8e28b-9bef-4188-94f0-668e5d8df76c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:48.751581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-124a659c-b077-41b1-a335-0f642db209e5 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2f834c8724e4e619a6b5dc832c27b9f', 'request_id': 'req-124a659c-b077-41b1-a335-0f642db209e5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:48.767978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-124a659c-b077-41b1-a335-0f642db209e5 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] The token cache is being invalidate because project c2f834c8724e4e619a6b5dc832c27b9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:48.769139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-124a659c-b077-41b1-a335-0f642db209e5 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:48.769966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-124a659c-b077-41b1-a335-0f642db209e5 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:48.771930 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3096/6227] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:48 2022] DELETE /identity/v3/projects/c2f834c8724e4e619a6b5dc832c27b9f => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:48.776788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a659c-b077-41b1-a335-0f642db209e5 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:48.778410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-124a659c-b077-41b1-a335-0f642db209e5 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:48.849166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a659c-b077-41b1-a335-0f642db209e5 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:48.858607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b2e8e28b-9bef-4188-94f0-668e5d8df76c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da0769045cc34c80ad5cd2e5d9089558', 'request_id': 'req-b2e8e28b-9bef-4188-94f0-668e5d8df76c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:48.864324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b2e8e28b-9bef-4188-94f0-668e5d8df76c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da0769045cc34c80ad5cd2e5d9089558', 'request_id': 'req-b2e8e28b-9bef-4188-94f0-668e5d8df76c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:48.906279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463697e-a798-416e-950b-5ae8d63d9558 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:48.906953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463697e-a798-416e-950b-5ae8d63d9558 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:48.907245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463697e-a798-416e-950b-5ae8d63d9558 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:48.907526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463697e-a798-416e-950b-5ae8d63d9558 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:48.908214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9463697e-a798-416e-950b-5ae8d63d9558 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:48.910139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9463697e-a798-416e-950b-5ae8d63d9558 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:48.929001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b2e8e28b-9bef-4188-94f0-668e5d8df76c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da0769045cc34c80ad5cd2e5d9089558', 'request_id': 'req-b2e8e28b-9bef-4188-94f0-668e5d8df76c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:48.941182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b2e8e28b-9bef-4188-94f0-668e5d8df76c admin admin] The token cache is being invalidate because project da0769045cc34c80ad5cd2e5d9089558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:48.941477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b2e8e28b-9bef-4188-94f0-668e5d8df76c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:48.942169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b2e8e28b-9bef-4188-94f0-668e5d8df76c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:48.944106 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3132/6228] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:48 2022] DELETE /identity/v3/projects/da0769045cc34c80ad5cd2e5d9089558 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:25:48.949270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d2fff-0dd0-4689-b6bf-5012e78bf52e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:48.949609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d2fff-0dd0-4689-b6bf-5012e78bf52e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:48.949897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d2fff-0dd0-4689-b6bf-5012e78bf52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:48.995007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0c7d2fff-0dd0-4689-b6bf-5012e78bf52e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:48.996896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0c7d2fff-0dd0-4689-b6bf-5012e78bf52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:49.086690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:49.089131 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3097/6229] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:25:48 2022] POST /identity/v3/projects => generated 348 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:25:49.101108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463697e-a798-416e-950b-5ae8d63d9558 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:49.124022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9463697e-a798-416e-950b-5ae8d63d9558 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:49.126786 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3133/6230] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:25:49.128395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936c1d7-4107-48f3-b590-f15095f265ad None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:49.128702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936c1d7-4107-48f3-b590-f15095f265ad None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:49.132369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936c1d7-4107-48f3-b590-f15095f265ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:49.163495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:49.176294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a936c1d7-4107-48f3-b590-f15095f265ad None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:49.179021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a936c1d7-4107-48f3-b590-f15095f265ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:49.273556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463697e-a798-416e-950b-5ae8d63d9558 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:49.283900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483abc50-dead-450d-b47d-cdff0cadb90a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:49.287028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483abc50-dead-450d-b47d-cdff0cadb90a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:49.287028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483abc50-dead-450d-b47d-cdff0cadb90a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:49.287028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483abc50-dead-450d-b47d-cdff0cadb90a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/bf80de40af534f938a54bfb57e75ce37` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:49.288320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483abc50-dead-450d-b47d-cdff0cadb90a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:get_project(project_id=bf80de40af534f938a54bfb57e75ce37)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:49.293793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483abc50-dead-450d-b47d-cdff0cadb90a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:49.298218 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3098/6231] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:25:49 2022] GET /identity/v3/projects/bf80de40af534f938a54bfb57e75ce37 => generated 348 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:49.311164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483abc50-dead-450d-b47d-cdff0cadb90a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:49.318108 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3134/6232] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:49.322712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936c1d7-4107-48f3-b590-f15095f265ad None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:49.325110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483abc50-dead-450d-b47d-cdff0cadb90a tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:49.334888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a936c1d7-4107-48f3-b590-f15095f265ad None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:49.339326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577619e8-9ca0-4ce8-ae24-9f588944950d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:49.340596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577619e8-9ca0-4ce8-ae24-9f588944950d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:49.341047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577619e8-9ca0-4ce8-ae24-9f588944950d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:49.341495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577619e8-9ca0-4ce8-ae24-9f588944950d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/bf80de40af534f938a54bfb57e75ce37` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:49.343130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577619e8-9ca0-4ce8-ae24-9f588944950d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bf80de40af534f938a54bfb57e75ce37)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:49.347243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577619e8-9ca0-4ce8-ae24-9f588944950d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:49.356593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74862a-6f49-4c5a-8cae-dfbed08df8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:49.357715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74862a-6f49-4c5a-8cae-dfbed08df8df admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:49.358204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74862a-6f49-4c5a-8cae-dfbed08df8df admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:49.360736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74862a-6f49-4c5a-8cae-dfbed08df8df admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:49.361655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74862a-6f49-4c5a-8cae-dfbed08df8df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:25:49.362409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74862a-6f49-4c5a-8cae-dfbed08df8df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:49.366016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74862a-6f49-4c5a-8cae-dfbed08df8df admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:49.415298 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3135/6233] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:25:49.419500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74862a-6f49-4c5a-8cae-dfbed08df8df admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:49.442081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74862a-6f49-4c5a-8cae-dfbed08df8df admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:49.448844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85d2d0-0b8e-43c4-a726-c93fccd37327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:49.449675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85d2d0-0b8e-43c4-a726-c93fccd37327 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:49.450066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85d2d0-0b8e-43c4-a726-c93fccd37327 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:49.450439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb85d2d0-0b8e-43c4-a726-c93fccd37327 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:49.451321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85d2d0-0b8e-43c4-a726-c93fccd37327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:49.454868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb85d2d0-0b8e-43c4-a726-c93fccd37327 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:49.563555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-577619e8-9ca0-4ce8-ae24-9f588944950d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf80de40af534f938a54bfb57e75ce37', 'request_id': 'req-577619e8-9ca0-4ce8-ae24-9f588944950d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:49.575516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-577619e8-9ca0-4ce8-ae24-9f588944950d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf80de40af534f938a54bfb57e75ce37', 'request_id': 'req-577619e8-9ca0-4ce8-ae24-9f588944950d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:49.584269 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3136/6234] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:49 2022] POST /identity/v3/projects => generated 383 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:49.589641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85d2d0-0b8e-43c4-a726-c93fccd37327 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:49.660250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-577619e8-9ca0-4ce8-ae24-9f588944950d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf80de40af534f938a54bfb57e75ce37', 'request_id': 'req-577619e8-9ca0-4ce8-ae24-9f588944950d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:49.684584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb85d2d0-0b8e-43c4-a726-c93fccd37327 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:49.686425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-577619e8-9ca0-4ce8-ae24-9f588944950d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] The token cache is being invalidate because project bf80de40af534f938a54bfb57e75ce37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:49.686726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-577619e8-9ca0-4ce8-ae24-9f588944950d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:49.687555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-577619e8-9ca0-4ce8-ae24-9f588944950d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:49.689783 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3099/6235] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:49 2022] DELETE /identity/v3/projects/bf80de40af534f938a54bfb57e75ce37 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:49.696396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577619e8-9ca0-4ce8-ae24-9f588944950d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:49.697642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedbd71-b9ba-4f72-b180-952a377d0b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:49.698445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedbd71-b9ba-4f72-b180-952a377d0b95 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:49.698565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-577619e8-9ca0-4ce8-ae24-9f588944950d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:49.698740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedbd71-b9ba-4f72-b180-952a377d0b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:49.699167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedbd71-b9ba-4f72-b180-952a377d0b95 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:49.699946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedbd71-b9ba-4f72-b180-952a377d0b95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:49.702107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedbd71-b9ba-4f72-b180-952a377d0b95 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:49.788314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577619e8-9ca0-4ce8-ae24-9f588944950d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:49.813915 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3137/6236] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:25:49 2022] POST /identity/v3/users => generated 292 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:49.813915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedbd71-b9ba-4f72-b180-952a377d0b95 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:49.821782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6bedbd71-b9ba-4f72-b180-952a377d0b95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:49.849679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedbd71-b9ba-4f72-b180-952a377d0b95 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:49.856272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ce513-0071-427b-896c-d6646a84c29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:49.858313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ce513-0071-427b-896c-d6646a84c29e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:49.859194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ce513-0071-427b-896c-d6646a84c29e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:49.859703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ce513-0071-427b-896c-d6646a84c29e admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:49.860765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ce513-0071-427b-896c-d6646a84c29e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:49.862805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638ce513-0071-427b-896c-d6646a84c29e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:49.878646 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3138/6237] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:25:49.883258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ce513-0071-427b-896c-d6646a84c29e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:49.895998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ce513-0071-427b-896c-d6646a84c29e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:49.902382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785d3c0-53d9-4f12-a08d-2e1a13a62084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:49.903014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785d3c0-53d9-4f12-a08d-2e1a13a62084 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:49.903527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785d3c0-53d9-4f12-a08d-2e1a13a62084 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:49.903640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785d3c0-53d9-4f12-a08d-2e1a13a62084 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:49.904403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785d3c0-53d9-4f12-a08d-2e1a13a62084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:49.906519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785d3c0-53d9-4f12-a08d-2e1a13a62084 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:49.914479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342350eb-3531-4481-b576-b0f1c8e496ca tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:49.915485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342350eb-3531-4481-b576-b0f1c8e496ca tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:49.916150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342350eb-3531-4481-b576-b0f1c8e496ca tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:49.916814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342350eb-3531-4481-b576-b0f1c8e496ca tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:49.917825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342350eb-3531-4481-b576-b0f1c8e496ca tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:49.918723 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3139/6238] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:25:49.919997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342350eb-3531-4481-b576-b0f1c8e496ca tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:49.922518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785d3c0-53d9-4f12-a08d-2e1a13a62084 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:49.936155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785d3c0-53d9-4f12-a08d-2e1a13a62084 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:49.941449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39eac33-5518-4978-97e2-2b492e7f218a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:49.942388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39eac33-5518-4978-97e2-2b492e7f218a admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:49.942672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39eac33-5518-4978-97e2-2b492e7f218a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:49.943412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39eac33-5518-4978-97e2-2b492e7f218a admin admin] PATH_INFO: `/v3/projects/30abceacd3ec4b9db2a28bc92b949570/users/1166808eb99545978b43831f7f18c0df/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:49.961471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39eac33-5518-4978-97e2-2b492e7f218a admin admin] RBAC: Authorizing `identity:create_grant(project_id=30abceacd3ec4b9db2a28bc92b949570, user_id=1166808eb99545978b43831f7f18c0df, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:49.964362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39eac33-5518-4978-97e2-2b492e7f218a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:50.008465 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3100/6239] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:25:49 2022] POST /identity/v3/domains => generated 275 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:50.011557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342350eb-3531-4481-b576-b0f1c8e496ca tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:50.104582 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3140/6240] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:25:49 2022] PUT /identity/v3/projects/30abceacd3ec4b9db2a28bc92b949570/users/1166808eb99545978b43831f7f18c0df/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:25:50.114975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737f064-4444-4f09-a4f1-7236702f5624 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:50.115261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737f064-4444-4f09-a4f1-7236702f5624 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:50.115553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737f064-4444-4f09-a4f1-7236702f5624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:50.116190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342350eb-3531-4481-b576-b0f1c8e496ca tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:50.146795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6737f064-4444-4f09-a4f1-7236702f5624 None None] MFA Rules not processed for user `1166808eb99545978b43831f7f18c0df`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:50.153170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6737f064-4444-4f09-a4f1-7236702f5624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:50.299729 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3141/6241] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:25:50.301504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1d5a8-89b0-4c2b-9832-b1fb9eb8558d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:50.302915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1d5a8-89b0-4c2b-9832-b1fb9eb8558d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:50.303446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1d5a8-89b0-4c2b-9832-b1fb9eb8558d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:50.305108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1d5a8-89b0-4c2b-9832-b1fb9eb8558d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:50.307702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d1d5a8-89b0-4c2b-9832-b1fb9eb8558d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:50.321293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d1d5a8-89b0-4c2b-9832-b1fb9eb8558d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:50.321944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737f064-4444-4f09-a4f1-7236702f5624 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:50.322690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6737f064-4444-4f09-a4f1-7236702f5624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:50.481029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737f064-4444-4f09-a4f1-7236702f5624 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:50.486937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd705e-bc81-4e32-b431-bf4d0cb4d6f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:50.488244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd705e-bc81-4e32-b431-bf4d0cb4d6f3 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:50.488538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd705e-bc81-4e32-b431-bf4d0cb4d6f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:50.492599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd705e-bc81-4e32-b431-bf4d0cb4d6f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:50.505302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd705e-bc81-4e32-b431-bf4d0cb4d6f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:50.506854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd705e-bc81-4e32-b431-bf4d0cb4d6f3 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:50.520510 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3101/6242] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:50 2022] POST /identity/v3/projects => generated 401 bytes in 511 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:25:50.527662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1d5a8-89b0-4c2b-9832-b1fb9eb8558d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:50.536211 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3142/6243] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:25:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:25:50.681475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1d5a8-89b0-4c2b-9832-b1fb9eb8558d tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:50.704942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa396a-2672-4512-9e06-a56649b41ca3 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:50.715992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa396a-2672-4512-9e06-a56649b41ca3 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:50.716293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa396a-2672-4512-9e06-a56649b41ca3 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:50.721549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa396a-2672-4512-9e06-a56649b41ca3 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/774ba5b28ccb4a468dcc38320e910909` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:50.734832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa396a-2672-4512-9e06-a56649b41ca3 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:get_project(project_id=774ba5b28ccb4a468dcc38320e910909)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:50.736954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa396a-2672-4512-9e06-a56649b41ca3 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:50.746901 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3102/6244] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:25:50 2022] GET /identity/v3/projects/774ba5b28ccb4a468dcc38320e910909 => generated 401 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:50.748440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd705e-bc81-4e32-b431-bf4d0cb4d6f3 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:50.764593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd705e-bc81-4e32-b431-bf4d0cb4d6f3 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:50.775978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc0941-b621-4233-ad7e-1f01bc272a39 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:50.777098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc0941-b621-4233-ad7e-1f01bc272a39 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:50.777546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc0941-b621-4233-ad7e-1f01bc272a39 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:50.778032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc0941-b621-4233-ad7e-1f01bc272a39 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/774ba5b28ccb4a468dcc38320e910909` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:50.779599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc0941-b621-4233-ad7e-1f01bc272a39 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:delete_project(project_id=774ba5b28ccb4a468dcc38320e910909)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:50.782077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc0941-b621-4233-ad7e-1f01bc272a39 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:50.966395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-15fc0941-b621-4233-ad7e-1f01bc272a39 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '774ba5b28ccb4a468dcc38320e910909', 'request_id': 'req-15fc0941-b621-4233-ad7e-1f01bc272a39'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:50.973359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-15fc0941-b621-4233-ad7e-1f01bc272a39 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '774ba5b28ccb4a468dcc38320e910909', 'request_id': 'req-15fc0941-b621-4233-ad7e-1f01bc272a39'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.072999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-15fc0941-b621-4233-ad7e-1f01bc272a39 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '774ba5b28ccb4a468dcc38320e910909', 'request_id': 'req-15fc0941-b621-4233-ad7e-1f01bc272a39'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.093105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-15fc0941-b621-4233-ad7e-1f01bc272a39 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] The token cache is being invalidate because project 774ba5b28ccb4a468dcc38320e910909 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:51.093931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-15fc0941-b621-4233-ad7e-1f01bc272a39 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.094854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-15fc0941-b621-4233-ad7e-1f01bc272a39 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.097412 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3143/6245] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:50 2022] DELETE /identity/v3/projects/774ba5b28ccb4a468dcc38320e910909 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:51.107239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa396a-2672-4512-9e06-a56649b41ca3 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:51.115310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0faa396a-2672-4512-9e06-a56649b41ca3 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:51.140349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa396a-2672-4512-9e06-a56649b41ca3 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:51.152710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:UPDATE=1,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:51.291572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c9ee8-7f74-47d3-84e4-df03521ee392 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:51.294943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c9ee8-7f74-47d3-84e4-df03521ee392 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:51.295241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c9ee8-7f74-47d3-84e4-df03521ee392 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:51.295527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28c9ee8-7f74-47d3-84e4-df03521ee392 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/domains/a79c774eda474d7bb1ce4542e39b144e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:51.297181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c9ee8-7f74-47d3-84e4-df03521ee392 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=a79c774eda474d7bb1ce4542e39b144e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:51.298337 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-b28c9ee8-7f74-47d3-84e4-df03521ee392 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:25:51.298337 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:25:51.298337 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:25:51.299306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28c9ee8-7f74-47d3-84e4-df03521ee392 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:51.325229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b28c9ee8-7f74-47d3-84e4-df03521ee392 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'a79c774eda474d7bb1ce4542e39b144e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.325997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b28c9ee8-7f74-47d3-84e4-df03521ee392 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'a79c774eda474d7bb1ce4542e39b144e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.455035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b28c9ee8-7f74-47d3-84e4-df03521ee392 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'a79c774eda474d7bb1ce4542e39b144e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.456738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b28c9ee8-7f74-47d3-84e4-df03521ee392 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'a79c774eda474d7bb1ce4542e39b144e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.461755 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3103/6246] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:25:51 2022] PATCH /identity/v3/domains/a79c774eda474d7bb1ce4542e39b144e => generated 276 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:51.465761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc0941-b621-4233-ad7e-1f01bc272a39 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:51.491539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc0941-b621-4233-ad7e-1f01bc272a39 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:51.572451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:51.573633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:51.574123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:51.574481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/domains/a79c774eda474d7bb1ce4542e39b144e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:51.603771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=a79c774eda474d7bb1ce4542e39b144e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:51.604851 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:25:51.604851 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:25:51.604851 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:25:51.605776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:51.637882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'a79c774eda474d7bb1ce4542e39b144e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.647416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'a79c774eda474d7bb1ce4542e39b144e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.839111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a79c774eda474d7bb1ce4542e39b144e', 'request_id': 'req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.846270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a79c774eda474d7bb1ce4542e39b144e', 'request_id': 'req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.905819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a79c774eda474d7bb1ce4542e39b144e', 'request_id': 'req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.920597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] The token cache is being invalidate because project a79c774eda474d7bb1ce4542e39b144e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:51.920892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.921581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.935319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:51.936829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'a79c774eda474d7bb1ce4542e39b144e', 'request_id': 'req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.938286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'a79c774eda474d7bb1ce4542e39b144e', 'request_id': 'req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.943175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'a79c774eda474d7bb1ce4542e39b144e', 'request_id': 'req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:51.951756 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3144/6247] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:25:51 2022] DELETE /identity/v3/domains/a79c774eda474d7bb1ce4542e39b144e => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:51.957201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c9ee8-7f74-47d3-84e4-df03521ee392 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:51.958680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b28c9ee8-7f74-47d3-84e4-df03521ee392 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:51.975309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28c9ee8-7f74-47d3-84e4-df03521ee392 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:52.056566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77807805-3d0c-4f3a-9ce8-07ad2f033ce0 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:52.057347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77807805-3d0c-4f3a-9ce8-07ad2f033ce0 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:52.057713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77807805-3d0c-4f3a-9ce8-07ad2f033ce0 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:52.058113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77807805-3d0c-4f3a-9ce8-07ad2f033ce0 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:52.058998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77807805-3d0c-4f3a-9ce8-07ad2f033ce0 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:52.062772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77807805-3d0c-4f3a-9ce8-07ad2f033ce0 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:52.133244 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3104/6248] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:25:51 2022] POST /identity/v3/domains => generated 275 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:25:52.137163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:52.139795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:52.163213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88a262a-5353-4efb-b01d-1ee9a65fdd4f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:52.241362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1fa1e-2bcf-4d39-9d3a-cc821c7715d2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:52.243200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1fa1e-2bcf-4d39-9d3a-cc821c7715d2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:52.244082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1fa1e-2bcf-4d39-9d3a-cc821c7715d2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:52.244374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c1fa1e-2bcf-4d39-9d3a-cc821c7715d2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:52.245085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1fa1e-2bcf-4d39-9d3a-cc821c7715d2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:52.247047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c1fa1e-2bcf-4d39-9d3a-cc821c7715d2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:52.367847 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3145/6249] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:52 2022] POST /identity/v3/projects => generated 409 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:52.371104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77807805-3d0c-4f3a-9ce8-07ad2f033ce0 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:52.386830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77807805-3d0c-4f3a-9ce8-07ad2f033ce0 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:52.496509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc29441-de4f-403b-bff9-533c4355bec4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:52.496509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc29441-de4f-403b-bff9-533c4355bec4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:52.496509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc29441-de4f-403b-bff9-533c4355bec4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:52.496933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc29441-de4f-403b-bff9-533c4355bec4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:52.496933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc29441-de4f-403b-bff9-533c4355bec4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:52.496933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc29441-de4f-403b-bff9-533c4355bec4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:52.593801 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3105/6250] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:52 2022] POST /identity/v3/projects => generated 404 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:52.594381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1fa1e-2bcf-4d39-9d3a-cc821c7715d2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:52.607876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c1fa1e-2bcf-4d39-9d3a-cc821c7715d2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:52.690496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58997402-f5da-4794-9481-121d94526cf7 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:52.691766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58997402-f5da-4794-9481-121d94526cf7 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:52.692062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58997402-f5da-4794-9481-121d94526cf7 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:52.692483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58997402-f5da-4794-9481-121d94526cf7 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/9c6f355af1ab4113bf62be34bcc180a7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:52.694060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58997402-f5da-4794-9481-121d94526cf7 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9c6f355af1ab4113bf62be34bcc180a7)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:52.696899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58997402-f5da-4794-9481-121d94526cf7 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:52.835844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-58997402-f5da-4794-9481-121d94526cf7 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c6f355af1ab4113bf62be34bcc180a7', 'request_id': 'req-58997402-f5da-4794-9481-121d94526cf7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:52.843600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-58997402-f5da-4794-9481-121d94526cf7 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c6f355af1ab4113bf62be34bcc180a7', 'request_id': 'req-58997402-f5da-4794-9481-121d94526cf7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:52.950869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-58997402-f5da-4794-9481-121d94526cf7 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c6f355af1ab4113bf62be34bcc180a7', 'request_id': 'req-58997402-f5da-4794-9481-121d94526cf7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:52.962828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-58997402-f5da-4794-9481-121d94526cf7 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] The token cache is being invalidate because project 9c6f355af1ab4113bf62be34bcc180a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:52.963184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-58997402-f5da-4794-9481-121d94526cf7 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:52.966759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-58997402-f5da-4794-9481-121d94526cf7 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:52.966759 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3146/6251] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:52 2022] DELETE /identity/v3/projects/9c6f355af1ab4113bf62be34bcc180a7 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:25:52.969472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc29441-de4f-403b-bff9-533c4355bec4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:52.971256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6cc29441-de4f-403b-bff9-533c4355bec4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:52.988492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc29441-de4f-403b-bff9-533c4355bec4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:53.063079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffe55b-9f96-4c31-8c52-990d063b2378 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:53.063930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffe55b-9f96-4c31-8c52-990d063b2378 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:53.064215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffe55b-9f96-4c31-8c52-990d063b2378 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:53.064490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffe55b-9f96-4c31-8c52-990d063b2378 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/d314c4da25044f109abf95d5f7de2f0b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:53.065740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffe55b-9f96-4c31-8c52-990d063b2378 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d314c4da25044f109abf95d5f7de2f0b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:53.067733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffe55b-9f96-4c31-8c52-990d063b2378 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:53.202946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-70ffe55b-9f96-4c31-8c52-990d063b2378 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd314c4da25044f109abf95d5f7de2f0b', 'request_id': 'req-70ffe55b-9f96-4c31-8c52-990d063b2378'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:53.208396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-70ffe55b-9f96-4c31-8c52-990d063b2378 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd314c4da25044f109abf95d5f7de2f0b', 'request_id': 'req-70ffe55b-9f96-4c31-8c52-990d063b2378'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:53.287996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-70ffe55b-9f96-4c31-8c52-990d063b2378 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd314c4da25044f109abf95d5f7de2f0b', 'request_id': 'req-70ffe55b-9f96-4c31-8c52-990d063b2378'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:53.320380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-70ffe55b-9f96-4c31-8c52-990d063b2378 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] The token cache is being invalidate because project d314c4da25044f109abf95d5f7de2f0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:53.320732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-70ffe55b-9f96-4c31-8c52-990d063b2378 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:53.321455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-70ffe55b-9f96-4c31-8c52-990d063b2378 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:53.326033 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3106/6252] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:52 2022] DELETE /identity/v3/projects/d314c4da25044f109abf95d5f7de2f0b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:53.327534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58997402-f5da-4794-9481-121d94526cf7 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:53.339114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58997402-f5da-4794-9481-121d94526cf7 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:53.362661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:53.403073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:53.421792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bae55-d919-4003-9f98-d501644bf748 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:53.422711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bae55-d919-4003-9f98-d501644bf748 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:53.423299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bae55-d919-4003-9f98-d501644bf748 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:53.423592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6bae55-d919-4003-9f98-d501644bf748 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/domains/1087da19bb76455f98a920fdf6f81d8b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:53.424959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6bae55-d919-4003-9f98-d501644bf748 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=1087da19bb76455f98a920fdf6f81d8b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:53.426662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6bae55-d919-4003-9f98-d501644bf748 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:53.449749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4a6bae55-d919-4003-9f98-d501644bf748 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '1087da19bb76455f98a920fdf6f81d8b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:53.450613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4a6bae55-d919-4003-9f98-d501644bf748 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '1087da19bb76455f98a920fdf6f81d8b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:53.562620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4a6bae55-d919-4003-9f98-d501644bf748 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '1087da19bb76455f98a920fdf6f81d8b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:53.563242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4a6bae55-d919-4003-9f98-d501644bf748 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '1087da19bb76455f98a920fdf6f81d8b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:53.568764 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3147/6253] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:25:53 2022] PATCH /identity/v3/domains/1087da19bb76455f98a920fdf6f81d8b => generated 276 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:53.570335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffe55b-9f96-4c31-8c52-990d063b2378 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:53.571739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-70ffe55b-9f96-4c31-8c52-990d063b2378 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:53.585253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffe55b-9f96-4c31-8c52-990d063b2378 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:53.650242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:53.650536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:53.650832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:53.651392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/domains/1087da19bb76455f98a920fdf6f81d8b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:53.669317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=1087da19bb76455f98a920fdf6f81d8b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:53.671088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:53.696551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '1087da19bb76455f98a920fdf6f81d8b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:53.700086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '1087da19bb76455f98a920fdf6f81d8b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:53.888990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1087da19bb76455f98a920fdf6f81d8b', 'request_id': 'req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:53.895918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1087da19bb76455f98a920fdf6f81d8b', 'request_id': 'req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:53.999687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1087da19bb76455f98a920fdf6f81d8b', 'request_id': 'req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:54.014520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] The token cache is being invalidate because project 1087da19bb76455f98a920fdf6f81d8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:54.014877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:54.015488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:54.026807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '1087da19bb76455f98a920fdf6f81d8b', 'request_id': 'req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:54.027511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '1087da19bb76455f98a920fdf6f81d8b', 'request_id': 'req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:54.035686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '1087da19bb76455f98a920fdf6f81d8b', 'request_id': 'req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:54.050351 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3107/6254] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:25:53 2022] DELETE /identity/v3/domains/1087da19bb76455f98a920fdf6f81d8b => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:54.060811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bae55-d919-4003-9f98-d501644bf748 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:54.062566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4a6bae55-d919-4003-9f98-d501644bf748 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:54.082503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6bae55-d919-4003-9f98-d501644bf748 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:54.157763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:54.190781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99c727-cada-45fb-928c-2ff305a23ba2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:54.191536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99c727-cada-45fb-928c-2ff305a23ba2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:54.191895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99c727-cada-45fb-928c-2ff305a23ba2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:54.192246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99c727-cada-45fb-928c-2ff305a23ba2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:54.193064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99c727-cada-45fb-928c-2ff305a23ba2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:54.195929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99c727-cada-45fb-928c-2ff305a23ba2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:54.275123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:54.277088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:54.352429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0134b8e-d2b0-4352-a5b3-e6ece27badbb tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:54.364198 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3148/6255] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:54 2022] POST /identity/v3/projects => generated 376 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:54.366021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99c727-cada-45fb-928c-2ff305a23ba2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:54.367809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36273429-2185-46b7-b8ab-fbacc8f44a5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:54.368486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36273429-2185-46b7-b8ab-fbacc8f44a5f service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:54.369303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36273429-2185-46b7-b8ab-fbacc8f44a5f service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:54.369589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36273429-2185-46b7-b8ab-fbacc8f44a5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:54.371139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1a99c727-cada-45fb-928c-2ff305a23ba2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:54.371447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-36273429-2185-46b7-b8ab-fbacc8f44a5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:54.388621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36273429-2185-46b7-b8ab-fbacc8f44a5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:54.390924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36273429-2185-46b7-b8ab-fbacc8f44a5f service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:54.405472 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3108/6256] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:25:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:25:54.456946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99c727-cada-45fb-928c-2ff305a23ba2 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:54.466590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059be04-df93-482a-9e0c-52bec4821d1f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:54.467676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059be04-df93-482a-9e0c-52bec4821d1f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:54.467980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059be04-df93-482a-9e0c-52bec4821d1f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:54.468271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059be04-df93-482a-9e0c-52bec4821d1f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/2b8b3b60379d41199ff75e3bc90c834d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:54.469652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059be04-df93-482a-9e0c-52bec4821d1f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:get_project(project_id=2b8b3b60379d41199ff75e3bc90c834d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:54.473254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059be04-df93-482a-9e0c-52bec4821d1f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:54.477620 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3149/6257] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:25:54 2022] GET /identity/v3/projects/2b8b3b60379d41199ff75e3bc90c834d => generated 376 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:54.481929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36273429-2185-46b7-b8ab-fbacc8f44a5f service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:54.493511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36273429-2185-46b7-b8ab-fbacc8f44a5f service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:54.498849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263330f2-a4e0-4209-9f11-fcc9fcd45eed tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:54.499514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263330f2-a4e0-4209-9f11-fcc9fcd45eed tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:54.499810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263330f2-a4e0-4209-9f11-fcc9fcd45eed tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:54.500100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263330f2-a4e0-4209-9f11-fcc9fcd45eed tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:54.500839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263330f2-a4e0-4209-9f11-fcc9fcd45eed tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:list_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:54.502765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263330f2-a4e0-4209-9f11-fcc9fcd45eed tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:54.522230 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3109/6258] 10.222.0.33 () {58 vars in 1130 bytes} [Sat May 28 21:25:54 2022] GET /identity/v3/projects => generated 6242 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:25:54.526951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059be04-df93-482a-9e0c-52bec4821d1f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:54.540284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059be04-df93-482a-9e0c-52bec4821d1f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:54.553252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bdfd9f-856d-47f9-9394-a90bbd02dd85 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:54.553252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bdfd9f-856d-47f9-9394-a90bbd02dd85 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:54.553252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bdfd9f-856d-47f9-9394-a90bbd02dd85 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:54.554137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bdfd9f-856d-47f9-9394-a90bbd02dd85 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/2b8b3b60379d41199ff75e3bc90c834d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:54.554137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bdfd9f-856d-47f9-9394-a90bbd02dd85 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2b8b3b60379d41199ff75e3bc90c834d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:54.554137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bdfd9f-856d-47f9-9394-a90bbd02dd85 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:54.768641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:54.777482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c8bdfd9f-856d-47f9-9394-a90bbd02dd85 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b8b3b60379d41199ff75e3bc90c834d', 'request_id': 'req-c8bdfd9f-856d-47f9-9394-a90bbd02dd85'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:54.786258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c8bdfd9f-856d-47f9-9394-a90bbd02dd85 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b8b3b60379d41199ff75e3bc90c834d', 'request_id': 'req-c8bdfd9f-856d-47f9-9394-a90bbd02dd85'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:54.893994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c8bdfd9f-856d-47f9-9394-a90bbd02dd85 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b8b3b60379d41199ff75e3bc90c834d', 'request_id': 'req-c8bdfd9f-856d-47f9-9394-a90bbd02dd85'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:54.913190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c8bdfd9f-856d-47f9-9394-a90bbd02dd85 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] The token cache is being invalidate because project 2b8b3b60379d41199ff75e3bc90c834d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:54.913500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c8bdfd9f-856d-47f9-9394-a90bbd02dd85 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:54.922477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c8bdfd9f-856d-47f9-9394-a90bbd02dd85 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:54.927185 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3150/6259] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:54 2022] DELETE /identity/v3/projects/2b8b3b60379d41199ff75e3bc90c834d => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:54.934515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263330f2-a4e0-4209-9f11-fcc9fcd45eed tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:54.937512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-263330f2-a4e0-4209-9f11-fcc9fcd45eed tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:54.977464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263330f2-a4e0-4209-9f11-fcc9fcd45eed tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:55.084479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646077b-0bbd-43f6-ac69-01187001b1cc tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:55.085625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646077b-0bbd-43f6-ac69-01187001b1cc tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:55.085625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646077b-0bbd-43f6-ac69-01187001b1cc tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:55.085977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7646077b-0bbd-43f6-ac69-01187001b1cc tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:55.087129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646077b-0bbd-43f6-ac69-01187001b1cc tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:55.089419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7646077b-0bbd-43f6-ac69-01187001b1cc tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:55.208295 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3110/6260] 10.222.0.33 () {60 vars in 1151 bytes} [Sat May 28 21:25:54 2022] POST /identity/v3/projects => generated 347 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:25:55.208423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bdfd9f-856d-47f9-9394-a90bbd02dd85 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:55.220540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bdfd9f-856d-47f9-9394-a90bbd02dd85 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:55.309305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de1903-6ccf-446d-8204-3718df311e48 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:55.311665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de1903-6ccf-446d-8204-3718df311e48 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:55.311665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de1903-6ccf-446d-8204-3718df311e48 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:55.311665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de1903-6ccf-446d-8204-3718df311e48 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/f176f440315c43dda99b8ef664dda3e3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:55.312398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de1903-6ccf-446d-8204-3718df311e48 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:update_project(project_id=f176f440315c43dda99b8ef664dda3e3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:55.313425 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-31de1903-6ccf-446d-8204-3718df311e48 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:25:55.313425 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:25:55.313425 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:25:55.315396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31de1903-6ccf-446d-8204-3718df311e48 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:55.428484 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3151/6261] 10.222.0.33 () {60 vars in 1251 bytes} [Sat May 28 21:25:55 2022] PATCH /identity/v3/projects/f176f440315c43dda99b8ef664dda3e3 => generated 361 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:55.433340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646077b-0bbd-43f6-ac69-01187001b1cc tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:55.454482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7646077b-0bbd-43f6-ac69-01187001b1cc tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:55.455013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa31d3f-3ff0-4938-9e5e-d4d01346b350 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:55.456331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa31d3f-3ff0-4938-9e5e-d4d01346b350 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:55.456735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa31d3f-3ff0-4938-9e5e-d4d01346b350 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:55.457121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa31d3f-3ff0-4938-9e5e-d4d01346b350 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/f176f440315c43dda99b8ef664dda3e3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:55.474515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa31d3f-3ff0-4938-9e5e-d4d01346b350 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:get_project(project_id=f176f440315c43dda99b8ef664dda3e3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:55.479762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa31d3f-3ff0-4938-9e5e-d4d01346b350 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:55.485091 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3111/6262] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:25:55 2022] GET /identity/v3/projects/f176f440315c43dda99b8ef664dda3e3 => generated 348 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:25:55.491172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de1903-6ccf-446d-8204-3718df311e48 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:55.515854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de1903-6ccf-446d-8204-3718df311e48 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:55.532135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151120d-66d1-498a-9946-f4f9272ca791 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:55.533220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151120d-66d1-498a-9946-f4f9272ca791 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:55.533861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151120d-66d1-498a-9946-f4f9272ca791 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:55.534336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4151120d-66d1-498a-9946-f4f9272ca791 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/f176f440315c43dda99b8ef664dda3e3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:55.535657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151120d-66d1-498a-9946-f4f9272ca791 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f176f440315c43dda99b8ef664dda3e3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:55.538007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4151120d-66d1-498a-9946-f4f9272ca791 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:55.597336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ad7d8-cb3f-46e0-8ebb-9674b9b620c4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:55.598675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ad7d8-cb3f-46e0-8ebb-9674b9b620c4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:55.599186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ad7d8-cb3f-46e0-8ebb-9674b9b620c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:55.625716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-324ad7d8-cb3f-46e0-8ebb-9674b9b620c4 None None] MFA Rules not processed for user `ebc8e00965334bd1bb811a0eefd64b84`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:55.625716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-324ad7d8-cb3f-46e0-8ebb-9674b9b620c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:55.695666 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3112/6263] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:55.708119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4151120d-66d1-498a-9946-f4f9272ca791 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f176f440315c43dda99b8ef664dda3e3', 'request_id': 'req-4151120d-66d1-498a-9946-f4f9272ca791'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:55.717073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4151120d-66d1-498a-9946-f4f9272ca791 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f176f440315c43dda99b8ef664dda3e3', 'request_id': 'req-4151120d-66d1-498a-9946-f4f9272ca791'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:55.819775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4151120d-66d1-498a-9946-f4f9272ca791 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f176f440315c43dda99b8ef664dda3e3', 'request_id': 'req-4151120d-66d1-498a-9946-f4f9272ca791'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:55.832901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4151120d-66d1-498a-9946-f4f9272ca791 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] The token cache is being invalidate because project f176f440315c43dda99b8ef664dda3e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:55.833203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4151120d-66d1-498a-9946-f4f9272ca791 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:55.833924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4151120d-66d1-498a-9946-f4f9272ca791 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:55.840458 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3152/6264] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:55 2022] DELETE /identity/v3/projects/f176f440315c43dda99b8ef664dda3e3 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:55.841508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ad7d8-cb3f-46e0-8ebb-9674b9b620c4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:55.854287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ad7d8-cb3f-46e0-8ebb-9674b9b620c4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:55.930650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74362cd-a42f-4156-a7cd-4737ea05e775 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:55.931348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74362cd-a42f-4156-a7cd-4737ea05e775 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:55.931646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74362cd-a42f-4156-a7cd-4737ea05e775 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:55.932018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74362cd-a42f-4156-a7cd-4737ea05e775 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:55.932629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74362cd-a42f-4156-a7cd-4737ea05e775 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:55.935551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74362cd-a42f-4156-a7cd-4737ea05e775 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:56.097451 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3113/6265] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:55 2022] POST /identity/v3/projects => generated 360 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:25:56.098711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151120d-66d1-498a-9946-f4f9272ca791 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:56.105717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4151120d-66d1-498a-9946-f4f9272ca791 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:56.136349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4151120d-66d1-498a-9946-f4f9272ca791 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:56.262820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed3233-3f33-487b-8241-ea2077833381 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:56.264678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed3233-3f33-487b-8241-ea2077833381 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:56.265671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed3233-3f33-487b-8241-ea2077833381 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:56.266223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed3233-3f33-487b-8241-ea2077833381 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/b32f426a7f4140b6b973eeb98ef7ff6e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:56.268763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed3233-3f33-487b-8241-ea2077833381 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:update_project(project_id=b32f426a7f4140b6b973eeb98ef7ff6e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:56.277027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eed3233-3f33-487b-8241-ea2077833381 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:56.421298 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3153/6266] 10.222.0.33 () {60 vars in 1251 bytes} [Sat May 28 21:25:56 2022] PATCH /identity/v3/projects/b32f426a7f4140b6b973eeb98ef7ff6e => generated 372 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:56.421716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74362cd-a42f-4156-a7cd-4737ea05e775 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:56.435517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74362cd-a42f-4156-a7cd-4737ea05e775 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:56.440814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ddcf6-1a55-43f8-a0a9-a3236fbc645e tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:56.442245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ddcf6-1a55-43f8-a0a9-a3236fbc645e tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:56.442733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ddcf6-1a55-43f8-a0a9-a3236fbc645e tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:56.443014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ddcf6-1a55-43f8-a0a9-a3236fbc645e tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/b32f426a7f4140b6b973eeb98ef7ff6e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:56.454241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:56.457949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ddcf6-1a55-43f8-a0a9-a3236fbc645e tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:get_project(project_id=b32f426a7f4140b6b973eeb98ef7ff6e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:56.461079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8ddcf6-1a55-43f8-a0a9-a3236fbc645e tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:56.475582 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3114/6267] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:25:56 2022] GET /identity/v3/projects/b32f426a7f4140b6b973eeb98ef7ff6e => generated 359 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:56.480059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed3233-3f33-487b-8241-ea2077833381 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:56.510108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed3233-3f33-487b-8241-ea2077833381 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:56.515576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8719a22-13b1-417c-91a2-1a4543c9d101 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:56.516704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8719a22-13b1-417c-91a2-1a4543c9d101 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:56.517314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8719a22-13b1-417c-91a2-1a4543c9d101 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:56.517602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8719a22-13b1-417c-91a2-1a4543c9d101 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/b32f426a7f4140b6b973eeb98ef7ff6e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:56.518906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8719a22-13b1-417c-91a2-1a4543c9d101 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b32f426a7f4140b6b973eeb98ef7ff6e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:56.520878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8719a22-13b1-417c-91a2-1a4543c9d101 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:56.638754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e8719a22-13b1-417c-91a2-1a4543c9d101 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b32f426a7f4140b6b973eeb98ef7ff6e', 'request_id': 'req-e8719a22-13b1-417c-91a2-1a4543c9d101'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:56.645297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e8719a22-13b1-417c-91a2-1a4543c9d101 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b32f426a7f4140b6b973eeb98ef7ff6e', 'request_id': 'req-e8719a22-13b1-417c-91a2-1a4543c9d101'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:56.814138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e8719a22-13b1-417c-91a2-1a4543c9d101 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b32f426a7f4140b6b973eeb98ef7ff6e', 'request_id': 'req-e8719a22-13b1-417c-91a2-1a4543c9d101'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:56.827328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e8719a22-13b1-417c-91a2-1a4543c9d101 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] The token cache is being invalidate because project b32f426a7f4140b6b973eeb98ef7ff6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:56.827635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e8719a22-13b1-417c-91a2-1a4543c9d101 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:56.831517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e8719a22-13b1-417c-91a2-1a4543c9d101 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:56.833291 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3154/6268] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:56 2022] DELETE /identity/v3/projects/b32f426a7f4140b6b973eeb98ef7ff6e => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:56.838452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ddcf6-1a55-43f8-a0a9-a3236fbc645e tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:56.840365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6d8ddcf6-1a55-43f8-a0a9-a3236fbc645e tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:56.862069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8ddcf6-1a55-43f8-a0a9-a3236fbc645e tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:56.955533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6c910-ccc5-4db1-a24f-f8a1034cea5f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:56.956737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6c910-ccc5-4db1-a24f-f8a1034cea5f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:56.957233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6c910-ccc5-4db1-a24f-f8a1034cea5f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:56.958306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd6c910-ccc5-4db1-a24f-f8a1034cea5f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:56.959255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6c910-ccc5-4db1-a24f-f8a1034cea5f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:56.966986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd6c910-ccc5-4db1-a24f-f8a1034cea5f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:57.082372 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3115/6269] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:25:56 2022] POST /identity/v3/projects => generated 355 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:25:57.087011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8719a22-13b1-417c-91a2-1a4543c9d101 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:57.099712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8719a22-13b1-417c-91a2-1a4543c9d101 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:57.174672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d25de-5f6f-4149-b70c-74c489236050 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:57.175962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d25de-5f6f-4149-b70c-74c489236050 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:57.176465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d25de-5f6f-4149-b70c-74c489236050 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:57.176968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d25de-5f6f-4149-b70c-74c489236050 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/7437325d221e4dae958cf63e0f08abb2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:57.179831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d25de-5f6f-4149-b70c-74c489236050 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:update_project(project_id=7437325d221e4dae958cf63e0f08abb2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:57.183101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d25de-5f6f-4149-b70c-74c489236050 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:57.288644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:57.300469 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3155/6270] 10.222.0.33 () {60 vars in 1251 bytes} [Sat May 28 21:25:57 2022] PATCH /identity/v3/projects/7437325d221e4dae958cf63e0f08abb2 => generated 369 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:57.302614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6c910-ccc5-4db1-a24f-f8a1034cea5f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:57.316410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd6c910-ccc5-4db1-a24f-f8a1034cea5f tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:57.321505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356e429-1c74-450c-bc63-b1abd570e85c tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:57.322796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356e429-1c74-450c-bc63-b1abd570e85c tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:57.323270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356e429-1c74-450c-bc63-b1abd570e85c tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:57.323726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356e429-1c74-450c-bc63-b1abd570e85c tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/7437325d221e4dae958cf63e0f08abb2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:57.345583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356e429-1c74-450c-bc63-b1abd570e85c tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:get_project(project_id=7437325d221e4dae958cf63e0f08abb2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:57.345583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356e429-1c74-450c-bc63-b1abd570e85c tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:57.345583 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3116/6271] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:25:57 2022] GET /identity/v3/projects/7437325d221e4dae958cf63e0f08abb2 => generated 356 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:25:57.346027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d25de-5f6f-4149-b70c-74c489236050 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:57.357174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d25de-5f6f-4149-b70c-74c489236050 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:57.362740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b2653-a430-4ae9-9248-0c1af74277b4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:57.363921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b2653-a430-4ae9-9248-0c1af74277b4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:57.364442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b2653-a430-4ae9-9248-0c1af74277b4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:57.364793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b2653-a430-4ae9-9248-0c1af74277b4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/projects/7437325d221e4dae958cf63e0f08abb2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:57.366112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b2653-a430-4ae9-9248-0c1af74277b4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7437325d221e4dae958cf63e0f08abb2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:57.368274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b2653-a430-4ae9-9248-0c1af74277b4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:57.536199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c36b2653-a430-4ae9-9248-0c1af74277b4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7437325d221e4dae958cf63e0f08abb2', 'request_id': 'req-c36b2653-a430-4ae9-9248-0c1af74277b4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:57.546595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c36b2653-a430-4ae9-9248-0c1af74277b4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7437325d221e4dae958cf63e0f08abb2', 'request_id': 'req-c36b2653-a430-4ae9-9248-0c1af74277b4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:57.626334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c36b2653-a430-4ae9-9248-0c1af74277b4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7437325d221e4dae958cf63e0f08abb2', 'request_id': 'req-c36b2653-a430-4ae9-9248-0c1af74277b4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:57.641414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c36b2653-a430-4ae9-9248-0c1af74277b4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] The token cache is being invalidate because project 7437325d221e4dae958cf63e0f08abb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:57.641690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c36b2653-a430-4ae9-9248-0c1af74277b4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:57.642565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c36b2653-a430-4ae9-9248-0c1af74277b4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:57.649605 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3156/6272] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:57 2022] DELETE /identity/v3/projects/7437325d221e4dae958cf63e0f08abb2 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:57.651108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356e429-1c74-450c-bc63-b1abd570e85c tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:57.659002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e356e429-1c74-450c-bc63-b1abd570e85c tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:57.678392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356e429-1c74-450c-bc63-b1abd570e85c tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:57.739939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1123237-2f9b-4bea-84fc-d7256746c746 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:57.740842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1123237-2f9b-4bea-84fc-d7256746c746 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:57.741095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1123237-2f9b-4bea-84fc-d7256746c746 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:57.741396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1123237-2f9b-4bea-84fc-d7256746c746 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/domains/1087da19bb76455f98a920fdf6f81d8b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:57.749261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1123237-2f9b-4bea-84fc-d7256746c746 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:25:57.759740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-c1123237-2f9b-4bea-84fc-d7256746c746 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Could not find domain: 1087da19bb76455f98a920fdf6f81d8b.: keystone.exception.DomainNotFound: Could not find domain: 1087da19bb76455f98a920fdf6f81d8b. May 28 21:25:57.759740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:25:57.759740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 21:25:57.759740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 21:25:57.759740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:25:57.759740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:25:57.759740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:25:57.759740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:25:57.759740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 1087da19bb76455f98a920fdf6f81d8b. May 28 21:25:57.759740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:25:57.759740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:25:57.759740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:25:57.759740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:25:57.759740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:25:57.759740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:25:57.759740 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:25:57.761283 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:25:57.761283 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:25:57.761283 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:25:57.761283 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:25:57.761283 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:25:57.761283 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:25:57.761283 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:25:57.761283 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch May 28 21:25:57.761283 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') May 28 21:25:57.761283 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:25:57.761283 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:25:57.761283 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:25:57.761283 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:25:57.761283 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:25:57.761283 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:25:57.761283 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:25:57.762564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:25:57.762564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:25:57.762564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:25:57.762564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:25:57.762564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 21:25:57.762564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:25:57.762564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 21:25:57.762564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:25:57.762564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:25:57.762564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:25:57.762564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 21:25:57.762564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:25:57.762564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 21:25:57.762564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 21:25:57.762564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 21:25:57.762564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 1087da19bb76455f98a920fdf6f81d8b. May 28 21:25:57.762564 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:25:57.764013 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3117/6273] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:25:57 2022] PATCH /identity/v3/domains/1087da19bb76455f98a920fdf6f81d8b => generated 112 bytes in 111 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:25:57.764973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b2653-a430-4ae9-9248-0c1af74277b4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:57.776727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b2653-a430-4ae9-9248-0c1af74277b4 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:57.782582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb73d15-8583-400c-a23d-e6be394044f1 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4f328d2b2d476fa245063334995618', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '31f42fca38a6448eb1d1cc64062a8ca7', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:57.784036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb73d15-8583-400c-a23d-e6be394044f1 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:57.784036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb73d15-8583-400c-a23d-e6be394044f1 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:57.784036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb73d15-8583-400c-a23d-e6be394044f1 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] PATH_INFO: `/v3/domains/a79c774eda474d7bb1ce4542e39b144e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:57.793088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb73d15-8583-400c-a23d-e6be394044f1 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:25:57.803471 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-ccb73d15-8583-400c-a23d-e6be394044f1 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Could not find domain: a79c774eda474d7bb1ce4542e39b144e.: keystone.exception.DomainNotFound: Could not find domain: a79c774eda474d7bb1ce4542e39b144e. May 28 21:25:57.803471 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:25:57.803471 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 21:25:57.803471 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 21:25:57.803471 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:25:57.803471 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:25:57.803471 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:25:57.803471 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:25:57.803471 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: a79c774eda474d7bb1ce4542e39b144e. May 28 21:25:57.803471 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:25:57.803471 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:25:57.803471 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:25:57.803471 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:25:57.803471 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:25:57.803471 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:25:57.803471 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:25:57.805341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:25:57.805341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:25:57.805341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:25:57.805341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:25:57.805341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:25:57.805341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:25:57.805341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:25:57.805341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch May 28 21:25:57.805341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') May 28 21:25:57.805341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:25:57.805341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:25:57.805341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:25:57.805341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:25:57.805341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:25:57.805341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:25:57.805341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:25:57.806511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:25:57.806511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:25:57.806511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:25:57.806511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:25:57.806511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 21:25:57.806511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:25:57.806511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 21:25:57.806511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:25:57.806511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:25:57.806511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:25:57.806511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 21:25:57.806511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:25:57.806511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 21:25:57.806511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 21:25:57.806511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 21:25:57.806511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: a79c774eda474d7bb1ce4542e39b144e. May 28 21:25:57.806511 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:25:57.807569 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3157/6274] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:25:57 2022] PATCH /identity/v3/domains/a79c774eda474d7bb1ce4542e39b144e => generated 112 bytes in 41 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) May 28 21:25:57.809131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1123237-2f9b-4bea-84fc-d7256746c746 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:57.810908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c1123237-2f9b-4bea-84fc-d7256746c746 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:57.881795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1123237-2f9b-4bea-84fc-d7256746c746 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:57.887173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc882685-5e57-47fc-a573-1aa1453481ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:57.888046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc882685-5e57-47fc-a573-1aa1453481ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:57.888331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc882685-5e57-47fc-a573-1aa1453481ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:57.888611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc882685-5e57-47fc-a573-1aa1453481ca admin admin] PATH_INFO: `/v3/users/499596c2b18248deb72e59b42859df04` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:57.890061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc882685-5e57-47fc-a573-1aa1453481ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=499596c2b18248deb72e59b42859df04)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:57.892104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc882685-5e57-47fc-a573-1aa1453481ca admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:58.126396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dc882685-5e57-47fc-a573-1aa1453481ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '499596c2b18248deb72e59b42859df04'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:58.130262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dc882685-5e57-47fc-a573-1aa1453481ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '499596c2b18248deb72e59b42859df04'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:58.140359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dc882685-5e57-47fc-a573-1aa1453481ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '499596c2b18248deb72e59b42859df04'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:58.287148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb73d15-8583-400c-a23d-e6be394044f1 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:58.289543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ccb73d15-8583-400c-a23d-e6be394044f1 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:58.305209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb73d15-8583-400c-a23d-e6be394044f1 tempest-ProjectsTestJSON-1779905977 tempest-ProjectsTestJSON-1779905977-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:58.310038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d5ab0-e5ec-4dfb-b9c7-5c6bfeab7bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:58.310977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d5ab0-e5ec-4dfb-b9c7-5c6bfeab7bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:58.311349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d5ab0-e5ec-4dfb-b9c7-5c6bfeab7bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:58.311680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0d5ab0-e5ec-4dfb-b9c7-5c6bfeab7bd2 admin admin] PATH_INFO: `/v3/users/2267c55067d549e79963425bfc21f45c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:58.313079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d5ab0-e5ec-4dfb-b9c7-5c6bfeab7bd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2267c55067d549e79963425bfc21f45c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:58.315147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0d5ab0-e5ec-4dfb-b9c7-5c6bfeab7bd2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:58.343651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dc882685-5e57-47fc-a573-1aa1453481ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '499596c2b18248deb72e59b42859df04'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:58.343651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dc882685-5e57-47fc-a573-1aa1453481ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '499596c2b18248deb72e59b42859df04'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:58.360008 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3118/6275] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:57 2022] DELETE /identity/v3/users/499596c2b18248deb72e59b42859df04 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:58.371219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e66683-1bb1-45bb-9540-d9b3e5894166 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:58.371836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e66683-1bb1-45bb-9540-d9b3e5894166 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:58.372113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e66683-1bb1-45bb-9540-d9b3e5894166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:58.409437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e9e66683-1bb1-45bb-9540-d9b3e5894166 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:58.410744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e9e66683-1bb1-45bb-9540-d9b3e5894166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:58.583428 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3119/6276] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:58.595767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e66683-1bb1-45bb-9540-d9b3e5894166 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:58.600057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e9e66683-1bb1-45bb-9540-d9b3e5894166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:58.655560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3d0d5ab0-e5ec-4dfb-b9c7-5c6bfeab7bd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2267c55067d549e79963425bfc21f45c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:58.673952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3d0d5ab0-e5ec-4dfb-b9c7-5c6bfeab7bd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2267c55067d549e79963425bfc21f45c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:58.678204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3d0d5ab0-e5ec-4dfb-b9c7-5c6bfeab7bd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2267c55067d549e79963425bfc21f45c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:58.679179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3d0d5ab0-e5ec-4dfb-b9c7-5c6bfeab7bd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2267c55067d549e79963425bfc21f45c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:58.716738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e66683-1bb1-45bb-9540-d9b3e5894166 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:58.720701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55103592-78da-41a3-8a7d-d9a45e38dbe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:58.721357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55103592-78da-41a3-8a7d-d9a45e38dbe2 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:58.721650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55103592-78da-41a3-8a7d-d9a45e38dbe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:58.722235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55103592-78da-41a3-8a7d-d9a45e38dbe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:58.731054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-55103592-78da-41a3-8a7d-d9a45e38dbe2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:58.747920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55103592-78da-41a3-8a7d-d9a45e38dbe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:58.749607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55103592-78da-41a3-8a7d-d9a45e38dbe2 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:58.769681 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3120/6277] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:25:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:25:58.819394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3d0d5ab0-e5ec-4dfb-b9c7-5c6bfeab7bd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2267c55067d549e79963425bfc21f45c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:58.852976 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3158/6278] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:58 2022] DELETE /identity/v3/users/2267c55067d549e79963425bfc21f45c => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:25:58.854778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3761237-95f2-47a6-81e2-49fc2bb61ae6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:58.855238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3761237-95f2-47a6-81e2-49fc2bb61ae6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:58.855574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3761237-95f2-47a6-81e2-49fc2bb61ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:58.884258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b3761237-95f2-47a6-81e2-49fc2bb61ae6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:25:58.887180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b3761237-95f2-47a6-81e2-49fc2bb61ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:58.991393 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3121/6279] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:25:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:25:59.041039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d5ab0-e5ec-4dfb-b9c7-5c6bfeab7bd2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:59.044192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3d0d5ab0-e5ec-4dfb-b9c7-5c6bfeab7bd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:59.125725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0d5ab0-e5ec-4dfb-b9c7-5c6bfeab7bd2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:59.129530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c215a-16f8-4099-8587-4f269770c5a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:59.130116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c215a-16f8-4099-8587-4f269770c5a2 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:59.130408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c215a-16f8-4099-8587-4f269770c5a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:59.130690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c215a-16f8-4099-8587-4f269770c5a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:59.141750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c215a-16f8-4099-8587-4f269770c5a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:59.145756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c215a-16f8-4099-8587-4f269770c5a2 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:59.155085 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3159/6280] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:25:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:25:59.367631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3761237-95f2-47a6-81e2-49fc2bb61ae6 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:59.367631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b3761237-95f2-47a6-81e2-49fc2bb61ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:25:59.380134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3761237-95f2-47a6-81e2-49fc2bb61ae6 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:59.388863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:59.389988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:59.390592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:59.390909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97 admin admin] PATH_INFO: `/v3/projects/f851982a0cf6417cbde471869958da3a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:59.392918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f851982a0cf6417cbde471869958da3a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:59.395233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:25:59.516621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f851982a0cf6417cbde471869958da3a', 'request_id': 'req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:59.527802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f851982a0cf6417cbde471869958da3a', 'request_id': 'req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:59.626424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f851982a0cf6417cbde471869958da3a', 'request_id': 'req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:59.653823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97 admin admin] The token cache is being invalidate because project f851982a0cf6417cbde471869958da3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:25:59.654161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:59.655522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:25:59.657655 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3122/6281] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:25:59 2022] DELETE /identity/v3/projects/f851982a0cf6417cbde471869958da3a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:25:59.661433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c215a-16f8-4099-8587-4f269770c5a2 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:25:59.677811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c215a-16f8-4099-8587-4f269770c5a2 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:25:59.771458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cdd06-6da6-437e-b9e7-9401691cfdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:25:59.772507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cdd06-6da6-437e-b9e7-9401691cfdb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:25:59.772794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cdd06-6da6-437e-b9e7-9401691cfdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:25:59.773414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cdd06-6da6-437e-b9e7-9401691cfdb1 admin admin] PATH_INFO: `/v3/users/9b4f328d2b2d476fa245063334995618` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:25:59.775530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cdd06-6da6-437e-b9e7-9401691cfdb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b4f328d2b2d476fa245063334995618)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:25:59.779309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cdd06-6da6-437e-b9e7-9401691cfdb1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:00.047876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:00.049585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:00.055677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-253cdd06-6da6-437e-b9e7-9401691cfdb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b4f328d2b2d476fa245063334995618'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:00.064284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-253cdd06-6da6-437e-b9e7-9401691cfdb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b4f328d2b2d476fa245063334995618'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:00.065035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-253cdd06-6da6-437e-b9e7-9401691cfdb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b4f328d2b2d476fa245063334995618'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:00.065697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-253cdd06-6da6-437e-b9e7-9401691cfdb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b4f328d2b2d476fa245063334995618'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:00.085950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb14b55-b7ce-4032-8c6b-9f5ca7c8ee97 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:00.094458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49799ea8-ca64-4b4c-8565-b86b3a7fa2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:00.095870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49799ea8-ca64-4b4c-8565-b86b3a7fa2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:00.096229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49799ea8-ca64-4b4c-8565-b86b3a7fa2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:00.096629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49799ea8-ca64-4b4c-8565-b86b3a7fa2b2 admin admin] PATH_INFO: `/v3/projects/f791936ea1fc49cfb07986f26a47cabe` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:00.098508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49799ea8-ca64-4b4c-8565-b86b3a7fa2b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f791936ea1fc49cfb07986f26a47cabe)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:00.101062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49799ea8-ca64-4b4c-8565-b86b3a7fa2b2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:00.190862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-253cdd06-6da6-437e-b9e7-9401691cfdb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b4f328d2b2d476fa245063334995618'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:00.219707 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3160/6282] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:25:59 2022] DELETE /identity/v3/users/9b4f328d2b2d476fa245063334995618 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:00.264971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-49799ea8-ca64-4b4c-8565-b86b3a7fa2b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f791936ea1fc49cfb07986f26a47cabe', 'request_id': 'req-49799ea8-ca64-4b4c-8565-b86b3a7fa2b2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:00.271417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-49799ea8-ca64-4b4c-8565-b86b3a7fa2b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f791936ea1fc49cfb07986f26a47cabe', 'request_id': 'req-49799ea8-ca64-4b4c-8565-b86b3a7fa2b2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:00.408115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-49799ea8-ca64-4b4c-8565-b86b3a7fa2b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f791936ea1fc49cfb07986f26a47cabe', 'request_id': 'req-49799ea8-ca64-4b4c-8565-b86b3a7fa2b2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:00.423080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-49799ea8-ca64-4b4c-8565-b86b3a7fa2b2 admin admin] The token cache is being invalidate because project f791936ea1fc49cfb07986f26a47cabe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:00.423330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-49799ea8-ca64-4b4c-8565-b86b3a7fa2b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:00.424213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-49799ea8-ca64-4b4c-8565-b86b3a7fa2b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:00.425849 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3123/6283] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:26:00 2022] DELETE /identity/v3/projects/f791936ea1fc49cfb07986f26a47cabe => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:00.431132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140d46d-b0e1-4ca8-8572-4338d5d2d3a4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:00.431432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140d46d-b0e1-4ca8-8572-4338d5d2d3a4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:00.431721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140d46d-b0e1-4ca8-8572-4338d5d2d3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:00.466012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d140d46d-b0e1-4ca8-8572-4338d5d2d3a4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:00.469661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d140d46d-b0e1-4ca8-8572-4338d5d2d3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:00.527272 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3161/6284] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:00.535160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4a171-aa28-475a-a340-10c213f89b84 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:00.535505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4a171-aa28-475a-a340-10c213f89b84 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:00.535831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad4a171-aa28-475a-a340-10c213f89b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:00.554525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0ad4a171-aa28-475a-a340-10c213f89b84 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:00.555775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0ad4a171-aa28-475a-a340-10c213f89b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:00.563791 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3124/6285] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:00.568081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140d46d-b0e1-4ca8-8572-4338d5d2d3a4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:00.576212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:00.581557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140d46d-b0e1-4ca8-8572-4338d5d2d3a4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:00.585204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465b5a9-2fe5-447e-8b90-4643f4b546e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:00.586469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465b5a9-2fe5-447e-8b90-4643f4b546e9 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:00.586762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465b5a9-2fe5-447e-8b90-4643f4b546e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:00.587048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465b5a9-2fe5-447e-8b90-4643f4b546e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:00.587803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465b5a9-2fe5-447e-8b90-4643f4b546e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:00.588139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465b5a9-2fe5-447e-8b90-4643f4b546e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:00.589764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5465b5a9-2fe5-447e-8b90-4643f4b546e9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:00.605867 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3162/6286] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:26:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:00.607057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4a171-aa28-475a-a340-10c213f89b84 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:00.616767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad4a171-aa28-475a-a340-10c213f89b84 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:00.624208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df0468-e59a-408f-b431-fb3586e51a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:00.624888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df0468-e59a-408f-b431-fb3586e51a91 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:00.625164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df0468-e59a-408f-b431-fb3586e51a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:00.625440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df0468-e59a-408f-b431-fb3586e51a91 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:00.626187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df0468-e59a-408f-b431-fb3586e51a91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:00.628105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df0468-e59a-408f-b431-fb3586e51a91 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:00.746692 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3125/6287] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:00 2022] POST /identity/v3/projects => generated 369 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:00.750898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465b5a9-2fe5-447e-8b90-4643f4b546e9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:00.762640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465b5a9-2fe5-447e-8b90-4643f4b546e9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:00.821642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df0468-e59a-408f-b431-fb3586e51a91 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:00.823746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-40df0468-e59a-408f-b431-fb3586e51a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:00.851272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbbcc1-3b71-4626-a3b8-125a373f6409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:00.852259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbbcc1-3b71-4626-a3b8-125a373f6409 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:00.852547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbbcc1-3b71-4626-a3b8-125a373f6409 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:00.853184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbbcc1-3b71-4626-a3b8-125a373f6409 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:00.853965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adbbcc1-3b71-4626-a3b8-125a373f6409 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:00.855948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adbbcc1-3b71-4626-a3b8-125a373f6409 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:00.890958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df0468-e59a-408f-b431-fb3586e51a91 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:00.895433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0b1ec-d198-4429-bb04-d0c027a15956 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:00.896092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0b1ec-d198-4429-bb04-d0c027a15956 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:00.898041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0b1ec-d198-4429-bb04-d0c027a15956 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:00.898041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0b1ec-d198-4429-bb04-d0c027a15956 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:00.898577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3cf0b1ec-d198-4429-bb04-d0c027a15956 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:00.912105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0b1ec-d198-4429-bb04-d0c027a15956 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:00.913587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0b1ec-d198-4429-bb04-d0c027a15956 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:00.932949 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3126/6288] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:00.932949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0b1ec-d198-4429-bb04-d0c027a15956 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:00.932949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3cf0b1ec-d198-4429-bb04-d0c027a15956 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:00.941169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0b1ec-d198-4429-bb04-d0c027a15956 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:00.953541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b41089-43bd-40a2-af71-e171c86f17ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:00.954789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b41089-43bd-40a2-af71-e171c86f17ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:00.955233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b41089-43bd-40a2-af71-e171c86f17ba admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:00.955727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b41089-43bd-40a2-af71-e171c86f17ba admin admin] PATH_INFO: `/v3/projects/31f42fca38a6448eb1d1cc64062a8ca7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:00.957330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b41089-43bd-40a2-af71-e171c86f17ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=31f42fca38a6448eb1d1cc64062a8ca7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:00.958890 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3163/6289] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:00 2022] POST /identity/v3/users => generated 285 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:00.959604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b41089-43bd-40a2-af71-e171c86f17ba admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:00.970056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbbcc1-3b71-4626-a3b8-125a373f6409 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:00.986323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbbcc1-3b71-4626-a3b8-125a373f6409 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:00.992879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40203f93-51cf-4208-8239-2d3fad6e4166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:00.995557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40203f93-51cf-4208-8239-2d3fad6e4166 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:00.995557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40203f93-51cf-4208-8239-2d3fad6e4166 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:00.995557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40203f93-51cf-4208-8239-2d3fad6e4166 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:00.996026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40203f93-51cf-4208-8239-2d3fad6e4166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:00.997640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40203f93-51cf-4208-8239-2d3fad6e4166 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:01.010352 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3164/6290] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:01.014990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40203f93-51cf-4208-8239-2d3fad6e4166 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:01.030539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40203f93-51cf-4208-8239-2d3fad6e4166 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:01.037334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5763dd-ff14-4ac8-a94a-8ddd373aca5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:01.038336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5763dd-ff14-4ac8-a94a-8ddd373aca5a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:01.039199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5763dd-ff14-4ac8-a94a-8ddd373aca5a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:01.039479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5763dd-ff14-4ac8-a94a-8ddd373aca5a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:01.040211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5763dd-ff14-4ac8-a94a-8ddd373aca5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:01.041820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5763dd-ff14-4ac8-a94a-8ddd373aca5a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:01.052742 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3165/6291] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:01.068672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5763dd-ff14-4ac8-a94a-8ddd373aca5a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:01.086436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5763dd-ff14-4ac8-a94a-8ddd373aca5a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:01.092927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ea232d-3d66-428f-a6e6-c983812d3a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:01.093860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea232d-3d66-428f-a6e6-c983812d3a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:01.094169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea232d-3d66-428f-a6e6-c983812d3a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:01.095496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea232d-3d66-428f-a6e6-c983812d3a88 admin admin] PATH_INFO: `/v3/projects/13837973df4240769de72a63520a0cbd/users/98f2a78a037c417ea3cff70dab6b0e99/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:01.104301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-22b41089-43bd-40a2-af71-e171c86f17ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31f42fca38a6448eb1d1cc64062a8ca7', 'request_id': 'req-22b41089-43bd-40a2-af71-e171c86f17ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:01.110483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-22b41089-43bd-40a2-af71-e171c86f17ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31f42fca38a6448eb1d1cc64062a8ca7', 'request_id': 'req-22b41089-43bd-40a2-af71-e171c86f17ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:01.115248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea232d-3d66-428f-a6e6-c983812d3a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13837973df4240769de72a63520a0cbd, user_id=98f2a78a037c417ea3cff70dab6b0e99, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:01.118102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ea232d-3d66-428f-a6e6-c983812d3a88 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:01.249236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:01.250878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-22b41089-43bd-40a2-af71-e171c86f17ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31f42fca38a6448eb1d1cc64062a8ca7', 'request_id': 'req-22b41089-43bd-40a2-af71-e171c86f17ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:01.258163 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3166/6292] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:01 2022] PUT /identity/v3/projects/13837973df4240769de72a63520a0cbd/users/98f2a78a037c417ea3cff70dab6b0e99/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:01.267317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-22b41089-43bd-40a2-af71-e171c86f17ba admin admin] The token cache is being invalidate because project 31f42fca38a6448eb1d1cc64062a8ca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:01.273749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a6e94-fe6e-4a6b-93d6-37583e61b5e5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:01.273749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a6e94-fe6e-4a6b-93d6-37583e61b5e5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:01.274094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-22b41089-43bd-40a2-af71-e171c86f17ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:01.274300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a6e94-fe6e-4a6b-93d6-37583e61b5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:01.274487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-22b41089-43bd-40a2-af71-e171c86f17ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:01.276036 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3127/6293] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:26:00 2022] DELETE /identity/v3/projects/31f42fca38a6448eb1d1cc64062a8ca7 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:01.281152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1a02a-f9ad-484a-9477-e3438ec36916 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:01.281443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1a02a-f9ad-484a-9477-e3438ec36916 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:01.284708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1a02a-f9ad-484a-9477-e3438ec36916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:01.309065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4bd1a02a-f9ad-484a-9477-e3438ec36916 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:01.310449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-070a6e94-fe6e-4a6b-93d6-37583e61b5e5 None None] MFA Rules not processed for user `98f2a78a037c417ea3cff70dab6b0e99`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:01.310577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4bd1a02a-f9ad-484a-9477-e3438ec36916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:01.317534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-070a6e94-fe6e-4a6b-93d6-37583e61b5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:01.387591 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3128/6294] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:26:01.396377 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3167/6295] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:01 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:01.396610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d32c6b-0d04-4252-8c41-b12f163c52ae None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:01.396610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d32c6b-0d04-4252-8c41-b12f163c52ae None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:01.396610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d32c6b-0d04-4252-8c41-b12f163c52ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:01.407669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a6e94-fe6e-4a6b-93d6-37583e61b5e5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:01.409585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-070a6e94-fe6e-4a6b-93d6-37583e61b5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:01.418509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-66d32c6b-0d04-4252-8c41-b12f163c52ae None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:01.419868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-66d32c6b-0d04-4252-8c41-b12f163c52ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:01.434195 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3129/6296] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:01.435289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d32c6b-0d04-4252-8c41-b12f163c52ae None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:01.445094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d32c6b-0d04-4252-8c41-b12f163c52ae None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:01.452768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee1440-aac7-48dd-948e-9a866746abb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:01.453776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee1440-aac7-48dd-948e-9a866746abb0 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:01.454164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee1440-aac7-48dd-948e-9a866746abb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:01.454519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee1440-aac7-48dd-948e-9a866746abb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:01.455340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee1440-aac7-48dd-948e-9a866746abb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:01.455769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee1440-aac7-48dd-948e-9a866746abb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:01.457604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee1440-aac7-48dd-948e-9a866746abb0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:01.475821 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3130/6297] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:01.477100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee1440-aac7-48dd-948e-9a866746abb0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:01.488695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee1440-aac7-48dd-948e-9a866746abb0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:01.496753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d15468-c9ad-4e52-be9f-6729d334071e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:01.497679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d15468-c9ad-4e52-be9f-6729d334071e admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:01.498058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d15468-c9ad-4e52-be9f-6729d334071e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:01.501474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d15468-c9ad-4e52-be9f-6729d334071e admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:01.501474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d15468-c9ad-4e52-be9f-6729d334071e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:01.502337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d15468-c9ad-4e52-be9f-6729d334071e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:01.546595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a6e94-fe6e-4a6b-93d6-37583e61b5e5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:01.553029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279063f4-7d6c-4c6f-96e8-0aa7ba30825c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:01.553747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279063f4-7d6c-4c6f-96e8-0aa7ba30825c service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:01.559080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279063f4-7d6c-4c6f-96e8-0aa7ba30825c service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:01.559080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279063f4-7d6c-4c6f-96e8-0aa7ba30825c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:01.564736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279063f4-7d6c-4c6f-96e8-0aa7ba30825c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:01.566690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279063f4-7d6c-4c6f-96e8-0aa7ba30825c service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:01.579710 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3168/6298] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:01 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:26:01.609263 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3131/6299] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:01 2022] POST /identity/v3/projects => generated 365 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:01.613425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279063f4-7d6c-4c6f-96e8-0aa7ba30825c service glance] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:01.625761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279063f4-7d6c-4c6f-96e8-0aa7ba30825c service glance] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:01.686630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a2869-0ef4-419f-8c3e-773a102a67d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:01.693938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a2869-0ef4-419f-8c3e-773a102a67d7 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:01.693938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a2869-0ef4-419f-8c3e-773a102a67d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:01.693938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a2869-0ef4-419f-8c3e-773a102a67d7 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:01.693938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a2869-0ef4-419f-8c3e-773a102a67d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:01.693938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a2869-0ef4-419f-8c3e-773a102a67d7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:01.793665 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3169/6300] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:01 2022] POST /identity/v3/users => generated 283 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:01.798272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d15468-c9ad-4e52-be9f-6729d334071e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:01.815211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d15468-c9ad-4e52-be9f-6729d334071e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:01.828061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f314e-01fc-498a-9a1c-8f44ff0d6e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:01.828432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f314e-01fc-498a-9a1c-8f44ff0d6e8d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:01.828688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f314e-01fc-498a-9a1c-8f44ff0d6e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:01.828894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f314e-01fc-498a-9a1c-8f44ff0d6e8d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:01.832431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f314e-01fc-498a-9a1c-8f44ff0d6e8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:01.843976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f314e-01fc-498a-9a1c-8f44ff0d6e8d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:01.863077 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3132/6301] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:01.867127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a2869-0ef4-419f-8c3e-773a102a67d7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:01.880935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a2869-0ef4-419f-8c3e-773a102a67d7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:01.889713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2f4bd-299b-4e9b-a1a3-357fe3ddb0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:01.890767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2f4bd-299b-4e9b-a1a3-357fe3ddb0e5 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:01.891654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2f4bd-299b-4e9b-a1a3-357fe3ddb0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:01.892042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2f4bd-299b-4e9b-a1a3-357fe3ddb0e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:01.893570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2f4bd-299b-4e9b-a1a3-357fe3ddb0e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:01.896480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2f4bd-299b-4e9b-a1a3-357fe3ddb0e5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:01.909671 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3170/6302] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:01.913920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f314e-01fc-498a-9a1c-8f44ff0d6e8d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:01.927353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f314e-01fc-498a-9a1c-8f44ff0d6e8d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:01.933030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67f295-0fa6-4e6b-a854-0702ba51211a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:01.934090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67f295-0fa6-4e6b-a854-0702ba51211a admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:01.939537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67f295-0fa6-4e6b-a854-0702ba51211a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:01.939537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b67f295-0fa6-4e6b-a854-0702ba51211a admin admin] PATH_INFO: `/v3/projects/6b0d918e277d4bc78e562134e0e42339/users/a1b4089f23c746a8ae580d7720de81b1/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:01.958244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b67f295-0fa6-4e6b-a854-0702ba51211a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b0d918e277d4bc78e562134e0e42339, user_id=a1b4089f23c746a8ae580d7720de81b1, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:01.963041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b67f295-0fa6-4e6b-a854-0702ba51211a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:02.055315 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3133/6303] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:01 2022] PUT /identity/v3/projects/6b0d918e277d4bc78e562134e0e42339/users/a1b4089f23c746a8ae580d7720de81b1/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:02.064022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:02.073134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f56de8-4fe2-498a-af65-5bdb55a72264 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:02.073134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f56de8-4fe2-498a-af65-5bdb55a72264 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:02.073485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f56de8-4fe2-498a-af65-5bdb55a72264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:02.109946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-02f56de8-4fe2-498a-af65-5bdb55a72264 None None] MFA Rules not processed for user `a1b4089f23c746a8ae580d7720de81b1`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:02.111749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-02f56de8-4fe2-498a-af65-5bdb55a72264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:02.192530 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3171/6304] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:02 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:02.198725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67f295-0fa6-4e6b-a854-0702ba51211a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:02.283056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b67f295-0fa6-4e6b-a854-0702ba51211a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:02.288477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4f39b-8ab0-4391-850b-59f1b9177a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:02.289162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4f39b-8ab0-4391-850b-59f1b9177a98 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:02.289445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4f39b-8ab0-4391-850b-59f1b9177a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:02.289803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f4f39b-8ab0-4391-850b-59f1b9177a98 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:02.294772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4f39b-8ab0-4391-850b-59f1b9177a98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:02.294772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f4f39b-8ab0-4391-850b-59f1b9177a98 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:02.429823 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3134/6305] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:02 2022] POST /identity/v3/projects => generated 367 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:02.431933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f56de8-4fe2-498a-af65-5bdb55a72264 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:02.454035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f56de8-4fe2-498a-af65-5bdb55a72264 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:02.550730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70412307-d597-4ebf-a35f-70eff1884ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:02.551633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70412307-d597-4ebf-a35f-70eff1884ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:02.552185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70412307-d597-4ebf-a35f-70eff1884ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:02.555132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70412307-d597-4ebf-a35f-70eff1884ca4 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:02.555132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70412307-d597-4ebf-a35f-70eff1884ca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:02.556925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70412307-d597-4ebf-a35f-70eff1884ca4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:02.576170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:UPDATE=1,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:02.683821 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3172/6306] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:02 2022] POST /identity/v3/users => generated 290 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:02.687927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4f39b-8ab0-4391-850b-59f1b9177a98 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:02.698976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f4f39b-8ab0-4391-850b-59f1b9177a98 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:02.705179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a364a-4617-4b5c-adde-6fbd5573406d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:02.707361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a364a-4617-4b5c-adde-6fbd5573406d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:02.707361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a364a-4617-4b5c-adde-6fbd5573406d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:02.707361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a364a-4617-4b5c-adde-6fbd5573406d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:02.707361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a364a-4617-4b5c-adde-6fbd5573406d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:02.708821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a364a-4617-4b5c-adde-6fbd5573406d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:02.720911 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3135/6307] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:02.725399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70412307-d597-4ebf-a35f-70eff1884ca4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:02.736404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70412307-d597-4ebf-a35f-70eff1884ca4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:02.743170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c5f5e-00b0-4eb0-aaee-0ec394de48d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:02.744233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c5f5e-00b0-4eb0-aaee-0ec394de48d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:02.744577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c5f5e-00b0-4eb0-aaee-0ec394de48d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:02.744869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c5f5e-00b0-4eb0-aaee-0ec394de48d5 admin admin] PATH_INFO: `/v3/projects/b8f0c827196d4ac7b046c431195aa987/users/6047f89d3fad4f8eae1b5f451a862540/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:02.762225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c5f5e-00b0-4eb0-aaee-0ec394de48d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8f0c827196d4ac7b046c431195aa987, user_id=6047f89d3fad4f8eae1b5f451a862540, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:02.767749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c5f5e-00b0-4eb0-aaee-0ec394de48d5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:02.836118 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3173/6308] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:02 2022] PUT /identity/v3/projects/b8f0c827196d4ac7b046c431195aa987/users/6047f89d3fad4f8eae1b5f451a862540/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:02.840205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a364a-4617-4b5c-adde-6fbd5573406d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:02.852500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a364a-4617-4b5c-adde-6fbd5573406d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:02.965175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907fb483-b245-444c-8f2a-6351a24b48c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:02.965999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fb483-b245-444c-8f2a-6351a24b48c6 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:02.966362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fb483-b245-444c-8f2a-6351a24b48c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:02.966735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907fb483-b245-444c-8f2a-6351a24b48c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:02.967632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fb483-b245-444c-8f2a-6351a24b48c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:02.969339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907fb483-b245-444c-8f2a-6351a24b48c6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:02.981440 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3136/6309] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:02.985407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c5f5e-00b0-4eb0-aaee-0ec394de48d5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:03.030828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c5f5e-00b0-4eb0-aaee-0ec394de48d5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:03.039235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b0c38-edb7-4750-88f2-af773ee05f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:03.040276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b0c38-edb7-4750-88f2-af773ee05f58 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:03.040482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b0c38-edb7-4750-88f2-af773ee05f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:03.040820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b0c38-edb7-4750-88f2-af773ee05f58 admin admin] PATH_INFO: `/v3/projects/b8f0c827196d4ac7b046c431195aa987/users/6047f89d3fad4f8eae1b5f451a862540/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:03.043076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b0c38-edb7-4750-88f2-af773ee05f58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8f0c827196d4ac7b046c431195aa987, user_id=6047f89d3fad4f8eae1b5f451a862540, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:03.046333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b0c38-edb7-4750-88f2-af773ee05f58 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:03.073801 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3174/6310] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:02 2022] PUT /identity/v3/projects/b8f0c827196d4ac7b046c431195aa987/users/6047f89d3fad4f8eae1b5f451a862540/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:03.076413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395effd6-422d-4af7-ac4d-063bcf21bf4b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:03.076413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395effd6-422d-4af7-ac4d-063bcf21bf4b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:03.076413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395effd6-422d-4af7-ac4d-063bcf21bf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:03.112587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-395effd6-422d-4af7-ac4d-063bcf21bf4b None None] MFA Rules not processed for user `6047f89d3fad4f8eae1b5f451a862540`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:03.114567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-395effd6-422d-4af7-ac4d-063bcf21bf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:03.198565 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3137/6311] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:03 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:03.202902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b0c38-edb7-4750-88f2-af773ee05f58 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:03.217496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b0c38-edb7-4750-88f2-af773ee05f58 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:03.223523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30efad2b-e8d5-4fc5-ad61-891b0516b6c9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:03.224278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efad2b-e8d5-4fc5-ad61-891b0516b6c9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:03.224884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efad2b-e8d5-4fc5-ad61-891b0516b6c9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:03.225395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efad2b-e8d5-4fc5-ad61-891b0516b6c9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:03.226910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30efad2b-e8d5-4fc5-ad61-891b0516b6c9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:03.229080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30efad2b-e8d5-4fc5-ad61-891b0516b6c9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:03.303931 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3175/6312] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:26:03 2022] POST /identity/v3/roles => generated 235 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:03.308090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395effd6-422d-4af7-ac4d-063bcf21bf4b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:03.319108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395effd6-422d-4af7-ac4d-063bcf21bf4b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:03.325969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55bdfca-30c9-4ad2-8180-4fa14b3062d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:03.327082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bdfca-30c9-4ad2-8180-4fa14b3062d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:03.327461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bdfca-30c9-4ad2-8180-4fa14b3062d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:03.331908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bdfca-30c9-4ad2-8180-4fa14b3062d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:03.331908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55bdfca-30c9-4ad2-8180-4fa14b3062d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:03.331908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55bdfca-30c9-4ad2-8180-4fa14b3062d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:03.427899 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3138/6313] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:26:03 2022] POST /identity/v3/roles => generated 235 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:03.436000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30efad2b-e8d5-4fc5-ad61-891b0516b6c9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:03.447425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30efad2b-e8d5-4fc5-ad61-891b0516b6c9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:03.452476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e3538-b616-4c6f-8f1d-c52b84e43a22 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:03.453401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e3538-b616-4c6f-8f1d-c52b84e43a22 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:03.453715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e3538-b616-4c6f-8f1d-c52b84e43a22 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:03.454254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e3538-b616-4c6f-8f1d-c52b84e43a22 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:03.455021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e3538-b616-4c6f-8f1d-c52b84e43a22 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:03.456637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e3538-b616-4c6f-8f1d-c52b84e43a22 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:03.515658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55bdfca-30c9-4ad2-8180-4fa14b3062d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:03.529231 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3176/6314] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:26:03 2022] POST /identity/v3/roles => generated 235 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:03.531067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55bdfca-30c9-4ad2-8180-4fa14b3062d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:03.539155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fccbd-ea05-4aa2-817c-10a63351abd8 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:03.540176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fccbd-ea05-4aa2-817c-10a63351abd8 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:03.540844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fccbd-ea05-4aa2-817c-10a63351abd8 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:03.541146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fccbd-ea05-4aa2-817c-10a63351abd8 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:03.542088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fccbd-ea05-4aa2-817c-10a63351abd8 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:03.543940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fccbd-ea05-4aa2-817c-10a63351abd8 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:03.628598 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3139/6315] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:26:03 2022] POST /identity/v3/domains => generated 274 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:03.629967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e3538-b616-4c6f-8f1d-c52b84e43a22 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:03.651886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e3538-b616-4c6f-8f1d-c52b84e43a22 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:03.730730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964497c-1237-4af4-9bbf-a94677eab8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:03.731533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964497c-1237-4af4-9bbf-a94677eab8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:03.731988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964497c-1237-4af4-9bbf-a94677eab8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:03.734482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964497c-1237-4af4-9bbf-a94677eab8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:03.735257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964497c-1237-4af4-9bbf-a94677eab8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:03.737229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964497c-1237-4af4-9bbf-a94677eab8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:03.843869 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3177/6316] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:03 2022] POST /identity/v3/projects => generated 400 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:03.848229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fccbd-ea05-4aa2-817c-10a63351abd8 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:03.859074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fccbd-ea05-4aa2-817c-10a63351abd8 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:03.939232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3808d-b798-4b6e-a472-6060db4898e2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:03.940115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3808d-b798-4b6e-a472-6060db4898e2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:03.943720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3808d-b798-4b6e-a472-6060db4898e2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:03.943720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3808d-b798-4b6e-a472-6060db4898e2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/groups` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:03.943720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3808d-b798-4b6e-a472-6060db4898e2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:03.944330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3808d-b798-4b6e-a472-6060db4898e2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:04.017662 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3140/6317] 10.222.0.33 () {60 vars in 1146 bytes} [Sat May 28 21:26:03 2022] POST /identity/v3/groups => generated 301 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:04.024752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964497c-1237-4af4-9bbf-a94677eab8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:04.037490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964497c-1237-4af4-9bbf-a94677eab8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:04.045194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365110b6-8f26-4e62-a324-4f2012e72250 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:04.045910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365110b6-8f26-4e62-a324-4f2012e72250 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:04.046363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365110b6-8f26-4e62-a324-4f2012e72250 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:04.050470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365110b6-8f26-4e62-a324-4f2012e72250 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:04.050470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365110b6-8f26-4e62-a324-4f2012e72250 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:04.050470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365110b6-8f26-4e62-a324-4f2012e72250 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:04.127814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:04.143197 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3178/6318] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:26:04 2022] POST /identity/v3/roles => generated 235 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:04.143438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3808d-b798-4b6e-a472-6060db4898e2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:04.161944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3808d-b798-4b6e-a472-6060db4898e2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:04.167829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b945df-6aaf-48b5-93b1-4cb7f1306a7d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:04.168900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b945df-6aaf-48b5-93b1-4cb7f1306a7d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:04.169370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b945df-6aaf-48b5-93b1-4cb7f1306a7d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:04.169658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b945df-6aaf-48b5-93b1-4cb7f1306a7d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:04.170505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b945df-6aaf-48b5-93b1-4cb7f1306a7d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:04.172987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b945df-6aaf-48b5-93b1-4cb7f1306a7d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:04.262411 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3141/6319] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:04 2022] POST /identity/v3/users => generated 442 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:04.269361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365110b6-8f26-4e62-a324-4f2012e72250 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:04.280543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365110b6-8f26-4e62-a324-4f2012e72250 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:04.286617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575efbc6-bd89-4c83-8c90-6c7c95e59d7f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:04.287406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575efbc6-bd89-4c83-8c90-6c7c95e59d7f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:04.299597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575efbc6-bd89-4c83-8c90-6c7c95e59d7f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:04.299597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575efbc6-bd89-4c83-8c90-6c7c95e59d7f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:04.299597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575efbc6-bd89-4c83-8c90-6c7c95e59d7f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:04.299597 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-575efbc6-bd89-4c83-8c90-6c7c95e59d7f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:04.299597 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:26:04.299597 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:26:04.300856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575efbc6-bd89-4c83-8c90-6c7c95e59d7f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:04.391268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:04.399132 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3179/6320] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:26:04 2022] POST /identity/v3/roles => generated 270 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:04.405155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b945df-6aaf-48b5-93b1-4cb7f1306a7d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:04.418392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b945df-6aaf-48b5-93b1-4cb7f1306a7d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:04.425433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f520d3-2a25-4b03-bf7f-6fe9f95a7cff tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:04.430904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f520d3-2a25-4b03-bf7f-6fe9f95a7cff tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:04.430904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f520d3-2a25-4b03-bf7f-6fe9f95a7cff tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:04.430904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f520d3-2a25-4b03-bf7f-6fe9f95a7cff tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/projects/b321a6894a5d4c79952076f6c0408168/users/1f04e663b88b4151ae962359fbc2451d/roles/f46aaf4203cb4fb1aca20e2c00ec2839` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:04.452687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f520d3-2a25-4b03-bf7f-6fe9f95a7cff tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_grant(project_id=b321a6894a5d4c79952076f6c0408168, user_id=1f04e663b88b4151ae962359fbc2451d, role_id=f46aaf4203cb4fb1aca20e2c00ec2839)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:04.455854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f520d3-2a25-4b03-bf7f-6fe9f95a7cff tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:04.508678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575efbc6-bd89-4c83-8c90-6c7c95e59d7f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:04.517263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-575efbc6-bd89-4c83-8c90-6c7c95e59d7f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:04.540362 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3142/6321] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:04 2022] PUT /identity/v3/projects/b321a6894a5d4c79952076f6c0408168/users/1f04e663b88b4151ae962359fbc2451d/roles/f46aaf4203cb4fb1aca20e2c00ec2839 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:04.546042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f520d3-2a25-4b03-bf7f-6fe9f95a7cff tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:04.614332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575efbc6-bd89-4c83-8c90-6c7c95e59d7f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:04.628592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f520d3-2a25-4b03-bf7f-6fe9f95a7cff tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:04.637931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f32aa-6722-4ef0-bbe5-d0adbcb7eaed tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:04.641488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f32aa-6722-4ef0-bbe5-d0adbcb7eaed tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:04.641488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f32aa-6722-4ef0-bbe5-d0adbcb7eaed tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:04.641488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f32aa-6722-4ef0-bbe5-d0adbcb7eaed tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:04.641488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f32aa-6722-4ef0-bbe5-d0adbcb7eaed tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Adding query filter params (scope.project.id=b321a6894a5d4c79952076f6c0408168, user.id=1f04e663b88b4151ae962359fbc2451d) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:04.641964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f32aa-6722-4ef0-bbe5-d0adbcb7eaed tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:04.642477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f32aa-6722-4ef0-bbe5-d0adbcb7eaed tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:04.696374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753f380-68f0-4fa8-8213-88d04797382b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:04.697690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753f380-68f0-4fa8-8213-88d04797382b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:04.698187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753f380-68f0-4fa8-8213-88d04797382b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:04.698632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753f380-68f0-4fa8-8213-88d04797382b admin admin] PATH_INFO: `/v3/users/1166808eb99545978b43831f7f18c0df` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:04.700072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753f380-68f0-4fa8-8213-88d04797382b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1166808eb99545978b43831f7f18c0df)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:04.702687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753f380-68f0-4fa8-8213-88d04797382b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:04.752366 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3143/6322] 10.222.0.33 () {58 vars in 1355 bytes} [Sat May 28 21:26:04 2022] GET /identity/v3/role_assignments?scope.project.id=b321a6894a5d4c79952076f6c0408168&user.id=1f04e663b88b4151ae962359fbc2451d&effective => generated 228 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:04.756616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f32aa-6722-4ef0-bbe5-d0adbcb7eaed tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:04.769485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f32aa-6722-4ef0-bbe5-d0adbcb7eaed tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:04.774664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f67a5-b893-4046-b8d0-e64b264996c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:04.775647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f67a5-b893-4046-b8d0-e64b264996c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:04.776010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f67a5-b893-4046-b8d0-e64b264996c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:04.776369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f67a5-b893-4046-b8d0-e64b264996c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/projects/b321a6894a5d4c79952076f6c0408168/users/1f04e663b88b4151ae962359fbc2451d/roles/f46aaf4203cb4fb1aca20e2c00ec2839` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:04.778984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f67a5-b893-4046-b8d0-e64b264996c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=b321a6894a5d4c79952076f6c0408168, user_id=1f04e663b88b4151ae962359fbc2451d, role_id=f46aaf4203cb4fb1aca20e2c00ec2839)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:04.781819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116f67a5-b893-4046-b8d0-e64b264996c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:04.789338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-116f67a5-b893-4046-b8d0-e64b264996c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invalidating the token cache because role f46aaf4203cb4fb1aca20e2c00ec2839 was removed from user 1f04e663b88b4151ae962359fbc2451d on project b321a6894a5d4c79952076f6c0408168. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:04.789608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-116f67a5-b893-4046-b8d0-e64b264996c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:04.790667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-116f67a5-b893-4046-b8d0-e64b264996c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:04.941647 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3144/6323] 10.222.0.33 () {58 vars in 1466 bytes} [Sat May 28 21:26:04 2022] DELETE /identity/v3/projects/b321a6894a5d4c79952076f6c0408168/users/1f04e663b88b4151ae962359fbc2451d/roles/f46aaf4203cb4fb1aca20e2c00ec2839 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:04.946670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f67a5-b893-4046-b8d0-e64b264996c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:04.954109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-116f67a5-b893-4046-b8d0-e64b264996c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:04.980576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a753f380-68f0-4fa8-8213-88d04797382b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1166808eb99545978b43831f7f18c0df'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:04.995130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a753f380-68f0-4fa8-8213-88d04797382b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1166808eb99545978b43831f7f18c0df'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:04.996600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a753f380-68f0-4fa8-8213-88d04797382b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1166808eb99545978b43831f7f18c0df'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:04.997988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a753f380-68f0-4fa8-8213-88d04797382b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1166808eb99545978b43831f7f18c0df'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:05.044588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f67a5-b893-4046-b8d0-e64b264996c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:05.049694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a187c-c426-40bb-be32-a743884a984e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:05.052056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a187c-c426-40bb-be32-a743884a984e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:05.052056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a187c-c426-40bb-be32-a743884a984e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:05.052056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a187c-c426-40bb-be32-a743884a984e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/f46aaf4203cb4fb1aca20e2c00ec2839` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:05.052943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a187c-c426-40bb-be32-a743884a984e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=f46aaf4203cb4fb1aca20e2c00ec2839)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:05.054755 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-2c8a187c-c426-40bb-be32-a743884a984e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:05.054755 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:26:05.054755 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:26:05.055753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8a187c-c426-40bb-be32-a743884a984e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:05.092004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a753f380-68f0-4fa8-8213-88d04797382b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1166808eb99545978b43831f7f18c0df'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:05.111411 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3180/6324] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:04 2022] DELETE /identity/v3/users/1166808eb99545978b43831f7f18c0df => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:05.115983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955d94c-410a-422c-8d2f-6888377fe038 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:05.116275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955d94c-410a-422c-8d2f-6888377fe038 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:05.117115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955d94c-410a-422c-8d2f-6888377fe038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:05.133906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9955d94c-410a-422c-8d2f-6888377fe038 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:05.135116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9955d94c-410a-422c-8d2f-6888377fe038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:05.205886 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3181/6325] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:26:05.219703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955d94c-410a-422c-8d2f-6888377fe038 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:05.221289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9955d94c-410a-422c-8d2f-6888377fe038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:05.234790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2c8a187c-c426-40bb-be32-a743884a984e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invalidating the token cache because role f46aaf4203cb4fb1aca20e2c00ec2839 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:05.235096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2c8a187c-c426-40bb-be32-a743884a984e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:05.239924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2c8a187c-c426-40bb-be32-a743884a984e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:05.242782 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3145/6326] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:04 2022] DELETE /identity/v3/roles/f46aaf4203cb4fb1aca20e2c00ec2839 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:05.248275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a187c-c426-40bb-be32-a743884a984e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:05.249700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2c8a187c-c426-40bb-be32-a743884a984e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:05.297110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955d94c-410a-422c-8d2f-6888377fe038 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:05.334901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8a187c-c426-40bb-be32-a743884a984e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:05.341592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56291446-3f08-4be0-b5e3-9a078ffb8418 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:05.342816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56291446-3f08-4be0-b5e3-9a078ffb8418 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:05.343291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56291446-3f08-4be0-b5e3-9a078ffb8418 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:05.343736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56291446-3f08-4be0-b5e3-9a078ffb8418 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/projects/b321a6894a5d4c79952076f6c0408168/users/1f04e663b88b4151ae962359fbc2451d/roles/5df3bb3dada94e2e8d1d425c4b107106` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:05.346999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56291446-3f08-4be0-b5e3-9a078ffb8418 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_grant(project_id=b321a6894a5d4c79952076f6c0408168, user_id=1f04e663b88b4151ae962359fbc2451d, role_id=5df3bb3dada94e2e8d1d425c4b107106)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:05.349906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56291446-3f08-4be0-b5e3-9a078ffb8418 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:05.363584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9adf8-ced3-44b0-94c8-074ff3a7e738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:05.364258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9adf8-ced3-44b0-94c8-074ff3a7e738 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:05.364558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9adf8-ced3-44b0-94c8-074ff3a7e738 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:05.364868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9adf8-ced3-44b0-94c8-074ff3a7e738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:05.366742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b8d9adf8-ced3-44b0-94c8-074ff3a7e738 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:05.433504 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3146/6327] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:05 2022] PUT /identity/v3/projects/b321a6894a5d4c79952076f6c0408168/users/1f04e663b88b4151ae962359fbc2451d/roles/5df3bb3dada94e2e8d1d425c4b107106 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:05.444969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56291446-3f08-4be0-b5e3-9a078ffb8418 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:05.444969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-56291446-3f08-4be0-b5e3-9a078ffb8418 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:05.445234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9adf8-ced3-44b0-94c8-074ff3a7e738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:05.445234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9adf8-ced3-44b0-94c8-074ff3a7e738 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:05.452794 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3182/6328] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:05.467456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9adf8-ced3-44b0-94c8-074ff3a7e738 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:05.481463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9adf8-ced3-44b0-94c8-074ff3a7e738 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:05.542979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56291446-3f08-4be0-b5e3-9a078ffb8418 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:05.545742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c9f51-1463-4944-b7ea-a06290f097b6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:05.547102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c9f51-1463-4944-b7ea-a06290f097b6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:05.547249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c9f51-1463-4944-b7ea-a06290f097b6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:05.547512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241c9f51-1463-4944-b7ea-a06290f097b6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:05.549209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae5878-6d4c-469f-8c2f-04503b0040c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:05.550092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae5878-6d4c-469f-8c2f-04503b0040c2 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:05.550384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae5878-6d4c-469f-8c2f-04503b0040c2 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:05.551333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae5878-6d4c-469f-8c2f-04503b0040c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:05.551763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c9f51-1463-4944-b7ea-a06290f097b6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=9ff56f0c05ab49039e0ff27f5ae8fbb3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:05.553044 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-241c9f51-1463-4944-b7ea-a06290f097b6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:05.553044 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:26:05.553044 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:26:05.553605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d3ae5878-6d4c-469f-8c2f-04503b0040c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:05.554318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241c9f51-1463-4944-b7ea-a06290f097b6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:05.577097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae5878-6d4c-469f-8c2f-04503b0040c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:05.579799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae5878-6d4c-469f-8c2f-04503b0040c2 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:05.596095 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3147/6329] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:05.597462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae5878-6d4c-469f-8c2f-04503b0040c2 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:05.598200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:05.599443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d3ae5878-6d4c-469f-8c2f-04503b0040c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:05.669173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae5878-6d4c-469f-8c2f-04503b0040c2 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:05.673725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e99c5b-d702-4143-bbda-0df8d755fafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:05.674652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e99c5b-d702-4143-bbda-0df8d755fafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:05.674939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e99c5b-d702-4143-bbda-0df8d755fafd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:05.675243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e99c5b-d702-4143-bbda-0df8d755fafd admin admin] PATH_INFO: `/v3/users/857a887b6c7c41588afb2845eed6f309` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:05.676474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e99c5b-d702-4143-bbda-0df8d755fafd admin admin] RBAC: Authorizing `identity:delete_user(user_id=857a887b6c7c41588afb2845eed6f309)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:05.678460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e99c5b-d702-4143-bbda-0df8d755fafd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:05.756151 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3183/6330] 10.222.0.33 () {60 vars in 1362 bytes} [Sat May 28 21:26:05 2022] PUT /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3 => generated 521 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:05.760915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c9f51-1463-4944-b7ea-a06290f097b6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:05.824971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241c9f51-1463-4944-b7ea-a06290f097b6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:05.829978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5cdcc-af3d-4e09-bdcb-7f26272f1403 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:05.830700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5cdcc-af3d-4e09-bdcb-7f26272f1403 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:05.830998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5cdcc-af3d-4e09-bdcb-7f26272f1403 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:05.831426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5cdcc-af3d-4e09-bdcb-7f26272f1403 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:05.832363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5cdcc-af3d-4e09-bdcb-7f26272f1403 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Adding query filter params (scope.project.id=b321a6894a5d4c79952076f6c0408168, user.id=1f04e663b88b4151ae962359fbc2451d) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:05.832674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5cdcc-af3d-4e09-bdcb-7f26272f1403 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:05.834502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5cdcc-af3d-4e09-bdcb-7f26272f1403 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:05.884916 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3184/6331] 10.222.0.33 () {58 vars in 1355 bytes} [Sat May 28 21:26:05 2022] GET /identity/v3/role_assignments?scope.project.id=b321a6894a5d4c79952076f6c0408168&user.id=1f04e663b88b4151ae962359fbc2451d&effective => generated 1061 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:26:05.885994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5cdcc-af3d-4e09-bdcb-7f26272f1403 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:05.891065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-69b5cdcc-af3d-4e09-bdcb-7f26272f1403 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:05.953418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5cdcc-af3d-4e09-bdcb-7f26272f1403 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:05.958360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed1388-42d4-4c27-9201-00bd24a7a460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:05.960018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed1388-42d4-4c27-9201-00bd24a7a460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:05.960018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed1388-42d4-4c27-9201-00bd24a7a460 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:05.960018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed1388-42d4-4c27-9201-00bd24a7a460 admin admin] PATH_INFO: `/v3/projects/30abceacd3ec4b9db2a28bc92b949570` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:05.961718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed1388-42d4-4c27-9201-00bd24a7a460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30abceacd3ec4b9db2a28bc92b949570)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:05.963667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed1388-42d4-4c27-9201-00bd24a7a460 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:05.995587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-20e99c5b-d702-4143-bbda-0df8d755fafd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '857a887b6c7c41588afb2845eed6f309'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:05.995587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-20e99c5b-d702-4143-bbda-0df8d755fafd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '857a887b6c7c41588afb2845eed6f309'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:06.001175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-20e99c5b-d702-4143-bbda-0df8d755fafd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '857a887b6c7c41588afb2845eed6f309'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:06.150847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20ed1388-42d4-4c27-9201-00bd24a7a460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30abceacd3ec4b9db2a28bc92b949570', 'request_id': 'req-20ed1388-42d4-4c27-9201-00bd24a7a460'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:06.157116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20ed1388-42d4-4c27-9201-00bd24a7a460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30abceacd3ec4b9db2a28bc92b949570', 'request_id': 'req-20ed1388-42d4-4c27-9201-00bd24a7a460'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:06.180095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-20e99c5b-d702-4143-bbda-0df8d755fafd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '857a887b6c7c41588afb2845eed6f309'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:06.180730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-20e99c5b-d702-4143-bbda-0df8d755fafd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '857a887b6c7c41588afb2845eed6f309'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:06.199897 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3148/6332] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:05 2022] DELETE /identity/v3/users/857a887b6c7c41588afb2845eed6f309 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:06.200854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e99c5b-d702-4143-bbda-0df8d755fafd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:06.202268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-20e99c5b-d702-4143-bbda-0df8d755fafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:06.260292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:06.272397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e99c5b-d702-4143-bbda-0df8d755fafd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:06.277058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d747ed-6499-4073-b423-f6094dd7c1a6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:06.278012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d747ed-6499-4073-b423-f6094dd7c1a6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:06.278307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d747ed-6499-4073-b423-f6094dd7c1a6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:06.278590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d747ed-6499-4073-b423-f6094dd7c1a6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:06.280476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d747ed-6499-4073-b423-f6094dd7c1a6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=9ff56f0c05ab49039e0ff27f5ae8fbb3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:06.281155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20ed1388-42d4-4c27-9201-00bd24a7a460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30abceacd3ec4b9db2a28bc92b949570', 'request_id': 'req-20ed1388-42d4-4c27-9201-00bd24a7a460'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:06.281783 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-34d747ed-6499-4073-b423-f6094dd7c1a6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:06.281783 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:26:06.281783 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:26:06.282778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d747ed-6499-4073-b423-f6094dd7c1a6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:06.294271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20ed1388-42d4-4c27-9201-00bd24a7a460 admin admin] The token cache is being invalidate because project 30abceacd3ec4b9db2a28bc92b949570 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:06.294569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20ed1388-42d4-4c27-9201-00bd24a7a460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:06.295362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-20ed1388-42d4-4c27-9201-00bd24a7a460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:06.296996 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3185/6333] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:26:05 2022] DELETE /identity/v3/projects/30abceacd3ec4b9db2a28bc92b949570 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:06.302104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec011469-974a-48af-bc20-762fdb3192d9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:06.302464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec011469-974a-48af-bc20-762fdb3192d9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:06.302812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec011469-974a-48af-bc20-762fdb3192d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:06.322658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ec011469-974a-48af-bc20-762fdb3192d9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:06.323895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ec011469-974a-48af-bc20-762fdb3192d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:06.385106 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3149/6334] 10.222.0.33 () {58 vars in 1346 bytes} [Sat May 28 21:26:06 2022] DELETE /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:06.385877 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3186/6335] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:26:06.389279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d747ed-6499-4073-b423-f6094dd7c1a6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:06.391024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-34d747ed-6499-4073-b423-f6094dd7c1a6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:06.394309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc369f-9fe4-4945-b90f-0e263ebda1b6 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:06.394710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc369f-9fe4-4945-b90f-0e263ebda1b6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:06.394994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc369f-9fe4-4945-b90f-0e263ebda1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:06.414496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f3bc369f-9fe4-4945-b90f-0e263ebda1b6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:06.416468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f3bc369f-9fe4-4945-b90f-0e263ebda1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:06.477811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d747ed-6499-4073-b423-f6094dd7c1a6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:06.483290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9956894-4b3d-4499-90e4-3f1177d8d8a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:06.483964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9956894-4b3d-4499-90e4-3f1177d8d8a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:06.484238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9956894-4b3d-4499-90e4-3f1177d8d8a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:06.484522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9956894-4b3d-4499-90e4-3f1177d8d8a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/role_assignments` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:06.485368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9956894-4b3d-4499-90e4-3f1177d8d8a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Adding query filter params (scope.project.id=b321a6894a5d4c79952076f6c0408168, user.id=1f04e663b88b4151ae962359fbc2451d) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:06.485780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9956894-4b3d-4499-90e4-3f1177d8d8a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:06.488280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9956894-4b3d-4499-90e4-3f1177d8d8a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:06.504568 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3187/6336] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:06.505579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc369f-9fe4-4945-b90f-0e263ebda1b6 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:06.507422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f3bc369f-9fe4-4945-b90f-0e263ebda1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:06.523339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc369f-9fe4-4945-b90f-0e263ebda1b6 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:06.531751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e8ca3-5c51-4558-8048-5b62f8c6778c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:06.532779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e8ca3-5c51-4558-8048-5b62f8c6778c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:06.532779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e8ca3-5c51-4558-8048-5b62f8c6778c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:06.533100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e8ca3-5c51-4558-8048-5b62f8c6778c admin admin] PATH_INFO: `/v3/projects/5a8841fa441d413a92a15510a9d082c6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:06.535445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e8ca3-5c51-4558-8048-5b62f8c6778c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a8841fa441d413a92a15510a9d082c6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:06.537884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e8ca3-5c51-4558-8048-5b62f8c6778c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:06.577737 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3150/6337] 10.222.0.33 () {58 vars in 1355 bytes} [Sat May 28 21:26:06 2022] GET /identity/v3/role_assignments?scope.project.id=b321a6894a5d4c79952076f6c0408168&user.id=1f04e663b88b4151ae962359fbc2451d&effective => generated 577 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:06.577737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9956894-4b3d-4499-90e4-3f1177d8d8a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:06.588540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9956894-4b3d-4499-90e4-3f1177d8d8a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:06.594059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3797c5-7852-48d3-9dd3-0b8c75884b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:06.595050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3797c5-7852-48d3-9dd3-0b8c75884b61 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:06.595230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3797c5-7852-48d3-9dd3-0b8c75884b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:06.595675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3797c5-7852-48d3-9dd3-0b8c75884b61 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:06.596448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3797c5-7852-48d3-9dd3-0b8c75884b61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:06.597069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3797c5-7852-48d3-9dd3-0b8c75884b61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:06.598792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3797c5-7852-48d3-9dd3-0b8c75884b61 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:06.616059 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3151/6338] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:06.616059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3797c5-7852-48d3-9dd3-0b8c75884b61 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:06.626907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3797c5-7852-48d3-9dd3-0b8c75884b61 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:06.631761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e66718-7aff-4a98-a6b0-a11ea5b4c5b7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:06.632708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e66718-7aff-4a98-a6b0-a11ea5b4c5b7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:06.632990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e66718-7aff-4a98-a6b0-a11ea5b4c5b7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:06.633268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e66718-7aff-4a98-a6b0-a11ea5b4c5b7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:06.637174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e66718-7aff-4a98-a6b0-a11ea5b4c5b7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=9ff56f0c05ab49039e0ff27f5ae8fbb3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:06.640285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e66718-7aff-4a98-a6b0-a11ea5b4c5b7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:06.675748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b09e8ca3-5c51-4558-8048-5b62f8c6778c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a8841fa441d413a92a15510a9d082c6', 'request_id': 'req-b09e8ca3-5c51-4558-8048-5b62f8c6778c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:06.682074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b09e8ca3-5c51-4558-8048-5b62f8c6778c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a8841fa441d413a92a15510a9d082c6', 'request_id': 'req-b09e8ca3-5c51-4558-8048-5b62f8c6778c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:06.684435 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-f8e66718-7aff-4a98-a6b0-a11ea5b4c5b7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] 5df3bb3dada94e2e8d1d425c4b107106 does not imply 9ff56f0c05ab49039e0ff27f5ae8fbb3.: keystone.exception.ImpliedRoleNotFound: 5df3bb3dada94e2e8d1d425c4b107106 does not imply 9ff56f0c05ab49039e0ff27f5ae8fbb3. May 28 21:26:06.684435 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:26:06.684435 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role May 28 21:26:06.684435 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = query.one() May 28 21:26:06.684435 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2865, in one May 28 21:26:06.684435 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._iter().one() May 28 21:26:06.684435 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1417, in one May 28 21:26:06.684435 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._only_one_row( May 28 21:26:06.684435 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row May 28 21:26:06.684435 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exc.NoResultFound( May 28 21:26:06.684435 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required May 28 21:26:06.684435 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:26:06.684435 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:26:06.684435 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:26:06.684435 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:26:06.684435 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:26:06.689101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:26:06.689101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:26:06.689101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:26:06.689101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:26:06.689101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:26:06.689101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:26:06.689101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:26:06.689101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:26:06.689101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:26:06.689101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete May 28 21:26:06.689101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) May 28 21:26:06.689101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:26:06.689101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:26:06.689101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role May 28 21:26:06.689101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) May 28 21:26:06.689101 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role May 28 21:26:06.690334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, May 28 21:26:06.690334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role May 28 21:26:06.690334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( May 28 21:26:06.690334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: 5df3bb3dada94e2e8d1d425c4b107106 does not imply 9ff56f0c05ab49039e0ff27f5ae8fbb3. May 28 21:26:06.690334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:26:06.690334 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3152/6339] 10.222.0.33 () {58 vars in 1346 bytes} [Sat May 28 21:26:06 2022] DELETE /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3 => generated 137 bytes in 72 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:26:06.690334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e66718-7aff-4a98-a6b0-a11ea5b4c5b7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:06.697015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e66718-7aff-4a98-a6b0-a11ea5b4c5b7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:06.753866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f200fc0-d57e-4dfd-b7a6-7bc514ebeef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:06.754513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f200fc0-d57e-4dfd-b7a6-7bc514ebeef8 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:06.754801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f200fc0-d57e-4dfd-b7a6-7bc514ebeef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:06.755079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f200fc0-d57e-4dfd-b7a6-7bc514ebeef8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:06.755762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f200fc0-d57e-4dfd-b7a6-7bc514ebeef8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:06.757700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f200fc0-d57e-4dfd-b7a6-7bc514ebeef8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:06.772332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b09e8ca3-5c51-4558-8048-5b62f8c6778c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a8841fa441d413a92a15510a9d082c6', 'request_id': 'req-b09e8ca3-5c51-4558-8048-5b62f8c6778c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:06.784895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b09e8ca3-5c51-4558-8048-5b62f8c6778c admin admin] The token cache is being invalidate because project 5a8841fa441d413a92a15510a9d082c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:06.785192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b09e8ca3-5c51-4558-8048-5b62f8c6778c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:06.785909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b09e8ca3-5c51-4558-8048-5b62f8c6778c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:06.787486 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3188/6340] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:26:06 2022] DELETE /identity/v3/projects/5a8841fa441d413a92a15510a9d082c6 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:06.788936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e8ca3-5c51-4558-8048-5b62f8c6778c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:06.791519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b09e8ca3-5c51-4558-8048-5b62f8c6778c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:06.849474 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3153/6341] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:06 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:06.851656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17825e-6b39-48e5-8200-25473c46307d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:06.852531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17825e-6b39-48e5-8200-25473c46307d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:06.852820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf17825e-6b39-48e5-8200-25473c46307d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:06.866988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e8ca3-5c51-4558-8048-5b62f8c6778c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:06.871648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-cf17825e-6b39-48e5-8200-25473c46307d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:06.872864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cf17825e-6b39-48e5-8200-25473c46307d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:06.923442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf697d86-550b-4544-8150-1bd091b90234 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:06.924339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf697d86-550b-4544-8150-1bd091b90234 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:06.924625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf697d86-550b-4544-8150-1bd091b90234 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:06.924911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf697d86-550b-4544-8150-1bd091b90234 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/projects/b321a6894a5d4c79952076f6c0408168/users/1f04e663b88b4151ae962359fbc2451d/roles/5df3bb3dada94e2e8d1d425c4b107106` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:06.927268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf697d86-550b-4544-8150-1bd091b90234 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=b321a6894a5d4c79952076f6c0408168, user_id=1f04e663b88b4151ae962359fbc2451d, role_id=5df3bb3dada94e2e8d1d425c4b107106)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:06.930014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf697d86-550b-4544-8150-1bd091b90234 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:06.935180 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3154/6342] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:06.935180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17825e-6b39-48e5-8200-25473c46307d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:06.942176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cf697d86-550b-4544-8150-1bd091b90234 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invalidating the token cache because role 5df3bb3dada94e2e8d1d425c4b107106 was removed from user 1f04e663b88b4151ae962359fbc2451d on project b321a6894a5d4c79952076f6c0408168. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:06.942176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cf697d86-550b-4544-8150-1bd091b90234 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:06.942366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cf17825e-6b39-48e5-8200-25473c46307d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:06.942481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cf697d86-550b-4544-8150-1bd091b90234 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:06.950248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf17825e-6b39-48e5-8200-25473c46307d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:06.954947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976b39d-d3ba-487c-a672-7f7e1a5fffaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:06.959528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976b39d-d3ba-487c-a672-7f7e1a5fffaa admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:06.959528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976b39d-d3ba-487c-a672-7f7e1a5fffaa admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:06.959528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0976b39d-d3ba-487c-a672-7f7e1a5fffaa admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:06.959528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976b39d-d3ba-487c-a672-7f7e1a5fffaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:06.959528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0976b39d-d3ba-487c-a672-7f7e1a5fffaa admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:07.008168 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3189/6343] 10.222.0.33 () {58 vars in 1466 bytes} [Sat May 28 21:26:06 2022] DELETE /identity/v3/projects/b321a6894a5d4c79952076f6c0408168/users/1f04e663b88b4151ae962359fbc2451d/roles/5df3bb3dada94e2e8d1d425c4b107106 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:07.009388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a47e4-3057-4483-9a2d-7f9094dcd7b2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:07.009679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a47e4-3057-4483-9a2d-7f9094dcd7b2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:07.010002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a47e4-3057-4483-9a2d-7f9094dcd7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:07.029867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-760a47e4-3057-4483-9a2d-7f9094dcd7b2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:07.032116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-760a47e4-3057-4483-9a2d-7f9094dcd7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:07.032245 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3155/6344] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:06 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:07.032245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976b39d-d3ba-487c-a672-7f7e1a5fffaa admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:07.033773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0976b39d-d3ba-487c-a672-7f7e1a5fffaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:07.047422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0976b39d-d3ba-487c-a672-7f7e1a5fffaa admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:07.095477 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3190/6345] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:07.100911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a47e4-3057-4483-9a2d-7f9094dcd7b2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:07.100911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-760a47e4-3057-4483-9a2d-7f9094dcd7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:07.111395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a47e4-3057-4483-9a2d-7f9094dcd7b2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:07.113728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbecc7-b094-482b-a92e-86486188c057 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:07.114595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbecc7-b094-482b-a92e-86486188c057 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:07.115002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbecc7-b094-482b-a92e-86486188c057 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:07.115384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbecc7-b094-482b-a92e-86486188c057 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:07.116150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8651d-cd82-4fdf-9df1-4381254e0e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:07.116290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbecc7-b094-482b-a92e-86486188c057 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:07.116824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8651d-cd82-4fdf-9df1-4381254e0e9b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:07.117110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8651d-cd82-4fdf-9df1-4381254e0e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:07.117396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e8651d-cd82-4fdf-9df1-4381254e0e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:07.117832 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-06cbecc7-b094-482b-a92e-86486188c057 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:07.117832 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:26:07.117832 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:26:07.118363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8651d-cd82-4fdf-9df1-4381254e0e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:07.119197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbecc7-b094-482b-a92e-86486188c057 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:07.119786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e8651d-cd82-4fdf-9df1-4381254e0e9b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:07.132970 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3191/6346] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:07 2022] GET /identity/v3/roles => generated 2324 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:07.134153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8651d-cd82-4fdf-9df1-4381254e0e9b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:07.144335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e8651d-cd82-4fdf-9df1-4381254e0e9b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:07.149737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f648859-e614-45f4-ab7c-a7374574f5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:07.150481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f648859-e614-45f4-ab7c-a7374574f5e2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:07.150792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f648859-e614-45f4-ab7c-a7374574f5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:07.151072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f648859-e614-45f4-ab7c-a7374574f5e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:07.151836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f648859-e614-45f4-ab7c-a7374574f5e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:07.152148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f648859-e614-45f4-ab7c-a7374574f5e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:07.153744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f648859-e614-45f4-ab7c-a7374574f5e2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:07.171782 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3192/6347] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:07.171782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f648859-e614-45f4-ab7c-a7374574f5e2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:07.181551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f648859-e614-45f4-ab7c-a7374574f5e2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:07.186702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea190a3-f806-4d9e-98e4-9936c6231e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:07.187408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea190a3-f806-4d9e-98e4-9936c6231e27 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:07.187691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea190a3-f806-4d9e-98e4-9936c6231e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:07.188032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea190a3-f806-4d9e-98e4-9936c6231e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:07.189049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea190a3-f806-4d9e-98e4-9936c6231e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:07.193699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea190a3-f806-4d9e-98e4-9936c6231e27 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:07.200575 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3156/6348] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:07 2022] POST /identity/v3/roles => generated 271 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:07.202453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbecc7-b094-482b-a92e-86486188c057 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:07.206654 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3193/6349] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:07 2022] GET /identity/v3/roles => generated 2324 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:07.211063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea190a3-f806-4d9e-98e4-9936c6231e27 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:07.216264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbecc7-b094-482b-a92e-86486188c057 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:07.221409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a05e9-71e1-4ab4-832f-365e936f162c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:07.222591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a05e9-71e1-4ab4-832f-365e936f162c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:07.222896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea190a3-f806-4d9e-98e4-9936c6231e27 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:07.223018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a05e9-71e1-4ab4-832f-365e936f162c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:07.223215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592a05e9-71e1-4ab4-832f-365e936f162c admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:07.223912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a05e9-71e1-4ab4-832f-365e936f162c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:07.226626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592a05e9-71e1-4ab4-832f-365e936f162c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:07.228424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acdfa0-53b4-4347-bee3-e357cfb4c2f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:07.229111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acdfa0-53b4-4347-bee3-e357cfb4c2f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:07.229400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acdfa0-53b4-4347-bee3-e357cfb4c2f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:07.229716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57acdfa0-53b4-4347-bee3-e357cfb4c2f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:07.230473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acdfa0-53b4-4347-bee3-e357cfb4c2f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Adding query filter params (domain_id=d7087720d030435a9edc125e584a7690) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:07.230781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acdfa0-53b4-4347-bee3-e357cfb4c2f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:07.231790 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-57acdfa0-53b4-4347-bee3-e357cfb4c2f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domain_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:07.231790 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:26:07.231790 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:26:07.232672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57acdfa0-53b4-4347-bee3-e357cfb4c2f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:07.241065 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3194/6350] 10.222.0.33 () {58 vars in 1206 bytes} [Sat May 28 21:26:07 2022] GET /identity/v3/roles?domain_id=d7087720d030435a9edc125e584a7690 => generated 408 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:07.243133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acdfa0-53b4-4347-bee3-e357cfb4c2f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:07.255108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57acdfa0-53b4-4347-bee3-e357cfb4c2f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:07.261964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa3d68-7b6a-4e09-814c-0493630ed54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:07.262802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa3d68-7b6a-4e09-814c-0493630ed54c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:07.263084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa3d68-7b6a-4e09-814c-0493630ed54c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:07.265330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa3d68-7b6a-4e09-814c-0493630ed54c admin admin] PATH_INFO: `/v3/projects/aaed16ff29234c81b161434120a4bf12/users/41c3af0358084a9da8f56f0b37ed7834/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:07.279465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa3d68-7b6a-4e09-814c-0493630ed54c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaed16ff29234c81b161434120a4bf12, user_id=41c3af0358084a9da8f56f0b37ed7834, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:07.282229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa3d68-7b6a-4e09-814c-0493630ed54c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:07.316809 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3157/6351] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:07 2022] POST /identity/v3/projects => generated 403 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:07.317980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a05e9-71e1-4ab4-832f-365e936f162c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:07.379983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592a05e9-71e1-4ab4-832f-365e936f162c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:07.384422 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3195/6352] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:07 2022] PUT /identity/v3/projects/aaed16ff29234c81b161434120a4bf12/users/41c3af0358084a9da8f56f0b37ed7834/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:07.385306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa3d68-7b6a-4e09-814c-0493630ed54c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:07.438346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644d747-5b09-49fa-a6c7-6ee52c3952e9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:07.439252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644d747-5b09-49fa-a6c7-6ee52c3952e9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:07.439557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644d747-5b09-49fa-a6c7-6ee52c3952e9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:07.439860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f644d747-5b09-49fa-a6c7-6ee52c3952e9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/94ceae7b671f4394956e47876edbe879` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:07.441079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644d747-5b09-49fa-a6c7-6ee52c3952e9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=94ceae7b671f4394956e47876edbe879)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:07.442728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f644d747-5b09-49fa-a6c7-6ee52c3952e9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:07.448705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa3d68-7b6a-4e09-814c-0493630ed54c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:07.453515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33227409-be84-4bd9-896b-9ee019a2dc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:07.454296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33227409-be84-4bd9-896b-9ee019a2dc07 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:07.454587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33227409-be84-4bd9-896b-9ee019a2dc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:07.454871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33227409-be84-4bd9-896b-9ee019a2dc07 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:07.456509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33227409-be84-4bd9-896b-9ee019a2dc07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:07.457564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33227409-be84-4bd9-896b-9ee019a2dc07 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:07.639797 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3196/6353] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:07 2022] POST /identity/v3/users => generated 302 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:07.639797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df854b5a-d471-41c2-84f7-fcfee28b9beb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:07.639797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df854b5a-d471-41c2-84f7-fcfee28b9beb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:07.639797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df854b5a-d471-41c2-84f7-fcfee28b9beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:07.659918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f644d747-5b09-49fa-a6c7-6ee52c3952e9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invalidating the token cache because role 94ceae7b671f4394956e47876edbe879 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:07.660381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f644d747-5b09-49fa-a6c7-6ee52c3952e9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:07.660960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f644d747-5b09-49fa-a6c7-6ee52c3952e9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:07.663688 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3158/6354] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:07 2022] DELETE /identity/v3/roles/94ceae7b671f4394956e47876edbe879 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:07.665005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644d747-5b09-49fa-a6c7-6ee52c3952e9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:07.666368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f644d747-5b09-49fa-a6c7-6ee52c3952e9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:07.674070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-df854b5a-d471-41c2-84f7-fcfee28b9beb None None] MFA Rules not processed for user `41c3af0358084a9da8f56f0b37ed7834`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:07.675297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-df854b5a-d471-41c2-84f7-fcfee28b9beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:07.718478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:07.739922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f644d747-5b09-49fa-a6c7-6ee52c3952e9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:07.744978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680caed-96cf-404f-8a79-389c061b7a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:07.745650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680caed-96cf-404f-8a79-389c061b7a9a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:07.745970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680caed-96cf-404f-8a79-389c061b7a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:07.746247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d680caed-96cf-404f-8a79-389c061b7a9a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:07.747006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680caed-96cf-404f-8a79-389c061b7a9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:07.748649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d680caed-96cf-404f-8a79-389c061b7a9a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:07.754128 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3197/6355] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:07.755125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df854b5a-d471-41c2-84f7-fcfee28b9beb None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:07.756534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-df854b5a-d471-41c2-84f7-fcfee28b9beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:07.760692 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3159/6356] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:07 2022] GET /identity/v3/roles => generated 2324 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:07.762060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680caed-96cf-404f-8a79-389c061b7a9a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:07.768166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d680caed-96cf-404f-8a79-389c061b7a9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:07.774685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:07.783382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d680caed-96cf-404f-8a79-389c061b7a9a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:07.789269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a67548-95ee-4265-adde-aaf529b990ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:07.790165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a67548-95ee-4265-adde-aaf529b990ce admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:07.790303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a67548-95ee-4265-adde-aaf529b990ce admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:07.790525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a67548-95ee-4265-adde-aaf529b990ce admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:07.791219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a67548-95ee-4265-adde-aaf529b990ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:07.793249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a67548-95ee-4265-adde-aaf529b990ce admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:07.833248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df854b5a-d471-41c2-84f7-fcfee28b9beb None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:07.838712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07215d49-d5f5-4bca-93ce-0e8b49b7970e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:07.839384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07215d49-d5f5-4bca-93ce-0e8b49b7970e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:07.839673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07215d49-d5f5-4bca-93ce-0e8b49b7970e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:07.839951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07215d49-d5f5-4bca-93ce-0e8b49b7970e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:07.840696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07215d49-d5f5-4bca-93ce-0e8b49b7970e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Adding query filter params (domain_id=d7087720d030435a9edc125e584a7690) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:07.841044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07215d49-d5f5-4bca-93ce-0e8b49b7970e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:list_domain_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:07.842651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07215d49-d5f5-4bca-93ce-0e8b49b7970e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:07.852041 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3198/6357] 10.222.0.33 () {58 vars in 1206 bytes} [Sat May 28 21:26:07 2022] GET /identity/v3/roles?domain_id=d7087720d030435a9edc125e584a7690 => generated 148 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:07.852856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07215d49-d5f5-4bca-93ce-0e8b49b7970e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:07.863360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07215d49-d5f5-4bca-93ce-0e8b49b7970e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:07.871249 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3160/6358] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:07 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:07.871359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c139fc-a70c-4ea8-a604-46cd82025f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:07.871479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a67548-95ee-4265-adde-aaf529b990ce admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:07.871571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c139fc-a70c-4ea8-a604-46cd82025f01 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:07.871571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c139fc-a70c-4ea8-a604-46cd82025f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:07.871571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c139fc-a70c-4ea8-a604-46cd82025f01 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:07.871571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c139fc-a70c-4ea8-a604-46cd82025f01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:07.873085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c139fc-a70c-4ea8-a604-46cd82025f01 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:07.885721 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3199/6359] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:07 2022] GET /identity/v3/roles => generated 2324 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:07.889615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c139fc-a70c-4ea8-a604-46cd82025f01 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:07.900668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c139fc-a70c-4ea8-a604-46cd82025f01 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:07.945628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a67548-95ee-4265-adde-aaf529b990ce admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:07.952754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d2b04-9913-4e00-b5f1-83c0b8df3450 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:07.954476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d2b04-9913-4e00-b5f1-83c0b8df3450 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:07.954763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d2b04-9913-4e00-b5f1-83c0b8df3450 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:07.955355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d2b04-9913-4e00-b5f1-83c0b8df3450 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/94ceae7b671f4394956e47876edbe879` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:07.967437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3d2b04-9913-4e00-b5f1-83c0b8df3450 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:26:07.978007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-1e3d2b04-9913-4e00-b5f1-83c0b8df3450 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Could not find role: 94ceae7b671f4394956e47876edbe879.: keystone.exception.RoleNotFound: Could not find role: 94ceae7b671f4394956e47876edbe879. May 28 21:26:07.978007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:26:07.978007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:26:07.978007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:26:07.978007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:26:07.978007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:26:07.978007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:26:07.978007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:26:07.978007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:26:07.978007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:26:07.978007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:26:07.978007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:26:07.978007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 165, in delete May 28 21:26:07.978007 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_role') May 28 21:26:07.979135 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:26:07.979135 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:26:07.979135 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:26:07.979135 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:26:07.979135 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:26:07.979135 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:26:07.979135 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:26:07.979135 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:26:07.979135 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:26:07.979135 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:26:07.979135 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:26:07.979135 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 21:26:07.979135 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:26:07.979135 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 21:26:07.979135 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:26:07.979135 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:26:07.980190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:26:07.980190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 21:26:07.980190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:26:07.980190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 21:26:07.980190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 28 21:26:07.980190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 28 21:26:07.980190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 28 21:26:07.980190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 28 21:26:07.980190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 28 21:26:07.980190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 28 21:26:07.980190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: 94ceae7b671f4394956e47876edbe879. May 28 21:26:07.980190 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:26:07.980190 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3161/6360] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:07 2022] DELETE /identity/v3/roles/94ceae7b671f4394956e47876edbe879 => generated 110 bytes in 111 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:26:07.981114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d2b04-9913-4e00-b5f1-83c0b8df3450 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:07.982772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd9c16-50d6-47e7-a137-e305a7bf30d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:07.983883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd9c16-50d6-47e7-a137-e305a7bf30d9 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:07.984317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd9c16-50d6-47e7-a137-e305a7bf30d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:07.984614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defd9c16-50d6-47e7-a137-e305a7bf30d9 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:07.985555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd9c16-50d6-47e7-a137-e305a7bf30d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:07.988450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defd9c16-50d6-47e7-a137-e305a7bf30d9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:07.998387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d2b04-9913-4e00-b5f1-83c0b8df3450 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:08.004846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5a821-2621-413f-8dbe-bd283fa95e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:08.010048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5a821-2621-413f-8dbe-bd283fa95e36 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:08.010048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5a821-2621-413f-8dbe-bd283fa95e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:08.010048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5a821-2621-413f-8dbe-bd283fa95e36 admin admin] PATH_INFO: `/v3/projects/1034dc0c96de4f71873b657b88c20c79/users/572c2ace0562496ebd07f81d37a1522c/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:08.021927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5a821-2621-413f-8dbe-bd283fa95e36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1034dc0c96de4f71873b657b88c20c79, user_id=572c2ace0562496ebd07f81d37a1522c, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:08.025187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5a821-2621-413f-8dbe-bd283fa95e36 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:08.101434 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3200/6361] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:07 2022] POST /identity/v3/users => generated 292 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:08.101434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd9c16-50d6-47e7-a137-e305a7bf30d9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:08.110370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defd9c16-50d6-47e7-a137-e305a7bf30d9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:08.119109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e12cc1-ddfe-4d57-932a-f3b59fe04a95 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:08.119109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e12cc1-ddfe-4d57-932a-f3b59fe04a95 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:08.119109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e12cc1-ddfe-4d57-932a-f3b59fe04a95 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:08.119404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e12cc1-ddfe-4d57-932a-f3b59fe04a95 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/domains/d7087720d030435a9edc125e584a7690/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:08.131779 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3162/6362] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:07 2022] PUT /identity/v3/projects/1034dc0c96de4f71873b657b88c20c79/users/572c2ace0562496ebd07f81d37a1522c/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:08.131779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5a821-2621-413f-8dbe-bd283fa95e36 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:08.134465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e12cc1-ddfe-4d57-932a-f3b59fe04a95 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=d7087720d030435a9edc125e584a7690, group_id=83dc424272e5493a9a3b46bf902dde1c, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:08.137693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e12cc1-ddfe-4d57-932a-f3b59fe04a95 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:08.191346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5a821-2621-413f-8dbe-bd283fa95e36 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:08.196679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e38de8-a8e4-49fc-9467-47111a6e7fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:08.197344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e38de8-a8e4-49fc-9467-47111a6e7fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:08.197633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e38de8-a8e4-49fc-9467-47111a6e7fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:08.197947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e38de8-a8e4-49fc-9467-47111a6e7fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:08.198684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e38de8-a8e4-49fc-9467-47111a6e7fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:08.200303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e38de8-a8e4-49fc-9467-47111a6e7fa2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:08.212562 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3163/6363] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:08 2022] GET /identity/v3/roles => generated 2324 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:08.213964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e535f75-66bb-47e9-b104-6438ee9045a8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:08.214258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e535f75-66bb-47e9-b104-6438ee9045a8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:08.214541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e535f75-66bb-47e9-b104-6438ee9045a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:08.238246 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3201/6364] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:08 2022] PUT /identity/v3/domains/d7087720d030435a9edc125e584a7690/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:08.238246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e12cc1-ddfe-4d57-932a-f3b59fe04a95 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:08.249336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3e535f75-66bb-47e9-b104-6438ee9045a8 None None] MFA Rules not processed for user `572c2ace0562496ebd07f81d37a1522c`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:08.250671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3e535f75-66bb-47e9-b104-6438ee9045a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:08.259903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:08.313303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e12cc1-ddfe-4d57-932a-f3b59fe04a95 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:08.318074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730cb35-298b-4ee2-a6ff-bea5e8ebc70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:08.318976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730cb35-298b-4ee2-a6ff-bea5e8ebc70f admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:08.319271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730cb35-298b-4ee2-a6ff-bea5e8ebc70f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:08.319584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730cb35-298b-4ee2-a6ff-bea5e8ebc70f admin admin] PATH_INFO: `/v3/projects/f8bcdab945614ab5b0a5658102c17dc3/users/7ad3cddbfb354764a781ea4eb37b518f/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:08.325394 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3164/6365] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:08 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:08.325394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e535f75-66bb-47e9-b104-6438ee9045a8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:08.336774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730cb35-298b-4ee2-a6ff-bea5e8ebc70f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8bcdab945614ab5b0a5658102c17dc3, user_id=7ad3cddbfb354764a781ea4eb37b518f, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:08.339594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730cb35-298b-4ee2-a6ff-bea5e8ebc70f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:08.386485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e535f75-66bb-47e9-b104-6438ee9045a8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:08.391209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2ebe6-22aa-405c-a2ac-62c12513ae44 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:08.392062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2ebe6-22aa-405c-a2ac-62c12513ae44 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:08.392350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2ebe6-22aa-405c-a2ac-62c12513ae44 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:08.392631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb2ebe6-22aa-405c-a2ac-62c12513ae44 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/domains/d7087720d030435a9edc125e584a7690/groups/83dc424272e5493a9a3b46bf902dde1c/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:08.394420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2ebe6-22aa-405c-a2ac-62c12513ae44 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=d7087720d030435a9edc125e584a7690, group_id=83dc424272e5493a9a3b46bf902dde1c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:08.397097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb2ebe6-22aa-405c-a2ac-62c12513ae44 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:08.409060 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3165/6366] 10.222.0.33 () {58 vars in 1364 bytes} [Sat May 28 21:26:08 2022] GET /identity/v3/domains/d7087720d030435a9edc125e584a7690/groups/83dc424272e5493a9a3b46bf902dde1c/roles => generated 410 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:08.409821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2ebe6-22aa-405c-a2ac-62c12513ae44 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:08.419857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb2ebe6-22aa-405c-a2ac-62c12513ae44 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:08.424469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590199f-568a-44c2-addc-afdbc64a7493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:08.425074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590199f-568a-44c2-addc-afdbc64a7493 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:08.431869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590199f-568a-44c2-addc-afdbc64a7493 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:08.431869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c590199f-568a-44c2-addc-afdbc64a7493 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:08.431869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590199f-568a-44c2-addc-afdbc64a7493 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:08.431869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c590199f-568a-44c2-addc-afdbc64a7493 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:08.462961 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3202/6367] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:08 2022] PUT /identity/v3/projects/f8bcdab945614ab5b0a5658102c17dc3/users/7ad3cddbfb354764a781ea4eb37b518f/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:08.462961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730cb35-298b-4ee2-a6ff-bea5e8ebc70f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:08.520866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730cb35-298b-4ee2-a6ff-bea5e8ebc70f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:08.526159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f34d10-76c8-4943-8948-380a14c02031 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:08.527011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f34d10-76c8-4943-8948-380a14c02031 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `HEAD` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:08.527330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f34d10-76c8-4943-8948-380a14c02031 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:08.527603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f34d10-76c8-4943-8948-380a14c02031 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/domains/d7087720d030435a9edc125e584a7690/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:08.530054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f34d10-76c8-4943-8948-380a14c02031 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=d7087720d030435a9edc125e584a7690, group_id=83dc424272e5493a9a3b46bf902dde1c, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:08.532915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f34d10-76c8-4943-8948-380a14c02031 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:08.539061 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3203/6368] 10.222.0.33 () {58 vars in 1464 bytes} [Sat May 28 21:26:08 2022] HEAD /identity/v3/domains/d7087720d030435a9edc125e584a7690/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:08.539906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f34d10-76c8-4943-8948-380a14c02031 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:08.558065 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3166/6369] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:08 2022] POST /identity/v3/projects => generated 403 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:08.558997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590199f-568a-44c2-addc-afdbc64a7493 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:08.605320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f34d10-76c8-4943-8948-380a14c02031 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:08.635660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c590199f-568a-44c2-addc-afdbc64a7493 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:08.639125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc06c38-3954-4e39-8349-54f9e9047179 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:08.639719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc06c38-3954-4e39-8349-54f9e9047179 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:08.641041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc06c38-3954-4e39-8349-54f9e9047179 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:08.641437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc06c38-3954-4e39-8349-54f9e9047179 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/domains/d7087720d030435a9edc125e584a7690/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:08.643979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc06c38-3954-4e39-8349-54f9e9047179 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=d7087720d030435a9edc125e584a7690, group_id=83dc424272e5493a9a3b46bf902dde1c, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:08.646841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc06c38-3954-4e39-8349-54f9e9047179 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:08.653531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bbc06c38-3954-4e39-8349-54f9e9047179 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invalidating the token cache because role 0e46a1ddecc44597be78d09c35eb3779 was removed from group 83dc424272e5493a9a3b46bf902dde1c on domain d7087720d030435a9edc125e584a7690. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:08.658223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bbc06c38-3954-4e39-8349-54f9e9047179 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:08.658223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bbc06c38-3954-4e39-8349-54f9e9047179 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:08.668327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bda41-e847-4923-935c-16bb8ae7864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:08.669038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bda41-e847-4923-935c-16bb8ae7864c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:08.669325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bda41-e847-4923-935c-16bb8ae7864c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:08.669654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bda41-e847-4923-935c-16bb8ae7864c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:08.670428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6bda41-e847-4923-935c-16bb8ae7864c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:08.672062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6bda41-e847-4923-935c-16bb8ae7864c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:08.683831 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3204/6370] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:08 2022] GET /identity/v3/roles => generated 2324 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:08.687016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bda41-e847-4923-935c-16bb8ae7864c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:08.687016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1b6bda41-e847-4923-935c-16bb8ae7864c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:08.698908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bda41-e847-4923-935c-16bb8ae7864c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:08.703557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974cd8be-8454-4a66-9e6b-6c300d26da99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:08.704212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974cd8be-8454-4a66-9e6b-6c300d26da99 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:08.704483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974cd8be-8454-4a66-9e6b-6c300d26da99 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:08.704761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974cd8be-8454-4a66-9e6b-6c300d26da99 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:08.705441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974cd8be-8454-4a66-9e6b-6c300d26da99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:08.707525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974cd8be-8454-4a66-9e6b-6c300d26da99 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:08.736563 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3167/6371] 10.222.0.33 () {58 vars in 1466 bytes} [Sat May 28 21:26:08 2022] DELETE /identity/v3/domains/d7087720d030435a9edc125e584a7690/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:08.736563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc06c38-3954-4e39-8349-54f9e9047179 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:08.737199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bbc06c38-3954-4e39-8349-54f9e9047179 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:08.794365 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3205/6372] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:08 2022] POST /identity/v3/users => generated 308 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:08.794365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974cd8be-8454-4a66-9e6b-6c300d26da99 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:08.794365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-974cd8be-8454-4a66-9e6b-6c300d26da99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:08.801940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc06c38-3954-4e39-8349-54f9e9047179 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:08.807347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2efaa7-9814-4f46-894c-aa3bc753a87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:08.808239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2efaa7-9814-4f46-894c-aa3bc753a87d admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:08.808527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2efaa7-9814-4f46-894c-aa3bc753a87d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:08.808641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974cd8be-8454-4a66-9e6b-6c300d26da99 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:08.808809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2efaa7-9814-4f46-894c-aa3bc753a87d admin admin] PATH_INFO: `/v3/projects/f8bcdab945614ab5b0a5658102c17dc3/users/7ad3cddbfb354764a781ea4eb37b518f/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:08.811055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2efaa7-9814-4f46-894c-aa3bc753a87d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8bcdab945614ab5b0a5658102c17dc3, user_id=7ad3cddbfb354764a781ea4eb37b518f, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:08.814144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2efaa7-9814-4f46-894c-aa3bc753a87d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:08.825241 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3168/6373] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:08 2022] PUT /identity/v3/projects/f8bcdab945614ab5b0a5658102c17dc3/users/7ad3cddbfb354764a781ea4eb37b518f/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:08.864710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2efaa7-9814-4f46-894c-aa3bc753a87d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:08.894064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:UPDATE=1,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:08.899146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2efaa7-9814-4f46-894c-aa3bc753a87d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:08.906866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055059ae-c317-4c9e-8d65-2a4813c9495b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:08.907172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70518af-6ca7-4707-a2f0-86e3a5cff839 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:08.907554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055059ae-c317-4c9e-8d65-2a4813c9495b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:08.907834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055059ae-c317-4c9e-8d65-2a4813c9495b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:08.908092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70518af-6ca7-4707-a2f0-86e3a5cff839 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:08.908215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055059ae-c317-4c9e-8d65-2a4813c9495b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:08.908392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70518af-6ca7-4707-a2f0-86e3a5cff839 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:08.908682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70518af-6ca7-4707-a2f0-86e3a5cff839 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/projects/b321a6894a5d4c79952076f6c0408168/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:08.908813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055059ae-c317-4c9e-8d65-2a4813c9495b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:08.910611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055059ae-c317-4c9e-8d65-2a4813c9495b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:08.911355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70518af-6ca7-4707-a2f0-86e3a5cff839 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_grant(project_id=b321a6894a5d4c79952076f6c0408168, group_id=83dc424272e5493a9a3b46bf902dde1c, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:08.915041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70518af-6ca7-4707-a2f0-86e3a5cff839 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:08.926681 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3169/6374] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:08 2022] GET /identity/v3/roles => generated 2324 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:08.926681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedaa2b-8ae8-4b42-8f10-0495b6443926 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:08.926681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedaa2b-8ae8-4b42-8f10-0495b6443926 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:08.926681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedaa2b-8ae8-4b42-8f10-0495b6443926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:08.960007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bfedaa2b-8ae8-4b42-8f10-0495b6443926 None None] MFA Rules not processed for user `7ad3cddbfb354764a781ea4eb37b518f`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:08.961288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bfedaa2b-8ae8-4b42-8f10-0495b6443926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:08.996169 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3206/6375] 10.222.0.33 () {60 vars in 1485 bytes} [Sat May 28 21:26:08 2022] PUT /identity/v3/projects/b321a6894a5d4c79952076f6c0408168/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:08.996169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70518af-6ca7-4707-a2f0-86e3a5cff839 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:09.051031 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3170/6376] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:08 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:09.051959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedaa2b-8ae8-4b42-8f10-0495b6443926 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:09.075387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70518af-6ca7-4707-a2f0-86e3a5cff839 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:09.080272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fc627-af46-43f6-a5af-8061e9d1f8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:09.082932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fc627-af46-43f6-a5af-8061e9d1f8e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:09.082932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fc627-af46-43f6-a5af-8061e9d1f8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:09.082932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238fc627-af46-43f6-a5af-8061e9d1f8e7 admin admin] PATH_INFO: `/v3/projects/61056f8aa6a94cd18edb1dea01ff284b/users/db835547a3b743869fb3675e3bf70b11/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:09.099784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fc627-af46-43f6-a5af-8061e9d1f8e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61056f8aa6a94cd18edb1dea01ff284b, user_id=db835547a3b743869fb3675e3bf70b11, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:09.103106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238fc627-af46-43f6-a5af-8061e9d1f8e7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:09.121160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedaa2b-8ae8-4b42-8f10-0495b6443926 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:09.156435 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3207/6377] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:08 2022] PUT /identity/v3/projects/61056f8aa6a94cd18edb1dea01ff284b/users/db835547a3b743869fb3675e3bf70b11/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:09.157453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fc627-af46-43f6-a5af-8061e9d1f8e7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:09.160280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-238fc627-af46-43f6-a5af-8061e9d1f8e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:09.178172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01585a-2146-47d8-9718-785e6ada6f9e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:09.179095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01585a-2146-47d8-9718-785e6ada6f9e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:09.179383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01585a-2146-47d8-9718-785e6ada6f9e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:09.179661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d01585a-2146-47d8-9718-785e6ada6f9e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/projects/b321a6894a5d4c79952076f6c0408168/groups/83dc424272e5493a9a3b46bf902dde1c/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:09.181539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01585a-2146-47d8-9718-785e6ada6f9e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:list_grants(project_id=b321a6894a5d4c79952076f6c0408168, group_id=83dc424272e5493a9a3b46bf902dde1c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:09.184211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d01585a-2146-47d8-9718-785e6ada6f9e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:09.196044 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3171/6378] 10.222.0.33 () {58 vars in 1367 bytes} [Sat May 28 21:26:09 2022] GET /identity/v3/projects/b321a6894a5d4c79952076f6c0408168/groups/83dc424272e5493a9a3b46bf902dde1c/roles => generated 411 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:09.196044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01585a-2146-47d8-9718-785e6ada6f9e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:09.209346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d01585a-2146-47d8-9718-785e6ada6f9e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:09.220687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238fc627-af46-43f6-a5af-8061e9d1f8e7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:09.227453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef5966-6889-4c8e-9deb-17bc3699bd9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:09.227453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef5966-6889-4c8e-9deb-17bc3699bd9a service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:09.227453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef5966-6889-4c8e-9deb-17bc3699bd9a service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:09.227453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef5966-6889-4c8e-9deb-17bc3699bd9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:09.266366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a70696-ad08-4751-a708-2a6d0b6a8499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:09.266973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a70696-ad08-4751-a708-2a6d0b6a8499 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:09.267262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a70696-ad08-4751-a708-2a6d0b6a8499 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:09.267542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a70696-ad08-4751-a708-2a6d0b6a8499 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:09.268305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a70696-ad08-4751-a708-2a6d0b6a8499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:09.269938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a70696-ad08-4751-a708-2a6d0b6a8499 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:09.284767 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3172/6379] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:09 2022] GET /identity/v3/roles => generated 2324 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:09.285615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a70696-ad08-4751-a708-2a6d0b6a8499 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:09.287992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef5966-6889-4c8e-9deb-17bc3699bd9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:09.290081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef5966-6889-4c8e-9deb-17bc3699bd9a service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:09.299796 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3208/6380] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:09 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:09.300416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef5966-6889-4c8e-9deb-17bc3699bd9a service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:09.312614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef5966-6889-4c8e-9deb-17bc3699bd9a service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:09.317389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383147d-cc22-4af2-9549-6ec6d42e1d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:09.318317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383147d-cc22-4af2-9549-6ec6d42e1d73 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:09.318602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383147d-cc22-4af2-9549-6ec6d42e1d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:09.318878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383147d-cc22-4af2-9549-6ec6d42e1d73 admin admin] PATH_INFO: `/v3/projects/61056f8aa6a94cd18edb1dea01ff284b/users/db835547a3b743869fb3675e3bf70b11/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:09.320928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383147d-cc22-4af2-9549-6ec6d42e1d73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61056f8aa6a94cd18edb1dea01ff284b, user_id=db835547a3b743869fb3675e3bf70b11, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:09.323638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383147d-cc22-4af2-9549-6ec6d42e1d73 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:09.336414 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3209/6381] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:09 2022] PUT /identity/v3/projects/61056f8aa6a94cd18edb1dea01ff284b/users/db835547a3b743869fb3675e3bf70b11/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:09.347895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876495ee-e7f7-4a4d-80e9-9a2b45e85feb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:09.348224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876495ee-e7f7-4a4d-80e9-9a2b45e85feb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:09.348464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876495ee-e7f7-4a4d-80e9-9a2b45e85feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:09.350568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a70696-ad08-4751-a708-2a6d0b6a8499 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:09.383395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-876495ee-e7f7-4a4d-80e9-9a2b45e85feb None None] MFA Rules not processed for user `db835547a3b743869fb3675e3bf70b11`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:09.383395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-876495ee-e7f7-4a4d-80e9-9a2b45e85feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:09.410401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c8eaa-4333-4164-a9aa-563ce73917be tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:09.411852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c8eaa-4333-4164-a9aa-563ce73917be tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:09.411852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c8eaa-4333-4164-a9aa-563ce73917be tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:09.412069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c8eaa-4333-4164-a9aa-563ce73917be tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/groups/83dc424272e5493a9a3b46bf902dde1c/users/1f04e663b88b4151ae962359fbc2451d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:09.414181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c8eaa-4333-4164-a9aa-563ce73917be tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=83dc424272e5493a9a3b46bf902dde1c, user_id=1f04e663b88b4151ae962359fbc2451d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:09.416281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c8eaa-4333-4164-a9aa-563ce73917be tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:09.436734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:09.453879 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3210/6382] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:09 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:26:09.458794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876495ee-e7f7-4a4d-80e9-9a2b45e85feb None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:09.468186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876495ee-e7f7-4a4d-80e9-9a2b45e85feb None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:09.473136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cde2ff-b43b-404f-975e-8f2f86f27ad1 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db835547a3b743869fb3675e3bf70b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61056f8aa6a94cd18edb1dea01ff284b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:09.473808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cde2ff-b43b-404f-975e-8f2f86f27ad1 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:09.474120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cde2ff-b43b-404f-975e-8f2f86f27ad1 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:09.474429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cde2ff-b43b-404f-975e-8f2f86f27ad1 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:09.475217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cde2ff-b43b-404f-975e-8f2f86f27ad1 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:09.476863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cde2ff-b43b-404f-975e-8f2f86f27ad1 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:09.511263 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3173/6383] 10.222.0.33 () {60 vars in 1359 bytes} [Sat May 28 21:26:09 2022] PUT /identity/v3/groups/83dc424272e5493a9a3b46bf902dde1c/users/1f04e663b88b4151ae962359fbc2451d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:09.515859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b28c8-ae4b-4bfe-a4c6-aeca5fffd11a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:09.516191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b28c8-ae4b-4bfe-a4c6-aeca5fffd11a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:09.516740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b28c8-ae4b-4bfe-a4c6-aeca5fffd11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:09.553902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4e2b28c8-ae4b-4bfe-a4c6-aeca5fffd11a None None] MFA Rules not processed for user `1f04e663b88b4151ae962359fbc2451d`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:09.555618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4e2b28c8-ae4b-4bfe-a4c6-aeca5fffd11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:09.576531 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3211/6384] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:26:09 2022] POST /identity/v3/roles => generated 239 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:09.577866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cde2ff-b43b-404f-975e-8f2f86f27ad1 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:09.579566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-82cde2ff-b43b-404f-975e-8f2f86f27ad1 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:09.644665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cde2ff-b43b-404f-975e-8f2f86f27ad1 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:09.658540 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3174/6385] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:09 2022] POST /identity/v3/auth/tokens => generated 3255 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:26:09.659438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b28c8-ae4b-4bfe-a4c6-aeca5fffd11a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:09.693060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deeee7d-4236-4acb-9b12-6ddc071c3fa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:09.693992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deeee7d-4236-4acb-9b12-6ddc071c3fa5 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:09.694351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deeee7d-4236-4acb-9b12-6ddc071c3fa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:09.695187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deeee7d-4236-4acb-9b12-6ddc071c3fa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:09.697195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9deeee7d-4236-4acb-9b12-6ddc071c3fa5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:09.722158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b28c8-ae4b-4bfe-a4c6-aeca5fffd11a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:09.726999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132fa95-2fc7-4f69-b829-dd518e046501 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db835547a3b743869fb3675e3bf70b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61056f8aa6a94cd18edb1dea01ff284b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:09.728020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132fa95-2fc7-4f69-b829-dd518e046501 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:09.728407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132fa95-2fc7-4f69-b829-dd518e046501 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:09.728792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132fa95-2fc7-4f69-b829-dd518e046501 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] PATH_INFO: `/v3/projects/1034dc0c96de4f71873b657b88c20c79/users/572c2ace0562496ebd07f81d37a1522c/roles/a4e8f22c4bcc41439f6e77c489e07864` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:09.731230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132fa95-2fc7-4f69-b829-dd518e046501 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] RBAC: Authorizing `identity:create_grant(project_id=1034dc0c96de4f71873b657b88c20c79, user_id=572c2ace0562496ebd07f81d37a1522c, role_id=a4e8f22c4bcc41439f6e77c489e07864)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:09.734175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132fa95-2fc7-4f69-b829-dd518e046501 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:09.763554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deeee7d-4236-4acb-9b12-6ddc071c3fa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:09.765006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deeee7d-4236-4acb-9b12-6ddc071c3fa5 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:09.774057 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3212/6386] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:09.774858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deeee7d-4236-4acb-9b12-6ddc071c3fa5 service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:09.836550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deeee7d-4236-4acb-9b12-6ddc071c3fa5 service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:09.841355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12225035-c0ce-48cf-bcbe-19d4021e9964 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:09.842290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12225035-c0ce-48cf-bcbe-19d4021e9964 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `HEAD` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:09.842576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12225035-c0ce-48cf-bcbe-19d4021e9964 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:09.842859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12225035-c0ce-48cf-bcbe-19d4021e9964 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/projects/b321a6894a5d4c79952076f6c0408168/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:09.845095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12225035-c0ce-48cf-bcbe-19d4021e9964 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:check_grant(project_id=b321a6894a5d4c79952076f6c0408168, group_id=83dc424272e5493a9a3b46bf902dde1c, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:09.848082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12225035-c0ce-48cf-bcbe-19d4021e9964 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:09.848419 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3175/6387] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:09 2022] PUT /identity/v3/projects/1034dc0c96de4f71873b657b88c20c79/users/572c2ace0562496ebd07f81d37a1522c/roles/a4e8f22c4bcc41439f6e77c489e07864 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:09.852690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132fa95-2fc7-4f69-b829-dd518e046501 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:09.854297 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3213/6388] 10.222.0.33 () {58 vars in 1467 bytes} [Sat May 28 21:26:09 2022] HEAD /identity/v3/projects/b321a6894a5d4c79952076f6c0408168/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:09.854441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f132fa95-2fc7-4f69-b829-dd518e046501 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:09.858132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12225035-c0ce-48cf-bcbe-19d4021e9964 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:09.868203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12225035-c0ce-48cf-bcbe-19d4021e9964 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:09.918853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132fa95-2fc7-4f69-b829-dd518e046501 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:09.923682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d334d19-61ac-4e17-8fbb-5365c3dfc344 tempest-ApplicationCredentialsV3AdminTest-1718076502 tempest-ApplicationCredentialsV3AdminTest-1718076502-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '572c2ace0562496ebd07f81d37a1522c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1034dc0c96de4f71873b657b88c20c79', 'project_domain_id': 'default', 'roles': ['member', 'tempest-test_role-914577071', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:09.926027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d334d19-61ac-4e17-8fbb-5365c3dfc344 tempest-ApplicationCredentialsV3AdminTest-1718076502 tempest-ApplicationCredentialsV3AdminTest-1718076502-project] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:09.926027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d334d19-61ac-4e17-8fbb-5365c3dfc344 tempest-ApplicationCredentialsV3AdminTest-1718076502 tempest-ApplicationCredentialsV3AdminTest-1718076502-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:09.926287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed57ebe-d082-404f-8da9-b97aaca7100e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:09.926425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d334d19-61ac-4e17-8fbb-5365c3dfc344 tempest-ApplicationCredentialsV3AdminTest-1718076502 tempest-ApplicationCredentialsV3AdminTest-1718076502-project] PATH_INFO: `/v3/users/572c2ace0562496ebd07f81d37a1522c/application_credentials` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:09.926538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed57ebe-d082-404f-8da9-b97aaca7100e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:09.926639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d334d19-61ac-4e17-8fbb-5365c3dfc344 tempest-ApplicationCredentialsV3AdminTest-1718076502 tempest-ApplicationCredentialsV3AdminTest-1718076502-project] RBAC: Authorizing `identity:create_application_credential(user_id=572c2ace0562496ebd07f81d37a1522c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:09.926729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed57ebe-d082-404f-8da9-b97aaca7100e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:09.926729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed57ebe-d082-404f-8da9-b97aaca7100e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/projects/b321a6894a5d4c79952076f6c0408168/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:09.927047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d334d19-61ac-4e17-8fbb-5365c3dfc344 tempest-ApplicationCredentialsV3AdminTest-1718076502 tempest-ApplicationCredentialsV3AdminTest-1718076502-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:09.928694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed57ebe-d082-404f-8da9-b97aaca7100e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=b321a6894a5d4c79952076f6c0408168, group_id=83dc424272e5493a9a3b46bf902dde1c, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:09.931961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed57ebe-d082-404f-8da9-b97aaca7100e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:09.937370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fed57ebe-d082-404f-8da9-b97aaca7100e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invalidating the token cache because role 0e46a1ddecc44597be78d09c35eb3779 was removed from group 83dc424272e5493a9a3b46bf902dde1c on project b321a6894a5d4c79952076f6c0408168. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:09.937774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fed57ebe-d082-404f-8da9-b97aaca7100e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:09.938532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fed57ebe-d082-404f-8da9-b97aaca7100e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:10.048172 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3214/6389] 10.222.0.33 () {58 vars in 1469 bytes} [Sat May 28 21:26:09 2022] DELETE /identity/v3/projects/b321a6894a5d4c79952076f6c0408168/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:10.052164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed57ebe-d082-404f-8da9-b97aaca7100e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:10.053559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fed57ebe-d082-404f-8da9-b97aaca7100e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:10.077733 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3176/6390] 10.222.0.33 () {60 vars in 1314 bytes} [Sat May 28 21:26:09 2022] POST /identity/v3/users/572c2ace0562496ebd07f81d37a1522c/application_credentials => generated 658 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:10.079408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e92453-370b-45f0-8fb7-a6e32eef68c5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:10.080170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e92453-370b-45f0-8fb7-a6e32eef68c5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:10.080467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e92453-370b-45f0-8fb7-a6e32eef68c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:10.103437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b7e92453-370b-45f0-8fb7-a6e32eef68c5 None None] MFA Rules not processed for user `572c2ace0562496ebd07f81d37a1522c`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:10.104879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b7e92453-370b-45f0-8fb7-a6e32eef68c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:10.142144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed57ebe-d082-404f-8da9-b97aaca7100e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:10.158386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ecef1-423d-4ada-88f1-8ede1f107922 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:10.159497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ecef1-423d-4ada-88f1-8ede1f107922 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:10.159791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ecef1-423d-4ada-88f1-8ede1f107922 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:10.160067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ecef1-423d-4ada-88f1-8ede1f107922 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/groups/83dc424272e5493a9a3b46bf902dde1c/users/1f04e663b88b4151ae962359fbc2451d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:10.162640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ecef1-423d-4ada-88f1-8ede1f107922 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=83dc424272e5493a9a3b46bf902dde1c, user_id=1f04e663b88b4151ae962359fbc2451d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:10.166058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ecef1-423d-4ada-88f1-8ede1f107922 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:10.236865 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3177/6391] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:10 2022] POST /identity/v3/auth/tokens => generated 3385 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) May 28 21:26:10.241210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e92453-370b-45f0-8fb7-a6e32eef68c5 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:10.243279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b7e92453-370b-45f0-8fb7-a6e32eef68c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:10.247551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a16ecef1-423d-4ada-88f1-8ede1f107922 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '1f04e663b88b4151ae962359fbc2451d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:10.318821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e92453-370b-45f0-8fb7-a6e32eef68c5 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:10.322673 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3215/6392] 10.222.0.33 () {58 vars in 1343 bytes} [Sat May 28 21:26:10 2022] DELETE /identity/v3/groups/83dc424272e5493a9a3b46bf902dde1c/users/1f04e663b88b4151ae962359fbc2451d => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:10.327630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ecef1-423d-4ada-88f1-8ede1f107922 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:10.383779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ade6de-bfe1-4d9e-9d95-2df6788fa387 tempest-ApplicationCredentialsV3AdminTest-1718076502 tempest-ApplicationCredentialsV3AdminTest-1718076502-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '572c2ace0562496ebd07f81d37a1522c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1034dc0c96de4f71873b657b88c20c79', 'project_domain_id': 'default', 'roles': ['member', 'tempest-test_role-914577071', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:10.384739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ade6de-bfe1-4d9e-9d95-2df6788fa387 tempest-ApplicationCredentialsV3AdminTest-1718076502 tempest-ApplicationCredentialsV3AdminTest-1718076502-project] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:10.385063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ade6de-bfe1-4d9e-9d95-2df6788fa387 tempest-ApplicationCredentialsV3AdminTest-1718076502 tempest-ApplicationCredentialsV3AdminTest-1718076502-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:10.385354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ade6de-bfe1-4d9e-9d95-2df6788fa387 tempest-ApplicationCredentialsV3AdminTest-1718076502 tempest-ApplicationCredentialsV3AdminTest-1718076502-project] PATH_INFO: `/v3/users/572c2ace0562496ebd07f81d37a1522c/application_credentials/468201f9ac9340219df7c9fc4e6ff772` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:10.386655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ade6de-bfe1-4d9e-9d95-2df6788fa387 tempest-ApplicationCredentialsV3AdminTest-1718076502 tempest-ApplicationCredentialsV3AdminTest-1718076502-project] RBAC: Authorizing `identity:delete_application_credential(user_id=572c2ace0562496ebd07f81d37a1522c, application_credential_id=468201f9ac9340219df7c9fc4e6ff772)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:10.388165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ade6de-bfe1-4d9e-9d95-2df6788fa387 tempest-ApplicationCredentialsV3AdminTest-1718076502 tempest-ApplicationCredentialsV3AdminTest-1718076502-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:10.394249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ecef1-423d-4ada-88f1-8ede1f107922 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:10.399722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80827757-6239-4261-ab5c-6a73192dc0da tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:10.400766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80827757-6239-4261-ab5c-6a73192dc0da tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:10.401056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80827757-6239-4261-ab5c-6a73192dc0da tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:10.401341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80827757-6239-4261-ab5c-6a73192dc0da tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/system/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:10.403383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80827757-6239-4261-ab5c-6a73192dc0da tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_system_grant_for_group(group_id=83dc424272e5493a9a3b46bf902dde1c, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:10.406043 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-80827757-6239-4261-ab5c-6a73192dc0da tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:10.406043 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:26:10.406043 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:26:10.406043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80827757-6239-4261-ab5c-6a73192dc0da tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:10.425001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:UPDATE=2,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:10.482836 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3178/6393] 10.222.0.33 () {58 vars in 1394 bytes} [Sat May 28 21:26:10 2022] DELETE /identity/v3/users/572c2ace0562496ebd07f81d37a1522c/application_credentials/468201f9ac9340219df7c9fc4e6ff772 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:10.483716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ade6de-bfe1-4d9e-9d95-2df6788fa387 tempest-ApplicationCredentialsV3AdminTest-1718076502 tempest-ApplicationCredentialsV3AdminTest-1718076502-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:10.485369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-75ade6de-bfe1-4d9e-9d95-2df6788fa387 tempest-ApplicationCredentialsV3AdminTest-1718076502 tempest-ApplicationCredentialsV3AdminTest-1718076502-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:10.532453 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3216/6394] 10.222.0.33 () {60 vars in 1380 bytes} [Sat May 28 21:26:10 2022] PUT /identity/v3/system/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:10.536446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80827757-6239-4261-ab5c-6a73192dc0da tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:10.550595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80827757-6239-4261-ab5c-6a73192dc0da tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:10.555351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc4225-7f94-4faf-b541-31eb81c4d873 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:10.556200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc4225-7f94-4faf-b541-31eb81c4d873 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:10.556548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc4225-7f94-4faf-b541-31eb81c4d873 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:10.556842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc4225-7f94-4faf-b541-31eb81c4d873 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/system/groups/83dc424272e5493a9a3b46bf902dde1c/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:10.558127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc4225-7f94-4faf-b541-31eb81c4d873 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:list_system_grants_for_group(group_id=83dc424272e5493a9a3b46bf902dde1c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:10.559156 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-74cc4225-7f94-4faf-b541-31eb81c4d873 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:10.559156 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:26:10.559156 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:26:10.559737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ade6de-bfe1-4d9e-9d95-2df6788fa387 tempest-ApplicationCredentialsV3AdminTest-1718076502 tempest-ApplicationCredentialsV3AdminTest-1718076502-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:10.560258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc4225-7f94-4faf-b541-31eb81c4d873 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:10.570055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a7d09-524d-459f-b322-fdbee527ed22 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'db835547a3b743869fb3675e3bf70b11', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '61056f8aa6a94cd18edb1dea01ff284b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:10.570958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a7d09-524d-459f-b322-fdbee527ed22 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:10.571241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a7d09-524d-459f-b322-fdbee527ed22 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:10.571526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a7d09-524d-459f-b322-fdbee527ed22 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] PATH_INFO: `/v3/roles/a4e8f22c4bcc41439f6e77c489e07864` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:10.573128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a7d09-524d-459f-b322-fdbee527ed22 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] RBAC: Authorizing `identity:delete_role(role_id=a4e8f22c4bcc41439f6e77c489e07864)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:10.573604 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3217/6395] 10.222.0.33 () {58 vars in 1262 bytes} [Sat May 28 21:26:10 2022] GET /identity/v3/system/groups/83dc424272e5493a9a3b46bf902dde1c/roles => generated 376 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:10.574966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103a7d09-524d-459f-b322-fdbee527ed22 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:10.577345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc4225-7f94-4faf-b541-31eb81c4d873 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:10.588563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc4225-7f94-4faf-b541-31eb81c4d873 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:10.593405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ed042-dce7-4f1e-9e3a-1b20d919e946 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:10.594311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ed042-dce7-4f1e-9e3a-1b20d919e946 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `HEAD` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:10.594721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ed042-dce7-4f1e-9e3a-1b20d919e946 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:10.594941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412ed042-dce7-4f1e-9e3a-1b20d919e946 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/system/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:10.597072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ed042-dce7-4f1e-9e3a-1b20d919e946 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:check_system_grant_for_group(group_id=83dc424272e5493a9a3b46bf902dde1c, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:10.598151 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-412ed042-dce7-4f1e-9e3a-1b20d919e946 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_group": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:10.598151 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:26:10.598151 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:26:10.599359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412ed042-dce7-4f1e-9e3a-1b20d919e946 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:10.606760 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3218/6396] 10.222.0.33 () {58 vars in 1362 bytes} [Sat May 28 21:26:10 2022] HEAD /identity/v3/system/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 30 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:10.610840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ed042-dce7-4f1e-9e3a-1b20d919e946 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:10.621861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412ed042-dce7-4f1e-9e3a-1b20d919e946 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:10.630763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4bb022-2231-4d03-9584-720b19ad9a4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:10.631882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bb022-2231-4d03-9584-720b19ad9a4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:10.632408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bb022-2231-4d03-9584-720b19ad9a4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:10.632760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4bb022-2231-4d03-9584-720b19ad9a4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/system/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:10.634923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4bb022-2231-4d03-9584-720b19ad9a4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_group(group_id=83dc424272e5493a9a3b46bf902dde1c, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:10.636448 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-7e4bb022-2231-4d03-9584-720b19ad9a4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_group": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:10.636448 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:26:10.636448 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:26:10.643061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4bb022-2231-4d03-9584-720b19ad9a4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:10.757164 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3219/6397] 10.222.0.33 () {58 vars in 1364 bytes} [Sat May 28 21:26:10 2022] DELETE /identity/v3/system/groups/83dc424272e5493a9a3b46bf902dde1c/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:10.762173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4bb022-2231-4d03-9584-720b19ad9a4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:10.773178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4bb022-2231-4d03-9584-720b19ad9a4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:10.778364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef3cfa-eee8-4eaa-820f-452f0d995181 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:10.783618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef3cfa-eee8-4eaa-820f-452f0d995181 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:10.783618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef3cfa-eee8-4eaa-820f-452f0d995181 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:10.783618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef3cfa-eee8-4eaa-820f-452f0d995181 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/domains/d7087720d030435a9edc125e584a7690/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:10.783618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef3cfa-eee8-4eaa-820f-452f0d995181 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=d7087720d030435a9edc125e584a7690, user_id=1f04e663b88b4151ae962359fbc2451d, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:10.784944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef3cfa-eee8-4eaa-820f-452f0d995181 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:10.877889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-103a7d09-524d-459f-b322-fdbee527ed22 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] Invalidating the token cache because role a4e8f22c4bcc41439f6e77c489e07864 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:10.880883 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3220/6398] 10.222.0.33 () {60 vars in 1479 bytes} [Sat May 28 21:26:10 2022] PUT /identity/v3/domains/d7087720d030435a9edc125e584a7690/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:10.881268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-103a7d09-524d-459f-b322-fdbee527ed22 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:10.882540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-103a7d09-524d-459f-b322-fdbee527ed22 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:10.882686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef3cfa-eee8-4eaa-820f-452f0d995181 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:10.884624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-35ef3cfa-eee8-4eaa-820f-452f0d995181 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:10.884755 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3179/6399] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:10 2022] DELETE /identity/v3/roles/a4e8f22c4bcc41439f6e77c489e07864 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:10.889873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a7d09-524d-459f-b322-fdbee527ed22 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:10.891225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-103a7d09-524d-459f-b322-fdbee527ed22 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:10.964690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef3cfa-eee8-4eaa-820f-452f0d995181 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:10.969283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a7d09-524d-459f-b322-fdbee527ed22 tempest-ApplicationCredentialsV3AdminTest-1225472380 tempest-ApplicationCredentialsV3AdminTest-1225472380-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:10.974084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:10.978759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b58c9f-dbf4-4c1a-8316-b89f224629f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:10.979993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b58c9f-dbf4-4c1a-8316-b89f224629f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:10.980379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b58c9f-dbf4-4c1a-8316-b89f224629f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:10.980671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b58c9f-dbf4-4c1a-8316-b89f224629f1 admin admin] PATH_INFO: `/v3/users/572c2ace0562496ebd07f81d37a1522c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:10.982038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b58c9f-dbf4-4c1a-8316-b89f224629f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=572c2ace0562496ebd07f81d37a1522c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:10.984608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b58c9f-dbf4-4c1a-8316-b89f224629f1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:11.033046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39c81c-5266-424b-b941-fb3ea03930ad tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:11.034035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39c81c-5266-424b-b941-fb3ea03930ad tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:11.034478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39c81c-5266-424b-b941-fb3ea03930ad tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:11.034770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb39c81c-5266-424b-b941-fb3ea03930ad tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/domains/d7087720d030435a9edc125e584a7690/users/1f04e663b88b4151ae962359fbc2451d/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:11.036635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39c81c-5266-424b-b941-fb3ea03930ad tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:list_grants(domain_id=d7087720d030435a9edc125e584a7690, user_id=1f04e663b88b4151ae962359fbc2451d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:11.039349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb39c81c-5266-424b-b941-fb3ea03930ad tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:11.052810 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3221/6400] 10.222.0.33 () {58 vars in 1361 bytes} [Sat May 28 21:26:10 2022] GET /identity/v3/domains/d7087720d030435a9edc125e584a7690/users/1f04e663b88b4151ae962359fbc2451d/roles => generated 409 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:26:11.057374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39c81c-5266-424b-b941-fb3ea03930ad tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:11.072558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb39c81c-5266-424b-b941-fb3ea03930ad tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:11.080339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501cd98-84a5-48c6-bc71-1f57b244fd15 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:11.080895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501cd98-84a5-48c6-bc71-1f57b244fd15 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `HEAD` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:11.081190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501cd98-84a5-48c6-bc71-1f57b244fd15 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:11.081475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501cd98-84a5-48c6-bc71-1f57b244fd15 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/domains/d7087720d030435a9edc125e584a7690/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:11.084953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501cd98-84a5-48c6-bc71-1f57b244fd15 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:check_grant(domain_id=d7087720d030435a9edc125e584a7690, user_id=1f04e663b88b4151ae962359fbc2451d, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:11.088459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501cd98-84a5-48c6-bc71-1f57b244fd15 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:11.095601 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3222/6401] 10.222.0.33 () {58 vars in 1461 bytes} [Sat May 28 21:26:11 2022] HEAD /identity/v3/domains/d7087720d030435a9edc125e584a7690/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:11.100213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501cd98-84a5-48c6-bc71-1f57b244fd15 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:11.111746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501cd98-84a5-48c6-bc71-1f57b244fd15 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:11.117215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754d8fa-f5db-4c98-b572-fd0acdd54468 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:11.118201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754d8fa-f5db-4c98-b572-fd0acdd54468 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:11.118492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754d8fa-f5db-4c98-b572-fd0acdd54468 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:11.118781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754d8fa-f5db-4c98-b572-fd0acdd54468 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/domains/d7087720d030435a9edc125e584a7690/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:11.121106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754d8fa-f5db-4c98-b572-fd0acdd54468 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=d7087720d030435a9edc125e584a7690, user_id=1f04e663b88b4151ae962359fbc2451d, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:11.123841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754d8fa-f5db-4c98-b572-fd0acdd54468 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:11.129968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b754d8fa-f5db-4c98-b572-fd0acdd54468 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invalidating the token cache because role 0e46a1ddecc44597be78d09c35eb3779 was removed from user 1f04e663b88b4151ae962359fbc2451d on domain d7087720d030435a9edc125e584a7690. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:11.133191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b754d8fa-f5db-4c98-b572-fd0acdd54468 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:11.133191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b754d8fa-f5db-4c98-b572-fd0acdd54468 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:11.144894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-69b58c9f-dbf4-4c1a-8316-b89f224629f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '572c2ace0562496ebd07f81d37a1522c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:11.145668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-69b58c9f-dbf4-4c1a-8316-b89f224629f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '572c2ace0562496ebd07f81d37a1522c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:11.153442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-69b58c9f-dbf4-4c1a-8316-b89f224629f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '572c2ace0562496ebd07f81d37a1522c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:11.260032 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3223/6402] 10.222.0.33 () {58 vars in 1463 bytes} [Sat May 28 21:26:11 2022] DELETE /identity/v3/domains/d7087720d030435a9edc125e584a7690/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:11.265144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754d8fa-f5db-4c98-b572-fd0acdd54468 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:11.266578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b754d8fa-f5db-4c98-b572-fd0acdd54468 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:11.276337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-69b58c9f-dbf4-4c1a-8316-b89f224629f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '572c2ace0562496ebd07f81d37a1522c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:11.277415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-69b58c9f-dbf4-4c1a-8316-b89f224629f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '572c2ace0562496ebd07f81d37a1522c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:11.301251 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3180/6403] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:10 2022] DELETE /identity/v3/users/572c2ace0562496ebd07f81d37a1522c => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:11.309139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1f5c9-74cf-4d42-8841-4b5f4a81407f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:11.309373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1f5c9-74cf-4d42-8841-4b5f4a81407f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:11.309655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d1f5c9-74cf-4d42-8841-4b5f4a81407f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:11.337885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-93d1f5c9-74cf-4d42-8841-4b5f4a81407f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:11.342140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-93d1f5c9-74cf-4d42-8841-4b5f4a81407f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:11.344243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754d8fa-f5db-4c98-b572-fd0acdd54468 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:11.418635 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3181/6404] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:26:11.430052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1f5c9-74cf-4d42-8841-4b5f4a81407f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:11.431709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-93d1f5c9-74cf-4d42-8841-4b5f4a81407f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:11.437957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e125b93-559a-4d0e-b028-5b03930bfd1b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:11.439004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e125b93-559a-4d0e-b028-5b03930bfd1b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:11.439361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e125b93-559a-4d0e-b028-5b03930bfd1b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:11.439715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e125b93-559a-4d0e-b028-5b03930bfd1b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/projects/b321a6894a5d4c79952076f6c0408168/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:11.442180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e125b93-559a-4d0e-b028-5b03930bfd1b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_grant(project_id=b321a6894a5d4c79952076f6c0408168, user_id=1f04e663b88b4151ae962359fbc2451d, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:11.445633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e125b93-559a-4d0e-b028-5b03930bfd1b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:11.502235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d1f5c9-74cf-4d42-8841-4b5f4a81407f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:11.507730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38b211-bcc7-44c1-b9de-2bc02becb217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:11.508044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38b211-bcc7-44c1-b9de-2bc02becb217 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:11.508436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38b211-bcc7-44c1-b9de-2bc02becb217 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:11.508774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f38b211-bcc7-44c1-b9de-2bc02becb217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:11.521357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38b211-bcc7-44c1-b9de-2bc02becb217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:11.522987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f38b211-bcc7-44c1-b9de-2bc02becb217 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:11.533034 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3182/6405] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:26:11.561952 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3224/6406] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:11 2022] PUT /identity/v3/projects/b321a6894a5d4c79952076f6c0408168/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:11.568330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38b211-bcc7-44c1-b9de-2bc02becb217 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:11.570452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4f38b211-bcc7-44c1-b9de-2bc02becb217 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:11.584521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f38b211-bcc7-44c1-b9de-2bc02becb217 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:11.647047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104c60b-2019-4fa4-a8f1-5073b671a24b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:11.647966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104c60b-2019-4fa4-a8f1-5073b671a24b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:11.648253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104c60b-2019-4fa4-a8f1-5073b671a24b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:11.648538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104c60b-2019-4fa4-a8f1-5073b671a24b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/projects/b321a6894a5d4c79952076f6c0408168/users/1f04e663b88b4151ae962359fbc2451d/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:11.650297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104c60b-2019-4fa4-a8f1-5073b671a24b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:list_grants(project_id=b321a6894a5d4c79952076f6c0408168, user_id=1f04e663b88b4151ae962359fbc2451d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:11.652833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104c60b-2019-4fa4-a8f1-5073b671a24b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:11.665177 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3183/6407] 10.222.0.33 () {58 vars in 1364 bytes} [Sat May 28 21:26:11 2022] GET /identity/v3/projects/b321a6894a5d4c79952076f6c0408168/users/1f04e663b88b4151ae962359fbc2451d/roles => generated 410 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:11.669752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e125b93-559a-4d0e-b028-5b03930bfd1b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:11.684302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e125b93-559a-4d0e-b028-5b03930bfd1b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:11.689716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7679a25-37b5-4078-8244-9eaaa4fb8943 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:11.690796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7679a25-37b5-4078-8244-9eaaa4fb8943 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `HEAD` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:11.691793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7679a25-37b5-4078-8244-9eaaa4fb8943 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:11.691793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7679a25-37b5-4078-8244-9eaaa4fb8943 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/projects/b321a6894a5d4c79952076f6c0408168/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:11.694260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7679a25-37b5-4078-8244-9eaaa4fb8943 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:check_grant(project_id=b321a6894a5d4c79952076f6c0408168, user_id=1f04e663b88b4151ae962359fbc2451d, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:11.697142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7679a25-37b5-4078-8244-9eaaa4fb8943 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:11.703784 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3225/6408] 10.222.0.33 () {58 vars in 1464 bytes} [Sat May 28 21:26:11 2022] HEAD /identity/v3/projects/b321a6894a5d4c79952076f6c0408168/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:11.707904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104c60b-2019-4fa4-a8f1-5073b671a24b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:11.718764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104c60b-2019-4fa4-a8f1-5073b671a24b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:11.723790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760df25-024d-44ff-aac6-fb59a95393f2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:11.728598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760df25-024d-44ff-aac6-fb59a95393f2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:11.728598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760df25-024d-44ff-aac6-fb59a95393f2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:11.728598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760df25-024d-44ff-aac6-fb59a95393f2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/projects/b321a6894a5d4c79952076f6c0408168/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:11.728598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760df25-024d-44ff-aac6-fb59a95393f2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=b321a6894a5d4c79952076f6c0408168, user_id=1f04e663b88b4151ae962359fbc2451d, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:11.730753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b760df25-024d-44ff-aac6-fb59a95393f2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:11.736327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b760df25-024d-44ff-aac6-fb59a95393f2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invalidating the token cache because role 0e46a1ddecc44597be78d09c35eb3779 was removed from user 1f04e663b88b4151ae962359fbc2451d on project b321a6894a5d4c79952076f6c0408168. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:11.736623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b760df25-024d-44ff-aac6-fb59a95393f2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:11.737299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b760df25-024d-44ff-aac6-fb59a95393f2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:11.833432 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3184/6409] 10.222.0.33 () {58 vars in 1466 bytes} [Sat May 28 21:26:11 2022] DELETE /identity/v3/projects/b321a6894a5d4c79952076f6c0408168/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:11.838523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7679a25-37b5-4078-8244-9eaaa4fb8943 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:11.847406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a7679a25-37b5-4078-8244-9eaaa4fb8943 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:11.861732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7679a25-37b5-4078-8244-9eaaa4fb8943 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:11.933719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be0a34-f1e6-454a-9332-d1da3dd1422f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:11.934808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be0a34-f1e6-454a-9332-d1da3dd1422f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:11.935100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be0a34-f1e6-454a-9332-d1da3dd1422f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:11.935386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be0a34-f1e6-454a-9332-d1da3dd1422f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/system/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:11.937213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be0a34-f1e6-454a-9332-d1da3dd1422f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1f04e663b88b4151ae962359fbc2451d, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:11.938273 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-60be0a34-f1e6-454a-9332-d1da3dd1422f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:11.938273 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:26:11.938273 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:26:11.941570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be0a34-f1e6-454a-9332-d1da3dd1422f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:12.083736 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3226/6410] 10.222.0.33 () {60 vars in 1377 bytes} [Sat May 28 21:26:11 2022] PUT /identity/v3/system/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:12.085551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760df25-024d-44ff-aac6-fb59a95393f2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:12.095873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b760df25-024d-44ff-aac6-fb59a95393f2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:12.101351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0da6b-5044-40d3-9cb1-be2fa9d114fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:12.102228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0da6b-5044-40d3-9cb1-be2fa9d114fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:12.103323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0da6b-5044-40d3-9cb1-be2fa9d114fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:12.103323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0da6b-5044-40d3-9cb1-be2fa9d114fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/system/users/1f04e663b88b4151ae962359fbc2451d/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:12.104371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b0da6b-5044-40d3-9cb1-be2fa9d114fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:list_system_grants_for_user(user_id=1f04e663b88b4151ae962359fbc2451d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:12.105441 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-d3b0da6b-5044-40d3-9cb1-be2fa9d114fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_system_grants_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:12.105441 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:26:12.105441 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:26:12.106375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b0da6b-5044-40d3-9cb1-be2fa9d114fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:12.118142 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3185/6411] 10.222.0.33 () {58 vars in 1259 bytes} [Sat May 28 21:26:12 2022] GET /identity/v3/system/users/1f04e663b88b4151ae962359fbc2451d/roles => generated 375 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:12.122503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be0a34-f1e6-454a-9332-d1da3dd1422f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:12.132435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be0a34-f1e6-454a-9332-d1da3dd1422f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:12.137107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87039b90-a2fd-4d21-9cc0-89e168e680df tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:12.137989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87039b90-a2fd-4d21-9cc0-89e168e680df tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `HEAD` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:12.138278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87039b90-a2fd-4d21-9cc0-89e168e680df tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:12.138558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87039b90-a2fd-4d21-9cc0-89e168e680df tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/system/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:12.140328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87039b90-a2fd-4d21-9cc0-89e168e680df tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:check_system_grant_for_user(user_id=1f04e663b88b4151ae962359fbc2451d, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:12.141350 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-87039b90-a2fd-4d21-9cc0-89e168e680df tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_system_grant_for_user": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:12.141350 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:26:12.141350 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:26:12.142272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87039b90-a2fd-4d21-9cc0-89e168e680df tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:12.147070 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3227/6412] 10.222.0.33 () {58 vars in 1359 bytes} [Sat May 28 21:26:12 2022] HEAD /identity/v3/system/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 25 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:12.151487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0da6b-5044-40d3-9cb1-be2fa9d114fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:12.161616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0da6b-5044-40d3-9cb1-be2fa9d114fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:12.167902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091d9b6-67fb-4af9-8159-496e87ca3b6a tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:12.168228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091d9b6-67fb-4af9-8159-496e87ca3b6a tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:12.168522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091d9b6-67fb-4af9-8159-496e87ca3b6a tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:12.168893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091d9b6-67fb-4af9-8159-496e87ca3b6a tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/system/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:12.171271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091d9b6-67fb-4af9-8159-496e87ca3b6a tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:revoke_system_grant_for_user(user_id=1f04e663b88b4151ae962359fbc2451d, role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:12.172290 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-6091d9b6-67fb-4af9-8159-496e87ca3b6a tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:revoke_system_grant_for_user": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:12.172290 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:26:12.172290 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:26:12.173169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091d9b6-67fb-4af9-8159-496e87ca3b6a tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:12.182875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87039b90-a2fd-4d21-9cc0-89e168e680df tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:12.185310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-87039b90-a2fd-4d21-9cc0-89e168e680df tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:12.263484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87039b90-a2fd-4d21-9cc0-89e168e680df tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:12.268533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a32340-c5a9-4e2d-a163-05cf6cbd14db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:12.269451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a32340-c5a9-4e2d-a163-05cf6cbd14db admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:12.271910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a32340-c5a9-4e2d-a163-05cf6cbd14db admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:12.271910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a32340-c5a9-4e2d-a163-05cf6cbd14db admin admin] PATH_INFO: `/v3/projects/1034dc0c96de4f71873b657b88c20c79` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:12.271910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a32340-c5a9-4e2d-a163-05cf6cbd14db admin admin] RBAC: Authorizing `identity:delete_project(project_id=1034dc0c96de4f71873b657b88c20c79)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:12.274041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a32340-c5a9-4e2d-a163-05cf6cbd14db admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:12.297656 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3186/6413] 10.222.0.33 () {58 vars in 1361 bytes} [Sat May 28 21:26:12 2022] DELETE /identity/v3/system/users/1f04e663b88b4151ae962359fbc2451d/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:12.298719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091d9b6-67fb-4af9-8159-496e87ca3b6a tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:12.311452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091d9b6-67fb-4af9-8159-496e87ca3b6a tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:12.316539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cc8cd-af68-4c9d-b158-bd577bf14da6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:12.317361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cc8cd-af68-4c9d-b158-bd577bf14da6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:12.317653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cc8cd-af68-4c9d-b158-bd577bf14da6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:12.318079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0cc8cd-af68-4c9d-b158-bd577bf14da6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:12.319487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0cc8cd-af68-4c9d-b158-bd577bf14da6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:12.321654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0cc8cd-af68-4c9d-b158-bd577bf14da6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:12.407709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-74a32340-c5a9-4e2d-a163-05cf6cbd14db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1034dc0c96de4f71873b657b88c20c79', 'request_id': 'req-74a32340-c5a9-4e2d-a163-05cf6cbd14db'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:12.413798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-74a32340-c5a9-4e2d-a163-05cf6cbd14db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1034dc0c96de4f71873b657b88c20c79', 'request_id': 'req-74a32340-c5a9-4e2d-a163-05cf6cbd14db'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:12.423922 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3187/6414] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:26:12 2022] POST /identity/v3/roles => generated 269 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:12.425095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cc8cd-af68-4c9d-b158-bd577bf14da6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:12.435240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0cc8cd-af68-4c9d-b158-bd577bf14da6 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:12.500167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77762624-e65e-4072-8e39-deae4d705032 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:12.500909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77762624-e65e-4072-8e39-deae4d705032 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:12.501264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77762624-e65e-4072-8e39-deae4d705032 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:12.501562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77762624-e65e-4072-8e39-deae4d705032 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:12.502353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77762624-e65e-4072-8e39-deae4d705032 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:12.503967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77762624-e65e-4072-8e39-deae4d705032 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:12.539301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-74a32340-c5a9-4e2d-a163-05cf6cbd14db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1034dc0c96de4f71873b657b88c20c79', 'request_id': 'req-74a32340-c5a9-4e2d-a163-05cf6cbd14db'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:12.574086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-74a32340-c5a9-4e2d-a163-05cf6cbd14db admin admin] The token cache is being invalidate because project 1034dc0c96de4f71873b657b88c20c79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:12.574086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-74a32340-c5a9-4e2d-a163-05cf6cbd14db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:12.574086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-74a32340-c5a9-4e2d-a163-05cf6cbd14db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:12.574086 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3228/6415] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:26:12 2022] DELETE /identity/v3/projects/1034dc0c96de4f71873b657b88c20c79 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:12.575924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a32340-c5a9-4e2d-a163-05cf6cbd14db admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:12.577682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-74a32340-c5a9-4e2d-a163-05cf6cbd14db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:12.608495 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3188/6416] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:26:12 2022] POST /identity/v3/roles => generated 270 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:12.612105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77762624-e65e-4072-8e39-deae4d705032 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:12.614245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-77762624-e65e-4072-8e39-deae4d705032 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:12.629215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77762624-e65e-4072-8e39-deae4d705032 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:12.636935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e7c77-7b29-4c99-a21f-8add22239d1b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:12.636935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e7c77-7b29-4c99-a21f-8add22239d1b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:12.636935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e7c77-7b29-4c99-a21f-8add22239d1b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:12.637290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e7c77-7b29-4c99-a21f-8add22239d1b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/0582d4ee67cc43ed9cbe297d38f4bc0f/implies/cd1d451ad0eb463e953b8dd5d998ff54` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:12.638711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e7c77-7b29-4c99-a21f-8add22239d1b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=0582d4ee67cc43ed9cbe297d38f4bc0f, implied_role_id=cd1d451ad0eb463e953b8dd5d998ff54)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:12.639750 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-e76e7c77-7b29-4c99-a21f-8add22239d1b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:12.639750 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:26:12.639750 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:26:12.641185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e7c77-7b29-4c99-a21f-8add22239d1b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:12.657443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a32340-c5a9-4e2d-a163-05cf6cbd14db admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:12.663049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0feb6f-2bd4-40df-aa60-8fc1b978578f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:12.664385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0feb6f-2bd4-40df-aa60-8fc1b978578f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:12.664681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0feb6f-2bd4-40df-aa60-8fc1b978578f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:12.664963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0feb6f-2bd4-40df-aa60-8fc1b978578f admin admin] PATH_INFO: `/v3/users/db835547a3b743869fb3675e3bf70b11` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:12.666257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0feb6f-2bd4-40df-aa60-8fc1b978578f admin admin] RBAC: Authorizing `identity:delete_user(user_id=db835547a3b743869fb3675e3bf70b11)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:12.674096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0feb6f-2bd4-40df-aa60-8fc1b978578f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:12.750796 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3189/6417] 10.222.0.33 () {60 vars in 1362 bytes} [Sat May 28 21:26:12 2022] PUT /identity/v3/roles/0582d4ee67cc43ed9cbe297d38f4bc0f/implies/cd1d451ad0eb463e953b8dd5d998ff54 => generated 530 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:12.754539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e7c77-7b29-4c99-a21f-8add22239d1b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:12.819378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e7c77-7b29-4c99-a21f-8add22239d1b tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:12.825004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f08c5-3a31-4640-8a30-bd8a6bdd7647 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:12.825621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f08c5-3a31-4640-8a30-bd8a6bdd7647 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:12.826215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f08c5-3a31-4640-8a30-bd8a6bdd7647 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:12.826215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f08c5-3a31-4640-8a30-bd8a6bdd7647 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:12.826952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f08c5-3a31-4640-8a30-bd8a6bdd7647 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:12.828691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f08c5-3a31-4640-8a30-bd8a6bdd7647 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:12.890300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:12.918660 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3190/6418] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:26:12 2022] POST /identity/v3/domains => generated 277 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:12.922344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f08c5-3a31-4640-8a30-bd8a6bdd7647 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:12.995188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f08c5-3a31-4640-8a30-bd8a6bdd7647 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:13.000580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228147c-8a8b-4767-8692-47b4a73e3abf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:13.012418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228147c-8a8b-4767-8692-47b4a73e3abf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:13.012897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228147c-8a8b-4767-8692-47b4a73e3abf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:13.013024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9228147c-8a8b-4767-8692-47b4a73e3abf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:13.013812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228147c-8a8b-4767-8692-47b4a73e3abf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_domain_role()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:13.020054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9228147c-8a8b-4767-8692-47b4a73e3abf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:13.026100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4a0feb6f-2bd4-40df-aa60-8fc1b978578f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db835547a3b743869fb3675e3bf70b11'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:13.034233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4a0feb6f-2bd4-40df-aa60-8fc1b978578f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db835547a3b743869fb3675e3bf70b11'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:13.035000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4a0feb6f-2bd4-40df-aa60-8fc1b978578f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db835547a3b743869fb3675e3bf70b11'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:13.035613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4a0feb6f-2bd4-40df-aa60-8fc1b978578f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db835547a3b743869fb3675e3bf70b11'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:13.053299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:13.114715 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3191/6419] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:26:12 2022] POST /identity/v3/roles => generated 269 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:13.116614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4a0feb6f-2bd4-40df-aa60-8fc1b978578f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db835547a3b743869fb3675e3bf70b11'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:13.116887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228147c-8a8b-4767-8692-47b4a73e3abf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:13.119438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9228147c-8a8b-4767-8692-47b4a73e3abf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:13.134402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9228147c-8a8b-4767-8692-47b4a73e3abf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:13.139147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef22a35-4901-430d-8832-988561167ddf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:13.145378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef22a35-4901-430d-8832-988561167ddf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:13.145378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef22a35-4901-430d-8832-988561167ddf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:13.145378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef22a35-4901-430d-8832-988561167ddf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/0582d4ee67cc43ed9cbe297d38f4bc0f/implies/fc10739412a6425d8b47fc35483d5d91` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:13.145686 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3229/6420] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:12 2022] DELETE /identity/v3/users/db835547a3b743869fb3675e3bf70b11 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:13.145963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef22a35-4901-430d-8832-988561167ddf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=0582d4ee67cc43ed9cbe297d38f4bc0f, implied_role_id=fc10739412a6425d8b47fc35483d5d91)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:13.145963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef22a35-4901-430d-8832-988561167ddf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:13.316183 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3192/6421] 10.222.0.33 () {60 vars in 1362 bytes} [Sat May 28 21:26:13 2022] PUT /identity/v3/roles/0582d4ee67cc43ed9cbe297d38f4bc0f/implies/fc10739412a6425d8b47fc35483d5d91 => generated 529 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:13.332112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0feb6f-2bd4-40df-aa60-8fc1b978578f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:13.392964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0feb6f-2bd4-40df-aa60-8fc1b978578f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:13.486045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9efaf5-17de-473f-a2b6-150dce6aff77 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:13.487114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9efaf5-17de-473f-a2b6-150dce6aff77 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:13.487413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9efaf5-17de-473f-a2b6-150dce6aff77 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:13.487698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9efaf5-17de-473f-a2b6-150dce6aff77 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/0582d4ee67cc43ed9cbe297d38f4bc0f/implies/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:13.495203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9efaf5-17de-473f-a2b6-150dce6aff77 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=0582d4ee67cc43ed9cbe297d38f4bc0f, implied_role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:13.497054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9efaf5-17de-473f-a2b6-150dce6aff77 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:13.631859 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3230/6422] 10.222.0.33 () {60 vars in 1362 bytes} [Sat May 28 21:26:13 2022] PUT /identity/v3/roles/0582d4ee67cc43ed9cbe297d38f4bc0f/implies/0e46a1ddecc44597be78d09c35eb3779 => generated 525 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:13.632934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef22a35-4901-430d-8832-988561167ddf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:13.640705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2ef22a35-4901-430d-8832-988561167ddf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:13.641813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9efaf5-17de-473f-a2b6-150dce6aff77 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:13.718665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef22a35-4901-430d-8832-988561167ddf tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:13.733672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9efaf5-17de-473f-a2b6-150dce6aff77 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:13.739690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9e579-988f-4333-a807-aa8c3711f7a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:13.740636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9e579-988f-4333-a807-aa8c3711f7a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:13.740930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9e579-988f-4333-a807-aa8c3711f7a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:13.741570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9e579-988f-4333-a807-aa8c3711f7a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/0e46a1ddecc44597be78d09c35eb3779/implies/0582d4ee67cc43ed9cbe297d38f4bc0f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:13.743583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9e579-988f-4333-a807-aa8c3711f7a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=0e46a1ddecc44597be78d09c35eb3779, implied_role_id=0582d4ee67cc43ed9cbe297d38f4bc0f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:13.745016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9e579-988f-4333-a807-aa8c3711f7a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:13.775393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-55c9e579-988f-4333-a807-aa8c3711f7a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] 0582d4ee67cc43ed9cbe297d38f4bc0f cannot be an implied roles.: keystone.exception.InvalidImpliedRole: 0582d4ee67cc43ed9cbe297d38f4bc0f cannot be an implied roles. May 28 21:26:13.775393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:26:13.775393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:26:13.775393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:26:13.775393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:26:13.775393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:26:13.775393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:26:13.775393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:26:13.775393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:26:13.775393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:26:13.775393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:26:13.775393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:26:13.775393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 259, in put May 28 21:26:13.775393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application PROVIDERS.role_api.create_implied_role(prior_role_id, implied_role_id) May 28 21:26:13.776857 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:26:13.776857 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:26:13.776857 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1361, in create_implied_role May 28 21:26:13.776857 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.InvalidImpliedRole(msg, May 28 21:26:13.776857 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.InvalidImpliedRole: 0582d4ee67cc43ed9cbe297d38f4bc0f cannot be an implied roles. May 28 21:26:13.776857 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:26:13.780389 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3231/6423] 10.222.0.33 () {60 vars in 1362 bytes} [Sat May 28 21:26:13 2022] PUT /identity/v3/roles/0e46a1ddecc44597be78d09c35eb3779/implies/0582d4ee67cc43ed9cbe297d38f4bc0f => generated 116 bytes in 139 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) May 28 21:26:13.781279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9e579-988f-4333-a807-aa8c3711f7a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:13.792803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9e579-988f-4333-a807-aa8c3711f7a4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:13.797651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98805b47-0c8a-4612-a4ce-29744e68840c tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:13.798609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98805b47-0c8a-4612-a4ce-29744e68840c tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:13.798897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98805b47-0c8a-4612-a4ce-29744e68840c tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:13.799225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98805b47-0c8a-4612-a4ce-29744e68840c tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/0582d4ee67cc43ed9cbe297d38f4bc0f/implies/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:13.801154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98805b47-0c8a-4612-a4ce-29744e68840c tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=0582d4ee67cc43ed9cbe297d38f4bc0f, implied_role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:13.802466 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-98805b47-0c8a-4612-a4ce-29744e68840c tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_implied_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:13.802466 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:26:13.802466 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:26:13.807498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98805b47-0c8a-4612-a4ce-29744e68840c tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:13.808043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2218ee7-bc53-4a66-8fcc-a36c18ef8b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:13.808911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2218ee7-bc53-4a66-8fcc-a36c18ef8b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:13.809196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2218ee7-bc53-4a66-8fcc-a36c18ef8b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:13.809474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2218ee7-bc53-4a66-8fcc-a36c18ef8b2d admin admin] PATH_INFO: `/v3/projects/61056f8aa6a94cd18edb1dea01ff284b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:13.810754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2218ee7-bc53-4a66-8fcc-a36c18ef8b2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=61056f8aa6a94cd18edb1dea01ff284b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:13.812791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2218ee7-bc53-4a66-8fcc-a36c18ef8b2d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:13.900393 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3232/6424] 10.222.0.33 () {58 vars in 1346 bytes} [Sat May 28 21:26:13 2022] DELETE /identity/v3/roles/0582d4ee67cc43ed9cbe297d38f4bc0f/implies/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:13.904493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98805b47-0c8a-4612-a4ce-29744e68840c tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:13.933571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b2218ee7-bc53-4a66-8fcc-a36c18ef8b2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61056f8aa6a94cd18edb1dea01ff284b', 'request_id': 'req-b2218ee7-bc53-4a66-8fcc-a36c18ef8b2d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:13.939932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b2218ee7-bc53-4a66-8fcc-a36c18ef8b2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61056f8aa6a94cd18edb1dea01ff284b', 'request_id': 'req-b2218ee7-bc53-4a66-8fcc-a36c18ef8b2d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:13.975261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98805b47-0c8a-4612-a4ce-29744e68840c tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:14.018019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b2218ee7-bc53-4a66-8fcc-a36c18ef8b2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61056f8aa6a94cd18edb1dea01ff284b', 'request_id': 'req-b2218ee7-bc53-4a66-8fcc-a36c18ef8b2d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:14.030221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b2218ee7-bc53-4a66-8fcc-a36c18ef8b2d admin admin] The token cache is being invalidate because project 61056f8aa6a94cd18edb1dea01ff284b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:14.030510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b2218ee7-bc53-4a66-8fcc-a36c18ef8b2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:14.031143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b2218ee7-bc53-4a66-8fcc-a36c18ef8b2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:14.032062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004dc9a4-64e5-48f9-b43f-a87421260c5e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:14.032936 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3193/6425] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:26:13 2022] DELETE /identity/v3/projects/61056f8aa6a94cd18edb1dea01ff284b => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:14.033114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dc9a4-64e5-48f9-b43f-a87421260c5e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:14.033402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dc9a4-64e5-48f9-b43f-a87421260c5e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:14.033684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dc9a4-64e5-48f9-b43f-a87421260c5e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/0582d4ee67cc43ed9cbe297d38f4bc0f/implies/fc10739412a6425d8b47fc35483d5d91` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:14.035454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004dc9a4-64e5-48f9-b43f-a87421260c5e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=0582d4ee67cc43ed9cbe297d38f4bc0f, implied_role_id=fc10739412a6425d8b47fc35483d5d91)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:14.037352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004dc9a4-64e5-48f9-b43f-a87421260c5e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:14.039745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9590a25-2062-48cd-a4d5-0482332d7f4d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:14.040040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9590a25-2062-48cd-a4d5-0482332d7f4d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:14.040425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9590a25-2062-48cd-a4d5-0482332d7f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:14.062105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b9590a25-2062-48cd-a4d5-0482332d7f4d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:14.063480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b9590a25-2062-48cd-a4d5-0482332d7f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:14.136429 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3194/6426] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:26:14.144223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1a3aa-13f8-4c9a-b11b-c93a23af955e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:14.144521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1a3aa-13f8-4c9a-b11b-c93a23af955e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:14.144892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1a3aa-13f8-4c9a-b11b-c93a23af955e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:14.158051 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3233/6427] 10.222.0.33 () {58 vars in 1346 bytes} [Sat May 28 21:26:13 2022] DELETE /identity/v3/roles/0582d4ee67cc43ed9cbe297d38f4bc0f/implies/fc10739412a6425d8b47fc35483d5d91 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:14.164026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004dc9a4-64e5-48f9-b43f-a87421260c5e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:14.167974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-004dc9a4-64e5-48f9-b43f-a87421260c5e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:14.177710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b1d1a3aa-13f8-4c9a-b11b-c93a23af955e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:14.179014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b1d1a3aa-13f8-4c9a-b11b-c93a23af955e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:14.247189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004dc9a4-64e5-48f9-b43f-a87421260c5e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:14.247472 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3195/6428] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:26:14.251760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1a3aa-13f8-4c9a-b11b-c93a23af955e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:14.254179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c37024-589c-49f4-86a9-b35f431abf29 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:14.255201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c37024-589c-49f4-86a9-b35f431abf29 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:14.255586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c37024-589c-49f4-86a9-b35f431abf29 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:14.255945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c37024-589c-49f4-86a9-b35f431abf29 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/fc10739412a6425d8b47fc35483d5d91` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:14.257971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c37024-589c-49f4-86a9-b35f431abf29 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=fc10739412a6425d8b47fc35483d5d91)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:14.259005 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-d6c37024-589c-49f4-86a9-b35f431abf29 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:14.259005 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:26:14.259005 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:26:14.259900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c37024-589c-49f4-86a9-b35f431abf29 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:14.262292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1a3aa-13f8-4c9a-b11b-c93a23af955e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:14.267402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb02a56-585c-4067-96bf-9142f06af064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:14.268067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb02a56-585c-4067-96bf-9142f06af064 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:14.268355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb02a56-585c-4067-96bf-9142f06af064 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:14.268641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb02a56-585c-4067-96bf-9142f06af064 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:14.272758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb02a56-585c-4067-96bf-9142f06af064 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:14.272758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb02a56-585c-4067-96bf-9142f06af064 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:14.272758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb02a56-585c-4067-96bf-9142f06af064 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:14.283347 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3196/6429] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:14.287584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb02a56-585c-4067-96bf-9142f06af064 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:14.297310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb02a56-585c-4067-96bf-9142f06af064 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:14.301943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea564e00-446f-4a2e-be00-64a645d933ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:14.302579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564e00-446f-4a2e-be00-64a645d933ec admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:14.302869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564e00-446f-4a2e-be00-64a645d933ec admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:14.303152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea564e00-446f-4a2e-be00-64a645d933ec admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:14.304403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea564e00-446f-4a2e-be00-64a645d933ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:14.306378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea564e00-446f-4a2e-be00-64a645d933ec admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:14.354470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d6c37024-589c-49f4-86a9-b35f431abf29 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invalidating the token cache because role fc10739412a6425d8b47fc35483d5d91 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:14.354788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d6c37024-589c-49f4-86a9-b35f431abf29 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:14.355465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d6c37024-589c-49f4-86a9-b35f431abf29 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:14.357427 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3234/6430] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:14 2022] DELETE /identity/v3/roles/fc10739412a6425d8b47fc35483d5d91 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:14.361374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c37024-589c-49f4-86a9-b35f431abf29 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:14.362733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d6c37024-589c-49f4-86a9-b35f431abf29 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:14.417539 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3197/6431] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:14 2022] POST /identity/v3/projects => generated 379 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:14.418561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea564e00-446f-4a2e-be00-64a645d933ec admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:14.420169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ea564e00-446f-4a2e-be00-64a645d933ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:14.429865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c37024-589c-49f4-86a9-b35f431abf29 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:14.489803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea564e00-446f-4a2e-be00-64a645d933ec admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:14.492058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8aad60-fa3d-4eab-933e-c7192da72406 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:14.492938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8aad60-fa3d-4eab-933e-c7192da72406 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:14.493223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8aad60-fa3d-4eab-933e-c7192da72406 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:14.493507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8aad60-fa3d-4eab-933e-c7192da72406 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/domains/d99c791bb3ab4a2eb2b67301f993b944` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:14.495855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c97f6-eaaa-4eb4-bda0-091422256623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:14.496789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c97f6-eaaa-4eb4-bda0-091422256623 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:14.497209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c97f6-eaaa-4eb4-bda0-091422256623 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:14.497589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c97f6-eaaa-4eb4-bda0-091422256623 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:14.498410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c97f6-eaaa-4eb4-bda0-091422256623 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:14.500453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c97f6-eaaa-4eb4-bda0-091422256623 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:14.504887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8aad60-fa3d-4eab-933e-c7192da72406 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=d99c791bb3ab4a2eb2b67301f993b944)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:14.506677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8aad60-fa3d-4eab-933e-c7192da72406 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:14.526770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2c8aad60-fa3d-4eab-933e-c7192da72406 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'd99c791bb3ab4a2eb2b67301f993b944'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:14.527569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2c8aad60-fa3d-4eab-933e-c7192da72406 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'd99c791bb3ab4a2eb2b67301f993b944'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:14.613470 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3198/6432] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:14 2022] POST /identity/v3/users => generated 290 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:14.617962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c97f6-eaaa-4eb4-bda0-091422256623 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:14.619561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2a5c97f6-eaaa-4eb4-bda0-091422256623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:14.631646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c97f6-eaaa-4eb4-bda0-091422256623 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:14.665529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2c8aad60-fa3d-4eab-933e-c7192da72406 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'd99c791bb3ab4a2eb2b67301f993b944'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:14.666192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2c8aad60-fa3d-4eab-933e-c7192da72406 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'd99c791bb3ab4a2eb2b67301f993b944'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:14.671639 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3235/6433] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:26:14 2022] PATCH /identity/v3/domains/d99c791bb3ab4a2eb2b67301f993b944 => generated 278 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:26:14.672625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8aad60-fa3d-4eab-933e-c7192da72406 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:14.674024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2c8aad60-fa3d-4eab-933e-c7192da72406 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:14.689344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e229c-f66f-41d7-a83e-600a5bd9080f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:14.690056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e229c-f66f-41d7-a83e-600a5bd9080f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:14.690350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e229c-f66f-41d7-a83e-600a5bd9080f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:14.690633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e229c-f66f-41d7-a83e-600a5bd9080f admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:14.691370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e229c-f66f-41d7-a83e-600a5bd9080f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:14.692997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e229c-f66f-41d7-a83e-600a5bd9080f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:14.704139 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3199/6434] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:14 2022] GET /identity/v3/roles => generated 2324 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:26:14.709169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e229c-f66f-41d7-a83e-600a5bd9080f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:14.709962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-478e229c-f66f-41d7-a83e-600a5bd9080f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:14.722462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e229c-f66f-41d7-a83e-600a5bd9080f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:14.727865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab42c3f-6b5c-4121-a588-89abb6b89c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:14.728527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab42c3f-6b5c-4121-a588-89abb6b89c03 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:14.732129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab42c3f-6b5c-4121-a588-89abb6b89c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:14.732129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab42c3f-6b5c-4121-a588-89abb6b89c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:14.732129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab42c3f-6b5c-4121-a588-89abb6b89c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:14.732129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab42c3f-6b5c-4121-a588-89abb6b89c03 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:14.739086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8aad60-fa3d-4eab-933e-c7192da72406 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:14.743245 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3200/6435] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:14 2022] GET /identity/v3/roles => generated 2324 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:26:14.744710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:14.745585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:14.745904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:14.746612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/domains/d99c791bb3ab4a2eb2b67301f993b944` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:14.747385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab42c3f-6b5c-4121-a588-89abb6b89c03 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:14.757249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab42c3f-6b5c-4121-a588-89abb6b89c03 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:14.760838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=d99c791bb3ab4a2eb2b67301f993b944)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:14.760838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:14.762239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6497310a-2a5e-4609-9651-9f6f22e1d2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:14.763112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497310a-2a5e-4609-9651-9f6f22e1d2a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:14.763404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497310a-2a5e-4609-9651-9f6f22e1d2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:14.763692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6497310a-2a5e-4609-9651-9f6f22e1d2a6 admin admin] PATH_INFO: `/v3/projects/178a0db5e64d44e79b19f773932da2e6/users/535ee7ca345b4654a96365391c908ec6/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:14.779259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6497310a-2a5e-4609-9651-9f6f22e1d2a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=178a0db5e64d44e79b19f773932da2e6, user_id=535ee7ca345b4654a96365391c908ec6, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:14.781997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'd99c791bb3ab4a2eb2b67301f993b944'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:14.783748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6497310a-2a5e-4609-9651-9f6f22e1d2a6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:14.785312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'd99c791bb3ab4a2eb2b67301f993b944'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:14.866458 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3201/6436] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:14 2022] PUT /identity/v3/projects/178a0db5e64d44e79b19f773932da2e6/users/535ee7ca345b4654a96365391c908ec6/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:14.877296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69994cbb-776d-48c0-9158-8f51419c1b2f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:14.877580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69994cbb-776d-48c0-9158-8f51419c1b2f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:14.877886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69994cbb-776d-48c0-9158-8f51419c1b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:14.902196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd99c791bb3ab4a2eb2b67301f993b944', 'request_id': 'req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:14.906921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-69994cbb-776d-48c0-9158-8f51419c1b2f None None] MFA Rules not processed for user `535ee7ca345b4654a96365391c908ec6`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:14.908133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-69994cbb-776d-48c0-9158-8f51419c1b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:14.914801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd99c791bb3ab4a2eb2b67301f993b944', 'request_id': 'req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:14.977830 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3202/6437] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:26:14.990377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69994cbb-776d-48c0-9158-8f51419c1b2f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:14.992384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-69994cbb-776d-48c0-9158-8f51419c1b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:14.994651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd99c791bb3ab4a2eb2b67301f993b944', 'request_id': 'req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:15.007645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] The token cache is being invalidate because project d99c791bb3ab4a2eb2b67301f993b944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:15.007928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:15.008604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:15.016243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'd99c791bb3ab4a2eb2b67301f993b944', 'request_id': 'req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:15.016999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'd99c791bb3ab4a2eb2b67301f993b944', 'request_id': 'req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:15.020404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'd99c791bb3ab4a2eb2b67301f993b944', 'request_id': 'req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:15.026956 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3236/6438] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:26:14 2022] DELETE /identity/v3/domains/d99c791bb3ab4a2eb2b67301f993b944 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:15.031111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:15.032468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:15.052734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69994cbb-776d-48c0-9158-8f51419c1b2f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:15.057341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efb864-f258-4a07-8fa6-237a99b77778 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:15.058048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efb864-f258-4a07-8fa6-237a99b77778 service glance] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:15.058339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efb864-f258-4a07-8fa6-237a99b77778 service glance] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:15.058616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efb864-f258-4a07-8fa6-237a99b77778 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:15.064571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c7efb864-f258-4a07-8fa6-237a99b77778 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:15.077059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7efb864-f258-4a07-8fa6-237a99b77778 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:15.078635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7efb864-f258-4a07-8fa6-237a99b77778 service glance] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:15.088532 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3203/6439] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:26:15.099389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec31fe4-46d9-4511-a3dc-764f9bb2f4fb tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:15.105563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978a6d2-c59e-494a-95cf-7256044a93ca tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:15.106885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a6d2-c59e-494a-95cf-7256044a93ca tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:15.107267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a6d2-c59e-494a-95cf-7256044a93ca tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:15.107645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978a6d2-c59e-494a-95cf-7256044a93ca tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/0582d4ee67cc43ed9cbe297d38f4bc0f/implies/cd1d451ad0eb463e953b8dd5d998ff54` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:15.109633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978a6d2-c59e-494a-95cf-7256044a93ca tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=0582d4ee67cc43ed9cbe297d38f4bc0f, implied_role_id=cd1d451ad0eb463e953b8dd5d998ff54)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:15.112036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978a6d2-c59e-494a-95cf-7256044a93ca tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:15.133451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:15.197563 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3237/6440] 10.222.0.33 () {58 vars in 1346 bytes} [Sat May 28 21:26:15 2022] DELETE /identity/v3/roles/0582d4ee67cc43ed9cbe297d38f4bc0f/implies/cd1d451ad0eb463e953b8dd5d998ff54 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:15.201717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efb864-f258-4a07-8fa6-237a99b77778 service glance] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:15.214832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7efb864-f258-4a07-8fa6-237a99b77778 service glance] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:15.263703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:15.273327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7f7b2-95af-49d4-92aa-8c7eb0af22d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:15.274259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7f7b2-95af-49d4-92aa-8c7eb0af22d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:15.274555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7f7b2-95af-49d4-92aa-8c7eb0af22d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:15.274888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae7f7b2-95af-49d4-92aa-8c7eb0af22d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/cd1d451ad0eb463e953b8dd5d998ff54` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:15.276125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7f7b2-95af-49d4-92aa-8c7eb0af22d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=cd1d451ad0eb463e953b8dd5d998ff54)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:15.278030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae7f7b2-95af-49d4-92aa-8c7eb0af22d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:15.378752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3ae7f7b2-95af-49d4-92aa-8c7eb0af22d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invalidating the token cache because role cd1d451ad0eb463e953b8dd5d998ff54 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:15.378752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3ae7f7b2-95af-49d4-92aa-8c7eb0af22d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:15.378752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3ae7f7b2-95af-49d4-92aa-8c7eb0af22d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:15.378752 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3204/6441] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:15 2022] DELETE /identity/v3/roles/cd1d451ad0eb463e953b8dd5d998ff54 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:15.380355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978a6d2-c59e-494a-95cf-7256044a93ca tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:15.383169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b978a6d2-c59e-494a-95cf-7256044a93ca tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:15.396893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978a6d2-c59e-494a-95cf-7256044a93ca tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:15.461831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ed217-ae38-4d2a-bdcc-46b62894b105 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:15.462785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ed217-ae38-4d2a-bdcc-46b62894b105 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:15.463063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ed217-ae38-4d2a-bdcc-46b62894b105 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:15.464882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ed217-ae38-4d2a-bdcc-46b62894b105 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/0582d4ee67cc43ed9cbe297d38f4bc0f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:15.464882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ed217-ae38-4d2a-bdcc-46b62894b105 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_domain_role(role_id=0582d4ee67cc43ed9cbe297d38f4bc0f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:15.466409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ed217-ae38-4d2a-bdcc-46b62894b105 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:15.476114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7f7b2-95af-49d4-92aa-8c7eb0af22d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:15.477517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3ae7f7b2-95af-49d4-92aa-8c7eb0af22d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:15.547329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae7f7b2-95af-49d4-92aa-8c7eb0af22d3 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:15.552261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9a3df-d6de-42a8-b801-ee964596dd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:15.553233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9a3df-d6de-42a8-b801-ee964596dd25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:15.553486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9a3df-d6de-42a8-b801-ee964596dd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:15.553780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9a3df-d6de-42a8-b801-ee964596dd25 admin admin] PATH_INFO: `/v3/users/535ee7ca345b4654a96365391c908ec6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:15.555170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9a3df-d6de-42a8-b801-ee964596dd25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=535ee7ca345b4654a96365391c908ec6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:15.557114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9a3df-d6de-42a8-b801-ee964596dd25 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:15.607546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d73ed217-ae38-4d2a-bdcc-46b62894b105 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invalidating the token cache because role 0582d4ee67cc43ed9cbe297d38f4bc0f has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:15.608049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d73ed217-ae38-4d2a-bdcc-46b62894b105 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:15.608722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d73ed217-ae38-4d2a-bdcc-46b62894b105 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:15.611282 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3238/6442] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:15 2022] DELETE /identity/v3/roles/0582d4ee67cc43ed9cbe297d38f4bc0f => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:15.616940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ed217-ae38-4d2a-bdcc-46b62894b105 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:15.618926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d73ed217-ae38-4d2a-bdcc-46b62894b105 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:15.686596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ed217-ae38-4d2a-bdcc-46b62894b105 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:15.691381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5096134-014e-42ca-8aa9-2dca4cb55952 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:15.692311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5096134-014e-42ca-8aa9-2dca4cb55952 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:15.692599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5096134-014e-42ca-8aa9-2dca4cb55952 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:15.692881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5096134-014e-42ca-8aa9-2dca4cb55952 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:15.695193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5096134-014e-42ca-8aa9-2dca4cb55952 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=9ff56f0c05ab49039e0ff27f5ae8fbb3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:15.696812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5096134-014e-42ca-8aa9-2dca4cb55952 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:15.864445 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3239/6443] 10.222.0.33 () {60 vars in 1362 bytes} [Sat May 28 21:26:15 2022] PUT /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3 => generated 521 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:15.868795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5096134-014e-42ca-8aa9-2dca4cb55952 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:15.874669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-23b9a3df-d6de-42a8-b801-ee964596dd25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '535ee7ca345b4654a96365391c908ec6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:15.875325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-23b9a3df-d6de-42a8-b801-ee964596dd25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '535ee7ca345b4654a96365391c908ec6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:15.885042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-23b9a3df-d6de-42a8-b801-ee964596dd25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '535ee7ca345b4654a96365391c908ec6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:15.932510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5096134-014e-42ca-8aa9-2dca4cb55952 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:15.937571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6aacf-51ae-47fa-9145-3c2100455d36 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:15.938541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6aacf-51ae-47fa-9145-3c2100455d36 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `HEAD` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:15.938886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6aacf-51ae-47fa-9145-3c2100455d36 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:15.939097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6aacf-51ae-47fa-9145-3c2100455d36 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:15.940836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6aacf-51ae-47fa-9145-3c2100455d36 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:check_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=9ff56f0c05ab49039e0ff27f5ae8fbb3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:15.941928 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-90d6aacf-51ae-47fa-9145-3c2100455d36 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:check_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:15.941928 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:26:15.941928 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:26:15.942781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6aacf-51ae-47fa-9145-3c2100455d36 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:15.943721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6aacf-51ae-47fa-9145-3c2100455d36 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=9ff56f0c05ab49039e0ff27f5ae8fbb3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:15.945222 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-90d6aacf-51ae-47fa-9145-3c2100455d36 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_implied_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:15.945222 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:26:15.945222 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:26:15.946110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6aacf-51ae-47fa-9145-3c2100455d36 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:15.952228 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3240/6444] 10.222.0.33 () {58 vars in 1344 bytes} [Sat May 28 21:26:15 2022] HEAD /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:15.956179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6aacf-51ae-47fa-9145-3c2100455d36 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:15.966154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6aacf-51ae-47fa-9145-3c2100455d36 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:15.970855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55afbaa-ed33-4e2b-aab6-6ab2b6c1b3d7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:15.972032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55afbaa-ed33-4e2b-aab6-6ab2b6c1b3d7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:15.972330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55afbaa-ed33-4e2b-aab6-6ab2b6c1b3d7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:15.972819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55afbaa-ed33-4e2b-aab6-6ab2b6c1b3d7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:15.975337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55afbaa-ed33-4e2b-aab6-6ab2b6c1b3d7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=9ff56f0c05ab49039e0ff27f5ae8fbb3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:15.977534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55afbaa-ed33-4e2b-aab6-6ab2b6c1b3d7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:15.986094 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3241/6445] 10.222.0.33 () {58 vars in 1343 bytes} [Sat May 28 21:26:15 2022] GET /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3 => generated 521 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:26:15.986974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55afbaa-ed33-4e2b-aab6-6ab2b6c1b3d7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:15.999292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55afbaa-ed33-4e2b-aab6-6ab2b6c1b3d7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:16.004242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d780c-9973-47d7-b18a-daff61f934d9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:16.005180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d780c-9973-47d7-b18a-daff61f934d9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:16.005467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d780c-9973-47d7-b18a-daff61f934d9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:16.005777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484d780c-9973-47d7-b18a-daff61f934d9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:16.008260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d780c-9973-47d7-b18a-daff61f934d9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=9ff56f0c05ab49039e0ff27f5ae8fbb3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:16.009910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484d780c-9973-47d7-b18a-daff61f934d9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:16.071234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-23b9a3df-d6de-42a8-b801-ee964596dd25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '535ee7ca345b4654a96365391c908ec6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:16.071986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-23b9a3df-d6de-42a8-b801-ee964596dd25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '535ee7ca345b4654a96365391c908ec6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:16.090944 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3205/6446] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:15 2022] DELETE /identity/v3/users/535ee7ca345b4654a96365391c908ec6 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:16.101319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c9192-3486-4fb6-ba51-8e9df95dde44 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:16.101614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c9192-3486-4fb6-ba51-8e9df95dde44 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:16.101931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c9192-3486-4fb6-ba51-8e9df95dde44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:16.121559 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3242/6447] 10.222.0.33 () {58 vars in 1346 bytes} [Sat May 28 21:26:15 2022] DELETE /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:16.127273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d780c-9973-47d7-b18a-daff61f934d9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:16.129410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-484d780c-9973-47d7-b18a-daff61f934d9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:16.144115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0a5c9192-3486-4fb6-ba51-8e9df95dde44 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:16.144115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0a5c9192-3486-4fb6-ba51-8e9df95dde44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:16.212307 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3206/6448] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:26:16.218898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c9192-3486-4fb6-ba51-8e9df95dde44 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:16.219091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484d780c-9973-47d7-b18a-daff61f934d9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:16.220831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0a5c9192-3486-4fb6-ba51-8e9df95dde44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:16.224215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92f372-70b5-48e4-9830-c4170c764f4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:16.227837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92f372-70b5-48e4-9830-c4170c764f4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:16.227837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92f372-70b5-48e4-9830-c4170c764f4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:16.227837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92f372-70b5-48e4-9830-c4170c764f4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:16.228377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92f372-70b5-48e4-9830-c4170c764f4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:get_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=9ff56f0c05ab49039e0ff27f5ae8fbb3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:16.230050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92f372-70b5-48e4-9830-c4170c764f4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:16.251480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-3f92f372-70b5-48e4-9830-c4170c764f4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] 5df3bb3dada94e2e8d1d425c4b107106 does not imply 9ff56f0c05ab49039e0ff27f5ae8fbb3.: keystone.exception.ImpliedRoleNotFound: 5df3bb3dada94e2e8d1d425c4b107106 does not imply 9ff56f0c05ab49039e0ff27f5ae8fbb3. May 28 21:26:16.251480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:26:16.251480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role May 28 21:26:16.251480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = query.one() May 28 21:26:16.251480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2865, in one May 28 21:26:16.251480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._iter().one() May 28 21:26:16.251480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1417, in one May 28 21:26:16.251480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._only_one_row( May 28 21:26:16.251480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row May 28 21:26:16.251480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exc.NoResultFound( May 28 21:26:16.251480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required May 28 21:26:16.251480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:26:16.251480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:26:16.251480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:26:16.251480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:26:16.251480 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:26:16.252822 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:26:16.252822 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:26:16.252822 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:26:16.252822 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:26:16.252822 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:26:16.252822 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:26:16.252822 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:26:16.252822 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:26:16.252822 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:26:16.252822 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 235, in get May 28 21:26:16.252822 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_implied_role(prior_role_id, implied_role_id) May 28 21:26:16.252822 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 239, in _get_implied_role May 28 21:26:16.252822 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application PROVIDERS.role_api.get_implied_role( May 28 21:26:16.252822 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 149, in get_implied_role May 28 21:26:16.252822 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, May 28 21:26:16.252822 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role May 28 21:26:16.254341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( May 28 21:26:16.254341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: 5df3bb3dada94e2e8d1d425c4b107106 does not imply 9ff56f0c05ab49039e0ff27f5ae8fbb3. May 28 21:26:16.254341 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:26:16.257183 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3243/6449] 10.222.0.33 () {58 vars in 1343 bytes} [Sat May 28 21:26:16 2022] GET /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3 => generated 137 bytes in 131 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:26:16.261176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92f372-70b5-48e4-9830-c4170c764f4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:16.282549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92f372-70b5-48e4-9830-c4170c764f4d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:16.290393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e055db3-689b-47b2-b75d-444e4fd476b5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:16.291374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e055db3-689b-47b2-b75d-444e4fd476b5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:16.291810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e055db3-689b-47b2-b75d-444e4fd476b5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:16.292101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e055db3-689b-47b2-b75d-444e4fd476b5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:16.294024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e055db3-689b-47b2-b75d-444e4fd476b5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=9ff56f0c05ab49039e0ff27f5ae8fbb3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:16.296301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e055db3-689b-47b2-b75d-444e4fd476b5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:16.309825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c9192-3486-4fb6-ba51-8e9df95dde44 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:16.314070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37105dec-9581-4816-9292-e0de42391b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:16.314797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37105dec-9581-4816-9292-e0de42391b1c service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:16.315156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37105dec-9581-4816-9292-e0de42391b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:16.315528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37105dec-9581-4816-9292-e0de42391b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:16.318419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-4e055db3-689b-47b2-b75d-444e4fd476b5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] 5df3bb3dada94e2e8d1d425c4b107106 does not imply 9ff56f0c05ab49039e0ff27f5ae8fbb3.: keystone.exception.ImpliedRoleNotFound: 5df3bb3dada94e2e8d1d425c4b107106 does not imply 9ff56f0c05ab49039e0ff27f5ae8fbb3. May 28 21:26:16.318419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:26:16.318419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 105, in _get_implied_role May 28 21:26:16.318419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = query.one() May 28 21:26:16.318419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 2865, in one May 28 21:26:16.318419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._iter().one() May 28 21:26:16.318419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 1417, in one May 28 21:26:16.318419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._only_one_row( May 28 21:26:16.318419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/result.py", line 562, in _only_one_row May 28 21:26:16.318419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exc.NoResultFound( May 28 21:26:16.318419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application sqlalchemy.exc.NoResultFound: No row was found when one was required May 28 21:26:16.318419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:26:16.318419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:26:16.318419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:26:16.318419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:26:16.318419 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:26:16.319776 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:26:16.319776 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:26:16.319776 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:26:16.319776 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:26:16.319776 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:26:16.319776 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:26:16.319776 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:26:16.319776 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:26:16.319776 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:26:16.319776 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 270, in delete May 28 21:26:16.319776 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application PROVIDERS.role_api.delete_implied_role(prior_role_id, implied_role_id) May 28 21:26:16.319776 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:26:16.319776 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:26:16.319776 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1369, in delete_implied_role May 28 21:26:16.319776 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.driver.delete_implied_role(prior_role_id, implied_role_id) May 28 21:26:16.319776 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 129, in delete_implied_role May 28 21:26:16.321156 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = self._get_implied_role(session, prior_role_id, May 28 21:26:16.321156 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 107, in _get_implied_role May 28 21:26:16.321156 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ImpliedRoleNotFound( May 28 21:26:16.321156 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ImpliedRoleNotFound: 5df3bb3dada94e2e8d1d425c4b107106 does not imply 9ff56f0c05ab49039e0ff27f5ae8fbb3. May 28 21:26:16.321156 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:26:16.321156 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3244/6450] 10.222.0.33 () {58 vars in 1346 bytes} [Sat May 28 21:26:16 2022] DELETE /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3 => generated 137 bytes in 59 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:26:16.325151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e055db3-689b-47b2-b75d-444e4fd476b5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:16.329092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37105dec-9581-4816-9292-e0de42391b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:16.330891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37105dec-9581-4816-9292-e0de42391b1c service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:16.340365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e055db3-689b-47b2-b75d-444e4fd476b5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:16.342680 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3207/6451] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:26:16.346207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f728a85-073d-43c4-9bd8-064409fac19d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:16.347385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f728a85-073d-43c4-9bd8-064409fac19d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:16.348654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f728a85-073d-43c4-9bd8-064409fac19d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:16.348654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f728a85-073d-43c4-9bd8-064409fac19d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:16.349910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f728a85-073d-43c4-9bd8-064409fac19d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=9ff56f0c05ab49039e0ff27f5ae8fbb3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:16.352101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f728a85-073d-43c4-9bd8-064409fac19d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:16.434024 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3245/6452] 10.222.0.33 () {60 vars in 1362 bytes} [Sat May 28 21:26:16 2022] PUT /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3 => generated 521 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:16.435370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37105dec-9581-4816-9292-e0de42391b1c service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:16.446368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37105dec-9581-4816-9292-e0de42391b1c service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:16.509577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65045e8-132b-4076-aed7-3e58a4518384 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:16.510532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65045e8-132b-4076-aed7-3e58a4518384 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:16.510820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65045e8-132b-4076-aed7-3e58a4518384 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:16.511221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65045e8-132b-4076-aed7-3e58a4518384 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/5ef04c834f42436b886862d8a3ee6e84` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:16.513027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65045e8-132b-4076-aed7-3e58a4518384 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=5ef04c834f42436b886862d8a3ee6e84)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:16.518482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65045e8-132b-4076-aed7-3e58a4518384 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:16.550816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:16.659736 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3208/6453] 10.222.0.33 () {60 vars in 1362 bytes} [Sat May 28 21:26:16 2022] PUT /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/5ef04c834f42436b886862d8a3ee6e84 => generated 521 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:16.660389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f728a85-073d-43c4-9bd8-064409fac19d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:16.673996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f728a85-073d-43c4-9bd8-064409fac19d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:16.760922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540856cb-9b48-41b3-98a1-8350d6b667d1 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:16.761926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540856cb-9b48-41b3-98a1-8350d6b667d1 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:16.762219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540856cb-9b48-41b3-98a1-8350d6b667d1 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:16.762499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540856cb-9b48-41b3-98a1-8350d6b667d1 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5ef04c834f42436b886862d8a3ee6e84/implies/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:16.764248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540856cb-9b48-41b3-98a1-8350d6b667d1 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=5ef04c834f42436b886862d8a3ee6e84, implied_role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:16.766058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540856cb-9b48-41b3-98a1-8350d6b667d1 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:16.888342 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3246/6454] 10.222.0.33 () {60 vars in 1362 bytes} [Sat May 28 21:26:16 2022] PUT /identity/v3/roles/5ef04c834f42436b886862d8a3ee6e84/implies/0e46a1ddecc44597be78d09c35eb3779 => generated 521 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:16.890321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65045e8-132b-4076-aed7-3e58a4518384 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:16.892079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540856cb-9b48-41b3-98a1-8350d6b667d1 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:16.892615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f65045e8-132b-4076-aed7-3e58a4518384 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:16.965464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65045e8-132b-4076-aed7-3e58a4518384 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:16.977368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540856cb-9b48-41b3-98a1-8350d6b667d1 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:16.982086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e8e76-5846-459f-b0b4-9d7202daaf4f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:16.982758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e8e76-5846-459f-b0b4-9d7202daaf4f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:16.983049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e8e76-5846-459f-b0b4-9d7202daaf4f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:16.983337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e8e76-5846-459f-b0b4-9d7202daaf4f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/role_inferences` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:16.984015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e8e76-5846-459f-b0b4-9d7202daaf4f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:list_role_inference_rules()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:16.985036 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-e48e8e76-5846-459f-b0b4-9d7202daaf4f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_role_inference_rules": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:16.985036 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:26:16.985036 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:26:16.985957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48e8e76-5846-459f-b0b4-9d7202daaf4f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:17.006796 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3247/6455] 10.222.0.33 () {58 vars in 1151 bytes} [Sat May 28 21:26:16 2022] GET /identity/v3/role_inferences => generated 1608 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:17.008196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e8e76-5846-459f-b0b4-9d7202daaf4f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:17.018557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48e8e76-5846-459f-b0b4-9d7202daaf4f tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:17.023453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a16c3-ac2c-412c-8843-51123ea17724 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:17.024406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a16c3-ac2c-412c-8843-51123ea17724 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:17.024695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a16c3-ac2c-412c-8843-51123ea17724 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:17.024876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af486c-d2aa-4181-affe-32230a2db0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:17.025000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863a16c3-ac2c-412c-8843-51123ea17724 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5ef04c834f42436b886862d8a3ee6e84/implies/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:17.025729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af486c-d2aa-4181-affe-32230a2db0a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:17.026055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af486c-d2aa-4181-affe-32230a2db0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:17.026338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af486c-d2aa-4181-affe-32230a2db0a1 admin admin] PATH_INFO: `/v3/projects/178a0db5e64d44e79b19f773932da2e6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:17.026957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a16c3-ac2c-412c-8843-51123ea17724 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=5ef04c834f42436b886862d8a3ee6e84, implied_role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:17.028051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af486c-d2aa-4181-affe-32230a2db0a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=178a0db5e64d44e79b19f773932da2e6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:17.028577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863a16c3-ac2c-412c-8843-51123ea17724 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:17.030459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af486c-d2aa-4181-affe-32230a2db0a1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:17.133127 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3248/6456] 10.222.0.33 () {58 vars in 1346 bytes} [Sat May 28 21:26:17 2022] DELETE /identity/v3/roles/5ef04c834f42436b886862d8a3ee6e84/implies/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:17.137416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a16c3-ac2c-412c-8843-51123ea17724 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:17.151346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-93af486c-d2aa-4181-affe-32230a2db0a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '178a0db5e64d44e79b19f773932da2e6', 'request_id': 'req-93af486c-d2aa-4181-affe-32230a2db0a1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:17.156902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-93af486c-d2aa-4181-affe-32230a2db0a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '178a0db5e64d44e79b19f773932da2e6', 'request_id': 'req-93af486c-d2aa-4181-affe-32230a2db0a1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:17.199942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863a16c3-ac2c-412c-8843-51123ea17724 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:17.268753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8cc08-5d03-4f4b-8d32-1e912383380e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:17.270093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8cc08-5d03-4f4b-8d32-1e912383380e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:17.270424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8cc08-5d03-4f4b-8d32-1e912383380e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:17.270714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8cc08-5d03-4f4b-8d32-1e912383380e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/5ef04c834f42436b886862d8a3ee6e84` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:17.274465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8cc08-5d03-4f4b-8d32-1e912383380e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=5ef04c834f42436b886862d8a3ee6e84)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:17.276190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee8cc08-5d03-4f4b-8d32-1e912383380e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:17.298052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-93af486c-d2aa-4181-affe-32230a2db0a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '178a0db5e64d44e79b19f773932da2e6', 'request_id': 'req-93af486c-d2aa-4181-affe-32230a2db0a1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:17.325719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-93af486c-d2aa-4181-affe-32230a2db0a1 admin admin] The token cache is being invalidate because project 178a0db5e64d44e79b19f773932da2e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:17.326964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-93af486c-d2aa-4181-affe-32230a2db0a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:17.330059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-93af486c-d2aa-4181-affe-32230a2db0a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:17.334007 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3209/6457] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:26:16 2022] DELETE /identity/v3/projects/178a0db5e64d44e79b19f773932da2e6 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:17.341327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204b16e-d580-40cf-ac1c-2ebb2f4f2a77 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:17.341897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204b16e-d580-40cf-ac1c-2ebb2f4f2a77 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:17.342812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5204b16e-d580-40cf-ac1c-2ebb2f4f2a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:17.350238 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3249/6458] 10.222.0.33 () {58 vars in 1346 bytes} [Sat May 28 21:26:17 2022] DELETE /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/5ef04c834f42436b886862d8a3ee6e84 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:17.360647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8cc08-5d03-4f4b-8d32-1e912383380e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:17.364987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fee8cc08-5d03-4f4b-8d32-1e912383380e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:17.398581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5204b16e-d580-40cf-ac1c-2ebb2f4f2a77 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:17.400130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5204b16e-d580-40cf-ac1c-2ebb2f4f2a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:17.465039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:17.480408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8cc08-5d03-4f4b-8d32-1e912383380e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:17.485558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828fb36-afe6-4235-b5a3-75e642176622 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:17.486509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828fb36-afe6-4235-b5a3-75e642176622 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:17.486950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828fb36-afe6-4235-b5a3-75e642176622 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:17.487247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828fb36-afe6-4235-b5a3-75e642176622 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:17.489008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2828fb36-afe6-4235-b5a3-75e642176622 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=9ff56f0c05ab49039e0ff27f5ae8fbb3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:17.490676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2828fb36-afe6-4235-b5a3-75e642176622 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:17.497481 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3210/6459] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:17.505161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fbffc-c74f-4feb-891f-7924fa8ad996 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:17.505478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fbffc-c74f-4feb-891f-7924fa8ad996 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:17.505758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243fbffc-c74f-4feb-891f-7924fa8ad996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:17.524532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-243fbffc-c74f-4feb-891f-7924fa8ad996 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:17.525744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-243fbffc-c74f-4feb-891f-7924fa8ad996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:17.533677 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3211/6460] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:26:17.537773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fbffc-c74f-4feb-891f-7924fa8ad996 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:17.547302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243fbffc-c74f-4feb-891f-7924fa8ad996 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:17.551890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff16fd-f05a-432e-872a-c4eb341235c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:17.552521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff16fd-f05a-432e-872a-c4eb341235c7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:17.552808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff16fd-f05a-432e-872a-c4eb341235c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:17.553088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff16fd-f05a-432e-872a-c4eb341235c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:17.553908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff16fd-f05a-432e-872a-c4eb341235c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:17.558541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff16fd-f05a-432e-872a-c4eb341235c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:17.558541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ff16fd-f05a-432e-872a-c4eb341235c7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:17.567148 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3212/6461] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:26:17.571146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff16fd-f05a-432e-872a-c4eb341235c7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:17.580316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff16fd-f05a-432e-872a-c4eb341235c7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:17.584910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37eda24-44b7-4c5a-b873-07cf2108e31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:17.586090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37eda24-44b7-4c5a-b873-07cf2108e31e admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:17.586379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37eda24-44b7-4c5a-b873-07cf2108e31e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:17.586658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37eda24-44b7-4c5a-b873-07cf2108e31e admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:17.587359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37eda24-44b7-4c5a-b873-07cf2108e31e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:17.589181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37eda24-44b7-4c5a-b873-07cf2108e31e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:17.604959 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3250/6462] 10.222.0.33 () {58 vars in 1346 bytes} [Sat May 28 21:26:17 2022] DELETE /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:17.610074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828fb36-afe6-4235-b5a3-75e642176622 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:17.672287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828fb36-afe6-4235-b5a3-75e642176622 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:17.677028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3b14b-2906-4ab6-b8db-4724153ae410 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:17.677725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3b14b-2906-4ab6-b8db-4724153ae410 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:17.678059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3b14b-2906-4ab6-b8db-4724153ae410 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:17.678345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3b14b-2906-4ab6-b8db-4724153ae410 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:17.679079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3b14b-2906-4ab6-b8db-4724153ae410 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:17.680675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3b14b-2906-4ab6-b8db-4724153ae410 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:17.692251 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3251/6463] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:17 2022] GET /identity/v3/roles => generated 2324 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:26:17.698165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3b14b-2906-4ab6-b8db-4724153ae410 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:17.708107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3b14b-2906-4ab6-b8db-4724153ae410 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:17.714365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952cc32-d6bb-47ec-9b12-5b3b35cd44b2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:17.715084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952cc32-d6bb-47ec-9b12-5b3b35cd44b2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:17.715380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952cc32-d6bb-47ec-9b12-5b3b35cd44b2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:17.715666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952cc32-d6bb-47ec-9b12-5b3b35cd44b2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:17.716455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952cc32-d6bb-47ec-9b12-5b3b35cd44b2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:17.718106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952cc32-d6bb-47ec-9b12-5b3b35cd44b2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:17.769648 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3213/6464] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:17 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:17.774250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37eda24-44b7-4c5a-b873-07cf2108e31e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:17.796242 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3252/6465] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:26:17 2022] POST /identity/v3/roles => generated 234 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:17.799017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952cc32-d6bb-47ec-9b12-5b3b35cd44b2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:17.813284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952cc32-d6bb-47ec-9b12-5b3b35cd44b2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:17.841151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37eda24-44b7-4c5a-b873-07cf2108e31e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:17.846297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e5e61-6f44-412d-bfe5-3177bb2cf13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:17.847234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e5e61-6f44-412d-bfe5-3177bb2cf13b admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:17.847524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e5e61-6f44-412d-bfe5-3177bb2cf13b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:17.847805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e5e61-6f44-412d-bfe5-3177bb2cf13b admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:17.848514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e5e61-6f44-412d-bfe5-3177bb2cf13b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:17.850521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e5e61-6f44-412d-bfe5-3177bb2cf13b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:17.871106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701aff81-d241-4779-9366-32561eafacb8 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:17.871968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701aff81-d241-4779-9366-32561eafacb8 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:17.872268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701aff81-d241-4779-9366-32561eafacb8 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:17.872535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701aff81-d241-4779-9366-32561eafacb8 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/dbd7834ada014cf199896f01ec2dc021` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:17.874139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701aff81-d241-4779-9366-32561eafacb8 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:update_role(role_id=dbd7834ada014cf199896f01ec2dc021)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:17.875314 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-701aff81-d241-4779-9366-32561eafacb8 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_role": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:17.875314 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:26:17.875314 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:26:17.876205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701aff81-d241-4779-9366-32561eafacb8 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:17.887714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:17.933522 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3214/6466] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:17 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:17.937534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e5e61-6f44-412d-bfe5-3177bb2cf13b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:17.948027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e5e61-6f44-412d-bfe5-3177bb2cf13b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:17.952879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b255599-07d2-4476-bca0-0dac9063f5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:17.953545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b255599-07d2-4476-bca0-0dac9063f5a0 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:17.953856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b255599-07d2-4476-bca0-0dac9063f5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:17.954146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b255599-07d2-4476-bca0-0dac9063f5a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:17.954873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b255599-07d2-4476-bca0-0dac9063f5a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:17.956488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b255599-07d2-4476-bca0-0dac9063f5a0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:17.958320 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3253/6467] 10.222.0.33 () {60 vars in 1242 bytes} [Sat May 28 21:26:17 2022] PATCH /identity/v3/roles/dbd7834ada014cf199896f01ec2dc021 => generated 237 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:26:17.965057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701aff81-d241-4779-9366-32561eafacb8 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:17.971083 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3215/6468] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:17 2022] GET /identity/v3/roles => generated 2552 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:26:17.975507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b255599-07d2-4476-bca0-0dac9063f5a0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:17.975686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701aff81-d241-4779-9366-32561eafacb8 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:17.980636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077898ab-a4f6-433c-b9e2-52f6751017c9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:17.981511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077898ab-a4f6-433c-b9e2-52f6751017c9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:17.981803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077898ab-a4f6-433c-b9e2-52f6751017c9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:17.982121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077898ab-a4f6-433c-b9e2-52f6751017c9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/dbd7834ada014cf199896f01ec2dc021` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:17.986166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b255599-07d2-4476-bca0-0dac9063f5a0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:17.990928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00ac9f-f8bf-4019-955f-a574a38a6597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:17.991198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077898ab-a4f6-433c-b9e2-52f6751017c9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:get_role(role_id=dbd7834ada014cf199896f01ec2dc021)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:17.992288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00ac9f-f8bf-4019-955f-a574a38a6597 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:17.992580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00ac9f-f8bf-4019-955f-a574a38a6597 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:17.992763 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-077898ab-a4f6-433c-b9e2-52f6751017c9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_role": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:17.992763 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:26:17.992763 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:26:17.993076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a00ac9f-f8bf-4019-955f-a574a38a6597 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:17.993563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00ac9f-f8bf-4019-955f-a574a38a6597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:17.993674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077898ab-a4f6-433c-b9e2-52f6751017c9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:17.998054 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3254/6469] 10.222.0.33 () {58 vars in 1220 bytes} [Sat May 28 21:26:17 2022] GET /identity/v3/roles/dbd7834ada014cf199896f01ec2dc021 => generated 237 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:26:17.998196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a00ac9f-f8bf-4019-955f-a574a38a6597 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:17.998966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077898ab-a4f6-433c-b9e2-52f6751017c9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:18.011289 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3216/6470] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:17 2022] GET /identity/v3/roles => generated 2552 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:18.011618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077898ab-a4f6-433c-b9e2-52f6751017c9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:18.012241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00ac9f-f8bf-4019-955f-a574a38a6597 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:18.015169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6d034-2ab8-4e60-9397-46e9b9f8fb66 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:18.015850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6d034-2ab8-4e60-9397-46e9b9f8fb66 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:18.016141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6d034-2ab8-4e60-9397-46e9b9f8fb66 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:18.016428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6d034-2ab8-4e60-9397-46e9b9f8fb66 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:18.017161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6d034-2ab8-4e60-9397-46e9b9f8fb66 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:18.018833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6d034-2ab8-4e60-9397-46e9b9f8fb66 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:18.021862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a00ac9f-f8bf-4019-955f-a574a38a6597 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:18.027086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bfe18a-5f81-40b7-9f4c-f1a4f27b9c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:18.028048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfe18a-5f81-40b7-9f4c-f1a4f27b9c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:18.030693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfe18a-5f81-40b7-9f4c-f1a4f27b9c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:18.030693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bfe18a-5f81-40b7-9f4c-f1a4f27b9c0d admin admin] PATH_INFO: `/v3/projects/2cc7b77a514e46ffbd027af670ce667f/users/7371c0456d29408b8523989c0a1d75ba/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:18.031414 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3255/6471] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:17 2022] GET /identity/v3/roles => generated 2552 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:18.035541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6d034-2ab8-4e60-9397-46e9b9f8fb66 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:18.043871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfe18a-5f81-40b7-9f4c-f1a4f27b9c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc7b77a514e46ffbd027af670ce667f, user_id=7371c0456d29408b8523989c0a1d75ba, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:18.048750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bfe18a-5f81-40b7-9f4c-f1a4f27b9c0d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:18.048870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6d034-2ab8-4e60-9397-46e9b9f8fb66 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:18.055389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5b0a5-6d0f-43f8-b7d5-60d44703cae2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:18.056536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5b0a5-6d0f-43f8-b7d5-60d44703cae2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:18.056843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5b0a5-6d0f-43f8-b7d5-60d44703cae2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:18.057328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5b0a5-6d0f-43f8-b7d5-60d44703cae2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/dbd7834ada014cf199896f01ec2dc021` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:18.059431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5b0a5-6d0f-43f8-b7d5-60d44703cae2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_role(role_id=dbd7834ada014cf199896f01ec2dc021)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:18.061298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5b0a5-6d0f-43f8-b7d5-60d44703cae2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:18.123198 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3217/6472] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:18 2022] PUT /identity/v3/projects/2cc7b77a514e46ffbd027af670ce667f/users/7371c0456d29408b8523989c0a1d75ba/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:18.133962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827aaf8f-8d0a-49f8-bd56-c13314475261 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:18.134259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827aaf8f-8d0a-49f8-bd56-c13314475261 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:18.134548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827aaf8f-8d0a-49f8-bd56-c13314475261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:18.161604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-827aaf8f-8d0a-49f8-bd56-c13314475261 None None] MFA Rules not processed for user `7371c0456d29408b8523989c0a1d75ba`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:18.162864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-827aaf8f-8d0a-49f8-bd56-c13314475261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:18.232657 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3218/6473] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:26:18.237144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827aaf8f-8d0a-49f8-bd56-c13314475261 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:18.263218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ded5b0a5-6d0f-43f8-b7d5-60d44703cae2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invalidating the token cache because role dbd7834ada014cf199896f01ec2dc021 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:18.263530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ded5b0a5-6d0f-43f8-b7d5-60d44703cae2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:18.264388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ded5b0a5-6d0f-43f8-b7d5-60d44703cae2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:18.266570 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3256/6474] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:18 2022] DELETE /identity/v3/roles/dbd7834ada014cf199896f01ec2dc021 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:18.271409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5b0a5-6d0f-43f8-b7d5-60d44703cae2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:18.272840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ded5b0a5-6d0f-43f8-b7d5-60d44703cae2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:18.306001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827aaf8f-8d0a-49f8-bd56-c13314475261 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:18.344012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5b0a5-6d0f-43f8-b7d5-60d44703cae2 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:18.350217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9e275-ec89-49f0-af00-c51b2ecfbe34 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:18.351457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9e275-ec89-49f0-af00-c51b2ecfbe34 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:18.351748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9e275-ec89-49f0-af00-c51b2ecfbe34 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:18.352259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9e275-ec89-49f0-af00-c51b2ecfbe34 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:18.354084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9e275-ec89-49f0-af00-c51b2ecfbe34 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=9ff56f0c05ab49039e0ff27f5ae8fbb3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:18.355700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9e275-ec89-49f0-af00-c51b2ecfbe34 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:18.369445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0cc87d-de46-4394-aa02-519797298567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:18.370261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cc87d-de46-4394-aa02-519797298567 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:18.370532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cc87d-de46-4394-aa02-519797298567 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:18.370816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cc87d-de46-4394-aa02-519797298567 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:18.371510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0cc87d-de46-4394-aa02-519797298567 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:18.373504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0cc87d-de46-4394-aa02-519797298567 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:18.444195 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3257/6475] 10.222.0.33 () {60 vars in 1362 bytes} [Sat May 28 21:26:18 2022] PUT /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3 => generated 521 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:18.446540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:18.449068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9e275-ec89-49f0-af00-c51b2ecfbe34 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:18.454262 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3219/6476] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:18 2022] POST /identity/v3/projects => generated 383 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:18.458602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0cc87d-de46-4394-aa02-519797298567 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:18.467441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0f0cc87d-de46-4394-aa02-519797298567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:18.540522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9e275-ec89-49f0-af00-c51b2ecfbe34 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:18.573472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0cc87d-de46-4394-aa02-519797298567 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:18.578666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25ece4-0e5e-4d4f-8888-a3399db400f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:18.579435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25ece4-0e5e-4d4f-8888-a3399db400f8 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:18.579910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25ece4-0e5e-4d4f-8888-a3399db400f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:18.580261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e25ece4-0e5e-4d4f-8888-a3399db400f8 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:18.581080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25ece4-0e5e-4d4f-8888-a3399db400f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:18.583597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e25ece4-0e5e-4d4f-8888-a3399db400f8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:18.612595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbd675-f91c-4352-947b-0ec4ef7690b4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:18.613256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbd675-f91c-4352-947b-0ec4ef7690b4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:18.613549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbd675-f91c-4352-947b-0ec4ef7690b4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:18.613983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbbd675-f91c-4352-947b-0ec4ef7690b4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/5ef04c834f42436b886862d8a3ee6e84` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:18.615677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbd675-f91c-4352-947b-0ec4ef7690b4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=5ef04c834f42436b886862d8a3ee6e84)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:18.617898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbbd675-f91c-4352-947b-0ec4ef7690b4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:18.693582 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3220/6477] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:18 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:18.697370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25ece4-0e5e-4d4f-8888-a3399db400f8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:18.706611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e25ece4-0e5e-4d4f-8888-a3399db400f8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:18.712098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cca98-0f0f-4a00-a309-c7e506c59212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:18.712791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cca98-0f0f-4a00-a309-c7e506c59212 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:18.713074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cca98-0f0f-4a00-a309-c7e506c59212 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:18.713357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cca98-0f0f-4a00-a309-c7e506c59212 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:18.714127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cca98-0f0f-4a00-a309-c7e506c59212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:18.715876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cca98-0f0f-4a00-a309-c7e506c59212 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:18.716398 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3258/6478] 10.222.0.33 () {60 vars in 1362 bytes} [Sat May 28 21:26:18 2022] PUT /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/5ef04c834f42436b886862d8a3ee6e84 => generated 521 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:18.720260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbd675-f91c-4352-947b-0ec4ef7690b4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:18.729368 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3221/6479] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:18 2022] GET /identity/v3/roles => generated 2324 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:26:18.733181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cca98-0f0f-4a00-a309-c7e506c59212 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:18.744014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cca98-0f0f-4a00-a309-c7e506c59212 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:18.783553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbbd675-f91c-4352-947b-0ec4ef7690b4 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:18.789137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c467f-d123-4531-8f10-0f5d5309ad2d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:18.790107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c467f-d123-4531-8f10-0f5d5309ad2d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:18.790401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c467f-d123-4531-8f10-0f5d5309ad2d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:18.790682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c467f-d123-4531-8f10-0f5d5309ad2d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5ef04c834f42436b886862d8a3ee6e84/implies/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:18.792438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c467f-d123-4531-8f10-0f5d5309ad2d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:create_implied_role(prior_role_id=5ef04c834f42436b886862d8a3ee6e84, implied_role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:18.794274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c467f-d123-4531-8f10-0f5d5309ad2d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:18.800530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d08ee2-f2f2-4419-bdba-9d330d4184fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:18.801194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d08ee2-f2f2-4419-bdba-9d330d4184fc admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:18.801481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d08ee2-f2f2-4419-bdba-9d330d4184fc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:18.801758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d08ee2-f2f2-4419-bdba-9d330d4184fc admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:18.802545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d08ee2-f2f2-4419-bdba-9d330d4184fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:18.804216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d08ee2-f2f2-4419-bdba-9d330d4184fc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:18.819926 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3222/6480] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:18 2022] GET /identity/v3/roles => generated 2324 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:26:18.820837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d08ee2-f2f2-4419-bdba-9d330d4184fc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:18.830771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d08ee2-f2f2-4419-bdba-9d330d4184fc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:18.836057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a503d-9f5f-4f4c-af50-e3ab7a84fc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:18.836939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a503d-9f5f-4f4c-af50-e3ab7a84fc6f admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:18.838988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a503d-9f5f-4f4c-af50-e3ab7a84fc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:18.838988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a503d-9f5f-4f4c-af50-e3ab7a84fc6f admin admin] PATH_INFO: `/v3/projects/82cf103697074afcb641b3da49f5761a/users/23e706bdbc8c4423b2616997c328f737/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:18.852890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a503d-9f5f-4f4c-af50-e3ab7a84fc6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=82cf103697074afcb641b3da49f5761a, user_id=23e706bdbc8c4423b2616997c328f737, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:18.855668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a503d-9f5f-4f4c-af50-e3ab7a84fc6f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:18.893434 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3259/6481] 10.222.0.33 () {60 vars in 1362 bytes} [Sat May 28 21:26:18 2022] PUT /identity/v3/roles/5ef04c834f42436b886862d8a3ee6e84/implies/0e46a1ddecc44597be78d09c35eb3779 => generated 521 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:18.897437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c467f-d123-4531-8f10-0f5d5309ad2d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:18.946022 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3223/6482] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:18 2022] PUT /identity/v3/projects/82cf103697074afcb641b3da49f5761a/users/23e706bdbc8c4423b2616997c328f737/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:18.958499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787199a-7553-4a89-8cef-650fe5cbf670 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:18.958850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787199a-7553-4a89-8cef-650fe5cbf670 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:18.959063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787199a-7553-4a89-8cef-650fe5cbf670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:18.967319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c467f-d123-4531-8f10-0f5d5309ad2d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:18.988225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5787199a-7553-4a89-8cef-650fe5cbf670 None None] MFA Rules not processed for user `23e706bdbc8c4423b2616997c328f737`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:18.990112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5787199a-7553-4a89-8cef-650fe5cbf670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:19.024680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41910969-db05-47b9-820a-c6ab4550f89e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:19.025638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41910969-db05-47b9-820a-c6ab4550f89e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:19.025960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41910969-db05-47b9-820a-c6ab4550f89e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:19.026271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41910969-db05-47b9-820a-c6ab4550f89e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5ef04c834f42436b886862d8a3ee6e84/implies` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:19.027522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41910969-db05-47b9-820a-c6ab4550f89e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=5ef04c834f42436b886862d8a3ee6e84)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:19.028547 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-41910969-db05-47b9-820a-c6ab4550f89e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_implied_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:26:19.028547 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:26:19.028547 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:26:19.029441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41910969-db05-47b9-820a-c6ab4550f89e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:19.038208 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3260/6483] 10.222.0.33 () {58 vars in 1244 bytes} [Sat May 28 21:26:18 2022] GET /identity/v3/roles/5ef04c834f42436b886862d8a3ee6e84/implies => generated 490 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:26:19.039837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41910969-db05-47b9-820a-c6ab4550f89e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:19.049461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41910969-db05-47b9-820a-c6ab4550f89e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:19.054443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bcc8d-97c7-4b35-a5d6-0a230cd9ed76 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:19.054606 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3224/6484] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:26:19.058676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bcc8d-97c7-4b35-a5d6-0a230cd9ed76 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:19.058972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bcc8d-97c7-4b35-a5d6-0a230cd9ed76 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:19.059189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787199a-7553-4a89-8cef-650fe5cbf670 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:19.059332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3bcc8d-97c7-4b35-a5d6-0a230cd9ed76 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:19.060460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3bcc8d-97c7-4b35-a5d6-0a230cd9ed76 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:list_implied_roles(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:19.062114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3bcc8d-97c7-4b35-a5d6-0a230cd9ed76 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:19.071858 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3261/6485] 10.222.0.33 () {58 vars in 1244 bytes} [Sat May 28 21:26:19 2022] GET /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies => generated 661 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:19.076395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bcc8d-97c7-4b35-a5d6-0a230cd9ed76 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:19.087459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3bcc8d-97c7-4b35-a5d6-0a230cd9ed76 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:19.093445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161ea2f-4d22-42e2-b409-481129843cac tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:19.094405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161ea2f-4d22-42e2-b409-481129843cac tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:19.094825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161ea2f-4d22-42e2-b409-481129843cac tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:19.095114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161ea2f-4d22-42e2-b409-481129843cac tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5ef04c834f42436b886862d8a3ee6e84/implies/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:19.097306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161ea2f-4d22-42e2-b409-481129843cac tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=5ef04c834f42436b886862d8a3ee6e84, implied_role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:19.098975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161ea2f-4d22-42e2-b409-481129843cac tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:19.125448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787199a-7553-4a89-8cef-650fe5cbf670 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:19.130583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f9579-42cd-4eed-9d83-ab8ac1fdbc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:19.131252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f9579-42cd-4eed-9d83-ab8ac1fdbc49 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:19.131571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f9579-42cd-4eed-9d83-ab8ac1fdbc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:19.132454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70f9579-42cd-4eed-9d83-ab8ac1fdbc49 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:19.133226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f9579-42cd-4eed-9d83-ab8ac1fdbc49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:19.135220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70f9579-42cd-4eed-9d83-ab8ac1fdbc49 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:19.168311 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3262/6486] 10.222.0.33 () {58 vars in 1346 bytes} [Sat May 28 21:26:19 2022] DELETE /identity/v3/roles/5ef04c834f42436b886862d8a3ee6e84/implies/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:19.172268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161ea2f-4d22-42e2-b409-481129843cac tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:19.212829 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3225/6487] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:19 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:19.214224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f9579-42cd-4eed-9d83-ab8ac1fdbc49 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:19.235821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161ea2f-4d22-42e2-b409-481129843cac tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:19.285602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70f9579-42cd-4eed-9d83-ab8ac1fdbc49 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:19.291989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739cf71-6646-42fa-b083-58f6485655bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:19.292666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739cf71-6646-42fa-b083-58f6485655bd admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:19.292958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739cf71-6646-42fa-b083-58f6485655bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:19.293241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a739cf71-6646-42fa-b083-58f6485655bd admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:19.294036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739cf71-6646-42fa-b083-58f6485655bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:19.294155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c2d95-f64a-4b4a-b6d3-55a1448c8db0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:19.295002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c2d95-f64a-4b4a-b6d3-55a1448c8db0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:19.295296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c2d95-f64a-4b4a-b6d3-55a1448c8db0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:19.295584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c2d95-f64a-4b4a-b6d3-55a1448c8db0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/5ef04c834f42436b886862d8a3ee6e84` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:19.295976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a739cf71-6646-42fa-b083-58f6485655bd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:19.297334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c2d95-f64a-4b4a-b6d3-55a1448c8db0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=5ef04c834f42436b886862d8a3ee6e84)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:19.299095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c2d95-f64a-4b4a-b6d3-55a1448c8db0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:19.408114 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3263/6488] 10.222.0.33 () {58 vars in 1346 bytes} [Sat May 28 21:26:19 2022] DELETE /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/5ef04c834f42436b886862d8a3ee6e84 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:19.412416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c2d95-f64a-4b4a-b6d3-55a1448c8db0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:19.454914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:19.461892 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3226/6489] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:19 2022] POST /identity/v3/users => generated 296 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:19.465772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739cf71-6646-42fa-b083-58f6485655bd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:19.473801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c2d95-f64a-4b4a-b6d3-55a1448c8db0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:19.476184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a739cf71-6646-42fa-b083-58f6485655bd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:19.478849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd00d8b-359c-42a8-a9e3-7334916ce61e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:19.479805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd00d8b-359c-42a8-a9e3-7334916ce61e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:19.485154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd00d8b-359c-42a8-a9e3-7334916ce61e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:19.485154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd00d8b-359c-42a8-a9e3-7334916ce61e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:19.485154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd00d8b-359c-42a8-a9e3-7334916ce61e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_implied_role(prior_role_id=5df3bb3dada94e2e8d1d425c4b107106, implied_role_id=9ff56f0c05ab49039e0ff27f5ae8fbb3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:19.485154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd00d8b-359c-42a8-a9e3-7334916ce61e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:19.533007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e095e-fb02-488a-81d7-8bc7035e301a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:19.533682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e095e-fb02-488a-81d7-8bc7035e301a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:19.539973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e095e-fb02-488a-81d7-8bc7035e301a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:19.539973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e095e-fb02-488a-81d7-8bc7035e301a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:19.539973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e095e-fb02-488a-81d7-8bc7035e301a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:19.539973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666e095e-fb02-488a-81d7-8bc7035e301a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:19.551073 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3227/6490] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:19 2022] GET /identity/v3/roles => generated 2324 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:26:19.552110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e095e-fb02-488a-81d7-8bc7035e301a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:19.563137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e095e-fb02-488a-81d7-8bc7035e301a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:19.568347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41741f-a36b-472e-9cb9-3a5ea0e6567e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:19.569314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41741f-a36b-472e-9cb9-3a5ea0e6567e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:19.570195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41741f-a36b-472e-9cb9-3a5ea0e6567e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:19.570614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41741f-a36b-472e-9cb9-3a5ea0e6567e admin admin] PATH_INFO: `/v3/projects/62a17c0b6f9d41d0a2c29db7cd06ee92/users/b97daa65b17c4d93a668db600666e37d/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:19.585597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41741f-a36b-472e-9cb9-3a5ea0e6567e admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a17c0b6f9d41d0a2c29db7cd06ee92, user_id=b97daa65b17c4d93a668db600666e37d, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:19.588311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41741f-a36b-472e-9cb9-3a5ea0e6567e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:19.603554 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3264/6491] 10.222.0.33 () {58 vars in 1346 bytes} [Sat May 28 21:26:19 2022] DELETE /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106/implies/9ff56f0c05ab49039e0ff27f5ae8fbb3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:19.603554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd00d8b-359c-42a8-a9e3-7334916ce61e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:19.613534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:19.669667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd00d8b-359c-42a8-a9e3-7334916ce61e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:19.675128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419c9c5-4fb8-4b8e-9aeb-91497f8a7c9e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:19.676172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419c9c5-4fb8-4b8e-9aeb-91497f8a7c9e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:19.676536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419c9c5-4fb8-4b8e-9aeb-91497f8a7c9e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:19.676877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419c9c5-4fb8-4b8e-9aeb-91497f8a7c9e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/domains/d99c791bb3ab4a2eb2b67301f993b944` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:19.684684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419c9c5-4fb8-4b8e-9aeb-91497f8a7c9e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:26:19.694782 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-d419c9c5-4fb8-4b8e-9aeb-91497f8a7c9e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Could not find domain: d99c791bb3ab4a2eb2b67301f993b944.: keystone.exception.DomainNotFound: Could not find domain: d99c791bb3ab4a2eb2b67301f993b944. May 28 21:26:19.694782 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:26:19.694782 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 21:26:19.694782 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 21:26:19.694782 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:26:19.694782 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:26:19.694782 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:26:19.694782 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:26:19.694782 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: d99c791bb3ab4a2eb2b67301f993b944. May 28 21:26:19.694782 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:26:19.694782 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:26:19.694782 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:26:19.694782 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:26:19.694782 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:26:19.694782 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:26:19.694782 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:26:19.696595 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:26:19.696595 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:26:19.696595 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:26:19.696595 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:26:19.696595 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:26:19.696595 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:26:19.696595 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:26:19.696595 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch May 28 21:26:19.696595 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') May 28 21:26:19.696595 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:26:19.696595 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:26:19.696595 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:26:19.696595 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:26:19.696595 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:26:19.696595 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:26:19.696595 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:26:19.697910 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:26:19.697910 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:26:19.697910 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:26:19.697910 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:26:19.697910 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 21:26:19.697910 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:26:19.697910 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 21:26:19.697910 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:26:19.697910 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:26:19.697910 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:26:19.697910 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 21:26:19.697910 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:26:19.697910 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 21:26:19.697910 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 21:26:19.697910 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 21:26:19.697910 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: d99c791bb3ab4a2eb2b67301f993b944. May 28 21:26:19.697910 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:26:19.699054 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3265/6492] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:26:19 2022] PATCH /identity/v3/domains/d99c791bb3ab4a2eb2b67301f993b944 => generated 112 bytes in 95 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) May 28 21:26:19.700902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419c9c5-4fb8-4b8e-9aeb-91497f8a7c9e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:19.707505 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3228/6493] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:19 2022] PUT /identity/v3/projects/62a17c0b6f9d41d0a2c29db7cd06ee92/users/b97daa65b17c4d93a668db600666e37d/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:19.711906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419c9c5-4fb8-4b8e-9aeb-91497f8a7c9e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:19.714323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41741f-a36b-472e-9cb9-3a5ea0e6567e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:19.787056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ae2dc-3b89-4bcb-b875-eb58cb62eba0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:19.788467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ae2dc-3b89-4bcb-b875-eb58cb62eba0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:19.788864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ae2dc-3b89-4bcb-b875-eb58cb62eba0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:19.789145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41741f-a36b-472e-9cb9-3a5ea0e6567e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:19.789341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ae2dc-3b89-4bcb-b875-eb58cb62eba0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/users/1f04e663b88b4151ae962359fbc2451d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:19.791415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ae2dc-3b89-4bcb-b875-eb58cb62eba0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_user(user_id=1f04e663b88b4151ae962359fbc2451d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:19.793083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ae2dc-3b89-4bcb-b875-eb58cb62eba0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:19.796423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82e249-9743-4d86-9cc9-c9786c3270a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:19.797177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82e249-9743-4d86-9cc9-c9786c3270a5 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:19.797552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82e249-9743-4d86-9cc9-c9786c3270a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:19.797916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82e249-9743-4d86-9cc9-c9786c3270a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:19.799576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f82e249-9743-4d86-9cc9-c9786c3270a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:19.801044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f82e249-9743-4d86-9cc9-c9786c3270a5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:19.813824 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3229/6494] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:19 2022] GET /identity/v3/roles => generated 2324 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:26:19.817882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82e249-9743-4d86-9cc9-c9786c3270a5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:19.830133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82e249-9743-4d86-9cc9-c9786c3270a5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:19.835518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e72e7dd-fc87-4542-8c9c-a9638d096a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:19.836450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72e7dd-fc87-4542-8c9c-a9638d096a50 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:19.836742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72e7dd-fc87-4542-8c9c-a9638d096a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:19.837023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e72e7dd-fc87-4542-8c9c-a9638d096a50 admin admin] PATH_INFO: `/v3/projects/62a17c0b6f9d41d0a2c29db7cd06ee92/users/b97daa65b17c4d93a668db600666e37d/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:19.839075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72e7dd-fc87-4542-8c9c-a9638d096a50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a17c0b6f9d41d0a2c29db7cd06ee92, user_id=b97daa65b17c4d93a668db600666e37d, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:19.841777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e72e7dd-fc87-4542-8c9c-a9638d096a50 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:19.859036 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3230/6495] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:19 2022] PUT /identity/v3/projects/62a17c0b6f9d41d0a2c29db7cd06ee92/users/b97daa65b17c4d93a668db600666e37d/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:19.877902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9044f-ea37-4cd1-ad54-a0c1c7aaa49c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:19.878196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9044f-ea37-4cd1-ad54-a0c1c7aaa49c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:19.878476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb9044f-ea37-4cd1-ad54-a0c1c7aaa49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:19.905877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-511ae2dc-3b89-4bcb-b875-eb58cb62eba0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f04e663b88b4151ae962359fbc2451d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:19.909774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bbb9044f-ea37-4cd1-ad54-a0c1c7aaa49c None None] MFA Rules not processed for user `b97daa65b17c4d93a668db600666e37d`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:19.911044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bbb9044f-ea37-4cd1-ad54-a0c1c7aaa49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:19.915279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-511ae2dc-3b89-4bcb-b875-eb58cb62eba0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f04e663b88b4151ae962359fbc2451d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:19.915981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-511ae2dc-3b89-4bcb-b875-eb58cb62eba0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f04e663b88b4151ae962359fbc2451d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:19.916676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-511ae2dc-3b89-4bcb-b875-eb58cb62eba0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f04e663b88b4151ae962359fbc2451d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:19.984818 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3231/6496] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:19 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:26:19.995589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9044f-ea37-4cd1-ad54-a0c1c7aaa49c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:19.997022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bbb9044f-ea37-4cd1-ad54-a0c1c7aaa49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:20.018744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-511ae2dc-3b89-4bcb-b875-eb58cb62eba0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f04e663b88b4151ae962359fbc2451d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:20.038010 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3266/6497] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:19 2022] DELETE /identity/v3/users/1f04e663b88b4151ae962359fbc2451d => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:20.042448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ae2dc-3b89-4bcb-b875-eb58cb62eba0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:20.043856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-511ae2dc-3b89-4bcb-b875-eb58cb62eba0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:20.062295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb9044f-ea37-4cd1-ad54-a0c1c7aaa49c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:20.115891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ae2dc-3b89-4bcb-b875-eb58cb62eba0 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:20.117925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0c087-3216-482c-a9c0-b8085884ed9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:20.118595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0c087-3216-482c-a9c0-b8085884ed9f service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:20.118883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0c087-3216-482c-a9c0-b8085884ed9f service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:20.119161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0c087-3216-482c-a9c0-b8085884ed9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:20.121440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-94b0c087-3216-482c-a9c0-b8085884ed9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:20.122272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55449a-76e4-4fec-8f95-e57fb2bcb192 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:20.123280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55449a-76e4-4fec-8f95-e57fb2bcb192 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:20.123578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55449a-76e4-4fec-8f95-e57fb2bcb192 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:20.123926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55449a-76e4-4fec-8f95-e57fb2bcb192 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/0e46a1ddecc44597be78d09c35eb3779` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:20.125518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55449a-76e4-4fec-8f95-e57fb2bcb192 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_role(role_id=0e46a1ddecc44597be78d09c35eb3779)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:20.127276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55449a-76e4-4fec-8f95-e57fb2bcb192 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:20.189915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0c087-3216-482c-a9c0-b8085884ed9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:20.191403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0c087-3216-482c-a9c0-b8085884ed9f service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:20.202235 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3232/6498] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:26:20.252903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bd55449a-76e4-4fec-8f95-e57fb2bcb192 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invalidating the token cache because role 0e46a1ddecc44597be78d09c35eb3779 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:20.253221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bd55449a-76e4-4fec-8f95-e57fb2bcb192 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:20.253925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bd55449a-76e4-4fec-8f95-e57fb2bcb192 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:20.255850 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3267/6499] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:20 2022] DELETE /identity/v3/roles/0e46a1ddecc44597be78d09c35eb3779 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:20.259915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0c087-3216-482c-a9c0-b8085884ed9f service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:20.271286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0c087-3216-482c-a9c0-b8085884ed9f service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:20.328195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e23e0-5968-4200-8faf-dd552f200ab9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:20.329074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e23e0-5968-4200-8faf-dd552f200ab9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:20.329365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e23e0-5968-4200-8faf-dd552f200ab9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:20.329645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e23e0-5968-4200-8faf-dd552f200ab9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/groups/83dc424272e5493a9a3b46bf902dde1c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:20.330961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e23e0-5968-4200-8faf-dd552f200ab9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_group(group_id=83dc424272e5493a9a3b46bf902dde1c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:20.332956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8e23e0-5968-4200-8faf-dd552f200ab9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:20.439889 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3233/6500] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:26:20 2022] DELETE /identity/v3/groups/83dc424272e5493a9a3b46bf902dde1c => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:20.443749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55449a-76e4-4fec-8f95-e57fb2bcb192 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:20.445486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bd55449a-76e4-4fec-8f95-e57fb2bcb192 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:20.458956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55449a-76e4-4fec-8f95-e57fb2bcb192 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:20.519253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8de45-0a98-4f8c-a60d-ade83449a10d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:20.520171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8de45-0a98-4f8c-a60d-ade83449a10d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:20.520456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8de45-0a98-4f8c-a60d-ade83449a10d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:20.520782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8de45-0a98-4f8c-a60d-ade83449a10d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/projects/b321a6894a5d4c79952076f6c0408168` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:20.521948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8de45-0a98-4f8c-a60d-ade83449a10d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b321a6894a5d4c79952076f6c0408168)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:20.524665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb8de45-0a98-4f8c-a60d-ade83449a10d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:20.648593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:20.695032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2eb8de45-0a98-4f8c-a60d-ade83449a10d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b321a6894a5d4c79952076f6c0408168', 'request_id': 'req-2eb8de45-0a98-4f8c-a60d-ade83449a10d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:20.702298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2eb8de45-0a98-4f8c-a60d-ade83449a10d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b321a6894a5d4c79952076f6c0408168', 'request_id': 'req-2eb8de45-0a98-4f8c-a60d-ade83449a10d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:20.837335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2eb8de45-0a98-4f8c-a60d-ade83449a10d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b321a6894a5d4c79952076f6c0408168', 'request_id': 'req-2eb8de45-0a98-4f8c-a60d-ade83449a10d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:20.859595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2eb8de45-0a98-4f8c-a60d-ade83449a10d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] The token cache is being invalidate because project b321a6894a5d4c79952076f6c0408168 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:20.860160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2eb8de45-0a98-4f8c-a60d-ade83449a10d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:20.865209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2eb8de45-0a98-4f8c-a60d-ade83449a10d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:20.868336 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3268/6501] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:26:20 2022] DELETE /identity/v3/projects/b321a6894a5d4c79952076f6c0408168 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:20.874302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e23e0-5968-4200-8faf-dd552f200ab9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:20.876404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6d8e23e0-5968-4200-8faf-dd552f200ab9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:20.894693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8e23e0-5968-4200-8faf-dd552f200ab9 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:21.000414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef9a86-1302-4851-b928-4cbb43f93861 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:21.001729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef9a86-1302-4851-b928-4cbb43f93861 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:21.002270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef9a86-1302-4851-b928-4cbb43f93861 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:21.002800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef9a86-1302-4851-b928-4cbb43f93861 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/domains/d7087720d030435a9edc125e584a7690` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:21.004536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef9a86-1302-4851-b928-4cbb43f93861 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=d7087720d030435a9edc125e584a7690)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:21.007548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef9a86-1302-4851-b928-4cbb43f93861 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:21.030998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-eeef9a86-1302-4851-b928-4cbb43f93861 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'd7087720d030435a9edc125e584a7690'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:21.032106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-eeef9a86-1302-4851-b928-4cbb43f93861 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'd7087720d030435a9edc125e584a7690'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:21.058409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:21.129379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-eeef9a86-1302-4851-b928-4cbb43f93861 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'd7087720d030435a9edc125e584a7690'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:21.130346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-eeef9a86-1302-4851-b928-4cbb43f93861 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'd7087720d030435a9edc125e584a7690'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:21.135156 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3234/6502] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:26:20 2022] PATCH /identity/v3/domains/d7087720d030435a9edc125e584a7690 => generated 275 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:21.140252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8de45-0a98-4f8c-a60d-ade83449a10d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:21.152130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8de45-0a98-4f8c-a60d-ade83449a10d tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:21.216557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:21.217499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:21.217789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:21.218108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/domains/d7087720d030435a9edc125e584a7690` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:21.232666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=d7087720d030435a9edc125e584a7690)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:21.235346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:21.265360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'd7087720d030435a9edc125e584a7690'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:21.269849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'd7087720d030435a9edc125e584a7690'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:21.407799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7087720d030435a9edc125e584a7690', 'request_id': 'req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:21.414344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7087720d030435a9edc125e584a7690', 'request_id': 'req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:21.532676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7087720d030435a9edc125e584a7690', 'request_id': 'req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:21.545601 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] The token cache is being invalidate because project d7087720d030435a9edc125e584a7690 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:21.545935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:21.546579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:21.554741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'd7087720d030435a9edc125e584a7690', 'request_id': 'req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:21.555383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'd7087720d030435a9edc125e584a7690', 'request_id': 'req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:21.558669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'd7087720d030435a9edc125e584a7690', 'request_id': 'req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:21.564297 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3269/6503] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:26:21 2022] DELETE /identity/v3/domains/d7087720d030435a9edc125e584a7690 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:21.568278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef9a86-1302-4851-b928-4cbb43f93861 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:21.569684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-eeef9a86-1302-4851-b928-4cbb43f93861 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:21.582522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef9a86-1302-4851-b928-4cbb43f93861 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:21.648347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e84bfc-3fb6-47a9-bab5-3bfc55f6e8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:21.649627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e84bfc-3fb6-47a9-bab5-3bfc55f6e8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:21.650049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e84bfc-3fb6-47a9-bab5-3bfc55f6e8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:21.650430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e84bfc-3fb6-47a9-bab5-3bfc55f6e8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5ef04c834f42436b886862d8a3ee6e84` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:21.653053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e84bfc-3fb6-47a9-bab5-3bfc55f6e8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_role(role_id=5ef04c834f42436b886862d8a3ee6e84)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:21.654972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e84bfc-3fb6-47a9-bab5-3bfc55f6e8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:21.825698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-90e84bfc-3fb6-47a9-bab5-3bfc55f6e8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invalidating the token cache because role 5ef04c834f42436b886862d8a3ee6e84 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:21.826045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-90e84bfc-3fb6-47a9-bab5-3bfc55f6e8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:21.826733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-90e84bfc-3fb6-47a9-bab5-3bfc55f6e8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:21.828660 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3235/6504] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:21 2022] DELETE /identity/v3/roles/5ef04c834f42436b886862d8a3ee6e84 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:21.832615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:21.834110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:21.848248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecaed8a-7be0-4e93-83b2-4680766dc7f7 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:21.905663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb7090-5464-442e-81ce-c7902c18ed2e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:21.906515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb7090-5464-442e-81ce-c7902c18ed2e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:21.906806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb7090-5464-442e-81ce-c7902c18ed2e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:21.907088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cb7090-5464-442e-81ce-c7902c18ed2e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/9ff56f0c05ab49039e0ff27f5ae8fbb3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:21.908645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb7090-5464-442e-81ce-c7902c18ed2e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_role(role_id=9ff56f0c05ab49039e0ff27f5ae8fbb3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:21.910313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cb7090-5464-442e-81ce-c7902c18ed2e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:22.001330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d1cb7090-5464-442e-81ce-c7902c18ed2e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invalidating the token cache because role 9ff56f0c05ab49039e0ff27f5ae8fbb3 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:22.001630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d1cb7090-5464-442e-81ce-c7902c18ed2e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:22.002327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d1cb7090-5464-442e-81ce-c7902c18ed2e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:22.004347 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3270/6505] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:21 2022] DELETE /identity/v3/roles/9ff56f0c05ab49039e0ff27f5ae8fbb3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:22.008263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e84bfc-3fb6-47a9-bab5-3bfc55f6e8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:22.009714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-90e84bfc-3fb6-47a9-bab5-3bfc55f6e8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:22.023008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e84bfc-3fb6-47a9-bab5-3bfc55f6e8c5 tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:22.086099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed28a56-4827-46ef-ae50-f402dbec54fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6047f89d3fad4f8eae1b5f451a862540', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b8f0c827196d4ac7b046c431195aa987', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:22.087111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed28a56-4827-46ef-ae50-f402dbec54fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:22.094696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed28a56-4827-46ef-ae50-f402dbec54fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:22.094696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed28a56-4827-46ef-ae50-f402dbec54fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] PATH_INFO: `/v3/roles/5df3bb3dada94e2e8d1d425c4b107106` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:22.094696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed28a56-4827-46ef-ae50-f402dbec54fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorizing `identity:delete_role(role_id=5df3bb3dada94e2e8d1d425c4b107106)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:22.094696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed28a56-4827-46ef-ae50-f402dbec54fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:22.179919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:22.206885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aed28a56-4827-46ef-ae50-f402dbec54fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invalidating the token cache because role 5df3bb3dada94e2e8d1d425c4b107106 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:22.208821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aed28a56-4827-46ef-ae50-f402dbec54fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:22.208821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-aed28a56-4827-46ef-ae50-f402dbec54fa tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:22.211060 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3236/6506] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:22 2022] DELETE /identity/v3/roles/5df3bb3dada94e2e8d1d425c4b107106 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:22.215084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb7090-5464-442e-81ce-c7902c18ed2e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:22.216554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d1cb7090-5464-442e-81ce-c7902c18ed2e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:22.278650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:22.288217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cb7090-5464-442e-81ce-c7902c18ed2e tempest-RolesV3TestJSON-1181967757 tempest-RolesV3TestJSON-1181967757-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:22.345746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2683980-a8da-438e-af15-c5953ecf9476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:22.346998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2683980-a8da-438e-af15-c5953ecf9476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:22.347219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2683980-a8da-438e-af15-c5953ecf9476 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:22.347508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2683980-a8da-438e-af15-c5953ecf9476 admin admin] PATH_INFO: `/v3/users/a1b4089f23c746a8ae580d7720de81b1` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:22.348777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2683980-a8da-438e-af15-c5953ecf9476 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1b4089f23c746a8ae580d7720de81b1)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:22.350840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2683980-a8da-438e-af15-c5953ecf9476 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:22.583291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2683980-a8da-438e-af15-c5953ecf9476 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1b4089f23c746a8ae580d7720de81b1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:22.590659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2683980-a8da-438e-af15-c5953ecf9476 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1b4089f23c746a8ae580d7720de81b1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:22.591400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2683980-a8da-438e-af15-c5953ecf9476 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1b4089f23c746a8ae580d7720de81b1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:22.592052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2683980-a8da-438e-af15-c5953ecf9476 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1b4089f23c746a8ae580d7720de81b1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:22.687727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e2683980-a8da-438e-af15-c5953ecf9476 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1b4089f23c746a8ae580d7720de81b1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:22.708125 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3271/6507] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:22 2022] DELETE /identity/v3/users/a1b4089f23c746a8ae580d7720de81b1 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:22.712823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b383fe-4cd1-4972-b26f-8866d05b401c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:22.713122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b383fe-4cd1-4972-b26f-8866d05b401c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:22.713365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b383fe-4cd1-4972-b26f-8866d05b401c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:22.732716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-76b383fe-4cd1-4972-b26f-8866d05b401c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:22.733999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-76b383fe-4cd1-4972-b26f-8866d05b401c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:22.798598 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3237/6508] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:22.809032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2683980-a8da-438e-af15-c5953ecf9476 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:22.810546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e2683980-a8da-438e-af15-c5953ecf9476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:22.872682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2683980-a8da-438e-af15-c5953ecf9476 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:22.878038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caef56-4714-4831-b3e4-6d198b2e0042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:22.878785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caef56-4714-4831-b3e4-6d198b2e0042 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:22.879078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caef56-4714-4831-b3e4-6d198b2e0042 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:22.879366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3caef56-4714-4831-b3e4-6d198b2e0042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:22.890378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3caef56-4714-4831-b3e4-6d198b2e0042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:22.892611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3caef56-4714-4831-b3e4-6d198b2e0042 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:22.902521 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3272/6509] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:22.956724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:23.195234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:23.450592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b383fe-4cd1-4972-b26f-8866d05b401c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:23.453364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-76b383fe-4cd1-4972-b26f-8866d05b401c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:23.465755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b383fe-4cd1-4972-b26f-8866d05b401c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:23.474623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc4439-0c61-468d-a24f-8d1a102910f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:23.474623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc4439-0c61-468d-a24f-8d1a102910f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:23.474623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc4439-0c61-468d-a24f-8d1a102910f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:23.474623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc4439-0c61-468d-a24f-8d1a102910f0 admin admin] PATH_INFO: `/v3/projects/6b0d918e277d4bc78e562134e0e42339` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:23.475046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc4439-0c61-468d-a24f-8d1a102910f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b0d918e277d4bc78e562134e0e42339)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:23.475256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc4439-0c61-468d-a24f-8d1a102910f0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:23.630881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cfcc4439-0c61-468d-a24f-8d1a102910f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b0d918e277d4bc78e562134e0e42339', 'request_id': 'req-cfcc4439-0c61-468d-a24f-8d1a102910f0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:23.637590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cfcc4439-0c61-468d-a24f-8d1a102910f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b0d918e277d4bc78e562134e0e42339', 'request_id': 'req-cfcc4439-0c61-468d-a24f-8d1a102910f0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:23.704221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cfcc4439-0c61-468d-a24f-8d1a102910f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b0d918e277d4bc78e562134e0e42339', 'request_id': 'req-cfcc4439-0c61-468d-a24f-8d1a102910f0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:23.717437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cfcc4439-0c61-468d-a24f-8d1a102910f0 admin admin] The token cache is being invalidate because project 6b0d918e277d4bc78e562134e0e42339 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:23.717698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cfcc4439-0c61-468d-a24f-8d1a102910f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:23.718985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cfcc4439-0c61-468d-a24f-8d1a102910f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:23.720257 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3238/6510] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:26:23 2022] DELETE /identity/v3/projects/6b0d918e277d4bc78e562134e0e42339 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:23.724228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caef56-4714-4831-b3e4-6d198b2e0042 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:23.740503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3caef56-4714-4831-b3e4-6d198b2e0042 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:23.801799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22194a3d-0cd6-466a-b652-6a81a0f0b87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:23.802713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22194a3d-0cd6-466a-b652-6a81a0f0b87d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:23.803036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22194a3d-0cd6-466a-b652-6a81a0f0b87d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:23.803316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22194a3d-0cd6-466a-b652-6a81a0f0b87d admin admin] PATH_INFO: `/v3/users/6047f89d3fad4f8eae1b5f451a862540` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:23.804561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22194a3d-0cd6-466a-b652-6a81a0f0b87d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6047f89d3fad4f8eae1b5f451a862540)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:23.806548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22194a3d-0cd6-466a-b652-6a81a0f0b87d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:24.087499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-22194a3d-0cd6-466a-b652-6a81a0f0b87d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6047f89d3fad4f8eae1b5f451a862540'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:24.096773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-22194a3d-0cd6-466a-b652-6a81a0f0b87d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6047f89d3fad4f8eae1b5f451a862540'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:24.099531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-22194a3d-0cd6-466a-b652-6a81a0f0b87d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6047f89d3fad4f8eae1b5f451a862540'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:24.100776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-22194a3d-0cd6-466a-b652-6a81a0f0b87d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6047f89d3fad4f8eae1b5f451a862540'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:24.236095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-22194a3d-0cd6-466a-b652-6a81a0f0b87d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6047f89d3fad4f8eae1b5f451a862540'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:24.260590 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3273/6511] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:23 2022] DELETE /identity/v3/users/6047f89d3fad4f8eae1b5f451a862540 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:24.915962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:25.021169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc4439-0c61-468d-a24f-8d1a102910f0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:25.023946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cfcc4439-0c61-468d-a24f-8d1a102910f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:25.039592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc4439-0c61-468d-a24f-8d1a102910f0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:25.093143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:25.135336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de755bb-8777-470a-9e9e-9ca5e6f765b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:25.136669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de755bb-8777-470a-9e9e-9ca5e6f765b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:25.137378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de755bb-8777-470a-9e9e-9ca5e6f765b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:25.137801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de755bb-8777-470a-9e9e-9ca5e6f765b1 admin admin] PATH_INFO: `/v3/projects/b8f0c827196d4ac7b046c431195aa987` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:25.139135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de755bb-8777-470a-9e9e-9ca5e6f765b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8f0c827196d4ac7b046c431195aa987)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:25.143014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de755bb-8777-470a-9e9e-9ca5e6f765b1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:25.294812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1de755bb-8777-470a-9e9e-9ca5e6f765b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8f0c827196d4ac7b046c431195aa987', 'request_id': 'req-1de755bb-8777-470a-9e9e-9ca5e6f765b1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:25.304339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1de755bb-8777-470a-9e9e-9ca5e6f765b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8f0c827196d4ac7b046c431195aa987', 'request_id': 'req-1de755bb-8777-470a-9e9e-9ca5e6f765b1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:25.372771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1de755bb-8777-470a-9e9e-9ca5e6f765b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8f0c827196d4ac7b046c431195aa987', 'request_id': 'req-1de755bb-8777-470a-9e9e-9ca5e6f765b1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:25.385634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1de755bb-8777-470a-9e9e-9ca5e6f765b1 admin admin] The token cache is being invalidate because project b8f0c827196d4ac7b046c431195aa987 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:25.385961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1de755bb-8777-470a-9e9e-9ca5e6f765b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:25.386627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1de755bb-8777-470a-9e9e-9ca5e6f765b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:25.388236 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3239/6512] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:26:25 2022] DELETE /identity/v3/projects/b8f0c827196d4ac7b046c431195aa987 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:25.395077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457987b-7e68-41c6-8b6c-7f1ac1db8b61 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:25.396076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457987b-7e68-41c6-8b6c-7f1ac1db8b61 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:25.396364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457987b-7e68-41c6-8b6c-7f1ac1db8b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:25.416818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c457987b-7e68-41c6-8b6c-7f1ac1db8b61 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:25.418075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c457987b-7e68-41c6-8b6c-7f1ac1db8b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:25.485720 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3274/6513] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:25.495319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2dc3d-8994-4b23-aa82-a185853b051d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:25.495698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2dc3d-8994-4b23-aa82-a185853b051d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:25.495983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e2dc3d-8994-4b23-aa82-a185853b051d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:25.521080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b5e2dc3d-8994-4b23-aa82-a185853b051d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:25.522447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b5e2dc3d-8994-4b23-aa82-a185853b051d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:25.531630 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3240/6514] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:25.536021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457987b-7e68-41c6-8b6c-7f1ac1db8b61 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:25.547447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457987b-7e68-41c6-8b6c-7f1ac1db8b61 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:25.568550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7b32d-1cc2-495d-8fdd-541a01afdfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:25.569415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7b32d-1cc2-495d-8fdd-541a01afdfec admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:25.571554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7b32d-1cc2-495d-8fdd-541a01afdfec admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:25.571554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa7b32d-1cc2-495d-8fdd-541a01afdfec admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:25.571554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7b32d-1cc2-495d-8fdd-541a01afdfec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:25.578690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7b32d-1cc2-495d-8fdd-541a01afdfec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:25.578690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa7b32d-1cc2-495d-8fdd-541a01afdfec admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:25.594324 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3275/6515] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:26:25.601153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e2dc3d-8994-4b23-aa82-a185853b051d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:25.610445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e2dc3d-8994-4b23-aa82-a185853b051d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:25.617315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f7f70-e6b9-4575-b326-b9b992029be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:25.618050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f7f70-e6b9-4575-b326-b9b992029be2 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:25.618340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f7f70-e6b9-4575-b326-b9b992029be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:25.618620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f7f70-e6b9-4575-b326-b9b992029be2 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:25.619342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f7f70-e6b9-4575-b326-b9b992029be2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:25.621914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f7f70-e6b9-4575-b326-b9b992029be2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:25.749432 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3241/6516] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:25 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:25.753546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7b32d-1cc2-495d-8fdd-541a01afdfec admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:25.767376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa7b32d-1cc2-495d-8fdd-541a01afdfec admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:25.859217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed1b86-e9c1-4945-97dd-b1932796d7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:25.860008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed1b86-e9c1-4945-97dd-b1932796d7da admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:25.860371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed1b86-e9c1-4945-97dd-b1932796d7da admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:25.860723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed1b86-e9c1-4945-97dd-b1932796d7da admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:25.861613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed1b86-e9c1-4945-97dd-b1932796d7da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:25.863887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed1b86-e9c1-4945-97dd-b1932796d7da admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:26.032896 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3276/6517] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:25 2022] POST /identity/v3/users => generated 286 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:26.087972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f7f70-e6b9-4575-b326-b9b992029be2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:26.094528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f7f70-e6b9-4575-b326-b9b992029be2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:26.101061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3cd3d-d924-4475-bc2b-8a639163dcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:26.102110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3cd3d-d924-4475-bc2b-8a639163dcad admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:26.102638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3cd3d-d924-4475-bc2b-8a639163dcad admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:26.103349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a3cd3d-d924-4475-bc2b-8a639163dcad admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:26.105492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a3cd3d-d924-4475-bc2b-8a639163dcad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:26.108140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a3cd3d-d924-4475-bc2b-8a639163dcad admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:26.124534 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3242/6518] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:26.125095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed1b86-e9c1-4945-97dd-b1932796d7da admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:26.136983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed1b86-e9c1-4945-97dd-b1932796d7da admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:26.144808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db799f3-8842-4878-a828-ba0e880acaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:26.145825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db799f3-8842-4878-a828-ba0e880acaa7 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:26.146400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db799f3-8842-4878-a828-ba0e880acaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:26.146932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db799f3-8842-4878-a828-ba0e880acaa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:26.148322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db799f3-8842-4878-a828-ba0e880acaa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:26.150329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db799f3-8842-4878-a828-ba0e880acaa7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:26.171187 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3277/6519] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:26.174891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3cd3d-d924-4475-bc2b-8a639163dcad admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:26.188347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a3cd3d-d924-4475-bc2b-8a639163dcad admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:26.194945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525f3b3-7280-4202-b76a-7039515a8118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:26.196062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525f3b3-7280-4202-b76a-7039515a8118 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:26.196360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525f3b3-7280-4202-b76a-7039515a8118 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:26.196639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525f3b3-7280-4202-b76a-7039515a8118 admin admin] PATH_INFO: `/v3/projects/e3a2e6f34ea2403b8b4bbde3e8deb06b/users/90b5d9b369a04940a10424cf0efb95ac/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:26.234901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525f3b3-7280-4202-b76a-7039515a8118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a2e6f34ea2403b8b4bbde3e8deb06b, user_id=90b5d9b369a04940a10424cf0efb95ac, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:26.237973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525f3b3-7280-4202-b76a-7039515a8118 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:26.375131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:26.401384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b06bec-85dd-4cb4-ba56-fde43268ab7d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:26.401681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b06bec-85dd-4cb4-ba56-fde43268ab7d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:26.402918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b06bec-85dd-4cb4-ba56-fde43268ab7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:26.405771 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3243/6520] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:26 2022] PUT /identity/v3/projects/e3a2e6f34ea2403b8b4bbde3e8deb06b/users/90b5d9b369a04940a10424cf0efb95ac/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:26.440878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e4b06bec-85dd-4cb4-ba56-fde43268ab7d None None] MFA Rules not processed for user `90b5d9b369a04940a10424cf0efb95ac`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:26.442526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e4b06bec-85dd-4cb4-ba56-fde43268ab7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:26.558785 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3278/6521] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:26.575957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525f3b3-7280-4202-b76a-7039515a8118 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:26.578192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1525f3b3-7280-4202-b76a-7039515a8118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:26.657917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525f3b3-7280-4202-b76a-7039515a8118 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:26.662463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475abd2b-8ed9-4026-86fb-62ca544d44b5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:26.663173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475abd2b-8ed9-4026-86fb-62ca544d44b5 service glance] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:26.663462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475abd2b-8ed9-4026-86fb-62ca544d44b5 service glance] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:26.663735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475abd2b-8ed9-4026-86fb-62ca544d44b5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:26.681924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475abd2b-8ed9-4026-86fb-62ca544d44b5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:26.681924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475abd2b-8ed9-4026-86fb-62ca544d44b5 service glance] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:26.692499 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3244/6522] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:27.114270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fee65-4b67-4b05-a221-a071b5182fc0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:27.114566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fee65-4b67-4b05-a221-a071b5182fc0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:27.114846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fee65-4b67-4b05-a221-a071b5182fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:27.167726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c51fee65-4b67-4b05-a221-a071b5182fc0 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:27.168944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c51fee65-4b67-4b05-a221-a071b5182fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:27.239849 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3279/6523] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:26:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:27.275105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:27.485978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475abd2b-8ed9-4026-86fb-62ca544d44b5 service glance] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:27.487473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-475abd2b-8ed9-4026-86fb-62ca544d44b5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:27.593558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475abd2b-8ed9-4026-86fb-62ca544d44b5 service glance] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:27.601360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc03699-1bc2-4a96-9f02-5065d44a34c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:27.602346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc03699-1bc2-4a96-9f02-5065d44a34c3 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:27.602858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc03699-1bc2-4a96-9f02-5065d44a34c3 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:27.603514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc03699-1bc2-4a96-9f02-5065d44a34c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:27.615206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc03699-1bc2-4a96-9f02-5065d44a34c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:27.617431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc03699-1bc2-4a96-9f02-5065d44a34c3 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:27.627323 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3245/6524] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:26:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:26:28.219018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:28.310505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9de9d8-382b-4562-ad70-3e09ae1586a4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:28.318943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9de9d8-382b-4562-ad70-3e09ae1586a4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:28.318943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9de9d8-382b-4562-ad70-3e09ae1586a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:28.332076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2c9de9d8-382b-4562-ad70-3e09ae1586a4 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:28.334701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2c9de9d8-382b-4562-ad70-3e09ae1586a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:28.358781 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3280/6525] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:26:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:28.371519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc03699-1bc2-4a96-9f02-5065d44a34c3 service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:28.386005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc03699-1bc2-4a96-9f02-5065d44a34c3 service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:28.391510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:28.391920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9d984-9996-42bf-a131-1a07bdba8531 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:28.392978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9d984-9996-42bf-a131-1a07bdba8531 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:28.393495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9d984-9996-42bf-a131-1a07bdba8531 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:28.393777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9d984-9996-42bf-a131-1a07bdba8531 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:28.408115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9d984-9996-42bf-a131-1a07bdba8531 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:28.409724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9d984-9996-42bf-a131-1a07bdba8531 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:28.422986 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3246/6526] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:26:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:26:28.562351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:29.025053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:29.275919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:29.462491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec73837-0eb1-47ef-ba53-29dbe65adb75 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:29.463173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec73837-0eb1-47ef-ba53-29dbe65adb75 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:29.463316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec73837-0eb1-47ef-ba53-29dbe65adb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:29.503015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fec73837-0eb1-47ef-ba53-29dbe65adb75 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:29.504497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fec73837-0eb1-47ef-ba53-29dbe65adb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:29.522977 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3281/6527] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:26:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:29.544240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9d984-9996-42bf-a131-1a07bdba8531 service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:29.566353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9d984-9996-42bf-a131-1a07bdba8531 service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:29.576841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca1f9b-e7d6-41bf-bc5a-757e2614f25e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:29.577793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca1f9b-e7d6-41bf-bc5a-757e2614f25e service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:29.583168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca1f9b-e7d6-41bf-bc5a-757e2614f25e service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:29.583168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca1f9b-e7d6-41bf-bc5a-757e2614f25e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:29.591517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca1f9b-e7d6-41bf-bc5a-757e2614f25e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:29.593277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ca1f9b-e7d6-41bf-bc5a-757e2614f25e service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:29.603906 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3247/6528] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:26:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:26:29.864529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:30.145218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:30.383448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec73837-0eb1-47ef-ba53-29dbe65adb75 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:30.383448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fec73837-0eb1-47ef-ba53-29dbe65adb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:30.494223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec73837-0eb1-47ef-ba53-29dbe65adb75 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:30.501087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da409fd1-cfe1-458a-b83a-d9ade30aad9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:30.501806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da409fd1-cfe1-458a-b83a-d9ade30aad9e service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:30.502266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da409fd1-cfe1-458a-b83a-d9ade30aad9e service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:30.502724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da409fd1-cfe1-458a-b83a-d9ade30aad9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:30.505169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-da409fd1-cfe1-458a-b83a-d9ade30aad9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:30.540851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9357a31-ba7e-4283-a147-27644373cbde None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:30.541202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9357a31-ba7e-4283-a147-27644373cbde None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:30.541492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9357a31-ba7e-4283-a147-27644373cbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:30.571792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a9357a31-ba7e-4283-a147-27644373cbde None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:30.573076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a9357a31-ba7e-4283-a147-27644373cbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:30.582378 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3248/6529] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:26:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:26:30.597610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9357a31-ba7e-4283-a147-27644373cbde None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:30.612005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da409fd1-cfe1-458a-b83a-d9ade30aad9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:30.612981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9357a31-ba7e-4283-a147-27644373cbde None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:30.613436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da409fd1-cfe1-458a-b83a-d9ade30aad9e service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:30.619137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d01e40-6058-4261-97d2-48d125b9df80 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:30.621974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d01e40-6058-4261-97d2-48d125b9df80 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:30.621974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d01e40-6058-4261-97d2-48d125b9df80 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:30.621974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d01e40-6058-4261-97d2-48d125b9df80 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:30.624205 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3282/6530] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:30.629453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d01e40-6058-4261-97d2-48d125b9df80 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:30.630892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d01e40-6058-4261-97d2-48d125b9df80 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:30.639333 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3249/6531] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:26:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 21:26:30.905121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:30.994480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da409fd1-cfe1-458a-b83a-d9ade30aad9e service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:31.023938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da409fd1-cfe1-458a-b83a-d9ade30aad9e service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:31.056136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419d5e5-4c5e-4425-8d4f-303c057d3198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:31.082486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419d5e5-4c5e-4425-8d4f-303c057d3198 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:31.083804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419d5e5-4c5e-4425-8d4f-303c057d3198 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:31.084101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b419d5e5-4c5e-4425-8d4f-303c057d3198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:31.090764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b419d5e5-4c5e-4425-8d4f-303c057d3198 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:31.131029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b419d5e5-4c5e-4425-8d4f-303c057d3198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:31.146989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b419d5e5-4c5e-4425-8d4f-303c057d3198 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:31.192150 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3283/6532] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:31.201938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:31.408608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d53d8-9839-4a9e-aa6b-187dbe55e3b0 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:31.409139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d53d8-9839-4a9e-aa6b-187dbe55e3b0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:31.409595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d53d8-9839-4a9e-aa6b-187dbe55e3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:31.428010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c72d53d8-9839-4a9e-aa6b-187dbe55e3b0 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:31.429354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c72d53d8-9839-4a9e-aa6b-187dbe55e3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:31.438035 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3250/6533] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:26:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:31.451640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419d5e5-4c5e-4425-8d4f-303c057d3198 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:31.453713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f3a81-20a9-44c3-b703-823d74ce62d6 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:31.454164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f3a81-20a9-44c3-b703-823d74ce62d6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:31.454687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4f3a81-20a9-44c3-b703-823d74ce62d6 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:31.459368 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3251/6534] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:26:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:26:31.469109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cbcd6-ea17-4a0f-95b8-2f6108f88ef9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:31.469275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cbcd6-ea17-4a0f-95b8-2f6108f88ef9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:31.470073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788cbcd6-ea17-4a0f-95b8-2f6108f88ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:31.470999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b419d5e5-4c5e-4425-8d4f-303c057d3198 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:31.476644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435b38b-ab3b-4a42-ab44-18d9aaade012 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:31.477307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435b38b-ab3b-4a42-ab44-18d9aaade012 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:31.477596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435b38b-ab3b-4a42-ab44-18d9aaade012 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:31.477952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435b38b-ab3b-4a42-ab44-18d9aaade012 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:31.491799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435b38b-ab3b-4a42-ab44-18d9aaade012 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:31.492625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435b38b-ab3b-4a42-ab44-18d9aaade012 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:31.493790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-788cbcd6-ea17-4a0f-95b8-2f6108f88ef9 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:31.495062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-788cbcd6-ea17-4a0f-95b8-2f6108f88ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:31.515763 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3284/6535] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:26:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 21:26:31.572168 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3252/6536] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:26:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:31.597143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435b38b-ab3b-4a42-ab44-18d9aaade012 service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:31.597143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7435b38b-ab3b-4a42-ab44-18d9aaade012 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:31.609027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86868eab-8b2e-4cae-b4c9-56c0e85001c3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:31.610937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86868eab-8b2e-4cae-b4c9-56c0e85001c3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:31.611225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86868eab-8b2e-4cae-b4c9-56c0e85001c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:31.620940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435b38b-ab3b-4a42-ab44-18d9aaade012 service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:31.626941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355e987-9145-4f3b-b3b9-73d1ab74eb86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:31.633293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-86868eab-8b2e-4cae-b4c9-56c0e85001c3 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:31.634527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355e987-9145-4f3b-b3b9-73d1ab74eb86 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:31.634803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355e987-9145-4f3b-b3b9-73d1ab74eb86 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:31.635022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-86868eab-8b2e-4cae-b4c9-56c0e85001c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:31.635139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355e987-9145-4f3b-b3b9-73d1ab74eb86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:31.653473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355e987-9145-4f3b-b3b9-73d1ab74eb86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:31.654852 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3253/6537] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:26:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:31.656093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355e987-9145-4f3b-b3b9-73d1ab74eb86 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:31.669755 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3285/6538] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:31.683629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86868eab-8b2e-4cae-b4c9-56c0e85001c3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:31.709428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86868eab-8b2e-4cae-b4c9-56c0e85001c3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:31.715509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166ac4a-c43f-49e8-aac6-8751aacd7b75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:31.716788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166ac4a-c43f-49e8-aac6-8751aacd7b75 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:31.717352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166ac4a-c43f-49e8-aac6-8751aacd7b75 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:31.718071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166ac4a-c43f-49e8-aac6-8751aacd7b75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:31.732941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166ac4a-c43f-49e8-aac6-8751aacd7b75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:31.736065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166ac4a-c43f-49e8-aac6-8751aacd7b75 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:31.747059 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3254/6539] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:26:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:26:32.036352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355e987-9145-4f3b-b3b9-73d1ab74eb86 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:32.037595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0355e987-9145-4f3b-b3b9-73d1ab74eb86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:32.055137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355e987-9145-4f3b-b3b9-73d1ab74eb86 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:32.061055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da90ff-2490-49f4-a741-2d71d5aa6e59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:32.062188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da90ff-2490-49f4-a741-2d71d5aa6e59 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:32.062694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da90ff-2490-49f4-a741-2d71d5aa6e59 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:32.063197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da90ff-2490-49f4-a741-2d71d5aa6e59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:32.065300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-74da90ff-2490-49f4-a741-2d71d5aa6e59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:32.241973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da90ff-2490-49f4-a741-2d71d5aa6e59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:32.246499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da90ff-2490-49f4-a741-2d71d5aa6e59 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:32.316076 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3286/6540] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:32.447016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:32.768885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd281b-0d42-4616-99d1-c87a5023f117 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:32.769520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd281b-0d42-4616-99d1-c87a5023f117 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:32.769520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd281b-0d42-4616-99d1-c87a5023f117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:32.809177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8ecd281b-0d42-4616-99d1-c87a5023f117 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:32.810731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8ecd281b-0d42-4616-99d1-c87a5023f117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:32.818813 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3255/6541] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:26:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:32.842496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da90ff-2490-49f4-a741-2d71d5aa6e59 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:32.887200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da90ff-2490-49f4-a741-2d71d5aa6e59 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:32.905990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30777f-afd6-46b2-90f8-7a1d469b4fba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:32.906705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30777f-afd6-46b2-90f8-7a1d469b4fba service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:32.907016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30777f-afd6-46b2-90f8-7a1d469b4fba service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:32.908031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af30777f-afd6-46b2-90f8-7a1d469b4fba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:32.933256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30777f-afd6-46b2-90f8-7a1d469b4fba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:32.936712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af30777f-afd6-46b2-90f8-7a1d469b4fba service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:32.947891 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3287/6542] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:26:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:26:33.052020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:34.206260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:34.333527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:34.700616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d190a8d-817d-478c-ac45-82c065e65298 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:34.701099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d190a8d-817d-478c-ac45-82c065e65298 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:34.701535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d190a8d-817d-478c-ac45-82c065e65298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:34.733467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8d190a8d-817d-478c-ac45-82c065e65298 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:34.734999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8d190a8d-817d-478c-ac45-82c065e65298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:34.750018 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3256/6543] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:26:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:26:34.768851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30777f-afd6-46b2-90f8-7a1d469b4fba service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:34.785466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af30777f-afd6-46b2-90f8-7a1d469b4fba service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:34.792638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbebbd-345a-4865-8c95-248ec4564b3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:34.796704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbebbd-345a-4865-8c95-248ec4564b3f service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:34.796704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbebbd-345a-4865-8c95-248ec4564b3f service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:34.796704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbebbd-345a-4865-8c95-248ec4564b3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:34.814143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbebbd-345a-4865-8c95-248ec4564b3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:34.816779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbebbd-345a-4865-8c95-248ec4564b3f service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:34.833644 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3288/6544] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:26:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:26:35.439196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cb24f-0dfb-4b6b-9aaa-46ba509170e9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:35.444888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cb24f-0dfb-4b6b-9aaa-46ba509170e9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:35.444888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418cb24f-0dfb-4b6b-9aaa-46ba509170e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:35.469242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-418cb24f-0dfb-4b6b-9aaa-46ba509170e9 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:35.470887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-418cb24f-0dfb-4b6b-9aaa-46ba509170e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:35.484541 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3257/6545] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:26:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:35.550821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbebbd-345a-4865-8c95-248ec4564b3f service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:35.572032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbebbd-345a-4865-8c95-248ec4564b3f service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:35.578372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec764c-4af0-44aa-89b1-882446bbca22 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:35.579269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec764c-4af0-44aa-89b1-882446bbca22 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:35.579734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec764c-4af0-44aa-89b1-882446bbca22 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:35.580277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccec764c-4af0-44aa-89b1-882446bbca22 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:35.592903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec764c-4af0-44aa-89b1-882446bbca22 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:35.595647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccec764c-4af0-44aa-89b1-882446bbca22 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:35.621528 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3289/6546] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:26:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:26:36.277401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54960cdd-812b-4c0d-9ab7-59a2980116f3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:36.277752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54960cdd-812b-4c0d-9ab7-59a2980116f3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:36.278101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54960cdd-812b-4c0d-9ab7-59a2980116f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:36.298750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-54960cdd-812b-4c0d-9ab7-59a2980116f3 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:36.300007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-54960cdd-812b-4c0d-9ab7-59a2980116f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:36.308002 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3258/6547] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:26:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:36.339977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec764c-4af0-44aa-89b1-882446bbca22 service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:36.339977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccec764c-4af0-44aa-89b1-882446bbca22 service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:36.347207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8ed86-4690-4a09-858e-d5d03a42e259 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:36.350114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8ed86-4690-4a09-858e-d5d03a42e259 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:36.351929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8ed86-4690-4a09-858e-d5d03a42e259 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:36.352875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d8ed86-4690-4a09-858e-d5d03a42e259 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:36.371865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8ed86-4690-4a09-858e-d5d03a42e259 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:36.375782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d8ed86-4690-4a09-858e-d5d03a42e259 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:36.389198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:36.395152 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3290/6548] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:26:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:26:37.030412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:37.270273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d2b86-6a3a-444c-99d2-ce1f17502fb1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:37.271170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d2b86-6a3a-444c-99d2-ce1f17502fb1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:37.271657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d2b86-6a3a-444c-99d2-ce1f17502fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:37.293786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d38d2b86-6a3a-444c-99d2-ce1f17502fb1 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:37.295379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d38d2b86-6a3a-444c-99d2-ce1f17502fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:37.318185 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3259/6549] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:26:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:37.329565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8ed86-4690-4a09-858e-d5d03a42e259 service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:37.346994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d8ed86-4690-4a09-858e-d5d03a42e259 service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:37.358999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bff72-8b88-4d8f-9f90-9ad32554ad38 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:37.360974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bff72-8b88-4d8f-9f90-9ad32554ad38 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:37.361883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bff72-8b88-4d8f-9f90-9ad32554ad38 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:37.362370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710bff72-8b88-4d8f-9f90-9ad32554ad38 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:37.377264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710bff72-8b88-4d8f-9f90-9ad32554ad38 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:37.384424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710bff72-8b88-4d8f-9f90-9ad32554ad38 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:37.395294 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3291/6550] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:26:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:26:38.041467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d06403-0de2-458f-be9b-9a173251938e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:38.042555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d06403-0de2-458f-be9b-9a173251938e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:38.042993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d06403-0de2-458f-be9b-9a173251938e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:38.077086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:38.097398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e3d06403-0de2-458f-be9b-9a173251938e None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:38.099402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e3d06403-0de2-458f-be9b-9a173251938e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:38.114404 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3260/6551] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:26:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:38.134787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bff72-8b88-4d8f-9f90-9ad32554ad38 service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:38.146761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d06403-0de2-458f-be9b-9a173251938e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:38.163049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710bff72-8b88-4d8f-9f90-9ad32554ad38 service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:38.168362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227e0c1-7ae7-41aa-9776-350c2d3efefc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:38.169381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227e0c1-7ae7-41aa-9776-350c2d3efefc service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:38.169988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227e0c1-7ae7-41aa-9776-350c2d3efefc service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:38.170337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227e0c1-7ae7-41aa-9776-350c2d3efefc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:38.172383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e227e0c1-7ae7-41aa-9776-350c2d3efefc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:38.183060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d06403-0de2-458f-be9b-9a173251938e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:38.192649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55080948-ea0e-4b1d-9428-ffea302e6920 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:38.193660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55080948-ea0e-4b1d-9428-ffea302e6920 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:38.194755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55080948-ea0e-4b1d-9428-ffea302e6920 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:38.195123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55080948-ea0e-4b1d-9428-ffea302e6920 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:38.205346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55080948-ea0e-4b1d-9428-ffea302e6920 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:38.208937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55080948-ea0e-4b1d-9428-ffea302e6920 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:38.221564 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3261/6552] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:26:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:26:38.269497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227e0c1-7ae7-41aa-9776-350c2d3efefc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:38.271184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e227e0c1-7ae7-41aa-9776-350c2d3efefc service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:38.282412 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3292/6553] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:38.315184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:38.868219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e955d4-7b5b-48e2-ac67-febaaa21089e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:38.868505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e955d4-7b5b-48e2-ac67-febaaa21089e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:38.868959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e955d4-7b5b-48e2-ac67-febaaa21089e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:38.900200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c5e955d4-7b5b-48e2-ac67-febaaa21089e None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:38.901409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c5e955d4-7b5b-48e2-ac67-febaaa21089e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:38.925517 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3262/6554] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:26:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:38.948645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227e0c1-7ae7-41aa-9776-350c2d3efefc service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:38.969682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e227e0c1-7ae7-41aa-9776-350c2d3efefc service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:38.978180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b741a53-794b-4e22-bb05-950269e1589f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:38.979110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b741a53-794b-4e22-bb05-950269e1589f service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:38.979562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b741a53-794b-4e22-bb05-950269e1589f service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:38.980033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b741a53-794b-4e22-bb05-950269e1589f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:38.992225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b741a53-794b-4e22-bb05-950269e1589f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:38.994337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b741a53-794b-4e22-bb05-950269e1589f service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:39.004520 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3293/6555] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:26:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:26:39.415872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:39.469777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e955d4-7b5b-48e2-ac67-febaaa21089e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:39.569456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e955d4-7b5b-48e2-ac67-febaaa21089e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:39.576120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1f297-fde3-4f94-a3d6-5450e9d7a93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:39.577606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1f297-fde3-4f94-a3d6-5450e9d7a93d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:39.577920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1f297-fde3-4f94-a3d6-5450e9d7a93d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:39.578418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e1f297-fde3-4f94-a3d6-5450e9d7a93d admin admin] PATH_INFO: `/v3/users/90b5d9b369a04940a10424cf0efb95ac` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:39.582538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1f297-fde3-4f94-a3d6-5450e9d7a93d admin admin] RBAC: Authorizing `identity:delete_user(user_id=90b5d9b369a04940a10424cf0efb95ac)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:39.582538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e1f297-fde3-4f94-a3d6-5450e9d7a93d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:39.779139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:39.826292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-09e1f297-fde3-4f94-a3d6-5450e9d7a93d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90b5d9b369a04940a10424cf0efb95ac'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:39.826989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-09e1f297-fde3-4f94-a3d6-5450e9d7a93d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90b5d9b369a04940a10424cf0efb95ac'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:39.837631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-09e1f297-fde3-4f94-a3d6-5450e9d7a93d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90b5d9b369a04940a10424cf0efb95ac'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:40.297685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-09e1f297-fde3-4f94-a3d6-5450e9d7a93d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90b5d9b369a04940a10424cf0efb95ac'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:40.298604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-09e1f297-fde3-4f94-a3d6-5450e9d7a93d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90b5d9b369a04940a10424cf0efb95ac'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:40.323610 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3263/6556] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:39 2022] DELETE /identity/v3/users/90b5d9b369a04940a10424cf0efb95ac => generated 0 bytes in 854 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:40.328301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38f033-c2b1-48f2-a80c-04723d4dd8ea None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:40.328848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38f033-c2b1-48f2-a80c-04723d4dd8ea None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:40.328982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38f033-c2b1-48f2-a80c-04723d4dd8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:40.348437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-df38f033-c2b1-48f2-a80c-04723d4dd8ea None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:40.354193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-df38f033-c2b1-48f2-a80c-04723d4dd8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:40.456829 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3294/6557] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:26:40.480032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1f297-fde3-4f94-a3d6-5450e9d7a93d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:40.482292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-09e1f297-fde3-4f94-a3d6-5450e9d7a93d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:40.579668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e1f297-fde3-4f94-a3d6-5450e9d7a93d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:40.586546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83b84b-e83e-406c-8db6-f987d0b0c727 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:40.587593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83b84b-e83e-406c-8db6-f987d0b0c727 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:40.588060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83b84b-e83e-406c-8db6-f987d0b0c727 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:40.588604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83b84b-e83e-406c-8db6-f987d0b0c727 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:40.602542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83b84b-e83e-406c-8db6-f987d0b0c727 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:40.605654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83b84b-e83e-406c-8db6-f987d0b0c727 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:40.619536 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3264/6558] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:40.634647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:41.433351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38f033-c2b1-48f2-a80c-04723d4dd8ea None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:41.435814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-df38f033-c2b1-48f2-a80c-04723d4dd8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:41.464403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38f033-c2b1-48f2-a80c-04723d4dd8ea None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:41.481853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfc9ae-0106-4e1c-86c6-bb2b5b3f083b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:41.483780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfc9ae-0106-4e1c-86c6-bb2b5b3f083b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:41.484666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfc9ae-0106-4e1c-86c6-bb2b5b3f083b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:41.485407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfc9ae-0106-4e1c-86c6-bb2b5b3f083b admin admin] PATH_INFO: `/v3/projects/e3a2e6f34ea2403b8b4bbde3e8deb06b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:41.487939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfc9ae-0106-4e1c-86c6-bb2b5b3f083b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3a2e6f34ea2403b8b4bbde3e8deb06b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:41.495212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfc9ae-0106-4e1c-86c6-bb2b5b3f083b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:41.578992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:41.634545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7bcfc9ae-0106-4e1c-86c6-bb2b5b3f083b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3a2e6f34ea2403b8b4bbde3e8deb06b', 'request_id': 'req-7bcfc9ae-0106-4e1c-86c6-bb2b5b3f083b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:41.641607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7bcfc9ae-0106-4e1c-86c6-bb2b5b3f083b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3a2e6f34ea2403b8b4bbde3e8deb06b', 'request_id': 'req-7bcfc9ae-0106-4e1c-86c6-bb2b5b3f083b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:41.778106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7bcfc9ae-0106-4e1c-86c6-bb2b5b3f083b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3a2e6f34ea2403b8b4bbde3e8deb06b', 'request_id': 'req-7bcfc9ae-0106-4e1c-86c6-bb2b5b3f083b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:41.798363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7bcfc9ae-0106-4e1c-86c6-bb2b5b3f083b admin admin] The token cache is being invalidate because project e3a2e6f34ea2403b8b4bbde3e8deb06b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:41.798677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7bcfc9ae-0106-4e1c-86c6-bb2b5b3f083b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:41.801983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7bcfc9ae-0106-4e1c-86c6-bb2b5b3f083b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:41.809888 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3295/6559] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:26:41 2022] DELETE /identity/v3/projects/e3a2e6f34ea2403b8b4bbde3e8deb06b => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:41.815487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ddec2-74aa-440a-8d56-fdad5505456b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:41.815487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ddec2-74aa-440a-8d56-fdad5505456b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:41.815487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ddec2-74aa-440a-8d56-fdad5505456b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:41.852257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5d1ddec2-74aa-440a-8d56-fdad5505456b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:41.854264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5d1ddec2-74aa-440a-8d56-fdad5505456b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:41.969546 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3265/6560] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:41.971901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37f285-153d-4fce-a098-610c6f28ce1f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:41.972191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37f285-153d-4fce-a098-610c6f28ce1f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:41.972472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37f285-153d-4fce-a098-610c6f28ce1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:42.000783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-dd37f285-153d-4fce-a098-610c6f28ce1f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:42.002634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dd37f285-153d-4fce-a098-610c6f28ce1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:42.016754 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3296/6561] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:42.018075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ddec2-74aa-440a-8d56-fdad5505456b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:42.036659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ddec2-74aa-440a-8d56-fdad5505456b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:42.043221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6685e7-9f07-41fb-b850-176b299f9afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:42.043909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6685e7-9f07-41fb-b850-176b299f9afa admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:42.044194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6685e7-9f07-41fb-b850-176b299f9afa admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:42.044463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6685e7-9f07-41fb-b850-176b299f9afa admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:42.046023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6685e7-9f07-41fb-b850-176b299f9afa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:42.046498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6685e7-9f07-41fb-b850-176b299f9afa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:42.048731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6685e7-9f07-41fb-b850-176b299f9afa admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:42.077705 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3266/6562] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:42.082425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37f285-153d-4fce-a098-610c6f28ce1f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:42.115944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37f285-153d-4fce-a098-610c6f28ce1f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:42.131840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc1ac5-83da-46d0-bdad-23694ebd3c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:42.133136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc1ac5-83da-46d0-bdad-23694ebd3c3d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:42.133435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc1ac5-83da-46d0-bdad-23694ebd3c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:42.133777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc1ac5-83da-46d0-bdad-23694ebd3c3d admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:42.138004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc1ac5-83da-46d0-bdad-23694ebd3c3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:42.141734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc1ac5-83da-46d0-bdad-23694ebd3c3d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:42.314997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:42.329893 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3297/6563] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:42 2022] POST /identity/v3/projects => generated 389 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:42.330168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6685e7-9f07-41fb-b850-176b299f9afa admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:42.346388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6685e7-9f07-41fb-b850-176b299f9afa admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:42.424231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193dafe7-444e-4fc5-ba69-b150df62329c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:42.424919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193dafe7-444e-4fc5-ba69-b150df62329c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:42.428258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193dafe7-444e-4fc5-ba69-b150df62329c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:42.428258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193dafe7-444e-4fc5-ba69-b150df62329c admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:42.428258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193dafe7-444e-4fc5-ba69-b150df62329c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:42.428258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193dafe7-444e-4fc5-ba69-b150df62329c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:42.528301 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3267/6564] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:42 2022] POST /identity/v3/users => generated 295 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:26:42.532374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc1ac5-83da-46d0-bdad-23694ebd3c3d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:42.544901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc1ac5-83da-46d0-bdad-23694ebd3c3d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:42.551036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74784699-a921-48c4-8c08-cbcf0e91c00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:42.551036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74784699-a921-48c4-8c08-cbcf0e91c00c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:42.551941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74784699-a921-48c4-8c08-cbcf0e91c00c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:42.551941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74784699-a921-48c4-8c08-cbcf0e91c00c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:42.552214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74784699-a921-48c4-8c08-cbcf0e91c00c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:42.553790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74784699-a921-48c4-8c08-cbcf0e91c00c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:42.568236 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3298/6565] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:26:42.570446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193dafe7-444e-4fc5-ba69-b150df62329c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:42.581784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193dafe7-444e-4fc5-ba69-b150df62329c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:42.587951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88128e7a-2580-462b-a6a1-dd32df46d387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:42.589122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88128e7a-2580-462b-a6a1-dd32df46d387 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:42.589514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88128e7a-2580-462b-a6a1-dd32df46d387 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:42.589885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88128e7a-2580-462b-a6a1-dd32df46d387 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:42.590723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88128e7a-2580-462b-a6a1-dd32df46d387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:42.592462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88128e7a-2580-462b-a6a1-dd32df46d387 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:42.606624 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3268/6566] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:26:42.611570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74784699-a921-48c4-8c08-cbcf0e91c00c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:42.622604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74784699-a921-48c4-8c08-cbcf0e91c00c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:42.630903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41179b-f0c6-4562-9c95-e07c5cd74984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:42.632949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41179b-f0c6-4562-9c95-e07c5cd74984 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:42.633454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41179b-f0c6-4562-9c95-e07c5cd74984 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:42.633963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c41179b-f0c6-4562-9c95-e07c5cd74984 admin admin] PATH_INFO: `/v3/projects/05b7dbe542c947208257997235c401ee/users/34f80b87840d40a9a4d9d5e5d2c79a34/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:42.655875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c41179b-f0c6-4562-9c95-e07c5cd74984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b7dbe542c947208257997235c401ee, user_id=34f80b87840d40a9a4d9d5e5d2c79a34, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:42.659667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c41179b-f0c6-4562-9c95-e07c5cd74984 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:42.802544 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3299/6567] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:42 2022] PUT /identity/v3/projects/05b7dbe542c947208257997235c401ee/users/34f80b87840d40a9a4d9d5e5d2c79a34/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:42.807637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02e185-bc81-41d3-a967-464ea3036d8d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:42.807637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02e185-bc81-41d3-a967-464ea3036d8d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:42.807637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02e185-bc81-41d3-a967-464ea3036d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:42.870649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-da02e185-bc81-41d3-a967-464ea3036d8d None None] MFA Rules not processed for user `34f80b87840d40a9a4d9d5e5d2c79a34`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:42.873474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-da02e185-bc81-41d3-a967-464ea3036d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:42.970552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41179b-f0c6-4562-9c95-e07c5cd74984 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:42.975164 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3269/6568] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:43.077719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c41179b-f0c6-4562-9c95-e07c5cd74984 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:43.083499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e99249-0fb4-4b8d-9248-24783f6b3bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:43.084187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e99249-0fb4-4b8d-9248-24783f6b3bf9 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:43.084508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e99249-0fb4-4b8d-9248-24783f6b3bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:43.084790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e99249-0fb4-4b8d-9248-24783f6b3bf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:43.085670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e99249-0fb4-4b8d-9248-24783f6b3bf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:43.087738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e99249-0fb4-4b8d-9248-24783f6b3bf9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:43.251793 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3300/6569] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:42 2022] POST /identity/v3/projects => generated 391 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:43.255743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02e185-bc81-41d3-a967-464ea3036d8d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:43.259700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:43.272430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02e185-bc81-41d3-a967-464ea3036d8d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:43.381608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6833c-eedb-4349-81c6-03c16e13b30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:43.381608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6833c-eedb-4349-81c6-03c16e13b30c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:43.381608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6833c-eedb-4349-81c6-03c16e13b30c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:43.381608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6833c-eedb-4349-81c6-03c16e13b30c admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:43.382235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6833c-eedb-4349-81c6-03c16e13b30c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:43.382546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6833c-eedb-4349-81c6-03c16e13b30c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:43.475660 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3270/6570] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:43 2022] POST /identity/v3/users => generated 302 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:43.480021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e99249-0fb4-4b8d-9248-24783f6b3bf9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:43.503914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e99249-0fb4-4b8d-9248-24783f6b3bf9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:43.525879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce824c9-f45b-42c1-b393-16fe4af991b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:43.526578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce824c9-f45b-42c1-b393-16fe4af991b2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:43.526864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce824c9-f45b-42c1-b393-16fe4af991b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:43.527144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce824c9-f45b-42c1-b393-16fe4af991b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:43.527953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce824c9-f45b-42c1-b393-16fe4af991b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:43.531129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce824c9-f45b-42c1-b393-16fe4af991b2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:43.564043 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3301/6571] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:43.566813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6833c-eedb-4349-81c6-03c16e13b30c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:43.580699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6833c-eedb-4349-81c6-03c16e13b30c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:43.587122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bddc7b-4efd-4398-a1bf-8e07d20ad7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:43.589302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bddc7b-4efd-4398-a1bf-8e07d20ad7ff admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:43.589801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bddc7b-4efd-4398-a1bf-8e07d20ad7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:43.590424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bddc7b-4efd-4398-a1bf-8e07d20ad7ff admin admin] PATH_INFO: `/v3/projects/4b5f0721a04c40abbe499605937e48c9/users/1797230aa4884648a350efee03638d94/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:43.610279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bddc7b-4efd-4398-a1bf-8e07d20ad7ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b5f0721a04c40abbe499605937e48c9, user_id=1797230aa4884648a350efee03638d94, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:43.613615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bddc7b-4efd-4398-a1bf-8e07d20ad7ff admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:43.712973 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3271/6572] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:43 2022] PUT /identity/v3/projects/4b5f0721a04c40abbe499605937e48c9/users/1797230aa4884648a350efee03638d94/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:43.717183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce824c9-f45b-42c1-b393-16fe4af991b2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:43.730590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce824c9-f45b-42c1-b393-16fe4af991b2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:43.799638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4daf8-7893-4b59-9c5d-44116a660b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:43.800865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4daf8-7893-4b59-9c5d-44116a660b66 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:43.801400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4daf8-7893-4b59-9c5d-44116a660b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:43.801676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4daf8-7893-4b59-9c5d-44116a660b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:43.802616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4daf8-7893-4b59-9c5d-44116a660b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:43.804512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4daf8-7893-4b59-9c5d-44116a660b66 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:43.819223 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3302/6573] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:43.822870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bddc7b-4efd-4398-a1bf-8e07d20ad7ff admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:43.841768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bddc7b-4efd-4398-a1bf-8e07d20ad7ff admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:43.850062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6600e68-7f45-4d31-b1fa-61bad18c2a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:43.852198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6600e68-7f45-4d31-b1fa-61bad18c2a91 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:43.852697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6600e68-7f45-4d31-b1fa-61bad18c2a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:43.853029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6600e68-7f45-4d31-b1fa-61bad18c2a91 admin admin] PATH_INFO: `/v3/projects/4b5f0721a04c40abbe499605937e48c9/users/1797230aa4884648a350efee03638d94/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:43.855190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6600e68-7f45-4d31-b1fa-61bad18c2a91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b5f0721a04c40abbe499605937e48c9, user_id=1797230aa4884648a350efee03638d94, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:43.858411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6600e68-7f45-4d31-b1fa-61bad18c2a91 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:43.870623 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3272/6574] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:43 2022] PUT /identity/v3/projects/4b5f0721a04c40abbe499605937e48c9/users/1797230aa4884648a350efee03638d94/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:43.881768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f21340-4fea-457e-9b87-65a5f268fa80 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:43.882044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f21340-4fea-457e-9b87-65a5f268fa80 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:43.883032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f21340-4fea-457e-9b87-65a5f268fa80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:43.922539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-87f21340-4fea-457e-9b87-65a5f268fa80 None None] MFA Rules not processed for user `1797230aa4884648a350efee03638d94`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:43.924342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-87f21340-4fea-457e-9b87-65a5f268fa80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:43.967263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:44.083486 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3303/6575] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:43 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:44.095138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6600e68-7f45-4d31-b1fa-61bad18c2a91 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:44.098447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b6600e68-7f45-4d31-b1fa-61bad18c2a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:44.217725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6600e68-7f45-4d31-b1fa-61bad18c2a91 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:44.222117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ece903-8dd1-4ca7-9936-8e05042502f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:44.223034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ece903-8dd1-4ca7-9936-8e05042502f5 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:44.223508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ece903-8dd1-4ca7-9936-8e05042502f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:44.223791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ece903-8dd1-4ca7-9936-8e05042502f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:44.307584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ece903-8dd1-4ca7-9936-8e05042502f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:44.309409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ece903-8dd1-4ca7-9936-8e05042502f5 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:44.321113 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3273/6576] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:44.950848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:UPDATE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:45.722172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:46.909249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:47.812429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:49.149584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:49.177798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f21340-4fea-457e-9b87-65a5f268fa80 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:49.181409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-87f21340-4fea-457e-9b87-65a5f268fa80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:49.205735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f21340-4fea-457e-9b87-65a5f268fa80 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:49.209689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120ccff5-1e56-4469-b93f-026a555df797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:49.211089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ccff5-1e56-4469-b93f-026a555df797 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:49.211448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ccff5-1e56-4469-b93f-026a555df797 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:49.212304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120ccff5-1e56-4469-b93f-026a555df797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:49.224335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ccff5-1e56-4469-b93f-026a555df797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:49.227713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120ccff5-1e56-4469-b93f-026a555df797 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:49.242868 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3304/6577] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:49.557459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:49.587698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ece903-8dd1-4ca7-9936-8e05042502f5 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:49.589858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a9ece903-8dd1-4ca7-9936-8e05042502f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:49.664836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ece903-8dd1-4ca7-9936-8e05042502f5 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:49.670533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd6185-1123-431c-b5ee-03b239b758ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:49.671433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd6185-1123-431c-b5ee-03b239b758ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:49.674101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd6185-1123-431c-b5ee-03b239b758ff admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:49.674101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd6185-1123-431c-b5ee-03b239b758ff admin admin] PATH_INFO: `/v3/users/7371c0456d29408b8523989c0a1d75ba` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:49.674101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd6185-1123-431c-b5ee-03b239b758ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=7371c0456d29408b8523989c0a1d75ba)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:49.675980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd6185-1123-431c-b5ee-03b239b758ff admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:49.920564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-29bd6185-1123-431c-b5ee-03b239b758ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7371c0456d29408b8523989c0a1d75ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:49.925129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-29bd6185-1123-431c-b5ee-03b239b758ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7371c0456d29408b8523989c0a1d75ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:49.930822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-29bd6185-1123-431c-b5ee-03b239b758ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7371c0456d29408b8523989c0a1d75ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:50.002050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-29bd6185-1123-431c-b5ee-03b239b758ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7371c0456d29408b8523989c0a1d75ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:50.002729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-29bd6185-1123-431c-b5ee-03b239b758ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7371c0456d29408b8523989c0a1d75ba'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:50.069937 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3274/6578] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:49 2022] DELETE /identity/v3/users/7371c0456d29408b8523989c0a1d75ba => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:50.075311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768ac84-b85b-4369-9866-f030d223ce7b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:50.075602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768ac84-b85b-4369-9866-f030d223ce7b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:50.076051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768ac84-b85b-4369-9866-f030d223ce7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:50.110610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e768ac84-b85b-4369-9866-f030d223ce7b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:50.112073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e768ac84-b85b-4369-9866-f030d223ce7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:50.183871 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3305/6579] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:50.196348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd6185-1123-431c-b5ee-03b239b758ff admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:50.197830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-29bd6185-1123-431c-b5ee-03b239b758ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:50.288707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd6185-1123-431c-b5ee-03b239b758ff admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:50.292591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43da82b-5a33-473e-94bf-8964a75e9039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:50.293251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43da82b-5a33-473e-94bf-8964a75e9039 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:50.293538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43da82b-5a33-473e-94bf-8964a75e9039 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:50.293818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43da82b-5a33-473e-94bf-8964a75e9039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:50.303720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43da82b-5a33-473e-94bf-8964a75e9039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:50.305426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43da82b-5a33-473e-94bf-8964a75e9039 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:50.315608 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3275/6580] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:50.881049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768ac84-b85b-4369-9866-f030d223ce7b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:50.882756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e768ac84-b85b-4369-9866-f030d223ce7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:50.901593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768ac84-b85b-4369-9866-f030d223ce7b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:50.907880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:50.910798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:50.915110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:50.915110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a admin admin] PATH_INFO: `/v3/projects/2cc7b77a514e46ffbd027af670ce667f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:50.915110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cc7b77a514e46ffbd027af670ce667f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:50.918152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:51.151585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cc7b77a514e46ffbd027af670ce667f', 'request_id': 'req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:51.166902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:51.173034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cc7b77a514e46ffbd027af670ce667f', 'request_id': 'req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:51.264234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cc7b77a514e46ffbd027af670ce667f', 'request_id': 'req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:51.280098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a admin admin] The token cache is being invalidate because project 2cc7b77a514e46ffbd027af670ce667f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:51.280396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:51.281074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:51.285763 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3306/6581] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:26:50 2022] DELETE /identity/v3/projects/2cc7b77a514e46ffbd027af670ce667f => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:51.287561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43da82b-5a33-473e-94bf-8964a75e9039 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:51.299916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43da82b-5a33-473e-94bf-8964a75e9039 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:51.463384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b5ba7-d188-48cb-9f68-dc3d6db6a25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:51.466643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b5ba7-d188-48cb-9f68-dc3d6db6a25e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:51.466643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b5ba7-d188-48cb-9f68-dc3d6db6a25e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:51.467978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b5ba7-d188-48cb-9f68-dc3d6db6a25e admin admin] PATH_INFO: `/v3/users/23e706bdbc8c4423b2616997c328f737` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:51.472129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b5ba7-d188-48cb-9f68-dc3d6db6a25e admin admin] RBAC: Authorizing `identity:delete_user(user_id=23e706bdbc8c4423b2616997c328f737)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:51.477283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b5ba7-d188-48cb-9f68-dc3d6db6a25e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:51.516417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:51.746516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2b0b5ba7-d188-48cb-9f68-dc3d6db6a25e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23e706bdbc8c4423b2616997c328f737'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:51.747329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2b0b5ba7-d188-48cb-9f68-dc3d6db6a25e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23e706bdbc8c4423b2616997c328f737'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:51.755847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2b0b5ba7-d188-48cb-9f68-dc3d6db6a25e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23e706bdbc8c4423b2616997c328f737'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:51.859598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2b0b5ba7-d188-48cb-9f68-dc3d6db6a25e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23e706bdbc8c4423b2616997c328f737'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:51.870691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2b0b5ba7-d188-48cb-9f68-dc3d6db6a25e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23e706bdbc8c4423b2616997c328f737'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:51.926559 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3276/6582] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:51 2022] DELETE /identity/v3/users/23e706bdbc8c4423b2616997c328f737 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:52.520995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:52.857344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:52.859969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:52.892365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b5ba7-d188-48cb-9f68-dc3d6db6a25e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:52.895261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2b0b5ba7-d188-48cb-9f68-dc3d6db6a25e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:52.969310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9598ebb-e4e6-46d5-a5d7-748a78d7ca3a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:53.020336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b5ba7-d188-48cb-9f68-dc3d6db6a25e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:53.028671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee9392-5594-4ddd-baff-9a4082685605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:53.029745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee9392-5594-4ddd-baff-9a4082685605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:53.030073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee9392-5594-4ddd-baff-9a4082685605 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:53.030359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee9392-5594-4ddd-baff-9a4082685605 admin admin] PATH_INFO: `/v3/projects/82cf103697074afcb641b3da49f5761a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:53.033407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee9392-5594-4ddd-baff-9a4082685605 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82cf103697074afcb641b3da49f5761a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:53.038938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee9392-5594-4ddd-baff-9a4082685605 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:53.059205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e31fc-d1bd-41d4-9dab-c071ada74406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:53.060501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e31fc-d1bd-41d4-9dab-c071ada74406 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:53.061287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e31fc-d1bd-41d4-9dab-c071ada74406 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:53.061738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242e31fc-d1bd-41d4-9dab-c071ada74406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:53.070802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-242e31fc-d1bd-41d4-9dab-c071ada74406 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:53.182575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9eee9392-5594-4ddd-baff-9a4082685605 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82cf103697074afcb641b3da49f5761a', 'request_id': 'req-9eee9392-5594-4ddd-baff-9a4082685605'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:53.186815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e31fc-d1bd-41d4-9dab-c071ada74406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:53.189269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9eee9392-5594-4ddd-baff-9a4082685605 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82cf103697074afcb641b3da49f5761a', 'request_id': 'req-9eee9392-5594-4ddd-baff-9a4082685605'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:53.189411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242e31fc-d1bd-41d4-9dab-c071ada74406 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:53.207536 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3307/6583] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:53.286884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9eee9392-5594-4ddd-baff-9a4082685605 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82cf103697074afcb641b3da49f5761a', 'request_id': 'req-9eee9392-5594-4ddd-baff-9a4082685605'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:53.312131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9eee9392-5594-4ddd-baff-9a4082685605 admin admin] The token cache is being invalidate because project 82cf103697074afcb641b3da49f5761a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:53.313575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9eee9392-5594-4ddd-baff-9a4082685605 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:53.314834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9eee9392-5594-4ddd-baff-9a4082685605 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:53.319096 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3277/6584] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:26:52 2022] DELETE /identity/v3/projects/82cf103697074afcb641b3da49f5761a => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:53.324483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e31fc-d1bd-41d4-9dab-c071ada74406 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:53.340628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242e31fc-d1bd-41d4-9dab-c071ada74406 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:53.526990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3fe41-0869-4207-9c91-649d36a9f605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:53.527980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3fe41-0869-4207-9c91-649d36a9f605 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:53.528331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3fe41-0869-4207-9c91-649d36a9f605 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:53.528683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3fe41-0869-4207-9c91-649d36a9f605 admin admin] PATH_INFO: `/v3/users/b97daa65b17c4d93a668db600666e37d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:53.530433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3fe41-0869-4207-9c91-649d36a9f605 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b97daa65b17c4d93a668db600666e37d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:53.532864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3fe41-0869-4207-9c91-649d36a9f605 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:53.804157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-41d3fe41-0869-4207-9c91-649d36a9f605 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b97daa65b17c4d93a668db600666e37d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:53.811199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-41d3fe41-0869-4207-9c91-649d36a9f605 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b97daa65b17c4d93a668db600666e37d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:53.812055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-41d3fe41-0869-4207-9c91-649d36a9f605 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b97daa65b17c4d93a668db600666e37d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:53.812906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-41d3fe41-0869-4207-9c91-649d36a9f605 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b97daa65b17c4d93a668db600666e37d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:53.876157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-41d3fe41-0869-4207-9c91-649d36a9f605 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b97daa65b17c4d93a668db600666e37d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:53.899983 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3308/6585] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:53 2022] DELETE /identity/v3/users/b97daa65b17c4d93a668db600666e37d => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:54.039653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:54.235866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:54.453103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee9392-5594-4ddd-baff-9a4082685605 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:54.454698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9eee9392-5594-4ddd-baff-9a4082685605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:54.475817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee9392-5594-4ddd-baff-9a4082685605 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:54.559784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8823d1-c16d-4d55-9b51-fb5bd3c796a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:54.561989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8823d1-c16d-4d55-9b51-fb5bd3c796a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:54.562519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8823d1-c16d-4d55-9b51-fb5bd3c796a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:54.563123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8823d1-c16d-4d55-9b51-fb5bd3c796a9 admin admin] PATH_INFO: `/v3/users/98f2a78a037c417ea3cff70dab6b0e99` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:54.564587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8823d1-c16d-4d55-9b51-fb5bd3c796a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98f2a78a037c417ea3cff70dab6b0e99)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:54.566938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8823d1-c16d-4d55-9b51-fb5bd3c796a9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:54.858249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2d8823d1-c16d-4d55-9b51-fb5bd3c796a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98f2a78a037c417ea3cff70dab6b0e99'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:54.859293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2d8823d1-c16d-4d55-9b51-fb5bd3c796a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98f2a78a037c417ea3cff70dab6b0e99'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:54.869706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3fe41-0869-4207-9c91-649d36a9f605 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:54.871832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-41d3fe41-0869-4207-9c91-649d36a9f605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:54.876649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2d8823d1-c16d-4d55-9b51-fb5bd3c796a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98f2a78a037c417ea3cff70dab6b0e99'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:54.890646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3fe41-0869-4207-9c91-649d36a9f605 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:54.899268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34c9c0-bcd0-445e-94ea-6544b1df56f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:54.900176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34c9c0-bcd0-445e-94ea-6544b1df56f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:54.900464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34c9c0-bcd0-445e-94ea-6544b1df56f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:54.900745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d34c9c0-bcd0-445e-94ea-6544b1df56f7 admin admin] PATH_INFO: `/v3/users/34f80b87840d40a9a4d9d5e5d2c79a34` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:54.902073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34c9c0-bcd0-445e-94ea-6544b1df56f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34f80b87840d40a9a4d9d5e5d2c79a34)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:54.904023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d34c9c0-bcd0-445e-94ea-6544b1df56f7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:54.966830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2d8823d1-c16d-4d55-9b51-fb5bd3c796a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98f2a78a037c417ea3cff70dab6b0e99'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:54.967458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2d8823d1-c16d-4d55-9b51-fb5bd3c796a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98f2a78a037c417ea3cff70dab6b0e99'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:54.999372 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3278/6586] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:54 2022] DELETE /identity/v3/users/98f2a78a037c417ea3cff70dab6b0e99 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:55.001244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6478c-2167-4179-8f25-205a97c3caf8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:55.001744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6478c-2167-4179-8f25-205a97c3caf8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:55.002141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6478c-2167-4179-8f25-205a97c3caf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:55.006445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:UPDATE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:55.032649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7af6478c-2167-4179-8f25-205a97c3caf8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:55.034617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7af6478c-2167-4179-8f25-205a97c3caf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:55.135323 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3279/6587] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:55.144263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6478c-2167-4179-8f25-205a97c3caf8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:55.144263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7af6478c-2167-4179-8f25-205a97c3caf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:55.148940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d34c9c0-bcd0-445e-94ea-6544b1df56f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34f80b87840d40a9a4d9d5e5d2c79a34'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:55.157911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d34c9c0-bcd0-445e-94ea-6544b1df56f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34f80b87840d40a9a4d9d5e5d2c79a34'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:55.159183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d34c9c0-bcd0-445e-94ea-6544b1df56f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34f80b87840d40a9a4d9d5e5d2c79a34'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:55.159326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6478c-2167-4179-8f25-205a97c3caf8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:55.160176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d34c9c0-bcd0-445e-94ea-6544b1df56f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34f80b87840d40a9a4d9d5e5d2c79a34'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:55.164503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c875a-b5b8-4a83-af03-c64f2e30f052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:55.167065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c875a-b5b8-4a83-af03-c64f2e30f052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:55.167065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c875a-b5b8-4a83-af03-c64f2e30f052 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:55.167065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c875a-b5b8-4a83-af03-c64f2e30f052 admin admin] PATH_INFO: `/v3/projects/62a17c0b6f9d41d0a2c29db7cd06ee92` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:55.167848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c875a-b5b8-4a83-af03-c64f2e30f052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62a17c0b6f9d41d0a2c29db7cd06ee92)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:55.170167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960c875a-b5b8-4a83-af03-c64f2e30f052 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:55.212044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d34c9c0-bcd0-445e-94ea-6544b1df56f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34f80b87840d40a9a4d9d5e5d2c79a34'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:55.241609 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3309/6588] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:54 2022] DELETE /identity/v3/users/34f80b87840d40a9a4d9d5e5d2c79a34 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:55.242888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34c9c0-bcd0-445e-94ea-6544b1df56f7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:55.244594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5d34c9c0-bcd0-445e-94ea-6544b1df56f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:55.322770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-960c875a-b5b8-4a83-af03-c64f2e30f052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62a17c0b6f9d41d0a2c29db7cd06ee92', 'request_id': 'req-960c875a-b5b8-4a83-af03-c64f2e30f052'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:55.330286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-960c875a-b5b8-4a83-af03-c64f2e30f052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62a17c0b6f9d41d0a2c29db7cd06ee92', 'request_id': 'req-960c875a-b5b8-4a83-af03-c64f2e30f052'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:55.341126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d34c9c0-bcd0-445e-94ea-6544b1df56f7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:55.389818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-960c875a-b5b8-4a83-af03-c64f2e30f052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62a17c0b6f9d41d0a2c29db7cd06ee92', 'request_id': 'req-960c875a-b5b8-4a83-af03-c64f2e30f052'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:55.400808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06220b22-6ea8-499e-b1df-09f67369eb68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:55.401525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06220b22-6ea8-499e-b1df-09f67369eb68 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:55.402036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06220b22-6ea8-499e-b1df-09f67369eb68 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:55.402459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06220b22-6ea8-499e-b1df-09f67369eb68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:55.403732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-960c875a-b5b8-4a83-af03-c64f2e30f052 admin admin] The token cache is being invalidate because project 62a17c0b6f9d41d0a2c29db7cd06ee92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:55.404033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-960c875a-b5b8-4a83-af03-c64f2e30f052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:55.404699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-960c875a-b5b8-4a83-af03-c64f2e30f052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:55.405323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-06220b22-6ea8-499e-b1df-09f67369eb68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:55.408334 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3280/6589] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:26:55 2022] DELETE /identity/v3/projects/62a17c0b6f9d41d0a2c29db7cd06ee92 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:55.409891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2443ca-8aca-4566-b268-683b21b9c8df None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:55.410034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2443ca-8aca-4566-b268-683b21b9c8df None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:55.418630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2443ca-8aca-4566-b268-683b21b9c8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:55.428398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4a2443ca-8aca-4566-b268-683b21b9c8df None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:55.430021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4a2443ca-8aca-4566-b268-683b21b9c8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:55.485916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06220b22-6ea8-499e-b1df-09f67369eb68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:55.487478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06220b22-6ea8-499e-b1df-09f67369eb68 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:55.498776 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3310/6590] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:55.500404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bb5cc-8f66-45a6-830b-d97354095774 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:55.500856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bb5cc-8f66-45a6-830b-d97354095774 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:55.501424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bb5cc-8f66-45a6-830b-d97354095774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:55.511228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:55.521340 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3281/6591] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:55.526193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-470bb5cc-8f66-45a6-830b-d97354095774 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:55.528580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-470bb5cc-8f66-45a6-830b-d97354095774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:55.532071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2443ca-8aca-4566-b268-683b21b9c8df None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:55.534253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4a2443ca-8aca-4566-b268-683b21b9c8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:55.540880 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3311/6592] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:55.551435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd11c5-f295-4928-931f-eb6faab7b018 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:55.552130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd11c5-f295-4928-931f-eb6faab7b018 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:55.552835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebd11c5-f295-4928-931f-eb6faab7b018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:55.575908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2443ca-8aca-4566-b268-683b21b9c8df None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:55.585130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ab077-5ee9-4c83-a6f2-7560e346813f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:55.585994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ab077-5ee9-4c83-a6f2-7560e346813f service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:55.586284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ab077-5ee9-4c83-a6f2-7560e346813f service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:55.586560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ab077-5ee9-4c83-a6f2-7560e346813f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:55.592630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4ebd11c5-f295-4928-931f-eb6faab7b018 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:55.594965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4ebd11c5-f295-4928-931f-eb6faab7b018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:55.618912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ab077-5ee9-4c83-a6f2-7560e346813f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:55.623264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ab077-5ee9-4c83-a6f2-7560e346813f service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:55.629291 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3312/6593] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:55.630566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd11c5-f295-4928-931f-eb6faab7b018 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:55.649090 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3282/6594] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:26:55.651034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebd11c5-f295-4928-931f-eb6faab7b018 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:55.657619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3221cbe-f8d2-4202-95f9-a340a391c6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:55.658455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3221cbe-f8d2-4202-95f9-a340a391c6ec admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:55.658806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3221cbe-f8d2-4202-95f9-a340a391c6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:55.659157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3221cbe-f8d2-4202-95f9-a340a391c6ec admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:55.660035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3221cbe-f8d2-4202-95f9-a340a391c6ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:55.660602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3221cbe-f8d2-4202-95f9-a340a391c6ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:55.662494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3221cbe-f8d2-4202-95f9-a340a391c6ec admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:55.679907 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3313/6595] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:55.681899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ab077-5ee9-4c83-a6f2-7560e346813f service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:55.697758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ab077-5ee9-4c83-a6f2-7560e346813f service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:55.710580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c6e4e-8f90-4731-95eb-c76c3dd52a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:55.712661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c6e4e-8f90-4731-95eb-c76c3dd52a32 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:55.713227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c6e4e-8f90-4731-95eb-c76c3dd52a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:55.714241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398c6e4e-8f90-4731-95eb-c76c3dd52a32 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:55.715254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c6e4e-8f90-4731-95eb-c76c3dd52a32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:55.723793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398c6e4e-8f90-4731-95eb-c76c3dd52a32 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:55.892926 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3283/6596] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:55 2022] POST /identity/v3/projects => generated 371 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:55.896884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3221cbe-f8d2-4202-95f9-a340a391c6ec admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:55.913461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3221cbe-f8d2-4202-95f9-a340a391c6ec admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:56.033262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85e8c8-075d-4a0b-9e30-2c1879869fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:56.034293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85e8c8-075d-4a0b-9e30-2c1879869fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:56.034768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85e8c8-075d-4a0b-9e30-2c1879869fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:56.035212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85e8c8-075d-4a0b-9e30-2c1879869fc7 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:56.036153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85e8c8-075d-4a0b-9e30-2c1879869fc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:56.038896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85e8c8-075d-4a0b-9e30-2c1879869fc7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:56.070379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c6e4e-8f90-4731-95eb-c76c3dd52a32 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:56.073621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-398c6e4e-8f90-4731-95eb-c76c3dd52a32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:56.138978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398c6e4e-8f90-4731-95eb-c76c3dd52a32 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:56.151566 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3314/6597] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:55 2022] POST /identity/v3/users => generated 286 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:56.151818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29bed96-3651-46be-bd9b-832067aab843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:56.152918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bed96-3651-46be-bd9b-832067aab843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:56.153209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bed96-3651-46be-bd9b-832067aab843 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:56.156295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29bed96-3651-46be-bd9b-832067aab843 admin admin] PATH_INFO: `/v3/projects/05b7dbe542c947208257997235c401ee` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:56.158499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bed96-3651-46be-bd9b-832067aab843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05b7dbe542c947208257997235c401ee)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:56.161791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29bed96-3651-46be-bd9b-832067aab843 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:56.162341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85e8c8-075d-4a0b-9e30-2c1879869fc7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:56.187518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85e8c8-075d-4a0b-9e30-2c1879869fc7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:56.199997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3ec4b-17b8-48cd-b513-f54286f4b0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:56.202669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3ec4b-17b8-48cd-b513-f54286f4b0af admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:56.202955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3ec4b-17b8-48cd-b513-f54286f4b0af admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:56.203232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac3ec4b-17b8-48cd-b513-f54286f4b0af admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:56.203967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac3ec4b-17b8-48cd-b513-f54286f4b0af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:56.206990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac3ec4b-17b8-48cd-b513-f54286f4b0af admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:56.218993 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3315/6598] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:56.223840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3ec4b-17b8-48cd-b513-f54286f4b0af admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:56.242100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac3ec4b-17b8-48cd-b513-f54286f4b0af admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:56.248608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fa313-9ac1-4692-b54f-af32ab6aff36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:56.251265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fa313-9ac1-4692-b54f-af32ab6aff36 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:56.254729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fa313-9ac1-4692-b54f-af32ab6aff36 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:56.254729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fa313-9ac1-4692-b54f-af32ab6aff36 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:56.254948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fa313-9ac1-4692-b54f-af32ab6aff36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:56.256712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fa313-9ac1-4692-b54f-af32ab6aff36 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:56.270155 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3316/6599] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:26:56.280800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fa313-9ac1-4692-b54f-af32ab6aff36 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:56.292459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fa313-9ac1-4692-b54f-af32ab6aff36 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:56.298947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdd3da-2ae4-48aa-bd89-ea6c96b13136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:56.300104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdd3da-2ae4-48aa-bd89-ea6c96b13136 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:56.303891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdd3da-2ae4-48aa-bd89-ea6c96b13136 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:56.303891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdd3da-2ae4-48aa-bd89-ea6c96b13136 admin admin] PATH_INFO: `/v3/projects/a097d0a752f8498a93e37e68e619a99f/users/82fcd8933bf6495ea6ecc17d8cefeb48/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:56.327334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdd3da-2ae4-48aa-bd89-ea6c96b13136 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a097d0a752f8498a93e37e68e619a99f, user_id=82fcd8933bf6495ea6ecc17d8cefeb48, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:56.330339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdd3da-2ae4-48aa-bd89-ea6c96b13136 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:56.341626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c29bed96-3651-46be-bd9b-832067aab843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05b7dbe542c947208257997235c401ee', 'request_id': 'req-c29bed96-3651-46be-bd9b-832067aab843'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:56.348045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c29bed96-3651-46be-bd9b-832067aab843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05b7dbe542c947208257997235c401ee', 'request_id': 'req-c29bed96-3651-46be-bd9b-832067aab843'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:56.430498 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3317/6600] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:56 2022] PUT /identity/v3/projects/a097d0a752f8498a93e37e68e619a99f/users/82fcd8933bf6495ea6ecc17d8cefeb48/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:26:56.431688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdd3da-2ae4-48aa-bd89-ea6c96b13136 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:56.433570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5ccdd3da-2ae4-48aa-bd89-ea6c96b13136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:56.444026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c29bed96-3651-46be-bd9b-832067aab843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05b7dbe542c947208257997235c401ee', 'request_id': 'req-c29bed96-3651-46be-bd9b-832067aab843'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:56.459997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c29bed96-3651-46be-bd9b-832067aab843 admin admin] The token cache is being invalidate because project 05b7dbe542c947208257997235c401ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:56.460478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c29bed96-3651-46be-bd9b-832067aab843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:56.461392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c29bed96-3651-46be-bd9b-832067aab843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:56.466237 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3284/6601] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:26:56 2022] DELETE /identity/v3/projects/05b7dbe542c947208257997235c401ee => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:56.467934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d44e4-5c83-4eec-8434-503202705e53 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:56.468383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d44e4-5c83-4eec-8434-503202705e53 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:56.468959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48d44e4-5c83-4eec-8434-503202705e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:56.526983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d48d44e4-5c83-4eec-8434-503202705e53 None None] MFA Rules not processed for user `82fcd8933bf6495ea6ecc17d8cefeb48`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:56.532106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d48d44e4-5c83-4eec-8434-503202705e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:56.540629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdd3da-2ae4-48aa-bd89-ea6c96b13136 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:56.544243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189a264-9a5e-47eb-935d-bce579ecf1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:56.545212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189a264-9a5e-47eb-935d-bce579ecf1c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:56.545527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189a264-9a5e-47eb-935d-bce579ecf1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:56.546783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189a264-9a5e-47eb-935d-bce579ecf1c4 admin admin] PATH_INFO: `/v3/projects/13837973df4240769de72a63520a0cbd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:56.548087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7189a264-9a5e-47eb-935d-bce579ecf1c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13837973df4240769de72a63520a0cbd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:56.550070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7189a264-9a5e-47eb-935d-bce579ecf1c4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:56.643085 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3285/6602] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:56.644251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d44e4-5c83-4eec-8434-503202705e53 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:56.645609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d48d44e4-5c83-4eec-8434-503202705e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:56.662152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48d44e4-5c83-4eec-8434-503202705e53 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:56.671475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3abff6-8a03-4b5e-a862-adbf0f4dd444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:56.672376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3abff6-8a03-4b5e-a862-adbf0f4dd444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:56.672811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3abff6-8a03-4b5e-a862-adbf0f4dd444 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:56.673096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3abff6-8a03-4b5e-a862-adbf0f4dd444 admin admin] PATH_INFO: `/v3/users/1797230aa4884648a350efee03638d94` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:56.674526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3abff6-8a03-4b5e-a862-adbf0f4dd444 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1797230aa4884648a350efee03638d94)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:56.676611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3abff6-8a03-4b5e-a862-adbf0f4dd444 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:56.715353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7189a264-9a5e-47eb-935d-bce579ecf1c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13837973df4240769de72a63520a0cbd', 'request_id': 'req-7189a264-9a5e-47eb-935d-bce579ecf1c4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:56.722285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7189a264-9a5e-47eb-935d-bce579ecf1c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13837973df4240769de72a63520a0cbd', 'request_id': 'req-7189a264-9a5e-47eb-935d-bce579ecf1c4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:56.800943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7189a264-9a5e-47eb-935d-bce579ecf1c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13837973df4240769de72a63520a0cbd', 'request_id': 'req-7189a264-9a5e-47eb-935d-bce579ecf1c4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:56.815592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7189a264-9a5e-47eb-935d-bce579ecf1c4 admin admin] The token cache is being invalidate because project 13837973df4240769de72a63520a0cbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:56.815968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7189a264-9a5e-47eb-935d-bce579ecf1c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:56.817051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7189a264-9a5e-47eb-935d-bce579ecf1c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:56.818571 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3318/6603] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:26:56 2022] DELETE /identity/v3/projects/13837973df4240769de72a63520a0cbd => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:56.819951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189a264-9a5e-47eb-935d-bce579ecf1c4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:56.821546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7189a264-9a5e-47eb-935d-bce579ecf1c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:56.922032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189a264-9a5e-47eb-935d-bce579ecf1c4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:56.932132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f976f57-8d8d-4ab4-bfcb-02947dc34bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:56.932879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f976f57-8d8d-4ab4-bfcb-02947dc34bba admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:56.933314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f976f57-8d8d-4ab4-bfcb-02947dc34bba admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:56.933444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f976f57-8d8d-4ab4-bfcb-02947dc34bba admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:56.934322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f976f57-8d8d-4ab4-bfcb-02947dc34bba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:56.936422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f976f57-8d8d-4ab4-bfcb-02947dc34bba admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:57.136980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4d3abff6-8a03-4b5e-a862-adbf0f4dd444 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1797230aa4884648a350efee03638d94'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:57.137652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4d3abff6-8a03-4b5e-a862-adbf0f4dd444 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1797230aa4884648a350efee03638d94'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:57.144867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4d3abff6-8a03-4b5e-a862-adbf0f4dd444 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1797230aa4884648a350efee03638d94'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:57.220449 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3319/6604] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:56 2022] POST /identity/v3/projects => generated 373 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:57.222338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486883e3-6624-4ea5-a0bb-ed0363e17f60 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:57.222632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486883e3-6624-4ea5-a0bb-ed0363e17f60 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:57.223081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486883e3-6624-4ea5-a0bb-ed0363e17f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:57.234981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4d3abff6-8a03-4b5e-a862-adbf0f4dd444 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1797230aa4884648a350efee03638d94'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:57.236493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:57.237135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4d3abff6-8a03-4b5e-a862-adbf0f4dd444 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1797230aa4884648a350efee03638d94'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:57.257761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-486883e3-6624-4ea5-a0bb-ed0363e17f60 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:57.259036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-486883e3-6624-4ea5-a0bb-ed0363e17f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:57.261877 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3286/6605] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:26:56 2022] DELETE /identity/v3/users/1797230aa4884648a350efee03638d94 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:57.266843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3abff6-8a03-4b5e-a862-adbf0f4dd444 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:57.266843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4d3abff6-8a03-4b5e-a862-adbf0f4dd444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:57.412883 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3320/6606] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:57.418220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfddf38-4a3d-4486-86e7-6a58fdeeb6a1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:57.418733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfddf38-4a3d-4486-86e7-6a58fdeeb6a1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:57.420655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfddf38-4a3d-4486-86e7-6a58fdeeb6a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:57.446901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3abff6-8a03-4b5e-a862-adbf0f4dd444 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:57.457948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6bfddf38-4a3d-4486-86e7-6a58fdeeb6a1 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:57.458981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2dd1e-9346-4e5b-affc-2b5f96a3a5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:57.460009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6bfddf38-4a3d-4486-86e7-6a58fdeeb6a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:57.460442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2dd1e-9346-4e5b-affc-2b5f96a3a5b9 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:57.460807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2dd1e-9346-4e5b-affc-2b5f96a3a5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:57.461236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2dd1e-9346-4e5b-affc-2b5f96a3a5b9 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:57.462440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2dd1e-9346-4e5b-affc-2b5f96a3a5b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:57.466790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce2dd1e-9346-4e5b-affc-2b5f96a3a5b9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:57.524131 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3321/6607] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:26:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:57.525238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276a69b-7bd7-4c21-bb51-0a2f58fa62b1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:57.525528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276a69b-7bd7-4c21-bb51-0a2f58fa62b1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:57.525982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4276a69b-7bd7-4c21-bb51-0a2f58fa62b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:57.546748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4276a69b-7bd7-4c21-bb51-0a2f58fa62b1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:57.547908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4276a69b-7bd7-4c21-bb51-0a2f58fa62b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:57.557999 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3322/6608] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:57.562348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276a69b-7bd7-4c21-bb51-0a2f58fa62b1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:57.563999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4276a69b-7bd7-4c21-bb51-0a2f58fa62b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:57.584385 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3287/6609] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:57 2022] POST /identity/v3/users => generated 293 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:57.584385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2dd1e-9346-4e5b-affc-2b5f96a3a5b9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:57.600165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2dd1e-9346-4e5b-affc-2b5f96a3a5b9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:57.618022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1adf21-5373-4b8c-995e-43af3339acec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:57.619275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1adf21-5373-4b8c-995e-43af3339acec admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:57.619829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1adf21-5373-4b8c-995e-43af3339acec admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:57.620275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1adf21-5373-4b8c-995e-43af3339acec admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:57.621318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1adf21-5373-4b8c-995e-43af3339acec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:57.622137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1adf21-5373-4b8c-995e-43af3339acec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:57.626075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1adf21-5373-4b8c-995e-43af3339acec admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:57.658904 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3288/6610] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:57.659514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1adf21-5373-4b8c-995e-43af3339acec admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:57.662647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4276a69b-7bd7-4c21-bb51-0a2f58fa62b1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:57.684613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1adf21-5373-4b8c-995e-43af3339acec admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:57.687137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecffee-685d-41a2-bd89-e88a1e512d76 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:57.688153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecffee-685d-41a2-bd89-e88a1e512d76 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:57.688745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecffee-685d-41a2-bd89-e88a1e512d76 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:57.689383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecffee-685d-41a2-bd89-e88a1e512d76 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:57.690683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ef253-aa2a-4560-a954-546feb162cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:57.695043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ef253-aa2a-4560-a954-546feb162cc1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:57.695043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ef253-aa2a-4560-a954-546feb162cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:57.695043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ef253-aa2a-4560-a954-546feb162cc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:57.695043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ef253-aa2a-4560-a954-546feb162cc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:57.695668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ef253-aa2a-4560-a954-546feb162cc1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:57.702173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ecffee-685d-41a2-bd89-e88a1e512d76 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:57.704268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ecffee-685d-41a2-bd89-e88a1e512d76 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:57.708385 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3289/6611] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:57.709138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ef253-aa2a-4560-a954-546feb162cc1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:57.716507 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3323/6612] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:26:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:26:57.718098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecffee-685d-41a2-bd89-e88a1e512d76 service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:57.722993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ef253-aa2a-4560-a954-546feb162cc1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:57.729508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fabb4-2427-4c36-9c03-2b0c2dc617d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:57.729809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecffee-685d-41a2-bd89-e88a1e512d76 service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:57.730486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fabb4-2427-4c36-9c03-2b0c2dc617d7 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:57.730970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fabb4-2427-4c36-9c03-2b0c2dc617d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:57.731170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fabb4-2427-4c36-9c03-2b0c2dc617d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:57.731941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655fabb4-2427-4c36-9c03-2b0c2dc617d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:57.734086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655fabb4-2427-4c36-9c03-2b0c2dc617d7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:57.736248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1708d15-4afb-4cf2-8113-27b363371112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:57.737487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1708d15-4afb-4cf2-8113-27b363371112 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:57.737972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1708d15-4afb-4cf2-8113-27b363371112 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:57.738412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1708d15-4afb-4cf2-8113-27b363371112 admin admin] PATH_INFO: `/v3/projects/3e5df82139b74685bfea7060efa740e8/users/afdf454219b740809c10e5feebd64116/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:57.764066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1708d15-4afb-4cf2-8113-27b363371112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e5df82139b74685bfea7060efa740e8, user_id=afdf454219b740809c10e5feebd64116, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:57.767611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1708d15-4afb-4cf2-8113-27b363371112 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:57.848514 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3290/6613] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:57 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:57.861231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fabb4-2427-4c36-9c03-2b0c2dc617d7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:57.875869 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3324/6614] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:57 2022] PUT /identity/v3/projects/3e5df82139b74685bfea7060efa740e8/users/afdf454219b740809c10e5feebd64116/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:57.880094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1708d15-4afb-4cf2-8113-27b363371112 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:58.033585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fabb4-2427-4c36-9c03-2b0c2dc617d7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:58.040153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8941a-ebb5-4531-ae78-af3a694f7a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:58.040841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8941a-ebb5-4531-ae78-af3a694f7a97 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:58.041136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8941a-ebb5-4531-ae78-af3a694f7a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:58.041556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8941a-ebb5-4531-ae78-af3a694f7a97 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:58.042255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8941a-ebb5-4531-ae78-af3a694f7a97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:58.044856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8941a-ebb5-4531-ae78-af3a694f7a97 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:58.056484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1708d15-4afb-4cf2-8113-27b363371112 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:58.072832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e61c4-694f-410d-857b-524541335506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:58.074523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e61c4-694f-410d-857b-524541335506 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:58.074523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e61c4-694f-410d-857b-524541335506 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:58.074523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e61c4-694f-410d-857b-524541335506 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:58.075366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e61c4-694f-410d-857b-524541335506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:58.077228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e61c4-694f-410d-857b-524541335506 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:58.090076 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3325/6615] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:58.092092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e61c4-694f-410d-857b-524541335506 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:58.094423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cf8e61c4-694f-410d-857b-524541335506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:58.111267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e61c4-694f-410d-857b-524541335506 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:58.119245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c68e6a5-9244-4473-8e1a-ca84c5533cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:58.120718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68e6a5-9244-4473-8e1a-ca84c5533cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:58.121570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68e6a5-9244-4473-8e1a-ca84c5533cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:58.122357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c68e6a5-9244-4473-8e1a-ca84c5533cfe admin admin] PATH_INFO: `/v3/projects/4b5f0721a04c40abbe499605937e48c9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:58.124366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68e6a5-9244-4473-8e1a-ca84c5533cfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b5f0721a04c40abbe499605937e48c9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:58.127234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c68e6a5-9244-4473-8e1a-ca84c5533cfe admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:58.173007 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3291/6616] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:57 2022] POST /identity/v3/users => generated 288 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:58.173007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8941a-ebb5-4531-ae78-af3a694f7a97 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:58.186626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8941a-ebb5-4531-ae78-af3a694f7a97 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:58.196525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae69eb2-50e0-4663-bf22-a0401d0e9218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:58.197670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae69eb2-50e0-4663-bf22-a0401d0e9218 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:58.198132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae69eb2-50e0-4663-bf22-a0401d0e9218 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:58.198301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae69eb2-50e0-4663-bf22-a0401d0e9218 admin admin] PATH_INFO: `/v3/projects/3e5df82139b74685bfea7060efa740e8/users/afdf454219b740809c10e5feebd64116/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:58.200975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae69eb2-50e0-4663-bf22-a0401d0e9218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e5df82139b74685bfea7060efa740e8, user_id=afdf454219b740809c10e5feebd64116, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:58.204230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae69eb2-50e0-4663-bf22-a0401d0e9218 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:58.251046 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3292/6617] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:58 2022] PUT /identity/v3/projects/3e5df82139b74685bfea7060efa740e8/users/afdf454219b740809c10e5feebd64116/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:58.251046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae69eb2-50e0-4663-bf22-a0401d0e9218 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:58.277099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c68e6a5-9244-4473-8e1a-ca84c5533cfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b5f0721a04c40abbe499605937e48c9', 'request_id': 'req-4c68e6a5-9244-4473-8e1a-ca84c5533cfe'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:58.290985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c68e6a5-9244-4473-8e1a-ca84c5533cfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b5f0721a04c40abbe499605937e48c9', 'request_id': 'req-4c68e6a5-9244-4473-8e1a-ca84c5533cfe'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:58.371606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae69eb2-50e0-4663-bf22-a0401d0e9218 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:58.395885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c68e6a5-9244-4473-8e1a-ca84c5533cfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b5f0721a04c40abbe499605937e48c9', 'request_id': 'req-4c68e6a5-9244-4473-8e1a-ca84c5533cfe'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:58.411262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c68e6a5-9244-4473-8e1a-ca84c5533cfe admin admin] The token cache is being invalidate because project 4b5f0721a04c40abbe499605937e48c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:26:58.412070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c68e6a5-9244-4473-8e1a-ca84c5533cfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:58.413570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c68e6a5-9244-4473-8e1a-ca84c5533cfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:26:58.415680 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3326/6618] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:26:58 2022] DELETE /identity/v3/projects/4b5f0721a04c40abbe499605937e48c9 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:58.417160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce1967-7331-41b9-a47d-4bb71d765af2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:58.417741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce1967-7331-41b9-a47d-4bb71d765af2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:58.419135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce1967-7331-41b9-a47d-4bb71d765af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:58.466964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a3ce1967-7331-41b9-a47d-4bb71d765af2 None None] MFA Rules not processed for user `afdf454219b740809c10e5feebd64116`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:58.473343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a3ce1967-7331-41b9-a47d-4bb71d765af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:58.476611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202e9da-9daf-4753-a880-a66c4f132bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:58.477791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202e9da-9daf-4753-a880-a66c4f132bfe admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:58.478339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202e9da-9daf-4753-a880-a66c4f132bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:58.479536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202e9da-9daf-4753-a880-a66c4f132bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:58.481089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202e9da-9daf-4753-a880-a66c4f132bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:58.483554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202e9da-9daf-4753-a880-a66c4f132bfe admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:58.497705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:58.508867 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3293/6619] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:58.510015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c70c1f-e578-4454-a3dd-684d803e2144 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:58.510321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c70c1f-e578-4454-a3dd-684d803e2144 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:58.510612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c70c1f-e578-4454-a3dd-684d803e2144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:58.542639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-83c70c1f-e578-4454-a3dd-684d803e2144 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:58.544249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-83c70c1f-e578-4454-a3dd-684d803e2144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:58.556983 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3294/6620] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:58.558026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c70c1f-e578-4454-a3dd-684d803e2144 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:58.559580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-83c70c1f-e578-4454-a3dd-684d803e2144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:58.573187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:58.579057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c70c1f-e578-4454-a3dd-684d803e2144 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:58.585155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdb208-74fa-4a4f-b513-1b0231f4e413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:58.586058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdb208-74fa-4a4f-b513-1b0231f4e413 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:58.586502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdb208-74fa-4a4f-b513-1b0231f4e413 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:58.586729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbdb208-74fa-4a4f-b513-1b0231f4e413 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:58.587584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbdb208-74fa-4a4f-b513-1b0231f4e413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:58.589471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbdb208-74fa-4a4f-b513-1b0231f4e413 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:58.602711 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3327/6621] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:58.605323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3c49f-25f4-4367-a019-75fbfd11570d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:58.605885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3c49f-25f4-4367-a019-75fbfd11570d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:58.606589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb3c49f-25f4-4367-a019-75fbfd11570d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:58.609104 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3295/6622] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:58.610032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdb208-74fa-4a4f-b513-1b0231f4e413 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:58.623074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbdb208-74fa-4a4f-b513-1b0231f4e413 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:58.630506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c6948-76cf-4eab-a94a-87d0b3b6087d tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf454219b740809c10e5feebd64116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e5df82139b74685bfea7060efa740e8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:58.631425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c6948-76cf-4eab-a94a-87d0b3b6087d tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:58.631730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c6948-76cf-4eab-a94a-87d0b3b6087d tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:58.632017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c6948-76cf-4eab-a94a-87d0b3b6087d tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:58.633274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c6948-76cf-4eab-a94a-87d0b3b6087d tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:58.633610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c6948-76cf-4eab-a94a-87d0b3b6087d tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:58.635556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c6948-76cf-4eab-a94a-87d0b3b6087d tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:58.639503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ffb3c49f-25f4-4367-a019-75fbfd11570d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:58.642626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ffb3c49f-25f4-4367-a019-75fbfd11570d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:58.654589 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3296/6623] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:58.655721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c6948-76cf-4eab-a94a-87d0b3b6087d tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:58.678212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c6948-76cf-4eab-a94a-87d0b3b6087d tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:58.681966 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3328/6624] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:58.683052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3c49f-25f4-4367-a019-75fbfd11570d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:58.686768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e39468-2722-41c2-abb4-1e0dcc29e09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:58.688117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e39468-2722-41c2-abb4-1e0dcc29e09c admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:58.688875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e39468-2722-41c2-abb4-1e0dcc29e09c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:58.689550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e39468-2722-41c2-abb4-1e0dcc29e09c admin admin] PATH_INFO: `/v3/projects/760e1d0cdcff4948bc9f8849ecc56e1c/users/d470c6d9a88d4b758d9df497c450cffb/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:58.695615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb3c49f-25f4-4367-a019-75fbfd11570d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:58.704269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fce4cd-07ae-4330-a390-166773867c0d tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf454219b740809c10e5feebd64116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e5df82139b74685bfea7060efa740e8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:58.706717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fce4cd-07ae-4330-a390-166773867c0d tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:58.707011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fce4cd-07ae-4330-a390-166773867c0d tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:58.707300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fce4cd-07ae-4330-a390-166773867c0d tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:58.708030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fce4cd-07ae-4330-a390-166773867c0d tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:58.710636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fce4cd-07ae-4330-a390-166773867c0d tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:58.741161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e39468-2722-41c2-abb4-1e0dcc29e09c admin admin] RBAC: Authorizing `identity:create_grant(project_id=760e1d0cdcff4948bc9f8849ecc56e1c, user_id=d470c6d9a88d4b758d9df497c450cffb, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:58.744938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e39468-2722-41c2-abb4-1e0dcc29e09c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:58.817593 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3329/6625] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:58 2022] POST /identity/v3/projects => generated 348 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:58.819056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fce4cd-07ae-4330-a390-166773867c0d tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:58.825929 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3297/6626] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:58 2022] PUT /identity/v3/projects/760e1d0cdcff4948bc9f8849ecc56e1c/users/d470c6d9a88d4b758d9df497c450cffb/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:58.825929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e39468-2722-41c2-abb4-1e0dcc29e09c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:58.954317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fce4cd-07ae-4330-a390-166773867c0d tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:58.983265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e39468-2722-41c2-abb4-1e0dcc29e09c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:59.001672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a15659-7c0c-43d6-8202-227a3652e415 tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf454219b740809c10e5feebd64116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e5df82139b74685bfea7060efa740e8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:59.002408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a15659-7c0c-43d6-8202-227a3652e415 tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:59.002740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a15659-7c0c-43d6-8202-227a3652e415 tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:59.003054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a15659-7c0c-43d6-8202-227a3652e415 tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:59.003798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a15659-7c0c-43d6-8202-227a3652e415 tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:59.004120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a15659-7c0c-43d6-8202-227a3652e415 tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:59.009020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a15659-7c0c-43d6-8202-227a3652e415 tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:59.033158 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3298/6627] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:59.033158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e651d8-e89e-4620-89c1-ab386c749983 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:59.033158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e651d8-e89e-4620-89c1-ab386c749983 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:59.033158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e651d8-e89e-4620-89c1-ab386c749983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:59.039531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e2c6c-9750-4929-aede-181dde4c7ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:59.040717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e2c6c-9750-4929-aede-181dde4c7ffd admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:59.040717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e2c6c-9750-4929-aede-181dde4c7ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:59.040943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e2c6c-9750-4929-aede-181dde4c7ffd admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:59.041555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e2c6c-9750-4929-aede-181dde4c7ffd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:26:59.041894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e2c6c-9750-4929-aede-181dde4c7ffd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:59.043750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e2c6c-9750-4929-aede-181dde4c7ffd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:59.062506 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3330/6628] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:26:59.062506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e2c6c-9750-4929-aede-181dde4c7ffd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:59.066982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-966e2c6c-9750-4929-aede-181dde4c7ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:59.090046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c0e651d8-e89e-4620-89c1-ab386c749983 None None] MFA Rules not processed for user `d470c6d9a88d4b758d9df497c450cffb`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:26:59.092611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c0e651d8-e89e-4620-89c1-ab386c749983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:59.169934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e2c6c-9750-4929-aede-181dde4c7ffd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:59.174328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23e23c-780e-4d26-bbaf-aae1ed3733c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:59.175889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23e23c-780e-4d26-bbaf-aae1ed3733c1 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:59.177987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23e23c-780e-4d26-bbaf-aae1ed3733c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:59.178348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23e23c-780e-4d26-bbaf-aae1ed3733c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:59.189002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23e23c-780e-4d26-bbaf-aae1ed3733c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:59.190962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23e23c-780e-4d26-bbaf-aae1ed3733c1 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:59.196495 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3299/6629] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:26:59.197431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e651d8-e89e-4620-89c1-ab386c749983 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:59.211120 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3331/6630] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:59.212528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23e23c-780e-4d26-bbaf-aae1ed3733c1 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:59.213806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5c23e23c-780e-4d26-bbaf-aae1ed3733c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:26:59.221211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:59.223212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e651d8-e89e-4620-89c1-ab386c749983 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:59.239646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee393c4-817d-4a0e-b811-eb7df9050b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:59.240391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee393c4-817d-4a0e-b811-eb7df9050b58 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:59.241351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee393c4-817d-4a0e-b811-eb7df9050b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:59.241644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee393c4-817d-4a0e-b811-eb7df9050b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:59.242444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee393c4-817d-4a0e-b811-eb7df9050b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:59.244499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee393c4-817d-4a0e-b811-eb7df9050b58 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:59.250646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:59.300085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23e23c-780e-4d26-bbaf-aae1ed3733c1 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:59.306177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537c4cc-a684-46a2-96b2-f34c1f243c94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:59.306852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537c4cc-a684-46a2-96b2-f34c1f243c94 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:59.316172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537c4cc-a684-46a2-96b2-f34c1f243c94 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:59.316172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537c4cc-a684-46a2-96b2-f34c1f243c94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:59.317719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537c4cc-a684-46a2-96b2-f34c1f243c94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:59.319984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537c4cc-a684-46a2-96b2-f34c1f243c94 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:59.339356 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3332/6631] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:26:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:26:59.368526 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3300/6632] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:26:59 2022] POST /identity/v3/projects => generated 393 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:59.371468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537c4cc-a684-46a2-96b2-f34c1f243c94 service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:59.388263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537c4cc-a684-46a2-96b2-f34c1f243c94 service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:59.478915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47828c6e-c0d9-4869-9411-be61350c2b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:59.480631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47828c6e-c0d9-4869-9411-be61350c2b1b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:59.481531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47828c6e-c0d9-4869-9411-be61350c2b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:59.482514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47828c6e-c0d9-4869-9411-be61350c2b1b admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:59.485317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47828c6e-c0d9-4869-9411-be61350c2b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:59.486360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47828c6e-c0d9-4869-9411-be61350c2b1b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:59.657244 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3333/6633] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:26:59 2022] POST /identity/v3/users => generated 297 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:26:59.662026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee393c4-817d-4a0e-b811-eb7df9050b58 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:59.682877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee393c4-817d-4a0e-b811-eb7df9050b58 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:59.687861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b0215-741b-4d53-9cb7-9bcd72f556e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:59.688611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b0215-741b-4d53-9cb7-9bcd72f556e5 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:59.688931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b0215-741b-4d53-9cb7-9bcd72f556e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:59.689230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b0215-741b-4d53-9cb7-9bcd72f556e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:59.690195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b0215-741b-4d53-9cb7-9bcd72f556e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:59.691912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b0215-741b-4d53-9cb7-9bcd72f556e5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:59.705798 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3301/6634] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:59.710022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47828c6e-c0d9-4869-9411-be61350c2b1b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:59.724126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47828c6e-c0d9-4869-9411-be61350c2b1b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:59.730410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00051148-4adc-40ed-8aa5-ff1a3ae5570a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:59.731300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00051148-4adc-40ed-8aa5-ff1a3ae5570a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:59.731594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00051148-4adc-40ed-8aa5-ff1a3ae5570a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:59.731874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00051148-4adc-40ed-8aa5-ff1a3ae5570a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:59.732671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00051148-4adc-40ed-8aa5-ff1a3ae5570a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:59.735172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00051148-4adc-40ed-8aa5-ff1a3ae5570a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:59.751528 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3334/6635] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:26:59.755959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b0215-741b-4d53-9cb7-9bcd72f556e5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:26:59.785751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b0215-741b-4d53-9cb7-9bcd72f556e5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:26:59.808704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c32bb7-e0ce-4ef8-85e8-e42c097af227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:26:59.809613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c32bb7-e0ce-4ef8-85e8-e42c097af227 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:59.810047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c32bb7-e0ce-4ef8-85e8-e42c097af227 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:59.810372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c32bb7-e0ce-4ef8-85e8-e42c097af227 admin admin] PATH_INFO: `/v3/projects/eec013a9ed56405da1386dbfb3934dac/users/54499025e1474eeaaeceb3fb318d4c5c/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:26:59.830549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c32bb7-e0ce-4ef8-85e8-e42c097af227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eec013a9ed56405da1386dbfb3934dac, user_id=54499025e1474eeaaeceb3fb318d4c5c, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:26:59.833820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c32bb7-e0ce-4ef8-85e8-e42c097af227 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:26:59.909725 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3302/6636] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:26:59 2022] PUT /identity/v3/projects/eec013a9ed56405da1386dbfb3934dac/users/54499025e1474eeaaeceb3fb318d4c5c/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:26:59.920868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598753fb-ce45-431f-aa06-4eb0280ecf2f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:26:59.921221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598753fb-ce45-431f-aa06-4eb0280ecf2f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:26:59.921515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598753fb-ce45-431f-aa06-4eb0280ecf2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:00.001213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:00.014907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-598753fb-ce45-431f-aa06-4eb0280ecf2f None None] MFA Rules not processed for user `54499025e1474eeaaeceb3fb318d4c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:00.019454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-598753fb-ce45-431f-aa06-4eb0280ecf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:00.069832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:00.212073 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3335/6637] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:26:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:00.216616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c32bb7-e0ce-4ef8-85e8-e42c097af227 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:00.395030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c32bb7-e0ce-4ef8-85e8-e42c097af227 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:00.416685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5d1d2-67d2-4f48-8502-1c123e49a217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:00.417385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5d1d2-67d2-4f48-8502-1c123e49a217 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:00.419423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5d1d2-67d2-4f48-8502-1c123e49a217 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:00.419917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be5d1d2-67d2-4f48-8502-1c123e49a217 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:00.422285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be5d1d2-67d2-4f48-8502-1c123e49a217 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:00.424290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be5d1d2-67d2-4f48-8502-1c123e49a217 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:00.589948 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3303/6638] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:27:00 2022] POST /identity/v3/projects => generated 391 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:00.593903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598753fb-ce45-431f-aa06-4eb0280ecf2f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:00.615061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598753fb-ce45-431f-aa06-4eb0280ecf2f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:00.780385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf18df-809a-4fb2-a1a4-e0afcfed3efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:00.781624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf18df-809a-4fb2-a1a4-e0afcfed3efe admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:00.781624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf18df-809a-4fb2-a1a4-e0afcfed3efe admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:00.782009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf18df-809a-4fb2-a1a4-e0afcfed3efe admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:00.782841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf18df-809a-4fb2-a1a4-e0afcfed3efe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:00.785322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf18df-809a-4fb2-a1a4-e0afcfed3efe admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:00.897478 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3336/6639] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:27:00 2022] POST /identity/v3/users => generated 302 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:00.898027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5d1d2-67d2-4f48-8502-1c123e49a217 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:00.921992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be5d1d2-67d2-4f48-8502-1c123e49a217 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:00.948631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27ca81-924f-4231-874c-3de21277c72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:00.949181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27ca81-924f-4231-874c-3de21277c72b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:00.949440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27ca81-924f-4231-874c-3de21277c72b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:00.949825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27ca81-924f-4231-874c-3de21277c72b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:00.950509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27ca81-924f-4231-874c-3de21277c72b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:00.952213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27ca81-924f-4231-874c-3de21277c72b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:00.976258 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3304/6640] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:27:00.977681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf18df-809a-4fb2-a1a4-e0afcfed3efe admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:00.991063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf18df-809a-4fb2-a1a4-e0afcfed3efe admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:00.999171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5c845-a5ac-41e7-bbbb-e40c54fd0c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:01.000392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5c845-a5ac-41e7-bbbb-e40c54fd0c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:01.000852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5c845-a5ac-41e7-bbbb-e40c54fd0c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:01.001307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be5c845-a5ac-41e7-bbbb-e40c54fd0c07 admin admin] PATH_INFO: `/v3/projects/a88603ae79c74bd9809e32a2deae247a/users/7cecb8cbdba644599b4c3cdcd527276e/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:01.025005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:01.031977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5c845-a5ac-41e7-bbbb-e40c54fd0c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a88603ae79c74bd9809e32a2deae247a, user_id=7cecb8cbdba644599b4c3cdcd527276e, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:01.054619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be5c845-a5ac-41e7-bbbb-e40c54fd0c07 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:01.109088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:01.174263 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3337/6641] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:00 2022] PUT /identity/v3/projects/a88603ae79c74bd9809e32a2deae247a/users/7cecb8cbdba644599b4c3cdcd527276e/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:01.178746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27ca81-924f-4231-874c-3de21277c72b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:01.201530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27ca81-924f-4231-874c-3de21277c72b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:01.394922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195963a-57de-4c99-8df2-32eb1496423c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:01.395672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195963a-57de-4c99-8df2-32eb1496423c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:01.396294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195963a-57de-4c99-8df2-32eb1496423c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:01.397223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195963a-57de-4c99-8df2-32eb1496423c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:01.398112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195963a-57de-4c99-8df2-32eb1496423c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:01.400210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195963a-57de-4c99-8df2-32eb1496423c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:01.422064 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3305/6642] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:01.422269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5c845-a5ac-41e7-bbbb-e40c54fd0c07 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:01.434672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be5c845-a5ac-41e7-bbbb-e40c54fd0c07 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:01.449119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9dbd4-7eb0-4d9c-b9fc-30c111637550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:01.451527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9dbd4-7eb0-4d9c-b9fc-30c111637550 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:01.452628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9dbd4-7eb0-4d9c-b9fc-30c111637550 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:01.454494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9dbd4-7eb0-4d9c-b9fc-30c111637550 admin admin] PATH_INFO: `/v3/projects/a88603ae79c74bd9809e32a2deae247a/users/7cecb8cbdba644599b4c3cdcd527276e/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:01.460192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9dbd4-7eb0-4d9c-b9fc-30c111637550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a88603ae79c74bd9809e32a2deae247a, user_id=7cecb8cbdba644599b4c3cdcd527276e, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:01.467182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9dbd4-7eb0-4d9c-b9fc-30c111637550 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:01.485370 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3338/6643] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:01 2022] PUT /identity/v3/projects/a88603ae79c74bd9809e32a2deae247a/users/7cecb8cbdba644599b4c3cdcd527276e/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:01.497142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea2472-01b0-406b-9dac-ca708b21e55a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:01.499191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea2472-01b0-406b-9dac-ca708b21e55a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:01.499191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea2472-01b0-406b-9dac-ca708b21e55a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:01.542592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c8ea2472-01b0-406b-9dac-ca708b21e55a None None] MFA Rules not processed for user `7cecb8cbdba644599b4c3cdcd527276e`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:01.543832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c8ea2472-01b0-406b-9dac-ca708b21e55a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:01.737830 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3306/6644] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:01 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:01.783572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9dbd4-7eb0-4d9c-b9fc-30c111637550 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:01.930010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9dbd4-7eb0-4d9c-b9fc-30c111637550 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:01.934758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975033c1-1196-4fe5-b921-49a8dacd8ea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:01.951091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975033c1-1196-4fe5-b921-49a8dacd8ea9 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:01.951717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975033c1-1196-4fe5-b921-49a8dacd8ea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:01.953072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975033c1-1196-4fe5-b921-49a8dacd8ea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:01.984429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975033c1-1196-4fe5-b921-49a8dacd8ea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:01.989668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975033c1-1196-4fe5-b921-49a8dacd8ea9 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:02.021094 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3339/6645] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:02.494254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07aeb7-286c-49cb-97da-2393a5f82326 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:02.494936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07aeb7-286c-49cb-97da-2393a5f82326 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:02.495272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07aeb7-286c-49cb-97da-2393a5f82326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:02.536683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3c07aeb7-286c-49cb-97da-2393a5f82326 None None] MFA Rules not processed for user `d470c6d9a88d4b758d9df497c450cffb`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:02.538745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3c07aeb7-286c-49cb-97da-2393a5f82326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:02.649895 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3307/6646] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:02.989821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:03.052595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:03.708961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:04.120984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:04.871300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975033c1-1196-4fe5-b921-49a8dacd8ea9 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:05.007165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975033c1-1196-4fe5-b921-49a8dacd8ea9 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:05.019903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62ac1a-7165-4e63-9de4-21402eef16ef tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf454219b740809c10e5feebd64116', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e5df82139b74685bfea7060efa740e8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:05.019903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62ac1a-7165-4e63-9de4-21402eef16ef tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:05.019903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62ac1a-7165-4e63-9de4-21402eef16ef tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:05.020347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62ac1a-7165-4e63-9de4-21402eef16ef tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] PATH_INFO: `/v3/projects/cd0a3808792b421fa5be261ca47316aa` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:05.020347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62ac1a-7165-4e63-9de4-21402eef16ef tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cd0a3808792b421fa5be261ca47316aa)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:05.020347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62ac1a-7165-4e63-9de4-21402eef16ef tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:05.170824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eb62ac1a-7165-4e63-9de4-21402eef16ef tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd0a3808792b421fa5be261ca47316aa', 'request_id': 'req-eb62ac1a-7165-4e63-9de4-21402eef16ef'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:05.178741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eb62ac1a-7165-4e63-9de4-21402eef16ef tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd0a3808792b421fa5be261ca47316aa', 'request_id': 'req-eb62ac1a-7165-4e63-9de4-21402eef16ef'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:05.254275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eb62ac1a-7165-4e63-9de4-21402eef16ef tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd0a3808792b421fa5be261ca47316aa', 'request_id': 'req-eb62ac1a-7165-4e63-9de4-21402eef16ef'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:05.272229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eb62ac1a-7165-4e63-9de4-21402eef16ef tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] The token cache is being invalidate because project cd0a3808792b421fa5be261ca47316aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:27:05.274221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eb62ac1a-7165-4e63-9de4-21402eef16ef tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:05.278331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eb62ac1a-7165-4e63-9de4-21402eef16ef tempest-QuotasNegativeTest-1837499772 tempest-QuotasNegativeTest-1837499772-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:05.282233 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3340/6647] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:27:04 2022] DELETE /identity/v3/projects/cd0a3808792b421fa5be261ca47316aa => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:05.287094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07aeb7-286c-49cb-97da-2393a5f82326 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:05.297526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3c07aeb7-286c-49cb-97da-2393a5f82326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:05.379265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07aeb7-286c-49cb-97da-2393a5f82326 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:05.400400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:05.455132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8ea3b-a369-4ff8-9184-e5e59930d085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:05.456132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8ea3b-a369-4ff8-9184-e5e59930d085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:05.456846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8ea3b-a369-4ff8-9184-e5e59930d085 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:05.457295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8ea3b-a369-4ff8-9184-e5e59930d085 admin admin] PATH_INFO: `/v3/users/82fcd8933bf6495ea6ecc17d8cefeb48` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:05.459512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8ea3b-a369-4ff8-9184-e5e59930d085 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82fcd8933bf6495ea6ecc17d8cefeb48)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:05.461950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d8ea3b-a369-4ff8-9184-e5e59930d085 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:05.727168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-26d8ea3b-a369-4ff8-9184-e5e59930d085 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82fcd8933bf6495ea6ecc17d8cefeb48'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:05.730468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-26d8ea3b-a369-4ff8-9184-e5e59930d085 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82fcd8933bf6495ea6ecc17d8cefeb48'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:05.752654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-26d8ea3b-a369-4ff8-9184-e5e59930d085 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82fcd8933bf6495ea6ecc17d8cefeb48'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:05.864193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-26d8ea3b-a369-4ff8-9184-e5e59930d085 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82fcd8933bf6495ea6ecc17d8cefeb48'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:05.867311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-26d8ea3b-a369-4ff8-9184-e5e59930d085 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82fcd8933bf6495ea6ecc17d8cefeb48'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:05.912882 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3308/6648] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:27:05 2022] DELETE /identity/v3/users/82fcd8933bf6495ea6ecc17d8cefeb48 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:05.920079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b5b02-5579-464a-bef0-5b55a1e1d519 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:05.920517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b5b02-5579-464a-bef0-5b55a1e1d519 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:05.920953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b5b02-5579-464a-bef0-5b55a1e1d519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:05.929517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:05.954180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c92b5b02-5579-464a-bef0-5b55a1e1d519 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:05.955427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c92b5b02-5579-464a-bef0-5b55a1e1d519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:06.069685 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3341/6649] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:06.081342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8ea3b-a369-4ff8-9184-e5e59930d085 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:06.082858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-26d8ea3b-a369-4ff8-9184-e5e59930d085 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:06.178263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8ea3b-a369-4ff8-9184-e5e59930d085 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:06.182000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782aa166-8fd6-47fa-b7c1-932f1034eebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:06.187976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782aa166-8fd6-47fa-b7c1-932f1034eebd service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:06.187976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782aa166-8fd6-47fa-b7c1-932f1034eebd service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:06.187976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782aa166-8fd6-47fa-b7c1-932f1034eebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:06.192473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782aa166-8fd6-47fa-b7c1-932f1034eebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:06.194557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782aa166-8fd6-47fa-b7c1-932f1034eebd service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:06.204513 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3309/6650] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:06.543509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b5b02-5579-464a-bef0-5b55a1e1d519 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:06.545035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c92b5b02-5579-464a-bef0-5b55a1e1d519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:06.557927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b5b02-5579-464a-bef0-5b55a1e1d519 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:06.562608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aa8baa-2a50-40e2-91e6-9bce7ed2f178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:06.563533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa8baa-2a50-40e2-91e6-9bce7ed2f178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:06.563765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa8baa-2a50-40e2-91e6-9bce7ed2f178 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:06.564044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aa8baa-2a50-40e2-91e6-9bce7ed2f178 admin admin] PATH_INFO: `/v3/users/54499025e1474eeaaeceb3fb318d4c5c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:06.565365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa8baa-2a50-40e2-91e6-9bce7ed2f178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54499025e1474eeaaeceb3fb318d4c5c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:06.567358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aa8baa-2a50-40e2-91e6-9bce7ed2f178 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:06.832228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782aa166-8fd6-47fa-b7c1-932f1034eebd service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:06.834437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-782aa166-8fd6-47fa-b7c1-932f1034eebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:06.841712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-97aa8baa-2a50-40e2-91e6-9bce7ed2f178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54499025e1474eeaaeceb3fb318d4c5c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:06.849626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-97aa8baa-2a50-40e2-91e6-9bce7ed2f178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54499025e1474eeaaeceb3fb318d4c5c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:06.851226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-97aa8baa-2a50-40e2-91e6-9bce7ed2f178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54499025e1474eeaaeceb3fb318d4c5c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:06.851854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-97aa8baa-2a50-40e2-91e6-9bce7ed2f178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54499025e1474eeaaeceb3fb318d4c5c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:06.862461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782aa166-8fd6-47fa-b7c1-932f1034eebd service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:06.868534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee2632-f248-471e-b761-d754971e99ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:06.869671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee2632-f248-471e-b761-d754971e99ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:06.869983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee2632-f248-471e-b761-d754971e99ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:06.870358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee2632-f248-471e-b761-d754971e99ad admin admin] PATH_INFO: `/v3/projects/a097d0a752f8498a93e37e68e619a99f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:06.877473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee2632-f248-471e-b761-d754971e99ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=a097d0a752f8498a93e37e68e619a99f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:06.879526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee2632-f248-471e-b761-d754971e99ad admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:06.927971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-97aa8baa-2a50-40e2-91e6-9bce7ed2f178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54499025e1474eeaaeceb3fb318d4c5c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:06.957189 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3342/6651] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:27:06 2022] DELETE /identity/v3/users/54499025e1474eeaaeceb3fb318d4c5c => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:06.973024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6c80f-7a6d-4f7e-82a6-4370f435d835 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:06.977664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6c80f-7a6d-4f7e-82a6-4370f435d835 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:06.978142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6c80f-7a6d-4f7e-82a6-4370f435d835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:07.012488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:07.026684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-98ee2632-f248-471e-b761-d754971e99ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a097d0a752f8498a93e37e68e619a99f', 'request_id': 'req-98ee2632-f248-471e-b761-d754971e99ad'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:07.029229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-92c6c80f-7a6d-4f7e-82a6-4370f435d835 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:07.034061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-98ee2632-f248-471e-b761-d754971e99ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a097d0a752f8498a93e37e68e619a99f', 'request_id': 'req-98ee2632-f248-471e-b761-d754971e99ad'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:07.046923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-92c6c80f-7a6d-4f7e-82a6-4370f435d835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:07.128164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-98ee2632-f248-471e-b761-d754971e99ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a097d0a752f8498a93e37e68e619a99f', 'request_id': 'req-98ee2632-f248-471e-b761-d754971e99ad'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:07.150856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-98ee2632-f248-471e-b761-d754971e99ad admin admin] The token cache is being invalidate because project a097d0a752f8498a93e37e68e619a99f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:27:07.153126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-98ee2632-f248-471e-b761-d754971e99ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:07.153126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-98ee2632-f248-471e-b761-d754971e99ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:07.165423 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3310/6652] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:27:06 2022] DELETE /identity/v3/projects/a097d0a752f8498a93e37e68e619a99f => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:07.174778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee2632-f248-471e-b761-d754971e99ad admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:07.177441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-98ee2632-f248-471e-b761-d754971e99ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:07.282669 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3343/6653] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:07.285515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:07.316498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6c80f-7a6d-4f7e-82a6-4370f435d835 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:07.318530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-92c6c80f-7a6d-4f7e-82a6-4370f435d835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:07.334570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee2632-f248-471e-b761-d754971e99ad admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:07.340323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9e167-6472-4632-800c-b6d2c00e70a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:07.341410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9e167-6472-4632-800c-b6d2c00e70a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:07.342063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9e167-6472-4632-800c-b6d2c00e70a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:07.342424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9e167-6472-4632-800c-b6d2c00e70a6 admin admin] PATH_INFO: `/v3/users/afdf454219b740809c10e5feebd64116` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:07.356949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9e167-6472-4632-800c-b6d2c00e70a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afdf454219b740809c10e5feebd64116)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:07.359282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9e167-6472-4632-800c-b6d2c00e70a6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:07.505431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6c80f-7a6d-4f7e-82a6-4370f435d835 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:07.513211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32768e-97f6-451c-8432-4aa3a0fa5b12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:07.516543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32768e-97f6-451c-8432-4aa3a0fa5b12 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:07.516543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32768e-97f6-451c-8432-4aa3a0fa5b12 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:07.516801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32768e-97f6-451c-8432-4aa3a0fa5b12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:07.552092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32768e-97f6-451c-8432-4aa3a0fa5b12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:07.556596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32768e-97f6-451c-8432-4aa3a0fa5b12 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:07.584559 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3344/6654] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:07.589174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-44f9e167-6472-4632-800c-b6d2c00e70a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afdf454219b740809c10e5feebd64116'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:07.590003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-44f9e167-6472-4632-800c-b6d2c00e70a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afdf454219b740809c10e5feebd64116'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:07.601914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-44f9e167-6472-4632-800c-b6d2c00e70a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afdf454219b740809c10e5feebd64116'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:07.704976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-44f9e167-6472-4632-800c-b6d2c00e70a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afdf454219b740809c10e5feebd64116'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:07.705919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-44f9e167-6472-4632-800c-b6d2c00e70a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afdf454219b740809c10e5feebd64116'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:07.741489 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3311/6655] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:27:07 2022] DELETE /identity/v3/users/afdf454219b740809c10e5feebd64116 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:08.276256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:08.725431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:08.870481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32768e-97f6-451c-8432-4aa3a0fa5b12 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:08.887358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32768e-97f6-451c-8432-4aa3a0fa5b12 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:08.898845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9e167-6472-4632-800c-b6d2c00e70a6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:08.900652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-44f9e167-6472-4632-800c-b6d2c00e70a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:09.064465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9e167-6472-4632-800c-b6d2c00e70a6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:09.082058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784de37-d0c4-4a10-afab-32e24960f16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:09.083269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784de37-d0c4-4a10-afab-32e24960f16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:09.083787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784de37-d0c4-4a10-afab-32e24960f16b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:09.084313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784de37-d0c4-4a10-afab-32e24960f16b admin admin] PATH_INFO: `/v3/projects/eec013a9ed56405da1386dbfb3934dac` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:09.087489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784de37-d0c4-4a10-afab-32e24960f16b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eec013a9ed56405da1386dbfb3934dac)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:09.091578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784de37-d0c4-4a10-afab-32e24960f16b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:09.094197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170d6b7-b265-413e-8ba4-57757ad99cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:09.095881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170d6b7-b265-413e-8ba4-57757ad99cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:09.096177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170d6b7-b265-413e-8ba4-57757ad99cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:09.100058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170d6b7-b265-413e-8ba4-57757ad99cd7 admin admin] PATH_INFO: `/v3/projects/3e5df82139b74685bfea7060efa740e8` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:09.106884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170d6b7-b265-413e-8ba4-57757ad99cd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e5df82139b74685bfea7060efa740e8)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:09.112657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2170d6b7-b265-413e-8ba4-57757ad99cd7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:09.272940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3784de37-d0c4-4a10-afab-32e24960f16b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eec013a9ed56405da1386dbfb3934dac', 'request_id': 'req-3784de37-d0c4-4a10-afab-32e24960f16b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:09.279616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3784de37-d0c4-4a10-afab-32e24960f16b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eec013a9ed56405da1386dbfb3934dac', 'request_id': 'req-3784de37-d0c4-4a10-afab-32e24960f16b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:09.338908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2170d6b7-b265-413e-8ba4-57757ad99cd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e5df82139b74685bfea7060efa740e8', 'request_id': 'req-2170d6b7-b265-413e-8ba4-57757ad99cd7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:09.349917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2170d6b7-b265-413e-8ba4-57757ad99cd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e5df82139b74685bfea7060efa740e8', 'request_id': 'req-2170d6b7-b265-413e-8ba4-57757ad99cd7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:09.657500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3784de37-d0c4-4a10-afab-32e24960f16b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eec013a9ed56405da1386dbfb3934dac', 'request_id': 'req-3784de37-d0c4-4a10-afab-32e24960f16b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:09.682688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3784de37-d0c4-4a10-afab-32e24960f16b admin admin] The token cache is being invalidate because project eec013a9ed56405da1386dbfb3934dac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:27:09.683011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3784de37-d0c4-4a10-afab-32e24960f16b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:09.685524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3784de37-d0c4-4a10-afab-32e24960f16b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:09.688001 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3312/6656] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:27:08 2022] DELETE /identity/v3/projects/eec013a9ed56405da1386dbfb3934dac => generated 0 bytes in 790 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:09.693783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784de37-d0c4-4a10-afab-32e24960f16b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:09.698404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3784de37-d0c4-4a10-afab-32e24960f16b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:09.733699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2170d6b7-b265-413e-8ba4-57757ad99cd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e5df82139b74685bfea7060efa740e8', 'request_id': 'req-2170d6b7-b265-413e-8ba4-57757ad99cd7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:09.761087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2170d6b7-b265-413e-8ba4-57757ad99cd7 admin admin] The token cache is being invalidate because project 3e5df82139b74685bfea7060efa740e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:27:09.763175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2170d6b7-b265-413e-8ba4-57757ad99cd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:09.767979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2170d6b7-b265-413e-8ba4-57757ad99cd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:09.770726 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3345/6657] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:27:08 2022] DELETE /identity/v3/projects/3e5df82139b74685bfea7060efa740e8 => generated 0 bytes in 901 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:09.778061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184fd1f-e58b-4edc-bec2-d5ce053fa722 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:09.778440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184fd1f-e58b-4edc-bec2-d5ce053fa722 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:09.778763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4184fd1f-e58b-4edc-bec2-d5ce053fa722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:09.832750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4184fd1f-e58b-4edc-bec2-d5ce053fa722 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:09.837153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4184fd1f-e58b-4edc-bec2-d5ce053fa722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:09.882561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784de37-d0c4-4a10-afab-32e24960f16b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:09.948314 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3346/6658] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:09.948314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8293b-e7aa-4622-911d-1dad7e759923 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:09.948314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8293b-e7aa-4622-911d-1dad7e759923 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:09.948748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c8293b-e7aa-4622-911d-1dad7e759923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:09.988184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-43c8293b-e7aa-4622-911d-1dad7e759923 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:09.993048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-43c8293b-e7aa-4622-911d-1dad7e759923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:09.999686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2af89-47df-4c95-a9b2-ab52dca1554f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:10.002773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2af89-47df-4c95-a9b2-ab52dca1554f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:10.002773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2af89-47df-4c95-a9b2-ab52dca1554f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:10.002773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2af89-47df-4c95-a9b2-ab52dca1554f admin admin] PATH_INFO: `/v3/users/7cecb8cbdba644599b4c3cdcd527276e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:10.004293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2af89-47df-4c95-a9b2-ab52dca1554f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cecb8cbdba644599b4c3cdcd527276e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:10.008569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2af89-47df-4c95-a9b2-ab52dca1554f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:10.012877 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3347/6659] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:27:10.016842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8293b-e7aa-4622-911d-1dad7e759923 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:10.040019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c8293b-e7aa-4622-911d-1dad7e759923 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:10.047937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a9276-32eb-4da6-ba88-e931873535e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:10.056362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a9276-32eb-4da6-ba88-e931873535e8 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:10.062326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a9276-32eb-4da6-ba88-e931873535e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:10.063603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a9276-32eb-4da6-ba88-e931873535e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:10.065285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a9276-32eb-4da6-ba88-e931873535e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:27:10.065869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a9276-32eb-4da6-ba88-e931873535e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:10.070027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a9276-32eb-4da6-ba88-e931873535e8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:10.090654 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3348/6660] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:27:10.095318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a9276-32eb-4da6-ba88-e931873535e8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:10.110932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a9276-32eb-4da6-ba88-e931873535e8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:10.122511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ceba6a-92c5-435b-ac9a-2d88e0043b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:10.123652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ceba6a-92c5-435b-ac9a-2d88e0043b8e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:10.126938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ceba6a-92c5-435b-ac9a-2d88e0043b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:10.129299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ceba6a-92c5-435b-ac9a-2d88e0043b8e admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:10.129724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ceba6a-92c5-435b-ac9a-2d88e0043b8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:10.133106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ceba6a-92c5-435b-ac9a-2d88e0043b8e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:10.268522 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3349/6661] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:27:10 2022] POST /identity/v3/projects => generated 375 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:10.273149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ceba6a-92c5-435b-ac9a-2d88e0043b8e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:10.294524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f1b2af89-47df-4c95-a9b2-ab52dca1554f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cecb8cbdba644599b4c3cdcd527276e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:10.298910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f1b2af89-47df-4c95-a9b2-ab52dca1554f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cecb8cbdba644599b4c3cdcd527276e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:10.309826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f1b2af89-47df-4c95-a9b2-ab52dca1554f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cecb8cbdba644599b4c3cdcd527276e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:10.357881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ceba6a-92c5-435b-ac9a-2d88e0043b8e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:10.363114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee46fef-995f-428f-b306-9774e1784cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:10.363976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee46fef-995f-428f-b306-9774e1784cfa admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:10.364442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee46fef-995f-428f-b306-9774e1784cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:10.364721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee46fef-995f-428f-b306-9774e1784cfa admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:10.365681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee46fef-995f-428f-b306-9774e1784cfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:10.368002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee46fef-995f-428f-b306-9774e1784cfa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:10.418104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f1b2af89-47df-4c95-a9b2-ab52dca1554f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cecb8cbdba644599b4c3cdcd527276e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:10.419025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f1b2af89-47df-4c95-a9b2-ab52dca1554f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cecb8cbdba644599b4c3cdcd527276e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:10.440595 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3313/6662] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:27:09 2022] DELETE /identity/v3/users/7cecb8cbdba644599b4c3cdcd527276e => generated 0 bytes in 749 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:27:10.511799 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3350/6663] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:27:10 2022] POST /identity/v3/users => generated 288 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:10.516081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2af89-47df-4c95-a9b2-ab52dca1554f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:10.517666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f1b2af89-47df-4c95-a9b2-ab52dca1554f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:10.624648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2af89-47df-4c95-a9b2-ab52dca1554f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:10.641495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b3c55-163e-4826-9dfa-8d4d04afb120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:10.653546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b3c55-163e-4826-9dfa-8d4d04afb120 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:10.653546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b3c55-163e-4826-9dfa-8d4d04afb120 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:10.653546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b3c55-163e-4826-9dfa-8d4d04afb120 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:10.653546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b3c55-163e-4826-9dfa-8d4d04afb120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:10.653546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b3c55-163e-4826-9dfa-8d4d04afb120 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:10.665878 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3314/6664] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:10.667338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee46fef-995f-428f-b306-9774e1784cfa admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:10.678584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:10.684638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee46fef-995f-428f-b306-9774e1784cfa admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:10.689612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ad003-0710-45c4-8944-bf7243705bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:10.690347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad003-0710-45c4-8944-bf7243705bce admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:10.690632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad003-0710-45c4-8944-bf7243705bce admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:10.691295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ad003-0710-45c4-8944-bf7243705bce admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:10.692371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ad003-0710-45c4-8944-bf7243705bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:10.694625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ad003-0710-45c4-8944-bf7243705bce admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:10.717603 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3351/6665] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:10.719577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b3c55-163e-4826-9dfa-8d4d04afb120 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:10.737046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b3c55-163e-4826-9dfa-8d4d04afb120 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:10.744686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed80af9-f9a8-4df0-9c5e-7c62cd2bf009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:10.745661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed80af9-f9a8-4df0-9c5e-7c62cd2bf009 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:10.745904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed80af9-f9a8-4df0-9c5e-7c62cd2bf009 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:10.746192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed80af9-f9a8-4df0-9c5e-7c62cd2bf009 admin admin] PATH_INFO: `/v3/projects/a800fc7014f942b8b0681e738b3adef4/users/0f00ea0259c34a468d66d894e10a3d05/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:10.764664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed80af9-f9a8-4df0-9c5e-7c62cd2bf009 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a800fc7014f942b8b0681e738b3adef4, user_id=0f00ea0259c34a468d66d894e10a3d05, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:10.767755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed80af9-f9a8-4df0-9c5e-7c62cd2bf009 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:10.793554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ad003-0710-45c4-8944-bf7243705bce admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:10.795471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e49ad003-0710-45c4-8944-bf7243705bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:10.809220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ad003-0710-45c4-8944-bf7243705bce admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:10.811433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:10.821073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74de458-3a20-4dcd-8a6d-617b83048b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:10.821073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74de458-3a20-4dcd-8a6d-617b83048b87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:10.821073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74de458-3a20-4dcd-8a6d-617b83048b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:10.821073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74de458-3a20-4dcd-8a6d-617b83048b87 admin admin] PATH_INFO: `/v3/projects/a88603ae79c74bd9809e32a2deae247a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:10.822776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74de458-3a20-4dcd-8a6d-617b83048b87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a88603ae79c74bd9809e32a2deae247a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:10.826689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74de458-3a20-4dcd-8a6d-617b83048b87 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:10.886023 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3315/6666] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:10 2022] PUT /identity/v3/projects/a800fc7014f942b8b0681e738b3adef4/users/0f00ea0259c34a468d66d894e10a3d05/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:27:10.898330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6c0f5-0b9c-4efe-a910-9ca11d2fa109 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:10.898330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6c0f5-0b9c-4efe-a910-9ca11d2fa109 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:10.901002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d6c0f5-0b9c-4efe-a910-9ca11d2fa109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:10.929630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-31d6c0f5-0b9c-4efe-a910-9ca11d2fa109 None None] MFA Rules not processed for user `0f00ea0259c34a468d66d894e10a3d05`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:10.935558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-31d6c0f5-0b9c-4efe-a910-9ca11d2fa109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:10.958845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a74de458-3a20-4dcd-8a6d-617b83048b87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a88603ae79c74bd9809e32a2deae247a', 'request_id': 'req-a74de458-3a20-4dcd-8a6d-617b83048b87'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:10.981219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a74de458-3a20-4dcd-8a6d-617b83048b87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a88603ae79c74bd9809e32a2deae247a', 'request_id': 'req-a74de458-3a20-4dcd-8a6d-617b83048b87'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:11.029590 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3316/6667] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:11.034593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6c0f5-0b9c-4efe-a910-9ca11d2fa109 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:11.068833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a74de458-3a20-4dcd-8a6d-617b83048b87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a88603ae79c74bd9809e32a2deae247a', 'request_id': 'req-a74de458-3a20-4dcd-8a6d-617b83048b87'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:11.088430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a74de458-3a20-4dcd-8a6d-617b83048b87 admin admin] The token cache is being invalidate because project a88603ae79c74bd9809e32a2deae247a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:27:11.089162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a74de458-3a20-4dcd-8a6d-617b83048b87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:11.090140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a74de458-3a20-4dcd-8a6d-617b83048b87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:11.097963 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3352/6668] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:27:10 2022] DELETE /identity/v3/projects/a88603ae79c74bd9809e32a2deae247a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:27:11.103448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa5977-826e-4c9f-97cc-c64c40ab78d7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:11.103739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa5977-826e-4c9f-97cc-c64c40ab78d7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:11.107452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa5977-826e-4c9f-97cc-c64c40ab78d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:11.127984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-62aa5977-826e-4c9f-97cc-c64c40ab78d7 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:11.129495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-62aa5977-826e-4c9f-97cc-c64c40ab78d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:11.158014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d6c0f5-0b9c-4efe-a910-9ca11d2fa109 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:11.226125 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3353/6669] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:11.234936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e5ff5-7a48-4b11-ad8c-37693ba89864 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:11.235258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e5ff5-7a48-4b11-ad8c-37693ba89864 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:11.235487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a57abe-2832-4400-b937-99919f422ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:11.235648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e5ff5-7a48-4b11-ad8c-37693ba89864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:11.236283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a57abe-2832-4400-b937-99919f422ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:11.236635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a57abe-2832-4400-b937-99919f422ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:11.237383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a57abe-2832-4400-b937-99919f422ab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:11.238198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a57abe-2832-4400-b937-99919f422ab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:11.240796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a57abe-2832-4400-b937-99919f422ab8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:11.259293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a87e5ff5-7a48-4b11-ad8c-37693ba89864 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:11.261052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a87e5ff5-7a48-4b11-ad8c-37693ba89864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:11.269901 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3354/6670] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:27:11.274510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e5ff5-7a48-4b11-ad8c-37693ba89864 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:11.283722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e5ff5-7a48-4b11-ad8c-37693ba89864 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:11.289081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568dda2a-f5db-4ea0-9d72-b43914ab9182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:11.289747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568dda2a-f5db-4ea0-9d72-b43914ab9182 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:11.292624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568dda2a-f5db-4ea0-9d72-b43914ab9182 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:11.292624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568dda2a-f5db-4ea0-9d72-b43914ab9182 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:11.292624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568dda2a-f5db-4ea0-9d72-b43914ab9182 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:27:11.292624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568dda2a-f5db-4ea0-9d72-b43914ab9182 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:11.293106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568dda2a-f5db-4ea0-9d72-b43914ab9182 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:11.308800 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3355/6671] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:27:11.310556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568dda2a-f5db-4ea0-9d72-b43914ab9182 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:11.324446 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3317/6672] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:27:11 2022] POST /identity/v3/projects => generated 375 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:27:11.328337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a57abe-2832-4400-b937-99919f422ab8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:11.331411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-47a57abe-2832-4400-b937-99919f422ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:11.340339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568dda2a-f5db-4ea0-9d72-b43914ab9182 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:11.433248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb36922-e281-484f-9f4f-0b5e6e8f3ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:11.437663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb36922-e281-484f-9f4f-0b5e6e8f3ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:11.437663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb36922-e281-484f-9f4f-0b5e6e8f3ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:11.437663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb36922-e281-484f-9f4f-0b5e6e8f3ed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:11.437663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb36922-e281-484f-9f4f-0b5e6e8f3ed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:11.438318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a57abe-2832-4400-b937-99919f422ab8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:11.438459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb36922-e281-484f-9f4f-0b5e6e8f3ed5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:11.442551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b2e12-e46e-4c25-b2bf-3609ed8e8806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:11.443592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b2e12-e46e-4c25-b2bf-3609ed8e8806 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:11.443777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b2e12-e46e-4c25-b2bf-3609ed8e8806 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:11.444456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b2e12-e46e-4c25-b2bf-3609ed8e8806 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:11.445059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b2e12-e46e-4c25-b2bf-3609ed8e8806 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:11.447081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b2e12-e46e-4c25-b2bf-3609ed8e8806 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:11.460704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:11.552218 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3318/6673] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:27:11 2022] POST /identity/v3/users => generated 294 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:27:11.557778 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3356/6674] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:27:11 2022] POST /identity/v3/projects => generated 375 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:11.559356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b2e12-e46e-4c25-b2bf-3609ed8e8806 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:11.561523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb36922-e281-484f-9f4f-0b5e6e8f3ed5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:11.574909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b2e12-e46e-4c25-b2bf-3609ed8e8806 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:11.645644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb36922-e281-484f-9f4f-0b5e6e8f3ed5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:11.647301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f6bcd-8919-4f2e-8ce4-a9a7fafcb158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:11.648064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f6bcd-8919-4f2e-8ce4-a9a7fafcb158 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:11.648536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f6bcd-8919-4f2e-8ce4-a9a7fafcb158 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:11.648707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f6bcd-8919-4f2e-8ce4-a9a7fafcb158 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:11.649574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f6bcd-8919-4f2e-8ce4-a9a7fafcb158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:11.651324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f6bcd-8919-4f2e-8ce4-a9a7fafcb158 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:11.653198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b13e68-6077-42b1-abe1-eaf3b49457f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:11.653964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b13e68-6077-42b1-abe1-eaf3b49457f7 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:11.654221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b13e68-6077-42b1-abe1-eaf3b49457f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:11.654504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b13e68-6077-42b1-abe1-eaf3b49457f7 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:11.655218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b13e68-6077-42b1-abe1-eaf3b49457f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:11.657338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b13e68-6077-42b1-abe1-eaf3b49457f7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:11.665510 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3319/6675] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:11.670342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f6bcd-8919-4f2e-8ce4-a9a7fafcb158 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:11.681445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f6bcd-8919-4f2e-8ce4-a9a7fafcb158 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:11.686015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc5614-6807-426b-9f7e-7e4fea19ae32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:11.687008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc5614-6807-426b-9f7e-7e4fea19ae32 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:11.687303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc5614-6807-426b-9f7e-7e4fea19ae32 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:11.687589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc5614-6807-426b-9f7e-7e4fea19ae32 admin admin] PATH_INFO: `/v3/projects/433606efac7643e892a147f76d0c47f2/users/50b290f7bef44156beeff98f4df1af72/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:11.706380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc5614-6807-426b-9f7e-7e4fea19ae32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=433606efac7643e892a147f76d0c47f2, user_id=50b290f7bef44156beeff98f4df1af72, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:11.710027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccc5614-6807-426b-9f7e-7e4fea19ae32 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:11.810081 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3357/6676] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:27:11 2022] POST /identity/v3/users => generated 288 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:11.811660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b13e68-6077-42b1-abe1-eaf3b49457f7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:11.819005 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3320/6677] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:11 2022] PUT /identity/v3/projects/433606efac7643e892a147f76d0c47f2/users/50b290f7bef44156beeff98f4df1af72/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:11.822956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b13e68-6077-42b1-abe1-eaf3b49457f7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:11.823125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc5614-6807-426b-9f7e-7e4fea19ae32 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:11.908270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc5614-6807-426b-9f7e-7e4fea19ae32 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:11.916361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc18e9-a3df-436f-b631-6f4419920688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:11.917084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc18e9-a3df-436f-b631-6f4419920688 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:11.918176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc18e9-a3df-436f-b631-6f4419920688 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:11.918416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91980966-d71e-4705-aff1-fb2d07c13f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:11.918576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc18e9-a3df-436f-b631-6f4419920688 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:11.919105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91980966-d71e-4705-aff1-fb2d07c13f31 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:11.919358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc18e9-a3df-436f-b631-6f4419920688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:11.919501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91980966-d71e-4705-aff1-fb2d07c13f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:11.920219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91980966-d71e-4705-aff1-fb2d07c13f31 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:11.920608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91980966-d71e-4705-aff1-fb2d07c13f31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:11.921015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cc18e9-a3df-436f-b631-6f4419920688 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:11.923262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91980966-d71e-4705-aff1-fb2d07c13f31 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:11.932931 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3321/6678] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:11.939168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc18e9-a3df-436f-b631-6f4419920688 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:11.948254 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3358/6679] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:27:11.952971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91980966-d71e-4705-aff1-fb2d07c13f31 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:11.953947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc18e9-a3df-436f-b631-6f4419920688 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:11.959563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00f299-6ce7-48b0-97d1-623d0d69f1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:11.960649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00f299-6ce7-48b0-97d1-623d0d69f1a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:11.961884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00f299-6ce7-48b0-97d1-623d0d69f1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:11.962179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00f299-6ce7-48b0-97d1-623d0d69f1a0 admin admin] PATH_INFO: `/v3/projects/433606efac7643e892a147f76d0c47f2/users/50b290f7bef44156beeff98f4df1af72/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:11.964401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00f299-6ce7-48b0-97d1-623d0d69f1a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=433606efac7643e892a147f76d0c47f2, user_id=50b290f7bef44156beeff98f4df1af72, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:11.967559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f00f299-6ce7-48b0-97d1-623d0d69f1a0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:11.969933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91980966-d71e-4705-aff1-fb2d07c13f31 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:11.979418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5ae40-fcc2-4084-acd9-09507080a390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:11.980113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5ae40-fcc2-4084-acd9-09507080a390 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:11.980397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5ae40-fcc2-4084-acd9-09507080a390 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:11.980675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5ae40-fcc2-4084-acd9-09507080a390 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:11.986121 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3322/6680] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:11 2022] PUT /identity/v3/projects/433606efac7643e892a147f76d0c47f2/users/50b290f7bef44156beeff98f4df1af72/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:11.986334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5ae40-fcc2-4084-acd9-09507080a390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:11.986334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5ae40-fcc2-4084-acd9-09507080a390 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:11.993039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc357cd-5a40-4c9e-bcf3-3e191d07bcfe None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:11.995970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc357cd-5a40-4c9e-bcf3-3e191d07bcfe None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:11.995970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc357cd-5a40-4c9e-bcf3-3e191d07bcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:12.005580 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3359/6681] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:12.006781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5ae40-fcc2-4084-acd9-09507080a390 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:12.015285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:12.022543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5ae40-fcc2-4084-acd9-09507080a390 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:12.023064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-fbc357cd-5a40-4c9e-bcf3-3e191d07bcfe None None] MFA Rules not processed for user `50b290f7bef44156beeff98f4df1af72`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:12.024635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fbc357cd-5a40-4c9e-bcf3-3e191d07bcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:12.129265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee68b2f-31a9-4f1c-9130-14c37dadc07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:12.130193 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3323/6682] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:27:12.134351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee68b2f-31a9-4f1c-9130-14c37dadc07d admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:12.134692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee68b2f-31a9-4f1c-9130-14c37dadc07d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:12.134991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee68b2f-31a9-4f1c-9130-14c37dadc07d admin admin] PATH_INFO: `/v3/projects/33eec9e0a220410998fb241b74255e5a/users/cd29030cd5284eaf82dd2c6fe17bba73/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:12.140882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc357cd-5a40-4c9e-bcf3-3e191d07bcfe None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:12.142906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fbc357cd-5a40-4c9e-bcf3-3e191d07bcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:12.153424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee68b2f-31a9-4f1c-9130-14c37dadc07d admin admin] RBAC: Authorizing `identity:create_grant(project_id=33eec9e0a220410998fb241b74255e5a, user_id=cd29030cd5284eaf82dd2c6fe17bba73, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:12.157150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee68b2f-31a9-4f1c-9130-14c37dadc07d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:12.212751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc357cd-5a40-4c9e-bcf3-3e191d07bcfe None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:12.216883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90ef6c-7e95-4658-8473-88c7284508b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:12.217748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90ef6c-7e95-4658-8473-88c7284508b1 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:12.218068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90ef6c-7e95-4658-8473-88c7284508b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:12.218383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90ef6c-7e95-4658-8473-88c7284508b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:12.229156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90ef6c-7e95-4658-8473-88c7284508b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:12.230667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90ef6c-7e95-4658-8473-88c7284508b1 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:12.245339 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3324/6683] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:27:12.258440 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3360/6684] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:12 2022] PUT /identity/v3/projects/33eec9e0a220410998fb241b74255e5a/users/cd29030cd5284eaf82dd2c6fe17bba73/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:12.268991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae32a1-9d50-49b9-96d4-79ff5306542e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:12.269285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae32a1-9d50-49b9-96d4-79ff5306542e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:12.270066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aae32a1-9d50-49b9-96d4-79ff5306542e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:12.297082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7aae32a1-9d50-49b9-96d4-79ff5306542e None None] MFA Rules not processed for user `cd29030cd5284eaf82dd2c6fe17bba73`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:12.298358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7aae32a1-9d50-49b9-96d4-79ff5306542e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:12.370217 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3325/6685] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:12.384655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee68b2f-31a9-4f1c-9130-14c37dadc07d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:12.386632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eee68b2f-31a9-4f1c-9130-14c37dadc07d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:12.454609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee68b2f-31a9-4f1c-9130-14c37dadc07d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:12.458355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb8baed-b43c-4b4f-9bb8-ab7450df65a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:12.459571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8baed-b43c-4b4f-9bb8-ab7450df65a4 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:12.459859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8baed-b43c-4b4f-9bb8-ab7450df65a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:12.460137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb8baed-b43c-4b4f-9bb8-ab7450df65a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:12.468614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8baed-b43c-4b4f-9bb8-ab7450df65a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:12.472428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb8baed-b43c-4b4f-9bb8-ab7450df65a4 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:12.479116 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3361/6686] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:12.491891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae32a1-9d50-49b9-96d4-79ff5306542e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:12.559246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aae32a1-9d50-49b9-96d4-79ff5306542e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:12.563871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e781f-8c19-4ce3-a8e2-7b7d4e312a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:12.564981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e781f-8c19-4ce3-a8e2-7b7d4e312a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:12.565272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e781f-8c19-4ce3-a8e2-7b7d4e312a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:12.565550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e781f-8c19-4ce3-a8e2-7b7d4e312a50 admin admin] PATH_INFO: `/v3/users/cd29030cd5284eaf82dd2c6fe17bba73` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:12.566777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e781f-8c19-4ce3-a8e2-7b7d4e312a50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd29030cd5284eaf82dd2c6fe17bba73)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:12.570028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e781f-8c19-4ce3-a8e2-7b7d4e312a50 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:12.618110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:12.774070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-328e781f-8c19-4ce3-a8e2-7b7d4e312a50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd29030cd5284eaf82dd2c6fe17bba73'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:12.774743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-328e781f-8c19-4ce3-a8e2-7b7d4e312a50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd29030cd5284eaf82dd2c6fe17bba73'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:12.782151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-328e781f-8c19-4ce3-a8e2-7b7d4e312a50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd29030cd5284eaf82dd2c6fe17bba73'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:12.841973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-328e781f-8c19-4ce3-a8e2-7b7d4e312a50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd29030cd5284eaf82dd2c6fe17bba73'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:12.852650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-328e781f-8c19-4ce3-a8e2-7b7d4e312a50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd29030cd5284eaf82dd2c6fe17bba73'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:12.873764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df43d0d-958f-458d-b4be-7ff9e2894f90 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:12.874271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df43d0d-958f-458d-b4be-7ff9e2894f90 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:12.874427 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3326/6687] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:27:12 2022] DELETE /identity/v3/users/cd29030cd5284eaf82dd2c6fe17bba73 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:12.874536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df43d0d-958f-458d-b4be-7ff9e2894f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:12.898089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7df43d0d-958f-458d-b4be-7ff9e2894f90 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:12.899301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7df43d0d-958f-458d-b4be-7ff9e2894f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:12.958939 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3362/6688] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:12.968524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e781f-8c19-4ce3-a8e2-7b7d4e312a50 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:12.970272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-328e781f-8c19-4ce3-a8e2-7b7d4e312a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:13.033279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e781f-8c19-4ce3-a8e2-7b7d4e312a50 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:13.036998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbe095f-3065-4077-8376-f0153f3e19d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:13.037649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe095f-3065-4077-8376-f0153f3e19d6 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:13.037966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe095f-3065-4077-8376-f0153f3e19d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:13.038244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe095f-3065-4077-8376-f0153f3e19d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:13.047956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe095f-3065-4077-8376-f0153f3e19d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:13.049425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbe095f-3065-4077-8376-f0153f3e19d6 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:13.060265 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3327/6689] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:13.424743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:13.486401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df43d0d-958f-458d-b4be-7ff9e2894f90 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:13.488203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7df43d0d-958f-458d-b4be-7ff9e2894f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:13.508454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df43d0d-958f-458d-b4be-7ff9e2894f90 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:13.516578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63fef54-8ef1-47a8-ae56-aafad938411e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:13.517685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fef54-8ef1-47a8-ae56-aafad938411e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:13.518141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fef54-8ef1-47a8-ae56-aafad938411e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:13.518498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fef54-8ef1-47a8-ae56-aafad938411e admin admin] PATH_INFO: `/v3/projects/33eec9e0a220410998fb241b74255e5a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:13.520111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63fef54-8ef1-47a8-ae56-aafad938411e admin admin] RBAC: Authorizing `identity:delete_project(project_id=33eec9e0a220410998fb241b74255e5a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:13.523541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63fef54-8ef1-47a8-ae56-aafad938411e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:13.649233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f63fef54-8ef1-47a8-ae56-aafad938411e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33eec9e0a220410998fb241b74255e5a', 'request_id': 'req-f63fef54-8ef1-47a8-ae56-aafad938411e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:13.650078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f63fef54-8ef1-47a8-ae56-aafad938411e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33eec9e0a220410998fb241b74255e5a', 'request_id': 'req-f63fef54-8ef1-47a8-ae56-aafad938411e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:13.754641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f63fef54-8ef1-47a8-ae56-aafad938411e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33eec9e0a220410998fb241b74255e5a', 'request_id': 'req-f63fef54-8ef1-47a8-ae56-aafad938411e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:13.767061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f63fef54-8ef1-47a8-ae56-aafad938411e admin admin] The token cache is being invalidate because project 33eec9e0a220410998fb241b74255e5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:27:13.767359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f63fef54-8ef1-47a8-ae56-aafad938411e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:13.768086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f63fef54-8ef1-47a8-ae56-aafad938411e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:13.770449 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3363/6690] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:27:13 2022] DELETE /identity/v3/projects/33eec9e0a220410998fb241b74255e5a => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:13.775990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956344e-05e8-495a-a576-ee655220c321 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:13.776620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956344e-05e8-495a-a576-ee655220c321 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:13.776620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f956344e-05e8-495a-a576-ee655220c321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:13.796420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f956344e-05e8-495a-a576-ee655220c321 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:13.797639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f956344e-05e8-495a-a576-ee655220c321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:13.864396 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3328/6691] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:27:13.872405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf294a1b-317d-4b8d-877e-ad9d68e44602 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:13.873017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf294a1b-317d-4b8d-877e-ad9d68e44602 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:13.873475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf294a1b-317d-4b8d-877e-ad9d68e44602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:13.901580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cf294a1b-317d-4b8d-877e-ad9d68e44602 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:13.906548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cf294a1b-317d-4b8d-877e-ad9d68e44602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:13.913502 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3364/6692] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:13.917369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956344e-05e8-495a-a576-ee655220c321 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:13.929539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f956344e-05e8-495a-a576-ee655220c321 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:13.937831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7578d4c-c85d-4491-b3f4-3efd02d85135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:13.939061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7578d4c-c85d-4491-b3f4-3efd02d85135 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:13.939614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7578d4c-c85d-4491-b3f4-3efd02d85135 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:13.939900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7578d4c-c85d-4491-b3f4-3efd02d85135 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:13.940646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7578d4c-c85d-4491-b3f4-3efd02d85135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:27:13.940963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7578d4c-c85d-4491-b3f4-3efd02d85135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:13.942833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7578d4c-c85d-4491-b3f4-3efd02d85135 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:13.962106 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3329/6693] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:27:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:27:13.964337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf294a1b-317d-4b8d-877e-ad9d68e44602 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:13.982079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf294a1b-317d-4b8d-877e-ad9d68e44602 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:13.993267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a7b71-07ce-4052-a130-b1f15736b245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:13.994233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a7b71-07ce-4052-a130-b1f15736b245 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:13.994528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a7b71-07ce-4052-a130-b1f15736b245 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:13.994819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a7b71-07ce-4052-a130-b1f15736b245 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:13.995536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a7b71-07ce-4052-a130-b1f15736b245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:13.997473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a7b71-07ce-4052-a130-b1f15736b245 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:14.137032 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3365/6694] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:27:13 2022] POST /identity/v3/projects => generated 357 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:14.138317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7578d4c-c85d-4491-b3f4-3efd02d85135 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:14.149483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7578d4c-c85d-4491-b3f4-3efd02d85135 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:14.217207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe28fc0-207c-4149-9bd3-167d694a012b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:14.218722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe28fc0-207c-4149-9bd3-167d694a012b admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:14.219008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe28fc0-207c-4149-9bd3-167d694a012b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:14.219283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe28fc0-207c-4149-9bd3-167d694a012b admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:14.220603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe28fc0-207c-4149-9bd3-167d694a012b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:14.223794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe28fc0-207c-4149-9bd3-167d694a012b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:14.229034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:14.309819 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3330/6695] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:27:14 2022] POST /identity/v3/users => generated 279 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:14.314118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a7b71-07ce-4052-a130-b1f15736b245 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:14.332864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a7b71-07ce-4052-a130-b1f15736b245 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:14.332864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fd431-a312-4a5c-8e7d-846b74d74fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:14.332864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fd431-a312-4a5c-8e7d-846b74d74fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:14.332864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fd431-a312-4a5c-8e7d-846b74d74fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:14.333285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031fd431-a312-4a5c-8e7d-846b74d74fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:14.333285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031fd431-a312-4a5c-8e7d-846b74d74fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:14.333622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031fd431-a312-4a5c-8e7d-846b74d74fa2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:14.344161 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3366/6696] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:14.348280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe28fc0-207c-4149-9bd3-167d694a012b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:14.358856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe28fc0-207c-4149-9bd3-167d694a012b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:14.363774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158056d5-6b15-4857-9d93-55bc4bd0dfdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:14.364688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158056d5-6b15-4857-9d93-55bc4bd0dfdf admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:14.364979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158056d5-6b15-4857-9d93-55bc4bd0dfdf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:14.365763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158056d5-6b15-4857-9d93-55bc4bd0dfdf admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:14.366533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158056d5-6b15-4857-9d93-55bc4bd0dfdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:14.368160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158056d5-6b15-4857-9d93-55bc4bd0dfdf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:14.382106 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3331/6697] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:14.383464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fd431-a312-4a5c-8e7d-846b74d74fa2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:14.396386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031fd431-a312-4a5c-8e7d-846b74d74fa2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:14.408576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c6d11-ac0a-4993-a7a4-d5786236b44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:14.409572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c6d11-ac0a-4993-a7a4-d5786236b44c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:14.409873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c6d11-ac0a-4993-a7a4-d5786236b44c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:14.410493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c6d11-ac0a-4993-a7a4-d5786236b44c admin admin] PATH_INFO: `/v3/projects/9b13e9b82ca8467b8008de69cce86516/users/7cc1828624064f028d1060ee393d2da3/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:14.426901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c6d11-ac0a-4993-a7a4-d5786236b44c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b13e9b82ca8467b8008de69cce86516, user_id=7cc1828624064f028d1060ee393d2da3, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:14.430457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c6d11-ac0a-4993-a7a4-d5786236b44c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:14.510000 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3367/6698] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:14 2022] PUT /identity/v3/projects/9b13e9b82ca8467b8008de69cce86516/users/7cc1828624064f028d1060ee393d2da3/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:14.517749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:14.527844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13624218-9693-4853-a235-ad3dd143022b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:14.528145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13624218-9693-4853-a235-ad3dd143022b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:14.528459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13624218-9693-4853-a235-ad3dd143022b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:14.575253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-13624218-9693-4853-a235-ad3dd143022b None None] MFA Rules not processed for user `7cc1828624064f028d1060ee393d2da3`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:14.581812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-13624218-9693-4853-a235-ad3dd143022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:14.674572 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3332/6699] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:14 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:14.679281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c6d11-ac0a-4993-a7a4-d5786236b44c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:14.764874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c6d11-ac0a-4993-a7a4-d5786236b44c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:14.771181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16d04a-fb89-4483-89fc-60a38ca62f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:14.771826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16d04a-fb89-4483-89fc-60a38ca62f36 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:14.772107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16d04a-fb89-4483-89fc-60a38ca62f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:14.772578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16d04a-fb89-4483-89fc-60a38ca62f36 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:14.773278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16d04a-fb89-4483-89fc-60a38ca62f36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:14.775240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be16d04a-fb89-4483-89fc-60a38ca62f36 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:15.286683 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3368/6700] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:27:14 2022] POST /identity/v3/projects => generated 357 bytes in 608 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:15.291508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13624218-9693-4853-a235-ad3dd143022b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:15.302468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13624218-9693-4853-a235-ad3dd143022b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:15.391310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee0f77-e2d3-4083-b675-8dca98ca92f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:15.392010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee0f77-e2d3-4083-b675-8dca98ca92f6 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:15.392289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee0f77-e2d3-4083-b675-8dca98ca92f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:15.392570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee0f77-e2d3-4083-b675-8dca98ca92f6 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:15.393977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee0f77-e2d3-4083-b675-8dca98ca92f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:15.395355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee0f77-e2d3-4083-b675-8dca98ca92f6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:15.517797 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3333/6701] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:27:15 2022] POST /identity/v3/users => generated 285 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:15.522482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16d04a-fb89-4483-89fc-60a38ca62f36 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:15.536238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16d04a-fb89-4483-89fc-60a38ca62f36 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:15.540012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7fe09-6c6e-4548-91ae-41932ded6b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:15.541136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7fe09-6c6e-4548-91ae-41932ded6b14 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:15.541427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7fe09-6c6e-4548-91ae-41932ded6b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:15.541734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a7fe09-6c6e-4548-91ae-41932ded6b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:15.542555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7fe09-6c6e-4548-91ae-41932ded6b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:15.544330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a7fe09-6c6e-4548-91ae-41932ded6b14 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:15.559542 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3369/6702] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:27:15.560842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee0f77-e2d3-4083-b675-8dca98ca92f6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:15.572319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee0f77-e2d3-4083-b675-8dca98ca92f6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:15.582096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b14a0-612a-4013-8c19-1bf48f27be9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:15.583059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b14a0-612a-4013-8c19-1bf48f27be9a admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:15.584734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b14a0-612a-4013-8c19-1bf48f27be9a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:15.584734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b14a0-612a-4013-8c19-1bf48f27be9a admin admin] PATH_INFO: `/v3/projects/95df1c4813ec4c6680dc0da790d9eaa2/users/39cae3a116be44429add7517136bfefb/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:15.602451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b14a0-612a-4013-8c19-1bf48f27be9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=95df1c4813ec4c6680dc0da790d9eaa2, user_id=39cae3a116be44429add7517136bfefb, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:15.605333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b14a0-612a-4013-8c19-1bf48f27be9a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:15.742344 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3334/6703] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:15 2022] PUT /identity/v3/projects/95df1c4813ec4c6680dc0da790d9eaa2/users/39cae3a116be44429add7517136bfefb/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:15.744341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7fe09-6c6e-4548-91ae-41932ded6b14 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:15.755533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a7fe09-6c6e-4548-91ae-41932ded6b14 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:15.812321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28d35a-715f-4e64-a1c6-1209f83f95f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:15.812918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28d35a-715f-4e64-a1c6-1209f83f95f8 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:15.813232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28d35a-715f-4e64-a1c6-1209f83f95f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:15.813888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28d35a-715f-4e64-a1c6-1209f83f95f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:15.814636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28d35a-715f-4e64-a1c6-1209f83f95f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:15.816251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28d35a-715f-4e64-a1c6-1209f83f95f8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:15.826672 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3370/6704] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:27:15.831178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b14a0-612a-4013-8c19-1bf48f27be9a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:15.841640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b14a0-612a-4013-8c19-1bf48f27be9a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:15.846774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294ab89-a27d-4cda-8c4e-ce26429b8af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:15.847657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294ab89-a27d-4cda-8c4e-ce26429b8af9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:15.847942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294ab89-a27d-4cda-8c4e-ce26429b8af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:15.848223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5294ab89-a27d-4cda-8c4e-ce26429b8af9 admin admin] PATH_INFO: `/v3/projects/95df1c4813ec4c6680dc0da790d9eaa2/users/39cae3a116be44429add7517136bfefb/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:15.850434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5294ab89-a27d-4cda-8c4e-ce26429b8af9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95df1c4813ec4c6680dc0da790d9eaa2, user_id=39cae3a116be44429add7517136bfefb, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:15.854245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5294ab89-a27d-4cda-8c4e-ce26429b8af9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:15.863736 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3335/6705] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:15 2022] PUT /identity/v3/projects/95df1c4813ec4c6680dc0da790d9eaa2/users/39cae3a116be44429add7517136bfefb/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:15.874934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230de50-16ad-4a4c-bcf4-4b6d4096fe28 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:15.877508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230de50-16ad-4a4c-bcf4-4b6d4096fe28 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:15.877508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230de50-16ad-4a4c-bcf4-4b6d4096fe28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:15.906069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7230de50-16ad-4a4c-bcf4-4b6d4096fe28 None None] MFA Rules not processed for user `39cae3a116be44429add7517136bfefb`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:15.907315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7230de50-16ad-4a4c-bcf4-4b6d4096fe28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:15.996485 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3371/6706] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:15 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:27:16.010241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294ab89-a27d-4cda-8c4e-ce26429b8af9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:16.012690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5294ab89-a27d-4cda-8c4e-ce26429b8af9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:16.109498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5294ab89-a27d-4cda-8c4e-ce26429b8af9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:16.114389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e16698-8101-48d8-8a71-64d568825d6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:16.116477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e16698-8101-48d8-8a71-64d568825d6b service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:16.116477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e16698-8101-48d8-8a71-64d568825d6b service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:16.116477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e16698-8101-48d8-8a71-64d568825d6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:16.127132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e16698-8101-48d8-8a71-64d568825d6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:16.128699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e16698-8101-48d8-8a71-64d568825d6b service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:16.141935 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3336/6707] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:16 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:16.382628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230de50-16ad-4a4c-bcf4-4b6d4096fe28 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:16.394450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230de50-16ad-4a4c-bcf4-4b6d4096fe28 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:16.398940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3aa7a-8db7-4346-a3af-53d2bd60f84d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:16.399580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3aa7a-8db7-4346-a3af-53d2bd60f84d service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:16.399868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3aa7a-8db7-4346-a3af-53d2bd60f84d service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:16.400147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f3aa7a-8db7-4346-a3af-53d2bd60f84d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:16.400533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:16.456420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3aa7a-8db7-4346-a3af-53d2bd60f84d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:16.457913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f3aa7a-8db7-4346-a3af-53d2bd60f84d service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:16.468054 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3372/6708] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:16 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:17.953676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:18.331684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:18.718865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbc1fc-bed8-4857-a0dc-a8fbf87308c4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:18.721126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbc1fc-bed8-4857-a0dc-a8fbf87308c4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:18.721600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbc1fc-bed8-4857-a0dc-a8fbf87308c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:18.744937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-38fbc1fc-bed8-4857-a0dc-a8fbf87308c4 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:18.747467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-38fbc1fc-bed8-4857-a0dc-a8fbf87308c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:18.819046 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3337/6709] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:27:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:18.833923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3aa7a-8db7-4346-a3af-53d2bd60f84d service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:18.833923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f5f3aa7a-8db7-4346-a3af-53d2bd60f84d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:18.911180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f3aa7a-8db7-4346-a3af-53d2bd60f84d service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:18.916327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4997d-3815-43e9-9295-c58822a4eb2b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:18.916987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4997d-3815-43e9-9295-c58822a4eb2b service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:18.917275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4997d-3815-43e9-9295-c58822a4eb2b service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:18.917557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad4997d-3815-43e9-9295-c58822a4eb2b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:18.926265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4997d-3815-43e9-9295-c58822a4eb2b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:18.928415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad4997d-3815-43e9-9295-c58822a4eb2b service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:18.938034 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3373/6710] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:27:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:27:19.285495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbc1fc-bed8-4857-a0dc-a8fbf87308c4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:19.287746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-38fbc1fc-bed8-4857-a0dc-a8fbf87308c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:19.360043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbc1fc-bed8-4857-a0dc-a8fbf87308c4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:19.364604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7ff4b-6d35-4620-9d89-a69fc34a0bea tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b290f7bef44156beeff98f4df1af72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '433606efac7643e892a147f76d0c47f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:19.365888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7ff4b-6d35-4620-9d89-a69fc34a0bea tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:19.366185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7ff4b-6d35-4620-9d89-a69fc34a0bea tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:19.366465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e7ff4b-6d35-4620-9d89-a69fc34a0bea tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:19.367201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7ff4b-6d35-4620-9d89-a69fc34a0bea tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:27:19.367517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7ff4b-6d35-4620-9d89-a69fc34a0bea tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:19.369134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e7ff4b-6d35-4620-9d89-a69fc34a0bea tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:19.381669 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3338/6711] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:27:19.386509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4997d-3815-43e9-9295-c58822a4eb2b service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:19.398185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad4997d-3815-43e9-9295-c58822a4eb2b service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:19.403860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c0539-7f2f-494b-84fd-6aed39db54ca tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b290f7bef44156beeff98f4df1af72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '433606efac7643e892a147f76d0c47f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:19.404540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c0539-7f2f-494b-84fd-6aed39db54ca tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:19.405009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c0539-7f2f-494b-84fd-6aed39db54ca tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:19.405366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c0539-7f2f-494b-84fd-6aed39db54ca tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:19.406155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c0539-7f2f-494b-84fd-6aed39db54ca tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:19.410233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c0539-7f2f-494b-84fd-6aed39db54ca tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:19.486283 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3374/6712] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:27:19 2022] POST /identity/v3/projects => generated 347 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:19.489418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7ff4b-6d35-4620-9d89-a69fc34a0bea tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:19.501636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e7ff4b-6d35-4620-9d89-a69fc34a0bea tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:19.562297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9793164-42d0-4ad6-ae7c-e271373b0c01 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b290f7bef44156beeff98f4df1af72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '433606efac7643e892a147f76d0c47f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:19.563331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9793164-42d0-4ad6-ae7c-e271373b0c01 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:19.563530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9793164-42d0-4ad6-ae7c-e271373b0c01 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:19.563810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9793164-42d0-4ad6-ae7c-e271373b0c01 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:19.564553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9793164-42d0-4ad6-ae7c-e271373b0c01 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:27:19.564868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9793164-42d0-4ad6-ae7c-e271373b0c01 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:19.566638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9793164-42d0-4ad6-ae7c-e271373b0c01 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:19.584307 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3339/6713] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:27:20.051679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:20.576773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:20.844972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c0539-7f2f-494b-84fd-6aed39db54ca tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:20.855977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c0539-7f2f-494b-84fd-6aed39db54ca tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:20.860793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e4d61-954a-42f3-ae0e-6f5c231e4b83 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50b290f7bef44156beeff98f4df1af72', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '433606efac7643e892a147f76d0c47f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:20.861730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e4d61-954a-42f3-ae0e-6f5c231e4b83 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:20.862126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e4d61-954a-42f3-ae0e-6f5c231e4b83 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:20.862475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e4d61-954a-42f3-ae0e-6f5c231e4b83 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] PATH_INFO: `/v3/projects/d42e213380dc46a2a2811d27eb20af41` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:20.863783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e4d61-954a-42f3-ae0e-6f5c231e4b83 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d42e213380dc46a2a2811d27eb20af41)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:20.865789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e4d61-954a-42f3-ae0e-6f5c231e4b83 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:20.961654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6c1e4d61-954a-42f3-ae0e-6f5c231e4b83 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd42e213380dc46a2a2811d27eb20af41', 'request_id': 'req-6c1e4d61-954a-42f3-ae0e-6f5c231e4b83'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:20.967314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6c1e4d61-954a-42f3-ae0e-6f5c231e4b83 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd42e213380dc46a2a2811d27eb20af41', 'request_id': 'req-6c1e4d61-954a-42f3-ae0e-6f5c231e4b83'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:21.061989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6c1e4d61-954a-42f3-ae0e-6f5c231e4b83 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd42e213380dc46a2a2811d27eb20af41', 'request_id': 'req-6c1e4d61-954a-42f3-ae0e-6f5c231e4b83'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:21.079550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6c1e4d61-954a-42f3-ae0e-6f5c231e4b83 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] The token cache is being invalidate because project d42e213380dc46a2a2811d27eb20af41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:27:21.080024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6c1e4d61-954a-42f3-ae0e-6f5c231e4b83 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:21.080915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6c1e4d61-954a-42f3-ae0e-6f5c231e4b83 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:21.082683 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3375/6714] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:27:20 2022] DELETE /identity/v3/projects/d42e213380dc46a2a2811d27eb20af41 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:21.094626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9793164-42d0-4ad6-ae7c-e271373b0c01 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:21.100389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b9793164-42d0-4ad6-ae7c-e271373b0c01 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:21.166968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9793164-42d0-4ad6-ae7c-e271373b0c01 tempest-RoutersIpV6AdminTest-785713032 tempest-RoutersIpV6AdminTest-785713032-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:21.225884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4aa61-a42c-49d6-9394-de5dc9594aa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:21.226481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4aa61-a42c-49d6-9394-de5dc9594aa1 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:21.227052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4aa61-a42c-49d6-9394-de5dc9594aa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:21.227346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c4aa61-a42c-49d6-9394-de5dc9594aa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:21.229525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-47c4aa61-a42c-49d6-9394-de5dc9594aa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:21.279335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:21.340182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4aa61-a42c-49d6-9394-de5dc9594aa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:21.341986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c4aa61-a42c-49d6-9394-de5dc9594aa1 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:21.357021 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3340/6715] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:22.216971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:22.970945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:23.405807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:24.675138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:25.231285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:25.861860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41793b-2d28-4e9c-a41a-3f23e2f872f5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:25.863035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41793b-2d28-4e9c-a41a-3f23e2f872f5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:25.863553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41793b-2d28-4e9c-a41a-3f23e2f872f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:25.884005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8e41793b-2d28-4e9c-a41a-3f23e2f872f5 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:25.885512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8e41793b-2d28-4e9c-a41a-3f23e2f872f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:25.952319 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3376/6716] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:27:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:25.975704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4aa61-a42c-49d6-9394-de5dc9594aa1 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:25.977685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-47c4aa61-a42c-49d6-9394-de5dc9594aa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:26.077269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c4aa61-a42c-49d6-9394-de5dc9594aa1 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:26.084105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eedea8-c590-4eae-96b7-6d4eb784aa8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:26.087804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eedea8-c590-4eae-96b7-6d4eb784aa8e service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:26.087804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eedea8-c590-4eae-96b7-6d4eb784aa8e service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:26.087804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eedea8-c590-4eae-96b7-6d4eb784aa8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:26.098464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eedea8-c590-4eae-96b7-6d4eb784aa8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:26.100832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eedea8-c590-4eae-96b7-6d4eb784aa8e service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:26.113519 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3341/6717] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:27:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 21:27:26.886354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:27.143667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:27.293108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c031d-fdd1-4110-8922-205e59ca7b0a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:27.293456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c031d-fdd1-4110-8922-205e59ca7b0a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:27.293738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c031d-fdd1-4110-8922-205e59ca7b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:27.314230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-524c031d-fdd1-4110-8922-205e59ca7b0a None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:27.315468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-524c031d-fdd1-4110-8922-205e59ca7b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:27.323709 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3377/6718] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:27:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:27.347303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eedea8-c590-4eae-96b7-6d4eb784aa8e service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:27.360944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eedea8-c590-4eae-96b7-6d4eb784aa8e service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:27.366922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc6d1d-7785-4790-92df-936cad7600dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:27.367581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc6d1d-7785-4790-92df-936cad7600dc service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:27.367872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc6d1d-7785-4790-92df-936cad7600dc service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:27.368151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc6d1d-7785-4790-92df-936cad7600dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:27.377578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc6d1d-7785-4790-92df-936cad7600dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:27.379884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc6d1d-7785-4790-92df-936cad7600dc service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:27.405966 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3342/6719] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:27:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:27:28.177203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b640d7c-6dc2-4e2a-bd03-9521e75fde77 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:28.177619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b640d7c-6dc2-4e2a-bd03-9521e75fde77 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:28.178024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b640d7c-6dc2-4e2a-bd03-9521e75fde77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:28.200619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8b640d7c-6dc2-4e2a-bd03-9521e75fde77 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:28.202699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8b640d7c-6dc2-4e2a-bd03-9521e75fde77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:28.216496 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3378/6720] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:27:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:28.228165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc6d1d-7785-4790-92df-936cad7600dc service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:28.246073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc6d1d-7785-4790-92df-936cad7600dc service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:28.265611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32185fa2-414d-404c-a494-4afc82c3a7e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:28.266818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32185fa2-414d-404c-a494-4afc82c3a7e0 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:28.267229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32185fa2-414d-404c-a494-4afc82c3a7e0 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:28.271657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32185fa2-414d-404c-a494-4afc82c3a7e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:28.279828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32185fa2-414d-404c-a494-4afc82c3a7e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:28.281980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32185fa2-414d-404c-a494-4afc82c3a7e0 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:28.295623 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3343/6721] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:27:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:27:28.939989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:29.200913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc9af4-9423-4914-9394-c6fd9741ec24 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:29.201875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc9af4-9423-4914-9394-c6fd9741ec24 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:29.202508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc9af4-9423-4914-9394-c6fd9741ec24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:29.225470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-53cc9af4-9423-4914-9394-c6fd9741ec24 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:29.227591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-53cc9af4-9423-4914-9394-c6fd9741ec24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:29.244819 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3379/6722] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:27:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:29.272479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32185fa2-414d-404c-a494-4afc82c3a7e0 service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:29.304728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32185fa2-414d-404c-a494-4afc82c3a7e0 service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:29.325659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae2c6a-691a-407c-96e9-9c47096f1b12 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:29.326355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae2c6a-691a-407c-96e9-9c47096f1b12 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:29.326627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae2c6a-691a-407c-96e9-9c47096f1b12 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:29.327120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae2c6a-691a-407c-96e9-9c47096f1b12 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:29.339724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae2c6a-691a-407c-96e9-9c47096f1b12 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:29.343295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ae2c6a-691a-407c-96e9-9c47096f1b12 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:29.363365 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3344/6723] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:27:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:27:29.909210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:30.257914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:30.286196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:31.183962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:31.284693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:32.834381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:32.862839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:33.485828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:33.739980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc9af4-9423-4914-9394-c6fd9741ec24 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:33.743524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:33.746732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-53cc9af4-9423-4914-9394-c6fd9741ec24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:33.838958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc9af4-9423-4914-9394-c6fd9741ec24 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:33.844482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351edc21-003c-48a9-aadd-36f024b26a92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:33.845412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351edc21-003c-48a9-aadd-36f024b26a92 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:33.845704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351edc21-003c-48a9-aadd-36f024b26a92 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:33.846023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351edc21-003c-48a9-aadd-36f024b26a92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:33.847855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-351edc21-003c-48a9-aadd-36f024b26a92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:33.911629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351edc21-003c-48a9-aadd-36f024b26a92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:33.913403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351edc21-003c-48a9-aadd-36f024b26a92 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:33.924353 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3380/6724] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:34.360522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:35.102832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:36.296789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:36.851823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:37.439379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:38.399125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae2c6a-691a-407c-96e9-9c47096f1b12 service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:38.400598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c5ae2c6a-691a-407c-96e9-9c47096f1b12 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:38.434742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:38.469324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae2c6a-691a-407c-96e9-9c47096f1b12 service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:38.475435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6aa68-ad0f-4e53-b589-2a694edcdcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:38.476489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6aa68-ad0f-4e53-b589-2a694edcdcd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:38.476774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6aa68-ad0f-4e53-b589-2a694edcdcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:38.477530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6aa68-ad0f-4e53-b589-2a694edcdcd1 admin admin] PATH_INFO: `/v3/users/d470c6d9a88d4b758d9df497c450cffb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:38.479077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6aa68-ad0f-4e53-b589-2a694edcdcd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d470c6d9a88d4b758d9df497c450cffb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:38.482577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6aa68-ad0f-4e53-b589-2a694edcdcd1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:38.667339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:38.681233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e1f6aa68-ad0f-4e53-b589-2a694edcdcd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd470c6d9a88d4b758d9df497c450cffb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:38.681970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e1f6aa68-ad0f-4e53-b589-2a694edcdcd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd470c6d9a88d4b758d9df497c450cffb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:38.689406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e1f6aa68-ad0f-4e53-b589-2a694edcdcd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd470c6d9a88d4b758d9df497c450cffb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:38.770108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e1f6aa68-ad0f-4e53-b589-2a694edcdcd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd470c6d9a88d4b758d9df497c450cffb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:38.770766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e1f6aa68-ad0f-4e53-b589-2a694edcdcd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd470c6d9a88d4b758d9df497c450cffb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:38.789436 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3345/6725] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:27:38 2022] DELETE /identity/v3/users/d470c6d9a88d4b758d9df497c450cffb => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:38.794630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808594df-1441-4c8d-86df-4ea0cc9a8603 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:38.803376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808594df-1441-4c8d-86df-4ea0cc9a8603 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:38.803376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808594df-1441-4c8d-86df-4ea0cc9a8603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:38.813650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-808594df-1441-4c8d-86df-4ea0cc9a8603 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:38.814913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-808594df-1441-4c8d-86df-4ea0cc9a8603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:38.877508 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3381/6726] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:38.887495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6aa68-ad0f-4e53-b589-2a694edcdcd1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:38.888921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e1f6aa68-ad0f-4e53-b589-2a694edcdcd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:38.957277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6aa68-ad0f-4e53-b589-2a694edcdcd1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:38.961509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a196a02-0664-4759-82d5-6df63264257b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:38.962310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a196a02-0664-4759-82d5-6df63264257b service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:38.962606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a196a02-0664-4759-82d5-6df63264257b service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:38.962890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a196a02-0664-4759-82d5-6df63264257b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:38.973590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a196a02-0664-4759-82d5-6df63264257b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:38.975292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a196a02-0664-4759-82d5-6df63264257b service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:38.985171 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3346/6727] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:38.988858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:39.246840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808594df-1441-4c8d-86df-4ea0cc9a8603 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:39.248266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-808594df-1441-4c8d-86df-4ea0cc9a8603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:39.262503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808594df-1441-4c8d-86df-4ea0cc9a8603 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:39.270514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afffa12-eb47-4a46-82c3-6ec19ad85369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:39.273343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afffa12-eb47-4a46-82c3-6ec19ad85369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:39.273877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afffa12-eb47-4a46-82c3-6ec19ad85369 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:39.274548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afffa12-eb47-4a46-82c3-6ec19ad85369 admin admin] PATH_INFO: `/v3/projects/760e1d0cdcff4948bc9f8849ecc56e1c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:39.276541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afffa12-eb47-4a46-82c3-6ec19ad85369 admin admin] RBAC: Authorizing `identity:delete_project(project_id=760e1d0cdcff4948bc9f8849ecc56e1c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:39.278769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afffa12-eb47-4a46-82c3-6ec19ad85369 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:39.406685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4afffa12-eb47-4a46-82c3-6ec19ad85369 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '760e1d0cdcff4948bc9f8849ecc56e1c', 'request_id': 'req-4afffa12-eb47-4a46-82c3-6ec19ad85369'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:39.414829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4afffa12-eb47-4a46-82c3-6ec19ad85369 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '760e1d0cdcff4948bc9f8849ecc56e1c', 'request_id': 'req-4afffa12-eb47-4a46-82c3-6ec19ad85369'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:39.524188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4afffa12-eb47-4a46-82c3-6ec19ad85369 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '760e1d0cdcff4948bc9f8849ecc56e1c', 'request_id': 'req-4afffa12-eb47-4a46-82c3-6ec19ad85369'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:39.538084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4afffa12-eb47-4a46-82c3-6ec19ad85369 admin admin] The token cache is being invalidate because project 760e1d0cdcff4948bc9f8849ecc56e1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:27:39.538377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4afffa12-eb47-4a46-82c3-6ec19ad85369 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:39.539246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4afffa12-eb47-4a46-82c3-6ec19ad85369 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:39.540821 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3382/6728] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:27:39 2022] DELETE /identity/v3/projects/760e1d0cdcff4948bc9f8849ecc56e1c => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:39.546184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5543438-79be-4592-ab8b-cb649ec78921 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:39.547172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5543438-79be-4592-ab8b-cb649ec78921 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:39.547487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5543438-79be-4592-ab8b-cb649ec78921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:39.566487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c5543438-79be-4592-ab8b-cb649ec78921 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:39.567606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c5543438-79be-4592-ab8b-cb649ec78921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:39.623612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:39.638893 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3347/6729] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:27:39.647495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eca90-e9cb-4451-b72f-5655b65a13f4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:39.647792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eca90-e9cb-4451-b72f-5655b65a13f4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:39.648102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1eca90-e9cb-4451-b72f-5655b65a13f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:39.666458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9f1eca90-e9cb-4451-b72f-5655b65a13f4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:39.667590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9f1eca90-e9cb-4451-b72f-5655b65a13f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:39.676175 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3383/6730] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:39.680506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5543438-79be-4592-ab8b-cb649ec78921 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:39.690398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5543438-79be-4592-ab8b-cb649ec78921 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:39.695757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16afd9-9825-44f8-b615-787640baba67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:39.696455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16afd9-9825-44f8-b615-787640baba67 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:39.696742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16afd9-9825-44f8-b615-787640baba67 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:39.697023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb16afd9-9825-44f8-b615-787640baba67 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:39.697763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16afd9-9825-44f8-b615-787640baba67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:27:39.698122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16afd9-9825-44f8-b615-787640baba67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:39.699765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb16afd9-9825-44f8-b615-787640baba67 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:39.721654 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3348/6731] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:27:39.723726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eca90-e9cb-4451-b72f-5655b65a13f4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:39.735828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1eca90-e9cb-4451-b72f-5655b65a13f4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:39.744532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff300c-a977-4ffc-9f63-2c69a86e2eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:39.745462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff300c-a977-4ffc-9f63-2c69a86e2eda admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:39.746109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff300c-a977-4ffc-9f63-2c69a86e2eda admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:39.746549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff300c-a977-4ffc-9f63-2c69a86e2eda admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:39.747785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff300c-a977-4ffc-9f63-2c69a86e2eda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:39.750032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff300c-a977-4ffc-9f63-2c69a86e2eda admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:39.898357 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3384/6732] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:27:39 2022] POST /identity/v3/projects => generated 399 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:39.901822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16afd9-9825-44f8-b615-787640baba67 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:39.911846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb16afd9-9825-44f8-b615-787640baba67 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:39.971119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:39.975804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7704c-afc1-4d63-a392-e2189c1117bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:39.980025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7704c-afc1-4d63-a392-e2189c1117bd admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:39.980025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7704c-afc1-4d63-a392-e2189c1117bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:39.980025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7704c-afc1-4d63-a392-e2189c1117bd admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:39.980025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7704c-afc1-4d63-a392-e2189c1117bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:39.980025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7704c-afc1-4d63-a392-e2189c1117bd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:40.087736 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3349/6733] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:27:39 2022] POST /identity/v3/users => generated 300 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:40.093767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff300c-a977-4ffc-9f63-2c69a86e2eda admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:40.106320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff300c-a977-4ffc-9f63-2c69a86e2eda admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:40.115184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e66d9-3fd6-4d7e-a777-078137dffdbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:40.117413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e66d9-3fd6-4d7e-a777-078137dffdbc admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:40.117952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e66d9-3fd6-4d7e-a777-078137dffdbc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:40.119012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e66d9-3fd6-4d7e-a777-078137dffdbc admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:40.119978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e66d9-3fd6-4d7e-a777-078137dffdbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:40.121982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e66d9-3fd6-4d7e-a777-078137dffdbc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:40.159535 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3385/6734] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:40.160133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7704c-afc1-4d63-a392-e2189c1117bd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:40.170254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7704c-afc1-4d63-a392-e2189c1117bd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:40.180039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1da78a-733a-40e8-aabe-89ead308604b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:40.180730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1da78a-733a-40e8-aabe-89ead308604b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:40.181207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1da78a-733a-40e8-aabe-89ead308604b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:40.181671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1da78a-733a-40e8-aabe-89ead308604b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:40.182626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1da78a-733a-40e8-aabe-89ead308604b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:40.189383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1da78a-733a-40e8-aabe-89ead308604b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:40.199817 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3350/6735] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:40.201287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e66d9-3fd6-4d7e-a777-078137dffdbc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:40.211953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e66d9-3fd6-4d7e-a777-078137dffdbc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:40.219714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb6003-68e0-42a1-9d5d-3d218d0c0f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:40.220877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb6003-68e0-42a1-9d5d-3d218d0c0f03 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:40.221159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb6003-68e0-42a1-9d5d-3d218d0c0f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:40.221437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcb6003-68e0-42a1-9d5d-3d218d0c0f03 admin admin] PATH_INFO: `/v3/projects/bcf76791bd3e47b68ca0381e604c3c53/users/aa0d39e663594fb596b7be9def994e22/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:40.237301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb6003-68e0-42a1-9d5d-3d218d0c0f03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcf76791bd3e47b68ca0381e604c3c53, user_id=aa0d39e663594fb596b7be9def994e22, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:40.241621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcb6003-68e0-42a1-9d5d-3d218d0c0f03 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:40.316164 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3386/6736] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:40 2022] PUT /identity/v3/projects/bcf76791bd3e47b68ca0381e604c3c53/users/aa0d39e663594fb596b7be9def994e22/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:40.327312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5903fd3-b35a-4dae-9bda-048dec97ae37 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:40.337954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5903fd3-b35a-4dae-9bda-048dec97ae37 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:40.337954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5903fd3-b35a-4dae-9bda-048dec97ae37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:40.358630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a5903fd3-b35a-4dae-9bda-048dec97ae37 None None] MFA Rules not processed for user `aa0d39e663594fb596b7be9def994e22`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:40.360882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a5903fd3-b35a-4dae-9bda-048dec97ae37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:40.411778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb6003-68e0-42a1-9d5d-3d218d0c0f03 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:40.416115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cbcb6003-68e0-42a1-9d5d-3d218d0c0f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:40.455463 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3351/6737] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:40.456630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5903fd3-b35a-4dae-9bda-048dec97ae37 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:40.459832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a5903fd3-b35a-4dae-9bda-048dec97ae37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:40.533415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcb6003-68e0-42a1-9d5d-3d218d0c0f03 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:40.541228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c42bb3-c776-46f3-8bd5-55e86c57f856 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:40.542191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c42bb3-c776-46f3-8bd5-55e86c57f856 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:40.550092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c42bb3-c776-46f3-8bd5-55e86c57f856 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:40.550092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c42bb3-c776-46f3-8bd5-55e86c57f856 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:40.550092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-43c42bb3-c776-46f3-8bd5-55e86c57f856 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:40.551035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5903fd3-b35a-4dae-9bda-048dec97ae37 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:40.554642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211a7ce-3705-4961-ad4f-184ab91a4b59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:40.555369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211a7ce-3705-4961-ad4f-184ab91a4b59 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:40.555654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211a7ce-3705-4961-ad4f-184ab91a4b59 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:40.556005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211a7ce-3705-4961-ad4f-184ab91a4b59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:40.568128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211a7ce-3705-4961-ad4f-184ab91a4b59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:40.578808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211a7ce-3705-4961-ad4f-184ab91a4b59 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:40.591612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:40.594424 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3352/6738] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:40 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:40.626670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c42bb3-c776-46f3-8bd5-55e86c57f856 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:40.628286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c42bb3-c776-46f3-8bd5-55e86c57f856 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:40.640858 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3387/6739] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:40 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:40.879301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:40.939558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211a7ce-3705-4961-ad4f-184ab91a4b59 service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:40.941403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b211a7ce-3705-4961-ad4f-184ab91a4b59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:40.961333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211a7ce-3705-4961-ad4f-184ab91a4b59 service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:40.968493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd834da-ebce-4173-9337-5464caf4306f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:40.969305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd834da-ebce-4173-9337-5464caf4306f service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:40.969759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd834da-ebce-4173-9337-5464caf4306f service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:40.970331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd834da-ebce-4173-9337-5464caf4306f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:40.972547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4bd834da-ebce-4173-9337-5464caf4306f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:41.056880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd834da-ebce-4173-9337-5464caf4306f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:41.058593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd834da-ebce-4173-9337-5464caf4306f service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:41.086075 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3353/6740] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:40 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:42.191451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac29a5-203a-456f-b627-fe76629d6e35 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:42.192320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac29a5-203a-456f-b627-fe76629d6e35 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:42.192609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fac29a5-203a-456f-b627-fe76629d6e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:42.216795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0fac29a5-203a-456f-b627-fe76629d6e35 None None] MFA Rules not processed for user `02d5fbb072814fd7923ecf12c7ecad7b`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:42.217973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0fac29a5-203a-456f-b627-fe76629d6e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:42.226041 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3388/6741] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:42.378986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd834da-ebce-4173-9337-5464caf4306f service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:42.381391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4bd834da-ebce-4173-9337-5464caf4306f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:42.492583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd834da-ebce-4173-9337-5464caf4306f service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:42.503695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbcbfa-6fe5-4ac0-aa08-09f3a768886b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:42.504820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbcbfa-6fe5-4ac0-aa08-09f3a768886b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:42.505289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbcbfa-6fe5-4ac0-aa08-09f3a768886b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:42.505752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fbcbfa-6fe5-4ac0-aa08-09f3a768886b admin admin] PATH_INFO: `/v3/users/41c3af0358084a9da8f56f0b37ed7834` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:42.507301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fbcbfa-6fe5-4ac0-aa08-09f3a768886b admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c3af0358084a9da8f56f0b37ed7834)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:42.509427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fbcbfa-6fe5-4ac0-aa08-09f3a768886b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:42.587742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac29a5-203a-456f-b627-fe76629d6e35 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:42.589700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0fac29a5-203a-456f-b627-fe76629d6e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:42.656062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fac29a5-203a-456f-b627-fe76629d6e35 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:42.660840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85192a71-0f8c-4db1-8ce1-09e4004cff2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:42.661510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85192a71-0f8c-4db1-8ce1-09e4004cff2f service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:42.661800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85192a71-0f8c-4db1-8ce1-09e4004cff2f service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:42.662630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85192a71-0f8c-4db1-8ce1-09e4004cff2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:42.664499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-85192a71-0f8c-4db1-8ce1-09e4004cff2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:42.677007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85192a71-0f8c-4db1-8ce1-09e4004cff2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:42.682407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85192a71-0f8c-4db1-8ce1-09e4004cff2f service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:42.691948 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3389/6742] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:27:42.894089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-56fbcbfa-6fe5-4ac0-aa08-09f3a768886b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c3af0358084a9da8f56f0b37ed7834'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:42.894853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-56fbcbfa-6fe5-4ac0-aa08-09f3a768886b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c3af0358084a9da8f56f0b37ed7834'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:42.922711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-56fbcbfa-6fe5-4ac0-aa08-09f3a768886b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c3af0358084a9da8f56f0b37ed7834'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:43.032846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-56fbcbfa-6fe5-4ac0-aa08-09f3a768886b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c3af0358084a9da8f56f0b37ed7834'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:43.033755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-56fbcbfa-6fe5-4ac0-aa08-09f3a768886b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c3af0358084a9da8f56f0b37ed7834'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:43.068790 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3354/6743] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:27:42 2022] DELETE /identity/v3/users/41c3af0358084a9da8f56f0b37ed7834 => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:43.073349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ea733-b743-47ab-947a-5905bc0d1141 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:43.073735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ea733-b743-47ab-947a-5905bc0d1141 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:43.074105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ea733-b743-47ab-947a-5905bc0d1141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:43.106990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8c5ea733-b743-47ab-947a-5905bc0d1141 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:43.108252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8c5ea733-b743-47ab-947a-5905bc0d1141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:43.255138 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3390/6744] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:43.269580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbcbfa-6fe5-4ac0-aa08-09f3a768886b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:43.271184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-56fbcbfa-6fe5-4ac0-aa08-09f3a768886b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:43.291683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:43.396172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fbcbfa-6fe5-4ac0-aa08-09f3a768886b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:43.404140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296e897-def5-47a6-8a19-860c123ea816 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:43.405090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296e897-def5-47a6-8a19-860c123ea816 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:43.405794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296e897-def5-47a6-8a19-860c123ea816 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:43.406345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9296e897-def5-47a6-8a19-860c123ea816 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:43.433668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9296e897-def5-47a6-8a19-860c123ea816 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:43.447409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9296e897-def5-47a6-8a19-860c123ea816 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:43.482230 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3355/6745] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:44.172670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ea733-b743-47ab-947a-5905bc0d1141 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:44.174815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8c5ea733-b743-47ab-947a-5905bc0d1141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:44.213378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ea733-b743-47ab-947a-5905bc0d1141 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:44.219292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b9179-eb1c-442e-89f8-305af74e11e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:44.220616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b9179-eb1c-442e-89f8-305af74e11e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:44.220855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b9179-eb1c-442e-89f8-305af74e11e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:44.221564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b9179-eb1c-442e-89f8-305af74e11e8 admin admin] PATH_INFO: `/v3/projects/aaed16ff29234c81b161434120a4bf12` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:44.223032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b9179-eb1c-442e-89f8-305af74e11e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaed16ff29234c81b161434120a4bf12)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:44.226150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b9179-eb1c-442e-89f8-305af74e11e8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:44.417936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fb4b9179-eb1c-442e-89f8-305af74e11e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaed16ff29234c81b161434120a4bf12', 'request_id': 'req-fb4b9179-eb1c-442e-89f8-305af74e11e8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:44.426335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fb4b9179-eb1c-442e-89f8-305af74e11e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaed16ff29234c81b161434120a4bf12', 'request_id': 'req-fb4b9179-eb1c-442e-89f8-305af74e11e8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:44.434107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:44.544690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fb4b9179-eb1c-442e-89f8-305af74e11e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaed16ff29234c81b161434120a4bf12', 'request_id': 'req-fb4b9179-eb1c-442e-89f8-305af74e11e8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:44.559323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fb4b9179-eb1c-442e-89f8-305af74e11e8 admin admin] The token cache is being invalidate because project aaed16ff29234c81b161434120a4bf12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:27:44.559857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fb4b9179-eb1c-442e-89f8-305af74e11e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:44.560866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fb4b9179-eb1c-442e-89f8-305af74e11e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:44.563146 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3391/6746] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:27:44 2022] DELETE /identity/v3/projects/aaed16ff29234c81b161434120a4bf12 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:44.567461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296e897-def5-47a6-8a19-860c123ea816 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:44.583697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9296e897-def5-47a6-8a19-860c123ea816 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:44.664273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb84e908-5041-41c3-876a-986ed6bc2996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:44.665184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84e908-5041-41c3-876a-986ed6bc2996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:44.665470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84e908-5041-41c3-876a-986ed6bc2996 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:44.665948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb84e908-5041-41c3-876a-986ed6bc2996 admin admin] PATH_INFO: `/v3/users/7ad3cddbfb354764a781ea4eb37b518f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:44.667215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84e908-5041-41c3-876a-986ed6bc2996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ad3cddbfb354764a781ea4eb37b518f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:44.669240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb84e908-5041-41c3-876a-986ed6bc2996 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:44.957025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb84e908-5041-41c3-876a-986ed6bc2996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ad3cddbfb354764a781ea4eb37b518f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:44.957726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb84e908-5041-41c3-876a-986ed6bc2996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ad3cddbfb354764a781ea4eb37b518f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:44.966025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb84e908-5041-41c3-876a-986ed6bc2996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ad3cddbfb354764a781ea4eb37b518f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:45.092962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb84e908-5041-41c3-876a-986ed6bc2996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ad3cddbfb354764a781ea4eb37b518f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:45.096933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fb84e908-5041-41c3-876a-986ed6bc2996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ad3cddbfb354764a781ea4eb37b518f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:45.128610 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3356/6747] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:27:44 2022] DELETE /identity/v3/users/7ad3cddbfb354764a781ea4eb37b518f => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:27:45.913455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b9179-eb1c-442e-89f8-305af74e11e8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:45.915071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fb4b9179-eb1c-442e-89f8-305af74e11e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:45.930838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b9179-eb1c-442e-89f8-305af74e11e8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:46.067298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8955e01-2264-4117-9f9c-b15359d3b858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:46.068866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8955e01-2264-4117-9f9c-b15359d3b858 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:46.069341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8955e01-2264-4117-9f9c-b15359d3b858 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:46.071361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8955e01-2264-4117-9f9c-b15359d3b858 admin admin] PATH_INFO: `/v3/projects/f8bcdab945614ab5b0a5658102c17dc3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:46.073306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8955e01-2264-4117-9f9c-b15359d3b858 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8bcdab945614ab5b0a5658102c17dc3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:46.075935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8955e01-2264-4117-9f9c-b15359d3b858 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:46.244813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a8955e01-2264-4117-9f9c-b15359d3b858 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8bcdab945614ab5b0a5658102c17dc3', 'request_id': 'req-a8955e01-2264-4117-9f9c-b15359d3b858'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:46.262698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a8955e01-2264-4117-9f9c-b15359d3b858 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8bcdab945614ab5b0a5658102c17dc3', 'request_id': 'req-a8955e01-2264-4117-9f9c-b15359d3b858'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:46.365737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a8955e01-2264-4117-9f9c-b15359d3b858 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8bcdab945614ab5b0a5658102c17dc3', 'request_id': 'req-a8955e01-2264-4117-9f9c-b15359d3b858'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:46.387655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a8955e01-2264-4117-9f9c-b15359d3b858 admin admin] The token cache is being invalidate because project f8bcdab945614ab5b0a5658102c17dc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:27:46.390725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a8955e01-2264-4117-9f9c-b15359d3b858 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:46.392851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a8955e01-2264-4117-9f9c-b15359d3b858 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:46.394601 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3392/6748] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:27:45 2022] DELETE /identity/v3/projects/f8bcdab945614ab5b0a5658102c17dc3 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:46.400104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d06eca-6971-4289-acf1-2476ca425cbe None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:46.400654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d06eca-6971-4289-acf1-2476ca425cbe None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:46.400654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d06eca-6971-4289-acf1-2476ca425cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:46.442725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-08d06eca-6971-4289-acf1-2476ca425cbe None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:46.442725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-08d06eca-6971-4289-acf1-2476ca425cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:46.476150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:46.525583 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3357/6749] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:46.533682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc3918-59b9-419e-a010-84ac072a6a4e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:46.534020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc3918-59b9-419e-a010-84ac072a6a4e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:46.534348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc3918-59b9-419e-a010-84ac072a6a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:46.558384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3bfc3918-59b9-419e-a010-84ac072a6a4e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:46.560742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3bfc3918-59b9-419e-a010-84ac072a6a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:46.571759 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3393/6750] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:46.576311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d06eca-6971-4289-acf1-2476ca425cbe None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:46.586566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d06eca-6971-4289-acf1-2476ca425cbe None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:46.591428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61461652-c930-4d82-9160-b9723f7598e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:46.592100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61461652-c930-4d82-9160-b9723f7598e2 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:46.592392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61461652-c930-4d82-9160-b9723f7598e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:46.592676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61461652-c930-4d82-9160-b9723f7598e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:46.593452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61461652-c930-4d82-9160-b9723f7598e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:27:46.593763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61461652-c930-4d82-9160-b9723f7598e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:46.595997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61461652-c930-4d82-9160-b9723f7598e2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:46.613503 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3358/6751] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:27:46.614172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc3918-59b9-419e-a010-84ac072a6a4e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:46.625732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc3918-59b9-419e-a010-84ac072a6a4e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:46.631654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc1389-17d1-4adb-a5c5-5ee9037d8137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:46.632424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc1389-17d1-4adb-a5c5-5ee9037d8137 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:46.632714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc1389-17d1-4adb-a5c5-5ee9037d8137 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:46.635988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc1389-17d1-4adb-a5c5-5ee9037d8137 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:46.635988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc1389-17d1-4adb-a5c5-5ee9037d8137 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:46.635988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dc1389-17d1-4adb-a5c5-5ee9037d8137 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:46.812369 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3394/6752] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:27:46 2022] POST /identity/v3/projects => generated 381 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:46.813964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61461652-c930-4d82-9160-b9723f7598e2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:46.828199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61461652-c930-4d82-9160-b9723f7598e2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:46.915954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ee2c6-bbb4-41ec-8d33-28f6406f4244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:46.916655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ee2c6-bbb4-41ec-8d33-28f6406f4244 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:46.916937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ee2c6-bbb4-41ec-8d33-28f6406f4244 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:46.917455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6ee2c6-bbb4-41ec-8d33-28f6406f4244 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:46.918390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ee2c6-bbb4-41ec-8d33-28f6406f4244 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:46.920647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6ee2c6-bbb4-41ec-8d33-28f6406f4244 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:47.059302 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3359/6753] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:27:46 2022] POST /identity/v3/users => generated 291 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:47.059853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc1389-17d1-4adb-a5c5-5ee9037d8137 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:47.071012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dc1389-17d1-4adb-a5c5-5ee9037d8137 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:47.086802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2df525-5756-43de-9134-c925d5d0012e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:47.087594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2df525-5756-43de-9134-c925d5d0012e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:47.087947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2df525-5756-43de-9134-c925d5d0012e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:47.088596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2df525-5756-43de-9134-c925d5d0012e admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:47.090137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2df525-5756-43de-9134-c925d5d0012e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:47.091912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2df525-5756-43de-9134-c925d5d0012e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:47.105886 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3395/6754] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:27:47.109074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ee2c6-bbb4-41ec-8d33-28f6406f4244 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:47.131570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6ee2c6-bbb4-41ec-8d33-28f6406f4244 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:47.138706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf66ecc-9328-46f7-9ca6-d45a8bc1981e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:47.139751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf66ecc-9328-46f7-9ca6-d45a8bc1981e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:47.140738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf66ecc-9328-46f7-9ca6-d45a8bc1981e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:47.141463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf66ecc-9328-46f7-9ca6-d45a8bc1981e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:47.142646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf66ecc-9328-46f7-9ca6-d45a8bc1981e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:47.145579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf66ecc-9328-46f7-9ca6-d45a8bc1981e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:47.173346 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3360/6755] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:47.181091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2df525-5756-43de-9134-c925d5d0012e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:47.192718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2df525-5756-43de-9134-c925d5d0012e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:47.198944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1a4ab-935c-4eef-b38b-70269db1a4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:47.199921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1a4ab-935c-4eef-b38b-70269db1a4dd admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:47.200211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1a4ab-935c-4eef-b38b-70269db1a4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:47.200492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b1a4ab-935c-4eef-b38b-70269db1a4dd admin admin] PATH_INFO: `/v3/projects/7f6b1c0445ee4734bd092870f7ba1d4c/users/fbd2e63ff9c04ae38831b88193a574b2/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:47.223838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1a4ab-935c-4eef-b38b-70269db1a4dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f6b1c0445ee4734bd092870f7ba1d4c, user_id=fbd2e63ff9c04ae38831b88193a574b2, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:47.227127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b1a4ab-935c-4eef-b38b-70269db1a4dd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:47.315686 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3396/6756] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:47 2022] PUT /identity/v3/projects/7f6b1c0445ee4734bd092870f7ba1d4c/users/fbd2e63ff9c04ae38831b88193a574b2/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:47.329393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e21ce-a18f-44d8-bfd6-f7841c234ef7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:47.329687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e21ce-a18f-44d8-bfd6-f7841c234ef7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:47.330173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918e21ce-a18f-44d8-bfd6-f7841c234ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:47.390933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-918e21ce-a18f-44d8-bfd6-f7841c234ef7 None None] MFA Rules not processed for user `fbd2e63ff9c04ae38831b88193a574b2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:47.392234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-918e21ce-a18f-44d8-bfd6-f7841c234ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:47.533142 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3361/6757] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:47.533766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1a4ab-935c-4eef-b38b-70269db1a4dd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:47.560926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:47.648428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b1a4ab-935c-4eef-b38b-70269db1a4dd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:47.653475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6953da-ea31-45a5-9713-73a37da3da63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:47.654203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6953da-ea31-45a5-9713-73a37da3da63 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:47.654491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6953da-ea31-45a5-9713-73a37da3da63 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:47.654771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6953da-ea31-45a5-9713-73a37da3da63 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:47.655499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6953da-ea31-45a5-9713-73a37da3da63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:47.657457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6953da-ea31-45a5-9713-73a37da3da63 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:47.806455 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3397/6758] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:27:47 2022] POST /identity/v3/projects => generated 379 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:27:47.812261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e21ce-a18f-44d8-bfd6-f7841c234ef7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:47.832154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918e21ce-a18f-44d8-bfd6-f7841c234ef7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:47.934808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cffd40-cb25-4336-89ac-35755195ba5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:47.935718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cffd40-cb25-4336-89ac-35755195ba5c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:47.936180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cffd40-cb25-4336-89ac-35755195ba5c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:47.936633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cffd40-cb25-4336-89ac-35755195ba5c admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:47.937519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cffd40-cb25-4336-89ac-35755195ba5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:47.940497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cffd40-cb25-4336-89ac-35755195ba5c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:48.118139 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3362/6759] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:27:47 2022] POST /identity/v3/users => generated 296 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:48.121944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6953da-ea31-45a5-9713-73a37da3da63 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:48.143465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6953da-ea31-45a5-9713-73a37da3da63 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:48.149085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6f4f1-7fbc-474d-9e00-e4becfd606ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:48.150590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6f4f1-7fbc-474d-9e00-e4becfd606ac admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:48.151136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6f4f1-7fbc-474d-9e00-e4becfd606ac admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:48.151628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f6f4f1-7fbc-474d-9e00-e4becfd606ac admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:48.152587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f6f4f1-7fbc-474d-9e00-e4becfd606ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:48.155638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f6f4f1-7fbc-474d-9e00-e4becfd606ac admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:48.175160 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3398/6760] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:48.179874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cffd40-cb25-4336-89ac-35755195ba5c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:48.232047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cffd40-cb25-4336-89ac-35755195ba5c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:48.241607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0d152-b640-4567-872b-16af89f12232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:48.242595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0d152-b640-4567-872b-16af89f12232 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:48.242858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0d152-b640-4567-872b-16af89f12232 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:48.243182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0d152-b640-4567-872b-16af89f12232 admin admin] PATH_INFO: `/v3/projects/34da6c8d235543f1b559991425abaf25/users/695f6c2d6d774c6291f0dae701a508ad/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:48.267337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0d152-b640-4567-872b-16af89f12232 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34da6c8d235543f1b559991425abaf25, user_id=695f6c2d6d774c6291f0dae701a508ad, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:48.270314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0d152-b640-4567-872b-16af89f12232 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:48.353224 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3363/6761] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:48 2022] PUT /identity/v3/projects/34da6c8d235543f1b559991425abaf25/users/695f6c2d6d774c6291f0dae701a508ad/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:48.355045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6f4f1-7fbc-474d-9e00-e4becfd606ac admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:48.367007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f6f4f1-7fbc-474d-9e00-e4becfd606ac admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:48.405938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:48.488297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606b666-b556-4af1-bd5f-806f1b29ffda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:48.489720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606b666-b556-4af1-bd5f-806f1b29ffda admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:48.490612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606b666-b556-4af1-bd5f-806f1b29ffda admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:48.490902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606b666-b556-4af1-bd5f-806f1b29ffda admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:48.491852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606b666-b556-4af1-bd5f-806f1b29ffda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:48.493494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606b666-b556-4af1-bd5f-806f1b29ffda admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:48.505462 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3399/6762] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:27:48.510952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0d152-b640-4567-872b-16af89f12232 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:48.523988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0d152-b640-4567-872b-16af89f12232 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:48.529159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815d0e0-88a8-4695-b6d9-5f8b452232fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:48.535126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815d0e0-88a8-4695-b6d9-5f8b452232fb admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:48.545082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815d0e0-88a8-4695-b6d9-5f8b452232fb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:48.545082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815d0e0-88a8-4695-b6d9-5f8b452232fb admin admin] PATH_INFO: `/v3/projects/34da6c8d235543f1b559991425abaf25/users/695f6c2d6d774c6291f0dae701a508ad/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:48.545082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815d0e0-88a8-4695-b6d9-5f8b452232fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=34da6c8d235543f1b559991425abaf25, user_id=695f6c2d6d774c6291f0dae701a508ad, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:48.545082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815d0e0-88a8-4695-b6d9-5f8b452232fb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:48.555686 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3364/6763] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:48 2022] PUT /identity/v3/projects/34da6c8d235543f1b559991425abaf25/users/695f6c2d6d774c6291f0dae701a508ad/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:48.570675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae0ed1-4231-4870-94ea-f0352629b5b1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:48.572166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae0ed1-4231-4870-94ea-f0352629b5b1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:48.572166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aae0ed1-4231-4870-94ea-f0352629b5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:48.636862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0aae0ed1-4231-4870-94ea-f0352629b5b1 None None] MFA Rules not processed for user `695f6c2d6d774c6291f0dae701a508ad`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:48.637549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0aae0ed1-4231-4870-94ea-f0352629b5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:48.734303 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3400/6764] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:48.748484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815d0e0-88a8-4695-b6d9-5f8b452232fb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:48.751220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7815d0e0-88a8-4695-b6d9-5f8b452232fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:48.822181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815d0e0-88a8-4695-b6d9-5f8b452232fb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:48.826217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3e4d0-dfde-4745-8740-97491652bac9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:48.835313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3e4d0-dfde-4745-8740-97491652bac9 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:48.835629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3e4d0-dfde-4745-8740-97491652bac9 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:48.835918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3e4d0-dfde-4745-8740-97491652bac9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:48.852499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3e4d0-dfde-4745-8740-97491652bac9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:48.854442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3e4d0-dfde-4745-8740-97491652bac9 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:48.865367 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3365/6765] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:49.125320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae0ed1-4231-4870-94ea-f0352629b5b1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:49.145447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aae0ed1-4231-4870-94ea-f0352629b5b1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:49.152583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efef17b-263c-4615-940e-da846f7e1abd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:49.153635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efef17b-263c-4615-940e-da846f7e1abd service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:49.154126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efef17b-263c-4615-940e-da846f7e1abd service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:49.156793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efef17b-263c-4615-940e-da846f7e1abd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:49.285100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efef17b-263c-4615-940e-da846f7e1abd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:49.287427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efef17b-263c-4615-940e-da846f7e1abd service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:49.307794 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3401/6766] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:49.405593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:50.416736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3e4d0-dfde-4745-8740-97491652bac9 service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:50.419480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c3c3e4d0-dfde-4745-8740-97491652bac9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:50.501552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3e4d0-dfde-4745-8740-97491652bac9 service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:50.516769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e08cc-3ba2-4e00-b637-a9dfdda1705b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:50.516769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e08cc-3ba2-4e00-b637-a9dfdda1705b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:50.516769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e08cc-3ba2-4e00-b637-a9dfdda1705b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:50.516769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e08cc-3ba2-4e00-b637-a9dfdda1705b admin admin] PATH_INFO: `/v3/users/7cc1828624064f028d1060ee393d2da3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:50.518227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e08cc-3ba2-4e00-b637-a9dfdda1705b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cc1828624064f028d1060ee393d2da3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:50.520870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e08cc-3ba2-4e00-b637-a9dfdda1705b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:50.571829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:50.766150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-116e08cc-3ba2-4e00-b637-a9dfdda1705b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cc1828624064f028d1060ee393d2da3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:50.767621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-116e08cc-3ba2-4e00-b637-a9dfdda1705b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cc1828624064f028d1060ee393d2da3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:50.779431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-116e08cc-3ba2-4e00-b637-a9dfdda1705b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cc1828624064f028d1060ee393d2da3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:50.875893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-116e08cc-3ba2-4e00-b637-a9dfdda1705b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cc1828624064f028d1060ee393d2da3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:50.877165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-116e08cc-3ba2-4e00-b637-a9dfdda1705b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cc1828624064f028d1060ee393d2da3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:50.904770 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3366/6767] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:27:50 2022] DELETE /identity/v3/users/7cc1828624064f028d1060ee393d2da3 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:50.909260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab43455-f5d8-433c-be1d-9754d5b66565 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:50.909790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab43455-f5d8-433c-be1d-9754d5b66565 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:50.909933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab43455-f5d8-433c-be1d-9754d5b66565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:50.935805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2ab43455-f5d8-433c-be1d-9754d5b66565 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:50.937947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2ab43455-f5d8-433c-be1d-9754d5b66565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:51.058103 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3402/6768] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:51.104070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e08cc-3ba2-4e00-b637-a9dfdda1705b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:51.107488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-116e08cc-3ba2-4e00-b637-a9dfdda1705b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:51.160679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab43455-f5d8-433c-be1d-9754d5b66565 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:51.163816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2ab43455-f5d8-433c-be1d-9754d5b66565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:51.187864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab43455-f5d8-433c-be1d-9754d5b66565 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:51.200728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64324e55-26a4-412e-9284-0cbed30fe118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:51.201643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64324e55-26a4-412e-9284-0cbed30fe118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:51.202311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64324e55-26a4-412e-9284-0cbed30fe118 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:51.202588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64324e55-26a4-412e-9284-0cbed30fe118 admin admin] PATH_INFO: `/v3/users/ebc8e00965334bd1bb811a0eefd64b84` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:51.206403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64324e55-26a4-412e-9284-0cbed30fe118 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebc8e00965334bd1bb811a0eefd64b84)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:51.209352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64324e55-26a4-412e-9284-0cbed30fe118 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:51.275008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e08cc-3ba2-4e00-b637-a9dfdda1705b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:51.282949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6808632-a603-4a9f-b83f-df5c45dc5258 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:51.284359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6808632-a603-4a9f-b83f-df5c45dc5258 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:51.285067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6808632-a603-4a9f-b83f-df5c45dc5258 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:51.285067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6808632-a603-4a9f-b83f-df5c45dc5258 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:51.299327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6808632-a603-4a9f-b83f-df5c45dc5258 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:51.301085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6808632-a603-4a9f-b83f-df5c45dc5258 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:51.312903 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3367/6769] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:51.500193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-64324e55-26a4-412e-9284-0cbed30fe118 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebc8e00965334bd1bb811a0eefd64b84'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:51.511573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-64324e55-26a4-412e-9284-0cbed30fe118 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebc8e00965334bd1bb811a0eefd64b84'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:51.512663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-64324e55-26a4-412e-9284-0cbed30fe118 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebc8e00965334bd1bb811a0eefd64b84'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:51.513871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-64324e55-26a4-412e-9284-0cbed30fe118 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebc8e00965334bd1bb811a0eefd64b84'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:51.622565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-64324e55-26a4-412e-9284-0cbed30fe118 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebc8e00965334bd1bb811a0eefd64b84'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:51.681494 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3403/6770] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:27:51 2022] DELETE /identity/v3/users/ebc8e00965334bd1bb811a0eefd64b84 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:27:52.126679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6808632-a603-4a9f-b83f-df5c45dc5258 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:52.128176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a6808632-a603-4a9f-b83f-df5c45dc5258 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:52.166117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6808632-a603-4a9f-b83f-df5c45dc5258 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:52.175391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64324e55-26a4-412e-9284-0cbed30fe118 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:52.177586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-64324e55-26a4-412e-9284-0cbed30fe118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:52.235502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:52.296172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaa064-123c-4dd5-b8bd-722d013c7e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:52.298863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaa064-123c-4dd5-b8bd-722d013c7e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:52.300382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaa064-123c-4dd5-b8bd-722d013c7e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:52.300836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abaa064-123c-4dd5-b8bd-722d013c7e53 admin admin] PATH_INFO: `/v3/projects/9b13e9b82ca8467b8008de69cce86516` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:52.303722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abaa064-123c-4dd5-b8bd-722d013c7e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b13e9b82ca8467b8008de69cce86516)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:52.307878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abaa064-123c-4dd5-b8bd-722d013c7e53 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:52.323705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64324e55-26a4-412e-9284-0cbed30fe118 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:52.338811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad7879-b45e-4cbd-be07-e2d3148d3892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:52.339848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad7879-b45e-4cbd-be07-e2d3148d3892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:52.340250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad7879-b45e-4cbd-be07-e2d3148d3892 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:52.340788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad7879-b45e-4cbd-be07-e2d3148d3892 admin admin] PATH_INFO: `/v3/projects/01fa7893b58741ce84a316632efe7882` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:52.342484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad7879-b45e-4cbd-be07-e2d3148d3892 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01fa7893b58741ce84a316632efe7882)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:52.346672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad7879-b45e-4cbd-be07-e2d3148d3892 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:52.522744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a1ad7879-b45e-4cbd-be07-e2d3148d3892 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01fa7893b58741ce84a316632efe7882', 'request_id': 'req-a1ad7879-b45e-4cbd-be07-e2d3148d3892'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:52.523697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2abaa064-123c-4dd5-b8bd-722d013c7e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b13e9b82ca8467b8008de69cce86516', 'request_id': 'req-2abaa064-123c-4dd5-b8bd-722d013c7e53'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:52.531477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2abaa064-123c-4dd5-b8bd-722d013c7e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b13e9b82ca8467b8008de69cce86516', 'request_id': 'req-2abaa064-123c-4dd5-b8bd-722d013c7e53'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:52.537259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a1ad7879-b45e-4cbd-be07-e2d3148d3892 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01fa7893b58741ce84a316632efe7882', 'request_id': 'req-a1ad7879-b45e-4cbd-be07-e2d3148d3892'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:52.639990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2abaa064-123c-4dd5-b8bd-722d013c7e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b13e9b82ca8467b8008de69cce86516', 'request_id': 'req-2abaa064-123c-4dd5-b8bd-722d013c7e53'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:52.670464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2abaa064-123c-4dd5-b8bd-722d013c7e53 admin admin] The token cache is being invalidate because project 9b13e9b82ca8467b8008de69cce86516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:27:52.670768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2abaa064-123c-4dd5-b8bd-722d013c7e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:52.672221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2abaa064-123c-4dd5-b8bd-722d013c7e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:52.678370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:52.681180 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3368/6771] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:27:52 2022] DELETE /identity/v3/projects/9b13e9b82ca8467b8008de69cce86516 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:52.684972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaa064-123c-4dd5-b8bd-722d013c7e53 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:52.689662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2abaa064-123c-4dd5-b8bd-722d013c7e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:52.753815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a1ad7879-b45e-4cbd-be07-e2d3148d3892 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01fa7893b58741ce84a316632efe7882', 'request_id': 'req-a1ad7879-b45e-4cbd-be07-e2d3148d3892'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:52.772767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a1ad7879-b45e-4cbd-be07-e2d3148d3892 admin admin] The token cache is being invalidate because project 01fa7893b58741ce84a316632efe7882 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:27:52.773169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a1ad7879-b45e-4cbd-be07-e2d3148d3892 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:52.781459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a1ad7879-b45e-4cbd-be07-e2d3148d3892 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:52.781459 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3404/6772] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:27:52 2022] DELETE /identity/v3/projects/01fa7893b58741ce84a316632efe7882 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:27:52.786468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df822e-79f8-46ba-b5d3-0d1611b25a81 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:52.786925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df822e-79f8-46ba-b5d3-0d1611b25a81 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:52.787300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df822e-79f8-46ba-b5d3-0d1611b25a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:52.830682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-23df822e-79f8-46ba-b5d3-0d1611b25a81 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:52.831846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-23df822e-79f8-46ba-b5d3-0d1611b25a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:52.845825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abaa064-123c-4dd5-b8bd-722d013c7e53 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:52.931072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977cec7-b9f6-4db3-abe9-c8fa2b9a05a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:52.932346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977cec7-b9f6-4db3-abe9-c8fa2b9a05a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:52.932965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977cec7-b9f6-4db3-abe9-c8fa2b9a05a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:52.933252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977cec7-b9f6-4db3-abe9-c8fa2b9a05a2 admin admin] PATH_INFO: `/v3/users/39cae3a116be44429add7517136bfefb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:52.934500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977cec7-b9f6-4db3-abe9-c8fa2b9a05a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39cae3a116be44429add7517136bfefb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:52.935366 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3405/6773] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:52.936603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d977cec7-b9f6-4db3-abe9-c8fa2b9a05a2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:52.943264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a2f92-63e2-4254-a46d-1c688187186c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:52.943560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a2f92-63e2-4254-a46d-1c688187186c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:52.943840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a2f92-63e2-4254-a46d-1c688187186c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:52.962060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f48a2f92-63e2-4254-a46d-1c688187186c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:52.963298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f48a2f92-63e2-4254-a46d-1c688187186c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:52.982627 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3406/6774] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:27:52.986743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a2f92-63e2-4254-a46d-1c688187186c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:52.997856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a2f92-63e2-4254-a46d-1c688187186c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:53.004049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3fc0b-7a83-4781-881a-cec92fe4710b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:53.004049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3fc0b-7a83-4781-881a-cec92fe4710b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:53.004049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3fc0b-7a83-4781-881a-cec92fe4710b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:53.004049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3fc0b-7a83-4781-881a-cec92fe4710b admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:53.005089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3fc0b-7a83-4781-881a-cec92fe4710b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:27:53.005502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3fc0b-7a83-4781-881a-cec92fe4710b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:53.007260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3fc0b-7a83-4781-881a-cec92fe4710b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:53.019407 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3407/6775] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:27:53.025063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3fc0b-7a83-4781-881a-cec92fe4710b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:53.035084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3fc0b-7a83-4781-881a-cec92fe4710b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:53.041128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b9f41-bed3-484f-b182-50927f79e50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:53.041805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b9f41-bed3-484f-b182-50927f79e50f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:53.042746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b9f41-bed3-484f-b182-50927f79e50f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:53.042746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b9f41-bed3-484f-b182-50927f79e50f admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:53.043102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b9f41-bed3-484f-b182-50927f79e50f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:53.045031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b9f41-bed3-484f-b182-50927f79e50f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:53.188099 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3408/6776] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:27:53 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:27:53.189491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b9f41-bed3-484f-b182-50927f79e50f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:53.201716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d977cec7-b9f6-4db3-abe9-c8fa2b9a05a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39cae3a116be44429add7517136bfefb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:53.203162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d977cec7-b9f6-4db3-abe9-c8fa2b9a05a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39cae3a116be44429add7517136bfefb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:53.228824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d977cec7-b9f6-4db3-abe9-c8fa2b9a05a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39cae3a116be44429add7517136bfefb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:53.302551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b9f41-bed3-484f-b182-50927f79e50f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:53.308521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6ccd1-cf34-42ac-b93a-bc964b3fd39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:53.313875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6ccd1-cf34-42ac-b93a-bc964b3fd39b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:53.313875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6ccd1-cf34-42ac-b93a-bc964b3fd39b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:53.313875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6ccd1-cf34-42ac-b93a-bc964b3fd39b admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:53.313875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6ccd1-cf34-42ac-b93a-bc964b3fd39b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:53.315084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6ccd1-cf34-42ac-b93a-bc964b3fd39b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:53.336201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d977cec7-b9f6-4db3-abe9-c8fa2b9a05a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39cae3a116be44429add7517136bfefb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:53.338883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d977cec7-b9f6-4db3-abe9-c8fa2b9a05a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39cae3a116be44429add7517136bfefb'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:53.366262 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3369/6777] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:27:52 2022] DELETE /identity/v3/users/39cae3a116be44429add7517136bfefb => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:53.454458 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3409/6778] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:27:53 2022] POST /identity/v3/users => generated 289 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:27:53.459215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977cec7-b9f6-4db3-abe9-c8fa2b9a05a2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:53.461240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d977cec7-b9f6-4db3-abe9-c8fa2b9a05a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:53.579396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977cec7-b9f6-4db3-abe9-c8fa2b9a05a2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:53.584455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79296961-8092-44b0-a996-65e822d810cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:53.585330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79296961-8092-44b0-a996-65e822d810cb admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:53.585619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79296961-8092-44b0-a996-65e822d810cb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:53.585958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79296961-8092-44b0-a996-65e822d810cb admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:53.586710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79296961-8092-44b0-a996-65e822d810cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:53.589062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79296961-8092-44b0-a996-65e822d810cb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:53.608770 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3370/6779] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:53.608904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6ccd1-cf34-42ac-b93a-bc964b3fd39b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:53.626647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6ccd1-cf34-42ac-b93a-bc964b3fd39b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:53.634625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc69e2bd-d866-420c-bd7d-c8fa72ca5939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:53.635694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69e2bd-d866-420c-bd7d-c8fa72ca5939 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:53.635694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69e2bd-d866-420c-bd7d-c8fa72ca5939 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:53.643151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc69e2bd-d866-420c-bd7d-c8fa72ca5939 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:53.643151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69e2bd-d866-420c-bd7d-c8fa72ca5939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:53.643151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc69e2bd-d866-420c-bd7d-c8fa72ca5939 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:53.653982 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3410/6780] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:27:53.658218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79296961-8092-44b0-a996-65e822d810cb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:53.672908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79296961-8092-44b0-a996-65e822d810cb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:53.679344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f33dc-efa5-43a0-954f-3ee06a1d6016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:53.680986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f33dc-efa5-43a0-954f-3ee06a1d6016 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:53.681408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f33dc-efa5-43a0-954f-3ee06a1d6016 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:53.681922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f33dc-efa5-43a0-954f-3ee06a1d6016 admin admin] PATH_INFO: `/v3/projects/e3cc8c5f2fe44b87bf5c05eadb62936c/users/93f78df0f96c47ac97a200a4442db6da/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:53.711097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f33dc-efa5-43a0-954f-3ee06a1d6016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3cc8c5f2fe44b87bf5c05eadb62936c, user_id=93f78df0f96c47ac97a200a4442db6da, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:53.713997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f33dc-efa5-43a0-954f-3ee06a1d6016 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:53.829817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ff34c-3953-4eb9-8200-2b4829b3890e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:53.830200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ff34c-3953-4eb9-8200-2b4829b3890e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:53.830493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ff34c-3953-4eb9-8200-2b4829b3890e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:53.831897 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3371/6781] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:53 2022] PUT /identity/v3/projects/e3cc8c5f2fe44b87bf5c05eadb62936c/users/93f78df0f96c47ac97a200a4442db6da/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:53.862816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-184ff34c-3953-4eb9-8200-2b4829b3890e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:53.864009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-184ff34c-3953-4eb9-8200-2b4829b3890e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:53.947869 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3411/6782] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:53.960702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f33dc-efa5-43a0-954f-3ee06a1d6016 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:53.962578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4a9f33dc-efa5-43a0-954f-3ee06a1d6016 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:54.079759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f33dc-efa5-43a0-954f-3ee06a1d6016 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:54.085357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407e850-9eb3-447f-b3d4-2cbdf79310cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:54.086078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407e850-9eb3-447f-b3d4-2cbdf79310cc service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:54.086422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407e850-9eb3-447f-b3d4-2cbdf79310cc service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:54.088715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407e850-9eb3-447f-b3d4-2cbdf79310cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:54.098171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407e850-9eb3-447f-b3d4-2cbdf79310cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:54.100781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407e850-9eb3-447f-b3d4-2cbdf79310cc service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:54.117388 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3372/6783] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:54.405823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:54.888636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ff34c-3953-4eb9-8200-2b4829b3890e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:54.890555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-184ff34c-3953-4eb9-8200-2b4829b3890e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:54.907771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ff34c-3953-4eb9-8200-2b4829b3890e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:54.917475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd13701-80db-4973-9f4a-1ac0879637e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:54.918599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd13701-80db-4973-9f4a-1ac0879637e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:54.918894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd13701-80db-4973-9f4a-1ac0879637e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:54.919276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd13701-80db-4973-9f4a-1ac0879637e3 admin admin] PATH_INFO: `/v3/users/aa0d39e663594fb596b7be9def994e22` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:54.922889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407e850-9eb3-447f-b3d4-2cbdf79310cc service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:54.923427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd13701-80db-4973-9f4a-1ac0879637e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa0d39e663594fb596b7be9def994e22)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:54.925270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd13701-80db-4973-9f4a-1ac0879637e3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:54.925404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a407e850-9eb3-447f-b3d4-2cbdf79310cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:54.954533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407e850-9eb3-447f-b3d4-2cbdf79310cc service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:54.959980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe9225e-972d-487e-a417-4fb3dec64046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:54.961341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9225e-972d-487e-a417-4fb3dec64046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:54.961636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9225e-972d-487e-a417-4fb3dec64046 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:54.961968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe9225e-972d-487e-a417-4fb3dec64046 admin admin] PATH_INFO: `/v3/projects/95df1c4813ec4c6680dc0da790d9eaa2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:54.963334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9225e-972d-487e-a417-4fb3dec64046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95df1c4813ec4c6680dc0da790d9eaa2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:54.965303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe9225e-972d-487e-a417-4fb3dec64046 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:55.041380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:55.135567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3bd13701-80db-4973-9f4a-1ac0879637e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa0d39e663594fb596b7be9def994e22'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:55.135896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4fe9225e-972d-487e-a417-4fb3dec64046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95df1c4813ec4c6680dc0da790d9eaa2', 'request_id': 'req-4fe9225e-972d-487e-a417-4fb3dec64046'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:55.142359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4fe9225e-972d-487e-a417-4fb3dec64046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95df1c4813ec4c6680dc0da790d9eaa2', 'request_id': 'req-4fe9225e-972d-487e-a417-4fb3dec64046'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:55.149607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3bd13701-80db-4973-9f4a-1ac0879637e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa0d39e663594fb596b7be9def994e22'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:55.150395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3bd13701-80db-4973-9f4a-1ac0879637e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa0d39e663594fb596b7be9def994e22'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:55.151004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3bd13701-80db-4973-9f4a-1ac0879637e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa0d39e663594fb596b7be9def994e22'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:55.213367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4fe9225e-972d-487e-a417-4fb3dec64046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95df1c4813ec4c6680dc0da790d9eaa2', 'request_id': 'req-4fe9225e-972d-487e-a417-4fb3dec64046'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:55.231230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4fe9225e-972d-487e-a417-4fb3dec64046 admin admin] The token cache is being invalidate because project 95df1c4813ec4c6680dc0da790d9eaa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:27:55.231527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4fe9225e-972d-487e-a417-4fb3dec64046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:55.236336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4fe9225e-972d-487e-a417-4fb3dec64046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:55.244264 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3373/6784] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:27:54 2022] DELETE /identity/v3/projects/95df1c4813ec4c6680dc0da790d9eaa2 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:55.252419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e0c98-7e53-456f-939c-9012d363d987 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:55.252419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e0c98-7e53-456f-939c-9012d363d987 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:55.252419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e0c98-7e53-456f-939c-9012d363d987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:55.280657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-930e0c98-7e53-456f-939c-9012d363d987 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:55.283550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-930e0c98-7e53-456f-939c-9012d363d987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:55.302434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3bd13701-80db-4973-9f4a-1ac0879637e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa0d39e663594fb596b7be9def994e22'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:55.328300 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3412/6785] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:27:54 2022] DELETE /identity/v3/users/aa0d39e663594fb596b7be9def994e22 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:55.332423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8001ed-c375-4a55-abc9-329062eee4ef None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:55.332923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8001ed-c375-4a55-abc9-329062eee4ef None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:55.333371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8001ed-c375-4a55-abc9-329062eee4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:55.383306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fc8001ed-c375-4a55-abc9-329062eee4ef None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:55.388224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fc8001ed-c375-4a55-abc9-329062eee4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:55.444732 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3374/6786] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:27:55.549908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25c9dd-fba8-4ea1-aaf7-d73aacb92297 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:55.549908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25c9dd-fba8-4ea1-aaf7-d73aacb92297 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:55.549908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d25c9dd-fba8-4ea1-aaf7-d73aacb92297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:55.567632 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3413/6787] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:27:55.573448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7d25c9dd-fba8-4ea1-aaf7-d73aacb92297 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:55.573448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7d25c9dd-fba8-4ea1-aaf7-d73aacb92297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:55.583711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8001ed-c375-4a55-abc9-329062eee4ef None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:55.586605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fc8001ed-c375-4a55-abc9-329062eee4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:55.587306 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3375/6788] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:27:55.588215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25c9dd-fba8-4ea1-aaf7-d73aacb92297 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:55.598721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d25c9dd-fba8-4ea1-aaf7-d73aacb92297 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:55.604134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787b5d6-affd-4ada-a6b6-7b1dc48bf5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:55.604810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787b5d6-affd-4ada-a6b6-7b1dc48bf5b7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:55.605160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787b5d6-affd-4ada-a6b6-7b1dc48bf5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:55.605446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787b5d6-affd-4ada-a6b6-7b1dc48bf5b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:55.606246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787b5d6-affd-4ada-a6b6-7b1dc48bf5b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:27:55.606560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787b5d6-affd-4ada-a6b6-7b1dc48bf5b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:55.608839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3787b5d6-affd-4ada-a6b6-7b1dc48bf5b7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:55.625803 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3376/6789] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:27:55.628861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787b5d6-affd-4ada-a6b6-7b1dc48bf5b7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:55.643282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3787b5d6-affd-4ada-a6b6-7b1dc48bf5b7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:55.651318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10d21a-b8d3-4ae5-8579-d0e619efad98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:55.652950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10d21a-b8d3-4ae5-8579-d0e619efad98 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:55.653308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10d21a-b8d3-4ae5-8579-d0e619efad98 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:55.653600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da10d21a-b8d3-4ae5-8579-d0e619efad98 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:55.654439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10d21a-b8d3-4ae5-8579-d0e619efad98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:55.656520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da10d21a-b8d3-4ae5-8579-d0e619efad98 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:55.705242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8001ed-c375-4a55-abc9-329062eee4ef None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:55.710543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74b055-3f80-4a75-bdd9-c01540a3cc33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:55.711266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74b055-3f80-4a75-bdd9-c01540a3cc33 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:55.711560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74b055-3f80-4a75-bdd9-c01540a3cc33 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:55.711840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74b055-3f80-4a75-bdd9-c01540a3cc33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:55.724005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74b055-3f80-4a75-bdd9-c01540a3cc33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:55.725556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74b055-3f80-4a75-bdd9-c01540a3cc33 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:55.737967 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3414/6790] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:27:55.841532 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3377/6791] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:27:55 2022] POST /identity/v3/projects => generated 385 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:55.846712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74b055-3f80-4a75-bdd9-c01540a3cc33 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:55.860713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74b055-3f80-4a75-bdd9-c01540a3cc33 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:55.967375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f1bae-99c7-4503-bf76-42396b95c584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:55.968434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1bae-99c7-4503-bf76-42396b95c584 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:55.968912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1bae-99c7-4503-bf76-42396b95c584 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:55.969458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f1bae-99c7-4503-bf76-42396b95c584 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:55.971422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f1bae-99c7-4503-bf76-42396b95c584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:55.974000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f1bae-99c7-4503-bf76-42396b95c584 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:56.112354 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3415/6792] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:27:55 2022] POST /identity/v3/users => generated 293 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:56.115977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10d21a-b8d3-4ae5-8579-d0e619efad98 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:56.135024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da10d21a-b8d3-4ae5-8579-d0e619efad98 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:56.141228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bcff8-dd5c-42c4-bd3d-b18db873147d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:56.142093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bcff8-dd5c-42c4-bd3d-b18db873147d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:56.142380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bcff8-dd5c-42c4-bd3d-b18db873147d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:56.142702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bcff8-dd5c-42c4-bd3d-b18db873147d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:56.143612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bcff8-dd5c-42c4-bd3d-b18db873147d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:56.146410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bcff8-dd5c-42c4-bd3d-b18db873147d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:56.165020 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3378/6793] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:56.169791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f1bae-99c7-4503-bf76-42396b95c584 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:56.184046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f1bae-99c7-4503-bf76-42396b95c584 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:56.191130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5eb6d-89c4-40db-b5f3-1172de9d6b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:56.193181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5eb6d-89c4-40db-b5f3-1172de9d6b71 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:56.193750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5eb6d-89c4-40db-b5f3-1172de9d6b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:56.194395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5eb6d-89c4-40db-b5f3-1172de9d6b71 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:56.196319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5eb6d-89c4-40db-b5f3-1172de9d6b71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:56.198440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5eb6d-89c4-40db-b5f3-1172de9d6b71 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:56.217592 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3416/6794] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:56.218768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bcff8-dd5c-42c4-bd3d-b18db873147d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:56.238732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bcff8-dd5c-42c4-bd3d-b18db873147d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:56.245830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc42d27-810e-4251-b95f-45b0689dc94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:56.247589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc42d27-810e-4251-b95f-45b0689dc94f admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:56.248139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc42d27-810e-4251-b95f-45b0689dc94f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:56.248526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc42d27-810e-4251-b95f-45b0689dc94f admin admin] PATH_INFO: `/v3/projects/5c1e22aaafb24196bdc1c94804f3da74/users/13304de0ae094696885dda008095cbe9/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:56.268059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc42d27-810e-4251-b95f-45b0689dc94f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c1e22aaafb24196bdc1c94804f3da74, user_id=13304de0ae094696885dda008095cbe9, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:56.272904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc42d27-810e-4251-b95f-45b0689dc94f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:56.351160 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3379/6795] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:56 2022] PUT /identity/v3/projects/5c1e22aaafb24196bdc1c94804f3da74/users/13304de0ae094696885dda008095cbe9/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:56.364075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a427d2-bfa5-4a1e-9613-bfa291a692c7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:56.364432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a427d2-bfa5-4a1e-9613-bfa291a692c7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:56.365981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a427d2-bfa5-4a1e-9613-bfa291a692c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:56.406768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-80a427d2-bfa5-4a1e-9613-bfa291a692c7 None None] MFA Rules not processed for user `13304de0ae094696885dda008095cbe9`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:56.408008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-80a427d2-bfa5-4a1e-9613-bfa291a692c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:56.478627 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3417/6796] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:56.483104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc42d27-810e-4251-b95f-45b0689dc94f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:56.533978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a427d2-bfa5-4a1e-9613-bfa291a692c7 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:56.535555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-80a427d2-bfa5-4a1e-9613-bfa291a692c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:56.562109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc42d27-810e-4251-b95f-45b0689dc94f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:56.567196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c24b3-dd5f-4974-9db1-00b5d94f060d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:56.567661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c24b3-dd5f-4974-9db1-00b5d94f060d admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:56.567987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c24b3-dd5f-4974-9db1-00b5d94f060d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:56.568266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c24b3-dd5f-4974-9db1-00b5d94f060d admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:56.569171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c24b3-dd5f-4974-9db1-00b5d94f060d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:56.571368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c24b3-dd5f-4974-9db1-00b5d94f060d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:56.647176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a427d2-bfa5-4a1e-9613-bfa291a692c7 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:56.653887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69980a-ce99-41ca-b8d6-945ad1acb66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:56.654095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69980a-ce99-41ca-b8d6-945ad1acb66f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:56.654494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69980a-ce99-41ca-b8d6-945ad1acb66f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:56.654927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69980a-ce99-41ca-b8d6-945ad1acb66f admin admin] PATH_INFO: `/v3/projects/bcf76791bd3e47b68ca0381e604c3c53` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:56.657245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69980a-ce99-41ca-b8d6-945ad1acb66f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcf76791bd3e47b68ca0381e604c3c53)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:56.659342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69980a-ce99-41ca-b8d6-945ad1acb66f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:56.715456 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3380/6797] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:27:56 2022] POST /identity/v3/projects => generated 387 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:56.716657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c24b3-dd5f-4974-9db1-00b5d94f060d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:56.804623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c24b3-dd5f-4974-9db1-00b5d94f060d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:56.826328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9d69980a-ce99-41ca-b8d6-945ad1acb66f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcf76791bd3e47b68ca0381e604c3c53', 'request_id': 'req-9d69980a-ce99-41ca-b8d6-945ad1acb66f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:56.838364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9d69980a-ce99-41ca-b8d6-945ad1acb66f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcf76791bd3e47b68ca0381e604c3c53', 'request_id': 'req-9d69980a-ce99-41ca-b8d6-945ad1acb66f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:56.911600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6382059-863b-40b6-bfed-4040f471c04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:56.912852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6382059-863b-40b6-bfed-4040f471c04c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:56.913410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6382059-863b-40b6-bfed-4040f471c04c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:56.914026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6382059-863b-40b6-bfed-4040f471c04c admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:56.915110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6382059-863b-40b6-bfed-4040f471c04c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:56.917625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6382059-863b-40b6-bfed-4040f471c04c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:56.944075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9d69980a-ce99-41ca-b8d6-945ad1acb66f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcf76791bd3e47b68ca0381e604c3c53', 'request_id': 'req-9d69980a-ce99-41ca-b8d6-945ad1acb66f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:57.013148 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3381/6798] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:27:56 2022] POST /identity/v3/users => generated 300 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:57.014273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6382059-863b-40b6-bfed-4040f471c04c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:57.027393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9d69980a-ce99-41ca-b8d6-945ad1acb66f admin admin] The token cache is being invalidate because project bcf76791bd3e47b68ca0381e604c3c53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:27:57.034481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9d69980a-ce99-41ca-b8d6-945ad1acb66f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:57.034615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6382059-863b-40b6-bfed-4040f471c04c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:57.034737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9d69980a-ce99-41ca-b8d6-945ad1acb66f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:57.040269 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3418/6799] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:27:56 2022] DELETE /identity/v3/projects/bcf76791bd3e47b68ca0381e604c3c53 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:27:57.046706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a378-626f-4f46-a02c-efefc961e65f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:57.047275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a378-626f-4f46-a02c-efefc961e65f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:57.049041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655a378-626f-4f46-a02c-efefc961e65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:57.051236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a73c5e-4ab3-4762-b65d-45eef135e434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:57.053820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a73c5e-4ab3-4762-b65d-45eef135e434 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:57.054523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a73c5e-4ab3-4762-b65d-45eef135e434 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:57.054933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a73c5e-4ab3-4762-b65d-45eef135e434 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:57.055969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a73c5e-4ab3-4762-b65d-45eef135e434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:57.058993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a73c5e-4ab3-4762-b65d-45eef135e434 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:57.102072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1655a378-626f-4f46-a02c-efefc961e65f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:57.103481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1655a378-626f-4f46-a02c-efefc961e65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:57.115086 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3382/6800] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:57.119549 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3419/6801] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:27:57.130964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a73c5e-4ab3-4762-b65d-45eef135e434 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:57.133496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741019c-0122-448b-8ff2-a99fd5aad9dd None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:57.133809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741019c-0122-448b-8ff2-a99fd5aad9dd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:57.134252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1741019c-0122-448b-8ff2-a99fd5aad9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:57.142571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-39a73c5e-4ab3-4762-b65d-45eef135e434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:57.165581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1741019c-0122-448b-8ff2-a99fd5aad9dd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:57.167746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1741019c-0122-448b-8ff2-a99fd5aad9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:57.169808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a73c5e-4ab3-4762-b65d-45eef135e434 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:57.175540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18cfc8-faf6-4f9a-9760-f82f7c8bf1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:57.176534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18cfc8-faf6-4f9a-9760-f82f7c8bf1ac admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:57.176893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18cfc8-faf6-4f9a-9760-f82f7c8bf1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:57.177274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18cfc8-faf6-4f9a-9760-f82f7c8bf1ac admin admin] PATH_INFO: `/v3/projects/646ded7e066240d19e2c359696157e71/users/b09e61e059b1410885f8adebf12650b1/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:57.186780 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3420/6802] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:57.190891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741019c-0122-448b-8ff2-a99fd5aad9dd None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:57.198658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18cfc8-faf6-4f9a-9760-f82f7c8bf1ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=646ded7e066240d19e2c359696157e71, user_id=b09e61e059b1410885f8adebf12650b1, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:57.202556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18cfc8-faf6-4f9a-9760-f82f7c8bf1ac admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:57.205323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1741019c-0122-448b-8ff2-a99fd5aad9dd None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:57.211976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce5c9b-c8d7-4d1a-a8fb-693679ec95d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:57.212711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce5c9b-c8d7-4d1a-a8fb-693679ec95d2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:57.213001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce5c9b-c8d7-4d1a-a8fb-693679ec95d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:57.213279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce5c9b-c8d7-4d1a-a8fb-693679ec95d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:57.214065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce5c9b-c8d7-4d1a-a8fb-693679ec95d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:27:57.214897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce5c9b-c8d7-4d1a-a8fb-693679ec95d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:57.216992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce5c9b-c8d7-4d1a-a8fb-693679ec95d2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:57.233084 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3421/6803] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:27:57.237976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce5c9b-c8d7-4d1a-a8fb-693679ec95d2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:57.255082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce5c9b-c8d7-4d1a-a8fb-693679ec95d2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:57.260797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e88ed-394e-4843-89d9-4303d96fbdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:57.262170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e88ed-394e-4843-89d9-4303d96fbdca admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:57.262637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e88ed-394e-4843-89d9-4303d96fbdca admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:57.263140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217e88ed-394e-4843-89d9-4303d96fbdca admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:57.264241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e88ed-394e-4843-89d9-4303d96fbdca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:57.266649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217e88ed-394e-4843-89d9-4303d96fbdca admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:57.327171 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3383/6804] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:57 2022] PUT /identity/v3/projects/646ded7e066240d19e2c359696157e71/users/b09e61e059b1410885f8adebf12650b1/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:57.340745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18cfc8-faf6-4f9a-9760-f82f7c8bf1ac admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:57.410054 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3422/6805] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:27:57 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:57.414145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e88ed-394e-4843-89d9-4303d96fbdca admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:57.487694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18cfc8-faf6-4f9a-9760-f82f7c8bf1ac admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:57.547441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217e88ed-394e-4843-89d9-4303d96fbdca admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:57.553286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad9026c-7a42-4f21-ba34-ccd06d8c66ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:57.554248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9026c-7a42-4f21-ba34-ccd06d8c66ef admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:57.554644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9026c-7a42-4f21-ba34-ccd06d8c66ef admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:57.555033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad9026c-7a42-4f21-ba34-ccd06d8c66ef admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:57.555848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9026c-7a42-4f21-ba34-ccd06d8c66ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:57.558614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad9026c-7a42-4f21-ba34-ccd06d8c66ef admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:57.599682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf1d08-3071-451c-b8d2-0e45a0cdd991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:57.600480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf1d08-3071-451c-b8d2-0e45a0cdd991 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:57.600769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf1d08-3071-451c-b8d2-0e45a0cdd991 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:57.601074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf1d08-3071-451c-b8d2-0e45a0cdd991 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:57.601807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccf1d08-3071-451c-b8d2-0e45a0cdd991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:57.603506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccf1d08-3071-451c-b8d2-0e45a0cdd991 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:57.613633 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3384/6806] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:57.618810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf1d08-3071-451c-b8d2-0e45a0cdd991 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:57.628878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf1d08-3071-451c-b8d2-0e45a0cdd991 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:57.635781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ef784-05f7-463b-a76a-3cb03e63a3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:57.636681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ef784-05f7-463b-a76a-3cb03e63a3c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:57.639285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ef784-05f7-463b-a76a-3cb03e63a3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:57.639285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ef784-05f7-463b-a76a-3cb03e63a3c8 admin admin] PATH_INFO: `/v3/projects/646ded7e066240d19e2c359696157e71/users/b09e61e059b1410885f8adebf12650b1/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:57.639285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ef784-05f7-463b-a76a-3cb03e63a3c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=646ded7e066240d19e2c359696157e71, user_id=b09e61e059b1410885f8adebf12650b1, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:57.641960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38ef784-05f7-463b-a76a-3cb03e63a3c8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:57.655901 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3385/6807] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:57 2022] PUT /identity/v3/projects/646ded7e066240d19e2c359696157e71/users/b09e61e059b1410885f8adebf12650b1/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:57.661179 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3423/6808] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:27:57 2022] POST /identity/v3/users => generated 292 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:27:57.665640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ef784-05f7-463b-a76a-3cb03e63a3c8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:57.673323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61ea5d-4ba4-4456-ae7d-1337a777ffe4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:57.673632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61ea5d-4ba4-4456-ae7d-1337a777ffe4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:57.673958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61ea5d-4ba4-4456-ae7d-1337a777ffe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:57.711773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ec61ea5d-4ba4-4456-ae7d-1337a777ffe4 None None] MFA Rules not processed for user `b09e61e059b1410885f8adebf12650b1`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:57.711773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ec61ea5d-4ba4-4456-ae7d-1337a777ffe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:57.748803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:57.774214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38ef784-05f7-463b-a76a-3cb03e63a3c8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:57.780118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9865b-b4f4-477b-9694-6556bd8e4b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:57.780807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9865b-b4f4-477b-9694-6556bd8e4b06 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:57.781099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9865b-b4f4-477b-9694-6556bd8e4b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:57.781386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc9865b-b4f4-477b-9694-6556bd8e4b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:57.782184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9865b-b4f4-477b-9694-6556bd8e4b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:57.784203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc9865b-b4f4-477b-9694-6556bd8e4b06 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:57.798866 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3386/6809] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:57.810136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9865b-b4f4-477b-9694-6556bd8e4b06 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:57.824747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc9865b-b4f4-477b-9694-6556bd8e4b06 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:57.831143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce25b0-30df-471e-aeb4-3faed89f57d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:57.832187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce25b0-30df-471e-aeb4-3faed89f57d6 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:57.832320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce25b0-30df-471e-aeb4-3faed89f57d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:57.832586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce25b0-30df-471e-aeb4-3faed89f57d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:57.833580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce25b0-30df-471e-aeb4-3faed89f57d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:57.835388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce25b0-30df-471e-aeb4-3faed89f57d6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:57.847293 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3424/6810] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:57.857512 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3387/6811] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:57.864316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce25b0-30df-471e-aeb4-3faed89f57d6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:57.864492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61ea5d-4ba4-4456-ae7d-1337a777ffe4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:57.866908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-76ce25b0-30df-471e-aeb4-3faed89f57d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:57.882260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61ea5d-4ba4-4456-ae7d-1337a777ffe4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:57.889298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf7aa6-4f09-4e67-a858-19e127036c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:57.894433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf7aa6-4f09-4e67-a858-19e127036c8d admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:57.894433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf7aa6-4f09-4e67-a858-19e127036c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:57.894433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf7aa6-4f09-4e67-a858-19e127036c8d admin admin] PATH_INFO: `/v3/projects/58db866e3b13484abe9de5c091d9aa5f/users/b85ebcba8aa84b5c9634141b2f6155b2/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:57.931534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf7aa6-4f09-4e67-a858-19e127036c8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=58db866e3b13484abe9de5c091d9aa5f, user_id=b85ebcba8aa84b5c9634141b2f6155b2, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:57.934278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf7aa6-4f09-4e67-a858-19e127036c8d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:57.993784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce25b0-30df-471e-aeb4-3faed89f57d6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:57.998154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3644da-21c2-4e14-add8-b3fd438642e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:57.999153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3644da-21c2-4e14-add8-b3fd438642e7 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:57.999261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3644da-21c2-4e14-add8-b3fd438642e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:57.999538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3644da-21c2-4e14-add8-b3fd438642e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:58.011835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3644da-21c2-4e14-add8-b3fd438642e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:58.013511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3644da-21c2-4e14-add8-b3fd438642e7 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:58.025659 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3388/6812] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:58.054085 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3425/6813] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:57 2022] PUT /identity/v3/projects/58db866e3b13484abe9de5c091d9aa5f/users/b85ebcba8aa84b5c9634141b2f6155b2/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:58.068537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f348e3-5492-4490-9281-70f16e612254 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:58.068537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f348e3-5492-4490-9281-70f16e612254 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:58.068537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f348e3-5492-4490-9281-70f16e612254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:58.157075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-19f348e3-5492-4490-9281-70f16e612254 None None] MFA Rules not processed for user `b85ebcba8aa84b5c9634141b2f6155b2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:58.161377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-19f348e3-5492-4490-9281-70f16e612254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:58.163831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf7aa6-4f09-4e67-a858-19e127036c8d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:58.280815 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3389/6814] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:58.285375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f348e3-5492-4490-9281-70f16e612254 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:58.298644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf7aa6-4f09-4e67-a858-19e127036c8d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:58.304253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f348e3-5492-4490-9281-70f16e612254 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:58.310721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660a0c5-d5da-4915-b0d4-6b231800d633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:58.311703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660a0c5-d5da-4915-b0d4-6b231800d633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:58.312117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660a0c5-d5da-4915-b0d4-6b231800d633 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:58.312449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe91768-7869-4e4a-b258-a5d1b0cacb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:58.313022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660a0c5-d5da-4915-b0d4-6b231800d633 admin admin] PATH_INFO: `/v3/users/13304de0ae094696885dda008095cbe9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:58.314293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe91768-7869-4e4a-b258-a5d1b0cacb12 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:58.314731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660a0c5-d5da-4915-b0d4-6b231800d633 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13304de0ae094696885dda008095cbe9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:58.315159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe91768-7869-4e4a-b258-a5d1b0cacb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:58.316579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:58.317244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe91768-7869-4e4a-b258-a5d1b0cacb12 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:58.318245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe91768-7869-4e4a-b258-a5d1b0cacb12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:58.318816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660a0c5-d5da-4915-b0d4-6b231800d633 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:58.320715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe91768-7869-4e4a-b258-a5d1b0cacb12 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:58.417456 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3390/6815] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:27:58 2022] POST /identity/v3/projects => generated 387 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:58.418421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe91768-7869-4e4a-b258-a5d1b0cacb12 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:58.569566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe91768-7869-4e4a-b258-a5d1b0cacb12 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:58.597592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e132f-53c6-4dba-9fca-7b1bb9131118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:58.600102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e132f-53c6-4dba-9fca-7b1bb9131118 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:58.600650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e132f-53c6-4dba-9fca-7b1bb9131118 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:58.601544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e132f-53c6-4dba-9fca-7b1bb9131118 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:58.601992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6660a0c5-d5da-4915-b0d4-6b231800d633 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13304de0ae094696885dda008095cbe9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:58.602324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e132f-53c6-4dba-9fca-7b1bb9131118 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:58.614309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e132f-53c6-4dba-9fca-7b1bb9131118 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:58.633228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6660a0c5-d5da-4915-b0d4-6b231800d633 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13304de0ae094696885dda008095cbe9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:58.641814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6660a0c5-d5da-4915-b0d4-6b231800d633 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13304de0ae094696885dda008095cbe9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:58.643145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6660a0c5-d5da-4915-b0d4-6b231800d633 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13304de0ae094696885dda008095cbe9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:58.784610 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3391/6816] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:27:58 2022] POST /identity/v3/users => generated 301 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:27:58.790558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e132f-53c6-4dba-9fca-7b1bb9131118 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:58.792319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ac4e132f-53c6-4dba-9fca-7b1bb9131118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:58.806399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6660a0c5-d5da-4915-b0d4-6b231800d633 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13304de0ae094696885dda008095cbe9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:27:58.808037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e132f-53c6-4dba-9fca-7b1bb9131118 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:58.813633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138acac7-1215-4544-b726-e0713c821576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:58.814388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138acac7-1215-4544-b726-e0713c821576 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:58.814554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138acac7-1215-4544-b726-e0713c821576 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:58.814850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138acac7-1215-4544-b726-e0713c821576 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:58.815584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138acac7-1215-4544-b726-e0713c821576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:58.817328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138acac7-1215-4544-b726-e0713c821576 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:58.829738 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3392/6817] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:27:58.835029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138acac7-1215-4544-b726-e0713c821576 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:58.851185 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3426/6818] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:27:58 2022] DELETE /identity/v3/users/13304de0ae094696885dda008095cbe9 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:27:58.852856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515c0e5-f740-4634-8eac-3b2de60e23b2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:58.853628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515c0e5-f740-4634-8eac-3b2de60e23b2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:58.855518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138acac7-1215-4544-b726-e0713c821576 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:58.856481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9515c0e5-f740-4634-8eac-3b2de60e23b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:58.924109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9515c0e5-f740-4634-8eac-3b2de60e23b2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:58.929190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9515c0e5-f740-4634-8eac-3b2de60e23b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:59.043590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe923574-1d22-4092-a3b7-af977b3feb46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:59.044519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe923574-1d22-4092-a3b7-af977b3feb46 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:59.046017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe923574-1d22-4092-a3b7-af977b3feb46 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:59.046728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe923574-1d22-4092-a3b7-af977b3feb46 admin admin] PATH_INFO: `/v3/projects/43cc11d55355438a9100deeb5e77d877/users/b906b2f0012148d48d36be0d057d816a/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:59.073365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe923574-1d22-4092-a3b7-af977b3feb46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43cc11d55355438a9100deeb5e77d877, user_id=b906b2f0012148d48d36be0d057d816a, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:59.076518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe923574-1d22-4092-a3b7-af977b3feb46 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:59.093582 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3427/6819] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:59.112628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515c0e5-f740-4634-8eac-3b2de60e23b2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:59.115024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9515c0e5-f740-4634-8eac-3b2de60e23b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:59.195739 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3393/6820] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:27:58 2022] PUT /identity/v3/projects/43cc11d55355438a9100deeb5e77d877/users/b906b2f0012148d48d36be0d057d816a/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:27:59.198138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c897dd-5334-4e83-b151-b392f8001f2a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:59.198673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c897dd-5334-4e83-b151-b392f8001f2a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:59.199118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c897dd-5334-4e83-b151-b392f8001f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:59.236815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-51c897dd-5334-4e83-b151-b392f8001f2a None None] MFA Rules not processed for user `b906b2f0012148d48d36be0d057d816a`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:27:59.239027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-51c897dd-5334-4e83-b151-b392f8001f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:59.263625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9515c0e5-f740-4634-8eac-3b2de60e23b2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:59.341933 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3394/6821] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:27:59 2022] POST /identity/v3/auth/tokens => generated 3261 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:27:59.385743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c897dd-5334-4e83-b151-b392f8001f2a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:27:59.392700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-51c897dd-5334-4e83-b151-b392f8001f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:27:59.393374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ee0fa-213d-4336-81f8-74fa17aa6e2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:59.394198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ee0fa-213d-4336-81f8-74fa17aa6e2a service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:59.394494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ee0fa-213d-4336-81f8-74fa17aa6e2a service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:59.395503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ee0fa-213d-4336-81f8-74fa17aa6e2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:59.494407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ee0fa-213d-4336-81f8-74fa17aa6e2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:59.499919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ee0fa-213d-4336-81f8-74fa17aa6e2a service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:59.514771 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3428/6822] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:27:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:27:59.528784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c897dd-5334-4e83-b151-b392f8001f2a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:27:59.534871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e922b10-75ed-4376-96ab-91bedcdec6b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:27:59.535563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e922b10-75ed-4376-96ab-91bedcdec6b3 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:27:59.536352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e922b10-75ed-4376-96ab-91bedcdec6b3 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:27:59.536352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e922b10-75ed-4376-96ab-91bedcdec6b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:27:59.546528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e922b10-75ed-4376-96ab-91bedcdec6b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:27:59.548367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e922b10-75ed-4376-96ab-91bedcdec6b3 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:27:59.559561 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3395/6823] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:27:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 680 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:28:00.387374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:00.760480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ee0fa-213d-4336-81f8-74fa17aa6e2a service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:00.762357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8a4ee0fa-213d-4336-81f8-74fa17aa6e2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:00.785668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ee0fa-213d-4336-81f8-74fa17aa6e2a service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:00.806596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a47c2-a445-4e07-bc6c-bfbb7448d873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:00.807427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a47c2-a445-4e07-bc6c-bfbb7448d873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:00.807981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a47c2-a445-4e07-bc6c-bfbb7448d873 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:00.809127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a47c2-a445-4e07-bc6c-bfbb7448d873 admin admin] PATH_INFO: `/v3/projects/5c1e22aaafb24196bdc1c94804f3da74` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:00.812533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a47c2-a445-4e07-bc6c-bfbb7448d873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c1e22aaafb24196bdc1c94804f3da74)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:00.819208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a47c2-a445-4e07-bc6c-bfbb7448d873 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:01.032667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-839a47c2-a445-4e07-bc6c-bfbb7448d873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c1e22aaafb24196bdc1c94804f3da74', 'request_id': 'req-839a47c2-a445-4e07-bc6c-bfbb7448d873'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:01.078809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-839a47c2-a445-4e07-bc6c-bfbb7448d873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c1e22aaafb24196bdc1c94804f3da74', 'request_id': 'req-839a47c2-a445-4e07-bc6c-bfbb7448d873'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:01.176617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-839a47c2-a445-4e07-bc6c-bfbb7448d873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c1e22aaafb24196bdc1c94804f3da74', 'request_id': 'req-839a47c2-a445-4e07-bc6c-bfbb7448d873'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:01.186957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e922b10-75ed-4376-96ab-91bedcdec6b3 service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:01.205285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e922b10-75ed-4376-96ab-91bedcdec6b3 service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:01.206968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-839a47c2-a445-4e07-bc6c-bfbb7448d873 admin admin] The token cache is being invalidate because project 5c1e22aaafb24196bdc1c94804f3da74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:01.208427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-839a47c2-a445-4e07-bc6c-bfbb7448d873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:01.213059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-839a47c2-a445-4e07-bc6c-bfbb7448d873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:01.218732 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3429/6824] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:00 2022] DELETE /identity/v3/projects/5c1e22aaafb24196bdc1c94804f3da74 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:01.222405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a47c2-a445-4e07-bc6c-bfbb7448d873 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:01.227127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-839a47c2-a445-4e07-bc6c-bfbb7448d873 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:01.283200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d0658-2cbe-46b8-950f-34c4914d799f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:01.284010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d0658-2cbe-46b8-950f-34c4914d799f service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:01.284393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d0658-2cbe-46b8-950f-34c4914d799f service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:01.285364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d0658-2cbe-46b8-950f-34c4914d799f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:01.287553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-507d0658-2cbe-46b8-950f-34c4914d799f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:01.301554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a47c2-a445-4e07-bc6c-bfbb7448d873 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:01.312810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c2236-2cca-4d68-a0ca-c6020ca2697e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:01.314062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c2236-2cca-4d68-a0ca-c6020ca2697e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:01.314700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c2236-2cca-4d68-a0ca-c6020ca2697e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:01.316657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c2236-2cca-4d68-a0ca-c6020ca2697e admin admin] PATH_INFO: `/v3/users/b09e61e059b1410885f8adebf12650b1` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:01.319113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c2236-2cca-4d68-a0ca-c6020ca2697e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b09e61e059b1410885f8adebf12650b1)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:01.322892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c2236-2cca-4d68-a0ca-c6020ca2697e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:01.367355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:01.393643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d0658-2cbe-46b8-950f-34c4914d799f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:01.395744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d0658-2cbe-46b8-950f-34c4914d799f service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:01.416624 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3396/6825] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:28:01.522951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-764c2236-2cca-4d68-a0ca-c6020ca2697e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b09e61e059b1410885f8adebf12650b1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:01.551485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-764c2236-2cca-4d68-a0ca-c6020ca2697e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b09e61e059b1410885f8adebf12650b1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:01.564691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-764c2236-2cca-4d68-a0ca-c6020ca2697e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b09e61e059b1410885f8adebf12650b1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:01.565880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-764c2236-2cca-4d68-a0ca-c6020ca2697e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b09e61e059b1410885f8adebf12650b1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:01.640455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-764c2236-2cca-4d68-a0ca-c6020ca2697e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b09e61e059b1410885f8adebf12650b1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:01.686299 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3430/6826] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:01 2022] DELETE /identity/v3/users/b09e61e059b1410885f8adebf12650b1 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:02.514542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:03.158311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9d6a6-bace-44c8-937c-7520bc934119 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:03.158311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9d6a6-bace-44c8-937c-7520bc934119 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:03.158311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc9d6a6-bace-44c8-937c-7520bc934119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:03.196010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-cdc9d6a6-bace-44c8-937c-7520bc934119 None None] MFA Rules not processed for user `02d5fbb072814fd7923ecf12c7ecad7b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:03.198108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cdc9d6a6-bace-44c8-937c-7520bc934119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:03.229371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c2236-2cca-4d68-a0ca-c6020ca2697e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:03.231616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-764c2236-2cca-4d68-a0ca-c6020ca2697e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:03.311258 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3397/6827] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:03.343096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c2236-2cca-4d68-a0ca-c6020ca2697e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:03.367507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9d6a6-bace-44c8-937c-7520bc934119 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:03.369262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cdc9d6a6-bace-44c8-937c-7520bc934119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:03.374910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58db625-fe83-4c88-9d00-67dac51c3f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:03.376172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58db625-fe83-4c88-9d00-67dac51c3f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:03.378313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58db625-fe83-4c88-9d00-67dac51c3f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:03.378803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58db625-fe83-4c88-9d00-67dac51c3f90 admin admin] PATH_INFO: `/v3/users/b85ebcba8aa84b5c9634141b2f6155b2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:03.387164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58db625-fe83-4c88-9d00-67dac51c3f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b85ebcba8aa84b5c9634141b2f6155b2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:03.391543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58db625-fe83-4c88-9d00-67dac51c3f90 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:03.423096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc9d6a6-bace-44c8-937c-7520bc934119 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:03.448677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:03.451784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:03.454048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:03.454546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4 admin admin] PATH_INFO: `/v3/projects/646ded7e066240d19e2c359696157e71` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:03.459546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=646ded7e066240d19e2c359696157e71)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:03.464108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:03.473127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:03.633375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '646ded7e066240d19e2c359696157e71', 'request_id': 'req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:03.642477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '646ded7e066240d19e2c359696157e71', 'request_id': 'req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:03.658329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b58db625-fe83-4c88-9d00-67dac51c3f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b85ebcba8aa84b5c9634141b2f6155b2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:03.677135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b58db625-fe83-4c88-9d00-67dac51c3f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b85ebcba8aa84b5c9634141b2f6155b2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:03.680478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b58db625-fe83-4c88-9d00-67dac51c3f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b85ebcba8aa84b5c9634141b2f6155b2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:03.681453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b58db625-fe83-4c88-9d00-67dac51c3f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b85ebcba8aa84b5c9634141b2f6155b2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:03.737057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '646ded7e066240d19e2c359696157e71', 'request_id': 'req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:03.751543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4 admin admin] The token cache is being invalidate because project 646ded7e066240d19e2c359696157e71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:03.751900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:03.752809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:03.754758 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3398/6828] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:03 2022] DELETE /identity/v3/projects/646ded7e066240d19e2c359696157e71 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:28:03.755421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:03.756764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:03.788069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b58db625-fe83-4c88-9d00-67dac51c3f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b85ebcba8aa84b5c9634141b2f6155b2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:03.829895 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3431/6829] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:03 2022] DELETE /identity/v3/users/b85ebcba8aa84b5c9634141b2f6155b2 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:03.842683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96976d6f-b630-406d-934d-ab6b0bcabbad None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:03.848994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96976d6f-b630-406d-934d-ab6b0bcabbad None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:03.852412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96976d6f-b630-406d-934d-ab6b0bcabbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:03.888433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447e88d-a2e7-41eb-8c39-4feb09e37ba4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:03.891861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-96976d6f-b630-406d-934d-ab6b0bcabbad None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:03.893499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-96976d6f-b630-406d-934d-ab6b0bcabbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:03.976881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec57b4-f1d5-4201-b3fa-448ff0f64689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:03.977825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec57b4-f1d5-4201-b3fa-448ff0f64689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:03.978247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec57b4-f1d5-4201-b3fa-448ff0f64689 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:03.978607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec57b4-f1d5-4201-b3fa-448ff0f64689 admin admin] PATH_INFO: `/v3/users/0f00ea0259c34a468d66d894e10a3d05` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:03.986789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec57b4-f1d5-4201-b3fa-448ff0f64689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f00ea0259c34a468d66d894e10a3d05)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:03.989003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec57b4-f1d5-4201-b3fa-448ff0f64689 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:04.008817 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3432/6830] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:04.010762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc867210-f63e-4200-84ec-63989286b92f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:04.011149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc867210-f63e-4200-84ec-63989286b92f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:04.011497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc867210-f63e-4200-84ec-63989286b92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:04.042839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-bc867210-f63e-4200-84ec-63989286b92f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:04.044537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bc867210-f63e-4200-84ec-63989286b92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:04.053727 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3433/6831] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:04.055252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d646d-c5da-46d7-b1b0-69b3e2bcdd85 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:04.055611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d646d-c5da-46d7-b1b0-69b3e2bcdd85 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:04.056093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d646d-c5da-46d7-b1b0-69b3e2bcdd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:04.085436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-065d646d-c5da-46d7-b1b0-69b3e2bcdd85 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:04.089723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-065d646d-c5da-46d7-b1b0-69b3e2bcdd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:04.102814 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3434/6832] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:04.102814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d646d-c5da-46d7-b1b0-69b3e2bcdd85 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:04.104762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-065d646d-c5da-46d7-b1b0-69b3e2bcdd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:04.179354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d646d-c5da-46d7-b1b0-69b3e2bcdd85 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:04.183659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1373610-375c-48a7-b587-f9d182bfb273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:04.184827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1373610-375c-48a7-b587-f9d182bfb273 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:04.184827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1373610-375c-48a7-b587-f9d182bfb273 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:04.185247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1373610-375c-48a7-b587-f9d182bfb273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:04.198123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1373610-375c-48a7-b587-f9d182bfb273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:04.199615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1373610-375c-48a7-b587-f9d182bfb273 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:04.212353 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3435/6833] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:04.213005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1373610-375c-48a7-b587-f9d182bfb273 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:04.225228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1373610-375c-48a7-b587-f9d182bfb273 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:04.231074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b6250-ea1d-480a-845f-316133ffbf6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:04.232523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b6250-ea1d-480a-845f-316133ffbf6e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:04.232991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b6250-ea1d-480a-845f-316133ffbf6e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:04.233442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b6250-ea1d-480a-845f-316133ffbf6e admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:04.234597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b6250-ea1d-480a-845f-316133ffbf6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:28:04.235070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b6250-ea1d-480a-845f-316133ffbf6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:04.237190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b6250-ea1d-480a-845f-316133ffbf6e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:04.257489 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3436/6834] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:28:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:28:04.259824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b6250-ea1d-480a-845f-316133ffbf6e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:04.286617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b6250-ea1d-480a-845f-316133ffbf6e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:04.311236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd518f50-d4df-481b-a951-da7c8a2ff1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:04.313747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd518f50-d4df-481b-a951-da7c8a2ff1be admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:04.314383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd518f50-d4df-481b-a951-da7c8a2ff1be admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:04.318619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd518f50-d4df-481b-a951-da7c8a2ff1be admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:04.318619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd518f50-d4df-481b-a951-da7c8a2ff1be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:04.319536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94ec57b4-f1d5-4201-b3fa-448ff0f64689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f00ea0259c34a468d66d894e10a3d05'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:04.321240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd518f50-d4df-481b-a951-da7c8a2ff1be admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:04.321699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94ec57b4-f1d5-4201-b3fa-448ff0f64689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f00ea0259c34a468d66d894e10a3d05'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:04.337432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94ec57b4-f1d5-4201-b3fa-448ff0f64689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f00ea0259c34a468d66d894e10a3d05'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:04.500744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94ec57b4-f1d5-4201-b3fa-448ff0f64689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f00ea0259c34a468d66d894e10a3d05'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:04.504758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94ec57b4-f1d5-4201-b3fa-448ff0f64689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f00ea0259c34a468d66d894e10a3d05'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:04.527938 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3437/6835] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:28:04 2022] POST /identity/v3/projects => generated 385 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:04.529061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd518f50-d4df-481b-a951-da7c8a2ff1be admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:04.536531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dd518f50-d4df-481b-a951-da7c8a2ff1be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:04.575611 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3399/6836] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:03 2022] DELETE /identity/v3/users/0f00ea0259c34a468d66d894e10a3d05 => generated 0 bytes in 820 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:04.578788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec57b4-f1d5-4201-b3fa-448ff0f64689 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:04.580231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-94ec57b4-f1d5-4201-b3fa-448ff0f64689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:04.741475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd518f50-d4df-481b-a951-da7c8a2ff1be admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:04.801870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec57b4-f1d5-4201-b3fa-448ff0f64689 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:04.809781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234879e0-0bde-4d7f-9adc-8c04144fde83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:04.811012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234879e0-0bde-4d7f-9adc-8c04144fde83 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:04.811463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234879e0-0bde-4d7f-9adc-8c04144fde83 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:04.812522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234879e0-0bde-4d7f-9adc-8c04144fde83 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:04.814750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234879e0-0bde-4d7f-9adc-8c04144fde83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:04.817201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234879e0-0bde-4d7f-9adc-8c04144fde83 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:04.891539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9508f42-78eb-4d3a-bb84-197651025dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:04.891741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9508f42-78eb-4d3a-bb84-197651025dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:04.892148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9508f42-78eb-4d3a-bb84-197651025dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:04.892503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9508f42-78eb-4d3a-bb84-197651025dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:04.894850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e9508f42-78eb-4d3a-bb84-197651025dd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:04.962429 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3400/6837] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:28:04 2022] POST /identity/v3/users => generated 293 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:04.968448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800d8e7-0f90-402a-b578-77faa434beba None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:04.968791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800d8e7-0f90-402a-b578-77faa434beba None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:04.969082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800d8e7-0f90-402a-b578-77faa434beba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:04.991710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d800d8e7-0f90-402a-b578-77faa434beba None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:04.993223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d800d8e7-0f90-402a-b578-77faa434beba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:05.006380 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3401/6838] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:05.006380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800d8e7-0f90-402a-b578-77faa434beba None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:05.018387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800d8e7-0f90-402a-b578-77faa434beba None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:05.020482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9508f42-78eb-4d3a-bb84-197651025dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:05.023246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9508f42-78eb-4d3a-bb84-197651025dd3 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:05.023405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6d642-3b9e-4636-82c7-c3a7e8045b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:05.024536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6d642-3b9e-4636-82c7-c3a7e8045b6a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:05.024883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6d642-3b9e-4636-82c7-c3a7e8045b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:05.025455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6d642-3b9e-4636-82c7-c3a7e8045b6a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:05.026264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6d642-3b9e-4636-82c7-c3a7e8045b6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:05.028105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6d642-3b9e-4636-82c7-c3a7e8045b6a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:05.043584 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3402/6839] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:05.051375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6d642-3b9e-4636-82c7-c3a7e8045b6a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:05.058424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3bf6d642-3b9e-4636-82c7-c3a7e8045b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:05.060605 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3438/6840] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:05.062558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9508f42-78eb-4d3a-bb84-197651025dd3 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:05.068523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:05.084693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9508f42-78eb-4d3a-bb84-197651025dd3 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:05.086607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6d642-3b9e-4636-82c7-c3a7e8045b6a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:05.100218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7d9a1-8803-4837-9edd-7eabb2b7ff5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:05.100922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7d9a1-8803-4837-9edd-7eabb2b7ff5b service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:05.102653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7d9a1-8803-4837-9edd-7eabb2b7ff5b service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:05.102653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7d9a1-8803-4837-9edd-7eabb2b7ff5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:05.106220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:05.107172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:05.107459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:05.107737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f admin admin] PATH_INFO: `/v3/projects/58db866e3b13484abe9de5c091d9aa5f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:05.109007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=58db866e3b13484abe9de5c091d9aa5f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:05.111194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:05.123343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:05.140194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7d9a1-8803-4837-9edd-7eabb2b7ff5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:05.142197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7d9a1-8803-4837-9edd-7eabb2b7ff5b service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:05.160174 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3439/6841] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:05.162449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7d9a1-8803-4837-9edd-7eabb2b7ff5b service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:05.175606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7d9a1-8803-4837-9edd-7eabb2b7ff5b service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:05.183486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241b22c-12e5-4f22-930e-327fe69f07dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:05.184432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241b22c-12e5-4f22-930e-327fe69f07dd admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:05.185689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241b22c-12e5-4f22-930e-327fe69f07dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:05.186077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241b22c-12e5-4f22-930e-327fe69f07dd admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:05.186831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241b22c-12e5-4f22-930e-327fe69f07dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:05.188449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241b22c-12e5-4f22-930e-327fe69f07dd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:05.202288 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3440/6842] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:05.202288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c8a9a-dabd-4fd7-b6b7-ccc05f0bb3d3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:05.202288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c8a9a-dabd-4fd7-b6b7-ccc05f0bb3d3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:05.202288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353c8a9a-dabd-4fd7-b6b7-ccc05f0bb3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:05.256974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-353c8a9a-dabd-4fd7-b6b7-ccc05f0bb3d3 None None] MFA Rules not processed for user `93f78df0f96c47ac97a200a4442db6da`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:05.258606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-353c8a9a-dabd-4fd7-b6b7-ccc05f0bb3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:05.267155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58db866e3b13484abe9de5c091d9aa5f', 'request_id': 'req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:05.279691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58db866e3b13484abe9de5c091d9aa5f', 'request_id': 'req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:05.348034 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3441/6843] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:05.363098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c8a9a-dabd-4fd7-b6b7-ccc05f0bb3d3 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:05.375280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58db866e3b13484abe9de5c091d9aa5f', 'request_id': 'req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:05.395586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f admin admin] The token cache is being invalidate because project 58db866e3b13484abe9de5c091d9aa5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:05.396936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:05.398405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:05.400742 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3403/6844] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:05 2022] DELETE /identity/v3/projects/58db866e3b13484abe9de5c091d9aa5f => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:05.401811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:05.410688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:05.478873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353c8a9a-dabd-4fd7-b6b7-ccc05f0bb3d3 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:05.488729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d6218a-0558-4e0d-a3e2-a7a0b4ce2c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:05.490013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6218a-0558-4e0d-a3e2-a7a0b4ce2c05 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:05.490390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6218a-0558-4e0d-a3e2-a7a0b4ce2c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:05.490772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d6218a-0558-4e0d-a3e2-a7a0b4ce2c05 admin admin] PATH_INFO: `/v3/projects/ba8c8e4d8ae04970a098c2a95a7f6c1a/users/069331e912434397a486441396dc557e/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:05.516061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6218a-0558-4e0d-a3e2-a7a0b4ce2c05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba8c8e4d8ae04970a098c2a95a7f6c1a, user_id=069331e912434397a486441396dc557e, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:05.517764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d6218a-0558-4e0d-a3e2-a7a0b4ce2c05 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:05.543111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7c76c-6193-46f2-a7a6-aebb6aa8fc3f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:05.550708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a25c11-47cb-4b23-a91d-df0c10fe7843 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:05.551386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a25c11-47cb-4b23-a91d-df0c10fe7843 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:05.551670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a25c11-47cb-4b23-a91d-df0c10fe7843 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:05.551969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a25c11-47cb-4b23-a91d-df0c10fe7843 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:05.554062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e7a25c11-47cb-4b23-a91d-df0c10fe7843 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:05.570428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a25c11-47cb-4b23-a91d-df0c10fe7843 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:05.576735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a25c11-47cb-4b23-a91d-df0c10fe7843 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:05.590648 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3404/6845] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:05.591842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a25c11-47cb-4b23-a91d-df0c10fe7843 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:05.598650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e7a25c11-47cb-4b23-a91d-df0c10fe7843 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:05.616656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a25c11-47cb-4b23-a91d-df0c10fe7843 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:05.620338 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3442/6846] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:05 2022] PUT /identity/v3/projects/ba8c8e4d8ae04970a098c2a95a7f6c1a/users/069331e912434397a486441396dc557e/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:05.624063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a49ec-6656-4ada-b6a7-bc5a09157ae5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:05.624357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a49ec-6656-4ada-b6a7-bc5a09157ae5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:05.624641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a49ec-6656-4ada-b6a7-bc5a09157ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:05.657553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6a1a49ec-6656-4ada-b6a7-bc5a09157ae5 None None] MFA Rules not processed for user `069331e912434397a486441396dc557e`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:05.658843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6a1a49ec-6656-4ada-b6a7-bc5a09157ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:05.696611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c71a7a-ac4d-48bf-a83f-ca7c1aa05531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:05.699236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c71a7a-ac4d-48bf-a83f-ca7c1aa05531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:05.699761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c71a7a-ac4d-48bf-a83f-ca7c1aa05531 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:05.700854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c71a7a-ac4d-48bf-a83f-ca7c1aa05531 admin admin] PATH_INFO: `/v3/users/b906b2f0012148d48d36be0d057d816a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:05.702782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c71a7a-ac4d-48bf-a83f-ca7c1aa05531 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b906b2f0012148d48d36be0d057d816a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:05.706362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c71a7a-ac4d-48bf-a83f-ca7c1aa05531 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:05.757489 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3443/6847] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:05.761870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a49ec-6656-4ada-b6a7-bc5a09157ae5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:05.763207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6a1a49ec-6656-4ada-b6a7-bc5a09157ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:05.788063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1a49ec-6656-4ada-b6a7-bc5a09157ae5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:05.826572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3271f7-17d5-45b4-bb1d-3db3aedad76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:05.827267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3271f7-17d5-45b4-bb1d-3db3aedad76a admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:05.827557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3271f7-17d5-45b4-bb1d-3db3aedad76a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:05.827911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3271f7-17d5-45b4-bb1d-3db3aedad76a admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:05.828561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3271f7-17d5-45b4-bb1d-3db3aedad76a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:05.830625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3271f7-17d5-45b4-bb1d-3db3aedad76a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:05.924745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-53c71a7a-ac4d-48bf-a83f-ca7c1aa05531 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b906b2f0012148d48d36be0d057d816a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:05.925350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-53c71a7a-ac4d-48bf-a83f-ca7c1aa05531 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b906b2f0012148d48d36be0d057d816a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:05.959423 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3444/6848] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:28:05 2022] POST /identity/v3/projects => generated 387 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:28:05.959423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3271f7-17d5-45b4-bb1d-3db3aedad76a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:05.970014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bb3271f7-17d5-45b4-bb1d-3db3aedad76a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:05.979450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-53c71a7a-ac4d-48bf-a83f-ca7c1aa05531 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b906b2f0012148d48d36be0d057d816a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:06.067297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3271f7-17d5-45b4-bb1d-3db3aedad76a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:06.072187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-53c71a7a-ac4d-48bf-a83f-ca7c1aa05531 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b906b2f0012148d48d36be0d057d816a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:06.075146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-53c71a7a-ac4d-48bf-a83f-ca7c1aa05531 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b906b2f0012148d48d36be0d057d816a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:06.099298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e202d-79e8-44d9-b28c-03cd444d624c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:06.100395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e202d-79e8-44d9-b28c-03cd444d624c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:06.100690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e202d-79e8-44d9-b28c-03cd444d624c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:06.101109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e202d-79e8-44d9-b28c-03cd444d624c admin admin] PATH_INFO: `/v3/projects/a800fc7014f942b8b0681e738b3adef4` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:06.102608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e202d-79e8-44d9-b28c-03cd444d624c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a800fc7014f942b8b0681e738b3adef4)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:06.117500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e202d-79e8-44d9-b28c-03cd444d624c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:06.125144 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3405/6849] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:05 2022] DELETE /identity/v3/users/b906b2f0012148d48d36be0d057d816a => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:06.128327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c71a7a-ac4d-48bf-a83f-ca7c1aa05531 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:06.131043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-53c71a7a-ac4d-48bf-a83f-ca7c1aa05531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:06.167744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:06.211095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:06.271976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c71a7a-ac4d-48bf-a83f-ca7c1aa05531 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:06.278767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfab193-2df4-49d5-beaa-7a829c3f6364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:06.279860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfab193-2df4-49d5-beaa-7a829c3f6364 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:06.280391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfab193-2df4-49d5-beaa-7a829c3f6364 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:06.281471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfab193-2df4-49d5-beaa-7a829c3f6364 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:06.282977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfab193-2df4-49d5-beaa-7a829c3f6364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:06.286483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfab193-2df4-49d5-beaa-7a829c3f6364 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:06.508713 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3406/6850] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:28:06 2022] POST /identity/v3/users => generated 300 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:06.512567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfab193-2df4-49d5-beaa-7a829c3f6364 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:06.513656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fc9e202d-79e8-44d9-b28c-03cd444d624c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a800fc7014f942b8b0681e738b3adef4', 'request_id': 'req-fc9e202d-79e8-44d9-b28c-03cd444d624c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:06.524620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fc9e202d-79e8-44d9-b28c-03cd444d624c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a800fc7014f942b8b0681e738b3adef4', 'request_id': 'req-fc9e202d-79e8-44d9-b28c-03cd444d624c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:06.529829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfab193-2df4-49d5-beaa-7a829c3f6364 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:06.626293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fc9e202d-79e8-44d9-b28c-03cd444d624c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a800fc7014f942b8b0681e738b3adef4', 'request_id': 'req-fc9e202d-79e8-44d9-b28c-03cd444d624c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:06.645579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fc9e202d-79e8-44d9-b28c-03cd444d624c admin admin] The token cache is being invalidate because project a800fc7014f942b8b0681e738b3adef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:06.648289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fc9e202d-79e8-44d9-b28c-03cd444d624c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:06.650733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fc9e202d-79e8-44d9-b28c-03cd444d624c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:06.658801 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3445/6851] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:05 2022] DELETE /identity/v3/projects/a800fc7014f942b8b0681e738b3adef4 => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:06.660113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e202d-79e8-44d9-b28c-03cd444d624c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:06.663867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fc9e202d-79e8-44d9-b28c-03cd444d624c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:06.671778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790c546-91da-44d8-9f7a-cda2d012b2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:06.671778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790c546-91da-44d8-9f7a-cda2d012b2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:06.671778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790c546-91da-44d8-9f7a-cda2d012b2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:06.671778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790c546-91da-44d8-9f7a-cda2d012b2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:06.672369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790c546-91da-44d8-9f7a-cda2d012b2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:06.674502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790c546-91da-44d8-9f7a-cda2d012b2f1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:06.691653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e202d-79e8-44d9-b28c-03cd444d624c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:06.697621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ca8f8-7e2e-4315-95a8-adc04efb24f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:06.700188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ca8f8-7e2e-4315-95a8-adc04efb24f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:06.704500 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3407/6852] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:06.706210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ca8f8-7e2e-4315-95a8-adc04efb24f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:06.707138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ca8f8-7e2e-4315-95a8-adc04efb24f0 admin admin] PATH_INFO: `/v3/users/50b290f7bef44156beeff98f4df1af72` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:06.709249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ca8f8-7e2e-4315-95a8-adc04efb24f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50b290f7bef44156beeff98f4df1af72)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:06.711473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3ca8f8-7e2e-4315-95a8-adc04efb24f0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:06.724699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790c546-91da-44d8-9f7a-cda2d012b2f1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:06.724699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f790c546-91da-44d8-9f7a-cda2d012b2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:06.743289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790c546-91da-44d8-9f7a-cda2d012b2f1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:06.757543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf94cd-af78-449b-9d40-692c989049d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:06.758505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf94cd-af78-449b-9d40-692c989049d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:06.758793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf94cd-af78-449b-9d40-692c989049d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:06.759120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf94cd-af78-449b-9d40-692c989049d8 admin admin] PATH_INFO: `/v3/projects/b79b3dca6bb0411fb0e5a7bfde12d5ac/users/98dec2804f3e4dad802487f4899758c1/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:06.788373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf94cd-af78-449b-9d40-692c989049d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b79b3dca6bb0411fb0e5a7bfde12d5ac, user_id=98dec2804f3e4dad802487f4899758c1, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:06.794036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf94cd-af78-449b-9d40-692c989049d8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:06.981935 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3408/6853] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:06 2022] PUT /identity/v3/projects/b79b3dca6bb0411fb0e5a7bfde12d5ac/users/98dec2804f3e4dad802487f4899758c1/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:06.990988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf94cd-af78-449b-9d40-692c989049d8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:07.093646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0a3ca8f8-7e2e-4315-95a8-adc04efb24f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50b290f7bef44156beeff98f4df1af72'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:07.106039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0a3ca8f8-7e2e-4315-95a8-adc04efb24f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50b290f7bef44156beeff98f4df1af72'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:07.107481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0a3ca8f8-7e2e-4315-95a8-adc04efb24f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50b290f7bef44156beeff98f4df1af72'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:07.108383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0a3ca8f8-7e2e-4315-95a8-adc04efb24f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50b290f7bef44156beeff98f4df1af72'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:07.154547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf94cd-af78-449b-9d40-692c989049d8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:07.161628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ca8cb-c1ba-44bc-baa1-ca986487d09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:07.162515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ca8cb-c1ba-44bc-baa1-ca986487d09b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:07.162842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ca8cb-c1ba-44bc-baa1-ca986487d09b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:07.163228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ca8cb-c1ba-44bc-baa1-ca986487d09b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:07.164670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ca8cb-c1ba-44bc-baa1-ca986487d09b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:07.166508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ca8cb-c1ba-44bc-baa1-ca986487d09b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:07.191905 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3409/6854] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:07.202599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ca8cb-c1ba-44bc-baa1-ca986487d09b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:07.204845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-470ca8cb-c1ba-44bc-baa1-ca986487d09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:07.238870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0a3ca8f8-7e2e-4315-95a8-adc04efb24f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50b290f7bef44156beeff98f4df1af72'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:07.244916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ca8cb-c1ba-44bc-baa1-ca986487d09b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:07.260975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd7a1a-054a-41d5-aa7f-47188b275884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:07.268542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd7a1a-054a-41d5-aa7f-47188b275884 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:07.268542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd7a1a-054a-41d5-aa7f-47188b275884 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:07.268542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd7a1a-054a-41d5-aa7f-47188b275884 admin admin] PATH_INFO: `/v3/projects/b79b3dca6bb0411fb0e5a7bfde12d5ac/users/98dec2804f3e4dad802487f4899758c1/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:07.269952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd7a1a-054a-41d5-aa7f-47188b275884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b79b3dca6bb0411fb0e5a7bfde12d5ac, user_id=98dec2804f3e4dad802487f4899758c1, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:07.275954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd7a1a-054a-41d5-aa7f-47188b275884 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:07.278052 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3446/6855] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:06 2022] DELETE /identity/v3/users/50b290f7bef44156beeff98f4df1af72 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:07.278666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ca8f8-7e2e-4315-95a8-adc04efb24f0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:07.281851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0a3ca8f8-7e2e-4315-95a8-adc04efb24f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:07.312912 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3410/6856] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:07 2022] PUT /identity/v3/projects/b79b3dca6bb0411fb0e5a7bfde12d5ac/users/98dec2804f3e4dad802487f4899758c1/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:07.323302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795afc7-131f-4b6a-89dc-b5cb7eb77f45 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:07.323610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795afc7-131f-4b6a-89dc-b5cb7eb77f45 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:07.323960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795afc7-131f-4b6a-89dc-b5cb7eb77f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:07.405526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0795afc7-131f-4b6a-89dc-b5cb7eb77f45 None None] MFA Rules not processed for user `98dec2804f3e4dad802487f4899758c1`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:07.410717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0795afc7-131f-4b6a-89dc-b5cb7eb77f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:07.479293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ca8f8-7e2e-4315-95a8-adc04efb24f0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:07.582941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:07.586516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:07.587277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:07.587568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a admin admin] PATH_INFO: `/v3/projects/43cc11d55355438a9100deeb5e77d877` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:07.591005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a admin admin] RBAC: Authorizing `identity:delete_project(project_id=43cc11d55355438a9100deeb5e77d877)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:07.593405 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3411/6857] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:07 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:28:07.593972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:07.604500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795afc7-131f-4b6a-89dc-b5cb7eb77f45 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:07.608604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0795afc7-131f-4b6a-89dc-b5cb7eb77f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:07.748478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795afc7-131f-4b6a-89dc-b5cb7eb77f45 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:07.805626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43cc11d55355438a9100deeb5e77d877', 'request_id': 'req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:07.812762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43cc11d55355438a9100deeb5e77d877', 'request_id': 'req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:07.888176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c2ff2-4415-41df-b5ae-67f227cf942f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:07.888907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c2ff2-4415-41df-b5ae-67f227cf942f service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:07.889201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c2ff2-4415-41df-b5ae-67f227cf942f service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:07.889482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1c2ff2-4415-41df-b5ae-67f227cf942f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:07.898338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43cc11d55355438a9100deeb5e77d877', 'request_id': 'req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:07.912178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a admin admin] The token cache is being invalidate because project 43cc11d55355438a9100deeb5e77d877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:07.912460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:07.913212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:07.915262 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3447/6858] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:07 2022] DELETE /identity/v3/projects/43cc11d55355438a9100deeb5e77d877 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:08.022084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:08.083829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c2ff2-4415-41df-b5ae-67f227cf942f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:08.085491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1c2ff2-4415-41df-b5ae-67f227cf942f service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:08.127742 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3412/6859] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:07 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:28:08.198139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:08.199716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:08.490825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7c7aa-89e3-42d0-a6f4-b1e7be2e771a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:08.507691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110ee46-010b-4b22-9251-38f9feed5810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:08.512170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110ee46-010b-4b22-9251-38f9feed5810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:08.512470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110ee46-010b-4b22-9251-38f9feed5810 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:08.513409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110ee46-010b-4b22-9251-38f9feed5810 admin admin] PATH_INFO: `/v3/projects/433606efac7643e892a147f76d0c47f2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:08.515366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110ee46-010b-4b22-9251-38f9feed5810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=433606efac7643e892a147f76d0c47f2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:08.518369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110ee46-010b-4b22-9251-38f9feed5810 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:08.571802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c2ff2-4415-41df-b5ae-67f227cf942f service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:08.574485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6b1c2ff2-4415-41df-b5ae-67f227cf942f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:08.602156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1c2ff2-4415-41df-b5ae-67f227cf942f service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:08.628549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2150ec-a415-40b5-b183-b9d11eec854c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:08.631604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2150ec-a415-40b5-b183-b9d11eec854c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:08.636165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2150ec-a415-40b5-b183-b9d11eec854c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:08.636165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2150ec-a415-40b5-b183-b9d11eec854c admin admin] PATH_INFO: `/v3/users/fbd2e63ff9c04ae38831b88193a574b2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:08.636165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2150ec-a415-40b5-b183-b9d11eec854c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbd2e63ff9c04ae38831b88193a574b2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:08.636924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2150ec-a415-40b5-b183-b9d11eec854c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:08.761115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d110ee46-010b-4b22-9251-38f9feed5810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '433606efac7643e892a147f76d0c47f2', 'request_id': 'req-d110ee46-010b-4b22-9251-38f9feed5810'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:08.768776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d110ee46-010b-4b22-9251-38f9feed5810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '433606efac7643e892a147f76d0c47f2', 'request_id': 'req-d110ee46-010b-4b22-9251-38f9feed5810'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:08.857083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d110ee46-010b-4b22-9251-38f9feed5810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '433606efac7643e892a147f76d0c47f2', 'request_id': 'req-d110ee46-010b-4b22-9251-38f9feed5810'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:08.874330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d110ee46-010b-4b22-9251-38f9feed5810 admin admin] The token cache is being invalidate because project 433606efac7643e892a147f76d0c47f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:08.886946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d110ee46-010b-4b22-9251-38f9feed5810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:08.887886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d110ee46-010b-4b22-9251-38f9feed5810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:08.890236 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3448/6860] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:08 2022] DELETE /identity/v3/projects/433606efac7643e892a147f76d0c47f2 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:08.903726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a78ca1-2e2b-4112-9824-4745248cb3b1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:08.904059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a78ca1-2e2b-4112-9824-4745248cb3b1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:08.904508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a78ca1-2e2b-4112-9824-4745248cb3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:08.929761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c2a78ca1-2e2b-4112-9824-4745248cb3b1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:08.931539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c2a78ca1-2e2b-4112-9824-4745248cb3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:08.940851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9a2150ec-a415-40b5-b183-b9d11eec854c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbd2e63ff9c04ae38831b88193a574b2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:08.943237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9a2150ec-a415-40b5-b183-b9d11eec854c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbd2e63ff9c04ae38831b88193a574b2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:08.965664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:08.968998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9a2150ec-a415-40b5-b183-b9d11eec854c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbd2e63ff9c04ae38831b88193a574b2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:09.033417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9a2150ec-a415-40b5-b183-b9d11eec854c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbd2e63ff9c04ae38831b88193a574b2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:09.034216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9a2150ec-a415-40b5-b183-b9d11eec854c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbd2e63ff9c04ae38831b88193a574b2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:09.085097 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3449/6861] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:09.091734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf00d57-9a7a-46b6-973a-6b332e66a467 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:09.092218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf00d57-9a7a-46b6-973a-6b332e66a467 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:09.092704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf00d57-9a7a-46b6-973a-6b332e66a467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:09.129009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-baf00d57-9a7a-46b6-973a-6b332e66a467 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:09.135111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-baf00d57-9a7a-46b6-973a-6b332e66a467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:09.136788 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3413/6862] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:08 2022] DELETE /identity/v3/users/fbd2e63ff9c04ae38831b88193a574b2 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:28:09.141285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81943240-d762-4b6a-bb3e-f87fe09f892d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:09.141829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81943240-d762-4b6a-bb3e-f87fe09f892d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:09.147648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81943240-d762-4b6a-bb3e-f87fe09f892d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:09.209058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-81943240-d762-4b6a-bb3e-f87fe09f892d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:09.210374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-81943240-d762-4b6a-bb3e-f87fe09f892d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:09.254433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:09.278700 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3450/6863] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:09.283319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf00d57-9a7a-46b6-973a-6b332e66a467 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:09.307020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf00d57-9a7a-46b6-973a-6b332e66a467 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:09.332077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f5be3-e0b1-4034-84e0-45de8397ab6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:09.333235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f5be3-e0b1-4034-84e0-45de8397ab6c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:09.333531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f5be3-e0b1-4034-84e0-45de8397ab6c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:09.334595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f5be3-e0b1-4034-84e0-45de8397ab6c admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:09.338351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f5be3-e0b1-4034-84e0-45de8397ab6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:28:09.338731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f5be3-e0b1-4034-84e0-45de8397ab6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:09.340408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f5be3-e0b1-4034-84e0-45de8397ab6c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:09.358934 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3414/6864] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:09.372878 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3451/6865] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:28:09.381422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81943240-d762-4b6a-bb3e-f87fe09f892d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:09.395948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81943240-d762-4b6a-bb3e-f87fe09f892d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:09.402079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efc2ca-6770-45b1-8189-eb4d8f187374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:09.402854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efc2ca-6770-45b1-8189-eb4d8f187374 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:09.403212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efc2ca-6770-45b1-8189-eb4d8f187374 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:09.403778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efc2ca-6770-45b1-8189-eb4d8f187374 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:09.404589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63efc2ca-6770-45b1-8189-eb4d8f187374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:09.406951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63efc2ca-6770-45b1-8189-eb4d8f187374 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:09.429016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f5be3-e0b1-4034-84e0-45de8397ab6c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:09.437550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-036f5be3-e0b1-4034-84e0-45de8397ab6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:09.524368 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3415/6866] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:28:09 2022] POST /identity/v3/projects => generated 389 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:09.526093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efc2ca-6770-45b1-8189-eb4d8f187374 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:09.606722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f5be3-e0b1-4034-84e0-45de8397ab6c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:09.700858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efc2ca-6770-45b1-8189-eb4d8f187374 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:09.708676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b5843-fe41-45fb-9454-ddcf476a49a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:09.709366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b5843-fe41-45fb-9454-ddcf476a49a7 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:09.709999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b5843-fe41-45fb-9454-ddcf476a49a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:09.709999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b5843-fe41-45fb-9454-ddcf476a49a7 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:09.710648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b5843-fe41-45fb-9454-ddcf476a49a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:09.711461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc845cd-7b03-44c8-ba78-98830c9e9675 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:09.712384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc845cd-7b03-44c8-ba78-98830c9e9675 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:09.712785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc845cd-7b03-44c8-ba78-98830c9e9675 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:09.713028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b5843-fe41-45fb-9454-ddcf476a49a7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:09.713240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc845cd-7b03-44c8-ba78-98830c9e9675 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:09.734707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc845cd-7b03-44c8-ba78-98830c9e9675 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:09.736700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc845cd-7b03-44c8-ba78-98830c9e9675 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:09.751376 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3452/6867] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:09.840588 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3416/6868] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:28:09 2022] POST /identity/v3/users => generated 295 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:09.845049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc845cd-7b03-44c8-ba78-98830c9e9675 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:09.858543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc845cd-7b03-44c8-ba78-98830c9e9675 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:09.868363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921db57-2dbf-46dc-a37f-f6496217f3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:09.869590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921db57-2dbf-46dc-a37f-f6496217f3f3 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:09.869590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921db57-2dbf-46dc-a37f-f6496217f3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:09.869912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921db57-2dbf-46dc-a37f-f6496217f3f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:09.872509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921db57-2dbf-46dc-a37f-f6496217f3f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:09.877287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921db57-2dbf-46dc-a37f-f6496217f3f3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:09.901565 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3453/6869] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:09.904828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b5843-fe41-45fb-9454-ddcf476a49a7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:09.920943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b5843-fe41-45fb-9454-ddcf476a49a7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:09.940618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed447a-863e-4cf7-8676-7c1da2669416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:09.946362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed447a-863e-4cf7-8676-7c1da2669416 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:09.946362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed447a-863e-4cf7-8676-7c1da2669416 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:09.946362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed447a-863e-4cf7-8676-7c1da2669416 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:09.946362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed447a-863e-4cf7-8676-7c1da2669416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:09.949059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed447a-863e-4cf7-8676-7c1da2669416 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:09.970520 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3417/6870] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:09.974988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921db57-2dbf-46dc-a37f-f6496217f3f3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:09.988772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921db57-2dbf-46dc-a37f-f6496217f3f3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:09.995098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa71463-1d8f-4980-8708-c8f2ffc2ea3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:09.996205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa71463-1d8f-4980-8708-c8f2ffc2ea3f admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:09.996980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa71463-1d8f-4980-8708-c8f2ffc2ea3f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:09.998395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa71463-1d8f-4980-8708-c8f2ffc2ea3f admin admin] PATH_INFO: `/v3/projects/203815c99d9546cf95c3e16207bba421/users/63fdd4246d2047c39e3a7e4d4c4fbc45/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:10.027290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed447a-863e-4cf7-8676-7c1da2669416 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:10.030852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cfed447a-863e-4cf7-8676-7c1da2669416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:10.054905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa71463-1d8f-4980-8708-c8f2ffc2ea3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=203815c99d9546cf95c3e16207bba421, user_id=63fdd4246d2047c39e3a7e4d4c4fbc45, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:10.073777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa71463-1d8f-4980-8708-c8f2ffc2ea3f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:10.088805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed447a-863e-4cf7-8676-7c1da2669416 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:10.105206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909d6d4-89ea-4da3-8171-c2bbfb7077c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:10.106278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909d6d4-89ea-4da3-8171-c2bbfb7077c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:10.110677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909d6d4-89ea-4da3-8171-c2bbfb7077c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:10.110677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909d6d4-89ea-4da3-8171-c2bbfb7077c9 admin admin] PATH_INFO: `/v3/users/069331e912434397a486441396dc557e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:10.110677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909d6d4-89ea-4da3-8171-c2bbfb7077c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=069331e912434397a486441396dc557e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:10.117527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909d6d4-89ea-4da3-8171-c2bbfb7077c9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:10.202307 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3454/6871] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:09 2022] PUT /identity/v3/projects/203815c99d9546cf95c3e16207bba421/users/63fdd4246d2047c39e3a7e4d4c4fbc45/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:28:10.214041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44c008-38d1-49d7-9faf-21a5ef503789 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:10.214041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44c008-38d1-49d7-9faf-21a5ef503789 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:10.214339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d44c008-38d1-49d7-9faf-21a5ef503789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:10.270751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6d44c008-38d1-49d7-9faf-21a5ef503789 None None] MFA Rules not processed for user `63fdd4246d2047c39e3a7e4d4c4fbc45`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:10.272915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6d44c008-38d1-49d7-9faf-21a5ef503789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:10.400763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3909d6d4-89ea-4da3-8171-c2bbfb7077c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '069331e912434397a486441396dc557e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:10.402002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3909d6d4-89ea-4da3-8171-c2bbfb7077c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '069331e912434397a486441396dc557e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:10.412168 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3455/6872] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:28:10.421269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3909d6d4-89ea-4da3-8171-c2bbfb7077c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '069331e912434397a486441396dc557e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:10.468045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44c008-38d1-49d7-9faf-21a5ef503789 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:10.469802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6d44c008-38d1-49d7-9faf-21a5ef503789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:10.507580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3909d6d4-89ea-4da3-8171-c2bbfb7077c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '069331e912434397a486441396dc557e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:10.516921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3909d6d4-89ea-4da3-8171-c2bbfb7077c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '069331e912434397a486441396dc557e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:10.574492 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3418/6873] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:10 2022] DELETE /identity/v3/users/069331e912434397a486441396dc557e => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:10.579079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ccadf-cd66-432f-badd-363b6b9a65f8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:10.579944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ccadf-cd66-432f-badd-363b6b9a65f8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:10.581378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ccadf-cd66-432f-badd-363b6b9a65f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:10.597537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d44c008-38d1-49d7-9faf-21a5ef503789 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:10.631034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5e1ccadf-cd66-432f-badd-363b6b9a65f8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:10.632654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5e1ccadf-cd66-432f-badd-363b6b9a65f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:10.712309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928a2ec-16fc-4360-bff0-1a39dcb3c4c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:10.712532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928a2ec-16fc-4360-bff0-1a39dcb3c4c0 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:10.712734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928a2ec-16fc-4360-bff0-1a39dcb3c4c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:10.733558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3928a2ec-16fc-4360-bff0-1a39dcb3c4c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:10.741787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3928a2ec-16fc-4360-bff0-1a39dcb3c4c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:10.751525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:10.782361 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3419/6874] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:10.830165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ccadf-cd66-432f-badd-363b6b9a65f8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:10.831844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5e1ccadf-cd66-432f-badd-363b6b9a65f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:10.856340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ccadf-cd66-432f-badd-363b6b9a65f8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:10.861390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:10.868910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928a2ec-16fc-4360-bff0-1a39dcb3c4c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:10.874576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a15282-bb27-494a-adfa-ffd9596f204c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:10.874576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a15282-bb27-494a-adfa-ffd9596f204c service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:10.874576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a15282-bb27-494a-adfa-ffd9596f204c service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:10.874576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a15282-bb27-494a-adfa-ffd9596f204c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:10.875073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3928a2ec-16fc-4360-bff0-1a39dcb3c4c0 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:10.889230 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3456/6875] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:28:10.891912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a15282-bb27-494a-adfa-ffd9596f204c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:10.893921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a15282-bb27-494a-adfa-ffd9596f204c service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:10.918385 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3420/6876] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:28:11.285611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928a2ec-16fc-4360-bff0-1a39dcb3c4c0 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:11.287173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3928a2ec-16fc-4360-bff0-1a39dcb3c4c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:11.316935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3928a2ec-16fc-4360-bff0-1a39dcb3c4c0 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:11.332834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1707f4-d94d-4094-a3ae-8d6977491b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:11.333855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1707f4-d94d-4094-a3ae-8d6977491b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:11.334136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1707f4-d94d-4094-a3ae-8d6977491b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:11.334415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1707f4-d94d-4094-a3ae-8d6977491b21 admin admin] PATH_INFO: `/v3/users/63fdd4246d2047c39e3a7e4d4c4fbc45` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:11.336822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1707f4-d94d-4094-a3ae-8d6977491b21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63fdd4246d2047c39e3a7e4d4c4fbc45)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:11.338838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1707f4-d94d-4094-a3ae-8d6977491b21 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:11.523777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a15282-bb27-494a-adfa-ffd9596f204c service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:11.528556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-06a15282-bb27-494a-adfa-ffd9596f204c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:11.542031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a15282-bb27-494a-adfa-ffd9596f204c service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:11.548276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb9f65-4be6-4c8d-87d1-98d3abef1049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:11.551539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb9f65-4be6-4c8d-87d1-98d3abef1049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:11.551539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb9f65-4be6-4c8d-87d1-98d3abef1049 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:11.551539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb9f65-4be6-4c8d-87d1-98d3abef1049 admin admin] PATH_INFO: `/v3/projects/7f6b1c0445ee4734bd092870f7ba1d4c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:11.551539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb9f65-4be6-4c8d-87d1-98d3abef1049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f6b1c0445ee4734bd092870f7ba1d4c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:11.555109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bb9f65-4be6-4c8d-87d1-98d3abef1049 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:11.685005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4f1707f4-d94d-4094-a3ae-8d6977491b21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63fdd4246d2047c39e3a7e4d4c4fbc45'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:11.707683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4f1707f4-d94d-4094-a3ae-8d6977491b21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63fdd4246d2047c39e3a7e4d4c4fbc45'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:11.709473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4f1707f4-d94d-4094-a3ae-8d6977491b21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63fdd4246d2047c39e3a7e4d4c4fbc45'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:11.710243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4f1707f4-d94d-4094-a3ae-8d6977491b21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63fdd4246d2047c39e3a7e4d4c4fbc45'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:11.836222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4f1707f4-d94d-4094-a3ae-8d6977491b21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63fdd4246d2047c39e3a7e4d4c4fbc45'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:11.844474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-35bb9f65-4be6-4c8d-87d1-98d3abef1049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f6b1c0445ee4734bd092870f7ba1d4c', 'request_id': 'req-35bb9f65-4be6-4c8d-87d1-98d3abef1049'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:11.857946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-35bb9f65-4be6-4c8d-87d1-98d3abef1049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f6b1c0445ee4734bd092870f7ba1d4c', 'request_id': 'req-35bb9f65-4be6-4c8d-87d1-98d3abef1049'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:11.858941 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3457/6877] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:11 2022] DELETE /identity/v3/users/63fdd4246d2047c39e3a7e4d4c4fbc45 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:11.863898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d728812-0ef2-437d-9fda-b32e3e120cff None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:11.864189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d728812-0ef2-437d-9fda-b32e3e120cff None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:11.865226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d728812-0ef2-437d-9fda-b32e3e120cff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:11.899268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7d728812-0ef2-437d-9fda-b32e3e120cff None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:11.901638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7d728812-0ef2-437d-9fda-b32e3e120cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:11.936211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-35bb9f65-4be6-4c8d-87d1-98d3abef1049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f6b1c0445ee4734bd092870f7ba1d4c', 'request_id': 'req-35bb9f65-4be6-4c8d-87d1-98d3abef1049'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:11.956508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-35bb9f65-4be6-4c8d-87d1-98d3abef1049 admin admin] The token cache is being invalidate because project 7f6b1c0445ee4734bd092870f7ba1d4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:11.956508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-35bb9f65-4be6-4c8d-87d1-98d3abef1049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:11.957299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-35bb9f65-4be6-4c8d-87d1-98d3abef1049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:11.960318 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3421/6878] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:11 2022] DELETE /identity/v3/projects/7f6b1c0445ee4734bd092870f7ba1d4c => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:11.964118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb9f65-4be6-4c8d-87d1-98d3abef1049 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:11.965784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-35bb9f65-4be6-4c8d-87d1-98d3abef1049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:12.021955 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3458/6879] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:28:12.041105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d728812-0ef2-437d-9fda-b32e3e120cff None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:12.042604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7d728812-0ef2-437d-9fda-b32e3e120cff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:12.101372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bb9f65-4be6-4c8d-87d1-98d3abef1049 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:12.107788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f60f58-3b02-406d-b32c-343064ec46d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:12.108985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f60f58-3b02-406d-b32c-343064ec46d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:12.109439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f60f58-3b02-406d-b32c-343064ec46d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:12.109942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f60f58-3b02-406d-b32c-343064ec46d9 admin admin] PATH_INFO: `/v3/users/695f6c2d6d774c6291f0dae701a508ad` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:12.111370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f60f58-3b02-406d-b32c-343064ec46d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=695f6c2d6d774c6291f0dae701a508ad)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:12.113861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f60f58-3b02-406d-b32c-343064ec46d9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:12.156949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d728812-0ef2-437d-9fda-b32e3e120cff None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:12.162910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f276a-b80e-4a50-9d91-b4786aab06b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:12.164746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f276a-b80e-4a50-9d91-b4786aab06b2 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:12.165046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f276a-b80e-4a50-9d91-b4786aab06b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:12.165437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f276a-b80e-4a50-9d91-b4786aab06b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:12.186178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f276a-b80e-4a50-9d91-b4786aab06b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:12.187391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f276a-b80e-4a50-9d91-b4786aab06b2 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:12.198031 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3459/6880] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:12.200057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f276a-b80e-4a50-9d91-b4786aab06b2 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:12.201472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-354f276a-b80e-4a50-9d91-b4786aab06b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:12.217125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f276a-b80e-4a50-9d91-b4786aab06b2 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:12.222570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263cf89c-1c72-4d9e-a69e-f90150f33edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:12.223441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263cf89c-1c72-4d9e-a69e-f90150f33edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:12.224297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263cf89c-1c72-4d9e-a69e-f90150f33edf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:12.224858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263cf89c-1c72-4d9e-a69e-f90150f33edf admin admin] PATH_INFO: `/v3/projects/ba8c8e4d8ae04970a098c2a95a7f6c1a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:12.226153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263cf89c-1c72-4d9e-a69e-f90150f33edf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba8c8e4d8ae04970a098c2a95a7f6c1a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:12.228041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263cf89c-1c72-4d9e-a69e-f90150f33edf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:12.290612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:12.402943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-58f60f58-3b02-406d-b32c-343064ec46d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '695f6c2d6d774c6291f0dae701a508ad'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:12.404053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-58f60f58-3b02-406d-b32c-343064ec46d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '695f6c2d6d774c6291f0dae701a508ad'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:12.413897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-263cf89c-1c72-4d9e-a69e-f90150f33edf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba8c8e4d8ae04970a098c2a95a7f6c1a', 'request_id': 'req-263cf89c-1c72-4d9e-a69e-f90150f33edf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:12.424284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-58f60f58-3b02-406d-b32c-343064ec46d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '695f6c2d6d774c6291f0dae701a508ad'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:12.427358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-263cf89c-1c72-4d9e-a69e-f90150f33edf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba8c8e4d8ae04970a098c2a95a7f6c1a', 'request_id': 'req-263cf89c-1c72-4d9e-a69e-f90150f33edf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:12.524969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-58f60f58-3b02-406d-b32c-343064ec46d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '695f6c2d6d774c6291f0dae701a508ad'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:12.525621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-58f60f58-3b02-406d-b32c-343064ec46d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '695f6c2d6d774c6291f0dae701a508ad'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:12.556879 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3422/6881] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:11 2022] DELETE /identity/v3/users/695f6c2d6d774c6291f0dae701a508ad => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:28:12.582102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-263cf89c-1c72-4d9e-a69e-f90150f33edf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba8c8e4d8ae04970a098c2a95a7f6c1a', 'request_id': 'req-263cf89c-1c72-4d9e-a69e-f90150f33edf'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:12.643820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-263cf89c-1c72-4d9e-a69e-f90150f33edf admin admin] The token cache is being invalidate because project ba8c8e4d8ae04970a098c2a95a7f6c1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:12.644829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-263cf89c-1c72-4d9e-a69e-f90150f33edf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:12.649472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-263cf89c-1c72-4d9e-a69e-f90150f33edf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:12.653790 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3460/6882] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:12 2022] DELETE /identity/v3/projects/ba8c8e4d8ae04970a098c2a95a7f6c1a => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:12.660335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f60f58-3b02-406d-b32c-343064ec46d9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:12.662492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-58f60f58-3b02-406d-b32c-343064ec46d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:12.704526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ebb88-0214-4119-acf7-83f5b3495824 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:12.705190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ebb88-0214-4119-acf7-83f5b3495824 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:12.705650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ebb88-0214-4119-acf7-83f5b3495824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:12.729826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:12.771680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f29ebb88-0214-4119-acf7-83f5b3495824 None None] MFA Rules not processed for user `93f78df0f96c47ac97a200a4442db6da`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:12.773543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f29ebb88-0214-4119-acf7-83f5b3495824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:12.823157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f60f58-3b02-406d-b32c-343064ec46d9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:12.832704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b930938-6440-4c14-88ca-e04c1b41b5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:12.834781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b930938-6440-4c14-88ca-e04c1b41b5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:12.835334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b930938-6440-4c14-88ca-e04c1b41b5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:12.835842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b930938-6440-4c14-88ca-e04c1b41b5c8 admin admin] PATH_INFO: `/v3/users/98dec2804f3e4dad802487f4899758c1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:12.838021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b930938-6440-4c14-88ca-e04c1b41b5c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98dec2804f3e4dad802487f4899758c1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:12.840669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b930938-6440-4c14-88ca-e04c1b41b5c8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:12.898415 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3461/6883] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:13.080391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5b930938-6440-4c14-88ca-e04c1b41b5c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98dec2804f3e4dad802487f4899758c1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:13.083809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5b930938-6440-4c14-88ca-e04c1b41b5c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98dec2804f3e4dad802487f4899758c1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:13.101888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5b930938-6440-4c14-88ca-e04c1b41b5c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98dec2804f3e4dad802487f4899758c1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:13.210442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5b930938-6440-4c14-88ca-e04c1b41b5c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98dec2804f3e4dad802487f4899758c1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:13.212192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5b930938-6440-4c14-88ca-e04c1b41b5c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98dec2804f3e4dad802487f4899758c1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:13.254624 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3423/6884] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:12 2022] DELETE /identity/v3/users/98dec2804f3e4dad802487f4899758c1 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:13.340321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ebb88-0214-4119-acf7-83f5b3495824 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:13.341806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f29ebb88-0214-4119-acf7-83f5b3495824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:13.374676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ebb88-0214-4119-acf7-83f5b3495824 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:13.468168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da18a16-44d8-4991-8681-c3cd32ac0017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:13.469667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da18a16-44d8-4991-8681-c3cd32ac0017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:13.470107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da18a16-44d8-4991-8681-c3cd32ac0017 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:13.470657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da18a16-44d8-4991-8681-c3cd32ac0017 admin admin] PATH_INFO: `/v3/projects/203815c99d9546cf95c3e16207bba421` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:13.472025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da18a16-44d8-4991-8681-c3cd32ac0017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=203815c99d9546cf95c3e16207bba421)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:13.475223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da18a16-44d8-4991-8681-c3cd32ac0017 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:13.619216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7da18a16-44d8-4991-8681-c3cd32ac0017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '203815c99d9546cf95c3e16207bba421', 'request_id': 'req-7da18a16-44d8-4991-8681-c3cd32ac0017'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:13.629510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7da18a16-44d8-4991-8681-c3cd32ac0017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '203815c99d9546cf95c3e16207bba421', 'request_id': 'req-7da18a16-44d8-4991-8681-c3cd32ac0017'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:13.748017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7da18a16-44d8-4991-8681-c3cd32ac0017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '203815c99d9546cf95c3e16207bba421', 'request_id': 'req-7da18a16-44d8-4991-8681-c3cd32ac0017'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:13.760735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b930938-6440-4c14-88ca-e04c1b41b5c8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:13.762695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5b930938-6440-4c14-88ca-e04c1b41b5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:13.774632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7da18a16-44d8-4991-8681-c3cd32ac0017 admin admin] The token cache is being invalidate because project 203815c99d9546cf95c3e16207bba421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:13.775269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7da18a16-44d8-4991-8681-c3cd32ac0017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:13.779111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7da18a16-44d8-4991-8681-c3cd32ac0017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:13.789266 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3462/6885] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:13 2022] DELETE /identity/v3/projects/203815c99d9546cf95c3e16207bba421 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:13.789427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b930938-6440-4c14-88ca-e04c1b41b5c8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:13.793618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0e915-bf68-4e1e-884f-ebfadef32b8f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:13.793919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0e915-bf68-4e1e-884f-ebfadef32b8f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:13.794207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f0e915-bf68-4e1e-884f-ebfadef32b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:13.839786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a5f0e915-bf68-4e1e-884f-ebfadef32b8f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:13.842395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a5f0e915-bf68-4e1e-884f-ebfadef32b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:13.887844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eadad00-8d47-44b5-a874-a41ab4696d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:13.888807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadad00-8d47-44b5-a874-a41ab4696d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:13.889095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadad00-8d47-44b5-a874-a41ab4696d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:13.889879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eadad00-8d47-44b5-a874-a41ab4696d0e admin admin] PATH_INFO: `/v3/projects/34da6c8d235543f1b559991425abaf25` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:13.890902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eadad00-8d47-44b5-a874-a41ab4696d0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=34da6c8d235543f1b559991425abaf25)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:13.895892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eadad00-8d47-44b5-a874-a41ab4696d0e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:13.994339 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3463/6886] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:13.995352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bc3b4-4f7f-49b4-a308-b9020545ec08 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:13.995652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bc3b4-4f7f-49b4-a308-b9020545ec08 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:13.995981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bc3b4-4f7f-49b4-a308-b9020545ec08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:14.017259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-294bc3b4-4f7f-49b4-a308-b9020545ec08 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:14.018435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-294bc3b4-4f7f-49b4-a308-b9020545ec08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:14.027470 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3464/6887] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:28:14.033310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bc3b4-4f7f-49b4-a308-b9020545ec08 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:14.044073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bc3b4-4f7f-49b4-a308-b9020545ec08 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:14.052725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d425f-bbd3-4503-a75c-1d9ce35d0d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:14.053194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d425f-bbd3-4503-a75c-1d9ce35d0d7e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:14.053487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d425f-bbd3-4503-a75c-1d9ce35d0d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:14.053765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d425f-bbd3-4503-a75c-1d9ce35d0d7e admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:14.054855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d425f-bbd3-4503-a75c-1d9ce35d0d7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:28:14.054999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d425f-bbd3-4503-a75c-1d9ce35d0d7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:14.058979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d425f-bbd3-4503-a75c-1d9ce35d0d7e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:14.071737 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3465/6888] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:28:14.085582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d425f-bbd3-4503-a75c-1d9ce35d0d7e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:14.104453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d425f-bbd3-4503-a75c-1d9ce35d0d7e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:14.111193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4eadad00-8d47-44b5-a874-a41ab4696d0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34da6c8d235543f1b559991425abaf25', 'request_id': 'req-4eadad00-8d47-44b5-a874-a41ab4696d0e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:14.118945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4eadad00-8d47-44b5-a874-a41ab4696d0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34da6c8d235543f1b559991425abaf25', 'request_id': 'req-4eadad00-8d47-44b5-a874-a41ab4696d0e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:14.138623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:14.193274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4eadad00-8d47-44b5-a874-a41ab4696d0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34da6c8d235543f1b559991425abaf25', 'request_id': 'req-4eadad00-8d47-44b5-a874-a41ab4696d0e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:14.221314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4eadad00-8d47-44b5-a874-a41ab4696d0e admin admin] The token cache is being invalidate because project 34da6c8d235543f1b559991425abaf25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:14.222045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4eadad00-8d47-44b5-a874-a41ab4696d0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:14.225120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4eadad00-8d47-44b5-a874-a41ab4696d0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:14.230513 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3424/6889] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:13 2022] DELETE /identity/v3/projects/34da6c8d235543f1b559991425abaf25 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:14.233018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a6a1a-09b8-4dbe-a946-b7af43455da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:14.233885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75da8c3-54ee-4926-8ced-e831384b5bc1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:14.247966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75da8c3-54ee-4926-8ced-e831384b5bc1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:14.249138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a6a1a-09b8-4dbe-a946-b7af43455da3 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:14.249912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75da8c3-54ee-4926-8ced-e831384b5bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:14.250298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a6a1a-09b8-4dbe-a946-b7af43455da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:14.251925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a6a1a-09b8-4dbe-a946-b7af43455da3 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:14.253311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a6a1a-09b8-4dbe-a946-b7af43455da3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:14.263041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a6a1a-09b8-4dbe-a946-b7af43455da3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:14.298750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d75da8c3-54ee-4926-8ced-e831384b5bc1 None None] MFA Rules not processed for user `f3468a0b2d324603ab6c3080993bc0c8`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:14.306640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d75da8c3-54ee-4926-8ced-e831384b5bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:14.361641 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3466/6890] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:28:14 2022] POST /identity/v3/projects => generated 389 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:14.365139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:14.367002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665f302-a8fa-4b87-a88f-bcc4c14b79be None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:14.369267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665f302-a8fa-4b87-a88f-bcc4c14b79be None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:14.369791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665f302-a8fa-4b87-a88f-bcc4c14b79be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:14.412492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a665f302-a8fa-4b87-a88f-bcc4c14b79be None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:14.412492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a665f302-a8fa-4b87-a88f-bcc4c14b79be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:14.425867 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3425/6891] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:14.437520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75da8c3-54ee-4926-8ced-e831384b5bc1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:14.446979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d75da8c3-54ee-4926-8ced-e831384b5bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:14.471211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75da8c3-54ee-4926-8ced-e831384b5bc1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:14.478991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:14.526630 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3467/6892] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:14.535021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb81669-da72-4296-8a94-90c35b3004ac None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:14.536046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb81669-da72-4296-8a94-90c35b3004ac None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:14.537226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb81669-da72-4296-8a94-90c35b3004ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:14.554714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800a927-dd18-47d4-8ac0-5cd2c39d1fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:14.555414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800a927-dd18-47d4-8ac0-5cd2c39d1fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:14.555709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800a927-dd18-47d4-8ac0-5cd2c39d1fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:14.556003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800a927-dd18-47d4-8ac0-5cd2c39d1fb6 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:14.557528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800a927-dd18-47d4-8ac0-5cd2c39d1fb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:14.559743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800a927-dd18-47d4-8ac0-5cd2c39d1fb6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:14.577621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-abb81669-da72-4296-8a94-90c35b3004ac None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:14.579757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-abb81669-da72-4296-8a94-90c35b3004ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:14.593936 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3468/6893] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:28:14.598225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb81669-da72-4296-8a94-90c35b3004ac None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:14.613414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb81669-da72-4296-8a94-90c35b3004ac None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:14.621415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d31465-0065-4a46-8b42-f9f428b9b53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:14.622803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d31465-0065-4a46-8b42-f9f428b9b53e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:14.623436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d31465-0065-4a46-8b42-f9f428b9b53e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:14.624029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d31465-0065-4a46-8b42-f9f428b9b53e admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:14.625017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d31465-0065-4a46-8b42-f9f428b9b53e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:28:14.625486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d31465-0065-4a46-8b42-f9f428b9b53e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:14.627468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d31465-0065-4a46-8b42-f9f428b9b53e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:14.646948 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3469/6894] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:28:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:28:14.651147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d31465-0065-4a46-8b42-f9f428b9b53e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:14.662644 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3426/6895] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:28:14 2022] POST /identity/v3/users => generated 295 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:14.672068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800a927-dd18-47d4-8ac0-5cd2c39d1fb6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:14.682216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d31465-0065-4a46-8b42-f9f428b9b53e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:14.690255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800a927-dd18-47d4-8ac0-5cd2c39d1fb6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:14.696096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ec4f0-ae7a-4a18-9493-9e795d2f89a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:14.696096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ec4f0-ae7a-4a18-9493-9e795d2f89a5 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:14.696096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ec4f0-ae7a-4a18-9493-9e795d2f89a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:14.696096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ec4f0-ae7a-4a18-9493-9e795d2f89a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:14.696840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ec4f0-ae7a-4a18-9493-9e795d2f89a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:14.698979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ec4f0-ae7a-4a18-9493-9e795d2f89a5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:14.711195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579fa0d1-0e63-4121-a86b-ef194633e3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:14.712442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fa0d1-0e63-4121-a86b-ef194633e3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:14.713187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fa0d1-0e63-4121-a86b-ef194633e3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:14.714146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579fa0d1-0e63-4121-a86b-ef194633e3d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:14.715376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579fa0d1-0e63-4121-a86b-ef194633e3d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:14.720084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579fa0d1-0e63-4121-a86b-ef194633e3d1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:14.735876 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3427/6896] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:28:14.736856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579fa0d1-0e63-4121-a86b-ef194633e3d1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:14.739808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-579fa0d1-0e63-4121-a86b-ef194633e3d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:14.757768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579fa0d1-0e63-4121-a86b-ef194633e3d1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:14.768593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:14.771499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:14.772012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:14.772563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401 admin admin] PATH_INFO: `/v3/projects/b79b3dca6bb0411fb0e5a7bfde12d5ac` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:14.778058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b79b3dca6bb0411fb0e5a7bfde12d5ac)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:14.782980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:14.832375 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3470/6897] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:28:14 2022] POST /identity/v3/projects => generated 363 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:14.833879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ec4f0-ae7a-4a18-9493-9e795d2f89a5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:14.963886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ec4f0-ae7a-4a18-9493-9e795d2f89a5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:14.999216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b79b3dca6bb0411fb0e5a7bfde12d5ac', 'request_id': 'req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:15.012090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b79b3dca6bb0411fb0e5a7bfde12d5ac', 'request_id': 'req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:15.034798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e029c-0f88-483b-a810-538581c44ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:15.035508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e029c-0f88-483b-a810-538581c44ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:15.035839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e029c-0f88-483b-a810-538581c44ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:15.036293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e029c-0f88-483b-a810-538581c44ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:15.037106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e029c-0f88-483b-a810-538581c44ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:15.039146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e029c-0f88-483b-a810-538581c44ee2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:15.057171 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3471/6898] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:14 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:15.058832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e029c-0f88-483b-a810-538581c44ee2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:15.070026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e029c-0f88-483b-a810-538581c44ee2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:15.075125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b79b3dca6bb0411fb0e5a7bfde12d5ac', 'request_id': 'req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:15.076630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c1637-5efc-4da6-aaf5-21283b563237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:15.076919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c1637-5efc-4da6-aaf5-21283b563237 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:15.077361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c1637-5efc-4da6-aaf5-21283b563237 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:15.077651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c1637-5efc-4da6-aaf5-21283b563237 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:15.078568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c1637-5efc-4da6-aaf5-21283b563237 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:15.083230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c1637-5efc-4da6-aaf5-21283b563237 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:15.098066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401 admin admin] The token cache is being invalidate because project b79b3dca6bb0411fb0e5a7bfde12d5ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:15.098354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:15.099072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:15.100671 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3428/6899] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:14 2022] DELETE /identity/v3/projects/b79b3dca6bb0411fb0e5a7bfde12d5ac => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:15.101426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:15.111443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:15.125275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc46fc7-7bfe-4b80-92ff-d5ad92fe1401 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:15.132504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ae486-1f6f-4551-b8d2-7bd3eb273ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:15.134794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ae486-1f6f-4551-b8d2-7bd3eb273ad3 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:15.134794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ae486-1f6f-4551-b8d2-7bd3eb273ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:15.135335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ae486-1f6f-4551-b8d2-7bd3eb273ad3 admin admin] PATH_INFO: `/v3/projects/78489a3cef704363834055047d38d6c4/users/d68d5f72a3f9466cb9d119b4a9521708/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:15.156695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ae486-1f6f-4551-b8d2-7bd3eb273ad3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78489a3cef704363834055047d38d6c4, user_id=d68d5f72a3f9466cb9d119b4a9521708, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:15.160451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ae486-1f6f-4551-b8d2-7bd3eb273ad3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:15.193143 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3472/6900] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:28:15 2022] POST /identity/v3/users => generated 282 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:15.197892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1200482-b53d-40bd-9135-0aa17b5424d4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:15.198126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1200482-b53d-40bd-9135-0aa17b5424d4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:15.198591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1200482-b53d-40bd-9135-0aa17b5424d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:15.218677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c1200482-b53d-40bd-9135-0aa17b5424d4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:15.220371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c1200482-b53d-40bd-9135-0aa17b5424d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:15.231147 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3473/6901] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:15.233979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1200482-b53d-40bd-9135-0aa17b5424d4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:15.239770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c1200482-b53d-40bd-9135-0aa17b5424d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:15.256663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1200482-b53d-40bd-9135-0aa17b5424d4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:15.264328 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3429/6902] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:15 2022] PUT /identity/v3/projects/78489a3cef704363834055047d38d6c4/users/d68d5f72a3f9466cb9d119b4a9521708/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:15.274809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075c13e-e4b5-4b43-b523-9e25ed8852a5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:15.274809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075c13e-e4b5-4b43-b523-9e25ed8852a5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:15.274809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075c13e-e4b5-4b43-b523-9e25ed8852a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:15.297526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9075c13e-e4b5-4b43-b523-9e25ed8852a5 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:15.297526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9075c13e-e4b5-4b43-b523-9e25ed8852a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:15.339804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed7c23-a904-4424-b1e3-26238b978a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:15.340789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed7c23-a904-4424-b1e3-26238b978a1f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:15.341375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed7c23-a904-4424-b1e3-26238b978a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:15.342540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed7c23-a904-4424-b1e3-26238b978a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:15.343557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed7c23-a904-4424-b1e3-26238b978a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:15.347445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed7c23-a904-4424-b1e3-26238b978a1f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:15.365103 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3474/6903] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:15.367296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d1a87-5042-4c46-8fa4-65536f35b00d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:15.367748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d1a87-5042-4c46-8fa4-65536f35b00d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:15.368715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d1a87-5042-4c46-8fa4-65536f35b00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:15.429015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ad7d1a87-5042-4c46-8fa4-65536f35b00d None None] MFA Rules not processed for user `d68d5f72a3f9466cb9d119b4a9521708`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:15.430314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ad7d1a87-5042-4c46-8fa4-65536f35b00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:15.434491 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3430/6904] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:15.439635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075c13e-e4b5-4b43-b523-9e25ed8852a5 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:15.453927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075c13e-e4b5-4b43-b523-9e25ed8852a5 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:15.460139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb9f19-b856-4474-80e8-def544358fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:15.462130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb9f19-b856-4474-80e8-def544358fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:15.462609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb9f19-b856-4474-80e8-def544358fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:15.463051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fb9f19-b856-4474-80e8-def544358fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:15.464908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb9f19-b856-4474-80e8-def544358fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:15.468418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fb9f19-b856-4474-80e8-def544358fa3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:15.492122 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3431/6905] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:15 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:15.493072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb9f19-b856-4474-80e8-def544358fa3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:15.507818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fb9f19-b856-4474-80e8-def544358fa3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:15.515147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b44ce-3680-4875-8faf-dcccc9724add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:15.517217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b44ce-3680-4875-8faf-dcccc9724add admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:15.517689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b44ce-3680-4875-8faf-dcccc9724add admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:15.520281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b44ce-3680-4875-8faf-dcccc9724add admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:15.521263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b44ce-3680-4875-8faf-dcccc9724add admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:28:15.521734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b44ce-3680-4875-8faf-dcccc9724add admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:15.525288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b44ce-3680-4875-8faf-dcccc9724add admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:15.539425 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3475/6906] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:15.540387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d1a87-5042-4c46-8fa4-65536f35b00d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:15.552765 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3432/6907] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:28:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:28:15.559400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b44ce-3680-4875-8faf-dcccc9724add admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:15.562878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d1a87-5042-4c46-8fa4-65536f35b00d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:15.563057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-326b44ce-3680-4875-8faf-dcccc9724add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:15.576322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ea2798-0986-46fd-8c7e-05f8dd864cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:15.577249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea2798-0986-46fd-8c7e-05f8dd864cab admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:15.577560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea2798-0986-46fd-8c7e-05f8dd864cab admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:15.578022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ea2798-0986-46fd-8c7e-05f8dd864cab admin admin] PATH_INFO: `/v3/projects/590912c81c474229b156eceeb104f8e6/users/11db4b6aecfc40e8a30c2ef4d737f6ae/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:15.604984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea2798-0986-46fd-8c7e-05f8dd864cab admin admin] RBAC: Authorizing `identity:create_grant(project_id=590912c81c474229b156eceeb104f8e6, user_id=11db4b6aecfc40e8a30c2ef4d737f6ae, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:15.605646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ea2798-0986-46fd-8c7e-05f8dd864cab admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:15.640299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b44ce-3680-4875-8faf-dcccc9724add admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:15.644237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce77daa-245b-4d96-a3d2-9afb0df9e8a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:15.645023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce77daa-245b-4d96-a3d2-9afb0df9e8a1 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:15.645317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce77daa-245b-4d96-a3d2-9afb0df9e8a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:15.645635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce77daa-245b-4d96-a3d2-9afb0df9e8a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:15.655142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce77daa-245b-4d96-a3d2-9afb0df9e8a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:15.656886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce77daa-245b-4d96-a3d2-9afb0df9e8a1 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:15.667613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:15.668175 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3433/6908] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:15.669137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce77daa-245b-4d96-a3d2-9afb0df9e8a1 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:15.680953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce77daa-245b-4d96-a3d2-9afb0df9e8a1 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:15.688095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886af52-4a31-4b7c-8430-537229243d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:15.689304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886af52-4a31-4b7c-8430-537229243d47 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:15.690569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886af52-4a31-4b7c-8430-537229243d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:15.691045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886af52-4a31-4b7c-8430-537229243d47 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:15.691913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886af52-4a31-4b7c-8430-537229243d47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:15.695077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886af52-4a31-4b7c-8430-537229243d47 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:15.732430 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3476/6909] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:15 2022] PUT /identity/v3/projects/590912c81c474229b156eceeb104f8e6/users/11db4b6aecfc40e8a30c2ef4d737f6ae/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:15.745099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098fbe9-2f56-48bf-bad1-c1a445ed49c1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:15.745400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098fbe9-2f56-48bf-bad1-c1a445ed49c1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:15.745715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098fbe9-2f56-48bf-bad1-c1a445ed49c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:15.801239 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3434/6910] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:28:15 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:15.803018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886af52-4a31-4b7c-8430-537229243d47 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:15.825702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1098fbe9-2f56-48bf-bad1-c1a445ed49c1 None None] MFA Rules not processed for user `11db4b6aecfc40e8a30c2ef4d737f6ae`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:15.827056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1098fbe9-2f56-48bf-bad1-c1a445ed49c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:15.990576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886af52-4a31-4b7c-8430-537229243d47 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:16.004202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad2769-52dc-449a-bb60-b28f165193d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:16.006736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad2769-52dc-449a-bb60-b28f165193d1 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:16.009047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad2769-52dc-449a-bb60-b28f165193d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:16.009047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad2769-52dc-449a-bb60-b28f165193d1 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:16.010917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad2769-52dc-449a-bb60-b28f165193d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:16.015331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad2769-52dc-449a-bb60-b28f165193d1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:16.024983 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3477/6911] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:15 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:28:16.029527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098fbe9-2f56-48bf-bad1-c1a445ed49c1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:16.053226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098fbe9-2f56-48bf-bad1-c1a445ed49c1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:16.063639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5989c-44ad-4c57-8271-bf84ed72a688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:16.064493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5989c-44ad-4c57-8271-bf84ed72a688 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:16.064792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5989c-44ad-4c57-8271-bf84ed72a688 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:16.065080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf5989c-44ad-4c57-8271-bf84ed72a688 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:16.065800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5989c-44ad-4c57-8271-bf84ed72a688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:16.068062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf5989c-44ad-4c57-8271-bf84ed72a688 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:16.172128 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3435/6912] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:28:15 2022] POST /identity/v3/users => generated 288 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:28:16.176305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad2769-52dc-449a-bb60-b28f165193d1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:16.188608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:16.197827 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3478/6913] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:28:16 2022] POST /identity/v3/projects => generated 365 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:16.201093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5989c-44ad-4c57-8271-bf84ed72a688 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:16.222363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad2769-52dc-449a-bb60-b28f165193d1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:16.337951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf5989c-44ad-4c57-8271-bf84ed72a688 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:16.343901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468afa6-cd9e-420a-9282-2216b0305227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:16.344934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468afa6-cd9e-420a-9282-2216b0305227 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:16.345300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468afa6-cd9e-420a-9282-2216b0305227 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:16.345881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468afa6-cd9e-420a-9282-2216b0305227 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:16.346899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468afa6-cd9e-420a-9282-2216b0305227 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:16.349933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468afa6-cd9e-420a-9282-2216b0305227 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:16.387932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92174b95-d1b6-4ba0-b43c-862b24e30c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:16.389815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174b95-d1b6-4ba0-b43c-862b24e30c5a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:16.390142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174b95-d1b6-4ba0-b43c-862b24e30c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:16.390424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174b95-d1b6-4ba0-b43c-862b24e30c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:16.391231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92174b95-d1b6-4ba0-b43c-862b24e30c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:16.400422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92174b95-d1b6-4ba0-b43c-862b24e30c5a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:16.424566 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3436/6914] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:16.425744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92174b95-d1b6-4ba0-b43c-862b24e30c5a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:16.447893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92174b95-d1b6-4ba0-b43c-862b24e30c5a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:16.456967 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3479/6915] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:28:16 2022] POST /identity/v3/users => generated 289 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:28:16.461476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468afa6-cd9e-420a-9282-2216b0305227 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:16.462846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35678cb1-f35f-433d-aa5f-1e99647ba8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:16.463569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35678cb1-f35f-433d-aa5f-1e99647ba8ca admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:16.463856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35678cb1-f35f-433d-aa5f-1e99647ba8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:16.464137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35678cb1-f35f-433d-aa5f-1e99647ba8ca admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:16.464915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35678cb1-f35f-433d-aa5f-1e99647ba8ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:16.471927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35678cb1-f35f-433d-aa5f-1e99647ba8ca admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:16.478295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468afa6-cd9e-420a-9282-2216b0305227 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:16.483349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07877676-719a-427f-87d5-72050a395b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:16.484399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07877676-719a-427f-87d5-72050a395b40 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:16.484865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07877676-719a-427f-87d5-72050a395b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:16.485261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07877676-719a-427f-87d5-72050a395b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:16.485814 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3437/6916] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:28:16.487355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07877676-719a-427f-87d5-72050a395b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:16.489025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07877676-719a-427f-87d5-72050a395b40 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:16.494002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35678cb1-f35f-433d-aa5f-1e99647ba8ca admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:16.505800 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3480/6917] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:16.507108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35678cb1-f35f-433d-aa5f-1e99647ba8ca admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:16.508570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07877676-719a-427f-87d5-72050a395b40 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:16.512611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d77ee18-32d9-4466-b5ea-800f763a155e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:16.513706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77ee18-32d9-4466-b5ea-800f763a155e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:16.514065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77ee18-32d9-4466-b5ea-800f763a155e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:16.514419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d77ee18-32d9-4466-b5ea-800f763a155e admin admin] PATH_INFO: `/v3/projects/810549f80c104ef787180ca456d45c81/users/503314f28b4045f0bbaae8cdbfbfc989/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:16.518723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07877676-719a-427f-87d5-72050a395b40 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:16.524029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28572bd8-b450-4253-aada-39adb787a51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:16.524940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28572bd8-b450-4253-aada-39adb787a51c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:16.525365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28572bd8-b450-4253-aada-39adb787a51c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:16.525654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28572bd8-b450-4253-aada-39adb787a51c admin admin] PATH_INFO: `/v3/projects/6b3edb1aac4d4b828b20fdb13304570f/users/9fb5fb90eae145cb97c82bd6c195a557/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:16.536811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77ee18-32d9-4466-b5ea-800f763a155e admin admin] RBAC: Authorizing `identity:create_grant(project_id=810549f80c104ef787180ca456d45c81, user_id=503314f28b4045f0bbaae8cdbfbfc989, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:16.537208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d77ee18-32d9-4466-b5ea-800f763a155e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:16.556116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28572bd8-b450-4253-aada-39adb787a51c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b3edb1aac4d4b828b20fdb13304570f, user_id=9fb5fb90eae145cb97c82bd6c195a557, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:16.559264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28572bd8-b450-4253-aada-39adb787a51c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:16.655974 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3438/6918] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:16 2022] PUT /identity/v3/projects/810549f80c104ef787180ca456d45c81/users/503314f28b4045f0bbaae8cdbfbfc989/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:16.662514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4126091-d3cb-4bdf-8c95-ee8dd06f8bc4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:16.662514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4126091-d3cb-4bdf-8c95-ee8dd06f8bc4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:16.662514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4126091-d3cb-4bdf-8c95-ee8dd06f8bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:16.663002 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3481/6919] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:16 2022] PUT /identity/v3/projects/6b3edb1aac4d4b828b20fdb13304570f/users/9fb5fb90eae145cb97c82bd6c195a557/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:16.664911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28572bd8-b450-4253-aada-39adb787a51c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:16.708255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e4126091-d3cb-4bdf-8c95-ee8dd06f8bc4 None None] MFA Rules not processed for user `503314f28b4045f0bbaae8cdbfbfc989`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:16.710105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e4126091-d3cb-4bdf-8c95-ee8dd06f8bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:16.762040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28572bd8-b450-4253-aada-39adb787a51c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:16.808078 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3439/6920] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:16.813108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4126091-d3cb-4bdf-8c95-ee8dd06f8bc4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:16.846617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce672a42-b117-41ed-bd50-c62cf42835a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:16.847545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce672a42-b117-41ed-bd50-c62cf42835a4 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:16.849628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce672a42-b117-41ed-bd50-c62cf42835a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:16.850516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce672a42-b117-41ed-bd50-c62cf42835a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:16.850922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce672a42-b117-41ed-bd50-c62cf42835a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:16.857333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce672a42-b117-41ed-bd50-c62cf42835a4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:16.875739 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3482/6921] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:16.887287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce672a42-b117-41ed-bd50-c62cf42835a4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:16.893314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce672a42-b117-41ed-bd50-c62cf42835a4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:16.898029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4126091-d3cb-4bdf-8c95-ee8dd06f8bc4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:16.899015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ee1fa-daab-4e97-8a7e-0f8539fed9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:16.899938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ee1fa-daab-4e97-8a7e-0f8539fed9ca admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:16.900228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ee1fa-daab-4e97-8a7e-0f8539fed9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:16.900509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ee1fa-daab-4e97-8a7e-0f8539fed9ca admin admin] PATH_INFO: `/v3/projects/6b3edb1aac4d4b828b20fdb13304570f/users/9fb5fb90eae145cb97c82bd6c195a557/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:16.902838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ee1fa-daab-4e97-8a7e-0f8539fed9ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b3edb1aac4d4b828b20fdb13304570f, user_id=9fb5fb90eae145cb97c82bd6c195a557, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:16.905380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c53040-5dbf-40e3-999d-074403fdd0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:16.906255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ee1fa-daab-4e97-8a7e-0f8539fed9ca admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:16.906575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c53040-5dbf-40e3-999d-074403fdd0bd admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:16.907052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c53040-5dbf-40e3-999d-074403fdd0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:16.907519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c53040-5dbf-40e3-999d-074403fdd0bd admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:16.908430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c53040-5dbf-40e3-999d-074403fdd0bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:16.910668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c53040-5dbf-40e3-999d-074403fdd0bd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:16.919821 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3483/6922] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:16 2022] PUT /identity/v3/projects/6b3edb1aac4d4b828b20fdb13304570f/users/9fb5fb90eae145cb97c82bd6c195a557/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:16.935867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ab6a4-81f6-4522-bf46-3415251463e2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:16.936173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ab6a4-81f6-4522-bf46-3415251463e2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:16.936448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ab6a4-81f6-4522-bf46-3415251463e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:16.975690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-943ab6a4-81f6-4522-bf46-3415251463e2 None None] MFA Rules not processed for user `9fb5fb90eae145cb97c82bd6c195a557`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:16.978007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-943ab6a4-81f6-4522-bf46-3415251463e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:17.073371 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3440/6923] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:28:16 2022] POST /identity/v3/projects => generated 375 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:17.077291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c53040-5dbf-40e3-999d-074403fdd0bd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:17.109867 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3484/6924] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:16 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:28:17.126350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ab6a4-81f6-4522-bf46-3415251463e2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:17.128359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-943ab6a4-81f6-4522-bf46-3415251463e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:17.208895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c53040-5dbf-40e3-999d-074403fdd0bd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:17.238819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c2163-515b-4eec-a501-34286e2ad80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:17.239929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c2163-515b-4eec-a501-34286e2ad80d admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:17.241793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c2163-515b-4eec-a501-34286e2ad80d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:17.243527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c2163-515b-4eec-a501-34286e2ad80d admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:17.246032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c2163-515b-4eec-a501-34286e2ad80d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:17.254795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8c2163-515b-4eec-a501-34286e2ad80d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:17.293737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ab6a4-81f6-4522-bf46-3415251463e2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:17.354678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834770b-0b98-40ca-b70f-4c4059d40ee1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:17.355353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834770b-0b98-40ca-b70f-4c4059d40ee1 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:17.355803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834770b-0b98-40ca-b70f-4c4059d40ee1 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:17.356088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0834770b-0b98-40ca-b70f-4c4059d40ee1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:17.438942 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3441/6925] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:28:17 2022] POST /identity/v3/users => generated 294 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:28:17.440484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c2163-515b-4eec-a501-34286e2ad80d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:17.456087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c2163-515b-4eec-a501-34286e2ad80d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:17.458584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0834770b-0b98-40ca-b70f-4c4059d40ee1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:17.464217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c466f-5a74-43e3-baed-96c7a047c235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:17.464448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0834770b-0b98-40ca-b70f-4c4059d40ee1 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:17.465492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c466f-5a74-43e3-baed-96c7a047c235 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:17.465755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c466f-5a74-43e3-baed-96c7a047c235 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:17.466064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c466f-5a74-43e3-baed-96c7a047c235 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:17.466871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c466f-5a74-43e3-baed-96c7a047c235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:17.471167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06c466f-5a74-43e3-baed-96c7a047c235 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:17.492788 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3442/6926] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:17.493879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c466f-5a74-43e3-baed-96c7a047c235 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:17.508821 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3485/6927] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:17 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:17.530560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c466f-5a74-43e3-baed-96c7a047c235 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:17.542703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d421750-874e-413f-a79a-7807b79c639d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:17.543870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d421750-874e-413f-a79a-7807b79c639d admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:17.543870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d421750-874e-413f-a79a-7807b79c639d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:17.544219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d421750-874e-413f-a79a-7807b79c639d admin admin] PATH_INFO: `/v3/projects/9b41bb4bd67b4f9a8335db5864198100/users/aae76eca30504119b0fc9b95c285fce9/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:17.565403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d421750-874e-413f-a79a-7807b79c639d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b41bb4bd67b4f9a8335db5864198100, user_id=aae76eca30504119b0fc9b95c285fce9, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:17.569355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d421750-874e-413f-a79a-7807b79c639d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:17.718399 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3443/6928] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:17 2022] PUT /identity/v3/projects/9b41bb4bd67b4f9a8335db5864198100/users/aae76eca30504119b0fc9b95c285fce9/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:17.719577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834770b-0b98-40ca-b70f-4c4059d40ee1 service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:17.733983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0834770b-0b98-40ca-b70f-4c4059d40ee1 service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:17.810127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c68a1-bb7d-4b7e-846b-f6bffc7a95c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:17.810830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c68a1-bb7d-4b7e-846b-f6bffc7a95c4 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:17.811122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c68a1-bb7d-4b7e-846b-f6bffc7a95c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:17.811453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402c68a1-bb7d-4b7e-846b-f6bffc7a95c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:17.812808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c68a1-bb7d-4b7e-846b-f6bffc7a95c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:17.814712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402c68a1-bb7d-4b7e-846b-f6bffc7a95c4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:17.826450 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3486/6929] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:28:17.830728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d421750-874e-413f-a79a-7807b79c639d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:17.869084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c68a1-bb7d-4b7e-846b-f6bffc7a95c4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:17.873004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d421750-874e-413f-a79a-7807b79c639d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:17.887185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fd4c0-b02c-4054-800d-65fe1fd2b37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:17.888150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fd4c0-b02c-4054-800d-65fe1fd2b37d admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:17.888541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fd4c0-b02c-4054-800d-65fe1fd2b37d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:17.888925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fd4c0-b02c-4054-800d-65fe1fd2b37d admin admin] PATH_INFO: `/v3/projects/9b41bb4bd67b4f9a8335db5864198100/users/aae76eca30504119b0fc9b95c285fce9/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:17.893195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fd4c0-b02c-4054-800d-65fe1fd2b37d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b41bb4bd67b4f9a8335db5864198100, user_id=aae76eca30504119b0fc9b95c285fce9, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:17.896721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fd4c0-b02c-4054-800d-65fe1fd2b37d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:17.926910 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3444/6930] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:17 2022] PUT /identity/v3/projects/9b41bb4bd67b4f9a8335db5864198100/users/aae76eca30504119b0fc9b95c285fce9/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:17.938255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df51e3f-5782-490e-a825-12727a4c141b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:17.938476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df51e3f-5782-490e-a825-12727a4c141b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:17.938759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df51e3f-5782-490e-a825-12727a4c141b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:17.945062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402c68a1-bb7d-4b7e-846b-f6bffc7a95c4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:17.980456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2df51e3f-5782-490e-a825-12727a4c141b None None] MFA Rules not processed for user `aae76eca30504119b0fc9b95c285fce9`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:17.983779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2df51e3f-5782-490e-a825-12727a4c141b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:18.005752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f34dc4-ce15-446c-a24e-9db40772ffd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:18.006069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f34dc4-ce15-446c-a24e-9db40772ffd6 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:18.006356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f34dc4-ce15-446c-a24e-9db40772ffd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:18.006632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f34dc4-ce15-446c-a24e-9db40772ffd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:18.075509 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3445/6931] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:18.081311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f34dc4-ce15-446c-a24e-9db40772ffd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:18.087701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f34dc4-ce15-446c-a24e-9db40772ffd6 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:18.088177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df51e3f-5782-490e-a825-12727a4c141b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:18.088177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2df51e3f-5782-490e-a825-12727a4c141b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:18.096916 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3487/6932] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:17 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:28:18.106134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df51e3f-5782-490e-a825-12727a4c141b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:18.110219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a72d7-d114-4095-a5a9-c58c0b3c16db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:18.110949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a72d7-d114-4095-a5a9-c58c0b3c16db service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:18.111409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a72d7-d114-4095-a5a9-c58c0b3c16db service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:18.112857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a72d7-d114-4095-a5a9-c58c0b3c16db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:18.190801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a72d7-d114-4095-a5a9-c58c0b3c16db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:18.193131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a72d7-d114-4095-a5a9-c58c0b3c16db service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:18.202956 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3446/6933] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:18.409469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:18.993318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:21.257231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:21.839666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:22.484070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f34dc4-ce15-446c-a24e-9db40772ffd6 service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:22.635715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f34dc4-ce15-446c-a24e-9db40772ffd6 service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:22.645610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6344b7e-f64c-43b7-a955-e38d280107d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:22.647584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6344b7e-f64c-43b7-a955-e38d280107d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:22.648393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6344b7e-f64c-43b7-a955-e38d280107d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:22.649097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6344b7e-f64c-43b7-a955-e38d280107d3 admin admin] PATH_INFO: `/v3/users/11db4b6aecfc40e8a30c2ef4d737f6ae` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:22.650697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6344b7e-f64c-43b7-a955-e38d280107d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11db4b6aecfc40e8a30c2ef4d737f6ae)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:22.654842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6344b7e-f64c-43b7-a955-e38d280107d3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:22.950484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a6344b7e-f64c-43b7-a955-e38d280107d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11db4b6aecfc40e8a30c2ef4d737f6ae'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:22.973317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a6344b7e-f64c-43b7-a955-e38d280107d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11db4b6aecfc40e8a30c2ef4d737f6ae'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:22.976158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a6344b7e-f64c-43b7-a955-e38d280107d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11db4b6aecfc40e8a30c2ef4d737f6ae'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:22.977981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a6344b7e-f64c-43b7-a955-e38d280107d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11db4b6aecfc40e8a30c2ef4d737f6ae'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:23.135722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a6344b7e-f64c-43b7-a955-e38d280107d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11db4b6aecfc40e8a30c2ef4d737f6ae'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:23.175525 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3488/6934] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:22 2022] DELETE /identity/v3/users/11db4b6aecfc40e8a30c2ef4d737f6ae => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:23.181415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a94d4-a3e2-42b9-ac42-b53ed6f4acc6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:23.184040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a94d4-a3e2-42b9-ac42-b53ed6f4acc6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:23.184040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a94d4-a3e2-42b9-ac42-b53ed6f4acc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:23.213662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-da6a94d4-a3e2-42b9-ac42-b53ed6f4acc6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:23.215267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-da6a94d4-a3e2-42b9-ac42-b53ed6f4acc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:23.340825 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3447/6935] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:23.359939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6344b7e-f64c-43b7-a955-e38d280107d3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:23.362043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a6344b7e-f64c-43b7-a955-e38d280107d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:23.470426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6344b7e-f64c-43b7-a955-e38d280107d3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:23.476485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b2d5e-8efb-44a9-bce3-45c1efba4d32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:23.477463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b2d5e-8efb-44a9-bce3-45c1efba4d32 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:23.478147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b2d5e-8efb-44a9-bce3-45c1efba4d32 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:23.478463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b2d5e-8efb-44a9-bce3-45c1efba4d32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:23.489583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88b2d5e-8efb-44a9-bce3-45c1efba4d32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:23.491188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88b2d5e-8efb-44a9-bce3-45c1efba4d32 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:23.501891 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3489/6936] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:23.769633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:24.266417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:24.320540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a94d4-a3e2-42b9-ac42-b53ed6f4acc6 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:24.322613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-da6a94d4-a3e2-42b9-ac42-b53ed6f4acc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:24.341765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a94d4-a3e2-42b9-ac42-b53ed6f4acc6 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:24.349247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296b185-7ab5-40b3-89ac-625bafab8ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:24.351811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296b185-7ab5-40b3-89ac-625bafab8ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:24.352252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296b185-7ab5-40b3-89ac-625bafab8ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:24.352701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5296b185-7ab5-40b3-89ac-625bafab8ed4 admin admin] PATH_INFO: `/v3/projects/590912c81c474229b156eceeb104f8e6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:24.355042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296b185-7ab5-40b3-89ac-625bafab8ed4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=590912c81c474229b156eceeb104f8e6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:24.358400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5296b185-7ab5-40b3-89ac-625bafab8ed4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:24.524487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5296b185-7ab5-40b3-89ac-625bafab8ed4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '590912c81c474229b156eceeb104f8e6', 'request_id': 'req-5296b185-7ab5-40b3-89ac-625bafab8ed4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:24.530455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5296b185-7ab5-40b3-89ac-625bafab8ed4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '590912c81c474229b156eceeb104f8e6', 'request_id': 'req-5296b185-7ab5-40b3-89ac-625bafab8ed4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:24.649735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5296b185-7ab5-40b3-89ac-625bafab8ed4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '590912c81c474229b156eceeb104f8e6', 'request_id': 'req-5296b185-7ab5-40b3-89ac-625bafab8ed4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:24.664297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5296b185-7ab5-40b3-89ac-625bafab8ed4 admin admin] The token cache is being invalidate because project 590912c81c474229b156eceeb104f8e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:24.664683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5296b185-7ab5-40b3-89ac-625bafab8ed4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:24.665419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5296b185-7ab5-40b3-89ac-625bafab8ed4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:24.667141 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3448/6937] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:24 2022] DELETE /identity/v3/projects/590912c81c474229b156eceeb104f8e6 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:24.671764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b2d5e-8efb-44a9-bce3-45c1efba4d32 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:24.682954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b2d5e-8efb-44a9-bce3-45c1efba4d32 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:24.752180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b687f-5d14-4591-891a-abcf9c004668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:24.753230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b687f-5d14-4591-891a-abcf9c004668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:24.753525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b687f-5d14-4591-891a-abcf9c004668 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:24.758117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b687f-5d14-4591-891a-abcf9c004668 admin admin] PATH_INFO: `/v3/users/9fb5fb90eae145cb97c82bd6c195a557` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:24.758117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b687f-5d14-4591-891a-abcf9c004668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fb5fb90eae145cb97c82bd6c195a557)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:24.758117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b687f-5d14-4591-891a-abcf9c004668 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:24.930961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d33b687f-5d14-4591-891a-abcf9c004668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fb5fb90eae145cb97c82bd6c195a557'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:24.938977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d33b687f-5d14-4591-891a-abcf9c004668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fb5fb90eae145cb97c82bd6c195a557'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:24.939846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d33b687f-5d14-4591-891a-abcf9c004668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fb5fb90eae145cb97c82bd6c195a557'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:24.940536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d33b687f-5d14-4591-891a-abcf9c004668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fb5fb90eae145cb97c82bd6c195a557'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:25.020265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296b185-7ab5-40b3-89ac-625bafab8ed4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:25.021781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5296b185-7ab5-40b3-89ac-625bafab8ed4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:25.035611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5296b185-7ab5-40b3-89ac-625bafab8ed4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:25.038720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d33b687f-5d14-4591-891a-abcf9c004668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fb5fb90eae145cb97c82bd6c195a557'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:25.044190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec776e-1ed8-413a-8c81-21f18aca34c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:25.045294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec776e-1ed8-413a-8c81-21f18aca34c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:25.047642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec776e-1ed8-413a-8c81-21f18aca34c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:25.052208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec776e-1ed8-413a-8c81-21f18aca34c1 admin admin] PATH_INFO: `/v3/users/503314f28b4045f0bbaae8cdbfbfc989` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:25.053653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec776e-1ed8-413a-8c81-21f18aca34c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=503314f28b4045f0bbaae8cdbfbfc989)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:25.055735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec776e-1ed8-413a-8c81-21f18aca34c1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:25.059639 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3490/6938] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:24 2022] DELETE /identity/v3/users/9fb5fb90eae145cb97c82bd6c195a557 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:25.346014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-45ec776e-1ed8-413a-8c81-21f18aca34c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '503314f28b4045f0bbaae8cdbfbfc989'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:25.346860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-45ec776e-1ed8-413a-8c81-21f18aca34c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '503314f28b4045f0bbaae8cdbfbfc989'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:25.354903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-45ec776e-1ed8-413a-8c81-21f18aca34c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '503314f28b4045f0bbaae8cdbfbfc989'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:25.456556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-45ec776e-1ed8-413a-8c81-21f18aca34c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '503314f28b4045f0bbaae8cdbfbfc989'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:25.457197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-45ec776e-1ed8-413a-8c81-21f18aca34c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '503314f28b4045f0bbaae8cdbfbfc989'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:25.477896 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3449/6939] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:25 2022] DELETE /identity/v3/users/503314f28b4045f0bbaae8cdbfbfc989 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:25.483119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f279f-96cd-4c12-971d-0a5ab45602ca None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:25.483654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f279f-96cd-4c12-971d-0a5ab45602ca None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:25.483654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f279f-96cd-4c12-971d-0a5ab45602ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:25.502389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-371f279f-96cd-4c12-971d-0a5ab45602ca None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:25.504485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-371f279f-96cd-4c12-971d-0a5ab45602ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:25.575857 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3491/6940] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:28:25.586461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec776e-1ed8-413a-8c81-21f18aca34c1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:25.587890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-45ec776e-1ed8-413a-8c81-21f18aca34c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:25.674713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec776e-1ed8-413a-8c81-21f18aca34c1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:25.681772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9355e1-e16c-4a61-8a8a-e4ed317206ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:25.682780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9355e1-e16c-4a61-8a8a-e4ed317206ac service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:25.683237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9355e1-e16c-4a61-8a8a-e4ed317206ac service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:25.684197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9355e1-e16c-4a61-8a8a-e4ed317206ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:25.697185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9355e1-e16c-4a61-8a8a-e4ed317206ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:25.698945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9355e1-e16c-4a61-8a8a-e4ed317206ac service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:25.712941 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3450/6941] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:25.861472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:25.898958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f279f-96cd-4c12-971d-0a5ab45602ca None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:25.900656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-371f279f-96cd-4c12-971d-0a5ab45602ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:25.913056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f279f-96cd-4c12-971d-0a5ab45602ca None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:25.918170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2022390-5855-4e4e-ae24-f3973562c1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:25.919047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2022390-5855-4e4e-ae24-f3973562c1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:25.919335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2022390-5855-4e4e-ae24-f3973562c1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:25.919620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2022390-5855-4e4e-ae24-f3973562c1a2 admin admin] PATH_INFO: `/v3/projects/6b3edb1aac4d4b828b20fdb13304570f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:25.920843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2022390-5855-4e4e-ae24-f3973562c1a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b3edb1aac4d4b828b20fdb13304570f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:25.922822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2022390-5855-4e4e-ae24-f3973562c1a2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:26.021514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:26.066933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b2022390-5855-4e4e-ae24-f3973562c1a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b3edb1aac4d4b828b20fdb13304570f', 'request_id': 'req-b2022390-5855-4e4e-ae24-f3973562c1a2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:26.073714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b2022390-5855-4e4e-ae24-f3973562c1a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b3edb1aac4d4b828b20fdb13304570f', 'request_id': 'req-b2022390-5855-4e4e-ae24-f3973562c1a2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:26.181352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b2022390-5855-4e4e-ae24-f3973562c1a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b3edb1aac4d4b828b20fdb13304570f', 'request_id': 'req-b2022390-5855-4e4e-ae24-f3973562c1a2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:26.194178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b2022390-5855-4e4e-ae24-f3973562c1a2 admin admin] The token cache is being invalidate because project 6b3edb1aac4d4b828b20fdb13304570f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:26.194948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b2022390-5855-4e4e-ae24-f3973562c1a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:26.195330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b2022390-5855-4e4e-ae24-f3973562c1a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:26.197133 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3492/6942] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:25 2022] DELETE /identity/v3/projects/6b3edb1aac4d4b828b20fdb13304570f => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:26.202040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99706a8-c51a-494d-b33e-394a4c7f3335 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:26.202441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99706a8-c51a-494d-b33e-394a4c7f3335 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:26.202596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99706a8-c51a-494d-b33e-394a4c7f3335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:26.222264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c99706a8-c51a-494d-b33e-394a4c7f3335 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:26.223529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c99706a8-c51a-494d-b33e-394a4c7f3335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:26.249368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2022390-5855-4e4e-ae24-f3973562c1a2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:26.251275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b2022390-5855-4e4e-ae24-f3973562c1a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:26.330001 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3451/6943] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:26.337364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b549de-3563-4502-adc6-be3a573a6290 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:26.339605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b549de-3563-4502-adc6-be3a573a6290 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:26.339605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b549de-3563-4502-adc6-be3a573a6290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:26.367606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2022390-5855-4e4e-ae24-f3973562c1a2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:26.370129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:26.371191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:26.371642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:26.372085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add admin admin] PATH_INFO: `/v3/projects/810549f80c104ef787180ca456d45c81` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:26.373330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-11b549de-3563-4502-adc6-be3a573a6290 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:26.374054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add admin admin] RBAC: Authorizing `identity:delete_project(project_id=810549f80c104ef787180ca456d45c81)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:26.374709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-11b549de-3563-4502-adc6-be3a573a6290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:26.377090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:26.387213 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3452/6944] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:26.388494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b549de-3563-4502-adc6-be3a573a6290 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:26.398124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b549de-3563-4502-adc6-be3a573a6290 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:26.402869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42be0b-d386-4606-805e-7d1eb0f28ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:26.403529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42be0b-d386-4606-805e-7d1eb0f28ddf admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:26.403811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42be0b-d386-4606-805e-7d1eb0f28ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:26.404086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb42be0b-d386-4606-805e-7d1eb0f28ddf admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:26.404831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42be0b-d386-4606-805e-7d1eb0f28ddf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:28:26.405127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42be0b-d386-4606-805e-7d1eb0f28ddf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:26.406779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb42be0b-d386-4606-805e-7d1eb0f28ddf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:26.421982 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3453/6945] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:28:26.423067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42be0b-d386-4606-805e-7d1eb0f28ddf admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:26.434879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb42be0b-d386-4606-805e-7d1eb0f28ddf admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:26.440188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6553d-e5b1-4494-9799-d2dcc2ff3928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:26.440447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6553d-e5b1-4494-9799-d2dcc2ff3928 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:26.440636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6553d-e5b1-4494-9799-d2dcc2ff3928 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:26.440972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6553d-e5b1-4494-9799-d2dcc2ff3928 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:26.441669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6553d-e5b1-4494-9799-d2dcc2ff3928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:26.444418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6553d-e5b1-4494-9799-d2dcc2ff3928 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:26.497917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '810549f80c104ef787180ca456d45c81', 'request_id': 'req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:26.504404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '810549f80c104ef787180ca456d45c81', 'request_id': 'req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:26.544645 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3454/6946] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:28:26 2022] POST /identity/v3/projects => generated 367 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:26.550118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6553d-e5b1-4494-9799-d2dcc2ff3928 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:26.581442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '810549f80c104ef787180ca456d45c81', 'request_id': 'req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:26.599123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add admin admin] The token cache is being invalidate because project 810549f80c104ef787180ca456d45c81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:26.599430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:26.600128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:26.603131 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3493/6947] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:26 2022] DELETE /identity/v3/projects/810549f80c104ef787180ca456d45c81 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:26.607966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:26.610438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:26.643858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6553d-e5b1-4494-9799-d2dcc2ff3928 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:26.648977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb0368-57ef-47e9-86f8-99d8ae4475c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:26.649658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb0368-57ef-47e9-86f8-99d8ae4475c7 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:26.650113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb0368-57ef-47e9-86f8-99d8ae4475c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:26.650402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb0368-57ef-47e9-86f8-99d8ae4475c7 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:26.651237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb0368-57ef-47e9-86f8-99d8ae4475c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:26.653425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb0368-57ef-47e9-86f8-99d8ae4475c7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:26.691124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3aeb2-6cf4-4762-8cf4-21f1d93c6add admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:26.696028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11640f72-623e-4458-99a0-11a3c59754aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:26.696900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11640f72-623e-4458-99a0-11a3c59754aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:26.697183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11640f72-623e-4458-99a0-11a3c59754aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:26.697948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11640f72-623e-4458-99a0-11a3c59754aa admin admin] PATH_INFO: `/v3/users/aae76eca30504119b0fc9b95c285fce9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:26.698882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11640f72-623e-4458-99a0-11a3c59754aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=aae76eca30504119b0fc9b95c285fce9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:26.700992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11640f72-623e-4458-99a0-11a3c59754aa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:26.733686 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3455/6948] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:28:26 2022] POST /identity/v3/users => generated 284 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:28:26.735300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb0368-57ef-47e9-86f8-99d8ae4475c7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:26.737018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1fcb0368-57ef-47e9-86f8-99d8ae4475c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:26.757604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb0368-57ef-47e9-86f8-99d8ae4475c7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:26.758577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73379a41-7462-4c22-bfb1-25a0d17b4100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:26.759622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73379a41-7462-4c22-bfb1-25a0d17b4100 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:26.760117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73379a41-7462-4c22-bfb1-25a0d17b4100 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:26.760626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73379a41-7462-4c22-bfb1-25a0d17b4100 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:26.761573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73379a41-7462-4c22-bfb1-25a0d17b4100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:26.763843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73379a41-7462-4c22-bfb1-25a0d17b4100 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:26.777705 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3456/6949] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:28:26.789015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73379a41-7462-4c22-bfb1-25a0d17b4100 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:26.799201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73379a41-7462-4c22-bfb1-25a0d17b4100 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:26.806386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c07a9-76dd-4f04-997c-cc4afbcc9c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:26.807079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c07a9-76dd-4f04-997c-cc4afbcc9c4c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:26.807370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c07a9-76dd-4f04-997c-cc4afbcc9c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:26.807653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c07a9-76dd-4f04-997c-cc4afbcc9c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:26.808393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c07a9-76dd-4f04-997c-cc4afbcc9c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:26.810071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c07a9-76dd-4f04-997c-cc4afbcc9c4c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:26.820721 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3457/6950] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:28:26.824917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c07a9-76dd-4f04-997c-cc4afbcc9c4c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:26.836320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c07a9-76dd-4f04-997c-cc4afbcc9c4c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:26.849269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f269fc-c60d-4eff-9e65-d26c8c6731e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:26.850233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f269fc-c60d-4eff-9e65-d26c8c6731e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:26.850523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f269fc-c60d-4eff-9e65-d26c8c6731e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:26.850803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f269fc-c60d-4eff-9e65-d26c8c6731e7 admin admin] PATH_INFO: `/v3/projects/75c578e7e0564510a56e863631f0a526/users/4184b47f65294165a528eec274c97224/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:26.867628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f269fc-c60d-4eff-9e65-d26c8c6731e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75c578e7e0564510a56e863631f0a526, user_id=4184b47f65294165a528eec274c97224, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:26.871204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f269fc-c60d-4eff-9e65-d26c8c6731e7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:26.900592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-11640f72-623e-4458-99a0-11a3c59754aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aae76eca30504119b0fc9b95c285fce9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:26.908608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-11640f72-623e-4458-99a0-11a3c59754aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aae76eca30504119b0fc9b95c285fce9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:26.909342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-11640f72-623e-4458-99a0-11a3c59754aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aae76eca30504119b0fc9b95c285fce9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:26.910131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-11640f72-623e-4458-99a0-11a3c59754aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aae76eca30504119b0fc9b95c285fce9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:26.964180 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3458/6951] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:26 2022] PUT /identity/v3/projects/75c578e7e0564510a56e863631f0a526/users/4184b47f65294165a528eec274c97224/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:26.975487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954dc35-acc8-4abc-bd93-1a73b3df51ea None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:26.975783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954dc35-acc8-4abc-bd93-1a73b3df51ea None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:26.976062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954dc35-acc8-4abc-bd93-1a73b3df51ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:26.985995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-11640f72-623e-4458-99a0-11a3c59754aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aae76eca30504119b0fc9b95c285fce9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:27.011870 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3494/6952] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:26 2022] DELETE /identity/v3/users/aae76eca30504119b0fc9b95c285fce9 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:27.015203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b954dc35-acc8-4abc-bd93-1a73b3df51ea None None] MFA Rules not processed for user `4184b47f65294165a528eec274c97224`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:27.016481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b954dc35-acc8-4abc-bd93-1a73b3df51ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:27.118268 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3459/6953] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:28:27.119385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11640f72-623e-4458-99a0-11a3c59754aa admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:27.121039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-11640f72-623e-4458-99a0-11a3c59754aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:27.197639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11640f72-623e-4458-99a0-11a3c59754aa admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:27.203399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5ba9c-73d0-4321-b939-ae79957ed5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:27.203634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5ba9c-73d0-4321-b939-ae79957ed5ab admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:27.204447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5ba9c-73d0-4321-b939-ae79957ed5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:27.204579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5ba9c-73d0-4321-b939-ae79957ed5ab admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:27.205338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5ba9c-73d0-4321-b939-ae79957ed5ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:27.207454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5ba9c-73d0-4321-b939-ae79957ed5ab admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:27.322686 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3495/6954] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:28:27 2022] POST /identity/v3/projects => generated 369 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:27.323890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954dc35-acc8-4abc-bd93-1a73b3df51ea None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:27.336524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954dc35-acc8-4abc-bd93-1a73b3df51ea None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:27.456162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9965de-1fe4-4dc7-9120-7c4145183d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:27.457279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9965de-1fe4-4dc7-9120-7c4145183d3c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:27.458341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9965de-1fe4-4dc7-9120-7c4145183d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:27.458708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9965de-1fe4-4dc7-9120-7c4145183d3c admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:27.459515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9965de-1fe4-4dc7-9120-7c4145183d3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:27.463756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9965de-1fe4-4dc7-9120-7c4145183d3c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:27.516496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5ba9c-73d0-4321-b939-ae79957ed5ab admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:27.519292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1fd5ba9c-73d0-4321-b939-ae79957ed5ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:27.536710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5ba9c-73d0-4321-b939-ae79957ed5ab admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:27.547225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde99be-6a61-412f-8f12-d2ef1ad71744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:27.547225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde99be-6a61-412f-8f12-d2ef1ad71744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:27.547225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde99be-6a61-412f-8f12-d2ef1ad71744 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:27.547225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde99be-6a61-412f-8f12-d2ef1ad71744 admin admin] PATH_INFO: `/v3/projects/9b41bb4bd67b4f9a8335db5864198100` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:27.549385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde99be-6a61-412f-8f12-d2ef1ad71744 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b41bb4bd67b4f9a8335db5864198100)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:27.552461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde99be-6a61-412f-8f12-d2ef1ad71744 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:27.592151 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3460/6955] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:28:27 2022] POST /identity/v3/users => generated 291 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:27.596372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9965de-1fe4-4dc7-9120-7c4145183d3c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:27.614401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9965de-1fe4-4dc7-9120-7c4145183d3c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:27.619531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37561e8e-97ff-4b0d-9ab7-447e274a11f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:27.620332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37561e8e-97ff-4b0d-9ab7-447e274a11f0 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:27.620679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37561e8e-97ff-4b0d-9ab7-447e274a11f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:27.621022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37561e8e-97ff-4b0d-9ab7-447e274a11f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:27.621821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37561e8e-97ff-4b0d-9ab7-447e274a11f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:27.623598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37561e8e-97ff-4b0d-9ab7-447e274a11f0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:27.637589 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3461/6956] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:27.638622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37561e8e-97ff-4b0d-9ab7-447e274a11f0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:27.649403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37561e8e-97ff-4b0d-9ab7-447e274a11f0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:27.656098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b657ccb-3347-4c21-b5ae-5eaf569d4f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:27.657112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b657ccb-3347-4c21-b5ae-5eaf569d4f7b admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:27.657424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b657ccb-3347-4c21-b5ae-5eaf569d4f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:27.657876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b657ccb-3347-4c21-b5ae-5eaf569d4f7b admin admin] PATH_INFO: `/v3/projects/cb0424fd0ef14290889aa14ab316ff9f/users/504b13af4c4d49cebac0741a73fedc0b/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:27.680418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b657ccb-3347-4c21-b5ae-5eaf569d4f7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb0424fd0ef14290889aa14ab316ff9f, user_id=504b13af4c4d49cebac0741a73fedc0b, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:27.683685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b657ccb-3347-4c21-b5ae-5eaf569d4f7b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:27.696369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:27.766414 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3462/6957] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:27 2022] PUT /identity/v3/projects/cb0424fd0ef14290889aa14ab316ff9f/users/504b13af4c4d49cebac0741a73fedc0b/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:27.767800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b657ccb-3347-4c21-b5ae-5eaf569d4f7b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:27.790466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2bde99be-6a61-412f-8f12-d2ef1ad71744 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b41bb4bd67b4f9a8335db5864198100', 'request_id': 'req-2bde99be-6a61-412f-8f12-d2ef1ad71744'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:27.804219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2bde99be-6a61-412f-8f12-d2ef1ad71744 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b41bb4bd67b4f9a8335db5864198100', 'request_id': 'req-2bde99be-6a61-412f-8f12-d2ef1ad71744'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:27.852477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b657ccb-3347-4c21-b5ae-5eaf569d4f7b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:27.903851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2bde99be-6a61-412f-8f12-d2ef1ad71744 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b41bb4bd67b4f9a8335db5864198100', 'request_id': 'req-2bde99be-6a61-412f-8f12-d2ef1ad71744'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:27.919961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a38e3-4484-4ff8-be36-329c53fff6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:27.920647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a38e3-4484-4ff8-be36-329c53fff6f2 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:27.920937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a38e3-4484-4ff8-be36-329c53fff6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:27.921063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2bde99be-6a61-412f-8f12-d2ef1ad71744 admin admin] The token cache is being invalidate because project 9b41bb4bd67b4f9a8335db5864198100 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:27.921242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a38e3-4484-4ff8-be36-329c53fff6f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:27.921546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2bde99be-6a61-412f-8f12-d2ef1ad71744 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:27.922172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a38e3-4484-4ff8-be36-329c53fff6f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:27.922301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2bde99be-6a61-412f-8f12-d2ef1ad71744 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:27.924078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a38e3-4484-4ff8-be36-329c53fff6f2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:27.924208 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3496/6958] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:27 2022] DELETE /identity/v3/projects/9b41bb4bd67b4f9a8335db5864198100 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:27.930310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99a630-4333-4c76-867b-b1c7f9b63fa8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:27.930607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99a630-4333-4c76-867b-b1c7f9b63fa8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:27.930894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef99a630-4333-4c76-867b-b1c7f9b63fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:27.951271 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3463/6959] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:28:27.953764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a38e3-4484-4ff8-be36-329c53fff6f2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:27.955836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-622a38e3-4484-4ff8-be36-329c53fff6f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:27.961628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ef99a630-4333-4c76-867b-b1c7f9b63fa8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:27.963050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ef99a630-4333-4c76-867b-b1c7f9b63fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:27.971034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:27.981421 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3497/6960] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:27.983315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193dcb8-5222-4631-a99c-56f717d84584 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:27.983760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193dcb8-5222-4631-a99c-56f717d84584 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:27.984057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193dcb8-5222-4631-a99c-56f717d84584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:27.984355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a38e3-4484-4ff8-be36-329c53fff6f2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:27.996872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be80c594-1547-4685-a6c2-84c109d56b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:28.000384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80c594-1547-4685-a6c2-84c109d56b0d admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:28.000384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80c594-1547-4685-a6c2-84c109d56b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:28.000384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be80c594-1547-4685-a6c2-84c109d56b0d admin admin] PATH_INFO: `/v3/projects/cb0424fd0ef14290889aa14ab316ff9f/users/504b13af4c4d49cebac0741a73fedc0b/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:28.007010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80c594-1547-4685-a6c2-84c109d56b0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb0424fd0ef14290889aa14ab316ff9f, user_id=504b13af4c4d49cebac0741a73fedc0b, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:28.008328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4193dcb8-5222-4631-a99c-56f717d84584 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:28.009732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4193dcb8-5222-4631-a99c-56f717d84584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:28.010022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be80c594-1547-4685-a6c2-84c109d56b0d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:28.024194 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3498/6961] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:28.036457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193dcb8-5222-4631-a99c-56f717d84584 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:28.040583 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3464/6962] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:27 2022] PUT /identity/v3/projects/cb0424fd0ef14290889aa14ab316ff9f/users/504b13af4c4d49cebac0741a73fedc0b/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:28:28.051255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576fc0e-0158-41b8-95fa-ed209e88628f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:28.052171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576fc0e-0158-41b8-95fa-ed209e88628f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:28.052365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d576fc0e-0158-41b8-95fa-ed209e88628f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:28.070972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193dcb8-5222-4631-a99c-56f717d84584 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:28.112548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d576fc0e-0158-41b8-95fa-ed209e88628f None None] MFA Rules not processed for user `504b13af4c4d49cebac0741a73fedc0b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:28.115057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d576fc0e-0158-41b8-95fa-ed209e88628f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:28.183530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cb376-5d0e-4298-9dff-9e07c645c61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:28.185543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cb376-5d0e-4298-9dff-9e07c645c61c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:28.186491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cb376-5d0e-4298-9dff-9e07c645c61c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:28.187094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cb376-5d0e-4298-9dff-9e07c645c61c admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:28.188110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461cb376-5d0e-4298-9dff-9e07c645c61c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:28:28.188659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461cb376-5d0e-4298-9dff-9e07c645c61c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:28.191780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461cb376-5d0e-4298-9dff-9e07c645c61c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:28.210019 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3499/6963] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:28:28.214378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cb376-5d0e-4298-9dff-9e07c645c61c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:28.247504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cb376-5d0e-4298-9dff-9e07c645c61c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:28.255949 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3465/6964] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:28 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:28:28.271416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576fc0e-0158-41b8-95fa-ed209e88628f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:28.272712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad027f-253e-4ab5-917b-bddcd1bc3917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:28.273014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d576fc0e-0158-41b8-95fa-ed209e88628f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:28.273406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad027f-253e-4ab5-917b-bddcd1bc3917 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:28.273697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad027f-253e-4ab5-917b-bddcd1bc3917 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:28.274003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad027f-253e-4ab5-917b-bddcd1bc3917 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:28.274708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad027f-253e-4ab5-917b-bddcd1bc3917 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:28.277900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad027f-253e-4ab5-917b-bddcd1bc3917 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:28.366522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d576fc0e-0158-41b8-95fa-ed209e88628f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:28.371359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c4f66-084a-4963-88cf-442da417d3f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:28.371601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c4f66-084a-4963-88cf-442da417d3f7 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:28.371852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c4f66-084a-4963-88cf-442da417d3f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:28.372108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c4f66-084a-4963-88cf-442da417d3f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:28.383302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c4f66-084a-4963-88cf-442da417d3f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:28.385171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416c4f66-084a-4963-88cf-442da417d3f7 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:28.395723 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3466/6965] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:28 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:28:28.440968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62af64-ffd0-4363-94b4-6c0975ab3246 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:28.440968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62af64-ffd0-4363-94b4-6c0975ab3246 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:28.440968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62af64-ffd0-4363-94b4-6c0975ab3246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:28.443250 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3500/6966] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:28:28 2022] POST /identity/v3/projects => generated 375 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:28.450612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad027f-253e-4ab5-917b-bddcd1bc3917 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:28.468728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6f62af64-ffd0-4363-94b4-6c0975ab3246 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:28.471163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6f62af64-ffd0-4363-94b4-6c0975ab3246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:28.541251 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3467/6967] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:28:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:28:28.541480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad027f-253e-4ab5-917b-bddcd1bc3917 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:28.549576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3fd32-5859-4486-a409-c434ce5238e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:28.550653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3fd32-5859-4486-a409-c434ce5238e5 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:28.550945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3fd32-5859-4486-a409-c434ce5238e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:28.551229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3fd32-5859-4486-a409-c434ce5238e5 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:28.552215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe3fd32-5859-4486-a409-c434ce5238e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:28.554719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe3fd32-5859-4486-a409-c434ce5238e5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:28.570899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62af64-ffd0-4363-94b4-6c0975ab3246 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:28.572411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6f62af64-ffd0-4363-94b4-6c0975ab3246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:28.643657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62af64-ffd0-4363-94b4-6c0975ab3246 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:28.648998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64760692-9bd1-4bbf-b5f6-d649526188d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:28.649669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64760692-9bd1-4bbf-b5f6-d649526188d6 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:28.649982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64760692-9bd1-4bbf-b5f6-d649526188d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:28.650293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64760692-9bd1-4bbf-b5f6-d649526188d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:28.652193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-64760692-9bd1-4bbf-b5f6-d649526188d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:28.683377 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3501/6968] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:28:28 2022] POST /identity/v3/users => generated 288 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:28.684483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3fd32-5859-4486-a409-c434ce5238e5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:28.709190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3fd32-5859-4486-a409-c434ce5238e5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:28.714800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1c7ce-0c95-41ea-90bc-4cf7d624bf87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:28.715737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1c7ce-0c95-41ea-90bc-4cf7d624bf87 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:28.716132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1c7ce-0c95-41ea-90bc-4cf7d624bf87 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:28.716438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1c7ce-0c95-41ea-90bc-4cf7d624bf87 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:28.717251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1c7ce-0c95-41ea-90bc-4cf7d624bf87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:28.719524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae1c7ce-0c95-41ea-90bc-4cf7d624bf87 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:28.734233 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3502/6969] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:28:28.735127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64760692-9bd1-4bbf-b5f6-d649526188d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:28.735482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1c7ce-0c95-41ea-90bc-4cf7d624bf87 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:28.737210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64760692-9bd1-4bbf-b5f6-d649526188d6 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:28.737372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6ae1c7ce-0c95-41ea-90bc-4cf7d624bf87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:28.749616 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3468/6970] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:28.750819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64760692-9bd1-4bbf-b5f6-d649526188d6 service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:28.765119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64760692-9bd1-4bbf-b5f6-d649526188d6 service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:28.771954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9ddf8-c234-4fd9-9481-0e9d710c8155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:28.773049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9ddf8-c234-4fd9-9481-0e9d710c8155 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:28.773502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9ddf8-c234-4fd9-9481-0e9d710c8155 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:28.774154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9ddf8-c234-4fd9-9481-0e9d710c8155 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:28.775261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9ddf8-c234-4fd9-9481-0e9d710c8155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:28.777423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a9ddf8-c234-4fd9-9481-0e9d710c8155 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:28.795538 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3469/6971] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:28.796602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9ddf8-c234-4fd9-9481-0e9d710c8155 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:28.809276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a9ddf8-c234-4fd9-9481-0e9d710c8155 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:28.816380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ef31c-7f48-43cf-b1e4-8c32941707fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:28.817481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ef31c-7f48-43cf-b1e4-8c32941707fb admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:28.817772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ef31c-7f48-43cf-b1e4-8c32941707fb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:28.818079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ef31c-7f48-43cf-b1e4-8c32941707fb admin admin] PATH_INFO: `/v3/projects/7b6c916fa5074323b5c9a3e959f2f1e7/users/a9ca0f87ca18428fbdc5ae0eb5329342/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:28.841193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ef31c-7f48-43cf-b1e4-8c32941707fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b6c916fa5074323b5c9a3e959f2f1e7, user_id=a9ca0f87ca18428fbdc5ae0eb5329342, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:28.844312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ef31c-7f48-43cf-b1e4-8c32941707fb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:28.863110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1c7ce-0c95-41ea-90bc-4cf7d624bf87 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:28.869597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8b45d-d996-485d-81ab-dbccde758417 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:28.870446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8b45d-d996-485d-81ab-dbccde758417 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:28.871168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8b45d-d996-485d-81ab-dbccde758417 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:28.871168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8b45d-d996-485d-81ab-dbccde758417 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:28.885545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8b45d-d996-485d-81ab-dbccde758417 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:28.888394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8b45d-d996-485d-81ab-dbccde758417 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:28.904684 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3503/6972] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:28:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:28:28.952535 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3470/6973] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:28 2022] PUT /identity/v3/projects/7b6c916fa5074323b5c9a3e959f2f1e7/users/a9ca0f87ca18428fbdc5ae0eb5329342/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:28:28.963693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9a127-e30e-414b-9d92-86bd84e7931b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:28.963993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9a127-e30e-414b-9d92-86bd84e7931b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:28.964275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9a127-e30e-414b-9d92-86bd84e7931b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:29.034925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cae9a127-e30e-414b-9d92-86bd84e7931b None None] MFA Rules not processed for user `a9ca0f87ca18428fbdc5ae0eb5329342`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:29.041693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cae9a127-e30e-414b-9d92-86bd84e7931b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:29.240962 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3504/6974] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:29.242852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ef31c-7f48-43cf-b1e4-8c32941707fb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:29.354146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ef31c-7f48-43cf-b1e4-8c32941707fb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:29.368420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454653f5-5aa8-4d71-9452-38732bb7bbfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:29.369452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454653f5-5aa8-4d71-9452-38732bb7bbfb service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:29.370382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454653f5-5aa8-4d71-9452-38732bb7bbfb service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:29.370676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454653f5-5aa8-4d71-9452-38732bb7bbfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:29.388129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454653f5-5aa8-4d71-9452-38732bb7bbfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:29.389774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454653f5-5aa8-4d71-9452-38732bb7bbfb service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:29.402864 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3471/6975] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:30.444517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:30.702256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:32.787655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:33.024152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:34.134436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:34.380176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:34.856978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9a127-e30e-414b-9d92-86bd84e7931b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:34.969900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9a127-e30e-414b-9d92-86bd84e7931b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:34.987472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d987af2-ddc1-41f4-90d8-7098bb680f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:34.988499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d987af2-ddc1-41f4-90d8-7098bb680f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:34.990369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d987af2-ddc1-41f4-90d8-7098bb680f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:34.990751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d987af2-ddc1-41f4-90d8-7098bb680f21 admin admin] PATH_INFO: `/v3/users/4184b47f65294165a528eec274c97224` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:34.994716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d987af2-ddc1-41f4-90d8-7098bb680f21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4184b47f65294165a528eec274c97224)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:34.997149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d987af2-ddc1-41f4-90d8-7098bb680f21 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:35.376267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8d987af2-ddc1-41f4-90d8-7098bb680f21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4184b47f65294165a528eec274c97224'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:35.394910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8d987af2-ddc1-41f4-90d8-7098bb680f21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4184b47f65294165a528eec274c97224'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:35.396006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8d987af2-ddc1-41f4-90d8-7098bb680f21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4184b47f65294165a528eec274c97224'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:35.396558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8d987af2-ddc1-41f4-90d8-7098bb680f21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4184b47f65294165a528eec274c97224'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:35.506906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8d987af2-ddc1-41f4-90d8-7098bb680f21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4184b47f65294165a528eec274c97224'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:35.536830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:35.559636 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3505/6976] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:34 2022] DELETE /identity/v3/users/4184b47f65294165a528eec274c97224 => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:35.564426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedee6d-2cdb-44d4-8eb5-ce5df4fbc1ce None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:35.565787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedee6d-2cdb-44d4-8eb5-ce5df4fbc1ce None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:35.566405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedee6d-2cdb-44d4-8eb5-ce5df4fbc1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:35.607440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3dedee6d-2cdb-44d4-8eb5-ce5df4fbc1ce None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:35.608725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3dedee6d-2cdb-44d4-8eb5-ce5df4fbc1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:35.782384 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3472/6977] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:35.787636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:35.856525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d987af2-ddc1-41f4-90d8-7098bb680f21 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:35.859112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8d987af2-ddc1-41f4-90d8-7098bb680f21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:36.017570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d987af2-ddc1-41f4-90d8-7098bb680f21 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:36.050954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9551a67-d86b-4830-a05d-f9d9fbe30b87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:36.051937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9551a67-d86b-4830-a05d-f9d9fbe30b87 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:36.052610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9551a67-d86b-4830-a05d-f9d9fbe30b87 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:36.053075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9551a67-d86b-4830-a05d-f9d9fbe30b87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:36.082021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9551a67-d86b-4830-a05d-f9d9fbe30b87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:36.088891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9551a67-d86b-4830-a05d-f9d9fbe30b87 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:36.121758 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3506/6978] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:36.237658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:36.530125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:36.774978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedee6d-2cdb-44d4-8eb5-ce5df4fbc1ce None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:36.776646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3dedee6d-2cdb-44d4-8eb5-ce5df4fbc1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:36.799060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedee6d-2cdb-44d4-8eb5-ce5df4fbc1ce None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:36.806883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c1af1-cbde-470b-8836-6f299fc3fd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:36.809321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c1af1-cbde-470b-8836-6f299fc3fd4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:36.809965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c1af1-cbde-470b-8836-6f299fc3fd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:36.810434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c1af1-cbde-470b-8836-6f299fc3fd4d admin admin] PATH_INFO: `/v3/users/d68d5f72a3f9466cb9d119b4a9521708` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:36.812665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c1af1-cbde-470b-8836-6f299fc3fd4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d68d5f72a3f9466cb9d119b4a9521708)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:36.817603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c1af1-cbde-470b-8836-6f299fc3fd4d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:36.973901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9551a67-d86b-4830-a05d-f9d9fbe30b87 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:36.975571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b9551a67-d86b-4830-a05d-f9d9fbe30b87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:36.995769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9551a67-d86b-4830-a05d-f9d9fbe30b87 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:37.014537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff26979-ba10-44d8-8578-f17cc50bb4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:37.015703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff26979-ba10-44d8-8578-f17cc50bb4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:37.016167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff26979-ba10-44d8-8578-f17cc50bb4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:37.017231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff26979-ba10-44d8-8578-f17cc50bb4d8 admin admin] PATH_INFO: `/v3/projects/75c578e7e0564510a56e863631f0a526` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:37.018902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff26979-ba10-44d8-8578-f17cc50bb4d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75c578e7e0564510a56e863631f0a526)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:37.021555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff26979-ba10-44d8-8578-f17cc50bb4d8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:37.026155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3c7c1af1-cbde-470b-8836-6f299fc3fd4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd68d5f72a3f9466cb9d119b4a9521708'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:37.036102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3c7c1af1-cbde-470b-8836-6f299fc3fd4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd68d5f72a3f9466cb9d119b4a9521708'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:37.049424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3c7c1af1-cbde-470b-8836-6f299fc3fd4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd68d5f72a3f9466cb9d119b4a9521708'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:37.147895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3c7c1af1-cbde-470b-8836-6f299fc3fd4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd68d5f72a3f9466cb9d119b4a9521708'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:37.148550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3c7c1af1-cbde-470b-8836-6f299fc3fd4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd68d5f72a3f9466cb9d119b4a9521708'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:37.185553 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3473/6979] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:36 2022] DELETE /identity/v3/users/d68d5f72a3f9466cb9d119b4a9521708 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:37.197091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440fb40-bb01-4533-b49e-a6a1f7333e7e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:37.197091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440fb40-bb01-4533-b49e-a6a1f7333e7e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:37.197091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440fb40-bb01-4533-b49e-a6a1f7333e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:37.219915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c440fb40-bb01-4533-b49e-a6a1f7333e7e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:37.219915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c440fb40-bb01-4533-b49e-a6a1f7333e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:37.289092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:37.294361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bff26979-ba10-44d8-8578-f17cc50bb4d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75c578e7e0564510a56e863631f0a526', 'request_id': 'req-bff26979-ba10-44d8-8578-f17cc50bb4d8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:37.330792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bff26979-ba10-44d8-8578-f17cc50bb4d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75c578e7e0564510a56e863631f0a526', 'request_id': 'req-bff26979-ba10-44d8-8578-f17cc50bb4d8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:37.374985 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3474/6980] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:37.397410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440fb40-bb01-4533-b49e-a6a1f7333e7e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:37.398983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c440fb40-bb01-4533-b49e-a6a1f7333e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:37.415426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bff26979-ba10-44d8-8578-f17cc50bb4d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75c578e7e0564510a56e863631f0a526', 'request_id': 'req-bff26979-ba10-44d8-8578-f17cc50bb4d8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:37.435745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bff26979-ba10-44d8-8578-f17cc50bb4d8 admin admin] The token cache is being invalidate because project 75c578e7e0564510a56e863631f0a526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:37.436026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bff26979-ba10-44d8-8578-f17cc50bb4d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:37.439094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-bff26979-ba10-44d8-8578-f17cc50bb4d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:37.441037 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3507/6981] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:36 2022] DELETE /identity/v3/projects/75c578e7e0564510a56e863631f0a526 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:37.446648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff26979-ba10-44d8-8578-f17cc50bb4d8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:37.450139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bff26979-ba10-44d8-8578-f17cc50bb4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:37.472653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff26979-ba10-44d8-8578-f17cc50bb4d8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:37.487041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed2f6e-c66b-4134-b0ae-c01c0b4f9aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:37.500894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed2f6e-c66b-4134-b0ae-c01c0b4f9aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:37.500894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed2f6e-c66b-4134-b0ae-c01c0b4f9aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:37.500894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed2f6e-c66b-4134-b0ae-c01c0b4f9aaa admin admin] PATH_INFO: `/v3/users/504b13af4c4d49cebac0741a73fedc0b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:37.500894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed2f6e-c66b-4134-b0ae-c01c0b4f9aaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=504b13af4c4d49cebac0741a73fedc0b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:37.500894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed2f6e-c66b-4134-b0ae-c01c0b4f9aaa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:37.512207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440fb40-bb01-4533-b49e-a6a1f7333e7e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:37.520361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b23ce-9d68-4a9f-b7dc-70e12d98a831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:37.521678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b23ce-9d68-4a9f-b7dc-70e12d98a831 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:37.522015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b23ce-9d68-4a9f-b7dc-70e12d98a831 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:37.522300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b23ce-9d68-4a9f-b7dc-70e12d98a831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:37.525679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-960b23ce-9d68-4a9f-b7dc-70e12d98a831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:37.544435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b23ce-9d68-4a9f-b7dc-70e12d98a831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:37.547533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b23ce-9d68-4a9f-b7dc-70e12d98a831 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:37.571963 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3475/6982] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:28:37.845337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-69ed2f6e-c66b-4134-b0ae-c01c0b4f9aaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '504b13af4c4d49cebac0741a73fedc0b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:37.861580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-69ed2f6e-c66b-4134-b0ae-c01c0b4f9aaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '504b13af4c4d49cebac0741a73fedc0b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:37.863276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-69ed2f6e-c66b-4134-b0ae-c01c0b4f9aaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '504b13af4c4d49cebac0741a73fedc0b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:37.864189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-69ed2f6e-c66b-4134-b0ae-c01c0b4f9aaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '504b13af4c4d49cebac0741a73fedc0b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:38.002603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-69ed2f6e-c66b-4134-b0ae-c01c0b4f9aaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '504b13af4c4d49cebac0741a73fedc0b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:38.050393 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3508/6983] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:37 2022] DELETE /identity/v3/users/504b13af4c4d49cebac0741a73fedc0b => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:38.063868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:38.434077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b23ce-9d68-4a9f-b7dc-70e12d98a831 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:38.436011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-960b23ce-9d68-4a9f-b7dc-70e12d98a831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:38.466709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b23ce-9d68-4a9f-b7dc-70e12d98a831 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:38.583125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76eff5a-7055-4af7-a8ea-ec1af4c45a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:38.584242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76eff5a-7055-4af7-a8ea-ec1af4c45a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:38.584705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76eff5a-7055-4af7-a8ea-ec1af4c45a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:38.585229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76eff5a-7055-4af7-a8ea-ec1af4c45a53 admin admin] PATH_INFO: `/v3/projects/78489a3cef704363834055047d38d6c4` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:38.586727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76eff5a-7055-4af7-a8ea-ec1af4c45a53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78489a3cef704363834055047d38d6c4)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:38.594080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76eff5a-7055-4af7-a8ea-ec1af4c45a53 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:38.738233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b76eff5a-7055-4af7-a8ea-ec1af4c45a53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78489a3cef704363834055047d38d6c4', 'request_id': 'req-b76eff5a-7055-4af7-a8ea-ec1af4c45a53'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:38.746961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b76eff5a-7055-4af7-a8ea-ec1af4c45a53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78489a3cef704363834055047d38d6c4', 'request_id': 'req-b76eff5a-7055-4af7-a8ea-ec1af4c45a53'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:38.815829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b76eff5a-7055-4af7-a8ea-ec1af4c45a53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78489a3cef704363834055047d38d6c4', 'request_id': 'req-b76eff5a-7055-4af7-a8ea-ec1af4c45a53'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:38.835093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b76eff5a-7055-4af7-a8ea-ec1af4c45a53 admin admin] The token cache is being invalidate because project 78489a3cef704363834055047d38d6c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:38.835537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b76eff5a-7055-4af7-a8ea-ec1af4c45a53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:38.844963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b76eff5a-7055-4af7-a8ea-ec1af4c45a53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:38.849101 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3476/6984] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:38 2022] DELETE /identity/v3/projects/78489a3cef704363834055047d38d6c4 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:38.854644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909f0de-8f63-425d-a1b1-0e197281ac7a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:38.855365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909f0de-8f63-425d-a1b1-0e197281ac7a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:38.855966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8909f0de-8f63-425d-a1b1-0e197281ac7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:38.907026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:38.949050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8909f0de-8f63-425d-a1b1-0e197281ac7a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:38.954623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8909f0de-8f63-425d-a1b1-0e197281ac7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:39.063077 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3509/6985] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:28:39.063743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef8fca-0720-48f2-bd85-2617341cc466 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:39.063976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef8fca-0720-48f2-bd85-2617341cc466 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:39.064204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef8fca-0720-48f2-bd85-2617341cc466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:39.100047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a5ef8fca-0720-48f2-bd85-2617341cc466 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:39.107595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a5ef8fca-0720-48f2-bd85-2617341cc466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:39.122531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909f0de-8f63-425d-a1b1-0e197281ac7a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:39.124301 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3477/6986] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:39.140442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8909f0de-8f63-425d-a1b1-0e197281ac7a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:39.149982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2a5e7-289f-4d02-b35b-3e7c3810efdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:39.150985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2a5e7-289f-4d02-b35b-3e7c3810efdf admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:39.151457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2a5e7-289f-4d02-b35b-3e7c3810efdf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:39.151990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2a5e7-289f-4d02-b35b-3e7c3810efdf admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:39.153010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2a5e7-289f-4d02-b35b-3e7c3810efdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:28:39.153509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2a5e7-289f-4d02-b35b-3e7c3810efdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:39.155790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2a5e7-289f-4d02-b35b-3e7c3810efdf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:39.193012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef8fca-0720-48f2-bd85-2617341cc466 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:39.200297 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3510/6987] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:28:39.208119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef8fca-0720-48f2-bd85-2617341cc466 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:39.213998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16513572-eced-45bb-9c77-7c0b3ddf8a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:39.215312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16513572-eced-45bb-9c77-7c0b3ddf8a99 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:39.216103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16513572-eced-45bb-9c77-7c0b3ddf8a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:39.216273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16513572-eced-45bb-9c77-7c0b3ddf8a99 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:39.217117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16513572-eced-45bb-9c77-7c0b3ddf8a99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:39.219436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16513572-eced-45bb-9c77-7c0b3ddf8a99 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:39.260800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2a5e7-289f-4d02-b35b-3e7c3810efdf admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:39.262873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b6c2a5e7-289f-4d02-b35b-3e7c3810efdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:39.277770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2a5e7-289f-4d02-b35b-3e7c3810efdf admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:39.286145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c3470-9d5a-45cb-8daf-d4d01a49dd0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:39.286145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c3470-9d5a-45cb-8daf-d4d01a49dd0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:39.286779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c3470-9d5a-45cb-8daf-d4d01a49dd0c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:39.287389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c3470-9d5a-45cb-8daf-d4d01a49dd0c admin admin] PATH_INFO: `/v3/projects/cb0424fd0ef14290889aa14ab316ff9f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:39.289626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c3470-9d5a-45cb-8daf-d4d01a49dd0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb0424fd0ef14290889aa14ab316ff9f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:39.292842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c3470-9d5a-45cb-8daf-d4d01a49dd0c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:39.322312 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3478/6988] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:28:39 2022] POST /identity/v3/projects => generated 365 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:39.323294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16513572-eced-45bb-9c77-7c0b3ddf8a99 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:39.420563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b44c3470-9d5a-45cb-8daf-d4d01a49dd0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb0424fd0ef14290889aa14ab316ff9f', 'request_id': 'req-b44c3470-9d5a-45cb-8daf-d4d01a49dd0c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:39.433182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b44c3470-9d5a-45cb-8daf-d4d01a49dd0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb0424fd0ef14290889aa14ab316ff9f', 'request_id': 'req-b44c3470-9d5a-45cb-8daf-d4d01a49dd0c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:39.456402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16513572-eced-45bb-9c77-7c0b3ddf8a99 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:39.531150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4d05a-3fd4-4876-af38-1a3991820680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:39.532435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4d05a-3fd4-4876-af38-1a3991820680 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:39.532727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4d05a-3fd4-4876-af38-1a3991820680 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:39.533575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4d05a-3fd4-4876-af38-1a3991820680 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:39.534526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4d05a-3fd4-4876-af38-1a3991820680 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:39.537874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d4d05a-3fd4-4876-af38-1a3991820680 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:39.575427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b44c3470-9d5a-45cb-8daf-d4d01a49dd0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb0424fd0ef14290889aa14ab316ff9f', 'request_id': 'req-b44c3470-9d5a-45cb-8daf-d4d01a49dd0c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:39.590104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b44c3470-9d5a-45cb-8daf-d4d01a49dd0c admin admin] The token cache is being invalidate because project cb0424fd0ef14290889aa14ab316ff9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:39.590734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b44c3470-9d5a-45cb-8daf-d4d01a49dd0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:39.591673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b44c3470-9d5a-45cb-8daf-d4d01a49dd0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:39.593923 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3511/6989] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:39 2022] DELETE /identity/v3/projects/cb0424fd0ef14290889aa14ab316ff9f => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:39.610265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98a2db-536f-489d-9e22-abaf3b8bfc28 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:39.610865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98a2db-536f-489d-9e22-abaf3b8bfc28 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:39.611908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98a2db-536f-489d-9e22-abaf3b8bfc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:39.640896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1a98a2db-536f-489d-9e22-abaf3b8bfc28 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:39.642613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1a98a2db-536f-489d-9e22-abaf3b8bfc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:39.659174 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3479/6990] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:28:39 2022] POST /identity/v3/users => generated 283 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:39.663797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4d05a-3fd4-4876-af38-1a3991820680 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:39.670763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a4d4d05a-3fd4-4876-af38-1a3991820680 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:39.686042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:39.688094 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3512/6991] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:39.696724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4d05a-3fd4-4876-af38-1a3991820680 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:39.714930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16555023-d7a0-4947-b2e3-934eb3041d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:39.716905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16555023-d7a0-4947-b2e3-934eb3041d96 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:39.717306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16555023-d7a0-4947-b2e3-934eb3041d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:39.717691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16555023-d7a0-4947-b2e3-934eb3041d96 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:39.718609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16555023-d7a0-4947-b2e3-934eb3041d96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:39.722009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16555023-d7a0-4947-b2e3-934eb3041d96 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:39.753878 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3480/6992] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:39.757342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98a2db-536f-489d-9e22-abaf3b8bfc28 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:39.802852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58de3b-4814-4f65-a627-7c5be57cacef None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:39.803931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58de3b-4814-4f65-a627-7c5be57cacef None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:39.804323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e58de3b-4814-4f65-a627-7c5be57cacef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:39.805669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98a2db-536f-489d-9e22-abaf3b8bfc28 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:39.812768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b9cb3-cb44-4f4c-bfcd-2115f45259ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:39.813727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b9cb3-cb44-4f4c-bfcd-2115f45259ff admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:39.814247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b9cb3-cb44-4f4c-bfcd-2115f45259ff admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:39.814706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b9cb3-cb44-4f4c-bfcd-2115f45259ff admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:39.815634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b9cb3-cb44-4f4c-bfcd-2115f45259ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:39.818968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b9cb3-cb44-4f4c-bfcd-2115f45259ff admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:39.833298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8e58de3b-4814-4f65-a627-7c5be57cacef None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:39.835099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8e58de3b-4814-4f65-a627-7c5be57cacef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:39.840256 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3513/6993] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:39.844043 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3481/6994] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:39.844948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b9cb3-cb44-4f4c-bfcd-2115f45259ff admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:39.848248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58de3b-4814-4f65-a627-7c5be57cacef None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:39.858629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e58de3b-4814-4f65-a627-7c5be57cacef None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:39.865575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b9cb3-cb44-4f4c-bfcd-2115f45259ff admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:39.870278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad560747-9c3e-47d9-8582-7012d988521a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:39.870976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad560747-9c3e-47d9-8582-7012d988521a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:39.871828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad560747-9c3e-47d9-8582-7012d988521a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:39.872140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad560747-9c3e-47d9-8582-7012d988521a admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:39.872385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4fdea-6437-48ec-87c1-b1cf992aeeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:39.872889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad560747-9c3e-47d9-8582-7012d988521a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:28:39.873214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad560747-9c3e-47d9-8582-7012d988521a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:39.874187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4fdea-6437-48ec-87c1-b1cf992aeeba admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:39.874718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4fdea-6437-48ec-87c1-b1cf992aeeba admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:39.874979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad560747-9c3e-47d9-8582-7012d988521a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:39.875459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4fdea-6437-48ec-87c1-b1cf992aeeba admin admin] PATH_INFO: `/v3/projects/f11f466272ba42be98bf2ea8d724c080/users/6119da412dd34694b3b888f5c225b3f3/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:39.894291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4fdea-6437-48ec-87c1-b1cf992aeeba admin admin] RBAC: Authorizing `identity:create_grant(project_id=f11f466272ba42be98bf2ea8d724c080, user_id=6119da412dd34694b3b888f5c225b3f3, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:39.896931 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3482/6995] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:28:39.898999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4fdea-6437-48ec-87c1-b1cf992aeeba admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:39.903262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad560747-9c3e-47d9-8582-7012d988521a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:39.953904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad560747-9c3e-47d9-8582-7012d988521a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:39.967837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb78372-5504-4f22-ab12-39a127336bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:39.968518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb78372-5504-4f22-ab12-39a127336bbf admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:39.969066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb78372-5504-4f22-ab12-39a127336bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:39.969344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb78372-5504-4f22-ab12-39a127336bbf admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:39.970501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb78372-5504-4f22-ab12-39a127336bbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:39.972606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb78372-5504-4f22-ab12-39a127336bbf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:40.018977 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3514/6996] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:39 2022] PUT /identity/v3/projects/f11f466272ba42be98bf2ea8d724c080/users/6119da412dd34694b3b888f5c225b3f3/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:40.030165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad61fa-cfa8-4a16-8fd8-f05686dad5d7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:40.030829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad61fa-cfa8-4a16-8fd8-f05686dad5d7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:40.031363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad61fa-cfa8-4a16-8fd8-f05686dad5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:40.075686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-70ad61fa-cfa8-4a16-8fd8-f05686dad5d7 None None] MFA Rules not processed for user `6119da412dd34694b3b888f5c225b3f3`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:40.081555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-70ad61fa-cfa8-4a16-8fd8-f05686dad5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:40.167179 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3483/6997] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:28:39 2022] POST /identity/v3/projects => generated 381 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:40.171312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb78372-5504-4f22-ab12-39a127336bbf admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:40.183058 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3515/6998] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:40 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:28:40.200799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad61fa-cfa8-4a16-8fd8-f05686dad5d7 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:40.203058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-70ad61fa-cfa8-4a16-8fd8-f05686dad5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:40.304411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb78372-5504-4f22-ab12-39a127336bbf admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:40.314495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980af81-345c-44b7-a0c0-0bfff1e72aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:40.315185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980af81-345c-44b7-a0c0-0bfff1e72aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:40.315470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980af81-345c-44b7-a0c0-0bfff1e72aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:40.315748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980af81-345c-44b7-a0c0-0bfff1e72aa2 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:40.316614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980af81-345c-44b7-a0c0-0bfff1e72aa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:40.318613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980af81-345c-44b7-a0c0-0bfff1e72aa2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:40.330727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad61fa-cfa8-4a16-8fd8-f05686dad5d7 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:40.416212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9846ddf0-2606-45f5-bffc-af9e0fa6ca34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:40.417210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846ddf0-2606-45f5-bffc-af9e0fa6ca34 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:40.418051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846ddf0-2606-45f5-bffc-af9e0fa6ca34 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:40.418556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9846ddf0-2606-45f5-bffc-af9e0fa6ca34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:40.454880 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3484/6999] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:28:40 2022] POST /identity/v3/users => generated 291 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:28:40.465898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980af81-345c-44b7-a0c0-0bfff1e72aa2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:40.477531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980af81-345c-44b7-a0c0-0bfff1e72aa2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:40.497206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e750b-cf43-42d7-9925-baaa2fe59506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:40.500260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e750b-cf43-42d7-9925-baaa2fe59506 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:40.500769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e750b-cf43-42d7-9925-baaa2fe59506 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:40.501917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e750b-cf43-42d7-9925-baaa2fe59506 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:40.503394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e750b-cf43-42d7-9925-baaa2fe59506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:40.510498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e750b-cf43-42d7-9925-baaa2fe59506 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:40.534653 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3485/7000] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:28:40.537104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e750b-cf43-42d7-9925-baaa2fe59506 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:40.551128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e750b-cf43-42d7-9925-baaa2fe59506 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:40.562917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fffa7e-3a94-4896-a607-306f317567e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:40.564020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fffa7e-3a94-4896-a607-306f317567e0 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:40.565184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fffa7e-3a94-4896-a607-306f317567e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:40.565766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fffa7e-3a94-4896-a607-306f317567e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:40.568118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fffa7e-3a94-4896-a607-306f317567e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:40.576149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846ddf0-2606-45f5-bffc-af9e0fa6ca34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:40.578376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fffa7e-3a94-4896-a607-306f317567e0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:40.580772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9846ddf0-2606-45f5-bffc-af9e0fa6ca34 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:40.592630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:40.610805 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3486/7001] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:40 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:40.619644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fffa7e-3a94-4896-a607-306f317567e0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:40.620399 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3516/7002] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:40 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:40.631978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fffa7e-3a94-4896-a607-306f317567e0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:40.642891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ce4ba-f463-4b6c-ac8c-2ee3ccbcb40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:40.646141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ce4ba-f463-4b6c-ac8c-2ee3ccbcb40e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:40.646432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ce4ba-f463-4b6c-ac8c-2ee3ccbcb40e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:40.647167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ce4ba-f463-4b6c-ac8c-2ee3ccbcb40e admin admin] PATH_INFO: `/v3/projects/4ed2192b2a514d379a4d6292f49d5a52/users/20c731d95198430486c6bf5b3345efc7/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:40.672005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ce4ba-f463-4b6c-ac8c-2ee3ccbcb40e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ed2192b2a514d379a4d6292f49d5a52, user_id=20c731d95198430486c6bf5b3345efc7, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:40.675344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ce4ba-f463-4b6c-ac8c-2ee3ccbcb40e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:40.826131 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3487/7003] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:40 2022] PUT /identity/v3/projects/4ed2192b2a514d379a4d6292f49d5a52/users/20c731d95198430486c6bf5b3345efc7/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:40.837010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d32741-4e43-4d62-8507-569f370bb385 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:40.837315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d32741-4e43-4d62-8507-569f370bb385 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:40.837583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d32741-4e43-4d62-8507-569f370bb385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:40.882456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-33d32741-4e43-4d62-8507-569f370bb385 None None] MFA Rules not processed for user `20c731d95198430486c6bf5b3345efc7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:40.883688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-33d32741-4e43-4d62-8507-569f370bb385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:40.998156 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3517/7004] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:41.000901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ce4ba-f463-4b6c-ac8c-2ee3ccbcb40e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:41.139564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ce4ba-f463-4b6c-ac8c-2ee3ccbcb40e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:41.152401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419aef2f-9adb-4b06-b935-d6ca1792219c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:41.152401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419aef2f-9adb-4b06-b935-d6ca1792219c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:41.152401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419aef2f-9adb-4b06-b935-d6ca1792219c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:41.152401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419aef2f-9adb-4b06-b935-d6ca1792219c admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:41.152878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419aef2f-9adb-4b06-b935-d6ca1792219c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:41.155046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419aef2f-9adb-4b06-b935-d6ca1792219c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:41.292393 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3488/7005] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:28:40 2022] POST /identity/v3/projects => generated 383 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:41.296203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d32741-4e43-4d62-8507-569f370bb385 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:41.310169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d32741-4e43-4d62-8507-569f370bb385 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:41.342650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:41.407434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f8b53-dd22-492a-85fc-1cdecfcf37f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:41.409024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f8b53-dd22-492a-85fc-1cdecfcf37f4 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:41.409326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f8b53-dd22-492a-85fc-1cdecfcf37f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:41.409613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93f8b53-dd22-492a-85fc-1cdecfcf37f4 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:41.410567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f8b53-dd22-492a-85fc-1cdecfcf37f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:41.412526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93f8b53-dd22-492a-85fc-1cdecfcf37f4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:41.541228 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3518/7006] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:28:41 2022] POST /identity/v3/users => generated 298 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:41.545689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419aef2f-9adb-4b06-b935-d6ca1792219c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:41.564531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419aef2f-9adb-4b06-b935-d6ca1792219c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:41.573443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4f26e-160f-4c83-8def-e36cdac42054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:41.574512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4f26e-160f-4c83-8def-e36cdac42054 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:41.580743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4f26e-160f-4c83-8def-e36cdac42054 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:41.580743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4f26e-160f-4c83-8def-e36cdac42054 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:41.580743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4f26e-160f-4c83-8def-e36cdac42054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:41.580743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4f26e-160f-4c83-8def-e36cdac42054 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:41.595187 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3489/7007] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:41.596609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f8b53-dd22-492a-85fc-1cdecfcf37f4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:41.608164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93f8b53-dd22-492a-85fc-1cdecfcf37f4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:41.622735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c5e5e-3552-43b3-b43e-88abc3349490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:41.623774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c5e5e-3552-43b3-b43e-88abc3349490 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:41.624428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c5e5e-3552-43b3-b43e-88abc3349490 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:41.625295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c5e5e-3552-43b3-b43e-88abc3349490 admin admin] PATH_INFO: `/v3/projects/b5d3310e400f40f5b08828f1d07387e6/users/447d0608f3c347f38a99b7b36993f67f/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:41.645484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c5e5e-3552-43b3-b43e-88abc3349490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d3310e400f40f5b08828f1d07387e6, user_id=447d0608f3c347f38a99b7b36993f67f, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:41.649287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c5e5e-3552-43b3-b43e-88abc3349490 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:41.672940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:41.714268 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3519/7008] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:41 2022] PUT /identity/v3/projects/b5d3310e400f40f5b08828f1d07387e6/users/447d0608f3c347f38a99b7b36993f67f/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:41.718254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4f26e-160f-4c83-8def-e36cdac42054 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:41.736842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4f26e-160f-4c83-8def-e36cdac42054 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:41.837345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd4129-05f2-483e-bc68-c8e66ae75dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:41.838211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd4129-05f2-483e-bc68-c8e66ae75dbe admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:41.838570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd4129-05f2-483e-bc68-c8e66ae75dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:41.839107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd4129-05f2-483e-bc68-c8e66ae75dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:41.841155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd4129-05f2-483e-bc68-c8e66ae75dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:41.845374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd4129-05f2-483e-bc68-c8e66ae75dbe admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:41.865603 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3490/7009] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:41.870249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c5e5e-3552-43b3-b43e-88abc3349490 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:41.883831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c5e5e-3552-43b3-b43e-88abc3349490 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:41.895279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe09457-d7db-42ee-b7ad-b5da0585ec89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:41.896236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe09457-d7db-42ee-b7ad-b5da0585ec89 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:41.896526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe09457-d7db-42ee-b7ad-b5da0585ec89 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:41.896810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe09457-d7db-42ee-b7ad-b5da0585ec89 admin admin] PATH_INFO: `/v3/projects/b5d3310e400f40f5b08828f1d07387e6/users/447d0608f3c347f38a99b7b36993f67f/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:41.900295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe09457-d7db-42ee-b7ad-b5da0585ec89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5d3310e400f40f5b08828f1d07387e6, user_id=447d0608f3c347f38a99b7b36993f67f, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:41.903106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe09457-d7db-42ee-b7ad-b5da0585ec89 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:41.930163 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3520/7010] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:41 2022] PUT /identity/v3/projects/b5d3310e400f40f5b08828f1d07387e6/users/447d0608f3c347f38a99b7b36993f67f/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:41.933512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49ff6a-7246-4780-9b6b-112c20716ebc None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:41.935393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49ff6a-7246-4780-9b6b-112c20716ebc None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:41.935852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49ff6a-7246-4780-9b6b-112c20716ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:41.969473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0b49ff6a-7246-4780-9b6b-112c20716ebc None None] MFA Rules not processed for user `447d0608f3c347f38a99b7b36993f67f`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:41.971025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0b49ff6a-7246-4780-9b6b-112c20716ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:42.089613 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3491/7011] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:42.108832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe09457-d7db-42ee-b7ad-b5da0585ec89 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:42.110751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1fe09457-d7db-42ee-b7ad-b5da0585ec89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:42.252621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe09457-d7db-42ee-b7ad-b5da0585ec89 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:42.266754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc57fe-4452-43f0-b181-75b3bce71f86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:42.267489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc57fe-4452-43f0-b181-75b3bce71f86 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:42.267851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc57fe-4452-43f0-b181-75b3bce71f86 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:42.268137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc57fe-4452-43f0-b181-75b3bce71f86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:42.283381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc57fe-4452-43f0-b181-75b3bce71f86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:42.285046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc57fe-4452-43f0-b181-75b3bce71f86 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:42.298522 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3521/7012] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:42 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:42.452175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49ff6a-7246-4780-9b6b-112c20716ebc None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:42.471016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b49ff6a-7246-4780-9b6b-112c20716ebc None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:42.479412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbca0e-f5df-4595-b579-8fb06de3f612 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:42.480242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbca0e-f5df-4595-b579-8fb06de3f612 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:42.481355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbca0e-f5df-4595-b579-8fb06de3f612 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:42.481725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dbca0e-f5df-4595-b579-8fb06de3f612 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:42.565279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dbca0e-f5df-4595-b579-8fb06de3f612 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:42.567323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dbca0e-f5df-4595-b579-8fb06de3f612 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:42.579521 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3492/7013] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:42.730248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:43.410268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:43.884711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:45.501013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:45.680496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc57fe-4452-43f0-b181-75b3bce71f86 service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:45.751174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc57fe-4452-43f0-b181-75b3bce71f86 service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:45.755719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e134a-a813-4d02-8204-0d0114b44853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:45.758735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e134a-a813-4d02-8204-0d0114b44853 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:45.759472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e134a-a813-4d02-8204-0d0114b44853 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:45.759985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e134a-a813-4d02-8204-0d0114b44853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:45.762611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-669e134a-a813-4d02-8204-0d0114b44853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:45.782986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e134a-a813-4d02-8204-0d0114b44853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:45.785254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e134a-a813-4d02-8204-0d0114b44853 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:45.799660 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3522/7014] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:45.957720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:47.570513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:47.627462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:48.377941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:48.701249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:49.816465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:50.586984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:51.472188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:52.870064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbca0e-f5df-4595-b579-8fb06de3f612 service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:52.876097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:52.959381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e134a-a813-4d02-8204-0d0114b44853 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:52.961680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-669e134a-a813-4d02-8204-0d0114b44853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:52.977764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dbca0e-f5df-4595-b579-8fb06de3f612 service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:52.980634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e134a-a813-4d02-8204-0d0114b44853 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:52.982757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd152484-6d42-4b90-8980-cd688a9961e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:52.983708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd152484-6d42-4b90-8980-cd688a9961e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:52.984061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd152484-6d42-4b90-8980-cd688a9961e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:52.984633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd152484-6d42-4b90-8980-cd688a9961e2 admin admin] PATH_INFO: `/v3/users/20c731d95198430486c6bf5b3345efc7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:52.985631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded9375-2752-4040-9efd-616c3b400cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:52.985856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd152484-6d42-4b90-8980-cd688a9961e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20c731d95198430486c6bf5b3345efc7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:52.986558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded9375-2752-4040-9efd-616c3b400cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:52.986846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded9375-2752-4040-9efd-616c3b400cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:52.987123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded9375-2752-4040-9efd-616c3b400cf6 admin admin] PATH_INFO: `/v3/users/a9ca0f87ca18428fbdc5ae0eb5329342` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:52.988263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd152484-6d42-4b90-8980-cd688a9961e2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:52.989051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded9375-2752-4040-9efd-616c3b400cf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9ca0f87ca18428fbdc5ae0eb5329342)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:52.990943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded9375-2752-4040-9efd-616c3b400cf6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:53.021704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:53.200202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fded9375-2752-4040-9efd-616c3b400cf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9ca0f87ca18428fbdc5ae0eb5329342'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:53.213427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cd152484-6d42-4b90-8980-cd688a9961e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20c731d95198430486c6bf5b3345efc7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:53.214358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cd152484-6d42-4b90-8980-cd688a9961e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20c731d95198430486c6bf5b3345efc7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:53.215811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fded9375-2752-4040-9efd-616c3b400cf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9ca0f87ca18428fbdc5ae0eb5329342'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:53.216746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fded9375-2752-4040-9efd-616c3b400cf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9ca0f87ca18428fbdc5ae0eb5329342'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:53.218092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fded9375-2752-4040-9efd-616c3b400cf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9ca0f87ca18428fbdc5ae0eb5329342'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:53.229226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cd152484-6d42-4b90-8980-cd688a9961e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20c731d95198430486c6bf5b3345efc7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:53.296967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fded9375-2752-4040-9efd-616c3b400cf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9ca0f87ca18428fbdc5ae0eb5329342'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:53.323690 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3523/7015] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:52 2022] DELETE /identity/v3/users/a9ca0f87ca18428fbdc5ae0eb5329342 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:53.327950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9b12d-1a63-4ea1-b937-b2d0f9b2492f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:53.333197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9b12d-1a63-4ea1-b937-b2d0f9b2492f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:53.333197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9b12d-1a63-4ea1-b937-b2d0f9b2492f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:53.349721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-24b9b12d-1a63-4ea1-b937-b2d0f9b2492f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:53.351142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-24b9b12d-1a63-4ea1-b937-b2d0f9b2492f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:53.357667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cd152484-6d42-4b90-8980-cd688a9961e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20c731d95198430486c6bf5b3345efc7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:53.358675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cd152484-6d42-4b90-8980-cd688a9961e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20c731d95198430486c6bf5b3345efc7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:53.388042 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3493/7016] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:52 2022] DELETE /identity/v3/users/20c731d95198430486c6bf5b3345efc7 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:53.392417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77393fe-14f0-4998-b834-03afe8fbd963 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:53.392713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77393fe-14f0-4998-b834-03afe8fbd963 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:53.393039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77393fe-14f0-4998-b834-03afe8fbd963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:53.412649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e77393fe-14f0-4998-b834-03afe8fbd963 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:53.413933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e77393fe-14f0-4998-b834-03afe8fbd963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:53.423327 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3524/7017] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:28:53.431467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9b12d-1a63-4ea1-b937-b2d0f9b2492f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:53.433003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-24b9b12d-1a63-4ea1-b937-b2d0f9b2492f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:53.480190 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3494/7018] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:28:53.491521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77393fe-14f0-4998-b834-03afe8fbd963 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:53.494795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e77393fe-14f0-4998-b834-03afe8fbd963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:53.509667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9b12d-1a63-4ea1-b937-b2d0f9b2492f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:53.554699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:53.563462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77393fe-14f0-4998-b834-03afe8fbd963 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:53.563713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d92207-e77e-4527-957c-eb6f65b7bcf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:53.564465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d92207-e77e-4527-957c-eb6f65b7bcf7 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:53.564890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d92207-e77e-4527-957c-eb6f65b7bcf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:53.565166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d92207-e77e-4527-957c-eb6f65b7bcf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:53.568747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154780a7-c804-464b-a851-0225419bea81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:53.569466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154780a7-c804-464b-a851-0225419bea81 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:53.575281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154780a7-c804-464b-a851-0225419bea81 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:53.575281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154780a7-c804-464b-a851-0225419bea81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:53.576248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d92207-e77e-4527-957c-eb6f65b7bcf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:53.577774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d92207-e77e-4527-957c-eb6f65b7bcf7 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:53.579707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154780a7-c804-464b-a851-0225419bea81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:53.581172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154780a7-c804-464b-a851-0225419bea81 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:53.589680 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3525/7019] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:28:53.591772 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3495/7020] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:28:54.235945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d92207-e77e-4527-957c-eb6f65b7bcf7 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:54.237761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b5d92207-e77e-4527-957c-eb6f65b7bcf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:54.264745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d92207-e77e-4527-957c-eb6f65b7bcf7 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:54.274298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7122132-29b5-4fec-bbc1-26c6a25b5a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:54.275196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7122132-29b5-4fec-bbc1-26c6a25b5a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:54.275490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7122132-29b5-4fec-bbc1-26c6a25b5a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:54.276319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7122132-29b5-4fec-bbc1-26c6a25b5a45 admin admin] PATH_INFO: `/v3/projects/4ed2192b2a514d379a4d6292f49d5a52` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:54.278151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7122132-29b5-4fec-bbc1-26c6a25b5a45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ed2192b2a514d379a4d6292f49d5a52)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:54.281449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7122132-29b5-4fec-bbc1-26c6a25b5a45 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:54.303318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:54.429738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e7122132-29b5-4fec-bbc1-26c6a25b5a45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ed2192b2a514d379a4d6292f49d5a52', 'request_id': 'req-e7122132-29b5-4fec-bbc1-26c6a25b5a45'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:54.435528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154780a7-c804-464b-a851-0225419bea81 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:54.438190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-154780a7-c804-464b-a851-0225419bea81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:54.439449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e7122132-29b5-4fec-bbc1-26c6a25b5a45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ed2192b2a514d379a4d6292f49d5a52', 'request_id': 'req-e7122132-29b5-4fec-bbc1-26c6a25b5a45'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:54.455343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154780a7-c804-464b-a851-0225419bea81 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:54.500769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e7122132-29b5-4fec-bbc1-26c6a25b5a45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ed2192b2a514d379a4d6292f49d5a52', 'request_id': 'req-e7122132-29b5-4fec-bbc1-26c6a25b5a45'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:54.534255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e7122132-29b5-4fec-bbc1-26c6a25b5a45 admin admin] The token cache is being invalidate because project 4ed2192b2a514d379a4d6292f49d5a52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:54.536592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e7122132-29b5-4fec-bbc1-26c6a25b5a45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:54.536592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e7122132-29b5-4fec-bbc1-26c6a25b5a45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:54.541448 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3526/7021] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:54 2022] DELETE /identity/v3/projects/4ed2192b2a514d379a4d6292f49d5a52 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:54.542660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7122132-29b5-4fec-bbc1-26c6a25b5a45 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:54.544415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e7122132-29b5-4fec-bbc1-26c6a25b5a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:54.545328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ed4f85-2dad-42fb-907d-7a2c9e1f765d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:54.546591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed4f85-2dad-42fb-907d-7a2c9e1f765d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:54.546883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed4f85-2dad-42fb-907d-7a2c9e1f765d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:54.547174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ed4f85-2dad-42fb-907d-7a2c9e1f765d admin admin] PATH_INFO: `/v3/projects/7b6c916fa5074323b5c9a3e959f2f1e7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:54.548794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ed4f85-2dad-42fb-907d-7a2c9e1f765d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b6c916fa5074323b5c9a3e959f2f1e7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:54.551561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ed4f85-2dad-42fb-907d-7a2c9e1f765d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:54.559682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7122132-29b5-4fec-bbc1-26c6a25b5a45 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:54.565239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358184b-8fdd-4e84-bc84-7990c7deb344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:54.566317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358184b-8fdd-4e84-bc84-7990c7deb344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:54.566676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358184b-8fdd-4e84-bc84-7990c7deb344 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:54.567031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358184b-8fdd-4e84-bc84-7990c7deb344 admin admin] PATH_INFO: `/v3/users/447d0608f3c347f38a99b7b36993f67f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:54.570785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358184b-8fdd-4e84-bc84-7990c7deb344 admin admin] RBAC: Authorizing `identity:delete_user(user_id=447d0608f3c347f38a99b7b36993f67f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:54.573195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358184b-8fdd-4e84-bc84-7990c7deb344 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:54.708760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-31ed4f85-2dad-42fb-907d-7a2c9e1f765d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b6c916fa5074323b5c9a3e959f2f1e7', 'request_id': 'req-31ed4f85-2dad-42fb-907d-7a2c9e1f765d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:54.715247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-31ed4f85-2dad-42fb-907d-7a2c9e1f765d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b6c916fa5074323b5c9a3e959f2f1e7', 'request_id': 'req-31ed4f85-2dad-42fb-907d-7a2c9e1f765d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:54.787135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-31ed4f85-2dad-42fb-907d-7a2c9e1f765d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b6c916fa5074323b5c9a3e959f2f1e7', 'request_id': 'req-31ed4f85-2dad-42fb-907d-7a2c9e1f765d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:54.803961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1358184b-8fdd-4e84-bc84-7990c7deb344 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '447d0608f3c347f38a99b7b36993f67f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:54.806999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-31ed4f85-2dad-42fb-907d-7a2c9e1f765d admin admin] The token cache is being invalidate because project 7b6c916fa5074323b5c9a3e959f2f1e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:54.807524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-31ed4f85-2dad-42fb-907d-7a2c9e1f765d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:54.808930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-31ed4f85-2dad-42fb-907d-7a2c9e1f765d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:54.815426 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3496/7022] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:54 2022] DELETE /identity/v3/projects/7b6c916fa5074323b5c9a3e959f2f1e7 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:28:54.815769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1358184b-8fdd-4e84-bc84-7990c7deb344 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '447d0608f3c347f38a99b7b36993f67f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:54.815769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1358184b-8fdd-4e84-bc84-7990c7deb344 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '447d0608f3c347f38a99b7b36993f67f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:54.815769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1358184b-8fdd-4e84-bc84-7990c7deb344 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '447d0608f3c347f38a99b7b36993f67f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:54.817814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f979864-a2ad-41fa-aedf-a1890c67782f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:54.818478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f979864-a2ad-41fa-aedf-a1890c67782f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:54.818478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f979864-a2ad-41fa-aedf-a1890c67782f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:54.841389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5f979864-a2ad-41fa-aedf-a1890c67782f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:54.842679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5f979864-a2ad-41fa-aedf-a1890c67782f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:54.904814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1358184b-8fdd-4e84-bc84-7990c7deb344 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '447d0608f3c347f38a99b7b36993f67f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:54.920684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:54.922397 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3497/7023] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:54.930708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7e305-a4f6-4f6b-a11c-e880da808dd1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:54.931003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7e305-a4f6-4f6b-a11c-e880da808dd1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:54.931285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7e305-a4f6-4f6b-a11c-e880da808dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:54.936390 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3527/7024] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:54 2022] DELETE /identity/v3/users/447d0608f3c347f38a99b7b36993f67f => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:28:54.955021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c4d7e305-a4f6-4f6b-a11c-e880da808dd1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:54.955021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c4d7e305-a4f6-4f6b-a11c-e880da808dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:55.028407 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3498/7025] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:28:55.033147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358184b-8fdd-4e84-bc84-7990c7deb344 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:55.047145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358184b-8fdd-4e84-bc84-7990c7deb344 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:55.053918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef12cd6-409e-44d1-8b08-0f029abbe96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:55.054616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef12cd6-409e-44d1-8b08-0f029abbe96c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:55.054908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef12cd6-409e-44d1-8b08-0f029abbe96c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:55.055201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef12cd6-409e-44d1-8b08-0f029abbe96c admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:55.056013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef12cd6-409e-44d1-8b08-0f029abbe96c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:28:55.056328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef12cd6-409e-44d1-8b08-0f029abbe96c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:55.058136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef12cd6-409e-44d1-8b08-0f029abbe96c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:55.074138 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3528/7026] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:28:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:28:55.079099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7e305-a4f6-4f6b-a11c-e880da808dd1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:55.089059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7e305-a4f6-4f6b-a11c-e880da808dd1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:55.094152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0043ba-6fca-4519-ba64-9657be1ff35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:55.094817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0043ba-6fca-4519-ba64-9657be1ff35d admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:55.095106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0043ba-6fca-4519-ba64-9657be1ff35d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:55.095384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0043ba-6fca-4519-ba64-9657be1ff35d admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:55.096083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0043ba-6fca-4519-ba64-9657be1ff35d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:55.098638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0043ba-6fca-4519-ba64-9657be1ff35d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:55.213268 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3499/7027] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:28:55 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:55.216978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef12cd6-409e-44d1-8b08-0f029abbe96c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:55.229734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef12cd6-409e-44d1-8b08-0f029abbe96c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:55.291050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6b290-c6c0-44bf-aaa6-5cbb6181d7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:55.291754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6b290-c6c0-44bf-aaa6-5cbb6181d7f7 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:55.292755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6b290-c6c0-44bf-aaa6-5cbb6181d7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:55.292755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6b290-c6c0-44bf-aaa6-5cbb6181d7f7 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:55.293111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6b290-c6c0-44bf-aaa6-5cbb6181d7f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:55.295160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6b290-c6c0-44bf-aaa6-5cbb6181d7f7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:55.401371 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3529/7028] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:28:55 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:55.406242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0043ba-6fca-4519-ba64-9657be1ff35d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:55.418129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0043ba-6fca-4519-ba64-9657be1ff35d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:55.424292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799866eb-fa34-46f8-b88b-79463cda7a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:55.425336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799866eb-fa34-46f8-b88b-79463cda7a8e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:55.426302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799866eb-fa34-46f8-b88b-79463cda7a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:55.426963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799866eb-fa34-46f8-b88b-79463cda7a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:55.427712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799866eb-fa34-46f8-b88b-79463cda7a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:55.429507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799866eb-fa34-46f8-b88b-79463cda7a8e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:55.444964 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3500/7029] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:55.446162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6b290-c6c0-44bf-aaa6-5cbb6181d7f7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:55.457349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6b290-c6c0-44bf-aaa6-5cbb6181d7f7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:55.459766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799866eb-fa34-46f8-b88b-79463cda7a8e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:55.462010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-799866eb-fa34-46f8-b88b-79463cda7a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:55.463162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb84805-35f3-40b1-a84d-3c029ef75bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:55.463893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb84805-35f3-40b1-a84d-3c029ef75bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:55.464181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb84805-35f3-40b1-a84d-3c029ef75bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:55.464667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb84805-35f3-40b1-a84d-3c029ef75bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:55.465454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb84805-35f3-40b1-a84d-3c029ef75bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:55.467309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb84805-35f3-40b1-a84d-3c029ef75bd6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:55.483687 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3530/7030] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:55.490806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb84805-35f3-40b1-a84d-3c029ef75bd6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:55.504887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb84805-35f3-40b1-a84d-3c029ef75bd6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:55.512194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de72db2-6687-4d5f-b696-02618c9c22f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:55.515451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de72db2-6687-4d5f-b696-02618c9c22f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:55.515451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de72db2-6687-4d5f-b696-02618c9c22f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:55.515451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de72db2-6687-4d5f-b696-02618c9c22f4 admin admin] PATH_INFO: `/v3/projects/071f3ee8ccba4f4a891cac48aaa54d0a/users/38d2c184e2ad446cb0481b1421439771/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:55.544362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de72db2-6687-4d5f-b696-02618c9c22f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=071f3ee8ccba4f4a891cac48aaa54d0a, user_id=38d2c184e2ad446cb0481b1421439771, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:55.547716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de72db2-6687-4d5f-b696-02618c9c22f4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:55.585944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799866eb-fa34-46f8-b88b-79463cda7a8e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:55.589629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b5b3aa-980d-440b-aa11-89554e1e8d96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:55.590374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5b3aa-980d-440b-aa11-89554e1e8d96 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:55.590720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5b3aa-980d-440b-aa11-89554e1e8d96 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:55.590913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b5b3aa-980d-440b-aa11-89554e1e8d96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:55.592731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e9b5b3aa-980d-440b-aa11-89554e1e8d96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:55.606190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5b3aa-980d-440b-aa11-89554e1e8d96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:55.607709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b5b3aa-980d-440b-aa11-89554e1e8d96 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:55.618660 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3501/7031] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:28:55.701414 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3531/7032] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:55 2022] PUT /identity/v3/projects/071f3ee8ccba4f4a891cac48aaa54d0a/users/38d2c184e2ad446cb0481b1421439771/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:55.712409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffabf2-af39-4d91-8326-bdc4b3ad4730 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:55.712709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffabf2-af39-4d91-8326-bdc4b3ad4730 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:55.712993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ffabf2-af39-4d91-8326-bdc4b3ad4730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:55.740701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c2ffabf2-af39-4d91-8326-bdc4b3ad4730 None None] MFA Rules not processed for user `38d2c184e2ad446cb0481b1421439771`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:55.741950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c2ffabf2-af39-4d91-8326-bdc4b3ad4730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:55.754167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de72db2-6687-4d5f-b696-02618c9c22f4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:55.758156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3de72db2-6687-4d5f-b696-02618c9c22f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:55.830962 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3502/7033] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:55.840008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffabf2-af39-4d91-8326-bdc4b3ad4730 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:55.841973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c2ffabf2-af39-4d91-8326-bdc4b3ad4730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:55.860980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de72db2-6687-4d5f-b696-02618c9c22f4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:55.866606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70549287-a96b-4b07-98e5-74f34657b007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:55.867827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70549287-a96b-4b07-98e5-74f34657b007 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:55.868387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70549287-a96b-4b07-98e5-74f34657b007 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:55.868832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70549287-a96b-4b07-98e5-74f34657b007 admin admin] PATH_INFO: `/v3/projects/b5d3310e400f40f5b08828f1d07387e6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:55.870394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70549287-a96b-4b07-98e5-74f34657b007 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5d3310e400f40f5b08828f1d07387e6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:55.872512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70549287-a96b-4b07-98e5-74f34657b007 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:55.915800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ffabf2-af39-4d91-8326-bdc4b3ad4730 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:55.919582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954f1bf-7d2f-498d-84eb-9281414077f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:55.920267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954f1bf-7d2f-498d-84eb-9281414077f1 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:55.924961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954f1bf-7d2f-498d-84eb-9281414077f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:55.924961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954f1bf-7d2f-498d-84eb-9281414077f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:55.931026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954f1bf-7d2f-498d-84eb-9281414077f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:55.932625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954f1bf-7d2f-498d-84eb-9281414077f1 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:55.942590 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3503/7034] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:55.998643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:56.018831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-70549287-a96b-4b07-98e5-74f34657b007 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5d3310e400f40f5b08828f1d07387e6', 'request_id': 'req-70549287-a96b-4b07-98e5-74f34657b007'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:56.026498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-70549287-a96b-4b07-98e5-74f34657b007 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5d3310e400f40f5b08828f1d07387e6', 'request_id': 'req-70549287-a96b-4b07-98e5-74f34657b007'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:56.156863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-70549287-a96b-4b07-98e5-74f34657b007 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5d3310e400f40f5b08828f1d07387e6', 'request_id': 'req-70549287-a96b-4b07-98e5-74f34657b007'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:56.169534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-70549287-a96b-4b07-98e5-74f34657b007 admin admin] The token cache is being invalidate because project b5d3310e400f40f5b08828f1d07387e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:28:56.170290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-70549287-a96b-4b07-98e5-74f34657b007 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:56.171058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-70549287-a96b-4b07-98e5-74f34657b007 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:56.172744 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3532/7035] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:28:55 2022] DELETE /identity/v3/projects/b5d3310e400f40f5b08828f1d07387e6 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:28:56.177650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d9461-a0ad-4a0d-83b3-ea5b8f909a7a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:56.178108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d9461-a0ad-4a0d-83b3-ea5b8f909a7a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:56.178490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165d9461-a0ad-4a0d-83b3-ea5b8f909a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:56.197735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-165d9461-a0ad-4a0d-83b3-ea5b8f909a7a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:56.199114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-165d9461-a0ad-4a0d-83b3-ea5b8f909a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:56.286222 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3504/7036] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:28:56.293891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cab7fe-4999-4406-ac98-38f196ac9eab None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:56.294347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cab7fe-4999-4406-ac98-38f196ac9eab None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:56.294532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cab7fe-4999-4406-ac98-38f196ac9eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:56.315393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-78cab7fe-4999-4406-ac98-38f196ac9eab None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:56.317378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-78cab7fe-4999-4406-ac98-38f196ac9eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:56.328985 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3533/7037] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:56.330348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d9461-a0ad-4a0d-83b3-ea5b8f909a7a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:56.343835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165d9461-a0ad-4a0d-83b3-ea5b8f909a7a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:56.348641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da21bf6-18db-4c38-b661-76b14d3c2d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:56.349314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da21bf6-18db-4c38-b661-76b14d3c2d90 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:56.351355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da21bf6-18db-4c38-b661-76b14d3c2d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:56.351355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da21bf6-18db-4c38-b661-76b14d3c2d90 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:56.351355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da21bf6-18db-4c38-b661-76b14d3c2d90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:28:56.351355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da21bf6-18db-4c38-b661-76b14d3c2d90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:56.352827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da21bf6-18db-4c38-b661-76b14d3c2d90 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:56.367105 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3505/7038] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:28:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:28:56.371827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cab7fe-4999-4406-ac98-38f196ac9eab None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:56.382179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cab7fe-4999-4406-ac98-38f196ac9eab None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:56.387067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2e348-209a-4126-ba9a-5e973999786b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:56.387735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2e348-209a-4126-ba9a-5e973999786b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:56.388028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2e348-209a-4126-ba9a-5e973999786b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:56.388298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2e348-209a-4126-ba9a-5e973999786b admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:56.389009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2e348-209a-4126-ba9a-5e973999786b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:56.390960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece2e348-209a-4126-ba9a-5e973999786b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:56.474333 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3534/7039] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:28:56 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:56.478533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da21bf6-18db-4c38-b661-76b14d3c2d90 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:56.490154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da21bf6-18db-4c38-b661-76b14d3c2d90 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:56.552901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada2490-e17b-441c-ba0a-20c56ec69eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:56.553664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada2490-e17b-441c-ba0a-20c56ec69eb8 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:56.554050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada2490-e17b-441c-ba0a-20c56ec69eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:56.554396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ada2490-e17b-441c-ba0a-20c56ec69eb8 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:56.555194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada2490-e17b-441c-ba0a-20c56ec69eb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:56.557270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ada2490-e17b-441c-ba0a-20c56ec69eb8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:56.677134 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3506/7040] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:28:56 2022] POST /identity/v3/users => generated 291 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:56.677828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2e348-209a-4126-ba9a-5e973999786b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:56.699285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2e348-209a-4126-ba9a-5e973999786b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:56.715366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2c79f-105f-4194-b9d7-a4113eb427ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:56.717128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2c79f-105f-4194-b9d7-a4113eb427ad admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:56.717657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2c79f-105f-4194-b9d7-a4113eb427ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:56.718708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2c79f-105f-4194-b9d7-a4113eb427ad admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:56.721001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf2c79f-105f-4194-b9d7-a4113eb427ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:56.724030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf2c79f-105f-4194-b9d7-a4113eb427ad admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:56.744134 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3535/7041] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:56.749354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada2490-e17b-441c-ba0a-20c56ec69eb8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:56.759646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ada2490-e17b-441c-ba0a-20c56ec69eb8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:56.764676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2026eac-3ef3-4e62-b49b-5464fd4473d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:56.765337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2026eac-3ef3-4e62-b49b-5464fd4473d1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:56.765620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2026eac-3ef3-4e62-b49b-5464fd4473d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:56.766200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2026eac-3ef3-4e62-b49b-5464fd4473d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:56.767158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2026eac-3ef3-4e62-b49b-5464fd4473d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:56.773573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2026eac-3ef3-4e62-b49b-5464fd4473d1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:56.785867 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3507/7042] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:28:56.788181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2c79f-105f-4194-b9d7-a4113eb427ad admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:56.799195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2c79f-105f-4194-b9d7-a4113eb427ad admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:56.805346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511cdce-cd60-461e-9001-170f955b5dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:56.806317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511cdce-cd60-461e-9001-170f955b5dab admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:56.806603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511cdce-cd60-461e-9001-170f955b5dab admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:56.806887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511cdce-cd60-461e-9001-170f955b5dab admin admin] PATH_INFO: `/v3/projects/09d66d43c6b54b9cbd75a69b5e3acb3b/users/70979f8927cb48edb6b4e68a2e856b88/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:56.823358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511cdce-cd60-461e-9001-170f955b5dab admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d66d43c6b54b9cbd75a69b5e3acb3b, user_id=70979f8927cb48edb6b4e68a2e856b88, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:56.826054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:56.826243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511cdce-cd60-461e-9001-170f955b5dab admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:56.928935 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3536/7043] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:56 2022] PUT /identity/v3/projects/09d66d43c6b54b9cbd75a69b5e3acb3b/users/70979f8927cb48edb6b4e68a2e856b88/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:56.947070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c383ae8-3d8d-4969-9509-e6f37df94346 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:56.947367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c383ae8-3d8d-4969-9509-e6f37df94346 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:56.947649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c383ae8-3d8d-4969-9509-e6f37df94346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:56.976003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9c383ae8-3d8d-4969-9509-e6f37df94346 None None] MFA Rules not processed for user `70979f8927cb48edb6b4e68a2e856b88`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:56.977150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9c383ae8-3d8d-4969-9509-e6f37df94346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:57.076203 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3508/7044] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:57.077161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511cdce-cd60-461e-9001-170f955b5dab admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:57.153973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511cdce-cd60-461e-9001-170f955b5dab admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:57.161827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da48bb0-ff41-4bc3-96f3-e7b5a1d528d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:57.162934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da48bb0-ff41-4bc3-96f3-e7b5a1d528d3 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:57.163282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da48bb0-ff41-4bc3-96f3-e7b5a1d528d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:57.164837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da48bb0-ff41-4bc3-96f3-e7b5a1d528d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:57.165870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da48bb0-ff41-4bc3-96f3-e7b5a1d528d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:57.168666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da48bb0-ff41-4bc3-96f3-e7b5a1d528d3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:57.268272 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3537/7045] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:28:57 2022] POST /identity/v3/projects => generated 381 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:57.273017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c383ae8-3d8d-4969-9509-e6f37df94346 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:57.283958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c383ae8-3d8d-4969-9509-e6f37df94346 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:57.342961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9babb5e5-1c00-425c-b1b7-069450d5d549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:57.343897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babb5e5-1c00-425c-b1b7-069450d5d549 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:57.344192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babb5e5-1c00-425c-b1b7-069450d5d549 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:57.344528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9babb5e5-1c00-425c-b1b7-069450d5d549 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:57.345356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9babb5e5-1c00-425c-b1b7-069450d5d549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:57.347334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9babb5e5-1c00-425c-b1b7-069450d5d549 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:57.456139 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3509/7046] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:28:57 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:28:57.463760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da48bb0-ff41-4bc3-96f3-e7b5a1d528d3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:57.478410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da48bb0-ff41-4bc3-96f3-e7b5a1d528d3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:57.486961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1cf5f-7c23-4f1e-95b5-11e31634f006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:57.487961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1cf5f-7c23-4f1e-95b5-11e31634f006 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:57.488442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1cf5f-7c23-4f1e-95b5-11e31634f006 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:57.488722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1cf5f-7c23-4f1e-95b5-11e31634f006 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:57.489667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1cf5f-7c23-4f1e-95b5-11e31634f006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:57.491751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1cf5f-7c23-4f1e-95b5-11e31634f006 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:57.512119 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3538/7047] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:57.515184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9babb5e5-1c00-425c-b1b7-069450d5d549 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:57.526992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9babb5e5-1c00-425c-b1b7-069450d5d549 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:57.532437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d06a67-af83-401a-b4eb-a18d67c0d891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:57.533341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d06a67-af83-401a-b4eb-a18d67c0d891 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:57.533630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d06a67-af83-401a-b4eb-a18d67c0d891 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:57.534029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d06a67-af83-401a-b4eb-a18d67c0d891 admin admin] PATH_INFO: `/v3/projects/5ac1a4622e9d427980de4bc6949b16b8/users/7ba0e966f2574366b4612bb633106e77/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:57.555042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d06a67-af83-401a-b4eb-a18d67c0d891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac1a4622e9d427980de4bc6949b16b8, user_id=7ba0e966f2574366b4612bb633106e77, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:57.556613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d06a67-af83-401a-b4eb-a18d67c0d891 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:57.701529 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3510/7048] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:57 2022] PUT /identity/v3/projects/5ac1a4622e9d427980de4bc6949b16b8/users/7ba0e966f2574366b4612bb633106e77/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:28:57.705630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1cf5f-7c23-4f1e-95b5-11e31634f006 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:57.719280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1cf5f-7c23-4f1e-95b5-11e31634f006 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:57.836146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043fe201-eedf-47e4-89a0-94db826f4f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:57.836913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fe201-eedf-47e4-89a0-94db826f4f74 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:57.837273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fe201-eedf-47e4-89a0-94db826f4f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:57.837648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043fe201-eedf-47e4-89a0-94db826f4f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:57.838499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043fe201-eedf-47e4-89a0-94db826f4f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:57.840323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043fe201-eedf-47e4-89a0-94db826f4f74 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:57.853746 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3539/7049] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:28:57.859443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d06a67-af83-401a-b4eb-a18d67c0d891 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:57.872792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d06a67-af83-401a-b4eb-a18d67c0d891 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:57.878735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b62e38-4e50-4df8-87f8-cb177b3260eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:57.880167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b62e38-4e50-4df8-87f8-cb177b3260eb admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:57.880504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b62e38-4e50-4df8-87f8-cb177b3260eb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:57.880790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b62e38-4e50-4df8-87f8-cb177b3260eb admin admin] PATH_INFO: `/v3/projects/5ac1a4622e9d427980de4bc6949b16b8/users/7ba0e966f2574366b4612bb633106e77/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:57.882951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b62e38-4e50-4df8-87f8-cb177b3260eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ac1a4622e9d427980de4bc6949b16b8, user_id=7ba0e966f2574366b4612bb633106e77, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:57.886455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b62e38-4e50-4df8-87f8-cb177b3260eb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:57.896841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:57.906317 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3511/7050] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:28:57 2022] PUT /identity/v3/projects/5ac1a4622e9d427980de4bc6949b16b8/users/7ba0e966f2574366b4612bb633106e77/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:28:57.917772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb992c5-6830-4a9e-a3d2-bc0ff454c952 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:57.918395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb992c5-6830-4a9e-a3d2-bc0ff454c952 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:57.918603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb992c5-6830-4a9e-a3d2-bc0ff454c952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:57.953126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5eb992c5-6830-4a9e-a3d2-bc0ff454c952 None None] MFA Rules not processed for user `7ba0e966f2574366b4612bb633106e77`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:57.954414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5eb992c5-6830-4a9e-a3d2-bc0ff454c952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:58.065820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:58.079099 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3540/7051] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:58.098511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b62e38-4e50-4df8-87f8-cb177b3260eb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:58.100904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-85b62e38-4e50-4df8-87f8-cb177b3260eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:58.167474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b62e38-4e50-4df8-87f8-cb177b3260eb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:58.171442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2b2c0c-d3b9-4dff-b651-24eaed76f260 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:58.172106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b2c0c-d3b9-4dff-b651-24eaed76f260 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:58.172432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b2c0c-d3b9-4dff-b651-24eaed76f260 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:58.172683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2b2c0c-d3b9-4dff-b651-24eaed76f260 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:58.184164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b2c0c-d3b9-4dff-b651-24eaed76f260 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:58.185762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2b2c0c-d3b9-4dff-b651-24eaed76f260 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:58.200795 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3512/7052] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:58.790690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb992c5-6830-4a9e-a3d2-bc0ff454c952 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:58.856024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0db7d5-6bd8-47fb-8936-9393b8a9d212 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:58.856700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0db7d5-6bd8-47fb-8936-9393b8a9d212 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:58.857025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0db7d5-6bd8-47fb-8936-9393b8a9d212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:58.881659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-dd0db7d5-6bd8-47fb-8936-9393b8a9d212 None None] MFA Rules not processed for user `38d2c184e2ad446cb0481b1421439771`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:58.883415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-dd0db7d5-6bd8-47fb-8936-9393b8a9d212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:58.911889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:58.927398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb992c5-6830-4a9e-a3d2-bc0ff454c952 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:58.928169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502afa59-d369-4c2f-bf00-13aa2b7a947a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:58.929404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502afa59-d369-4c2f-bf00-13aa2b7a947a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:58.929933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502afa59-d369-4c2f-bf00-13aa2b7a947a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:58.930654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502afa59-d369-4c2f-bf00-13aa2b7a947a admin admin] PATH_INFO: `/v3/users/70979f8927cb48edb6b4e68a2e856b88` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:58.932979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502afa59-d369-4c2f-bf00-13aa2b7a947a admin admin] RBAC: Authorizing `identity:delete_user(user_id=70979f8927cb48edb6b4e68a2e856b88)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:58.935470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502afa59-d369-4c2f-bf00-13aa2b7a947a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:58.967725 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3513/7053] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:59.133496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-502afa59-d369-4c2f-bf00-13aa2b7a947a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70979f8927cb48edb6b4e68a2e856b88'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:59.146165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-502afa59-d369-4c2f-bf00-13aa2b7a947a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70979f8927cb48edb6b4e68a2e856b88'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:59.147328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-502afa59-d369-4c2f-bf00-13aa2b7a947a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70979f8927cb48edb6b4e68a2e856b88'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:59.148035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-502afa59-d369-4c2f-bf00-13aa2b7a947a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70979f8927cb48edb6b4e68a2e856b88'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:59.227911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-502afa59-d369-4c2f-bf00-13aa2b7a947a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70979f8927cb48edb6b4e68a2e856b88'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:28:59.277232 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3541/7054] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:28:58 2022] DELETE /identity/v3/users/70979f8927cb48edb6b4e68a2e856b88 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:28:59.282679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abb846-5c0d-43bd-95f4-8f64da091a60 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:59.282977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abb846-5c0d-43bd-95f4-8f64da091a60 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:59.283267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40abb846-5c0d-43bd-95f4-8f64da091a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:59.305781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-40abb846-5c0d-43bd-95f4-8f64da091a60 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:28:59.306406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:59.307409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-40abb846-5c0d-43bd-95f4-8f64da091a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:59.389456 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3514/7055] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:28:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:28:59.398244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502afa59-d369-4c2f-bf00-13aa2b7a947a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:59.400040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-502afa59-d369-4c2f-bf00-13aa2b7a947a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:59.491582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502afa59-d369-4c2f-bf00-13aa2b7a947a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:59.495899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c01c3-4a05-4f6e-8944-24ba5e7a0827 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:59.496581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c01c3-4a05-4f6e-8944-24ba5e7a0827 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:59.496871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c01c3-4a05-4f6e-8944-24ba5e7a0827 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:59.497154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384c01c3-4a05-4f6e-8944-24ba5e7a0827 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:59.522163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c01c3-4a05-4f6e-8944-24ba5e7a0827 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:59.524009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384c01c3-4a05-4f6e-8944-24ba5e7a0827 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:59.538071 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3542/7056] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:28:59.661519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abb846-5c0d-43bd-95f4-8f64da091a60 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:28:59.664633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-40abb846-5c0d-43bd-95f4-8f64da091a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:59.797658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40abb846-5c0d-43bd-95f4-8f64da091a60 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:28:59.807777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909737dd-d74c-423a-9c80-c96f4d046e4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:28:59.809617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909737dd-d74c-423a-9c80-c96f4d046e4d service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:28:59.809617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909737dd-d74c-423a-9c80-c96f4d046e4d service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:28:59.809617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909737dd-d74c-423a-9c80-c96f4d046e4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:28:59.819007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-909737dd-d74c-423a-9c80-c96f4d046e4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:28:59.836259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909737dd-d74c-423a-9c80-c96f4d046e4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:28:59.838130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909737dd-d74c-423a-9c80-c96f4d046e4d service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:28:59.857536 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3515/7057] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:28:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:29:00.438572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:29:00.439982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c01c3-4a05-4f6e-8944-24ba5e7a0827 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:00.442031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-384c01c3-4a05-4f6e-8944-24ba5e7a0827 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:00.455789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384c01c3-4a05-4f6e-8944-24ba5e7a0827 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:00.467948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dd68e-fd24-46bf-bc97-fca6062a5c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:00.469188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dd68e-fd24-46bf-bc97-fca6062a5c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:00.469864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dd68e-fd24-46bf-bc97-fca6062a5c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:00.470517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dd68e-fd24-46bf-bc97-fca6062a5c80 admin admin] PATH_INFO: `/v3/projects/09d66d43c6b54b9cbd75a69b5e3acb3b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:00.476049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936dd68e-fd24-46bf-bc97-fca6062a5c80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09d66d43c6b54b9cbd75a69b5e3acb3b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:00.476049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936dd68e-fd24-46bf-bc97-fca6062a5c80 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:00.592552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-936dd68e-fd24-46bf-bc97-fca6062a5c80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09d66d43c6b54b9cbd75a69b5e3acb3b', 'request_id': 'req-936dd68e-fd24-46bf-bc97-fca6062a5c80'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:00.605058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-936dd68e-fd24-46bf-bc97-fca6062a5c80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09d66d43c6b54b9cbd75a69b5e3acb3b', 'request_id': 'req-936dd68e-fd24-46bf-bc97-fca6062a5c80'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:00.682152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-936dd68e-fd24-46bf-bc97-fca6062a5c80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09d66d43c6b54b9cbd75a69b5e3acb3b', 'request_id': 'req-936dd68e-fd24-46bf-bc97-fca6062a5c80'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:00.704425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-936dd68e-fd24-46bf-bc97-fca6062a5c80 admin admin] The token cache is being invalidate because project 09d66d43c6b54b9cbd75a69b5e3acb3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:29:00.707846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-936dd68e-fd24-46bf-bc97-fca6062a5c80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:00.707846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-936dd68e-fd24-46bf-bc97-fca6062a5c80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:00.708919 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3543/7058] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:29:00 2022] DELETE /identity/v3/projects/09d66d43c6b54b9cbd75a69b5e3acb3b => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:29:00.713187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909737dd-d74c-423a-9c80-c96f4d046e4d service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:00.730543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909737dd-d74c-423a-9c80-c96f4d046e4d service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:00.758140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:29:00.810822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99435a-21de-4150-a580-91b1cfad4aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:00.811728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99435a-21de-4150-a580-91b1cfad4aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:00.812012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99435a-21de-4150-a580-91b1cfad4aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:00.812295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99435a-21de-4150-a580-91b1cfad4aa3 admin admin] PATH_INFO: `/v3/users/7ba0e966f2574366b4612bb633106e77` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:00.813740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99435a-21de-4150-a580-91b1cfad4aa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ba0e966f2574366b4612bb633106e77)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:00.815759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99435a-21de-4150-a580-91b1cfad4aa3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:01.111597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8c99435a-21de-4150-a580-91b1cfad4aa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ba0e966f2574366b4612bb633106e77'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:01.112263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8c99435a-21de-4150-a580-91b1cfad4aa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ba0e966f2574366b4612bb633106e77'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:01.120737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8c99435a-21de-4150-a580-91b1cfad4aa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ba0e966f2574366b4612bb633106e77'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:01.223725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8c99435a-21de-4150-a580-91b1cfad4aa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ba0e966f2574366b4612bb633106e77'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:01.224390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8c99435a-21de-4150-a580-91b1cfad4aa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ba0e966f2574366b4612bb633106e77'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:01.258704 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3516/7059] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:29:00 2022] DELETE /identity/v3/users/7ba0e966f2574366b4612bb633106e77 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:29:01.756536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dd68e-fd24-46bf-bc97-fca6062a5c80 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:01.758553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-936dd68e-fd24-46bf-bc97-fca6062a5c80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:01.773729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dd68e-fd24-46bf-bc97-fca6062a5c80 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:01.845454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c3e64-b192-421e-ae20-95b479ea0561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:01.846372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c3e64-b192-421e-ae20-95b479ea0561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:01.846684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c3e64-b192-421e-ae20-95b479ea0561 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:01.846962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c3e64-b192-421e-ae20-95b479ea0561 admin admin] PATH_INFO: `/v3/projects/5ac1a4622e9d427980de4bc6949b16b8` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:01.848687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c3e64-b192-421e-ae20-95b479ea0561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ac1a4622e9d427980de4bc6949b16b8)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:01.850708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c3e64-b192-421e-ae20-95b479ea0561 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:01.998857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dc4c3e64-b192-421e-ae20-95b479ea0561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ac1a4622e9d427980de4bc6949b16b8', 'request_id': 'req-dc4c3e64-b192-421e-ae20-95b479ea0561'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:02.015595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dc4c3e64-b192-421e-ae20-95b479ea0561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ac1a4622e9d427980de4bc6949b16b8', 'request_id': 'req-dc4c3e64-b192-421e-ae20-95b479ea0561'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:02.015595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:29:02.123645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dc4c3e64-b192-421e-ae20-95b479ea0561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ac1a4622e9d427980de4bc6949b16b8', 'request_id': 'req-dc4c3e64-b192-421e-ae20-95b479ea0561'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:02.137166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dc4c3e64-b192-421e-ae20-95b479ea0561 admin admin] The token cache is being invalidate because project 5ac1a4622e9d427980de4bc6949b16b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:29:02.137547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dc4c3e64-b192-421e-ae20-95b479ea0561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:02.138389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dc4c3e64-b192-421e-ae20-95b479ea0561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:02.144274 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3544/7060] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:29:01 2022] DELETE /identity/v3/projects/5ac1a4622e9d427980de4bc6949b16b8 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:29:02.147600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e6a2c-3515-450f-8db2-5a156e27dc63 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:02.148095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e6a2c-3515-450f-8db2-5a156e27dc63 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:02.148213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e6a2c-3515-450f-8db2-5a156e27dc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:02.167855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0f2e6a2c-3515-450f-8db2-5a156e27dc63 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:29:02.172922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0f2e6a2c-3515-450f-8db2-5a156e27dc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:02.236036 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3517/7061] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:29:02.244302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffa597-1ef0-498f-b6c2-4df432d1ca3e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:02.244596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffa597-1ef0-498f-b6c2-4df432d1ca3e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:02.244878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ffa597-1ef0-498f-b6c2-4df432d1ca3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:02.265574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b1ffa597-1ef0-498f-b6c2-4df432d1ca3e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:29:02.267441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b1ffa597-1ef0-498f-b6c2-4df432d1ca3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:02.276717 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3545/7062] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:29:02.281126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e6a2c-3515-450f-8db2-5a156e27dc63 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:02.296580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e6a2c-3515-450f-8db2-5a156e27dc63 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:02.302192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e5de5-776b-489c-8785-8f860d465218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:02.302772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e5de5-776b-489c-8785-8f860d465218 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:02.303056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e5de5-776b-489c-8785-8f860d465218 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:02.303478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e5de5-776b-489c-8785-8f860d465218 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:02.304216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e5de5-776b-489c-8785-8f860d465218 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:29:02.304653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e5de5-776b-489c-8785-8f860d465218 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:02.312750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777e5de5-776b-489c-8785-8f860d465218 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:02.329218 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3518/7063] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:29:02.335342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffa597-1ef0-498f-b6c2-4df432d1ca3e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:02.348659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ffa597-1ef0-498f-b6c2-4df432d1ca3e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:02.353557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b39882-5a8b-4fff-a612-addb0332b51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:02.354543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b39882-5a8b-4fff-a612-addb0332b51d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:02.354828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b39882-5a8b-4fff-a612-addb0332b51d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:02.355108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b39882-5a8b-4fff-a612-addb0332b51d admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:02.355811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b39882-5a8b-4fff-a612-addb0332b51d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:02.357880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b39882-5a8b-4fff-a612-addb0332b51d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:02.486209 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3546/7064] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:29:02 2022] POST /identity/v3/projects => generated 389 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:29:02.490536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e5de5-776b-489c-8785-8f860d465218 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:02.512454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e5de5-776b-489c-8785-8f860d465218 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:02.556003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:29:02.598087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169cd0c0-4ed6-4221-a781-d6ab5375494a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:02.599142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cd0c0-4ed6-4221-a781-d6ab5375494a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:02.600156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cd0c0-4ed6-4221-a781-d6ab5375494a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:02.600524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169cd0c0-4ed6-4221-a781-d6ab5375494a admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:02.601509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169cd0c0-4ed6-4221-a781-d6ab5375494a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:02.606109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169cd0c0-4ed6-4221-a781-d6ab5375494a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:02.730792 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3519/7065] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:29:02 2022] POST /identity/v3/users => generated 295 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:29:02.735594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b39882-5a8b-4fff-a612-addb0332b51d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:02.751607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b39882-5a8b-4fff-a612-addb0332b51d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:02.765472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e3ed0-a75f-4767-8183-531c1da6fc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:02.767060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e3ed0-a75f-4767-8183-531c1da6fc53 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:02.768560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e3ed0-a75f-4767-8183-531c1da6fc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:02.769038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e3ed0-a75f-4767-8183-531c1da6fc53 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:02.771123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e3ed0-a75f-4767-8183-531c1da6fc53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:02.774890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e3ed0-a75f-4767-8183-531c1da6fc53 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:02.794368 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3547/7066] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:29:02.798165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169cd0c0-4ed6-4221-a781-d6ab5375494a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:02.815399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169cd0c0-4ed6-4221-a781-d6ab5375494a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:02.832743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9560ef-20ed-4831-8f8b-d8f1db4a3a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:02.833611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9560ef-20ed-4831-8f8b-d8f1db4a3a2c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:02.833983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9560ef-20ed-4831-8f8b-d8f1db4a3a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:02.834268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9560ef-20ed-4831-8f8b-d8f1db4a3a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:02.835694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9560ef-20ed-4831-8f8b-d8f1db4a3a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:02.837688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9560ef-20ed-4831-8f8b-d8f1db4a3a2c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:02.854104 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3520/7067] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:29:02.855343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e3ed0-a75f-4767-8183-531c1da6fc53 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:02.870829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e3ed0-a75f-4767-8183-531c1da6fc53 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:02.879572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a328ea-c800-4d93-9ef4-11a6f6865f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:02.881046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a328ea-c800-4d93-9ef4-11a6f6865f66 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:02.881628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a328ea-c800-4d93-9ef4-11a6f6865f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:02.883922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a328ea-c800-4d93-9ef4-11a6f6865f66 admin admin] PATH_INFO: `/v3/projects/97cf01c13fa745a3bd04e35a0ce27088/users/642d08e104394dd28e6b4b502f5bc8cf/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:02.904317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a328ea-c800-4d93-9ef4-11a6f6865f66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97cf01c13fa745a3bd04e35a0ce27088, user_id=642d08e104394dd28e6b4b502f5bc8cf, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:02.907544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a328ea-c800-4d93-9ef4-11a6f6865f66 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:02.998373 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3548/7068] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:29:02 2022] PUT /identity/v3/projects/97cf01c13fa745a3bd04e35a0ce27088/users/642d08e104394dd28e6b4b502f5bc8cf/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:29:03.010814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7dc30-f2cc-4fb8-9dcc-963f5810c621 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:03.011162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7dc30-f2cc-4fb8-9dcc-963f5810c621 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:03.011462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b7dc30-f2cc-4fb8-9dcc-963f5810c621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:03.044141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-87b7dc30-f2cc-4fb8-9dcc-963f5810c621 None None] MFA Rules not processed for user `642d08e104394dd28e6b4b502f5bc8cf`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:29:03.045377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-87b7dc30-f2cc-4fb8-9dcc-963f5810c621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:03.123739 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3521/7069] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:29:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:29:03.124226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a328ea-c800-4d93-9ef4-11a6f6865f66 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:03.226728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a328ea-c800-4d93-9ef4-11a6f6865f66 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:03.232282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00290eb1-2d58-40e7-b0c7-57e034530186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:03.232974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00290eb1-2d58-40e7-b0c7-57e034530186 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:03.233350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00290eb1-2d58-40e7-b0c7-57e034530186 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:03.233640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00290eb1-2d58-40e7-b0c7-57e034530186 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:03.234394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00290eb1-2d58-40e7-b0c7-57e034530186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:03.236399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00290eb1-2d58-40e7-b0c7-57e034530186 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:03.349662 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3549/7070] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:29:03 2022] POST /identity/v3/projects => generated 385 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:29:03.354004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7dc30-f2cc-4fb8-9dcc-963f5810c621 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:03.366192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b7dc30-f2cc-4fb8-9dcc-963f5810c621 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:03.434455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e46cec-235f-4aca-bf5b-3b34f6421393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:03.435157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e46cec-235f-4aca-bf5b-3b34f6421393 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:03.435507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e46cec-235f-4aca-bf5b-3b34f6421393 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:03.435963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e46cec-235f-4aca-bf5b-3b34f6421393 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:03.436835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e46cec-235f-4aca-bf5b-3b34f6421393 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:03.439159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e46cec-235f-4aca-bf5b-3b34f6421393 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:03.532191 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3522/7071] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:29:03 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:29:03.534011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00290eb1-2d58-40e7-b0c7-57e034530186 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:03.546722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00290eb1-2d58-40e7-b0c7-57e034530186 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:03.553935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa4ac5-6313-40bc-9f72-b8b502a043c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:03.554723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa4ac5-6313-40bc-9f72-b8b502a043c9 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:03.555608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa4ac5-6313-40bc-9f72-b8b502a043c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:03.555608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa4ac5-6313-40bc-9f72-b8b502a043c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:03.556402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa4ac5-6313-40bc-9f72-b8b502a043c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:03.558156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfa4ac5-6313-40bc-9f72-b8b502a043c9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:03.571636 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3550/7072] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:29:03.576009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e46cec-235f-4aca-bf5b-3b34f6421393 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:03.588140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e46cec-235f-4aca-bf5b-3b34f6421393 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:03.593564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153c7f05-b1ca-4b67-8bc3-b32e58a557b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:03.594362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c7f05-b1ca-4b67-8bc3-b32e58a557b3 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:03.597200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c7f05-b1ca-4b67-8bc3-b32e58a557b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:03.597200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153c7f05-b1ca-4b67-8bc3-b32e58a557b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:03.597200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c7f05-b1ca-4b67-8bc3-b32e58a557b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:03.597698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153c7f05-b1ca-4b67-8bc3-b32e58a557b3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:03.613659 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3523/7073] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:29:03.615012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa4ac5-6313-40bc-9f72-b8b502a043c9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:03.630283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfa4ac5-6313-40bc-9f72-b8b502a043c9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:03.635872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8869e84-38af-4e79-bd67-fa7739f64295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:03.636696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8869e84-38af-4e79-bd67-fa7739f64295 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:03.636994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8869e84-38af-4e79-bd67-fa7739f64295 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:03.637284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8869e84-38af-4e79-bd67-fa7739f64295 admin admin] PATH_INFO: `/v3/projects/538959f5d1cc42698adcd7d2d5ae99e3/users/215d95f1b8cc43a2b027e05dfbff954c/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:03.661183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8869e84-38af-4e79-bd67-fa7739f64295 admin admin] RBAC: Authorizing `identity:create_grant(project_id=538959f5d1cc42698adcd7d2d5ae99e3, user_id=215d95f1b8cc43a2b027e05dfbff954c, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:03.665249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8869e84-38af-4e79-bd67-fa7739f64295 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:03.775836 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3551/7074] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:29:03 2022] PUT /identity/v3/projects/538959f5d1cc42698adcd7d2d5ae99e3/users/215d95f1b8cc43a2b027e05dfbff954c/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:29:03.788264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426be8f0-1b8e-4635-8406-ea705f226c97 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:03.788494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426be8f0-1b8e-4635-8406-ea705f226c97 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:03.788748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426be8f0-1b8e-4635-8406-ea705f226c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:03.820625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-426be8f0-1b8e-4635-8406-ea705f226c97 None None] MFA Rules not processed for user `215d95f1b8cc43a2b027e05dfbff954c`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:29:03.821990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-426be8f0-1b8e-4635-8406-ea705f226c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:03.828667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:29:03.929716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8869e84-38af-4e79-bd67-fa7739f64295 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:03.931587 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3524/7075] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:29:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:29:03.975214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:29:04.007664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8869e84-38af-4e79-bd67-fa7739f64295 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:04.013817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2fae5-3ad9-4d57-967f-de9ee9491e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:04.014545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2fae5-3ad9-4d57-967f-de9ee9491e60 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:04.014830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2fae5-3ad9-4d57-967f-de9ee9491e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:04.015114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d2fae5-3ad9-4d57-967f-de9ee9491e60 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:04.015821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d2fae5-3ad9-4d57-967f-de9ee9491e60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:04.018043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d2fae5-3ad9-4d57-967f-de9ee9491e60 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:04.117265 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3552/7076] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:29:03 2022] POST /identity/v3/projects => generated 389 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:29:04.122876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426be8f0-1b8e-4635-8406-ea705f226c97 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:04.136902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426be8f0-1b8e-4635-8406-ea705f226c97 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:04.224908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40ce8b-964d-4ba6-b413-773f54c94021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:04.225776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40ce8b-964d-4ba6-b413-773f54c94021 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:04.226107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40ce8b-964d-4ba6-b413-773f54c94021 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:04.226380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40ce8b-964d-4ba6-b413-773f54c94021 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:04.227071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40ce8b-964d-4ba6-b413-773f54c94021 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:04.229229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40ce8b-964d-4ba6-b413-773f54c94021 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:04.394445 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3525/7077] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:29:04 2022] POST /identity/v3/users => generated 301 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:29:04.395988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2fae5-3ad9-4d57-967f-de9ee9491e60 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:04.407824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d2fae5-3ad9-4d57-967f-de9ee9491e60 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:04.413088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0128e-b590-4d01-a9d5-4b5208bf438f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:04.413775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0128e-b590-4d01-a9d5-4b5208bf438f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:04.414113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0128e-b590-4d01-a9d5-4b5208bf438f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:04.414403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e0128e-b590-4d01-a9d5-4b5208bf438f admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:04.415194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0128e-b590-4d01-a9d5-4b5208bf438f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:04.416843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e0128e-b590-4d01-a9d5-4b5208bf438f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:04.428710 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3553/7078] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:29:04.432963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40ce8b-964d-4ba6-b413-773f54c94021 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:04.443592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40ce8b-964d-4ba6-b413-773f54c94021 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:04.452211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb05b0-fb45-4e52-a4fb-96d435ce3656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:04.452211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb05b0-fb45-4e52-a4fb-96d435ce3656 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:04.452211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb05b0-fb45-4e52-a4fb-96d435ce3656 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:04.452537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb05b0-fb45-4e52-a4fb-96d435ce3656 admin admin] PATH_INFO: `/v3/projects/2f92fc484a66469f8ccaeff81454ca7d/users/2858a8d717504381a81a93debfb03892/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:04.469743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb05b0-fb45-4e52-a4fb-96d435ce3656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f92fc484a66469f8ccaeff81454ca7d, user_id=2858a8d717504381a81a93debfb03892, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:04.475154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb05b0-fb45-4e52-a4fb-96d435ce3656 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:04.599914 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3526/7079] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:29:04 2022] PUT /identity/v3/projects/2f92fc484a66469f8ccaeff81454ca7d/users/2858a8d717504381a81a93debfb03892/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:29:04.612131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0128e-b590-4d01-a9d5-4b5208bf438f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:04.625728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e0128e-b590-4d01-a9d5-4b5208bf438f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:04.715811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3871d52-c072-4b4b-a21c-fceb991a8405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:04.716536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3871d52-c072-4b4b-a21c-fceb991a8405 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:04.716824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3871d52-c072-4b4b-a21c-fceb991a8405 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:04.717108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3871d52-c072-4b4b-a21c-fceb991a8405 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:04.717935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3871d52-c072-4b4b-a21c-fceb991a8405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:04.720026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3871d52-c072-4b4b-a21c-fceb991a8405 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:04.734250 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3554/7080] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:29:04.738551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb05b0-fb45-4e52-a4fb-96d435ce3656 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:04.750039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb05b0-fb45-4e52-a4fb-96d435ce3656 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:04.754826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc1d5b-241b-4e26-9244-6bb6a73dcce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:04.755764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc1d5b-241b-4e26-9244-6bb6a73dcce6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:04.756052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc1d5b-241b-4e26-9244-6bb6a73dcce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:04.756329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc1d5b-241b-4e26-9244-6bb6a73dcce6 admin admin] PATH_INFO: `/v3/projects/2f92fc484a66469f8ccaeff81454ca7d/users/2858a8d717504381a81a93debfb03892/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:04.758405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc1d5b-241b-4e26-9244-6bb6a73dcce6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f92fc484a66469f8ccaeff81454ca7d, user_id=2858a8d717504381a81a93debfb03892, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:04.761128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc1d5b-241b-4e26-9244-6bb6a73dcce6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:04.773148 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3527/7081] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:29:04 2022] PUT /identity/v3/projects/2f92fc484a66469f8ccaeff81454ca7d/users/2858a8d717504381a81a93debfb03892/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:29:04.788995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339877a-e902-43ce-9ddd-9387886bc9e2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:04.789606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339877a-e902-43ce-9ddd-9387886bc9e2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:04.789606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339877a-e902-43ce-9ddd-9387886bc9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:04.819749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a339877a-e902-43ce-9ddd-9387886bc9e2 None None] MFA Rules not processed for user `2858a8d717504381a81a93debfb03892`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:29:04.821029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a339877a-e902-43ce-9ddd-9387886bc9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:04.925878 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3555/7082] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:29:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:29:04.935191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc1d5b-241b-4e26-9244-6bb6a73dcce6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:04.937070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-abcc1d5b-241b-4e26-9244-6bb6a73dcce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:05.030622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:29:05.073524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc1d5b-241b-4e26-9244-6bb6a73dcce6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:05.090982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380da5c2-6c3e-4363-852a-e7bd5af7c4e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:05.091814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380da5c2-6c3e-4363-852a-e7bd5af7c4e6 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:05.092173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380da5c2-6c3e-4363-852a-e7bd5af7c4e6 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:05.092525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380da5c2-6c3e-4363-852a-e7bd5af7c4e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:05.169497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380da5c2-6c3e-4363-852a-e7bd5af7c4e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:05.171230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380da5c2-6c3e-4363-852a-e7bd5af7c4e6 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:05.181335 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3528/7083] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:29:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:29:05.335635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339877a-e902-43ce-9ddd-9387886bc9e2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:05.364693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339877a-e902-43ce-9ddd-9387886bc9e2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:05.376445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f84caf-4581-4339-a10e-0ffa4e9a480e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:05.377634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f84caf-4581-4339-a10e-0ffa4e9a480e service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:05.380129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f84caf-4581-4339-a10e-0ffa4e9a480e service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:05.380659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f84caf-4581-4339-a10e-0ffa4e9a480e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:05.384099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-86f84caf-4581-4339-a10e-0ffa4e9a480e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:05.402526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f84caf-4581-4339-a10e-0ffa4e9a480e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:05.404545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f84caf-4581-4339-a10e-0ffa4e9a480e service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:05.421236 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3556/7084] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:29:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:29:09.914133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380da5c2-6c3e-4363-852a-e7bd5af7c4e6 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:09.915370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-380da5c2-6c3e-4363-852a-e7bd5af7c4e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:09.989999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f84caf-4581-4339-a10e-0ffa4e9a480e service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:10.001959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380da5c2-6c3e-4363-852a-e7bd5af7c4e6 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:10.010935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f84caf-4581-4339-a10e-0ffa4e9a480e service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:10.017401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb6dade-4c5c-4df9-ba0d-522ad39c7a01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:10.018092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6dade-4c5c-4df9-ba0d-522ad39c7a01 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:10.018385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6dade-4c5c-4df9-ba0d-522ad39c7a01 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:10.018739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6dade-4c5c-4df9-ba0d-522ad39c7a01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:10.028137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098dc1c-ebbd-4bb2-ae07-69e294672bda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:10.029101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098dc1c-ebbd-4bb2-ae07-69e294672bda service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:10.029484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098dc1c-ebbd-4bb2-ae07-69e294672bda service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:10.029899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e098dc1c-ebbd-4bb2-ae07-69e294672bda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:10.034245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e098dc1c-ebbd-4bb2-ae07-69e294672bda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:10.037593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6dade-4c5c-4df9-ba0d-522ad39c7a01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:10.053968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb6dade-4c5c-4df9-ba0d-522ad39c7a01 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:10.068049 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3529/7085] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:29:09 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:29:10.072904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098dc1c-ebbd-4bb2-ae07-69e294672bda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:10.074700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e098dc1c-ebbd-4bb2-ae07-69e294672bda service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:10.108418 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3557/7086] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:29:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:29:12.209460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447830d-0b0e-4350-a904-ea495f358e7d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:12.211562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447830d-0b0e-4350-a904-ea495f358e7d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:12.212036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447830d-0b0e-4350-a904-ea495f358e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:12.240663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1447830d-0b0e-4350-a904-ea495f358e7d None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:29:12.242507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1447830d-0b0e-4350-a904-ea495f358e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:12.329655 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3530/7087] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:29:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:29:12.343767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098dc1c-ebbd-4bb2-ae07-69e294672bda service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:12.345217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e098dc1c-ebbd-4bb2-ae07-69e294672bda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:12.352265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:29:12.444891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e098dc1c-ebbd-4bb2-ae07-69e294672bda service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:12.460967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a946f7b-20a5-43c4-934b-8f55cc6ac8ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:12.462224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a946f7b-20a5-43c4-934b-8f55cc6ac8ec service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:12.463489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a946f7b-20a5-43c4-934b-8f55cc6ac8ec service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:12.464280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a946f7b-20a5-43c4-934b-8f55cc6ac8ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:12.482333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a946f7b-20a5-43c4-934b-8f55cc6ac8ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:12.486277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a946f7b-20a5-43c4-934b-8f55cc6ac8ec service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:12.498592 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3558/7088] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:29:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 21:29:15.190111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b01b04-3ef1-466f-afdb-eb6c3ce0a27f None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:15.194428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b01b04-3ef1-466f-afdb-eb6c3ce0a27f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:15.194428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b01b04-3ef1-466f-afdb-eb6c3ce0a27f None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:15.199757 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3531/7089] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:29:15 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:29:15.205873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc4375-2cf2-4c6f-8a80-2a26c5b6a44c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:15.207060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc4375-2cf2-4c6f-8a80-2a26c5b6a44c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:15.207060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc4375-2cf2-4c6f-8a80-2a26c5b6a44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:15.263371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-12dc4375-2cf2-4c6f-8a80-2a26c5b6a44c None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:29:15.264642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-12dc4375-2cf2-4c6f-8a80-2a26c5b6a44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:15.470853 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3559/7090] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:29:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:29:15.501149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b01b04-3ef1-466f-afdb-eb6c3ce0a27f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:15.503761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b6b01b04-3ef1-466f-afdb-eb6c3ce0a27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:15.567590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b01b04-3ef1-466f-afdb-eb6c3ce0a27f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:15.587261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1b68a-cf67-4a16-a579-4ff5ab1b670c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:15.588100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1b68a-cf67-4a16-a579-4ff5ab1b670c service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:15.589799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1b68a-cf67-4a16-a579-4ff5ab1b670c service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:15.597253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc1b68a-cf67-4a16-a579-4ff5ab1b670c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:15.605016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1b68a-cf67-4a16-a579-4ff5ab1b670c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:15.607990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc1b68a-cf67-4a16-a579-4ff5ab1b670c service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:15.626164 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3532/7091] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:29:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:29:16.137412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc4375-2cf2-4c6f-8a80-2a26c5b6a44c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:16.157367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc4375-2cf2-4c6f-8a80-2a26c5b6a44c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:16.174154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7680b94-7f5f-49bc-bab4-57eee78dd9a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:16.175894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7680b94-7f5f-49bc-bab4-57eee78dd9a2 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:16.180084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7680b94-7f5f-49bc-bab4-57eee78dd9a2 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:16.180084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7680b94-7f5f-49bc-bab4-57eee78dd9a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:16.181685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e7680b94-7f5f-49bc-bab4-57eee78dd9a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:16.220465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7680b94-7f5f-49bc-bab4-57eee78dd9a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:16.221545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7680b94-7f5f-49bc-bab4-57eee78dd9a2 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:16.258425 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3560/7092] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:29:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:29:16.928429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1b68a-cf67-4a16-a579-4ff5ab1b670c service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:16.930261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cfc1b68a-cf67-4a16-a579-4ff5ab1b670c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:16.966241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc1b68a-cf67-4a16-a579-4ff5ab1b670c service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:16.975270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703d28a-4039-45ad-bc38-fd7e079cd7ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:16.976763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703d28a-4039-45ad-bc38-fd7e079cd7ad service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:16.977630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703d28a-4039-45ad-bc38-fd7e079cd7ad service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:16.978281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703d28a-4039-45ad-bc38-fd7e079cd7ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:16.981044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f703d28a-4039-45ad-bc38-fd7e079cd7ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:17.001365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703d28a-4039-45ad-bc38-fd7e079cd7ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:17.004224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703d28a-4039-45ad-bc38-fd7e079cd7ad service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:17.035904 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3533/7093] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:29:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:29:18.098461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfacad5-ee66-4f96-ae27-6f073d2b4ab8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:18.098461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfacad5-ee66-4f96-ae27-6f073d2b4ab8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:18.098461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfacad5-ee66-4f96-ae27-6f073d2b4ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:18.120429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5dfacad5-ee66-4f96-ae27-6f073d2b4ab8 None None] MFA Rules not processed for user `642d08e104394dd28e6b4b502f5bc8cf`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:29:18.122029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5dfacad5-ee66-4f96-ae27-6f073d2b4ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:18.135807 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3561/7094] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:29:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:29:27.018678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:29:28.111479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=68 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:29:37.640591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703d28a-4039-45ad-bc38-fd7e079cd7ad service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:37.642732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f703d28a-4039-45ad-bc38-fd7e079cd7ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:37.669620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703d28a-4039-45ad-bc38-fd7e079cd7ad service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:37.683204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa940e6e-85b4-4f55-a214-e9a8e685d7d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:37.685078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa940e6e-85b4-4f55-a214-e9a8e685d7d8 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:37.687327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa940e6e-85b4-4f55-a214-e9a8e685d7d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:37.687819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa940e6e-85b4-4f55-a214-e9a8e685d7d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:37.692277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-aa940e6e-85b4-4f55-a214-e9a8e685d7d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:37.807051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa940e6e-85b4-4f55-a214-e9a8e685d7d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:37.809430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa940e6e-85b4-4f55-a214-e9a8e685d7d8 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:37.828371 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3534/7095] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:29:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:29:47.822430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=30 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:29:48.602915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfacad5-ee66-4f96-ae27-6f073d2b4ab8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:48.604538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5dfacad5-ee66-4f96-ae27-6f073d2b4ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:48.637711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfacad5-ee66-4f96-ae27-6f073d2b4ab8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:48.642806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f65e63-d744-4963-9531-79c755989306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:48.644262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f65e63-d744-4963-9531-79c755989306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:48.644555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f65e63-d744-4963-9531-79c755989306 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:48.644825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f65e63-d744-4963-9531-79c755989306 admin admin] PATH_INFO: `/v3/users/02d5fbb072814fd7923ecf12c7ecad7b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:48.646325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f65e63-d744-4963-9531-79c755989306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02d5fbb072814fd7923ecf12c7ecad7b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:48.648444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f65e63-d744-4963-9531-79c755989306 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:48.927231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c4f65e63-d744-4963-9531-79c755989306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02d5fbb072814fd7923ecf12c7ecad7b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:48.943716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c4f65e63-d744-4963-9531-79c755989306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02d5fbb072814fd7923ecf12c7ecad7b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:48.944480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c4f65e63-d744-4963-9531-79c755989306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02d5fbb072814fd7923ecf12c7ecad7b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:48.945109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c4f65e63-d744-4963-9531-79c755989306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02d5fbb072814fd7923ecf12c7ecad7b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:49.058866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c4f65e63-d744-4963-9531-79c755989306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02d5fbb072814fd7923ecf12c7ecad7b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:49.112950 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3562/7096] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:29:48 2022] DELETE /identity/v3/users/02d5fbb072814fd7923ecf12c7ecad7b => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:29:49.823631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa940e6e-85b4-4f55-a214-e9a8e685d7d8 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:49.918986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa940e6e-85b4-4f55-a214-e9a8e685d7d8 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:50.036558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961f4c7-450c-4e00-b250-60e4369ea3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:50.037774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961f4c7-450c-4e00-b250-60e4369ea3ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:50.038219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961f4c7-450c-4e00-b250-60e4369ea3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:50.038647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961f4c7-450c-4e00-b250-60e4369ea3ac admin admin] PATH_INFO: `/v3/projects/54447ea481574a128f06c24315f34e46` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:50.040211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961f4c7-450c-4e00-b250-60e4369ea3ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=54447ea481574a128f06c24315f34e46)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:50.043074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961f4c7-450c-4e00-b250-60e4369ea3ac admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:50.240113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b961f4c7-450c-4e00-b250-60e4369ea3ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54447ea481574a128f06c24315f34e46', 'request_id': 'req-b961f4c7-450c-4e00-b250-60e4369ea3ac'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:50.250806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b961f4c7-450c-4e00-b250-60e4369ea3ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54447ea481574a128f06c24315f34e46', 'request_id': 'req-b961f4c7-450c-4e00-b250-60e4369ea3ac'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:50.373270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b961f4c7-450c-4e00-b250-60e4369ea3ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54447ea481574a128f06c24315f34e46', 'request_id': 'req-b961f4c7-450c-4e00-b250-60e4369ea3ac'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:50.418893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b961f4c7-450c-4e00-b250-60e4369ea3ac admin admin] The token cache is being invalidate because project 54447ea481574a128f06c24315f34e46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:29:50.419196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b961f4c7-450c-4e00-b250-60e4369ea3ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:50.419881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b961f4c7-450c-4e00-b250-60e4369ea3ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:29:50.425813 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3535/7097] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:29:49 2022] DELETE /identity/v3/projects/54447ea481574a128f06c24315f34e46 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:29:50.429416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4053f7-65a4-40fc-b787-fc28e01b694d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:50.430608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4053f7-65a4-40fc-b787-fc28e01b694d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:50.431360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4053f7-65a4-40fc-b787-fc28e01b694d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:50.483273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1a4053f7-65a4-40fc-b787-fc28e01b694d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:29:50.484676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1a4053f7-65a4-40fc-b787-fc28e01b694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:50.651194 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3563/7098] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:29:50.659390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01423e-7b84-4660-aa2a-26c8eb658df3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:50.660571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01423e-7b84-4660-aa2a-26c8eb658df3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:50.660571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01423e-7b84-4660-aa2a-26c8eb658df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:50.686199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4f01423e-7b84-4660-aa2a-26c8eb658df3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:29:50.687997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4f01423e-7b84-4660-aa2a-26c8eb658df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:50.700236 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3536/7099] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:29:50.703284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4053f7-65a4-40fc-b787-fc28e01b694d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:50.734130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4053f7-65a4-40fc-b787-fc28e01b694d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:50.744067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b615727-ee55-4e6c-b540-a18a88a62d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:50.745045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b615727-ee55-4e6c-b540-a18a88a62d9b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:50.746275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b615727-ee55-4e6c-b540-a18a88a62d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:50.746752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b615727-ee55-4e6c-b540-a18a88a62d9b admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:50.751106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b615727-ee55-4e6c-b540-a18a88a62d9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:29:50.751615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b615727-ee55-4e6c-b540-a18a88a62d9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:50.754539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b615727-ee55-4e6c-b540-a18a88a62d9b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:50.788018 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3564/7100] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:29:50.789999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01423e-7b84-4660-aa2a-26c8eb658df3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:50.832368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01423e-7b84-4660-aa2a-26c8eb658df3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:50.840510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd640fd-4028-4847-b438-8d43e086607c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:50.840510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd640fd-4028-4847-b438-8d43e086607c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:50.840510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd640fd-4028-4847-b438-8d43e086607c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:50.840510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd640fd-4028-4847-b438-8d43e086607c admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:50.840957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd640fd-4028-4847-b438-8d43e086607c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:50.841998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd640fd-4028-4847-b438-8d43e086607c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:50.955784 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3537/7101] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:29:50 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:29:50.957117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b615727-ee55-4e6c-b540-a18a88a62d9b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:50.972455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b615727-ee55-4e6c-b540-a18a88a62d9b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:51.077930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:29:51.108381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27107752-6fdb-4ae8-972b-b54363184ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:51.109175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27107752-6fdb-4ae8-972b-b54363184ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:51.109562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27107752-6fdb-4ae8-972b-b54363184ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:51.109961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27107752-6fdb-4ae8-972b-b54363184ee5 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:51.110907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27107752-6fdb-4ae8-972b-b54363184ee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:51.113141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27107752-6fdb-4ae8-972b-b54363184ee5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:51.214405 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3565/7102] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:29:50 2022] POST /identity/v3/users => generated 292 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:29:51.218754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd640fd-4028-4847-b438-8d43e086607c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:51.230214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd640fd-4028-4847-b438-8d43e086607c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:51.243901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbbe92-6dd1-463b-bff6-3c2852c1f972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:51.248479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbbe92-6dd1-463b-bff6-3c2852c1f972 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:51.248479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbbe92-6dd1-463b-bff6-3c2852c1f972 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:51.248479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbbe92-6dd1-463b-bff6-3c2852c1f972 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:51.250576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbbe92-6dd1-463b-bff6-3c2852c1f972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:51.256192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffbbe92-6dd1-463b-bff6-3c2852c1f972 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:51.274768 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3538/7103] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:29:51.278976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27107752-6fdb-4ae8-972b-b54363184ee5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:51.299400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27107752-6fdb-4ae8-972b-b54363184ee5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:51.307110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a3868-66ea-4ccb-a565-d1bda07db380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:51.308298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a3868-66ea-4ccb-a565-d1bda07db380 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:51.308454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a3868-66ea-4ccb-a565-d1bda07db380 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:51.308613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a3868-66ea-4ccb-a565-d1bda07db380 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:51.309582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a3868-66ea-4ccb-a565-d1bda07db380 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:51.311540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a3868-66ea-4ccb-a565-d1bda07db380 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:51.326515 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3566/7104] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:29:51.331035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbbe92-6dd1-463b-bff6-3c2852c1f972 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:51.372788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbbe92-6dd1-463b-bff6-3c2852c1f972 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:51.380852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e4f21-ef32-4348-a313-a8d3bfb49406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:51.382134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e4f21-ef32-4348-a313-a8d3bfb49406 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:51.382403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e4f21-ef32-4348-a313-a8d3bfb49406 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:51.382875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1e4f21-ef32-4348-a313-a8d3bfb49406 admin admin] PATH_INFO: `/v3/projects/2ba32c313b5d4d728ceb5ebadab52c9c/users/4c36e5d440be496693c66e5bbc1ef060/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:51.411721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e4f21-ef32-4348-a313-a8d3bfb49406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba32c313b5d4d728ceb5ebadab52c9c, user_id=4c36e5d440be496693c66e5bbc1ef060, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:51.414688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1e4f21-ef32-4348-a313-a8d3bfb49406 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:51.517645 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3539/7105] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:29:51 2022] PUT /identity/v3/projects/2ba32c313b5d4d728ceb5ebadab52c9c/users/4c36e5d440be496693c66e5bbc1ef060/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:29:51.523782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e06b5-c12a-4cc7-b769-068fe8affa55 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:51.524280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e06b5-c12a-4cc7-b769-068fe8affa55 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:51.524280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e06b5-c12a-4cc7-b769-068fe8affa55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:51.542964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fc6e06b5-c12a-4cc7-b769-068fe8affa55 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:29:51.546478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fc6e06b5-c12a-4cc7-b769-068fe8affa55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:51.630411 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3567/7106] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:29:51.683460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e4f21-ef32-4348-a313-a8d3bfb49406 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:51.687230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2f1e4f21-ef32-4348-a313-a8d3bfb49406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:51.737976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:29:51.818224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1e4f21-ef32-4348-a313-a8d3bfb49406 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:51.818224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d1f12-b51e-4290-a236-3a773a09f045 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:51.818224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d1f12-b51e-4290-a236-3a773a09f045 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:51.818224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d1f12-b51e-4290-a236-3a773a09f045 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:51.818666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d1f12-b51e-4290-a236-3a773a09f045 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:51.828667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d1f12-b51e-4290-a236-3a773a09f045 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:51.830198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d1f12-b51e-4290-a236-3a773a09f045 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:51.841779 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3540/7107] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:29:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:29:53.655623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa94a2-380e-437f-8b07-b46590d00f49 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:53.656107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa94a2-380e-437f-8b07-b46590d00f49 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:53.656423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfa94a2-380e-437f-8b07-b46590d00f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:53.704320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2cfa94a2-380e-437f-8b07-b46590d00f49 None None] MFA Rules not processed for user `4c36e5d440be496693c66e5bbc1ef060`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:29:53.706127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2cfa94a2-380e-437f-8b07-b46590d00f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:53.810081 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3568/7108] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:29:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:29:53.821573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002b006-d9cd-4ce0-8f6b-67f52cce968f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:53.822127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002b006-d9cd-4ce0-8f6b-67f52cce968f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:53.822195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002b006-d9cd-4ce0-8f6b-67f52cce968f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:53.847525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1002b006-d9cd-4ce0-8f6b-67f52cce968f None None] MFA Rules not processed for user `4c36e5d440be496693c66e5bbc1ef060`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:29:53.848770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1002b006-d9cd-4ce0-8f6b-67f52cce968f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:53.857740 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3541/7109] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:29:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:29:53.871567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa94a2-380e-437f-8b07-b46590d00f49 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:29:53.873304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2cfa94a2-380e-437f-8b07-b46590d00f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:29:53.952282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:29:53.998876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfa94a2-380e-437f-8b07-b46590d00f49 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:29:54.009349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6672e-cc47-471f-a103-01a586fde31a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:29:54.010071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6672e-cc47-471f-a103-01a586fde31a service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:29:54.010403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6672e-cc47-471f-a103-01a586fde31a service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:29:54.022583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c6672e-cc47-471f-a103-01a586fde31a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:29:54.054947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c6672e-cc47-471f-a103-01a586fde31a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:29:54.057826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c6672e-cc47-471f-a103-01a586fde31a service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:29:54.090041 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3569/7110] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:29:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:30:03.893921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=25 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:04.773574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:05.160992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351151cf-729a-4cd3-a9eb-e9a762eab628 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:05.161927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351151cf-729a-4cd3-a9eb-e9a762eab628 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:05.162414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351151cf-729a-4cd3-a9eb-e9a762eab628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:05.182249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-351151cf-729a-4cd3-a9eb-e9a762eab628 None None] MFA Rules not processed for user `f3468a0b2d324603ab6c3080993bc0c8`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:05.183807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-351151cf-729a-4cd3-a9eb-e9a762eab628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:05.276921 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3542/7111] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:15.264180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=25 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:18.807673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6672e-cc47-471f-a103-01a586fde31a service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:18.833291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c6672e-cc47-471f-a103-01a586fde31a service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:18.850730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc8e00-3aa3-4e39-a766-eda96bee4a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:18.852536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc8e00-3aa3-4e39-a766-eda96bee4a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:18.853171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc8e00-3aa3-4e39-a766-eda96bee4a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:18.853794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc8e00-3aa3-4e39-a766-eda96bee4a23 admin admin] PATH_INFO: `/v3/users/4c36e5d440be496693c66e5bbc1ef060` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:18.855525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc8e00-3aa3-4e39-a766-eda96bee4a23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c36e5d440be496693c66e5bbc1ef060)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:18.857764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc8e00-3aa3-4e39-a766-eda96bee4a23 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:19.223885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6efc8e00-3aa3-4e39-a766-eda96bee4a23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c36e5d440be496693c66e5bbc1ef060'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:19.257803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6efc8e00-3aa3-4e39-a766-eda96bee4a23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c36e5d440be496693c66e5bbc1ef060'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:19.259034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6efc8e00-3aa3-4e39-a766-eda96bee4a23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c36e5d440be496693c66e5bbc1ef060'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:19.260101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6efc8e00-3aa3-4e39-a766-eda96bee4a23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c36e5d440be496693c66e5bbc1ef060'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:19.407886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6efc8e00-3aa3-4e39-a766-eda96bee4a23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c36e5d440be496693c66e5bbc1ef060'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:19.436165 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3570/7112] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:30:18 2022] DELETE /identity/v3/users/4c36e5d440be496693c66e5bbc1ef060 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:30:20.029714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351151cf-729a-4cd3-a9eb-e9a762eab628 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:20.049130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351151cf-729a-4cd3-a9eb-e9a762eab628 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:20.274415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004ad23-72b4-4c45-b6ff-3f09a6b74894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:20.276056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004ad23-72b4-4c45-b6ff-3f09a6b74894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:20.276458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004ad23-72b4-4c45-b6ff-3f09a6b74894 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:20.276809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004ad23-72b4-4c45-b6ff-3f09a6b74894 admin admin] PATH_INFO: `/v3/projects/2ba32c313b5d4d728ceb5ebadab52c9c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:20.278241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004ad23-72b4-4c45-b6ff-3f09a6b74894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ba32c313b5d4d728ceb5ebadab52c9c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:20.280267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004ad23-72b4-4c45-b6ff-3f09a6b74894 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:20.419547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6004ad23-72b4-4c45-b6ff-3f09a6b74894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ba32c313b5d4d728ceb5ebadab52c9c', 'request_id': 'req-6004ad23-72b4-4c45-b6ff-3f09a6b74894'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:20.439062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6004ad23-72b4-4c45-b6ff-3f09a6b74894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ba32c313b5d4d728ceb5ebadab52c9c', 'request_id': 'req-6004ad23-72b4-4c45-b6ff-3f09a6b74894'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:20.484146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc8e00-3aa3-4e39-a766-eda96bee4a23 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:20.485753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6efc8e00-3aa3-4e39-a766-eda96bee4a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:20.611655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6004ad23-72b4-4c45-b6ff-3f09a6b74894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ba32c313b5d4d728ceb5ebadab52c9c', 'request_id': 'req-6004ad23-72b4-4c45-b6ff-3f09a6b74894'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:20.635471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6004ad23-72b4-4c45-b6ff-3f09a6b74894 admin admin] The token cache is being invalidate because project 2ba32c313b5d4d728ceb5ebadab52c9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:30:20.635770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6004ad23-72b4-4c45-b6ff-3f09a6b74894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:20.636822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6004ad23-72b4-4c45-b6ff-3f09a6b74894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:20.640225 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3543/7113] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:30:20 2022] DELETE /identity/v3/projects/2ba32c313b5d4d728ceb5ebadab52c9c => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:30:20.649455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c1d42-59e6-4911-9fae-5d65f88d77a1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:20.649664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c1d42-59e6-4911-9fae-5d65f88d77a1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:20.650000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669c1d42-59e6-4911-9fae-5d65f88d77a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:20.666095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc8e00-3aa3-4e39-a766-eda96bee4a23 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:20.688333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-669c1d42-59e6-4911-9fae-5d65f88d77a1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:20.690442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-669c1d42-59e6-4911-9fae-5d65f88d77a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:20.815785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228f151-d361-4562-b9b9-7c2a78a665ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:20.816468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228f151-d361-4562-b9b9-7c2a78a665ab service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:20.816759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228f151-d361-4562-b9b9-7c2a78a665ab service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:20.817047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228f151-d361-4562-b9b9-7c2a78a665ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:20.820815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0228f151-d361-4562-b9b9-7c2a78a665ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:20.832814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:20.842915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228f151-d361-4562-b9b9-7c2a78a665ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:20.845587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0228f151-d361-4562-b9b9-7c2a78a665ab service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:20.864803 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3544/7114] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:20.865046 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3571/7115] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:30:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:30:20.879688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f84f2-9c82-439a-8820-0a76a45d0e23 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:20.879999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f84f2-9c82-439a-8820-0a76a45d0e23 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:20.880286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f84f2-9c82-439a-8820-0a76a45d0e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:20.942127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c71f84f2-9c82-439a-8820-0a76a45d0e23 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:20.943707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c71f84f2-9c82-439a-8820-0a76a45d0e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:20.964118 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3545/7116] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:20.968540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228f151-d361-4562-b9b9-7c2a78a665ab service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:20.991138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228f151-d361-4562-b9b9-7c2a78a665ab service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:20.991138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95843e7-9550-402e-a927-c7af335fa27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:20.991138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95843e7-9550-402e-a927-c7af335fa27a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:20.991138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95843e7-9550-402e-a927-c7af335fa27a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:20.991436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95843e7-9550-402e-a927-c7af335fa27a admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:20.991436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95843e7-9550-402e-a927-c7af335fa27a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:30:20.991436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95843e7-9550-402e-a927-c7af335fa27a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:20.992634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95843e7-9550-402e-a927-c7af335fa27a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:21.013637 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3572/7117] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:30:21.022806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f84f2-9c82-439a-8820-0a76a45d0e23 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:21.044533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f84f2-9c82-439a-8820-0a76a45d0e23 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:21.066592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663984fc-f219-46a6-b1c5-31c639efa023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:21.067284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663984fc-f219-46a6-b1c5-31c639efa023 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:21.068108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663984fc-f219-46a6-b1c5-31c639efa023 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:21.069650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663984fc-f219-46a6-b1c5-31c639efa023 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:21.070778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663984fc-f219-46a6-b1c5-31c639efa023 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:21.078615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663984fc-f219-46a6-b1c5-31c639efa023 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:21.326442 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3546/7118] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:30:21 2022] POST /identity/v3/projects => generated 399 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:30:21.330756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95843e7-9550-402e-a927-c7af335fa27a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:21.342872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95843e7-9550-402e-a927-c7af335fa27a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:21.451518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8a9e8d-5bef-48f9-b1c5-c91d14d9af67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:21.453035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a9e8d-5bef-48f9-b1c5-c91d14d9af67 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:21.454023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a9e8d-5bef-48f9-b1c5-c91d14d9af67 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:21.458366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a9e8d-5bef-48f9-b1c5-c91d14d9af67 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:21.458366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8a9e8d-5bef-48f9-b1c5-c91d14d9af67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:21.458366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8a9e8d-5bef-48f9-b1c5-c91d14d9af67 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:21.568053 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3573/7119] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:30:21 2022] POST /identity/v3/users => generated 300 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:30:21.572438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663984fc-f219-46a6-b1c5-31c639efa023 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:21.578535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:21.584223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663984fc-f219-46a6-b1c5-31c639efa023 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:21.590085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538c56c-d1f5-494c-a342-9337872dda83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:21.590841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538c56c-d1f5-494c-a342-9337872dda83 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:21.591212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538c56c-d1f5-494c-a342-9337872dda83 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:21.591856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a538c56c-d1f5-494c-a342-9337872dda83 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:21.592824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538c56c-d1f5-494c-a342-9337872dda83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:21.594561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a538c56c-d1f5-494c-a342-9337872dda83 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:21.609908 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3547/7120] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:30:21.613135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8a9e8d-5bef-48f9-b1c5-c91d14d9af67 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:21.629454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8a9e8d-5bef-48f9-b1c5-c91d14d9af67 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:21.633200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621ef74-67a2-4cac-8b3c-6bc3b4fc347d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:21.634088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621ef74-67a2-4cac-8b3c-6bc3b4fc347d admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:21.634443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621ef74-67a2-4cac-8b3c-6bc3b4fc347d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:21.634792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621ef74-67a2-4cac-8b3c-6bc3b4fc347d admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:21.635601 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621ef74-67a2-4cac-8b3c-6bc3b4fc347d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:21.637718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4621ef74-67a2-4cac-8b3c-6bc3b4fc347d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:21.653266 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3574/7121] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:30:21 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:30:21.660603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538c56c-d1f5-494c-a342-9337872dda83 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:21.672531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a538c56c-d1f5-494c-a342-9337872dda83 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:21.677536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99590a-7798-4192-a3e5-b7739d0edb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:21.678667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99590a-7798-4192-a3e5-b7739d0edb7e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:21.679143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99590a-7798-4192-a3e5-b7739d0edb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:21.679689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99590a-7798-4192-a3e5-b7739d0edb7e admin admin] PATH_INFO: `/v3/projects/239c9c3debdd456ab6e9c0545ab47fe2/users/5b4a37f2a8ec4160bc4db12c0d3e7de5/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:21.695927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99590a-7798-4192-a3e5-b7739d0edb7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=239c9c3debdd456ab6e9c0545ab47fe2, user_id=5b4a37f2a8ec4160bc4db12c0d3e7de5, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:21.698833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb99590a-7798-4192-a3e5-b7739d0edb7e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:21.770112 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3548/7122] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:30:21 2022] PUT /identity/v3/projects/239c9c3debdd456ab6e9c0545ab47fe2/users/5b4a37f2a8ec4160bc4db12c0d3e7de5/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:30:21.772128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64447cc1-d626-4820-8e7c-63fff67649e7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:21.772487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64447cc1-d626-4820-8e7c-63fff67649e7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:21.772860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64447cc1-d626-4820-8e7c-63fff67649e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:21.796011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-64447cc1-d626-4820-8e7c-63fff67649e7 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:21.797354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-64447cc1-d626-4820-8e7c-63fff67649e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:21.862751 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3575/7123] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:21.875898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99590a-7798-4192-a3e5-b7739d0edb7e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:21.877919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cb99590a-7798-4192-a3e5-b7739d0edb7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:21.958440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99590a-7798-4192-a3e5-b7739d0edb7e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:21.962881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f077d8-1a24-43d4-a151-4a5fdf6c85ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:21.963544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f077d8-1a24-43d4-a151-4a5fdf6c85ac service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:21.963835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f077d8-1a24-43d4-a151-4a5fdf6c85ac service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:21.964119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f077d8-1a24-43d4-a151-4a5fdf6c85ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:21.975863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f077d8-1a24-43d4-a151-4a5fdf6c85ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:21.977365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f077d8-1a24-43d4-a151-4a5fdf6c85ac service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:21.987999 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3549/7124] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:30:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:30:23.877204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93873c0f-a1cf-47b7-bf2b-caba8e342565 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:23.877754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93873c0f-a1cf-47b7-bf2b-caba8e342565 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:23.878095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93873c0f-a1cf-47b7-bf2b-caba8e342565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:23.898894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-93873c0f-a1cf-47b7-bf2b-caba8e342565 None None] MFA Rules not processed for user `642d08e104394dd28e6b4b502f5bc8cf`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:23.900830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-93873c0f-a1cf-47b7-bf2b-caba8e342565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:23.981210 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3576/7125] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:24.637375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18b34f-9ce6-444e-bdb5-6d5d43c5917b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:24.638300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18b34f-9ce6-444e-bdb5-6d5d43c5917b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:24.638736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18b34f-9ce6-444e-bdb5-6d5d43c5917b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:24.726544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1e18b34f-9ce6-444e-bdb5-6d5d43c5917b None None] MFA Rules not processed for user `5b4a37f2a8ec4160bc4db12c0d3e7de5`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:24.730400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1e18b34f-9ce6-444e-bdb5-6d5d43c5917b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:24.894885 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3550/7126] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:24.987553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb7815-9bcc-415d-b544-b902a66b85c1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:24.988549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb7815-9bcc-415d-b544-b902a66b85c1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:24.988957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb7815-9bcc-415d-b544-b902a66b85c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:25.008962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:25.025389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2afb7815-9bcc-415d-b544-b902a66b85c1 None None] MFA Rules not processed for user `5b4a37f2a8ec4160bc4db12c0d3e7de5`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:25.031526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2afb7815-9bcc-415d-b544-b902a66b85c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:25.075576 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3577/7127] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:25.382764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18b34f-9ce6-444e-bdb5-6d5d43c5917b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:25.384721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1e18b34f-9ce6-444e-bdb5-6d5d43c5917b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:25.540314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:25.576540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18b34f-9ce6-444e-bdb5-6d5d43c5917b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:25.582154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4141b-52bc-44b5-b399-3dded434265e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:25.582799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4141b-52bc-44b5-b399-3dded434265e service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:25.583085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4141b-52bc-44b5-b399-3dded434265e service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:25.583357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e4141b-52bc-44b5-b399-3dded434265e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:25.594590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4141b-52bc-44b5-b399-3dded434265e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:25.596250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e4141b-52bc-44b5-b399-3dded434265e service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:25.662231 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3551/7128] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:30:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:30:26.136555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb7815-9bcc-415d-b544-b902a66b85c1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:26.138035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2afb7815-9bcc-415d-b544-b902a66b85c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:26.152863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb7815-9bcc-415d-b544-b902a66b85c1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:26.168243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afada1d6-3e72-4f01-954d-031fd142cceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:26.168243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afada1d6-3e72-4f01-954d-031fd142cceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:26.168243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afada1d6-3e72-4f01-954d-031fd142cceb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:26.168243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afada1d6-3e72-4f01-954d-031fd142cceb admin admin] PATH_INFO: `/v3/users/6119da412dd34694b3b888f5c225b3f3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:26.168707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4141b-52bc-44b5-b399-3dded434265e service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:26.168799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afada1d6-3e72-4f01-954d-031fd142cceb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6119da412dd34694b3b888f5c225b3f3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:26.168875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-73e4141b-52bc-44b5-b399-3dded434265e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:26.168947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afada1d6-3e72-4f01-954d-031fd142cceb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:26.187392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e4141b-52bc-44b5-b399-3dded434265e service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:26.192881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e7ba63-7adb-4e13-8e10-20ce1c34de2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:26.196654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7ba63-7adb-4e13-8e10-20ce1c34de2d service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:26.196654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7ba63-7adb-4e13-8e10-20ce1c34de2d service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:26.196654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e7ba63-7adb-4e13-8e10-20ce1c34de2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:26.197814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-21e7ba63-7adb-4e13-8e10-20ce1c34de2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:26.225222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7ba63-7adb-4e13-8e10-20ce1c34de2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:26.226913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e7ba63-7adb-4e13-8e10-20ce1c34de2d service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:26.256640 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3552/7129] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:30:26 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:30:26.488583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-afada1d6-3e72-4f01-954d-031fd142cceb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6119da412dd34694b3b888f5c225b3f3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:26.504151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-afada1d6-3e72-4f01-954d-031fd142cceb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6119da412dd34694b3b888f5c225b3f3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:26.504952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-afada1d6-3e72-4f01-954d-031fd142cceb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6119da412dd34694b3b888f5c225b3f3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:26.505620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-afada1d6-3e72-4f01-954d-031fd142cceb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6119da412dd34694b3b888f5c225b3f3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:26.607790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-afada1d6-3e72-4f01-954d-031fd142cceb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6119da412dd34694b3b888f5c225b3f3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:26.631189 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3578/7130] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:30:26 2022] DELETE /identity/v3/users/6119da412dd34694b3b888f5c225b3f3 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:30:26.635633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38d787-8a16-4d3a-be2a-1da739f83646 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:26.636213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38d787-8a16-4d3a-be2a-1da739f83646 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:26.636675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd38d787-8a16-4d3a-be2a-1da739f83646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:26.659347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-cd38d787-8a16-4d3a-be2a-1da739f83646 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:26.661184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cd38d787-8a16-4d3a-be2a-1da739f83646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:26.783122 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3553/7131] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:26.793682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afada1d6-3e72-4f01-954d-031fd142cceb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:26.795269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-afada1d6-3e72-4f01-954d-031fd142cceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:26.902222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afada1d6-3e72-4f01-954d-031fd142cceb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:26.907517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d382b9-a5e6-4152-a7e0-b421d4e3338d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:26.908756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d382b9-a5e6-4152-a7e0-b421d4e3338d service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:26.909041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d382b9-a5e6-4152-a7e0-b421d4e3338d service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:26.909317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d382b9-a5e6-4152-a7e0-b421d4e3338d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:26.933715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d382b9-a5e6-4152-a7e0-b421d4e3338d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:26.934301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d382b9-a5e6-4152-a7e0-b421d4e3338d service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:26.974719 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3579/7132] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:30:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:30:27.422246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38d787-8a16-4d3a-be2a-1da739f83646 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:27.422246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cd38d787-8a16-4d3a-be2a-1da739f83646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:27.433238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd38d787-8a16-4d3a-be2a-1da739f83646 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:27.438719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5411f4-a89a-482b-a089-b55cb051634b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:27.440191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5411f4-a89a-482b-a089-b55cb051634b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:27.440727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5411f4-a89a-482b-a089-b55cb051634b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:27.441230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5411f4-a89a-482b-a089-b55cb051634b admin admin] PATH_INFO: `/v3/projects/f11f466272ba42be98bf2ea8d724c080` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:27.442936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5411f4-a89a-482b-a089-b55cb051634b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f11f466272ba42be98bf2ea8d724c080)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:27.445701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5411f4-a89a-482b-a089-b55cb051634b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:27.660606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0b5411f4-a89a-482b-a089-b55cb051634b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f11f466272ba42be98bf2ea8d724c080', 'request_id': 'req-0b5411f4-a89a-482b-a089-b55cb051634b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:27.666306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0b5411f4-a89a-482b-a089-b55cb051634b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f11f466272ba42be98bf2ea8d724c080', 'request_id': 'req-0b5411f4-a89a-482b-a089-b55cb051634b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:27.870228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0b5411f4-a89a-482b-a089-b55cb051634b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f11f466272ba42be98bf2ea8d724c080', 'request_id': 'req-0b5411f4-a89a-482b-a089-b55cb051634b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:27.888700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0b5411f4-a89a-482b-a089-b55cb051634b admin admin] The token cache is being invalidate because project f11f466272ba42be98bf2ea8d724c080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:30:27.894277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0b5411f4-a89a-482b-a089-b55cb051634b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:27.894277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0b5411f4-a89a-482b-a089-b55cb051634b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:27.894808 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3554/7133] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:30:27 2022] DELETE /identity/v3/projects/f11f466272ba42be98bf2ea8d724c080 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:30:27.900152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2b006-d192-4c83-a70b-77fbc71bb588 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:27.900697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2b006-d192-4c83-a70b-77fbc71bb588 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:27.901449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2b006-d192-4c83-a70b-77fbc71bb588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:27.928806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8ba2b006-d192-4c83-a70b-77fbc71bb588 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:27.930198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8ba2b006-d192-4c83-a70b-77fbc71bb588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:28.059936 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3580/7134] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:30:28.060180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128246c3-d32f-470c-9229-2662f5701683 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:28.060180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128246c3-d32f-470c-9229-2662f5701683 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:28.060180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128246c3-d32f-470c-9229-2662f5701683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:28.088084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-128246c3-d32f-470c-9229-2662f5701683 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:28.089321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-128246c3-d32f-470c-9229-2662f5701683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:28.102498 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3555/7135] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:28.106099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2b006-d192-4c83-a70b-77fbc71bb588 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:28.136007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2b006-d192-4c83-a70b-77fbc71bb588 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:28.161354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587725d-5373-4d2f-863b-ffc175094d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:28.162417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587725d-5373-4d2f-863b-ffc175094d66 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:28.163310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587725d-5373-4d2f-863b-ffc175094d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:28.163907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587725d-5373-4d2f-863b-ffc175094d66 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:28.165060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587725d-5373-4d2f-863b-ffc175094d66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:30:28.165566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587725d-5373-4d2f-863b-ffc175094d66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:28.167640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587725d-5373-4d2f-863b-ffc175094d66 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:28.190472 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3581/7136] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:30:28.194643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128246c3-d32f-470c-9229-2662f5701683 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:28.210822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128246c3-d32f-470c-9229-2662f5701683 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:28.215928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d0b9c-82f9-43bb-86f0-c59a9b7d4154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:28.216950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d0b9c-82f9-43bb-86f0-c59a9b7d4154 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:28.217238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d0b9c-82f9-43bb-86f0-c59a9b7d4154 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:28.217516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158d0b9c-82f9-43bb-86f0-c59a9b7d4154 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:28.218378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d0b9c-82f9-43bb-86f0-c59a9b7d4154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:28.221285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158d0b9c-82f9-43bb-86f0-c59a9b7d4154 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:28.359920 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3556/7137] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:30:28 2022] POST /identity/v3/projects => generated 369 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:30:28.363029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587725d-5373-4d2f-863b-ffc175094d66 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:28.369184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:28.375834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587725d-5373-4d2f-863b-ffc175094d66 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:28.464055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc5f38-342c-419d-a36c-56ef90079935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:28.464733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc5f38-342c-419d-a36c-56ef90079935 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:28.465028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc5f38-342c-419d-a36c-56ef90079935 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:28.465309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc5f38-342c-419d-a36c-56ef90079935 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:28.466161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc5f38-342c-419d-a36c-56ef90079935 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:28.468484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc5f38-342c-419d-a36c-56ef90079935 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:28.624579 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3582/7138] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:30:28 2022] POST /identity/v3/users => generated 285 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:30:28.626056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d0b9c-82f9-43bb-86f0-c59a9b7d4154 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:28.648310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158d0b9c-82f9-43bb-86f0-c59a9b7d4154 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:28.656211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6f297-5517-4bc4-a521-110b6673b624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:28.659765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6f297-5517-4bc4-a521-110b6673b624 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:28.659765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6f297-5517-4bc4-a521-110b6673b624 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:28.660329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee6f297-5517-4bc4-a521-110b6673b624 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:28.661085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee6f297-5517-4bc4-a521-110b6673b624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:28.663624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee6f297-5517-4bc4-a521-110b6673b624 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:28.685692 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3557/7139] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:30:28.686224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc5f38-342c-419d-a36c-56ef90079935 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:28.700355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc5f38-342c-419d-a36c-56ef90079935 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:28.706483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3d9e3-8e2d-44a2-aa46-c74e4cc6f111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:28.707171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3d9e3-8e2d-44a2-aa46-c74e4cc6f111 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:28.707459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3d9e3-8e2d-44a2-aa46-c74e4cc6f111 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:28.707740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3d9e3-8e2d-44a2-aa46-c74e4cc6f111 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:28.708485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3d9e3-8e2d-44a2-aa46-c74e4cc6f111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:28.710145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac3d9e3-8e2d-44a2-aa46-c74e4cc6f111 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:28.721604 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3583/7140] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:30:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:30:28.725705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6f297-5517-4bc4-a521-110b6673b624 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:28.742140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee6f297-5517-4bc4-a521-110b6673b624 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:28.748363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb565c-a0c0-4e5f-8482-95bd93fb36e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:28.749354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb565c-a0c0-4e5f-8482-95bd93fb36e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:28.749651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb565c-a0c0-4e5f-8482-95bd93fb36e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:28.749961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb565c-a0c0-4e5f-8482-95bd93fb36e5 admin admin] PATH_INFO: `/v3/projects/f78187f9e0e740a9b25764a1ebe50ac1/users/c4b139fe378a4d8586a7aef3745bfb6e/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:28.753193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:28.767318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb565c-a0c0-4e5f-8482-95bd93fb36e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f78187f9e0e740a9b25764a1ebe50ac1, user_id=c4b139fe378a4d8586a7aef3745bfb6e, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:28.770220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb565c-a0c0-4e5f-8482-95bd93fb36e5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:28.940162 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3558/7141] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:30:28 2022] PUT /identity/v3/projects/f78187f9e0e740a9b25764a1ebe50ac1/users/c4b139fe378a4d8586a7aef3745bfb6e/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:30:28.950991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14eb0ec-2fe9-4497-91a9-d1537ee060a6 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:28.951582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14eb0ec-2fe9-4497-91a9-d1537ee060a6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:28.951878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14eb0ec-2fe9-4497-91a9-d1537ee060a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:28.984349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a14eb0ec-2fe9-4497-91a9-d1537ee060a6 None None] MFA Rules not processed for user `c4b139fe378a4d8586a7aef3745bfb6e`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:28.985046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a14eb0ec-2fe9-4497-91a9-d1537ee060a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:29.081918 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3584/7142] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:29.129892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb565c-a0c0-4e5f-8482-95bd93fb36e5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:29.135072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-24eb565c-a0c0-4e5f-8482-95bd93fb36e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:29.242586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb565c-a0c0-4e5f-8482-95bd93fb36e5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:29.249708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbb63e-45fe-40a1-821d-99114885b681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:29.250410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbb63e-45fe-40a1-821d-99114885b681 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:29.250807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbb63e-45fe-40a1-821d-99114885b681 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:29.251287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdbb63e-45fe-40a1-821d-99114885b681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:29.265450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdbb63e-45fe-40a1-821d-99114885b681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:29.268060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdbb63e-45fe-40a1-821d-99114885b681 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:29.282149 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3559/7143] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:30:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:30:41.058138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:41.407763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:47.137040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14eb0ec-2fe9-4497-91a9-d1537ee060a6 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:47.139582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a14eb0ec-2fe9-4497-91a9-d1537ee060a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:47.251179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14eb0ec-2fe9-4497-91a9-d1537ee060a6 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:47.256425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29920e5-8e66-4d64-8361-a4229f40eb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:47.257581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29920e5-8e66-4d64-8361-a4229f40eb83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:47.258367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29920e5-8e66-4d64-8361-a4229f40eb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:47.258903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29920e5-8e66-4d64-8361-a4229f40eb83 admin admin] PATH_INFO: `/v3/users/5b4a37f2a8ec4160bc4db12c0d3e7de5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:47.260853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29920e5-8e66-4d64-8361-a4229f40eb83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b4a37f2a8ec4160bc4db12c0d3e7de5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:47.263121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29920e5-8e66-4d64-8361-a4229f40eb83 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:47.271097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbb63e-45fe-40a1-821d-99114885b681 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:47.285236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdbb63e-45fe-40a1-821d-99114885b681 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:47.296994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd82a2-c695-4f94-9a8a-87133626e51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:47.298379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd82a2-c695-4f94-9a8a-87133626e51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:47.298869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd82a2-c695-4f94-9a8a-87133626e51c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:47.299661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdd82a2-c695-4f94-9a8a-87133626e51c admin admin] PATH_INFO: `/v3/users/c4b139fe378a4d8586a7aef3745bfb6e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:47.304236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd82a2-c695-4f94-9a8a-87133626e51c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4b139fe378a4d8586a7aef3745bfb6e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:47.312450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdd82a2-c695-4f94-9a8a-87133626e51c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:47.472859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f29920e5-8e66-4d64-8361-a4229f40eb83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b4a37f2a8ec4160bc4db12c0d3e7de5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:47.479789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f29920e5-8e66-4d64-8361-a4229f40eb83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b4a37f2a8ec4160bc4db12c0d3e7de5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:47.480475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f29920e5-8e66-4d64-8361-a4229f40eb83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b4a37f2a8ec4160bc4db12c0d3e7de5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:47.481088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f29920e5-8e66-4d64-8361-a4229f40eb83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b4a37f2a8ec4160bc4db12c0d3e7de5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:47.494190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-abdd82a2-c695-4f94-9a8a-87133626e51c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4b139fe378a4d8586a7aef3745bfb6e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:47.494862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-abdd82a2-c695-4f94-9a8a-87133626e51c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4b139fe378a4d8586a7aef3745bfb6e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:47.501887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-abdd82a2-c695-4f94-9a8a-87133626e51c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4b139fe378a4d8586a7aef3745bfb6e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:47.560911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f29920e5-8e66-4d64-8361-a4229f40eb83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b4a37f2a8ec4160bc4db12c0d3e7de5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:47.587258 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3585/7144] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:30:47 2022] DELETE /identity/v3/users/5b4a37f2a8ec4160bc4db12c0d3e7de5 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:30:47.646426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-abdd82a2-c695-4f94-9a8a-87133626e51c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4b139fe378a4d8586a7aef3745bfb6e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:47.648565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-abdd82a2-c695-4f94-9a8a-87133626e51c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4b139fe378a4d8586a7aef3745bfb6e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:47.685468 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3560/7145] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:30:47 2022] DELETE /identity/v3/users/c4b139fe378a4d8586a7aef3745bfb6e => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:30:47.691027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efd7d3-2b3e-437b-8b09-355c4adbcec6 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:47.693880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efd7d3-2b3e-437b-8b09-355c4adbcec6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:47.693880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efd7d3-2b3e-437b-8b09-355c4adbcec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:47.723456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-15efd7d3-2b3e-437b-8b09-355c4adbcec6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:47.724879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-15efd7d3-2b3e-437b-8b09-355c4adbcec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:47.799305 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3586/7146] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:47.860267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd82a2-c695-4f94-9a8a-87133626e51c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:47.861880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-abdd82a2-c695-4f94-9a8a-87133626e51c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:47.930053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdd82a2-c695-4f94-9a8a-87133626e51c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:47.934405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3354b-300d-49e5-8804-bab0f24bb442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:47.935233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3354b-300d-49e5-8804-bab0f24bb442 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:47.935521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3354b-300d-49e5-8804-bab0f24bb442 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:47.935800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee3354b-300d-49e5-8804-bab0f24bb442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:47.946138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3354b-300d-49e5-8804-bab0f24bb442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:47.947647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee3354b-300d-49e5-8804-bab0f24bb442 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:47.958182 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3561/7147] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:30:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:30:48.014278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efd7d3-2b3e-437b-8b09-355c4adbcec6 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:48.036998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-15efd7d3-2b3e-437b-8b09-355c4adbcec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:48.036998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efd7d3-2b3e-437b-8b09-355c4adbcec6 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:48.036998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290ec43-ecdc-4bb9-b3e0-32d0fc575cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:48.038136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290ec43-ecdc-4bb9-b3e0-32d0fc575cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:48.038136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290ec43-ecdc-4bb9-b3e0-32d0fc575cee admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:48.038136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290ec43-ecdc-4bb9-b3e0-32d0fc575cee admin admin] PATH_INFO: `/v3/projects/239c9c3debdd456ab6e9c0545ab47fe2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:48.039746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290ec43-ecdc-4bb9-b3e0-32d0fc575cee admin admin] RBAC: Authorizing `identity:delete_project(project_id=239c9c3debdd456ab6e9c0545ab47fe2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:48.040314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290ec43-ecdc-4bb9-b3e0-32d0fc575cee admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:48.064575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:48.211906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3290ec43-ecdc-4bb9-b3e0-32d0fc575cee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '239c9c3debdd456ab6e9c0545ab47fe2', 'request_id': 'req-3290ec43-ecdc-4bb9-b3e0-32d0fc575cee'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:48.219220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3290ec43-ecdc-4bb9-b3e0-32d0fc575cee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '239c9c3debdd456ab6e9c0545ab47fe2', 'request_id': 'req-3290ec43-ecdc-4bb9-b3e0-32d0fc575cee'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:48.318193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3290ec43-ecdc-4bb9-b3e0-32d0fc575cee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '239c9c3debdd456ab6e9c0545ab47fe2', 'request_id': 'req-3290ec43-ecdc-4bb9-b3e0-32d0fc575cee'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:48.327963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3354b-300d-49e5-8804-bab0f24bb442 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:48.329465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-aee3354b-300d-49e5-8804-bab0f24bb442 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:48.344508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3290ec43-ecdc-4bb9-b3e0-32d0fc575cee admin admin] The token cache is being invalidate because project 239c9c3debdd456ab6e9c0545ab47fe2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:30:48.344508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3290ec43-ecdc-4bb9-b3e0-32d0fc575cee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:48.344693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee3354b-300d-49e5-8804-bab0f24bb442 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:48.344773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3290ec43-ecdc-4bb9-b3e0-32d0fc575cee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:48.349172 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3587/7148] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:30:48 2022] DELETE /identity/v3/projects/239c9c3debdd456ab6e9c0545ab47fe2 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:30:48.356404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f02ff-c79e-43ec-81d6-af2b6f2015eb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:48.356697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f02ff-c79e-43ec-81d6-af2b6f2015eb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:48.356989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f02ff-c79e-43ec-81d6-af2b6f2015eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:48.395221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-736f02ff-c79e-43ec-81d6-af2b6f2015eb None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:48.396578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-736f02ff-c79e-43ec-81d6-af2b6f2015eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:48.409702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f60d22-bf2d-4775-a54b-e3dae96d9ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:48.410619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f60d22-bf2d-4775-a54b-e3dae96d9ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:48.410894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f60d22-bf2d-4775-a54b-e3dae96d9ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:48.411168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f60d22-bf2d-4775-a54b-e3dae96d9ab5 admin admin] PATH_INFO: `/v3/projects/f78187f9e0e740a9b25764a1ebe50ac1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:48.413509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f60d22-bf2d-4775-a54b-e3dae96d9ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f78187f9e0e740a9b25764a1ebe50ac1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:48.415827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f60d22-bf2d-4775-a54b-e3dae96d9ab5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:48.443317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:48.490381 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3588/7149] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:48.499076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781883f3-7e4c-4cfe-b48d-c514bf5a38e1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:48.499606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781883f3-7e4c-4cfe-b48d-c514bf5a38e1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:48.500904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781883f3-7e4c-4cfe-b48d-c514bf5a38e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:48.525776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-781883f3-7e4c-4cfe-b48d-c514bf5a38e1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:48.527111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-781883f3-7e4c-4cfe-b48d-c514bf5a38e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:48.542130 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3589/7150] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:30:48.543085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781883f3-7e4c-4cfe-b48d-c514bf5a38e1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:48.553944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781883f3-7e4c-4cfe-b48d-c514bf5a38e1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:48.559361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c94b9e-7ac3-4220-8686-1d5b6863359e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:48.560241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c94b9e-7ac3-4220-8686-1d5b6863359e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:48.560571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c94b9e-7ac3-4220-8686-1d5b6863359e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:48.560856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c94b9e-7ac3-4220-8686-1d5b6863359e admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:48.561595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c94b9e-7ac3-4220-8686-1d5b6863359e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:30:48.561984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c94b9e-7ac3-4220-8686-1d5b6863359e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:48.565900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c94b9e-7ac3-4220-8686-1d5b6863359e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:48.579037 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3590/7151] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:30:48.583325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c94b9e-7ac3-4220-8686-1d5b6863359e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:48.595929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-38f60d22-bf2d-4775-a54b-e3dae96d9ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f78187f9e0e740a9b25764a1ebe50ac1', 'request_id': 'req-38f60d22-bf2d-4775-a54b-e3dae96d9ab5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:48.606693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-38f60d22-bf2d-4775-a54b-e3dae96d9ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f78187f9e0e740a9b25764a1ebe50ac1', 'request_id': 'req-38f60d22-bf2d-4775-a54b-e3dae96d9ab5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:48.616373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c94b9e-7ac3-4220-8686-1d5b6863359e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:48.690604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b2bbc-cefe-4bb9-9c30-0871047e3f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:48.691344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b2bbc-cefe-4bb9-9c30-0871047e3f84 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:48.696286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b2bbc-cefe-4bb9-9c30-0871047e3f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:48.696286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b2bbc-cefe-4bb9-9c30-0871047e3f84 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:48.696286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b2bbc-cefe-4bb9-9c30-0871047e3f84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:48.696286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b2bbc-cefe-4bb9-9c30-0871047e3f84 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:48.743422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-38f60d22-bf2d-4775-a54b-e3dae96d9ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f78187f9e0e740a9b25764a1ebe50ac1', 'request_id': 'req-38f60d22-bf2d-4775-a54b-e3dae96d9ab5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:48.756709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-38f60d22-bf2d-4775-a54b-e3dae96d9ab5 admin admin] The token cache is being invalidate because project f78187f9e0e740a9b25764a1ebe50ac1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:30:48.757002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-38f60d22-bf2d-4775-a54b-e3dae96d9ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:48.757899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-38f60d22-bf2d-4775-a54b-e3dae96d9ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:48.759916 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3562/7152] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:30:48 2022] DELETE /identity/v3/projects/f78187f9e0e740a9b25764a1ebe50ac1 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:30:48.764835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c960d39-7800-40c9-83d6-69bfbbd8feb4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:48.765131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c960d39-7800-40c9-83d6-69bfbbd8feb4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:48.765419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c960d39-7800-40c9-83d6-69bfbbd8feb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:48.792322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5c960d39-7800-40c9-83d6-69bfbbd8feb4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:48.793077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5c960d39-7800-40c9-83d6-69bfbbd8feb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:48.801983 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3563/7153] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:30:48.811906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bd712-c480-4aa7-83c7-bf15cf918b1a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:48.812426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bd712-c480-4aa7-83c7-bf15cf918b1a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:48.812706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bd712-c480-4aa7-83c7-bf15cf918b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:48.817500 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3591/7154] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:30:48 2022] POST /identity/v3/projects => generated 371 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:30:48.818554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b2bbc-cefe-4bb9-9c30-0871047e3f84 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:48.820256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7c8b2bbc-cefe-4bb9-9c30-0871047e3f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:48.834235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:48.848852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6f7bd712-c480-4aa7-83c7-bf15cf918b1a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:48.850179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6f7bd712-c480-4aa7-83c7-bf15cf918b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:48.912692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b2bbc-cefe-4bb9-9c30-0871047e3f84 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:48.918584 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3564/7155] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:48.919188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21753908-26a1-4b3f-85a0-c496db5a256e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:48.919951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21753908-26a1-4b3f-85a0-c496db5a256e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:48.920470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21753908-26a1-4b3f-85a0-c496db5a256e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:48.921082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21753908-26a1-4b3f-85a0-c496db5a256e admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:48.922269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21753908-26a1-4b3f-85a0-c496db5a256e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:48.925314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21753908-26a1-4b3f-85a0-c496db5a256e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:48.928499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bd712-c480-4aa7-83c7-bf15cf918b1a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:48.940428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bd712-c480-4aa7-83c7-bf15cf918b1a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:48.946202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d01ce8-207f-49c9-9283-1f79d82d9635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:48.947052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d01ce8-207f-49c9-9283-1f79d82d9635 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:48.947458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d01ce8-207f-49c9-9283-1f79d82d9635 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:48.947750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d01ce8-207f-49c9-9283-1f79d82d9635 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:48.949891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d01ce8-207f-49c9-9283-1f79d82d9635 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:30:48.949891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d01ce8-207f-49c9-9283-1f79d82d9635 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:48.950716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d01ce8-207f-49c9-9283-1f79d82d9635 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:48.965903 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3565/7156] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:30:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:30:48.970253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d01ce8-207f-49c9-9283-1f79d82d9635 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:48.981438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d01ce8-207f-49c9-9283-1f79d82d9635 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:48.989436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7336f-a2d3-4f32-83a1-ed555ba32335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:48.990434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7336f-a2d3-4f32-83a1-ed555ba32335 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:48.990740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7336f-a2d3-4f32-83a1-ed555ba32335 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:48.991029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7336f-a2d3-4f32-83a1-ed555ba32335 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:48.991872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7336f-a2d3-4f32-83a1-ed555ba32335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:48.993990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f7336f-a2d3-4f32-83a1-ed555ba32335 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:49.004667 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3592/7157] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:30:48 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:30:49.011000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21753908-26a1-4b3f-85a0-c496db5a256e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:49.017572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21753908-26a1-4b3f-85a0-c496db5a256e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:49.023365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5f5a0-7655-4ad7-b6b9-25162f5dbca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:49.024152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5f5a0-7655-4ad7-b6b9-25162f5dbca1 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:49.024509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5f5a0-7655-4ad7-b6b9-25162f5dbca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:49.024903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5f5a0-7655-4ad7-b6b9-25162f5dbca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:49.025750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5f5a0-7655-4ad7-b6b9-25162f5dbca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:49.027635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5f5a0-7655-4ad7-b6b9-25162f5dbca1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:49.039301 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3593/7158] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:30:49.043272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5f5a0-7655-4ad7-b6b9-25162f5dbca1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:49.054020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5f5a0-7655-4ad7-b6b9-25162f5dbca1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:49.059437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5f03a-0a64-41c2-b480-67189a14481c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:49.060115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5f03a-0a64-41c2-b480-67189a14481c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:49.060502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5f03a-0a64-41c2-b480-67189a14481c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:49.060859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5f03a-0a64-41c2-b480-67189a14481c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:49.061610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5f03a-0a64-41c2-b480-67189a14481c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:49.063431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5f03a-0a64-41c2-b480-67189a14481c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:49.078542 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3594/7159] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:30:49.090787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5f03a-0a64-41c2-b480-67189a14481c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:49.108027 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3566/7160] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:30:48 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:30:49.113031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5f03a-0a64-41c2-b480-67189a14481c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:49.113791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7336f-a2d3-4f32-83a1-ed555ba32335 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:49.183261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d30bc75-7a8d-485a-b84d-14d4e2c49d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:49.187611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30bc75-7a8d-485a-b84d-14d4e2c49d25 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:49.188180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30bc75-7a8d-485a-b84d-14d4e2c49d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:49.188773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d30bc75-7a8d-485a-b84d-14d4e2c49d25 admin admin] PATH_INFO: `/v3/projects/b98e2ebc3d3b4e7083b9c429628ce69f/users/c08e8a164d3248188ce965d1a9e04f03/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:49.206332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7336f-a2d3-4f32-83a1-ed555ba32335 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:49.211562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183f4b3-8e9d-4c79-83d4-ad950c7cb6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:49.212238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183f4b3-8e9d-4c79-83d4-ad950c7cb6a4 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:49.212524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183f4b3-8e9d-4c79-83d4-ad950c7cb6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:49.212803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183f4b3-8e9d-4c79-83d4-ad950c7cb6a4 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:49.213496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183f4b3-8e9d-4c79-83d4-ad950c7cb6a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:49.214296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30bc75-7a8d-485a-b84d-14d4e2c49d25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b98e2ebc3d3b4e7083b9c429628ce69f, user_id=c08e8a164d3248188ce965d1a9e04f03, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:49.215530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183f4b3-8e9d-4c79-83d4-ad950c7cb6a4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:49.218638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d30bc75-7a8d-485a-b84d-14d4e2c49d25 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:49.226051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:49.325649 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3595/7161] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:30:49 2022] PUT /identity/v3/projects/b98e2ebc3d3b4e7083b9c429628ce69f/users/c08e8a164d3248188ce965d1a9e04f03/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:30:49.331288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b2a51-254f-41be-a4b1-84d3b9b47d29 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:49.331566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b2a51-254f-41be-a4b1-84d3b9b47d29 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:49.331846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b2a51-254f-41be-a4b1-84d3b9b47d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:49.349159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-068b2a51-254f-41be-a4b1-84d3b9b47d29 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:49.350401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-068b2a51-254f-41be-a4b1-84d3b9b47d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:49.365865 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3567/7162] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:30:49 2022] POST /identity/v3/users => generated 287 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:30:49.409372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:49.418779 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3596/7163] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:30:49.429394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183f4b3-8e9d-4c79-83d4-ad950c7cb6a4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:49.431250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9183f4b3-8e9d-4c79-83d4-ad950c7cb6a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:49.441425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b2a51-254f-41be-a4b1-84d3b9b47d29 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:49.451466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b2a51-254f-41be-a4b1-84d3b9b47d29 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:49.456230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e31525-70fd-4a5c-b5e7-9bd97ff584a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:49.456898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e31525-70fd-4a5c-b5e7-9bd97ff584a4 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:49.457184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e31525-70fd-4a5c-b5e7-9bd97ff584a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:49.457464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e31525-70fd-4a5c-b5e7-9bd97ff584a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:49.458262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e31525-70fd-4a5c-b5e7-9bd97ff584a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:49.459884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e31525-70fd-4a5c-b5e7-9bd97ff584a4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:49.470767 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3597/7164] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:30:49.474867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e31525-70fd-4a5c-b5e7-9bd97ff584a4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:49.485030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e31525-70fd-4a5c-b5e7-9bd97ff584a4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:49.490279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6366fa-7d7c-4061-a950-e01f41e247a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:49.490938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6366fa-7d7c-4061-a950-e01f41e247a6 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:49.491220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6366fa-7d7c-4061-a950-e01f41e247a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:49.491520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6366fa-7d7c-4061-a950-e01f41e247a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:49.492239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6366fa-7d7c-4061-a950-e01f41e247a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:49.493825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6366fa-7d7c-4061-a950-e01f41e247a6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:49.495429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183f4b3-8e9d-4c79-83d4-ad950c7cb6a4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:49.505529 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3598/7165] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:30:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:30:49.509949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6366fa-7d7c-4061-a950-e01f41e247a6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:49.519974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6366fa-7d7c-4061-a950-e01f41e247a6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:49.525010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7aa1e-a351-4c5c-87b8-b733e87b448c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:49.525924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7aa1e-a351-4c5c-87b8-b733e87b448c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:49.526217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7aa1e-a351-4c5c-87b8-b733e87b448c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:49.526500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7aa1e-a351-4c5c-87b8-b733e87b448c admin admin] PATH_INFO: `/v3/projects/a25bae08df72428896f3ecd64fb26030/users/b18d5c8da8d842508ad9bd58a0bffc19/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:49.542678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7aa1e-a351-4c5c-87b8-b733e87b448c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a25bae08df72428896f3ecd64fb26030, user_id=b18d5c8da8d842508ad9bd58a0bffc19, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:49.545374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7aa1e-a351-4c5c-87b8-b733e87b448c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:49.548417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abddf6ea-1921-4f7f-a813-4f9cfc2336e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:49.549143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddf6ea-1921-4f7f-a813-4f9cfc2336e9 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:49.551574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddf6ea-1921-4f7f-a813-4f9cfc2336e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:49.551574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abddf6ea-1921-4f7f-a813-4f9cfc2336e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:49.559677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abddf6ea-1921-4f7f-a813-4f9cfc2336e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:49.561127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abddf6ea-1921-4f7f-a813-4f9cfc2336e9 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:49.571307 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3568/7166] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:30:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:30:49.664223 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3599/7167] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:30:49 2022] PUT /identity/v3/projects/a25bae08df72428896f3ecd64fb26030/users/b18d5c8da8d842508ad9bd58a0bffc19/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:30:49.674849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4b11b-b6ab-4a3f-9b61-53f3ef94c216 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:49.675144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4b11b-b6ab-4a3f-9b61-53f3ef94c216 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:49.675428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f4b11b-b6ab-4a3f-9b61-53f3ef94c216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:49.703514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-56f4b11b-b6ab-4a3f-9b61-53f3ef94c216 None None] MFA Rules not processed for user `b18d5c8da8d842508ad9bd58a0bffc19`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:49.705144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-56f4b11b-b6ab-4a3f-9b61-53f3ef94c216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:49.788178 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3569/7168] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:49.803394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7aa1e-a351-4c5c-87b8-b733e87b448c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:49.805544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-14d7aa1e-a351-4c5c-87b8-b733e87b448c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:49.868376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7aa1e-a351-4c5c-87b8-b733e87b448c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:49.872512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ad1e7-f004-4921-805b-2d691da58377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:49.873259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ad1e7-f004-4921-805b-2d691da58377 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:49.873556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ad1e7-f004-4921-805b-2d691da58377 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:49.873855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ad1e7-f004-4921-805b-2d691da58377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:49.884669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ad1e7-f004-4921-805b-2d691da58377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:49.886623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88ad1e7-f004-4921-805b-2d691da58377 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:49.898548 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3600/7169] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:30:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:30:50.922242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4b11b-b6ab-4a3f-9b61-53f3ef94c216 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:50.982102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:51.000558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f4b11b-b6ab-4a3f-9b61-53f3ef94c216 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:51.005515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101341a-bbbf-4183-bf64-78a81247fbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:51.006765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101341a-bbbf-4183-bf64-78a81247fbdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:51.007136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101341a-bbbf-4183-bf64-78a81247fbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:51.007473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101341a-bbbf-4183-bf64-78a81247fbdd admin admin] PATH_INFO: `/v3/users/b18d5c8da8d842508ad9bd58a0bffc19` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:51.008785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101341a-bbbf-4183-bf64-78a81247fbdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b18d5c8da8d842508ad9bd58a0bffc19)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:51.010895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101341a-bbbf-4183-bf64-78a81247fbdd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:51.288610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb845c2-8787-4c7b-b6ef-d312b7066807 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:51.289215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb845c2-8787-4c7b-b6ef-d312b7066807 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:51.289215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb845c2-8787-4c7b-b6ef-d312b7066807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:51.305100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0101341a-bbbf-4183-bf64-78a81247fbdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b18d5c8da8d842508ad9bd58a0bffc19'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:51.305766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0101341a-bbbf-4183-bf64-78a81247fbdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b18d5c8da8d842508ad9bd58a0bffc19'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:51.315771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0101341a-bbbf-4183-bf64-78a81247fbdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b18d5c8da8d842508ad9bd58a0bffc19'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:51.322712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cfb845c2-8787-4c7b-b6ef-d312b7066807 None None] MFA Rules not processed for user `c08e8a164d3248188ce965d1a9e04f03`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:51.324083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cfb845c2-8787-4c7b-b6ef-d312b7066807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:51.389045 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3601/7170] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:30:51.399232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7aeb4-8184-45ea-90e4-0f87901f7fee None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:51.399524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7aeb4-8184-45ea-90e4-0f87901f7fee None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:51.399807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7aeb4-8184-45ea-90e4-0f87901f7fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:51.419404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ece7aeb4-8184-45ea-90e4-0f87901f7fee None None] MFA Rules not processed for user `c08e8a164d3248188ce965d1a9e04f03`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:51.419404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ece7aeb4-8184-45ea-90e4-0f87901f7fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:51.425499 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3602/7171] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:30:51.436702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7aeb4-8184-45ea-90e4-0f87901f7fee None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:51.438181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ece7aeb4-8184-45ea-90e4-0f87901f7fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:51.469379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0101341a-bbbf-4183-bf64-78a81247fbdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b18d5c8da8d842508ad9bd58a0bffc19'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:51.470854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0101341a-bbbf-4183-bf64-78a81247fbdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b18d5c8da8d842508ad9bd58a0bffc19'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:51.484072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:51.495639 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3570/7172] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:30:50 2022] DELETE /identity/v3/users/b18d5c8da8d842508ad9bd58a0bffc19 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:30:51.500115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8aed19-7e78-4f78-b603-c916cf949ee2 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:51.500439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8aed19-7e78-4f78-b603-c916cf949ee2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:51.500676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8aed19-7e78-4f78-b603-c916cf949ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:51.517275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece7aeb4-8184-45ea-90e4-0f87901f7fee None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:51.517534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4b8aed19-7e78-4f78-b603-c916cf949ee2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:51.519749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4b8aed19-7e78-4f78-b603-c916cf949ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:51.581912 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3571/7173] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:30:51.582760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8aed19-7e78-4f78-b603-c916cf949ee2 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:51.584077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4b8aed19-7e78-4f78-b603-c916cf949ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:51.587871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b303c53-32f8-402d-af44-a4d2883a4a80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:51.588811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303c53-32f8-402d-af44-a4d2883a4a80 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:51.589521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303c53-32f8-402d-af44-a4d2883a4a80 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:51.590035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b303c53-32f8-402d-af44-a4d2883a4a80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:51.593516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1b303c53-32f8-402d-af44-a4d2883a4a80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:51.654883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b303c53-32f8-402d-af44-a4d2883a4a80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:51.656498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b303c53-32f8-402d-af44-a4d2883a4a80 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:51.662853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8aed19-7e78-4f78-b603-c916cf949ee2 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:51.665757 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3603/7174] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:30:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:30:51.666673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b303c53-32f8-402d-af44-a4d2883a4a80 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:51.668090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1b303c53-32f8-402d-af44-a4d2883a4a80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:51.668947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1753a84-882b-4b57-b38c-685900d399c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:51.672314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1753a84-882b-4b57-b38c-685900d399c1 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:51.672314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1753a84-882b-4b57-b38c-685900d399c1 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:51.672314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1753a84-882b-4b57-b38c-685900d399c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:51.672314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e1753a84-882b-4b57-b38c-685900d399c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:51.687958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1753a84-882b-4b57-b38c-685900d399c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:51.689638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1753a84-882b-4b57-b38c-685900d399c1 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:51.703354 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3572/7175] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:30:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:30:51.730456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b303c53-32f8-402d-af44-a4d2883a4a80 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:51.734428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c614435-01eb-4dee-8632-c131e05b40bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:51.735209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c614435-01eb-4dee-8632-c131e05b40bf service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:51.735898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c614435-01eb-4dee-8632-c131e05b40bf service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:51.741070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c614435-01eb-4dee-8632-c131e05b40bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:51.747220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c614435-01eb-4dee-8632-c131e05b40bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:51.748834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c614435-01eb-4dee-8632-c131e05b40bf service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:51.760213 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3604/7176] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:30:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:30:51.816846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:52.522087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1753a84-882b-4b57-b38c-685900d399c1 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:52.522087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e1753a84-882b-4b57-b38c-685900d399c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:52.536393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1753a84-882b-4b57-b38c-685900d399c1 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:52.540570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ecd67-e871-487e-b5d5-500d48095e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:52.541444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ecd67-e871-487e-b5d5-500d48095e3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:52.541748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ecd67-e871-487e-b5d5-500d48095e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:52.542017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ecd67-e871-487e-b5d5-500d48095e3b admin admin] PATH_INFO: `/v3/projects/a25bae08df72428896f3ecd64fb26030` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:52.543193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ecd67-e871-487e-b5d5-500d48095e3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a25bae08df72428896f3ecd64fb26030)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:52.545108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ecd67-e871-487e-b5d5-500d48095e3b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:52.691336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6d9ecd67-e871-487e-b5d5-500d48095e3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a25bae08df72428896f3ecd64fb26030', 'request_id': 'req-6d9ecd67-e871-487e-b5d5-500d48095e3b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:52.715500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6d9ecd67-e871-487e-b5d5-500d48095e3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a25bae08df72428896f3ecd64fb26030', 'request_id': 'req-6d9ecd67-e871-487e-b5d5-500d48095e3b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:52.834192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6d9ecd67-e871-487e-b5d5-500d48095e3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a25bae08df72428896f3ecd64fb26030', 'request_id': 'req-6d9ecd67-e871-487e-b5d5-500d48095e3b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:52.850474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6d9ecd67-e871-487e-b5d5-500d48095e3b admin admin] The token cache is being invalidate because project a25bae08df72428896f3ecd64fb26030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:30:52.850768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6d9ecd67-e871-487e-b5d5-500d48095e3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:52.851442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6d9ecd67-e871-487e-b5d5-500d48095e3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:52.853263 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3573/7177] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:30:52 2022] DELETE /identity/v3/projects/a25bae08df72428896f3ecd64fb26030 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:30:52.857906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76850da-2807-4feb-ae2f-de7f9af8c840 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:52.858360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76850da-2807-4feb-ae2f-de7f9af8c840 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:52.858655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76850da-2807-4feb-ae2f-de7f9af8c840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:52.877343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a76850da-2807-4feb-ae2f-de7f9af8c840 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:52.878646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a76850da-2807-4feb-ae2f-de7f9af8c840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:52.891161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:52.946137 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3605/7178] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:52.954158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ac4be-55c9-478b-a490-163e447d4574 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:52.954471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ac4be-55c9-478b-a490-163e447d4574 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:52.954758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ac4be-55c9-478b-a490-163e447d4574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:52.975280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6b1ac4be-55c9-478b-a490-163e447d4574 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:52.976509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6b1ac4be-55c9-478b-a490-163e447d4574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:52.985028 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3574/7179] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:52.989392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76850da-2807-4feb-ae2f-de7f9af8c840 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:52.999863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76850da-2807-4feb-ae2f-de7f9af8c840 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:53.005475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a43f9-0288-4b86-816f-254d8e6d5013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:53.008956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a43f9-0288-4b86-816f-254d8e6d5013 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:53.008956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a43f9-0288-4b86-816f-254d8e6d5013 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:53.008956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a43f9-0288-4b86-816f-254d8e6d5013 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:53.008956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a43f9-0288-4b86-816f-254d8e6d5013 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:30:53.008956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a43f9-0288-4b86-816f-254d8e6d5013 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:53.009440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a43f9-0288-4b86-816f-254d8e6d5013 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:53.022954 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3606/7180] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:30:53.027477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ac4be-55c9-478b-a490-163e447d4574 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:53.037871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ac4be-55c9-478b-a490-163e447d4574 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:53.042910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff169707-2637-4279-8364-e253e6033795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:53.043647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff169707-2637-4279-8364-e253e6033795 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:53.043981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff169707-2637-4279-8364-e253e6033795 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:53.044312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff169707-2637-4279-8364-e253e6033795 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:53.045047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff169707-2637-4279-8364-e253e6033795 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:53.047622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff169707-2637-4279-8364-e253e6033795 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:53.211528 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3575/7181] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:30:53 2022] POST /identity/v3/projects => generated 351 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:30:53.212885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a43f9-0288-4b86-816f-254d8e6d5013 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:53.224471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a43f9-0288-4b86-816f-254d8e6d5013 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:53.286380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa7bf8-883e-47bc-90ea-4c740bfc2f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:53.287072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa7bf8-883e-47bc-90ea-4c740bfc2f4b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:53.287360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa7bf8-883e-47bc-90ea-4c740bfc2f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:53.287637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaa7bf8-883e-47bc-90ea-4c740bfc2f4b admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:53.288836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa7bf8-883e-47bc-90ea-4c740bfc2f4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:53.290511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaa7bf8-883e-47bc-90ea-4c740bfc2f4b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:53.401595 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3607/7182] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:30:53 2022] POST /identity/v3/users => generated 276 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:30:53.408884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff169707-2637-4279-8364-e253e6033795 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:53.420399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff169707-2637-4279-8364-e253e6033795 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:53.425113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b8237-1599-4aae-a675-c988cd231db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:53.425800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b8237-1599-4aae-a675-c988cd231db9 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:53.426104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b8237-1599-4aae-a675-c988cd231db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:53.426384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b8237-1599-4aae-a675-c988cd231db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:53.427119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b8237-1599-4aae-a675-c988cd231db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:53.428834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b8237-1599-4aae-a675-c988cd231db9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:53.441060 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3576/7183] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:30:53.445125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa7bf8-883e-47bc-90ea-4c740bfc2f4b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:53.455477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaa7bf8-883e-47bc-90ea-4c740bfc2f4b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:53.460683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c96e76-165c-4512-bf48-d8a5e9853118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:53.461344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c96e76-165c-4512-bf48-d8a5e9853118 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:53.461628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c96e76-165c-4512-bf48-d8a5e9853118 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:53.461936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c96e76-165c-4512-bf48-d8a5e9853118 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:53.462673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c96e76-165c-4512-bf48-d8a5e9853118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:53.464268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c96e76-165c-4512-bf48-d8a5e9853118 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:53.475325 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3608/7184] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:30:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:30:53.479670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b8237-1599-4aae-a675-c988cd231db9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:53.490718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b8237-1599-4aae-a675-c988cd231db9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:53.495653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b58386-e1ac-4965-b46d-e4369d721526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:53.496592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b58386-e1ac-4965-b46d-e4369d721526 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:53.496908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b58386-e1ac-4965-b46d-e4369d721526 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:53.497190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b58386-e1ac-4965-b46d-e4369d721526 admin admin] PATH_INFO: `/v3/projects/e3a4ee2f5eb64d3ebd0caa02bc99be4e/users/51b204c08c254af1a8a668c1653ccd95/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:53.514418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b58386-e1ac-4965-b46d-e4369d721526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a4ee2f5eb64d3ebd0caa02bc99be4e, user_id=51b204c08c254af1a8a668c1653ccd95, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:53.517191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b58386-e1ac-4965-b46d-e4369d721526 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:53.588805 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3577/7185] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:30:53 2022] PUT /identity/v3/projects/e3a4ee2f5eb64d3ebd0caa02bc99be4e/users/51b204c08c254af1a8a668c1653ccd95/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:30:53.600642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c4d04-165f-4298-8dd6-d6229d85f095 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:53.600932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c4d04-165f-4298-8dd6-d6229d85f095 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:53.601213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c4d04-165f-4298-8dd6-d6229d85f095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:53.633609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8e4c4d04-165f-4298-8dd6-d6229d85f095 None None] MFA Rules not processed for user `51b204c08c254af1a8a668c1653ccd95`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:53.634869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8e4c4d04-165f-4298-8dd6-d6229d85f095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:53.707020 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3609/7186] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:53 2022] POST /identity/v3/auth/tokens => generated 3218 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:53.720625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b58386-e1ac-4965-b46d-e4369d721526 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:53.722873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-21b58386-e1ac-4965-b46d-e4369d721526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:53.807315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b58386-e1ac-4965-b46d-e4369d721526 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:53.811320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52270e0b-bcbb-416f-b10d-9262c3edafa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:53.811968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52270e0b-bcbb-416f-b10d-9262c3edafa3 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:53.812251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52270e0b-bcbb-416f-b10d-9262c3edafa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:53.812529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52270e0b-bcbb-416f-b10d-9262c3edafa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:53.821772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52270e0b-bcbb-416f-b10d-9262c3edafa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:53.823251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52270e0b-bcbb-416f-b10d-9262c3edafa3 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:53.833403 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3578/7187] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:30:53 2022] GET /identity/v3/auth/tokens => generated 3218 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:30:54.487771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c4d04-165f-4298-8dd6-d6229d85f095 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:54.494159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8e4c4d04-165f-4298-8dd6-d6229d85f095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:54.523785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:54.592266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c4d04-165f-4298-8dd6-d6229d85f095 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:54.597590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4d49a-44ee-43b7-9cab-18ffc53f9891 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:54.598505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4d49a-44ee-43b7-9cab-18ffc53f9891 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:54.599064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4d49a-44ee-43b7-9cab-18ffc53f9891 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:54.600113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4d49a-44ee-43b7-9cab-18ffc53f9891 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:54.601340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c3d4d49a-44ee-43b7-9cab-18ffc53f9891 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:54.616682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4d49a-44ee-43b7-9cab-18ffc53f9891 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:54.618577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4d49a-44ee-43b7-9cab-18ffc53f9891 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:54.632960 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3610/7188] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:30:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:30:57.102164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52270e0b-bcbb-416f-b10d-9262c3edafa3 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:57.130146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:57.268700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52270e0b-bcbb-416f-b10d-9262c3edafa3 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:57.281641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb16639-2268-48ac-b977-be2391bc8ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:57.282707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb16639-2268-48ac-b977-be2391bc8ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:57.282999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb16639-2268-48ac-b977-be2391bc8ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:57.283281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb16639-2268-48ac-b977-be2391bc8ed8 admin admin] PATH_INFO: `/v3/users/51b204c08c254af1a8a668c1653ccd95` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:57.290010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb16639-2268-48ac-b977-be2391bc8ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51b204c08c254af1a8a668c1653ccd95)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:57.291538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb16639-2268-48ac-b977-be2391bc8ed8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:57.534809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cfb16639-2268-48ac-b977-be2391bc8ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51b204c08c254af1a8a668c1653ccd95'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:57.535999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cfb16639-2268-48ac-b977-be2391bc8ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51b204c08c254af1a8a668c1653ccd95'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:57.543490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cfb16639-2268-48ac-b977-be2391bc8ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51b204c08c254af1a8a668c1653ccd95'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:57.663776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cfb16639-2268-48ac-b977-be2391bc8ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51b204c08c254af1a8a668c1653ccd95'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:57.665955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cfb16639-2268-48ac-b977-be2391bc8ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51b204c08c254af1a8a668c1653ccd95'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:57.699136 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3579/7189] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:30:57 2022] DELETE /identity/v3/users/51b204c08c254af1a8a668c1653ccd95 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:30:57.703810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f636d34-fdd7-47ae-8f88-146d56f9e2ba None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:57.704318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f636d34-fdd7-47ae-8f88-146d56f9e2ba None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:57.704426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f636d34-fdd7-47ae-8f88-146d56f9e2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:57.723821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0f636d34-fdd7-47ae-8f88-146d56f9e2ba None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:57.725439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0f636d34-fdd7-47ae-8f88-146d56f9e2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:57.759762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb16639-2268-48ac-b977-be2391bc8ed8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:57.761342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cfb16639-2268-48ac-b977-be2391bc8ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:57.801256 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3611/7190] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:57.811956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f636d34-fdd7-47ae-8f88-146d56f9e2ba None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:57.813873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0f636d34-fdd7-47ae-8f88-146d56f9e2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:57.838817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb16639-2268-48ac-b977-be2391bc8ed8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:57.843877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983adb8f-a819-4bea-b0da-17d5ea18c391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:57.844760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983adb8f-a819-4bea-b0da-17d5ea18c391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:57.845039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983adb8f-a819-4bea-b0da-17d5ea18c391 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:57.845311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983adb8f-a819-4bea-b0da-17d5ea18c391 admin admin] PATH_INFO: `/v3/users/642d08e104394dd28e6b4b502f5bc8cf` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:57.847174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983adb8f-a819-4bea-b0da-17d5ea18c391 admin admin] RBAC: Authorizing `identity:delete_user(user_id=642d08e104394dd28e6b4b502f5bc8cf)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:57.849166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983adb8f-a819-4bea-b0da-17d5ea18c391 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:57.895284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f636d34-fdd7-47ae-8f88-146d56f9e2ba None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:57.899008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee824587-cd9a-4100-84de-1b4b6b93e3c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:57.899659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee824587-cd9a-4100-84de-1b4b6b93e3c0 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:57.899944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee824587-cd9a-4100-84de-1b4b6b93e3c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:57.900224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee824587-cd9a-4100-84de-1b4b6b93e3c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:57.910692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee824587-cd9a-4100-84de-1b4b6b93e3c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:57.912326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee824587-cd9a-4100-84de-1b4b6b93e3c0 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:57.922790 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3612/7191] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:30:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:30:58.085045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-983adb8f-a819-4bea-b0da-17d5ea18c391 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '642d08e104394dd28e6b4b502f5bc8cf'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:58.088370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-983adb8f-a819-4bea-b0da-17d5ea18c391 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '642d08e104394dd28e6b4b502f5bc8cf'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:58.104926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-983adb8f-a819-4bea-b0da-17d5ea18c391 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '642d08e104394dd28e6b4b502f5bc8cf'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:58.201113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-983adb8f-a819-4bea-b0da-17d5ea18c391 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '642d08e104394dd28e6b4b502f5bc8cf'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:58.201954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-983adb8f-a819-4bea-b0da-17d5ea18c391 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '642d08e104394dd28e6b4b502f5bc8cf'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:58.210114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee824587-cd9a-4100-84de-1b4b6b93e3c0 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:58.212008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ee824587-cd9a-4100-84de-1b4b6b93e3c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:58.227788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:58.233554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee824587-cd9a-4100-84de-1b4b6b93e3c0 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:58.241279 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3580/7192] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:30:57 2022] DELETE /identity/v3/users/642d08e104394dd28e6b4b502f5bc8cf => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:30:58.245705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525f462-a86b-44e2-a352-23c87a80930b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:58.250262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525f462-a86b-44e2-a352-23c87a80930b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:58.250262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525f462-a86b-44e2-a352-23c87a80930b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:58.255980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444a2109-b4a0-4a73-89c4-4c40dd3294fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:58.256857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a2109-b4a0-4a73-89c4-4c40dd3294fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:58.257158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a2109-b4a0-4a73-89c4-4c40dd3294fc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:58.257440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444a2109-b4a0-4a73-89c4-4c40dd3294fc admin admin] PATH_INFO: `/v3/projects/e3a4ee2f5eb64d3ebd0caa02bc99be4e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:58.258695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a2109-b4a0-4a73-89c4-4c40dd3294fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3a4ee2f5eb64d3ebd0caa02bc99be4e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:58.260710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444a2109-b4a0-4a73-89c4-4c40dd3294fc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:58.275585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b525f462-a86b-44e2-a352-23c87a80930b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:58.276834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b525f462-a86b-44e2-a352-23c87a80930b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:58.347374 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3581/7193] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:58.357310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525f462-a86b-44e2-a352-23c87a80930b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:58.361653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b525f462-a86b-44e2-a352-23c87a80930b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:58.375719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:58.390603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-444a2109-b4a0-4a73-89c4-4c40dd3294fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3a4ee2f5eb64d3ebd0caa02bc99be4e', 'request_id': 'req-444a2109-b4a0-4a73-89c4-4c40dd3294fc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:58.402238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-444a2109-b4a0-4a73-89c4-4c40dd3294fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3a4ee2f5eb64d3ebd0caa02bc99be4e', 'request_id': 'req-444a2109-b4a0-4a73-89c4-4c40dd3294fc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:58.442967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525f462-a86b-44e2-a352-23c87a80930b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:58.501209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6cbda-6d7d-4a96-84a4-8635ee850de9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:58.502083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6cbda-6d7d-4a96-84a4-8635ee850de9 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:58.502370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6cbda-6d7d-4a96-84a4-8635ee850de9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:58.503302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6cbda-6d7d-4a96-84a4-8635ee850de9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:58.559715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-444a2109-b4a0-4a73-89c4-4c40dd3294fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3a4ee2f5eb64d3ebd0caa02bc99be4e', 'request_id': 'req-444a2109-b4a0-4a73-89c4-4c40dd3294fc'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:58.568310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6cbda-6d7d-4a96-84a4-8635ee850de9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:58.571469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6cbda-6d7d-4a96-84a4-8635ee850de9 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:58.573859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-444a2109-b4a0-4a73-89c4-4c40dd3294fc admin admin] The token cache is being invalidate because project e3a4ee2f5eb64d3ebd0caa02bc99be4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:30:58.574325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-444a2109-b4a0-4a73-89c4-4c40dd3294fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:58.575159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-444a2109-b4a0-4a73-89c4-4c40dd3294fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:58.576936 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3613/7194] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:30:58 2022] DELETE /identity/v3/projects/e3a4ee2f5eb64d3ebd0caa02bc99be4e => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:30:58.580121 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3582/7195] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:30:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:30:58.583031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a12b8-372b-4c44-821d-ac9d11a16ebd None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:58.583286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a12b8-372b-4c44-821d-ac9d11a16ebd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:58.583650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a12b8-372b-4c44-821d-ac9d11a16ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:58.617920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2a2a12b8-372b-4c44-821d-ac9d11a16ebd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:58.619235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2a2a12b8-372b-4c44-821d-ac9d11a16ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:58.629014 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3614/7196] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:58.636853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcd241-b270-40ce-a4b7-26100f557b1e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:58.637148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcd241-b270-40ce-a4b7-26100f557b1e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:58.637463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcd241-b270-40ce-a4b7-26100f557b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:58.661528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-53dcd241-b270-40ce-a4b7-26100f557b1e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:58.662895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-53dcd241-b270-40ce-a4b7-26100f557b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:58.674864 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3583/7197] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:30:58.679845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a12b8-372b-4c44-821d-ac9d11a16ebd None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:58.693219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a12b8-372b-4c44-821d-ac9d11a16ebd None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:58.698846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5673566-0f5e-4ae5-bc6a-c3b79f115e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:58.699960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5673566-0f5e-4ae5-bc6a-c3b79f115e9c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:58.700411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5673566-0f5e-4ae5-bc6a-c3b79f115e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:58.700763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5673566-0f5e-4ae5-bc6a-c3b79f115e9c admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:58.702733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5673566-0f5e-4ae5-bc6a-c3b79f115e9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:30:58.703108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5673566-0f5e-4ae5-bc6a-c3b79f115e9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:58.705890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5673566-0f5e-4ae5-bc6a-c3b79f115e9c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:58.725693 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3615/7198] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:30:58.725995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcd241-b270-40ce-a4b7-26100f557b1e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:58.735861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dcd241-b270-40ce-a4b7-26100f557b1e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:58.741905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ab2ed-aa97-4f4e-b30c-43616378e070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:58.742537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ab2ed-aa97-4f4e-b30c-43616378e070 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:58.742825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ab2ed-aa97-4f4e-b30c-43616378e070 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:58.743107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ab2ed-aa97-4f4e-b30c-43616378e070 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:58.743824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ab2ed-aa97-4f4e-b30c-43616378e070 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:58.745773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ab2ed-aa97-4f4e-b30c-43616378e070 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:58.880137 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3584/7199] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:30:58 2022] POST /identity/v3/projects => generated 405 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:30:58.881254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5673566-0f5e-4ae5-bc6a-c3b79f115e9c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:58.895883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ab2ed-aa97-4f4e-b30c-43616378e070 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:58.902174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bc9ab2ed-aa97-4f4e-b30c-43616378e070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:58.905606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5673566-0f5e-4ae5-bc6a-c3b79f115e9c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:59.020584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389ab65-e0b8-472a-a7c2-29d232778c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:59.021550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389ab65-e0b8-472a-a7c2-29d232778c4f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:59.022368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389ab65-e0b8-472a-a7c2-29d232778c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:59.022868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3389ab65-e0b8-472a-a7c2-29d232778c4f admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:59.026235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389ab65-e0b8-472a-a7c2-29d232778c4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:59.028591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3389ab65-e0b8-472a-a7c2-29d232778c4f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:59.047171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ab2ed-aa97-4f4e-b30c-43616378e070 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:59.052823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d506b9-5cfc-4902-9c3b-79779429b467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:59.053709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d506b9-5cfc-4902-9c3b-79779429b467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:59.054052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d506b9-5cfc-4902-9c3b-79779429b467 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:59.054330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d506b9-5cfc-4902-9c3b-79779429b467 admin admin] PATH_INFO: `/v3/projects/97cf01c13fa745a3bd04e35a0ce27088` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:59.055546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d506b9-5cfc-4902-9c3b-79779429b467 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97cf01c13fa745a3bd04e35a0ce27088)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:59.058171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d506b9-5cfc-4902-9c3b-79779429b467 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:59.214645 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3616/7200] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:30:58 2022] POST /identity/v3/users => generated 303 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:30:59.216230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389ab65-e0b8-472a-a7c2-29d232778c4f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:59.230023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f3d506b9-5cfc-4902-9c3b-79779429b467 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97cf01c13fa745a3bd04e35a0ce27088', 'request_id': 'req-f3d506b9-5cfc-4902-9c3b-79779429b467'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:59.230507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3389ab65-e0b8-472a-a7c2-29d232778c4f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:59.239298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f3d506b9-5cfc-4902-9c3b-79779429b467 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97cf01c13fa745a3bd04e35a0ce27088', 'request_id': 'req-f3d506b9-5cfc-4902-9c3b-79779429b467'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:59.307231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208c0e6-7af7-4b67-8687-c82d207803e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:59.307911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208c0e6-7af7-4b67-8687-c82d207803e2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:59.308199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208c0e6-7af7-4b67-8687-c82d207803e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:59.308478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208c0e6-7af7-4b67-8687-c82d207803e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:59.309208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208c0e6-7af7-4b67-8687-c82d207803e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:59.310844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208c0e6-7af7-4b67-8687-c82d207803e2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:59.324427 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3617/7201] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:30:59.326322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208c0e6-7af7-4b67-8687-c82d207803e2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:59.339515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208c0e6-7af7-4b67-8687-c82d207803e2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:59.352144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846374c4-5cef-4cb6-ac11-034ca6152985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:59.352924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846374c4-5cef-4cb6-ac11-034ca6152985 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:59.353228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846374c4-5cef-4cb6-ac11-034ca6152985 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:59.353556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846374c4-5cef-4cb6-ac11-034ca6152985 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:59.354480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846374c4-5cef-4cb6-ac11-034ca6152985 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:59.358286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846374c4-5cef-4cb6-ac11-034ca6152985 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:59.373601 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3618/7202] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:30:59.378303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846374c4-5cef-4cb6-ac11-034ca6152985 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:59.392478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846374c4-5cef-4cb6-ac11-034ca6152985 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:59.398259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a5d8f-5448-4f33-9269-33b6e1256f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:59.399147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a5d8f-5448-4f33-9269-33b6e1256f47 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:59.399428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a5d8f-5448-4f33-9269-33b6e1256f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:59.399704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a5d8f-5448-4f33-9269-33b6e1256f47 admin admin] PATH_INFO: `/v3/projects/6b54dfce3a2b41a8a13110ada4f335ae/users/a870b308cfc844af989fe1ce311855e3/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:59.418761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a5d8f-5448-4f33-9269-33b6e1256f47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b54dfce3a2b41a8a13110ada4f335ae, user_id=a870b308cfc844af989fe1ce311855e3, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:59.418956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f3d506b9-5cfc-4902-9c3b-79779429b467 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97cf01c13fa745a3bd04e35a0ce27088', 'request_id': 'req-f3d506b9-5cfc-4902-9c3b-79779429b467'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:59.420325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a5d8f-5448-4f33-9269-33b6e1256f47 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:59.432473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f3d506b9-5cfc-4902-9c3b-79779429b467 admin admin] The token cache is being invalidate because project 97cf01c13fa745a3bd04e35a0ce27088 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:30:59.432891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f3d506b9-5cfc-4902-9c3b-79779429b467 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:59.433502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f3d506b9-5cfc-4902-9c3b-79779429b467 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:59.435117 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3585/7203] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:30:58 2022] DELETE /identity/v3/projects/97cf01c13fa745a3bd04e35a0ce27088 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:30:59.438948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d506b9-5cfc-4902-9c3b-79779429b467 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:59.440949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f3d506b9-5cfc-4902-9c3b-79779429b467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:59.455054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d506b9-5cfc-4902-9c3b-79779429b467 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:59.459798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1019288-99b8-4c26-a062-185710fd1cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:59.460709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1019288-99b8-4c26-a062-185710fd1cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:59.460994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1019288-99b8-4c26-a062-185710fd1cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:59.461275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1019288-99b8-4c26-a062-185710fd1cf7 admin admin] PATH_INFO: `/v3/users/215d95f1b8cc43a2b027e05dfbff954c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:59.462490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1019288-99b8-4c26-a062-185710fd1cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=215d95f1b8cc43a2b027e05dfbff954c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:59.464439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1019288-99b8-4c26-a062-185710fd1cf7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:59.562610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:59.567790 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3619/7204] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:30:59 2022] PUT /identity/v3/projects/6b54dfce3a2b41a8a13110ada4f335ae/users/a870b308cfc844af989fe1ce311855e3/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:30:59.584197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e70cf3-fef6-4f8d-99b2-368d444fff85 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:59.584197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e70cf3-fef6-4f8d-99b2-368d444fff85 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:59.584197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e70cf3-fef6-4f8d-99b2-368d444fff85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:59.606799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b3e70cf3-fef6-4f8d-99b2-368d444fff85 None None] MFA Rules not processed for user `a870b308cfc844af989fe1ce311855e3`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:30:59.608031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b3e70cf3-fef6-4f8d-99b2-368d444fff85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:59.680376 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3620/7205] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:30:59 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:30:59.684742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e70cf3-fef6-4f8d-99b2-368d444fff85 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:59.686126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b3e70cf3-fef6-4f8d-99b2-368d444fff85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:59.730378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e1019288-99b8-4c26-a062-185710fd1cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '215d95f1b8cc43a2b027e05dfbff954c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:59.731299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e1019288-99b8-4c26-a062-185710fd1cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '215d95f1b8cc43a2b027e05dfbff954c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:59.740188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e1019288-99b8-4c26-a062-185710fd1cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '215d95f1b8cc43a2b027e05dfbff954c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:59.760910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e70cf3-fef6-4f8d-99b2-368d444fff85 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:59.765971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d9da1-1ada-473c-9af9-46da8335a638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:59.767901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d9da1-1ada-473c-9af9-46da8335a638 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:59.767901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d9da1-1ada-473c-9af9-46da8335a638 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:59.767901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d9da1-1ada-473c-9af9-46da8335a638 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:59.768304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d9da1-1ada-473c-9af9-46da8335a638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:59.769948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d9da1-1ada-473c-9af9-46da8335a638 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:30:59.846033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e1019288-99b8-4c26-a062-185710fd1cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '215d95f1b8cc43a2b027e05dfbff954c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:59.847204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e1019288-99b8-4c26-a062-185710fd1cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '215d95f1b8cc43a2b027e05dfbff954c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:30:59.853240 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3621/7206] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:30:59 2022] POST /identity/v3/projects => generated 405 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:30:59.857367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d9da1-1ada-473c-9af9-46da8335a638 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:30:59.858987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-985d9da1-1ada-473c-9af9-46da8335a638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:30:59.867492 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3586/7207] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:30:59 2022] DELETE /identity/v3/users/215d95f1b8cc43a2b027e05dfbff954c => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:30:59.927429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d9da1-1ada-473c-9af9-46da8335a638 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:30:59.972496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:59.985858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c75f2f-b607-4afb-b28e-84808eb6037c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:30:59.986658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c75f2f-b607-4afb-b28e-84808eb6037c admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:30:59.986953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c75f2f-b607-4afb-b28e-84808eb6037c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:30:59.987238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c75f2f-b607-4afb-b28e-84808eb6037c admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:30:59.987937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c75f2f-b607-4afb-b28e-84808eb6037c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:30:59.989937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c75f2f-b607-4afb-b28e-84808eb6037c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:00.082693 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3622/7208] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:30:59 2022] POST /identity/v3/users => generated 309 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:31:00.083863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1019288-99b8-4c26-a062-185710fd1cf7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:00.096128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1019288-99b8-4c26-a062-185710fd1cf7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:00.102121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699dc2a6-c0c8-476f-89e7-5d57137107a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:00.102799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dc2a6-c0c8-476f-89e7-5d57137107a3 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:00.103089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dc2a6-c0c8-476f-89e7-5d57137107a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:00.103398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dc2a6-c0c8-476f-89e7-5d57137107a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:00.104207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699dc2a6-c0c8-476f-89e7-5d57137107a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:00.105827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699dc2a6-c0c8-476f-89e7-5d57137107a3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:00.116840 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3587/7209] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:31:00.121527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c75f2f-b607-4afb-b28e-84808eb6037c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:00.132670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c75f2f-b607-4afb-b28e-84808eb6037c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:00.137439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb0a11-29dc-4cc8-91f0-3067671e7504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:00.138376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb0a11-29dc-4cc8-91f0-3067671e7504 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:00.138660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb0a11-29dc-4cc8-91f0-3067671e7504 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:00.138943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb0a11-29dc-4cc8-91f0-3067671e7504 admin admin] PATH_INFO: `/v3/projects/ecdf7823601b48b78593f51178ee02d2/users/4cf615e98f154a249b2dada2610dce9c/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:00.156686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb0a11-29dc-4cc8-91f0-3067671e7504 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecdf7823601b48b78593f51178ee02d2, user_id=4cf615e98f154a249b2dada2610dce9c, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:00.160611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb0a11-29dc-4cc8-91f0-3067671e7504 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:00.240497 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3623/7210] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:31:00 2022] PUT /identity/v3/projects/ecdf7823601b48b78593f51178ee02d2/users/4cf615e98f154a249b2dada2610dce9c/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:00.244489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699dc2a6-c0c8-476f-89e7-5d57137107a3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:00.255142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699dc2a6-c0c8-476f-89e7-5d57137107a3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:00.289434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb0a11-29dc-4cc8-91f0-3067671e7504 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:00.291209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ddcb0a11-29dc-4cc8-91f0-3067671e7504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:00.332319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d96cb-22e0-49ba-8b90-9c8f27f9ddba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:00.332319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d96cb-22e0-49ba-8b90-9c8f27f9ddba admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:00.332319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d96cb-22e0-49ba-8b90-9c8f27f9ddba admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:00.332319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217d96cb-22e0-49ba-8b90-9c8f27f9ddba admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:00.332710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d96cb-22e0-49ba-8b90-9c8f27f9ddba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:00.332710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217d96cb-22e0-49ba-8b90-9c8f27f9ddba admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:00.346497 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3588/7211] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:31:00.347597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d96cb-22e0-49ba-8b90-9c8f27f9ddba admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:00.358022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217d96cb-22e0-49ba-8b90-9c8f27f9ddba admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:00.365590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f955c5a-0e0e-4089-a347-b60138acd521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:00.366401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f955c5a-0e0e-4089-a347-b60138acd521 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:00.366689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f955c5a-0e0e-4089-a347-b60138acd521 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:00.366969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f955c5a-0e0e-4089-a347-b60138acd521 admin admin] PATH_INFO: `/v3/projects/ecdf7823601b48b78593f51178ee02d2/users/4cf615e98f154a249b2dada2610dce9c/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:00.367932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb0a11-29dc-4cc8-91f0-3067671e7504 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:00.369002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f955c5a-0e0e-4089-a347-b60138acd521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecdf7823601b48b78593f51178ee02d2, user_id=4cf615e98f154a249b2dada2610dce9c, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:00.373675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505a43a-31d9-400e-8d38-489283ca3c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:00.378124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f955c5a-0e0e-4089-a347-b60138acd521 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:00.378895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505a43a-31d9-400e-8d38-489283ca3c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:00.378895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505a43a-31d9-400e-8d38-489283ca3c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:00.378895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505a43a-31d9-400e-8d38-489283ca3c2e admin admin] PATH_INFO: `/v3/projects/538959f5d1cc42698adcd7d2d5ae99e3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:00.378895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505a43a-31d9-400e-8d38-489283ca3c2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=538959f5d1cc42698adcd7d2d5ae99e3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:00.380073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1505a43a-31d9-400e-8d38-489283ca3c2e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:00.393929 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3589/7212] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:31:00 2022] PUT /identity/v3/projects/ecdf7823601b48b78593f51178ee02d2/users/4cf615e98f154a249b2dada2610dce9c/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:00.408648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaafbf0-ea25-4918-892f-77cb5d316a0c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:00.408648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaafbf0-ea25-4918-892f-77cb5d316a0c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:00.408648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaafbf0-ea25-4918-892f-77cb5d316a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:00.444961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-abaafbf0-ea25-4918-892f-77cb5d316a0c None None] MFA Rules not processed for user `4cf615e98f154a249b2dada2610dce9c`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:00.446313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-abaafbf0-ea25-4918-892f-77cb5d316a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:00.531365 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3590/7213] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:00 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:00.538459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1505a43a-31d9-400e-8d38-489283ca3c2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '538959f5d1cc42698adcd7d2d5ae99e3', 'request_id': 'req-1505a43a-31d9-400e-8d38-489283ca3c2e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:00.543797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaafbf0-ea25-4918-892f-77cb5d316a0c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:00.544746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1505a43a-31d9-400e-8d38-489283ca3c2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '538959f5d1cc42698adcd7d2d5ae99e3', 'request_id': 'req-1505a43a-31d9-400e-8d38-489283ca3c2e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:00.545371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-abaafbf0-ea25-4918-892f-77cb5d316a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:00.621589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaafbf0-ea25-4918-892f-77cb5d316a0c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:00.634277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1505a43a-31d9-400e-8d38-489283ca3c2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '538959f5d1cc42698adcd7d2d5ae99e3', 'request_id': 'req-1505a43a-31d9-400e-8d38-489283ca3c2e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:00.646699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1505a43a-31d9-400e-8d38-489283ca3c2e admin admin] The token cache is being invalidate because project 538959f5d1cc42698adcd7d2d5ae99e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:31:00.646994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1505a43a-31d9-400e-8d38-489283ca3c2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:00.647654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1505a43a-31d9-400e-8d38-489283ca3c2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:00.649933 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3624/7214] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:31:00 2022] DELETE /identity/v3/projects/538959f5d1cc42698adcd7d2d5ae99e3 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:00.653669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505a43a-31d9-400e-8d38-489283ca3c2e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:00.655098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1505a43a-31d9-400e-8d38-489283ca3c2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:00.671673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb909a-6408-498b-b82d-e95b7679a486 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:00.672388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb909a-6408-498b-b82d-e95b7679a486 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:00.675098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb909a-6408-498b-b82d-e95b7679a486 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:00.675098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb909a-6408-498b-b82d-e95b7679a486 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:00.676770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f2eb909a-6408-498b-b82d-e95b7679a486 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:00.722969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1505a43a-31d9-400e-8d38-489283ca3c2e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:00.728013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca98ab2-b48b-4b8f-a5c7-56ff73d434d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:00.728977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca98ab2-b48b-4b8f-a5c7-56ff73d434d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:00.729202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca98ab2-b48b-4b8f-a5c7-56ff73d434d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:00.729481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca98ab2-b48b-4b8f-a5c7-56ff73d434d8 admin admin] PATH_INFO: `/v3/users/2858a8d717504381a81a93debfb03892` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:00.731384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca98ab2-b48b-4b8f-a5c7-56ff73d434d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2858a8d717504381a81a93debfb03892)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:00.733388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca98ab2-b48b-4b8f-a5c7-56ff73d434d8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:00.742401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb909a-6408-498b-b82d-e95b7679a486 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:00.744000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb909a-6408-498b-b82d-e95b7679a486 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:00.754706 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3591/7215] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:31:00 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:31:00.783493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:00.891365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb909a-6408-498b-b82d-e95b7679a486 service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:00.892854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f2eb909a-6408-498b-b82d-e95b7679a486 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:00.900979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2ca98ab2-b48b-4b8f-a5c7-56ff73d434d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2858a8d717504381a81a93debfb03892'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:00.906577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb909a-6408-498b-b82d-e95b7679a486 service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:00.923328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2ca98ab2-b48b-4b8f-a5c7-56ff73d434d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2858a8d717504381a81a93debfb03892'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:00.923328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2ca98ab2-b48b-4b8f-a5c7-56ff73d434d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2858a8d717504381a81a93debfb03892'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:00.923328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2ca98ab2-b48b-4b8f-a5c7-56ff73d434d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2858a8d717504381a81a93debfb03892'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:00.923625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e43580-d78f-4f8b-9697-dc8ff2a73571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:00.923625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e43580-d78f-4f8b-9697-dc8ff2a73571 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:00.923625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e43580-d78f-4f8b-9697-dc8ff2a73571 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:00.923625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e43580-d78f-4f8b-9697-dc8ff2a73571 admin admin] PATH_INFO: `/v3/users/a870b308cfc844af989fe1ce311855e3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:00.924012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e43580-d78f-4f8b-9697-dc8ff2a73571 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a870b308cfc844af989fe1ce311855e3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:00.924012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e43580-d78f-4f8b-9697-dc8ff2a73571 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:00.966323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:01.022933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2ca98ab2-b48b-4b8f-a5c7-56ff73d434d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2858a8d717504381a81a93debfb03892'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:01.043730 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3625/7216] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:31:00 2022] DELETE /identity/v3/users/2858a8d717504381a81a93debfb03892 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:31:01.188700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:01.196911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51e43580-d78f-4f8b-9697-dc8ff2a73571 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a870b308cfc844af989fe1ce311855e3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:01.197592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51e43580-d78f-4f8b-9697-dc8ff2a73571 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a870b308cfc844af989fe1ce311855e3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:01.204726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51e43580-d78f-4f8b-9697-dc8ff2a73571 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a870b308cfc844af989fe1ce311855e3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:01.296363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51e43580-d78f-4f8b-9697-dc8ff2a73571 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a870b308cfc844af989fe1ce311855e3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:01.297726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-51e43580-d78f-4f8b-9697-dc8ff2a73571 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a870b308cfc844af989fe1ce311855e3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:01.305695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca98ab2-b48b-4b8f-a5c7-56ff73d434d8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:01.306357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2ca98ab2-b48b-4b8f-a5c7-56ff73d434d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:01.328352 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3592/7217] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:31:00 2022] DELETE /identity/v3/users/a870b308cfc844af989fe1ce311855e3 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:31:01.333172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f0236-5199-4764-b49b-7897ae804698 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:01.334144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f0236-5199-4764-b49b-7897ae804698 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:01.334144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f0236-5199-4764-b49b-7897ae804698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:01.361689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4b8f0236-5199-4764-b49b-7897ae804698 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:01.364255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4b8f0236-5199-4764-b49b-7897ae804698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:01.402331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca98ab2-b48b-4b8f-a5c7-56ff73d434d8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:01.424603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:01.441989 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3593/7218] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:01.449553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f0236-5199-4764-b49b-7897ae804698 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:01.460166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4b8f0236-5199-4764-b49b-7897ae804698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:01.467462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc2ac4-43e3-4595-ab85-843f522152d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:01.468571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc2ac4-43e3-4595-ab85-843f522152d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:01.468860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc2ac4-43e3-4595-ab85-843f522152d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:01.469142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc2ac4-43e3-4595-ab85-843f522152d0 admin admin] PATH_INFO: `/v3/projects/2f92fc484a66469f8ccaeff81454ca7d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:01.470521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc2ac4-43e3-4595-ab85-843f522152d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f92fc484a66469f8ccaeff81454ca7d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:01.473091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc2ac4-43e3-4595-ab85-843f522152d0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:01.519828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f0236-5199-4764-b49b-7897ae804698 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:01.523488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aceefc-c674-4801-b392-be72bde2c0fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:01.524171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aceefc-c674-4801-b392-be72bde2c0fc service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:01.524454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aceefc-c674-4801-b392-be72bde2c0fc service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:01.524729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aceefc-c674-4801-b392-be72bde2c0fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:01.538196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aceefc-c674-4801-b392-be72bde2c0fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:01.539633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aceefc-c674-4801-b392-be72bde2c0fc service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:01.549152 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3594/7219] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:31:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:31:01.695342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3bc2ac4-43e3-4595-ab85-843f522152d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f92fc484a66469f8ccaeff81454ca7d', 'request_id': 'req-f3bc2ac4-43e3-4595-ab85-843f522152d0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:01.706004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3bc2ac4-43e3-4595-ab85-843f522152d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f92fc484a66469f8ccaeff81454ca7d', 'request_id': 'req-f3bc2ac4-43e3-4595-ab85-843f522152d0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:01.822240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3bc2ac4-43e3-4595-ab85-843f522152d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f92fc484a66469f8ccaeff81454ca7d', 'request_id': 'req-f3bc2ac4-43e3-4595-ab85-843f522152d0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:01.837231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3bc2ac4-43e3-4595-ab85-843f522152d0 admin admin] The token cache is being invalidate because project 2f92fc484a66469f8ccaeff81454ca7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:31:01.837604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3bc2ac4-43e3-4595-ab85-843f522152d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:01.838698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f3bc2ac4-43e3-4595-ab85-843f522152d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:01.840547 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3626/7220] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:31:01 2022] DELETE /identity/v3/projects/2f92fc484a66469f8ccaeff81454ca7d => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:31:01.845730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15516b9-a49c-4a9b-b4c2-396ddaec6a5c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:01.846321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15516b9-a49c-4a9b-b4c2-396ddaec6a5c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:01.846790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15516b9-a49c-4a9b-b4c2-396ddaec6a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:01.880985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c15516b9-a49c-4a9b-b4c2-396ddaec6a5c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:01.886326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c15516b9-a49c-4a9b-b4c2-396ddaec6a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:01.920509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:01.951519 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3595/7221] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:31:01.954470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e0901-4ad4-41f7-8abd-cea0cc1b6012 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:01.954833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e0901-4ad4-41f7-8abd-cea0cc1b6012 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:01.955193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e0901-4ad4-41f7-8abd-cea0cc1b6012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:01.976920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fb6e0901-4ad4-41f7-8abd-cea0cc1b6012 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:01.978292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fb6e0901-4ad4-41f7-8abd-cea0cc1b6012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:01.986695 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3627/7222] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:01.991244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15516b9-a49c-4a9b-b4c2-396ddaec6a5c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:02.000588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15516b9-a49c-4a9b-b4c2-396ddaec6a5c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:02.005827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320c277-6181-4d42-a7d6-943df83ffb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:02.007170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320c277-6181-4d42-a7d6-943df83ffb86 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:02.007170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320c277-6181-4d42-a7d6-943df83ffb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:02.007170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320c277-6181-4d42-a7d6-943df83ffb86 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:02.007744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320c277-6181-4d42-a7d6-943df83ffb86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:31:02.008052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320c277-6181-4d42-a7d6-943df83ffb86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:02.009699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320c277-6181-4d42-a7d6-943df83ffb86 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:02.021860 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3596/7223] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:31:02.026270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e0901-4ad4-41f7-8abd-cea0cc1b6012 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:02.035986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e0901-4ad4-41f7-8abd-cea0cc1b6012 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:02.041084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e80f0-65ef-4303-880b-2c8590472769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:02.041741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e80f0-65ef-4303-880b-2c8590472769 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:02.042065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e80f0-65ef-4303-880b-2c8590472769 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:02.042348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e80f0-65ef-4303-880b-2c8590472769 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:02.043065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e80f0-65ef-4303-880b-2c8590472769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:02.045033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e80f0-65ef-4303-880b-2c8590472769 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:02.071223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320c277-6181-4d42-a7d6-943df83ffb86 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:02.073113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7320c277-6181-4d42-a7d6-943df83ffb86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:02.091407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320c277-6181-4d42-a7d6-943df83ffb86 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:02.096387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56e6c4-76ff-4dac-baa7-e217a22de434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:02.097412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56e6c4-76ff-4dac-baa7-e217a22de434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:02.097694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56e6c4-76ff-4dac-baa7-e217a22de434 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:02.098003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e56e6c4-76ff-4dac-baa7-e217a22de434 admin admin] PATH_INFO: `/v3/projects/6b54dfce3a2b41a8a13110ada4f335ae` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:02.099232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56e6c4-76ff-4dac-baa7-e217a22de434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b54dfce3a2b41a8a13110ada4f335ae)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:02.101176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e56e6c4-76ff-4dac-baa7-e217a22de434 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:02.151614 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3628/7224] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:31:02 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:31:02.155779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e80f0-65ef-4303-880b-2c8590472769 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:02.228694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e80f0-65ef-4303-880b-2c8590472769 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:02.233453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434bea7d-fce1-41e6-bc47-a597a9851518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:02.234153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bea7d-fce1-41e6-bc47-a597a9851518 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:02.234440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bea7d-fce1-41e6-bc47-a597a9851518 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:02.234720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434bea7d-fce1-41e6-bc47-a597a9851518 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:02.235437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434bea7d-fce1-41e6-bc47-a597a9851518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:02.237992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434bea7d-fce1-41e6-bc47-a597a9851518 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:02.283193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5e56e6c4-76ff-4dac-baa7-e217a22de434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b54dfce3a2b41a8a13110ada4f335ae', 'request_id': 'req-5e56e6c4-76ff-4dac-baa7-e217a22de434'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:02.289989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5e56e6c4-76ff-4dac-baa7-e217a22de434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b54dfce3a2b41a8a13110ada4f335ae', 'request_id': 'req-5e56e6c4-76ff-4dac-baa7-e217a22de434'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:02.349265 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3629/7225] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:31:02 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:31:02.350833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434bea7d-fce1-41e6-bc47-a597a9851518 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:02.361717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434bea7d-fce1-41e6-bc47-a597a9851518 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:02.412211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:02.418963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6ec24-11a8-40b3-a68f-098f2b357ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:02.419815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6ec24-11a8-40b3-a68f-098f2b357ccf admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:02.420124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6ec24-11a8-40b3-a68f-098f2b357ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:02.420440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b6ec24-11a8-40b3-a68f-098f2b357ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:02.421181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6ec24-11a8-40b3-a68f-098f2b357ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:02.422837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b6ec24-11a8-40b3-a68f-098f2b357ccf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:02.436167 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3630/7226] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:31:02.437230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6ec24-11a8-40b3-a68f-098f2b357ccf admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:02.440106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5e56e6c4-76ff-4dac-baa7-e217a22de434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b54dfce3a2b41a8a13110ada4f335ae', 'request_id': 'req-5e56e6c4-76ff-4dac-baa7-e217a22de434'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:02.458144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b6ec24-11a8-40b3-a68f-098f2b357ccf admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:02.458144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e76db-cac0-41cb-9799-0808ae0dd650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:02.458144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e76db-cac0-41cb-9799-0808ae0dd650 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:02.458144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e76db-cac0-41cb-9799-0808ae0dd650 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:02.458729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e76db-cac0-41cb-9799-0808ae0dd650 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:02.458729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e76db-cac0-41cb-9799-0808ae0dd650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:02.458729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e76db-cac0-41cb-9799-0808ae0dd650 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:02.468392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5e56e6c4-76ff-4dac-baa7-e217a22de434 admin admin] The token cache is being invalidate because project 6b54dfce3a2b41a8a13110ada4f335ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:31:02.468689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5e56e6c4-76ff-4dac-baa7-e217a22de434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:02.469370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5e56e6c4-76ff-4dac-baa7-e217a22de434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:02.470445 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3631/7227] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:31:02.471820 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3597/7228] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:31:02 2022] DELETE /identity/v3/projects/6b54dfce3a2b41a8a13110ada4f335ae => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:02.474907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e76db-cac0-41cb-9799-0808ae0dd650 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:02.475036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56e6c4-76ff-4dac-baa7-e217a22de434 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:02.476356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5e56e6c4-76ff-4dac-baa7-e217a22de434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:02.476767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-329e76db-cac0-41cb-9799-0808ae0dd650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:02.490338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e56e6c4-76ff-4dac-baa7-e217a22de434 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:02.492279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e76db-cac0-41cb-9799-0808ae0dd650 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:02.495285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c53ca-bb00-4b10-b487-08965def8394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:02.496324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c53ca-bb00-4b10-b487-08965def8394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:02.496617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c53ca-bb00-4b10-b487-08965def8394 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:02.496894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c53ca-bb00-4b10-b487-08965def8394 admin admin] PATH_INFO: `/v3/users/4cf615e98f154a249b2dada2610dce9c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:02.497139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a5864-0058-457d-b0ce-ab19ec5337b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:02.498067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a5864-0058-457d-b0ce-ab19ec5337b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:02.498175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c53ca-bb00-4b10-b487-08965def8394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cf615e98f154a249b2dada2610dce9c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:02.498351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a5864-0058-457d-b0ce-ab19ec5337b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:02.498633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a5864-0058-457d-b0ce-ab19ec5337b0 admin admin] PATH_INFO: `/v3/projects/5241f87816134389b836a8eae1d53bd3/users/c93401a279c9403ca62627e829082968/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:02.500076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c53ca-bb00-4b10-b487-08965def8394 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:02.514753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a5864-0058-457d-b0ce-ab19ec5337b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5241f87816134389b836a8eae1d53bd3, user_id=c93401a279c9403ca62627e829082968, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:02.517765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a5864-0058-457d-b0ce-ab19ec5337b0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:02.619834 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3632/7229] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:31:02 2022] PUT /identity/v3/projects/5241f87816134389b836a8eae1d53bd3/users/c93401a279c9403ca62627e829082968/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:02.624848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc6466-5f77-4427-8661-6815bbb976ce None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:02.625138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc6466-5f77-4427-8661-6815bbb976ce None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:02.625420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc6466-5f77-4427-8661-6815bbb976ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:02.642659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8fdc6466-5f77-4427-8661-6815bbb976ce None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:02.643934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8fdc6466-5f77-4427-8661-6815bbb976ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:02.707154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-140c53ca-bb00-4b10-b487-08965def8394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cf615e98f154a249b2dada2610dce9c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:02.707892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-140c53ca-bb00-4b10-b487-08965def8394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cf615e98f154a249b2dada2610dce9c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:02.715116 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3633/7230] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:31:02.715408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-140c53ca-bb00-4b10-b487-08965def8394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cf615e98f154a249b2dada2610dce9c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:02.732290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc6466-5f77-4427-8661-6815bbb976ce None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:02.733636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8fdc6466-5f77-4427-8661-6815bbb976ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:02.800462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc6466-5f77-4427-8661-6815bbb976ce None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:02.804246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cb447-1df5-4269-8b7b-cbe4cdf7b6bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:02.804901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cb447-1df5-4269-8b7b-cbe4cdf7b6bd service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:02.805189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cb447-1df5-4269-8b7b-cbe4cdf7b6bd service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:02.805468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cb447-1df5-4269-8b7b-cbe4cdf7b6bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:02.814865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267cb447-1df5-4269-8b7b-cbe4cdf7b6bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:02.816417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267cb447-1df5-4269-8b7b-cbe4cdf7b6bd service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:02.826231 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3634/7231] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:31:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:31:02.834533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-140c53ca-bb00-4b10-b487-08965def8394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cf615e98f154a249b2dada2610dce9c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:02.835166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-140c53ca-bb00-4b10-b487-08965def8394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cf615e98f154a249b2dada2610dce9c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:02.841604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:02.861696 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3598/7232] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:31:02 2022] DELETE /identity/v3/users/4cf615e98f154a249b2dada2610dce9c => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:31:02.937972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:03.447114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cb447-1df5-4269-8b7b-cbe4cdf7b6bd service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:03.460273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cb447-1df5-4269-8b7b-cbe4cdf7b6bd service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:03.535465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600737d5-5799-4a93-9563-97a1cea675b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:03.536857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600737d5-5799-4a93-9563-97a1cea675b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:03.537151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600737d5-5799-4a93-9563-97a1cea675b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:03.537626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600737d5-5799-4a93-9563-97a1cea675b7 admin admin] PATH_INFO: `/v3/projects/ecdf7823601b48b78593f51178ee02d2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:03.538776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600737d5-5799-4a93-9563-97a1cea675b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecdf7823601b48b78593f51178ee02d2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:03.544359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600737d5-5799-4a93-9563-97a1cea675b7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:03.677824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-600737d5-5799-4a93-9563-97a1cea675b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecdf7823601b48b78593f51178ee02d2', 'request_id': 'req-600737d5-5799-4a93-9563-97a1cea675b7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:03.683982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-600737d5-5799-4a93-9563-97a1cea675b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecdf7823601b48b78593f51178ee02d2', 'request_id': 'req-600737d5-5799-4a93-9563-97a1cea675b7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:03.823347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-600737d5-5799-4a93-9563-97a1cea675b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecdf7823601b48b78593f51178ee02d2', 'request_id': 'req-600737d5-5799-4a93-9563-97a1cea675b7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:03.835971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-600737d5-5799-4a93-9563-97a1cea675b7 admin admin] The token cache is being invalidate because project ecdf7823601b48b78593f51178ee02d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:31:03.836568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-600737d5-5799-4a93-9563-97a1cea675b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:03.837049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-600737d5-5799-4a93-9563-97a1cea675b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:03.838873 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3635/7233] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:31:03 2022] DELETE /identity/v3/projects/ecdf7823601b48b78593f51178ee02d2 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:03.843954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd924ea-6094-48b7-baa1-812c1ebf377e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:03.844393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd924ea-6094-48b7-baa1-812c1ebf377e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:03.844521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd924ea-6094-48b7-baa1-812c1ebf377e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:03.863378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6dd924ea-6094-48b7-baa1-812c1ebf377e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:03.864597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6dd924ea-6094-48b7-baa1-812c1ebf377e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:03.923698 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3599/7234] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:03.932630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd0789-9455-44ae-a941-8cae7804a0be None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:03.932630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd0789-9455-44ae-a941-8cae7804a0be None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:03.932630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd0789-9455-44ae-a941-8cae7804a0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:03.952683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-61bd0789-9455-44ae-a941-8cae7804a0be None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:03.954033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-61bd0789-9455-44ae-a941-8cae7804a0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:03.963094 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3636/7235] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:03.967789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd924ea-6094-48b7-baa1-812c1ebf377e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:03.978163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd924ea-6094-48b7-baa1-812c1ebf377e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:03.983246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57522830-a676-4302-b1fd-ecf89b87d5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:03.984002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57522830-a676-4302-b1fd-ecf89b87d5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:03.984351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57522830-a676-4302-b1fd-ecf89b87d5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:03.984733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57522830-a676-4302-b1fd-ecf89b87d5d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:03.985465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57522830-a676-4302-b1fd-ecf89b87d5d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:31:03.985778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57522830-a676-4302-b1fd-ecf89b87d5d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:03.988788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57522830-a676-4302-b1fd-ecf89b87d5d1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:04.008900 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3600/7236] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:31:04.010407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd0789-9455-44ae-a941-8cae7804a0be None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:04.019943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd0789-9455-44ae-a941-8cae7804a0be None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:04.024587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3643d1-6514-45ff-b268-536cd87a0e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:04.025385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3643d1-6514-45ff-b268-536cd87a0e87 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:04.025652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3643d1-6514-45ff-b268-536cd87a0e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:04.026076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3643d1-6514-45ff-b268-536cd87a0e87 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:04.034438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3643d1-6514-45ff-b268-536cd87a0e87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:04.034438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3643d1-6514-45ff-b268-536cd87a0e87 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:04.156989 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3637/7237] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:31:04 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:31:04.161700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57522830-a676-4302-b1fd-ecf89b87d5d1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:04.172826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57522830-a676-4302-b1fd-ecf89b87d5d1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:04.245695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfef7cd-7c5e-4ab4-902c-69c31691a5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:04.246654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfef7cd-7c5e-4ab4-902c-69c31691a5eb admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:04.246943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfef7cd-7c5e-4ab4-902c-69c31691a5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:04.247224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfef7cd-7c5e-4ab4-902c-69c31691a5eb admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:04.248042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfef7cd-7c5e-4ab4-902c-69c31691a5eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:04.250498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfef7cd-7c5e-4ab4-902c-69c31691a5eb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:04.394946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:04.407762 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3601/7238] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:31:04 2022] POST /identity/v3/users => generated 287 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:31:04.409536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3643d1-6514-45ff-b268-536cd87a0e87 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:04.422985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3643d1-6514-45ff-b268-536cd87a0e87 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:04.428141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077039d6-352c-4579-9157-c9fecbc3ff21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:04.428841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077039d6-352c-4579-9157-c9fecbc3ff21 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:04.429130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077039d6-352c-4579-9157-c9fecbc3ff21 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:04.429553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077039d6-352c-4579-9157-c9fecbc3ff21 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:04.430193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077039d6-352c-4579-9157-c9fecbc3ff21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:04.431813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077039d6-352c-4579-9157-c9fecbc3ff21 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:04.448746 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3638/7239] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:31:04.449805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfef7cd-7c5e-4ab4-902c-69c31691a5eb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:04.463630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfef7cd-7c5e-4ab4-902c-69c31691a5eb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:04.469006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7892495-3326-41d7-8ebe-9eb9a79ed796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:04.469669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7892495-3326-41d7-8ebe-9eb9a79ed796 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:04.469989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7892495-3326-41d7-8ebe-9eb9a79ed796 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:04.470286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7892495-3326-41d7-8ebe-9eb9a79ed796 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:04.471046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7892495-3326-41d7-8ebe-9eb9a79ed796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:04.473362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7892495-3326-41d7-8ebe-9eb9a79ed796 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:04.487560 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3602/7240] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:31:04.498633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077039d6-352c-4579-9157-c9fecbc3ff21 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:04.517823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077039d6-352c-4579-9157-c9fecbc3ff21 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:04.524737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7ca31-ed62-41d1-bd30-c24eedde922d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:04.525659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7ca31-ed62-41d1-bd30-c24eedde922d admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:04.525975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7ca31-ed62-41d1-bd30-c24eedde922d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:04.526254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7ca31-ed62-41d1-bd30-c24eedde922d admin admin] PATH_INFO: `/v3/projects/7006c0f4463f4db0b9e6e481602439e1/users/499d79bbfe384d81a98d50c4e6bc0dc2/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:04.544933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7ca31-ed62-41d1-bd30-c24eedde922d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7006c0f4463f4db0b9e6e481602439e1, user_id=499d79bbfe384d81a98d50c4e6bc0dc2, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:04.556454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7ca31-ed62-41d1-bd30-c24eedde922d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:04.698777 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3639/7241] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:31:04 2022] PUT /identity/v3/projects/7006c0f4463f4db0b9e6e481602439e1/users/499d79bbfe384d81a98d50c4e6bc0dc2/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:04.711025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c53b1-c4cd-415a-b3f9-0f82cf913449 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:04.711560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c53b1-c4cd-415a-b3f9-0f82cf913449 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:04.712067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1c53b1-c4cd-415a-b3f9-0f82cf913449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:04.767554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:04.776146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8b1c53b1-c4cd-415a-b3f9-0f82cf913449 None None] MFA Rules not processed for user `499d79bbfe384d81a98d50c4e6bc0dc2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:04.777380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8b1c53b1-c4cd-415a-b3f9-0f82cf913449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:04.854698 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3603/7242] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:31:04.859670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7ca31-ed62-41d1-bd30-c24eedde922d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:04.931554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7ca31-ed62-41d1-bd30-c24eedde922d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:04.938719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8bf227-71fa-44e3-812c-8f9cedc373af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:04.939882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8bf227-71fa-44e3-812c-8f9cedc373af admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:04.940894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8bf227-71fa-44e3-812c-8f9cedc373af admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:04.941179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8bf227-71fa-44e3-812c-8f9cedc373af admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:04.942174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8bf227-71fa-44e3-812c-8f9cedc373af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:04.944124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8bf227-71fa-44e3-812c-8f9cedc373af admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:05.066925 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3640/7243] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:31:04 2022] POST /identity/v3/projects => generated 371 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:31:05.070569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c53b1-c4cd-415a-b3f9-0f82cf913449 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:05.086138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1c53b1-c4cd-415a-b3f9-0f82cf913449 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:05.152741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08c752-4009-4360-9758-d4ad8a55a000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:05.153432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08c752-4009-4360-9758-d4ad8a55a000 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:05.153723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08c752-4009-4360-9758-d4ad8a55a000 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:05.154040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08c752-4009-4360-9758-d4ad8a55a000 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:05.154761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08c752-4009-4360-9758-d4ad8a55a000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:05.157008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08c752-4009-4360-9758-d4ad8a55a000 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:05.288028 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3604/7244] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:31:05 2022] POST /identity/v3/users => generated 292 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:31:05.289271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8bf227-71fa-44e3-812c-8f9cedc373af admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:05.300407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8bf227-71fa-44e3-812c-8f9cedc373af admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:05.305275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab2d31-7310-4725-be17-f56d7e5676c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:05.306179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab2d31-7310-4725-be17-f56d7e5676c7 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:05.306329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab2d31-7310-4725-be17-f56d7e5676c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:05.306611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ab2d31-7310-4725-be17-f56d7e5676c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:05.307371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab2d31-7310-4725-be17-f56d7e5676c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:05.308985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ab2d31-7310-4725-be17-f56d7e5676c7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:05.321211 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3641/7245] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:31:05.328060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08c752-4009-4360-9758-d4ad8a55a000 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:05.340050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08c752-4009-4360-9758-d4ad8a55a000 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:05.356069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9087cc-5802-478d-8c00-c9078918aaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:05.357590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9087cc-5802-478d-8c00-c9078918aaf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:05.357962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9087cc-5802-478d-8c00-c9078918aaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:05.358240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9087cc-5802-478d-8c00-c9078918aaf9 admin admin] PATH_INFO: `/v3/projects/203c2c8c4e7c4197bf55fda8aa5d8b5c/users/18624e9a2fda48618d1b339d7749b41a/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:05.392587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9087cc-5802-478d-8c00-c9078918aaf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=203c2c8c4e7c4197bf55fda8aa5d8b5c, user_id=18624e9a2fda48618d1b339d7749b41a, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:05.395461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9087cc-5802-478d-8c00-c9078918aaf9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:05.542194 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3605/7246] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:31:05 2022] PUT /identity/v3/projects/203c2c8c4e7c4197bf55fda8aa5d8b5c/users/18624e9a2fda48618d1b339d7749b41a/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:05.550401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab2d31-7310-4725-be17-f56d7e5676c7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:05.572024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ab2d31-7310-4725-be17-f56d7e5676c7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:05.694533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa624144-bfa1-4b1a-bc16-76487f3ad2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:05.697112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa624144-bfa1-4b1a-bc16-76487f3ad2da admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:05.697112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa624144-bfa1-4b1a-bc16-76487f3ad2da admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:05.697334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa624144-bfa1-4b1a-bc16-76487f3ad2da admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:05.698165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa624144-bfa1-4b1a-bc16-76487f3ad2da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:05.700108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa624144-bfa1-4b1a-bc16-76487f3ad2da admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:05.732094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9087cc-5802-478d-8c00-c9078918aaf9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:05.739069 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3642/7247] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:31:05.746779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9087cc-5802-478d-8c00-c9078918aaf9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:05.753048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b173b86-32ee-4dc9-aa0d-edd5f0c8e6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:05.756554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b173b86-32ee-4dc9-aa0d-edd5f0c8e6ee admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:05.756554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b173b86-32ee-4dc9-aa0d-edd5f0c8e6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:05.756554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b173b86-32ee-4dc9-aa0d-edd5f0c8e6ee admin admin] PATH_INFO: `/v3/projects/203c2c8c4e7c4197bf55fda8aa5d8b5c/users/18624e9a2fda48618d1b339d7749b41a/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:05.760858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b173b86-32ee-4dc9-aa0d-edd5f0c8e6ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=203c2c8c4e7c4197bf55fda8aa5d8b5c, user_id=18624e9a2fda48618d1b339d7749b41a, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:05.764394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b173b86-32ee-4dc9-aa0d-edd5f0c8e6ee admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:05.782559 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3606/7248] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:31:05 2022] PUT /identity/v3/projects/203c2c8c4e7c4197bf55fda8aa5d8b5c/users/18624e9a2fda48618d1b339d7749b41a/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:05.792026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754b38b-032d-4e79-8906-e314468839e0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:05.792378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754b38b-032d-4e79-8906-e314468839e0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:05.792595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754b38b-032d-4e79-8906-e314468839e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:05.832571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f754b38b-032d-4e79-8906-e314468839e0 None None] MFA Rules not processed for user `18624e9a2fda48618d1b339d7749b41a`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:05.832571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f754b38b-032d-4e79-8906-e314468839e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:05.904375 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3643/7249] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:05 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:05.917358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b173b86-32ee-4dc9-aa0d-edd5f0c8e6ee admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:05.919205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5b173b86-32ee-4dc9-aa0d-edd5f0c8e6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:05.987823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b173b86-32ee-4dc9-aa0d-edd5f0c8e6ee admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:05.991656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983a816-b8fd-41e4-b447-b7367b7baf11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:05.992402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983a816-b8fd-41e4-b447-b7367b7baf11 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:05.992829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983a816-b8fd-41e4-b447-b7367b7baf11 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:06.000803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983a816-b8fd-41e4-b447-b7367b7baf11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:06.002554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983a816-b8fd-41e4-b447-b7367b7baf11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:06.003993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983a816-b8fd-41e4-b447-b7367b7baf11 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:06.014089 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3607/7250] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:31:05 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:31:06.134776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:07.332369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754b38b-032d-4e79-8906-e314468839e0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:07.440729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754b38b-032d-4e79-8906-e314468839e0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:07.447056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f53df-986d-4625-b771-ed2184236fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:07.447980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f53df-986d-4625-b771-ed2184236fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:07.448822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f53df-986d-4625-b771-ed2184236fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:07.449113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f53df-986d-4625-b771-ed2184236fe9 admin admin] PATH_INFO: `/v3/users/499d79bbfe384d81a98d50c4e6bc0dc2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:07.450423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f53df-986d-4625-b771-ed2184236fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=499d79bbfe384d81a98d50c4e6bc0dc2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:07.452856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f53df-986d-4625-b771-ed2184236fe9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:07.651158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:07.752865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7e0f53df-986d-4625-b771-ed2184236fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '499d79bbfe384d81a98d50c4e6bc0dc2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:07.764451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7e0f53df-986d-4625-b771-ed2184236fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '499d79bbfe384d81a98d50c4e6bc0dc2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:07.769239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7e0f53df-986d-4625-b771-ed2184236fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '499d79bbfe384d81a98d50c4e6bc0dc2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:07.770016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7e0f53df-986d-4625-b771-ed2184236fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '499d79bbfe384d81a98d50c4e6bc0dc2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:07.913233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7e0f53df-986d-4625-b771-ed2184236fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '499d79bbfe384d81a98d50c4e6bc0dc2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:07.932734 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3644/7251] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:31:07 2022] DELETE /identity/v3/users/499d79bbfe384d81a98d50c4e6bc0dc2 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:07.937532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0af69-be20-4377-8c7f-9c4c1b080ab9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:07.937820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0af69-be20-4377-8c7f-9c4c1b080ab9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:07.938140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0af69-be20-4377-8c7f-9c4c1b080ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:07.977923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-03f0af69-be20-4377-8c7f-9c4c1b080ab9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:07.979327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-03f0af69-be20-4377-8c7f-9c4c1b080ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:08.088675 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3608/7252] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:31:08.106791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f53df-986d-4625-b771-ed2184236fe9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:08.108811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7e0f53df-986d-4625-b771-ed2184236fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:08.254961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f53df-986d-4625-b771-ed2184236fe9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:08.259527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4df9d-6d9e-4be5-b7ea-7e7085bed881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:08.260422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4df9d-6d9e-4be5-b7ea-7e7085bed881 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:08.260910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4df9d-6d9e-4be5-b7ea-7e7085bed881 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:08.261370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4df9d-6d9e-4be5-b7ea-7e7085bed881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:08.274765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4df9d-6d9e-4be5-b7ea-7e7085bed881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:08.276470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4df9d-6d9e-4be5-b7ea-7e7085bed881 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:08.300096 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3645/7253] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:31:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:31:09.053342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0af69-be20-4377-8c7f-9c4c1b080ab9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:09.054920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-03f0af69-be20-4377-8c7f-9c4c1b080ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:09.080370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0af69-be20-4377-8c7f-9c4c1b080ab9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:09.084746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f54ec0-6f48-4db4-9f7d-2f109900ffff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:09.086094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f54ec0-6f48-4db4-9f7d-2f109900ffff admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:09.086668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f54ec0-6f48-4db4-9f7d-2f109900ffff admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:09.087096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f54ec0-6f48-4db4-9f7d-2f109900ffff admin admin] PATH_INFO: `/v3/projects/7006c0f4463f4db0b9e6e481602439e1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:09.088443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f54ec0-6f48-4db4-9f7d-2f109900ffff admin admin] RBAC: Authorizing `identity:delete_project(project_id=7006c0f4463f4db0b9e6e481602439e1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:09.091140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f54ec0-6f48-4db4-9f7d-2f109900ffff admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:09.259248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-01f54ec0-6f48-4db4-9f7d-2f109900ffff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7006c0f4463f4db0b9e6e481602439e1', 'request_id': 'req-01f54ec0-6f48-4db4-9f7d-2f109900ffff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:09.278052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:09.282986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-01f54ec0-6f48-4db4-9f7d-2f109900ffff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7006c0f4463f4db0b9e6e481602439e1', 'request_id': 'req-01f54ec0-6f48-4db4-9f7d-2f109900ffff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:09.372297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-01f54ec0-6f48-4db4-9f7d-2f109900ffff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7006c0f4463f4db0b9e6e481602439e1', 'request_id': 'req-01f54ec0-6f48-4db4-9f7d-2f109900ffff'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:09.387469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-01f54ec0-6f48-4db4-9f7d-2f109900ffff admin admin] The token cache is being invalidate because project 7006c0f4463f4db0b9e6e481602439e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:31:09.387759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-01f54ec0-6f48-4db4-9f7d-2f109900ffff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:09.388673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-01f54ec0-6f48-4db4-9f7d-2f109900ffff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:09.390811 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3609/7254] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:31:09 2022] DELETE /identity/v3/projects/7006c0f4463f4db0b9e6e481602439e1 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:09.395449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4df9d-6d9e-4be5-b7ea-7e7085bed881 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:09.407677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4df9d-6d9e-4be5-b7ea-7e7085bed881 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:09.538406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d389d7a-342f-4d06-9025-3678359e36c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:09.550239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d389d7a-342f-4d06-9025-3678359e36c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:09.550239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d389d7a-342f-4d06-9025-3678359e36c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:09.550239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d389d7a-342f-4d06-9025-3678359e36c7 admin admin] PATH_INFO: `/v3/users/18624e9a2fda48618d1b339d7749b41a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:09.550239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d389d7a-342f-4d06-9025-3678359e36c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18624e9a2fda48618d1b339d7749b41a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:09.550239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d389d7a-342f-4d06-9025-3678359e36c7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:09.573460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:09.788986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7d389d7a-342f-4d06-9025-3678359e36c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18624e9a2fda48618d1b339d7749b41a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:09.799053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7d389d7a-342f-4d06-9025-3678359e36c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18624e9a2fda48618d1b339d7749b41a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:09.801066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7d389d7a-342f-4d06-9025-3678359e36c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18624e9a2fda48618d1b339d7749b41a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:09.802223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7d389d7a-342f-4d06-9025-3678359e36c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18624e9a2fda48618d1b339d7749b41a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:09.897538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7d389d7a-342f-4d06-9025-3678359e36c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18624e9a2fda48618d1b339d7749b41a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:09.925356 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3646/7255] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:31:09 2022] DELETE /identity/v3/users/18624e9a2fda48618d1b339d7749b41a => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:09.935290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a1e8e-1568-408c-b187-8f83d62eb636 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:09.935290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a1e8e-1568-408c-b187-8f83d62eb636 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:09.935290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a1e8e-1568-408c-b187-8f83d62eb636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:09.984471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7e8a1e8e-1568-408c-b187-8f83d62eb636 None None] MFA Rules not processed for user `c93401a279c9403ca62627e829082968`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:09.986135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7e8a1e8e-1568-408c-b187-8f83d62eb636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:10.136269 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3610/7256] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:10.140544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d389d7a-342f-4d06-9025-3678359e36c7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:10.142180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7d389d7a-342f-4d06-9025-3678359e36c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:10.239762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d389d7a-342f-4d06-9025-3678359e36c7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:10.244948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463bc2b5-a27c-41e9-b1f9-5a3ba61d7cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:10.245666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bc2b5-a27c-41e9-b1f9-5a3ba61d7cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:10.246002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bc2b5-a27c-41e9-b1f9-5a3ba61d7cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:10.246300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463bc2b5-a27c-41e9-b1f9-5a3ba61d7cb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:10.247075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463bc2b5-a27c-41e9-b1f9-5a3ba61d7cb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:10.248949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463bc2b5-a27c-41e9-b1f9-5a3ba61d7cb6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:10.389349 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3647/7257] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:31:10 2022] POST /identity/v3/projects => generated 381 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:31:10.390992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a1e8e-1568-408c-b187-8f83d62eb636 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:10.403793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a1e8e-1568-408c-b187-8f83d62eb636 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:10.459838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463bc2b5-a27c-41e9-b1f9-5a3ba61d7cb6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:10.462912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256ee72-2656-40c9-b62f-ecbd8eb26932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:10.463036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-463bc2b5-a27c-41e9-b1f9-5a3ba61d7cb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:10.463115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256ee72-2656-40c9-b62f-ecbd8eb26932 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:10.463115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256ee72-2656-40c9-b62f-ecbd8eb26932 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:10.463257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256ee72-2656-40c9-b62f-ecbd8eb26932 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:10.463953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256ee72-2656-40c9-b62f-ecbd8eb26932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:10.466200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256ee72-2656-40c9-b62f-ecbd8eb26932 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:10.479922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463bc2b5-a27c-41e9-b1f9-5a3ba61d7cb6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:10.486213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efbb3fa-db82-480f-b09d-f90894436ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:10.487093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbb3fa-db82-480f-b09d-f90894436ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:10.487587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbb3fa-db82-480f-b09d-f90894436ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:10.487873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efbb3fa-db82-480f-b09d-f90894436ec2 admin admin] PATH_INFO: `/v3/projects/203c2c8c4e7c4197bf55fda8aa5d8b5c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:10.489403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efbb3fa-db82-480f-b09d-f90894436ec2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=203c2c8c4e7c4197bf55fda8aa5d8b5c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:10.491626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efbb3fa-db82-480f-b09d-f90894436ec2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:10.543539 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3611/7258] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:31:10 2022] POST /identity/v3/users => generated 297 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:31:10.546430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256ee72-2656-40c9-b62f-ecbd8eb26932 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:10.558533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256ee72-2656-40c9-b62f-ecbd8eb26932 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:10.563552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02e4fe-c8b6-4268-9c87-0b655f29f19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:10.564283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02e4fe-c8b6-4268-9c87-0b655f29f19d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:10.564616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02e4fe-c8b6-4268-9c87-0b655f29f19d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:10.564944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a02e4fe-c8b6-4268-9c87-0b655f29f19d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:10.566163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a02e4fe-c8b6-4268-9c87-0b655f29f19d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:10.567778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a02e4fe-c8b6-4268-9c87-0b655f29f19d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:10.579138 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3612/7259] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:31:10.583834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02e4fe-c8b6-4268-9c87-0b655f29f19d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:10.593917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a02e4fe-c8b6-4268-9c87-0b655f29f19d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:10.599305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d67a9b-08d6-4537-b5ac-ae6e1106239f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:10.600179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d67a9b-08d6-4537-b5ac-ae6e1106239f admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:10.600498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d67a9b-08d6-4537-b5ac-ae6e1106239f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:10.600733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d67a9b-08d6-4537-b5ac-ae6e1106239f admin admin] PATH_INFO: `/v3/projects/445b6918f25d420cb182ff4a8f1ef2d7/users/af49e980f06741acb92a7ef61c6f1034/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:10.616709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d67a9b-08d6-4537-b5ac-ae6e1106239f admin admin] RBAC: Authorizing `identity:create_grant(project_id=445b6918f25d420cb182ff4a8f1ef2d7, user_id=af49e980f06741acb92a7ef61c6f1034, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:10.619490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d67a9b-08d6-4537-b5ac-ae6e1106239f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:10.709599 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3613/7260] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:31:10 2022] PUT /identity/v3/projects/445b6918f25d420cb182ff4a8f1ef2d7/users/af49e980f06741acb92a7ef61c6f1034/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:31:10.714600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d67a9b-08d6-4537-b5ac-ae6e1106239f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:10.724983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6efbb3fa-db82-480f-b09d-f90894436ec2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '203c2c8c4e7c4197bf55fda8aa5d8b5c', 'request_id': 'req-6efbb3fa-db82-480f-b09d-f90894436ec2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:10.731124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6efbb3fa-db82-480f-b09d-f90894436ec2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '203c2c8c4e7c4197bf55fda8aa5d8b5c', 'request_id': 'req-6efbb3fa-db82-480f-b09d-f90894436ec2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:10.793539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d67a9b-08d6-4537-b5ac-ae6e1106239f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:10.810379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6efbb3fa-db82-480f-b09d-f90894436ec2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '203c2c8c4e7c4197bf55fda8aa5d8b5c', 'request_id': 'req-6efbb3fa-db82-480f-b09d-f90894436ec2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:10.828580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6efbb3fa-db82-480f-b09d-f90894436ec2 admin admin] The token cache is being invalidate because project 203c2c8c4e7c4197bf55fda8aa5d8b5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:31:10.828906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6efbb3fa-db82-480f-b09d-f90894436ec2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:10.829585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6efbb3fa-db82-480f-b09d-f90894436ec2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:10.831666 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3648/7261] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:31:10 2022] DELETE /identity/v3/projects/203c2c8c4e7c4197bf55fda8aa5d8b5c => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:10.836207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830358f8-fede-4f61-bd5c-0d5d19480555 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:10.836509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830358f8-fede-4f61-bd5c-0d5d19480555 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:10.837003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830358f8-fede-4f61-bd5c-0d5d19480555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:10.860016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-830358f8-fede-4f61-bd5c-0d5d19480555 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:10.861511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-830358f8-fede-4f61-bd5c-0d5d19480555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:10.862406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b523e88-7323-495b-9add-f5d035de5698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:10.863125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b523e88-7323-495b-9add-f5d035de5698 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:10.863408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b523e88-7323-495b-9add-f5d035de5698 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:10.863693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b523e88-7323-495b-9add-f5d035de5698 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:10.864506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b523e88-7323-495b-9add-f5d035de5698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:10.866167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b523e88-7323-495b-9add-f5d035de5698 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:10.870697 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3649/7262] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:31:10.876978 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3614/7263] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:31:10.878653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7033dfa-c2ca-435d-ab78-e5dd705d4ade None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:10.878948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7033dfa-c2ca-435d-ab78-e5dd705d4ade None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:10.879234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7033dfa-c2ca-435d-ab78-e5dd705d4ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:10.881387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b523e88-7323-495b-9add-f5d035de5698 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:10.884330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8b523e88-7323-495b-9add-f5d035de5698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:10.903925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f7033dfa-c2ca-435d-ab78-e5dd705d4ade None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:10.906296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b523e88-7323-495b-9add-f5d035de5698 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:10.912791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fc091-3169-42ac-acea-adcb95b45ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:10.914502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fc091-3169-42ac-acea-adcb95b45ebb admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:10.914960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fc091-3169-42ac-acea-adcb95b45ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:10.915259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fc091-3169-42ac-acea-adcb95b45ebb admin admin] PATH_INFO: `/v3/projects/445b6918f25d420cb182ff4a8f1ef2d7/users/af49e980f06741acb92a7ef61c6f1034/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:10.917376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fc091-3169-42ac-acea-adcb95b45ebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=445b6918f25d420cb182ff4a8f1ef2d7, user_id=af49e980f06741acb92a7ef61c6f1034, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:10.919737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f7033dfa-c2ca-435d-ab78-e5dd705d4ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:10.920132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fc091-3169-42ac-acea-adcb95b45ebb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:10.929774 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3650/7264] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:31:10.934251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7033dfa-c2ca-435d-ab78-e5dd705d4ade None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:10.937340 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3615/7265] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:31:10 2022] PUT /identity/v3/projects/445b6918f25d420cb182ff4a8f1ef2d7/users/af49e980f06741acb92a7ef61c6f1034/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:10.944883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7033dfa-c2ca-435d-ab78-e5dd705d4ade None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:11.014676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196f2e8-590b-46d6-91a9-6c23dfd06b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:11.015374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196f2e8-590b-46d6-91a9-6c23dfd06b85 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:11.015657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196f2e8-590b-46d6-91a9-6c23dfd06b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:11.015940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196f2e8-590b-46d6-91a9-6c23dfd06b85 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:11.016684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196f2e8-590b-46d6-91a9-6c23dfd06b85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:31:11.016994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196f2e8-590b-46d6-91a9-6c23dfd06b85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:11.018951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196f2e8-590b-46d6-91a9-6c23dfd06b85 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:11.031425 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3651/7266] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:31:11.037355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fc091-3169-42ac-acea-adcb95b45ebb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:11.060350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fc091-3169-42ac-acea-adcb95b45ebb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:11.064649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7311d-e2a6-4662-b997-10c96ec7c66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:11.065528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7311d-e2a6-4662-b997-10c96ec7c66a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:11.072671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7311d-e2a6-4662-b997-10c96ec7c66a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:11.073074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7311d-e2a6-4662-b997-10c96ec7c66a admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:11.073933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7311d-e2a6-4662-b997-10c96ec7c66a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:11.076131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7311d-e2a6-4662-b997-10c96ec7c66a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:11.241476 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3616/7267] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:31:11 2022] POST /identity/v3/projects => generated 387 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:31:11.242233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196f2e8-590b-46d6-91a9-6c23dfd06b85 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:11.273667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196f2e8-590b-46d6-91a9-6c23dfd06b85 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:11.370969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f411b-0ede-478d-b0d3-3566ade21aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:11.371692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f411b-0ede-478d-b0d3-3566ade21aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:11.371978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f411b-0ede-478d-b0d3-3566ade21aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:11.372256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f411b-0ede-478d-b0d3-3566ade21aa3 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:11.373090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f411b-0ede-478d-b0d3-3566ade21aa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:11.375175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f411b-0ede-478d-b0d3-3566ade21aa3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:11.510152 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3652/7268] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:31:11 2022] POST /identity/v3/users => generated 294 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:31:11.512165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7311d-e2a6-4662-b997-10c96ec7c66a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:11.524769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7311d-e2a6-4662-b997-10c96ec7c66a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:11.534696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e80f8e-09f3-4fb0-94ea-3f471ac9fd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:11.535927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e80f8e-09f3-4fb0-94ea-3f471ac9fd60 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:11.536217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e80f8e-09f3-4fb0-94ea-3f471ac9fd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:11.536930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e80f8e-09f3-4fb0-94ea-3f471ac9fd60 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:11.538836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e80f8e-09f3-4fb0-94ea-3f471ac9fd60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:11.541769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e80f8e-09f3-4fb0-94ea-3f471ac9fd60 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:11.553448 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3617/7269] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:31:11.558435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f411b-0ede-478d-b0d3-3566ade21aa3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:11.570662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f411b-0ede-478d-b0d3-3566ade21aa3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:11.576401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db2637-a452-4ee4-bb1b-ae898a7ceafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:11.577362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db2637-a452-4ee4-bb1b-ae898a7ceafe admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:11.577907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db2637-a452-4ee4-bb1b-ae898a7ceafe admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:11.578409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db2637-a452-4ee4-bb1b-ae898a7ceafe admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:11.579379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db2637-a452-4ee4-bb1b-ae898a7ceafe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:11.589068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db2637-a452-4ee4-bb1b-ae898a7ceafe admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:11.604673 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3653/7270] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:31:11.606548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e80f8e-09f3-4fb0-94ea-3f471ac9fd60 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:11.617488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e80f8e-09f3-4fb0-94ea-3f471ac9fd60 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:11.622882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e0989-d463-4d15-a152-bd8dd2f9495f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:11.623791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e0989-d463-4d15-a152-bd8dd2f9495f admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:11.624076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e0989-d463-4d15-a152-bd8dd2f9495f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:11.624357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e0989-d463-4d15-a152-bd8dd2f9495f admin admin] PATH_INFO: `/v3/projects/4f716173816046f5b9b33388eb4dc547/users/faab3cb5052c4e6284cc6ffb6e80cd3c/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:11.641291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e0989-d463-4d15-a152-bd8dd2f9495f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f716173816046f5b9b33388eb4dc547, user_id=faab3cb5052c4e6284cc6ffb6e80cd3c, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:11.645068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e0989-d463-4d15-a152-bd8dd2f9495f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:11.727540 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3618/7271] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:31:11 2022] PUT /identity/v3/projects/4f716173816046f5b9b33388eb4dc547/users/faab3cb5052c4e6284cc6ffb6e80cd3c/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:11.738942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e27628-58c3-4b69-8c13-ad7749e1b759 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:11.739132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e27628-58c3-4b69-8c13-ad7749e1b759 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:11.739276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e27628-58c3-4b69-8c13-ad7749e1b759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:11.772872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f6e27628-58c3-4b69-8c13-ad7749e1b759 None None] MFA Rules not processed for user `faab3cb5052c4e6284cc6ffb6e80cd3c`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:11.774163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f6e27628-58c3-4b69-8c13-ad7749e1b759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:11.871873 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3654/7272] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:31:11.876632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e0989-d463-4d15-a152-bd8dd2f9495f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:12.012531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e0989-d463-4d15-a152-bd8dd2f9495f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:12.020765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a3af8-2d63-4d2a-be50-0bb352619403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:12.021618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a3af8-2d63-4d2a-be50-0bb352619403 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:12.022030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a3af8-2d63-4d2a-be50-0bb352619403 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:12.022386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667a3af8-2d63-4d2a-be50-0bb352619403 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:12.024746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a3af8-2d63-4d2a-be50-0bb352619403 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:12.026991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667a3af8-2d63-4d2a-be50-0bb352619403 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:12.193435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:12.210634 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3619/7273] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:31:11 2022] POST /identity/v3/projects => generated 385 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:31:12.212131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e27628-58c3-4b69-8c13-ad7749e1b759 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:12.264188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e27628-58c3-4b69-8c13-ad7749e1b759 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:12.404573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25984071-e439-45d9-b064-12a7b9ba9a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:12.405362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25984071-e439-45d9-b064-12a7b9ba9a26 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:12.405712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25984071-e439-45d9-b064-12a7b9ba9a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:12.406118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25984071-e439-45d9-b064-12a7b9ba9a26 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:12.407094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25984071-e439-45d9-b064-12a7b9ba9a26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:12.409983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25984071-e439-45d9-b064-12a7b9ba9a26 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:12.650405 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3655/7274] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:31:12 2022] POST /identity/v3/users => generated 299 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:31:12.653395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a3af8-2d63-4d2a-be50-0bb352619403 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:12.671601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667a3af8-2d63-4d2a-be50-0bb352619403 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:12.687554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d095d-d6b0-481c-a8f8-1c14ca4358fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:12.688432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d095d-d6b0-481c-a8f8-1c14ca4358fa admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:12.688822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d095d-d6b0-481c-a8f8-1c14ca4358fa admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:12.689206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d095d-d6b0-481c-a8f8-1c14ca4358fa admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:12.690879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d095d-d6b0-481c-a8f8-1c14ca4358fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:12.692637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644d095d-d6b0-481c-a8f8-1c14ca4358fa admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:12.720507 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3620/7275] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:31:12.725490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25984071-e439-45d9-b064-12a7b9ba9a26 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:12.739204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25984071-e439-45d9-b064-12a7b9ba9a26 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:12.754897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6694c20-f2c5-48fd-839d-01ed41b05a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:12.760314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6694c20-f2c5-48fd-839d-01ed41b05a55 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:12.760314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6694c20-f2c5-48fd-839d-01ed41b05a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:12.760314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6694c20-f2c5-48fd-839d-01ed41b05a55 admin admin] PATH_INFO: `/v3/projects/da3ba581cfca41ca947fad46794677c5/users/427fa89aceb94de5a30430f870522f14/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:12.789823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6694c20-f2c5-48fd-839d-01ed41b05a55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da3ba581cfca41ca947fad46794677c5, user_id=427fa89aceb94de5a30430f870522f14, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:12.793570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6694c20-f2c5-48fd-839d-01ed41b05a55 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:12.907400 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3656/7276] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:31:12 2022] PUT /identity/v3/projects/da3ba581cfca41ca947fad46794677c5/users/427fa89aceb94de5a30430f870522f14/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:12.916241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d095d-d6b0-481c-a8f8-1c14ca4358fa admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:12.930984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d095d-d6b0-481c-a8f8-1c14ca4358fa admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:13.112647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd00b202-9879-452e-afce-42e11b3fa709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:13.119059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00b202-9879-452e-afce-42e11b3fa709 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:13.120293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00b202-9879-452e-afce-42e11b3fa709 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:13.120293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd00b202-9879-452e-afce-42e11b3fa709 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:13.122636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00b202-9879-452e-afce-42e11b3fa709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:13.131905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd00b202-9879-452e-afce-42e11b3fa709 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:13.152790 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3621/7277] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:31:13.154884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6694c20-f2c5-48fd-839d-01ed41b05a55 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:13.175524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6694c20-f2c5-48fd-839d-01ed41b05a55 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:13.183141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560facfc-2a1e-4f52-9db3-6fd6a4424f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:13.184797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560facfc-2a1e-4f52-9db3-6fd6a4424f54 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:13.185352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560facfc-2a1e-4f52-9db3-6fd6a4424f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:13.186223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560facfc-2a1e-4f52-9db3-6fd6a4424f54 admin admin] PATH_INFO: `/v3/projects/da3ba581cfca41ca947fad46794677c5/users/427fa89aceb94de5a30430f870522f14/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:13.188678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560facfc-2a1e-4f52-9db3-6fd6a4424f54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da3ba581cfca41ca947fad46794677c5, user_id=427fa89aceb94de5a30430f870522f14, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:13.191819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560facfc-2a1e-4f52-9db3-6fd6a4424f54 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:13.229673 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3657/7278] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:31:13 2022] PUT /identity/v3/projects/da3ba581cfca41ca947fad46794677c5/users/427fa89aceb94de5a30430f870522f14/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:13.241701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5472a0-5e84-4bc2-b7cf-a913c0b26caa None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:13.242260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5472a0-5e84-4bc2-b7cf-a913c0b26caa None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:13.242685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5472a0-5e84-4bc2-b7cf-a913c0b26caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:13.275805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ca5472a0-5e84-4bc2-b7cf-a913c0b26caa None None] MFA Rules not processed for user `427fa89aceb94de5a30430f870522f14`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:13.279625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ca5472a0-5e84-4bc2-b7cf-a913c0b26caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:13.375480 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3622/7279] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:13.388684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560facfc-2a1e-4f52-9db3-6fd6a4424f54 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:13.390815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-560facfc-2a1e-4f52-9db3-6fd6a4424f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:13.480684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560facfc-2a1e-4f52-9db3-6fd6a4424f54 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:13.485988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a162c6cb-4d29-4672-8776-cf552f5b8af4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:13.487054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162c6cb-4d29-4672-8776-cf552f5b8af4 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:13.487769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162c6cb-4d29-4672-8776-cf552f5b8af4 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:13.487769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a162c6cb-4d29-4672-8776-cf552f5b8af4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:13.506330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162c6cb-4d29-4672-8776-cf552f5b8af4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:13.508449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a162c6cb-4d29-4672-8776-cf552f5b8af4 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:13.522266 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3658/7280] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:31:13 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:31:13.720111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:13.860150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5472a0-5e84-4bc2-b7cf-a913c0b26caa None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:13.994586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5472a0-5e84-4bc2-b7cf-a913c0b26caa None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:14.002424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf676d-062c-4ceb-b4d4-9aa269b2a245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:14.003572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf676d-062c-4ceb-b4d4-9aa269b2a245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:14.004010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf676d-062c-4ceb-b4d4-9aa269b2a245 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:14.004311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf676d-062c-4ceb-b4d4-9aa269b2a245 admin admin] PATH_INFO: `/v3/users/faab3cb5052c4e6284cc6ffb6e80cd3c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:14.005765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf676d-062c-4ceb-b4d4-9aa269b2a245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faab3cb5052c4e6284cc6ffb6e80cd3c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:14.008082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf676d-062c-4ceb-b4d4-9aa269b2a245 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:14.410761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b8cf676d-062c-4ceb-b4d4-9aa269b2a245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faab3cb5052c4e6284cc6ffb6e80cd3c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:14.411669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b8cf676d-062c-4ceb-b4d4-9aa269b2a245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faab3cb5052c4e6284cc6ffb6e80cd3c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:14.440185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b8cf676d-062c-4ceb-b4d4-9aa269b2a245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faab3cb5052c4e6284cc6ffb6e80cd3c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:14.555340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b8cf676d-062c-4ceb-b4d4-9aa269b2a245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faab3cb5052c4e6284cc6ffb6e80cd3c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:14.556501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b8cf676d-062c-4ceb-b4d4-9aa269b2a245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faab3cb5052c4e6284cc6ffb6e80cd3c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:14.583573 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3623/7281] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:31:13 2022] DELETE /identity/v3/users/faab3cb5052c4e6284cc6ffb6e80cd3c => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:14.588273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f618041-68ca-413e-939c-5c9151e249f4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:14.588879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f618041-68ca-413e-939c-5c9151e249f4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:14.589082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f618041-68ca-413e-939c-5c9151e249f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:14.613609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4f618041-68ca-413e-939c-5c9151e249f4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:14.615503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4f618041-68ca-413e-939c-5c9151e249f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:14.702338 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3659/7282] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:31:14.715872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf676d-062c-4ceb-b4d4-9aa269b2a245 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:14.717475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b8cf676d-062c-4ceb-b4d4-9aa269b2a245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:14.804282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf676d-062c-4ceb-b4d4-9aa269b2a245 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:14.810032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17118d-b2c4-458a-ad3e-b604ae1b6801 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:14.810945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17118d-b2c4-458a-ad3e-b604ae1b6801 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:14.811085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17118d-b2c4-458a-ad3e-b604ae1b6801 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:14.811250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17118d-b2c4-458a-ad3e-b604ae1b6801 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:14.833875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17118d-b2c4-458a-ad3e-b604ae1b6801 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:14.835820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af17118d-b2c4-458a-ad3e-b604ae1b6801 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:14.851389 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3624/7283] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:31:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:31:16.012066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:16.027146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f618041-68ca-413e-939c-5c9151e249f4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:16.030023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4f618041-68ca-413e-939c-5c9151e249f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:16.070019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f618041-68ca-413e-939c-5c9151e249f4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:16.079226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a40043-ad50-4756-a1b6-a9b511af86f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:16.081092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a40043-ad50-4756-a1b6-a9b511af86f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:16.081382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a40043-ad50-4756-a1b6-a9b511af86f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:16.081664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a40043-ad50-4756-a1b6-a9b511af86f6 admin admin] PATH_INFO: `/v3/projects/4f716173816046f5b9b33388eb4dc547` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:16.083815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a40043-ad50-4756-a1b6-a9b511af86f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f716173816046f5b9b33388eb4dc547)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:16.086203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a40043-ad50-4756-a1b6-a9b511af86f6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:16.329464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-00a40043-ad50-4756-a1b6-a9b511af86f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f716173816046f5b9b33388eb4dc547', 'request_id': 'req-00a40043-ad50-4756-a1b6-a9b511af86f6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:16.341827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-00a40043-ad50-4756-a1b6-a9b511af86f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f716173816046f5b9b33388eb4dc547', 'request_id': 'req-00a40043-ad50-4756-a1b6-a9b511af86f6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:16.507260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-00a40043-ad50-4756-a1b6-a9b511af86f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f716173816046f5b9b33388eb4dc547', 'request_id': 'req-00a40043-ad50-4756-a1b6-a9b511af86f6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:16.507260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-00a40043-ad50-4756-a1b6-a9b511af86f6 admin admin] The token cache is being invalidate because project 4f716173816046f5b9b33388eb4dc547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:31:16.507260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-00a40043-ad50-4756-a1b6-a9b511af86f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:16.507260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-00a40043-ad50-4756-a1b6-a9b511af86f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:16.507260 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3660/7284] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:31:16 2022] DELETE /identity/v3/projects/4f716173816046f5b9b33388eb4dc547 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:16.507777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17118d-b2c4-458a-ad3e-b604ae1b6801 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:16.514748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17118d-b2c4-458a-ad3e-b604ae1b6801 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:16.574306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd761eb5-420c-47af-9829-ea799f1b3f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:16.575282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd761eb5-420c-47af-9829-ea799f1b3f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:16.575561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd761eb5-420c-47af-9829-ea799f1b3f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:16.575837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd761eb5-420c-47af-9829-ea799f1b3f68 admin admin] PATH_INFO: `/v3/users/427fa89aceb94de5a30430f870522f14` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:16.577085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd761eb5-420c-47af-9829-ea799f1b3f68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=427fa89aceb94de5a30430f870522f14)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:16.579118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd761eb5-420c-47af-9829-ea799f1b3f68 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:16.850923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fd761eb5-420c-47af-9829-ea799f1b3f68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '427fa89aceb94de5a30430f870522f14'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:16.852446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fd761eb5-420c-47af-9829-ea799f1b3f68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '427fa89aceb94de5a30430f870522f14'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:16.861071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fd761eb5-420c-47af-9829-ea799f1b3f68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '427fa89aceb94de5a30430f870522f14'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:16.912505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a40043-ad50-4756-a1b6-a9b511af86f6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:16.914088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-00a40043-ad50-4756-a1b6-a9b511af86f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:16.933348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a40043-ad50-4756-a1b6-a9b511af86f6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:16.939571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced94fb-3e46-4491-83cb-2be590fd93bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:16.941789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced94fb-3e46-4491-83cb-2be590fd93bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:16.943046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced94fb-3e46-4491-83cb-2be590fd93bb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:16.943570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced94fb-3e46-4491-83cb-2be590fd93bb admin admin] PATH_INFO: `/v3/users/c08e8a164d3248188ce965d1a9e04f03` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:16.945336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced94fb-3e46-4491-83cb-2be590fd93bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c08e8a164d3248188ce965d1a9e04f03)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:16.947956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced94fb-3e46-4491-83cb-2be590fd93bb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:16.953571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fd761eb5-420c-47af-9829-ea799f1b3f68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '427fa89aceb94de5a30430f870522f14'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:16.954509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fd761eb5-420c-47af-9829-ea799f1b3f68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '427fa89aceb94de5a30430f870522f14'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:16.984865 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3625/7285] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:31:16 2022] DELETE /identity/v3/users/427fa89aceb94de5a30430f870522f14 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:17.212896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dced94fb-3e46-4491-83cb-2be590fd93bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c08e8a164d3248188ce965d1a9e04f03'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:17.221652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dced94fb-3e46-4491-83cb-2be590fd93bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c08e8a164d3248188ce965d1a9e04f03'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:17.222611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dced94fb-3e46-4491-83cb-2be590fd93bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c08e8a164d3248188ce965d1a9e04f03'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:17.223865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dced94fb-3e46-4491-83cb-2be590fd93bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c08e8a164d3248188ce965d1a9e04f03'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:17.325634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dced94fb-3e46-4491-83cb-2be590fd93bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c08e8a164d3248188ce965d1a9e04f03'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:17.366687 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3661/7286] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:31:16 2022] DELETE /identity/v3/users/c08e8a164d3248188ce965d1a9e04f03 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:17.753798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd761eb5-420c-47af-9829-ea799f1b3f68 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:17.757245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced94fb-3e46-4491-83cb-2be590fd93bb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:17.758049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dced94fb-3e46-4491-83cb-2be590fd93bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:17.759366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fd761eb5-420c-47af-9829-ea799f1b3f68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:17.869490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced94fb-3e46-4491-83cb-2be590fd93bb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:17.875388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f179ca-c2b8-43bf-988a-801dd3e75d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:17.876582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f179ca-c2b8-43bf-988a-801dd3e75d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:17.877058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f179ca-c2b8-43bf-988a-801dd3e75d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:17.877504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f179ca-c2b8-43bf-988a-801dd3e75d21 admin admin] PATH_INFO: `/v3/projects/b98e2ebc3d3b4e7083b9c429628ce69f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:17.879430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f179ca-c2b8-43bf-988a-801dd3e75d21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b98e2ebc3d3b4e7083b9c429628ce69f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:17.881958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f179ca-c2b8-43bf-988a-801dd3e75d21 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:17.890989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd761eb5-420c-47af-9829-ea799f1b3f68 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:17.897051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35192e-56aa-45cb-a3fc-55bff11db591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:17.897972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35192e-56aa-45cb-a3fc-55bff11db591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:17.898439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35192e-56aa-45cb-a3fc-55bff11db591 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:17.899216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35192e-56aa-45cb-a3fc-55bff11db591 admin admin] PATH_INFO: `/v3/projects/da3ba581cfca41ca947fad46794677c5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:17.900767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35192e-56aa-45cb-a3fc-55bff11db591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da3ba581cfca41ca947fad46794677c5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:17.903369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35192e-56aa-45cb-a3fc-55bff11db591 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:18.031855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c6f179ca-c2b8-43bf-988a-801dd3e75d21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b98e2ebc3d3b4e7083b9c429628ce69f', 'request_id': 'req-c6f179ca-c2b8-43bf-988a-801dd3e75d21'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:18.043200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c6f179ca-c2b8-43bf-988a-801dd3e75d21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b98e2ebc3d3b4e7083b9c429628ce69f', 'request_id': 'req-c6f179ca-c2b8-43bf-988a-801dd3e75d21'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:18.083130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8c35192e-56aa-45cb-a3fc-55bff11db591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da3ba581cfca41ca947fad46794677c5', 'request_id': 'req-8c35192e-56aa-45cb-a3fc-55bff11db591'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:18.108971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8c35192e-56aa-45cb-a3fc-55bff11db591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da3ba581cfca41ca947fad46794677c5', 'request_id': 'req-8c35192e-56aa-45cb-a3fc-55bff11db591'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:18.140567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c6f179ca-c2b8-43bf-988a-801dd3e75d21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b98e2ebc3d3b4e7083b9c429628ce69f', 'request_id': 'req-c6f179ca-c2b8-43bf-988a-801dd3e75d21'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:18.166173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c6f179ca-c2b8-43bf-988a-801dd3e75d21 admin admin] The token cache is being invalidate because project b98e2ebc3d3b4e7083b9c429628ce69f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:31:18.166716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c6f179ca-c2b8-43bf-988a-801dd3e75d21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:18.167537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c6f179ca-c2b8-43bf-988a-801dd3e75d21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:18.173237 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3662/7287] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:31:17 2022] DELETE /identity/v3/projects/b98e2ebc3d3b4e7083b9c429628ce69f => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:31:18.175328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b9505-76b3-469d-a203-bc86565e90e8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:18.175777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b9505-76b3-469d-a203-bc86565e90e8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:18.176205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4b9505-76b3-469d-a203-bc86565e90e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:18.202228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3e4b9505-76b3-469d-a203-bc86565e90e8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:18.203495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3e4b9505-76b3-469d-a203-bc86565e90e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:18.239748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8c35192e-56aa-45cb-a3fc-55bff11db591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da3ba581cfca41ca947fad46794677c5', 'request_id': 'req-8c35192e-56aa-45cb-a3fc-55bff11db591'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:18.253705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8c35192e-56aa-45cb-a3fc-55bff11db591 admin admin] The token cache is being invalidate because project da3ba581cfca41ca947fad46794677c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:31:18.254368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8c35192e-56aa-45cb-a3fc-55bff11db591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:18.255305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8c35192e-56aa-45cb-a3fc-55bff11db591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:18.261058 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3626/7288] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:31:17 2022] DELETE /identity/v3/projects/da3ba581cfca41ca947fad46794677c5 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:18.262759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ffc58-800f-44a4-bb3e-7667dd0d24a2 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:18.263238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ffc58-800f-44a4-bb3e-7667dd0d24a2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:18.263698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ffc58-800f-44a4-bb3e-7667dd0d24a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:18.269777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:18.295658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-824ffc58-800f-44a4-bb3e-7667dd0d24a2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:18.298517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-824ffc58-800f-44a4-bb3e-7667dd0d24a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:18.304405 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3663/7289] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:18.313586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4c825-c812-4c34-bf81-b62192792a4a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:18.315439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4c825-c812-4c34-bf81-b62192792a4a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:18.315920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa4c825-c812-4c34-bf81-b62192792a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:18.322299 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3627/7290] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:31:18.333950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19355443-03cf-4321-a31f-57d2fd16e477 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:18.334239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19355443-03cf-4321-a31f-57d2fd16e477 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:18.334470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19355443-03cf-4321-a31f-57d2fd16e477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:18.338948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2aa4c825-c812-4c34-bf81-b62192792a4a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:18.340178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2aa4c825-c812-4c34-bf81-b62192792a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:18.357445 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3664/7291] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:18.358706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4c825-c812-4c34-bf81-b62192792a4a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:18.365716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-19355443-03cf-4321-a31f-57d2fd16e477 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:18.367001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-19355443-03cf-4321-a31f-57d2fd16e477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:18.376221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa4c825-c812-4c34-bf81-b62192792a4a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:18.380067 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3628/7292] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:18.383403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f7afc-909c-4af2-9d09-2cdc134d8224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:18.384309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f7afc-909c-4af2-9d09-2cdc134d8224 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:18.384634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f7afc-909c-4af2-9d09-2cdc134d8224 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:18.384921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f7afc-909c-4af2-9d09-2cdc134d8224 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:18.385668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f7afc-909c-4af2-9d09-2cdc134d8224 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:31:18.386113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f7afc-909c-4af2-9d09-2cdc134d8224 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:18.387089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19355443-03cf-4321-a31f-57d2fd16e477 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:18.387742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f7afc-909c-4af2-9d09-2cdc134d8224 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:18.404102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19355443-03cf-4321-a31f-57d2fd16e477 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:18.404748 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3665/7293] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:31:18.409628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f7afc-909c-4af2-9d09-2cdc134d8224 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:18.419348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a927470-82ce-4456-bd05-d0658908b6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:18.420318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a927470-82ce-4456-bd05-d0658908b6be admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:18.422518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a927470-82ce-4456-bd05-d0658908b6be admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:18.422997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a927470-82ce-4456-bd05-d0658908b6be admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:18.423845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f7afc-909c-4af2-9d09-2cdc134d8224 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:18.424544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a927470-82ce-4456-bd05-d0658908b6be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:31:18.425162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a927470-82ce-4456-bd05-d0658908b6be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:18.427672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a927470-82ce-4456-bd05-d0658908b6be admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:18.429506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a3ce2-10f9-4625-a426-8f4a7ee2c8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:18.430395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a3ce2-10f9-4625-a426-8f4a7ee2c8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:18.430742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a3ce2-10f9-4625-a426-8f4a7ee2c8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:18.431100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a3ce2-10f9-4625-a426-8f4a7ee2c8b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:18.431903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a3ce2-10f9-4625-a426-8f4a7ee2c8b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:18.433931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a3ce2-10f9-4625-a426-8f4a7ee2c8b8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:18.443525 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3629/7294] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:31:18.465390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a927470-82ce-4456-bd05-d0658908b6be admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:18.482618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a927470-82ce-4456-bd05-d0658908b6be admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:18.490589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bf407-cd6e-4840-b85e-3efe29d6f89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:18.491864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bf407-cd6e-4840-b85e-3efe29d6f89d admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:18.492100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bf407-cd6e-4840-b85e-3efe29d6f89d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:18.492377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bf407-cd6e-4840-b85e-3efe29d6f89d admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:18.493593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bf407-cd6e-4840-b85e-3efe29d6f89d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:18.496610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bf407-cd6e-4840-b85e-3efe29d6f89d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:18.554556 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3666/7295] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:31:18 2022] POST /identity/v3/projects => generated 371 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:31:18.558912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a3ce2-10f9-4625-a426-8f4a7ee2c8b8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:18.637698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a3ce2-10f9-4625-a426-8f4a7ee2c8b8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:18.649963 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3630/7296] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:31:18 2022] POST /identity/v3/projects => generated 391 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:31:18.654585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bf407-cd6e-4840-b85e-3efe29d6f89d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:18.658778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed0806-b164-4d5e-a241-5e2ff8d21d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:18.660489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed0806-b164-4d5e-a241-5e2ff8d21d21 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:18.660780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed0806-b164-4d5e-a241-5e2ff8d21d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:18.661095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beed0806-b164-4d5e-a241-5e2ff8d21d21 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:18.666322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed0806-b164-4d5e-a241-5e2ff8d21d21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:18.671799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beed0806-b164-4d5e-a241-5e2ff8d21d21 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:18.744158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bf407-cd6e-4840-b85e-3efe29d6f89d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:18.749148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309f122-9f80-42bd-97a5-6b8447c152c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:18.749974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309f122-9f80-42bd-97a5-6b8447c152c8 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:18.750266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309f122-9f80-42bd-97a5-6b8447c152c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:18.750545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b309f122-9f80-42bd-97a5-6b8447c152c8 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:18.751249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309f122-9f80-42bd-97a5-6b8447c152c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:18.753801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b309f122-9f80-42bd-97a5-6b8447c152c8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:18.825502 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3667/7297] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:31:18 2022] POST /identity/v3/users => generated 286 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:31:18.836084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed0806-b164-4d5e-a241-5e2ff8d21d21 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:18.846865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beed0806-b164-4d5e-a241-5e2ff8d21d21 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:18.852237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c9c78-58f3-4824-b146-ac00413c82fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:18.853141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c9c78-58f3-4824-b146-ac00413c82fa admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:18.853499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c9c78-58f3-4824-b146-ac00413c82fa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:18.853787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3c9c78-58f3-4824-b146-ac00413c82fa admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:18.854714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c9c78-58f3-4824-b146-ac00413c82fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:18.856428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3c9c78-58f3-4824-b146-ac00413c82fa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:18.857166 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3631/7298] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:31:18 2022] POST /identity/v3/users => generated 296 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:31:18.861225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309f122-9f80-42bd-97a5-6b8447c152c8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:18.868791 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3668/7299] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:31:18.872809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c9c78-58f3-4824-b146-ac00413c82fa admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:18.876662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b309f122-9f80-42bd-97a5-6b8447c152c8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:18.883927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3c9c78-58f3-4824-b146-ac00413c82fa admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:18.884964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c1c63-c9be-4f29-8896-8113905edd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:18.885655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c1c63-c9be-4f29-8896-8113905edd15 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:18.885970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c1c63-c9be-4f29-8896-8113905edd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:18.886254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c1c63-c9be-4f29-8896-8113905edd15 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:18.886985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c1c63-c9be-4f29-8896-8113905edd15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:18.889529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c1c63-c9be-4f29-8896-8113905edd15 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:18.899311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda01dd8-4e92-4786-aaeb-f6eb8a3948bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:18.900395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda01dd8-4e92-4786-aaeb-f6eb8a3948bc admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:18.900650 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3632/7300] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:31:18.901290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda01dd8-4e92-4786-aaeb-f6eb8a3948bc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:18.901579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda01dd8-4e92-4786-aaeb-f6eb8a3948bc admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:18.902402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda01dd8-4e92-4786-aaeb-f6eb8a3948bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:18.904508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c1c63-c9be-4f29-8896-8113905edd15 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:18.904891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda01dd8-4e92-4786-aaeb-f6eb8a3948bc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:18.918485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c1c63-c9be-4f29-8896-8113905edd15 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:18.928815 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3669/7301] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:31:18.933862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c4bb1-1cdd-4b2f-9ce4-6ab250d61203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:18.934706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c4bb1-1cdd-4b2f-9ce4-6ab250d61203 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:18.935135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda01dd8-4e92-4786-aaeb-f6eb8a3948bc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:18.935250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c4bb1-1cdd-4b2f-9ce4-6ab250d61203 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:18.935699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c4bb1-1cdd-4b2f-9ce4-6ab250d61203 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:18.936882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c4bb1-1cdd-4b2f-9ce4-6ab250d61203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:18.938282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164c4bb1-1cdd-4b2f-9ce4-6ab250d61203 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:18.952321 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3633/7302] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:31:18.955932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda01dd8-4e92-4786-aaeb-f6eb8a3948bc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:18.957479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c4bb1-1cdd-4b2f-9ce4-6ab250d61203 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:18.969211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c4bb1-1cdd-4b2f-9ce4-6ab250d61203 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:18.969704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f835a-cf52-4b83-a4dc-47c02b2d74d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:18.971002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f835a-cf52-4b83-a4dc-47c02b2d74d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:18.971748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f835a-cf52-4b83-a4dc-47c02b2d74d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:18.972053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f835a-cf52-4b83-a4dc-47c02b2d74d8 admin admin] PATH_INFO: `/v3/projects/8c9a59b25d3d479caeafce7ad25cf423/users/d15d14a609054950959cb446223dad78/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:18.983573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f6cecb-a240-4a1e-98b1-ea3c3e1ed44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:18.984547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6cecb-a240-4a1e-98b1-ea3c3e1ed44d admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:18.984834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6cecb-a240-4a1e-98b1-ea3c3e1ed44d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:18.994851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f6cecb-a240-4a1e-98b1-ea3c3e1ed44d admin admin] PATH_INFO: `/v3/projects/5a8892908ee44862a6f48555703188b6/users/ab1c8bf8b24c4b1083e1e26840065a7a/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:18.996374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f835a-cf52-4b83-a4dc-47c02b2d74d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c9a59b25d3d479caeafce7ad25cf423, user_id=d15d14a609054950959cb446223dad78, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:18.999789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f835a-cf52-4b83-a4dc-47c02b2d74d8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:19.026260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6cecb-a240-4a1e-98b1-ea3c3e1ed44d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a8892908ee44862a6f48555703188b6, user_id=ab1c8bf8b24c4b1083e1e26840065a7a, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:19.029239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f6cecb-a240-4a1e-98b1-ea3c3e1ed44d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:19.177690 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3634/7303] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:31:18 2022] PUT /identity/v3/projects/5a8892908ee44862a6f48555703188b6/users/ab1c8bf8b24c4b1083e1e26840065a7a/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:19.177913 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3670/7304] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:31:18 2022] PUT /identity/v3/projects/8c9a59b25d3d479caeafce7ad25cf423/users/d15d14a609054950959cb446223dad78/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:31:19.179935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50110e91-1b8d-41fd-bb57-0bdd358b4b75 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:19.180228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50110e91-1b8d-41fd-bb57-0bdd358b4b75 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:19.180516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50110e91-1b8d-41fd-bb57-0bdd358b4b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:19.181496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1458a-1385-43af-83c1-7dcf4da8e952 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:19.181926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1458a-1385-43af-83c1-7dcf4da8e952 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:19.182279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1458a-1385-43af-83c1-7dcf4da8e952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:19.247172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-50110e91-1b8d-41fd-bb57-0bdd358b4b75 None None] MFA Rules not processed for user `ab1c8bf8b24c4b1083e1e26840065a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:19.248758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-50110e91-1b8d-41fd-bb57-0bdd358b4b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:19.255183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f6b1458a-1385-43af-83c1-7dcf4da8e952 None None] MFA Rules not processed for user `d15d14a609054950959cb446223dad78`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:19.256694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f6b1458a-1385-43af-83c1-7dcf4da8e952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:19.399495 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3635/7305] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:19.406307 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3671/7306] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:31:19.407240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50110e91-1b8d-41fd-bb57-0bdd358b4b75 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:19.408437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d89c0b-a661-4364-9eb2-01d391b31885 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:19.408729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d89c0b-a661-4364-9eb2-01d391b31885 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:19.409031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d89c0b-a661-4364-9eb2-01d391b31885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:19.454542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-06d89c0b-a661-4364-9eb2-01d391b31885 None None] MFA Rules not processed for user `d15d14a609054950959cb446223dad78`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:19.454542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-06d89c0b-a661-4364-9eb2-01d391b31885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:19.479350 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3672/7307] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:19.496474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d89c0b-a661-4364-9eb2-01d391b31885 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:19.496474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-06d89c0b-a661-4364-9eb2-01d391b31885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:19.572272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50110e91-1b8d-41fd-bb57-0bdd358b4b75 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:19.579027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb94ff-0d33-474a-803f-a6638b7378f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:19.579730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb94ff-0d33-474a-803f-a6638b7378f7 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:19.580017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb94ff-0d33-474a-803f-a6638b7378f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:19.580335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb94ff-0d33-474a-803f-a6638b7378f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:19.581282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb94ff-0d33-474a-803f-a6638b7378f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:19.586238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb94ff-0d33-474a-803f-a6638b7378f7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:19.667901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d89c0b-a661-4364-9eb2-01d391b31885 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:19.676978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a73b1-e6f3-464a-a069-129e7decc6ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:19.677759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a73b1-e6f3-464a-a069-129e7decc6ac service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:19.678145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a73b1-e6f3-464a-a069-129e7decc6ac service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:19.678828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a73b1-e6f3-464a-a069-129e7decc6ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:19.697198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a73b1-e6f3-464a-a069-129e7decc6ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:19.698847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a73b1-e6f3-464a-a069-129e7decc6ac service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:19.712386 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3673/7308] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:31:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:31:19.734390 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3636/7309] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:31:19 2022] POST /identity/v3/projects => generated 393 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:31:19.744236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a73b1-e6f3-464a-a069-129e7decc6ac service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:19.759016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a73b1-e6f3-464a-a069-129e7decc6ac service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:19.878926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85adc8-692e-472f-abe5-d2deee6d1cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:19.880253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85adc8-692e-472f-abe5-d2deee6d1cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:19.880620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85adc8-692e-472f-abe5-d2deee6d1cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:19.881511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85adc8-692e-472f-abe5-d2deee6d1cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:19.882285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb85adc8-692e-472f-abe5-d2deee6d1cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:19.885265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb85adc8-692e-472f-abe5-d2deee6d1cb7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:19.997232 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3674/7310] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:31:19 2022] POST /identity/v3/users => generated 303 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:31:20.002911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb94ff-0d33-474a-803f-a6638b7378f7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:20.072684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb94ff-0d33-474a-803f-a6638b7378f7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:20.089137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd66cb7-6dfa-4158-b594-bd8ced90216f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:20.090078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd66cb7-6dfa-4158-b594-bd8ced90216f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:20.090736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd66cb7-6dfa-4158-b594-bd8ced90216f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:20.094729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd66cb7-6dfa-4158-b594-bd8ced90216f admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:20.096672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd66cb7-6dfa-4158-b594-bd8ced90216f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:20.139200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd66cb7-6dfa-4158-b594-bd8ced90216f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:20.172773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:20.180764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85adc8-692e-472f-abe5-d2deee6d1cb7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:20.181392 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3637/7311] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:31:20.193809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85adc8-692e-472f-abe5-d2deee6d1cb7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:20.200200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2aedc-8c67-45c3-a6d5-d4b879a98215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:20.200591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2aedc-8c67-45c3-a6d5-d4b879a98215 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:20.201432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2aedc-8c67-45c3-a6d5-d4b879a98215 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:20.202124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a2aedc-8c67-45c3-a6d5-d4b879a98215 admin admin] PATH_INFO: `/v3/projects/365d61be2186463c8cf97b83812d3530/users/471273eb72014ec79f501a02271a227d/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:20.231891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2aedc-8c67-45c3-a6d5-d4b879a98215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=365d61be2186463c8cf97b83812d3530, user_id=471273eb72014ec79f501a02271a227d, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:20.234999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a2aedc-8c67-45c3-a6d5-d4b879a98215 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:20.478495 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3675/7312] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:31:20 2022] PUT /identity/v3/projects/365d61be2186463c8cf97b83812d3530/users/471273eb72014ec79f501a02271a227d/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:20.485891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd66cb7-6dfa-4158-b594-bd8ced90216f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:20.499014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd66cb7-6dfa-4158-b594-bd8ced90216f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:20.620886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78cb11-bf69-434a-8691-f6fc25565fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:20.627599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78cb11-bf69-434a-8691-f6fc25565fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:20.627599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78cb11-bf69-434a-8691-f6fc25565fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:20.627599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78cb11-bf69-434a-8691-f6fc25565fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:20.627599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78cb11-bf69-434a-8691-f6fc25565fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:20.632399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78cb11-bf69-434a-8691-f6fc25565fa6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:20.646793 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3638/7313] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:31:20.650865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2aedc-8c67-45c3-a6d5-d4b879a98215 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:20.668516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a2aedc-8c67-45c3-a6d5-d4b879a98215 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:20.680256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c8ea7-e248-4d9c-b5a4-108eedf25be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:20.681325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c8ea7-e248-4d9c-b5a4-108eedf25be9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:20.681690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c8ea7-e248-4d9c-b5a4-108eedf25be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:20.683376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2c8ea7-e248-4d9c-b5a4-108eedf25be9 admin admin] PATH_INFO: `/v3/projects/365d61be2186463c8cf97b83812d3530/users/471273eb72014ec79f501a02271a227d/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:20.686908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c8ea7-e248-4d9c-b5a4-108eedf25be9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=365d61be2186463c8cf97b83812d3530, user_id=471273eb72014ec79f501a02271a227d, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:20.691602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2c8ea7-e248-4d9c-b5a4-108eedf25be9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:20.718895 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3676/7314] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:31:20 2022] PUT /identity/v3/projects/365d61be2186463c8cf97b83812d3530/users/471273eb72014ec79f501a02271a227d/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:20.734615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74540fa0-a95b-49e5-a652-774b5f58698e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:20.735715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74540fa0-a95b-49e5-a652-774b5f58698e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:20.736213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74540fa0-a95b-49e5-a652-774b5f58698e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:20.788769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-74540fa0-a95b-49e5-a652-774b5f58698e None None] MFA Rules not processed for user `471273eb72014ec79f501a02271a227d`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:20.793065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-74540fa0-a95b-49e5-a652-774b5f58698e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:20.913073 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3639/7315] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:20.922819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c8ea7-e248-4d9c-b5a4-108eedf25be9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:20.928541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:20.928711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7f2c8ea7-e248-4d9c-b5a4-108eedf25be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:21.065215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2c8ea7-e248-4d9c-b5a4-108eedf25be9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:21.075384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58592e-7e69-477e-b613-de1be11b293c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:21.078502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58592e-7e69-477e-b613-de1be11b293c service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:21.079571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58592e-7e69-477e-b613-de1be11b293c service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:21.080136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58592e-7e69-477e-b613-de1be11b293c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:21.095411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58592e-7e69-477e-b613-de1be11b293c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:21.098210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58592e-7e69-477e-b613-de1be11b293c service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:21.100986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c785f2-ed61-473a-b933-e0607a461148 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:21.102669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c785f2-ed61-473a-b933-e0607a461148 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:21.103181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c785f2-ed61-473a-b933-e0607a461148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:21.110631 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3677/7316] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:31:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:31:21.142225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e6c785f2-ed61-473a-b933-e0607a461148 None None] MFA Rules not processed for user `af49e980f06741acb92a7ef61c6f1034`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:21.143651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e6c785f2-ed61-473a-b933-e0607a461148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:21.169349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58592e-7e69-477e-b613-de1be11b293c service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:21.171597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cf58592e-7e69-477e-b613-de1be11b293c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:21.195279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58592e-7e69-477e-b613-de1be11b293c service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:21.205779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd8147-ebff-48ef-a244-7b0109aa433b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:21.207439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd8147-ebff-48ef-a244-7b0109aa433b service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:21.208666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd8147-ebff-48ef-a244-7b0109aa433b service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:21.209075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd8147-ebff-48ef-a244-7b0109aa433b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:21.272901 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3640/7317] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:31:21.280901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c785f2-ed61-473a-b933-e0607a461148 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:21.284409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e6c785f2-ed61-473a-b933-e0607a461148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:21.328281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd8147-ebff-48ef-a244-7b0109aa433b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:21.328281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd8147-ebff-48ef-a244-7b0109aa433b service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:21.335443 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3678/7318] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:31:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:31:21.361514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c785f2-ed61-473a-b933-e0607a461148 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:21.365415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fc862-4857-4ac8-97bb-94d345f33c5a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:21.366112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fc862-4857-4ac8-97bb-94d345f33c5a service glance] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:21.366550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fc862-4857-4ac8-97bb-94d345f33c5a service glance] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:21.366848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fc862-4857-4ac8-97bb-94d345f33c5a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:21.368845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bd1fc862-4857-4ac8-97bb-94d345f33c5a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:21.468640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1fc862-4857-4ac8-97bb-94d345f33c5a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:21.470153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1fc862-4857-4ac8-97bb-94d345f33c5a service glance] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:21.482363 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3641/7319] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:31:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:31:21.597399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72323d19-8440-4df5-bceb-c9b80938bfe8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:21.597915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72323d19-8440-4df5-bceb-c9b80938bfe8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:21.598196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72323d19-8440-4df5-bceb-c9b80938bfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:21.627139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-72323d19-8440-4df5-bceb-c9b80938bfe8 None None] MFA Rules not processed for user `93f78df0f96c47ac97a200a4442db6da`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:21.628353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-72323d19-8440-4df5-bceb-c9b80938bfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:21.700962 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3679/7320] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:22.206110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:23.044394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:24.977777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:26.485599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:27.325557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:28.975707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:29.982310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:30.365140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:30.873858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:38.332276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fc862-4857-4ac8-97bb-94d345f33c5a service glance] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:38.351506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:38.442234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fc862-4857-4ac8-97bb-94d345f33c5a service glance] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:38.448638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cb948-ce5b-4e14-b463-6f191c2533db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:38.449970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cb948-ce5b-4e14-b463-6f191c2533db admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:38.450870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cb948-ce5b-4e14-b463-6f191c2533db admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:38.451233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cb948-ce5b-4e14-b463-6f191c2533db admin admin] PATH_INFO: `/v3/users/ab1c8bf8b24c4b1083e1e26840065a7a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:38.453633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268cb948-ce5b-4e14-b463-6f191c2533db admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab1c8bf8b24c4b1083e1e26840065a7a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:38.456253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268cb948-ce5b-4e14-b463-6f191c2533db admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:38.651280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-268cb948-ce5b-4e14-b463-6f191c2533db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab1c8bf8b24c4b1083e1e26840065a7a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:38.652505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-268cb948-ce5b-4e14-b463-6f191c2533db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab1c8bf8b24c4b1083e1e26840065a7a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:38.667020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-268cb948-ce5b-4e14-b463-6f191c2533db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab1c8bf8b24c4b1083e1e26840065a7a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:38.742107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-268cb948-ce5b-4e14-b463-6f191c2533db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab1c8bf8b24c4b1083e1e26840065a7a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:38.742768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-268cb948-ce5b-4e14-b463-6f191c2533db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab1c8bf8b24c4b1083e1e26840065a7a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:38.773401 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3642/7321] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:31:38 2022] DELETE /identity/v3/users/ab1c8bf8b24c4b1083e1e26840065a7a => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:38.778145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181d9f4-071b-4c56-9719-fb76cb8dd9e2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:38.778672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181d9f4-071b-4c56-9719-fb76cb8dd9e2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:38.778817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181d9f4-071b-4c56-9719-fb76cb8dd9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:38.797917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6181d9f4-071b-4c56-9719-fb76cb8dd9e2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:38.799233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6181d9f4-071b-4c56-9719-fb76cb8dd9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:38.880856 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3680/7322] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:38.891126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cb948-ce5b-4e14-b463-6f191c2533db admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:38.893297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-268cb948-ce5b-4e14-b463-6f191c2533db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:38.956885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268cb948-ce5b-4e14-b463-6f191c2533db admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:38.961028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac962e-9594-4fa5-a70b-b6ae300d46b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:38.961627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac962e-9594-4fa5-a70b-b6ae300d46b1 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:38.961962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac962e-9594-4fa5-a70b-b6ae300d46b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:38.962390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac962e-9594-4fa5-a70b-b6ae300d46b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:38.972174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac962e-9594-4fa5-a70b-b6ae300d46b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:38.973634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac962e-9594-4fa5-a70b-b6ae300d46b1 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:38.985775 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3643/7323] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:31:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:31:39.521045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181d9f4-071b-4c56-9719-fb76cb8dd9e2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:39.524478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6181d9f4-071b-4c56-9719-fb76cb8dd9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:39.537536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181d9f4-071b-4c56-9719-fb76cb8dd9e2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:39.543940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee241b6-a5e9-448f-bf35-4a8f0298d187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:39.544959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee241b6-a5e9-448f-bf35-4a8f0298d187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:39.545253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee241b6-a5e9-448f-bf35-4a8f0298d187 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:39.545537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee241b6-a5e9-448f-bf35-4a8f0298d187 admin admin] PATH_INFO: `/v3/projects/5a8892908ee44862a6f48555703188b6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:39.547365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee241b6-a5e9-448f-bf35-4a8f0298d187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a8892908ee44862a6f48555703188b6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:39.550244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee241b6-a5e9-448f-bf35-4a8f0298d187 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:39.665968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3ee241b6-a5e9-448f-bf35-4a8f0298d187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a8892908ee44862a6f48555703188b6', 'request_id': 'req-3ee241b6-a5e9-448f-bf35-4a8f0298d187'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:39.672210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3ee241b6-a5e9-448f-bf35-4a8f0298d187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a8892908ee44862a6f48555703188b6', 'request_id': 'req-3ee241b6-a5e9-448f-bf35-4a8f0298d187'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:39.747608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3ee241b6-a5e9-448f-bf35-4a8f0298d187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a8892908ee44862a6f48555703188b6', 'request_id': 'req-3ee241b6-a5e9-448f-bf35-4a8f0298d187'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:39.759648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3ee241b6-a5e9-448f-bf35-4a8f0298d187 admin admin] The token cache is being invalidate because project 5a8892908ee44862a6f48555703188b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:31:39.759938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3ee241b6-a5e9-448f-bf35-4a8f0298d187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:39.760574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3ee241b6-a5e9-448f-bf35-4a8f0298d187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:39.762201 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3681/7324] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:31:39 2022] DELETE /identity/v3/projects/5a8892908ee44862a6f48555703188b6 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:39.766458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac962e-9594-4fa5-a70b-b6ae300d46b1 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:39.777204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac962e-9594-4fa5-a70b-b6ae300d46b1 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:39.811648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:39.837545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7d4eb7-55ff-41c7-886c-85270a1bec8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:39.838464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d4eb7-55ff-41c7-886c-85270a1bec8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:39.839719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d4eb7-55ff-41c7-886c-85270a1bec8b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:39.839719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7d4eb7-55ff-41c7-886c-85270a1bec8b admin admin] PATH_INFO: `/v3/users/471273eb72014ec79f501a02271a227d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:39.840555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d4eb7-55ff-41c7-886c-85270a1bec8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=471273eb72014ec79f501a02271a227d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:39.842556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7d4eb7-55ff-41c7-886c-85270a1bec8b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:40.100399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1d7d4eb7-55ff-41c7-886c-85270a1bec8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '471273eb72014ec79f501a02271a227d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:40.103303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1d7d4eb7-55ff-41c7-886c-85270a1bec8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '471273eb72014ec79f501a02271a227d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:40.110492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1d7d4eb7-55ff-41c7-886c-85270a1bec8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '471273eb72014ec79f501a02271a227d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:40.211206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1d7d4eb7-55ff-41c7-886c-85270a1bec8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '471273eb72014ec79f501a02271a227d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:40.211827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1d7d4eb7-55ff-41c7-886c-85270a1bec8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '471273eb72014ec79f501a02271a227d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:40.234175 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3644/7325] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:31:39 2022] DELETE /identity/v3/users/471273eb72014ec79f501a02271a227d => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:40.737627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee241b6-a5e9-448f-bf35-4a8f0298d187 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:40.739753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3ee241b6-a5e9-448f-bf35-4a8f0298d187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:40.761638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee241b6-a5e9-448f-bf35-4a8f0298d187 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:40.771794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:40.824501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97155cf8-d913-4233-b4a6-c8c56bf1c8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:40.825336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97155cf8-d913-4233-b4a6-c8c56bf1c8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:40.825625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97155cf8-d913-4233-b4a6-c8c56bf1c8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:40.825961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97155cf8-d913-4233-b4a6-c8c56bf1c8fb admin admin] PATH_INFO: `/v3/projects/365d61be2186463c8cf97b83812d3530` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:40.827177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97155cf8-d913-4233-b4a6-c8c56bf1c8fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=365d61be2186463c8cf97b83812d3530)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:40.829104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97155cf8-d913-4233-b4a6-c8c56bf1c8fb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:40.952763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-97155cf8-d913-4233-b4a6-c8c56bf1c8fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '365d61be2186463c8cf97b83812d3530', 'request_id': 'req-97155cf8-d913-4233-b4a6-c8c56bf1c8fb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:40.958284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-97155cf8-d913-4233-b4a6-c8c56bf1c8fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '365d61be2186463c8cf97b83812d3530', 'request_id': 'req-97155cf8-d913-4233-b4a6-c8c56bf1c8fb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:41.085404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-97155cf8-d913-4233-b4a6-c8c56bf1c8fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '365d61be2186463c8cf97b83812d3530', 'request_id': 'req-97155cf8-d913-4233-b4a6-c8c56bf1c8fb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:41.097589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-97155cf8-d913-4233-b4a6-c8c56bf1c8fb admin admin] The token cache is being invalidate because project 365d61be2186463c8cf97b83812d3530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:31:41.097923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-97155cf8-d913-4233-b4a6-c8c56bf1c8fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:41.098605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-97155cf8-d913-4233-b4a6-c8c56bf1c8fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:31:41.101024 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3682/7326] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:31:40 2022] DELETE /identity/v3/projects/365d61be2186463c8cf97b83812d3530 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:41.106237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9b81a-2817-4db3-82b4-7a67f9107b10 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:41.106725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9b81a-2817-4db3-82b4-7a67f9107b10 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:41.106725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9b81a-2817-4db3-82b4-7a67f9107b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:41.124676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-82c9b81a-2817-4db3-82b4-7a67f9107b10 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:41.125911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-82c9b81a-2817-4db3-82b4-7a67f9107b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:41.191219 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3645/7327] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:41.199618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24857693-675c-4d98-a9c4-18d4e17d9eed None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:41.199893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24857693-675c-4d98-a9c4-18d4e17d9eed None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:41.201603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24857693-675c-4d98-a9c4-18d4e17d9eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:41.224158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-24857693-675c-4d98-a9c4-18d4e17d9eed None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:41.226297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-24857693-675c-4d98-a9c4-18d4e17d9eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:41.237634 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3683/7328] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:41.242173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9b81a-2817-4db3-82b4-7a67f9107b10 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:41.252086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9b81a-2817-4db3-82b4-7a67f9107b10 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:41.257442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d83edd-e536-4de5-8f9c-392cc6bb68e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:41.258036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d83edd-e536-4de5-8f9c-392cc6bb68e8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:41.258326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d83edd-e536-4de5-8f9c-392cc6bb68e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:41.258630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d83edd-e536-4de5-8f9c-392cc6bb68e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:41.259378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d83edd-e536-4de5-8f9c-392cc6bb68e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:31:41.259673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d83edd-e536-4de5-8f9c-392cc6bb68e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:41.262025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d83edd-e536-4de5-8f9c-392cc6bb68e8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:41.275533 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3646/7329] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:31:41.280158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24857693-675c-4d98-a9c4-18d4e17d9eed None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:41.290612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24857693-675c-4d98-a9c4-18d4e17d9eed None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:41.297744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fed42-7efa-4419-9908-928df12d5a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:41.298596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fed42-7efa-4419-9908-928df12d5a09 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:41.298889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fed42-7efa-4419-9908-928df12d5a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:41.299268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fed42-7efa-4419-9908-928df12d5a09 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:41.299858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fed42-7efa-4419-9908-928df12d5a09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:41.302143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fed42-7efa-4419-9908-928df12d5a09 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:41.446187 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3684/7330] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:31:41 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:31:41.450392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d83edd-e536-4de5-8f9c-392cc6bb68e8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:41.464105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d83edd-e536-4de5-8f9c-392cc6bb68e8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:41.503965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:41.520898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c5e66-d816-4566-896f-2dd51a47490f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:41.521692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c5e66-d816-4566-896f-2dd51a47490f admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:41.521899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c5e66-d816-4566-896f-2dd51a47490f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:41.522273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053c5e66-d816-4566-896f-2dd51a47490f admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:41.523555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c5e66-d816-4566-896f-2dd51a47490f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:41.525568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053c5e66-d816-4566-896f-2dd51a47490f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:41.608919 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3647/7331] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:31:41 2022] POST /identity/v3/users => generated 291 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:31:41.610403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fed42-7efa-4419-9908-928df12d5a09 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:41.621815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fed42-7efa-4419-9908-928df12d5a09 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:41.626504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363041e-8d31-41f2-86b6-44fa6bda0b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:41.627165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363041e-8d31-41f2-86b6-44fa6bda0b92 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:41.627452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363041e-8d31-41f2-86b6-44fa6bda0b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:41.627734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363041e-8d31-41f2-86b6-44fa6bda0b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:41.628475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363041e-8d31-41f2-86b6-44fa6bda0b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:41.630112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363041e-8d31-41f2-86b6-44fa6bda0b92 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:41.640608 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3685/7332] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:31:41.644578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c5e66-d816-4566-896f-2dd51a47490f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:41.654578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053c5e66-d816-4566-896f-2dd51a47490f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:41.659475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec731f-c924-477d-82fe-2a3f50bbb220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:41.660238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec731f-c924-477d-82fe-2a3f50bbb220 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:41.660523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec731f-c924-477d-82fe-2a3f50bbb220 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:41.660802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec731f-c924-477d-82fe-2a3f50bbb220 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:41.661565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec731f-c924-477d-82fe-2a3f50bbb220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:41.663844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec731f-c924-477d-82fe-2a3f50bbb220 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:41.674868 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3648/7333] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:31:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:31:41.680964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363041e-8d31-41f2-86b6-44fa6bda0b92 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:41.694971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363041e-8d31-41f2-86b6-44fa6bda0b92 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:41.699696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8de42-b635-48b2-8e25-027219d9a06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:41.700588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8de42-b635-48b2-8e25-027219d9a06b admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:41.700871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8de42-b635-48b2-8e25-027219d9a06b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:41.701151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8de42-b635-48b2-8e25-027219d9a06b admin admin] PATH_INFO: `/v3/projects/6ed82f578e214948a9311cbdbb55a027/users/8d033e99f9294e348b01c59a970931d5/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:41.716288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8de42-b635-48b2-8e25-027219d9a06b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ed82f578e214948a9311cbdbb55a027, user_id=8d033e99f9294e348b01c59a970931d5, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:41.719148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8de42-b635-48b2-8e25-027219d9a06b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:41.895353 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3686/7334] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:31:41 2022] PUT /identity/v3/projects/6ed82f578e214948a9311cbdbb55a027/users/8d033e99f9294e348b01c59a970931d5/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:31:41.900940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ab2d3-5bd4-4297-8c15-81ae9c5099ce None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:41.901169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ab2d3-5bd4-4297-8c15-81ae9c5099ce None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:41.901452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ab2d3-5bd4-4297-8c15-81ae9c5099ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:41.919988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c20ab2d3-5bd4-4297-8c15-81ae9c5099ce None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:41.921252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c20ab2d3-5bd4-4297-8c15-81ae9c5099ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:41.981790 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3649/7335] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:41.991298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8de42-b635-48b2-8e25-027219d9a06b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:41.992989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8cb8de42-b635-48b2-8e25-027219d9a06b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:42.039699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:42.056101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8de42-b635-48b2-8e25-027219d9a06b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:42.060241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1597f-510d-4e19-a3f7-45ab171e06fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:42.060903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1597f-510d-4e19-a3f7-45ab171e06fd service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:42.061196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1597f-510d-4e19-a3f7-45ab171e06fd service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:42.061480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1597f-510d-4e19-a3f7-45ab171e06fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:42.072600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1597f-510d-4e19-a3f7-45ab171e06fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:42.074087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1597f-510d-4e19-a3f7-45ab171e06fd service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:42.084208 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3687/7336] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:31:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:31:43.475016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b5fb6-d2dd-41d4-a940-bd41f045c8f4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:43.476140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b5fb6-d2dd-41d4-a940-bd41f045c8f4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:43.476140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b5fb6-d2dd-41d4-a940-bd41f045c8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:43.506913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a61b5fb6-d2dd-41d4-a940-bd41f045c8f4 None None] MFA Rules not processed for user `8d033e99f9294e348b01c59a970931d5`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:31:43.508268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a61b5fb6-d2dd-41d4-a940-bd41f045c8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:43.586211 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3650/7337] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:31:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:31:43.599089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1597f-510d-4e19-a3f7-45ab171e06fd service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:43.602238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-53f1597f-510d-4e19-a3f7-45ab171e06fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:43.661623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1597f-510d-4e19-a3f7-45ab171e06fd service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:43.665726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcdfd1-26f1-407f-99bf-a78992678cd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:43.666400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcdfd1-26f1-407f-99bf-a78992678cd9 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:43.666686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcdfd1-26f1-407f-99bf-a78992678cd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:43.666963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcdfd1-26f1-407f-99bf-a78992678cd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:43.675410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fcdfd1-26f1-407f-99bf-a78992678cd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:43.676836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fcdfd1-26f1-407f-99bf-a78992678cd9 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:43.686233 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3688/7338] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:31:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:31:53.583173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:53.682222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=34 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:59.216799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b5fb6-d2dd-41d4-a940-bd41f045c8f4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:59.220147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a61b5fb6-d2dd-41d4-a940-bd41f045c8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:59.298743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b5fb6-d2dd-41d4-a940-bd41f045c8f4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:59.303917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec317b99-b43b-4d04-b79a-aa88f8372bb3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:59.304570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec317b99-b43b-4d04-b79a-aa88f8372bb3 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:59.304885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec317b99-b43b-4d04-b79a-aa88f8372bb3 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:59.305170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec317b99-b43b-4d04-b79a-aa88f8372bb3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:59.307541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ec317b99-b43b-4d04-b79a-aa88f8372bb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:59.324083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec317b99-b43b-4d04-b79a-aa88f8372bb3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:59.325793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec317b99-b43b-4d04-b79a-aa88f8372bb3 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:31:59.343444 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3651/7339] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:31:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:31:59.711608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcdfd1-26f1-407f-99bf-a78992678cd9 service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:31:59.713557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-19fcdfd1-26f1-407f-99bf-a78992678cd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:31:59.728987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcdfd1-26f1-407f-99bf-a78992678cd9 service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:31:59.733567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb051da-a007-41bf-9095-205e7479d436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:31:59.734518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb051da-a007-41bf-9095-205e7479d436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:31:59.734804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb051da-a007-41bf-9095-205e7479d436 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:31:59.735089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb051da-a007-41bf-9095-205e7479d436 admin admin] PATH_INFO: `/v3/users/f3468a0b2d324603ab6c3080993bc0c8` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:31:59.736298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb051da-a007-41bf-9095-205e7479d436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3468a0b2d324603ab6c3080993bc0c8)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:31:59.738270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb051da-a007-41bf-9095-205e7479d436 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:00.095372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5cb051da-a007-41bf-9095-205e7479d436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3468a0b2d324603ab6c3080993bc0c8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:00.103527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5cb051da-a007-41bf-9095-205e7479d436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3468a0b2d324603ab6c3080993bc0c8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:00.104449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5cb051da-a007-41bf-9095-205e7479d436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3468a0b2d324603ab6c3080993bc0c8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:00.105067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5cb051da-a007-41bf-9095-205e7479d436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3468a0b2d324603ab6c3080993bc0c8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:00.217250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5cb051da-a007-41bf-9095-205e7479d436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3468a0b2d324603ab6c3080993bc0c8'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:00.245888 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3689/7340] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:31:59 2022] DELETE /identity/v3/users/f3468a0b2d324603ab6c3080993bc0c8 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:32:00.247168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0eae6-65ce-4a3d-b4be-e65c9e08ad80 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:00.255357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0eae6-65ce-4a3d-b4be-e65c9e08ad80 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:00.255357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0eae6-65ce-4a3d-b4be-e65c9e08ad80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:00.283528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-37c0eae6-65ce-4a3d-b4be-e65c9e08ad80 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:32:00.288650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-37c0eae6-65ce-4a3d-b4be-e65c9e08ad80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:00.396827 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3652/7341] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:32:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:32:00.404738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb051da-a007-41bf-9095-205e7479d436 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:00.406259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5cb051da-a007-41bf-9095-205e7479d436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:00.472776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb051da-a007-41bf-9095-205e7479d436 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:00.477373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e96f43-9b78-40a1-be75-dd2d6608065d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:00.478115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e96f43-9b78-40a1-be75-dd2d6608065d service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:00.478407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e96f43-9b78-40a1-be75-dd2d6608065d service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:00.478690 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e96f43-9b78-40a1-be75-dd2d6608065d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:00.488397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e96f43-9b78-40a1-be75-dd2d6608065d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:00.489866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e96f43-9b78-40a1-be75-dd2d6608065d service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:00.501410 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3690/7342] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:32:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:32:00.835107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0eae6-65ce-4a3d-b4be-e65c9e08ad80 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:00.836655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-37c0eae6-65ce-4a3d-b4be-e65c9e08ad80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:00.853233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0eae6-65ce-4a3d-b4be-e65c9e08ad80 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:00.858632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6414fa6-2b23-45e4-b0b1-2d98304bab91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:00.859497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6414fa6-2b23-45e4-b0b1-2d98304bab91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:00.859786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6414fa6-2b23-45e4-b0b1-2d98304bab91 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:00.860052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6414fa6-2b23-45e4-b0b1-2d98304bab91 admin admin] PATH_INFO: `/v3/projects/c3588eab36044e439158f876836ea514` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:00.862138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6414fa6-2b23-45e4-b0b1-2d98304bab91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3588eab36044e439158f876836ea514)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:00.864166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6414fa6-2b23-45e4-b0b1-2d98304bab91 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:01.035126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f6414fa6-2b23-45e4-b0b1-2d98304bab91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3588eab36044e439158f876836ea514', 'request_id': 'req-f6414fa6-2b23-45e4-b0b1-2d98304bab91'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:01.041713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f6414fa6-2b23-45e4-b0b1-2d98304bab91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3588eab36044e439158f876836ea514', 'request_id': 'req-f6414fa6-2b23-45e4-b0b1-2d98304bab91'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:01.122896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f6414fa6-2b23-45e4-b0b1-2d98304bab91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3588eab36044e439158f876836ea514', 'request_id': 'req-f6414fa6-2b23-45e4-b0b1-2d98304bab91'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:01.131677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f6414fa6-2b23-45e4-b0b1-2d98304bab91 admin admin] The token cache is being invalidate because project c3588eab36044e439158f876836ea514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:32:01.131987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f6414fa6-2b23-45e4-b0b1-2d98304bab91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:01.132621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f6414fa6-2b23-45e4-b0b1-2d98304bab91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:01.134331 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3653/7343] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:32:00 2022] DELETE /identity/v3/projects/c3588eab36044e439158f876836ea514 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:32:01.138273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e96f43-9b78-40a1-be75-dd2d6608065d service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:01.148397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e96f43-9b78-40a1-be75-dd2d6608065d service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:01.218842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a2935-bd9f-4116-ae54-89f65300e683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:01.220040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a2935-bd9f-4116-ae54-89f65300e683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:01.220495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a2935-bd9f-4116-ae54-89f65300e683 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:01.220959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a2935-bd9f-4116-ae54-89f65300e683 admin admin] PATH_INFO: `/v3/users/e2da25bb9fdc42fbbc3665d6618ecd18` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:01.222725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a2935-bd9f-4116-ae54-89f65300e683 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2da25bb9fdc42fbbc3665d6618ecd18)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:01.226209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a2935-bd9f-4116-ae54-89f65300e683 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:01.253460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:01.498090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e16a2935-bd9f-4116-ae54-89f65300e683 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2da25bb9fdc42fbbc3665d6618ecd18'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:01.504933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e16a2935-bd9f-4116-ae54-89f65300e683 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2da25bb9fdc42fbbc3665d6618ecd18'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:01.505595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e16a2935-bd9f-4116-ae54-89f65300e683 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2da25bb9fdc42fbbc3665d6618ecd18'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:01.506309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e16a2935-bd9f-4116-ae54-89f65300e683 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2da25bb9fdc42fbbc3665d6618ecd18'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:01.596974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e16a2935-bd9f-4116-ae54-89f65300e683 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2da25bb9fdc42fbbc3665d6618ecd18'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:01.616705 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3691/7344] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:32:01 2022] DELETE /identity/v3/users/e2da25bb9fdc42fbbc3665d6618ecd18 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:32:01.851294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6414fa6-2b23-45e4-b0b1-2d98304bab91 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:01.853387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f6414fa6-2b23-45e4-b0b1-2d98304bab91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:01.870422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6414fa6-2b23-45e4-b0b1-2d98304bab91 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:01.880461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:01.928248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de2db2f-de62-4149-9503-a741c205abdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:01.929049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2db2f-de62-4149-9503-a741c205abdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:01.929335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2db2f-de62-4149-9503-a741c205abdc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:01.929615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de2db2f-de62-4149-9503-a741c205abdc admin admin] PATH_INFO: `/v3/projects/7dae9dd6b0b54bf2954f3d8427e092ee` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:01.931433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2db2f-de62-4149-9503-a741c205abdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dae9dd6b0b54bf2954f3d8427e092ee)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:01.933364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de2db2f-de62-4149-9503-a741c205abdc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:02.034545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2de2db2f-de62-4149-9503-a741c205abdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dae9dd6b0b54bf2954f3d8427e092ee', 'request_id': 'req-2de2db2f-de62-4149-9503-a741c205abdc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:02.040080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2de2db2f-de62-4149-9503-a741c205abdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dae9dd6b0b54bf2954f3d8427e092ee', 'request_id': 'req-2de2db2f-de62-4149-9503-a741c205abdc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:02.084293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a2935-bd9f-4116-ae54-89f65300e683 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:02.084293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e16a2935-bd9f-4116-ae54-89f65300e683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:02.097741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a2935-bd9f-4116-ae54-89f65300e683 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:02.107366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2de2db2f-de62-4149-9503-a741c205abdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dae9dd6b0b54bf2954f3d8427e092ee', 'request_id': 'req-2de2db2f-de62-4149-9503-a741c205abdc'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:02.120184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2de2db2f-de62-4149-9503-a741c205abdc admin admin] The token cache is being invalidate because project 7dae9dd6b0b54bf2954f3d8427e092ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:32:02.120625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2de2db2f-de62-4149-9503-a741c205abdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:02.121353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2de2db2f-de62-4149-9503-a741c205abdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:02.122933 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3654/7345] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:32:01 2022] DELETE /identity/v3/projects/7dae9dd6b0b54bf2954f3d8427e092ee => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:32:02.158176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08e91b-3796-4fc5-b071-21c616de42c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:02.159100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08e91b-3796-4fc5-b071-21c616de42c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:02.159386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08e91b-3796-4fc5-b071-21c616de42c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:02.159666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08e91b-3796-4fc5-b071-21c616de42c3 admin admin] PATH_INFO: `/v3/users/d15d14a609054950959cb446223dad78` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:02.161198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08e91b-3796-4fc5-b071-21c616de42c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d15d14a609054950959cb446223dad78)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:02.163269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f08e91b-3796-4fc5-b071-21c616de42c3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:02.420705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0f08e91b-3796-4fc5-b071-21c616de42c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd15d14a609054950959cb446223dad78'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:02.443147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0f08e91b-3796-4fc5-b071-21c616de42c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd15d14a609054950959cb446223dad78'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:02.444033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0f08e91b-3796-4fc5-b071-21c616de42c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd15d14a609054950959cb446223dad78'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:02.444736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0f08e91b-3796-4fc5-b071-21c616de42c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd15d14a609054950959cb446223dad78'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:02.512747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0f08e91b-3796-4fc5-b071-21c616de42c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd15d14a609054950959cb446223dad78'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:02.536407 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3692/7346] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:32:02 2022] DELETE /identity/v3/users/d15d14a609054950959cb446223dad78 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:32:02.541366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d940e-f2a0-468e-923f-08a4bc757ea1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:02.541766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d940e-f2a0-468e-923f-08a4bc757ea1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:02.542144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d940e-f2a0-468e-923f-08a4bc757ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:02.561300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6a8d940e-f2a0-468e-923f-08a4bc757ea1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:32:02.562576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6a8d940e-f2a0-468e-923f-08a4bc757ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:02.627784 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3655/7347] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:32:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:32:02.637935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08e91b-3796-4fc5-b071-21c616de42c3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:02.639272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0f08e91b-3796-4fc5-b071-21c616de42c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:02.656185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:02.718417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f08e91b-3796-4fc5-b071-21c616de42c3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:02.728790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdef731-6312-4763-8233-151e04eb5f62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:02.729491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdef731-6312-4763-8233-151e04eb5f62 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:02.730474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdef731-6312-4763-8233-151e04eb5f62 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:02.730785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdef731-6312-4763-8233-151e04eb5f62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:02.741290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdef731-6312-4763-8233-151e04eb5f62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:02.742808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdef731-6312-4763-8233-151e04eb5f62 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:02.757877 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3693/7348] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:32:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:32:03.245823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d940e-f2a0-468e-923f-08a4bc757ea1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:03.247375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6a8d940e-f2a0-468e-923f-08a4bc757ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:03.260882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d940e-f2a0-468e-923f-08a4bc757ea1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:03.266206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77adcee-a8b3-4a05-a421-abaa73a94808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:03.267166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77adcee-a8b3-4a05-a421-abaa73a94808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:03.267517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77adcee-a8b3-4a05-a421-abaa73a94808 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:03.267862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77adcee-a8b3-4a05-a421-abaa73a94808 admin admin] PATH_INFO: `/v3/projects/8c9a59b25d3d479caeafce7ad25cf423` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:03.269148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77adcee-a8b3-4a05-a421-abaa73a94808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c9a59b25d3d479caeafce7ad25cf423)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:03.271492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77adcee-a8b3-4a05-a421-abaa73a94808 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:03.365533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:03.373553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c77adcee-a8b3-4a05-a421-abaa73a94808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c9a59b25d3d479caeafce7ad25cf423', 'request_id': 'req-c77adcee-a8b3-4a05-a421-abaa73a94808'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:03.379889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c77adcee-a8b3-4a05-a421-abaa73a94808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c9a59b25d3d479caeafce7ad25cf423', 'request_id': 'req-c77adcee-a8b3-4a05-a421-abaa73a94808'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:03.447185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c77adcee-a8b3-4a05-a421-abaa73a94808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c9a59b25d3d479caeafce7ad25cf423', 'request_id': 'req-c77adcee-a8b3-4a05-a421-abaa73a94808'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:03.462759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c77adcee-a8b3-4a05-a421-abaa73a94808 admin admin] The token cache is being invalidate because project 8c9a59b25d3d479caeafce7ad25cf423 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:32:03.463060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c77adcee-a8b3-4a05-a421-abaa73a94808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:03.463744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c77adcee-a8b3-4a05-a421-abaa73a94808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:03.466797 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3656/7349] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:32:03 2022] DELETE /identity/v3/projects/8c9a59b25d3d479caeafce7ad25cf423 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:32:03.471338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a3b8b-1bc5-4617-b993-c463ddbaa569 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:03.471857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a3b8b-1bc5-4617-b993-c463ddbaa569 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:03.471993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5a3b8b-1bc5-4617-b993-c463ddbaa569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:03.492680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3b5a3b8b-1bc5-4617-b993-c463ddbaa569 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:32:03.494514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3b5a3b8b-1bc5-4617-b993-c463ddbaa569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:03.558143 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3694/7350] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:32:03.566159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546769e-ab11-47d7-a6b5-73733407a610 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:03.566485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546769e-ab11-47d7-a6b5-73733407a610 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:03.566796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546769e-ab11-47d7-a6b5-73733407a610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:03.585993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a546769e-ab11-47d7-a6b5-73733407a610 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:32:03.587232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a546769e-ab11-47d7-a6b5-73733407a610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:03.595552 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3657/7351] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:32:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:32:03.599638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a3b8b-1bc5-4617-b993-c463ddbaa569 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:03.614326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5a3b8b-1bc5-4617-b993-c463ddbaa569 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:03.620107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864885e5-edf4-4386-a36f-541b42dbde34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:03.620810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864885e5-edf4-4386-a36f-541b42dbde34 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:03.621093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864885e5-edf4-4386-a36f-541b42dbde34 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:03.622260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864885e5-edf4-4386-a36f-541b42dbde34 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:03.622260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864885e5-edf4-4386-a36f-541b42dbde34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:32:03.622585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864885e5-edf4-4386-a36f-541b42dbde34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:03.624191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864885e5-edf4-4386-a36f-541b42dbde34 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:03.639680 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3695/7352] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:32:03.642733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546769e-ab11-47d7-a6b5-73733407a610 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:03.652885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546769e-ab11-47d7-a6b5-73733407a610 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:03.658387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f7ce4-5d2a-47fe-8b1d-f7af7c0b4d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:03.659048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f7ce4-5d2a-47fe-8b1d-f7af7c0b4d3f admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:03.659368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f7ce4-5d2a-47fe-8b1d-f7af7c0b4d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:03.659651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f7ce4-5d2a-47fe-8b1d-f7af7c0b4d3f admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:03.660380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f7ce4-5d2a-47fe-8b1d-f7af7c0b4d3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:03.662341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6f7ce4-5d2a-47fe-8b1d-f7af7c0b4d3f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:03.768857 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3658/7353] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:32:03 2022] POST /identity/v3/projects => generated 405 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:32:03.773002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864885e5-edf4-4386-a36f-541b42dbde34 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:03.783191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864885e5-edf4-4386-a36f-541b42dbde34 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:03.843597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a287989-b9a7-4283-ae95-9a6d6bf6f79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:03.844265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a287989-b9a7-4283-ae95-9a6d6bf6f79f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:03.844550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a287989-b9a7-4283-ae95-9a6d6bf6f79f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:03.844827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a287989-b9a7-4283-ae95-9a6d6bf6f79f admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:03.845513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a287989-b9a7-4283-ae95-9a6d6bf6f79f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:03.847541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a287989-b9a7-4283-ae95-9a6d6bf6f79f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:03.919266 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3696/7354] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:32:03 2022] POST /identity/v3/users => generated 303 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:32:03.923406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f7ce4-5d2a-47fe-8b1d-f7af7c0b4d3f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:03.935408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f7ce4-5d2a-47fe-8b1d-f7af7c0b4d3f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:03.940283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a19d1-7654-41a2-935b-33c08eda6484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:03.940944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a19d1-7654-41a2-935b-33c08eda6484 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:03.941230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a19d1-7654-41a2-935b-33c08eda6484 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:03.941512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a19d1-7654-41a2-935b-33c08eda6484 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:03.942295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a19d1-7654-41a2-935b-33c08eda6484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:03.943906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a19d1-7654-41a2-935b-33c08eda6484 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:03.955460 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3659/7355] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:32:03.959433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a287989-b9a7-4283-ae95-9a6d6bf6f79f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:03.969713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a287989-b9a7-4283-ae95-9a6d6bf6f79f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:03.974353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f28099-12d1-49da-ac3e-254f75fad17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:03.975004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f28099-12d1-49da-ac3e-254f75fad17c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:03.975312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f28099-12d1-49da-ac3e-254f75fad17c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:03.975589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f28099-12d1-49da-ac3e-254f75fad17c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:03.976339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f28099-12d1-49da-ac3e-254f75fad17c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:03.978481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f28099-12d1-49da-ac3e-254f75fad17c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:03.989306 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3697/7356] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:32:03.994663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a19d1-7654-41a2-935b-33c08eda6484 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:04.004860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a19d1-7654-41a2-935b-33c08eda6484 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:04.009917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea2607-35d8-4988-9181-9dcc63e9e6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:04.010811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea2607-35d8-4988-9181-9dcc63e9e6fb admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:04.011100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea2607-35d8-4988-9181-9dcc63e9e6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:04.011370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea2607-35d8-4988-9181-9dcc63e9e6fb admin admin] PATH_INFO: `/v3/projects/8a50a2e9cce04058b89adb71008e6397/users/1d2aebad2d24440fb9cfdb279afc9290/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:04.026025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea2607-35d8-4988-9181-9dcc63e9e6fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a50a2e9cce04058b89adb71008e6397, user_id=1d2aebad2d24440fb9cfdb279afc9290, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:04.028708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ea2607-35d8-4988-9181-9dcc63e9e6fb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:04.108108 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3660/7357] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:32:03 2022] PUT /identity/v3/projects/8a50a2e9cce04058b89adb71008e6397/users/1d2aebad2d24440fb9cfdb279afc9290/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:32:04.113456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e5cc0-e4dc-4802-9c5f-659cb56269d3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:04.113751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e5cc0-e4dc-4802-9c5f-659cb56269d3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:04.114059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e5cc0-e4dc-4802-9c5f-659cb56269d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:04.132447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ff0e5cc0-e4dc-4802-9c5f-659cb56269d3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:32:04.133662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ff0e5cc0-e4dc-4802-9c5f-659cb56269d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:04.150072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:04.191793 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3698/7358] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:32:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:32:04.201614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea2607-35d8-4988-9181-9dcc63e9e6fb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:04.203406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-43ea2607-35d8-4988-9181-9dcc63e9e6fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:04.264384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea2607-35d8-4988-9181-9dcc63e9e6fb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:04.268048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f06ce-ae7c-437b-9d65-ffbae44a0878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:04.268701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f06ce-ae7c-437b-9d65-ffbae44a0878 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:04.268992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f06ce-ae7c-437b-9d65-ffbae44a0878 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:04.269272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f06ce-ae7c-437b-9d65-ffbae44a0878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:04.279293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f06ce-ae7c-437b-9d65-ffbae44a0878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:04.280741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f06ce-ae7c-437b-9d65-ffbae44a0878 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:04.291057 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3661/7359] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:32:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:32:05.617043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6bdab-44b9-4e1f-82e0-2583399fd02c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:05.617550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6bdab-44b9-4e1f-82e0-2583399fd02c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:05.617649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6bdab-44b9-4e1f-82e0-2583399fd02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:05.648503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-eaf6bdab-44b9-4e1f-82e0-2583399fd02c None None] MFA Rules not processed for user `1d2aebad2d24440fb9cfdb279afc9290`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:32:05.650534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eaf6bdab-44b9-4e1f-82e0-2583399fd02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:05.732097 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3699/7360] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:32:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:32:05.735546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f06ce-ae7c-437b-9d65-ffbae44a0878 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:05.746122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f06ce-ae7c-437b-9d65-ffbae44a0878 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:05.750721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b362e6-04e5-405a-a01c-0828cf2cf22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:05.755601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b362e6-04e5-405a-a01c-0828cf2cf22f admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:05.755601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b362e6-04e5-405a-a01c-0828cf2cf22f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:05.755601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b362e6-04e5-405a-a01c-0828cf2cf22f admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:05.755601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b362e6-04e5-405a-a01c-0828cf2cf22f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:05.755601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b362e6-04e5-405a-a01c-0828cf2cf22f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:05.840432 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3662/7361] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:32:05 2022] POST /identity/v3/projects => generated 403 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:32:05.844534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6bdab-44b9-4e1f-82e0-2583399fd02c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:05.855844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6bdab-44b9-4e1f-82e0-2583399fd02c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:05.917042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ca26f-4209-4766-8d78-3f5a934fbf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:05.917724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ca26f-4209-4766-8d78-3f5a934fbf57 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:05.918118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ca26f-4209-4766-8d78-3f5a934fbf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:05.918435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ca26f-4209-4766-8d78-3f5a934fbf57 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:05.919148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ca26f-4209-4766-8d78-3f5a934fbf57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:05.921085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ca26f-4209-4766-8d78-3f5a934fbf57 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:06.033480 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3700/7362] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:32:05 2022] POST /identity/v3/users => generated 302 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:32:06.037357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b362e6-04e5-405a-a01c-0828cf2cf22f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:06.053952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b362e6-04e5-405a-a01c-0828cf2cf22f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:06.065350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9711f-de40-4b91-8da1-e287f98b2f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:06.065489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9711f-de40-4b91-8da1-e287f98b2f81 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:06.065698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9711f-de40-4b91-8da1-e287f98b2f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:06.066026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9711f-de40-4b91-8da1-e287f98b2f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:06.066776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9711f-de40-4b91-8da1-e287f98b2f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:06.068558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9711f-de40-4b91-8da1-e287f98b2f81 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:06.085880 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3663/7363] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:32:06.087340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ca26f-4209-4766-8d78-3f5a934fbf57 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:06.097988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ca26f-4209-4766-8d78-3f5a934fbf57 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:06.103209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a1d45-1045-4794-9004-0c13f53230ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:06.103863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a1d45-1045-4794-9004-0c13f53230ac admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:06.104149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a1d45-1045-4794-9004-0c13f53230ac admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:06.104426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a1d45-1045-4794-9004-0c13f53230ac admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:06.105182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a1d45-1045-4794-9004-0c13f53230ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:06.109002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a1d45-1045-4794-9004-0c13f53230ac admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:06.120237 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3701/7364] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:32:06.124759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9711f-de40-4b91-8da1-e287f98b2f81 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:06.136183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9711f-de40-4b91-8da1-e287f98b2f81 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:06.140929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02b57c6-157a-4028-90b5-74e7f09c894b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:06.141812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b57c6-157a-4028-90b5-74e7f09c894b admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:06.142202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b57c6-157a-4028-90b5-74e7f09c894b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:06.142483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02b57c6-157a-4028-90b5-74e7f09c894b admin admin] PATH_INFO: `/v3/projects/a63778cceaba4431b706a6de830ced40/users/cb35028198084d9c9eeb72b28e0e9117/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:06.157034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b57c6-157a-4028-90b5-74e7f09c894b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a63778cceaba4431b706a6de830ced40, user_id=cb35028198084d9c9eeb72b28e0e9117, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:06.159747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02b57c6-157a-4028-90b5-74e7f09c894b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:06.239617 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3664/7365] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:32:06 2022] PUT /identity/v3/projects/a63778cceaba4431b706a6de830ced40/users/cb35028198084d9c9eeb72b28e0e9117/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:32:07.582211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa35046-20f6-4ac1-9e5c-b77c146d1f00 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:07.582702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa35046-20f6-4ac1-9e5c-b77c146d1f00 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:07.582924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa35046-20f6-4ac1-9e5c-b77c146d1f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:07.620988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cfa35046-20f6-4ac1-9e5c-b77c146d1f00 None None] MFA Rules not processed for user `cb35028198084d9c9eeb72b28e0e9117`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:32:07.622304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cfa35046-20f6-4ac1-9e5c-b77c146d1f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:07.643905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:07.691603 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3702/7366] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:32:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:32:07.703264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8165a-db0a-43b6-af97-b03dec67c51f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:07.703814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8165a-db0a-43b6-af97-b03dec67c51f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:07.703814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8165a-db0a-43b6-af97-b03dec67c51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:07.721797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e9e8165a-db0a-43b6-af97-b03dec67c51f None None] MFA Rules not processed for user `1d2aebad2d24440fb9cfdb279afc9290`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:32:07.723093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e9e8165a-db0a-43b6-af97-b03dec67c51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:07.727201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:07.786104 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3665/7367] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:32:07 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:32:07.798274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa35046-20f6-4ac1-9e5c-b77c146d1f00 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:07.799655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cfa35046-20f6-4ac1-9e5c-b77c146d1f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:07.882810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa35046-20f6-4ac1-9e5c-b77c146d1f00 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:07.888547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e05040-29ee-4397-bc0b-eb658f1ccccd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:07.889212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e05040-29ee-4397-bc0b-eb658f1ccccd service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:07.889498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e05040-29ee-4397-bc0b-eb658f1ccccd service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:07.889781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e05040-29ee-4397-bc0b-eb658f1ccccd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:07.898052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e05040-29ee-4397-bc0b-eb658f1ccccd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:07.899656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e05040-29ee-4397-bc0b-eb658f1ccccd service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:07.908622 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3703/7368] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:32:07 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:32:17.838268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=21 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:17.904522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=52 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:22.242255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8165a-db0a-43b6-af97-b03dec67c51f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:22.258321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8165a-db0a-43b6-af97-b03dec67c51f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:22.264700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379401b-2f3d-438a-8401-9827a7d7e8b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:22.265419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379401b-2f3d-438a-8401-9827a7d7e8b7 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:22.265737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379401b-2f3d-438a-8401-9827a7d7e8b7 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:22.266042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379401b-2f3d-438a-8401-9827a7d7e8b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:22.274355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379401b-2f3d-438a-8401-9827a7d7e8b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:22.276590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379401b-2f3d-438a-8401-9827a7d7e8b7 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:22.285929 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3666/7369] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:32:22 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:32:27.880727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e05040-29ee-4397-bc0b-eb658f1ccccd service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:27.950714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e05040-29ee-4397-bc0b-eb658f1ccccd service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:27.955441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b4e0b-cf36-48ab-a1c0-509db82507c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:27.956320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b4e0b-cf36-48ab-a1c0-509db82507c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:27.956608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b4e0b-cf36-48ab-a1c0-509db82507c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:27.956890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902b4e0b-cf36-48ab-a1c0-509db82507c6 admin admin] PATH_INFO: `/v3/users/1d2aebad2d24440fb9cfdb279afc9290` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:27.958130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b4e0b-cf36-48ab-a1c0-509db82507c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d2aebad2d24440fb9cfdb279afc9290)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:27.960073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902b4e0b-cf36-48ab-a1c0-509db82507c6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:28.225553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-902b4e0b-cf36-48ab-a1c0-509db82507c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d2aebad2d24440fb9cfdb279afc9290'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:28.233435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-902b4e0b-cf36-48ab-a1c0-509db82507c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d2aebad2d24440fb9cfdb279afc9290'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:28.234136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-902b4e0b-cf36-48ab-a1c0-509db82507c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d2aebad2d24440fb9cfdb279afc9290'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:28.234792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-902b4e0b-cf36-48ab-a1c0-509db82507c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d2aebad2d24440fb9cfdb279afc9290'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:28.295945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-902b4e0b-cf36-48ab-a1c0-509db82507c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d2aebad2d24440fb9cfdb279afc9290'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:28.325292 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3704/7370] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:32:27 2022] DELETE /identity/v3/users/1d2aebad2d24440fb9cfdb279afc9290 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:32:28.594360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379401b-2f3d-438a-8401-9827a7d7e8b7 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:28.606410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379401b-2f3d-438a-8401-9827a7d7e8b7 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:28.672760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38fad2-32a5-4255-a854-6b5f814f0a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:28.673721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38fad2-32a5-4255-a854-6b5f814f0a62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:28.674042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38fad2-32a5-4255-a854-6b5f814f0a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:28.674322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38fad2-32a5-4255-a854-6b5f814f0a62 admin admin] PATH_INFO: `/v3/projects/8a50a2e9cce04058b89adb71008e6397` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:28.675705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38fad2-32a5-4255-a854-6b5f814f0a62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a50a2e9cce04058b89adb71008e6397)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:28.678131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38fad2-32a5-4255-a854-6b5f814f0a62 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:28.805346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dd38fad2-32a5-4255-a854-6b5f814f0a62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a50a2e9cce04058b89adb71008e6397', 'request_id': 'req-dd38fad2-32a5-4255-a854-6b5f814f0a62'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:28.812122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dd38fad2-32a5-4255-a854-6b5f814f0a62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a50a2e9cce04058b89adb71008e6397', 'request_id': 'req-dd38fad2-32a5-4255-a854-6b5f814f0a62'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:28.887844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dd38fad2-32a5-4255-a854-6b5f814f0a62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a50a2e9cce04058b89adb71008e6397', 'request_id': 'req-dd38fad2-32a5-4255-a854-6b5f814f0a62'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:28.899491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dd38fad2-32a5-4255-a854-6b5f814f0a62 admin admin] The token cache is being invalidate because project 8a50a2e9cce04058b89adb71008e6397 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:32:28.899780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dd38fad2-32a5-4255-a854-6b5f814f0a62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:28.900409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dd38fad2-32a5-4255-a854-6b5f814f0a62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:28.902494 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3667/7371] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:32:28 2022] DELETE /identity/v3/projects/8a50a2e9cce04058b89adb71008e6397 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:32:28.906544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b4e0b-cf36-48ab-a1c0-509db82507c6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:28.907959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-902b4e0b-cf36-48ab-a1c0-509db82507c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:28.929046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902b4e0b-cf36-48ab-a1c0-509db82507c6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:28.982240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382085a-40a1-4aaa-a141-5e2d70295495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:28.983124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382085a-40a1-4aaa-a141-5e2d70295495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:28.983534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382085a-40a1-4aaa-a141-5e2d70295495 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:28.983828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382085a-40a1-4aaa-a141-5e2d70295495 admin admin] PATH_INFO: `/v3/users/cb35028198084d9c9eeb72b28e0e9117` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:28.985121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382085a-40a1-4aaa-a141-5e2d70295495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb35028198084d9c9eeb72b28e0e9117)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:28.987087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382085a-40a1-4aaa-a141-5e2d70295495 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:29.011582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:29.194984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d382085a-40a1-4aaa-a141-5e2d70295495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb35028198084d9c9eeb72b28e0e9117'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:29.202011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d382085a-40a1-4aaa-a141-5e2d70295495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb35028198084d9c9eeb72b28e0e9117'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:29.202718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d382085a-40a1-4aaa-a141-5e2d70295495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb35028198084d9c9eeb72b28e0e9117'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:29.203331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d382085a-40a1-4aaa-a141-5e2d70295495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb35028198084d9c9eeb72b28e0e9117'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:29.329323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d382085a-40a1-4aaa-a141-5e2d70295495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb35028198084d9c9eeb72b28e0e9117'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:29.350890 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3705/7372] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:32:28 2022] DELETE /identity/v3/users/cb35028198084d9c9eeb72b28e0e9117 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:32:29.754210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38fad2-32a5-4255-a854-6b5f814f0a62 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:29.757113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-dd38fad2-32a5-4255-a854-6b5f814f0a62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:29.770972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38fad2-32a5-4255-a854-6b5f814f0a62 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:29.834447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956fd7dd-0020-4d0b-aee3-ba3c28fa5990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:29.835469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956fd7dd-0020-4d0b-aee3-ba3c28fa5990 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:29.835857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956fd7dd-0020-4d0b-aee3-ba3c28fa5990 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:29.836204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956fd7dd-0020-4d0b-aee3-ba3c28fa5990 admin admin] PATH_INFO: `/v3/projects/a63778cceaba4431b706a6de830ced40` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:29.837451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956fd7dd-0020-4d0b-aee3-ba3c28fa5990 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a63778cceaba4431b706a6de830ced40)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:29.839764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956fd7dd-0020-4d0b-aee3-ba3c28fa5990 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:29.862767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:29.975105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-956fd7dd-0020-4d0b-aee3-ba3c28fa5990 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a63778cceaba4431b706a6de830ced40', 'request_id': 'req-956fd7dd-0020-4d0b-aee3-ba3c28fa5990'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:29.980421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-956fd7dd-0020-4d0b-aee3-ba3c28fa5990 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a63778cceaba4431b706a6de830ced40', 'request_id': 'req-956fd7dd-0020-4d0b-aee3-ba3c28fa5990'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:30.157458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-956fd7dd-0020-4d0b-aee3-ba3c28fa5990 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a63778cceaba4431b706a6de830ced40', 'request_id': 'req-956fd7dd-0020-4d0b-aee3-ba3c28fa5990'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:30.170073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-956fd7dd-0020-4d0b-aee3-ba3c28fa5990 admin admin] The token cache is being invalidate because project a63778cceaba4431b706a6de830ced40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:32:30.170364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-956fd7dd-0020-4d0b-aee3-ba3c28fa5990 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:30.171023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-956fd7dd-0020-4d0b-aee3-ba3c28fa5990 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:30.172643 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3668/7373] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:32:29 2022] DELETE /identity/v3/projects/a63778cceaba4431b706a6de830ced40 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:32:30.177543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7183b-7475-492e-a4d7-bd420a85112c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:30.178080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7183b-7475-492e-a4d7-bd420a85112c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:30.178195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d7183b-7475-492e-a4d7-bd420a85112c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:30.197096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d5d7183b-7475-492e-a4d7-bd420a85112c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:32:30.198354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d5d7183b-7475-492e-a4d7-bd420a85112c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:30.258543 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3706/7374] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:32:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:32:30.266564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34159e41-c95a-42f2-bb30-94a2cd3707ce None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:30.266855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34159e41-c95a-42f2-bb30-94a2cd3707ce None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:30.267134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34159e41-c95a-42f2-bb30-94a2cd3707ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:30.285787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-34159e41-c95a-42f2-bb30-94a2cd3707ce None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:32:30.287045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-34159e41-c95a-42f2-bb30-94a2cd3707ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:30.295586 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3669/7375] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:32:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:32:30.299761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7183b-7475-492e-a4d7-bd420a85112c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:30.310343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d7183b-7475-492e-a4d7-bd420a85112c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:30.315296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48595827-2132-49bf-ab17-985ad3a146c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:30.315967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48595827-2132-49bf-ab17-985ad3a146c9 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:30.316461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48595827-2132-49bf-ab17-985ad3a146c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:30.316575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48595827-2132-49bf-ab17-985ad3a146c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:30.317252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48595827-2132-49bf-ab17-985ad3a146c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:32:30.317563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48595827-2132-49bf-ab17-985ad3a146c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:30.319226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48595827-2132-49bf-ab17-985ad3a146c9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:30.335903 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3707/7376] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:32:30.340365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34159e41-c95a-42f2-bb30-94a2cd3707ce None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:30.350513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34159e41-c95a-42f2-bb30-94a2cd3707ce None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:30.356239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5133b-c39e-4e22-89e1-9f6fbdf1eccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:30.356919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5133b-c39e-4e22-89e1-9f6fbdf1eccc admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:30.357206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5133b-c39e-4e22-89e1-9f6fbdf1eccc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:30.357570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5133b-c39e-4e22-89e1-9f6fbdf1eccc admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:30.358214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5133b-c39e-4e22-89e1-9f6fbdf1eccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:30.360187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5133b-c39e-4e22-89e1-9f6fbdf1eccc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:30.514213 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3670/7377] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:32:30 2022] POST /identity/v3/projects => generated 371 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:32:30.518436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48595827-2132-49bf-ab17-985ad3a146c9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:30.531603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48595827-2132-49bf-ab17-985ad3a146c9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:30.598569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fecef2-934d-4192-b4aa-8b02760441fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:30.599315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fecef2-934d-4192-b4aa-8b02760441fe admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:30.599889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fecef2-934d-4192-b4aa-8b02760441fe admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:30.600180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fecef2-934d-4192-b4aa-8b02760441fe admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:30.601645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fecef2-934d-4192-b4aa-8b02760441fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:30.605595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fecef2-934d-4192-b4aa-8b02760441fe admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:30.721461 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3708/7378] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:32:30 2022] POST /identity/v3/users => generated 286 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:32:30.723303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5133b-c39e-4e22-89e1-9f6fbdf1eccc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:30.737870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5133b-c39e-4e22-89e1-9f6fbdf1eccc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:30.746060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fafef-732b-4831-aa4e-880df7d4fa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:30.746883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fafef-732b-4831-aa4e-880df7d4fa74 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:30.747198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fafef-732b-4831-aa4e-880df7d4fa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:30.747469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fafef-732b-4831-aa4e-880df7d4fa74 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:30.748728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9fafef-732b-4831-aa4e-880df7d4fa74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:30.751186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9fafef-732b-4831-aa4e-880df7d4fa74 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:30.768899 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3671/7379] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:32:30.771205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fecef2-934d-4192-b4aa-8b02760441fe admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:30.784432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fecef2-934d-4192-b4aa-8b02760441fe admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:30.790780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46668636-e57b-4b5a-ba05-3eb27e3594eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:30.793902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46668636-e57b-4b5a-ba05-3eb27e3594eb admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:30.793902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46668636-e57b-4b5a-ba05-3eb27e3594eb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:30.793902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46668636-e57b-4b5a-ba05-3eb27e3594eb admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:30.793902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46668636-e57b-4b5a-ba05-3eb27e3594eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:30.795072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46668636-e57b-4b5a-ba05-3eb27e3594eb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:30.808652 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3709/7380] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:32:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:32:30.813037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fafef-732b-4831-aa4e-880df7d4fa74 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:30.823768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9fafef-732b-4831-aa4e-880df7d4fa74 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:30.829915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0800a78-0409-4b1b-961d-5aad4a5ab7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:30.831512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0800a78-0409-4b1b-961d-5aad4a5ab7d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:30.832006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0800a78-0409-4b1b-961d-5aad4a5ab7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:30.832476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0800a78-0409-4b1b-961d-5aad4a5ab7d4 admin admin] PATH_INFO: `/v3/projects/0ce4c7a9c74d441581a634afac6b8f8d/users/7882449f34df41708755bf3f902cf043/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:30.856612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0800a78-0409-4b1b-961d-5aad4a5ab7d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce4c7a9c74d441581a634afac6b8f8d, user_id=7882449f34df41708755bf3f902cf043, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:30.860118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0800a78-0409-4b1b-961d-5aad4a5ab7d4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:30.970332 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3672/7381] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:32:30 2022] PUT /identity/v3/projects/0ce4c7a9c74d441581a634afac6b8f8d/users/7882449f34df41708755bf3f902cf043/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:32:30.980759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0044d40-e878-480c-8dff-e485b87c3ad1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:30.981061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0044d40-e878-480c-8dff-e485b87c3ad1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:30.981339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0044d40-e878-480c-8dff-e485b87c3ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:30.990086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:31.009506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f0044d40-e878-480c-8dff-e485b87c3ad1 None None] MFA Rules not processed for user `7882449f34df41708755bf3f902cf043`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:32:31.010879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f0044d40-e878-480c-8dff-e485b87c3ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:31.078503 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3710/7382] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:32:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:32:31.090046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0800a78-0409-4b1b-961d-5aad4a5ab7d4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:31.093359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e0800a78-0409-4b1b-961d-5aad4a5ab7d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:31.181859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0800a78-0409-4b1b-961d-5aad4a5ab7d4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:31.194657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a3aa5-a66f-4e98-b633-8dc0308676ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:31.195366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a3aa5-a66f-4e98-b633-8dc0308676ba service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:31.195650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a3aa5-a66f-4e98-b633-8dc0308676ba service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:31.195927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a3aa5-a66f-4e98-b633-8dc0308676ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:31.205170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a3aa5-a66f-4e98-b633-8dc0308676ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:31.206831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a3aa5-a66f-4e98-b633-8dc0308676ba service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:31.216192 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3673/7383] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:32:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:32:41.208747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=30 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:41.211778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:43.727364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0044d40-e878-480c-8dff-e485b87c3ad1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:43.730166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f0044d40-e878-480c-8dff-e485b87c3ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:43.825072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0044d40-e878-480c-8dff-e485b87c3ad1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:43.832001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0a259-2d16-440d-9f4f-34be12e7dd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:43.832204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0a259-2d16-440d-9f4f-34be12e7dd97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:43.832391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0a259-2d16-440d-9f4f-34be12e7dd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:43.832674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e0a259-2d16-440d-9f4f-34be12e7dd97 admin admin] PATH_INFO: `/v3/users/38d2c184e2ad446cb0481b1421439771` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:43.834014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0a259-2d16-440d-9f4f-34be12e7dd97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38d2c184e2ad446cb0481b1421439771)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:43.840634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e0a259-2d16-440d-9f4f-34be12e7dd97 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:44.120699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-88e0a259-2d16-440d-9f4f-34be12e7dd97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38d2c184e2ad446cb0481b1421439771'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:44.130068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-88e0a259-2d16-440d-9f4f-34be12e7dd97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38d2c184e2ad446cb0481b1421439771'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:44.130810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-88e0a259-2d16-440d-9f4f-34be12e7dd97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38d2c184e2ad446cb0481b1421439771'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:44.131420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-88e0a259-2d16-440d-9f4f-34be12e7dd97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38d2c184e2ad446cb0481b1421439771'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:44.225090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-88e0a259-2d16-440d-9f4f-34be12e7dd97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38d2c184e2ad446cb0481b1421439771'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:44.244536 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3711/7384] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:32:43 2022] DELETE /identity/v3/users/38d2c184e2ad446cb0481b1421439771 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:32:44.249032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d19d2-851a-485b-90c0-edcee30e205e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:44.249540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d19d2-851a-485b-90c0-edcee30e205e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:44.249643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d19d2-851a-485b-90c0-edcee30e205e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:44.268044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ad0d19d2-851a-485b-90c0-edcee30e205e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:32:44.270009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ad0d19d2-851a-485b-90c0-edcee30e205e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:44.328604 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3674/7385] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:32:44.340813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0a259-2d16-440d-9f4f-34be12e7dd97 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:44.343459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-88e0a259-2d16-440d-9f4f-34be12e7dd97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:44.403872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e0a259-2d16-440d-9f4f-34be12e7dd97 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:44.407422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a31512-c6c9-476f-988b-441751383ccb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:44.408097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a31512-c6c9-476f-988b-441751383ccb service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:44.408386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a31512-c6c9-476f-988b-441751383ccb service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:44.408665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a31512-c6c9-476f-988b-441751383ccb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:44.418505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a31512-c6c9-476f-988b-441751383ccb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:44.419943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a31512-c6c9-476f-988b-441751383ccb service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:44.429589 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3712/7386] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:32:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:32:44.696499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d19d2-851a-485b-90c0-edcee30e205e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:44.697970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ad0d19d2-851a-485b-90c0-edcee30e205e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:44.711237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d19d2-851a-485b-90c0-edcee30e205e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:44.716263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d2ffdc-5eaa-43da-b65e-3d940754cc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:44.717224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2ffdc-5eaa-43da-b65e-3d940754cc75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:44.717583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2ffdc-5eaa-43da-b65e-3d940754cc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:44.717979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d2ffdc-5eaa-43da-b65e-3d940754cc75 admin admin] PATH_INFO: `/v3/projects/071f3ee8ccba4f4a891cac48aaa54d0a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:44.719198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2ffdc-5eaa-43da-b65e-3d940754cc75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=071f3ee8ccba4f4a891cac48aaa54d0a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:44.721199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d2ffdc-5eaa-43da-b65e-3d940754cc75 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:44.850134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-03d2ffdc-5eaa-43da-b65e-3d940754cc75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '071f3ee8ccba4f4a891cac48aaa54d0a', 'request_id': 'req-03d2ffdc-5eaa-43da-b65e-3d940754cc75'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:44.855681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-03d2ffdc-5eaa-43da-b65e-3d940754cc75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '071f3ee8ccba4f4a891cac48aaa54d0a', 'request_id': 'req-03d2ffdc-5eaa-43da-b65e-3d940754cc75'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:44.940118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-03d2ffdc-5eaa-43da-b65e-3d940754cc75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '071f3ee8ccba4f4a891cac48aaa54d0a', 'request_id': 'req-03d2ffdc-5eaa-43da-b65e-3d940754cc75'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:44.957234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-03d2ffdc-5eaa-43da-b65e-3d940754cc75 admin admin] The token cache is being invalidate because project 071f3ee8ccba4f4a891cac48aaa54d0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:32:44.957521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-03d2ffdc-5eaa-43da-b65e-3d940754cc75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:44.958218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-03d2ffdc-5eaa-43da-b65e-3d940754cc75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:32:44.959758 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3675/7387] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:32:44 2022] DELETE /identity/v3/projects/071f3ee8ccba4f4a891cac48aaa54d0a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:32:44.964835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16a3e0-326d-4486-a88d-6e0ee59e6e3d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:44.965315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16a3e0-326d-4486-a88d-6e0ee59e6e3d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:44.965419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16a3e0-326d-4486-a88d-6e0ee59e6e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:44.983045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ba16a3e0-326d-4486-a88d-6e0ee59e6e3d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:32:44.984312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ba16a3e0-326d-4486-a88d-6e0ee59e6e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:45.001901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:45.043717 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3713/7388] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:32:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:32:45.051937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fa394-152a-4868-aa8d-1ff677aaeb78 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:45.052228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fa394-152a-4868-aa8d-1ff677aaeb78 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:45.052509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fa394-152a-4868-aa8d-1ff677aaeb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:45.071455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8b4fa394-152a-4868-aa8d-1ff677aaeb78 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:32:45.072648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8b4fa394-152a-4868-aa8d-1ff677aaeb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:45.080690 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3676/7389] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:32:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:32:45.084937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16a3e0-326d-4486-a88d-6e0ee59e6e3d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:45.095190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16a3e0-326d-4486-a88d-6e0ee59e6e3d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:45.100133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86078c3b-bc3f-44bd-8a64-99f4eda14234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:45.109935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86078c3b-bc3f-44bd-8a64-99f4eda14234 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:45.109935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86078c3b-bc3f-44bd-8a64-99f4eda14234 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:45.109935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86078c3b-bc3f-44bd-8a64-99f4eda14234 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:45.109935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86078c3b-bc3f-44bd-8a64-99f4eda14234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:32:45.109935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86078c3b-bc3f-44bd-8a64-99f4eda14234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:45.109935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86078c3b-bc3f-44bd-8a64-99f4eda14234 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:45.117284 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3714/7390] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:32:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:32:45.121340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fa394-152a-4868-aa8d-1ff677aaeb78 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:45.133354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fa394-152a-4868-aa8d-1ff677aaeb78 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:45.138332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b173f-d570-411f-9a70-d29865c5391a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:45.139019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b173f-d570-411f-9a70-d29865c5391a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:45.139303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b173f-d570-411f-9a70-d29865c5391a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:45.139582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4b173f-d570-411f-9a70-d29865c5391a admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:45.140330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b173f-d570-411f-9a70-d29865c5391a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:45.142327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4b173f-d570-411f-9a70-d29865c5391a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:45.256306 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3677/7391] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:32:45 2022] POST /identity/v3/projects => generated 363 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:32:45.257366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86078c3b-bc3f-44bd-8a64-99f4eda14234 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:45.270536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86078c3b-bc3f-44bd-8a64-99f4eda14234 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:45.332714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c99d64-5a48-43ef-b987-d4dd96bc1067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:45.333390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c99d64-5a48-43ef-b987-d4dd96bc1067 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:45.333683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c99d64-5a48-43ef-b987-d4dd96bc1067 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:45.334003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c99d64-5a48-43ef-b987-d4dd96bc1067 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:45.339339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c99d64-5a48-43ef-b987-d4dd96bc1067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:45.339339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c99d64-5a48-43ef-b987-d4dd96bc1067 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:45.434948 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3715/7392] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:32:45 2022] POST /identity/v3/users => generated 282 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:32:45.436467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b173f-d570-411f-9a70-d29865c5391a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:45.448095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4b173f-d570-411f-9a70-d29865c5391a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:45.452679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41cd0d-3dd5-4967-a027-f215137ad3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:45.453342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41cd0d-3dd5-4967-a027-f215137ad3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:45.453623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41cd0d-3dd5-4967-a027-f215137ad3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:45.453937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41cd0d-3dd5-4967-a027-f215137ad3d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:45.454664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41cd0d-3dd5-4967-a027-f215137ad3d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:45.456929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a41cd0d-3dd5-4967-a027-f215137ad3d7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:45.467724 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3678/7393] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:32:45.472096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c99d64-5a48-43ef-b987-d4dd96bc1067 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:45.482699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c99d64-5a48-43ef-b987-d4dd96bc1067 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:45.487452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62a339-bde4-473e-a90a-106ea77f9700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:45.488194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62a339-bde4-473e-a90a-106ea77f9700 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:45.488445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62a339-bde4-473e-a90a-106ea77f9700 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:45.488727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62a339-bde4-473e-a90a-106ea77f9700 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:45.489460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62a339-bde4-473e-a90a-106ea77f9700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:45.491099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62a339-bde4-473e-a90a-106ea77f9700 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:45.501037 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3716/7394] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:32:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:32:45.505749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41cd0d-3dd5-4967-a027-f215137ad3d7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:45.516038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41cd0d-3dd5-4967-a027-f215137ad3d7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:45.521312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1479aae-3b7a-40b8-b5a5-d7de5ba55c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:45.522269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1479aae-3b7a-40b8-b5a5-d7de5ba55c2b admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:45.522552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1479aae-3b7a-40b8-b5a5-d7de5ba55c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:45.522828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1479aae-3b7a-40b8-b5a5-d7de5ba55c2b admin admin] PATH_INFO: `/v3/projects/045cd28b35b3467e91274fff8140771b/users/1d9d97eaf9f74d58adf5ea7e90918c22/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:45.537261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1479aae-3b7a-40b8-b5a5-d7de5ba55c2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=045cd28b35b3467e91274fff8140771b, user_id=1d9d97eaf9f74d58adf5ea7e90918c22, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:45.541302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1479aae-3b7a-40b8-b5a5-d7de5ba55c2b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:45.626050 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3679/7395] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:32:45 2022] PUT /identity/v3/projects/045cd28b35b3467e91274fff8140771b/users/1d9d97eaf9f74d58adf5ea7e90918c22/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:32:45.636844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa88a4-b37f-4898-9d99-b271a38c3206 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:45.637135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa88a4-b37f-4898-9d99-b271a38c3206 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:45.637453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa88a4-b37f-4898-9d99-b271a38c3206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:45.671632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-90fa88a4-b37f-4898-9d99-b271a38c3206 None None] MFA Rules not processed for user `1d9d97eaf9f74d58adf5ea7e90918c22`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:32:45.672868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-90fa88a4-b37f-4898-9d99-b271a38c3206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:45.741068 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3717/7396] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:32:45 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:32:45.753680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1479aae-3b7a-40b8-b5a5-d7de5ba55c2b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:32:45.755492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b1479aae-3b7a-40b8-b5a5-d7de5ba55c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:32:45.785276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:45.844476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1479aae-3b7a-40b8-b5a5-d7de5ba55c2b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:32:45.848220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383195a-2af2-45a4-a582-a603521179d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:32:45.848876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383195a-2af2-45a4-a582-a603521179d6 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:32:45.849211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383195a-2af2-45a4-a582-a603521179d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:32:45.849440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383195a-2af2-45a4-a582-a603521179d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:32:45.857999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383195a-2af2-45a4-a582-a603521179d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:32:45.859441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383195a-2af2-45a4-a582-a603521179d6 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:32:45.868732 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3680/7397] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:32:45 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:32:55.738188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:55.935016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=22 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:03.335322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa88a4-b37f-4898-9d99-b271a38c3206 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:03.350857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa88a4-b37f-4898-9d99-b271a38c3206 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:03.354686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f80111-1c1c-4b8a-af46-00eeb4b4b769 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:03.355332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f80111-1c1c-4b8a-af46-00eeb4b4b769 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:03.355615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f80111-1c1c-4b8a-af46-00eeb4b4b769 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:03.355894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f80111-1c1c-4b8a-af46-00eeb4b4b769 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:03.357747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-95f80111-1c1c-4b8a-af46-00eeb4b4b769 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:03.490144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f80111-1c1c-4b8a-af46-00eeb4b4b769 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:03.491628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f80111-1c1c-4b8a-af46-00eeb4b4b769 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:03.505752 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3718/7398] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:33:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:33:04.576883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383195a-2af2-45a4-a582-a603521179d6 service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:04.576883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2383195a-2af2-45a4-a582-a603521179d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:04.599687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383195a-2af2-45a4-a582-a603521179d6 service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:04.603476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a1fdd-9ec1-47d0-9650-4e49d44927df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:04.604151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a1fdd-9ec1-47d0-9650-4e49d44927df service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:04.604436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a1fdd-9ec1-47d0-9650-4e49d44927df service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:04.604738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a1fdd-9ec1-47d0-9650-4e49d44927df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:04.606538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4e1a1fdd-9ec1-47d0-9650-4e49d44927df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:04.622473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a1fdd-9ec1-47d0-9650-4e49d44927df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:04.623906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a1fdd-9ec1-47d0-9650-4e49d44927df service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:04.638159 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3681/7399] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:33:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:33:05.579573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f80111-1c1c-4b8a-af46-00eeb4b4b769 service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:05.600462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f80111-1c1c-4b8a-af46-00eeb4b4b769 service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:05.602090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563e1b35-2132-419a-bccb-a797791f87f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:05.602773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e1b35-2132-419a-bccb-a797791f87f8 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:05.603087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e1b35-2132-419a-bccb-a797791f87f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:05.603368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e1b35-2132-419a-bccb-a797791f87f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:05.605170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-563e1b35-2132-419a-bccb-a797791f87f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:05.666333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e1b35-2132-419a-bccb-a797791f87f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:05.667775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563e1b35-2132-419a-bccb-a797791f87f8 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:05.677981 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3719/7400] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:33:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:33:06.801253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a1fdd-9ec1-47d0-9650-4e49d44927df service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:06.801761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4e1a1fdd-9ec1-47d0-9650-4e49d44927df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:06.890030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a1fdd-9ec1-47d0-9650-4e49d44927df service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:06.894579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a00e8c-90fa-42a8-9c34-2c56ad006aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:06.895957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a00e8c-90fa-42a8-9c34-2c56ad006aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:06.896237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a00e8c-90fa-42a8-9c34-2c56ad006aff admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:06.896516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a00e8c-90fa-42a8-9c34-2c56ad006aff admin admin] PATH_INFO: `/v3/users/7882449f34df41708755bf3f902cf043` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:06.902962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a00e8c-90fa-42a8-9c34-2c56ad006aff admin admin] RBAC: Authorizing `identity:delete_user(user_id=7882449f34df41708755bf3f902cf043)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:06.902962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a00e8c-90fa-42a8-9c34-2c56ad006aff admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:07.160078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e1a00e8c-90fa-42a8-9c34-2c56ad006aff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7882449f34df41708755bf3f902cf043'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:07.160812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e1a00e8c-90fa-42a8-9c34-2c56ad006aff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7882449f34df41708755bf3f902cf043'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:07.168013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e1a00e8c-90fa-42a8-9c34-2c56ad006aff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7882449f34df41708755bf3f902cf043'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:07.255281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e1a00e8c-90fa-42a8-9c34-2c56ad006aff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7882449f34df41708755bf3f902cf043'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:07.255970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e1a00e8c-90fa-42a8-9c34-2c56ad006aff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7882449f34df41708755bf3f902cf043'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:07.275034 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3682/7401] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:06 2022] DELETE /identity/v3/users/7882449f34df41708755bf3f902cf043 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:07.279880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecddf3b-73a5-4047-a49d-9a1665d721a2 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:07.280384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecddf3b-73a5-4047-a49d-9a1665d721a2 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:07.280476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecddf3b-73a5-4047-a49d-9a1665d721a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:07.307862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fecddf3b-73a5-4047-a49d-9a1665d721a2 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:07.309096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fecddf3b-73a5-4047-a49d-9a1665d721a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:07.315404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:07.370161 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3720/7402] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:07.380459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a00e8c-90fa-42a8-9c34-2c56ad006aff admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:07.381971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e1a00e8c-90fa-42a8-9c34-2c56ad006aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:07.425698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:07.458307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a00e8c-90fa-42a8-9c34-2c56ad006aff admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:07.462125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e5afd-9e3d-4f56-a1df-dade61c6b7bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:07.462778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e5afd-9e3d-4f56-a1df-dade61c6b7bf service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:07.463099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e5afd-9e3d-4f56-a1df-dade61c6b7bf service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:07.463380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e5afd-9e3d-4f56-a1df-dade61c6b7bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:07.472922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e5afd-9e3d-4f56-a1df-dade61c6b7bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:07.474622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e5afd-9e3d-4f56-a1df-dade61c6b7bf service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:07.484245 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3683/7403] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:33:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:33:07.982866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecddf3b-73a5-4047-a49d-9a1665d721a2 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:07.984347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fecddf3b-73a5-4047-a49d-9a1665d721a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:07.997912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecddf3b-73a5-4047-a49d-9a1665d721a2 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:08.002679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2fad92-9f57-4984-a3b2-f4cf4f0d88aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:08.003514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fad92-9f57-4984-a3b2-f4cf4f0d88aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:08.003795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fad92-9f57-4984-a3b2-f4cf4f0d88aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:08.004681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2fad92-9f57-4984-a3b2-f4cf4f0d88aa admin admin] PATH_INFO: `/v3/projects/0ce4c7a9c74d441581a634afac6b8f8d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:08.005277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2fad92-9f57-4984-a3b2-f4cf4f0d88aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ce4c7a9c74d441581a634afac6b8f8d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:08.007239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2fad92-9f57-4984-a3b2-f4cf4f0d88aa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:08.113379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d2fad92-9f57-4984-a3b2-f4cf4f0d88aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce4c7a9c74d441581a634afac6b8f8d', 'request_id': 'req-5d2fad92-9f57-4984-a3b2-f4cf4f0d88aa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:08.120912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d2fad92-9f57-4984-a3b2-f4cf4f0d88aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce4c7a9c74d441581a634afac6b8f8d', 'request_id': 'req-5d2fad92-9f57-4984-a3b2-f4cf4f0d88aa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:08.203821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d2fad92-9f57-4984-a3b2-f4cf4f0d88aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce4c7a9c74d441581a634afac6b8f8d', 'request_id': 'req-5d2fad92-9f57-4984-a3b2-f4cf4f0d88aa'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:08.216524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d2fad92-9f57-4984-a3b2-f4cf4f0d88aa admin admin] The token cache is being invalidate because project 0ce4c7a9c74d441581a634afac6b8f8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:08.216811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d2fad92-9f57-4984-a3b2-f4cf4f0d88aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:08.217452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5d2fad92-9f57-4984-a3b2-f4cf4f0d88aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:08.219053 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3721/7404] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:07 2022] DELETE /identity/v3/projects/0ce4c7a9c74d441581a634afac6b8f8d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:08.225701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e6623-08f1-4bc6-b25d-a8ab877309e4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:08.226252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e6623-08f1-4bc6-b25d-a8ab877309e4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:08.226347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e6623-08f1-4bc6-b25d-a8ab877309e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:08.273659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a96e6623-08f1-4bc6-b25d-a8ab877309e4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:08.274892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a96e6623-08f1-4bc6-b25d-a8ab877309e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:08.334275 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3684/7405] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:33:08.342300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9238c-96d2-4b56-9a5c-40c169e9bb21 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:08.342847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9238c-96d2-4b56-9a5c-40c169e9bb21 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:08.342847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9238c-96d2-4b56-9a5c-40c169e9bb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:08.362154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cac9238c-96d2-4b56-9a5c-40c169e9bb21 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:08.363374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cac9238c-96d2-4b56-9a5c-40c169e9bb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:08.373227 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3722/7406] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:08.383326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e6623-08f1-4bc6-b25d-a8ab877309e4 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:08.393242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e6623-08f1-4bc6-b25d-a8ab877309e4 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:08.399200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7b62a-4aea-41b5-9edd-3310492ee5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:08.399452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7b62a-4aea-41b5-9edd-3310492ee5ce admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:08.399739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7b62a-4aea-41b5-9edd-3310492ee5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:08.400021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7b62a-4aea-41b5-9edd-3310492ee5ce admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:08.400765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc7b62a-4aea-41b5-9edd-3310492ee5ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:33:08.401076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc7b62a-4aea-41b5-9edd-3310492ee5ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:08.402750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc7b62a-4aea-41b5-9edd-3310492ee5ce admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:08.415741 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3685/7407] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:33:08.420140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9238c-96d2-4b56-9a5c-40c169e9bb21 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:08.430004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9238c-96d2-4b56-9a5c-40c169e9bb21 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:08.434785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bbccf-a1a4-4ee2-aaeb-d9a29be49a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:08.435719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bbccf-a1a4-4ee2-aaeb-d9a29be49a27 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:08.435928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bbccf-a1a4-4ee2-aaeb-d9a29be49a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:08.436014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bbccf-a1a4-4ee2-aaeb-d9a29be49a27 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:08.436717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11bbccf-a1a4-4ee2-aaeb-d9a29be49a27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:08.439030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11bbccf-a1a4-4ee2-aaeb-d9a29be49a27 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:08.547602 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3723/7408] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:08 2022] POST /identity/v3/projects => generated 389 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:08.551629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7b62a-4aea-41b5-9edd-3310492ee5ce admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:08.561885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7b62a-4aea-41b5-9edd-3310492ee5ce admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:08.622870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a3167-3ce7-4278-9ea9-2d2cdbfd320c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:08.623538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a3167-3ce7-4278-9ea9-2d2cdbfd320c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:08.623822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a3167-3ce7-4278-9ea9-2d2cdbfd320c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:08.624100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a3167-3ce7-4278-9ea9-2d2cdbfd320c admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:08.624779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a3167-3ce7-4278-9ea9-2d2cdbfd320c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:08.626780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a3167-3ce7-4278-9ea9-2d2cdbfd320c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:08.740432 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3686/7409] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:08 2022] POST /identity/v3/users => generated 295 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:08.744484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bbccf-a1a4-4ee2-aaeb-d9a29be49a27 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:08.755633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bbccf-a1a4-4ee2-aaeb-d9a29be49a27 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:08.764459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c985ac-7f9d-4f38-876e-16731a3c7e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:08.765157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c985ac-7f9d-4f38-876e-16731a3c7e8a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:08.765443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c985ac-7f9d-4f38-876e-16731a3c7e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:08.765723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c985ac-7f9d-4f38-876e-16731a3c7e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:08.766542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c985ac-7f9d-4f38-876e-16731a3c7e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:08.768181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c985ac-7f9d-4f38-876e-16731a3c7e8a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:08.780244 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3724/7410] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:08.784355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a3167-3ce7-4278-9ea9-2d2cdbfd320c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:08.794583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a3167-3ce7-4278-9ea9-2d2cdbfd320c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:08.799283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ff46f-a4ab-444e-9d99-af010a6ec76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:08.799880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ff46f-a4ab-444e-9d99-af010a6ec76b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:08.800168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ff46f-a4ab-444e-9d99-af010a6ec76b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:08.800449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ff46f-a4ab-444e-9d99-af010a6ec76b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:08.801175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ff46f-a4ab-444e-9d99-af010a6ec76b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:08.803328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ff46f-a4ab-444e-9d99-af010a6ec76b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:08.818550 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3687/7411] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:08.822582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c985ac-7f9d-4f38-876e-16731a3c7e8a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:08.832708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c985ac-7f9d-4f38-876e-16731a3c7e8a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:08.837709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0e92c-4086-4c0d-945f-3771d684f305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:08.838637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0e92c-4086-4c0d-945f-3771d684f305 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:08.838987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0e92c-4086-4c0d-945f-3771d684f305 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:08.839449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0e92c-4086-4c0d-945f-3771d684f305 admin admin] PATH_INFO: `/v3/projects/284c98cdbd0645388b350270ee5d61e0/users/a61283b09f6c47118cd431e8e2696b0e/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:08.857714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0e92c-4086-4c0d-945f-3771d684f305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=284c98cdbd0645388b350270ee5d61e0, user_id=a61283b09f6c47118cd431e8e2696b0e, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:08.860440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0e92c-4086-4c0d-945f-3771d684f305 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:08.929467 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3725/7412] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:08 2022] PUT /identity/v3/projects/284c98cdbd0645388b350270ee5d61e0/users/a61283b09f6c47118cd431e8e2696b0e/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:08.945001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13439829-cdc5-420e-8f14-9f44f28174e3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:08.945363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13439829-cdc5-420e-8f14-9f44f28174e3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:08.945712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13439829-cdc5-420e-8f14-9f44f28174e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:08.964727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:08.983285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-13439829-cdc5-420e-8f14-9f44f28174e3 None None] MFA Rules not processed for user `a61283b09f6c47118cd431e8e2696b0e`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:08.983285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-13439829-cdc5-420e-8f14-9f44f28174e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:09.041243 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3688/7413] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:09.049231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0e92c-4086-4c0d-945f-3771d684f305 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:09.097624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:09.125425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0e92c-4086-4c0d-945f-3771d684f305 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:09.130264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0440d41-73fe-4039-aaab-1e3a9d9ee54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:09.132094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0440d41-73fe-4039-aaab-1e3a9d9ee54d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:09.132094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0440d41-73fe-4039-aaab-1e3a9d9ee54d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:09.132094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0440d41-73fe-4039-aaab-1e3a9d9ee54d admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:09.132613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0440d41-73fe-4039-aaab-1e3a9d9ee54d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:09.134573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0440d41-73fe-4039-aaab-1e3a9d9ee54d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:09.219449 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3726/7414] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:09 2022] POST /identity/v3/projects => generated 389 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:09.221114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13439829-cdc5-420e-8f14-9f44f28174e3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:09.232664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13439829-cdc5-420e-8f14-9f44f28174e3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:09.300488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fa08d-0c12-4835-88bc-af8d7eb1928b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:09.301159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fa08d-0c12-4835-88bc-af8d7eb1928b admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:09.302035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fa08d-0c12-4835-88bc-af8d7eb1928b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:09.302312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17fa08d-0c12-4835-88bc-af8d7eb1928b admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:09.302992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17fa08d-0c12-4835-88bc-af8d7eb1928b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:09.304917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17fa08d-0c12-4835-88bc-af8d7eb1928b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:09.409149 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3689/7415] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:09 2022] POST /identity/v3/users => generated 301 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:09.416140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0440d41-73fe-4039-aaab-1e3a9d9ee54d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:09.427365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0440d41-73fe-4039-aaab-1e3a9d9ee54d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:09.432086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874709a5-6cd7-42b0-879b-736631443b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:09.432772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874709a5-6cd7-42b0-879b-736631443b1a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:09.433125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874709a5-6cd7-42b0-879b-736631443b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:09.433411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874709a5-6cd7-42b0-879b-736631443b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:09.434196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874709a5-6cd7-42b0-879b-736631443b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:09.435870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874709a5-6cd7-42b0-879b-736631443b1a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:09.446904 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3727/7416] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:09.451368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fa08d-0c12-4835-88bc-af8d7eb1928b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:09.461889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17fa08d-0c12-4835-88bc-af8d7eb1928b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:09.466845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475d37f-60a5-4c3f-a043-3c9aacf031bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:09.467732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475d37f-60a5-4c3f-a043-3c9aacf031bf admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:09.468021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475d37f-60a5-4c3f-a043-3c9aacf031bf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:09.468331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475d37f-60a5-4c3f-a043-3c9aacf031bf admin admin] PATH_INFO: `/v3/projects/11b18f811e944325bcf90bc62fe13694/users/d39d4a62e6884a6799a8b034788673c0/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:09.484364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475d37f-60a5-4c3f-a043-3c9aacf031bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=11b18f811e944325bcf90bc62fe13694, user_id=d39d4a62e6884a6799a8b034788673c0, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:09.487177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475d37f-60a5-4c3f-a043-3c9aacf031bf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:09.600165 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3690/7417] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:09 2022] PUT /identity/v3/projects/11b18f811e944325bcf90bc62fe13694/users/d39d4a62e6884a6799a8b034788673c0/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:09.604490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874709a5-6cd7-42b0-879b-736631443b1a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:09.615387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874709a5-6cd7-42b0-879b-736631443b1a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:09.674157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81e57e-2e8a-4f45-8c20-56a25a82863c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:09.674788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81e57e-2e8a-4f45-8c20-56a25a82863c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:09.675108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81e57e-2e8a-4f45-8c20-56a25a82863c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:09.675422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81e57e-2e8a-4f45-8c20-56a25a82863c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:09.676241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81e57e-2e8a-4f45-8c20-56a25a82863c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:09.677940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81e57e-2e8a-4f45-8c20-56a25a82863c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:09.688126 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3728/7418] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:09.692399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475d37f-60a5-4c3f-a043-3c9aacf031bf admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:09.733290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475d37f-60a5-4c3f-a043-3c9aacf031bf admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:09.733290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4b9f5-8e40-4bcf-81e8-76344207b9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:09.733290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4b9f5-8e40-4bcf-81e8-76344207b9d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:09.733290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4b9f5-8e40-4bcf-81e8-76344207b9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:09.733724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4b9f5-8e40-4bcf-81e8-76344207b9d1 admin admin] PATH_INFO: `/v3/projects/11b18f811e944325bcf90bc62fe13694/users/d39d4a62e6884a6799a8b034788673c0/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:09.733724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4b9f5-8e40-4bcf-81e8-76344207b9d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11b18f811e944325bcf90bc62fe13694, user_id=d39d4a62e6884a6799a8b034788673c0, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:09.733724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4b9f5-8e40-4bcf-81e8-76344207b9d1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:09.733724 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3691/7419] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:09 2022] PUT /identity/v3/projects/11b18f811e944325bcf90bc62fe13694/users/d39d4a62e6884a6799a8b034788673c0/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:09.743761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdccbc4-6566-4dfe-9d60-99041a149dfb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:09.744060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdccbc4-6566-4dfe-9d60-99041a149dfb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:09.744347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdccbc4-6566-4dfe-9d60-99041a149dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:09.771440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4fdccbc4-6566-4dfe-9d60-99041a149dfb None None] MFA Rules not processed for user `d39d4a62e6884a6799a8b034788673c0`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:09.772646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4fdccbc4-6566-4dfe-9d60-99041a149dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:09.840009 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3729/7420] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:09 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:09.845125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4b9f5-8e40-4bcf-81e8-76344207b9d1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:09.857134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4b9f5-8e40-4bcf-81e8-76344207b9d1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:09.862340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3905b3-71fb-44c8-96a0-279230bc7c71 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:09.863064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3905b3-71fb-44c8-96a0-279230bc7c71 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:09.863375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3905b3-71fb-44c8-96a0-279230bc7c71 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:09.863690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3905b3-71fb-44c8-96a0-279230bc7c71 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:09.865111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3905b3-71fb-44c8-96a0-279230bc7c71 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:09.867632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3905b3-71fb-44c8-96a0-279230bc7c71 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:10.324514 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3692/7421] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:33:09 2022] POST /identity/v3/domains => generated 276 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:10.328814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdccbc4-6566-4dfe-9d60-99041a149dfb None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:10.338484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdccbc4-6566-4dfe-9d60-99041a149dfb None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:10.363791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:10.396390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d04406-e291-4a98-bd91-12eab905b0bd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:10.397249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d04406-e291-4a98-bd91-12eab905b0bd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:10.397537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d04406-e291-4a98-bd91-12eab905b0bd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:10.397819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d04406-e291-4a98-bd91-12eab905b0bd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/210781bdf2f442789e269b9bae58c2dd/config` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:10.398609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d04406-e291-4a98-bd91-12eab905b0bd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=210781bdf2f442789e269b9bae58c2dd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:10.399624 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-c2d04406-e291-4a98-bd91-12eab905b0bd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:33:10.399624 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:33:10.399624 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:33:10.400517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d04406-e291-4a98-bd91-12eab905b0bd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:10.625246 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3730/7422] 10.222.0.33 () {60 vars in 1268 bytes} [Sat May 28 21:33:10 2022] PUT /identity/v3/domains/210781bdf2f442789e269b9bae58c2dd/config => generated 138 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:10.629316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3905b3-71fb-44c8-96a0-279230bc7c71 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:10.640179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3905b3-71fb-44c8-96a0-279230bc7c71 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:10.644769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34088a-af4e-4350-8f7c-d8e379024df9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:10.645657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34088a-af4e-4350-8f7c-d8e379024df9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:10.645976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34088a-af4e-4350-8f7c-d8e379024df9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:10.646259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34088a-af4e-4350-8f7c-d8e379024df9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/210781bdf2f442789e269b9bae58c2dd/config/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:10.647461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34088a-af4e-4350-8f7c-d8e379024df9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=210781bdf2f442789e269b9bae58c2dd, group=identity)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:10.648640 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-3e34088a-af4e-4350-8f7c-d8e379024df9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:33:10.648640 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:33:10.648640 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:33:10.649568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34088a-af4e-4350-8f7c-d8e379024df9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:10.656770 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3693/7423] 10.222.0.33 () {58 vars in 1274 bytes} [Sat May 28 21:33:10 2022] GET /identity/v3/domains/210781bdf2f442789e269b9bae58c2dd/config/identity => generated 45 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:10.661312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d04406-e291-4a98-bd91-12eab905b0bd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:10.671395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d04406-e291-4a98-bd91-12eab905b0bd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:10.676117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939247f-fe2f-420b-9ef2-4d9dc36b8cc6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:10.677631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939247f-fe2f-420b-9ef2-4d9dc36b8cc6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:10.677954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939247f-fe2f-420b-9ef2-4d9dc36b8cc6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:10.678223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939247f-fe2f-420b-9ef2-4d9dc36b8cc6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/210781bdf2f442789e269b9bae58c2dd/config/identity/driver` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:10.679498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939247f-fe2f-420b-9ef2-4d9dc36b8cc6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=210781bdf2f442789e269b9bae58c2dd, group=identity, option=driver)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:10.680506 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-8939247f-fe2f-420b-9ef2-4d9dc36b8cc6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:33:10.680506 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:33:10.680506 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:33:10.681397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939247f-fe2f-420b-9ef2-4d9dc36b8cc6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:10.690081 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3731/7424] 10.222.0.33 () {58 vars in 1295 bytes} [Sat May 28 21:33:10 2022] GET /identity/v3/domains/210781bdf2f442789e269b9bae58c2dd/config/identity/driver => generated 31 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:10.691269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34088a-af4e-4350-8f7c-d8e379024df9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:10.702052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34088a-af4e-4350-8f7c-d8e379024df9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:10.707196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb20518-e2e1-4cdb-9e4a-ee7f9df5c1a0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:10.708078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb20518-e2e1-4cdb-9e4a-ee7f9df5c1a0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:10.708366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb20518-e2e1-4cdb-9e4a-ee7f9df5c1a0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:10.708646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb20518-e2e1-4cdb-9e4a-ee7f9df5c1a0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/210781bdf2f442789e269b9bae58c2dd/config/ldap` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:10.709899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb20518-e2e1-4cdb-9e4a-ee7f9df5c1a0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=210781bdf2f442789e269b9bae58c2dd, group=ldap)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:10.711555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb20518-e2e1-4cdb-9e4a-ee7f9df5c1a0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:10.717126 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3694/7425] 10.222.0.33 () {58 vars in 1262 bytes} [Sat May 28 21:33:10 2022] GET /identity/v3/domains/210781bdf2f442789e269b9bae58c2dd/config/ldap => generated 106 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:33:10.721266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939247f-fe2f-420b-9ef2-4d9dc36b8cc6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:10.731918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939247f-fe2f-420b-9ef2-4d9dc36b8cc6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:10.736614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92c520-d460-4fa7-8b4c-d99f8c042480 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:10.737476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92c520-d460-4fa7-8b4c-d99f8c042480 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:10.737766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92c520-d460-4fa7-8b4c-d99f8c042480 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:10.738083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92c520-d460-4fa7-8b4c-d99f8c042480 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/210781bdf2f442789e269b9bae58c2dd/config/ldap/url` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:10.739868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a92c520-d460-4fa7-8b4c-d99f8c042480 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=210781bdf2f442789e269b9bae58c2dd, group=ldap, option=url)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:10.741476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a92c520-d460-4fa7-8b4c-d99f8c042480 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:10.747226 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3732/7426] 10.222.0.33 () {58 vars in 1274 bytes} [Sat May 28 21:33:10 2022] GET /identity/v3/domains/210781bdf2f442789e269b9bae58c2dd/config/ldap/url => generated 46 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:10.750568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb20518-e2e1-4cdb-9e4a-ee7f9df5c1a0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:10.763467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb20518-e2e1-4cdb-9e4a-ee7f9df5c1a0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:10.768210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76ef48-440e-42b7-902e-000ef3c130ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:10.769109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76ef48-440e-42b7-902e-000ef3c130ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:10.769392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76ef48-440e-42b7-902e-000ef3c130ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:10.769667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76ef48-440e-42b7-902e-000ef3c130ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/210781bdf2f442789e269b9bae58c2dd/config/ldap/user_tree_dn` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:10.770930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76ef48-440e-42b7-902e-000ef3c130ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=210781bdf2f442789e269b9bae58c2dd, group=ldap, option=user_tree_dn)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:10.772563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76ef48-440e-42b7-902e-000ef3c130ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:10.778590 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3695/7427] 10.222.0.33 () {58 vars in 1301 bytes} [Sat May 28 21:33:10 2022] GET /identity/v3/domains/210781bdf2f442789e269b9bae58c2dd/config/ldap/user_tree_dn => generated 63 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) May 28 21:33:10.782669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92c520-d460-4fa7-8b4c-d99f8c042480 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:10.793275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92c520-d460-4fa7-8b4c-d99f8c042480 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:10.797960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69dd46-b0fa-451f-b589-12cdb58bc7af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:10.798806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69dd46-b0fa-451f-b589-12cdb58bc7af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:10.799097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69dd46-b0fa-451f-b589-12cdb58bc7af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:10.799383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa69dd46-b0fa-451f-b589-12cdb58bc7af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/210781bdf2f442789e269b9bae58c2dd/config` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:10.800040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69dd46-b0fa-451f-b589-12cdb58bc7af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=210781bdf2f442789e269b9bae58c2dd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:10.801054 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-aa69dd46-b0fa-451f-b589-12cdb58bc7af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:33:10.801054 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:33:10.801054 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:33:10.802606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa69dd46-b0fa-451f-b589-12cdb58bc7af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:10.898250 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3733/7428] 10.222.0.33 () {58 vars in 1250 bytes} [Sat May 28 21:33:10 2022] DELETE /identity/v3/domains/210781bdf2f442789e269b9bae58c2dd/config => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:10.900223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76ef48-440e-42b7-902e-000ef3c130ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:10.911288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76ef48-440e-42b7-902e-000ef3c130ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:10.915996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f78dac-be93-4be5-8854-971ca2e66523 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:10.921970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f78dac-be93-4be5-8854-971ca2e66523 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:10.921970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f78dac-be93-4be5-8854-971ca2e66523 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:10.921970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f78dac-be93-4be5-8854-971ca2e66523 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/210781bdf2f442789e269b9bae58c2dd` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:10.921970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f78dac-be93-4be5-8854-971ca2e66523 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=210781bdf2f442789e269b9bae58c2dd)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:10.922430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f78dac-be93-4be5-8854-971ca2e66523 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:10.928669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:10.941569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f0f78dac-be93-4be5-8854-971ca2e66523 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '210781bdf2f442789e269b9bae58c2dd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:10.942259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f0f78dac-be93-4be5-8854-971ca2e66523 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '210781bdf2f442789e269b9bae58c2dd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:11.013163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f0f78dac-be93-4be5-8854-971ca2e66523 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '210781bdf2f442789e269b9bae58c2dd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:11.013873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f0f78dac-be93-4be5-8854-971ca2e66523 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '210781bdf2f442789e269b9bae58c2dd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:11.022784 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3696/7429] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:10 2022] PATCH /identity/v3/domains/210781bdf2f442789e269b9bae58c2dd => generated 277 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:11.025543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69dd46-b0fa-451f-b589-12cdb58bc7af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:11.027445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-aa69dd46-b0fa-451f-b589-12cdb58bc7af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:11.041257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa69dd46-b0fa-451f-b589-12cdb58bc7af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:11.101908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:11.102783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:11.103160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:11.103603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/210781bdf2f442789e269b9bae58c2dd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:11.114470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=210781bdf2f442789e269b9bae58c2dd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:11.116099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:11.137783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '210781bdf2f442789e269b9bae58c2dd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:11.141031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '210781bdf2f442789e269b9bae58c2dd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:11.269790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:11.276947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '210781bdf2f442789e269b9bae58c2dd', 'request_id': 'req-145bb354-1f76-49a6-866a-5920b096ea63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:11.282626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '210781bdf2f442789e269b9bae58c2dd', 'request_id': 'req-145bb354-1f76-49a6-866a-5920b096ea63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:11.414530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '210781bdf2f442789e269b9bae58c2dd', 'request_id': 'req-145bb354-1f76-49a6-866a-5920b096ea63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:11.427109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] The token cache is being invalidate because project 210781bdf2f442789e269b9bae58c2dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:11.427405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:11.428065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:11.435786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '210781bdf2f442789e269b9bae58c2dd', 'request_id': 'req-145bb354-1f76-49a6-866a-5920b096ea63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:11.436510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '210781bdf2f442789e269b9bae58c2dd', 'request_id': 'req-145bb354-1f76-49a6-866a-5920b096ea63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:11.439901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '210781bdf2f442789e269b9bae58c2dd', 'request_id': 'req-145bb354-1f76-49a6-866a-5920b096ea63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:11.447004 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3734/7430] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:11 2022] DELETE /identity/v3/domains/210781bdf2f442789e269b9bae58c2dd => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:11.452669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f78dac-be93-4be5-8854-971ca2e66523 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:11.463844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f78dac-be93-4be5-8854-971ca2e66523 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:11.522337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bc32a-281e-4d4f-a4d6-494f6c0bf45f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:11.523036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bc32a-281e-4d4f-a4d6-494f6c0bf45f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:11.523346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bc32a-281e-4d4f-a4d6-494f6c0bf45f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:11.523623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bc32a-281e-4d4f-a4d6-494f6c0bf45f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:11.524393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52bc32a-281e-4d4f-a4d6-494f6c0bf45f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:11.526012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52bc32a-281e-4d4f-a4d6-494f6c0bf45f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:11.590193 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3697/7431] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:33:11 2022] POST /identity/v3/domains => generated 274 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:11.594480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:11.596075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:11.608692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bb354-1f76-49a6-866a-5920b096ea63 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:11.667448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1315aae-16bb-4de7-a2a2-d9397f4014b9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:11.668301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1315aae-16bb-4de7-a2a2-d9397f4014b9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:11.668592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1315aae-16bb-4de7-a2a2-d9397f4014b9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:11.668877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1315aae-16bb-4de7-a2a2-d9397f4014b9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/3a23c9961dd74b44b7600e7d45cb7791/config` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:11.669628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1315aae-16bb-4de7-a2a2-d9397f4014b9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=3a23c9961dd74b44b7600e7d45cb7791)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:11.671270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1315aae-16bb-4de7-a2a2-d9397f4014b9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:11.819986 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3735/7432] 10.222.0.33 () {60 vars in 1268 bytes} [Sat May 28 21:33:11 2022] PUT /identity/v3/domains/3a23c9961dd74b44b7600e7d45cb7791/config => generated 138 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:11.823911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bc32a-281e-4d4f-a4d6-494f6c0bf45f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:11.834512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bc32a-281e-4d4f-a4d6-494f6c0bf45f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:11.839122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbe17d-ab3e-4489-9725-64e423dbbd78 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:11.840218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbe17d-ab3e-4489-9725-64e423dbbd78 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:11.840502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbe17d-ab3e-4489-9725-64e423dbbd78 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:11.840783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbe17d-ab3e-4489-9725-64e423dbbd78 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/3a23c9961dd74b44b7600e7d45cb7791/config` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:11.841570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbe17d-ab3e-4489-9725-64e423dbbd78 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=3a23c9961dd74b44b7600e7d45cb7791)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:11.846948 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-99bbe17d-ab3e-4489-9725-64e423dbbd78 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:33:11.846948 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:33:11.846948 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:33:11.846948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbe17d-ab3e-4489-9725-64e423dbbd78 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:11.940529 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3698/7433] 10.222.0.33 () {60 vars in 1270 bytes} [Sat May 28 21:33:11 2022] PATCH /identity/v3/domains/3a23c9961dd74b44b7600e7d45cb7791/config => generated 141 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:11.945230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1315aae-16bb-4de7-a2a2-d9397f4014b9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:11.955733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1315aae-16bb-4de7-a2a2-d9397f4014b9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:11.960346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da00b4f-196f-4ef5-b808-621ee215a239 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:11.961179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da00b4f-196f-4ef5-b808-621ee215a239 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:11.961465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da00b4f-196f-4ef5-b808-621ee215a239 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:11.961741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da00b4f-196f-4ef5-b808-621ee215a239 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/3a23c9961dd74b44b7600e7d45cb7791/config` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:11.963003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da00b4f-196f-4ef5-b808-621ee215a239 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=3a23c9961dd74b44b7600e7d45cb7791)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:11.964873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da00b4f-196f-4ef5-b808-621ee215a239 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:11.969472 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3736/7434] 10.222.0.33 () {58 vars in 1247 bytes} [Sat May 28 21:33:11 2022] GET /identity/v3/domains/3a23c9961dd74b44b7600e7d45cb7791/config => generated 141 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:11.973450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbe17d-ab3e-4489-9725-64e423dbbd78 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:11.983382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbe17d-ab3e-4489-9725-64e423dbbd78 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:11.988061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540e8a9-55e8-4fd9-8241-307c984aa809 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:11.989511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540e8a9-55e8-4fd9-8241-307c984aa809 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:11.989934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540e8a9-55e8-4fd9-8241-307c984aa809 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:11.990123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540e8a9-55e8-4fd9-8241-307c984aa809 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/3a23c9961dd74b44b7600e7d45cb7791/config` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:11.990777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540e8a9-55e8-4fd9-8241-307c984aa809 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=3a23c9961dd74b44b7600e7d45cb7791)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:11.991817 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-c540e8a9-55e8-4fd9-8241-307c984aa809 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:33:11.991817 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:33:11.991817 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:33:11.992708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540e8a9-55e8-4fd9-8241-307c984aa809 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:12.080808 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3699/7435] 10.222.0.33 () {58 vars in 1250 bytes} [Sat May 28 21:33:11 2022] DELETE /identity/v3/domains/3a23c9961dd74b44b7600e7d45cb7791/config => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:12.082412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da00b4f-196f-4ef5-b808-621ee215a239 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:12.093143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da00b4f-196f-4ef5-b808-621ee215a239 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:12.098302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038f6f1-108c-4693-9c7f-030edd5e013e tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:12.099419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038f6f1-108c-4693-9c7f-030edd5e013e tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:12.099689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038f6f1-108c-4693-9c7f-030edd5e013e tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:12.099990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038f6f1-108c-4693-9c7f-030edd5e013e tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/3a23c9961dd74b44b7600e7d45cb7791/config` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:12.101239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038f6f1-108c-4693-9c7f-030edd5e013e tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=3a23c9961dd74b44b7600e7d45cb7791)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:12.102968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038f6f1-108c-4693-9c7f-030edd5e013e tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:12.117964 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-7038f6f1-108c-4693-9c7f-030edd5e013e tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Could not find any options in domain configuration for domain 3a23c9961dd74b44b7600e7d45cb7791.: keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain 3a23c9961dd74b44b7600e7d45cb7791. May 28 21:33:12.117964 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:12.117964 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:12.117964 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:12.117964 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:12.117964 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:12.117964 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:12.117964 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:12.117964 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:12.117964 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:12.117964 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:12.117964 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:12.117964 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get May 28 21:33:12.118992 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) May 28 21:33:12.118992 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config May 28 21:33:12.118992 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( May 28 21:33:12.118992 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:12.118992 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:12.118992 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config May 28 21:33:12.118992 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( May 28 21:33:12.118992 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find any options in domain configuration for domain 3a23c9961dd74b44b7600e7d45cb7791. May 28 21:33:12.118992 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:12.127236 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3737/7436] 10.222.0.33 () {58 vars in 1247 bytes} [Sat May 28 21:33:12 2022] GET /identity/v3/domains/3a23c9961dd74b44b7600e7d45cb7791/config => generated 151 bytes in 37 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:33:12.127435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540e8a9-55e8-4fd9-8241-307c984aa809 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:12.134253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540e8a9-55e8-4fd9-8241-307c984aa809 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:12.139105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39f973-0505-40b2-988d-2d59c7ad3eb9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:12.139940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39f973-0505-40b2-988d-2d59c7ad3eb9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:12.140228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39f973-0505-40b2-988d-2d59c7ad3eb9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:12.140569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed39f973-0505-40b2-988d-2d59c7ad3eb9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/3a23c9961dd74b44b7600e7d45cb7791/config` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:12.141222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39f973-0505-40b2-988d-2d59c7ad3eb9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=3a23c9961dd74b44b7600e7d45cb7791)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:12.142854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed39f973-0505-40b2-988d-2d59c7ad3eb9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:12.148853 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3700/7437] 10.222.0.33 () {58 vars in 1250 bytes} [Sat May 28 21:33:12 2022] DELETE /identity/v3/domains/3a23c9961dd74b44b7600e7d45cb7791/config => generated 0 bytes in 26 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:12.158115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038f6f1-108c-4693-9c7f-030edd5e013e tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:12.168552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038f6f1-108c-4693-9c7f-030edd5e013e tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:12.173272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45735abf-af6c-4cb2-a85d-a0daef76a801 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:12.178473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45735abf-af6c-4cb2-a85d-a0daef76a801 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:12.178473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45735abf-af6c-4cb2-a85d-a0daef76a801 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:12.178473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45735abf-af6c-4cb2-a85d-a0daef76a801 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/3a23c9961dd74b44b7600e7d45cb7791` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:12.178473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45735abf-af6c-4cb2-a85d-a0daef76a801 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=3a23c9961dd74b44b7600e7d45cb7791)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:12.178835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45735abf-af6c-4cb2-a85d-a0daef76a801 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:12.200190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-45735abf-af6c-4cb2-a85d-a0daef76a801 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '3a23c9961dd74b44b7600e7d45cb7791'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:12.201059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-45735abf-af6c-4cb2-a85d-a0daef76a801 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '3a23c9961dd74b44b7600e7d45cb7791'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:12.327007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-45735abf-af6c-4cb2-a85d-a0daef76a801 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '3a23c9961dd74b44b7600e7d45cb7791'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:12.327646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-45735abf-af6c-4cb2-a85d-a0daef76a801 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '3a23c9961dd74b44b7600e7d45cb7791'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:12.330132 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3738/7438] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:12 2022] PATCH /identity/v3/domains/3a23c9961dd74b44b7600e7d45cb7791 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:12.334311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39f973-0505-40b2-988d-2d59c7ad3eb9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:12.336353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ed39f973-0505-40b2-988d-2d59c7ad3eb9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:12.350094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed39f973-0505-40b2-988d-2d59c7ad3eb9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:12.411298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:12.412174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:12.412456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:12.412733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/3a23c9961dd74b44b7600e7d45cb7791` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:12.423133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=3a23c9961dd74b44b7600e7d45cb7791)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:12.424745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:12.427610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:UPDATE=1,keystone:INSERT=4,keystone:DELETE=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:12.450739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '3a23c9961dd74b44b7600e7d45cb7791'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:12.454735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '3a23c9961dd74b44b7600e7d45cb7791'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:12.592165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a23c9961dd74b44b7600e7d45cb7791', 'request_id': 'req-127a165d-891d-4d75-9db1-278fe32b11aa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:12.608828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a23c9961dd74b44b7600e7d45cb7791', 'request_id': 'req-127a165d-891d-4d75-9db1-278fe32b11aa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:12.688818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a23c9961dd74b44b7600e7d45cb7791', 'request_id': 'req-127a165d-891d-4d75-9db1-278fe32b11aa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:12.701190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] The token cache is being invalidate because project 3a23c9961dd74b44b7600e7d45cb7791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:12.701482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:12.702170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:12.709999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '3a23c9961dd74b44b7600e7d45cb7791', 'request_id': 'req-127a165d-891d-4d75-9db1-278fe32b11aa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:12.710687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '3a23c9961dd74b44b7600e7d45cb7791', 'request_id': 'req-127a165d-891d-4d75-9db1-278fe32b11aa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:12.713823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '3a23c9961dd74b44b7600e7d45cb7791', 'request_id': 'req-127a165d-891d-4d75-9db1-278fe32b11aa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:12.719639 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3701/7439] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:12 2022] DELETE /identity/v3/domains/3a23c9961dd74b44b7600e7d45cb7791 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:12.725646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45735abf-af6c-4cb2-a85d-a0daef76a801 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:12.736550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45735abf-af6c-4cb2-a85d-a0daef76a801 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:12.759142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:12.799034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c946000-fa19-4013-b52d-694b0c5c21f5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:12.799820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c946000-fa19-4013-b52d-694b0c5c21f5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:12.800180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c946000-fa19-4013-b52d-694b0c5c21f5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:12.800535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c946000-fa19-4013-b52d-694b0c5c21f5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:12.801397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c946000-fa19-4013-b52d-694b0c5c21f5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:12.802536 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-5c946000-fa19-4013-b52d-694b0c5c21f5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:33:12.802536 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:33:12.802536 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:33:12.803618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c946000-fa19-4013-b52d-694b0c5c21f5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:12.867868 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3739/7440] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:33:12 2022] POST /identity/v3/domains => generated 276 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:12.872018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:12.873467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:12.886487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a165d-891d-4d75-9db1-278fe32b11aa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:12.945196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607ddf8-bf8c-413d-9293-71ef9dcd70d3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:12.946072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607ddf8-bf8c-413d-9293-71ef9dcd70d3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:12.946357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607ddf8-bf8c-413d-9293-71ef9dcd70d3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:12.946632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f607ddf8-bf8c-413d-9293-71ef9dcd70d3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/b19c70d319c649f1a83592410437ef0a/config` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:12.947372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607ddf8-bf8c-413d-9293-71ef9dcd70d3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:create_domain_config(domain_id=b19c70d319c649f1a83592410437ef0a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:12.948397 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-f607ddf8-bf8c-413d-9293-71ef9dcd70d3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:33:12.948397 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:33:12.948397 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:33:12.949301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f607ddf8-bf8c-413d-9293-71ef9dcd70d3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:13.081004 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3702/7441] 10.222.0.33 () {60 vars in 1268 bytes} [Sat May 28 21:33:12 2022] PUT /identity/v3/domains/b19c70d319c649f1a83592410437ef0a/config => generated 138 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:13.085727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c946000-fa19-4013-b52d-694b0c5c21f5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:13.096561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c946000-fa19-4013-b52d-694b0c5c21f5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:13.101388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19af9d5-4a6a-4823-9ca9-b8fb7901896b tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:13.102287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19af9d5-4a6a-4823-9ca9-b8fb7901896b tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:13.102580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19af9d5-4a6a-4823-9ca9-b8fb7901896b tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:13.102864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19af9d5-4a6a-4823-9ca9-b8fb7901896b tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/b19c70d319c649f1a83592410437ef0a/config/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:13.103656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19af9d5-4a6a-4823-9ca9-b8fb7901896b tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=b19c70d319c649f1a83592410437ef0a, group=identity)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:13.105064 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-d19af9d5-4a6a-4823-9ca9-b8fb7901896b tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_domain_config": "role:admin and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:33:13.105064 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:33:13.105064 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:33:13.106201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19af9d5-4a6a-4823-9ca9-b8fb7901896b tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:13.191274 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3740/7442] 10.222.0.33 () {60 vars in 1296 bytes} [Sat May 28 21:33:13 2022] PATCH /identity/v3/domains/b19c70d319c649f1a83592410437ef0a/config/identity => generated 176 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:13.205966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607ddf8-bf8c-413d-9293-71ef9dcd70d3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:13.217258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f607ddf8-bf8c-413d-9293-71ef9dcd70d3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:13.221893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1779b4-b4b4-4c87-8e7e-3ddc1fb834ea tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:13.225723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1779b4-b4b4-4c87-8e7e-3ddc1fb834ea tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:13.225723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1779b4-b4b4-4c87-8e7e-3ddc1fb834ea tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:13.225723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1779b4-b4b4-4c87-8e7e-3ddc1fb834ea tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/b19c70d319c649f1a83592410437ef0a/config/identity/driver` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:13.225723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1779b4-b4b4-4c87-8e7e-3ddc1fb834ea tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:update_domain_config(domain_id=b19c70d319c649f1a83592410437ef0a, group=identity, option=driver)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:13.226112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1779b4-b4b4-4c87-8e7e-3ddc1fb834ea tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:13.238812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:13.322931 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3703/7443] 10.222.0.33 () {60 vars in 1317 bytes} [Sat May 28 21:33:13 2022] PATCH /identity/v3/domains/b19c70d319c649f1a83592410437ef0a/config/identity/driver => generated 177 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:13.327068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19af9d5-4a6a-4823-9ca9-b8fb7901896b tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:13.337101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19af9d5-4a6a-4823-9ca9-b8fb7901896b tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:13.341946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96926909-88c7-4a9d-8da3-8f7365ff0cfe tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:13.342810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96926909-88c7-4a9d-8da3-8f7365ff0cfe tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:13.343098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96926909-88c7-4a9d-8da3-8f7365ff0cfe tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:13.343378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96926909-88c7-4a9d-8da3-8f7365ff0cfe tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/b19c70d319c649f1a83592410437ef0a/config/identity/driver` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:13.344041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96926909-88c7-4a9d-8da3-8f7365ff0cfe tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=b19c70d319c649f1a83592410437ef0a, group=identity, option=driver)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:13.345653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96926909-88c7-4a9d-8da3-8f7365ff0cfe tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:13.427485 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3741/7444] 10.222.0.33 () {58 vars in 1298 bytes} [Sat May 28 21:33:13 2022] DELETE /identity/v3/domains/b19c70d319c649f1a83592410437ef0a/config/identity/driver => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:13.431819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1779b4-b4b4-4c87-8e7e-3ddc1fb834ea tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:13.442760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1779b4-b4b4-4c87-8e7e-3ddc1fb834ea tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:13.447365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7e601-0c8e-46b7-b41e-6c58029fef52 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:13.448225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7e601-0c8e-46b7-b41e-6c58029fef52 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:13.448508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7e601-0c8e-46b7-b41e-6c58029fef52 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:13.448788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7e601-0c8e-46b7-b41e-6c58029fef52 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/b19c70d319c649f1a83592410437ef0a/config/identity/driver` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:13.450046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7e601-0c8e-46b7-b41e-6c58029fef52 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=b19c70d319c649f1a83592410437ef0a, group=identity, option=driver)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:13.451650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7e601-0c8e-46b7-b41e-6c58029fef52 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:13.466200 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-c1b7e601-0c8e-46b7-b41e-6c58029fef52 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Could not find option driver in group identity in domain configuration for domain b19c70d319c649f1a83592410437ef0a.: keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain b19c70d319c649f1a83592410437ef0a. May 28 21:33:13.466200 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:13.466200 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:13.466200 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:13.466200 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:13.466200 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:13.466200 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:13.466200 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:13.466200 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:13.466200 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:13.466200 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:13.466200 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:13.466200 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get May 28 21:33:13.467776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) May 28 21:33:13.467776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config May 28 21:33:13.467776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( May 28 21:33:13.467776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:13.467776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:13.467776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config May 28 21:33:13.467776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( May 28 21:33:13.467776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find option driver in group identity in domain configuration for domain b19c70d319c649f1a83592410437ef0a. May 28 21:33:13.467776 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:13.467776 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3704/7445] 10.222.0.33 () {58 vars in 1295 bytes} [Sat May 28 21:33:13 2022] GET /identity/v3/domains/b19c70d319c649f1a83592410437ef0a/config/identity/driver => generated 171 bytes in 36 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:33:13.471653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96926909-88c7-4a9d-8da3-8f7365ff0cfe tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:13.484906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96926909-88c7-4a9d-8da3-8f7365ff0cfe tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:13.491089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6620a7a-1945-424c-bdd5-7c83993aa9a8 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:13.491423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6620a7a-1945-424c-bdd5-7c83993aa9a8 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:13.491709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6620a7a-1945-424c-bdd5-7c83993aa9a8 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:13.491994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6620a7a-1945-424c-bdd5-7c83993aa9a8 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/b19c70d319c649f1a83592410437ef0a/config/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:13.492725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6620a7a-1945-424c-bdd5-7c83993aa9a8 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=b19c70d319c649f1a83592410437ef0a, group=identity)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:13.494382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6620a7a-1945-424c-bdd5-7c83993aa9a8 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:13.568922 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3742/7446] 10.222.0.33 () {58 vars in 1277 bytes} [Sat May 28 21:33:13 2022] DELETE /identity/v3/domains/b19c70d319c649f1a83592410437ef0a/config/identity => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:13.572927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7e601-0c8e-46b7-b41e-6c58029fef52 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:13.583050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7e601-0c8e-46b7-b41e-6c58029fef52 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:13.587579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d20983-a16f-4f54-ae74-c18d09fe2f51 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:13.588420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d20983-a16f-4f54-ae74-c18d09fe2f51 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:13.588706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d20983-a16f-4f54-ae74-c18d09fe2f51 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:13.588984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d20983-a16f-4f54-ae74-c18d09fe2f51 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/b19c70d319c649f1a83592410437ef0a/config/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:13.590476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d20983-a16f-4f54-ae74-c18d09fe2f51 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config(domain_id=b19c70d319c649f1a83592410437ef0a, group=identity)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:13.592066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d20983-a16f-4f54-ae74-c18d09fe2f51 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:13.605055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-a7d20983-a16f-4f54-ae74-c18d09fe2f51 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Could not find group identity in domain configuration for domain b19c70d319c649f1a83592410437ef0a.: keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain b19c70d319c649f1a83592410437ef0a. May 28 21:33:13.605055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:13.605055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:13.605055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:13.605055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:13.605055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:13.605055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:13.605055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:13.605055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:13.605055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:13.605055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:13.605055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:13.605055 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 179, in get May 28 21:33:13.606023 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application config = self._get_config(domain_id, group, option) May 28 21:33:13.606023 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 186, in _get_config May 28 21:33:13.606023 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return PROVIDERS.domain_config_api.get_config( May 28 21:33:13.606023 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:13.606023 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:13.606023 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 1266, in get_config May 28 21:33:13.606023 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainConfigNotFound( May 28 21:33:13.606023 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainConfigNotFound: Could not find group identity in domain configuration for domain b19c70d319c649f1a83592410437ef0a. May 28 21:33:13.606023 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:13.606567 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3705/7447] 10.222.0.33 () {58 vars in 1274 bytes} [Sat May 28 21:33:13 2022] GET /identity/v3/domains/b19c70d319c649f1a83592410437ef0a/config/identity => generated 154 bytes in 34 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:33:13.610242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6620a7a-1945-424c-bdd5-7c83993aa9a8 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:13.620828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6620a7a-1945-424c-bdd5-7c83993aa9a8 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:13.625397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a7049-ebfe-4be1-a7fd-7e561bdaeebd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:13.626264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a7049-ebfe-4be1-a7fd-7e561bdaeebd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:13.626563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a7049-ebfe-4be1-a7fd-7e561bdaeebd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:13.626836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a7049-ebfe-4be1-a7fd-7e561bdaeebd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/b19c70d319c649f1a83592410437ef0a/config` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:13.627520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a7049-ebfe-4be1-a7fd-7e561bdaeebd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:delete_domain_config(domain_id=b19c70d319c649f1a83592410437ef0a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:13.629111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424a7049-ebfe-4be1-a7fd-7e561bdaeebd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:13.717383 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3743/7448] 10.222.0.33 () {58 vars in 1250 bytes} [Sat May 28 21:33:13 2022] DELETE /identity/v3/domains/b19c70d319c649f1a83592410437ef0a/config => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:13.724337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d20983-a16f-4f54-ae74-c18d09fe2f51 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:13.734742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d20983-a16f-4f54-ae74-c18d09fe2f51 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:13.740176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fcdb28-ccb3-4603-8424-8ee8b9641ab0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:13.741038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fcdb28-ccb3-4603-8424-8ee8b9641ab0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:13.741326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fcdb28-ccb3-4603-8424-8ee8b9641ab0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:13.741638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fcdb28-ccb3-4603-8424-8ee8b9641ab0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/b19c70d319c649f1a83592410437ef0a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:13.743046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fcdb28-ccb3-4603-8424-8ee8b9641ab0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=b19c70d319c649f1a83592410437ef0a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:13.744708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fcdb28-ccb3-4603-8424-8ee8b9641ab0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:13.765197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-79fcdb28-ccb3-4603-8424-8ee8b9641ab0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'b19c70d319c649f1a83592410437ef0a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:13.765931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-79fcdb28-ccb3-4603-8424-8ee8b9641ab0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'b19c70d319c649f1a83592410437ef0a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:13.848207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-79fcdb28-ccb3-4603-8424-8ee8b9641ab0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'b19c70d319c649f1a83592410437ef0a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:13.849344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-79fcdb28-ccb3-4603-8424-8ee8b9641ab0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'b19c70d319c649f1a83592410437ef0a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:13.851808 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3706/7449] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:13 2022] PATCH /identity/v3/domains/b19c70d319c649f1a83592410437ef0a => generated 277 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:13.855999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a7049-ebfe-4be1-a7fd-7e561bdaeebd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:13.857828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-424a7049-ebfe-4be1-a7fd-7e561bdaeebd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:13.871558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a7049-ebfe-4be1-a7fd-7e561bdaeebd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:13.931636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:13.932498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:13.932783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:13.933061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/b19c70d319c649f1a83592410437ef0a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:13.944085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=b19c70d319c649f1a83592410437ef0a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:13.951311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:13.968869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'b19c70d319c649f1a83592410437ef0a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:13.970778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:13.972476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'b19c70d319c649f1a83592410437ef0a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:14.100629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b19c70d319c649f1a83592410437ef0a', 'request_id': 'req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:14.106018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b19c70d319c649f1a83592410437ef0a', 'request_id': 'req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:14.198915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b19c70d319c649f1a83592410437ef0a', 'request_id': 'req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:14.215030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] The token cache is being invalidate because project b19c70d319c649f1a83592410437ef0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:14.215329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:14.216003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:14.224601 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'b19c70d319c649f1a83592410437ef0a', 'request_id': 'req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:14.225739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'b19c70d319c649f1a83592410437ef0a', 'request_id': 'req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:14.229007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'b19c70d319c649f1a83592410437ef0a', 'request_id': 'req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:14.234776 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3744/7450] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:13 2022] DELETE /identity/v3/domains/b19c70d319c649f1a83592410437ef0a => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:14.245368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fcdb28-ccb3-4603-8424-8ee8b9641ab0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.250132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fcdb28-ccb3-4603-8424-8ee8b9641ab0 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.306719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6fd59-34ce-42d3-b916-62cdbd77dac6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.307348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6fd59-34ce-42d3-b916-62cdbd77dac6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.307637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6fd59-34ce-42d3-b916-62cdbd77dac6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.307916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6fd59-34ce-42d3-b916-62cdbd77dac6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.308558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6fd59-34ce-42d3-b916-62cdbd77dac6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.309601 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-88e6fd59-34ce-42d3-b916-62cdbd77dac6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:33:14.309601 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:33:14.309601 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:33:14.310597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6fd59-34ce-42d3-b916-62cdbd77dac6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.314924 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3707/7451] 10.222.0.33 () {58 vars in 1172 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/default => generated 1527 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:14.319342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.320770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:14.334394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f450c2-2d6d-45ae-83d0-9b9f2ad2da0f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.339073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da6ea2-2677-4d84-8d14-885984decb4d tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.345694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da6ea2-2677-4d84-8d14-885984decb4d tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.345694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da6ea2-2677-4d84-8d14-885984decb4d tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.345694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5da6ea2-2677-4d84-8d14-885984decb4d tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/identity/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.345694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da6ea2-2677-4d84-8d14-885984decb4d tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.346094 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-c5da6ea2-2677-4d84-8d14-885984decb4d tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_domain_config_default": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 28 21:33:14.346094 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:33:14.346094 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:33:14.346094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5da6ea2-2677-4d84-8d14-885984decb4d tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.346094 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3745/7452] 10.222.0.33 () {58 vars in 1199 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/identity/default => generated 64 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.349037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6fd59-34ce-42d3-b916-62cdbd77dac6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.358944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6fd59-34ce-42d3-b916-62cdbd77dac6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.364644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95728c63-221d-49e1-a86f-11da0707fec5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.365502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95728c63-221d-49e1-a86f-11da0707fec5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.365794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95728c63-221d-49e1-a86f-11da0707fec5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.366134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95728c63-221d-49e1-a86f-11da0707fec5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/identity/driver/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.366810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95728c63-221d-49e1-a86f-11da0707fec5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=driver)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.368487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95728c63-221d-49e1-a86f-11da0707fec5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.369662 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3708/7453] 10.222.0.33 () {58 vars in 1220 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/identity/driver/default => generated 30 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) May 28 21:33:14.376591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da6ea2-2677-4d84-8d14-885984decb4d tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.386840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5da6ea2-2677-4d84-8d14-885984decb4d tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.391507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18972e0-e975-4b98-87e3-9d3f532988b8 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.392314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18972e0-e975-4b98-87e3-9d3f532988b8 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.392604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18972e0-e975-4b98-87e3-9d3f532988b8 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.392889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18972e0-e975-4b98-87e3-9d3f532988b8 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/identity/list_limit/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.393579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18972e0-e975-4b98-87e3-9d3f532988b8 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=identity, option=list_limit)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.395732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18972e0-e975-4b98-87e3-9d3f532988b8 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.396840 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3746/7454] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/identity/list_limit/default => generated 33 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.400599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95728c63-221d-49e1-a86f-11da0707fec5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.416189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95728c63-221d-49e1-a86f-11da0707fec5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.421008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b68ce-73fe-427a-a730-844c60f8e14c tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.421763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b68ce-73fe-427a-a730-844c60f8e14c tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.422109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b68ce-73fe-427a-a730-844c60f8e14c tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.422394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970b68ce-73fe-427a-a730-844c60f8e14c tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.423054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b68ce-73fe-427a-a730-844c60f8e14c tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.424694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970b68ce-73fe-427a-a730-844c60f8e14c tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.426361 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3709/7455] 10.222.0.33 () {58 vars in 1187 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/default => generated 1476 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:14.430619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18972e0-e975-4b98-87e3-9d3f532988b8 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.440790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18972e0-e975-4b98-87e3-9d3f532988b8 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.446057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df08096-dd5c-4652-8ceb-ffccca1f6d77 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.446825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df08096-dd5c-4652-8ceb-ffccca1f6d77 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.447115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df08096-dd5c-4652-8ceb-ffccca1f6d77 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.447399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df08096-dd5c-4652-8ceb-ffccca1f6d77 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/url/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.448070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df08096-dd5c-4652-8ceb-ffccca1f6d77 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=url)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.449687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df08096-dd5c-4652-8ceb-ffccca1f6d77 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.451014 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3747/7456] 10.222.0.33 () {58 vars in 1199 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/url/default => generated 40 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.457353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b68ce-73fe-427a-a730-844c60f8e14c tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.468307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970b68ce-73fe-427a-a730-844c60f8e14c tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.473046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf98d4f-104c-4c09-8db2-dff39a0a5d17 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.473793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf98d4f-104c-4c09-8db2-dff39a0a5d17 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.474098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf98d4f-104c-4c09-8db2-dff39a0a5d17 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.474384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf98d4f-104c-4c09-8db2-dff39a0a5d17 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.475042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf98d4f-104c-4c09-8db2-dff39a0a5d17 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.476643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf98d4f-104c-4c09-8db2-dff39a0a5d17 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.477755 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3710/7457] 10.222.0.33 () {58 vars in 1202 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/user/default => generated 27 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.481658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df08096-dd5c-4652-8ceb-ffccca1f6d77 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.492405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df08096-dd5c-4652-8ceb-ffccca1f6d77 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.497174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241bbc8f-9d97-433c-9e3b-90ef8ec51aab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.497989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bbc8f-9d97-433c-9e3b-90ef8ec51aab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.498283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bbc8f-9d97-433c-9e3b-90ef8ec51aab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.498569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241bbc8f-9d97-433c-9e3b-90ef8ec51aab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/suffix/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.499238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241bbc8f-9d97-433c-9e3b-90ef8ec51aab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=suffix)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.500849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241bbc8f-9d97-433c-9e3b-90ef8ec51aab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.503161 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3748/7458] 10.222.0.33 () {58 vars in 1208 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/suffix/default => generated 44 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.508999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf98d4f-104c-4c09-8db2-dff39a0a5d17 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.519302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf98d4f-104c-4c09-8db2-dff39a0a5d17 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.524658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68542c-b97d-49f1-9918-1bb8f400fcad tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.525428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68542c-b97d-49f1-9918-1bb8f400fcad tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.525715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68542c-b97d-49f1-9918-1bb8f400fcad tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.526039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b68542c-b97d-49f1-9918-1bb8f400fcad tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/query_scope/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.526697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68542c-b97d-49f1-9918-1bb8f400fcad tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=query_scope)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.528296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b68542c-b97d-49f1-9918-1bb8f400fcad tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.529415 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3711/7459] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/query_scope/default => generated 35 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.537376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241bbc8f-9d97-433c-9e3b-90ef8ec51aab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.547917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241bbc8f-9d97-433c-9e3b-90ef8ec51aab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.552635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662986fb-7815-41ac-b4e9-f318fdf4c6d6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.553438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662986fb-7815-41ac-b4e9-f318fdf4c6d6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.553727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662986fb-7815-41ac-b4e9-f318fdf4c6d6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.554051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662986fb-7815-41ac-b4e9-f318fdf4c6d6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/page_size/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.558004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662986fb-7815-41ac-b4e9-f318fdf4c6d6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=page_size)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.558004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662986fb-7815-41ac-b4e9-f318fdf4c6d6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.558004 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3749/7460] 10.222.0.33 () {58 vars in 1217 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/page_size/default => generated 29 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) May 28 21:33:14.561262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68542c-b97d-49f1-9918-1bb8f400fcad tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.572098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b68542c-b97d-49f1-9918-1bb8f400fcad tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.576773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceeff88-55fd-4cb8-b971-79fb3db1385e tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.577562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceeff88-55fd-4cb8-b971-79fb3db1385e tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.577873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceeff88-55fd-4cb8-b971-79fb3db1385e tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.578174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceeff88-55fd-4cb8-b971-79fb3db1385e tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/alias_dereferencing/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.578827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceeff88-55fd-4cb8-b971-79fb3db1385e tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=alias_dereferencing)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.580505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceeff88-55fd-4cb8-b971-79fb3db1385e tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.581582 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3712/7461] 10.222.0.33 () {58 vars in 1247 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/alias_dereferencing/default => generated 47 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.586117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662986fb-7815-41ac-b4e9-f318fdf4c6d6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.600945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662986fb-7815-41ac-b4e9-f318fdf4c6d6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.605761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc5f4e-2b52-41f8-bf78-8bef2afde6c7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.606569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc5f4e-2b52-41f8-bf78-8bef2afde6c7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.606863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc5f4e-2b52-41f8-bf78-8bef2afde6c7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.607149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bc5f4e-2b52-41f8-bf78-8bef2afde6c7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/debug_level/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.608294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc5f4e-2b52-41f8-bf78-8bef2afde6c7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=debug_level)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.610259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bc5f4e-2b52-41f8-bf78-8bef2afde6c7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.611497 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3750/7462] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/debug_level/default => generated 34 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.615461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceeff88-55fd-4cb8-b971-79fb3db1385e tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.626001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceeff88-55fd-4cb8-b971-79fb3db1385e tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.631947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440a374-3740-45f5-95e4-cdf2c8d93b00 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.632095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440a374-3740-45f5-95e4-cdf2c8d93b00 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.632258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440a374-3740-45f5-95e4-cdf2c8d93b00 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.632542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e440a374-3740-45f5-95e4-cdf2c8d93b00 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/chase_referrals/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.633204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440a374-3740-45f5-95e4-cdf2c8d93b00 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=chase_referrals)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.634830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e440a374-3740-45f5-95e4-cdf2c8d93b00 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.635942 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3713/7463] 10.222.0.33 () {58 vars in 1235 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/chase_referrals/default => generated 38 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.639471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc5f4e-2b52-41f8-bf78-8bef2afde6c7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.650480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bc5f4e-2b52-41f8-bf78-8bef2afde6c7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.655234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7f972-663a-4610-bfdc-f4dbd498b082 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.657411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7f972-663a-4610-bfdc-f4dbd498b082 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.657411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7f972-663a-4610-bfdc-f4dbd498b082 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.657411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7f972-663a-4610-bfdc-f4dbd498b082 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_tree_dn/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.657411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7f972-663a-4610-bfdc-f4dbd498b082 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_tree_dn)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.658849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7f972-663a-4610-bfdc-f4dbd498b082 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.660017 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3751/7464] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/user_tree_dn/default => generated 35 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.663694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440a374-3740-45f5-95e4-cdf2c8d93b00 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.673820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e440a374-3740-45f5-95e4-cdf2c8d93b00 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.679046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01963f90-634f-445c-b531-2e18acff68af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.679825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01963f90-634f-445c-b531-2e18acff68af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.680151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01963f90-634f-445c-b531-2e18acff68af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.680461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01963f90-634f-445c-b531-2e18acff68af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_filter/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.681168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01963f90-634f-445c-b531-2e18acff68af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_filter)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.682795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01963f90-634f-445c-b531-2e18acff68af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.684116 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3714/7465] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/user_filter/default => generated 34 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.687955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7f972-663a-4610-bfdc-f4dbd498b082 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.698681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7f972-663a-4610-bfdc-f4dbd498b082 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.703454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca730608-96c7-4563-854d-33363b877285 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.704231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca730608-96c7-4563-854d-33363b877285 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.704515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca730608-96c7-4563-854d-33363b877285 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.704799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca730608-96c7-4563-854d-33363b877285 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_objectclass/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.705465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca730608-96c7-4563-854d-33363b877285 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_objectclass)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.707081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca730608-96c7-4563-854d-33363b877285 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.708914 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3752/7466] 10.222.0.33 () {58 vars in 1238 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/user_objectclass/default => generated 50 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.712586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01963f90-634f-445c-b531-2e18acff68af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.722928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01963f90-634f-445c-b531-2e18acff68af tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.727738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de157a-e3b8-4b22-ad8d-63b1d3f1f24d tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.728387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de157a-e3b8-4b22-ad8d-63b1d3f1f24d tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.728678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de157a-e3b8-4b22-ad8d-63b1d3f1f24d tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.729163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de157a-e3b8-4b22-ad8d-63b1d3f1f24d tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_id_attribute/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.729759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de157a-e3b8-4b22-ad8d-63b1d3f1f24d tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_id_attribute)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.731380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de157a-e3b8-4b22-ad8d-63b1d3f1f24d tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.733528 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3715/7467] 10.222.0.33 () {58 vars in 1241 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/user_id_attribute/default => generated 40 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.738655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca730608-96c7-4563-854d-33363b877285 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.748806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca730608-96c7-4563-854d-33363b877285 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.753767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac816dc2-c1a1-4883-9000-45e66c3a8191 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.755094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac816dc2-c1a1-4883-9000-45e66c3a8191 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.755384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac816dc2-c1a1-4883-9000-45e66c3a8191 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.755682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac816dc2-c1a1-4883-9000-45e66c3a8191 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_name_attribute/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.756344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac816dc2-c1a1-4883-9000-45e66c3a8191 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_name_attribute)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.758006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac816dc2-c1a1-4883-9000-45e66c3a8191 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.759240 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3753/7468] 10.222.0.33 () {58 vars in 1247 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/user_name_attribute/default => generated 42 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.763009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de157a-e3b8-4b22-ad8d-63b1d3f1f24d tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.773692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de157a-e3b8-4b22-ad8d-63b1d3f1f24d tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.778370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853928bb-5962-45e7-9abd-dfda8b483b1c tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.779144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853928bb-5962-45e7-9abd-dfda8b483b1c tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.779433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853928bb-5962-45e7-9abd-dfda8b483b1c tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.779714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853928bb-5962-45e7-9abd-dfda8b483b1c tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_mail_attribute/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.780371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853928bb-5962-45e7-9abd-dfda8b483b1c tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_mail_attribute)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.781987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853928bb-5962-45e7-9abd-dfda8b483b1c tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.783096 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3716/7469] 10.222.0.33 () {58 vars in 1247 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/user_mail_attribute/default => generated 44 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.786595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac816dc2-c1a1-4883-9000-45e66c3a8191 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.796469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac816dc2-c1a1-4883-9000-45e66c3a8191 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.801086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274bff1-e15b-40bf-9ae6-012874a82805 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.802405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274bff1-e15b-40bf-9ae6-012874a82805 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.802693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274bff1-e15b-40bf-9ae6-012874a82805 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.802977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274bff1-e15b-40bf-9ae6-012874a82805 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_description_attribute/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.803631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274bff1-e15b-40bf-9ae6-012874a82805 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_description_attribute)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.805197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274bff1-e15b-40bf-9ae6-012874a82805 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.806420 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3754/7470] 10.222.0.33 () {58 vars in 1268 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/user_description_attribute/default => generated 58 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) May 28 21:33:14.813078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853928bb-5962-45e7-9abd-dfda8b483b1c tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.823370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853928bb-5962-45e7-9abd-dfda8b483b1c tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.828426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ea58d-af8f-4b0c-b499-4e69f80dae13 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.829196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ea58d-af8f-4b0c-b499-4e69f80dae13 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.829566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ea58d-af8f-4b0c-b499-4e69f80dae13 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.829857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37ea58d-af8f-4b0c-b499-4e69f80dae13 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_pass_attribute/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.830530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ea58d-af8f-4b0c-b499-4e69f80dae13 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_pass_attribute)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.832179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37ea58d-af8f-4b0c-b499-4e69f80dae13 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.833285 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3717/7471] 10.222.0.33 () {58 vars in 1247 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/user_pass_attribute/default => generated 52 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.836935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274bff1-e15b-40bf-9ae6-012874a82805 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.847670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274bff1-e15b-40bf-9ae6-012874a82805 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.852561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7a4a4-12c7-41b1-ada6-9358535259bf tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.853425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7a4a4-12c7-41b1-ada6-9358535259bf tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.853716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7a4a4-12c7-41b1-ada6-9358535259bf tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.854074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7a4a4-12c7-41b1-ada6-9358535259bf tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_attribute/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.854774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7a4a4-12c7-41b1-ada6-9358535259bf tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_attribute)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.856374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7a4a4-12c7-41b1-ada6-9358535259bf tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.860269 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3755/7472] 10.222.0.33 () {58 vars in 1256 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/user_enabled_attribute/default => generated 50 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.861315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ea58d-af8f-4b0c-b499-4e69f80dae13 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.871846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37ea58d-af8f-4b0c-b499-4e69f80dae13 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.876526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41068aaf-a1fb-4649-a4ce-47ef7737abe7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.877307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41068aaf-a1fb-4649-a4ce-47ef7737abe7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.877600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41068aaf-a1fb-4649-a4ce-47ef7737abe7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.877919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41068aaf-a1fb-4649-a4ce-47ef7737abe7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_invert/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.878589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41068aaf-a1fb-4649-a4ce-47ef7737abe7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_invert)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.880795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41068aaf-a1fb-4649-a4ce-47ef7737abe7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.881921 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3718/7473] 10.222.0.33 () {58 vars in 1247 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/user_enabled_invert/default => generated 43 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.885455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7a4a4-12c7-41b1-ada6-9358535259bf tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.896021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7a4a4-12c7-41b1-ada6-9358535259bf tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.900604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905303c7-678a-4487-971a-9d903d1776c3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.901367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905303c7-678a-4487-971a-9d903d1776c3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.901652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905303c7-678a-4487-971a-9d903d1776c3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.901977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905303c7-678a-4487-971a-9d903d1776c3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_mask/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.902635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905303c7-678a-4487-971a-9d903d1776c3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_mask)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.904212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905303c7-678a-4487-971a-9d903d1776c3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.905385 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3756/7474] 10.222.0.33 () {58 vars in 1241 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/user_enabled_mask/default => generated 37 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) May 28 21:33:14.908996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41068aaf-a1fb-4649-a4ce-47ef7737abe7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.919753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41068aaf-a1fb-4649-a4ce-47ef7737abe7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.924325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecef161-754c-4729-baac-8aa28a2689b7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.925092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecef161-754c-4729-baac-8aa28a2689b7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.925382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecef161-754c-4729-baac-8aa28a2689b7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.925688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecef161-754c-4729-baac-8aa28a2689b7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_default/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.926402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecef161-754c-4729-baac-8aa28a2689b7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_default)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.927982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecef161-754c-4729-baac-8aa28a2689b7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.929118 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3719/7475] 10.222.0.33 () {58 vars in 1250 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/user_enabled_default/default => generated 45 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.932765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905303c7-678a-4487-971a-9d903d1776c3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.942974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905303c7-678a-4487-971a-9d903d1776c3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.947670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c59f1-7b8e-40ba-ae52-aad526e08472 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.948458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c59f1-7b8e-40ba-ae52-aad526e08472 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.948745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c59f1-7b8e-40ba-ae52-aad526e08472 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.949028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c59f1-7b8e-40ba-ae52-aad526e08472 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_attribute_ignore/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.949716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c59f1-7b8e-40ba-ae52-aad526e08472 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_attribute_ignore)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.951338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c59f1-7b8e-40ba-ae52-aad526e08472 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.952575 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3757/7476] 10.222.0.33 () {58 vars in 1253 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/user_attribute_ignore/default => generated 62 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.956504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecef161-754c-4729-baac-8aa28a2689b7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.967511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecef161-754c-4729-baac-8aa28a2689b7 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:14.972170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e343588-845c-4282-824c-c780126341e6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:14.972971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e343588-845c-4282-824c-c780126341e6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:14.973928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e343588-845c-4282-824c-c780126341e6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:14.974229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e343588-845c-4282-824c-c780126341e6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_default_project_id_attribute/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:14.974897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e343588-845c-4282-824c-c780126341e6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_default_project_id_attribute)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:14.976503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e343588-845c-4282-824c-c780126341e6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:14.977801 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3720/7477] 10.222.0.33 () {58 vars in 1289 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/user_default_project_id_attribute/default => generated 56 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:14.987368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c59f1-7b8e-40ba-ae52-aad526e08472 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:14.998563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c59f1-7b8e-40ba-ae52-aad526e08472 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.003196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aef7c3-f82c-4b13-8d38-01acb8f1ab38 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.003974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aef7c3-f82c-4b13-8d38-01acb8f1ab38 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.004262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aef7c3-f82c-4b13-8d38-01acb8f1ab38 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.004609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aef7c3-f82c-4b13-8d38-01acb8f1ab38 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.005265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aef7c3-f82c-4b13-8d38-01acb8f1ab38 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.006882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aef7c3-f82c-4b13-8d38-01acb8f1ab38 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.008076 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3758/7478] 10.222.0.33 () {58 vars in 1256 bytes} [Sat May 28 21:33:14 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation/default => generated 46 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.011996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e343588-845c-4282-824c-c780126341e6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.022465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e343588-845c-4282-824c-c780126341e6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.030575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1bc38-1349-4bf8-b519-2427997845ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.031399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1bc38-1349-4bf8-b519-2427997845ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.031692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1bc38-1349-4bf8-b519-2427997845ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.031978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1bc38-1349-4bf8-b519-2427997845ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_dn/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.032652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1bc38-1349-4bf8-b519-2427997845ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_dn)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.034327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1bc38-1349-4bf8-b519-2427997845ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.035508 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3721/7479] 10.222.0.33 () {58 vars in 1265 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_dn/default => generated 48 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.040357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aef7c3-f82c-4b13-8d38-01acb8f1ab38 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.050571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aef7c3-f82c-4b13-8d38-01acb8f1ab38 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.055383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e24c4-16df-406d-bdc2-2e43e90cca65 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.056236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e24c4-16df-406d-bdc2-2e43e90cca65 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.056528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e24c4-16df-406d-bdc2-2e43e90cca65 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.056932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e24c4-16df-406d-bdc2-2e43e90cca65 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.057618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e24c4-16df-406d-bdc2-2e43e90cca65 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_enabled_emulation_use_group_config)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.059291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e24c4-16df-406d-bdc2-2e43e90cca65 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.063442 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3759/7480] 10.222.0.33 () {58 vars in 1307 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/user_enabled_emulation_use_group_config/default => generated 63 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.065173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1bc38-1349-4bf8-b519-2427997845ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.075219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1bc38-1349-4bf8-b519-2427997845ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.079968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d4c8e-c209-4cd6-8db5-f826600392f6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.080812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d4c8e-c209-4cd6-8db5-f826600392f6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.081104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d4c8e-c209-4cd6-8db5-f826600392f6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.081388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467d4c8e-c209-4cd6-8db5-f826600392f6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/user_additional_attribute_mapping/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.082133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d4c8e-c209-4cd6-8db5-f826600392f6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=user_additional_attribute_mapping)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.084271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467d4c8e-c209-4cd6-8db5-f826600392f6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.085410 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3722/7481] 10.222.0.33 () {58 vars in 1289 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/user_additional_attribute_mapping/default => generated 54 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.089649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e24c4-16df-406d-bdc2-2e43e90cca65 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.100755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e24c4-16df-406d-bdc2-2e43e90cca65 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.105360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f38b6-bc10-4fab-9eb8-908735081cd5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.106170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f38b6-bc10-4fab-9eb8-908735081cd5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.106457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f38b6-bc10-4fab-9eb8-908735081cd5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.106738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79f38b6-bc10-4fab-9eb8-908735081cd5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_tree_dn/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.107402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f38b6-bc10-4fab-9eb8-908735081cd5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_tree_dn)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.108983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79f38b6-bc10-4fab-9eb8-908735081cd5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.110203 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3760/7482] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/group_tree_dn/default => generated 36 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.113870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d4c8e-c209-4cd6-8db5-f826600392f6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.123894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467d4c8e-c209-4cd6-8db5-f826600392f6 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.128682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bba2c-dc47-4646-aad7-8fba625523d3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.129441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bba2c-dc47-4646-aad7-8fba625523d3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.129732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bba2c-dc47-4646-aad7-8fba625523d3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.130043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854bba2c-dc47-4646-aad7-8fba625523d3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_filter/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.130752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854bba2c-dc47-4646-aad7-8fba625523d3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_filter)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.132456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854bba2c-dc47-4646-aad7-8fba625523d3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.133560 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3723/7483] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/group_filter/default => generated 35 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.137024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f38b6-bc10-4fab-9eb8-908735081cd5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.151197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79f38b6-bc10-4fab-9eb8-908735081cd5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.156311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cc38b-30fb-45b5-98ac-06df40884c08 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.157117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cc38b-30fb-45b5-98ac-06df40884c08 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.157411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cc38b-30fb-45b5-98ac-06df40884c08 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.157707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cc38b-30fb-45b5-98ac-06df40884c08 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_objectclass/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.158419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024cc38b-30fb-45b5-98ac-06df40884c08 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_objectclass)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.160026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024cc38b-30fb-45b5-98ac-06df40884c08 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.162056 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3761/7484] 10.222.0.33 () {58 vars in 1241 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/group_objectclass/default => generated 50 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.166186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bba2c-dc47-4646-aad7-8fba625523d3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.176670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854bba2c-dc47-4646-aad7-8fba625523d3 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.181561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14411e-9b3b-4f1a-8f3c-fcd4285b84ed tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.182511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14411e-9b3b-4f1a-8f3c-fcd4285b84ed tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.182804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14411e-9b3b-4f1a-8f3c-fcd4285b84ed tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.183092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14411e-9b3b-4f1a-8f3c-fcd4285b84ed tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_id_attribute/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.183757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14411e-9b3b-4f1a-8f3c-fcd4285b84ed tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_id_attribute)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.185393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14411e-9b3b-4f1a-8f3c-fcd4285b84ed tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.186552 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3724/7485] 10.222.0.33 () {58 vars in 1244 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/group_id_attribute/default => generated 41 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.190498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cc38b-30fb-45b5-98ac-06df40884c08 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.201295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cc38b-30fb-45b5-98ac-06df40884c08 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.205922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291e199-b456-403d-b530-ba0b858653ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.206708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291e199-b456-403d-b530-ba0b858653ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.208607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291e199-b456-403d-b530-ba0b858653ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.208607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291e199-b456-403d-b530-ba0b858653ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_name_attribute/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.208607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291e199-b456-403d-b530-ba0b858653ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_name_attribute)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.210149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291e199-b456-403d-b530-ba0b858653ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.211378 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3762/7486] 10.222.0.33 () {58 vars in 1250 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/group_name_attribute/default => generated 43 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.215531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14411e-9b3b-4f1a-8f3c-fcd4285b84ed tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.225450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14411e-9b3b-4f1a-8f3c-fcd4285b84ed tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.234248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafc354-8f94-4226-ba00-bac2d0779106 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.234248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafc354-8f94-4226-ba00-bac2d0779106 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.234248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafc354-8f94-4226-ba00-bac2d0779106 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.234671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafc354-8f94-4226-ba00-bac2d0779106 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_members_are_ids/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.234671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfafc354-8f94-4226-ba00-bac2d0779106 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_members_are_ids)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.234671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfafc354-8f94-4226-ba00-bac2d0779106 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.234999 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3725/7487] 10.222.0.33 () {58 vars in 1253 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/group_members_are_ids/default => generated 45 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.238895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291e199-b456-403d-b530-ba0b858653ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.248984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291e199-b456-403d-b530-ba0b858653ab tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.254089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443df4d6-1e77-4ea6-950f-dd4f511c7fd9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.255253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443df4d6-1e77-4ea6-950f-dd4f511c7fd9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.255533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443df4d6-1e77-4ea6-950f-dd4f511c7fd9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.255814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443df4d6-1e77-4ea6-950f-dd4f511c7fd9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_member_attribute/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.256477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443df4d6-1e77-4ea6-950f-dd4f511c7fd9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_member_attribute)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.258094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443df4d6-1e77-4ea6-950f-dd4f511c7fd9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.259434 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3763/7488] 10.222.0.33 () {58 vars in 1256 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/group_member_attribute/default => generated 49 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.264172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafc354-8f94-4226-ba00-bac2d0779106 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.275252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafc354-8f94-4226-ba00-bac2d0779106 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.279988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7c797-5555-47c4-bef7-9885bfeb1c07 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.280778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7c797-5555-47c4-bef7-9885bfeb1c07 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.281064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7c797-5555-47c4-bef7-9885bfeb1c07 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.281344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d7c797-5555-47c4-bef7-9885bfeb1c07 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_desc_attribute/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.282035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7c797-5555-47c4-bef7-9885bfeb1c07 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_desc_attribute)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.283623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d7c797-5555-47c4-bef7-9885bfeb1c07 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.284755 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3726/7489] 10.222.0.33 () {58 vars in 1250 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/group_desc_attribute/default => generated 52 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) May 28 21:33:15.294847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443df4d6-1e77-4ea6-950f-dd4f511c7fd9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.305342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443df4d6-1e77-4ea6-950f-dd4f511c7fd9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.310164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df2177-8fef-4cc6-9479-5a40e01a1bf5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.311018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df2177-8fef-4cc6-9479-5a40e01a1bf5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.311314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df2177-8fef-4cc6-9479-5a40e01a1bf5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.311605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df2177-8fef-4cc6-9479-5a40e01a1bf5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_attribute_ignore/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.312315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df2177-8fef-4cc6-9479-5a40e01a1bf5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_attribute_ignore)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.314332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df2177-8fef-4cc6-9479-5a40e01a1bf5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.317876 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3764/7490] 10.222.0.33 () {58 vars in 1256 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/group_attribute_ignore/default => generated 43 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.320254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7c797-5555-47c4-bef7-9885bfeb1c07 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.330685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d7c797-5555-47c4-bef7-9885bfeb1c07 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.336069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036706d-259a-4eb2-900c-7dfd3891f080 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.336894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036706d-259a-4eb2-900c-7dfd3891f080 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.337185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036706d-259a-4eb2-900c-7dfd3891f080 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.337467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036706d-259a-4eb2-900c-7dfd3891f080 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/group_additional_attribute_mapping/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.338216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036706d-259a-4eb2-900c-7dfd3891f080 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=group_additional_attribute_mapping)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.341415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036706d-259a-4eb2-900c-7dfd3891f080 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.342609 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3727/7491] 10.222.0.33 () {58 vars in 1292 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/group_additional_attribute_mapping/default => generated 55 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.346701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df2177-8fef-4cc6-9479-5a40e01a1bf5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.357196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df2177-8fef-4cc6-9479-5a40e01a1bf5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.362107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028b9f4-d12b-45cd-8f5b-114f141d2688 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.366431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028b9f4-d12b-45cd-8f5b-114f141d2688 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.366431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028b9f4-d12b-45cd-8f5b-114f141d2688 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.366431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028b9f4-d12b-45cd-8f5b-114f141d2688 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertfile/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.366431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a028b9f4-d12b-45cd-8f5b-114f141d2688 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertfile)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.366773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a028b9f4-d12b-45cd-8f5b-114f141d2688 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.367056 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3765/7492] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/tls_cacertfile/default => generated 37 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.370620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036706d-259a-4eb2-900c-7dfd3891f080 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.381338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036706d-259a-4eb2-900c-7dfd3891f080 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.385969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3610058-9513-4ba8-ad56-a461bc7fb4fa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.386744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3610058-9513-4ba8-ad56-a461bc7fb4fa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.387035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3610058-9513-4ba8-ad56-a461bc7fb4fa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.387322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3610058-9513-4ba8-ad56-a461bc7fb4fa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_cacertdir/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.387983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3610058-9513-4ba8-ad56-a461bc7fb4fa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_cacertdir)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.389582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3610058-9513-4ba8-ad56-a461bc7fb4fa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.390824 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3728/7493] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/tls_cacertdir/default => generated 36 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.394493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028b9f4-d12b-45cd-8f5b-114f141d2688 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.405391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a028b9f4-d12b-45cd-8f5b-114f141d2688 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.410218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898d369-e123-4b6b-8ee1-f8d1c5cc4136 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.410995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898d369-e123-4b6b-8ee1-f8d1c5cc4136 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.411284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898d369-e123-4b6b-8ee1-f8d1c5cc4136 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.411563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898d369-e123-4b6b-8ee1-f8d1c5cc4136 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_tls/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.412215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898d369-e123-4b6b-8ee1-f8d1c5cc4136 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_tls)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.413786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c898d369-e123-4b6b-8ee1-f8d1c5cc4136 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.415022 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3766/7494] 10.222.0.33 () {58 vars in 1211 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/use_tls/default => generated 31 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.418459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3610058-9513-4ba8-ad56-a461bc7fb4fa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.425946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=1,keystone:SELECT=98,keystone:UPDATE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:15.429677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3610058-9513-4ba8-ad56-a461bc7fb4fa tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.434208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5316f93-42d7-4e7b-b384-0738339f021a tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.435955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5316f93-42d7-4e7b-b384-0738339f021a tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.435955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5316f93-42d7-4e7b-b384-0738339f021a tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.435955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5316f93-42d7-4e7b-b384-0738339f021a tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/tls_req_cert/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.436216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5316f93-42d7-4e7b-b384-0738339f021a tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=tls_req_cert)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.437722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5316f93-42d7-4e7b-b384-0738339f021a tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.438970 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3729/7495] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/tls_req_cert/default => generated 39 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.443000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898d369-e123-4b6b-8ee1-f8d1c5cc4136 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.453287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c898d369-e123-4b6b-8ee1-f8d1c5cc4136 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.458786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfb3a1-a754-40a8-9524-33e4dc2f4419 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.459548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfb3a1-a754-40a8-9524-33e4dc2f4419 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.459836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfb3a1-a754-40a8-9524-33e4dc2f4419 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.460115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dfb3a1-a754-40a8-9524-33e4dc2f4419 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_pool/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.460774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dfb3a1-a754-40a8-9524-33e4dc2f4419 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_pool)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.462423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dfb3a1-a754-40a8-9524-33e4dc2f4419 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.463773 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3767/7496] 10.222.0.33 () {58 vars in 1214 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/use_pool/default => generated 31 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.467715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5316f93-42d7-4e7b-b384-0738339f021a tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.478214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5316f93-42d7-4e7b-b384-0738339f021a tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.483576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c1d05-f97d-46c1-aacf-e359e53c999f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.486869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c1d05-f97d-46c1-aacf-e359e53c999f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.486869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c1d05-f97d-46c1-aacf-e359e53c999f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.486869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c1d05-f97d-46c1-aacf-e359e53c999f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_size/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.486869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c1d05-f97d-46c1-aacf-e359e53c999f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_size)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.487284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51c1d05-f97d-46c1-aacf-e359e53c999f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.488417 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3730/7497] 10.222.0.33 () {58 vars in 1217 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/pool_size/default => generated 30 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) May 28 21:33:15.492185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfb3a1-a754-40a8-9524-33e4dc2f4419 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.502220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dfb3a1-a754-40a8-9524-33e4dc2f4419 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.507547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414f51e-086f-45f8-be8f-d9f0ee27e9cd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.508331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414f51e-086f-45f8-be8f-d9f0ee27e9cd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.509867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414f51e-086f-45f8-be8f-d9f0ee27e9cd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.509867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414f51e-086f-45f8-be8f-d9f0ee27e9cd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_max/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.509867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414f51e-086f-45f8-be8f-d9f0ee27e9cd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_max)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.511256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414f51e-086f-45f8-be8f-d9f0ee27e9cd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.515069 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3768/7498] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/pool_retry_max/default => generated 34 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.516093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c1d05-f97d-46c1-aacf-e359e53c999f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.526913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51c1d05-f97d-46c1-aacf-e359e53c999f tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.531810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa6adc-207a-4720-8554-769dcde27f23 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.532606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa6adc-207a-4720-8554-769dcde27f23 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.533021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa6adc-207a-4720-8554-769dcde27f23 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.533576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baa6adc-207a-4720-8554-769dcde27f23 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_retry_delay/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.534330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa6adc-207a-4720-8554-769dcde27f23 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_retry_delay)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.536286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baa6adc-207a-4720-8554-769dcde27f23 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.540570 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3731/7499] 10.222.0.33 () {58 vars in 1238 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/pool_retry_delay/default => generated 38 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (2 switches on core 0) May 28 21:33:15.541997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414f51e-086f-45f8-be8f-d9f0ee27e9cd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.552871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414f51e-086f-45f8-be8f-d9f0ee27e9cd tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.557737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0124a-f65e-4a95-979e-56d1ea9003a9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.558594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0124a-f65e-4a95-979e-56d1ea9003a9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.558888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0124a-f65e-4a95-979e-56d1ea9003a9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.559178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0124a-f65e-4a95-979e-56d1ea9003a9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_timeout/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.559880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0124a-f65e-4a95-979e-56d1ea9003a9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_timeout)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.561486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0124a-f65e-4a95-979e-56d1ea9003a9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.562746 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3769/7500] 10.222.0.33 () {58 vars in 1259 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/pool_connection_timeout/default => generated 44 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.570405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa6adc-207a-4720-8554-769dcde27f23 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.580354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baa6adc-207a-4720-8554-769dcde27f23 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.585216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1024de6-2e40-45d9-9603-e6a45fa5a22b tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.586044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1024de6-2e40-45d9-9603-e6a45fa5a22b tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.586332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1024de6-2e40-45d9-9603-e6a45fa5a22b tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.586613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1024de6-2e40-45d9-9603-e6a45fa5a22b tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/pool_connection_lifetime/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.587272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1024de6-2e40-45d9-9603-e6a45fa5a22b tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=pool_connection_lifetime)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.589002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1024de6-2e40-45d9-9603-e6a45fa5a22b tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.590481 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3732/7501] 10.222.0.33 () {58 vars in 1262 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/pool_connection_lifetime/default => generated 46 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.594154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0124a-f65e-4a95-979e-56d1ea9003a9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.604598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0124a-f65e-4a95-979e-56d1ea9003a9 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.609810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d80cfa-e9b4-4faa-bacf-6177eb17f626 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.610717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d80cfa-e9b4-4faa-bacf-6177eb17f626 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.611016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d80cfa-e9b4-4faa-bacf-6177eb17f626 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.611313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d80cfa-e9b4-4faa-bacf-6177eb17f626 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/use_auth_pool/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.611975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d80cfa-e9b4-4faa-bacf-6177eb17f626 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=use_auth_pool)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.613618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d80cfa-e9b4-4faa-bacf-6177eb17f626 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.618548 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3770/7502] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/use_auth_pool/default => generated 36 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.619737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1024de6-2e40-45d9-9603-e6a45fa5a22b tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.631053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1024de6-2e40-45d9-9603-e6a45fa5a22b tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.635632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28618d-74e9-45ac-8c77-b2b7c3d408f5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.636454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28618d-74e9-45ac-8c77-b2b7c3d408f5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.636738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28618d-74e9-45ac-8c77-b2b7c3d408f5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.637017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28618d-74e9-45ac-8c77-b2b7c3d408f5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_size/default` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.637668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28618d-74e9-45ac-8c77-b2b7c3d408f5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_size)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.639290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb28618d-74e9-45ac-8c77-b2b7c3d408f5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.640388 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3733/7503] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/auth_pool_size/default => generated 36 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.644140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d80cfa-e9b4-4faa-bacf-6177eb17f626 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.654671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d80cfa-e9b4-4faa-bacf-6177eb17f626 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.659791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14846c8c-7caf-41be-9a86-a874cb3164fe tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.660718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14846c8c-7caf-41be-9a86-a874cb3164fe tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.660945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14846c8c-7caf-41be-9a86-a874cb3164fe tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.661229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14846c8c-7caf-41be-9a86-a874cb3164fe tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/config/ldap/auth_pool_connection_lifetime/default` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.661970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14846c8c-7caf-41be-9a86-a874cb3164fe tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorizing `identity:get_domain_config_default(group=ldap, option=auth_pool_connection_lifetime)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.665035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14846c8c-7caf-41be-9a86-a874cb3164fe tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:15.665035 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3771/7504] 10.222.0.33 () {58 vars in 1277 bytes} [Sat May 28 21:33:15 2022] GET /identity/v3/domains/config/ldap/auth_pool_connection_lifetime/default => generated 50 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 176 bytes (1 switches on core 0) May 28 21:33:15.670365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28618d-74e9-45ac-8c77-b2b7c3d408f5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.681286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28618d-74e9-45ac-8c77-b2b7c3d408f5 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.686129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34ad2b-af8f-4e06-999d-aa519a59b93a tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.687017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34ad2b-af8f-4e06-999d-aa519a59b93a tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.687310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34ad2b-af8f-4e06-999d-aa519a59b93a tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.687596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34ad2b-af8f-4e06-999d-aa519a59b93a tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/b19c70d319c649f1a83592410437ef0a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.696721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34ad2b-af8f-4e06-999d-aa519a59b93a tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:33:15.706800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-ee34ad2b-af8f-4e06-999d-aa519a59b93a tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Could not find domain: b19c70d319c649f1a83592410437ef0a.: keystone.exception.DomainNotFound: Could not find domain: b19c70d319c649f1a83592410437ef0a. May 28 21:33:15.706800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:15.706800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 21:33:15.706800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 21:33:15.706800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:33:15.706800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:33:15.706800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:33:15.706800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:33:15.706800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: b19c70d319c649f1a83592410437ef0a. May 28 21:33:15.706800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:15.706800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:33:15.706800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:15.706800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:15.706800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:15.706800 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:15.707875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:15.707875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:15.707875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:15.707875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:15.707875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:15.707875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:15.707875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:15.707875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:15.707875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch May 28 21:33:15.707875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') May 28 21:33:15.707875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:33:15.707875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:33:15.707875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:33:15.707875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:33:15.707875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:15.707875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:15.708873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:33:15.708873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:33:15.708873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:33:15.708873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:33:15.708873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:33:15.708873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 21:33:15.708873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:33:15.708873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 21:33:15.708873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:33:15.708873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:33:15.708873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:33:15.708873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 21:33:15.708873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:33:15.708873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 21:33:15.708873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 21:33:15.708873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 21:33:15.708873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: b19c70d319c649f1a83592410437ef0a. May 28 21:33:15.710191 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:15.710191 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3734/7505] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:15 2022] PATCH /identity/v3/domains/b19c70d319c649f1a83592410437ef0a => generated 112 bytes in 38 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:33:15.712234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14846c8c-7caf-41be-9a86-a874cb3164fe tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.722581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14846c8c-7caf-41be-9a86-a874cb3164fe tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.727381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fa11e-de35-4ff8-a6f0-70bd379c77a4 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.728236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fa11e-de35-4ff8-a6f0-70bd379c77a4 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.728525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fa11e-de35-4ff8-a6f0-70bd379c77a4 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.728805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3fa11e-de35-4ff8-a6f0-70bd379c77a4 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/3a23c9961dd74b44b7600e7d45cb7791` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.731932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:15.741950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3fa11e-de35-4ff8-a6f0-70bd379c77a4 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:33:15.747849 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-ed3fa11e-de35-4ff8-a6f0-70bd379c77a4 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Could not find domain: 3a23c9961dd74b44b7600e7d45cb7791.: keystone.exception.DomainNotFound: Could not find domain: 3a23c9961dd74b44b7600e7d45cb7791. May 28 21:33:15.747849 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:15.747849 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 21:33:15.747849 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 21:33:15.747849 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:33:15.747849 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:33:15.747849 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:33:15.747849 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:33:15.747849 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 3a23c9961dd74b44b7600e7d45cb7791. May 28 21:33:15.747849 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:15.747849 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:33:15.747849 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:15.747849 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:15.747849 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:15.747849 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:15.748927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:15.748927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:15.748927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:15.748927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:15.748927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:15.748927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:15.748927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:15.748927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:15.748927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch May 28 21:33:15.748927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') May 28 21:33:15.748927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:33:15.748927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:33:15.748927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:33:15.748927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:33:15.748927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:15.748927 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:15.754371 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:33:15.754371 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:33:15.754371 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:33:15.754371 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:33:15.754371 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:33:15.754371 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 21:33:15.754371 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:33:15.754371 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 21:33:15.754371 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:33:15.754371 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:33:15.754371 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:33:15.754371 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 21:33:15.754371 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:33:15.754371 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 21:33:15.754371 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 21:33:15.754371 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 21:33:15.754371 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 3a23c9961dd74b44b7600e7d45cb7791. May 28 21:33:15.756086 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:15.756086 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3772/7506] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:15 2022] PATCH /identity/v3/domains/3a23c9961dd74b44b7600e7d45cb7791 => generated 112 bytes in 37 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) May 28 21:33:15.756955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34ad2b-af8f-4e06-999d-aa519a59b93a tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.767169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34ad2b-af8f-4e06-999d-aa519a59b93a tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.771667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc3739-538e-4a63-a06d-ebceb5f03269 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd39d4a62e6884a6799a8b034788673c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '11b18f811e944325bcf90bc62fe13694', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.772516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc3739-538e-4a63-a06d-ebceb5f03269 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.772800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc3739-538e-4a63-a06d-ebceb5f03269 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.773121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc3739-538e-4a63-a06d-ebceb5f03269 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] PATH_INFO: `/v3/domains/210781bdf2f442789e269b9bae58c2dd` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.779999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc3739-538e-4a63-a06d-ebceb5f03269 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:33:15.789907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-a6dc3739-538e-4a63-a06d-ebceb5f03269 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Could not find domain: 210781bdf2f442789e269b9bae58c2dd.: keystone.exception.DomainNotFound: Could not find domain: 210781bdf2f442789e269b9bae58c2dd. May 28 21:33:15.789907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:15.789907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 21:33:15.789907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 21:33:15.789907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:33:15.789907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:33:15.789907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:33:15.789907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:33:15.789907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 210781bdf2f442789e269b9bae58c2dd. May 28 21:33:15.789907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:15.789907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:33:15.789907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:15.789907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:15.789907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:15.789907 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:15.790970 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:15.790970 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:15.790970 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:15.790970 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:15.790970 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:15.790970 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:15.790970 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:15.790970 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:15.790970 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch May 28 21:33:15.790970 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') May 28 21:33:15.790970 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:33:15.790970 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:33:15.790970 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:33:15.790970 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:33:15.790970 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:15.790970 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:15.792245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:33:15.792245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:33:15.792245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:33:15.792245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:33:15.792245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:33:15.792245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 21:33:15.792245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:33:15.792245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 21:33:15.792245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:33:15.792245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:33:15.792245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:33:15.792245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 21:33:15.792245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:33:15.792245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 21:33:15.792245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 21:33:15.792245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 21:33:15.792245 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 210781bdf2f442789e269b9bae58c2dd. May 28 21:33:15.793371 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:15.793695 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3735/7507] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:15 2022] PATCH /identity/v3/domains/210781bdf2f442789e269b9bae58c2dd => generated 112 bytes in 35 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) May 28 21:33:15.794884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fa11e-de35-4ff8-a6f0-70bd379c77a4 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:15.796608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ed3fa11e-de35-4ff8-a6f0-70bd379c77a4 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:15.864752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3fa11e-de35-4ff8-a6f0-70bd379c77a4 tempest-DomainConfigurationTestJSON-485737103 tempest-DomainConfigurationTestJSON-485737103-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:15.869594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5b2ad-3987-4aa8-a8ce-caf4007b7992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:15.870499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5b2ad-3987-4aa8-a8ce-caf4007b7992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:15.870790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5b2ad-3987-4aa8-a8ce-caf4007b7992 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:15.871067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5b2ad-3987-4aa8-a8ce-caf4007b7992 admin admin] PATH_INFO: `/v3/users/a61283b09f6c47118cd431e8e2696b0e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:15.872284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5b2ad-3987-4aa8-a8ce-caf4007b7992 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a61283b09f6c47118cd431e8e2696b0e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:15.874257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5b2ad-3987-4aa8-a8ce-caf4007b7992 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:16.137289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0cc5b2ad-3987-4aa8-a8ce-caf4007b7992 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a61283b09f6c47118cd431e8e2696b0e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:16.144244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0cc5b2ad-3987-4aa8-a8ce-caf4007b7992 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a61283b09f6c47118cd431e8e2696b0e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:16.144947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0cc5b2ad-3987-4aa8-a8ce-caf4007b7992 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a61283b09f6c47118cd431e8e2696b0e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:16.145566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0cc5b2ad-3987-4aa8-a8ce-caf4007b7992 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a61283b09f6c47118cd431e8e2696b0e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:16.203787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0cc5b2ad-3987-4aa8-a8ce-caf4007b7992 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a61283b09f6c47118cd431e8e2696b0e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:16.225354 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3773/7508] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:15 2022] DELETE /identity/v3/users/a61283b09f6c47118cd431e8e2696b0e => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:16.229795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76052dda-915e-4275-8384-041e383772d6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:16.230127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76052dda-915e-4275-8384-041e383772d6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:16.230429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76052dda-915e-4275-8384-041e383772d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:16.249507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-76052dda-915e-4275-8384-041e383772d6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:16.250764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-76052dda-915e-4275-8384-041e383772d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:16.310333 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3736/7509] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:16.320051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5b2ad-3987-4aa8-a8ce-caf4007b7992 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:16.321528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0cc5b2ad-3987-4aa8-a8ce-caf4007b7992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:16.383288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5b2ad-3987-4aa8-a8ce-caf4007b7992 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:16.386881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029116a3-1576-4e3b-b087-9eae0cceadb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:16.387524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029116a3-1576-4e3b-b087-9eae0cceadb7 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:16.387809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029116a3-1576-4e3b-b087-9eae0cceadb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:16.388087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029116a3-1576-4e3b-b087-9eae0cceadb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:16.397765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029116a3-1576-4e3b-b087-9eae0cceadb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:16.399218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029116a3-1576-4e3b-b087-9eae0cceadb7 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:16.408648 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3774/7510] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:33:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:33:16.912194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76052dda-915e-4275-8384-041e383772d6 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:16.913767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-76052dda-915e-4275-8384-041e383772d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:16.927985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76052dda-915e-4275-8384-041e383772d6 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:16.933197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a42f8-f40d-4126-a58f-55125592a51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:16.934392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a42f8-f40d-4126-a58f-55125592a51f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:16.934541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a42f8-f40d-4126-a58f-55125592a51f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:16.934817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a42f8-f40d-4126-a58f-55125592a51f admin admin] PATH_INFO: `/v3/projects/284c98cdbd0645388b350270ee5d61e0` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:16.936009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a42f8-f40d-4126-a58f-55125592a51f admin admin] RBAC: Authorizing `identity:delete_project(project_id=284c98cdbd0645388b350270ee5d61e0)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:16.938086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a42f8-f40d-4126-a58f-55125592a51f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:17.055759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c73a42f8-f40d-4126-a58f-55125592a51f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '284c98cdbd0645388b350270ee5d61e0', 'request_id': 'req-c73a42f8-f40d-4126-a58f-55125592a51f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:17.061418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c73a42f8-f40d-4126-a58f-55125592a51f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '284c98cdbd0645388b350270ee5d61e0', 'request_id': 'req-c73a42f8-f40d-4126-a58f-55125592a51f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:17.160745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c73a42f8-f40d-4126-a58f-55125592a51f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '284c98cdbd0645388b350270ee5d61e0', 'request_id': 'req-c73a42f8-f40d-4126-a58f-55125592a51f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:17.175916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c73a42f8-f40d-4126-a58f-55125592a51f admin admin] The token cache is being invalidate because project 284c98cdbd0645388b350270ee5d61e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:17.176226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c73a42f8-f40d-4126-a58f-55125592a51f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:17.177474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c73a42f8-f40d-4126-a58f-55125592a51f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:17.179248 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3737/7511] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:16 2022] DELETE /identity/v3/projects/284c98cdbd0645388b350270ee5d61e0 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:17.184377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029116a3-1576-4e3b-b087-9eae0cceadb7 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:17.203987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029116a3-1576-4e3b-b087-9eae0cceadb7 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:17.229328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:17.270897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491d11a3-7217-4bd5-b302-6c3ac8dc1d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:17.271784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d11a3-7217-4bd5-b302-6c3ac8dc1d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:17.272067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d11a3-7217-4bd5-b302-6c3ac8dc1d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:17.272344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491d11a3-7217-4bd5-b302-6c3ac8dc1d54 admin admin] PATH_INFO: `/v3/users/d39d4a62e6884a6799a8b034788673c0` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:17.273544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d11a3-7217-4bd5-b302-6c3ac8dc1d54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d39d4a62e6884a6799a8b034788673c0)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:17.276016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491d11a3-7217-4bd5-b302-6c3ac8dc1d54 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:17.579178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-491d11a3-7217-4bd5-b302-6c3ac8dc1d54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd39d4a62e6884a6799a8b034788673c0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:17.588283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-491d11a3-7217-4bd5-b302-6c3ac8dc1d54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd39d4a62e6884a6799a8b034788673c0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:17.589029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-491d11a3-7217-4bd5-b302-6c3ac8dc1d54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd39d4a62e6884a6799a8b034788673c0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:17.589965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-491d11a3-7217-4bd5-b302-6c3ac8dc1d54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd39d4a62e6884a6799a8b034788673c0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:17.687732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-491d11a3-7217-4bd5-b302-6c3ac8dc1d54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd39d4a62e6884a6799a8b034788673c0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:17.706393 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3775/7512] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:17 2022] DELETE /identity/v3/users/d39d4a62e6884a6799a8b034788673c0 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:18.165868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a42f8-f40d-4126-a58f-55125592a51f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:18.167333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c73a42f8-f40d-4126-a58f-55125592a51f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:18.180367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a42f8-f40d-4126-a58f-55125592a51f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:18.228010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:18.240584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae6c57-3d70-4de3-94cc-9b452e88e7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:18.243304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae6c57-3d70-4de3-94cc-9b452e88e7fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:18.243304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae6c57-3d70-4de3-94cc-9b452e88e7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:18.243304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae6c57-3d70-4de3-94cc-9b452e88e7fa admin admin] PATH_INFO: `/v3/projects/11b18f811e944325bcf90bc62fe13694` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:18.243574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae6c57-3d70-4de3-94cc-9b452e88e7fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=11b18f811e944325bcf90bc62fe13694)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:18.245382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae6c57-3d70-4de3-94cc-9b452e88e7fa admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:18.393170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-66ae6c57-3d70-4de3-94cc-9b452e88e7fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11b18f811e944325bcf90bc62fe13694', 'request_id': 'req-66ae6c57-3d70-4de3-94cc-9b452e88e7fa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:18.399241 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-66ae6c57-3d70-4de3-94cc-9b452e88e7fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11b18f811e944325bcf90bc62fe13694', 'request_id': 'req-66ae6c57-3d70-4de3-94cc-9b452e88e7fa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:18.472115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-66ae6c57-3d70-4de3-94cc-9b452e88e7fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11b18f811e944325bcf90bc62fe13694', 'request_id': 'req-66ae6c57-3d70-4de3-94cc-9b452e88e7fa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:18.483948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-66ae6c57-3d70-4de3-94cc-9b452e88e7fa admin admin] The token cache is being invalidate because project 11b18f811e944325bcf90bc62fe13694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:18.484329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-66ae6c57-3d70-4de3-94cc-9b452e88e7fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:18.484973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-66ae6c57-3d70-4de3-94cc-9b452e88e7fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:18.486586 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3738/7513] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:18 2022] DELETE /identity/v3/projects/11b18f811e944325bcf90bc62fe13694 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:18.494441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1923d-beba-4905-bda8-545052181f95 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:18.494441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1923d-beba-4905-bda8-545052181f95 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:18.494929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb1923d-beba-4905-bda8-545052181f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:18.521499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7eb1923d-beba-4905-bda8-545052181f95 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:18.524837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7eb1923d-beba-4905-bda8-545052181f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:18.590217 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3776/7514] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:18.598013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced4cd6-56d6-41a9-8437-848dcc6d66f3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:18.598925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced4cd6-56d6-41a9-8437-848dcc6d66f3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:18.599254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced4cd6-56d6-41a9-8437-848dcc6d66f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:18.620675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1ced4cd6-56d6-41a9-8437-848dcc6d66f3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:18.622126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1ced4cd6-56d6-41a9-8437-848dcc6d66f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:18.635501 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3739/7515] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:18.639827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1923d-beba-4905-bda8-545052181f95 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:18.650884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb1923d-beba-4905-bda8-545052181f95 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:18.654953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efc641-466a-4774-a71e-fb296ce7c8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:18.655662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efc641-466a-4774-a71e-fb296ce7c8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:18.655983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efc641-466a-4774-a71e-fb296ce7c8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:18.656260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efc641-466a-4774-a71e-fb296ce7c8d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:18.657009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efc641-466a-4774-a71e-fb296ce7c8d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:33:18.657361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efc641-466a-4774-a71e-fb296ce7c8d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:18.659004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efc641-466a-4774-a71e-fb296ce7c8d9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:18.672836 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3777/7516] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:18.677084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced4cd6-56d6-41a9-8437-848dcc6d66f3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:18.686786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced4cd6-56d6-41a9-8437-848dcc6d66f3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:18.691706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460930e3-d285-4e7e-bf92-ff87e8fbf884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:18.692393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460930e3-d285-4e7e-bf92-ff87e8fbf884 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:18.692801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460930e3-d285-4e7e-bf92-ff87e8fbf884 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:18.693076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460930e3-d285-4e7e-bf92-ff87e8fbf884 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:18.693935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460930e3-d285-4e7e-bf92-ff87e8fbf884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:18.695841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460930e3-d285-4e7e-bf92-ff87e8fbf884 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:18.781566 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3740/7517] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:18 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:18.785867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efc641-466a-4774-a71e-fb296ce7c8d9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:18.795969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efc641-466a-4774-a71e-fb296ce7c8d9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:18.823359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:18.853219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac2f190-a178-4ff4-ab44-722f3270f204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:18.853959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac2f190-a178-4ff4-ab44-722f3270f204 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:18.854247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac2f190-a178-4ff4-ab44-722f3270f204 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:18.854527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac2f190-a178-4ff4-ab44-722f3270f204 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:18.855225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac2f190-a178-4ff4-ab44-722f3270f204 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:18.857171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac2f190-a178-4ff4-ab44-722f3270f204 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:18.956624 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3778/7518] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:18 2022] POST /identity/v3/users => generated 283 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:18.960714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460930e3-d285-4e7e-bf92-ff87e8fbf884 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:18.971695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460930e3-d285-4e7e-bf92-ff87e8fbf884 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:18.977073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e52952-3126-45c4-88ea-69c70025a2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:18.977733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e52952-3126-45c4-88ea-69c70025a2be admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:18.978059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e52952-3126-45c4-88ea-69c70025a2be admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:18.978343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e52952-3126-45c4-88ea-69c70025a2be admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:18.979116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e52952-3126-45c4-88ea-69c70025a2be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:18.980849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e52952-3126-45c4-88ea-69c70025a2be admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:18.991812 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3741/7519] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:18.995694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac2f190-a178-4ff4-ab44-722f3270f204 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:19.005781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac2f190-a178-4ff4-ab44-722f3270f204 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:19.010607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77427d78-3b09-4ee2-8771-ebe11ad78ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:19.011333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77427d78-3b09-4ee2-8771-ebe11ad78ebc admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:19.011708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77427d78-3b09-4ee2-8771-ebe11ad78ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:19.011988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77427d78-3b09-4ee2-8771-ebe11ad78ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:19.012723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77427d78-3b09-4ee2-8771-ebe11ad78ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:19.014364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77427d78-3b09-4ee2-8771-ebe11ad78ebc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:19.025144 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3779/7520] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:19.028917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e52952-3126-45c4-88ea-69c70025a2be admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:19.039436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e52952-3126-45c4-88ea-69c70025a2be admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:19.044148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127de2c-4e41-451d-9d0e-6febbe2dccc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:19.045019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127de2c-4e41-451d-9d0e-6febbe2dccc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:19.045301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127de2c-4e41-451d-9d0e-6febbe2dccc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:19.045575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127de2c-4e41-451d-9d0e-6febbe2dccc4 admin admin] PATH_INFO: `/v3/projects/31effd15ea534ccc8744bec9ee1fe055/users/0228d086c0d64996a8047184a6c6a46d/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:19.071737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127de2c-4e41-451d-9d0e-6febbe2dccc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31effd15ea534ccc8744bec9ee1fe055, user_id=0228d086c0d64996a8047184a6c6a46d, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:19.074625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127de2c-4e41-451d-9d0e-6febbe2dccc4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:19.184984 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3742/7521] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:19 2022] PUT /identity/v3/projects/31effd15ea534ccc8744bec9ee1fe055/users/0228d086c0d64996a8047184a6c6a46d/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:19.195733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f14844-7bf1-4512-9be6-e7f1af1254a4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:19.196047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f14844-7bf1-4512-9be6-e7f1af1254a4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:19.196330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f14844-7bf1-4512-9be6-e7f1af1254a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:19.223458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d0f14844-7bf1-4512-9be6-e7f1af1254a4 None None] MFA Rules not processed for user `0228d086c0d64996a8047184a6c6a46d`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:19.224667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d0f14844-7bf1-4512-9be6-e7f1af1254a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:19.289976 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3780/7522] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:19 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:19.298443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127de2c-4e41-451d-9d0e-6febbe2dccc4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:19.367821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127de2c-4e41-451d-9d0e-6febbe2dccc4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:19.372299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e8c03-7783-410c-891f-3972931bef67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:19.373053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e8c03-7783-410c-891f-3972931bef67 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:19.373335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e8c03-7783-410c-891f-3972931bef67 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:19.373614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e8c03-7783-410c-891f-3972931bef67 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:19.374342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e8c03-7783-410c-891f-3972931bef67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:19.376348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e8c03-7783-410c-891f-3972931bef67 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:19.491599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:19.496587 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3743/7523] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:19 2022] POST /identity/v3/projects => generated 367 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:19.500414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f14844-7bf1-4512-9be6-e7f1af1254a4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:19.512090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f14844-7bf1-4512-9be6-e7f1af1254a4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:19.571785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7d39d-802c-4b22-b276-18a6049d0389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:19.572559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7d39d-802c-4b22-b276-18a6049d0389 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:19.572844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7d39d-802c-4b22-b276-18a6049d0389 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:19.573121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d7d39d-802c-4b22-b276-18a6049d0389 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:19.573822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7d39d-802c-4b22-b276-18a6049d0389 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:19.575809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d7d39d-802c-4b22-b276-18a6049d0389 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:19.678753 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3781/7524] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:19 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:19.682999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e8c03-7783-410c-891f-3972931bef67 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:19.693468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e8c03-7783-410c-891f-3972931bef67 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:19.700085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81582b-1201-489b-9abe-1ba36723da45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:19.700085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81582b-1201-489b-9abe-1ba36723da45 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:19.700085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81582b-1201-489b-9abe-1ba36723da45 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:19.700085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81582b-1201-489b-9abe-1ba36723da45 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:19.700745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81582b-1201-489b-9abe-1ba36723da45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:19.702624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81582b-1201-489b-9abe-1ba36723da45 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:19.715395 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3744/7525] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:19.726510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7d39d-802c-4b22-b276-18a6049d0389 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:19.736800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d7d39d-802c-4b22-b276-18a6049d0389 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:19.742237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c246dd6-865d-470d-b493-422a6e9d2318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:19.743125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c246dd6-865d-470d-b493-422a6e9d2318 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:19.743410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c246dd6-865d-470d-b493-422a6e9d2318 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:19.743688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c246dd6-865d-470d-b493-422a6e9d2318 admin admin] PATH_INFO: `/v3/projects/b47788cd7cb8432aa3527c703bc16704/users/4fa7d879b17b46eeb13a829b7b5a9d93/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:19.758725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c246dd6-865d-470d-b493-422a6e9d2318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b47788cd7cb8432aa3527c703bc16704, user_id=4fa7d879b17b46eeb13a829b7b5a9d93, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:19.761427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c246dd6-865d-470d-b493-422a6e9d2318 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:19.889359 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3782/7526] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:19 2022] PUT /identity/v3/projects/b47788cd7cb8432aa3527c703bc16704/users/4fa7d879b17b46eeb13a829b7b5a9d93/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:19.893400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81582b-1201-489b-9abe-1ba36723da45 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:19.903564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81582b-1201-489b-9abe-1ba36723da45 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:19.961681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf845b2-d459-428a-a19b-23e8685f2125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:19.962379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf845b2-d459-428a-a19b-23e8685f2125 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:19.962667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf845b2-d459-428a-a19b-23e8685f2125 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:19.962945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf845b2-d459-428a-a19b-23e8685f2125 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:19.963664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf845b2-d459-428a-a19b-23e8685f2125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:19.965578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf845b2-d459-428a-a19b-23e8685f2125 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:19.975700 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3745/7527] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:19.979975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c246dd6-865d-470d-b493-422a6e9d2318 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:19.986576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:19.992755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c246dd6-865d-470d-b493-422a6e9d2318 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:19.997500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3fcbd8-5e6f-4dbb-bbdf-cdc68c0f6bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:19.998422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fcbd8-5e6f-4dbb-bbdf-cdc68c0f6bf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:19.998710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fcbd8-5e6f-4dbb-bbdf-cdc68c0f6bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:19.998992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3fcbd8-5e6f-4dbb-bbdf-cdc68c0f6bf4 admin admin] PATH_INFO: `/v3/projects/b47788cd7cb8432aa3527c703bc16704/users/4fa7d879b17b46eeb13a829b7b5a9d93/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:20.001027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3fcbd8-5e6f-4dbb-bbdf-cdc68c0f6bf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b47788cd7cb8432aa3527c703bc16704, user_id=4fa7d879b17b46eeb13a829b7b5a9d93, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:20.003858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3fcbd8-5e6f-4dbb-bbdf-cdc68c0f6bf4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:20.014278 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3783/7528] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:19 2022] PUT /identity/v3/projects/b47788cd7cb8432aa3527c703bc16704/users/4fa7d879b17b46eeb13a829b7b5a9d93/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:20.027542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe0960-6644-452b-bdfe-3a9c89165d14 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:20.027542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe0960-6644-452b-bdfe-3a9c89165d14 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:20.027542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe0960-6644-452b-bdfe-3a9c89165d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:20.054339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e7fe0960-6644-452b-bdfe-3a9c89165d14 None None] MFA Rules not processed for user `4fa7d879b17b46eeb13a829b7b5a9d93`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:20.055593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e7fe0960-6644-452b-bdfe-3a9c89165d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:20.127244 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3746/7529] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:20 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:20.131864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3fcbd8-5e6f-4dbb-bbdf-cdc68c0f6bf4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:20.142605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3fcbd8-5e6f-4dbb-bbdf-cdc68c0f6bf4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:20.147759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f5c87-d46a-4394-acdb-e6449209ceac tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:20.148522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f5c87-d46a-4394-acdb-e6449209ceac tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:20.148812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f5c87-d46a-4394-acdb-e6449209ceac tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:20.149111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f5c87-d46a-4394-acdb-e6449209ceac tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:20.149914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f5c87-d46a-4394-acdb-e6449209ceac tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:20.153900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f5c87-d46a-4394-acdb-e6449209ceac tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:20.233145 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3784/7530] 10.222.0.33 () {60 vars in 1149 bytes} [Sat May 28 21:33:20 2022] POST /identity/v3/domains => generated 277 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:20.237198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe0960-6644-452b-bdfe-3a9c89165d14 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:20.246824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe0960-6644-452b-bdfe-3a9c89165d14 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:20.303858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee7d4b-2dd0-48c1-bee6-259fc9879ee3 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:20.304553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee7d4b-2dd0-48c1-bee6-259fc9879ee3 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:20.304846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee7d4b-2dd0-48c1-bee6-259fc9879ee3 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:20.305158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afee7d4b-2dd0-48c1-bee6-259fc9879ee3 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:20.306004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee7d4b-2dd0-48c1-bee6-259fc9879ee3 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:20.307655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afee7d4b-2dd0-48c1-bee6-259fc9879ee3 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:20.433854 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3747/7531] 10.222.0.33 () {60 vars in 1149 bytes} [Sat May 28 21:33:20 2022] POST /identity/v3/domains => generated 277 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:33:20.437871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f5c87-d46a-4394-acdb-e6449209ceac tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:20.449164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f5c87-d46a-4394-acdb-e6449209ceac tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:20.506844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710081e-8360-4d8c-b322-68e3d1f4bd03 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:20.507473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710081e-8360-4d8c-b322-68e3d1f4bd03 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:20.507684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710081e-8360-4d8c-b322-68e3d1f4bd03 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:20.508154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710081e-8360-4d8c-b322-68e3d1f4bd03 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:20.508703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710081e-8360-4d8c-b322-68e3d1f4bd03 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:20.510364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710081e-8360-4d8c-b322-68e3d1f4bd03 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:20.621579 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3785/7532] 10.222.0.33 () {60 vars in 1149 bytes} [Sat May 28 21:33:20 2022] POST /identity/v3/domains => generated 278 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:20.622556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee7d4b-2dd0-48c1-bee6-259fc9879ee3 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:20.629401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:20.634190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afee7d4b-2dd0-48c1-bee6-259fc9879ee3 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:20.694926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec5621-0840-4798-bc93-50877063b002 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:20.695593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec5621-0840-4798-bc93-50877063b002 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:20.695879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec5621-0840-4798-bc93-50877063b002 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:20.698715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ec5621-0840-4798-bc93-50877063b002 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:20.698715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec5621-0840-4798-bc93-50877063b002 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:20.698715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ec5621-0840-4798-bc93-50877063b002 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:20.867719 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3748/7533] 10.222.0.33 () {60 vars in 1149 bytes} [Sat May 28 21:33:20 2022] POST /identity/v3/domains => generated 279 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:33:20.872128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710081e-8360-4d8c-b322-68e3d1f4bd03 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:20.883486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710081e-8360-4d8c-b322-68e3d1f4bd03 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:20.938835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:20.939700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:20.939983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:20.940259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/678f9c486da24aee86bd930938fe3430` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:20.951160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=678f9c486da24aee86bd930938fe3430)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:20.952794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:20.975748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '678f9c486da24aee86bd930938fe3430'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:20.978994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '678f9c486da24aee86bd930938fe3430'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:21.062537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:21.070083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '678f9c486da24aee86bd930938fe3430', 'request_id': 'req-15f7511a-3b58-46b6-a694-72a1673754cd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:21.075537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '678f9c486da24aee86bd930938fe3430', 'request_id': 'req-15f7511a-3b58-46b6-a694-72a1673754cd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:21.198170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '678f9c486da24aee86bd930938fe3430', 'request_id': 'req-15f7511a-3b58-46b6-a694-72a1673754cd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:21.210961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] The token cache is being invalidate because project 678f9c486da24aee86bd930938fe3430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:21.211257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:21.214890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:21.219533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '678f9c486da24aee86bd930938fe3430', 'request_id': 'req-15f7511a-3b58-46b6-a694-72a1673754cd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:21.220188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '678f9c486da24aee86bd930938fe3430', 'request_id': 'req-15f7511a-3b58-46b6-a694-72a1673754cd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:21.223498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '678f9c486da24aee86bd930938fe3430', 'request_id': 'req-15f7511a-3b58-46b6-a694-72a1673754cd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:21.229881 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3786/7534] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:20 2022] DELETE /identity/v3/domains/678f9c486da24aee86bd930938fe3430 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:21.234735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec5621-0840-4798-bc93-50877063b002 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:21.236504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-87ec5621-0840-4798-bc93-50877063b002 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:21.249688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ec5621-0840-4798-bc93-50877063b002 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:21.309746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70ad48-2592-4b26-ac8c-b95007edd216 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:21.310519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ad48-2592-4b26-ac8c-b95007edd216 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:21.310809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ad48-2592-4b26-ac8c-b95007edd216 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:21.311090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70ad48-2592-4b26-ac8c-b95007edd216 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:21.311831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70ad48-2592-4b26-ac8c-b95007edd216 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:21.313461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70ad48-2592-4b26-ac8c-b95007edd216 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:21.468303 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3749/7535] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:33:21 2022] POST /identity/v3/domains => generated 249 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:33:21.472218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:21.482603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f7511a-3b58-46b6-a694-72a1673754cd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:21.539275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47e044-bdc4-46f1-b5fa-5315de9e7434 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:21.540147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47e044-bdc4-46f1-b5fa-5315de9e7434 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:21.540435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47e044-bdc4-46f1-b5fa-5315de9e7434 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:21.540717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da47e044-bdc4-46f1-b5fa-5315de9e7434 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/67f1a1efec994a2e82153d1d41397a05` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:21.552084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47e044-bdc4-46f1-b5fa-5315de9e7434 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=67f1a1efec994a2e82153d1d41397a05)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:21.553744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da47e044-bdc4-46f1-b5fa-5315de9e7434 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:21.575068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da47e044-bdc4-46f1-b5fa-5315de9e7434 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '67f1a1efec994a2e82153d1d41397a05'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:21.575789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da47e044-bdc4-46f1-b5fa-5315de9e7434 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '67f1a1efec994a2e82153d1d41397a05'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:21.708815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da47e044-bdc4-46f1-b5fa-5315de9e7434 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '67f1a1efec994a2e82153d1d41397a05'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:21.709487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da47e044-bdc4-46f1-b5fa-5315de9e7434 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '67f1a1efec994a2e82153d1d41397a05'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:21.712205 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3787/7536] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:21 2022] PATCH /identity/v3/domains/67f1a1efec994a2e82153d1d41397a05 => generated 250 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:21.717918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70ad48-2592-4b26-ac8c-b95007edd216 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:21.719691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9a70ad48-2592-4b26-ac8c-b95007edd216 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:21.732323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70ad48-2592-4b26-ac8c-b95007edd216 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:21.796303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:21.797193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:21.797480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:21.797757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/67f1a1efec994a2e82153d1d41397a05` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:21.811753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=67f1a1efec994a2e82153d1d41397a05)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:21.813477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:21.836775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '67f1a1efec994a2e82153d1d41397a05'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:21.840298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '67f1a1efec994a2e82153d1d41397a05'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:21.841489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:21.964584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67f1a1efec994a2e82153d1d41397a05', 'request_id': 'req-933f4540-5c67-4961-bff1-3b4e520d3ba6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:21.972454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67f1a1efec994a2e82153d1d41397a05', 'request_id': 'req-933f4540-5c67-4961-bff1-3b4e520d3ba6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.071769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67f1a1efec994a2e82153d1d41397a05', 'request_id': 'req-933f4540-5c67-4961-bff1-3b4e520d3ba6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.085193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] The token cache is being invalidate because project 67f1a1efec994a2e82153d1d41397a05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:22.085487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.086158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.093934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '67f1a1efec994a2e82153d1d41397a05', 'request_id': 'req-933f4540-5c67-4961-bff1-3b4e520d3ba6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.094585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '67f1a1efec994a2e82153d1d41397a05', 'request_id': 'req-933f4540-5c67-4961-bff1-3b4e520d3ba6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.097893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '67f1a1efec994a2e82153d1d41397a05', 'request_id': 'req-933f4540-5c67-4961-bff1-3b4e520d3ba6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.103494 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3750/7537] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:21 2022] DELETE /identity/v3/domains/67f1a1efec994a2e82153d1d41397a05 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:22.108412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47e044-bdc4-46f1-b5fa-5315de9e7434 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:22.119515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da47e044-bdc4-46f1-b5fa-5315de9e7434 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:22.177928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dc045-e545-49f5-82f4-a6d263a86684 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:22.180786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dc045-e545-49f5-82f4-a6d263a86684 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:22.180786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dc045-e545-49f5-82f4-a6d263a86684 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:22.180786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dc045-e545-49f5-82f4-a6d263a86684 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:22.180786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5dc045-e545-49f5-82f4-a6d263a86684 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:22.181469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5dc045-e545-49f5-82f4-a6d263a86684 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:22.272950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:22.280133 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3788/7538] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:33:22 2022] POST /identity/v3/domains => generated 279 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:22.284385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:22.285918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:22.298639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f4540-5c67-4961-bff1-3b4e520d3ba6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:22.356048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823c957-e087-441b-8e66-8f9342600d5c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:22.356972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823c957-e087-441b-8e66-8f9342600d5c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:22.357251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823c957-e087-441b-8e66-8f9342600d5c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:22.357526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3823c957-e087-441b-8e66-8f9342600d5c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/a5d7ed5741e148b186fa6b36568adc7c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:22.369006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823c957-e087-441b-8e66-8f9342600d5c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=a5d7ed5741e148b186fa6b36568adc7c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:22.370658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3823c957-e087-441b-8e66-8f9342600d5c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:22.390288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3823c957-e087-441b-8e66-8f9342600d5c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'a5d7ed5741e148b186fa6b36568adc7c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.391002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3823c957-e087-441b-8e66-8f9342600d5c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'a5d7ed5741e148b186fa6b36568adc7c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.562874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3823c957-e087-441b-8e66-8f9342600d5c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'a5d7ed5741e148b186fa6b36568adc7c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.563407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3823c957-e087-441b-8e66-8f9342600d5c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'a5d7ed5741e148b186fa6b36568adc7c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.566023 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3751/7539] 10.222.0.33 () {60 vars in 1249 bytes} [Sat May 28 21:33:22 2022] PATCH /identity/v3/domains/a5d7ed5741e148b186fa6b36568adc7c => generated 279 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:22.570673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dc045-e545-49f5-82f4-a6d263a86684 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:22.572372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9f5dc045-e545-49f5-82f4-a6d263a86684 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:22.585138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dc045-e545-49f5-82f4-a6d263a86684 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:22.650345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdeeac-e864-4b86-9b50-bee26241c58e tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:22.651355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdeeac-e864-4b86-9b50-bee26241c58e tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:22.651725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdeeac-e864-4b86-9b50-bee26241c58e tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:22.652091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcdeeac-e864-4b86-9b50-bee26241c58e tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/a5d7ed5741e148b186fa6b36568adc7c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:22.664689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdeeac-e864-4b86-9b50-bee26241c58e tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=a5d7ed5741e148b186fa6b36568adc7c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:22.666719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcdeeac-e864-4b86-9b50-bee26241c58e tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:22.668426 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3789/7540] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:33:22 2022] GET /identity/v3/domains/a5d7ed5741e148b186fa6b36568adc7c => generated 279 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:22.673779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823c957-e087-441b-8e66-8f9342600d5c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:22.684684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3823c957-e087-441b-8e66-8f9342600d5c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:22.689314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:22.690199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:22.690490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:22.690772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/a5d7ed5741e148b186fa6b36568adc7c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:22.692066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=a5d7ed5741e148b186fa6b36568adc7c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:22.693754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:22.721410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'a5d7ed5741e148b186fa6b36568adc7c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.721410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:22.722459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'a5d7ed5741e148b186fa6b36568adc7c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.884030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5d7ed5741e148b186fa6b36568adc7c', 'request_id': 'req-fad5dd61-34ce-4f34-870f-98e605c4a112'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.889356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5d7ed5741e148b186fa6b36568adc7c', 'request_id': 'req-fad5dd61-34ce-4f34-870f-98e605c4a112'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.974712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5d7ed5741e148b186fa6b36568adc7c', 'request_id': 'req-fad5dd61-34ce-4f34-870f-98e605c4a112'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.986691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] The token cache is being invalidate because project a5d7ed5741e148b186fa6b36568adc7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:22.987008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.987659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.995730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'a5d7ed5741e148b186fa6b36568adc7c', 'request_id': 'req-fad5dd61-34ce-4f34-870f-98e605c4a112'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.996419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'a5d7ed5741e148b186fa6b36568adc7c', 'request_id': 'req-fad5dd61-34ce-4f34-870f-98e605c4a112'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:22.999703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'a5d7ed5741e148b186fa6b36568adc7c', 'request_id': 'req-fad5dd61-34ce-4f34-870f-98e605c4a112'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:23.005285 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3752/7541] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:22 2022] DELETE /identity/v3/domains/a5d7ed5741e148b186fa6b36568adc7c => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:23.009224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdeeac-e864-4b86-9b50-bee26241c58e tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:23.010953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4bcdeeac-e864-4b86-9b50-bee26241c58e tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:23.025437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcdeeac-e864-4b86-9b50-bee26241c58e tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:23.084667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81277edf-d42f-45df-aeee-ccacd240b2d6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:23.085330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81277edf-d42f-45df-aeee-ccacd240b2d6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:23.085620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81277edf-d42f-45df-aeee-ccacd240b2d6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:23.085929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81277edf-d42f-45df-aeee-ccacd240b2d6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:23.086754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81277edf-d42f-45df-aeee-ccacd240b2d6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:23.088350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81277edf-d42f-45df-aeee-ccacd240b2d6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:23.107295 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3790/7542] 10.222.0.33 () {58 vars in 1127 bytes} [Sat May 28 21:33:23 2022] GET /identity/v3/domains => generated 1350 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:23.111534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:23.121527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad5dd61-34ce-4f34-870f-98e605c4a112 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:23.126255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c9700-8e4d-47ce-9a73-60881993e700 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:23.127114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c9700-8e4d-47ce-9a73-60881993e700 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:23.127419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c9700-8e4d-47ce-9a73-60881993e700 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:23.127684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c9700-8e4d-47ce-9a73-60881993e700 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/a5d7ed5741e148b186fa6b36568adc7c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:23.134741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c9700-8e4d-47ce-9a73-60881993e700 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:33:23.144845 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-543c9700-8e4d-47ce-9a73-60881993e700 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Could not find domain: a5d7ed5741e148b186fa6b36568adc7c.: keystone.exception.DomainNotFound: Could not find domain: a5d7ed5741e148b186fa6b36568adc7c. May 28 21:33:23.144845 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:23.144845 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 21:33:23.144845 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 21:33:23.144845 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:33:23.144845 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:33:23.144845 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:33:23.144845 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:33:23.144845 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: a5d7ed5741e148b186fa6b36568adc7c. May 28 21:33:23.144845 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:23.144845 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:33:23.144845 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:23.144845 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:23.144845 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:23.144845 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:23.144845 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:23.152442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:23.152442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:23.152442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:23.152442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:23.152442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:23.152442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:23.152442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:23.152442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch May 28 21:33:23.152442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') May 28 21:33:23.152442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:33:23.152442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:33:23.152442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:33:23.152442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:33:23.152442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:23.152442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:23.152442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:33:23.153558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:33:23.153558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:33:23.153558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:33:23.153558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:33:23.153558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 21:33:23.153558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:33:23.153558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 21:33:23.153558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:33:23.153558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:33:23.153558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:33:23.153558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 21:33:23.153558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:33:23.153558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 21:33:23.153558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 21:33:23.153558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 21:33:23.153558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: a5d7ed5741e148b186fa6b36568adc7c. May 28 21:33:23.153558 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:23.154805 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3753/7543] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:23 2022] PATCH /identity/v3/domains/a5d7ed5741e148b186fa6b36568adc7c => generated 112 bytes in 35 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:33:23.154893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81277edf-d42f-45df-aeee-ccacd240b2d6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:23.162138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81277edf-d42f-45df-aeee-ccacd240b2d6 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:23.166773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec659e6-a5fe-42b8-97c5-317d3afe9ada tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:23.167427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec659e6-a5fe-42b8-97c5-317d3afe9ada tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:23.167717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec659e6-a5fe-42b8-97c5-317d3afe9ada tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:23.167998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec659e6-a5fe-42b8-97c5-317d3afe9ada tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:23.168799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec659e6-a5fe-42b8-97c5-317d3afe9ada tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:23.170436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec659e6-a5fe-42b8-97c5-317d3afe9ada tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:23.233582 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3791/7544] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:33:23 2022] POST /identity/v3/domains => generated 277 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:23.237921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c9700-8e4d-47ce-9a73-60881993e700 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:23.248074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c9700-8e4d-47ce-9a73-60881993e700 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:23.304797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89b07a-881c-45b4-b047-c2313a1ec01e tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:23.305505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89b07a-881c-45b4-b047-c2313a1ec01e tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:23.305795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89b07a-881c-45b4-b047-c2313a1ec01e tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:23.306111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89b07a-881c-45b4-b047-c2313a1ec01e tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:23.306817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89b07a-881c-45b4-b047-c2313a1ec01e tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:23.308759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89b07a-881c-45b4-b047-c2313a1ec01e tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:23.455244 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3754/7545] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:23 2022] POST /identity/v3/users => generated 348 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:23.467607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec659e6-a5fe-42b8-97c5-317d3afe9ada tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:23.479145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec659e6-a5fe-42b8-97c5-317d3afe9ada tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:23.483655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be265ebf-9417-4fbe-b9e5-92c556f016dd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:23.484312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be265ebf-9417-4fbe-b9e5-92c556f016dd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:23.484599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be265ebf-9417-4fbe-b9e5-92c556f016dd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:23.484882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be265ebf-9417-4fbe-b9e5-92c556f016dd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/groups` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:23.485602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be265ebf-9417-4fbe-b9e5-92c556f016dd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:23.487538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be265ebf-9417-4fbe-b9e5-92c556f016dd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:23.563017 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3792/7546] 10.222.0.33 () {60 vars in 1146 bytes} [Sat May 28 21:33:23 2022] POST /identity/v3/groups => generated 324 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:23.567782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89b07a-881c-45b4-b047-c2313a1ec01e tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:23.577880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89b07a-881c-45b4-b047-c2313a1ec01e tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:23.582461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131f072-c091-47c3-8232-e40e8bd0869d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:23.583869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131f072-c091-47c3-8232-e40e8bd0869d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:23.584158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131f072-c091-47c3-8232-e40e8bd0869d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:23.584523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131f072-c091-47c3-8232-e40e8bd0869d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/f900c412cf24451f99f336e6d21f07d7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:23.585796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131f072-c091-47c3-8232-e40e8bd0869d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=f900c412cf24451f99f336e6d21f07d7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:23.587436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c131f072-c091-47c3-8232-e40e8bd0869d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:23.607509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c131f072-c091-47c3-8232-e40e8bd0869d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'f900c412cf24451f99f336e6d21f07d7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:23.608221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c131f072-c091-47c3-8232-e40e8bd0869d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'f900c412cf24451f99f336e6d21f07d7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:23.737010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=84,keystone:DELETE=11,keystone:INSERT=4,keystone:UPDATE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:23.745654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c131f072-c091-47c3-8232-e40e8bd0869d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'f900c412cf24451f99f336e6d21f07d7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:23.746317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c131f072-c091-47c3-8232-e40e8bd0869d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'f900c412cf24451f99f336e6d21f07d7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:23.748759 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3755/7547] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:23 2022] PATCH /identity/v3/domains/f900c412cf24451f99f336e6d21f07d7 => generated 278 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:23.753597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be265ebf-9417-4fbe-b9e5-92c556f016dd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:23.756737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-be265ebf-9417-4fbe-b9e5-92c556f016dd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:23.769660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be265ebf-9417-4fbe-b9e5-92c556f016dd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:23.831398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:23.832280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:23.832588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:23.832870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/f900c412cf24451f99f336e6d21f07d7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:23.835058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:23.844913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=f900c412cf24451f99f336e6d21f07d7)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:23.847531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:23.869343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'f900c412cf24451f99f336e6d21f07d7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:23.872598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'f900c412cf24451f99f336e6d21f07d7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:24.317309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef0e81b58b9496bba32ed5c74afc2b3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:24.324860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef0e81b58b9496bba32ed5c74afc2b3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:24.325740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef0e81b58b9496bba32ed5c74afc2b3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:24.326627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef0e81b58b9496bba32ed5c74afc2b3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:24.460601 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef0e81b58b9496bba32ed5c74afc2b3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:24.579851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:24.581018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f900c412cf24451f99f336e6d21f07d7', 'request_id': 'req-e75639c3-8da3-4c4c-b518-f8fa38e5a009'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:24.586967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f900c412cf24451f99f336e6d21f07d7', 'request_id': 'req-e75639c3-8da3-4c4c-b518-f8fa38e5a009'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:24.724100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f900c412cf24451f99f336e6d21f07d7', 'request_id': 'req-e75639c3-8da3-4c4c-b518-f8fa38e5a009'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:24.736821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] The token cache is being invalidate because project f900c412cf24451f99f336e6d21f07d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:24.737117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:24.737769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:24.745820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'f900c412cf24451f99f336e6d21f07d7', 'request_id': 'req-e75639c3-8da3-4c4c-b518-f8fa38e5a009'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:24.746536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'f900c412cf24451f99f336e6d21f07d7', 'request_id': 'req-e75639c3-8da3-4c4c-b518-f8fa38e5a009'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:24.749807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'f900c412cf24451f99f336e6d21f07d7', 'request_id': 'req-e75639c3-8da3-4c4c-b518-f8fa38e5a009'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:24.755879 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3793/7548] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:23 2022] DELETE /identity/v3/domains/f900c412cf24451f99f336e6d21f07d7 => generated 0 bytes in 1003 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:24.765602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131f072-c091-47c3-8232-e40e8bd0869d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:24.776465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c131f072-c091-47c3-8232-e40e8bd0869d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:24.852177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25eed1-9fb6-4072-aedc-9e64b969ebae tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:24.853056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25eed1-9fb6-4072-aedc-9e64b969ebae tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:24.853343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25eed1-9fb6-4072-aedc-9e64b969ebae tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:24.853620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25eed1-9fb6-4072-aedc-9e64b969ebae tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/f900c412cf24451f99f336e6d21f07d7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:24.868571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25eed1-9fb6-4072-aedc-9e64b969ebae tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:get_domain(domain_id=f900c412cf24451f99f336e6d21f07d7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:24.868571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25eed1-9fb6-4072-aedc-9e64b969ebae tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:24.882119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-ff25eed1-9fb6-4072-aedc-9e64b969ebae tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Could not find domain: f900c412cf24451f99f336e6d21f07d7.: keystone.exception.DomainNotFound: Could not find domain: f900c412cf24451f99f336e6d21f07d7. May 28 21:33:24.882119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:24.882119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 21:33:24.882119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 21:33:24.882119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:33:24.882119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:33:24.882119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:33:24.882119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:33:24.882119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: f900c412cf24451f99f336e6d21f07d7. May 28 21:33:24.882119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:24.882119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:33:24.882119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:24.882119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:24.882119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:24.882119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:24.882119 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:24.883277 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:24.883277 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:24.883277 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:24.883277 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:24.883277 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:24.883277 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:24.883277 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:24.883277 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 28 21:33:24.883277 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 28 21:33:24.883277 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 28 21:33:24.883277 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 28 21:33:24.883277 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:24.883277 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:24.883277 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:33:24.883277 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:33:24.883277 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:33:24.883277 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:33:24.884559 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:33:24.884559 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 21:33:24.884559 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:33:24.884559 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 21:33:24.884559 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:33:24.884559 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:33:24.884559 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:33:24.884559 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 21:33:24.884559 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:33:24.884559 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 21:33:24.884559 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 21:33:24.884559 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 21:33:24.884559 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: f900c412cf24451f99f336e6d21f07d7. May 28 21:33:24.884559 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:24.884559 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3756/7549] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:33:24 2022] GET /identity/v3/domains/f900c412cf24451f99f336e6d21f07d7 => generated 112 bytes in 118 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:33:24.887669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:24.889216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:24.902904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75639c3-8da3-4c4c-b518-f8fa38e5a009 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:24.907507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8364ea3-27f8-4d8f-9149-0969a1576cda tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:24.908384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8364ea3-27f8-4d8f-9149-0969a1576cda tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:24.908671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8364ea3-27f8-4d8f-9149-0969a1576cda tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:24.908951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8364ea3-27f8-4d8f-9149-0969a1576cda tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/users/2ef0e81b58b9496bba32ed5c74afc2b3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:24.918205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8364ea3-27f8-4d8f-9149-0969a1576cda tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:get_user(user_id=2ef0e81b58b9496bba32ed5c74afc2b3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:24.921190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8364ea3-27f8-4d8f-9149-0969a1576cda tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:24.950206 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-d8364ea3-27f8-4d8f-9149-0969a1576cda tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Could not find user: 2ef0e81b58b9496bba32ed5c74afc2b3.: keystone.exception.UserNotFound: Could not find user: 2ef0e81b58b9496bba32ed5c74afc2b3. May 28 21:33:24.950206 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:24.950206 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:24.950206 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:24.950206 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:24.950206 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:24.950206 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:24.950206 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:24.950206 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:24.950206 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:24.950206 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:24.950206 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:24.950206 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 28 21:33:24.950206 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_user(user_id) May 28 21:33:24.951321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 28 21:33:24.951321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 28 21:33:24.951321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:24.951321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:24.951321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 21:33:24.951321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 21:33:24.951321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 21:33:24.951321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 21:33:24.951321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:33:24.951321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:33:24.951321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:33:24.951321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:33:24.951321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:33:24.951321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 21:33:24.951321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:33:24.951321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 21:33:24.951321 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:33:24.952393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:33:24.952393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:33:24.952393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 21:33:24.952393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:33:24.952393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 21:33:24.952393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 21:33:24.952393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 21:33:24.952393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 21:33:24.952393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 21:33:24.952393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 21:33:24.952393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 21:33:24.952393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: 2ef0e81b58b9496bba32ed5c74afc2b3. May 28 21:33:24.952393 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:24.952393 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3794/7550] 10.222.0.33 () {58 vars in 1220 bytes} [Sat May 28 21:33:24 2022] GET /identity/v3/users/2ef0e81b58b9496bba32ed5c74afc2b3 => generated 110 bytes in 64 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:33:24.956228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25eed1-9fb6-4072-aedc-9e64b969ebae tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:24.968495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25eed1-9fb6-4072-aedc-9e64b969ebae tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:24.973004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cd0c8-0f0c-43b7-9ecc-05867dff4170 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:24.973935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cd0c8-0f0c-43b7-9ecc-05867dff4170 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:24.974244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cd0c8-0f0c-43b7-9ecc-05867dff4170 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:24.974521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7cd0c8-0f0c-43b7-9ecc-05867dff4170 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/groups/6b7580d62a3a47059e52bf52ae498536` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:24.979009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7cd0c8-0f0c-43b7-9ecc-05867dff4170 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:get_group(group_id=6b7580d62a3a47059e52bf52ae498536)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:24.980899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7cd0c8-0f0c-43b7-9ecc-05867dff4170 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:25.014072 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-aa7cd0c8-0f0c-43b7-9ecc-05867dff4170 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Could not find group: 6b7580d62a3a47059e52bf52ae498536.: keystone.exception.GroupNotFound: Could not find group: 6b7580d62a3a47059e52bf52ae498536. May 28 21:33:25.014072 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:25.014072 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:25.014072 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:25.014072 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:25.014072 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:25.014072 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:25.014072 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:25.014072 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:25.014072 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:25.014072 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:25.014072 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:25.014072 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 58, in get May 28 21:33:25.014072 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_group(group_id) May 28 21:33:25.014995 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 70, in _get_group May 28 21:33:25.014995 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.identity_api.get_group(group_id)) May 28 21:33:25.014995 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:25.014995 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:25.014995 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 21:33:25.014995 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 21:33:25.014995 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 21:33:25.014995 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 21:33:25.014995 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:33:25.014995 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:33:25.014995 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:33:25.014995 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:33:25.014995 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:33:25.014995 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 21:33:25.014995 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:33:25.014995 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 21:33:25.014995 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:33:25.016051 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:33:25.016051 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:33:25.016051 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 21:33:25.016051 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:33:25.016051 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 21:33:25.016051 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group May 28 21:33:25.016051 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) May 28 21:33:25.016051 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group May 28 21:33:25.016051 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() May 28 21:33:25.016051 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group May 28 21:33:25.016051 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) May 28 21:33:25.016051 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: 6b7580d62a3a47059e52bf52ae498536. May 28 21:33:25.016051 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:25.016051 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3757/7551] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:24 2022] GET /identity/v3/groups/6b7580d62a3a47059e52bf52ae498536 => generated 111 bytes in 59 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:33:25.019477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8364ea3-27f8-4d8f-9149-0969a1576cda tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:25.030208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8364ea3-27f8-4d8f-9149-0969a1576cda tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:25.034802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfe9af-f30f-4691-89a0-ec4f7b822e74 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:25.035644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfe9af-f30f-4691-89a0-ec4f7b822e74 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:25.035927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfe9af-f30f-4691-89a0-ec4f7b822e74 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:25.036232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acfe9af-f30f-4691-89a0-ec4f7b822e74 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/groups/6b7580d62a3a47059e52bf52ae498536` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:25.040607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acfe9af-f30f-4691-89a0-ec4f7b822e74 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:33:25.052806 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-4acfe9af-f30f-4691-89a0-ec4f7b822e74 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Could not find group: 6b7580d62a3a47059e52bf52ae498536.: keystone.exception.GroupNotFound: Could not find group: 6b7580d62a3a47059e52bf52ae498536. May 28 21:33:25.052806 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:25.052806 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:25.052806 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:25.052806 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:25.052806 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:25.052806 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:25.052806 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:25.052806 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:25.052806 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:25.052806 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:25.052806 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:25.052806 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/groups.py", line 133, in delete May 28 21:33:25.052806 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_group') May 28 21:33:25.053795 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:33:25.053795 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:33:25.053795 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:33:25.053795 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:33:25.053795 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:25.053795 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:25.053795 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 21:33:25.053795 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 21:33:25.053795 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 21:33:25.053795 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 21:33:25.053795 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:33:25.053795 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:33:25.053795 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:33:25.053795 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:33:25.053795 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:33:25.053795 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 21:33:25.053795 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:33:25.054877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 21:33:25.054877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:33:25.054877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:33:25.054877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:33:25.054877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 21:33:25.054877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:33:25.054877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 21:33:25.054877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1262, in get_group May 28 21:33:25.054877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ref = driver.get_group(entity_id) May 28 21:33:25.054877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 417, in get_group May 28 21:33:25.054877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_group(session, group_id).to_dict() May 28 21:33:25.054877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 412, in _get_group May 28 21:33:25.054877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.GroupNotFound(group_id=group_id) May 28 21:33:25.054877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.GroupNotFound: Could not find group: 6b7580d62a3a47059e52bf52ae498536. May 28 21:33:25.054877 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:25.054877 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3795/7552] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:33:25 2022] DELETE /identity/v3/groups/6b7580d62a3a47059e52bf52ae498536 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:33:25.057727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cd0c8-0f0c-43b7-9ecc-05867dff4170 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:25.072069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7cd0c8-0f0c-43b7-9ecc-05867dff4170 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:25.077729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585b1b6-daa2-4bf3-a328-aa6561c1b7fe tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:25.078658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585b1b6-daa2-4bf3-a328-aa6561c1b7fe tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:25.078948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585b1b6-daa2-4bf3-a328-aa6561c1b7fe tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:25.079225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585b1b6-daa2-4bf3-a328-aa6561c1b7fe tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/users/2ef0e81b58b9496bba32ed5c74afc2b3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:25.088291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585b1b6-daa2-4bf3-a328-aa6561c1b7fe tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:delete_user(user_id=2ef0e81b58b9496bba32ed5c74afc2b3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:25.090582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585b1b6-daa2-4bf3-a328-aa6561c1b7fe tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:25.107231 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-8585b1b6-daa2-4bf3-a328-aa6561c1b7fe tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Could not find user: 2ef0e81b58b9496bba32ed5c74afc2b3.: keystone.exception.UserNotFound: Could not find user: 2ef0e81b58b9496bba32ed5c74afc2b3. May 28 21:33:25.107231 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:25.107231 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:25.107231 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:25.107231 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:25.107231 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:25.107231 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:25.107231 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:25.107231 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:25.107231 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:25.107231 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:25.107231 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:25.107231 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 275, in delete May 28 21:33:25.107231 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application PROVIDERS.identity_api.delete_user(user_id) May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1207, in delete_user May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application user_old = self.get_user(user_id) May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:33:25.108227 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: 2ef0e81b58b9496bba32ed5c74afc2b3. May 28 21:33:25.109246 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:25.110302 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3758/7553] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:25 2022] DELETE /identity/v3/users/2ef0e81b58b9496bba32ed5c74afc2b3 => generated 110 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:33:25.112728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfe9af-f30f-4691-89a0-ec4f7b822e74 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:25.123232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acfe9af-f30f-4691-89a0-ec4f7b822e74 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:25.128205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1c2c2-3612-439e-9462-ceea6c21d9cc tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:25.128767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1c2c2-3612-439e-9462-ceea6c21d9cc tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:25.129052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1c2c2-3612-439e-9462-ceea6c21d9cc tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:25.129334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1c2c2-3612-439e-9462-ceea6c21d9cc tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/f900c412cf24451f99f336e6d21f07d7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:25.137687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1c2c2-3612-439e-9462-ceea6c21d9cc tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:33:25.148385 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-50b1c2c2-3612-439e-9462-ceea6c21d9cc tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Could not find domain: f900c412cf24451f99f336e6d21f07d7.: keystone.exception.DomainNotFound: Could not find domain: f900c412cf24451f99f336e6d21f07d7. May 28 21:33:25.148385 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:25.148385 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 21:33:25.148385 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 21:33:25.148385 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:33:25.148385 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:33:25.148385 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:33:25.148385 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:33:25.148385 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: f900c412cf24451f99f336e6d21f07d7. May 28 21:33:25.148385 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:25.148385 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:33:25.148385 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:25.148385 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:25.148385 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:25.148385 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:25.148385 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:25.149611 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:25.149611 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:25.149611 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:25.149611 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:25.149611 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:25.149611 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:25.149611 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:25.149611 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch May 28 21:33:25.149611 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') May 28 21:33:25.149611 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:33:25.149611 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:33:25.149611 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:33:25.149611 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:33:25.149611 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:25.149611 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:25.149611 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:33:25.150578 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:33:25.150578 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:33:25.150578 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:33:25.150578 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:33:25.150578 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 21:33:25.150578 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:33:25.150578 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 21:33:25.150578 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:33:25.150578 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:33:25.150578 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:33:25.150578 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 21:33:25.150578 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:33:25.150578 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 21:33:25.150578 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 21:33:25.150578 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 21:33:25.150578 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: f900c412cf24451f99f336e6d21f07d7. May 28 21:33:25.150578 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:25.151610 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3796/7554] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:25 2022] PATCH /identity/v3/domains/f900c412cf24451f99f336e6d21f07d7 => generated 112 bytes in 37 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:33:25.160419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585b1b6-daa2-4bf3-a328-aa6561c1b7fe tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:25.170735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585b1b6-daa2-4bf3-a328-aa6561c1b7fe tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:25.175724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae60be-514f-4471-b074-dd1610f2f7d0 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:25.176419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae60be-514f-4471-b074-dd1610f2f7d0 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:25.176729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae60be-514f-4471-b074-dd1610f2f7d0 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:25.177033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae60be-514f-4471-b074-dd1610f2f7d0 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:25.177748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae60be-514f-4471-b074-dd1610f2f7d0 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:25.179426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae60be-514f-4471-b074-dd1610f2f7d0 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:25.194070 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3759/7555] 10.222.0.33 () {58 vars in 1127 bytes} [Sat May 28 21:33:25 2022] GET /identity/v3/domains => generated 1350 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:25.199103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1c2c2-3612-439e-9462-ceea6c21d9cc tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:25.210362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1c2c2-3612-439e-9462-ceea6c21d9cc tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:25.216513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f70a72-2f91-4c70-a8d3-721e494c4e33 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:25.217191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f70a72-2f91-4c70-a8d3-721e494c4e33 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:25.217552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f70a72-2f91-4c70-a8d3-721e494c4e33 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:25.217768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f70a72-2f91-4c70-a8d3-721e494c4e33 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:25.218551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f70a72-2f91-4c70-a8d3-721e494c4e33 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Adding query filter params (enabled=True) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:33:25.218914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f70a72-2f91-4c70-a8d3-721e494c4e33 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:25.220470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f70a72-2f91-4c70-a8d3-721e494c4e33 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:25.242841 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3797/7556] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:33:25 2022] GET /identity/v3/domains?enabled=True => generated 1096 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:25.247632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae60be-514f-4471-b074-dd1610f2f7d0 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:25.258314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae60be-514f-4471-b074-dd1610f2f7d0 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:25.263317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f10b96-f44e-4dd5-a787-90ad742e6012 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:25.263982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f10b96-f44e-4dd5-a787-90ad742e6012 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:25.264272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f10b96-f44e-4dd5-a787-90ad742e6012 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:25.264557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f10b96-f44e-4dd5-a787-90ad742e6012 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:25.265316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f10b96-f44e-4dd5-a787-90ad742e6012 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Adding query filter params (name=tempest-test_domain-1559148159) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:33:25.265596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f10b96-f44e-4dd5-a787-90ad742e6012 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:25.267370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f10b96-f44e-4dd5-a787-90ad742e6012 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:25.279592 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3760/7557] 10.222.0.33 () {58 vars in 1198 bytes} [Sat May 28 21:33:25 2022] GET /identity/v3/domains?name=tempest-test_domain-1559148159 => generated 409 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:25.284267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f70a72-2f91-4c70-a8d3-721e494c4e33 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:25.294871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f70a72-2f91-4c70-a8d3-721e494c4e33 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:25.299527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bf4fe-8600-4d60-be43-45a854736b02 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:25.300450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bf4fe-8600-4d60-be43-45a854736b02 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:25.300740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bf4fe-8600-4d60-be43-45a854736b02 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:25.301016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182bf4fe-8600-4d60-be43-45a854736b02 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/f900c412cf24451f99f336e6d21f07d7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:25.308610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182bf4fe-8600-4d60-be43-45a854736b02 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:33:25.319421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-182bf4fe-8600-4d60-be43-45a854736b02 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Could not find domain: f900c412cf24451f99f336e6d21f07d7.: keystone.exception.DomainNotFound: Could not find domain: f900c412cf24451f99f336e6d21f07d7. May 28 21:33:25.319421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:25.319421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 21:33:25.319421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 21:33:25.319421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:33:25.319421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:33:25.319421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:33:25.319421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:33:25.319421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: f900c412cf24451f99f336e6d21f07d7. May 28 21:33:25.319421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:25.319421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:33:25.319421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:25.319421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:25.319421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:25.319421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:25.319421 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:25.323386 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:25.323386 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:25.323386 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:25.323386 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:25.323386 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:25.323386 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:25.323386 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:25.323386 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch May 28 21:33:25.323386 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') May 28 21:33:25.323386 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:33:25.323386 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:33:25.323386 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:33:25.323386 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:33:25.323386 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:25.323386 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:25.323386 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:33:25.324527 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:33:25.324527 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:33:25.324527 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:33:25.324527 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:33:25.324527 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 21:33:25.324527 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:33:25.324527 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 21:33:25.324527 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:33:25.324527 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:33:25.324527 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:33:25.324527 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 21:33:25.324527 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:33:25.324527 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 21:33:25.324527 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 21:33:25.324527 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 21:33:25.324527 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: f900c412cf24451f99f336e6d21f07d7. May 28 21:33:25.324527 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:25.325715 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3798/7558] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:25 2022] PATCH /identity/v3/domains/f900c412cf24451f99f336e6d21f07d7 => generated 112 bytes in 37 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:33:25.325800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f10b96-f44e-4dd5-a787-90ad742e6012 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:25.335500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f10b96-f44e-4dd5-a787-90ad742e6012 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:25.341065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1b9dd-7ec7-41f1-9512-02107b7e37d3 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:25.341922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1b9dd-7ec7-41f1-9512-02107b7e37d3 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:25.342215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1b9dd-7ec7-41f1-9512-02107b7e37d3 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:25.342519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1b9dd-7ec7-41f1-9512-02107b7e37d3 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/886a027d755f49ad868e7a4fb7650a36` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:25.354159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1b9dd-7ec7-41f1-9512-02107b7e37d3 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=886a027d755f49ad868e7a4fb7650a36)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:25.355853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1b9dd-7ec7-41f1-9512-02107b7e37d3 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:25.397445 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3761/7559] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:25 2022] PATCH /identity/v3/domains/886a027d755f49ad868e7a4fb7650a36 => generated 278 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:25.401341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bf4fe-8600-4d60-be43-45a854736b02 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:25.411738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182bf4fe-8600-4d60-be43-45a854736b02 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:25.416437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:25.417354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:25.417641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:25.417963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/886a027d755f49ad868e7a4fb7650a36` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:25.430405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=886a027d755f49ad868e7a4fb7650a36)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:25.432029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:25.454052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '886a027d755f49ad868e7a4fb7650a36'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:25.457371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '886a027d755f49ad868e7a4fb7650a36'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:25.587755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '886a027d755f49ad868e7a4fb7650a36', 'request_id': 'req-880e9102-4a97-4c85-a994-6d312b7691b7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:25.593513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '886a027d755f49ad868e7a4fb7650a36', 'request_id': 'req-880e9102-4a97-4c85-a994-6d312b7691b7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:25.660370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '886a027d755f49ad868e7a4fb7650a36', 'request_id': 'req-880e9102-4a97-4c85-a994-6d312b7691b7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:25.673783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] The token cache is being invalidate because project 886a027d755f49ad868e7a4fb7650a36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:25.674123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:25.674797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:25.683411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '886a027d755f49ad868e7a4fb7650a36', 'request_id': 'req-880e9102-4a97-4c85-a994-6d312b7691b7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:25.684073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '886a027d755f49ad868e7a4fb7650a36', 'request_id': 'req-880e9102-4a97-4c85-a994-6d312b7691b7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:25.685586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:25.688026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '886a027d755f49ad868e7a4fb7650a36', 'request_id': 'req-880e9102-4a97-4c85-a994-6d312b7691b7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:25.693616 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3799/7560] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:25 2022] DELETE /identity/v3/domains/886a027d755f49ad868e7a4fb7650a36 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:25.697687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1b9dd-7ec7-41f1-9512-02107b7e37d3 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:25.699443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-01e1b9dd-7ec7-41f1-9512-02107b7e37d3 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:25.711888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1b9dd-7ec7-41f1-9512-02107b7e37d3 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:25.757270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:25.769264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c3dbf-a9c9-4165-8016-052ac55c05bd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:25.770161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c3dbf-a9c9-4165-8016-052ac55c05bd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:25.770465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c3dbf-a9c9-4165-8016-052ac55c05bd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:25.771155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35c3dbf-a9c9-4165-8016-052ac55c05bd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/90ff34ba05c34eaea0f85aa551cb63da` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:25.781964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c3dbf-a9c9-4165-8016-052ac55c05bd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=90ff34ba05c34eaea0f85aa551cb63da)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:25.783591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35c3dbf-a9c9-4165-8016-052ac55c05bd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:25.804287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a35c3dbf-a9c9-4165-8016-052ac55c05bd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '90ff34ba05c34eaea0f85aa551cb63da'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:25.805150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a35c3dbf-a9c9-4165-8016-052ac55c05bd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '90ff34ba05c34eaea0f85aa551cb63da'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:25.945306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a35c3dbf-a9c9-4165-8016-052ac55c05bd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '90ff34ba05c34eaea0f85aa551cb63da'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:25.945981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a35c3dbf-a9c9-4165-8016-052ac55c05bd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '90ff34ba05c34eaea0f85aa551cb63da'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:25.948925 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3762/7561] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:25 2022] PATCH /identity/v3/domains/90ff34ba05c34eaea0f85aa551cb63da => generated 278 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:25.964748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:25.975540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880e9102-4a97-4c85-a994-6d312b7691b7 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:26.032747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:26.033619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:26.033947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:26.034232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/90ff34ba05c34eaea0f85aa551cb63da` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:26.045457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=90ff34ba05c34eaea0f85aa551cb63da)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:26.047114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:26.069251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '90ff34ba05c34eaea0f85aa551cb63da'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.072671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '90ff34ba05c34eaea0f85aa551cb63da'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.210379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90ff34ba05c34eaea0f85aa551cb63da', 'request_id': 'req-5263f5b2-9e2b-4216-a842-078d29fcbad2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.216152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90ff34ba05c34eaea0f85aa551cb63da', 'request_id': 'req-5263f5b2-9e2b-4216-a842-078d29fcbad2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.311904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90ff34ba05c34eaea0f85aa551cb63da', 'request_id': 'req-5263f5b2-9e2b-4216-a842-078d29fcbad2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.325376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] The token cache is being invalidate because project 90ff34ba05c34eaea0f85aa551cb63da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:26.325671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.326340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.334309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '90ff34ba05c34eaea0f85aa551cb63da', 'request_id': 'req-5263f5b2-9e2b-4216-a842-078d29fcbad2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.334941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '90ff34ba05c34eaea0f85aa551cb63da', 'request_id': 'req-5263f5b2-9e2b-4216-a842-078d29fcbad2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.338356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '90ff34ba05c34eaea0f85aa551cb63da', 'request_id': 'req-5263f5b2-9e2b-4216-a842-078d29fcbad2'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.347435 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3800/7562] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:25 2022] DELETE /identity/v3/domains/90ff34ba05c34eaea0f85aa551cb63da => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:26.352201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c3dbf-a9c9-4165-8016-052ac55c05bd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:26.353760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a35c3dbf-a9c9-4165-8016-052ac55c05bd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:26.366612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35c3dbf-a9c9-4165-8016-052ac55c05bd tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:26.423493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e25bf2-c273-4f19-bce0-e313b7e4275d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:26.424286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e25bf2-c273-4f19-bce0-e313b7e4275d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:26.424571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e25bf2-c273-4f19-bce0-e313b7e4275d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:26.424845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e25bf2-c273-4f19-bce0-e313b7e4275d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/b6741cb5700943d996af7ec4e6768c2e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:26.435373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e25bf2-c273-4f19-bce0-e313b7e4275d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=b6741cb5700943d996af7ec4e6768c2e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:26.436997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e25bf2-c273-4f19-bce0-e313b7e4275d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:26.456719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-69e25bf2-c273-4f19-bce0-e313b7e4275d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'b6741cb5700943d996af7ec4e6768c2e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.457387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-69e25bf2-c273-4f19-bce0-e313b7e4275d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'b6741cb5700943d996af7ec4e6768c2e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.532368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-69e25bf2-c273-4f19-bce0-e313b7e4275d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'b6741cb5700943d996af7ec4e6768c2e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.533011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-69e25bf2-c273-4f19-bce0-e313b7e4275d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'b6741cb5700943d996af7ec4e6768c2e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.535487 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3763/7563] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:26 2022] PATCH /identity/v3/domains/b6741cb5700943d996af7ec4e6768c2e => generated 278 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:26.539522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:26.540916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:26.554147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5263f5b2-9e2b-4216-a842-078d29fcbad2 tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:26.584499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:26.612480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59970976-4abb-45e8-aadb-9357bfe0a79c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fa7d879b17b46eeb13a829b7b5a9d93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b47788cd7cb8432aa3527c703bc16704', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:26.613396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59970976-4abb-45e8-aadb-9357bfe0a79c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:26.613827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59970976-4abb-45e8-aadb-9357bfe0a79c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:26.614131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59970976-4abb-45e8-aadb-9357bfe0a79c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] PATH_INFO: `/v3/domains/b6741cb5700943d996af7ec4e6768c2e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:26.625760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59970976-4abb-45e8-aadb-9357bfe0a79c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=b6741cb5700943d996af7ec4e6768c2e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:26.627568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59970976-4abb-45e8-aadb-9357bfe0a79c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:26.655082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-59970976-4abb-45e8-aadb-9357bfe0a79c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'b6741cb5700943d996af7ec4e6768c2e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.658710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-59970976-4abb-45e8-aadb-9357bfe0a79c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'b6741cb5700943d996af7ec4e6768c2e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.762905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-59970976-4abb-45e8-aadb-9357bfe0a79c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6741cb5700943d996af7ec4e6768c2e', 'request_id': 'req-59970976-4abb-45e8-aadb-9357bfe0a79c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.768219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-59970976-4abb-45e8-aadb-9357bfe0a79c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6741cb5700943d996af7ec4e6768c2e', 'request_id': 'req-59970976-4abb-45e8-aadb-9357bfe0a79c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.871980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-59970976-4abb-45e8-aadb-9357bfe0a79c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6741cb5700943d996af7ec4e6768c2e', 'request_id': 'req-59970976-4abb-45e8-aadb-9357bfe0a79c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.885992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-59970976-4abb-45e8-aadb-9357bfe0a79c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] The token cache is being invalidate because project b6741cb5700943d996af7ec4e6768c2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:26.886287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-59970976-4abb-45e8-aadb-9357bfe0a79c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.887250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-59970976-4abb-45e8-aadb-9357bfe0a79c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.899163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-59970976-4abb-45e8-aadb-9357bfe0a79c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'b6741cb5700943d996af7ec4e6768c2e', 'request_id': 'req-59970976-4abb-45e8-aadb-9357bfe0a79c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.899871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-59970976-4abb-45e8-aadb-9357bfe0a79c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'b6741cb5700943d996af7ec4e6768c2e', 'request_id': 'req-59970976-4abb-45e8-aadb-9357bfe0a79c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.903377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-59970976-4abb-45e8-aadb-9357bfe0a79c tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'b6741cb5700943d996af7ec4e6768c2e', 'request_id': 'req-59970976-4abb-45e8-aadb-9357bfe0a79c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:26.918705 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3801/7564] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:26 2022] DELETE /identity/v3/domains/b6741cb5700943d996af7ec4e6768c2e => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:26.918846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e25bf2-c273-4f19-bce0-e313b7e4275d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:26.918846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-69e25bf2-c273-4f19-bce0-e313b7e4275d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:26.982569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e25bf2-c273-4f19-bce0-e313b7e4275d tempest-DomainsTestJSON-1556071758 tempest-DomainsTestJSON-1556071758-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:26.987198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:UPDATE=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:27.037909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4962d1-e33f-4853-a41f-8b1f1ee5c3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:27.038783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4962d1-e33f-4853-a41f-8b1f1ee5c3a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:27.039070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4962d1-e33f-4853-a41f-8b1f1ee5c3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:27.039346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4962d1-e33f-4853-a41f-8b1f1ee5c3a7 admin admin] PATH_INFO: `/v3/users/0228d086c0d64996a8047184a6c6a46d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:27.040581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4962d1-e33f-4853-a41f-8b1f1ee5c3a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0228d086c0d64996a8047184a6c6a46d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:27.042579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4962d1-e33f-4853-a41f-8b1f1ee5c3a7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:27.258164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec4962d1-e33f-4853-a41f-8b1f1ee5c3a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0228d086c0d64996a8047184a6c6a46d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:27.258849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec4962d1-e33f-4853-a41f-8b1f1ee5c3a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0228d086c0d64996a8047184a6c6a46d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:27.266705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec4962d1-e33f-4853-a41f-8b1f1ee5c3a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0228d086c0d64996a8047184a6c6a46d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:27.340785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec4962d1-e33f-4853-a41f-8b1f1ee5c3a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0228d086c0d64996a8047184a6c6a46d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:27.341434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ec4962d1-e33f-4853-a41f-8b1f1ee5c3a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0228d086c0d64996a8047184a6c6a46d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:27.362573 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3764/7565] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:26 2022] DELETE /identity/v3/users/0228d086c0d64996a8047184a6c6a46d => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:33:27.368095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d8543-9f15-4f2b-9abf-8de5b3a6be18 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:27.375386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d8543-9f15-4f2b-9abf-8de5b3a6be18 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:27.375386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d8543-9f15-4f2b-9abf-8de5b3a6be18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:27.385963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e27d8543-9f15-4f2b-9abf-8de5b3a6be18 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:27.387164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e27d8543-9f15-4f2b-9abf-8de5b3a6be18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:27.446618 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3802/7566] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:27.456196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4962d1-e33f-4853-a41f-8b1f1ee5c3a7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:27.457630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ec4962d1-e33f-4853-a41f-8b1f1ee5c3a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:27.516962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4962d1-e33f-4853-a41f-8b1f1ee5c3a7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:27.520920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e669971-2109-452a-a3f5-3602f887ca0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:27.522108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e669971-2109-452a-a3f5-3602f887ca0c service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:27.522405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e669971-2109-452a-a3f5-3602f887ca0c service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:27.522679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e669971-2109-452a-a3f5-3602f887ca0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:27.531780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e669971-2109-452a-a3f5-3602f887ca0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:27.533248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e669971-2109-452a-a3f5-3602f887ca0c service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:27.543236 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3765/7567] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:33:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:33:27.993268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d8543-9f15-4f2b-9abf-8de5b3a6be18 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:27.994809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e27d8543-9f15-4f2b-9abf-8de5b3a6be18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:28.016847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d8543-9f15-4f2b-9abf-8de5b3a6be18 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:28.024267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591db8e7-addf-497f-8f47-3ad35d216e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:28.025259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591db8e7-addf-497f-8f47-3ad35d216e83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:28.025677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591db8e7-addf-497f-8f47-3ad35d216e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:28.026069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591db8e7-addf-497f-8f47-3ad35d216e83 admin admin] PATH_INFO: `/v3/projects/31effd15ea534ccc8744bec9ee1fe055` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:28.027541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591db8e7-addf-497f-8f47-3ad35d216e83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31effd15ea534ccc8744bec9ee1fe055)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:28.030094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591db8e7-addf-497f-8f47-3ad35d216e83 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:28.033186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:28.158896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-591db8e7-addf-497f-8f47-3ad35d216e83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31effd15ea534ccc8744bec9ee1fe055', 'request_id': 'req-591db8e7-addf-497f-8f47-3ad35d216e83'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:28.164882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-591db8e7-addf-497f-8f47-3ad35d216e83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31effd15ea534ccc8744bec9ee1fe055', 'request_id': 'req-591db8e7-addf-497f-8f47-3ad35d216e83'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:28.215412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-591db8e7-addf-497f-8f47-3ad35d216e83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31effd15ea534ccc8744bec9ee1fe055', 'request_id': 'req-591db8e7-addf-497f-8f47-3ad35d216e83'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:28.228219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-591db8e7-addf-497f-8f47-3ad35d216e83 admin admin] The token cache is being invalidate because project 31effd15ea534ccc8744bec9ee1fe055 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:28.228520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-591db8e7-addf-497f-8f47-3ad35d216e83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:28.229183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-591db8e7-addf-497f-8f47-3ad35d216e83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:28.230876 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3803/7568] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:27 2022] DELETE /identity/v3/projects/31effd15ea534ccc8744bec9ee1fe055 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:28.234961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e669971-2109-452a-a3f5-3602f887ca0c service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:28.248389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e669971-2109-452a-a3f5-3602f887ca0c service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:28.277610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:28.305305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b98b71-f864-407e-b6b0-ba6e88c33bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:28.306229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b98b71-f864-407e-b6b0-ba6e88c33bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:28.306516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b98b71-f864-407e-b6b0-ba6e88c33bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:28.306792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b98b71-f864-407e-b6b0-ba6e88c33bd2 admin admin] PATH_INFO: `/v3/users/4fa7d879b17b46eeb13a829b7b5a9d93` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:28.308079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b98b71-f864-407e-b6b0-ba6e88c33bd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fa7d879b17b46eeb13a829b7b5a9d93)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:28.310041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b98b71-f864-407e-b6b0-ba6e88c33bd2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:28.491217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-28b98b71-f864-407e-b6b0-ba6e88c33bd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fa7d879b17b46eeb13a829b7b5a9d93'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:28.492028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-28b98b71-f864-407e-b6b0-ba6e88c33bd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fa7d879b17b46eeb13a829b7b5a9d93'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:28.499740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-28b98b71-f864-407e-b6b0-ba6e88c33bd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fa7d879b17b46eeb13a829b7b5a9d93'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:28.592270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-28b98b71-f864-407e-b6b0-ba6e88c33bd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fa7d879b17b46eeb13a829b7b5a9d93'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:28.592933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-28b98b71-f864-407e-b6b0-ba6e88c33bd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fa7d879b17b46eeb13a829b7b5a9d93'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:28.611314 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3766/7569] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:28 2022] DELETE /identity/v3/users/4fa7d879b17b46eeb13a829b7b5a9d93 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:33:29.044838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591db8e7-addf-497f-8f47-3ad35d216e83 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:29.046326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-591db8e7-addf-497f-8f47-3ad35d216e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:29.063581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591db8e7-addf-497f-8f47-3ad35d216e83 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:29.132157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12f94a-60f0-42e7-ac7c-444b27482b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:29.133026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12f94a-60f0-42e7-ac7c-444b27482b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:29.133337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12f94a-60f0-42e7-ac7c-444b27482b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:29.133616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12f94a-60f0-42e7-ac7c-444b27482b31 admin admin] PATH_INFO: `/v3/projects/b47788cd7cb8432aa3527c703bc16704` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:29.134839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12f94a-60f0-42e7-ac7c-444b27482b31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b47788cd7cb8432aa3527c703bc16704)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:29.136763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12f94a-60f0-42e7-ac7c-444b27482b31 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:29.284065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ef12f94a-60f0-42e7-ac7c-444b27482b31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b47788cd7cb8432aa3527c703bc16704', 'request_id': 'req-ef12f94a-60f0-42e7-ac7c-444b27482b31'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:29.290213 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ef12f94a-60f0-42e7-ac7c-444b27482b31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b47788cd7cb8432aa3527c703bc16704', 'request_id': 'req-ef12f94a-60f0-42e7-ac7c-444b27482b31'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:29.397618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ef12f94a-60f0-42e7-ac7c-444b27482b31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b47788cd7cb8432aa3527c703bc16704', 'request_id': 'req-ef12f94a-60f0-42e7-ac7c-444b27482b31'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:29.410058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ef12f94a-60f0-42e7-ac7c-444b27482b31 admin admin] The token cache is being invalidate because project b47788cd7cb8432aa3527c703bc16704 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:29.410349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ef12f94a-60f0-42e7-ac7c-444b27482b31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:29.410972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ef12f94a-60f0-42e7-ac7c-444b27482b31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:29.412508 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3804/7570] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:29 2022] DELETE /identity/v3/projects/b47788cd7cb8432aa3527c703bc16704 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:29.417443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696cdb47-ce36-45a0-82c8-98cffeed9906 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:29.417944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696cdb47-ce36-45a0-82c8-98cffeed9906 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:29.418039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696cdb47-ce36-45a0-82c8-98cffeed9906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:29.435901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-696cdb47-ce36-45a0-82c8-98cffeed9906 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:29.439212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-696cdb47-ce36-45a0-82c8-98cffeed9906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:29.495578 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3767/7571] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:29.503328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18ee12-b39e-4231-bf1f-784770716153 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:29.503622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18ee12-b39e-4231-bf1f-784770716153 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:29.503905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18ee12-b39e-4231-bf1f-784770716153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:29.522577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fa18ee12-b39e-4231-bf1f-784770716153 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:29.523834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fa18ee12-b39e-4231-bf1f-784770716153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:29.531924 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3805/7572] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:29.536126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696cdb47-ce36-45a0-82c8-98cffeed9906 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:29.546122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696cdb47-ce36-45a0-82c8-98cffeed9906 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:29.550601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e2ee2-c81c-4f3c-a30f-b03c5d3d35eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:29.551248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e2ee2-c81c-4f3c-a30f-b03c5d3d35eb admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:29.551533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e2ee2-c81c-4f3c-a30f-b03c5d3d35eb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:29.551809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e2ee2-c81c-4f3c-a30f-b03c5d3d35eb admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:29.552526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e2ee2-c81c-4f3c-a30f-b03c5d3d35eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:33:29.552836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e2ee2-c81c-4f3c-a30f-b03c5d3d35eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:29.554524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e2ee2-c81c-4f3c-a30f-b03c5d3d35eb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:29.571712 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3768/7573] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:29.572851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18ee12-b39e-4231-bf1f-784770716153 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:29.582422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18ee12-b39e-4231-bf1f-784770716153 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:29.587225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4eb1a-1c21-4ebd-81eb-c3d93bb4ada4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:29.588435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4eb1a-1c21-4ebd-81eb-c3d93bb4ada4 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:29.588435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4eb1a-1c21-4ebd-81eb-c3d93bb4ada4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:29.588435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4eb1a-1c21-4ebd-81eb-c3d93bb4ada4 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:29.589106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4eb1a-1c21-4ebd-81eb-c3d93bb4ada4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:29.591346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4eb1a-1c21-4ebd-81eb-c3d93bb4ada4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:29.604823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:29.695669 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3806/7574] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:29 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:33:29.700382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e2ee2-c81c-4f3c-a30f-b03c5d3d35eb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:29.711806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e2ee2-c81c-4f3c-a30f-b03c5d3d35eb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:29.746603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:29.780534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16986d-09eb-4aa1-9f42-14d72759c4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:29.781215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16986d-09eb-4aa1-9f42-14d72759c4ef admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:29.781505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16986d-09eb-4aa1-9f42-14d72759c4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:29.781789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f16986d-09eb-4aa1-9f42-14d72759c4ef admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:29.782539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16986d-09eb-4aa1-9f42-14d72759c4ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:29.784491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f16986d-09eb-4aa1-9f42-14d72759c4ef admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:29.885103 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3769/7575] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:29 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:33:29.886377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4eb1a-1c21-4ebd-81eb-c3d93bb4ada4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:29.897272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4eb1a-1c21-4ebd-81eb-c3d93bb4ada4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:29.902055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d8879-2535-44ba-9a50-2f135294dfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:29.902834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d8879-2535-44ba-9a50-2f135294dfe4 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:29.903067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d8879-2535-44ba-9a50-2f135294dfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:29.903358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d8879-2535-44ba-9a50-2f135294dfe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:29.904732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d8879-2535-44ba-9a50-2f135294dfe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:29.905875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d8879-2535-44ba-9a50-2f135294dfe4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:29.918055 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3807/7576] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:29.922058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16986d-09eb-4aa1-9f42-14d72759c4ef admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:29.933163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f16986d-09eb-4aa1-9f42-14d72759c4ef admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:29.938000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3af7d-d7b1-465d-84d4-2eeb429ffe2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:29.938642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3af7d-d7b1-465d-84d4-2eeb429ffe2a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:29.938924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3af7d-d7b1-465d-84d4-2eeb429ffe2a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:29.939205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3af7d-d7b1-465d-84d4-2eeb429ffe2a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:29.939945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3af7d-d7b1-465d-84d4-2eeb429ffe2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:29.941562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d3af7d-d7b1-465d-84d4-2eeb429ffe2a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:29.952356 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3770/7577] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:29.956550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d8879-2535-44ba-9a50-2f135294dfe4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:29.967362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d8879-2535-44ba-9a50-2f135294dfe4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:29.972131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b2751-bf63-4898-a743-cea3c2a8b48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:29.973015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b2751-bf63-4898-a743-cea3c2a8b48e admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:29.975677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b2751-bf63-4898-a743-cea3c2a8b48e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:29.975677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0b2751-bf63-4898-a743-cea3c2a8b48e admin admin] PATH_INFO: `/v3/projects/16cc1fdcab5c44fabe915217750364fa/users/471a767f42574cdbbdbc4c1f5a0e28a7/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:29.990008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b2751-bf63-4898-a743-cea3c2a8b48e admin admin] RBAC: Authorizing `identity:create_grant(project_id=16cc1fdcab5c44fabe915217750364fa, user_id=471a767f42574cdbbdbc4c1f5a0e28a7, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:29.993547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0b2751-bf63-4898-a743-cea3c2a8b48e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:30.076836 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3808/7578] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:29 2022] PUT /identity/v3/projects/16cc1fdcab5c44fabe915217750364fa/users/471a767f42574cdbbdbc4c1f5a0e28a7/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:30.087109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785d455-e8ed-414f-a1ed-d88ccbc7beea None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:30.087396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785d455-e8ed-414f-a1ed-d88ccbc7beea None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:30.087674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b785d455-e8ed-414f-a1ed-d88ccbc7beea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:30.121313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b785d455-e8ed-414f-a1ed-d88ccbc7beea None None] MFA Rules not processed for user `471a767f42574cdbbdbc4c1f5a0e28a7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:30.122869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b785d455-e8ed-414f-a1ed-d88ccbc7beea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:30.188390 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3771/7579] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:30.192762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b2751-bf63-4898-a743-cea3c2a8b48e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:30.255777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0b2751-bf63-4898-a743-cea3c2a8b48e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:30.260505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d770d6-9ab8-49a2-8f11-426a7335213a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:30.261187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d770d6-9ab8-49a2-8f11-426a7335213a admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:30.261468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d770d6-9ab8-49a2-8f11-426a7335213a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:30.261744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d770d6-9ab8-49a2-8f11-426a7335213a admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:30.262580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d770d6-9ab8-49a2-8f11-426a7335213a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:30.264471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d770d6-9ab8-49a2-8f11-426a7335213a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:30.376085 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3809/7580] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:30 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:30.380783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785d455-e8ed-414f-a1ed-d88ccbc7beea None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:30.391672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b785d455-e8ed-414f-a1ed-d88ccbc7beea None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:30.449715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c0afc-376a-4382-8a11-131876eff7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:30.450470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c0afc-376a-4382-8a11-131876eff7fe admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:30.452518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c0afc-376a-4382-8a11-131876eff7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:30.452518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c0afc-376a-4382-8a11-131876eff7fe admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:30.452518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c0afc-376a-4382-8a11-131876eff7fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:30.453631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c0afc-376a-4382-8a11-131876eff7fe admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:30.534888 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3772/7581] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:30 2022] POST /identity/v3/users => generated 298 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:30.538746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d770d6-9ab8-49a2-8f11-426a7335213a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:30.549307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d770d6-9ab8-49a2-8f11-426a7335213a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:30.554055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c1b38-06dc-48e5-8fe3-61ee827f1744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:30.554719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c1b38-06dc-48e5-8fe3-61ee827f1744 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:30.555010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c1b38-06dc-48e5-8fe3-61ee827f1744 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:30.555287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c1b38-06dc-48e5-8fe3-61ee827f1744 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:30.556013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c1b38-06dc-48e5-8fe3-61ee827f1744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:30.557614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c1b38-06dc-48e5-8fe3-61ee827f1744 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:30.568309 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3810/7582] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:30.572976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c0afc-376a-4382-8a11-131876eff7fe admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:30.583074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c0afc-376a-4382-8a11-131876eff7fe admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:30.588222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc118f-fdfb-4bd1-981d-bc93622d73aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:30.589083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc118f-fdfb-4bd1-981d-bc93622d73aa admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:30.589626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc118f-fdfb-4bd1-981d-bc93622d73aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:30.589942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc118f-fdfb-4bd1-981d-bc93622d73aa admin admin] PATH_INFO: `/v3/projects/ae692deba7c542e1b17f76c5770f856b/users/283cd42c229e4d5b9110f740fa2b68ab/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:30.605779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc118f-fdfb-4bd1-981d-bc93622d73aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae692deba7c542e1b17f76c5770f856b, user_id=283cd42c229e4d5b9110f740fa2b68ab, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:30.608796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc118f-fdfb-4bd1-981d-bc93622d73aa admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:30.695050 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3773/7583] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:30 2022] PUT /identity/v3/projects/ae692deba7c542e1b17f76c5770f856b/users/283cd42c229e4d5b9110f740fa2b68ab/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:30.699633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c1b38-06dc-48e5-8fe3-61ee827f1744 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:30.710094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c1b38-06dc-48e5-8fe3-61ee827f1744 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:30.768152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57252bba-9c3d-4b4e-b64a-0618ca5a7044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:30.768828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57252bba-9c3d-4b4e-b64a-0618ca5a7044 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:30.769116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57252bba-9c3d-4b4e-b64a-0618ca5a7044 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:30.769396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57252bba-9c3d-4b4e-b64a-0618ca5a7044 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:30.770202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57252bba-9c3d-4b4e-b64a-0618ca5a7044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:30.771904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57252bba-9c3d-4b4e-b64a-0618ca5a7044 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:30.782906 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3811/7584] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:30.787511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc118f-fdfb-4bd1-981d-bc93622d73aa admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:30.797654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc118f-fdfb-4bd1-981d-bc93622d73aa admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:30.802323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b0897-83c7-4e4f-a03e-888af7fb8dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:30.803192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b0897-83c7-4e4f-a03e-888af7fb8dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:30.803475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b0897-83c7-4e4f-a03e-888af7fb8dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:30.803752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b0897-83c7-4e4f-a03e-888af7fb8dfe admin admin] PATH_INFO: `/v3/projects/ae692deba7c542e1b17f76c5770f856b/users/283cd42c229e4d5b9110f740fa2b68ab/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:30.809917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b0897-83c7-4e4f-a03e-888af7fb8dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae692deba7c542e1b17f76c5770f856b, user_id=283cd42c229e4d5b9110f740fa2b68ab, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:30.812578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b0897-83c7-4e4f-a03e-888af7fb8dfe admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:30.818539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:30.824273 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3774/7585] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:30 2022] PUT /identity/v3/projects/ae692deba7c542e1b17f76c5770f856b/users/283cd42c229e4d5b9110f740fa2b68ab/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:30.844125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b88fe8-e14b-48c4-868a-bc3e34361ea4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:30.844420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b88fe8-e14b-48c4-868a-bc3e34361ea4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:30.844706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b88fe8-e14b-48c4-868a-bc3e34361ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:30.872242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-12b88fe8-e14b-48c4-868a-bc3e34361ea4 None None] MFA Rules not processed for user `283cd42c229e4d5b9110f740fa2b68ab`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:30.873466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-12b88fe8-e14b-48c4-868a-bc3e34361ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:30.890702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:30.945511 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3812/7586] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:30.951291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b0897-83c7-4e4f-a03e-888af7fb8dfe admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:30.961974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b0897-83c7-4e4f-a03e-888af7fb8dfe admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:30.966813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25e32f-3344-4c80-a745-35d423e2acff tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '283cd42c229e4d5b9110f740fa2b68ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae692deba7c542e1b17f76c5770f856b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:30.967503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25e32f-3344-4c80-a745-35d423e2acff tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:30.967787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25e32f-3344-4c80-a745-35d423e2acff tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:30.968061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25e32f-3344-4c80-a745-35d423e2acff tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:30.968782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25e32f-3344-4c80-a745-35d423e2acff tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:30.970524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25e32f-3344-4c80-a745-35d423e2acff tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-7c25e32f-3344-4c80-a745-35d423e2acff tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invalid input for field 'name': '' is too short May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: Failed validating 'minLength' in schema['properties']['name']: May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: On instance['name']: May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: '': keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: Failed validating 'minLength' in schema['properties']['name']: May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: On instance['name']: May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: '' May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise error May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: '' is too short May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application On instance['name']: May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application '' May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:30.976587 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:33:30.978526 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:30.978526 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:30.978526 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:30.978526 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:30.978526 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:30.978526 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:30.978526 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:30.978526 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:30.978526 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:30.978526 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:30.978526 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:30.978526 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:30.978526 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post May 28 21:33:30.978526 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) May 28 21:33:30.978526 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate May 28 21:33:30.978526 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) May 28 21:33:30.978526 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate May 28 21:33:30.979707 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) May 28 21:33:30.979707 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': '' is too short May 28 21:33:30.979707 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:30.979707 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Failed validating 'minLength' in schema['properties']['name']: May 28 21:33:30.979707 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} May 28 21:33:30.979707 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:30.979707 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application On instance['name']: May 28 21:33:30.979707 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application '' May 28 21:33:30.979707 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:30.979707 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3775/7587] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:33:30 2022] POST /identity/v3/domains => generated 283 bytes in 27 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) May 28 21:33:30.982757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b88fe8-e14b-48c4-868a-bc3e34361ea4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:30.992137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b88fe8-e14b-48c4-868a-bc3e34361ea4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:30.997345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91506b74-b3da-497e-8e5b-4218e66b1d51 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '283cd42c229e4d5b9110f740fa2b68ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae692deba7c542e1b17f76c5770f856b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:30.998039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91506b74-b3da-497e-8e5b-4218e66b1d51 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:30.998354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91506b74-b3da-497e-8e5b-4218e66b1d51 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:30.998593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91506b74-b3da-497e-8e5b-4218e66b1d51 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:30.999336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91506b74-b3da-497e-8e5b-4218e66b1d51 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:31.000931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91506b74-b3da-497e-8e5b-4218e66b1d51 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-91506b74-b3da-497e-8e5b-4218e66b1d51 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: Failed validating 'maxLength' in schema['properties']['name']: May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: On instance['name']: May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa': keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: Failed validating 'maxLength' in schema['properties']['name']: May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: On instance['name']: May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 71, in validate May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.validator.validate(*args, **kwargs) May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/lib/python3/dist-packages/jsonschema/validators.py", line 353, in validate May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise error May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application jsonschema.exceptions.ValidationError: 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} May 28 21:33:31.006934 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application On instance['name']: May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 115, in post May 28 21:33:31.008996 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application validation.lazy_validate(schema.domain_create, domain) May 28 21:33:31.010477 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/__init__.py", line 30, in lazy_validate May 28 21:33:31.010477 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application schema_validator.validate(resource_to_validate) May 28 21:33:31.010477 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/validation/validators.py", line 89, in validate May 28 21:33:31.010477 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.SchemaValidationError(detail=detail) May 28 21:33:31.010477 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.SchemaValidationError: Invalid input for field 'name': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' is too long May 28 21:33:31.010477 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:31.010477 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Failed validating 'maxLength' in schema['properties']['name']: May 28 21:33:31.010477 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application {'maxLength': 64, 'minLength': 1, 'pattern': '[\\S]+', 'type': 'string'} May 28 21:33:31.010477 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:31.010477 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application On instance['name']: May 28 21:33:31.010477 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa' May 28 21:33:31.010477 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:31.010477 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3813/7588] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:33:30 2022] POST /identity/v3/domains => generated 412 bytes in 27 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) May 28 21:33:31.016219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25e32f-3344-4c80-a745-35d423e2acff tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:31.027351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25e32f-3344-4c80-a745-35d423e2acff tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:31.031991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595955b-fa7e-4bff-a916-eaeb25002f39 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '283cd42c229e4d5b9110f740fa2b68ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae692deba7c542e1b17f76c5770f856b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:31.032633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595955b-fa7e-4bff-a916-eaeb25002f39 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:31.032922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595955b-fa7e-4bff-a916-eaeb25002f39 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:31.033211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595955b-fa7e-4bff-a916-eaeb25002f39 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:31.033946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595955b-fa7e-4bff-a916-eaeb25002f39 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:31.035556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595955b-fa7e-4bff-a916-eaeb25002f39 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:31.153699 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3776/7589] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:33:31 2022] POST /identity/v3/domains => generated 277 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:31.157888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91506b74-b3da-497e-8e5b-4218e66b1d51 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:31.168586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91506b74-b3da-497e-8e5b-4218e66b1d51 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:31.236337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afa03f-fccc-4689-9df7-fbeb96a6a4d5 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '283cd42c229e4d5b9110f740fa2b68ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae692deba7c542e1b17f76c5770f856b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:31.243277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afa03f-fccc-4689-9df7-fbeb96a6a4d5 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:31.243277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afa03f-fccc-4689-9df7-fbeb96a6a4d5 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:31.243277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afa03f-fccc-4689-9df7-fbeb96a6a4d5 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] PATH_INFO: `/v3/domains/bbaff86ca9c3448d910dca4e8878ce53` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:31.249262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93afa03f-fccc-4689-9df7-fbeb96a6a4d5 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=bbaff86ca9c3448d910dca4e8878ce53)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:31.250923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93afa03f-fccc-4689-9df7-fbeb96a6a4d5 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:31.270469 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-93afa03f-fccc-4689-9df7-fbeb96a6a4d5 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Cannot delete a domain that is enabled, please disable it first.: keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. May 28 21:33:31.270469 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:31.270469 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:31.270469 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:31.270469 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:31.270469 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:31.270469 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:31.270469 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:31.270469 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:31.270469 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:31.270469 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:31.270469 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:31.270469 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 151, in delete May 28 21:33:31.270469 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_domain( May 28 21:33:31.271607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:31.271607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:31.271607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 802, in delete_domain May 28 21:33:31.271607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._delete_domain(domain, initiator) May 28 21:33:31.271607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 815, in _delete_domain May 28 21:33:31.271607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( May 28 21:33:31.271607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: Cannot delete a domain that is enabled, please disable it first. May 28 21:33:31.271607 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:31.272449 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3814/7590] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:31 2022] DELETE /identity/v3/domains/bbaff86ca9c3448d910dca4e8878ce53 => generated 120 bytes in 115 msecs (HTTP/1.1 403) 5 headers in 184 bytes (2 switches on core 0) May 28 21:33:31.275856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595955b-fa7e-4bff-a916-eaeb25002f39 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:31.286898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595955b-fa7e-4bff-a916-eaeb25002f39 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:31.292297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab780e08-fc85-45a6-95ba-e1decd498e25 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '283cd42c229e4d5b9110f740fa2b68ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae692deba7c542e1b17f76c5770f856b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:31.293159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab780e08-fc85-45a6-95ba-e1decd498e25 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:31.293452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab780e08-fc85-45a6-95ba-e1decd498e25 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:31.293733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab780e08-fc85-45a6-95ba-e1decd498e25 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] PATH_INFO: `/v3/domains/bbaff86ca9c3448d910dca4e8878ce53` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:31.295253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab780e08-fc85-45a6-95ba-e1decd498e25 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=bbaff86ca9c3448d910dca4e8878ce53)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:31.296885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab780e08-fc85-45a6-95ba-e1decd498e25 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:31.318293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ab780e08-fc85-45a6-95ba-e1decd498e25 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'bbaff86ca9c3448d910dca4e8878ce53'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:31.318925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ab780e08-fc85-45a6-95ba-e1decd498e25 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'bbaff86ca9c3448d910dca4e8878ce53'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:31.436598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ab780e08-fc85-45a6-95ba-e1decd498e25 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'bbaff86ca9c3448d910dca4e8878ce53'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:31.437243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ab780e08-fc85-45a6-95ba-e1decd498e25 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'bbaff86ca9c3448d910dca4e8878ce53'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:31.439670 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3777/7591] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:31 2022] PATCH /identity/v3/domains/bbaff86ca9c3448d910dca4e8878ce53 => generated 278 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:31.443731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afa03f-fccc-4689-9df7-fbeb96a6a4d5 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:31.445390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-93afa03f-fccc-4689-9df7-fbeb96a6a4d5 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:31.458651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afa03f-fccc-4689-9df7-fbeb96a6a4d5 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:31.516127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '283cd42c229e4d5b9110f740fa2b68ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae692deba7c542e1b17f76c5770f856b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:31.516988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:31.517280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:31.517562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] PATH_INFO: `/v3/domains/bbaff86ca9c3448d910dca4e8878ce53` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:31.529296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=bbaff86ca9c3448d910dca4e8878ce53)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:31.531092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:31.554290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'bbaff86ca9c3448d910dca4e8878ce53'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:31.558308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'bbaff86ca9c3448d910dca4e8878ce53'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:31.560219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:31.670547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbaff86ca9c3448d910dca4e8878ce53', 'request_id': 'req-ae645a51-ef4a-469b-97e7-a4ee555538af'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:31.676382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbaff86ca9c3448d910dca4e8878ce53', 'request_id': 'req-ae645a51-ef4a-469b-97e7-a4ee555538af'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:31.762904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbaff86ca9c3448d910dca4e8878ce53', 'request_id': 'req-ae645a51-ef4a-469b-97e7-a4ee555538af'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:31.775964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] The token cache is being invalidate because project bbaff86ca9c3448d910dca4e8878ce53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:31.776257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:31.776890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:31.784545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'bbaff86ca9c3448d910dca4e8878ce53', 'request_id': 'req-ae645a51-ef4a-469b-97e7-a4ee555538af'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:31.785218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'bbaff86ca9c3448d910dca4e8878ce53', 'request_id': 'req-ae645a51-ef4a-469b-97e7-a4ee555538af'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:31.788653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'bbaff86ca9c3448d910dca4e8878ce53', 'request_id': 'req-ae645a51-ef4a-469b-97e7-a4ee555538af'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:31.794996 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3815/7592] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:31 2022] DELETE /identity/v3/domains/bbaff86ca9c3448d910dca4e8878ce53 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:31.800085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab780e08-fc85-45a6-95ba-e1decd498e25 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:31.810942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab780e08-fc85-45a6-95ba-e1decd498e25 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:31.868463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee40c0-f5d3-4efe-8776-637f4e00eb58 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '283cd42c229e4d5b9110f740fa2b68ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae692deba7c542e1b17f76c5770f856b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:31.869332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee40c0-f5d3-4efe-8776-637f4e00eb58 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:31.869621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee40c0-f5d3-4efe-8776-637f4e00eb58 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:31.869942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee40c0-f5d3-4efe-8776-637f4e00eb58 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] PATH_INFO: `/v3/domains/4be321d3db9f40ff8d0e92785af17c5f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:31.876423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee40c0-f5d3-4efe-8776-637f4e00eb58 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:33:31.886399 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-d6ee40c0-f5d3-4efe-8776-637f4e00eb58 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Could not find domain: 4be321d3db9f40ff8d0e92785af17c5f.: keystone.exception.DomainNotFound: Could not find domain: 4be321d3db9f40ff8d0e92785af17c5f. May 28 21:33:31.886399 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:31.886399 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 21:33:31.886399 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 21:33:31.886399 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:33:31.886399 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:33:31.886399 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:33:31.886399 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:33:31.886399 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 4be321d3db9f40ff8d0e92785af17c5f. May 28 21:33:31.886399 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:31.886399 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:33:31.886399 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:31.886399 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:31.886399 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:31.886399 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:31.887539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:31.887539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:31.887539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:31.887539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:31.887539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:31.887539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:31.887539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:31.887539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:31.887539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 150, in delete May 28 21:33:31.887539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:delete_domain') May 28 21:33:31.887539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:33:31.887539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:33:31.887539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:33:31.887539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:33:31.887539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:31.887539 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:31.892643 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:33:31.892643 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:33:31.892643 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:33:31.892643 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:33:31.892643 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:33:31.892643 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 21:33:31.892643 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:33:31.892643 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 21:33:31.892643 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:33:31.892643 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:33:31.892643 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:33:31.892643 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 21:33:31.892643 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:33:31.892643 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 21:33:31.892643 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 21:33:31.892643 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 21:33:31.892643 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 4be321d3db9f40ff8d0e92785af17c5f. May 28 21:33:31.893805 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:31.893805 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3778/7593] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:31 2022] DELETE /identity/v3/domains/4be321d3db9f40ff8d0e92785af17c5f => generated 112 bytes in 88 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:33:31.894005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:31.894774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:31.912950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae645a51-ef4a-469b-97e7-a4ee555538af tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:31.918532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b8c90-bc14-4a22-94bb-165c0855475e tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '283cd42c229e4d5b9110f740fa2b68ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae692deba7c542e1b17f76c5770f856b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:31.920490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8c90-bc14-4a22-94bb-165c0855475e tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:31.920490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8c90-bc14-4a22-94bb-165c0855475e tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:31.920490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b8c90-bc14-4a22-94bb-165c0855475e tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:31.920490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b8c90-bc14-4a22-94bb-165c0855475e tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:31.922176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b8c90-bc14-4a22-94bb-165c0855475e tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:32.024070 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3816/7594] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:33:31 2022] POST /identity/v3/domains => generated 253 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:32.025802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee40c0-f5d3-4efe-8776-637f4e00eb58 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:32.036895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee40c0-f5d3-4efe-8776-637f4e00eb58 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:32.095117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ad6c1-4168-41ff-a27b-d55576acb522 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '283cd42c229e4d5b9110f740fa2b68ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae692deba7c542e1b17f76c5770f856b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:32.095819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ad6c1-4168-41ff-a27b-d55576acb522 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:32.096147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ad6c1-4168-41ff-a27b-d55576acb522 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:32.096437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12ad6c1-4168-41ff-a27b-d55576acb522 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:32.097151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ad6c1-4168-41ff-a27b-d55576acb522 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:32.098841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12ad6c1-4168-41ff-a27b-d55576acb522 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:32.294554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.sql.core [None req-c12ad6c1-4168-41ff-a27b-d55576acb522 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-1835297797' for key 'project.ixu_project_name_domain_id'") May 28 21:33:32.294554 ncdv-master-1 devstack@keystone.service[103104]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 28 21:33:32.294554 ncdv-master-1 devstack@keystone.service[103104]: [parameters: {'id': '58274950df1c44a6866b6f062dbe209f', 'name': 'tempest-domain-dup-1835297797', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] May 28 21:33:32.294554 ncdv-master-1 devstack@keystone.service[103104]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=103104) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 28 21:33:32.319103 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-c12ad6c1-4168-41ff-a27b-d55576acb522 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1835297797.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1835297797. May 28 21:33:32.319103 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:32.319103 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 21:33:32.319103 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 21:33:32.319103 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 21:33:32.319103 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 21:33:32.319103 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 21:33:32.319103 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application result = self._query(query) May 28 21:33:32.319103 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 21:33:32.319103 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application conn.query(q) May 28 21:33:32.319103 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 21:33:32.319103 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 21:33:32.319103 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application result.read() May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-tempest-domain-dup-1835297797' for key 'project.ixu_project_name_domain_id'") May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 28 21:33:32.320238 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return project_ref.to_dict() May 28 21:33:32.321672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 21:33:32.321672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application next(self.gen) May 28 21:33:32.321672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 28 21:33:32.321672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application yield resource May 28 21:33:32.321672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 21:33:32.321672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application next(self.gen) May 28 21:33:32.321672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 28 21:33:32.321672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.session.rollback() May 28 21:33:32.321672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 28 21:33:32.321672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.force_reraise() May 28 21:33:32.321672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 28 21:33:32.321672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise self.value May 28 21:33:32.321672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 28 21:33:32.321672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 28 21:33:32.321672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 28 21:33:32.321672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application session.commit() May 28 21:33:32.321672 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 28 21:33:32.323018 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 28 21:33:32.323018 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 28 21:33:32.323018 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._prepare_impl() May 28 21:33:32.323018 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 28 21:33:32.323018 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.session.flush() May 28 21:33:32.323018 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 28 21:33:32.323018 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._flush(objects) May 28 21:33:32.323018 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 28 21:33:32.323018 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 28 21:33:32.323018 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 28 21:33:32.323018 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application compat.raise_( May 28 21:33:32.323018 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 21:33:32.323018 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception May 28 21:33:32.323018 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 28 21:33:32.323018 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application flush_context.execute() May 28 21:33:32.323018 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 28 21:33:32.323018 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rec.execute(self) May 28 21:33:32.324862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 28 21:33:32.324862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 28 21:33:32.324862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 28 21:33:32.324862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application _emit_insert_statements( May 28 21:33:32.324862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 28 21:33:32.324862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application c = connection._execute_20( May 28 21:33:32.324862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 28 21:33:32.324862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 28 21:33:32.324862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 28 21:33:32.324862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 28 21:33:32.324862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 28 21:33:32.324862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ret = self._execute_context( May 28 21:33:32.324862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 28 21:33:32.324862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 28 21:33:32.324862 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 28 21:33:32.326106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 28 21:33:32.326106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 21:33:32.326106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception May 28 21:33:32.326106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 28 21:33:32.326106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.dialect.do_execute( May 28 21:33:32.326106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 28 21:33:32.326106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 28 21:33:32.326106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 28 21:33:32.326106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application result = self._query(query) May 28 21:33:32.326106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 28 21:33:32.326106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application conn.query(q) May 28 21:33:32.326106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 28 21:33:32.326106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 28 21:33:32.326106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 28 21:33:32.326106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application result.read() May 28 21:33:32.326106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 28 21:33:32.326106 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 28 21:33:32.327291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 28 21:33:32.327291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application packet.raise_for_error() May 28 21:33:32.327291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 28 21:33:32.327291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 28 21:33:32.327291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 28 21:33:32.327291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 28 21:33:32.327291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-tempest-domain-dup-1835297797' for key 'project.ixu_project_name_domain_id'") May 28 21:33:32.327291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 28 21:33:32.327291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [parameters: {'id': '58274950df1c44a6866b6f062dbe209f', 'name': 'tempest-domain-dup-1835297797', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] May 28 21:33:32.327291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 28 21:33:32.327291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:32.327291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:33:32.327291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:32.327291 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:32.328321 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 28 21:33:32.328321 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 28 21:33:32.328321 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 28 21:33:32.328321 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 28 21:33:32.328321 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name tempest-domain-dup-1835297797. May 28 21:33:32.328321 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:32.328321 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:33:32.328321 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:32.328321 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:32.328321 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:32.328321 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:32.328321 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:32.328321 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:32.328321 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:32.328321 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:32.328321 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:32.328321 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:32.329568 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:32.329568 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:32.329568 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post May 28 21:33:32.329568 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( May 28 21:33:32.329568 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:32.329568 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:32.329568 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain May 28 21:33:32.329568 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application is_domain_project = self.create_project( May 28 21:33:32.329568 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:32.329568 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:32.329568 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 28 21:33:32.329568 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.Conflict( May 28 21:33:32.329568 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: tempest-domain-dup-1835297797. May 28 21:33:32.329568 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:32.329568 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3779/7595] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:33:32 2022] POST /identity/v3/domains => generated 212 bytes in 295 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 28 21:33:32.330735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b8c90-bc14-4a22-94bb-165c0855475e tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:32.335726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b8c90-bc14-4a22-94bb-165c0855475e tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:32.340856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da40cd1-761c-43c0-95e8-7331f019872a tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '283cd42c229e4d5b9110f740fa2b68ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae692deba7c542e1b17f76c5770f856b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:32.341749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da40cd1-761c-43c0-95e8-7331f019872a tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:32.342084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da40cd1-761c-43c0-95e8-7331f019872a tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:32.342369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da40cd1-761c-43c0-95e8-7331f019872a tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] PATH_INFO: `/v3/domains/012314f0cf4440dd9c4607264b94ceab` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:32.354355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da40cd1-761c-43c0-95e8-7331f019872a tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=012314f0cf4440dd9c4607264b94ceab)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:32.355997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da40cd1-761c-43c0-95e8-7331f019872a tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:32.375488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2da40cd1-761c-43c0-95e8-7331f019872a tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '012314f0cf4440dd9c4607264b94ceab'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:32.376138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2da40cd1-761c-43c0-95e8-7331f019872a tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '012314f0cf4440dd9c4607264b94ceab'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:32.526894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2da40cd1-761c-43c0-95e8-7331f019872a tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '012314f0cf4440dd9c4607264b94ceab'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:32.527551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2da40cd1-761c-43c0-95e8-7331f019872a tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '012314f0cf4440dd9c4607264b94ceab'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:32.530108 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3817/7596] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:32 2022] PATCH /identity/v3/domains/012314f0cf4440dd9c4607264b94ceab => generated 254 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:32.534349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ad6c1-4168-41ff-a27b-d55576acb522 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:32.536098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c12ad6c1-4168-41ff-a27b-d55576acb522 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:32.551624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12ad6c1-4168-41ff-a27b-d55576acb522 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:32.585332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:32.611058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '283cd42c229e4d5b9110f740fa2b68ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae692deba7c542e1b17f76c5770f856b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:32.611931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:32.612222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:32.612503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] PATH_INFO: `/v3/domains/012314f0cf4440dd9c4607264b94ceab` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:32.622991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=012314f0cf4440dd9c4607264b94ceab)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:32.624615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:32.646181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '012314f0cf4440dd9c4607264b94ceab'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:32.650011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '012314f0cf4440dd9c4607264b94ceab'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:32.780633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '012314f0cf4440dd9c4607264b94ceab', 'request_id': 'req-daaf94bd-4b17-4eba-9e37-9e7d5a969339'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:32.786059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '012314f0cf4440dd9c4607264b94ceab', 'request_id': 'req-daaf94bd-4b17-4eba-9e37-9e7d5a969339'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:32.916154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '012314f0cf4440dd9c4607264b94ceab', 'request_id': 'req-daaf94bd-4b17-4eba-9e37-9e7d5a969339'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:32.928137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] The token cache is being invalidate because project 012314f0cf4440dd9c4607264b94ceab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:32.928426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:32.929046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:32.936254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '012314f0cf4440dd9c4607264b94ceab', 'request_id': 'req-daaf94bd-4b17-4eba-9e37-9e7d5a969339'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:32.936896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '012314f0cf4440dd9c4607264b94ceab', 'request_id': 'req-daaf94bd-4b17-4eba-9e37-9e7d5a969339'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:32.940135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '012314f0cf4440dd9c4607264b94ceab', 'request_id': 'req-daaf94bd-4b17-4eba-9e37-9e7d5a969339'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:32.949335 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3780/7597] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:32 2022] DELETE /identity/v3/domains/012314f0cf4440dd9c4607264b94ceab => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:33:32.954635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da40cd1-761c-43c0-95e8-7331f019872a tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:32.966932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da40cd1-761c-43c0-95e8-7331f019872a tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:33.027102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49ab39-18ab-4d7f-b129-a93821b3df4b tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '283cd42c229e4d5b9110f740fa2b68ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ae692deba7c542e1b17f76c5770f856b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:33.027968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49ab39-18ab-4d7f-b129-a93821b3df4b tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:33.028255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49ab39-18ab-4d7f-b129-a93821b3df4b tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:33.028536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49ab39-18ab-4d7f-b129-a93821b3df4b tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] PATH_INFO: `/v3/domains/bbaff86ca9c3448d910dca4e8878ce53` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:33.036910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49ab39-18ab-4d7f-b129-a93821b3df4b tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:33:33.047231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-5e49ab39-18ab-4d7f-b129-a93821b3df4b tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Could not find domain: bbaff86ca9c3448d910dca4e8878ce53.: keystone.exception.DomainNotFound: Could not find domain: bbaff86ca9c3448d910dca4e8878ce53. May 28 21:33:33.047231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:33.047231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 21:33:33.047231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 21:33:33.047231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:33:33.047231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:33:33.047231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:33:33.047231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:33:33.047231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: bbaff86ca9c3448d910dca4e8878ce53. May 28 21:33:33.047231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:33.047231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:33:33.047231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:33.047231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:33.047231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:33.047231 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:33.048333 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:33.048333 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:33.048333 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:33.048333 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:33.048333 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:33.048333 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:33.048333 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:33.048333 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:33.048333 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch May 28 21:33:33.048333 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') May 28 21:33:33.048333 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:33:33.048333 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:33:33.048333 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:33:33.048333 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:33:33.048333 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:33.048333 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:33.049273 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:33:33.049273 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:33:33.049273 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:33:33.049273 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:33:33.049273 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:33:33.049273 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 21:33:33.049273 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:33:33.049273 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 21:33:33.049273 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:33:33.049273 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:33:33.049273 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:33:33.049273 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 21:33:33.049273 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:33:33.049273 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 21:33:33.049273 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 21:33:33.049273 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 21:33:33.049273 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: bbaff86ca9c3448d910dca4e8878ce53. May 28 21:33:33.050397 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:33:33.050397 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3818/7598] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:32 2022] PATCH /identity/v3/domains/bbaff86ca9c3448d910dca4e8878ce53 => generated 112 bytes in 94 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) May 28 21:33:33.053880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:33.055374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:33.131701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf94bd-4b17-4eba-9e37-9e7d5a969339 tempest-DomainsNegativeTestJSON-2119190974 tempest-DomainsNegativeTestJSON-2119190974-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:33.137059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63648763-5f1f-4e04-ab3f-1e462360cc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:33.137974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63648763-5f1f-4e04-ab3f-1e462360cc32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:33.138262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63648763-5f1f-4e04-ab3f-1e462360cc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:33.138540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63648763-5f1f-4e04-ab3f-1e462360cc32 admin admin] PATH_INFO: `/v3/users/471a767f42574cdbbdbc4c1f5a0e28a7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:33.139757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63648763-5f1f-4e04-ab3f-1e462360cc32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=471a767f42574cdbbdbc4c1f5a0e28a7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:33.141747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63648763-5f1f-4e04-ab3f-1e462360cc32 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:33.155525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:33.366131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-63648763-5f1f-4e04-ab3f-1e462360cc32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '471a767f42574cdbbdbc4c1f5a0e28a7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:33.366789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-63648763-5f1f-4e04-ab3f-1e462360cc32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '471a767f42574cdbbdbc4c1f5a0e28a7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:33.374226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-63648763-5f1f-4e04-ab3f-1e462360cc32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '471a767f42574cdbbdbc4c1f5a0e28a7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:33.448956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-63648763-5f1f-4e04-ab3f-1e462360cc32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '471a767f42574cdbbdbc4c1f5a0e28a7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:33.449596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-63648763-5f1f-4e04-ab3f-1e462360cc32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '471a767f42574cdbbdbc4c1f5a0e28a7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:33.468056 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3781/7599] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:33 2022] DELETE /identity/v3/users/471a767f42574cdbbdbc4c1f5a0e28a7 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:33.472467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84164d7d-9197-423d-af95-7b1e9bc07417 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:33.472954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84164d7d-9197-423d-af95-7b1e9bc07417 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:33.473060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84164d7d-9197-423d-af95-7b1e9bc07417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:33.633200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=2,keystone:UPDATE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:33.639370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-84164d7d-9197-423d-af95-7b1e9bc07417 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:33.645751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-84164d7d-9197-423d-af95-7b1e9bc07417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:33.700355 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3819/7600] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:33.710791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63648763-5f1f-4e04-ab3f-1e462360cc32 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:33.711939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-63648763-5f1f-4e04-ab3f-1e462360cc32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:33.772221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63648763-5f1f-4e04-ab3f-1e462360cc32 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:33.775820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859acb3-a08b-47ea-af49-8ae5b83a788c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:33.776483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859acb3-a08b-47ea-af49-8ae5b83a788c service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:33.776766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859acb3-a08b-47ea-af49-8ae5b83a788c service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:33.777046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859acb3-a08b-47ea-af49-8ae5b83a788c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:33.786237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859acb3-a08b-47ea-af49-8ae5b83a788c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:33.787716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859acb3-a08b-47ea-af49-8ae5b83a788c service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:33.797810 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3782/7601] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:33:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:33:34.222120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84164d7d-9197-423d-af95-7b1e9bc07417 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:34.223573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-84164d7d-9197-423d-af95-7b1e9bc07417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:34.237995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84164d7d-9197-423d-af95-7b1e9bc07417 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:34.243380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c12a11-939f-4bc4-880c-0429199e83d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:34.244875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c12a11-939f-4bc4-880c-0429199e83d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:34.244875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c12a11-939f-4bc4-880c-0429199e83d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:34.245126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c12a11-939f-4bc4-880c-0429199e83d9 admin admin] PATH_INFO: `/v3/projects/16cc1fdcab5c44fabe915217750364fa` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:34.246321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c12a11-939f-4bc4-880c-0429199e83d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16cc1fdcab5c44fabe915217750364fa)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:34.248245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c12a11-939f-4bc4-880c-0429199e83d9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:34.380313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-80c12a11-939f-4bc4-880c-0429199e83d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16cc1fdcab5c44fabe915217750364fa', 'request_id': 'req-80c12a11-939f-4bc4-880c-0429199e83d9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:34.386785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-80c12a11-939f-4bc4-880c-0429199e83d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16cc1fdcab5c44fabe915217750364fa', 'request_id': 'req-80c12a11-939f-4bc4-880c-0429199e83d9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:34.482058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-80c12a11-939f-4bc4-880c-0429199e83d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16cc1fdcab5c44fabe915217750364fa', 'request_id': 'req-80c12a11-939f-4bc4-880c-0429199e83d9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:34.493523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-80c12a11-939f-4bc4-880c-0429199e83d9 admin admin] The token cache is being invalidate because project 16cc1fdcab5c44fabe915217750364fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:34.493803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-80c12a11-939f-4bc4-880c-0429199e83d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:34.494575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-80c12a11-939f-4bc4-880c-0429199e83d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:34.496211 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3820/7602] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:34 2022] DELETE /identity/v3/projects/16cc1fdcab5c44fabe915217750364fa => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:34.500158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859acb3-a08b-47ea-af49-8ae5b83a788c service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:34.510453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859acb3-a08b-47ea-af49-8ae5b83a788c service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:34.572659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4658708-9e53-4046-ae5b-d02cd92327f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:34.579615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4658708-9e53-4046-ae5b-d02cd92327f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:34.579615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4658708-9e53-4046-ae5b-d02cd92327f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:34.579615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4658708-9e53-4046-ae5b-d02cd92327f7 admin admin] PATH_INFO: `/v3/users/283cd42c229e4d5b9110f740fa2b68ab` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:34.579615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4658708-9e53-4046-ae5b-d02cd92327f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=283cd42c229e4d5b9110f740fa2b68ab)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:34.579615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4658708-9e53-4046-ae5b-d02cd92327f7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:34.797633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:34.806346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a4658708-9e53-4046-ae5b-d02cd92327f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '283cd42c229e4d5b9110f740fa2b68ab'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:34.807006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a4658708-9e53-4046-ae5b-d02cd92327f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '283cd42c229e4d5b9110f740fa2b68ab'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:34.813745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a4658708-9e53-4046-ae5b-d02cd92327f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '283cd42c229e4d5b9110f740fa2b68ab'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:34.902325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a4658708-9e53-4046-ae5b-d02cd92327f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '283cd42c229e4d5b9110f740fa2b68ab'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:34.902952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a4658708-9e53-4046-ae5b-d02cd92327f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '283cd42c229e4d5b9110f740fa2b68ab'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:34.921515 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3783/7603] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:34 2022] DELETE /identity/v3/users/283cd42c229e4d5b9110f740fa2b68ab => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:35.369465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c12a11-939f-4bc4-880c-0429199e83d9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:35.372731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-80c12a11-939f-4bc4-880c-0429199e83d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:35.385356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c12a11-939f-4bc4-880c-0429199e83d9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:35.446138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140f17e-cccb-4d45-94b2-ffdf2aab08c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:35.447665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140f17e-cccb-4d45-94b2-ffdf2aab08c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:35.447954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140f17e-cccb-4d45-94b2-ffdf2aab08c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:35.448233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140f17e-cccb-4d45-94b2-ffdf2aab08c1 admin admin] PATH_INFO: `/v3/projects/ae692deba7c542e1b17f76c5770f856b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:35.449440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140f17e-cccb-4d45-94b2-ffdf2aab08c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae692deba7c542e1b17f76c5770f856b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:35.451536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140f17e-cccb-4d45-94b2-ffdf2aab08c1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:35.621430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:35.632893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8140f17e-cccb-4d45-94b2-ffdf2aab08c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae692deba7c542e1b17f76c5770f856b', 'request_id': 'req-8140f17e-cccb-4d45-94b2-ffdf2aab08c1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:35.641192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8140f17e-cccb-4d45-94b2-ffdf2aab08c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae692deba7c542e1b17f76c5770f856b', 'request_id': 'req-8140f17e-cccb-4d45-94b2-ffdf2aab08c1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:35.751958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8140f17e-cccb-4d45-94b2-ffdf2aab08c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae692deba7c542e1b17f76c5770f856b', 'request_id': 'req-8140f17e-cccb-4d45-94b2-ffdf2aab08c1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:35.764259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8140f17e-cccb-4d45-94b2-ffdf2aab08c1 admin admin] The token cache is being invalidate because project ae692deba7c542e1b17f76c5770f856b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:35.764550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8140f17e-cccb-4d45-94b2-ffdf2aab08c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:35.765178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8140f17e-cccb-4d45-94b2-ffdf2aab08c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:35.766750 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3821/7604] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:35 2022] DELETE /identity/v3/projects/ae692deba7c542e1b17f76c5770f856b => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:35.771729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5da2e-f71d-4027-811f-ce3ecb3bc563 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:35.772232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5da2e-f71d-4027-811f-ce3ecb3bc563 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:35.772649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5da2e-f71d-4027-811f-ce3ecb3bc563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:35.792719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f8b5da2e-f71d-4027-811f-ce3ecb3bc563 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:35.793940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f8b5da2e-f71d-4027-811f-ce3ecb3bc563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:35.852100 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3784/7605] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:35.860247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2892b-4e63-4126-b4b0-560845f1a9e9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:35.860532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2892b-4e63-4126-b4b0-560845f1a9e9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:35.860809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2892b-4e63-4126-b4b0-560845f1a9e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:35.878985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9aa2892b-4e63-4126-b4b0-560845f1a9e9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:35.881505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9aa2892b-4e63-4126-b4b0-560845f1a9e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:35.888069 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3822/7606] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:35.892192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5da2e-f71d-4027-811f-ce3ecb3bc563 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:35.902518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5da2e-f71d-4027-811f-ce3ecb3bc563 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:35.907255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d09c1b-b238-4eab-ba69-edb01d5b296d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:35.907923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d09c1b-b238-4eab-ba69-edb01d5b296d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:35.908205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d09c1b-b238-4eab-ba69-edb01d5b296d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:35.908480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d09c1b-b238-4eab-ba69-edb01d5b296d admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:35.909260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d09c1b-b238-4eab-ba69-edb01d5b296d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:33:35.909565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d09c1b-b238-4eab-ba69-edb01d5b296d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:35.911240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d09c1b-b238-4eab-ba69-edb01d5b296d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:35.923813 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3785/7607] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:35.928116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2892b-4e63-4126-b4b0-560845f1a9e9 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:35.939322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2892b-4e63-4126-b4b0-560845f1a9e9 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:35.944071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50525d5f-4b00-45bd-adac-922dd3a92423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:35.944721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50525d5f-4b00-45bd-adac-922dd3a92423 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:35.945003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50525d5f-4b00-45bd-adac-922dd3a92423 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:35.945278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50525d5f-4b00-45bd-adac-922dd3a92423 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:35.946001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50525d5f-4b00-45bd-adac-922dd3a92423 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:35.947899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50525d5f-4b00-45bd-adac-922dd3a92423 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:36.098623 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3823/7608] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:35 2022] POST /identity/v3/projects => generated 367 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:36.103420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d09c1b-b238-4eab-ba69-edb01d5b296d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:36.114332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d09c1b-b238-4eab-ba69-edb01d5b296d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:36.184779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed665de9-ca7f-4ff0-8841-dcd93043c02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:36.185458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed665de9-ca7f-4ff0-8841-dcd93043c02a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:36.185745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed665de9-ca7f-4ff0-8841-dcd93043c02a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:36.186062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed665de9-ca7f-4ff0-8841-dcd93043c02a admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:36.186766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed665de9-ca7f-4ff0-8841-dcd93043c02a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:36.188685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed665de9-ca7f-4ff0-8841-dcd93043c02a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:36.272406 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3786/7609] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:36 2022] POST /identity/v3/users => generated 284 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:36.277293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50525d5f-4b00-45bd-adac-922dd3a92423 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:36.287683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50525d5f-4b00-45bd-adac-922dd3a92423 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:36.293186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8f60e-bb0b-4c5e-b094-ede9f31a5d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:36.293864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8f60e-bb0b-4c5e-b094-ede9f31a5d87 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:36.294159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8f60e-bb0b-4c5e-b094-ede9f31a5d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:36.294437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8f60e-bb0b-4c5e-b094-ede9f31a5d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:36.295173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8f60e-bb0b-4c5e-b094-ede9f31a5d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:36.296762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8f60e-bb0b-4c5e-b094-ede9f31a5d87 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:36.306743 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3824/7610] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:36.310684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed665de9-ca7f-4ff0-8841-dcd93043c02a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:36.320878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed665de9-ca7f-4ff0-8841-dcd93043c02a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:36.325553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cebb51-9650-4c62-8206-3ee7f922ac70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:36.326253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cebb51-9650-4c62-8206-3ee7f922ac70 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:36.326542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cebb51-9650-4c62-8206-3ee7f922ac70 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:36.326824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cebb51-9650-4c62-8206-3ee7f922ac70 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:36.327561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cebb51-9650-4c62-8206-3ee7f922ac70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:36.329171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cebb51-9650-4c62-8206-3ee7f922ac70 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:36.339802 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3787/7611] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:36.343751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8f60e-bb0b-4c5e-b094-ede9f31a5d87 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:36.355009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8f60e-bb0b-4c5e-b094-ede9f31a5d87 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:36.359708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f46f6-5bdc-4af7-885a-83492a8016ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:36.360601 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f46f6-5bdc-4af7-885a-83492a8016ab admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:36.363480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f46f6-5bdc-4af7-885a-83492a8016ab admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:36.363480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f46f6-5bdc-4af7-885a-83492a8016ab admin admin] PATH_INFO: `/v3/projects/4fe55c90ce30473d92c1a8ad013719ab/users/b88542c6d3a54954a5e897093c7229b9/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:36.375919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f46f6-5bdc-4af7-885a-83492a8016ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fe55c90ce30473d92c1a8ad013719ab, user_id=b88542c6d3a54954a5e897093c7229b9, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:36.378648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f46f6-5bdc-4af7-885a-83492a8016ab admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:36.478578 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3825/7612] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:36 2022] PUT /identity/v3/projects/4fe55c90ce30473d92c1a8ad013719ab/users/b88542c6d3a54954a5e897093c7229b9/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:33:36.489573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1da202-b08b-44cf-88d7-dafa4cd1072d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:36.489898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1da202-b08b-44cf-88d7-dafa4cd1072d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:36.490186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1da202-b08b-44cf-88d7-dafa4cd1072d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:36.516708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ac1da202-b08b-44cf-88d7-dafa4cd1072d None None] MFA Rules not processed for user `b88542c6d3a54954a5e897093c7229b9`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:36.517970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ac1da202-b08b-44cf-88d7-dafa4cd1072d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:36.527100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:36.582355 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3788/7613] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:36.586903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f46f6-5bdc-4af7-885a-83492a8016ab admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:36.653926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f46f6-5bdc-4af7-885a-83492a8016ab admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:36.658714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d388826-13bf-40a5-8117-7770c0555bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:36.659359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d388826-13bf-40a5-8117-7770c0555bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:36.659647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d388826-13bf-40a5-8117-7770c0555bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:36.659935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d388826-13bf-40a5-8117-7770c0555bd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:36.660650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d388826-13bf-40a5-8117-7770c0555bd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:36.662548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d388826-13bf-40a5-8117-7770c0555bd5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:36.768520 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3826/7614] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:36 2022] POST /identity/v3/projects => generated 367 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:36.772461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1da202-b08b-44cf-88d7-dafa4cd1072d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:36.784053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1da202-b08b-44cf-88d7-dafa4cd1072d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:36.844254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c3918-9cff-413d-90ee-d84e0dd8dbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:36.844994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c3918-9cff-413d-90ee-d84e0dd8dbee admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:36.845230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c3918-9cff-413d-90ee-d84e0dd8dbee admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:36.845510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463c3918-9cff-413d-90ee-d84e0dd8dbee admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:36.846374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c3918-9cff-413d-90ee-d84e0dd8dbee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:36.848326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463c3918-9cff-413d-90ee-d84e0dd8dbee admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:36.962528 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3789/7615] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:36 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:36.968707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d388826-13bf-40a5-8117-7770c0555bd5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:36.979370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d388826-13bf-40a5-8117-7770c0555bd5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:36.984166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6fcda-cc6a-414f-877d-f3868ac4a187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:36.984826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6fcda-cc6a-414f-877d-f3868ac4a187 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:36.985112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6fcda-cc6a-414f-877d-f3868ac4a187 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:36.985389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a6fcda-cc6a-414f-877d-f3868ac4a187 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:36.986161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6fcda-cc6a-414f-877d-f3868ac4a187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:36.987776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a6fcda-cc6a-414f-877d-f3868ac4a187 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:36.998652 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3827/7616] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:37.002600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c3918-9cff-413d-90ee-d84e0dd8dbee admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:37.013102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463c3918-9cff-413d-90ee-d84e0dd8dbee admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:37.017653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eea468-a496-423f-a8ae-f820952e5df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:37.018563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eea468-a496-423f-a8ae-f820952e5df9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:37.018849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eea468-a496-423f-a8ae-f820952e5df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:37.019124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eea468-a496-423f-a8ae-f820952e5df9 admin admin] PATH_INFO: `/v3/projects/813d44d35f774f42ae7d5d78fce74798/users/abc3a19461344532a15b743cc207d11c/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:37.033639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eea468-a496-423f-a8ae-f820952e5df9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=813d44d35f774f42ae7d5d78fce74798, user_id=abc3a19461344532a15b743cc207d11c, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:37.036319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eea468-a496-423f-a8ae-f820952e5df9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:37.209552 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3790/7617] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:37 2022] PUT /identity/v3/projects/813d44d35f774f42ae7d5d78fce74798/users/abc3a19461344532a15b743cc207d11c/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:33:37.215170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6fcda-cc6a-414f-877d-f3868ac4a187 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:37.225106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a6fcda-cc6a-414f-877d-f3868ac4a187 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:37.260928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:37.288043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dddc8a-cba7-4fe4-8871-a8184c5a5f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:37.288728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dddc8a-cba7-4fe4-8871-a8184c5a5f4e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:37.288991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dddc8a-cba7-4fe4-8871-a8184c5a5f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:37.289264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dddc8a-cba7-4fe4-8871-a8184c5a5f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:37.290556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dddc8a-cba7-4fe4-8871-a8184c5a5f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:37.292152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dddc8a-cba7-4fe4-8871-a8184c5a5f4e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:37.301906 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3828/7618] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:37.306785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eea468-a496-423f-a8ae-f820952e5df9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:37.317325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eea468-a496-423f-a8ae-f820952e5df9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:37.322116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b534c4-4db7-429c-be95-0e8d04d682ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:37.322990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b534c4-4db7-429c-be95-0e8d04d682ed admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:37.323277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b534c4-4db7-429c-be95-0e8d04d682ed admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:37.323551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b534c4-4db7-429c-be95-0e8d04d682ed admin admin] PATH_INFO: `/v3/projects/813d44d35f774f42ae7d5d78fce74798/users/abc3a19461344532a15b743cc207d11c/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:37.325516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b534c4-4db7-429c-be95-0e8d04d682ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=813d44d35f774f42ae7d5d78fce74798, user_id=abc3a19461344532a15b743cc207d11c, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:37.328205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b534c4-4db7-429c-be95-0e8d04d682ed admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:37.338496 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3791/7619] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:37 2022] PUT /identity/v3/projects/813d44d35f774f42ae7d5d78fce74798/users/abc3a19461344532a15b743cc207d11c/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:37.348945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74adc38-32ef-461f-96ca-022547f5e86e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:37.349357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74adc38-32ef-461f-96ca-022547f5e86e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:37.349714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74adc38-32ef-461f-96ca-022547f5e86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:37.379011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e74adc38-32ef-461f-96ca-022547f5e86e None None] MFA Rules not processed for user `abc3a19461344532a15b743cc207d11c`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:37.380211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e74adc38-32ef-461f-96ca-022547f5e86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:37.447623 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3829/7620] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:37 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:37.452027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b534c4-4db7-429c-be95-0e8d04d682ed admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:37.463412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b534c4-4db7-429c-be95-0e8d04d682ed admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:37.469030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b4a30-50bf-4488-958c-e9eaeec84a37 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abc3a19461344532a15b743cc207d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813d44d35f774f42ae7d5d78fce74798', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:37.469646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b4a30-50bf-4488-958c-e9eaeec84a37 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:37.470040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b4a30-50bf-4488-958c-e9eaeec84a37 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:37.470369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b4a30-50bf-4488-958c-e9eaeec84a37 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:37.471135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b4a30-50bf-4488-958c-e9eaeec84a37 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:37.472832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b4a30-50bf-4488-958c-e9eaeec84a37 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:37.591406 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3792/7621] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:33:37 2022] POST /identity/v3/domains => generated 276 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:37.595816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74adc38-32ef-461f-96ca-022547f5e86e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:37.606120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74adc38-32ef-461f-96ca-022547f5e86e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:37.663269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb62e62-8b38-4ba2-a014-c47e0cf543fc tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abc3a19461344532a15b743cc207d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813d44d35f774f42ae7d5d78fce74798', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:37.663972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb62e62-8b38-4ba2-a014-c47e0cf543fc tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:37.664232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb62e62-8b38-4ba2-a014-c47e0cf543fc tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:37.664521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb62e62-8b38-4ba2-a014-c47e0cf543fc tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:37.665877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb62e62-8b38-4ba2-a014-c47e0cf543fc tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:37.667912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb62e62-8b38-4ba2-a014-c47e0cf543fc tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:37.750968 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3830/7622] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:37 2022] POST /identity/v3/users => generated 347 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:37.754688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b4a30-50bf-4488-958c-e9eaeec84a37 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:37.765440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b4a30-50bf-4488-958c-e9eaeec84a37 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:37.770392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f6ffa-6a59-435b-8e55-7cabba9c0d20 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abc3a19461344532a15b743cc207d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813d44d35f774f42ae7d5d78fce74798', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:37.771068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f6ffa-6a59-435b-8e55-7cabba9c0d20 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:37.771348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f6ffa-6a59-435b-8e55-7cabba9c0d20 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:37.771627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f6ffa-6a59-435b-8e55-7cabba9c0d20 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:37.772329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294f6ffa-6a59-435b-8e55-7cabba9c0d20 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:37.774372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294f6ffa-6a59-435b-8e55-7cabba9c0d20 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:37.859808 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3793/7623] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:37 2022] POST /identity/v3/users => generated 323 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:33:37.864766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb62e62-8b38-4ba2-a014-c47e0cf543fc tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:37.875037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb62e62-8b38-4ba2-a014-c47e0cf543fc tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:37.879844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d383870-8309-45c9-8e8b-2114c482b1e4 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abc3a19461344532a15b743cc207d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813d44d35f774f42ae7d5d78fce74798', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:37.880797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d383870-8309-45c9-8e8b-2114c482b1e4 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:37.881087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d383870-8309-45c9-8e8b-2114c482b1e4 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:37.881366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d383870-8309-45c9-8e8b-2114c482b1e4 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] PATH_INFO: `/v3/users/4b752fc7b47745a1a7c2aa144af039ef` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:37.895451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d383870-8309-45c9-8e8b-2114c482b1e4 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorizing `identity:get_user(user_id=4b752fc7b47745a1a7c2aa144af039ef)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:37.897694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d383870-8309-45c9-8e8b-2114c482b1e4 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:37.899924 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3831/7624] 10.222.0.33 () {58 vars in 1220 bytes} [Sat May 28 21:33:37 2022] GET /identity/v3/users/4b752fc7b47745a1a7c2aa144af039ef => generated 347 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:37.905219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f6ffa-6a59-435b-8e55-7cabba9c0d20 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:37.916466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f6ffa-6a59-435b-8e55-7cabba9c0d20 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:37.921216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bae85-440f-42b2-9be9-b53969abec2d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abc3a19461344532a15b743cc207d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813d44d35f774f42ae7d5d78fce74798', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:37.921914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bae85-440f-42b2-9be9-b53969abec2d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:37.922211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bae85-440f-42b2-9be9-b53969abec2d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:37.922510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bae85-440f-42b2-9be9-b53969abec2d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:37.923320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566bae85-440f-42b2-9be9-b53969abec2d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Adding query filter params (domain_id=b3186a1dccef4e948c742adea2e3632c) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:33:37.923624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566bae85-440f-42b2-9be9-b53969abec2d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:37.925659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566bae85-440f-42b2-9be9-b53969abec2d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:37.940604 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3794/7625] 10.222.0.33 () {58 vars in 1206 bytes} [Sat May 28 21:33:37 2022] GET /identity/v3/users?domain_id=b3186a1dccef4e948c742adea2e3632c => generated 484 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:37.945707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d383870-8309-45c9-8e8b-2114c482b1e4 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:37.955897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d383870-8309-45c9-8e8b-2114c482b1e4 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:37.960579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d0adc-74fb-413c-bd64-8bb5c06ca58a tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abc3a19461344532a15b743cc207d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813d44d35f774f42ae7d5d78fce74798', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:37.961251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d0adc-74fb-413c-bd64-8bb5c06ca58a tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:37.961570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d0adc-74fb-413c-bd64-8bb5c06ca58a tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:37.962441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d0adc-74fb-413c-bd64-8bb5c06ca58a tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:37.963251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d0adc-74fb-413c-bd64-8bb5c06ca58a tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:37.965550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d0adc-74fb-413c-bd64-8bb5c06ca58a tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:37.993013 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3832/7626] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:37 2022] GET /identity/v3/users => generated 7731 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:33:37.998213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bae85-440f-42b2-9be9-b53969abec2d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:38.008501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bae85-440f-42b2-9be9-b53969abec2d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:38.013660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da17af9-3761-4e77-abcd-aae2ec392756 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abc3a19461344532a15b743cc207d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813d44d35f774f42ae7d5d78fce74798', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:38.014354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da17af9-3761-4e77-abcd-aae2ec392756 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:38.014647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da17af9-3761-4e77-abcd-aae2ec392756 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:38.014935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da17af9-3761-4e77-abcd-aae2ec392756 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:38.015745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da17af9-3761-4e77-abcd-aae2ec392756 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Adding query filter params (name=tempest-test_user-2027027914) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:33:38.016056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da17af9-3761-4e77-abcd-aae2ec392756 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:38.017954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da17af9-3761-4e77-abcd-aae2ec392756 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:38.053054 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3795/7627] 10.222.0.33 () {58 vars in 1188 bytes} [Sat May 28 21:33:37 2022] GET /identity/v3/users?name=tempest-test_user-2027027914 => generated 475 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:38.058323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d0adc-74fb-413c-bd64-8bb5c06ca58a tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:38.069016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d0adc-74fb-413c-bd64-8bb5c06ca58a tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:38.074333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623354d8-225b-4ff2-a328-47cababd3f88 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abc3a19461344532a15b743cc207d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813d44d35f774f42ae7d5d78fce74798', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:38.075004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623354d8-225b-4ff2-a328-47cababd3f88 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:38.075291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623354d8-225b-4ff2-a328-47cababd3f88 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:38.075578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623354d8-225b-4ff2-a328-47cababd3f88 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:38.076393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623354d8-225b-4ff2-a328-47cababd3f88 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Adding query filter params (enabled=False) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:33:38.076728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623354d8-225b-4ff2-a328-47cababd3f88 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:38.078600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623354d8-225b-4ff2-a328-47cababd3f88 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:38.081162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:38.100929 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3833/7628] 10.222.0.33 () {58 vars in 1148 bytes} [Sat May 28 21:33:38 2022] GET /identity/v3/users?enabled=False => generated 431 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:38.105924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da17af9-3761-4e77-abcd-aae2ec392756 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:38.112459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:38.116252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da17af9-3761-4e77-abcd-aae2ec392756 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:38.121060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1f56b-c699-46fd-8990-375f139f66f8 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abc3a19461344532a15b743cc207d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813d44d35f774f42ae7d5d78fce74798', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:38.122143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1f56b-c699-46fd-8990-375f139f66f8 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:38.122451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1f56b-c699-46fd-8990-375f139f66f8 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:38.122694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1f56b-c699-46fd-8990-375f139f66f8 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] PATH_INFO: `/v3/users/36a660f030c84d4a8c6ca70ddeb8ad00` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:38.137724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1f56b-c699-46fd-8990-375f139f66f8 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorizing `identity:delete_user(user_id=36a660f030c84d4a8c6ca70ddeb8ad00)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:38.139710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1f56b-c699-46fd-8990-375f139f66f8 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:38.277744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-19c1f56b-c699-46fd-8990-375f139f66f8 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36a660f030c84d4a8c6ca70ddeb8ad00'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:38.278548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-19c1f56b-c699-46fd-8990-375f139f66f8 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36a660f030c84d4a8c6ca70ddeb8ad00'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:38.285805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-19c1f56b-c699-46fd-8990-375f139f66f8 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36a660f030c84d4a8c6ca70ddeb8ad00'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:38.380090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-19c1f56b-c699-46fd-8990-375f139f66f8 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36a660f030c84d4a8c6ca70ddeb8ad00'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:38.380827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-19c1f56b-c699-46fd-8990-375f139f66f8 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36a660f030c84d4a8c6ca70ddeb8ad00'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:38.399523 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3796/7629] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:38 2022] DELETE /identity/v3/users/36a660f030c84d4a8c6ca70ddeb8ad00 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:38.403444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623354d8-225b-4ff2-a328-47cababd3f88 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:38.405141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-623354d8-225b-4ff2-a328-47cababd3f88 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:38.418096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623354d8-225b-4ff2-a328-47cababd3f88 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:38.479722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7409a-5ca9-4137-b344-fcf458d9022f tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abc3a19461344532a15b743cc207d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813d44d35f774f42ae7d5d78fce74798', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:38.480610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7409a-5ca9-4137-b344-fcf458d9022f tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:38.480902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7409a-5ca9-4137-b344-fcf458d9022f tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:38.481186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7409a-5ca9-4137-b344-fcf458d9022f tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] PATH_INFO: `/v3/users/4b752fc7b47745a1a7c2aa144af039ef` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:38.482450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7409a-5ca9-4137-b344-fcf458d9022f tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorizing `identity:delete_user(user_id=4b752fc7b47745a1a7c2aa144af039ef)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:38.484371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7409a-5ca9-4137-b344-fcf458d9022f tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:38.592157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5af7409a-5ca9-4137-b344-fcf458d9022f tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b752fc7b47745a1a7c2aa144af039ef'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:38.599280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5af7409a-5ca9-4137-b344-fcf458d9022f tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b752fc7b47745a1a7c2aa144af039ef'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:38.599941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5af7409a-5ca9-4137-b344-fcf458d9022f tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b752fc7b47745a1a7c2aa144af039ef'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:38.600548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5af7409a-5ca9-4137-b344-fcf458d9022f tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b752fc7b47745a1a7c2aa144af039ef'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:38.659909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5af7409a-5ca9-4137-b344-fcf458d9022f tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b752fc7b47745a1a7c2aa144af039ef'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:38.678405 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3834/7630] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:38 2022] DELETE /identity/v3/users/4b752fc7b47745a1a7c2aa144af039ef => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:38.682945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1f56b-c699-46fd-8990-375f139f66f8 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:38.693325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1f56b-c699-46fd-8990-375f139f66f8 tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:38.756954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85885c39-e8a5-431f-b6f6-c65ca312a64d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abc3a19461344532a15b743cc207d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813d44d35f774f42ae7d5d78fce74798', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:38.757833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85885c39-e8a5-431f-b6f6-c65ca312a64d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:38.758153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85885c39-e8a5-431f-b6f6-c65ca312a64d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:38.758436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85885c39-e8a5-431f-b6f6-c65ca312a64d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] PATH_INFO: `/v3/domains/b3186a1dccef4e948c742adea2e3632c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:38.759761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85885c39-e8a5-431f-b6f6-c65ca312a64d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=b3186a1dccef4e948c742adea2e3632c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:38.761873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85885c39-e8a5-431f-b6f6-c65ca312a64d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:38.781537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-85885c39-e8a5-431f-b6f6-c65ca312a64d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': 'b3186a1dccef4e948c742adea2e3632c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:38.782201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-85885c39-e8a5-431f-b6f6-c65ca312a64d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': 'b3186a1dccef4e948c742adea2e3632c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:38.860916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-85885c39-e8a5-431f-b6f6-c65ca312a64d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': 'b3186a1dccef4e948c742adea2e3632c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:38.861533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-85885c39-e8a5-431f-b6f6-c65ca312a64d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': 'b3186a1dccef4e948c742adea2e3632c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:38.866830 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3797/7631] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:33:38 2022] PATCH /identity/v3/domains/b3186a1dccef4e948c742adea2e3632c => generated 277 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:38.867970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7409a-5ca9-4137-b344-fcf458d9022f tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:38.869411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5af7409a-5ca9-4137-b344-fcf458d9022f tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:38.882851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7409a-5ca9-4137-b344-fcf458d9022f tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:38.939910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212bbec-acd2-488d-8b88-cd562c976ecd tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'abc3a19461344532a15b743cc207d11c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '813d44d35f774f42ae7d5d78fce74798', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:38.940783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212bbec-acd2-488d-8b88-cd562c976ecd tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:38.941068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212bbec-acd2-488d-8b88-cd562c976ecd tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:38.941345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212bbec-acd2-488d-8b88-cd562c976ecd tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] PATH_INFO: `/v3/domains/b3186a1dccef4e948c742adea2e3632c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:38.952007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212bbec-acd2-488d-8b88-cd562c976ecd tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=b3186a1dccef4e948c742adea2e3632c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:38.953610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212bbec-acd2-488d-8b88-cd562c976ecd tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:38.961656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:38.975953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b212bbec-acd2-488d-8b88-cd562c976ecd tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': 'b3186a1dccef4e948c742adea2e3632c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:38.979880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b212bbec-acd2-488d-8b88-cd562c976ecd tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': 'b3186a1dccef4e948c742adea2e3632c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:39.158242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b212bbec-acd2-488d-8b88-cd562c976ecd tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3186a1dccef4e948c742adea2e3632c', 'request_id': 'req-b212bbec-acd2-488d-8b88-cd562c976ecd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:39.164566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b212bbec-acd2-488d-8b88-cd562c976ecd tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3186a1dccef4e948c742adea2e3632c', 'request_id': 'req-b212bbec-acd2-488d-8b88-cd562c976ecd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:39.237642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b212bbec-acd2-488d-8b88-cd562c976ecd tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3186a1dccef4e948c742adea2e3632c', 'request_id': 'req-b212bbec-acd2-488d-8b88-cd562c976ecd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:39.251259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b212bbec-acd2-488d-8b88-cd562c976ecd tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] The token cache is being invalidate because project b3186a1dccef4e948c742adea2e3632c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:39.251491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b212bbec-acd2-488d-8b88-cd562c976ecd tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:39.252218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b212bbec-acd2-488d-8b88-cd562c976ecd tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:39.259866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b212bbec-acd2-488d-8b88-cd562c976ecd tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': 'b3186a1dccef4e948c742adea2e3632c', 'request_id': 'req-b212bbec-acd2-488d-8b88-cd562c976ecd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:39.260547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b212bbec-acd2-488d-8b88-cd562c976ecd tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': 'b3186a1dccef4e948c742adea2e3632c', 'request_id': 'req-b212bbec-acd2-488d-8b88-cd562c976ecd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:39.264352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b212bbec-acd2-488d-8b88-cd562c976ecd tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': 'b3186a1dccef4e948c742adea2e3632c', 'request_id': 'req-b212bbec-acd2-488d-8b88-cd562c976ecd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:39.270081 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3835/7632] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:38 2022] DELETE /identity/v3/domains/b3186a1dccef4e948c742adea2e3632c => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:39.274133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85885c39-e8a5-431f-b6f6-c65ca312a64d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:39.275711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-85885c39-e8a5-431f-b6f6-c65ca312a64d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:39.312913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=1,keystone:UPDATE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:39.355013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85885c39-e8a5-431f-b6f6-c65ca312a64d tempest-UsersV3TestJSON-1243899168 tempest-UsersV3TestJSON-1243899168-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:39.426013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac529d5-73b2-4eb0-b9fd-01cc9ead6e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:39.426916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac529d5-73b2-4eb0-b9fd-01cc9ead6e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:39.427213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac529d5-73b2-4eb0-b9fd-01cc9ead6e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:39.427597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac529d5-73b2-4eb0-b9fd-01cc9ead6e04 admin admin] PATH_INFO: `/v3/users/b88542c6d3a54954a5e897093c7229b9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:39.428941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac529d5-73b2-4eb0-b9fd-01cc9ead6e04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b88542c6d3a54954a5e897093c7229b9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:39.431026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac529d5-73b2-4eb0-b9fd-01cc9ead6e04 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:39.721357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7ac529d5-73b2-4eb0-b9fd-01cc9ead6e04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b88542c6d3a54954a5e897093c7229b9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:39.722059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7ac529d5-73b2-4eb0-b9fd-01cc9ead6e04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b88542c6d3a54954a5e897093c7229b9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:39.729104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7ac529d5-73b2-4eb0-b9fd-01cc9ead6e04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b88542c6d3a54954a5e897093c7229b9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:39.825021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7ac529d5-73b2-4eb0-b9fd-01cc9ead6e04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b88542c6d3a54954a5e897093c7229b9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:39.828126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7ac529d5-73b2-4eb0-b9fd-01cc9ead6e04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b88542c6d3a54954a5e897093c7229b9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:39.844181 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3798/7633] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:39 2022] DELETE /identity/v3/users/b88542c6d3a54954a5e897093c7229b9 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:39.848643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24a282-0889-417a-8f69-2a34de46b706 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:39.849142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24a282-0889-417a-8f69-2a34de46b706 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:39.849236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24a282-0889-417a-8f69-2a34de46b706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:39.866070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1f24a282-0889-417a-8f69-2a34de46b706 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:39.867210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1f24a282-0889-417a-8f69-2a34de46b706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:39.927013 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3836/7634] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:39.937567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac529d5-73b2-4eb0-b9fd-01cc9ead6e04 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:39.938992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7ac529d5-73b2-4eb0-b9fd-01cc9ead6e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:40.000661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac529d5-73b2-4eb0-b9fd-01cc9ead6e04 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:40.004600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defd94f-6985-407f-80bd-cd0d57840271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:40.005257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defd94f-6985-407f-80bd-cd0d57840271 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:40.005548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defd94f-6985-407f-80bd-cd0d57840271 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:40.005833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1defd94f-6985-407f-80bd-cd0d57840271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:40.015659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defd94f-6985-407f-80bd-cd0d57840271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:40.017676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1defd94f-6985-407f-80bd-cd0d57840271 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:40.022064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:40.026740 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3799/7635] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:33:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:33:40.468226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24a282-0889-417a-8f69-2a34de46b706 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:40.469702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1f24a282-0889-417a-8f69-2a34de46b706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:40.484511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24a282-0889-417a-8f69-2a34de46b706 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:40.489315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165891e3-148e-4b3e-a810-81fc14be1ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:40.490260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165891e3-148e-4b3e-a810-81fc14be1ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:40.490543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165891e3-148e-4b3e-a810-81fc14be1ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:40.490848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165891e3-148e-4b3e-a810-81fc14be1ad0 admin admin] PATH_INFO: `/v3/projects/4fe55c90ce30473d92c1a8ad013719ab` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:40.492105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165891e3-148e-4b3e-a810-81fc14be1ad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fe55c90ce30473d92c1a8ad013719ab)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:40.494100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165891e3-148e-4b3e-a810-81fc14be1ad0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:40.608669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:40.609974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-165891e3-148e-4b3e-a810-81fc14be1ad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fe55c90ce30473d92c1a8ad013719ab', 'request_id': 'req-165891e3-148e-4b3e-a810-81fc14be1ad0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:40.615930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-165891e3-148e-4b3e-a810-81fc14be1ad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fe55c90ce30473d92c1a8ad013719ab', 'request_id': 'req-165891e3-148e-4b3e-a810-81fc14be1ad0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:40.723572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-165891e3-148e-4b3e-a810-81fc14be1ad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fe55c90ce30473d92c1a8ad013719ab', 'request_id': 'req-165891e3-148e-4b3e-a810-81fc14be1ad0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:40.735697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-165891e3-148e-4b3e-a810-81fc14be1ad0 admin admin] The token cache is being invalidate because project 4fe55c90ce30473d92c1a8ad013719ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:40.735996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-165891e3-148e-4b3e-a810-81fc14be1ad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:40.736642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-165891e3-148e-4b3e-a810-81fc14be1ad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:40.738295 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3837/7636] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:40 2022] DELETE /identity/v3/projects/4fe55c90ce30473d92c1a8ad013719ab => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:40.742410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defd94f-6985-407f-80bd-cd0d57840271 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:40.752379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1defd94f-6985-407f-80bd-cd0d57840271 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:40.809359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aaf471-489e-4e7a-8258-6e7c65bc52d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:40.810274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aaf471-489e-4e7a-8258-6e7c65bc52d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:40.810558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aaf471-489e-4e7a-8258-6e7c65bc52d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:40.810837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aaf471-489e-4e7a-8258-6e7c65bc52d4 admin admin] PATH_INFO: `/v3/users/abc3a19461344532a15b743cc207d11c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:40.812047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aaf471-489e-4e7a-8258-6e7c65bc52d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abc3a19461344532a15b743cc207d11c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:40.814084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aaf471-489e-4e7a-8258-6e7c65bc52d4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:41.001793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-56aaf471-489e-4e7a-8258-6e7c65bc52d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abc3a19461344532a15b743cc207d11c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:41.002441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-56aaf471-489e-4e7a-8258-6e7c65bc52d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abc3a19461344532a15b743cc207d11c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:41.009482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-56aaf471-489e-4e7a-8258-6e7c65bc52d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abc3a19461344532a15b743cc207d11c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:41.123375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-56aaf471-489e-4e7a-8258-6e7c65bc52d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abc3a19461344532a15b743cc207d11c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:41.124119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-56aaf471-489e-4e7a-8258-6e7c65bc52d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abc3a19461344532a15b743cc207d11c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:41.144471 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3800/7637] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:40 2022] DELETE /identity/v3/users/abc3a19461344532a15b743cc207d11c => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:41.525434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165891e3-148e-4b3e-a810-81fc14be1ad0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:41.527454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-165891e3-148e-4b3e-a810-81fc14be1ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:41.540317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165891e3-148e-4b3e-a810-81fc14be1ad0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:41.601128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c7462-64a0-465d-a5b5-cb3b43812379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:41.602094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c7462-64a0-465d-a5b5-cb3b43812379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:41.602379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c7462-64a0-465d-a5b5-cb3b43812379 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:41.602654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c7462-64a0-465d-a5b5-cb3b43812379 admin admin] PATH_INFO: `/v3/projects/813d44d35f774f42ae7d5d78fce74798` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:41.603831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c7462-64a0-465d-a5b5-cb3b43812379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=813d44d35f774f42ae7d5d78fce74798)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:41.605751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c7462-64a0-465d-a5b5-cb3b43812379 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:41.730570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6e4c7462-64a0-465d-a5b5-cb3b43812379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '813d44d35f774f42ae7d5d78fce74798', 'request_id': 'req-6e4c7462-64a0-465d-a5b5-cb3b43812379'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:41.736645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6e4c7462-64a0-465d-a5b5-cb3b43812379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '813d44d35f774f42ae7d5d78fce74798', 'request_id': 'req-6e4c7462-64a0-465d-a5b5-cb3b43812379'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:41.820105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6e4c7462-64a0-465d-a5b5-cb3b43812379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '813d44d35f774f42ae7d5d78fce74798', 'request_id': 'req-6e4c7462-64a0-465d-a5b5-cb3b43812379'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:41.831640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6e4c7462-64a0-465d-a5b5-cb3b43812379 admin admin] The token cache is being invalidate because project 813d44d35f774f42ae7d5d78fce74798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:41.831933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6e4c7462-64a0-465d-a5b5-cb3b43812379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:41.832561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6e4c7462-64a0-465d-a5b5-cb3b43812379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:41.834224 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3838/7638] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:41 2022] DELETE /identity/v3/projects/813d44d35f774f42ae7d5d78fce74798 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:41.839344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d819d73-c75d-4190-b153-d155958f1733 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:41.839899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d819d73-c75d-4190-b153-d155958f1733 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:41.840175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d819d73-c75d-4190-b153-d155958f1733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:41.859953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3d819d73-c75d-4190-b153-d155958f1733 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:41.861206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3d819d73-c75d-4190-b153-d155958f1733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:41.925445 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3801/7639] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:41.934002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4219e6-c4d4-45e5-9396-f581a2e4aea4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:41.934293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4219e6-c4d4-45e5-9396-f581a2e4aea4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:41.934571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4219e6-c4d4-45e5-9396-f581a2e4aea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:41.955048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8b4219e6-c4d4-45e5-9396-f581a2e4aea4 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:41.956276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8b4219e6-c4d4-45e5-9396-f581a2e4aea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:41.964344 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3839/7640] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:41.973437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d819d73-c75d-4190-b153-d155958f1733 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:41.983872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d819d73-c75d-4190-b153-d155958f1733 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:41.988448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daea1a3-084b-48f2-a891-5a53e4016b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:41.989129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daea1a3-084b-48f2-a891-5a53e4016b75 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:41.989409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daea1a3-084b-48f2-a891-5a53e4016b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:41.989687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daea1a3-084b-48f2-a891-5a53e4016b75 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:41.990444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daea1a3-084b-48f2-a891-5a53e4016b75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:33:41.990753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daea1a3-084b-48f2-a891-5a53e4016b75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:41.992353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daea1a3-084b-48f2-a891-5a53e4016b75 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:42.004809 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3802/7641] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:42.009205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4219e6-c4d4-45e5-9396-f581a2e4aea4 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:42.019442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4219e6-c4d4-45e5-9396-f581a2e4aea4 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:42.024238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cca8bc-b398-4f88-a164-3d48f4d35c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:42.024890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cca8bc-b398-4f88-a164-3d48f4d35c9d admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:42.025177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cca8bc-b398-4f88-a164-3d48f4d35c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:42.025457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cca8bc-b398-4f88-a164-3d48f4d35c9d admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:42.026187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cca8bc-b398-4f88-a164-3d48f4d35c9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:42.028130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cca8bc-b398-4f88-a164-3d48f4d35c9d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:42.135361 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3840/7642] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:42 2022] POST /identity/v3/projects => generated 365 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:42.140013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daea1a3-084b-48f2-a891-5a53e4016b75 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:42.150325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daea1a3-084b-48f2-a891-5a53e4016b75 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:42.154544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:42.206122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe233f5-137a-46a0-9374-b20d449f51f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:42.206818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe233f5-137a-46a0-9374-b20d449f51f3 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:42.207165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe233f5-137a-46a0-9374-b20d449f51f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:42.207450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe233f5-137a-46a0-9374-b20d449f51f3 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:42.208152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe233f5-137a-46a0-9374-b20d449f51f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:42.210137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe233f5-137a-46a0-9374-b20d449f51f3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:42.375446 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3803/7643] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:42 2022] POST /identity/v3/users => generated 283 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:42.379482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cca8bc-b398-4f88-a164-3d48f4d35c9d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:42.389991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cca8bc-b398-4f88-a164-3d48f4d35c9d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:42.394669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e26e1-1ec6-4c4c-9ecf-e3f759abc595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:42.395336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e26e1-1ec6-4c4c-9ecf-e3f759abc595 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:42.395620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e26e1-1ec6-4c4c-9ecf-e3f759abc595 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:42.395900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501e26e1-1ec6-4c4c-9ecf-e3f759abc595 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:42.396675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e26e1-1ec6-4c4c-9ecf-e3f759abc595 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:42.398396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501e26e1-1ec6-4c4c-9ecf-e3f759abc595 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:42.409658 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3841/7644] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:42.413791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe233f5-137a-46a0-9374-b20d449f51f3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:42.424738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe233f5-137a-46a0-9374-b20d449f51f3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:42.429403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b870a-1f49-4e6f-8946-f9d0a28213c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:42.430107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b870a-1f49-4e6f-8946-f9d0a28213c4 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:42.430391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b870a-1f49-4e6f-8946-f9d0a28213c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:42.430671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b870a-1f49-4e6f-8946-f9d0a28213c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:42.431435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b870a-1f49-4e6f-8946-f9d0a28213c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:42.433068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b870a-1f49-4e6f-8946-f9d0a28213c4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:42.442978 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3804/7645] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:33:42.446944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e26e1-1ec6-4c4c-9ecf-e3f759abc595 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:42.457523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501e26e1-1ec6-4c4c-9ecf-e3f759abc595 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:42.462097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa41ad-b584-4eca-bc2e-fdb2dacff56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:42.462966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa41ad-b584-4eca-bc2e-fdb2dacff56c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:42.463250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa41ad-b584-4eca-bc2e-fdb2dacff56c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:42.463530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa41ad-b584-4eca-bc2e-fdb2dacff56c admin admin] PATH_INFO: `/v3/projects/78088426c39843329565a38ef1567431/users/c15d86b3c2cd4991944bb8aaeff7cd63/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:42.483008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa41ad-b584-4eca-bc2e-fdb2dacff56c admin admin] RBAC: Authorizing `identity:create_grant(project_id=78088426c39843329565a38ef1567431, user_id=c15d86b3c2cd4991944bb8aaeff7cd63, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:42.485765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa41ad-b584-4eca-bc2e-fdb2dacff56c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:42.581639 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3842/7646] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:42 2022] PUT /identity/v3/projects/78088426c39843329565a38ef1567431/users/c15d86b3c2cd4991944bb8aaeff7cd63/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:42.593438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281fdc9-75a0-44e8-ab52-90b3f50087e6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:42.593732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281fdc9-75a0-44e8-ab52-90b3f50087e6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:42.594073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281fdc9-75a0-44e8-ab52-90b3f50087e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:42.623115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6281fdc9-75a0-44e8-ab52-90b3f50087e6 None None] MFA Rules not processed for user `c15d86b3c2cd4991944bb8aaeff7cd63`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:42.624875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6281fdc9-75a0-44e8-ab52-90b3f50087e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:42.689395 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3805/7647] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:42.693873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa41ad-b584-4eca-bc2e-fdb2dacff56c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:42.698515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:42.760044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa41ad-b584-4eca-bc2e-fdb2dacff56c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:42.763691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a1626-36ac-4dce-b107-a64f418f0800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:42.764386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a1626-36ac-4dce-b107-a64f418f0800 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:42.764966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a1626-36ac-4dce-b107-a64f418f0800 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:42.765253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a1626-36ac-4dce-b107-a64f418f0800 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:42.766006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a1626-36ac-4dce-b107-a64f418f0800 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:42.767991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a1626-36ac-4dce-b107-a64f418f0800 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:42.879139 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3843/7648] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:42 2022] POST /identity/v3/projects => generated 367 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:42.883393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281fdc9-75a0-44e8-ab52-90b3f50087e6 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:42.895314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281fdc9-75a0-44e8-ab52-90b3f50087e6 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:42.957899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028f2cc-b103-4c2c-b914-0c8e20960de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:42.959522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028f2cc-b103-4c2c-b914-0c8e20960de7 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:42.959522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028f2cc-b103-4c2c-b914-0c8e20960de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:42.959522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e028f2cc-b103-4c2c-b914-0c8e20960de7 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:42.960119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028f2cc-b103-4c2c-b914-0c8e20960de7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:42.961949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e028f2cc-b103-4c2c-b914-0c8e20960de7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:43.060281 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3806/7649] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:42 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:43.067554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a1626-36ac-4dce-b107-a64f418f0800 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:43.078966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a1626-36ac-4dce-b107-a64f418f0800 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:43.083644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23eaad4-88f6-42ba-9fcb-a5084774dae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:43.084304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eaad4-88f6-42ba-9fcb-a5084774dae8 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:43.084588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eaad4-88f6-42ba-9fcb-a5084774dae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:43.084866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23eaad4-88f6-42ba-9fcb-a5084774dae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:43.085596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23eaad4-88f6-42ba-9fcb-a5084774dae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:43.087315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23eaad4-88f6-42ba-9fcb-a5084774dae8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:43.098560 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3844/7650] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:43.102651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028f2cc-b103-4c2c-b914-0c8e20960de7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:43.109638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:43.113467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e028f2cc-b103-4c2c-b914-0c8e20960de7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:43.118629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d719846-ca94-45cd-99fe-a856842d7616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:43.119533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d719846-ca94-45cd-99fe-a856842d7616 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:43.125647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d719846-ca94-45cd-99fe-a856842d7616 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:43.125647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d719846-ca94-45cd-99fe-a856842d7616 admin admin] PATH_INFO: `/v3/projects/ebc5e3a401fb457c926c5351325cc6fb/users/652362beeec344afb1eb6737eb3e2bdd/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:43.134549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d719846-ca94-45cd-99fe-a856842d7616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc5e3a401fb457c926c5351325cc6fb, user_id=652362beeec344afb1eb6737eb3e2bdd, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:43.137330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d719846-ca94-45cd-99fe-a856842d7616 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:43.213901 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3807/7651] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:43 2022] PUT /identity/v3/projects/ebc5e3a401fb457c926c5351325cc6fb/users/652362beeec344afb1eb6737eb3e2bdd/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:43.218720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23eaad4-88f6-42ba-9fcb-a5084774dae8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:43.228704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23eaad4-88f6-42ba-9fcb-a5084774dae8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:43.287961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd92303-09c4-48f6-8724-82536aefe9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:43.288704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd92303-09c4-48f6-8724-82536aefe9fa admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:43.289018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd92303-09c4-48f6-8724-82536aefe9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:43.289325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd92303-09c4-48f6-8724-82536aefe9fa admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:43.290121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd92303-09c4-48f6-8724-82536aefe9fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:43.291828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd92303-09c4-48f6-8724-82536aefe9fa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:43.301429 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3845/7652] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:43.305691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d719846-ca94-45cd-99fe-a856842d7616 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:43.316157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d719846-ca94-45cd-99fe-a856842d7616 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:43.320807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55407762-4df9-4d9e-9174-bed9fcc5cb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:43.321691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55407762-4df9-4d9e-9174-bed9fcc5cb94 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:43.322033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55407762-4df9-4d9e-9174-bed9fcc5cb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:43.322310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55407762-4df9-4d9e-9174-bed9fcc5cb94 admin admin] PATH_INFO: `/v3/projects/ebc5e3a401fb457c926c5351325cc6fb/users/652362beeec344afb1eb6737eb3e2bdd/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:43.324938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55407762-4df9-4d9e-9174-bed9fcc5cb94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc5e3a401fb457c926c5351325cc6fb, user_id=652362beeec344afb1eb6737eb3e2bdd, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:43.327541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55407762-4df9-4d9e-9174-bed9fcc5cb94 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:43.338219 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3808/7653] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:43 2022] PUT /identity/v3/projects/ebc5e3a401fb457c926c5351325cc6fb/users/652362beeec344afb1eb6737eb3e2bdd/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:43.349174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307be688-065f-4346-94e3-fd67f6db682e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:43.349456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307be688-065f-4346-94e3-fd67f6db682e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:43.350432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307be688-065f-4346-94e3-fd67f6db682e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:43.376248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-307be688-065f-4346-94e3-fd67f6db682e None None] MFA Rules not processed for user `652362beeec344afb1eb6737eb3e2bdd`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:43.377455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-307be688-065f-4346-94e3-fd67f6db682e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:43.442078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:43.447342 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3846/7654] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:43 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:43.452956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55407762-4df9-4d9e-9174-bed9fcc5cb94 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:43.463715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55407762-4df9-4d9e-9174-bed9fcc5cb94 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:43.468994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb901704-3913-496f-b616-7289c78b6c96 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:43.469583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb901704-3913-496f-b616-7289c78b6c96 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:43.469906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb901704-3913-496f-b616-7289c78b6c96 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:43.470194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb901704-3913-496f-b616-7289c78b6c96 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:43.470879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb901704-3913-496f-b616-7289c78b6c96 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:43.472730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb901704-3913-496f-b616-7289c78b6c96 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:43.609950 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3809/7655] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:43 2022] POST /identity/v3/projects => generated 359 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:43.613923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307be688-065f-4346-94e3-fd67f6db682e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:43.623814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307be688-065f-4346-94e3-fd67f6db682e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:43.686311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3b6d0-8f17-4447-b5f9-e506061ab960 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:43.686995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3b6d0-8f17-4447-b5f9-e506061ab960 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:43.687285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3b6d0-8f17-4447-b5f9-e506061ab960 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:43.688001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3b6d0-8f17-4447-b5f9-e506061ab960 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:43.688363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3b6d0-8f17-4447-b5f9-e506061ab960 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:43.690282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b3b6d0-8f17-4447-b5f9-e506061ab960 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:43.829368 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3847/7656] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:43 2022] POST /identity/v3/users => generated 373 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:43.833675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb901704-3913-496f-b616-7289c78b6c96 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:43.844708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb901704-3913-496f-b616-7289c78b6c96 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:43.849637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b639d-0b0f-4ce0-a7ba-7bd3bea45ae0 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:43.850545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b639d-0b0f-4ce0-a7ba-7bd3bea45ae0 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:43.850865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b639d-0b0f-4ce0-a7ba-7bd3bea45ae0 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:43.851175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807b639d-0b0f-4ce0-a7ba-7bd3bea45ae0 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users/7897d08a1c6d425399a886b41be53929` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:43.865180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b639d-0b0f-4ce0-a7ba-7bd3bea45ae0 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:get_user(user_id=7897d08a1c6d425399a886b41be53929)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:43.867458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807b639d-0b0f-4ce0-a7ba-7bd3bea45ae0 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:43.869082 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3810/7657] 10.222.0.33 () {58 vars in 1220 bytes} [Sat May 28 21:33:43 2022] GET /identity/v3/users/7897d08a1c6d425399a886b41be53929 => generated 373 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:43.878292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3b6d0-8f17-4447-b5f9-e506061ab960 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:43.889400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3b6d0-8f17-4447-b5f9-e506061ab960 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:43.894252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc46b02-3679-47a5-a73a-680174def5de tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:43.895143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc46b02-3679-47a5-a73a-680174def5de tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:43.898143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc46b02-3679-47a5-a73a-680174def5de tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:43.898143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc46b02-3679-47a5-a73a-680174def5de tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users/7897d08a1c6d425399a886b41be53929` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:43.898143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc46b02-3679-47a5-a73a-680174def5de tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:delete_user(user_id=7897d08a1c6d425399a886b41be53929)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:43.899093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc46b02-3679-47a5-a73a-680174def5de tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:44.020438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2fc46b02-3679-47a5-a73a-680174def5de tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7897d08a1c6d425399a886b41be53929'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:44.027763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2fc46b02-3679-47a5-a73a-680174def5de tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7897d08a1c6d425399a886b41be53929'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:44.028537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2fc46b02-3679-47a5-a73a-680174def5de tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7897d08a1c6d425399a886b41be53929'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:44.029218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2fc46b02-3679-47a5-a73a-680174def5de tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7897d08a1c6d425399a886b41be53929'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:44.163085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2fc46b02-3679-47a5-a73a-680174def5de tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7897d08a1c6d425399a886b41be53929'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:44.182080 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3848/7658] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:43 2022] DELETE /identity/v3/users/7897d08a1c6d425399a886b41be53929 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:44.192754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b639d-0b0f-4ce0-a7ba-7bd3bea45ae0 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:44.194873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-807b639d-0b0f-4ce0-a7ba-7bd3bea45ae0 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:44.208073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807b639d-0b0f-4ce0-a7ba-7bd3bea45ae0 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:44.264744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:44.265607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:44.265926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:44.266233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/projects/6421a964486549688723bd8837ff8203` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:44.267415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6421a964486549688723bd8837ff8203)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:44.269392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:44.397594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6421a964486549688723bd8837ff8203', 'request_id': 'req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:44.403709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6421a964486549688723bd8837ff8203', 'request_id': 'req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:44.470315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6421a964486549688723bd8837ff8203', 'request_id': 'req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:44.482536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] The token cache is being invalidate because project 6421a964486549688723bd8837ff8203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:44.482829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:44.483585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:44.485253 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3811/7659] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:44 2022] DELETE /identity/v3/projects/6421a964486549688723bd8837ff8203 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:44.490315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc46b02-3679-47a5-a73a-680174def5de tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:44.500936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc46b02-3679-47a5-a73a-680174def5de tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:44.561630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f31c3b-601b-44c8-96f0-7b2804348f14 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:44.562334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f31c3b-601b-44c8-96f0-7b2804348f14 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:44.562628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f31c3b-601b-44c8-96f0-7b2804348f14 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:44.562916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f31c3b-601b-44c8-96f0-7b2804348f14 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:44.563618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f31c3b-601b-44c8-96f0-7b2804348f14 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:44.565472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f31c3b-601b-44c8-96f0-7b2804348f14 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:44.569456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:44.704475 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3849/7660] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:44 2022] POST /identity/v3/projects => generated 358 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:33:44.708623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:44.710106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:44.723799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627cdf9-1be6-4290-af3d-8a2a4ff428ef tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:44.740442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:44.780220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a923b-845f-4089-8014-0aab247a093c tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:44.780899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a923b-845f-4089-8014-0aab247a093c tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:44.781189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a923b-845f-4089-8014-0aab247a093c tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:44.781470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a923b-845f-4089-8014-0aab247a093c tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:44.782221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a923b-845f-4089-8014-0aab247a093c tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:44.784145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8a923b-845f-4089-8014-0aab247a093c tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:44.915582 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3812/7661] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:44 2022] POST /identity/v3/users => generated 417 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:44.919950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f31c3b-601b-44c8-96f0-7b2804348f14 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:44.930933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f31c3b-601b-44c8-96f0-7b2804348f14 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:44.935590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beadf454-3dd9-44e6-9e2e-faa26170aa35 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:44.936249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadf454-3dd9-44e6-9e2e-faa26170aa35 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:44.936536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadf454-3dd9-44e6-9e2e-faa26170aa35 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:44.936814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beadf454-3dd9-44e6-9e2e-faa26170aa35 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:44.937559 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beadf454-3dd9-44e6-9e2e-faa26170aa35 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:44.939440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beadf454-3dd9-44e6-9e2e-faa26170aa35 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:45.021247 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3850/7662] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:33:44 2022] POST /identity/v3/roles => generated 240 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:33:45.025126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a923b-845f-4089-8014-0aab247a093c tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:45.036684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a923b-845f-4089-8014-0aab247a093c tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:45.042337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764854a9-95c2-4e73-86dc-83b9d7d38adf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:45.043223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764854a9-95c2-4e73-86dc-83b9d7d38adf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:45.043509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764854a9-95c2-4e73-86dc-83b9d7d38adf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:45.043790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764854a9-95c2-4e73-86dc-83b9d7d38adf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users/a6105c2f13b8440587c03071bea843c1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:45.060820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764854a9-95c2-4e73-86dc-83b9d7d38adf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:get_user(user_id=a6105c2f13b8440587c03071bea843c1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:45.063335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764854a9-95c2-4e73-86dc-83b9d7d38adf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:45.068651 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3813/7663] 10.222.0.33 () {58 vars in 1220 bytes} [Sat May 28 21:33:45 2022] GET /identity/v3/users/a6105c2f13b8440587c03071bea843c1 => generated 417 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:45.069147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beadf454-3dd9-44e6-9e2e-faa26170aa35 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:45.080086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beadf454-3dd9-44e6-9e2e-faa26170aa35 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:45.084672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84748b7e-93b5-4451-b6c4-0e1c82cf1603 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:45.085532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84748b7e-93b5-4451-b6c4-0e1c82cf1603 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:45.085821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84748b7e-93b5-4451-b6c4-0e1c82cf1603 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:45.086135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84748b7e-93b5-4451-b6c4-0e1c82cf1603 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/roles/2621e597bbfe49d5ae90336bc40394c9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:45.087624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84748b7e-93b5-4451-b6c4-0e1c82cf1603 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:get_role(role_id=2621e597bbfe49d5ae90336bc40394c9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:45.089230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84748b7e-93b5-4451-b6c4-0e1c82cf1603 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:45.090884 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3851/7664] 10.222.0.33 () {58 vars in 1220 bytes} [Sat May 28 21:33:45 2022] GET /identity/v3/roles/2621e597bbfe49d5ae90336bc40394c9 => generated 240 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:33:45.094697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764854a9-95c2-4e73-86dc-83b9d7d38adf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:45.104654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764854a9-95c2-4e73-86dc-83b9d7d38adf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:45.109311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c4de1-d564-4b5b-9188-6b7e1b4930c3 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:45.110000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c4de1-d564-4b5b-9188-6b7e1b4930c3 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:45.110344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c4de1-d564-4b5b-9188-6b7e1b4930c3 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:45.110625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c4de1-d564-4b5b-9188-6b7e1b4930c3 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:45.111341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c4de1-d564-4b5b-9188-6b7e1b4930c3 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:45.113274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c4de1-d564-4b5b-9188-6b7e1b4930c3 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:45.293991 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3814/7665] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:45 2022] POST /identity/v3/projects => generated 359 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:45.298061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84748b7e-93b5-4451-b6c4-0e1c82cf1603 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:45.308940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84748b7e-93b5-4451-b6c4-0e1c82cf1603 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:45.378144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f402979-6db7-45be-a794-0b94bbe9e7dd tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:45.379470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f402979-6db7-45be-a794-0b94bbe9e7dd tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:45.379470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f402979-6db7-45be-a794-0b94bbe9e7dd tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:45.379752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f402979-6db7-45be-a794-0b94bbe9e7dd tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/projects/033b220538754846bdb48257f46dce14` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:45.381047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f402979-6db7-45be-a794-0b94bbe9e7dd tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:get_project(project_id=033b220538754846bdb48257f46dce14)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:45.383181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f402979-6db7-45be-a794-0b94bbe9e7dd tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:45.385303 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3852/7666] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:45 2022] GET /identity/v3/projects/033b220538754846bdb48257f46dce14 => generated 359 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:45.395267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c4de1-d564-4b5b-9188-6b7e1b4930c3 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:45.406306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c4de1-d564-4b5b-9188-6b7e1b4930c3 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:45.411263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48576e8f-2079-408d-91e2-97f49c8c71b6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:45.412154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48576e8f-2079-408d-91e2-97f49c8c71b6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:45.412446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48576e8f-2079-408d-91e2-97f49c8c71b6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:45.412728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48576e8f-2079-408d-91e2-97f49c8c71b6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/projects/033b220538754846bdb48257f46dce14/users/a6105c2f13b8440587c03071bea843c1/roles/2621e597bbfe49d5ae90336bc40394c9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:45.415073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48576e8f-2079-408d-91e2-97f49c8c71b6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:create_grant(project_id=033b220538754846bdb48257f46dce14, user_id=a6105c2f13b8440587c03071bea843c1, role_id=2621e597bbfe49d5ae90336bc40394c9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:45.418570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48576e8f-2079-408d-91e2-97f49c8c71b6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:45.529788 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3815/7667] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:45 2022] PUT /identity/v3/projects/033b220538754846bdb48257f46dce14/users/a6105c2f13b8440587c03071bea843c1/roles/2621e597bbfe49d5ae90336bc40394c9 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:45.534043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f402979-6db7-45be-a794-0b94bbe9e7dd tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:45.544749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f402979-6db7-45be-a794-0b94bbe9e7dd tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:45.619916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411123c-ea64-4d47-9713-13e5e9d4098d tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:45.621494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411123c-ea64-4d47-9713-13e5e9d4098d tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:45.621789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411123c-ea64-4d47-9713-13e5e9d4098d tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:45.622089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411123c-ea64-4d47-9713-13e5e9d4098d tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:45.623228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411123c-ea64-4d47-9713-13e5e9d4098d tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:45.624914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411123c-ea64-4d47-9713-13e5e9d4098d tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:45.765052 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3853/7668] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:45 2022] POST /identity/v3/projects => generated 360 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:45.769265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48576e8f-2079-408d-91e2-97f49c8c71b6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:45.779854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48576e8f-2079-408d-91e2-97f49c8c71b6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:45.837527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8c807-80ba-4904-8192-d233b28a931c tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:45.838369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8c807-80ba-4904-8192-d233b28a931c tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:45.838663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8c807-80ba-4904-8192-d233b28a931c tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:45.839683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8c807-80ba-4904-8192-d233b28a931c tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/projects/90d9fbaba0cc4e83aa51655b32b44514` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:45.840902 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8c807-80ba-4904-8192-d233b28a931c tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:get_project(project_id=90d9fbaba0cc4e83aa51655b32b44514)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:45.843006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8c807-80ba-4904-8192-d233b28a931c tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:45.847514 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3816/7669] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:33:45 2022] GET /identity/v3/projects/90d9fbaba0cc4e83aa51655b32b44514 => generated 360 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:45.848964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411123c-ea64-4d47-9713-13e5e9d4098d tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:45.860376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411123c-ea64-4d47-9713-13e5e9d4098d tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:45.866020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd234f-4c74-4e26-a68f-6fcfa2031103 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:45.866883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd234f-4c74-4e26-a68f-6fcfa2031103 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:45.867238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd234f-4c74-4e26-a68f-6fcfa2031103 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:45.867528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd234f-4c74-4e26-a68f-6fcfa2031103 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/projects/90d9fbaba0cc4e83aa51655b32b44514/users/a6105c2f13b8440587c03071bea843c1/roles/2621e597bbfe49d5ae90336bc40394c9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:45.869868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd234f-4c74-4e26-a68f-6fcfa2031103 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:create_grant(project_id=90d9fbaba0cc4e83aa51655b32b44514, user_id=a6105c2f13b8440587c03071bea843c1, role_id=2621e597bbfe49d5ae90336bc40394c9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:45.872726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd234f-4c74-4e26-a68f-6fcfa2031103 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:45.987898 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3854/7670] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:45 2022] PUT /identity/v3/projects/90d9fbaba0cc4e83aa51655b32b44514/users/a6105c2f13b8440587c03071bea843c1/roles/2621e597bbfe49d5ae90336bc40394c9 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:45.988806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8c807-80ba-4904-8192-d233b28a931c tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:45.998695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8c807-80ba-4904-8192-d233b28a931c tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:46.059699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774adf64-29fb-4c9f-8c37-72d6501e7b08 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:46.060585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774adf64-29fb-4c9f-8c37-72d6501e7b08 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:46.060959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774adf64-29fb-4c9f-8c37-72d6501e7b08 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:46.061158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774adf64-29fb-4c9f-8c37-72d6501e7b08 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users/a6105c2f13b8440587c03071bea843c1/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:46.062477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774adf64-29fb-4c9f-8c37-72d6501e7b08 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:list_user_projects(user_id=a6105c2f13b8440587c03071bea843c1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:46.064618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774adf64-29fb-4c9f-8c37-72d6501e7b08 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:46.068035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:46.107016 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3817/7671] 10.222.0.33 () {58 vars in 1247 bytes} [Sat May 28 21:33:45 2022] GET /identity/v3/users/a6105c2f13b8440587c03071bea843c1/projects => generated 843 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:46.111417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd234f-4c74-4e26-a68f-6fcfa2031103 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:46.122219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd234f-4c74-4e26-a68f-6fcfa2031103 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:46.127116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d39a40-8daf-4346-8a4d-23561945c106 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:46.127982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d39a40-8daf-4346-8a4d-23561945c106 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:46.128271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d39a40-8daf-4346-8a4d-23561945c106 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:46.128555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d39a40-8daf-4346-8a4d-23561945c106 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/projects/90d9fbaba0cc4e83aa51655b32b44514` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:46.130560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d39a40-8daf-4346-8a4d-23561945c106 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:delete_project(project_id=90d9fbaba0cc4e83aa51655b32b44514)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:46.132001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d39a40-8daf-4346-8a4d-23561945c106 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:46.403425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:46.412803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-52d39a40-8daf-4346-8a4d-23561945c106 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90d9fbaba0cc4e83aa51655b32b44514', 'request_id': 'req-52d39a40-8daf-4346-8a4d-23561945c106'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:46.418768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-52d39a40-8daf-4346-8a4d-23561945c106 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90d9fbaba0cc4e83aa51655b32b44514', 'request_id': 'req-52d39a40-8daf-4346-8a4d-23561945c106'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:46.517212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-52d39a40-8daf-4346-8a4d-23561945c106 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90d9fbaba0cc4e83aa51655b32b44514', 'request_id': 'req-52d39a40-8daf-4346-8a4d-23561945c106'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:46.530204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-52d39a40-8daf-4346-8a4d-23561945c106 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] The token cache is being invalidate because project 90d9fbaba0cc4e83aa51655b32b44514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:46.530493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-52d39a40-8daf-4346-8a4d-23561945c106 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:46.531216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-52d39a40-8daf-4346-8a4d-23561945c106 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:46.533164 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3855/7672] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:46 2022] DELETE /identity/v3/projects/90d9fbaba0cc4e83aa51655b32b44514 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:46.537226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774adf64-29fb-4c9f-8c37-72d6501e7b08 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:46.539191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-774adf64-29fb-4c9f-8c37-72d6501e7b08 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:46.551854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774adf64-29fb-4c9f-8c37-72d6501e7b08 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:46.617737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89bd177-bb25-4828-97e7-f5fec5fd560a tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:46.618545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bd177-bb25-4828-97e7-f5fec5fd560a tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:46.619014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bd177-bb25-4828-97e7-f5fec5fd560a tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:46.619261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89bd177-bb25-4828-97e7-f5fec5fd560a tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/projects/033b220538754846bdb48257f46dce14` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:46.620943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89bd177-bb25-4828-97e7-f5fec5fd560a tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:delete_project(project_id=033b220538754846bdb48257f46dce14)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:46.622889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89bd177-bb25-4828-97e7-f5fec5fd560a tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:46.806094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d89bd177-bb25-4828-97e7-f5fec5fd560a tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '033b220538754846bdb48257f46dce14', 'request_id': 'req-d89bd177-bb25-4828-97e7-f5fec5fd560a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:46.812299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d89bd177-bb25-4828-97e7-f5fec5fd560a tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '033b220538754846bdb48257f46dce14', 'request_id': 'req-d89bd177-bb25-4828-97e7-f5fec5fd560a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:46.900807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d89bd177-bb25-4828-97e7-f5fec5fd560a tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '033b220538754846bdb48257f46dce14', 'request_id': 'req-d89bd177-bb25-4828-97e7-f5fec5fd560a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:46.912579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d89bd177-bb25-4828-97e7-f5fec5fd560a tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] The token cache is being invalidate because project 033b220538754846bdb48257f46dce14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:46.912870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d89bd177-bb25-4828-97e7-f5fec5fd560a tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:46.913595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d89bd177-bb25-4828-97e7-f5fec5fd560a tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:46.915235 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3818/7673] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:46 2022] DELETE /identity/v3/projects/033b220538754846bdb48257f46dce14 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:46.919968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d39a40-8daf-4346-8a4d-23561945c106 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:46.935578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d39a40-8daf-4346-8a4d-23561945c106 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:46.991364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08216985-2f39-4569-8f66-74fc287b3808 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:46.992216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08216985-2f39-4569-8f66-74fc287b3808 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:46.992501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08216985-2f39-4569-8f66-74fc287b3808 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:46.992779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08216985-2f39-4569-8f66-74fc287b3808 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/roles/2621e597bbfe49d5ae90336bc40394c9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:46.996575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08216985-2f39-4569-8f66-74fc287b3808 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:delete_role(role_id=2621e597bbfe49d5ae90336bc40394c9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:46.996575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08216985-2f39-4569-8f66-74fc287b3808 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:47.117457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-08216985-2f39-4569-8f66-74fc287b3808 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invalidating the token cache because role 2621e597bbfe49d5ae90336bc40394c9 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:47.117764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-08216985-2f39-4569-8f66-74fc287b3808 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:47.118522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-08216985-2f39-4569-8f66-74fc287b3808 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:47.120547 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3856/7674] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:46 2022] DELETE /identity/v3/roles/2621e597bbfe49d5ae90336bc40394c9 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:47.124741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89bd177-bb25-4828-97e7-f5fec5fd560a tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:47.126336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d89bd177-bb25-4828-97e7-f5fec5fd560a tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:47.140706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89bd177-bb25-4828-97e7-f5fec5fd560a tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:47.197115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74f8fa-251d-4675-a463-beb3811d967e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:47.198087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74f8fa-251d-4675-a463-beb3811d967e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:47.198889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74f8fa-251d-4675-a463-beb3811d967e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:47.199170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74f8fa-251d-4675-a463-beb3811d967e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users/a6105c2f13b8440587c03071bea843c1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:47.200375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74f8fa-251d-4675-a463-beb3811d967e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:delete_user(user_id=a6105c2f13b8440587c03071bea843c1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:47.202373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd74f8fa-251d-4675-a463-beb3811d967e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:47.204724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:47.385104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cd74f8fa-251d-4675-a463-beb3811d967e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6105c2f13b8440587c03071bea843c1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:47.385772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cd74f8fa-251d-4675-a463-beb3811d967e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6105c2f13b8440587c03071bea843c1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:47.393217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cd74f8fa-251d-4675-a463-beb3811d967e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6105c2f13b8440587c03071bea843c1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:47.488100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cd74f8fa-251d-4675-a463-beb3811d967e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6105c2f13b8440587c03071bea843c1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:47.491491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cd74f8fa-251d-4675-a463-beb3811d967e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6105c2f13b8440587c03071bea843c1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:47.508772 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3819/7675] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:47 2022] DELETE /identity/v3/users/a6105c2f13b8440587c03071bea843c1 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:47.514458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08216985-2f39-4569-8f66-74fc287b3808 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:47.515148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-08216985-2f39-4569-8f66-74fc287b3808 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:47.527945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08216985-2f39-4569-8f66-74fc287b3808 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:47.584502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40507e-ec1f-475d-8a0a-dc3ecac7f120 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:47.585344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40507e-ec1f-475d-8a0a-dc3ecac7f120 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:47.585634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40507e-ec1f-475d-8a0a-dc3ecac7f120 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:47.585950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40507e-ec1f-475d-8a0a-dc3ecac7f120 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/projects/8dfcafd4555b400eb4f967ececa2d1e0` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:47.587144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40507e-ec1f-475d-8a0a-dc3ecac7f120 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8dfcafd4555b400eb4f967ececa2d1e0)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:47.589052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40507e-ec1f-475d-8a0a-dc3ecac7f120 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:47.740540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:47.752787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0d40507e-ec1f-475d-8a0a-dc3ecac7f120 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dfcafd4555b400eb4f967ececa2d1e0', 'request_id': 'req-0d40507e-ec1f-475d-8a0a-dc3ecac7f120'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:47.758205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0d40507e-ec1f-475d-8a0a-dc3ecac7f120 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dfcafd4555b400eb4f967ececa2d1e0', 'request_id': 'req-0d40507e-ec1f-475d-8a0a-dc3ecac7f120'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:47.837738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0d40507e-ec1f-475d-8a0a-dc3ecac7f120 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dfcafd4555b400eb4f967ececa2d1e0', 'request_id': 'req-0d40507e-ec1f-475d-8a0a-dc3ecac7f120'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:47.850178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0d40507e-ec1f-475d-8a0a-dc3ecac7f120 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] The token cache is being invalidate because project 8dfcafd4555b400eb4f967ececa2d1e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:47.850468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0d40507e-ec1f-475d-8a0a-dc3ecac7f120 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:47.851095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0d40507e-ec1f-475d-8a0a-dc3ecac7f120 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:47.852680 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3857/7676] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:47 2022] DELETE /identity/v3/projects/8dfcafd4555b400eb4f967ececa2d1e0 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:47.858571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74f8fa-251d-4675-a463-beb3811d967e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:47.860215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cd74f8fa-251d-4675-a463-beb3811d967e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:47.873738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74f8fa-251d-4675-a463-beb3811d967e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:47.931388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17390eef-5de5-4c7b-bffa-d4b09c891241 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:47.932281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17390eef-5de5-4c7b-bffa-d4b09c891241 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:47.932567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17390eef-5de5-4c7b-bffa-d4b09c891241 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:47.933522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17390eef-5de5-4c7b-bffa-d4b09c891241 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users/c15d86b3c2cd4991944bb8aaeff7cd63` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:47.934852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17390eef-5de5-4c7b-bffa-d4b09c891241 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:update_user(user_id=c15d86b3c2cd4991944bb8aaeff7cd63)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:47.935894 ncdv-master-1 devstack@keystone.service[103104]: WARNING py.warnings [None req-17390eef-5de5-4c7b-bffa-d4b09c891241 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:33:47.935894 ncdv-master-1 devstack@keystone.service[103104]: warnings.warn(msg) May 28 21:33:47.935894 ncdv-master-1 devstack@keystone.service[103104]: May 28 21:33:47.937111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17390eef-5de5-4c7b-bffa-d4b09c891241 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:48.063864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-17390eef-5de5-4c7b-bffa-d4b09c891241 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'c15d86b3c2cd4991944bb8aaeff7cd63'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:48.169483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-17390eef-5de5-4c7b-bffa-d4b09c891241 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invalidating the token cache because user c15d86b3c2cd4991944bb8aaeff7cd63 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:48.169808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-17390eef-5de5-4c7b-bffa-d4b09c891241 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:48.170567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-17390eef-5de5-4c7b-bffa-d4b09c891241 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:48.172503 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3820/7677] 10.222.0.33 () {60 vars in 1242 bytes} [Sat May 28 21:33:47 2022] PATCH /identity/v3/users/c15d86b3c2cd4991944bb8aaeff7cd63 => generated 296 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:48.178259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828472f-fbd0-4f3c-8ea9-b888a249185e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:48.178259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828472f-fbd0-4f3c-8ea9-b888a249185e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:48.178259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828472f-fbd0-4f3c-8ea9-b888a249185e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:48.209971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b828472f-fbd0-4f3c-8ea9-b888a249185e None None] MFA Rules not processed for user `c15d86b3c2cd4991944bb8aaeff7cd63`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:48.211196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b828472f-fbd0-4f3c-8ea9-b888a249185e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:48.219627 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3858/7678] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:48 2022] POST /identity/v3/auth/tokens => generated 348 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 21:33:48.223856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17390eef-5de5-4c7b-bffa-d4b09c891241 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:48.225343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-17390eef-5de5-4c7b-bffa-d4b09c891241 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:48.238468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17390eef-5de5-4c7b-bffa-d4b09c891241 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:48.243072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe456ee-2f2f-4e7e-bda2-f604a7ab16e6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:48.243938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe456ee-2f2f-4e7e-bda2-f604a7ab16e6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:48.244223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe456ee-2f2f-4e7e-bda2-f604a7ab16e6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:48.244791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe456ee-2f2f-4e7e-bda2-f604a7ab16e6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users/c15d86b3c2cd4991944bb8aaeff7cd63` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:48.246130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe456ee-2f2f-4e7e-bda2-f604a7ab16e6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:update_user(user_id=c15d86b3c2cd4991944bb8aaeff7cd63)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:48.248090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe456ee-2f2f-4e7e-bda2-f604a7ab16e6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:48.358689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bbe456ee-2f2f-4e7e-bda2-f604a7ab16e6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'c15d86b3c2cd4991944bb8aaeff7cd63'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:48.467820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bbe456ee-2f2f-4e7e-bda2-f604a7ab16e6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invalidating the token cache because user c15d86b3c2cd4991944bb8aaeff7cd63 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:48.468210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bbe456ee-2f2f-4e7e-bda2-f604a7ab16e6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:48.469005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bbe456ee-2f2f-4e7e-bda2-f604a7ab16e6 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:48.471198 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3821/7679] 10.222.0.33 () {60 vars in 1242 bytes} [Sat May 28 21:33:48 2022] PATCH /identity/v3/users/c15d86b3c2cd4991944bb8aaeff7cd63 => generated 296 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:48.480938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828472f-fbd0-4f3c-8ea9-b888a249185e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:48.493058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828472f-fbd0-4f3c-8ea9-b888a249185e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:48.497615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19d73b-91f2-4cd7-9f8b-7a2f2588c7ee tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:48.498508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19d73b-91f2-4cd7-9f8b-7a2f2588c7ee tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:48.498801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19d73b-91f2-4cd7-9f8b-7a2f2588c7ee tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:48.499080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19d73b-91f2-4cd7-9f8b-7a2f2588c7ee tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users/c15d86b3c2cd4991944bb8aaeff7cd63` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:48.513641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19d73b-91f2-4cd7-9f8b-7a2f2588c7ee tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:update_user(user_id=c15d86b3c2cd4991944bb8aaeff7cd63)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:48.514761 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-3a19d73b-91f2-4cd7-9f8b-7a2f2588c7ee tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:update_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:33:48.514761 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:33:48.514761 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:33:48.515974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19d73b-91f2-4cd7-9f8b-7a2f2588c7ee tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:48.623892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3a19d73b-91f2-4cd7-9f8b-7a2f2588c7ee tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'c15d86b3c2cd4991944bb8aaeff7cd63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:48.693129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3a19d73b-91f2-4cd7-9f8b-7a2f2588c7ee tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invalidating the token cache because user c15d86b3c2cd4991944bb8aaeff7cd63 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:48.693479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3a19d73b-91f2-4cd7-9f8b-7a2f2588c7ee tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:48.694134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3a19d73b-91f2-4cd7-9f8b-7a2f2588c7ee tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:48.696984 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3859/7680] 10.222.0.33 () {60 vars in 1242 bytes} [Sat May 28 21:33:48 2022] PATCH /identity/v3/users/c15d86b3c2cd4991944bb8aaeff7cd63 => generated 296 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:48.700755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c642809-bd1a-4c7d-844a-3cf2f8ec830e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:48.701052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c642809-bd1a-4c7d-844a-3cf2f8ec830e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:48.701349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c642809-bd1a-4c7d-844a-3cf2f8ec830e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:48.732100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2c642809-bd1a-4c7d-844a-3cf2f8ec830e None None] MFA Rules not processed for user `c15d86b3c2cd4991944bb8aaeff7cd63`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:48.733298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2c642809-bd1a-4c7d-844a-3cf2f8ec830e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:48.736394 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3822/7681] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:48 2022] POST /identity/v3/auth/tokens => generated 348 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 21:33:48.740663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19d73b-91f2-4cd7-9f8b-7a2f2588c7ee tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:48.742050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3a19d73b-91f2-4cd7-9f8b-7a2f2588c7ee tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:48.755547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19d73b-91f2-4cd7-9f8b-7a2f2588c7ee tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:48.760168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c9ec0-442a-4075-b2c2-0025976c7abf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:48.761147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c9ec0-442a-4075-b2c2-0025976c7abf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:48.761429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c9ec0-442a-4075-b2c2-0025976c7abf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:48.761703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c9ec0-442a-4075-b2c2-0025976c7abf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users/c15d86b3c2cd4991944bb8aaeff7cd63` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:48.763011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c9ec0-442a-4075-b2c2-0025976c7abf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:update_user(user_id=c15d86b3c2cd4991944bb8aaeff7cd63)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:48.764942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c9ec0-442a-4075-b2c2-0025976c7abf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:48.907525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-047c9ec0-442a-4075-b2c2-0025976c7abf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'c15d86b3c2cd4991944bb8aaeff7cd63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:48.984645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-047c9ec0-442a-4075-b2c2-0025976c7abf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invalidating the token cache because user c15d86b3c2cd4991944bb8aaeff7cd63 was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:48.984952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-047c9ec0-442a-4075-b2c2-0025976c7abf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:48.985588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-047c9ec0-442a-4075-b2c2-0025976c7abf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:48.987521 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3860/7682] 10.222.0.33 () {60 vars in 1242 bytes} [Sat May 28 21:33:48 2022] PATCH /identity/v3/users/c15d86b3c2cd4991944bb8aaeff7cd63 => generated 296 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:48.992699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c642809-bd1a-4c7d-844a-3cf2f8ec830e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:49.004758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c642809-bd1a-4c7d-844a-3cf2f8ec830e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:49.009285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a8a09-1f72-4432-a58a-6b40bd74f0c4 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:49.009981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a8a09-1f72-4432-a58a-6b40bd74f0c4 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:49.010268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a8a09-1f72-4432-a58a-6b40bd74f0c4 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:49.010547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a8a09-1f72-4432-a58a-6b40bd74f0c4 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:49.011320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a8a09-1f72-4432-a58a-6b40bd74f0c4 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:49.013203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a8a09-1f72-4432-a58a-6b40bd74f0c4 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:49.022673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=80,keystone:DELETE=12,keystone:INSERT=6,keystone:UPDATE=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:49.131139 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3823/7683] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:33:48 2022] POST /identity/v3/users => generated 265 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:33:49.135255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c9ec0-442a-4075-b2c2-0025976c7abf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:49.136677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-047c9ec0-442a-4075-b2c2-0025976c7abf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:49.149145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c9ec0-442a-4075-b2c2-0025976c7abf tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:49.156042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa6818-a92e-4ef3-a28e-000036e5dd91 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:49.156922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa6818-a92e-4ef3-a28e-000036e5dd91 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:49.157210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa6818-a92e-4ef3-a28e-000036e5dd91 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:49.157822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa6818-a92e-4ef3-a28e-000036e5dd91 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users/493d2469d47e4fab959c13532dfc9285/password` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:49.340360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-32aa6818-a92e-4ef3-a28e-000036e5dd91 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '493d2469d47e4fab959c13532dfc9285'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:49.472597 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3861/7684] 10.222.0.33 () {60 vars in 1268 bytes} [Sat May 28 21:33:49 2022] POST /identity/v3/users/493d2469d47e4fab959c13532dfc9285/password => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:33:50.478419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d8c14-4397-4cd4-9b1d-28aac7c8ac96 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:50.478929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d8c14-4397-4cd4-9b1d-28aac7c8ac96 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:50.479364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d8c14-4397-4cd4-9b1d-28aac7c8ac96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:50.496825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ea8d8c14-4397-4cd4-9b1d-28aac7c8ac96 None None] MFA Rules not processed for user `493d2469d47e4fab959c13532dfc9285`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:50.498394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ea8d8c14-4397-4cd4-9b1d-28aac7c8ac96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:50.501689 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3824/7685] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:50 2022] POST /identity/v3/auth/tokens => generated 330 bytes in 25 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 21:33:50.505816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa6818-a92e-4ef3-a28e-000036e5dd91 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:50.516553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa6818-a92e-4ef3-a28e-000036e5dd91 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:50.529523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12731e-56fa-4a40-8a3f-0331a3f82852 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:50.529523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12731e-56fa-4a40-8a3f-0331a3f82852 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:50.529523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12731e-56fa-4a40-8a3f-0331a3f82852 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:50.529801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12731e-56fa-4a40-8a3f-0331a3f82852 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:50.533380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12731e-56fa-4a40-8a3f-0331a3f82852 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:50.535622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12731e-56fa-4a40-8a3f-0331a3f82852 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:50.541951 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3862/7686] 10.222.0.33 () {56 vars in 1262 bytes} [Sat May 28 21:33:50 2022] GET /identity/v3/auth/tokens => generated 330 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 21:33:50.550555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d8c14-4397-4cd4-9b1d-28aac7c8ac96 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:50.558957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d8c14-4397-4cd4-9b1d-28aac7c8ac96 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:50.563409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bf5e7-2449-46fb-8c16-4030b92d5eda tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:50.564269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bf5e7-2449-46fb-8c16-4030b92d5eda tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:50.564553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bf5e7-2449-46fb-8c16-4030b92d5eda tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:50.564829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bf5e7-2449-46fb-8c16-4030b92d5eda tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users/493d2469d47e4fab959c13532dfc9285` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:50.566154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9bf5e7-2449-46fb-8c16-4030b92d5eda tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:delete_user(user_id=493d2469d47e4fab959c13532dfc9285)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:50.568230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9bf5e7-2449-46fb-8c16-4030b92d5eda tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:50.712031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1f9bf5e7-2449-46fb-8c16-4030b92d5eda tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '493d2469d47e4fab959c13532dfc9285'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:50.712797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1f9bf5e7-2449-46fb-8c16-4030b92d5eda tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '493d2469d47e4fab959c13532dfc9285'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:50.720311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1f9bf5e7-2449-46fb-8c16-4030b92d5eda tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '493d2469d47e4fab959c13532dfc9285'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:50.797828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1f9bf5e7-2449-46fb-8c16-4030b92d5eda tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '493d2469d47e4fab959c13532dfc9285'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:50.799493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1f9bf5e7-2449-46fb-8c16-4030b92d5eda tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '493d2469d47e4fab959c13532dfc9285'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:50.818260 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3825/7687] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:50 2022] DELETE /identity/v3/users/493d2469d47e4fab959c13532dfc9285 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:50.823292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12731e-56fa-4a40-8a3f-0331a3f82852 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:50.834021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12731e-56fa-4a40-8a3f-0331a3f82852 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:50.853120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=80,keystone:DELETE=10,keystone:INSERT=7,keystone:UPDATE=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:50.890666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f764c0a-2327-4bae-93fc-75a902451450 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:50.891333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f764c0a-2327-4bae-93fc-75a902451450 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:50.891621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f764c0a-2327-4bae-93fc-75a902451450 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:50.891903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f764c0a-2327-4bae-93fc-75a902451450 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:50.892626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f764c0a-2327-4bae-93fc-75a902451450 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:50.894503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f764c0a-2327-4bae-93fc-75a902451450 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:50.974281 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3863/7688] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:50 2022] POST /identity/v3/users => generated 364 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:50.978566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bf5e7-2449-46fb-8c16-4030b92d5eda tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:50.980018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1f9bf5e7-2449-46fb-8c16-4030b92d5eda tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:50.993310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bf5e7-2449-46fb-8c16-4030b92d5eda tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:50.998042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80d4a0-6f3b-4aa5-95d1-73df8071de33 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:50.998701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80d4a0-6f3b-4aa5-95d1-73df8071de33 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:50.998986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80d4a0-6f3b-4aa5-95d1-73df8071de33 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:50.999262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80d4a0-6f3b-4aa5-95d1-73df8071de33 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:50.999944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80d4a0-6f3b-4aa5-95d1-73df8071de33 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:51.002176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80d4a0-6f3b-4aa5-95d1-73df8071de33 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:51.118045 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3826/7689] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:50 2022] POST /identity/v3/projects => generated 360 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:51.122276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f764c0a-2327-4bae-93fc-75a902451450 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:51.132748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f764c0a-2327-4bae-93fc-75a902451450 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:51.192750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7260a-5fd3-4d7d-9a1c-f7150840050f tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:51.193646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7260a-5fd3-4d7d-9a1c-f7150840050f tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:51.193985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7260a-5fd3-4d7d-9a1c-f7150840050f tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:51.194274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b7260a-5fd3-4d7d-9a1c-f7150840050f tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users/a756838d3f144354bc08e1dae19d61f4` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:51.208593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7260a-5fd3-4d7d-9a1c-f7150840050f tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:update_user(user_id=a756838d3f144354bc08e1dae19d61f4)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:51.210743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b7260a-5fd3-4d7d-9a1c-f7150840050f tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:51.282284 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3864/7690] 10.222.0.33 () {60 vars in 1243 bytes} [Sat May 28 21:33:51 2022] PATCH /identity/v3/users/a756838d3f144354bc08e1dae19d61f4 => generated 521 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:51.286392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80d4a0-6f3b-4aa5-95d1-73df8071de33 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:51.298045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80d4a0-6f3b-4aa5-95d1-73df8071de33 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:51.302956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281019d0-3a22-4324-bed9-ba83a751261e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:51.303868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281019d0-3a22-4324-bed9-ba83a751261e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:51.304186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281019d0-3a22-4324-bed9-ba83a751261e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:51.304492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281019d0-3a22-4324-bed9-ba83a751261e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users/a756838d3f144354bc08e1dae19d61f4` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:51.320343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281019d0-3a22-4324-bed9-ba83a751261e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:get_user(user_id=a756838d3f144354bc08e1dae19d61f4)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:51.322775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281019d0-3a22-4324-bed9-ba83a751261e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:51.327894 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3827/7691] 10.222.0.33 () {58 vars in 1220 bytes} [Sat May 28 21:33:51 2022] GET /identity/v3/users/a756838d3f144354bc08e1dae19d61f4 => generated 388 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:33:51.329424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7260a-5fd3-4d7d-9a1c-f7150840050f tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:51.341233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b7260a-5fd3-4d7d-9a1c-f7150840050f tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:51.346005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c056502-825c-4afa-a8cf-58c2eab91138 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:51.346901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c056502-825c-4afa-a8cf-58c2eab91138 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:51.347191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c056502-825c-4afa-a8cf-58c2eab91138 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:51.347652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c056502-825c-4afa-a8cf-58c2eab91138 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/projects/13f1dc5ac8764faab4f2ca8b854192dc` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:51.348719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c056502-825c-4afa-a8cf-58c2eab91138 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:delete_project(project_id=13f1dc5ac8764faab4f2ca8b854192dc)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:51.350747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c056502-825c-4afa-a8cf-58c2eab91138 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:51.439695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1c056502-825c-4afa-a8cf-58c2eab91138 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13f1dc5ac8764faab4f2ca8b854192dc', 'request_id': 'req-1c056502-825c-4afa-a8cf-58c2eab91138'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:51.445160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1c056502-825c-4afa-a8cf-58c2eab91138 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13f1dc5ac8764faab4f2ca8b854192dc', 'request_id': 'req-1c056502-825c-4afa-a8cf-58c2eab91138'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:51.517626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1c056502-825c-4afa-a8cf-58c2eab91138 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13f1dc5ac8764faab4f2ca8b854192dc', 'request_id': 'req-1c056502-825c-4afa-a8cf-58c2eab91138'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:51.532506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1c056502-825c-4afa-a8cf-58c2eab91138 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] The token cache is being invalidate because project 13f1dc5ac8764faab4f2ca8b854192dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:51.532803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1c056502-825c-4afa-a8cf-58c2eab91138 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:51.533615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1c056502-825c-4afa-a8cf-58c2eab91138 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:51.535809 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3865/7692] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:51 2022] DELETE /identity/v3/projects/13f1dc5ac8764faab4f2ca8b854192dc => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:51.539784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281019d0-3a22-4324-bed9-ba83a751261e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:51.541925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-281019d0-3a22-4324-bed9-ba83a751261e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:51.555923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281019d0-3a22-4324-bed9-ba83a751261e tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:51.615635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ab2f0-c7b0-495f-9147-88f551d04efb tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '652362beeec344afb1eb6737eb3e2bdd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ebc5e3a401fb457c926c5351325cc6fb', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:51.616517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ab2f0-c7b0-495f-9147-88f551d04efb tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:51.616805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ab2f0-c7b0-495f-9147-88f551d04efb tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:51.617081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ab2f0-c7b0-495f-9147-88f551d04efb tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] PATH_INFO: `/v3/users/a756838d3f144354bc08e1dae19d61f4` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:51.618328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ab2f0-c7b0-495f-9147-88f551d04efb tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorizing `identity:delete_user(user_id=a756838d3f144354bc08e1dae19d61f4)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:51.620323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ab2f0-c7b0-495f-9147-88f551d04efb tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:51.642258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=4,keystone:SELECT=83,keystone:DELETE=13 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:51.747468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a86ab2f0-c7b0-495f-9147-88f551d04efb tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a756838d3f144354bc08e1dae19d61f4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:51.748180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a86ab2f0-c7b0-495f-9147-88f551d04efb tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a756838d3f144354bc08e1dae19d61f4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:51.755066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a86ab2f0-c7b0-495f-9147-88f551d04efb tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a756838d3f144354bc08e1dae19d61f4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:51.841799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a86ab2f0-c7b0-495f-9147-88f551d04efb tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a756838d3f144354bc08e1dae19d61f4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:51.842577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a86ab2f0-c7b0-495f-9147-88f551d04efb tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a756838d3f144354bc08e1dae19d61f4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:51.860504 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3828/7693] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:51 2022] DELETE /identity/v3/users/a756838d3f144354bc08e1dae19d61f4 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:51.865643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c056502-825c-4afa-a8cf-58c2eab91138 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:51.868361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1c056502-825c-4afa-a8cf-58c2eab91138 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:51.877770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:UPDATE=2,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:51.931751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c056502-825c-4afa-a8cf-58c2eab91138 tempest-UsersV3TestJSON-1058086332 tempest-UsersV3TestJSON-1058086332-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:51.988507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0910076-5852-43e6-84e9-9a4eb9092b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:51.989463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0910076-5852-43e6-84e9-9a4eb9092b98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:51.989748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0910076-5852-43e6-84e9-9a4eb9092b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:51.990066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0910076-5852-43e6-84e9-9a4eb9092b98 admin admin] PATH_INFO: `/v3/users/c15d86b3c2cd4991944bb8aaeff7cd63` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:52.005495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0910076-5852-43e6-84e9-9a4eb9092b98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c15d86b3c2cd4991944bb8aaeff7cd63)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:52.007581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0910076-5852-43e6-84e9-9a4eb9092b98 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:52.223622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c0910076-5852-43e6-84e9-9a4eb9092b98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c15d86b3c2cd4991944bb8aaeff7cd63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:52.230629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c0910076-5852-43e6-84e9-9a4eb9092b98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c15d86b3c2cd4991944bb8aaeff7cd63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:52.231316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c0910076-5852-43e6-84e9-9a4eb9092b98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c15d86b3c2cd4991944bb8aaeff7cd63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:52.232137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c0910076-5852-43e6-84e9-9a4eb9092b98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c15d86b3c2cd4991944bb8aaeff7cd63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:52.317630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c0910076-5852-43e6-84e9-9a4eb9092b98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c15d86b3c2cd4991944bb8aaeff7cd63'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:52.336619 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3866/7694] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:51 2022] DELETE /identity/v3/users/c15d86b3c2cd4991944bb8aaeff7cd63 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:52.341983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38f715-4aef-4a02-ba04-3554bb20ab9a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:52.342280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38f715-4aef-4a02-ba04-3554bb20ab9a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:52.342563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38f715-4aef-4a02-ba04-3554bb20ab9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:52.360258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4d38f715-4aef-4a02-ba04-3554bb20ab9a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:52.361451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4d38f715-4aef-4a02-ba04-3554bb20ab9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:52.426288 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3829/7695] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:52.435946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0910076-5852-43e6-84e9-9a4eb9092b98 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:52.438113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c0910076-5852-43e6-84e9-9a4eb9092b98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:52.500051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0910076-5852-43e6-84e9-9a4eb9092b98 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:52.506441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9933a0-a8e9-42a0-be31-80a480b632b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:52.507114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9933a0-a8e9-42a0-be31-80a480b632b0 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:52.507408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9933a0-a8e9-42a0-be31-80a480b632b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:52.507805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9933a0-a8e9-42a0-be31-80a480b632b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:52.518763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9933a0-a8e9-42a0-be31-80a480b632b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:52.520865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9933a0-a8e9-42a0-be31-80a480b632b0 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:52.532399 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3867/7696] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:33:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:33:52.604081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:53.005213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38f715-4aef-4a02-ba04-3554bb20ab9a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:53.006678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4d38f715-4aef-4a02-ba04-3554bb20ab9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:53.024185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38f715-4aef-4a02-ba04-3554bb20ab9a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:53.030456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ce062-ddb0-4ffb-a155-bc71fdc809f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:53.031314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ce062-ddb0-4ffb-a155-bc71fdc809f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:53.031602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ce062-ddb0-4ffb-a155-bc71fdc809f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:53.031883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ce062-ddb0-4ffb-a155-bc71fdc809f5 admin admin] PATH_INFO: `/v3/projects/78088426c39843329565a38ef1567431` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:53.033675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ce062-ddb0-4ffb-a155-bc71fdc809f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78088426c39843329565a38ef1567431)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:53.035649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921ce062-ddb0-4ffb-a155-bc71fdc809f5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:53.146950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-921ce062-ddb0-4ffb-a155-bc71fdc809f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78088426c39843329565a38ef1567431', 'request_id': 'req-921ce062-ddb0-4ffb-a155-bc71fdc809f5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:53.152700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-921ce062-ddb0-4ffb-a155-bc71fdc809f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78088426c39843329565a38ef1567431', 'request_id': 'req-921ce062-ddb0-4ffb-a155-bc71fdc809f5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:53.254870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-921ce062-ddb0-4ffb-a155-bc71fdc809f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78088426c39843329565a38ef1567431', 'request_id': 'req-921ce062-ddb0-4ffb-a155-bc71fdc809f5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:53.270885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-921ce062-ddb0-4ffb-a155-bc71fdc809f5 admin admin] The token cache is being invalidate because project 78088426c39843329565a38ef1567431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:53.271238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-921ce062-ddb0-4ffb-a155-bc71fdc809f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:53.271910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-921ce062-ddb0-4ffb-a155-bc71fdc809f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:53.273809 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3830/7697] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:53 2022] DELETE /identity/v3/projects/78088426c39843329565a38ef1567431 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:53.278303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9933a0-a8e9-42a0-be31-80a480b632b0 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:53.288383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9933a0-a8e9-42a0-be31-80a480b632b0 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:53.344661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c505c-f090-4f52-9b9f-83562142cdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:53.345535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c505c-f090-4f52-9b9f-83562142cdea admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:53.345818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c505c-f090-4f52-9b9f-83562142cdea admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:53.346134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c505c-f090-4f52-9b9f-83562142cdea admin admin] PATH_INFO: `/v3/users/652362beeec344afb1eb6737eb3e2bdd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:53.347350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c505c-f090-4f52-9b9f-83562142cdea admin admin] RBAC: Authorizing `identity:delete_user(user_id=652362beeec344afb1eb6737eb3e2bdd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:53.349271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c505c-f090-4f52-9b9f-83562142cdea admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:53.621609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3d4c505c-f090-4f52-9b9f-83562142cdea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '652362beeec344afb1eb6737eb3e2bdd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:53.629754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3d4c505c-f090-4f52-9b9f-83562142cdea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '652362beeec344afb1eb6737eb3e2bdd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:53.630610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3d4c505c-f090-4f52-9b9f-83562142cdea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '652362beeec344afb1eb6737eb3e2bdd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:53.631227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3d4c505c-f090-4f52-9b9f-83562142cdea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '652362beeec344afb1eb6737eb3e2bdd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:53.710913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3d4c505c-f090-4f52-9b9f-83562142cdea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '652362beeec344afb1eb6737eb3e2bdd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:53.733075 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3868/7698] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:53 2022] DELETE /identity/v3/users/652362beeec344afb1eb6737eb3e2bdd => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:54.192076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ce062-ddb0-4ffb-a155-bc71fdc809f5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:54.192076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-921ce062-ddb0-4ffb-a155-bc71fdc809f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:54.231079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921ce062-ddb0-4ffb-a155-bc71fdc809f5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:54.238464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:54.275277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32829b1-ac4f-4663-a20e-15ba6e99e358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:54.276148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32829b1-ac4f-4663-a20e-15ba6e99e358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:54.280996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32829b1-ac4f-4663-a20e-15ba6e99e358 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:54.280996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32829b1-ac4f-4663-a20e-15ba6e99e358 admin admin] PATH_INFO: `/v3/projects/ebc5e3a401fb457c926c5351325cc6fb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:54.280996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32829b1-ac4f-4663-a20e-15ba6e99e358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebc5e3a401fb457c926c5351325cc6fb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:54.280996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32829b1-ac4f-4663-a20e-15ba6e99e358 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:54.381666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c32829b1-ac4f-4663-a20e-15ba6e99e358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebc5e3a401fb457c926c5351325cc6fb', 'request_id': 'req-c32829b1-ac4f-4663-a20e-15ba6e99e358'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:54.387222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c32829b1-ac4f-4663-a20e-15ba6e99e358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebc5e3a401fb457c926c5351325cc6fb', 'request_id': 'req-c32829b1-ac4f-4663-a20e-15ba6e99e358'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:54.443515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c32829b1-ac4f-4663-a20e-15ba6e99e358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebc5e3a401fb457c926c5351325cc6fb', 'request_id': 'req-c32829b1-ac4f-4663-a20e-15ba6e99e358'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:54.455673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c32829b1-ac4f-4663-a20e-15ba6e99e358 admin admin] The token cache is being invalidate because project ebc5e3a401fb457c926c5351325cc6fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:54.456132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c32829b1-ac4f-4663-a20e-15ba6e99e358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:54.456618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c32829b1-ac4f-4663-a20e-15ba6e99e358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:54.458211 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3831/7699] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:54 2022] DELETE /identity/v3/projects/ebc5e3a401fb457c926c5351325cc6fb => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:54.463303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a280f18-5b99-450d-b059-1dd589908a35 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:54.463541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a280f18-5b99-450d-b059-1dd589908a35 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:54.463821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a280f18-5b99-450d-b059-1dd589908a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:54.484015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6a280f18-5b99-450d-b059-1dd589908a35 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:54.485336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6a280f18-5b99-450d-b059-1dd589908a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:54.529568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:54.548026 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3869/7700] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:54.555961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dbf67-b6a2-488a-88ed-83843bda03e1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:54.556204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dbf67-b6a2-488a-88ed-83843bda03e1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:54.556493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dbf67-b6a2-488a-88ed-83843bda03e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:54.579813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-960dbf67-b6a2-488a-88ed-83843bda03e1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:54.581166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-960dbf67-b6a2-488a-88ed-83843bda03e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:54.589900 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3832/7701] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:54.594637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a280f18-5b99-450d-b059-1dd589908a35 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:54.606059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a280f18-5b99-450d-b059-1dd589908a35 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:54.611215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e75bf-96d8-4451-a84f-67144bf5f900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:54.611959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e75bf-96d8-4451-a84f-67144bf5f900 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:54.612245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e75bf-96d8-4451-a84f-67144bf5f900 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:54.612521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e75bf-96d8-4451-a84f-67144bf5f900 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:54.613265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e75bf-96d8-4451-a84f-67144bf5f900 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:33:54.613581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e75bf-96d8-4451-a84f-67144bf5f900 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:54.615255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e75bf-96d8-4451-a84f-67144bf5f900 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:54.635109 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3870/7702] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:54.636633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dbf67-b6a2-488a-88ed-83843bda03e1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:54.647270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dbf67-b6a2-488a-88ed-83843bda03e1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:54.654480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee24bb02-3181-40e3-8ce6-86b9ec1bdb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:54.655140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24bb02-3181-40e3-8ce6-86b9ec1bdb4a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:54.655502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24bb02-3181-40e3-8ce6-86b9ec1bdb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:54.655782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24bb02-3181-40e3-8ce6-86b9ec1bdb4a admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:54.657024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee24bb02-3181-40e3-8ce6-86b9ec1bdb4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:54.659070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee24bb02-3181-40e3-8ce6-86b9ec1bdb4a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:54.786972 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3833/7703] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:54 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:54.794082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e75bf-96d8-4451-a84f-67144bf5f900 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:54.801595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e75bf-96d8-4451-a84f-67144bf5f900 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:54.866375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150508c-c2b7-4fc1-9ff5-df83d1db1a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:54.867049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150508c-c2b7-4fc1-9ff5-df83d1db1a33 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:54.867338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150508c-c2b7-4fc1-9ff5-df83d1db1a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:54.867622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1150508c-c2b7-4fc1-9ff5-df83d1db1a33 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:54.868336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150508c-c2b7-4fc1-9ff5-df83d1db1a33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:54.870290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1150508c-c2b7-4fc1-9ff5-df83d1db1a33 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:54.975520 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3871/7704] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:54 2022] POST /identity/v3/users => generated 286 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:54.979304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee24bb02-3181-40e3-8ce6-86b9ec1bdb4a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:54.989934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee24bb02-3181-40e3-8ce6-86b9ec1bdb4a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:54.994675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7632e7-9c4b-44c8-9d27-08793e9e708d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:55.000360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7632e7-9c4b-44c8-9d27-08793e9e708d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:55.000360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7632e7-9c4b-44c8-9d27-08793e9e708d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:55.000360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7632e7-9c4b-44c8-9d27-08793e9e708d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:55.000360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7632e7-9c4b-44c8-9d27-08793e9e708d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:55.000360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7632e7-9c4b-44c8-9d27-08793e9e708d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:55.009227 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3834/7705] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:55.013796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150508c-c2b7-4fc1-9ff5-df83d1db1a33 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:55.024969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1150508c-c2b7-4fc1-9ff5-df83d1db1a33 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:55.029783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db92e89e-6b71-4345-8d59-9c6f20a56d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:55.030498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92e89e-6b71-4345-8d59-9c6f20a56d30 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:55.030814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92e89e-6b71-4345-8d59-9c6f20a56d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:55.031127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92e89e-6b71-4345-8d59-9c6f20a56d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:55.031862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92e89e-6b71-4345-8d59-9c6f20a56d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:55.033464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db92e89e-6b71-4345-8d59-9c6f20a56d30 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:55.043930 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3872/7706] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:55.048380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7632e7-9c4b-44c8-9d27-08793e9e708d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:55.059509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7632e7-9c4b-44c8-9d27-08793e9e708d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:55.064694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5fe6b8-0687-44a8-9468-cd3e8714aafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:55.065949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fe6b8-0687-44a8-9468-cd3e8714aafc admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:55.066238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fe6b8-0687-44a8-9468-cd3e8714aafc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:55.070163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5fe6b8-0687-44a8-9468-cd3e8714aafc admin admin] PATH_INFO: `/v3/projects/cd8b43d3bb904f48835951b08581b137/users/7188d7f819f848c89629d127b493f410/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:55.083486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5fe6b8-0687-44a8-9468-cd3e8714aafc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8b43d3bb904f48835951b08581b137, user_id=7188d7f819f848c89629d127b493f410, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:55.088166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5fe6b8-0687-44a8-9468-cd3e8714aafc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:55.175618 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3835/7707] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:55 2022] PUT /identity/v3/projects/cd8b43d3bb904f48835951b08581b137/users/7188d7f819f848c89629d127b493f410/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:55.278237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a0db6-fabc-4f16-ab3b-60d267a3d7f8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:55.278569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a0db6-fabc-4f16-ab3b-60d267a3d7f8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:55.278852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a0db6-fabc-4f16-ab3b-60d267a3d7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:55.306389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fe5a0db6-fabc-4f16-ab3b-60d267a3d7f8 None None] MFA Rules not processed for user `7188d7f819f848c89629d127b493f410`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:55.307611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fe5a0db6-fabc-4f16-ab3b-60d267a3d7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:55.375929 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3873/7708] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:55.380612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5fe6b8-0687-44a8-9468-cd3e8714aafc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:55.430785 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:55.445065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5fe6b8-0687-44a8-9468-cd3e8714aafc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:55.450532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365d18a-7935-416d-a63f-6075a66abf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:55.451201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365d18a-7935-416d-a63f-6075a66abf77 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:55.453928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365d18a-7935-416d-a63f-6075a66abf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:55.453928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365d18a-7935-416d-a63f-6075a66abf77 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:55.453928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365d18a-7935-416d-a63f-6075a66abf77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:55.454512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365d18a-7935-416d-a63f-6075a66abf77 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:55.560580 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3836/7709] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:55 2022] POST /identity/v3/projects => generated 371 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:55.564472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a0db6-fabc-4f16-ab3b-60d267a3d7f8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:55.575424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a0db6-fabc-4f16-ab3b-60d267a3d7f8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:55.615465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:55.634520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c744d-a94c-4b55-abcd-83ed36221c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:55.635197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c744d-a94c-4b55-abcd-83ed36221c3e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:55.635758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c744d-a94c-4b55-abcd-83ed36221c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:55.635758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c744d-a94c-4b55-abcd-83ed36221c3e admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:55.636491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c744d-a94c-4b55-abcd-83ed36221c3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:55.638514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805c744d-a94c-4b55-abcd-83ed36221c3e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:55.714075 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3874/7710] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:55 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:55.719732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365d18a-7935-416d-a63f-6075a66abf77 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:55.731016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365d18a-7935-416d-a63f-6075a66abf77 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:55.738201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde8187-8e86-4501-a6f5-e1dea5472fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:55.738201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde8187-8e86-4501-a6f5-e1dea5472fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:55.738201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde8187-8e86-4501-a6f5-e1dea5472fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:55.738474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde8187-8e86-4501-a6f5-e1dea5472fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:55.739183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde8187-8e86-4501-a6f5-e1dea5472fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:55.740770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde8187-8e86-4501-a6f5-e1dea5472fe8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:55.752097 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3837/7711] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:55.756205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c744d-a94c-4b55-abcd-83ed36221c3e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:55.766421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c744d-a94c-4b55-abcd-83ed36221c3e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:55.786044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde8187-8e86-4501-a6f5-e1dea5472fe8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:55.788676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ffde8187-8e86-4501-a6f5-e1dea5472fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:55.799477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e51de-a022-40b6-ac7a-4e2194a9df80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:55.801028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e51de-a022-40b6-ac7a-4e2194a9df80 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:55.802254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e51de-a022-40b6-ac7a-4e2194a9df80 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:55.802669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e51de-a022-40b6-ac7a-4e2194a9df80 admin admin] PATH_INFO: `/v3/projects/9121f455a2df4fa58f9291cbee4c6bae/users/101e48b5a21c4679af145230b23aa84d/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:55.832429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e51de-a022-40b6-ac7a-4e2194a9df80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9121f455a2df4fa58f9291cbee4c6bae, user_id=101e48b5a21c4679af145230b23aa84d, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:55.837077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e51de-a022-40b6-ac7a-4e2194a9df80 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:55.876621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde8187-8e86-4501-a6f5-e1dea5472fe8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:55.882180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e36ffd-dc64-445b-8226-ad8203ab3409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:55.882180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e36ffd-dc64-445b-8226-ad8203ab3409 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:55.882180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e36ffd-dc64-445b-8226-ad8203ab3409 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:55.882180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e36ffd-dc64-445b-8226-ad8203ab3409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:55.883438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-41e36ffd-dc64-445b-8226-ad8203ab3409 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:55.895621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e36ffd-dc64-445b-8226-ad8203ab3409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:55.897062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e36ffd-dc64-445b-8226-ad8203ab3409 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:55.907715 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3838/7712] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:33:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:33:55.946381 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3875/7713] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:55 2022] PUT /identity/v3/projects/9121f455a2df4fa58f9291cbee4c6bae/users/101e48b5a21c4679af145230b23aa84d/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:55.952042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e36ffd-dc64-445b-8226-ad8203ab3409 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:55.962506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e36ffd-dc64-445b-8226-ad8203ab3409 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:56.030097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856963e-90ee-4835-9c05-61c7ae60041a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:56.031250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856963e-90ee-4835-9c05-61c7ae60041a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:56.031875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856963e-90ee-4835-9c05-61c7ae60041a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:56.031991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856963e-90ee-4835-9c05-61c7ae60041a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:56.032841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856963e-90ee-4835-9c05-61c7ae60041a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:56.034557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856963e-90ee-4835-9c05-61c7ae60041a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:56.048188 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3839/7714] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:33:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:33:56.049347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e51de-a022-40b6-ac7a-4e2194a9df80 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:56.067170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e51de-a022-40b6-ac7a-4e2194a9df80 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:56.072341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6d063-07d1-4f9f-8e80-ca837eaf8fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:56.073414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6d063-07d1-4f9f-8e80-ca837eaf8fd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:56.073707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6d063-07d1-4f9f-8e80-ca837eaf8fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:56.074536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6d063-07d1-4f9f-8e80-ca837eaf8fd1 admin admin] PATH_INFO: `/v3/projects/9121f455a2df4fa58f9291cbee4c6bae/users/101e48b5a21c4679af145230b23aa84d/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:56.076874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6d063-07d1-4f9f-8e80-ca837eaf8fd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9121f455a2df4fa58f9291cbee4c6bae, user_id=101e48b5a21c4679af145230b23aa84d, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:56.079600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6d063-07d1-4f9f-8e80-ca837eaf8fd1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:56.092598 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3876/7715] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:33:56 2022] PUT /identity/v3/projects/9121f455a2df4fa58f9291cbee4c6bae/users/101e48b5a21c4679af145230b23aa84d/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:56.103138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c61d0-6b14-470c-969e-4084e469d5a3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:56.103460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c61d0-6b14-470c-969e-4084e469d5a3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:56.103746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c61d0-6b14-470c-969e-4084e469d5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:56.132592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-fa4c61d0-6b14-470c-969e-4084e469d5a3 None None] MFA Rules not processed for user `101e48b5a21c4679af145230b23aa84d`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:56.134121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fa4c61d0-6b14-470c-969e-4084e469d5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:56.208594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:56.234314 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3840/7716] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:56 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:56.239137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6d063-07d1-4f9f-8e80-ca837eaf8fd1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:56.251033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6d063-07d1-4f9f-8e80-ca837eaf8fd1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:56.257401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0488e-b593-4c7c-b8b6-1281b71ff74b tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101e48b5a21c4679af145230b23aa84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9121f455a2df4fa58f9291cbee4c6bae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:56.258167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0488e-b593-4c7c-b8b6-1281b71ff74b tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:56.258616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0488e-b593-4c7c-b8b6-1281b71ff74b tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:56.259063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f0488e-b593-4c7c-b8b6-1281b71ff74b tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:56.259820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0488e-b593-4c7c-b8b6-1281b71ff74b tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:56.262508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f0488e-b593-4c7c-b8b6-1281b71ff74b tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:56.431552 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3877/7717] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:33:56 2022] POST /identity/v3/projects => generated 359 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:56.436446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c61d0-6b14-470c-969e-4084e469d5a3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:56.445603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c61d0-6b14-470c-969e-4084e469d5a3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:56.517118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9b243-7316-4d89-b68f-6bcb2b83addf tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101e48b5a21c4679af145230b23aa84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9121f455a2df4fa58f9291cbee4c6bae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:56.517805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9b243-7316-4d89-b68f-6bcb2b83addf tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:56.518132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9b243-7316-4d89-b68f-6bcb2b83addf tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:56.518411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9b243-7316-4d89-b68f-6bcb2b83addf tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:56.519168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc9b243-7316-4d89-b68f-6bcb2b83addf tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:56.521195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc9b243-7316-4d89-b68f-6bcb2b83addf tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:56.651084 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3841/7718] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:56 2022] POST /identity/v3/users => generated 371 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:33:56.655484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0488e-b593-4c7c-b8b6-1281b71ff74b tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:56.668910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f0488e-b593-4c7c-b8b6-1281b71ff74b tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:56.673867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be505f96-943b-4d46-906e-432fcb35ae5c tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101e48b5a21c4679af145230b23aa84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9121f455a2df4fa58f9291cbee4c6bae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:56.674554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be505f96-943b-4d46-906e-432fcb35ae5c tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:56.675528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be505f96-943b-4d46-906e-432fcb35ae5c tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:56.675844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be505f96-943b-4d46-906e-432fcb35ae5c tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:56.676712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be505f96-943b-4d46-906e-432fcb35ae5c tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: Adding query filter params (domain_id=default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:33:56.677022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be505f96-943b-4d46-906e-432fcb35ae5c tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: Authorizing `identity:list_users()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:56.679108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be505f96-943b-4d46-906e-432fcb35ae5c tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:56.708121 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3878/7719] 10.222.0.33 () {58 vars in 1156 bytes} [Sat May 28 21:33:56 2022] GET /identity/v3/users?domain_id=default => generated 7161 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:33:56.712782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9b243-7316-4d89-b68f-6bcb2b83addf tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:56.725267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9b243-7316-4d89-b68f-6bcb2b83addf tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:56.730809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afd17d-f66f-4fb9-940e-ea0a3f656d87 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101e48b5a21c4679af145230b23aa84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9121f455a2df4fa58f9291cbee4c6bae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:56.731684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afd17d-f66f-4fb9-940e-ea0a3f656d87 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:56.731973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afd17d-f66f-4fb9-940e-ea0a3f656d87 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:56.732779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afd17d-f66f-4fb9-940e-ea0a3f656d87 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] PATH_INFO: `/v3/users/59a6a850103f456abcb994b9d4b6ad9a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:56.748487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7afd17d-f66f-4fb9-940e-ea0a3f656d87 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: Authorizing `identity:update_user(user_id=59a6a850103f456abcb994b9d4b6ad9a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:56.750739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7afd17d-f66f-4fb9-940e-ea0a3f656d87 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:56.902839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d7afd17d-f66f-4fb9-940e-ea0a3f656d87 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '59a6a850103f456abcb994b9d4b6ad9a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:57.036567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d7afd17d-f66f-4fb9-940e-ea0a3f656d87 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Invalidating the token cache because user 59a6a850103f456abcb994b9d4b6ad9a was enabled or disabled. Authorization will be calculated and enforced accordingly the next time they authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:57.044040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d7afd17d-f66f-4fb9-940e-ea0a3f656d87 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:57.044040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d7afd17d-f66f-4fb9-940e-ea0a3f656d87 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:57.044040 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3842/7720] 10.222.0.33 () {60 vars in 1243 bytes} [Sat May 28 21:33:56 2022] PATCH /identity/v3/users/59a6a850103f456abcb994b9d4b6ad9a => generated 485 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:33:57.045418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafba2f0-bb09-43d8-b9a8-191f24526a8c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:57.045711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafba2f0-bb09-43d8-b9a8-191f24526a8c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:57.046026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafba2f0-bb09-43d8-b9a8-191f24526a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:57.061338 ncdv-master-1 devstack@keystone.service[103103]: WARNING keystone.auth.plugins.core [None req-eafba2f0-bb09-43d8-b9a8-191f24526a8c None None] User is disabled: 59a6a850103f456abcb994b9d4b6ad9a: AssertionError: User is disabled: 59a6a850103f456abcb994b9d4b6ad9a May 28 21:33:57.078281 ncdv-master-1 devstack@keystone.service[103103]: WARNING keystone.server.flask.application [None req-eafba2f0-bb09-43d8-b9a8-191f24526a8c None None] Authorization failed. The request you have made requires authentication. from 10.222.0.33: keystone.exception.Unauthorized: The request you have made requires authentication. May 28 21:33:57.080238 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3879/7721] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:57 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 36 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) May 28 21:33:57.084955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afd17d-f66f-4fb9-940e-ea0a3f656d87 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:57.086881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d7afd17d-f66f-4fb9-940e-ea0a3f656d87 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:57.102974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afd17d-f66f-4fb9-940e-ea0a3f656d87 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:57.108964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb148b2-8f13-4571-83b3-01ace55ae98e tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101e48b5a21c4679af145230b23aa84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9121f455a2df4fa58f9291cbee4c6bae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:57.109979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb148b2-8f13-4571-83b3-01ace55ae98e tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:57.110433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb148b2-8f13-4571-83b3-01ace55ae98e tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:57.111534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb148b2-8f13-4571-83b3-01ace55ae98e tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] PATH_INFO: `/v3/users/59a6a850103f456abcb994b9d4b6ad9a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:57.141231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb148b2-8f13-4571-83b3-01ace55ae98e tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: Authorizing `identity:delete_user(user_id=59a6a850103f456abcb994b9d4b6ad9a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:57.152778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb148b2-8f13-4571-83b3-01ace55ae98e tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:57.338059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ecb148b2-8f13-4571-83b3-01ace55ae98e tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59a6a850103f456abcb994b9d4b6ad9a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:57.338715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ecb148b2-8f13-4571-83b3-01ace55ae98e tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59a6a850103f456abcb994b9d4b6ad9a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:57.349245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ecb148b2-8f13-4571-83b3-01ace55ae98e tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59a6a850103f456abcb994b9d4b6ad9a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:57.437789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ecb148b2-8f13-4571-83b3-01ace55ae98e tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59a6a850103f456abcb994b9d4b6ad9a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:57.439088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ecb148b2-8f13-4571-83b3-01ace55ae98e tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59a6a850103f456abcb994b9d4b6ad9a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:57.457779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=83,keystone:INSERT=5,keystone:UPDATE=1,keystone:DELETE=11 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:57.463889 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3843/7722] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:57 2022] DELETE /identity/v3/users/59a6a850103f456abcb994b9d4b6ad9a => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:57.468927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafba2f0-bb09-43d8-b9a8-191f24526a8c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:57.471255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eafba2f0-bb09-43d8-b9a8-191f24526a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:57.591676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafba2f0-bb09-43d8-b9a8-191f24526a8c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:57.591676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101e48b5a21c4679af145230b23aa84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9121f455a2df4fa58f9291cbee4c6bae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:57.591676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:57.592073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:57.592481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] PATH_INFO: `/v3/projects/73ecff0a060540b79fec7651d709ce71` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:57.596902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: Authorizing `identity:delete_project(project_id=73ecff0a060540b79fec7651d709ce71)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:57.596902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:57.759209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73ecff0a060540b79fec7651d709ce71', 'request_id': 'req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:57.764663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:57.766702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73ecff0a060540b79fec7651d709ce71', 'request_id': 'req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:57.872572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73ecff0a060540b79fec7651d709ce71', 'request_id': 'req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:57.887039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] The token cache is being invalidate because project 73ecff0a060540b79fec7651d709ce71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:57.887619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:57.888634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:57.890501 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3880/7723] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:57 2022] DELETE /identity/v3/projects/73ecff0a060540b79fec7651d709ce71 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:33:57.896038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb148b2-8f13-4571-83b3-01ace55ae98e tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:57.909014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb148b2-8f13-4571-83b3-01ace55ae98e tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:57.977780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9481281-cd63-4373-b3a7-376133fcf64a tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '101e48b5a21c4679af145230b23aa84d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9121f455a2df4fa58f9291cbee4c6bae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:57.978500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9481281-cd63-4373-b3a7-376133fcf64a tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:57.978952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9481281-cd63-4373-b3a7-376133fcf64a tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:57.979041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9481281-cd63-4373-b3a7-376133fcf64a tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:57.979761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9481281-cd63-4373-b3a7-376133fcf64a tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:57.982174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9481281-cd63-4373-b3a7-376133fcf64a tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:57.996848 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-b9481281-cd63-4373-b3a7-376133fcf64a tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Could not find domain: 0cf39f2475404c6c8722cfa388bb38f8.: keystone.exception.DomainNotFound: Could not find domain: 0cf39f2475404c6c8722cfa388bb38f8. May 28 21:33:57.996848 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:57.996848 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 21:33:57.996848 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 21:33:57.996848 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:33:57.996848 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:33:57.996848 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:33:57.996848 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:33:57.996848 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 0cf39f2475404c6c8722cfa388bb38f8. May 28 21:33:57.996848 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:57.996848 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:33:57.996848 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:57.996848 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:33:57.996848 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:33:57.996848 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:33:57.997975 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:33:57.997975 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:33:57.997975 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:33:57.997975 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:33:57.997975 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:33:57.997975 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:33:57.997975 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:33:57.997975 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:33:57.997975 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 28 21:33:57.997975 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 28 21:33:57.997975 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:57.997975 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:57.997975 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 21:33:57.997975 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 21:33:57.997975 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 28 21:33:57.997975 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 21:33:57.997975 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 987, in create_user May 28 21:33:57.998993 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application PROVIDERS.resource_api.get_domain(domain_id) May 28 21:33:57.998993 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:33:57.998993 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:33:57.998993 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:33:57.998993 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:33:57.998993 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:33:57.998993 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:33:57.998993 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:33:57.998993 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 21:33:57.998993 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:33:57.998993 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 21:33:57.998993 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:33:57.998993 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:33:57.998993 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:33:57.998993 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 21:33:57.998993 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:33:57.998993 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 21:33:58.000057 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 21:33:58.000057 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 21:33:58.000057 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 0cf39f2475404c6c8722cfa388bb38f8. May 28 21:33:58.000057 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:33:58.000057 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3844/7724] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:33:57 2022] POST /identity/v3/users => generated 112 bytes in 103 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:33:58.003190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:58.004653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:58.118518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898f5a7-3fb0-46f5-97ab-2a0c8acfb1a6 tempest-UsersNegativeTest-1650267973 tempest-UsersNegativeTest-1650267973-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:58.126051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3310a-279c-4371-a471-2e2cf30f4dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:58.127114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3310a-279c-4371-a471-2e2cf30f4dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:58.127457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3310a-279c-4371-a471-2e2cf30f4dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:58.131223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3310a-279c-4371-a471-2e2cf30f4dd7 admin admin] PATH_INFO: `/v3/users/7188d7f819f848c89629d127b493f410` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:58.131223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3310a-279c-4371-a471-2e2cf30f4dd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7188d7f819f848c89629d127b493f410)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:58.131223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3310a-279c-4371-a471-2e2cf30f4dd7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:58.169063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a9bdc-d03b-4e48-ba68-7f60c6d421d8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:58.169512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a9bdc-d03b-4e48-ba68-7f60c6d421d8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:58.169953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a9bdc-d03b-4e48-ba68-7f60c6d421d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:58.198960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-182a9bdc-d03b-4e48-ba68-7f60c6d421d8 None None] MFA Rules not processed for user `c93401a279c9403ca62627e829082968`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:58.201227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-182a9bdc-d03b-4e48-ba68-7f60c6d421d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:58.283481 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3845/7725] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:58.357040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b1c3310a-279c-4371-a471-2e2cf30f4dd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7188d7f819f848c89629d127b493f410'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:58.364227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b1c3310a-279c-4371-a471-2e2cf30f4dd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7188d7f819f848c89629d127b493f410'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:58.364913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b1c3310a-279c-4371-a471-2e2cf30f4dd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7188d7f819f848c89629d127b493f410'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:58.365532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b1c3310a-279c-4371-a471-2e2cf30f4dd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7188d7f819f848c89629d127b493f410'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:58.449826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b1c3310a-279c-4371-a471-2e2cf30f4dd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7188d7f819f848c89629d127b493f410'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:58.467965 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3881/7726] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:58 2022] DELETE /identity/v3/users/7188d7f819f848c89629d127b493f410 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:58.472992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a41b8-9ee5-48ef-8443-1170647bcb98 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:58.473494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a41b8-9ee5-48ef-8443-1170647bcb98 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:58.473625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a41b8-9ee5-48ef-8443-1170647bcb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:58.491595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-546a41b8-9ee5-48ef-8443-1170647bcb98 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:33:58.492824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-546a41b8-9ee5-48ef-8443-1170647bcb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:58.560479 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3846/7727] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:33:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:33:58.573140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3310a-279c-4371-a471-2e2cf30f4dd7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:58.574901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b1c3310a-279c-4371-a471-2e2cf30f4dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:58.654127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3310a-279c-4371-a471-2e2cf30f4dd7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:58.660372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb313c-a389-41f2-985c-f8fb5c26d722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:58.661058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb313c-a389-41f2-985c-f8fb5c26d722 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:58.661348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb313c-a389-41f2-985c-f8fb5c26d722 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:58.661627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb313c-a389-41f2-985c-f8fb5c26d722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:58.673272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb313c-a389-41f2-985c-f8fb5c26d722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:58.675809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb313c-a389-41f2-985c-f8fb5c26d722 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:58.687545 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3882/7728] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:33:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:33:59.180061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a41b8-9ee5-48ef-8443-1170647bcb98 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:59.181558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-546a41b8-9ee5-48ef-8443-1170647bcb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:59.211592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a41b8-9ee5-48ef-8443-1170647bcb98 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:59.221641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d6bd3-03f8-4df7-a603-533598e338af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:59.222495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d6bd3-03f8-4df7-a603-533598e338af service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:59.222873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d6bd3-03f8-4df7-a603-533598e338af service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:59.223244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463d6bd3-03f8-4df7-a603-533598e338af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:59.225371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-463d6bd3-03f8-4df7-a603-533598e338af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:59.244674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d6bd3-03f8-4df7-a603-533598e338af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:59.246417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463d6bd3-03f8-4df7-a603-533598e338af service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:59.259205 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3847/7729] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:33:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:33:59.531538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb313c-a389-41f2-985c-f8fb5c26d722 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:59.533173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-00bb313c-a389-41f2-985c-f8fb5c26d722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:33:59.544158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:59.563708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb313c-a389-41f2-985c-f8fb5c26d722 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:59.569693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b156a-aa37-44a5-a638-bc5dcf384e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:59.570592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b156a-aa37-44a5-a638-bc5dcf384e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:59.570876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b156a-aa37-44a5-a638-bc5dcf384e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:59.571592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9b156a-aa37-44a5-a638-bc5dcf384e68 admin admin] PATH_INFO: `/v3/projects/cd8b43d3bb904f48835951b08581b137` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:59.572579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b156a-aa37-44a5-a638-bc5dcf384e68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd8b43d3bb904f48835951b08581b137)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:59.581055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9b156a-aa37-44a5-a638-bc5dcf384e68 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:33:59.719069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8f9b156a-aa37-44a5-a638-bc5dcf384e68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd8b43d3bb904f48835951b08581b137', 'request_id': 'req-8f9b156a-aa37-44a5-a638-bc5dcf384e68'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:59.726183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8f9b156a-aa37-44a5-a638-bc5dcf384e68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd8b43d3bb904f48835951b08581b137', 'request_id': 'req-8f9b156a-aa37-44a5-a638-bc5dcf384e68'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:59.820164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8f9b156a-aa37-44a5-a638-bc5dcf384e68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd8b43d3bb904f48835951b08581b137', 'request_id': 'req-8f9b156a-aa37-44a5-a638-bc5dcf384e68'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:59.833046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8f9b156a-aa37-44a5-a638-bc5dcf384e68 admin admin] The token cache is being invalidate because project cd8b43d3bb904f48835951b08581b137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:33:59.833494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8f9b156a-aa37-44a5-a638-bc5dcf384e68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:59.834321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-8f9b156a-aa37-44a5-a638-bc5dcf384e68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:33:59.839031 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3883/7730] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:33:59 2022] DELETE /identity/v3/projects/cd8b43d3bb904f48835951b08581b137 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:33:59.840363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d6bd3-03f8-4df7-a603-533598e338af service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:33:59.859705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463d6bd3-03f8-4df7-a603-533598e338af service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:33:59.900108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:59.965810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969cfe7-77d0-4cd7-8a77-9a6b1a682c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:33:59.966755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969cfe7-77d0-4cd7-8a77-9a6b1a682c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:33:59.967039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969cfe7-77d0-4cd7-8a77-9a6b1a682c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:33:59.970155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969cfe7-77d0-4cd7-8a77-9a6b1a682c2a admin admin] PATH_INFO: `/v3/users/101e48b5a21c4679af145230b23aa84d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:33:59.970155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969cfe7-77d0-4cd7-8a77-9a6b1a682c2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=101e48b5a21c4679af145230b23aa84d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:33:59.970607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969cfe7-77d0-4cd7-8a77-9a6b1a682c2a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:00.330664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2969cfe7-77d0-4cd7-8a77-9a6b1a682c2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '101e48b5a21c4679af145230b23aa84d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:00.331409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2969cfe7-77d0-4cd7-8a77-9a6b1a682c2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '101e48b5a21c4679af145230b23aa84d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:00.339289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2969cfe7-77d0-4cd7-8a77-9a6b1a682c2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '101e48b5a21c4679af145230b23aa84d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:00.460705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2969cfe7-77d0-4cd7-8a77-9a6b1a682c2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '101e48b5a21c4679af145230b23aa84d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:00.461590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2969cfe7-77d0-4cd7-8a77-9a6b1a682c2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '101e48b5a21c4679af145230b23aa84d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:00.484475 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3848/7731] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:33:59 2022] DELETE /identity/v3/users/101e48b5a21c4679af145230b23aa84d => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:00.630105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b156a-aa37-44a5-a638-bc5dcf384e68 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:00.631578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8f9b156a-aa37-44a5-a638-bc5dcf384e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:00.646718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9b156a-aa37-44a5-a638-bc5dcf384e68 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:00.715833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969cfe7-77d0-4cd7-8a77-9a6b1a682c2a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:00.718034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2969cfe7-77d0-4cd7-8a77-9a6b1a682c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:00.738051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099b0c7-fc1a-4d35-ad32-6a688f437800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:00.739242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099b0c7-fc1a-4d35-ad32-6a688f437800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:00.739703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099b0c7-fc1a-4d35-ad32-6a688f437800 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:00.740347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099b0c7-fc1a-4d35-ad32-6a688f437800 admin admin] PATH_INFO: `/v3/users/93f78df0f96c47ac97a200a4442db6da` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:00.742433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099b0c7-fc1a-4d35-ad32-6a688f437800 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93f78df0f96c47ac97a200a4442db6da)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:00.744872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099b0c7-fc1a-4d35-ad32-6a688f437800 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:00.970393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1099b0c7-fc1a-4d35-ad32-6a688f437800 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93f78df0f96c47ac97a200a4442db6da'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:00.985046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1099b0c7-fc1a-4d35-ad32-6a688f437800 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93f78df0f96c47ac97a200a4442db6da'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:00.986165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1099b0c7-fc1a-4d35-ad32-6a688f437800 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93f78df0f96c47ac97a200a4442db6da'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:00.986893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1099b0c7-fc1a-4d35-ad32-6a688f437800 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93f78df0f96c47ac97a200a4442db6da'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:00.987485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969cfe7-77d0-4cd7-8a77-9a6b1a682c2a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:00.992714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8c23a-5208-446f-9b3e-9f75fb1457b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:00.993374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8c23a-5208-446f-9b3e-9f75fb1457b4 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:00.993660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8c23a-5208-446f-9b3e-9f75fb1457b4 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:00.993990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8c23a-5208-446f-9b3e-9f75fb1457b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:00.995864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-74a8c23a-5208-446f-9b3e-9f75fb1457b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:01.010702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8c23a-5208-446f-9b3e-9f75fb1457b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:01.014943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8c23a-5208-446f-9b3e-9f75fb1457b4 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:01.025699 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3849/7732] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:34:01.027935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8c23a-5208-446f-9b3e-9f75fb1457b4 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:01.029373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-74a8c23a-5208-446f-9b3e-9f75fb1457b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:01.044537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8c23a-5208-446f-9b3e-9f75fb1457b4 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:01.049505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7fb6be-4b0a-4fbd-a5bc-f801c2b66891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:01.050395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fb6be-4b0a-4fbd-a5bc-f801c2b66891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:01.050676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fb6be-4b0a-4fbd-a5bc-f801c2b66891 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:01.050948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fb6be-4b0a-4fbd-a5bc-f801c2b66891 admin admin] PATH_INFO: `/v3/projects/9121f455a2df4fa58f9291cbee4c6bae` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:01.052117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7fb6be-4b0a-4fbd-a5bc-f801c2b66891 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9121f455a2df4fa58f9291cbee4c6bae)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:01.054046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7fb6be-4b0a-4fbd-a5bc-f801c2b66891 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:01.080367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:01.090291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1099b0c7-fc1a-4d35-ad32-6a688f437800 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93f78df0f96c47ac97a200a4442db6da'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:01.118952 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3884/7733] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:34:00 2022] DELETE /identity/v3/users/93f78df0f96c47ac97a200a4442db6da => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:34:01.241130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1e7fb6be-4b0a-4fbd-a5bc-f801c2b66891 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9121f455a2df4fa58f9291cbee4c6bae', 'request_id': 'req-1e7fb6be-4b0a-4fbd-a5bc-f801c2b66891'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:01.248298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1e7fb6be-4b0a-4fbd-a5bc-f801c2b66891 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9121f455a2df4fa58f9291cbee4c6bae', 'request_id': 'req-1e7fb6be-4b0a-4fbd-a5bc-f801c2b66891'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:01.324286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1e7fb6be-4b0a-4fbd-a5bc-f801c2b66891 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9121f455a2df4fa58f9291cbee4c6bae', 'request_id': 'req-1e7fb6be-4b0a-4fbd-a5bc-f801c2b66891'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:01.337675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1e7fb6be-4b0a-4fbd-a5bc-f801c2b66891 admin admin] The token cache is being invalidate because project 9121f455a2df4fa58f9291cbee4c6bae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:34:01.338010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1e7fb6be-4b0a-4fbd-a5bc-f801c2b66891 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:01.338669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1e7fb6be-4b0a-4fbd-a5bc-f801c2b66891 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:01.341171 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3850/7734] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:34:01 2022] DELETE /identity/v3/projects/9121f455a2df4fa58f9291cbee4c6bae => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:01.346136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74db0a-105f-4220-bd4e-2b47072bb882 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:01.349476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74db0a-105f-4220-bd4e-2b47072bb882 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:01.349476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74db0a-105f-4220-bd4e-2b47072bb882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:01.359295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:01.366729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4b74db0a-105f-4220-bd4e-2b47072bb882 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:01.368074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4b74db0a-105f-4220-bd4e-2b47072bb882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:01.427321 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3885/7735] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:01.438730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74db0a-105f-4220-bd4e-2b47072bb882 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:01.438990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9f1c7-185d-4928-82f8-7d09f43ada17 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:01.439481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9f1c7-185d-4928-82f8-7d09f43ada17 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:01.439915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9f1c7-185d-4928-82f8-7d09f43ada17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:01.440079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4b74db0a-105f-4220-bd4e-2b47072bb882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:01.457035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74db0a-105f-4220-bd4e-2b47072bb882 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:01.462571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-38a9f1c7-185d-4928-82f8-7d09f43ada17 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:01.464079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-38a9f1c7-185d-4928-82f8-7d09f43ada17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:01.470295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea541e3c-18a8-4b06-866c-8decd8327286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:01.471235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea541e3c-18a8-4b06-866c-8decd8327286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:01.471533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea541e3c-18a8-4b06-866c-8decd8327286 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:01.471820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea541e3c-18a8-4b06-866c-8decd8327286 admin admin] PATH_INFO: `/v3/projects/e3cc8c5f2fe44b87bf5c05eadb62936c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:01.473085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea541e3c-18a8-4b06-866c-8decd8327286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3cc8c5f2fe44b87bf5c05eadb62936c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:01.473389 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3851/7736] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:01.477697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9f1c7-185d-4928-82f8-7d09f43ada17 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:01.478469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea541e3c-18a8-4b06-866c-8decd8327286 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:01.487074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a9f1c7-185d-4928-82f8-7d09f43ada17 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:01.491748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed37166-db36-49fe-87e0-1b7d8a7fa567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:01.492404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed37166-db36-49fe-87e0-1b7d8a7fa567 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:01.492690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed37166-db36-49fe-87e0-1b7d8a7fa567 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:01.492968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed37166-db36-49fe-87e0-1b7d8a7fa567 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:01.493824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed37166-db36-49fe-87e0-1b7d8a7fa567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:34:01.494285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed37166-db36-49fe-87e0-1b7d8a7fa567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:01.496073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed37166-db36-49fe-87e0-1b7d8a7fa567 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:01.510180 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3852/7737] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:34:01.514967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed37166-db36-49fe-87e0-1b7d8a7fa567 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:01.524919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed37166-db36-49fe-87e0-1b7d8a7fa567 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:01.530673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42dd01-08bb-46c0-b0a0-2614e71c52b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:01.531319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42dd01-08bb-46c0-b0a0-2614e71c52b8 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:01.531602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42dd01-08bb-46c0-b0a0-2614e71c52b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:01.531877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42dd01-08bb-46c0-b0a0-2614e71c52b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:01.532595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42dd01-08bb-46c0-b0a0-2614e71c52b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:01.534845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42dd01-08bb-46c0-b0a0-2614e71c52b8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:01.609396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ea541e3c-18a8-4b06-866c-8decd8327286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3cc8c5f2fe44b87bf5c05eadb62936c', 'request_id': 'req-ea541e3c-18a8-4b06-866c-8decd8327286'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:01.614963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ea541e3c-18a8-4b06-866c-8decd8327286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3cc8c5f2fe44b87bf5c05eadb62936c', 'request_id': 'req-ea541e3c-18a8-4b06-866c-8decd8327286'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:01.642982 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3853/7738] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:34:01 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:34:01.644162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42dd01-08bb-46c0-b0a0-2614e71c52b8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:01.711456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42dd01-08bb-46c0-b0a0-2614e71c52b8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:01.716570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846f0a4-eeeb-4a5b-9480-babe020f47d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:01.717234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846f0a4-eeeb-4a5b-9480-babe020f47d9 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:01.717516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846f0a4-eeeb-4a5b-9480-babe020f47d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:01.717790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2846f0a4-eeeb-4a5b-9480-babe020f47d9 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:01.718535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2846f0a4-eeeb-4a5b-9480-babe020f47d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:01.720499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2846f0a4-eeeb-4a5b-9480-babe020f47d9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:01.731135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ea541e3c-18a8-4b06-866c-8decd8327286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3cc8c5f2fe44b87bf5c05eadb62936c', 'request_id': 'req-ea541e3c-18a8-4b06-866c-8decd8327286'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:01.740147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ea541e3c-18a8-4b06-866c-8decd8327286 admin admin] The token cache is being invalidate because project e3cc8c5f2fe44b87bf5c05eadb62936c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:34:01.740467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ea541e3c-18a8-4b06-866c-8decd8327286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:01.741171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ea541e3c-18a8-4b06-866c-8decd8327286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:01.742826 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3886/7739] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:34:01 2022] DELETE /identity/v3/projects/e3cc8c5f2fe44b87bf5c05eadb62936c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:01.748013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e864c-ead5-4414-b7bd-33aa2d872759 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:01.748304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e864c-ead5-4414-b7bd-33aa2d872759 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:01.748593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e864c-ead5-4414-b7bd-33aa2d872759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:01.766934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d43e864c-ead5-4414-b7bd-33aa2d872759 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:01.768185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d43e864c-ead5-4414-b7bd-33aa2d872759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:01.776520 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3887/7740] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:34:01.783815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ac03e-ff43-43e2-b713-db403bd4429c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:01.784109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ac03e-ff43-43e2-b713-db403bd4429c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:01.784395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ac03e-ff43-43e2-b713-db403bd4429c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:01.802379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3b5ac03e-ff43-43e2-b713-db403bd4429c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:01.803608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3b5ac03e-ff43-43e2-b713-db403bd4429c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:01.816053 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3888/7741] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:34:01.816998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ac03e-ff43-43e2-b713-db403bd4429c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:01.827121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ac03e-ff43-43e2-b713-db403bd4429c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:01.831915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d98e8-ecb2-40f4-a095-24ac562531dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:01.832576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d98e8-ecb2-40f4-a095-24ac562531dc admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:01.832863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d98e8-ecb2-40f4-a095-24ac562531dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:01.833146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2d98e8-ecb2-40f4-a095-24ac562531dc admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:01.833922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d98e8-ecb2-40f4-a095-24ac562531dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:34:01.834239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d98e8-ecb2-40f4-a095-24ac562531dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:01.835858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2d98e8-ecb2-40f4-a095-24ac562531dc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:01.853892 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3889/7742] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:34:01.856338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d98e8-ecb2-40f4-a095-24ac562531dc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:01.866406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2d98e8-ecb2-40f4-a095-24ac562531dc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:01.871551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930856ad-1f3e-422c-ad9e-fac763fbfc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:01.872252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930856ad-1f3e-422c-ad9e-fac763fbfc06 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:01.872540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930856ad-1f3e-422c-ad9e-fac763fbfc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:01.872815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930856ad-1f3e-422c-ad9e-fac763fbfc06 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:01.875794 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3854/7743] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:34:01 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:34:01.875942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930856ad-1f3e-422c-ad9e-fac763fbfc06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:01.875942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930856ad-1f3e-422c-ad9e-fac763fbfc06 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:01.876799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846f0a4-eeeb-4a5b-9480-babe020f47d9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:01.878413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2846f0a4-eeeb-4a5b-9480-babe020f47d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:01.891198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2846f0a4-eeeb-4a5b-9480-babe020f47d9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:01.895842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad5829-ddf6-413b-b09d-417b8fb1177c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:01.896498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad5829-ddf6-413b-b09d-417b8fb1177c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:01.896784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad5829-ddf6-413b-b09d-417b8fb1177c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:01.897063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad5829-ddf6-413b-b09d-417b8fb1177c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:01.897788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad5829-ddf6-413b-b09d-417b8fb1177c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:01.899429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad5829-ddf6-413b-b09d-417b8fb1177c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:01.909475 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3855/7744] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:01.919199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad5829-ddf6-413b-b09d-417b8fb1177c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:01.929122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad5829-ddf6-413b-b09d-417b8fb1177c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:01.934457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b335b5-3746-49c0-8884-b270c83e9fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:01.935115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b335b5-3746-49c0-8884-b270c83e9fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:01.935403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b335b5-3746-49c0-8884-b270c83e9fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:01.935685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b335b5-3746-49c0-8884-b270c83e9fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:01.936410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b335b5-3746-49c0-8884-b270c83e9fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:01.938045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b335b5-3746-49c0-8884-b270c83e9fc4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:01.949038 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3856/7745] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:34:01.953294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b335b5-3746-49c0-8884-b270c83e9fc4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:01.963279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b335b5-3746-49c0-8884-b270c83e9fc4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:01.968371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed447d-b54d-42be-9e12-ac3738941b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:01.969158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed447d-b54d-42be-9e12-ac3738941b78 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:01.969441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed447d-b54d-42be-9e12-ac3738941b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:01.969715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed447d-b54d-42be-9e12-ac3738941b78 admin admin] PATH_INFO: `/v3/projects/a528cc969a85428e80afad554aef0c01/users/b501d4b261ac4c3aa897b59fc7f424f7/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:01.986249 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3890/7746] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:34:01 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:34:01.992342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930856ad-1f3e-422c-ad9e-fac763fbfc06 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:01.992445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed447d-b54d-42be-9e12-ac3738941b78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a528cc969a85428e80afad554aef0c01, user_id=b501d4b261ac4c3aa897b59fc7f424f7, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:01.993181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed447d-b54d-42be-9e12-ac3738941b78 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:02.037419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:02.054530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930856ad-1f3e-422c-ad9e-fac763fbfc06 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:02.061967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1380d-b4a2-429b-84a9-e3321763edb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:02.064166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1380d-b4a2-429b-84a9-e3321763edb1 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:02.064166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1380d-b4a2-429b-84a9-e3321763edb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:02.064166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1380d-b4a2-429b-84a9-e3321763edb1 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:02.064166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c1380d-b4a2-429b-84a9-e3321763edb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:02.065936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c1380d-b4a2-429b-84a9-e3321763edb1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:02.089133 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3857/7747] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:34:01 2022] PUT /identity/v3/projects/a528cc969a85428e80afad554aef0c01/users/b501d4b261ac4c3aa897b59fc7f424f7/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:34:02.100353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cc848-8c22-473f-afdc-9230ee4884cb None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:02.100641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cc848-8c22-473f-afdc-9230ee4884cb None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:02.100918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cc848-8c22-473f-afdc-9230ee4884cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:02.120317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=7,keystone:SELECT=87,keystone:INSERT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:02.128716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ed6cc848-8c22-473f-afdc-9230ee4884cb None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:02.129979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ed6cc848-8c22-473f-afdc-9230ee4884cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:02.186757 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3891/7748] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:34:01 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:34:02.190749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1380d-b4a2-429b-84a9-e3321763edb1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:02.196262 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3858/7749] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:34:02.200968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1380d-b4a2-429b-84a9-e3321763edb1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:02.201139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6cc848-8c22-473f-afdc-9230ee4884cb None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:02.209877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6cc848-8c22-473f-afdc-9230ee4884cb None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:02.215675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ff4f2-4049-4975-9c6d-1883b77f6c32 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b501d4b261ac4c3aa897b59fc7f424f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a528cc969a85428e80afad554aef0c01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:02.216563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ff4f2-4049-4975-9c6d-1883b77f6c32 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:02.216845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ff4f2-4049-4975-9c6d-1883b77f6c32 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:02.217121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ff4f2-4049-4975-9c6d-1883b77f6c32 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] PATH_INFO: `/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:02.260779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627f269-6e62-4641-8e87-1efa071969e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:02.261432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627f269-6e62-4641-8e87-1efa071969e1 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:02.261765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627f269-6e62-4641-8e87-1efa071969e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:02.262079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627f269-6e62-4641-8e87-1efa071969e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:02.262836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627f269-6e62-4641-8e87-1efa071969e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:02.264477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627f269-6e62-4641-8e87-1efa071969e1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:02.282166 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3892/7750] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:02.286096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627f269-6e62-4641-8e87-1efa071969e1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:02.297361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627f269-6e62-4641-8e87-1efa071969e1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:02.302125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89e3f6-e216-4733-aa3c-de46dc8ec688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:02.302843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89e3f6-e216-4733-aa3c-de46dc8ec688 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:02.303133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89e3f6-e216-4733-aa3c-de46dc8ec688 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:02.303750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f89e3f6-e216-4733-aa3c-de46dc8ec688 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:02.304483 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89e3f6-e216-4733-aa3c-de46dc8ec688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:02.306176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f89e3f6-e216-4733-aa3c-de46dc8ec688 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:02.321169 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3893/7751] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:02.322999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e43ff4f2-4049-4975-9c6d-1883b77f6c32 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b501d4b261ac4c3aa897b59fc7f424f7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:02.325580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89e3f6-e216-4733-aa3c-de46dc8ec688 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:02.336848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f89e3f6-e216-4733-aa3c-de46dc8ec688 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:02.343963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1a871-70ea-438c-a709-d0f26b11a0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:02.344955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1a871-70ea-438c-a709-d0f26b11a0db admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:02.345239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1a871-70ea-438c-a709-d0f26b11a0db admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:02.345519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1a871-70ea-438c-a709-d0f26b11a0db admin admin] PATH_INFO: `/v3/projects/f9c9caba58394e1480acb632405111c5/users/b16d0e791edd4f119e646e7ced74bf74/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:02.362873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1a871-70ea-438c-a709-d0f26b11a0db admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9c9caba58394e1480acb632405111c5, user_id=b16d0e791edd4f119e646e7ced74bf74, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:02.366034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1a871-70ea-438c-a709-d0f26b11a0db admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:02.455324 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3859/7752] 10.222.0.33 () {60 vars in 1268 bytes} [Sat May 28 21:34:02 2022] POST /identity/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:34:02.472580 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3894/7753] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:34:02 2022] PUT /identity/v3/projects/f9c9caba58394e1480acb632405111c5/users/b16d0e791edd4f119e646e7ced74bf74/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:02.477639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b6f48-e120-4b68-9f00-9266259f3429 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:02.477976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b6f48-e120-4b68-9f00-9266259f3429 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:02.478260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b6f48-e120-4b68-9f00-9266259f3429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:02.497087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1b7b6f48-e120-4b68-9f00-9266259f3429 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:02.498354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1b7b6f48-e120-4b68-9f00-9266259f3429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:02.559873 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3860/7754] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:34:02.569377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1a871-70ea-438c-a709-d0f26b11a0db admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:02.571154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-68c1a871-70ea-438c-a709-d0f26b11a0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:02.631469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1a871-70ea-438c-a709-d0f26b11a0db admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:02.635296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015c900-9f98-49cb-ab0e-593028a25fcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:02.636043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015c900-9f98-49cb-ab0e-593028a25fcd service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:02.636359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015c900-9f98-49cb-ab0e-593028a25fcd service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:02.637303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015c900-9f98-49cb-ab0e-593028a25fcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:02.648139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015c900-9f98-49cb-ab0e-593028a25fcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:02.650476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015c900-9f98-49cb-ab0e-593028a25fcd service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:02.661056 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3895/7755] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:34:03.461344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0b163-1273-41e0-af60-c0585673458a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:03.461988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0b163-1273-41e0-af60-c0585673458a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:03.461988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0b163-1273-41e0-af60-c0585673458a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:03.480645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7fc0b163-1273-41e0-af60-c0585673458a None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:03.496376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7fc0b163-1273-41e0-af60-c0585673458a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:03.496376 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3861/7756] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:03 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 25 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 21:34:03.496663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02b1d2-6e08-41b9-8c52-6263e3453a2c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:03.496663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02b1d2-6e08-41b9-8c52-6263e3453a2c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:03.496663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02b1d2-6e08-41b9-8c52-6263e3453a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:03.508466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-eb02b1d2-6e08-41b9-8c52-6263e3453a2c None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:03.509696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eb02b1d2-6e08-41b9-8c52-6263e3453a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:03.547724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:03.572835 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3896/7757] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:03.577517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0b163-1273-41e0-af60-c0585673458a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:03.590838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0b163-1273-41e0-af60-c0585673458a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:03.594977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7154b85-4300-4150-abb7-4fae72e06904 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b501d4b261ac4c3aa897b59fc7f424f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a528cc969a85428e80afad554aef0c01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:03.595843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7154b85-4300-4150-abb7-4fae72e06904 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:03.596129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7154b85-4300-4150-abb7-4fae72e06904 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:03.596409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7154b85-4300-4150-abb7-4fae72e06904 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] PATH_INFO: `/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:03.645799 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-f7154b85-4300-4150-abb7-4fae72e06904 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. May 28 21:34:03.645799 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:34:03.645799 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:34:03.645799 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:34:03.645799 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:34:03.645799 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:34:03.645799 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:34:03.645799 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:34:03.645799 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:34:03.645799 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:34:03.645799 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:34:03.645799 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:34:03.645799 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper May 28 21:34:03.646940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(*args, **kwargs) May 28 21:34:03.646940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post May 28 21:34:03.646940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( May 28 21:34:03.646940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:34:03.646940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:34:03.646940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 21:34:03.646940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 21:34:03.646940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password May 28 21:34:03.646940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) May 28 21:34:03.646940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password May 28 21:34:03.646940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) May 28 21:34:03.646940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history May 28 21:34:03.646940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( May 28 21:34:03.646940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. May 28 21:34:03.646940 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:34:03.648329 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3862/7758] 10.222.0.33 () {60 vars in 1268 bytes} [Sat May 28 21:34:03 2022] POST /identity/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password => generated 188 bytes in 70 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) May 28 21:34:03.651304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02b1d2-6e08-41b9-8c52-6263e3453a2c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:03.660642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02b1d2-6e08-41b9-8c52-6263e3453a2c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:03.664968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cada450-454c-4695-8892-e30bb1e853ca tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b501d4b261ac4c3aa897b59fc7f424f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a528cc969a85428e80afad554aef0c01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:03.665968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cada450-454c-4695-8892-e30bb1e853ca tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:03.666332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cada450-454c-4695-8892-e30bb1e853ca tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:03.666684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cada450-454c-4695-8892-e30bb1e853ca tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] PATH_INFO: `/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:03.709963 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-8cada450-454c-4695-8892-e30bb1e853ca tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2.: keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. May 28 21:34:03.709963 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:34:03.709963 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:34:03.709963 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:34:03.709963 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:34:03.709963 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:34:03.709963 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:34:03.709963 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:34:03.709963 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:34:03.709963 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:34:03.709963 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:34:03.709963 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:34:03.709963 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper May 28 21:34:03.710962 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(*args, **kwargs) May 28 21:34:03.710962 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 291, in post May 28 21:34:03.710962 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application PROVIDERS.identity_api.change_password( May 28 21:34:03.710962 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:34:03.710962 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:34:03.710962 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 28 21:34:03.710962 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 28 21:34:03.710962 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1468, in change_password May 28 21:34:03.710962 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application driver.change_password(entity_id, new_password) May 28 21:34:03.710962 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 268, in change_password May 28 21:34:03.710962 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._validate_password_history(new_password, user_ref) May 28 21:34:03.710962 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 256, in _validate_password_history May 28 21:34:03.710962 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.PasswordHistoryValidationError( May 28 21:34:03.710962 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.PasswordHistoryValidationError: The new password cannot be identical to a previous password. The total number which includes the new password must be unique is 2. May 28 21:34:03.710962 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:34:03.712462 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3897/7759] 10.222.0.33 () {60 vars in 1268 bytes} [Sat May 28 21:34:03 2022] POST /identity/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password => generated 188 bytes in 60 msecs (HTTP/1.1 400) 5 headers in 186 bytes (1 switches on core 0) May 28 21:34:03.716258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7154b85-4300-4150-abb7-4fae72e06904 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:03.726364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7154b85-4300-4150-abb7-4fae72e06904 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:03.730687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b91c8-45de-4145-9da0-8c74fd3dc6da tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b501d4b261ac4c3aa897b59fc7f424f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a528cc969a85428e80afad554aef0c01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:03.731542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b91c8-45de-4145-9da0-8c74fd3dc6da tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:03.731826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b91c8-45de-4145-9da0-8c74fd3dc6da tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:03.732103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b91c8-45de-4145-9da0-8c74fd3dc6da tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] PATH_INFO: `/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:03.898316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b14b91c8-45de-4145-9da0-8c74fd3dc6da tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b501d4b261ac4c3aa897b59fc7f424f7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:03.988846 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3863/7760] 10.222.0.33 () {60 vars in 1268 bytes} [Sat May 28 21:34:03 2022] POST /identity/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:04.986924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403d038-b7a9-4c5e-b255-a92d35d6c35c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:04.987451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403d038-b7a9-4c5e-b255-a92d35d6c35c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:04.987544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2403d038-b7a9-4c5e-b255-a92d35d6c35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:05.023497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2403d038-b7a9-4c5e-b255-a92d35d6c35c None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:05.024755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2403d038-b7a9-4c5e-b255-a92d35d6c35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:05.039427 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3898/7761] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:04 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 21:34:05.041525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d1495-d479-4b4f-94a3-b69cbf2b52e6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:05.042446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d1495-d479-4b4f-94a3-b69cbf2b52e6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:05.043018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d1495-d479-4b4f-94a3-b69cbf2b52e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:05.067854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:UPDATE=2,keystone:INSERT=4,keystone:DELETE=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:05.086089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-228d1495-d479-4b4f-94a3-b69cbf2b52e6 None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:05.087562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-228d1495-d479-4b4f-94a3-b69cbf2b52e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:05.116544 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3864/7762] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:05.120091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403d038-b7a9-4c5e-b255-a92d35d6c35c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:05.153236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2403d038-b7a9-4c5e-b255-a92d35d6c35c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:05.158303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016799b5-761b-4fc8-8485-03d45f7a86a6 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b501d4b261ac4c3aa897b59fc7f424f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a528cc969a85428e80afad554aef0c01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:05.159581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016799b5-761b-4fc8-8485-03d45f7a86a6 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:05.160041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016799b5-761b-4fc8-8485-03d45f7a86a6 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:05.160485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016799b5-761b-4fc8-8485-03d45f7a86a6 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] PATH_INFO: `/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:05.358309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d1495-d479-4b4f-94a3-b69cbf2b52e6 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:05.360500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-228d1495-d479-4b4f-94a3-b69cbf2b52e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:05.379141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-016799b5-761b-4fc8-8485-03d45f7a86a6 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b501d4b261ac4c3aa897b59fc7f424f7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:05.382889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d1495-d479-4b4f-94a3-b69cbf2b52e6 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:05.403861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f4258-3569-4f1c-9620-83b72c857eb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:05.404536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f4258-3569-4f1c-9620-83b72c857eb4 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:05.404880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f4258-3569-4f1c-9620-83b72c857eb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:05.405086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f4258-3569-4f1c-9620-83b72c857eb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:05.407208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8f3f4258-3569-4f1c-9620-83b72c857eb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:05.443030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f4258-3569-4f1c-9620-83b72c857eb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:05.445348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3f4258-3569-4f1c-9620-83b72c857eb4 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:05.460192 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3865/7763] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:34:05.490769 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3899/7764] 10.222.0.33 () {60 vars in 1268 bytes} [Sat May 28 21:34:05 2022] POST /identity/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:06.505285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c9b34-80c3-4f31-a1c2-f1fad686a158 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:06.507024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c9b34-80c3-4f31-a1c2-f1fad686a158 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:06.507024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c9b34-80c3-4f31-a1c2-f1fad686a158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:06.544030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bd0c9b34-80c3-4f31-a1c2-f1fad686a158 None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:06.549601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bd0c9b34-80c3-4f31-a1c2-f1fad686a158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:06.551639 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3866/7765] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:06 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 21:34:06.556405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949db54-0b85-4301-ada4-f376f14681f0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:06.556872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949db54-0b85-4301-ada4-f376f14681f0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:06.556987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6949db54-0b85-4301-ada4-f376f14681f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:06.587945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6949db54-0b85-4301-ada4-f376f14681f0 None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:06.589915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6949db54-0b85-4301-ada4-f376f14681f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:06.616637 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3900/7766] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:34:06.622240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c9b34-80c3-4f31-a1c2-f1fad686a158 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:06.636097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c9b34-80c3-4f31-a1c2-f1fad686a158 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:06.646879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6f2e8-e921-4432-95fd-5d0cdec96011 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b501d4b261ac4c3aa897b59fc7f424f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a528cc969a85428e80afad554aef0c01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:06.649413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6f2e8-e921-4432-95fd-5d0cdec96011 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:06.651053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6f2e8-e921-4432-95fd-5d0cdec96011 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:06.653122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6f2e8-e921-4432-95fd-5d0cdec96011 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] PATH_INFO: `/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:06.868537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-86a6f2e8-e921-4432-95fd-5d0cdec96011 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b501d4b261ac4c3aa897b59fc7f424f7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:06.999148 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3867/7767] 10.222.0.33 () {60 vars in 1268 bytes} [Sat May 28 21:34:06 2022] POST /identity/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:08.004360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87126ef9-2801-405a-bf1f-97441b9bfa76 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:08.004878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87126ef9-2801-405a-bf1f-97441b9bfa76 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:08.004986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87126ef9-2801-405a-bf1f-97441b9bfa76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:08.028510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-87126ef9-2801-405a-bf1f-97441b9bfa76 None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:08.030145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-87126ef9-2801-405a-bf1f-97441b9bfa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:08.034388 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3901/7768] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:08 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 21:34:08.038824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f01a15-1ba2-4407-b1d9-948bb114003f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:08.039369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f01a15-1ba2-4407-b1d9-948bb114003f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:08.039369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f01a15-1ba2-4407-b1d9-948bb114003f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:08.077466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-24f01a15-1ba2-4407-b1d9-948bb114003f None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:08.084800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-24f01a15-1ba2-4407-b1d9-948bb114003f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:08.085675 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3868/7769] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:34:08.090699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87126ef9-2801-405a-bf1f-97441b9bfa76 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:08.101563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87126ef9-2801-405a-bf1f-97441b9bfa76 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:08.106153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36d996-d66c-4f97-9b50-3bba95d07292 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b501d4b261ac4c3aa897b59fc7f424f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a528cc969a85428e80afad554aef0c01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:08.107008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36d996-d66c-4f97-9b50-3bba95d07292 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:08.107302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36d996-d66c-4f97-9b50-3bba95d07292 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:08.107588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36d996-d66c-4f97-9b50-3bba95d07292 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] PATH_INFO: `/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:08.222701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7e36d996-d66c-4f97-9b50-3bba95d07292 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b501d4b261ac4c3aa897b59fc7f424f7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:08.301523 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3902/7770] 10.222.0.33 () {60 vars in 1268 bytes} [Sat May 28 21:34:08 2022] POST /identity/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:08.915944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba658fb1-792c-4a7f-bcbf-552ddd689f05 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:08.917025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba658fb1-792c-4a7f-bcbf-552ddd689f05 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:08.919248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba658fb1-792c-4a7f-bcbf-552ddd689f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:08.942123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ba658fb1-792c-4a7f-bcbf-552ddd689f05 None None] MFA Rules not processed for user `8d033e99f9294e348b01c59a970931d5`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:08.942123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ba658fb1-792c-4a7f-bcbf-552ddd689f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:09.012959 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3869/7771] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:09.308088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189ea0c-21ae-41ea-9c5b-6feb4d02e425 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:09.308650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189ea0c-21ae-41ea-9c5b-6feb4d02e425 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:09.308734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189ea0c-21ae-41ea-9c5b-6feb4d02e425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:09.336821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7189ea0c-21ae-41ea-9c5b-6feb4d02e425 None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:09.338672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7189ea0c-21ae-41ea-9c5b-6feb4d02e425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:09.345343 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3903/7772] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:09 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 21:34:09.384472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2b457-db84-4d6f-a43d-4850b4c7c55b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:09.384945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2b457-db84-4d6f-a43d-4850b4c7c55b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:09.385039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2b457-db84-4d6f-a43d-4850b4c7c55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:09.425266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8cb2b457-db84-4d6f-a43d-4850b4c7c55b None None] MFA Rules not processed for user `b16d0e791edd4f119e646e7ced74bf74`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:09.427279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8cb2b457-db84-4d6f-a43d-4850b4c7c55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:09.506038 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3870/7773] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:34:09.520192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189ea0c-21ae-41ea-9c5b-6feb4d02e425 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:09.521859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7189ea0c-21ae-41ea-9c5b-6feb4d02e425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:09.610703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189ea0c-21ae-41ea-9c5b-6feb4d02e425 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:09.616543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64d1071-2bbc-4878-81c6-fa9cd9f21718 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:09.617366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d1071-2bbc-4878-81c6-fa9cd9f21718 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:09.617491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d1071-2bbc-4878-81c6-fa9cd9f21718 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:09.617733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d1071-2bbc-4878-81c6-fa9cd9f21718 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:09.627386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d1071-2bbc-4878-81c6-fa9cd9f21718 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:09.629010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64d1071-2bbc-4878-81c6-fa9cd9f21718 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:09.640731 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3904/7774] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:34:10.351033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff255a1-828e-47be-8209-36f1b654ae0a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:10.351383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff255a1-828e-47be-8209-36f1b654ae0a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:10.351700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff255a1-828e-47be-8209-36f1b654ae0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:10.374592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-aff255a1-828e-47be-8209-36f1b654ae0a None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:10.376425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-aff255a1-828e-47be-8209-36f1b654ae0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:10.383685 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3871/7775] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:10.390912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca8452-a614-4e1d-b6c2-b627f76ed9b0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:10.391399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca8452-a614-4e1d-b6c2-b627f76ed9b0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:10.391532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca8452-a614-4e1d-b6c2-b627f76ed9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:10.403717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:UPDATE=2,keystone:INSERT=4,keystone:DELETE=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:10.411045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-95ca8452-a614-4e1d-b6c2-b627f76ed9b0 None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:10.413221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-95ca8452-a614-4e1d-b6c2-b627f76ed9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:10.422456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6a273-5a92-41eb-9d5e-65c455532ad4 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:10.423000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6a273-5a92-41eb-9d5e-65c455532ad4 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:10.423093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6a273-5a92-41eb-9d5e-65c455532ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:10.425089 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3905/7776] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:10 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 28 21:34:10.573170 ncdv-master-1 devstack@keystone.service[103104]: WARNING keystone.server.flask.application [None req-59b6a273-5a92-41eb-9d5e-65c455532ad4 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.33: keystone.exception.Unauthorized: The request you have made requires authentication. May 28 21:34:10.575214 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3872/7777] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:10 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 154 msecs (HTTP/1.1 401) 6 headers in 252 bytes (1 switches on core 0) May 28 21:34:10.579564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2598b86-c312-43de-8574-373648c322e1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:10.579975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2598b86-c312-43de-8574-373648c322e1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:10.580262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2598b86-c312-43de-8574-373648c322e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:10.709807 ncdv-master-1 devstack@keystone.service[103103]: WARNING keystone.server.flask.application [None req-e2598b86-c312-43de-8574-373648c322e1 None None] Authorization failed. The request you have made requires authentication. from 10.222.0.33: keystone.exception.Unauthorized: The request you have made requires authentication. May 28 21:34:10.711417 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3906/7778] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:10 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 133 msecs (HTTP/1.1 401) 6 headers in 252 bytes (2 switches on core 0) May 28 21:34:10.716670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbb245-7442-40fa-9dc2-b9ac6435df7b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:10.716956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbb245-7442-40fa-9dc2-b9ac6435df7b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:10.717232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbb245-7442-40fa-9dc2-b9ac6435df7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:10.731738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:UPDATE=2,keystone:INSERT=2,keystone:DELETE=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:10.741639 ncdv-master-1 devstack@keystone.service[103104]: WARNING keystone.server.flask.application [None req-a9dbb245-7442-40fa-9dc2-b9ac6435df7b None None] Authorization failed. The request you have made requires authentication. from 10.222.0.33: keystone.exception.Unauthorized: The request you have made requires authentication. May 28 21:34:10.743286 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3873/7779] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:10 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 28 msecs (HTTP/1.1 401) 6 headers in 252 bytes (2 switches on core 0) May 28 21:34:12.585265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f242146-1164-4cbc-ab15-5468537a36f8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:12.585265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f242146-1164-4cbc-ab15-5468537a36f8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:12.585265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f242146-1164-4cbc-ab15-5468537a36f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:12.604921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1f242146-1164-4cbc-ab15-5468537a36f8 None None] MFA Rules not processed for user `b16d0e791edd4f119e646e7ced74bf74`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:12.606237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1f242146-1164-4cbc-ab15-5468537a36f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:12.615070 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3907/7780] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:13.487463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dbb245-7442-40fa-9dc2-b9ac6435df7b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:13.512120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dbb245-7442-40fa-9dc2-b9ac6435df7b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:13.538255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9fb17-2022-495b-bee6-95cb93ffc83c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:13.539099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9fb17-2022-495b-bee6-95cb93ffc83c service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:13.539527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9fb17-2022-495b-bee6-95cb93ffc83c service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:13.539913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9fb17-2022-495b-bee6-95cb93ffc83c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:13.544873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-08c9fb17-2022-495b-bee6-95cb93ffc83c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:13.577659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9fb17-2022-495b-bee6-95cb93ffc83c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:13.579488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9fb17-2022-495b-bee6-95cb93ffc83c service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:13.612977 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3874/7781] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:34:14.685302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f242146-1164-4cbc-ab15-5468537a36f8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:14.698038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f242146-1164-4cbc-ab15-5468537a36f8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:14.702458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fae02c-7af0-4c51-b9b0-9e485ab5f62a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:14.703159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fae02c-7af0-4c51-b9b0-9e485ab5f62a service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:14.703614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fae02c-7af0-4c51-b9b0-9e485ab5f62a service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:14.709789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fae02c-7af0-4c51-b9b0-9e485ab5f62a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:14.709789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-85fae02c-7af0-4c51-b9b0-9e485ab5f62a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:14.719496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fae02c-7af0-4c51-b9b0-9e485ab5f62a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:14.720955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fae02c-7af0-4c51-b9b0-9e485ab5f62a service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:14.732666 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3908/7782] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:34:16.077461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9fb17-2022-495b-bee6-95cb93ffc83c service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:16.094736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9fb17-2022-495b-bee6-95cb93ffc83c service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:16.101139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d958d-a761-471d-a9b0-384feff4034a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:16.102035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d958d-a761-471d-a9b0-384feff4034a service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:16.102328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d958d-a761-471d-a9b0-384feff4034a service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:16.102636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d958d-a761-471d-a9b0-384feff4034a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:16.104565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ad1d958d-a761-471d-a9b0-384feff4034a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:16.130561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d958d-a761-471d-a9b0-384feff4034a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:16.132205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d958d-a761-471d-a9b0-384feff4034a service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:16.146845 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3875/7783] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:34:17.650184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fae02c-7af0-4c51-b9b0-9e485ab5f62a service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:17.662086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fae02c-7af0-4c51-b9b0-9e485ab5f62a service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:17.666032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b40f25-cae3-4105-ac25-344bd2b381cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:17.670885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b40f25-cae3-4105-ac25-344bd2b381cc service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:17.670885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b40f25-cae3-4105-ac25-344bd2b381cc service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:17.670885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b40f25-cae3-4105-ac25-344bd2b381cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:17.670885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b5b40f25-cae3-4105-ac25-344bd2b381cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:17.732868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b40f25-cae3-4105-ac25-344bd2b381cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:17.734360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b40f25-cae3-4105-ac25-344bd2b381cc service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:17.744129 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3909/7784] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:34:18.624647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d958d-a761-471d-a9b0-384feff4034a service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:18.626940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ad1d958d-a761-471d-a9b0-384feff4034a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:18.651353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d958d-a761-471d-a9b0-384feff4034a service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:18.661283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63722c9e-dcf1-404f-9c10-0160592c556e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:18.662270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63722c9e-dcf1-404f-9c10-0160592c556e service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:18.663628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63722c9e-dcf1-404f-9c10-0160592c556e service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:18.664129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63722c9e-dcf1-404f-9c10-0160592c556e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:18.667572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-63722c9e-dcf1-404f-9c10-0160592c556e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:18.692231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63722c9e-dcf1-404f-9c10-0160592c556e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:18.694169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63722c9e-dcf1-404f-9c10-0160592c556e service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:18.717134 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3876/7785] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:34:21.758859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70055b3c-720f-426b-a9d4-ee8becc46afa None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:21.759341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70055b3c-720f-426b-a9d4-ee8becc46afa None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:21.759545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70055b3c-720f-426b-a9d4-ee8becc46afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:21.932485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-70055b3c-720f-426b-a9d4-ee8becc46afa None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:21.933786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-70055b3c-720f-426b-a9d4-ee8becc46afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:21.937031 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3910/7786] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:21 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 21:34:21.942403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63722c9e-dcf1-404f-9c10-0160592c556e service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:21.955435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63722c9e-dcf1-404f-9c10-0160592c556e service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:21.960324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507dc24-ac63-4f8d-816e-75fdb6519025 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b501d4b261ac4c3aa897b59fc7f424f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a528cc969a85428e80afad554aef0c01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:21.961475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507dc24-ac63-4f8d-816e-75fdb6519025 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:21.961934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507dc24-ac63-4f8d-816e-75fdb6519025 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:21.962443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507dc24-ac63-4f8d-816e-75fdb6519025 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] PATH_INFO: `/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:22.116745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1507dc24-ac63-4f8d-816e-75fdb6519025 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b501d4b261ac4c3aa897b59fc7f424f7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:22.223854 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3877/7787] 10.222.0.33 () {60 vars in 1268 bytes} [Sat May 28 21:34:21 2022] POST /identity/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:23.227639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb456ca-b6fb-487d-9072-9c63a2c931ce None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:23.228114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb456ca-b6fb-487d-9072-9c63a2c931ce None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:23.228210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb456ca-b6fb-487d-9072-9c63a2c931ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:23.252472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8fb456ca-b6fb-487d-9072-9c63a2c931ce None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:23.253821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8fb456ca-b6fb-487d-9072-9c63a2c931ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:23.256920 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3911/7788] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:23 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 21:34:23.261416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaa1bf-bd6f-440f-83e9-082dc6520023 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:23.261897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaa1bf-bd6f-440f-83e9-082dc6520023 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:23.262150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcaa1bf-bd6f-440f-83e9-082dc6520023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:23.325825 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-ddcaa1bf-bd6f-440f-83e9-082dc6520023 None None] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token May 28 21:34:23.325825 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:34:23.325825 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:34:23.325825 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:34:23.325825 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:34:23.325825 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:34:23.325825 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:34:23.325825 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:34:23.325825 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:34:23.325825 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:34:23.325825 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:34:23.325825 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:34:23.325825 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/server/flask/common.py", line 1064, in wrapper May 28 21:34:23.325825 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return f(*args, **kwargs) May 28 21:34:23.325825 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 315, in post May 28 21:34:23.325825 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application token = authentication.authenticate_for_token(auth_data) May 28 21:34:23.326875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 185, in authenticate_for_token May 28 21:34:23.326875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application authenticate(auth_info, auth_context) May 28 21:34:23.326875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/_shared/authentication.py", line 152, in authenticate May 28 21:34:23.326875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = method.authenticate(auth_info.get_method_data(method_name)) May 28 21:34:23.326875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 42, in authenticate May 28 21:34:23.326875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application token = self._get_token_ref(auth_payload) May 28 21:34:23.326875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/auth/plugins/token.py", line 36, in _get_token_ref May 28 21:34:23.326875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return PROVIDERS.token_provider_api.validate_token(token_id) May 28 21:34:23.326875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:34:23.326875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:34:23.326875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token May 28 21:34:23.326875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) May 28 21:34:23.326875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token May 28 21:34:23.326875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.check_revocation(token) May 28 21:34:23.326875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:34:23.326875 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:34:23.327936 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation May 28 21:34:23.327936 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) May 28 21:34:23.327936 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:34:23.327936 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:34:23.327936 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 May 28 21:34:23.327936 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) May 28 21:34:23.327936 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:34:23.327936 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:34:23.327936 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token May 28 21:34:23.327936 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) May 28 21:34:23.327936 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token May 28 21:34:23.327936 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:34:23.330945 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3878/7789] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:23 2022] POST /identity/v3/auth/tokens => generated 80 bytes in 71 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 21:34:23.331873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e9a5a-c28a-49b7-ae54-d01d0a36616e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:23.332165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e9a5a-c28a-49b7-ae54-d01d0a36616e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:23.332449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e9a5a-c28a-49b7-ae54-d01d0a36616e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:23.441968 ncdv-master-1 devstack@keystone.service[103103]: WARNING keystone.server.flask.application [None req-416e9a5a-c28a-49b7-ae54-d01d0a36616e None None] Authorization failed. The request you have made requires authentication. from 10.222.0.33: keystone.exception.Unauthorized: The request you have made requires authentication. May 28 21:34:23.444136 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3912/7790] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:23 2022] POST /identity/v3/auth/tokens => generated 109 bytes in 113 msecs (HTTP/1.1 401) 6 headers in 252 bytes (2 switches on core 0) May 28 21:34:23.449210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934b6b5-006e-49d6-9064-b91ecbd4f826 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:23.449605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934b6b5-006e-49d6-9064-b91ecbd4f826 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:23.450047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9934b6b5-006e-49d6-9064-b91ecbd4f826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:23.570437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9934b6b5-006e-49d6-9064-b91ecbd4f826 None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:23.572433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9934b6b5-006e-49d6-9064-b91ecbd4f826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:23.581885 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3879/7791] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:23.584904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e9a5a-c28a-49b7-ae54-d01d0a36616e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:23.598131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e9a5a-c28a-49b7-ae54-d01d0a36616e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:23.603859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5450664-aca8-47c0-b0e5-1671c5928a83 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b501d4b261ac4c3aa897b59fc7f424f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a528cc969a85428e80afad554aef0c01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:23.604967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5450664-aca8-47c0-b0e5-1671c5928a83 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:23.605926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5450664-aca8-47c0-b0e5-1671c5928a83 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:23.607859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5450664-aca8-47c0-b0e5-1671c5928a83 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] PATH_INFO: `/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:23.727056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a5450664-aca8-47c0-b0e5-1671c5928a83 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b501d4b261ac4c3aa897b59fc7f424f7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:23.816671 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3913/7792] 10.222.0.33 () {60 vars in 1268 bytes} [Sat May 28 21:34:23 2022] POST /identity/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:24.822043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497e508-8996-4d31-9302-00121547b1c3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:24.822043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497e508-8996-4d31-9302-00121547b1c3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:24.822043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497e508-8996-4d31-9302-00121547b1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:24.839215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c497e508-8996-4d31-9302-00121547b1c3 None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:24.841083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c497e508-8996-4d31-9302-00121547b1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:24.844238 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3880/7793] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:24 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 25 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 21:34:24.854087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f04da-6b78-4988-bbd3-32b7321f8fa5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:24.854536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f04da-6b78-4988-bbd3-32b7321f8fa5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:24.854647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f04da-6b78-4988-bbd3-32b7321f8fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:24.873420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-433f04da-6b78-4988-bbd3-32b7321f8fa5 None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:24.874654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-433f04da-6b78-4988-bbd3-32b7321f8fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:24.881493 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3914/7794] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:34:24.885776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497e508-8996-4d31-9302-00121547b1c3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:24.897172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497e508-8996-4d31-9302-00121547b1c3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:24.901209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7568c-9174-4120-935b-72ac75050863 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b501d4b261ac4c3aa897b59fc7f424f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a528cc969a85428e80afad554aef0c01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:24.902622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7568c-9174-4120-935b-72ac75050863 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:24.902908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7568c-9174-4120-935b-72ac75050863 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:24.903183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7568c-9174-4120-935b-72ac75050863 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] PATH_INFO: `/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:25.059413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-44b7568c-9174-4120-935b-72ac75050863 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b501d4b261ac4c3aa897b59fc7f424f7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:25.170163 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3881/7795] 10.222.0.33 () {60 vars in 1268 bytes} [Sat May 28 21:34:24 2022] POST /identity/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:34:26.175922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965fe2f-a0d1-4478-b43a-b94886b9cf18 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:26.176416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965fe2f-a0d1-4478-b43a-b94886b9cf18 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:26.176546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965fe2f-a0d1-4478-b43a-b94886b9cf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:26.194985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c965fe2f-a0d1-4478-b43a-b94886b9cf18 None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:26.196186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c965fe2f-a0d1-4478-b43a-b94886b9cf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:26.199349 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3915/7796] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:26 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 24 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 21:34:26.204032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ce29-2ef6-4605-a27f-0086129895de None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:26.204511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ce29-2ef6-4605-a27f-0086129895de None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:26.204623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ce29-2ef6-4605-a27f-0086129895de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:26.223230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3564ce29-2ef6-4605-a27f-0086129895de None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:26.224434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3564ce29-2ef6-4605-a27f-0086129895de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:26.232091 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3882/7797] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:34:26.236265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965fe2f-a0d1-4478-b43a-b94886b9cf18 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:26.249009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965fe2f-a0d1-4478-b43a-b94886b9cf18 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:26.253207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bb1cc-43d3-4ab0-8139-1a831785e802 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b501d4b261ac4c3aa897b59fc7f424f7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a528cc969a85428e80afad554aef0c01', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:26.254110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bb1cc-43d3-4ab0-8139-1a831785e802 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:26.254427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bb1cc-43d3-4ab0-8139-1a831785e802 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:26.254686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bb1cc-43d3-4ab0-8139-1a831785e802 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] PATH_INFO: `/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:26.276270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:UPDATE=4,keystone:INSERT=2,keystone:DELETE=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:26.345223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e93bb1cc-43d3-4ab0-8139-1a831785e802 tempest-IdentityV3UsersTest-1369702259 tempest-IdentityV3UsersTest-1369702259-project] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': 'b501d4b261ac4c3aa897b59fc7f424f7'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:26.441019 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3916/7798] 10.222.0.33 () {60 vars in 1268 bytes} [Sat May 28 21:34:26 2022] POST /identity/v3/users/b501d4b261ac4c3aa897b59fc7f424f7/password => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:27.448748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6eec2-47ed-4d7e-84f0-6fac80d7806c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:27.449187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6eec2-47ed-4d7e-84f0-6fac80d7806c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:27.449350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a6eec2-47ed-4d7e-84f0-6fac80d7806c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:27.468088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-97a6eec2-47ed-4d7e-84f0-6fac80d7806c None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:27.469284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-97a6eec2-47ed-4d7e-84f0-6fac80d7806c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:27.472608 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3883/7799] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:27 2022] POST /identity/v3/auth/tokens => generated 353 bytes in 25 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 21:34:28.478507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f514b8-0766-4bb3-820f-9860a2840855 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:28.479011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f514b8-0766-4bb3-820f-9860a2840855 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:28.479095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f514b8-0766-4bb3-820f-9860a2840855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:28.504696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d4f514b8-0766-4bb3-820f-9860a2840855 None None] MFA Rules not processed for user `b501d4b261ac4c3aa897b59fc7f424f7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:28.504696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d4f514b8-0766-4bb3-820f-9860a2840855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:28.513157 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3917/7800] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:28.518503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6eec2-47ed-4d7e-84f0-6fac80d7806c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:28.536489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a6eec2-47ed-4d7e-84f0-6fac80d7806c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:28.545794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340046f4-83ab-41a4-84ec-91b18f049e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:28.548231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340046f4-83ab-41a4-84ec-91b18f049e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:28.548705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340046f4-83ab-41a4-84ec-91b18f049e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:28.548991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340046f4-83ab-41a4-84ec-91b18f049e76 admin admin] PATH_INFO: `/v3/users/b501d4b261ac4c3aa897b59fc7f424f7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:28.550659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340046f4-83ab-41a4-84ec-91b18f049e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b501d4b261ac4c3aa897b59fc7f424f7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:28.552802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340046f4-83ab-41a4-84ec-91b18f049e76 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:28.825129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-340046f4-83ab-41a4-84ec-91b18f049e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b501d4b261ac4c3aa897b59fc7f424f7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:28.826134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-340046f4-83ab-41a4-84ec-91b18f049e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b501d4b261ac4c3aa897b59fc7f424f7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:28.834134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-340046f4-83ab-41a4-84ec-91b18f049e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b501d4b261ac4c3aa897b59fc7f424f7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:28.839114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=80,keystone:UPDATE=3,keystone:INSERT=5,keystone:DELETE=12 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:28.900709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-340046f4-83ab-41a4-84ec-91b18f049e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b501d4b261ac4c3aa897b59fc7f424f7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:28.901462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-340046f4-83ab-41a4-84ec-91b18f049e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b501d4b261ac4c3aa897b59fc7f424f7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:28.922283 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3884/7801] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:34:28 2022] DELETE /identity/v3/users/b501d4b261ac4c3aa897b59fc7f424f7 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:28.927263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5b70d-4377-4e74-aa04-1366c86fe5bc None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:28.927491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5b70d-4377-4e74-aa04-1366c86fe5bc None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:28.927786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5b70d-4377-4e74-aa04-1366c86fe5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:28.946595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-30c5b70d-4377-4e74-aa04-1366c86fe5bc None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:28.947852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-30c5b70d-4377-4e74-aa04-1366c86fe5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:29.017220 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3918/7802] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:29.030351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340046f4-83ab-41a4-84ec-91b18f049e76 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:29.032952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-340046f4-83ab-41a4-84ec-91b18f049e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:29.114744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340046f4-83ab-41a4-84ec-91b18f049e76 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:29.119245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d4fe4-a0e0-4cae-89b9-5e9508899b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:29.120677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d4fe4-a0e0-4cae-89b9-5e9508899b98 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:29.121316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d4fe4-a0e0-4cae-89b9-5e9508899b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:29.123463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d4fe4-a0e0-4cae-89b9-5e9508899b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:29.138376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d4fe4-a0e0-4cae-89b9-5e9508899b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:29.140205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d4fe4-a0e0-4cae-89b9-5e9508899b98 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:29.153731 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3885/7803] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:34:29.892664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5b70d-4377-4e74-aa04-1366c86fe5bc None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:29.892664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-30c5b70d-4377-4e74-aa04-1366c86fe5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:29.905597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5b70d-4377-4e74-aa04-1366c86fe5bc None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:29.921628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ebfe7-d668-498e-9f07-bfb3c48d16e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:29.921628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ebfe7-d668-498e-9f07-bfb3c48d16e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:29.921628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ebfe7-d668-498e-9f07-bfb3c48d16e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:29.921628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ebfe7-d668-498e-9f07-bfb3c48d16e3 admin admin] PATH_INFO: `/v3/projects/a528cc969a85428e80afad554aef0c01` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:29.928004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ebfe7-d668-498e-9f07-bfb3c48d16e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a528cc969a85428e80afad554aef0c01)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:29.935495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ebfe7-d668-498e-9f07-bfb3c48d16e3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:30.095893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-348ebfe7-d668-498e-9f07-bfb3c48d16e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a528cc969a85428e80afad554aef0c01', 'request_id': 'req-348ebfe7-d668-498e-9f07-bfb3c48d16e3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:30.102988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-348ebfe7-d668-498e-9f07-bfb3c48d16e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a528cc969a85428e80afad554aef0c01', 'request_id': 'req-348ebfe7-d668-498e-9f07-bfb3c48d16e3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:30.177999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-348ebfe7-d668-498e-9f07-bfb3c48d16e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a528cc969a85428e80afad554aef0c01', 'request_id': 'req-348ebfe7-d668-498e-9f07-bfb3c48d16e3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:30.201014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-348ebfe7-d668-498e-9f07-bfb3c48d16e3 admin admin] The token cache is being invalidate because project a528cc969a85428e80afad554aef0c01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:34:30.201563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-348ebfe7-d668-498e-9f07-bfb3c48d16e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:30.203778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-348ebfe7-d668-498e-9f07-bfb3c48d16e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:30.206749 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3919/7804] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:34:29 2022] DELETE /identity/v3/projects/a528cc969a85428e80afad554aef0c01 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:30.222829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c97db-6e81-4999-b5ce-12fec582866b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:30.223953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c97db-6e81-4999-b5ce-12fec582866b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:30.224334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c97db-6e81-4999-b5ce-12fec582866b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:30.245622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-929c97db-6e81-4999-b5ce-12fec582866b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:30.247371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-929c97db-6e81-4999-b5ce-12fec582866b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:30.339413 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3886/7805] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:30.348080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621c080-83e2-4a1f-a79f-720bcb42d9d1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:30.348436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621c080-83e2-4a1f-a79f-720bcb42d9d1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:30.348782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621c080-83e2-4a1f-a79f-720bcb42d9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:30.369142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2621c080-83e2-4a1f-a79f-720bcb42d9d1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:30.370399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2621c080-83e2-4a1f-a79f-720bcb42d9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:30.378862 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3920/7806] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:30.384664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c97db-6e81-4999-b5ce-12fec582866b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:30.393411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c97db-6e81-4999-b5ce-12fec582866b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:30.398096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4c07d-8f09-4b28-bb8c-8a47a068a505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:30.398751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4c07d-8f09-4b28-bb8c-8a47a068a505 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:30.399039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4c07d-8f09-4b28-bb8c-8a47a068a505 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:30.399320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b4c07d-8f09-4b28-bb8c-8a47a068a505 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:30.400043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4c07d-8f09-4b28-bb8c-8a47a068a505 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:34:30.400352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4c07d-8f09-4b28-bb8c-8a47a068a505 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:30.402026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b4c07d-8f09-4b28-bb8c-8a47a068a505 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:30.413937 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3887/7807] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:34:30.418273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621c080-83e2-4a1f-a79f-720bcb42d9d1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:30.427447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621c080-83e2-4a1f-a79f-720bcb42d9d1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:30.432070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f4cab-870b-45d0-8bb9-a8f314e36175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:30.432704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f4cab-870b-45d0-8bb9-a8f314e36175 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:30.432988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f4cab-870b-45d0-8bb9-a8f314e36175 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:30.433267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021f4cab-870b-45d0-8bb9-a8f314e36175 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:30.434022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f4cab-870b-45d0-8bb9-a8f314e36175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:30.435905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021f4cab-870b-45d0-8bb9-a8f314e36175 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:30.519414 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3921/7808] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:34:30 2022] POST /identity/v3/projects => generated 397 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:34:30.523640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4c07d-8f09-4b28-bb8c-8a47a068a505 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:30.534679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b4c07d-8f09-4b28-bb8c-8a47a068a505 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:30.595150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3f18b-d1c6-4d6d-a3c7-1e0da572fd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:30.595150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3f18b-d1c6-4d6d-a3c7-1e0da572fd5c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:30.595150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3f18b-d1c6-4d6d-a3c7-1e0da572fd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:30.596781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3f18b-d1c6-4d6d-a3c7-1e0da572fd5c admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:30.596781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3f18b-d1c6-4d6d-a3c7-1e0da572fd5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:30.597869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3f18b-d1c6-4d6d-a3c7-1e0da572fd5c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:30.660103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:30.671439 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3888/7809] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:34:30 2022] POST /identity/v3/users => generated 305 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:34:30.672611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f4cab-870b-45d0-8bb9-a8f314e36175 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:30.684254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021f4cab-870b-45d0-8bb9-a8f314e36175 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:30.690988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64afd839-aeee-4bd0-9eac-c4246275de58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:30.691839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afd839-aeee-4bd0-9eac-c4246275de58 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:30.692197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afd839-aeee-4bd0-9eac-c4246275de58 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:30.692554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afd839-aeee-4bd0-9eac-c4246275de58 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:30.693361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64afd839-aeee-4bd0-9eac-c4246275de58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:30.695472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64afd839-aeee-4bd0-9eac-c4246275de58 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:30.710001 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3922/7810] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:30.713966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3f18b-d1c6-4d6d-a3c7-1e0da572fd5c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:30.724263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3f18b-d1c6-4d6d-a3c7-1e0da572fd5c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:30.730583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79707135-2b2e-4cd8-9983-2f5878e73650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:30.732079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79707135-2b2e-4cd8-9983-2f5878e73650 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:30.732552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79707135-2b2e-4cd8-9983-2f5878e73650 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:30.732788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79707135-2b2e-4cd8-9983-2f5878e73650 admin admin] PATH_INFO: `/v3/projects/d950fd46f5754ce4ade5113020932fb6/users/c73304be4db448c1b5e3f3673e6f3512/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:30.751197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79707135-2b2e-4cd8-9983-2f5878e73650 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d950fd46f5754ce4ade5113020932fb6, user_id=c73304be4db448c1b5e3f3673e6f3512, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:30.754672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79707135-2b2e-4cd8-9983-2f5878e73650 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:30.815095 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3889/7811] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:34:30 2022] PUT /identity/v3/projects/d950fd46f5754ce4ade5113020932fb6/users/c73304be4db448c1b5e3f3673e6f3512/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:34:30.819186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64afd839-aeee-4bd0-9eac-c4246275de58 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:30.829633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64afd839-aeee-4bd0-9eac-c4246275de58 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:30.849445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:UPDATE=1,keystone:INSERT=4,keystone:DELETE=9,keystone:SELECT=86 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:30.890062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107506a2-d717-4794-ba46-ba222690c3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:30.890768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107506a2-d717-4794-ba46-ba222690c3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:30.891053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107506a2-d717-4794-ba46-ba222690c3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:30.891332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107506a2-d717-4794-ba46-ba222690c3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:30.892070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107506a2-d717-4794-ba46-ba222690c3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:30.893668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107506a2-d717-4794-ba46-ba222690c3f8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:30.904308 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3923/7812] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:30.908968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79707135-2b2e-4cd8-9983-2f5878e73650 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:30.920834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79707135-2b2e-4cd8-9983-2f5878e73650 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:30.925498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357b95f-ca7f-4cf7-9cb5-a10894925a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:30.926531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357b95f-ca7f-4cf7-9cb5-a10894925a15 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:30.926697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357b95f-ca7f-4cf7-9cb5-a10894925a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:30.926973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357b95f-ca7f-4cf7-9cb5-a10894925a15 admin admin] PATH_INFO: `/v3/projects/d950fd46f5754ce4ade5113020932fb6/users/c73304be4db448c1b5e3f3673e6f3512/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:30.928997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357b95f-ca7f-4cf7-9cb5-a10894925a15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d950fd46f5754ce4ade5113020932fb6, user_id=c73304be4db448c1b5e3f3673e6f3512, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:30.931716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357b95f-ca7f-4cf7-9cb5-a10894925a15 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:30.943066 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3890/7813] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:34:30 2022] PUT /identity/v3/projects/d950fd46f5754ce4ade5113020932fb6/users/c73304be4db448c1b5e3f3673e6f3512/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:30.954745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1c6ac-7591-49fd-987c-bb51fd8f7bd1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:30.955038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1c6ac-7591-49fd-987c-bb51fd8f7bd1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:30.955392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1c6ac-7591-49fd-987c-bb51fd8f7bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:30.983080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9ed1c6ac-7591-49fd-987c-bb51fd8f7bd1 None None] MFA Rules not processed for user `c73304be4db448c1b5e3f3673e6f3512`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:30.984297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9ed1c6ac-7591-49fd-987c-bb51fd8f7bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:31.058461 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3924/7814] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:30 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:31.063501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357b95f-ca7f-4cf7-9cb5-a10894925a15 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:31.127101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357b95f-ca7f-4cf7-9cb5-a10894925a15 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:31.131924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8668a-c51d-43fa-b1e8-80eb3bec825c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:31.132585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8668a-c51d-43fa-b1e8-80eb3bec825c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:31.132874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8668a-c51d-43fa-b1e8-80eb3bec825c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:31.133154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8668a-c51d-43fa-b1e8-80eb3bec825c admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:31.133921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8668a-c51d-43fa-b1e8-80eb3bec825c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:31.135845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce8668a-c51d-43fa-b1e8-80eb3bec825c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:31.249912 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3891/7815] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:34:31 2022] POST /identity/v3/projects => generated 397 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:34:31.261620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1c6ac-7591-49fd-987c-bb51fd8f7bd1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:31.272604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1c6ac-7591-49fd-987c-bb51fd8f7bd1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:31.329308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3232c-5519-4faf-89f1-4148a86710d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:31.330045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3232c-5519-4faf-89f1-4148a86710d6 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:31.330337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3232c-5519-4faf-89f1-4148a86710d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:31.330620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3232c-5519-4faf-89f1-4148a86710d6 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:31.331354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3232c-5519-4faf-89f1-4148a86710d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:31.333298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3232c-5519-4faf-89f1-4148a86710d6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:31.424848 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3925/7816] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:34:31 2022] POST /identity/v3/users => generated 299 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:34:31.428991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8668a-c51d-43fa-b1e8-80eb3bec825c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:31.441874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8668a-c51d-43fa-b1e8-80eb3bec825c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:31.448123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39648ae6-e6c6-4a8f-94d3-5a427d08b0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:31.451895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39648ae6-e6c6-4a8f-94d3-5a427d08b0f7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:31.451895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39648ae6-e6c6-4a8f-94d3-5a427d08b0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:31.451895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39648ae6-e6c6-4a8f-94d3-5a427d08b0f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:31.451895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39648ae6-e6c6-4a8f-94d3-5a427d08b0f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:31.452474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39648ae6-e6c6-4a8f-94d3-5a427d08b0f7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:31.468404 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3892/7817] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:34:31.469686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3232c-5519-4faf-89f1-4148a86710d6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:31.479972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3232c-5519-4faf-89f1-4148a86710d6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:31.484913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17afc7a-7f02-48a6-a0d0-93271abd2e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:31.485641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17afc7a-7f02-48a6-a0d0-93271abd2e7b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:31.485941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17afc7a-7f02-48a6-a0d0-93271abd2e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:31.486224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17afc7a-7f02-48a6-a0d0-93271abd2e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:31.486962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17afc7a-7f02-48a6-a0d0-93271abd2e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:31.488598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17afc7a-7f02-48a6-a0d0-93271abd2e7b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:31.499588 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3926/7818] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:31.503982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39648ae6-e6c6-4a8f-94d3-5a427d08b0f7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:31.514937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39648ae6-e6c6-4a8f-94d3-5a427d08b0f7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:31.519571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d13bf-eebe-4d3b-af14-ab5005acfb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:31.520440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d13bf-eebe-4d3b-af14-ab5005acfb23 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:31.520722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d13bf-eebe-4d3b-af14-ab5005acfb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:31.520999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d13bf-eebe-4d3b-af14-ab5005acfb23 admin admin] PATH_INFO: `/v3/projects/0fe6b50020084cfc96b4c1832aca7a4c/users/2f674edb7a584beaa27bf50a2a6312d6/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:31.536380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d13bf-eebe-4d3b-af14-ab5005acfb23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe6b50020084cfc96b4c1832aca7a4c, user_id=2f674edb7a584beaa27bf50a2a6312d6, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:31.539249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520d13bf-eebe-4d3b-af14-ab5005acfb23 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:31.645556 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3893/7819] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:34:31 2022] PUT /identity/v3/projects/0fe6b50020084cfc96b4c1832aca7a4c/users/2f674edb7a584beaa27bf50a2a6312d6/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:31.656344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf1482-6d58-430f-9e3a-42ccde1dab7f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:31.656672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf1482-6d58-430f-9e3a-42ccde1dab7f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:31.656960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf1482-6d58-430f-9e3a-42ccde1dab7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:31.687778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c4cf1482-6d58-430f-9e3a-42ccde1dab7f None None] MFA Rules not processed for user `2f674edb7a584beaa27bf50a2a6312d6`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:31.689028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c4cf1482-6d58-430f-9e3a-42ccde1dab7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:31.698218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:31.773979 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3927/7820] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:31.774768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d13bf-eebe-4d3b-af14-ab5005acfb23 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:31.776534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-520d13bf-eebe-4d3b-af14-ab5005acfb23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:31.850633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d13bf-eebe-4d3b-af14-ab5005acfb23 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:31.854400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bda0a6-ff3d-43b1-9c26-2ab501ec6621 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:31.855057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bda0a6-ff3d-43b1-9c26-2ab501ec6621 service glance] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:31.855340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bda0a6-ff3d-43b1-9c26-2ab501ec6621 service glance] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:31.855618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bda0a6-ff3d-43b1-9c26-2ab501ec6621 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:31.882183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:31.919464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bda0a6-ff3d-43b1-9c26-2ab501ec6621 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:31.920914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bda0a6-ff3d-43b1-9c26-2ab501ec6621 service glance] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:31.931324 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3894/7821] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:31 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:34:32.924546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf1482-6d58-430f-9e3a-42ccde1dab7f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:32.988557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf1482-6d58-430f-9e3a-42ccde1dab7f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:32.993260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587c1e4-51fa-4082-a699-df6db11be4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:32.994168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587c1e4-51fa-4082-a699-df6db11be4d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:32.994455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587c1e4-51fa-4082-a699-df6db11be4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:32.994736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587c1e4-51fa-4082-a699-df6db11be4d9 admin admin] PATH_INFO: `/v3/users/c73304be4db448c1b5e3f3673e6f3512` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:32.995930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5587c1e4-51fa-4082-a699-df6db11be4d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c73304be4db448c1b5e3f3673e6f3512)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:32.998423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5587c1e4-51fa-4082-a699-df6db11be4d9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:33.194629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5587c1e4-51fa-4082-a699-df6db11be4d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c73304be4db448c1b5e3f3673e6f3512'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:33.202675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5587c1e4-51fa-4082-a699-df6db11be4d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c73304be4db448c1b5e3f3673e6f3512'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:33.203368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5587c1e4-51fa-4082-a699-df6db11be4d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c73304be4db448c1b5e3f3673e6f3512'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:33.203968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5587c1e4-51fa-4082-a699-df6db11be4d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c73304be4db448c1b5e3f3673e6f3512'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:33.266109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5587c1e4-51fa-4082-a699-df6db11be4d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c73304be4db448c1b5e3f3673e6f3512'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:33.288739 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3928/7822] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:34:32 2022] DELETE /identity/v3/users/c73304be4db448c1b5e3f3673e6f3512 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:33.293508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09301cd6-adc6-4ddd-91ba-1c7e8a7da810 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:33.294012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09301cd6-adc6-4ddd-91ba-1c7e8a7da810 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:33.294108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09301cd6-adc6-4ddd-91ba-1c7e8a7da810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:33.311594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-09301cd6-adc6-4ddd-91ba-1c7e8a7da810 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:33.312796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-09301cd6-adc6-4ddd-91ba-1c7e8a7da810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:33.373693 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3895/7823] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:33.383421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587c1e4-51fa-4082-a699-df6db11be4d9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:33.384905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5587c1e4-51fa-4082-a699-df6db11be4d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:33.418289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:33.445315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587c1e4-51fa-4082-a699-df6db11be4d9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:33.448972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a82d54-cef0-4878-a0cd-4a85ad18c081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:33.449616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a82d54-cef0-4878-a0cd-4a85ad18c081 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:33.449940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a82d54-cef0-4878-a0cd-4a85ad18c081 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:33.450224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a82d54-cef0-4878-a0cd-4a85ad18c081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:33.460622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a82d54-cef0-4878-a0cd-4a85ad18c081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:33.462089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a82d54-cef0-4878-a0cd-4a85ad18c081 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:33.472365 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3929/7824] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:34:34.020395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09301cd6-adc6-4ddd-91ba-1c7e8a7da810 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:34.020395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-09301cd6-adc6-4ddd-91ba-1c7e8a7da810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:34.034065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09301cd6-adc6-4ddd-91ba-1c7e8a7da810 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:34.038861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18edbe39-9a33-475b-b0d8-d85436cda20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:34.039772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edbe39-9a33-475b-b0d8-d85436cda20f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:34.040056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edbe39-9a33-475b-b0d8-d85436cda20f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:34.040335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18edbe39-9a33-475b-b0d8-d85436cda20f admin admin] PATH_INFO: `/v3/projects/d950fd46f5754ce4ade5113020932fb6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:34.041641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18edbe39-9a33-475b-b0d8-d85436cda20f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d950fd46f5754ce4ade5113020932fb6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:34.043637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18edbe39-9a33-475b-b0d8-d85436cda20f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:34.174705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-18edbe39-9a33-475b-b0d8-d85436cda20f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd950fd46f5754ce4ade5113020932fb6', 'request_id': 'req-18edbe39-9a33-475b-b0d8-d85436cda20f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:34.180425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-18edbe39-9a33-475b-b0d8-d85436cda20f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd950fd46f5754ce4ade5113020932fb6', 'request_id': 'req-18edbe39-9a33-475b-b0d8-d85436cda20f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:34.264555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-18edbe39-9a33-475b-b0d8-d85436cda20f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd950fd46f5754ce4ade5113020932fb6', 'request_id': 'req-18edbe39-9a33-475b-b0d8-d85436cda20f'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:34.276584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-18edbe39-9a33-475b-b0d8-d85436cda20f admin admin] The token cache is being invalidate because project d950fd46f5754ce4ade5113020932fb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:34:34.276882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-18edbe39-9a33-475b-b0d8-d85436cda20f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:34.277522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-18edbe39-9a33-475b-b0d8-d85436cda20f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:34.279533 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3896/7825] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:34:34 2022] DELETE /identity/v3/projects/d950fd46f5754ce4ade5113020932fb6 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:34.283475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a82d54-cef0-4878-a0cd-4a85ad18c081 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:34.293277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a82d54-cef0-4878-a0cd-4a85ad18c081 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:34.351092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d54f06-872a-435c-8278-7be706947270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:34.351893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d54f06-872a-435c-8278-7be706947270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:34.352177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d54f06-872a-435c-8278-7be706947270 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:34.352460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d54f06-872a-435c-8278-7be706947270 admin admin] PATH_INFO: `/v3/users/2f674edb7a584beaa27bf50a2a6312d6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:34.353651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d54f06-872a-435c-8278-7be706947270 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f674edb7a584beaa27bf50a2a6312d6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:34.355643 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d54f06-872a-435c-8278-7be706947270 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:34.518914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c3d54f06-872a-435c-8278-7be706947270 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f674edb7a584beaa27bf50a2a6312d6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:34.525913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c3d54f06-872a-435c-8278-7be706947270 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f674edb7a584beaa27bf50a2a6312d6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:34.526588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c3d54f06-872a-435c-8278-7be706947270 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f674edb7a584beaa27bf50a2a6312d6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:34.528035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c3d54f06-872a-435c-8278-7be706947270 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f674edb7a584beaa27bf50a2a6312d6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:34.608083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c3d54f06-872a-435c-8278-7be706947270 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f674edb7a584beaa27bf50a2a6312d6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:34.627049 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3930/7826] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:34:34 2022] DELETE /identity/v3/users/2f674edb7a584beaa27bf50a2a6312d6 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:35.052265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18edbe39-9a33-475b-b0d8-d85436cda20f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:35.053851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-18edbe39-9a33-475b-b0d8-d85436cda20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:35.081318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18edbe39-9a33-475b-b0d8-d85436cda20f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:35.127129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:35.142308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b02dbd-12e9-4821-8711-f7898d239402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:35.143179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b02dbd-12e9-4821-8711-f7898d239402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:35.143464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b02dbd-12e9-4821-8711-f7898d239402 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:35.143745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b02dbd-12e9-4821-8711-f7898d239402 admin admin] PATH_INFO: `/v3/projects/0fe6b50020084cfc96b4c1832aca7a4c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:35.144930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b02dbd-12e9-4821-8711-f7898d239402 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fe6b50020084cfc96b4c1832aca7a4c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:35.146907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b02dbd-12e9-4821-8711-f7898d239402 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:35.266830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-41b02dbd-12e9-4821-8711-f7898d239402 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fe6b50020084cfc96b4c1832aca7a4c', 'request_id': 'req-41b02dbd-12e9-4821-8711-f7898d239402'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:35.272422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-41b02dbd-12e9-4821-8711-f7898d239402 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fe6b50020084cfc96b4c1832aca7a4c', 'request_id': 'req-41b02dbd-12e9-4821-8711-f7898d239402'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:35.351700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-41b02dbd-12e9-4821-8711-f7898d239402 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fe6b50020084cfc96b4c1832aca7a4c', 'request_id': 'req-41b02dbd-12e9-4821-8711-f7898d239402'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:35.364600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-41b02dbd-12e9-4821-8711-f7898d239402 admin admin] The token cache is being invalidate because project 0fe6b50020084cfc96b4c1832aca7a4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:34:35.364889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-41b02dbd-12e9-4821-8711-f7898d239402 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:35.365553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-41b02dbd-12e9-4821-8711-f7898d239402 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:35.367252 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3897/7827] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:34:35 2022] DELETE /identity/v3/projects/0fe6b50020084cfc96b4c1832aca7a4c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:35.372157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5b9bf-231f-45f5-9855-21a9187feb82 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:35.372645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5b9bf-231f-45f5-9855-21a9187feb82 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:35.372753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5b9bf-231f-45f5-9855-21a9187feb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:35.392566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-95d5b9bf-231f-45f5-9855-21a9187feb82 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:35.393711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-95d5b9bf-231f-45f5-9855-21a9187feb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:35.438928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:35.452910 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3931/7828] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:35.460669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a633b87-718d-42df-8b0b-9d961232b333 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:35.461039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a633b87-718d-42df-8b0b-9d961232b333 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:35.461411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a633b87-718d-42df-8b0b-9d961232b333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:35.482186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0a633b87-718d-42df-8b0b-9d961232b333 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:35.483980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0a633b87-718d-42df-8b0b-9d961232b333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:35.491881 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3898/7829] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:35.496079 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5b9bf-231f-45f5-9855-21a9187feb82 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:35.505631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5b9bf-231f-45f5-9855-21a9187feb82 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:35.510222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c563ec-ae57-4c51-8ff7-5e6a43238590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:35.510875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c563ec-ae57-4c51-8ff7-5e6a43238590 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:35.511157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c563ec-ae57-4c51-8ff7-5e6a43238590 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:35.511436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c563ec-ae57-4c51-8ff7-5e6a43238590 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:35.512167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c563ec-ae57-4c51-8ff7-5e6a43238590 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:34:35.512476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c563ec-ae57-4c51-8ff7-5e6a43238590 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:35.514109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c563ec-ae57-4c51-8ff7-5e6a43238590 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:35.525874 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3932/7830] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:34:35.530655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a633b87-718d-42df-8b0b-9d961232b333 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:35.539986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a633b87-718d-42df-8b0b-9d961232b333 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:35.544659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4648b-b13a-442a-95b1-22d98cf2ae4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:35.545896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4648b-b13a-442a-95b1-22d98cf2ae4b admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:35.546181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4648b-b13a-442a-95b1-22d98cf2ae4b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:35.546451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4648b-b13a-442a-95b1-22d98cf2ae4b admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:35.547134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4648b-b13a-442a-95b1-22d98cf2ae4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:35.549026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4648b-b13a-442a-95b1-22d98cf2ae4b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:35.617189 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3899/7831] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:34:35 2022] POST /identity/v3/projects => generated 391 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:34:35.621542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c563ec-ae57-4c51-8ff7-5e6a43238590 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:35.631952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c563ec-ae57-4c51-8ff7-5e6a43238590 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:35.692222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4b02d-f27f-47d3-ad58-d3abeca3fc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:35.692903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4b02d-f27f-47d3-ad58-d3abeca3fc2b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:35.693265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4b02d-f27f-47d3-ad58-d3abeca3fc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:35.693544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f4b02d-f27f-47d3-ad58-d3abeca3fc2b admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:35.694286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f4b02d-f27f-47d3-ad58-d3abeca3fc2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:35.696203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f4b02d-f27f-47d3-ad58-d3abeca3fc2b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:35.784638 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3933/7832] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:34:35 2022] POST /identity/v3/users => generated 302 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:34:35.786434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4648b-b13a-442a-95b1-22d98cf2ae4b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:35.797566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4648b-b13a-442a-95b1-22d98cf2ae4b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:35.802496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d9df3-1eec-4d0a-bd6a-ecaa44b3dc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:35.803889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d9df3-1eec-4d0a-bd6a-ecaa44b3dc79 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:35.804204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d9df3-1eec-4d0a-bd6a-ecaa44b3dc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:35.804697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1d9df3-1eec-4d0a-bd6a-ecaa44b3dc79 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:35.806150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d9df3-1eec-4d0a-bd6a-ecaa44b3dc79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:35.808660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1d9df3-1eec-4d0a-bd6a-ecaa44b3dc79 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:35.820865 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3900/7833] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:35.825442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4b02d-f27f-47d3-ad58-d3abeca3fc2b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:35.836649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f4b02d-f27f-47d3-ad58-d3abeca3fc2b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:35.841604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33731b-7480-4a96-860b-dcb8986452d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:35.849713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33731b-7480-4a96-860b-dcb8986452d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:35.849713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33731b-7480-4a96-860b-dcb8986452d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:35.849713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f33731b-7480-4a96-860b-dcb8986452d7 admin admin] PATH_INFO: `/v3/projects/f1b8de1eea844589a138c0eaa218dd46/users/ad2104dd86ce4cfd8e603d5b8b192e2e/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:35.859059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33731b-7480-4a96-860b-dcb8986452d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b8de1eea844589a138c0eaa218dd46, user_id=ad2104dd86ce4cfd8e603d5b8b192e2e, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:35.861706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f33731b-7480-4a96-860b-dcb8986452d7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:35.906975 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3934/7834] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:34:35 2022] PUT /identity/v3/projects/f1b8de1eea844589a138c0eaa218dd46/users/ad2104dd86ce4cfd8e603d5b8b192e2e/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:34:35.911112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d9df3-1eec-4d0a-bd6a-ecaa44b3dc79 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:35.921066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1d9df3-1eec-4d0a-bd6a-ecaa44b3dc79 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:35.982210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4781d4-bba6-4079-b8db-78fb43d3f518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:35.982648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4781d4-bba6-4079-b8db-78fb43d3f518 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:35.982942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4781d4-bba6-4079-b8db-78fb43d3f518 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:35.983342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4781d4-bba6-4079-b8db-78fb43d3f518 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:35.984082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4781d4-bba6-4079-b8db-78fb43d3f518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:35.985739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4781d4-bba6-4079-b8db-78fb43d3f518 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:35.995595 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3901/7835] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:36.000653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33731b-7480-4a96-860b-dcb8986452d7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:36.011841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f33731b-7480-4a96-860b-dcb8986452d7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:36.018084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f895585-4f53-4d0e-a0a1-a6a269f1e25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:36.019157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f895585-4f53-4d0e-a0a1-a6a269f1e25e admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:36.019517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f895585-4f53-4d0e-a0a1-a6a269f1e25e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:36.019980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f895585-4f53-4d0e-a0a1-a6a269f1e25e admin admin] PATH_INFO: `/v3/projects/f1b8de1eea844589a138c0eaa218dd46/users/ad2104dd86ce4cfd8e603d5b8b192e2e/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:36.022265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f895585-4f53-4d0e-a0a1-a6a269f1e25e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b8de1eea844589a138c0eaa218dd46, user_id=ad2104dd86ce4cfd8e603d5b8b192e2e, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:36.025306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f895585-4f53-4d0e-a0a1-a6a269f1e25e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:36.036479 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3935/7836] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:34:35 2022] PUT /identity/v3/projects/f1b8de1eea844589a138c0eaa218dd46/users/ad2104dd86ce4cfd8e603d5b8b192e2e/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:36.047674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd49f4-041f-4707-ad90-7dbca132b212 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:36.050836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd49f4-041f-4707-ad90-7dbca132b212 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:36.050836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd49f4-041f-4707-ad90-7dbca132b212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:36.078466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-83bd49f4-041f-4707-ad90-7dbca132b212 None None] MFA Rules not processed for user `ad2104dd86ce4cfd8e603d5b8b192e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:36.079758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-83bd49f4-041f-4707-ad90-7dbca132b212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:36.102399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:36.153409 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3902/7837] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:36.158203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f895585-4f53-4d0e-a0a1-a6a269f1e25e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:36.225028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f895585-4f53-4d0e-a0a1-a6a269f1e25e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:36.229668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e181fc7-7ce6-436d-9353-7e08513b0b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:36.230400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e181fc7-7ce6-436d-9353-7e08513b0b17 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:36.230715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e181fc7-7ce6-436d-9353-7e08513b0b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:36.230996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e181fc7-7ce6-436d-9353-7e08513b0b17 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:36.231695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e181fc7-7ce6-436d-9353-7e08513b0b17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:36.233608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e181fc7-7ce6-436d-9353-7e08513b0b17 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:36.332454 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3936/7838] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:34:36 2022] POST /identity/v3/projects => generated 391 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:34:36.340099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd49f4-041f-4707-ad90-7dbca132b212 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:36.350982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd49f4-041f-4707-ad90-7dbca132b212 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:36.410262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbd053-2736-46bf-9284-708997ff0217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:36.411043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbd053-2736-46bf-9284-708997ff0217 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:36.411321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbd053-2736-46bf-9284-708997ff0217 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:36.411590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbd053-2736-46bf-9284-708997ff0217 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:36.412585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdbd053-2736-46bf-9284-708997ff0217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:36.414619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdbd053-2736-46bf-9284-708997ff0217 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:36.527981 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3903/7839] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:34:36 2022] POST /identity/v3/users => generated 296 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:34:36.532100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e181fc7-7ce6-436d-9353-7e08513b0b17 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:36.542534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e181fc7-7ce6-436d-9353-7e08513b0b17 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:36.547783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905c69e-bc36-4d98-b777-681da1349466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:36.548437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905c69e-bc36-4d98-b777-681da1349466 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:36.548721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905c69e-bc36-4d98-b777-681da1349466 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:36.549329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905c69e-bc36-4d98-b777-681da1349466 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:36.549746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905c69e-bc36-4d98-b777-681da1349466 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:36.551385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905c69e-bc36-4d98-b777-681da1349466 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:36.563213 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3937/7840] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:36.567287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbd053-2736-46bf-9284-708997ff0217 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:36.577441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbd053-2736-46bf-9284-708997ff0217 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:36.582167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01cef3-be37-4701-a557-4cda7d4dec3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:36.582825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01cef3-be37-4701-a557-4cda7d4dec3e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:36.583114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01cef3-be37-4701-a557-4cda7d4dec3e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:36.583392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01cef3-be37-4701-a557-4cda7d4dec3e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:36.584131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01cef3-be37-4701-a557-4cda7d4dec3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:36.585716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01cef3-be37-4701-a557-4cda7d4dec3e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:36.596408 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3904/7841] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:36.600556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905c69e-bc36-4d98-b777-681da1349466 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:36.607250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:36.611503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905c69e-bc36-4d98-b777-681da1349466 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:36.616159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd314605-e78f-41b8-8ff7-fbaa3ad27658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:36.617027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd314605-e78f-41b8-8ff7-fbaa3ad27658 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:36.617308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd314605-e78f-41b8-8ff7-fbaa3ad27658 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:36.617582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd314605-e78f-41b8-8ff7-fbaa3ad27658 admin admin] PATH_INFO: `/v3/projects/ba3522a65ca641c691b61c67f32147f0/users/5f5c065f2d734845a90cc55a32edb5d3/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:36.632686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd314605-e78f-41b8-8ff7-fbaa3ad27658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba3522a65ca641c691b61c67f32147f0, user_id=5f5c065f2d734845a90cc55a32edb5d3, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:36.635484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd314605-e78f-41b8-8ff7-fbaa3ad27658 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:36.757725 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3938/7842] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:34:36 2022] PUT /identity/v3/projects/ba3522a65ca641c691b61c67f32147f0/users/5f5c065f2d734845a90cc55a32edb5d3/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:34:36.768296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fc23e-ad82-42a7-98cc-f8ad6078cf81 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:36.768588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fc23e-ad82-42a7-98cc-f8ad6078cf81 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:36.768867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fc23e-ad82-42a7-98cc-f8ad6078cf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:36.796269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-809fc23e-ad82-42a7-98cc-f8ad6078cf81 None None] MFA Rules not processed for user `5f5c065f2d734845a90cc55a32edb5d3`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:36.798048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-809fc23e-ad82-42a7-98cc-f8ad6078cf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:36.861031 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3905/7843] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:36.872608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd314605-e78f-41b8-8ff7-fbaa3ad27658 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:36.874409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bd314605-e78f-41b8-8ff7-fbaa3ad27658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:36.932979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd314605-e78f-41b8-8ff7-fbaa3ad27658 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:36.936731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d07705-0e50-467d-a57f-4f231e168c3f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:36.937625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d07705-0e50-467d-a57f-4f231e168c3f service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:36.937945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d07705-0e50-467d-a57f-4f231e168c3f service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:36.938220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d07705-0e50-467d-a57f-4f231e168c3f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:36.999951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d07705-0e50-467d-a57f-4f231e168c3f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:37.001395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d07705-0e50-467d-a57f-4f231e168c3f service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:37.011301 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3939/7844] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:36 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:34:38.541743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fc23e-ad82-42a7-98cc-f8ad6078cf81 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:38.569939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:38.605823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fc23e-ad82-42a7-98cc-f8ad6078cf81 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:38.610717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99e5c2-ce35-404d-b7ba-292baa06ea86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:38.611592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99e5c2-ce35-404d-b7ba-292baa06ea86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:38.611880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99e5c2-ce35-404d-b7ba-292baa06ea86 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:38.612157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99e5c2-ce35-404d-b7ba-292baa06ea86 admin admin] PATH_INFO: `/v3/users/ad2104dd86ce4cfd8e603d5b8b192e2e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:38.613397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99e5c2-ce35-404d-b7ba-292baa06ea86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad2104dd86ce4cfd8e603d5b8b192e2e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:38.615373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99e5c2-ce35-404d-b7ba-292baa06ea86 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:38.861206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7c99e5c2-ce35-404d-b7ba-292baa06ea86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad2104dd86ce4cfd8e603d5b8b192e2e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:38.861881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7c99e5c2-ce35-404d-b7ba-292baa06ea86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad2104dd86ce4cfd8e603d5b8b192e2e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:38.868911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7c99e5c2-ce35-404d-b7ba-292baa06ea86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad2104dd86ce4cfd8e603d5b8b192e2e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:38.945377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7c99e5c2-ce35-404d-b7ba-292baa06ea86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad2104dd86ce4cfd8e603d5b8b192e2e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:38.946282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7c99e5c2-ce35-404d-b7ba-292baa06ea86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad2104dd86ce4cfd8e603d5b8b192e2e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:38.967625 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3906/7845] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:34:38 2022] DELETE /identity/v3/users/ad2104dd86ce4cfd8e603d5b8b192e2e => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:38.972913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43547d-6ddc-47ab-9630-62c3fe54fb40 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:38.973410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43547d-6ddc-47ab-9630-62c3fe54fb40 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:38.973754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43547d-6ddc-47ab-9630-62c3fe54fb40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:38.991537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1a43547d-6ddc-47ab-9630-62c3fe54fb40 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:38.992740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1a43547d-6ddc-47ab-9630-62c3fe54fb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:39.052908 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3940/7846] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:39.063824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99e5c2-ce35-404d-b7ba-292baa06ea86 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:39.065394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7c99e5c2-ce35-404d-b7ba-292baa06ea86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:39.130889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99e5c2-ce35-404d-b7ba-292baa06ea86 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:39.135146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faba13-0adf-4c19-913a-497ef632b6fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:39.135829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faba13-0adf-4c19-913a-497ef632b6fc service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:39.136114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faba13-0adf-4c19-913a-497ef632b6fc service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:39.136391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faba13-0adf-4c19-913a-497ef632b6fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:39.146496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faba13-0adf-4c19-913a-497ef632b6fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:39.147953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faba13-0adf-4c19-913a-497ef632b6fc service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:39.158483 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3907/7847] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:34:39.595508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43547d-6ddc-47ab-9630-62c3fe54fb40 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:39.597199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1a43547d-6ddc-47ab-9630-62c3fe54fb40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:39.610401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43547d-6ddc-47ab-9630-62c3fe54fb40 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:39.615844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738ca49-9ef7-4589-9d07-d67b140aa6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:39.616795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738ca49-9ef7-4589-9d07-d67b140aa6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:39.617115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738ca49-9ef7-4589-9d07-d67b140aa6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:39.617489 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738ca49-9ef7-4589-9d07-d67b140aa6ae admin admin] PATH_INFO: `/v3/projects/f1b8de1eea844589a138c0eaa218dd46` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:39.618784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738ca49-9ef7-4589-9d07-d67b140aa6ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b8de1eea844589a138c0eaa218dd46)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:39.620789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738ca49-9ef7-4589-9d07-d67b140aa6ae admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:39.741697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:39.758189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d738ca49-9ef7-4589-9d07-d67b140aa6ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b8de1eea844589a138c0eaa218dd46', 'request_id': 'req-d738ca49-9ef7-4589-9d07-d67b140aa6ae'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:39.767976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d738ca49-9ef7-4589-9d07-d67b140aa6ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b8de1eea844589a138c0eaa218dd46', 'request_id': 'req-d738ca49-9ef7-4589-9d07-d67b140aa6ae'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:39.865664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d738ca49-9ef7-4589-9d07-d67b140aa6ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b8de1eea844589a138c0eaa218dd46', 'request_id': 'req-d738ca49-9ef7-4589-9d07-d67b140aa6ae'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:39.878296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d738ca49-9ef7-4589-9d07-d67b140aa6ae admin admin] The token cache is being invalidate because project f1b8de1eea844589a138c0eaa218dd46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:34:39.878588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d738ca49-9ef7-4589-9d07-d67b140aa6ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:39.879216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d738ca49-9ef7-4589-9d07-d67b140aa6ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:39.880756 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3941/7848] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:34:39 2022] DELETE /identity/v3/projects/f1b8de1eea844589a138c0eaa218dd46 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:39.884769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faba13-0adf-4c19-913a-497ef632b6fc service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:39.894822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faba13-0adf-4c19-913a-497ef632b6fc service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:39.940507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:39.952903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e4d5f-b14a-4de8-ae6e-b6422815a320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:39.954368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e4d5f-b14a-4de8-ae6e-b6422815a320 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:39.954657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e4d5f-b14a-4de8-ae6e-b6422815a320 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:39.954934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e4d5f-b14a-4de8-ae6e-b6422815a320 admin admin] PATH_INFO: `/v3/users/5f5c065f2d734845a90cc55a32edb5d3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:39.956131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e4d5f-b14a-4de8-ae6e-b6422815a320 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f5c065f2d734845a90cc55a32edb5d3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:39.958111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e4d5f-b14a-4de8-ae6e-b6422815a320 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:40.229472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a17e4d5f-b14a-4de8-ae6e-b6422815a320 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f5c065f2d734845a90cc55a32edb5d3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:40.230161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a17e4d5f-b14a-4de8-ae6e-b6422815a320 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f5c065f2d734845a90cc55a32edb5d3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:40.237333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a17e4d5f-b14a-4de8-ae6e-b6422815a320 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f5c065f2d734845a90cc55a32edb5d3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:40.317633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a17e4d5f-b14a-4de8-ae6e-b6422815a320 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f5c065f2d734845a90cc55a32edb5d3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:40.318318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a17e4d5f-b14a-4de8-ae6e-b6422815a320 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f5c065f2d734845a90cc55a32edb5d3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:40.337251 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3908/7849] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:34:39 2022] DELETE /identity/v3/users/5f5c065f2d734845a90cc55a32edb5d3 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:40.791994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738ca49-9ef7-4589-9d07-d67b140aa6ae admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:40.795647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d738ca49-9ef7-4589-9d07-d67b140aa6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:40.808854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738ca49-9ef7-4589-9d07-d67b140aa6ae admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:40.871022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8502cf5-22a4-4a59-9180-75076d8be4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:40.871887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8502cf5-22a4-4a59-9180-75076d8be4a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:40.872171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8502cf5-22a4-4a59-9180-75076d8be4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:40.872451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8502cf5-22a4-4a59-9180-75076d8be4a5 admin admin] PATH_INFO: `/v3/projects/ba3522a65ca641c691b61c67f32147f0` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:40.873633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8502cf5-22a4-4a59-9180-75076d8be4a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba3522a65ca641c691b61c67f32147f0)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:40.876219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8502cf5-22a4-4a59-9180-75076d8be4a5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:40.998169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b8502cf5-22a4-4a59-9180-75076d8be4a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba3522a65ca641c691b61c67f32147f0', 'request_id': 'req-b8502cf5-22a4-4a59-9180-75076d8be4a5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:41.004386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b8502cf5-22a4-4a59-9180-75076d8be4a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba3522a65ca641c691b61c67f32147f0', 'request_id': 'req-b8502cf5-22a4-4a59-9180-75076d8be4a5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:41.071591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b8502cf5-22a4-4a59-9180-75076d8be4a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba3522a65ca641c691b61c67f32147f0', 'request_id': 'req-b8502cf5-22a4-4a59-9180-75076d8be4a5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:41.083738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b8502cf5-22a4-4a59-9180-75076d8be4a5 admin admin] The token cache is being invalidate because project ba3522a65ca641c691b61c67f32147f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:34:41.084066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b8502cf5-22a4-4a59-9180-75076d8be4a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:41.085351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b8502cf5-22a4-4a59-9180-75076d8be4a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:41.086361 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3942/7850] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:34:40 2022] DELETE /identity/v3/projects/ba3522a65ca641c691b61c67f32147f0 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:41.091503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed763c3c-685e-45b0-8bca-7f711ec00ab1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:41.091994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed763c3c-685e-45b0-8bca-7f711ec00ab1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:41.092109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed763c3c-685e-45b0-8bca-7f711ec00ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:41.111917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ed763c3c-685e-45b0-8bca-7f711ec00ab1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:41.117379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ed763c3c-685e-45b0-8bca-7f711ec00ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:41.174500 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3909/7851] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:41.182638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41faa5e7-f103-4844-99cd-4f8a62fc069c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:41.182979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41faa5e7-f103-4844-99cd-4f8a62fc069c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:41.183261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41faa5e7-f103-4844-99cd-4f8a62fc069c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:41.202138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-41faa5e7-f103-4844-99cd-4f8a62fc069c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:41.203329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-41faa5e7-f103-4844-99cd-4f8a62fc069c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:41.211316 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3943/7852] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:41.216095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed763c3c-685e-45b0-8bca-7f711ec00ab1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:41.227213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed763c3c-685e-45b0-8bca-7f711ec00ab1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:41.232044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845b96d-82eb-481f-87db-1e92b2493876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:41.235114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845b96d-82eb-481f-87db-1e92b2493876 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:41.235114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845b96d-82eb-481f-87db-1e92b2493876 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:41.235114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845b96d-82eb-481f-87db-1e92b2493876 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:41.235114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845b96d-82eb-481f-87db-1e92b2493876 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:34:41.235114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845b96d-82eb-481f-87db-1e92b2493876 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:41.236581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845b96d-82eb-481f-87db-1e92b2493876 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:41.248404 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3910/7853] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:34:41.253465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41faa5e7-f103-4844-99cd-4f8a62fc069c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:41.264031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41faa5e7-f103-4844-99cd-4f8a62fc069c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:41.269037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1bdca-b6a8-4b80-a94d-ad7c57b0e2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:41.269696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1bdca-b6a8-4b80-a94d-ad7c57b0e2d4 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:41.270027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1bdca-b6a8-4b80-a94d-ad7c57b0e2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:41.270361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1bdca-b6a8-4b80-a94d-ad7c57b0e2d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:41.271060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1bdca-b6a8-4b80-a94d-ad7c57b0e2d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:41.272971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1bdca-b6a8-4b80-a94d-ad7c57b0e2d4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:41.387799 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3944/7854] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:34:41 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:34:41.392021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845b96d-82eb-481f-87db-1e92b2493876 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:41.402351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845b96d-82eb-481f-87db-1e92b2493876 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:41.449796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:41.458672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f3cd9-7b1f-4bfe-9675-f907eeb2a2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:41.459348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f3cd9-7b1f-4bfe-9675-f907eeb2a2b3 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:41.459633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f3cd9-7b1f-4bfe-9675-f907eeb2a2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:41.459913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f3cd9-7b1f-4bfe-9675-f907eeb2a2b3 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:41.460598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f3cd9-7b1f-4bfe-9675-f907eeb2a2b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:41.462569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f3cd9-7b1f-4bfe-9675-f907eeb2a2b3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:41.597173 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3911/7855] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:34:41 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:34:41.601613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1bdca-b6a8-4b80-a94d-ad7c57b0e2d4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:41.612575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1bdca-b6a8-4b80-a94d-ad7c57b0e2d4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:41.617537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f11ad0-b91c-49cb-9fd8-7c9f265de083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:41.618244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f11ad0-b91c-49cb-9fd8-7c9f265de083 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:41.622398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f11ad0-b91c-49cb-9fd8-7c9f265de083 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:41.622398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f11ad0-b91c-49cb-9fd8-7c9f265de083 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:41.622398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f11ad0-b91c-49cb-9fd8-7c9f265de083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:41.622398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f11ad0-b91c-49cb-9fd8-7c9f265de083 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:41.636448 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3945/7856] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:41.640942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f3cd9-7b1f-4bfe-9675-f907eeb2a2b3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:41.651296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f3cd9-7b1f-4bfe-9675-f907eeb2a2b3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:41.656811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa8e57-cdb8-4e83-b0af-5917b2bba566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:41.657587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa8e57-cdb8-4e83-b0af-5917b2bba566 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:41.657923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa8e57-cdb8-4e83-b0af-5917b2bba566 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:41.658209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa8e57-cdb8-4e83-b0af-5917b2bba566 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:41.658972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa8e57-cdb8-4e83-b0af-5917b2bba566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:41.660617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa8e57-cdb8-4e83-b0af-5917b2bba566 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:41.671847 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3912/7857] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:41.676050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f11ad0-b91c-49cb-9fd8-7c9f265de083 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:41.682567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:41.686379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f11ad0-b91c-49cb-9fd8-7c9f265de083 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:41.691292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085799e-540f-4870-b87e-d730e22314f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:41.692404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085799e-540f-4870-b87e-d730e22314f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:41.693128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085799e-540f-4870-b87e-d730e22314f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:41.693128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085799e-540f-4870-b87e-d730e22314f9 admin admin] PATH_INFO: `/v3/projects/3edbb38548434fb0a8f800974973e137/users/a3b86a59aded4f61844b594c230344d2/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:41.707868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d085799e-540f-4870-b87e-d730e22314f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3edbb38548434fb0a8f800974973e137, user_id=a3b86a59aded4f61844b594c230344d2, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:41.710644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d085799e-540f-4870-b87e-d730e22314f9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:41.821022 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3946/7858] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:34:41 2022] PUT /identity/v3/projects/3edbb38548434fb0a8f800974973e137/users/a3b86a59aded4f61844b594c230344d2/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:41.836774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecb3f8-cd47-436d-b24c-e0ac46c2292f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:41.837090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecb3f8-cd47-436d-b24c-e0ac46c2292f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:41.837373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecb3f8-cd47-436d-b24c-e0ac46c2292f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:41.866309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7aecb3f8-cd47-436d-b24c-e0ac46c2292f None None] MFA Rules not processed for user `a3b86a59aded4f61844b594c230344d2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:41.867442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7aecb3f8-cd47-436d-b24c-e0ac46c2292f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:41.931577 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3913/7859] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:34:41.936082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085799e-540f-4870-b87e-d730e22314f9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:41.999545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085799e-540f-4870-b87e-d730e22314f9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:42.005049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3700caa-3c87-44c7-b9f1-d97d0639b8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:42.005712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3700caa-3c87-44c7-b9f1-d97d0639b8fa admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:42.006029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3700caa-3c87-44c7-b9f1-d97d0639b8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:42.006334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3700caa-3c87-44c7-b9f1-d97d0639b8fa admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:42.007062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3700caa-3c87-44c7-b9f1-d97d0639b8fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:42.009028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3700caa-3c87-44c7-b9f1-d97d0639b8fa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:42.108340 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3947/7860] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:34:41 2022] POST /identity/v3/projects => generated 383 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:34:42.119445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecb3f8-cd47-436d-b24c-e0ac46c2292f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:42.130892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aecb3f8-cd47-436d-b24c-e0ac46c2292f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:42.188443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a249e-94d0-4d92-8bbc-2cfaa1478cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:42.189118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a249e-94d0-4d92-8bbc-2cfaa1478cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:42.189403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a249e-94d0-4d92-8bbc-2cfaa1478cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:42.189683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a249e-94d0-4d92-8bbc-2cfaa1478cd4 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:42.190504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a249e-94d0-4d92-8bbc-2cfaa1478cd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:42.192474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a249e-94d0-4d92-8bbc-2cfaa1478cd4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:42.254780 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3914/7861] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:34:42 2022] POST /identity/v3/users => generated 298 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:34:42.259035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3700caa-3c87-44c7-b9f1-d97d0639b8fa admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:42.270326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3700caa-3c87-44c7-b9f1-d97d0639b8fa admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:42.275078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9839635-9a29-417c-858e-a5b3cb135c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:42.275738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9839635-9a29-417c-858e-a5b3cb135c45 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:42.276132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9839635-9a29-417c-858e-a5b3cb135c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:42.276415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9839635-9a29-417c-858e-a5b3cb135c45 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:42.277142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9839635-9a29-417c-858e-a5b3cb135c45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:42.278771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9839635-9a29-417c-858e-a5b3cb135c45 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:42.289705 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3948/7862] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:42.301135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a249e-94d0-4d92-8bbc-2cfaa1478cd4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:42.311343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a249e-94d0-4d92-8bbc-2cfaa1478cd4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:42.316656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3afa20-f12f-4a4f-b0c8-382dd1275ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:42.317526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3afa20-f12f-4a4f-b0c8-382dd1275ddd admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:42.317859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3afa20-f12f-4a4f-b0c8-382dd1275ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:42.318147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3afa20-f12f-4a4f-b0c8-382dd1275ddd admin admin] PATH_INFO: `/v3/projects/648fffac95a941b48088057d77a135f0/users/b4cb26ac8fa64c4886490503abc3f347/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:42.332643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3afa20-f12f-4a4f-b0c8-382dd1275ddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=648fffac95a941b48088057d77a135f0, user_id=b4cb26ac8fa64c4886490503abc3f347, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:42.336310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3afa20-f12f-4a4f-b0c8-382dd1275ddd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:42.438667 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3915/7863] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:34:42 2022] PUT /identity/v3/projects/648fffac95a941b48088057d77a135f0/users/b4cb26ac8fa64c4886490503abc3f347/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:42.442763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9839635-9a29-417c-858e-a5b3cb135c45 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:42.453371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9839635-9a29-417c-858e-a5b3cb135c45 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:42.509261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0d72a-2faf-41fc-8052-c25d37a2f76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:42.510166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0d72a-2faf-41fc-8052-c25d37a2f76e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:42.510460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0d72a-2faf-41fc-8052-c25d37a2f76e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:42.513942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0d72a-2faf-41fc-8052-c25d37a2f76e admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:42.513942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0d72a-2faf-41fc-8052-c25d37a2f76e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:42.513942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb0d72a-2faf-41fc-8052-c25d37a2f76e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:42.524905 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3949/7864] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:42 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:42.530725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3afa20-f12f-4a4f-b0c8-382dd1275ddd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:42.541332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3afa20-f12f-4a4f-b0c8-382dd1275ddd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:42.546108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee526e1e-323f-4730-881a-3b206b15896b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:42.546993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee526e1e-323f-4730-881a-3b206b15896b admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:42.547274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee526e1e-323f-4730-881a-3b206b15896b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:42.548065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee526e1e-323f-4730-881a-3b206b15896b admin admin] PATH_INFO: `/v3/projects/648fffac95a941b48088057d77a135f0/users/b4cb26ac8fa64c4886490503abc3f347/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:42.550104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee526e1e-323f-4730-881a-3b206b15896b admin admin] RBAC: Authorizing `identity:create_grant(project_id=648fffac95a941b48088057d77a135f0, user_id=b4cb26ac8fa64c4886490503abc3f347, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:42.552898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee526e1e-323f-4730-881a-3b206b15896b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:42.564204 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3916/7865] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:34:42 2022] PUT /identity/v3/projects/648fffac95a941b48088057d77a135f0/users/b4cb26ac8fa64c4886490503abc3f347/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:42.574440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188994ce-cae1-41a8-ad04-aa2f3500ca11 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:42.574738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188994ce-cae1-41a8-ad04-aa2f3500ca11 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:42.575020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188994ce-cae1-41a8-ad04-aa2f3500ca11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:42.602159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-188994ce-cae1-41a8-ad04-aa2f3500ca11 None None] MFA Rules not processed for user `b4cb26ac8fa64c4886490503abc3f347`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:42.603354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-188994ce-cae1-41a8-ad04-aa2f3500ca11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:42.643403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:42.675993 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3950/7866] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:42.686168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee526e1e-323f-4730-881a-3b206b15896b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:42.687866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ee526e1e-323f-4730-881a-3b206b15896b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:42.702634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:42.750444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee526e1e-323f-4730-881a-3b206b15896b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:42.754147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d006331-6c8c-4d13-bfed-3e9673175f08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:42.754843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d006331-6c8c-4d13-bfed-3e9673175f08 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:42.755124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d006331-6c8c-4d13-bfed-3e9673175f08 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:42.755399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d006331-6c8c-4d13-bfed-3e9673175f08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:42.812729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d006331-6c8c-4d13-bfed-3e9673175f08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:42.814611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d006331-6c8c-4d13-bfed-3e9673175f08 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:42.824563 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3917/7867] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:34:43.485050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188994ce-cae1-41a8-ad04-aa2f3500ca11 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:43.497575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188994ce-cae1-41a8-ad04-aa2f3500ca11 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:43.501711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c75d4-befe-4bc9-8743-702c963855f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:43.502421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c75d4-befe-4bc9-8743-702c963855f7 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:43.502710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c75d4-befe-4bc9-8743-702c963855f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:43.502994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c75d4-befe-4bc9-8743-702c963855f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:43.512419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c75d4-befe-4bc9-8743-702c963855f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:43.513883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c75d4-befe-4bc9-8743-702c963855f7 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:43.524206 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3951/7868] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:34:52.231407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d006331-6c8c-4d13-bfed-3e9673175f08 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:52.310967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d006331-6c8c-4d13-bfed-3e9673175f08 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:52.316492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe25c58-afc6-4774-888e-42eefa17ee28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:52.317502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe25c58-afc6-4774-888e-42eefa17ee28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:52.317929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe25c58-afc6-4774-888e-42eefa17ee28 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:52.318315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe25c58-afc6-4774-888e-42eefa17ee28 admin admin] PATH_INFO: `/v3/users/a3b86a59aded4f61844b594c230344d2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:52.319778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe25c58-afc6-4774-888e-42eefa17ee28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3b86a59aded4f61844b594c230344d2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:52.321945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe25c58-afc6-4774-888e-42eefa17ee28 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:52.537489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cbe25c58-afc6-4774-888e-42eefa17ee28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3b86a59aded4f61844b594c230344d2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:52.538124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cbe25c58-afc6-4774-888e-42eefa17ee28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3b86a59aded4f61844b594c230344d2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:52.544992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cbe25c58-afc6-4774-888e-42eefa17ee28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3b86a59aded4f61844b594c230344d2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:52.631376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cbe25c58-afc6-4774-888e-42eefa17ee28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3b86a59aded4f61844b594c230344d2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:52.632005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-cbe25c58-afc6-4774-888e-42eefa17ee28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3b86a59aded4f61844b594c230344d2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:52.637744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:52.649912 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3918/7869] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:34:52 2022] DELETE /identity/v3/users/a3b86a59aded4f61844b594c230344d2 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:52.654543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29edd4-b5e0-4f26-bc95-fdf1a0d8fd9d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:52.655037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29edd4-b5e0-4f26-bc95-fdf1a0d8fd9d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:52.669117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29edd4-b5e0-4f26-bc95-fdf1a0d8fd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:52.674972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-be29edd4-b5e0-4f26-bc95-fdf1a0d8fd9d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:52.676202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-be29edd4-b5e0-4f26-bc95-fdf1a0d8fd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:52.735027 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3952/7870] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:34:52.745293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe25c58-afc6-4774-888e-42eefa17ee28 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:52.746711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cbe25c58-afc6-4774-888e-42eefa17ee28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:52.806572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe25c58-afc6-4774-888e-42eefa17ee28 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:52.810220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb9e27-f444-40e9-ab78-0163d8793f5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:52.810863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb9e27-f444-40e9-ab78-0163d8793f5c service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:52.811148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb9e27-f444-40e9-ab78-0163d8793f5c service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:52.811460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb9e27-f444-40e9-ab78-0163d8793f5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:52.820903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb9e27-f444-40e9-ab78-0163d8793f5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:52.822386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb9e27-f444-40e9-ab78-0163d8793f5c service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:52.832656 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3919/7871] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:34:53.102680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29edd4-b5e0-4f26-bc95-fdf1a0d8fd9d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:53.104188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-be29edd4-b5e0-4f26-bc95-fdf1a0d8fd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:53.116521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29edd4-b5e0-4f26-bc95-fdf1a0d8fd9d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:53.121580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a29ade-4938-4519-89d5-b33ae4d516a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:53.122496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a29ade-4938-4519-89d5-b33ae4d516a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:53.123240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a29ade-4938-4519-89d5-b33ae4d516a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:53.123396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a29ade-4938-4519-89d5-b33ae4d516a0 admin admin] PATH_INFO: `/v3/projects/3edbb38548434fb0a8f800974973e137` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:53.124838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a29ade-4938-4519-89d5-b33ae4d516a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3edbb38548434fb0a8f800974973e137)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:53.126541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a29ade-4938-4519-89d5-b33ae4d516a0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:53.278736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10a29ade-4938-4519-89d5-b33ae4d516a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3edbb38548434fb0a8f800974973e137', 'request_id': 'req-10a29ade-4938-4519-89d5-b33ae4d516a0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:53.284550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10a29ade-4938-4519-89d5-b33ae4d516a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3edbb38548434fb0a8f800974973e137', 'request_id': 'req-10a29ade-4938-4519-89d5-b33ae4d516a0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:53.418026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10a29ade-4938-4519-89d5-b33ae4d516a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3edbb38548434fb0a8f800974973e137', 'request_id': 'req-10a29ade-4938-4519-89d5-b33ae4d516a0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:53.431331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10a29ade-4938-4519-89d5-b33ae4d516a0 admin admin] The token cache is being invalidate because project 3edbb38548434fb0a8f800974973e137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:34:53.437179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10a29ade-4938-4519-89d5-b33ae4d516a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:53.437179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10a29ade-4938-4519-89d5-b33ae4d516a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:53.437179 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3953/7872] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:34:53 2022] DELETE /identity/v3/projects/3edbb38548434fb0a8f800974973e137 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:53.438285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb9e27-f444-40e9-ab78-0163d8793f5c service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:53.449335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb9e27-f444-40e9-ab78-0163d8793f5c service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:53.517950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b0c0c-d677-414c-8141-8121532ea393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:53.518827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b0c0c-d677-414c-8141-8121532ea393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:53.519109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b0c0c-d677-414c-8141-8121532ea393 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:53.519387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b0c0c-d677-414c-8141-8121532ea393 admin admin] PATH_INFO: `/v3/users/b4cb26ac8fa64c4886490503abc3f347` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:53.520589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b0c0c-d677-414c-8141-8121532ea393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4cb26ac8fa64c4886490503abc3f347)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:53.522590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b0c0c-d677-414c-8141-8121532ea393 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:53.787058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6b0b0c0c-d677-414c-8141-8121532ea393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4cb26ac8fa64c4886490503abc3f347'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:53.787722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6b0b0c0c-d677-414c-8141-8121532ea393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4cb26ac8fa64c4886490503abc3f347'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:53.794613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6b0b0c0c-d677-414c-8141-8121532ea393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4cb26ac8fa64c4886490503abc3f347'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:53.877110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6b0b0c0c-d677-414c-8141-8121532ea393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4cb26ac8fa64c4886490503abc3f347'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:53.878263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6b0b0c0c-d677-414c-8141-8121532ea393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4cb26ac8fa64c4886490503abc3f347'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:53.895957 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3920/7873] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:34:53 2022] DELETE /identity/v3/users/b4cb26ac8fa64c4886490503abc3f347 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:34:54.137180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a29ade-4938-4519-89d5-b33ae4d516a0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:54.138834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-10a29ade-4938-4519-89d5-b33ae4d516a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:54.154272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a29ade-4938-4519-89d5-b33ae4d516a0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:54.211042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:54.224897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a97535-3534-414a-8a9d-16ae74b01fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:54.226007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a97535-3534-414a-8a9d-16ae74b01fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:54.226189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a97535-3534-414a-8a9d-16ae74b01fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:54.226481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a97535-3534-414a-8a9d-16ae74b01fa6 admin admin] PATH_INFO: `/v3/projects/648fffac95a941b48088057d77a135f0` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:54.227696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a97535-3534-414a-8a9d-16ae74b01fa6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=648fffac95a941b48088057d77a135f0)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:54.229676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a97535-3534-414a-8a9d-16ae74b01fa6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:54.325073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-56a97535-3534-414a-8a9d-16ae74b01fa6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '648fffac95a941b48088057d77a135f0', 'request_id': 'req-56a97535-3534-414a-8a9d-16ae74b01fa6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:54.331020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-56a97535-3534-414a-8a9d-16ae74b01fa6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '648fffac95a941b48088057d77a135f0', 'request_id': 'req-56a97535-3534-414a-8a9d-16ae74b01fa6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:54.437499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-56a97535-3534-414a-8a9d-16ae74b01fa6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '648fffac95a941b48088057d77a135f0', 'request_id': 'req-56a97535-3534-414a-8a9d-16ae74b01fa6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:54.449685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-56a97535-3534-414a-8a9d-16ae74b01fa6 admin admin] The token cache is being invalidate because project 648fffac95a941b48088057d77a135f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:34:54.450524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-56a97535-3534-414a-8a9d-16ae74b01fa6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:54.451179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-56a97535-3534-414a-8a9d-16ae74b01fa6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:54.453080 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3954/7874] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:34:54 2022] DELETE /identity/v3/projects/648fffac95a941b48088057d77a135f0 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:54.458456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ea016-81a4-4490-a063-f8509c24fa24 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:54.459037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ea016-81a4-4490-a063-f8509c24fa24 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:54.459293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ea016-81a4-4490-a063-f8509c24fa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:54.478519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7e4ea016-81a4-4490-a063-f8509c24fa24 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:54.479767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7e4ea016-81a4-4490-a063-f8509c24fa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:54.485118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:54.549389 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3921/7875] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:54.554118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a75e36-a5d3-4de9-96c9-7a6a6a2237f3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:54.554414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a75e36-a5d3-4de9-96c9-7a6a6a2237f3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:54.554700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a75e36-a5d3-4de9-96c9-7a6a6a2237f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:54.574617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c2a75e36-a5d3-4de9-96c9-7a6a6a2237f3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:54.575963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c2a75e36-a5d3-4de9-96c9-7a6a6a2237f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:54.585071 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3955/7876] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:54.590097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ea016-81a4-4490-a063-f8509c24fa24 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:54.601137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ea016-81a4-4490-a063-f8509c24fa24 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:54.607351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76baeb9f-c17c-4f1a-a05b-0a2788d528cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:54.608089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baeb9f-c17c-4f1a-a05b-0a2788d528cd admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:54.608446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baeb9f-c17c-4f1a-a05b-0a2788d528cd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:54.608796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76baeb9f-c17c-4f1a-a05b-0a2788d528cd admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:54.609603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76baeb9f-c17c-4f1a-a05b-0a2788d528cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:34:54.610075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76baeb9f-c17c-4f1a-a05b-0a2788d528cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:54.611945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76baeb9f-c17c-4f1a-a05b-0a2788d528cd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:54.628789 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3922/7877] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:34:54.630503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a75e36-a5d3-4de9-96c9-7a6a6a2237f3 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:54.639994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a75e36-a5d3-4de9-96c9-7a6a6a2237f3 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:54.645706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad2a49-411f-48e5-b217-e8b868d88885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:54.646411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad2a49-411f-48e5-b217-e8b868d88885 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:54.646697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad2a49-411f-48e5-b217-e8b868d88885 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:54.647451 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad2a49-411f-48e5-b217-e8b868d88885 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:54.647733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad2a49-411f-48e5-b217-e8b868d88885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:54.649642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad2a49-411f-48e5-b217-e8b868d88885 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:54.747707 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3956/7878] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:34:54 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:34:54.749055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76baeb9f-c17c-4f1a-a05b-0a2788d528cd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:54.766850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76baeb9f-c17c-4f1a-a05b-0a2788d528cd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:54.825374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376543d-667d-48b3-850a-e2ad57075be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:54.828803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376543d-667d-48b3-850a-e2ad57075be5 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:54.828803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376543d-667d-48b3-850a-e2ad57075be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:54.828803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376543d-667d-48b3-850a-e2ad57075be5 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:54.828803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376543d-667d-48b3-850a-e2ad57075be5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:54.829271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0376543d-667d-48b3-850a-e2ad57075be5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:54.900868 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3923/7879] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:34:54 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:34:54.906204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad2a49-411f-48e5-b217-e8b868d88885 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:54.916889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad2a49-411f-48e5-b217-e8b868d88885 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:54.921561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90d501-b643-44aa-a547-f629efbdbced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:54.922553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90d501-b643-44aa-a547-f629efbdbced admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:54.922848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90d501-b643-44aa-a547-f629efbdbced admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:54.923116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90d501-b643-44aa-a547-f629efbdbced admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:54.923853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90d501-b643-44aa-a547-f629efbdbced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:54.925456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90d501-b643-44aa-a547-f629efbdbced admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:54.935906 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3957/7880] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:54.940447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376543d-667d-48b3-850a-e2ad57075be5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:54.950952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0376543d-667d-48b3-850a-e2ad57075be5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:54.956395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd9004-d00d-4260-a7f6-1f5dc6a5e824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:54.957086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd9004-d00d-4260-a7f6-1f5dc6a5e824 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:54.957376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd9004-d00d-4260-a7f6-1f5dc6a5e824 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:54.957662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd9004-d00d-4260-a7f6-1f5dc6a5e824 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:54.958424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd9004-d00d-4260-a7f6-1f5dc6a5e824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:54.960031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd9004-d00d-4260-a7f6-1f5dc6a5e824 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:54.970445 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3924/7881] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:54.974531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90d501-b643-44aa-a547-f629efbdbced admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:54.985191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90d501-b643-44aa-a547-f629efbdbced admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:54.990437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbbc85-0cab-468a-9fe1-f93d8b907e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:54.991297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbbc85-0cab-468a-9fe1-f93d8b907e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:54.991584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbbc85-0cab-468a-9fe1-f93d8b907e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:54.991865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bbbc85-0cab-468a-9fe1-f93d8b907e09 admin admin] PATH_INFO: `/v3/projects/02842acd2aca4dda89c9504eb07fbed1/users/d7d76f77a0734b8594f1a1ac6cc50a2d/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:55.006810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbbc85-0cab-468a-9fe1-f93d8b907e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02842acd2aca4dda89c9504eb07fbed1, user_id=d7d76f77a0734b8594f1a1ac6cc50a2d, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:55.009531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bbbc85-0cab-468a-9fe1-f93d8b907e09 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:55.060246 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3958/7882] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:34:54 2022] PUT /identity/v3/projects/02842acd2aca4dda89c9504eb07fbed1/users/d7d76f77a0734b8594f1a1ac6cc50a2d/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:34:55.071112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78dad8-3a0a-499e-a91e-6678af36691d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:55.071505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78dad8-3a0a-499e-a91e-6678af36691d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:55.071877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b78dad8-3a0a-499e-a91e-6678af36691d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:55.098984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1b78dad8-3a0a-499e-a91e-6678af36691d None None] MFA Rules not processed for user `d7d76f77a0734b8594f1a1ac6cc50a2d`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:55.100248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1b78dad8-3a0a-499e-a91e-6678af36691d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:55.164508 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3925/7883] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:55.168964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbbc85-0cab-468a-9fe1-f93d8b907e09 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:55.233031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bbbc85-0cab-468a-9fe1-f93d8b907e09 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:55.238340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f98f8-0086-48f5-87a6-6bae5b6e5d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:55.239005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f98f8-0086-48f5-87a6-6bae5b6e5d19 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:55.239288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f98f8-0086-48f5-87a6-6bae5b6e5d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:55.239562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2f98f8-0086-48f5-87a6-6bae5b6e5d19 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:55.240247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f98f8-0086-48f5-87a6-6bae5b6e5d19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:55.242298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2f98f8-0086-48f5-87a6-6bae5b6e5d19 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:55.295214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:55.305411 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3959/7884] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:34:55 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:34:55.307107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78dad8-3a0a-499e-a91e-6678af36691d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:55.313796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:55.318728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b78dad8-3a0a-499e-a91e-6678af36691d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:55.375989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2103ac-677e-4b3c-ad7d-fbccd39a0fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:55.376669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2103ac-677e-4b3c-ad7d-fbccd39a0fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:55.376955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2103ac-677e-4b3c-ad7d-fbccd39a0fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:55.377329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2103ac-677e-4b3c-ad7d-fbccd39a0fd0 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:55.378643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2103ac-677e-4b3c-ad7d-fbccd39a0fd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:55.380799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2103ac-677e-4b3c-ad7d-fbccd39a0fd0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:55.449091 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3926/7885] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:34:55 2022] POST /identity/v3/users => generated 295 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:34:55.453013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f98f8-0086-48f5-87a6-6bae5b6e5d19 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:55.463501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2f98f8-0086-48f5-87a6-6bae5b6e5d19 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:55.470801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae18f3c-f11a-43bf-b03f-c5dff5c63c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:55.471474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae18f3c-f11a-43bf-b03f-c5dff5c63c4c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:55.472254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae18f3c-f11a-43bf-b03f-c5dff5c63c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:55.472532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae18f3c-f11a-43bf-b03f-c5dff5c63c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:55.473265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae18f3c-f11a-43bf-b03f-c5dff5c63c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:55.474989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae18f3c-f11a-43bf-b03f-c5dff5c63c4c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:55.485748 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3960/7886] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:55.491108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2103ac-677e-4b3c-ad7d-fbccd39a0fd0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:55.501249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2103ac-677e-4b3c-ad7d-fbccd39a0fd0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:55.506018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacb49c-3cab-44cb-a7e4-91a12b5e5a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:55.506898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacb49c-3cab-44cb-a7e4-91a12b5e5a1b admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:55.507186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacb49c-3cab-44cb-a7e4-91a12b5e5a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:55.507466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacb49c-3cab-44cb-a7e4-91a12b5e5a1b admin admin] PATH_INFO: `/v3/projects/067f660323b74665b17cb0dc78aa13c1/users/4667ca7591a74c0ca9b4d3bf0381197c/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:55.522067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacb49c-3cab-44cb-a7e4-91a12b5e5a1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=067f660323b74665b17cb0dc78aa13c1, user_id=4667ca7591a74c0ca9b4d3bf0381197c, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:55.525204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacb49c-3cab-44cb-a7e4-91a12b5e5a1b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:55.631438 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3927/7887] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:34:55 2022] PUT /identity/v3/projects/067f660323b74665b17cb0dc78aa13c1/users/4667ca7591a74c0ca9b4d3bf0381197c/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:34:55.635599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae18f3c-f11a-43bf-b03f-c5dff5c63c4c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:55.647185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae18f3c-f11a-43bf-b03f-c5dff5c63c4c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:55.703488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5dfd3-e51e-4acf-9801-9ab96d4e6f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:55.704087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5dfd3-e51e-4acf-9801-9ab96d4e6f6c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:55.704485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5dfd3-e51e-4acf-9801-9ab96d4e6f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:55.704774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5dfd3-e51e-4acf-9801-9ab96d4e6f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:55.705492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5dfd3-e51e-4acf-9801-9ab96d4e6f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:55.707127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5dfd3-e51e-4acf-9801-9ab96d4e6f6c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:55.717174 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3961/7888] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:34:55.721260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacb49c-3cab-44cb-a7e4-91a12b5e5a1b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:55.732273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacb49c-3cab-44cb-a7e4-91a12b5e5a1b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:55.738674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e6140-9217-483f-80a1-4d69e1271d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:55.739610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e6140-9217-483f-80a1-4d69e1271d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:55.739850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e6140-9217-483f-80a1-4d69e1271d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:55.740129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e6140-9217-483f-80a1-4d69e1271d18 admin admin] PATH_INFO: `/v3/projects/067f660323b74665b17cb0dc78aa13c1/users/4667ca7591a74c0ca9b4d3bf0381197c/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:55.742225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e6140-9217-483f-80a1-4d69e1271d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=067f660323b74665b17cb0dc78aa13c1, user_id=4667ca7591a74c0ca9b4d3bf0381197c, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:55.744899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e6140-9217-483f-80a1-4d69e1271d18 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:55.755463 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3928/7889] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:34:55 2022] PUT /identity/v3/projects/067f660323b74665b17cb0dc78aa13c1/users/4667ca7591a74c0ca9b4d3bf0381197c/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:34:55.766226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518e1bf-c392-4a8a-b459-77355853e23f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:55.766550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518e1bf-c392-4a8a-b459-77355853e23f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:55.766865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518e1bf-c392-4a8a-b459-77355853e23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:55.794414 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d518e1bf-c392-4a8a-b459-77355853e23f None None] MFA Rules not processed for user `4667ca7591a74c0ca9b4d3bf0381197c`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:55.795628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d518e1bf-c392-4a8a-b459-77355853e23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:55.863050 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3962/7890] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:55.873262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e6140-9217-483f-80a1-4d69e1271d18 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:55.874978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-328e6140-9217-483f-80a1-4d69e1271d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:55.934233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e6140-9217-483f-80a1-4d69e1271d18 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:55.938121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca1b0a-3648-49ef-b9a9-5f8857371463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:55.938764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca1b0a-3648-49ef-b9a9-5f8857371463 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:55.939051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca1b0a-3648-49ef-b9a9-5f8857371463 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:55.939346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca1b0a-3648-49ef-b9a9-5f8857371463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:55.948724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca1b0a-3648-49ef-b9a9-5f8857371463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:55.950228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca1b0a-3648-49ef-b9a9-5f8857371463 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:55.960504 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3929/7891] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:34:58.347068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518e1bf-c392-4a8a-b459-77355853e23f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:58.410938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518e1bf-c392-4a8a-b459-77355853e23f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:58.415516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0a6f7-740d-41f4-a7f8-2aeb743b67e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:58.416364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0a6f7-740d-41f4-a7f8-2aeb743b67e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:58.416661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0a6f7-740d-41f4-a7f8-2aeb743b67e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:58.416940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0a6f7-740d-41f4-a7f8-2aeb743b67e8 admin admin] PATH_INFO: `/v3/users/d7d76f77a0734b8594f1a1ac6cc50a2d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:58.418166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0a6f7-740d-41f4-a7f8-2aeb743b67e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7d76f77a0734b8594f1a1ac6cc50a2d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:58.420096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0a6f7-740d-41f4-a7f8-2aeb743b67e8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:58.431524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:58.628786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a7b0a6f7-740d-41f4-a7f8-2aeb743b67e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7d76f77a0734b8594f1a1ac6cc50a2d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:58.635667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a7b0a6f7-740d-41f4-a7f8-2aeb743b67e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7d76f77a0734b8594f1a1ac6cc50a2d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:58.636332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a7b0a6f7-740d-41f4-a7f8-2aeb743b67e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7d76f77a0734b8594f1a1ac6cc50a2d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:58.636946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a7b0a6f7-740d-41f4-a7f8-2aeb743b67e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7d76f77a0734b8594f1a1ac6cc50a2d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:58.746244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a7b0a6f7-740d-41f4-a7f8-2aeb743b67e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7d76f77a0734b8594f1a1ac6cc50a2d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:58.765239 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3963/7892] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:34:58 2022] DELETE /identity/v3/users/d7d76f77a0734b8594f1a1ac6cc50a2d => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:58.775180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc9b3e-ae5d-43f9-919d-08e1afd0e75e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:58.775658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc9b3e-ae5d-43f9-919d-08e1afd0e75e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:58.775758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc9b3e-ae5d-43f9-919d-08e1afd0e75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:58.794089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b2fc9b3e-ae5d-43f9-919d-08e1afd0e75e None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:34:58.795288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b2fc9b3e-ae5d-43f9-919d-08e1afd0e75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:58.847481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:58.856274 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3930/7893] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:34:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:34:58.866813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0a6f7-740d-41f4-a7f8-2aeb743b67e8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:58.868295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a7b0a6f7-740d-41f4-a7f8-2aeb743b67e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:58.931973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0a6f7-740d-41f4-a7f8-2aeb743b67e8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:58.935711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c328a16-6a83-4d6a-b5bd-4c4c792fda11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:58.936360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c328a16-6a83-4d6a-b5bd-4c4c792fda11 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:58.936646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c328a16-6a83-4d6a-b5bd-4c4c792fda11 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:58.936925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c328a16-6a83-4d6a-b5bd-4c4c792fda11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:58.947155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c328a16-6a83-4d6a-b5bd-4c4c792fda11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:58.948590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c328a16-6a83-4d6a-b5bd-4c4c792fda11 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:58.958207 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3964/7894] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:34:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:34:59.391052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc9b3e-ae5d-43f9-919d-08e1afd0e75e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:59.392409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b2fc9b3e-ae5d-43f9-919d-08e1afd0e75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:34:59.405043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc9b3e-ae5d-43f9-919d-08e1afd0e75e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:59.410299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47e845-47a9-4b05-8ec8-956a1aafad8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:59.411172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47e845-47a9-4b05-8ec8-956a1aafad8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:59.411584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47e845-47a9-4b05-8ec8-956a1aafad8b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:59.411868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47e845-47a9-4b05-8ec8-956a1aafad8b admin admin] PATH_INFO: `/v3/projects/02842acd2aca4dda89c9504eb07fbed1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:59.413073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de47e845-47a9-4b05-8ec8-956a1aafad8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=02842acd2aca4dda89c9504eb07fbed1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:59.415043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de47e845-47a9-4b05-8ec8-956a1aafad8b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:59.514483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-de47e845-47a9-4b05-8ec8-956a1aafad8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02842acd2aca4dda89c9504eb07fbed1', 'request_id': 'req-de47e845-47a9-4b05-8ec8-956a1aafad8b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:59.521204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-de47e845-47a9-4b05-8ec8-956a1aafad8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02842acd2aca4dda89c9504eb07fbed1', 'request_id': 'req-de47e845-47a9-4b05-8ec8-956a1aafad8b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:59.586851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-de47e845-47a9-4b05-8ec8-956a1aafad8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02842acd2aca4dda89c9504eb07fbed1', 'request_id': 'req-de47e845-47a9-4b05-8ec8-956a1aafad8b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:59.600797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-de47e845-47a9-4b05-8ec8-956a1aafad8b admin admin] The token cache is being invalidate because project 02842acd2aca4dda89c9504eb07fbed1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:34:59.601091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-de47e845-47a9-4b05-8ec8-956a1aafad8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:59.601763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-de47e845-47a9-4b05-8ec8-956a1aafad8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:59.603544 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3931/7895] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:34:59 2022] DELETE /identity/v3/projects/02842acd2aca4dda89c9504eb07fbed1 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:34:59.607672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c328a16-6a83-4d6a-b5bd-4c4c792fda11 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:34:59.617997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c328a16-6a83-4d6a-b5bd-4c4c792fda11 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:34:59.678034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f120c79-a2a1-44af-8a6f-c0aad8f8c018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:34:59.678940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f120c79-a2a1-44af-8a6f-c0aad8f8c018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:34:59.679235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f120c79-a2a1-44af-8a6f-c0aad8f8c018 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:34:59.679605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f120c79-a2a1-44af-8a6f-c0aad8f8c018 admin admin] PATH_INFO: `/v3/users/4667ca7591a74c0ca9b4d3bf0381197c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:34:59.680794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f120c79-a2a1-44af-8a6f-c0aad8f8c018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4667ca7591a74c0ca9b4d3bf0381197c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:34:59.682996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f120c79-a2a1-44af-8a6f-c0aad8f8c018 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:34:59.896554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:59.908440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6f120c79-a2a1-44af-8a6f-c0aad8f8c018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4667ca7591a74c0ca9b4d3bf0381197c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:59.918780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6f120c79-a2a1-44af-8a6f-c0aad8f8c018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4667ca7591a74c0ca9b4d3bf0381197c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:59.919667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6f120c79-a2a1-44af-8a6f-c0aad8f8c018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4667ca7591a74c0ca9b4d3bf0381197c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:59.920349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6f120c79-a2a1-44af-8a6f-c0aad8f8c018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4667ca7591a74c0ca9b4d3bf0381197c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:34:59.980871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6f120c79-a2a1-44af-8a6f-c0aad8f8c018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4667ca7591a74c0ca9b4d3bf0381197c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:00.000050 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3965/7896] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:34:59 2022] DELETE /identity/v3/users/4667ca7591a74c0ca9b4d3bf0381197c => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:00.474652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47e845-47a9-4b05-8ec8-956a1aafad8b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:00.476332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-de47e845-47a9-4b05-8ec8-956a1aafad8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:00.490195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47e845-47a9-4b05-8ec8-956a1aafad8b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:00.550584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c058227-fc7b-4088-86ea-25ce80c36e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:00.551481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c058227-fc7b-4088-86ea-25ce80c36e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:00.551770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c058227-fc7b-4088-86ea-25ce80c36e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:00.552195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c058227-fc7b-4088-86ea-25ce80c36e70 admin admin] PATH_INFO: `/v3/projects/067f660323b74665b17cb0dc78aa13c1` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:00.553329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c058227-fc7b-4088-86ea-25ce80c36e70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=067f660323b74665b17cb0dc78aa13c1)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:00.555447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c058227-fc7b-4088-86ea-25ce80c36e70 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:00.684958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2c058227-fc7b-4088-86ea-25ce80c36e70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '067f660323b74665b17cb0dc78aa13c1', 'request_id': 'req-2c058227-fc7b-4088-86ea-25ce80c36e70'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:00.690789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2c058227-fc7b-4088-86ea-25ce80c36e70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '067f660323b74665b17cb0dc78aa13c1', 'request_id': 'req-2c058227-fc7b-4088-86ea-25ce80c36e70'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:00.774930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2c058227-fc7b-4088-86ea-25ce80c36e70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '067f660323b74665b17cb0dc78aa13c1', 'request_id': 'req-2c058227-fc7b-4088-86ea-25ce80c36e70'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:00.788774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2c058227-fc7b-4088-86ea-25ce80c36e70 admin admin] The token cache is being invalidate because project 067f660323b74665b17cb0dc78aa13c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:35:00.789079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2c058227-fc7b-4088-86ea-25ce80c36e70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:00.789751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2c058227-fc7b-4088-86ea-25ce80c36e70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:00.791446 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3932/7897] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:35:00 2022] DELETE /identity/v3/projects/067f660323b74665b17cb0dc78aa13c1 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:00.796457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e10ec-41cd-4890-a2ec-0b4765746d87 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:00.796966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e10ec-41cd-4890-a2ec-0b4765746d87 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:00.797092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e10ec-41cd-4890-a2ec-0b4765746d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:00.815066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-514e10ec-41cd-4890-a2ec-0b4765746d87 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:00.816270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-514e10ec-41cd-4890-a2ec-0b4765746d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:00.876697 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3966/7898] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:35:00.886061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9708f-e023-4646-b55b-586f3fdd2cef None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:00.886355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9708f-e023-4646-b55b-586f3fdd2cef None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:00.886638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd9708f-e023-4646-b55b-586f3fdd2cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:00.905564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ecd9708f-e023-4646-b55b-586f3fdd2cef None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:00.906805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ecd9708f-e023-4646-b55b-586f3fdd2cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:00.915196 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3933/7899] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:35:00.919074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e10ec-41cd-4890-a2ec-0b4765746d87 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:00.929587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e10ec-41cd-4890-a2ec-0b4765746d87 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:00.934255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdff5c-c00a-42a4-95e9-fecda1431efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:00.934906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdff5c-c00a-42a4-95e9-fecda1431efb admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:00.935189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdff5c-c00a-42a4-95e9-fecda1431efb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:00.935466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdff5c-c00a-42a4-95e9-fecda1431efb admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:00.936222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdff5c-c00a-42a4-95e9-fecda1431efb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:35:00.936600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdff5c-c00a-42a4-95e9-fecda1431efb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:00.938151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdff5c-c00a-42a4-95e9-fecda1431efb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:00.950388 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3967/7900] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:35:00.954951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9708f-e023-4646-b55b-586f3fdd2cef None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:00.964455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd9708f-e023-4646-b55b-586f3fdd2cef None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:00.969709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cff250-4621-4a3e-91f2-bda47d1d73cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:00.970405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cff250-4621-4a3e-91f2-bda47d1d73cf admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:00.970691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cff250-4621-4a3e-91f2-bda47d1d73cf admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:00.970968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cff250-4621-4a3e-91f2-bda47d1d73cf admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:00.971660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cff250-4621-4a3e-91f2-bda47d1d73cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:00.973723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cff250-4621-4a3e-91f2-bda47d1d73cf admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:00.977307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:00.999492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdff5c-c00a-42a4-95e9-fecda1431efb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:01.001221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ffbdff5c-c00a-42a4-95e9-fecda1431efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:01.015144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdff5c-c00a-42a4-95e9-fecda1431efb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:01.019973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89d5ad-6e4d-45ff-9663-fd482fb66c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:01.020811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89d5ad-6e4d-45ff-9663-fd482fb66c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:01.021105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89d5ad-6e4d-45ff-9663-fd482fb66c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:01.021390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89d5ad-6e4d-45ff-9663-fd482fb66c1a admin admin] PATH_INFO: `/v3/users/1d9d97eaf9f74d58adf5ea7e90918c22` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:01.022649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89d5ad-6e4d-45ff-9663-fd482fb66c1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d9d97eaf9f74d58adf5ea7e90918c22)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:01.024701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89d5ad-6e4d-45ff-9663-fd482fb66c1a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:01.085321 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3934/7901] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:35:00 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:35:01.089190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cff250-4621-4a3e-91f2-bda47d1d73cf admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:01.149647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cff250-4621-4a3e-91f2-bda47d1d73cf admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:01.154895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8910ca-f85d-4524-8ea6-607c0a5d179e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:01.155610 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8910ca-f85d-4524-8ea6-607c0a5d179e admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:01.156019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8910ca-f85d-4524-8ea6-607c0a5d179e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:01.156306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8910ca-f85d-4524-8ea6-607c0a5d179e admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:01.156998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8910ca-f85d-4524-8ea6-607c0a5d179e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:01.159267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8910ca-f85d-4524-8ea6-607c0a5d179e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:01.261262 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3935/7902] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:35:01 2022] POST /identity/v3/users => generated 283 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:35:01.265577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8910ca-f85d-4524-8ea6-607c0a5d179e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:01.275716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8910ca-f85d-4524-8ea6-607c0a5d179e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:01.279644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2d89d5ad-6e4d-45ff-9663-fd482fb66c1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d9d97eaf9f74d58adf5ea7e90918c22'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:01.280528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0593ff-b3aa-48ac-ae50-515bbb5f0869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:01.281302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0593ff-b3aa-48ac-ae50-515bbb5f0869 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:01.281716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0593ff-b3aa-48ac-ae50-515bbb5f0869 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:01.282114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0593ff-b3aa-48ac-ae50-515bbb5f0869 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:01.282856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0593ff-b3aa-48ac-ae50-515bbb5f0869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:01.284474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0593ff-b3aa-48ac-ae50-515bbb5f0869 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:01.288374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2d89d5ad-6e4d-45ff-9663-fd482fb66c1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d9d97eaf9f74d58adf5ea7e90918c22'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:01.292843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2d89d5ad-6e4d-45ff-9663-fd482fb66c1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d9d97eaf9f74d58adf5ea7e90918c22'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:01.292843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2d89d5ad-6e4d-45ff-9663-fd482fb66c1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d9d97eaf9f74d58adf5ea7e90918c22'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:01.295398 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3936/7903] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:35:01.299222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0593ff-b3aa-48ac-ae50-515bbb5f0869 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:01.300894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1f0593ff-b3aa-48ac-ae50-515bbb5f0869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:01.313686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0593ff-b3aa-48ac-ae50-515bbb5f0869 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:01.319036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264b38f-1545-409c-8240-a8c3514cee57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:01.319676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264b38f-1545-409c-8240-a8c3514cee57 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:01.319963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264b38f-1545-409c-8240-a8c3514cee57 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:01.320242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264b38f-1545-409c-8240-a8c3514cee57 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:01.320958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264b38f-1545-409c-8240-a8c3514cee57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:01.322574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264b38f-1545-409c-8240-a8c3514cee57 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:01.332204 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3937/7904] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:35:01.336253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264b38f-1545-409c-8240-a8c3514cee57 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:01.346102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264b38f-1545-409c-8240-a8c3514cee57 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:01.351228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a555597b-c2d1-4dbf-a074-a6ffe223efc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:01.352069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555597b-c2d1-4dbf-a074-a6ffe223efc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:01.352350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555597b-c2d1-4dbf-a074-a6ffe223efc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:01.352624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a555597b-c2d1-4dbf-a074-a6ffe223efc0 admin admin] PATH_INFO: `/v3/projects/339e2550870747638bf4646e49d65ec9/users/2c0d89d588514c34b5e4c72ddef4fca6/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:01.362845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2d89d5ad-6e4d-45ff-9663-fd482fb66c1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d9d97eaf9f74d58adf5ea7e90918c22'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:01.367994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a555597b-c2d1-4dbf-a074-a6ffe223efc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=339e2550870747638bf4646e49d65ec9, user_id=2c0d89d588514c34b5e4c72ddef4fca6, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:01.370750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a555597b-c2d1-4dbf-a074-a6ffe223efc0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:01.382691 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3968/7905] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:35:00 2022] DELETE /identity/v3/users/1d9d97eaf9f74d58adf5ea7e90918c22 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:35:01.386990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e6286-36a4-4e55-b596-c117dad4e227 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:01.387281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e6286-36a4-4e55-b596-c117dad4e227 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:01.387561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e6286-36a4-4e55-b596-c117dad4e227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:01.403878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e44e6286-36a4-4e55-b596-c117dad4e227 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:01.405096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e44e6286-36a4-4e55-b596-c117dad4e227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:01.408986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:01.463114 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3969/7906] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:35:01.472810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e6286-36a4-4e55-b596-c117dad4e227 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:01.474680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e44e6286-36a4-4e55-b596-c117dad4e227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:01.511827 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3938/7907] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:35:01 2022] PUT /identity/v3/projects/339e2550870747638bf4646e49d65ec9/users/2c0d89d588514c34b5e4c72ddef4fca6/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:01.522053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d417c2e-0555-4aa5-b1d9-615f347ccc62 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:01.522341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d417c2e-0555-4aa5-b1d9-615f347ccc62 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:01.522620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d417c2e-0555-4aa5-b1d9-615f347ccc62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:01.538116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e6286-36a4-4e55-b596-c117dad4e227 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:01.550079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9d417c2e-0555-4aa5-b1d9-615f347ccc62 None None] MFA Rules not processed for user `2c0d89d588514c34b5e4c72ddef4fca6`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:01.551298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9d417c2e-0555-4aa5-b1d9-615f347ccc62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:01.587551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654f978-0a2c-4e4c-8cf9-b0701853e1c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:01.588210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654f978-0a2c-4e4c-8cf9-b0701853e1c7 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:01.588498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654f978-0a2c-4e4c-8cf9-b0701853e1c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:01.588782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654f978-0a2c-4e4c-8cf9-b0701853e1c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:01.622006 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3939/7908] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:35:01.622903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d417c2e-0555-4aa5-b1d9-615f347ccc62 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:01.641048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:01.653114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654f978-0a2c-4e4c-8cf9-b0701853e1c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:01.654832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654f978-0a2c-4e4c-8cf9-b0701853e1c7 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:01.665822 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3970/7909] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:35:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:35:01.686288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d417c2e-0555-4aa5-b1d9-615f347ccc62 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:01.691546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454cf52-9d36-40d1-a897-db258d827944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:01.692632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454cf52-9d36-40d1-a897-db258d827944 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:01.695328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454cf52-9d36-40d1-a897-db258d827944 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:01.695328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454cf52-9d36-40d1-a897-db258d827944 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:01.695328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454cf52-9d36-40d1-a897-db258d827944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:01.696230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454cf52-9d36-40d1-a897-db258d827944 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:01.831696 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3940/7910] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:35:01 2022] POST /identity/v3/projects => generated 365 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:35:01.836250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654f978-0a2c-4e4c-8cf9-b0701853e1c7 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:01.846667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654f978-0a2c-4e4c-8cf9-b0701853e1c7 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:01.858948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:01.903765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd042b-56a3-467e-827a-631546783595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:01.904443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd042b-56a3-467e-827a-631546783595 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:01.904723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd042b-56a3-467e-827a-631546783595 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:01.904998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd042b-56a3-467e-827a-631546783595 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:01.905687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd042b-56a3-467e-827a-631546783595 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:01.907657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd042b-56a3-467e-827a-631546783595 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:01.984493 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3971/7911] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:35:01 2022] POST /identity/v3/users => generated 289 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:35:01.986288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454cf52-9d36-40d1-a897-db258d827944 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:01.997051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454cf52-9d36-40d1-a897-db258d827944 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:02.002011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23462c-f3dc-4e0a-907b-983e022ee9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:02.002682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23462c-f3dc-4e0a-907b-983e022ee9bd admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:02.003029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23462c-f3dc-4e0a-907b-983e022ee9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:02.003251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc23462c-f3dc-4e0a-907b-983e022ee9bd admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:02.003988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23462c-f3dc-4e0a-907b-983e022ee9bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:02.005670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc23462c-f3dc-4e0a-907b-983e022ee9bd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:02.017151 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3941/7912] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:35:02.022072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd042b-56a3-467e-827a-631546783595 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:02.032324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd042b-56a3-467e-827a-631546783595 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:02.037749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c3c39-006f-4ebe-8213-7e162e6c4714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:02.038685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c3c39-006f-4ebe-8213-7e162e6c4714 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:02.038976 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c3c39-006f-4ebe-8213-7e162e6c4714 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:02.039258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c3c39-006f-4ebe-8213-7e162e6c4714 admin admin] PATH_INFO: `/v3/projects/5ff3470db2764b0fa924306ce210820a/users/4735c6374e104a0bbb2c211b56a06c7a/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:02.054567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c3c39-006f-4ebe-8213-7e162e6c4714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff3470db2764b0fa924306ce210820a, user_id=4735c6374e104a0bbb2c211b56a06c7a, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:02.062085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c3c39-006f-4ebe-8213-7e162e6c4714 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:02.119132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23462c-f3dc-4e0a-907b-983e022ee9bd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:02.121004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-dc23462c-f3dc-4e0a-907b-983e022ee9bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:02.138321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc23462c-f3dc-4e0a-907b-983e022ee9bd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:02.143630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913f4b6-f7a3-4e96-9439-0b1bc07e59ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:02.144671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913f4b6-f7a3-4e96-9439-0b1bc07e59ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:02.144960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913f4b6-f7a3-4e96-9439-0b1bc07e59ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:02.145905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913f4b6-f7a3-4e96-9439-0b1bc07e59ad admin admin] PATH_INFO: `/v3/projects/045cd28b35b3467e91274fff8140771b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:02.147122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913f4b6-f7a3-4e96-9439-0b1bc07e59ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=045cd28b35b3467e91274fff8140771b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:02.149313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913f4b6-f7a3-4e96-9439-0b1bc07e59ad admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:02.216050 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3972/7913] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:35:02 2022] PUT /identity/v3/projects/5ff3470db2764b0fa924306ce210820a/users/4735c6374e104a0bbb2c211b56a06c7a/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:02.220013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c3c39-006f-4ebe-8213-7e162e6c4714 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:02.275044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a913f4b6-f7a3-4e96-9439-0b1bc07e59ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '045cd28b35b3467e91274fff8140771b', 'request_id': 'req-a913f4b6-f7a3-4e96-9439-0b1bc07e59ad'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:02.280542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a913f4b6-f7a3-4e96-9439-0b1bc07e59ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '045cd28b35b3467e91274fff8140771b', 'request_id': 'req-a913f4b6-f7a3-4e96-9439-0b1bc07e59ad'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:02.284560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c3c39-006f-4ebe-8213-7e162e6c4714 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:02.349429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b336f-df0b-440d-ad4f-a7fa61177189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:02.350066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b336f-df0b-440d-ad4f-a7fa61177189 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:02.350355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b336f-df0b-440d-ad4f-a7fa61177189 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:02.350636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b336f-df0b-440d-ad4f-a7fa61177189 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:02.351377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b336f-df0b-440d-ad4f-a7fa61177189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:02.352989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b336f-df0b-440d-ad4f-a7fa61177189 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:02.362693 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3973/7914] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:35:02.366810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b336f-df0b-440d-ad4f-a7fa61177189 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:02.367499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a913f4b6-f7a3-4e96-9439-0b1bc07e59ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '045cd28b35b3467e91274fff8140771b', 'request_id': 'req-a913f4b6-f7a3-4e96-9439-0b1bc07e59ad'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:02.376701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b336f-df0b-440d-ad4f-a7fa61177189 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:02.380122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a913f4b6-f7a3-4e96-9439-0b1bc07e59ad admin admin] The token cache is being invalidate because project 045cd28b35b3467e91274fff8140771b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:35:02.380279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a913f4b6-f7a3-4e96-9439-0b1bc07e59ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:02.380923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a913f4b6-f7a3-4e96-9439-0b1bc07e59ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:02.382511 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3942/7915] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:35:02 2022] DELETE /identity/v3/projects/045cd28b35b3467e91274fff8140771b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:02.382835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124c0a7-69cc-4f1b-9246-8a2eb059677d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:02.383801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124c0a7-69cc-4f1b-9246-8a2eb059677d admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:02.384177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124c0a7-69cc-4f1b-9246-8a2eb059677d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:02.384462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124c0a7-69cc-4f1b-9246-8a2eb059677d admin admin] PATH_INFO: `/v3/projects/5ff3470db2764b0fa924306ce210820a/users/4735c6374e104a0bbb2c211b56a06c7a/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:02.386506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124c0a7-69cc-4f1b-9246-8a2eb059677d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff3470db2764b0fa924306ce210820a, user_id=4735c6374e104a0bbb2c211b56a06c7a, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:02.387271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73aca2-b5e5-4fa7-92be-d9de35062174 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:02.387561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73aca2-b5e5-4fa7-92be-d9de35062174 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:02.387843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73aca2-b5e5-4fa7-92be-d9de35062174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:02.389173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124c0a7-69cc-4f1b-9246-8a2eb059677d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:02.404530 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3974/7916] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:35:02 2022] PUT /identity/v3/projects/5ff3470db2764b0fa924306ce210820a/users/4735c6374e104a0bbb2c211b56a06c7a/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:02.411499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1c73aca2-b5e5-4fa7-92be-d9de35062174 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:02.412765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1c73aca2-b5e5-4fa7-92be-d9de35062174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:02.414681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6215a-7e4c-4848-9f90-c4c590b65acb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:02.414958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6215a-7e4c-4848-9f90-c4c590b65acb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:02.415252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6215a-7e4c-4848-9f90-c4c590b65acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:02.436727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:02.444901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-f8c6215a-7e4c-4848-9f90-c4c590b65acb None None] MFA Rules not processed for user `4735c6374e104a0bbb2c211b56a06c7a`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:02.446239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f8c6215a-7e4c-4848-9f90-c4c590b65acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:02.476915 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3943/7917] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:35:02.484572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd3b78-a7be-45b3-8507-604ab9eb9418 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:02.484866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd3b78-a7be-45b3-8507-604ab9eb9418 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:02.485147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd3b78-a7be-45b3-8507-604ab9eb9418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:02.504178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5cbd3b78-a7be-45b3-8507-604ab9eb9418 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:02.506065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5cbd3b78-a7be-45b3-8507-604ab9eb9418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:02.514387 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3944/7918] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:35:02.518279 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3975/7919] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:35:02.519031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd3b78-a7be-45b3-8507-604ab9eb9418 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:02.527566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6215a-7e4c-4848-9f90-c4c590b65acb None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:02.528957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f8c6215a-7e4c-4848-9f90-c4c590b65acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:02.529229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd3b78-a7be-45b3-8507-604ab9eb9418 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:02.535440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f564d-65ae-407a-b88a-c89cdee809f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:02.536350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f564d-65ae-407a-b88a-c89cdee809f7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:02.536801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f564d-65ae-407a-b88a-c89cdee809f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:02.537079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4f564d-65ae-407a-b88a-c89cdee809f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:02.537865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f564d-65ae-407a-b88a-c89cdee809f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:35:02.547622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f564d-65ae-407a-b88a-c89cdee809f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:02.547622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4f564d-65ae-407a-b88a-c89cdee809f7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:02.547622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:02.556187 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3945/7920] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:35:02.557181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f564d-65ae-407a-b88a-c89cdee809f7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:02.566844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4f564d-65ae-407a-b88a-c89cdee809f7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:02.572833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e45e7b-ab81-4dc3-8119-118d9598fe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:02.577605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e45e7b-ab81-4dc3-8119-118d9598fe73 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:02.577605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e45e7b-ab81-4dc3-8119-118d9598fe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:02.577605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e45e7b-ab81-4dc3-8119-118d9598fe73 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:02.577605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e45e7b-ab81-4dc3-8119-118d9598fe73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:02.577605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e45e7b-ab81-4dc3-8119-118d9598fe73 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:02.590912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6215a-7e4c-4848-9f90-c4c590b65acb None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:02.594725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c7405-4eea-4b23-8cf7-4fe27d4b0689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:02.595395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c7405-4eea-4b23-8cf7-4fe27d4b0689 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:02.595678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c7405-4eea-4b23-8cf7-4fe27d4b0689 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:02.596002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c7405-4eea-4b23-8cf7-4fe27d4b0689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:02.605600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c7405-4eea-4b23-8cf7-4fe27d4b0689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:02.607023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c7405-4eea-4b23-8cf7-4fe27d4b0689 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:02.617063 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3976/7921] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:35:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:35:02.673547 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3946/7922] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:35:02 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:35:02.678319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c7405-4eea-4b23-8cf7-4fe27d4b0689 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:02.688906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c7405-4eea-4b23-8cf7-4fe27d4b0689 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:02.746939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e77436-c26d-49da-99ba-70dbe3ab58fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:02.747996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e77436-c26d-49da-99ba-70dbe3ab58fc admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:02.748280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e77436-c26d-49da-99ba-70dbe3ab58fc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:02.748554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e77436-c26d-49da-99ba-70dbe3ab58fc admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:02.749298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e77436-c26d-49da-99ba-70dbe3ab58fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:02.751405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e77436-c26d-49da-99ba-70dbe3ab58fc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:02.912417 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3977/7923] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:35:02 2022] POST /identity/v3/users => generated 287 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:35:02.916583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e45e7b-ab81-4dc3-8119-118d9598fe73 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:02.927556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e45e7b-ab81-4dc3-8119-118d9598fe73 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:02.931915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fca7b-6504-4004-83fc-47a327bea7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:02.932918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fca7b-6504-4004-83fc-47a327bea7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:02.933148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fca7b-6504-4004-83fc-47a327bea7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:02.933464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763fca7b-6504-4004-83fc-47a327bea7d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:02.934237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763fca7b-6504-4004-83fc-47a327bea7d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:02.935873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763fca7b-6504-4004-83fc-47a327bea7d8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:02.946796 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3947/7924] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:35:02.950769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e77436-c26d-49da-99ba-70dbe3ab58fc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:02.961669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e77436-c26d-49da-99ba-70dbe3ab58fc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:02.966716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec87da32-557d-448b-931f-1fe19a028f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:02.967383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87da32-557d-448b-931f-1fe19a028f7d admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:02.967672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87da32-557d-448b-931f-1fe19a028f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:02.967958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec87da32-557d-448b-931f-1fe19a028f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:02.970664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87da32-557d-448b-931f-1fe19a028f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:02.970664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec87da32-557d-448b-931f-1fe19a028f7d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:02.980470 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3978/7925] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:35:02.984463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fca7b-6504-4004-83fc-47a327bea7d8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:02.995432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763fca7b-6504-4004-83fc-47a327bea7d8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:03.000189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2464448-9900-4392-a57f-b97dc76da442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:03.001080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2464448-9900-4392-a57f-b97dc76da442 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:03.001364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2464448-9900-4392-a57f-b97dc76da442 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:03.001645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2464448-9900-4392-a57f-b97dc76da442 admin admin] PATH_INFO: `/v3/projects/4b4e0b07528040a3aa0194c06fb942cb/users/13249a1c05fa43bd917b36aafb9db0d9/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:03.017793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2464448-9900-4392-a57f-b97dc76da442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b4e0b07528040a3aa0194c06fb942cb, user_id=13249a1c05fa43bd917b36aafb9db0d9, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:03.021142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2464448-9900-4392-a57f-b97dc76da442 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:03.099303 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3948/7926] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:35:02 2022] PUT /identity/v3/projects/4b4e0b07528040a3aa0194c06fb942cb/users/13249a1c05fa43bd917b36aafb9db0d9/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:03.104878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceb7ac-2bfc-4653-b269-2d1e29bff76f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:03.105222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceb7ac-2bfc-4653-b269-2d1e29bff76f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:03.105532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceb7ac-2bfc-4653-b269-2d1e29bff76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:03.124034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-48ceb7ac-2bfc-4653-b269-2d1e29bff76f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:03.125266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-48ceb7ac-2bfc-4653-b269-2d1e29bff76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:03.139668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:03.191955 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3979/7927] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:35:03.202411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2464448-9900-4392-a57f-b97dc76da442 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:03.206482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e2464448-9900-4392-a57f-b97dc76da442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:03.271188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2464448-9900-4392-a57f-b97dc76da442 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:03.275118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5aaf71-c0eb-49b7-a76b-05f71ee36736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:03.275791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5aaf71-c0eb-49b7-a76b-05f71ee36736 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:03.276079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5aaf71-c0eb-49b7-a76b-05f71ee36736 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:03.276392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5aaf71-c0eb-49b7-a76b-05f71ee36736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:03.286816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5aaf71-c0eb-49b7-a76b-05f71ee36736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:03.288295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5aaf71-c0eb-49b7-a76b-05f71ee36736 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:03.299188 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3949/7928] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:35:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:35:06.098674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ceb7ac-2bfc-4653-b269-2d1e29bff76f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:06.115928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ceb7ac-2bfc-4653-b269-2d1e29bff76f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:06.121933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e6dbe-aecf-42be-8b86-fd4f731e2077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:06.122951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e6dbe-aecf-42be-8b86-fd4f731e2077 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:06.123422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e6dbe-aecf-42be-8b86-fd4f731e2077 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:06.123811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e6dbe-aecf-42be-8b86-fd4f731e2077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:06.127218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b45e6dbe-aecf-42be-8b86-fd4f731e2077 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:06.193736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e6dbe-aecf-42be-8b86-fd4f731e2077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:06.194225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e6dbe-aecf-42be-8b86-fd4f731e2077 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:06.206258 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3980/7929] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:35:06 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:35:07.885677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5aaf71-c0eb-49b7-a76b-05f71ee36736 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:07.887557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1b5aaf71-c0eb-49b7-a76b-05f71ee36736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:07.905413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5aaf71-c0eb-49b7-a76b-05f71ee36736 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:07.910469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972a304-4ce6-4ff7-9dc0-79dc342f1a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:07.911361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972a304-4ce6-4ff7-9dc0-79dc342f1a4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:07.911707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972a304-4ce6-4ff7-9dc0-79dc342f1a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:07.911987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1972a304-4ce6-4ff7-9dc0-79dc342f1a4a admin admin] PATH_INFO: `/v3/users/2c0d89d588514c34b5e4c72ddef4fca6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:07.913533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1972a304-4ce6-4ff7-9dc0-79dc342f1a4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0d89d588514c34b5e4c72ddef4fca6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:07.915691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1972a304-4ce6-4ff7-9dc0-79dc342f1a4a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:08.170965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1972a304-4ce6-4ff7-9dc0-79dc342f1a4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0d89d588514c34b5e4c72ddef4fca6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:08.171745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1972a304-4ce6-4ff7-9dc0-79dc342f1a4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0d89d588514c34b5e4c72ddef4fca6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:08.182511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1972a304-4ce6-4ff7-9dc0-79dc342f1a4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0d89d588514c34b5e4c72ddef4fca6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:08.323442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1972a304-4ce6-4ff7-9dc0-79dc342f1a4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0d89d588514c34b5e4c72ddef4fca6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:08.332795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1972a304-4ce6-4ff7-9dc0-79dc342f1a4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0d89d588514c34b5e4c72ddef4fca6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:08.354482 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3950/7930] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:35:07 2022] DELETE /identity/v3/users/2c0d89d588514c34b5e4c72ddef4fca6 => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:08.359641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16107f-e88a-4e50-aa29-5b11d345fdf5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:08.367885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16107f-e88a-4e50-aa29-5b11d345fdf5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:08.367885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16107f-e88a-4e50-aa29-5b11d345fdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:08.388313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8d16107f-e88a-4e50-aa29-5b11d345fdf5 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:08.389750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8d16107f-e88a-4e50-aa29-5b11d345fdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:08.499311 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3981/7931] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:35:08.517823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972a304-4ce6-4ff7-9dc0-79dc342f1a4a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:08.519464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1972a304-4ce6-4ff7-9dc0-79dc342f1a4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:08.547849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:08.657601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1972a304-4ce6-4ff7-9dc0-79dc342f1a4a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:08.665529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894ed9c-ce11-4278-ad24-727b3c3e115f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:08.672265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894ed9c-ce11-4278-ad24-727b3c3e115f service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:08.672265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894ed9c-ce11-4278-ad24-727b3c3e115f service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:08.672265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894ed9c-ce11-4278-ad24-727b3c3e115f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:08.678147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894ed9c-ce11-4278-ad24-727b3c3e115f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:08.679813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894ed9c-ce11-4278-ad24-727b3c3e115f service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:08.690895 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3951/7932] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:35:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:35:09.262279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16107f-e88a-4e50-aa29-5b11d345fdf5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:09.263777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8d16107f-e88a-4e50-aa29-5b11d345fdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:09.290589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16107f-e88a-4e50-aa29-5b11d345fdf5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:09.306110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a3da5-e7d4-4a4a-967d-c8c642cb7d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:09.307010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a3da5-e7d4-4a4a-967d-c8c642cb7d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:09.307484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a3da5-e7d4-4a4a-967d-c8c642cb7d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:09.307993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a3da5-e7d4-4a4a-967d-c8c642cb7d1d admin admin] PATH_INFO: `/v3/projects/339e2550870747638bf4646e49d65ec9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:09.310167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a3da5-e7d4-4a4a-967d-c8c642cb7d1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=339e2550870747638bf4646e49d65ec9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:09.312405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a3da5-e7d4-4a4a-967d-c8c642cb7d1d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:09.442854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c59a3da5-e7d4-4a4a-967d-c8c642cb7d1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '339e2550870747638bf4646e49d65ec9', 'request_id': 'req-c59a3da5-e7d4-4a4a-967d-c8c642cb7d1d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:09.445689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:09.450415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c59a3da5-e7d4-4a4a-967d-c8c642cb7d1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '339e2550870747638bf4646e49d65ec9', 'request_id': 'req-c59a3da5-e7d4-4a4a-967d-c8c642cb7d1d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:09.525640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c59a3da5-e7d4-4a4a-967d-c8c642cb7d1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '339e2550870747638bf4646e49d65ec9', 'request_id': 'req-c59a3da5-e7d4-4a4a-967d-c8c642cb7d1d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:09.539845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c59a3da5-e7d4-4a4a-967d-c8c642cb7d1d admin admin] The token cache is being invalidate because project 339e2550870747638bf4646e49d65ec9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:35:09.540141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c59a3da5-e7d4-4a4a-967d-c8c642cb7d1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:09.541033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c59a3da5-e7d4-4a4a-967d-c8c642cb7d1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:09.542819 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3982/7933] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:35:09 2022] DELETE /identity/v3/projects/339e2550870747638bf4646e49d65ec9 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:09.546968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894ed9c-ce11-4278-ad24-727b3c3e115f service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:09.559582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894ed9c-ce11-4278-ad24-727b3c3e115f service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:09.695415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5d099-e773-42a5-90ce-3f9de5fa1e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:09.696325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5d099-e773-42a5-90ce-3f9de5fa1e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:09.696626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5d099-e773-42a5-90ce-3f9de5fa1e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:09.696948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5d099-e773-42a5-90ce-3f9de5fa1e51 admin admin] PATH_INFO: `/v3/users/4735c6374e104a0bbb2c211b56a06c7a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:09.699364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a5d099-e773-42a5-90ce-3f9de5fa1e51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4735c6374e104a0bbb2c211b56a06c7a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:09.701344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a5d099-e773-42a5-90ce-3f9de5fa1e51 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:09.767033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187eda90-0fe0-45ad-ab71-0e70e13a72e7 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:09.767338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187eda90-0fe0-45ad-ab71-0e70e13a72e7 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:09.767636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187eda90-0fe0-45ad-ab71-0e70e13a72e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:09.805492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-187eda90-0fe0-45ad-ab71-0e70e13a72e7 None None] MFA Rules not processed for user `13249a1c05fa43bd917b36aafb9db0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:09.807095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-187eda90-0fe0-45ad-ab71-0e70e13a72e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:09.879040 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3983/7934] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:35:09.883701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187eda90-0fe0-45ad-ab71-0e70e13a72e7 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:09.885491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-187eda90-0fe0-45ad-ab71-0e70e13a72e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:09.903695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187eda90-0fe0-45ad-ab71-0e70e13a72e7 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:09.909382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b341c2-02a2-424d-9a8b-e5342b546315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:09.910141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b341c2-02a2-424d-9a8b-e5342b546315 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:09.910377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b341c2-02a2-424d-9a8b-e5342b546315 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:09.910661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b341c2-02a2-424d-9a8b-e5342b546315 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:09.912010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b341c2-02a2-424d-9a8b-e5342b546315 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:09.914246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b341c2-02a2-424d-9a8b-e5342b546315 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:09.915772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b1a5d099-e773-42a5-90ce-3f9de5fa1e51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4735c6374e104a0bbb2c211b56a06c7a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:09.916527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b1a5d099-e773-42a5-90ce-3f9de5fa1e51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4735c6374e104a0bbb2c211b56a06c7a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:09.926382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b1a5d099-e773-42a5-90ce-3f9de5fa1e51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4735c6374e104a0bbb2c211b56a06c7a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:09.997201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b1a5d099-e773-42a5-90ce-3f9de5fa1e51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4735c6374e104a0bbb2c211b56a06c7a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:10.001268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b1a5d099-e773-42a5-90ce-3f9de5fa1e51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4735c6374e104a0bbb2c211b56a06c7a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:10.017209 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3952/7935] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:35:09 2022] DELETE /identity/v3/users/4735c6374e104a0bbb2c211b56a06c7a => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:10.028279 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3984/7936] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:35:09 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:35:10.032935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5d099-e773-42a5-90ce-3f9de5fa1e51 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:10.034487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b1a5d099-e773-42a5-90ce-3f9de5fa1e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:10.108969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:10.140263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5d099-e773-42a5-90ce-3f9de5fa1e51 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:10.215922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55a042-8e06-4e83-b9bf-0f26be608d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:10.216949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55a042-8e06-4e83-b9bf-0f26be608d3b admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:10.217466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55a042-8e06-4e83-b9bf-0f26be608d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:10.217764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b55a042-8e06-4e83-b9bf-0f26be608d3b admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:10.218532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55a042-8e06-4e83-b9bf-0f26be608d3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:10.223262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b55a042-8e06-4e83-b9bf-0f26be608d3b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:10.330086 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3953/7937] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:35:10 2022] POST /identity/v3/users => generated 293 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:35:10.334672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b341c2-02a2-424d-9a8b-e5342b546315 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:10.347490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b341c2-02a2-424d-9a8b-e5342b546315 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:10.355410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59372822-bf09-4596-9bd0-32f9133e858f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:10.355410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59372822-bf09-4596-9bd0-32f9133e858f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:10.355410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59372822-bf09-4596-9bd0-32f9133e858f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:10.355410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59372822-bf09-4596-9bd0-32f9133e858f admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:10.355820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59372822-bf09-4596-9bd0-32f9133e858f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:10.356804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59372822-bf09-4596-9bd0-32f9133e858f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:10.372840 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3985/7938] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:35:10.381321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55a042-8e06-4e83-b9bf-0f26be608d3b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:10.393238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b55a042-8e06-4e83-b9bf-0f26be608d3b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:10.404087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44515443-c458-468e-bc3b-7a4bd110d4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:10.405373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44515443-c458-468e-bc3b-7a4bd110d4d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:10.405865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44515443-c458-468e-bc3b-7a4bd110d4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:10.406306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44515443-c458-468e-bc3b-7a4bd110d4d2 admin admin] PATH_INFO: `/v3/projects/e5d9e07b1e18471e9a5c0667719672f6/users/bce779fb1eb547a5a077c6f1ac48d800/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:10.433433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44515443-c458-468e-bc3b-7a4bd110d4d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d9e07b1e18471e9a5c0667719672f6, user_id=bce779fb1eb547a5a077c6f1ac48d800, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:10.436818 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44515443-c458-468e-bc3b-7a4bd110d4d2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:10.552275 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3954/7939] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:35:10 2022] PUT /identity/v3/projects/e5d9e07b1e18471e9a5c0667719672f6/users/bce779fb1eb547a5a077c6f1ac48d800/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:10.552853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59372822-bf09-4596-9bd0-32f9133e858f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:10.554800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-59372822-bf09-4596-9bd0-32f9133e858f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:10.556951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44515443-c458-468e-bc3b-7a4bd110d4d2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:10.571964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59372822-bf09-4596-9bd0-32f9133e858f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:10.641436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44515443-c458-468e-bc3b-7a4bd110d4d2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:10.647635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757fb00-f0c2-4393-a8cb-bda81da02b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:10.648273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757fb00-f0c2-4393-a8cb-bda81da02b57 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:10.648560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757fb00-f0c2-4393-a8cb-bda81da02b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:10.648838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757fb00-f0c2-4393-a8cb-bda81da02b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:10.649636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757fb00-f0c2-4393-a8cb-bda81da02b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:10.651326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0757fb00-f0c2-4393-a8cb-bda81da02b57 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:10.651963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7de29-0173-41be-8ed2-e97dae1ca11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:10.652839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7de29-0173-41be-8ed2-e97dae1ca11c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:10.653204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7de29-0173-41be-8ed2-e97dae1ca11c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:10.653738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7de29-0173-41be-8ed2-e97dae1ca11c admin admin] PATH_INFO: `/v3/projects/5ff3470db2764b0fa924306ce210820a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:10.655073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7de29-0173-41be-8ed2-e97dae1ca11c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ff3470db2764b0fa924306ce210820a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:10.657194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7de29-0173-41be-8ed2-e97dae1ca11c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:10.663246 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3955/7940] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:35:10.669216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757fb00-f0c2-4393-a8cb-bda81da02b57 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:10.679787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757fb00-f0c2-4393-a8cb-bda81da02b57 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:10.685711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a25beb-4308-4565-9271-bac95ee48073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:10.686707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a25beb-4308-4565-9271-bac95ee48073 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:10.687068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a25beb-4308-4565-9271-bac95ee48073 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:10.687364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a25beb-4308-4565-9271-bac95ee48073 admin admin] PATH_INFO: `/v3/projects/e5d9e07b1e18471e9a5c0667719672f6/users/bce779fb1eb547a5a077c6f1ac48d800/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:10.689377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a25beb-4308-4565-9271-bac95ee48073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d9e07b1e18471e9a5c0667719672f6, user_id=bce779fb1eb547a5a077c6f1ac48d800, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:10.692744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a25beb-4308-4565-9271-bac95ee48073 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:10.703126 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3956/7941] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:35:10 2022] PUT /identity/v3/projects/e5d9e07b1e18471e9a5c0667719672f6/users/bce779fb1eb547a5a077c6f1ac48d800/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:35:10.746459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:10.753149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-edf7de29-0173-41be-8ed2-e97dae1ca11c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ff3470db2764b0fa924306ce210820a', 'request_id': 'req-edf7de29-0173-41be-8ed2-e97dae1ca11c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:10.759425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-edf7de29-0173-41be-8ed2-e97dae1ca11c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ff3470db2764b0fa924306ce210820a', 'request_id': 'req-edf7de29-0173-41be-8ed2-e97dae1ca11c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:10.837862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-edf7de29-0173-41be-8ed2-e97dae1ca11c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ff3470db2764b0fa924306ce210820a', 'request_id': 'req-edf7de29-0173-41be-8ed2-e97dae1ca11c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:10.851116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-edf7de29-0173-41be-8ed2-e97dae1ca11c admin admin] The token cache is being invalidate because project 5ff3470db2764b0fa924306ce210820a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:35:10.851503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-edf7de29-0173-41be-8ed2-e97dae1ca11c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:10.852322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-edf7de29-0173-41be-8ed2-e97dae1ca11c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:10.854308 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3986/7942] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:35:10 2022] DELETE /identity/v3/projects/5ff3470db2764b0fa924306ce210820a => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:10.859179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d539d88-e5fd-4ebe-a1e4-784f5594227f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:10.859631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d539d88-e5fd-4ebe-a1e4-784f5594227f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:10.860135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d539d88-e5fd-4ebe-a1e4-784f5594227f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:10.882901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2d539d88-e5fd-4ebe-a1e4-784f5594227f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:10.884473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2d539d88-e5fd-4ebe-a1e4-784f5594227f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:10.913453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:10.948830 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3957/7943] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:35:10.957764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad18edd-7fbd-491e-abe9-1dc59526fcf8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:10.958107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad18edd-7fbd-491e-abe9-1dc59526fcf8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:10.958390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad18edd-7fbd-491e-abe9-1dc59526fcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:10.978069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5ad18edd-7fbd-491e-abe9-1dc59526fcf8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:10.979438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5ad18edd-7fbd-491e-abe9-1dc59526fcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:10.992103 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3987/7944] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:35:10.992321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d539d88-e5fd-4ebe-a1e4-784f5594227f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:11.003267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d539d88-e5fd-4ebe-a1e4-784f5594227f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:11.018986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dbd8f0-fdef-4991-9c75-e1284712e320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:11.020044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbd8f0-fdef-4991-9c75-e1284712e320 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:11.020362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbd8f0-fdef-4991-9c75-e1284712e320 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:11.020640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbd8f0-fdef-4991-9c75-e1284712e320 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:11.021386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dbd8f0-fdef-4991-9c75-e1284712e320 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:35:11.021826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dbd8f0-fdef-4991-9c75-e1284712e320 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:11.026470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dbd8f0-fdef-4991-9c75-e1284712e320 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:11.036321 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3958/7945] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:35:11.041072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad18edd-7fbd-491e-abe9-1dc59526fcf8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:11.053787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad18edd-7fbd-491e-abe9-1dc59526fcf8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:11.062272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbd103-a6c4-4c8b-ad47-e6c6bf12bb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:11.062272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbd103-a6c4-4c8b-ad47-e6c6bf12bb08 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:11.062272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbd103-a6c4-4c8b-ad47-e6c6bf12bb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:11.062272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fbd103-a6c4-4c8b-ad47-e6c6bf12bb08 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:11.062619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbd103-a6c4-4c8b-ad47-e6c6bf12bb08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:11.066200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fbd103-a6c4-4c8b-ad47-e6c6bf12bb08 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:11.192714 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3988/7946] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:35:11 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:35:11.196047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dbd8f0-fdef-4991-9c75-e1284712e320 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:11.207397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dbd8f0-fdef-4991-9c75-e1284712e320 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:11.279100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae7588-4175-41c0-80bf-8bc7cce447d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:11.280048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae7588-4175-41c0-80bf-8bc7cce447d5 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:11.280528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae7588-4175-41c0-80bf-8bc7cce447d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:11.280809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ae7588-4175-41c0-80bf-8bc7cce447d5 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:11.281604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae7588-4175-41c0-80bf-8bc7cce447d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:11.291880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ae7588-4175-41c0-80bf-8bc7cce447d5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:11.462875 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3959/7947] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:35:11 2022] POST /identity/v3/users => generated 288 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:35:11.467450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbd103-a6c4-4c8b-ad47-e6c6bf12bb08 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:11.479469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fbd103-a6c4-4c8b-ad47-e6c6bf12bb08 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:11.484472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388fdf6-70b7-46fd-9fd3-3388a0d1853a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:11.489534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388fdf6-70b7-46fd-9fd3-3388a0d1853a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:11.489534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388fdf6-70b7-46fd-9fd3-3388a0d1853a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:11.489534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388fdf6-70b7-46fd-9fd3-3388a0d1853a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:11.489534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388fdf6-70b7-46fd-9fd3-3388a0d1853a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:11.489534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388fdf6-70b7-46fd-9fd3-3388a0d1853a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:11.504248 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3989/7948] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:35:11.506192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae7588-4175-41c0-80bf-8bc7cce447d5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:11.519671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ae7588-4175-41c0-80bf-8bc7cce447d5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:11.524763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d135ee2-36dc-414a-8fd7-f3776df652ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:11.525514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d135ee2-36dc-414a-8fd7-f3776df652ff admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:11.525970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d135ee2-36dc-414a-8fd7-f3776df652ff admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:11.526321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d135ee2-36dc-414a-8fd7-f3776df652ff admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:11.527454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d135ee2-36dc-414a-8fd7-f3776df652ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:11.528987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d135ee2-36dc-414a-8fd7-f3776df652ff admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:11.543158 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3960/7949] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:35:11.547580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388fdf6-70b7-46fd-9fd3-3388a0d1853a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:11.560090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388fdf6-70b7-46fd-9fd3-3388a0d1853a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:11.566954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e80cf-07d7-48e6-b680-52e99f16bcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:11.568528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e80cf-07d7-48e6-b680-52e99f16bcf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:11.568883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e80cf-07d7-48e6-b680-52e99f16bcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:11.572655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e80cf-07d7-48e6-b680-52e99f16bcf1 admin admin] PATH_INFO: `/v3/projects/12ad6488e7d94080abbff92aa863b639/users/00497cacd228474eb459ba64ebd6ff86/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:11.588281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e80cf-07d7-48e6-b680-52e99f16bcf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12ad6488e7d94080abbff92aa863b639, user_id=00497cacd228474eb459ba64ebd6ff86, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:11.592230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e80cf-07d7-48e6-b680-52e99f16bcf1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:11.726167 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3990/7950] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:35:11 2022] PUT /identity/v3/projects/12ad6488e7d94080abbff92aa863b639/users/00497cacd228474eb459ba64ebd6ff86/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:11.737998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb3808-5f12-47cc-b82b-2077fdc21584 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:11.738554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb3808-5f12-47cc-b82b-2077fdc21584 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:11.738554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb3808-5f12-47cc-b82b-2077fdc21584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:11.772183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ecdb3808-5f12-47cc-b82b-2077fdc21584 None None] MFA Rules not processed for user `00497cacd228474eb459ba64ebd6ff86`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:11.773957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ecdb3808-5f12-47cc-b82b-2077fdc21584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:11.848035 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3961/7951] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:35:11.858563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e80cf-07d7-48e6-b680-52e99f16bcf1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:11.860296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d60e80cf-07d7-48e6-b680-52e99f16bcf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:11.931115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e80cf-07d7-48e6-b680-52e99f16bcf1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:11.935653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eab5b0-151c-4efd-81a6-d31be3c393e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:11.936417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab5b0-151c-4efd-81a6-d31be3c393e7 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:11.936802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab5b0-151c-4efd-81a6-d31be3c393e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:11.937172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eab5b0-151c-4efd-81a6-d31be3c393e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:11.949362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eab5b0-151c-4efd-81a6-d31be3c393e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:11.949362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eab5b0-151c-4efd-81a6-d31be3c393e7 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:11.958110 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3991/7952] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:35:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:35:16.667645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d8efd-dcbe-4e82-ad32-c621bc8aacc6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:16.668377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d8efd-dcbe-4e82-ad32-c621bc8aacc6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:16.668922 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275d8efd-dcbe-4e82-ad32-c621bc8aacc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:16.691471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-275d8efd-dcbe-4e82-ad32-c621bc8aacc6 None None] MFA Rules not processed for user `b16d0e791edd4f119e646e7ced74bf74`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:16.693365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-275d8efd-dcbe-4e82-ad32-c621bc8aacc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:16.745854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:16.805672 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3962/7953] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:35:18.826131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce5378-07e1-4cfa-ad2c-94089676a167 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:18.827021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce5378-07e1-4cfa-ad2c-94089676a167 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:18.827359 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce5378-07e1-4cfa-ad2c-94089676a167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:18.867142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-84ce5378-07e1-4cfa-ad2c-94089676a167 None None] MFA Rules not processed for user `bce779fb1eb547a5a077c6f1ac48d800`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:18.869465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-84ce5378-07e1-4cfa-ad2c-94089676a167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:18.972126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:18.997997 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3992/7954] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:35:19.010723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d8efd-dcbe-4e82-ad32-c621bc8aacc6 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:19.013276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-275d8efd-dcbe-4e82-ad32-c621bc8aacc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:19.129226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275d8efd-dcbe-4e82-ad32-c621bc8aacc6 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:19.137057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eae69c-58a3-4f0e-bb78-fb6d6efbb51a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:19.137946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eae69c-58a3-4f0e-bb78-fb6d6efbb51a service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:19.138368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eae69c-58a3-4f0e-bb78-fb6d6efbb51a service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:19.138720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eae69c-58a3-4f0e-bb78-fb6d6efbb51a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:19.141062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f2eae69c-58a3-4f0e-bb78-fb6d6efbb51a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:19.217684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eae69c-58a3-4f0e-bb78-fb6d6efbb51a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:19.219616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eae69c-58a3-4f0e-bb78-fb6d6efbb51a service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:19.231286 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3963/7955] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:35:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:35:24.559761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ce587-4a38-44a0-90d6-d1cbe687d295 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:24.560873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ce587-4a38-44a0-90d6-d1cbe687d295 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:24.561316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ce587-4a38-44a0-90d6-d1cbe687d295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:24.582823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-734ce587-4a38-44a0-90d6-d1cbe687d295 None None] MFA Rules not processed for user `13249a1c05fa43bd917b36aafb9db0d9`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:24.584314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-734ce587-4a38-44a0-90d6-d1cbe687d295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:24.599185 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3993/7956] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:35:29.227127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=69 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:32.637889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eae69c-58a3-4f0e-bb78-fb6d6efbb51a service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:32.641326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f2eae69c-58a3-4f0e-bb78-fb6d6efbb51a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:32.794590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eae69c-58a3-4f0e-bb78-fb6d6efbb51a service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:32.800669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd6b0c-bb2c-4b52-b2b9-524a4e6f434d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:32.801377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd6b0c-bb2c-4b52-b2b9-524a4e6f434d service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:32.801724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd6b0c-bb2c-4b52-b2b9-524a4e6f434d service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:32.811079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd6b0c-bb2c-4b52-b2b9-524a4e6f434d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:32.816752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8fbd6b0c-bb2c-4b52-b2b9-524a4e6f434d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:32.836259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd6b0c-bb2c-4b52-b2b9-524a4e6f434d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:32.837943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd6b0c-bb2c-4b52-b2b9-524a4e6f434d service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:32.852980 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3964/7957] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:35:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:35:34.575060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:35.523259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ce587-4a38-44a0-90d6-d1cbe687d295 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:35.588089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ce587-4a38-44a0-90d6-d1cbe687d295 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:35.594637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b4ebe-4ee0-44cd-925c-c58029f962aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:35.595331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b4ebe-4ee0-44cd-925c-c58029f962aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:35.595619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b4ebe-4ee0-44cd-925c-c58029f962aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:35.595983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b4ebe-4ee0-44cd-925c-c58029f962aa admin admin] PATH_INFO: `/v3/users/00497cacd228474eb459ba64ebd6ff86` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:35.597280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b4ebe-4ee0-44cd-925c-c58029f962aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=00497cacd228474eb459ba64ebd6ff86)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:35.599261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b4ebe-4ee0-44cd-925c-c58029f962aa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:35.810560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a05b4ebe-4ee0-44cd-925c-c58029f962aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00497cacd228474eb459ba64ebd6ff86'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:35.818780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a05b4ebe-4ee0-44cd-925c-c58029f962aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00497cacd228474eb459ba64ebd6ff86'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:35.819694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a05b4ebe-4ee0-44cd-925c-c58029f962aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00497cacd228474eb459ba64ebd6ff86'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:35.820375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a05b4ebe-4ee0-44cd-925c-c58029f962aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00497cacd228474eb459ba64ebd6ff86'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:35.938916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a05b4ebe-4ee0-44cd-925c-c58029f962aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00497cacd228474eb459ba64ebd6ff86'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:35.958945 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3994/7958] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:35:35 2022] DELETE /identity/v3/users/00497cacd228474eb459ba64ebd6ff86 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:35.963562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ab732-b605-44de-b31e-763d849ed4e3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:35.963957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ab732-b605-44de-b31e-763d849ed4e3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:35.964132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ab732-b605-44de-b31e-763d849ed4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:35.982262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c60ab732-b605-44de-b31e-763d849ed4e3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:35.983462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c60ab732-b605-44de-b31e-763d849ed4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:36.042307 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3965/7959] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:35:36.052741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b4ebe-4ee0-44cd-925c-c58029f962aa admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:36.054183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a05b4ebe-4ee0-44cd-925c-c58029f962aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:36.117292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b4ebe-4ee0-44cd-925c-c58029f962aa admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:36.121960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379fafc-b02e-4ee5-a9dc-481a2e508bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:36.129876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379fafc-b02e-4ee5-a9dc-481a2e508bae service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:36.129876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379fafc-b02e-4ee5-a9dc-481a2e508bae service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:36.129876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379fafc-b02e-4ee5-a9dc-481a2e508bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:36.132982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379fafc-b02e-4ee5-a9dc-481a2e508bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:36.134513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379fafc-b02e-4ee5-a9dc-481a2e508bae service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:36.144803 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3995/7960] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:35:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:35:36.374437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ab732-b605-44de-b31e-763d849ed4e3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:36.380634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c60ab732-b605-44de-b31e-763d849ed4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:36.388285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ab732-b605-44de-b31e-763d849ed4e3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:36.393079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57dd658-3322-4ede-bcb8-daba805e8e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:36.393992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dd658-3322-4ede-bcb8-daba805e8e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:36.394364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dd658-3322-4ede-bcb8-daba805e8e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:36.394637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dd658-3322-4ede-bcb8-daba805e8e21 admin admin] PATH_INFO: `/v3/projects/12ad6488e7d94080abbff92aa863b639` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:36.396324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57dd658-3322-4ede-bcb8-daba805e8e21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12ad6488e7d94080abbff92aa863b639)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:36.398305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57dd658-3322-4ede-bcb8-daba805e8e21 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:36.519080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a57dd658-3322-4ede-bcb8-daba805e8e21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12ad6488e7d94080abbff92aa863b639', 'request_id': 'req-a57dd658-3322-4ede-bcb8-daba805e8e21'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:36.525713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a57dd658-3322-4ede-bcb8-daba805e8e21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12ad6488e7d94080abbff92aa863b639', 'request_id': 'req-a57dd658-3322-4ede-bcb8-daba805e8e21'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:36.636093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a57dd658-3322-4ede-bcb8-daba805e8e21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12ad6488e7d94080abbff92aa863b639', 'request_id': 'req-a57dd658-3322-4ede-bcb8-daba805e8e21'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:36.649536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a57dd658-3322-4ede-bcb8-daba805e8e21 admin admin] The token cache is being invalidate because project 12ad6488e7d94080abbff92aa863b639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:35:36.649827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a57dd658-3322-4ede-bcb8-daba805e8e21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:36.650611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a57dd658-3322-4ede-bcb8-daba805e8e21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:35:36.652253 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3966/7961] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:35:36 2022] DELETE /identity/v3/projects/12ad6488e7d94080abbff92aa863b639 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:36.657051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce6c88-7854-439c-bca6-41e326d249c1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:36.657537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce6c88-7854-439c-bca6-41e326d249c1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:36.657649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce6c88-7854-439c-bca6-41e326d249c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:36.675353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-76ce6c88-7854-439c-bca6-41e326d249c1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:36.676566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-76ce6c88-7854-439c-bca6-41e326d249c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:36.694809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:36.735822 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3996/7962] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:35:36.743747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6537b-2b90-4b18-b28c-5725d5398d0c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:36.744041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6537b-2b90-4b18-b28c-5725d5398d0c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:36.744319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6537b-2b90-4b18-b28c-5725d5398d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:36.762817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-57b6537b-2b90-4b18-b28c-5725d5398d0c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:36.764002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-57b6537b-2b90-4b18-b28c-5725d5398d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:36.772567 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3967/7963] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:35:36.776770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce6c88-7854-439c-bca6-41e326d249c1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:36.786595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce6c88-7854-439c-bca6-41e326d249c1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:36.791469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a0ecc-6d3b-47a4-9c32-253becdf790f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:36.792133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a0ecc-6d3b-47a4-9c32-253becdf790f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:36.792416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a0ecc-6d3b-47a4-9c32-253becdf790f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:36.792696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a0ecc-6d3b-47a4-9c32-253becdf790f admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:36.793422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a0ecc-6d3b-47a4-9c32-253becdf790f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:35:36.793731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a0ecc-6d3b-47a4-9c32-253becdf790f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:36.795353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a0ecc-6d3b-47a4-9c32-253becdf790f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:36.806917 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3997/7964] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:35:36.811211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6537b-2b90-4b18-b28c-5725d5398d0c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:36.821167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6537b-2b90-4b18-b28c-5725d5398d0c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:36.825936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97263c69-ba2a-4b44-81b3-e336b8854885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:36.828308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97263c69-ba2a-4b44-81b3-e336b8854885 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:36.828308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97263c69-ba2a-4b44-81b3-e336b8854885 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:36.828308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97263c69-ba2a-4b44-81b3-e336b8854885 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:36.828308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97263c69-ba2a-4b44-81b3-e336b8854885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:36.829938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97263c69-ba2a-4b44-81b3-e336b8854885 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:36.833699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:37.004398 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3968/7965] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:35:36 2022] POST /identity/v3/projects => generated 373 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:35:37.008981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a0ecc-6d3b-47a4-9c32-253becdf790f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:37.019281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a0ecc-6d3b-47a4-9c32-253becdf790f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:37.076080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04325d-2788-40de-a16f-922ef49f5aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:37.076762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04325d-2788-40de-a16f-922ef49f5aac admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:37.077049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04325d-2788-40de-a16f-922ef49f5aac admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:37.077333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04325d-2788-40de-a16f-922ef49f5aac admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:37.078082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04325d-2788-40de-a16f-922ef49f5aac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:37.080012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f04325d-2788-40de-a16f-922ef49f5aac admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:37.189698 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3998/7966] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:35:37 2022] POST /identity/v3/users => generated 287 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:35:37.194516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97263c69-ba2a-4b44-81b3-e336b8854885 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:37.204892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97263c69-ba2a-4b44-81b3-e336b8854885 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:37.210835 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bc0c5-e200-422a-837e-e996ed026177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:37.211503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bc0c5-e200-422a-837e-e996ed026177 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:37.211781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bc0c5-e200-422a-837e-e996ed026177 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:37.212064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5bc0c5-e200-422a-837e-e996ed026177 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:37.212824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5bc0c5-e200-422a-837e-e996ed026177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:37.214499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5bc0c5-e200-422a-837e-e996ed026177 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:37.224628 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3969/7967] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:35:37.228801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04325d-2788-40de-a16f-922ef49f5aac admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:37.239396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f04325d-2788-40de-a16f-922ef49f5aac admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:37.244216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373eae86-59d3-499e-9bd2-efb3e8f34c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:37.244896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eae86-59d3-499e-9bd2-efb3e8f34c50 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:37.245184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eae86-59d3-499e-9bd2-efb3e8f34c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:37.245465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eae86-59d3-499e-9bd2-efb3e8f34c50 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:37.246272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373eae86-59d3-499e-9bd2-efb3e8f34c50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:37.247909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373eae86-59d3-499e-9bd2-efb3e8f34c50 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:37.259785 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 3999/7968] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:35:37.262795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bc0c5-e200-422a-837e-e996ed026177 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:37.273202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5bc0c5-e200-422a-837e-e996ed026177 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:37.278531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6579a-b488-4c1f-8e85-95c914424218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:37.280808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6579a-b488-4c1f-8e85-95c914424218 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:37.280808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6579a-b488-4c1f-8e85-95c914424218 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:37.280808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6579a-b488-4c1f-8e85-95c914424218 admin admin] PATH_INFO: `/v3/projects/28f8fcc4afe8471c94db58bef7526459/users/b8372df2880a483e8f1b2b1b667ddf04/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:37.294664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6579a-b488-4c1f-8e85-95c914424218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28f8fcc4afe8471c94db58bef7526459, user_id=b8372df2880a483e8f1b2b1b667ddf04, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:37.297340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6579a-b488-4c1f-8e85-95c914424218 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:37.384952 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3970/7969] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:35:37 2022] PUT /identity/v3/projects/28f8fcc4afe8471c94db58bef7526459/users/b8372df2880a483e8f1b2b1b667ddf04/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:37.391564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efa17f-8224-4933-8448-5055ebad7eac None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:37.391847 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efa17f-8224-4933-8448-5055ebad7eac None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:37.392134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70efa17f-8224-4933-8448-5055ebad7eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:37.410159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-70efa17f-8224-4933-8448-5055ebad7eac None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:37.411878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-70efa17f-8224-4933-8448-5055ebad7eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:37.470574 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4000/7970] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:35:37.480649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6579a-b488-4c1f-8e85-95c914424218 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:37.482505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8bf6579a-b488-4c1f-8e85-95c914424218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:37.547592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6579a-b488-4c1f-8e85-95c914424218 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:37.551926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c258a1f-7e32-4b8f-bc49-fd8a254e323c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:37.552576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c258a1f-7e32-4b8f-bc49-fd8a254e323c service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:37.552895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c258a1f-7e32-4b8f-bc49-fd8a254e323c service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:37.553176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c258a1f-7e32-4b8f-bc49-fd8a254e323c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:37.562335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c258a1f-7e32-4b8f-bc49-fd8a254e323c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:37.564964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c258a1f-7e32-4b8f-bc49-fd8a254e323c service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:37.574027 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3971/7971] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:35:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:35:38.866477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a9a92-cec5-4614-8268-67da3289ceac None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:38.867027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a9a92-cec5-4614-8268-67da3289ceac None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:38.867136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a9a92-cec5-4614-8268-67da3289ceac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:38.898606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-088a9a92-cec5-4614-8268-67da3289ceac None None] MFA Rules not processed for user `b8372df2880a483e8f1b2b1b667ddf04`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:38.902599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-088a9a92-cec5-4614-8268-67da3289ceac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:38.920155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:38.973906 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4001/7972] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:35:38.978580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c258a1f-7e32-4b8f-bc49-fd8a254e323c service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:38.990613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c258a1f-7e32-4b8f-bc49-fd8a254e323c service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:38.995991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7d025-440a-4324-b7bc-c5aae9669f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:38.997345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7d025-440a-4324-b7bc-c5aae9669f67 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:38.997345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7d025-440a-4324-b7bc-c5aae9669f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:38.997345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e7d025-440a-4324-b7bc-c5aae9669f67 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:38.998351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7d025-440a-4324-b7bc-c5aae9669f67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:39.000727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e7d025-440a-4324-b7bc-c5aae9669f67 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:39.118181 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3972/7973] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:35:38 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:35:39.122287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a9a92-cec5-4614-8268-67da3289ceac None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:39.133500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a9a92-cec5-4614-8268-67da3289ceac None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:39.188684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6e3f7-7256-4b9e-8560-ba8d860eeb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:39.189284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6e3f7-7256-4b9e-8560-ba8d860eeb95 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:39.189568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6e3f7-7256-4b9e-8560-ba8d860eeb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:39.189868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d6e3f7-7256-4b9e-8560-ba8d860eeb95 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:39.190587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6e3f7-7256-4b9e-8560-ba8d860eeb95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:39.192574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d6e3f7-7256-4b9e-8560-ba8d860eeb95 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:39.289664 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4002/7974] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:35:39 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:35:39.293729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7d025-440a-4324-b7bc-c5aae9669f67 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:39.304638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e7d025-440a-4324-b7bc-c5aae9669f67 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:39.317468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e55c8f-2db1-45bf-89dc-d3275deeed43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:39.317468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e55c8f-2db1-45bf-89dc-d3275deeed43 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:39.317468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e55c8f-2db1-45bf-89dc-d3275deeed43 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:39.317468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e55c8f-2db1-45bf-89dc-d3275deeed43 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:39.317956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e55c8f-2db1-45bf-89dc-d3275deeed43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:39.317956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e55c8f-2db1-45bf-89dc-d3275deeed43 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:39.324213 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3973/7975] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:35:39.328456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6e3f7-7256-4b9e-8560-ba8d860eeb95 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:39.339388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d6e3f7-7256-4b9e-8560-ba8d860eeb95 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:39.346250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1f980-a7da-4ab9-8833-af99589e966c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:39.346250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1f980-a7da-4ab9-8833-af99589e966c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:39.346250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1f980-a7da-4ab9-8833-af99589e966c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:39.346591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1f980-a7da-4ab9-8833-af99589e966c admin admin] PATH_INFO: `/v3/projects/78570e37130442ada37177bb179a68bf/users/12a55a22ce4a432c8fa818b9ca61262c/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:39.361436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1f980-a7da-4ab9-8833-af99589e966c admin admin] RBAC: Authorizing `identity:create_grant(project_id=78570e37130442ada37177bb179a68bf, user_id=12a55a22ce4a432c8fa818b9ca61262c, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:39.364183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1f980-a7da-4ab9-8833-af99589e966c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:39.448866 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4003/7976] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:35:39 2022] PUT /identity/v3/projects/78570e37130442ada37177bb179a68bf/users/12a55a22ce4a432c8fa818b9ca61262c/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:39.452805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e55c8f-2db1-45bf-89dc-d3275deeed43 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:39.463215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e55c8f-2db1-45bf-89dc-d3275deeed43 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:39.519153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418808e-225c-4866-b657-83ba039c190e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:39.519816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418808e-225c-4866-b657-83ba039c190e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:39.520640 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418808e-225c-4866-b657-83ba039c190e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:39.520913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418808e-225c-4866-b657-83ba039c190e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:39.521635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418808e-225c-4866-b657-83ba039c190e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:39.523275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418808e-225c-4866-b657-83ba039c190e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:39.532852 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3974/7977] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:35:39 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:35:39.537561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1f980-a7da-4ab9-8833-af99589e966c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:39.548233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1f980-a7da-4ab9-8833-af99589e966c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:39.552958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189a0cd-f42c-4626-8ff5-bb8118bd2ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:39.553858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189a0cd-f42c-4626-8ff5-bb8118bd2ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:39.554156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189a0cd-f42c-4626-8ff5-bb8118bd2ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:39.554434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189a0cd-f42c-4626-8ff5-bb8118bd2ac8 admin admin] PATH_INFO: `/v3/projects/78570e37130442ada37177bb179a68bf/users/12a55a22ce4a432c8fa818b9ca61262c/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:39.556433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189a0cd-f42c-4626-8ff5-bb8118bd2ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78570e37130442ada37177bb179a68bf, user_id=12a55a22ce4a432c8fa818b9ca61262c, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:39.559250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189a0cd-f42c-4626-8ff5-bb8118bd2ac8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:39.570704 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4004/7978] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:35:39 2022] PUT /identity/v3/projects/78570e37130442ada37177bb179a68bf/users/12a55a22ce4a432c8fa818b9ca61262c/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:35:40.722428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418808e-225c-4866-b657-83ba039c190e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:40.722428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2418808e-225c-4866-b657-83ba039c190e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:40.745394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:40.782127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418808e-225c-4866-b657-83ba039c190e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:40.833499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07de2d-8bca-45cf-b6bb-e77a837c3ee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:40.834927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07de2d-8bca-45cf-b6bb-e77a837c3ee8 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:40.835275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07de2d-8bca-45cf-b6bb-e77a837c3ee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:40.837353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07de2d-8bca-45cf-b6bb-e77a837c3ee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:40.838164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5d07de2d-8bca-45cf-b6bb-e77a837c3ee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:40.906245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07de2d-8bca-45cf-b6bb-e77a837c3ee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:40.916720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07de2d-8bca-45cf-b6bb-e77a837c3ee8 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:40.920870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78285c75-3f5f-4e32-9d64-decafd678a40 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:40.920870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78285c75-3f5f-4e32-9d64-decafd678a40 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:40.920870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78285c75-3f5f-4e32-9d64-decafd678a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:40.921545 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3975/7979] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:35:40 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:35:40.948742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-78285c75-3f5f-4e32-9d64-decafd678a40 None None] MFA Rules not processed for user `12a55a22ce4a432c8fa818b9ca61262c`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:40.950222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-78285c75-3f5f-4e32-9d64-decafd678a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:41.024241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:41.033759 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4005/7980] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:35:41.046323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2df5f-4acf-44c4-b392-a2ef94ff04a7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:41.046813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2df5f-4acf-44c4-b392-a2ef94ff04a7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:41.049228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2df5f-4acf-44c4-b392-a2ef94ff04a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:41.068077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e7f2df5f-4acf-44c4-b392-a2ef94ff04a7 None None] MFA Rules not processed for user `b8372df2880a483e8f1b2b1b667ddf04`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:35:41.069570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e7f2df5f-4acf-44c4-b392-a2ef94ff04a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:41.131429 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3976/7981] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:35:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:35:41.142291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78285c75-3f5f-4e32-9d64-decafd678a40 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:41.143820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-78285c75-3f5f-4e32-9d64-decafd678a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:41.229750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78285c75-3f5f-4e32-9d64-decafd678a40 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:41.235948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e722e-ba48-422f-a604-a4c89c8c865a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:41.236796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e722e-ba48-422f-a604-a4c89c8c865a service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:41.237086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e722e-ba48-422f-a604-a4c89c8c865a service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:41.237368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e722e-ba48-422f-a604-a4c89c8c865a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:41.245830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e722e-ba48-422f-a604-a4c89c8c865a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:41.247492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e722e-ba48-422f-a604-a4c89c8c865a service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:41.257446 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4006/7982] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:35:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:35:51.123947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:51.252565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=36 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:57.436663 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2df5f-4acf-44c4-b392-a2ef94ff04a7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:35:57.438119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e7f2df5f-4acf-44c4-b392-a2ef94ff04a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:35:57.499828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2df5f-4acf-44c4-b392-a2ef94ff04a7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:35:57.503481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc67dd0-edff-40fb-9496-4506a21de668 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:35:57.506442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc67dd0-edff-40fb-9496-4506a21de668 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:35:57.506442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc67dd0-edff-40fb-9496-4506a21de668 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:35:57.506442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc67dd0-edff-40fb-9496-4506a21de668 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:35:57.514847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc67dd0-edff-40fb-9496-4506a21de668 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:35:57.516362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc67dd0-edff-40fb-9496-4506a21de668 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:35:57.526825 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3977/7983] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:35:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:36:00.136542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e722e-ba48-422f-a604-a4c89c8c865a service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:00.138032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-299e722e-ba48-422f-a604-a4c89c8c865a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:00.211731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e722e-ba48-422f-a604-a4c89c8c865a service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:00.217729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac46b57-63bf-4138-be16-aa4064a39102 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:00.218417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac46b57-63bf-4138-be16-aa4064a39102 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:00.218701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac46b57-63bf-4138-be16-aa4064a39102 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:00.218996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac46b57-63bf-4138-be16-aa4064a39102 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:00.220757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3ac46b57-63bf-4138-be16-aa4064a39102 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:00.238050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac46b57-63bf-4138-be16-aa4064a39102 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:00.239676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac46b57-63bf-4138-be16-aa4064a39102 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:00.252737 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4007/7984] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:36:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:36:05.276081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc67dd0-edff-40fb-9496-4506a21de668 service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:05.289514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc67dd0-edff-40fb-9496-4506a21de668 service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:05.294711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141e6df-9dce-4b61-b136-54366f29ba50 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:05.295361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141e6df-9dce-4b61-b136-54366f29ba50 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:05.295668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141e6df-9dce-4b61-b136-54366f29ba50 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:05.298185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141e6df-9dce-4b61-b136-54366f29ba50 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:05.298185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6141e6df-9dce-4b61-b136-54366f29ba50 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:05.312790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141e6df-9dce-4b61-b136-54366f29ba50 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:05.314449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141e6df-9dce-4b61-b136-54366f29ba50 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:05.327554 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3978/7985] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:36:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:36:10.245869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=32 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:36:15.320292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=32 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:36:17.554514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5121f3d-602d-4431-950f-bf3d413cc979 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:17.555044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5121f3d-602d-4431-950f-bf3d413cc979 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:17.555437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5121f3d-602d-4431-950f-bf3d413cc979 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:17.557064 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4008/7986] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:36:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:36:17.562573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351ff56-ed3f-4992-8e6e-7940e3643dce None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:17.563022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351ff56-ed3f-4992-8e6e-7940e3643dce None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:17.563122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3351ff56-ed3f-4992-8e6e-7940e3643dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:17.629162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3351ff56-ed3f-4992-8e6e-7940e3643dce None None] MFA Rules not processed for user `1b2a9ebcd3f84b758322a80c6e590de0`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:36:17.630417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3351ff56-ed3f-4992-8e6e-7940e3643dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:17.637887 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3979/7987] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:36:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:36:17.650206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5121f3d-602d-4431-950f-bf3d413cc979 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:17.664617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5121f3d-602d-4431-950f-bf3d413cc979 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:17.672361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2bef6-27d6-46d7-aab4-ecf311380680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:17.673018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2bef6-27d6-46d7-aab4-ecf311380680 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:17.673312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2bef6-27d6-46d7-aab4-ecf311380680 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:17.673583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2bef6-27d6-46d7-aab4-ecf311380680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:17.683587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2bef6-27d6-46d7-aab4-ecf311380680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:17.687004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2bef6-27d6-46d7-aab4-ecf311380680 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:17.696555 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4009/7988] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:36:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:36:22.460306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351ff56-ed3f-4992-8e6e-7940e3643dce None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:22.472593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3351ff56-ed3f-4992-8e6e-7940e3643dce None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:22.476393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5956716-7068-4dc9-b581-76261928e284 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:22.477020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5956716-7068-4dc9-b581-76261928e284 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:22.477302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5956716-7068-4dc9-b581-76261928e284 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:22.477578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5956716-7068-4dc9-b581-76261928e284 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:22.479457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f5956716-7068-4dc9-b581-76261928e284 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:22.542053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5956716-7068-4dc9-b581-76261928e284 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:22.543519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5956716-7068-4dc9-b581-76261928e284 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:22.552926 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3980/7989] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:36:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:36:24.821088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cae77f-4196-45f4-a8b5-62cde528ecc3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:24.821600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cae77f-4196-45f4-a8b5-62cde528ecc3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:24.821882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cae77f-4196-45f4-a8b5-62cde528ecc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:24.840894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-02cae77f-4196-45f4-a8b5-62cde528ecc3 None None] MFA Rules not processed for user `b16d0e791edd4f119e646e7ced74bf74`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:36:24.842766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-02cae77f-4196-45f4-a8b5-62cde528ecc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:24.897805 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4010/7990] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:36:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:36:25.581225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5956716-7068-4dc9-b581-76261928e284 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:25.598951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5956716-7068-4dc9-b581-76261928e284 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:25.602759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee4724-3307-41b6-bb2e-8aa865b78dfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:25.603442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee4724-3307-41b6-bb2e-8aa865b78dfd service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:25.603732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee4724-3307-41b6-bb2e-8aa865b78dfd service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:25.604014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee4724-3307-41b6-bb2e-8aa865b78dfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:25.605858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cbee4724-3307-41b6-bb2e-8aa865b78dfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:25.620282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee4724-3307-41b6-bb2e-8aa865b78dfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:25.621743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee4724-3307-41b6-bb2e-8aa865b78dfd service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:25.634331 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3981/7991] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:36:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:36:34.890696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=31 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:36:35.627801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=47 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:36:43.401816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cae77f-4196-45f4-a8b5-62cde528ecc3 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:43.414488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cae77f-4196-45f4-a8b5-62cde528ecc3 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:43.418376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e96f3d-7bec-4380-8469-f0e393d0257b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:43.419246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e96f3d-7bec-4380-8469-f0e393d0257b service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:43.419541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e96f3d-7bec-4380-8469-f0e393d0257b service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:43.419820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e96f3d-7bec-4380-8469-f0e393d0257b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:43.421624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-35e96f3d-7bec-4380-8469-f0e393d0257b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:43.487667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e96f3d-7bec-4380-8469-f0e393d0257b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:43.489107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e96f3d-7bec-4380-8469-f0e393d0257b service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:43.498829 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4011/7992] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:36:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:36:44.227878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee4724-3307-41b6-bb2e-8aa865b78dfd service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:44.229263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cbee4724-3307-41b6-bb2e-8aa865b78dfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:44.243829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee4724-3307-41b6-bb2e-8aa865b78dfd service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:44.247973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8187bb93-676d-4e2d-a573-34b08c63bd72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:44.248636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187bb93-676d-4e2d-a573-34b08c63bd72 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:44.248923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187bb93-676d-4e2d-a573-34b08c63bd72 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:44.249205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8187bb93-676d-4e2d-a573-34b08c63bd72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:44.252004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8187bb93-676d-4e2d-a573-34b08c63bd72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:44.316837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187bb93-676d-4e2d-a573-34b08c63bd72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:44.318446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8187bb93-676d-4e2d-a573-34b08c63bd72 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:44.329897 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3982/7993] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:36:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:36:51.279900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e96f3d-7bec-4380-8469-f0e393d0257b service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:51.282152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-35e96f3d-7bec-4380-8469-f0e393d0257b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:51.307321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e96f3d-7bec-4380-8469-f0e393d0257b service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:51.316186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799704f-dfac-4aec-8e07-0a1449243d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:51.316186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799704f-dfac-4aec-8e07-0a1449243d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:51.316186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799704f-dfac-4aec-8e07-0a1449243d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:51.316186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799704f-dfac-4aec-8e07-0a1449243d7a admin admin] PATH_INFO: `/v3/users/c93401a279c9403ca62627e829082968` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:51.316698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799704f-dfac-4aec-8e07-0a1449243d7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c93401a279c9403ca62627e829082968)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:51.318651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799704f-dfac-4aec-8e07-0a1449243d7a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:51.326164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc0912-5ffc-44b3-933a-0caecac7ab9e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:51.326658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc0912-5ffc-44b3-933a-0caecac7ab9e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:51.326802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc0912-5ffc-44b3-933a-0caecac7ab9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:51.344991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9dcc0912-5ffc-44b3-933a-0caecac7ab9e None None] MFA Rules not processed for user `8d033e99f9294e348b01c59a970931d5`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:36:51.346251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9dcc0912-5ffc-44b3-933a-0caecac7ab9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:51.353421 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3983/7994] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:36:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:36:51.590900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e799704f-dfac-4aec-8e07-0a1449243d7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c93401a279c9403ca62627e829082968'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:51.601067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e799704f-dfac-4aec-8e07-0a1449243d7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c93401a279c9403ca62627e829082968'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:51.601806 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e799704f-dfac-4aec-8e07-0a1449243d7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c93401a279c9403ca62627e829082968'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:51.602497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e799704f-dfac-4aec-8e07-0a1449243d7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c93401a279c9403ca62627e829082968'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:51.675554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e799704f-dfac-4aec-8e07-0a1449243d7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c93401a279c9403ca62627e829082968'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:51.697066 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4012/7995] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:36:51 2022] DELETE /identity/v3/users/c93401a279c9403ca62627e829082968 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:36:51.998151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc0912-5ffc-44b3-933a-0caecac7ab9e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:52.014738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc0912-5ffc-44b3-933a-0caecac7ab9e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:52.107676 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7045e9-afd7-44e7-b003-94d2687346d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:52.108654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7045e9-afd7-44e7-b003-94d2687346d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:52.108995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7045e9-afd7-44e7-b003-94d2687346d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:52.109281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7045e9-afd7-44e7-b003-94d2687346d8 admin admin] PATH_INFO: `/v3/projects/5241f87816134389b836a8eae1d53bd3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:52.111011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7045e9-afd7-44e7-b003-94d2687346d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5241f87816134389b836a8eae1d53bd3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:52.113067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7045e9-afd7-44e7-b003-94d2687346d8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:52.220566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ef7045e9-afd7-44e7-b003-94d2687346d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5241f87816134389b836a8eae1d53bd3', 'request_id': 'req-ef7045e9-afd7-44e7-b003-94d2687346d8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:52.226218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ef7045e9-afd7-44e7-b003-94d2687346d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5241f87816134389b836a8eae1d53bd3', 'request_id': 'req-ef7045e9-afd7-44e7-b003-94d2687346d8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:52.322304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ef7045e9-afd7-44e7-b003-94d2687346d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5241f87816134389b836a8eae1d53bd3', 'request_id': 'req-ef7045e9-afd7-44e7-b003-94d2687346d8'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:52.336082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ef7045e9-afd7-44e7-b003-94d2687346d8 admin admin] The token cache is being invalidate because project 5241f87816134389b836a8eae1d53bd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:36:52.336678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ef7045e9-afd7-44e7-b003-94d2687346d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:52.336979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ef7045e9-afd7-44e7-b003-94d2687346d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:52.338581 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3984/7996] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:36:51 2022] DELETE /identity/v3/projects/5241f87816134389b836a8eae1d53bd3 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:36:52.343385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799704f-dfac-4aec-8e07-0a1449243d7a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:52.344894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e799704f-dfac-4aec-8e07-0a1449243d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:52.362518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799704f-dfac-4aec-8e07-0a1449243d7a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:52.434822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e948791-a4e1-4684-9a76-93d0be7c0877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:52.435706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e948791-a4e1-4684-9a76-93d0be7c0877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:52.435994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e948791-a4e1-4684-9a76-93d0be7c0877 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:52.436307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e948791-a4e1-4684-9a76-93d0be7c0877 admin admin] PATH_INFO: `/v3/users/af49e980f06741acb92a7ef61c6f1034` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:52.437567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e948791-a4e1-4684-9a76-93d0be7c0877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af49e980f06741acb92a7ef61c6f1034)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:52.439588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e948791-a4e1-4684-9a76-93d0be7c0877 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:52.445646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:36:52.634795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1e948791-a4e1-4684-9a76-93d0be7c0877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af49e980f06741acb92a7ef61c6f1034'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:52.646183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1e948791-a4e1-4684-9a76-93d0be7c0877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af49e980f06741acb92a7ef61c6f1034'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:52.646940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1e948791-a4e1-4684-9a76-93d0be7c0877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af49e980f06741acb92a7ef61c6f1034'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:52.647567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1e948791-a4e1-4684-9a76-93d0be7c0877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af49e980f06741acb92a7ef61c6f1034'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:52.728457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-1e948791-a4e1-4684-9a76-93d0be7c0877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af49e980f06741acb92a7ef61c6f1034'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:52.749882 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4013/7997] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:36:52 2022] DELETE /identity/v3/users/af49e980f06741acb92a7ef61c6f1034 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:36:52.970935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7045e9-afd7-44e7-b003-94d2687346d8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:52.972395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ef7045e9-afd7-44e7-b003-94d2687346d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:52.985964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7045e9-afd7-44e7-b003-94d2687346d8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:53.005637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:36:53.044420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbec2bf-7014-40f4-b168-f13e98a17a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:53.045299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbec2bf-7014-40f4-b168-f13e98a17a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:53.045587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbec2bf-7014-40f4-b168-f13e98a17a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:53.045892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbec2bf-7014-40f4-b168-f13e98a17a57 admin admin] PATH_INFO: `/v3/projects/445b6918f25d420cb182ff4a8f1ef2d7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:53.047115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbec2bf-7014-40f4-b168-f13e98a17a57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=445b6918f25d420cb182ff4a8f1ef2d7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:53.049124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbec2bf-7014-40f4-b168-f13e98a17a57 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:53.166863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fcbec2bf-7014-40f4-b168-f13e98a17a57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '445b6918f25d420cb182ff4a8f1ef2d7', 'request_id': 'req-fcbec2bf-7014-40f4-b168-f13e98a17a57'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:53.172370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fcbec2bf-7014-40f4-b168-f13e98a17a57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '445b6918f25d420cb182ff4a8f1ef2d7', 'request_id': 'req-fcbec2bf-7014-40f4-b168-f13e98a17a57'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:53.275543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fcbec2bf-7014-40f4-b168-f13e98a17a57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '445b6918f25d420cb182ff4a8f1ef2d7', 'request_id': 'req-fcbec2bf-7014-40f4-b168-f13e98a17a57'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:53.288151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fcbec2bf-7014-40f4-b168-f13e98a17a57 admin admin] The token cache is being invalidate because project 445b6918f25d420cb182ff4a8f1ef2d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:36:53.288447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fcbec2bf-7014-40f4-b168-f13e98a17a57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:53.289082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fcbec2bf-7014-40f4-b168-f13e98a17a57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:53.290684 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3985/7998] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:36:52 2022] DELETE /identity/v3/projects/445b6918f25d420cb182ff4a8f1ef2d7 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:36:57.196432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e948791-a4e1-4684-9a76-93d0be7c0877 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:57.197892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1e948791-a4e1-4684-9a76-93d0be7c0877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:57.212034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e948791-a4e1-4684-9a76-93d0be7c0877 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:57.269235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07184060-a8c8-4b5b-9adc-1fbc8396d6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:57.270154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07184060-a8c8-4b5b-9adc-1fbc8396d6bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:57.270433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07184060-a8c8-4b5b-9adc-1fbc8396d6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:57.270709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07184060-a8c8-4b5b-9adc-1fbc8396d6bf admin admin] PATH_INFO: `/v3/users/b8372df2880a483e8f1b2b1b667ddf04` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:57.271891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07184060-a8c8-4b5b-9adc-1fbc8396d6bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8372df2880a483e8f1b2b1b667ddf04)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:57.273822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07184060-a8c8-4b5b-9adc-1fbc8396d6bf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:57.495002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-07184060-a8c8-4b5b-9adc-1fbc8396d6bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8372df2880a483e8f1b2b1b667ddf04'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:57.502694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-07184060-a8c8-4b5b-9adc-1fbc8396d6bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8372df2880a483e8f1b2b1b667ddf04'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:57.503361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-07184060-a8c8-4b5b-9adc-1fbc8396d6bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8372df2880a483e8f1b2b1b667ddf04'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:57.503985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-07184060-a8c8-4b5b-9adc-1fbc8396d6bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8372df2880a483e8f1b2b1b667ddf04'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:57.566153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-07184060-a8c8-4b5b-9adc-1fbc8396d6bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8372df2880a483e8f1b2b1b667ddf04'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:57.583880 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4014/7999] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:36:57 2022] DELETE /identity/v3/users/b8372df2880a483e8f1b2b1b667ddf04 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:36:57.913305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbec2bf-7014-40f4-b168-f13e98a17a57 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:57.914987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fcbec2bf-7014-40f4-b168-f13e98a17a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:57.935723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbec2bf-7014-40f4-b168-f13e98a17a57 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:57.994711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:57.995582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:57.995867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:57.996145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e admin admin] PATH_INFO: `/v3/projects/28f8fcc4afe8471c94db58bef7526459` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:57.997330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e admin admin] RBAC: Authorizing `identity:delete_project(project_id=28f8fcc4afe8471c94db58bef7526459)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:57.999294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:58.094052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28f8fcc4afe8471c94db58bef7526459', 'request_id': 'req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:58.098465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:36:58.100597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28f8fcc4afe8471c94db58bef7526459', 'request_id': 'req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:58.190489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28f8fcc4afe8471c94db58bef7526459', 'request_id': 'req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:58.201747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e admin admin] The token cache is being invalidate because project 28f8fcc4afe8471c94db58bef7526459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:36:58.203001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:58.203111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:58.205563 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3986/8000] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:36:57 2022] DELETE /identity/v3/projects/28f8fcc4afe8471c94db58bef7526459 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:36:58.209564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07184060-a8c8-4b5b-9adc-1fbc8396d6bf admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:58.211011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-07184060-a8c8-4b5b-9adc-1fbc8396d6bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:58.221199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:36:58.225136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07184060-a8c8-4b5b-9adc-1fbc8396d6bf admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:58.281689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b628f4c-85d8-4e21-90fe-27639ff35e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:58.282595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b628f4c-85d8-4e21-90fe-27639ff35e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:58.283557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b628f4c-85d8-4e21-90fe-27639ff35e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:58.283845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b628f4c-85d8-4e21-90fe-27639ff35e7d admin admin] PATH_INFO: `/v3/users/12a55a22ce4a432c8fa818b9ca61262c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:58.285037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b628f4c-85d8-4e21-90fe-27639ff35e7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=12a55a22ce4a432c8fa818b9ca61262c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:58.286990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b628f4c-85d8-4e21-90fe-27639ff35e7d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:58.466505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3b628f4c-85d8-4e21-90fe-27639ff35e7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12a55a22ce4a432c8fa818b9ca61262c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:58.474002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3b628f4c-85d8-4e21-90fe-27639ff35e7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12a55a22ce4a432c8fa818b9ca61262c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:58.474676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3b628f4c-85d8-4e21-90fe-27639ff35e7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12a55a22ce4a432c8fa818b9ca61262c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:58.475433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3b628f4c-85d8-4e21-90fe-27639ff35e7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12a55a22ce4a432c8fa818b9ca61262c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:58.534312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3b628f4c-85d8-4e21-90fe-27639ff35e7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12a55a22ce4a432c8fa818b9ca61262c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:58.553173 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4015/8001] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:36:58 2022] DELETE /identity/v3/users/12a55a22ce4a432c8fa818b9ca61262c => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:36:58.819642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:58.821116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:58.837641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b3b6e-73a3-4b89-bd9e-d884af79f25e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:58.896940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3282f-d25c-43e0-8736-81b15e2a2b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:58.897808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3282f-d25c-43e0-8736-81b15e2a2b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:58.898119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3282f-d25c-43e0-8736-81b15e2a2b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:58.898396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3282f-d25c-43e0-8736-81b15e2a2b17 admin admin] PATH_INFO: `/v3/projects/78570e37130442ada37177bb179a68bf` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:58.899572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3282f-d25c-43e0-8736-81b15e2a2b17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78570e37130442ada37177bb179a68bf)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:58.901512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3282f-d25c-43e0-8736-81b15e2a2b17 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:58.996840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-11e3282f-d25c-43e0-8736-81b15e2a2b17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78570e37130442ada37177bb179a68bf', 'request_id': 'req-11e3282f-d25c-43e0-8736-81b15e2a2b17'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:59.002781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-11e3282f-d25c-43e0-8736-81b15e2a2b17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78570e37130442ada37177bb179a68bf', 'request_id': 'req-11e3282f-d25c-43e0-8736-81b15e2a2b17'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:59.093859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-11e3282f-d25c-43e0-8736-81b15e2a2b17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78570e37130442ada37177bb179a68bf', 'request_id': 'req-11e3282f-d25c-43e0-8736-81b15e2a2b17'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:59.106931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-11e3282f-d25c-43e0-8736-81b15e2a2b17 admin admin] The token cache is being invalidate because project 78570e37130442ada37177bb179a68bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:36:59.107299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-11e3282f-d25c-43e0-8736-81b15e2a2b17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:59.108211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-11e3282f-d25c-43e0-8736-81b15e2a2b17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:36:59.110250 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3987/8002] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:36:58 2022] DELETE /identity/v3/projects/78570e37130442ada37177bb179a68bf => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:36:59.114925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55875198-f139-4287-b254-9cb3b6259c8f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:59.115221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55875198-f139-4287-b254-9cb3b6259c8f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:59.115536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55875198-f139-4287-b254-9cb3b6259c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:59.134866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-55875198-f139-4287-b254-9cb3b6259c8f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:36:59.136080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-55875198-f139-4287-b254-9cb3b6259c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:59.196620 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4016/8003] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:36:59.204269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec798c4-abf5-4bad-9349-499afe0ab28d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:59.204559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec798c4-abf5-4bad-9349-499afe0ab28d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:59.204841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec798c4-abf5-4bad-9349-499afe0ab28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:59.223762 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6ec798c4-abf5-4bad-9349-499afe0ab28d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:36:59.224950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6ec798c4-abf5-4bad-9349-499afe0ab28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:59.232839 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3988/8004] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:36:59.236902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55875198-f139-4287-b254-9cb3b6259c8f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:59.246611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55875198-f139-4287-b254-9cb3b6259c8f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:59.251191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a67f8-34f5-4178-83d7-d49491a9a9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:59.251870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a67f8-34f5-4178-83d7-d49491a9a9ad admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:59.252154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a67f8-34f5-4178-83d7-d49491a9a9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:59.252940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a67f8-34f5-4178-83d7-d49491a9a9ad admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:59.253696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a67f8-34f5-4178-83d7-d49491a9a9ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:36:59.254216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a67f8-34f5-4178-83d7-d49491a9a9ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:59.255845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a67f8-34f5-4178-83d7-d49491a9a9ad admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:59.267427 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4017/8005] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:36:59.276854 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec798c4-abf5-4bad-9349-499afe0ab28d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:59.286432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec798c4-abf5-4bad-9349-499afe0ab28d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:59.291242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66185713-8b7d-4e92-91cb-6fe2fd79af75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:59.291893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66185713-8b7d-4e92-91cb-6fe2fd79af75 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:59.292246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66185713-8b7d-4e92-91cb-6fe2fd79af75 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:59.292490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66185713-8b7d-4e92-91cb-6fe2fd79af75 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:59.293177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66185713-8b7d-4e92-91cb-6fe2fd79af75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:59.295133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66185713-8b7d-4e92-91cb-6fe2fd79af75 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:59.404051 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3989/8006] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:36:59 2022] POST /identity/v3/projects => generated 385 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:36:59.407939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a67f8-34f5-4178-83d7-d49491a9a9ad admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:59.418992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a67f8-34f5-4178-83d7-d49491a9a9ad admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:59.428108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:36:59.476354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222620b9-3f99-48aa-b860-89939ea3c330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:59.477049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222620b9-3f99-48aa-b860-89939ea3c330 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:59.477337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222620b9-3f99-48aa-b860-89939ea3c330 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:59.477613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222620b9-3f99-48aa-b860-89939ea3c330 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:59.478337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222620b9-3f99-48aa-b860-89939ea3c330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:59.480342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222620b9-3f99-48aa-b860-89939ea3c330 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:59.613524 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4018/8007] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:36:59 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:36:59.617039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66185713-8b7d-4e92-91cb-6fe2fd79af75 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:59.627367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66185713-8b7d-4e92-91cb-6fe2fd79af75 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:59.631993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74410b-221d-4dda-84c9-3f5cedcb678b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:59.633614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74410b-221d-4dda-84c9-3f5cedcb678b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:59.633614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74410b-221d-4dda-84c9-3f5cedcb678b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:59.633614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74410b-221d-4dda-84c9-3f5cedcb678b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:59.633978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74410b-221d-4dda-84c9-3f5cedcb678b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:59.635574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74410b-221d-4dda-84c9-3f5cedcb678b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:59.646576 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3990/8008] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:36:59.650254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222620b9-3f99-48aa-b860-89939ea3c330 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:59.661123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222620b9-3f99-48aa-b860-89939ea3c330 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:59.665921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3597cab-f6ec-4edb-8ada-85a10b0a70cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:59.666583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3597cab-f6ec-4edb-8ada-85a10b0a70cb admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:59.666897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3597cab-f6ec-4edb-8ada-85a10b0a70cb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:59.667196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3597cab-f6ec-4edb-8ada-85a10b0a70cb admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:59.667948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3597cab-f6ec-4edb-8ada-85a10b0a70cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:59.669560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3597cab-f6ec-4edb-8ada-85a10b0a70cb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:59.679589 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4019/8009] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:36:59.684626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74410b-221d-4dda-84c9-3f5cedcb678b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:59.694721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74410b-221d-4dda-84c9-3f5cedcb678b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:59.699412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32220ab8-7f05-4fd4-861f-13fb8e9bba8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:59.700291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32220ab8-7f05-4fd4-861f-13fb8e9bba8e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:59.700582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32220ab8-7f05-4fd4-861f-13fb8e9bba8e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:59.700865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32220ab8-7f05-4fd4-861f-13fb8e9bba8e admin admin] PATH_INFO: `/v3/projects/c2ffd00c53e444ae84569429b4a04f44/users/55d6461fd29b452d98cfcda3644a6902/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:59.716184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32220ab8-7f05-4fd4-861f-13fb8e9bba8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ffd00c53e444ae84569429b4a04f44, user_id=55d6461fd29b452d98cfcda3644a6902, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:36:59.718929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32220ab8-7f05-4fd4-861f-13fb8e9bba8e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:36:59.813825 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3991/8010] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:36:59 2022] PUT /identity/v3/projects/c2ffd00c53e444ae84569429b4a04f44/users/55d6461fd29b452d98cfcda3644a6902/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:36:59.824443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ee864-afef-4106-9a00-34f70fc18122 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:59.824749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ee864-afef-4106-9a00-34f70fc18122 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:59.825037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ee864-afef-4106-9a00-34f70fc18122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:36:59.853429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-311ee864-afef-4106-9a00-34f70fc18122 None None] MFA Rules not processed for user `55d6461fd29b452d98cfcda3644a6902`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:36:59.854712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-311ee864-afef-4106-9a00-34f70fc18122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:59.918561 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4020/8011] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:36:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:36:59.929415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32220ab8-7f05-4fd4-861f-13fb8e9bba8e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:36:59.931144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-32220ab8-7f05-4fd4-861f-13fb8e9bba8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:36:59.942541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:36:59.990073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32220ab8-7f05-4fd4-861f-13fb8e9bba8e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:36:59.994530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a9c21-4164-4229-8ab9-1ac9f41ded61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:36:59.996057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a9c21-4164-4229-8ab9-1ac9f41ded61 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:36:59.996401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a9c21-4164-4229-8ab9-1ac9f41ded61 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:36:59.996680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a9c21-4164-4229-8ab9-1ac9f41ded61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:00.006359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a9c21-4164-4229-8ab9-1ac9f41ded61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:00.007868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a9c21-4164-4229-8ab9-1ac9f41ded61 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:00.016881 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3992/8012] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:36:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:37:00.036346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ee864-afef-4106-9a00-34f70fc18122 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:00.104910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ee864-afef-4106-9a00-34f70fc18122 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:00.109640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab069ca-9a1b-4fd5-a80b-24f659bfc95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:00.110554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab069ca-9a1b-4fd5-a80b-24f659bfc95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:00.110846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab069ca-9a1b-4fd5-a80b-24f659bfc95d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:00.111128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab069ca-9a1b-4fd5-a80b-24f659bfc95d admin admin] PATH_INFO: `/v3/users/55d6461fd29b452d98cfcda3644a6902` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:00.112410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab069ca-9a1b-4fd5-a80b-24f659bfc95d admin admin] RBAC: Authorizing `identity:delete_user(user_id=55d6461fd29b452d98cfcda3644a6902)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:00.114390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab069ca-9a1b-4fd5-a80b-24f659bfc95d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:00.138034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:00.397858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eab069ca-9a1b-4fd5-a80b-24f659bfc95d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55d6461fd29b452d98cfcda3644a6902'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:00.405025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eab069ca-9a1b-4fd5-a80b-24f659bfc95d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55d6461fd29b452d98cfcda3644a6902'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:00.405725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eab069ca-9a1b-4fd5-a80b-24f659bfc95d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55d6461fd29b452d98cfcda3644a6902'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:00.406380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eab069ca-9a1b-4fd5-a80b-24f659bfc95d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55d6461fd29b452d98cfcda3644a6902'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:00.476455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-eab069ca-9a1b-4fd5-a80b-24f659bfc95d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55d6461fd29b452d98cfcda3644a6902'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:00.495557 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4021/8013] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:37:00 2022] DELETE /identity/v3/users/55d6461fd29b452d98cfcda3644a6902 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:00.501358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ce99f-e423-48a4-85fc-a2726cb5f31a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:00.501684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ce99f-e423-48a4-85fc-a2726cb5f31a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:00.502023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ce99f-e423-48a4-85fc-a2726cb5f31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:00.519888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c05ce99f-e423-48a4-85fc-a2726cb5f31a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:00.521128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c05ce99f-e423-48a4-85fc-a2726cb5f31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:00.588660 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3993/8014] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:37:00.599943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab069ca-9a1b-4fd5-a80b-24f659bfc95d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:00.601357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eab069ca-9a1b-4fd5-a80b-24f659bfc95d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:00.667998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab069ca-9a1b-4fd5-a80b-24f659bfc95d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:00.672577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e253a-57c6-412e-89f5-b3e889addf87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:00.673141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e253a-57c6-412e-89f5-b3e889addf87 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:00.673424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e253a-57c6-412e-89f5-b3e889addf87 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:00.673706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e253a-57c6-412e-89f5-b3e889addf87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:00.685249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e253a-57c6-412e-89f5-b3e889addf87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:00.687220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e253a-57c6-412e-89f5-b3e889addf87 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:00.697529 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4022/8015] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:37:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:37:01.128865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ce99f-e423-48a4-85fc-a2726cb5f31a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:01.131633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c05ce99f-e423-48a4-85fc-a2726cb5f31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:01.147714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ce99f-e423-48a4-85fc-a2726cb5f31a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:01.154874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c57ded-0f86-49e7-adfb-41e47fa14634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:01.156138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c57ded-0f86-49e7-adfb-41e47fa14634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:01.156420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c57ded-0f86-49e7-adfb-41e47fa14634 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:01.157275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c57ded-0f86-49e7-adfb-41e47fa14634 admin admin] PATH_INFO: `/v3/projects/c2ffd00c53e444ae84569429b4a04f44` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:01.157934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c57ded-0f86-49e7-adfb-41e47fa14634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2ffd00c53e444ae84569429b4a04f44)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:01.160856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c57ded-0f86-49e7-adfb-41e47fa14634 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:01.254131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-49c57ded-0f86-49e7-adfb-41e47fa14634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2ffd00c53e444ae84569429b4a04f44', 'request_id': 'req-49c57ded-0f86-49e7-adfb-41e47fa14634'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:01.259707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-49c57ded-0f86-49e7-adfb-41e47fa14634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2ffd00c53e444ae84569429b4a04f44', 'request_id': 'req-49c57ded-0f86-49e7-adfb-41e47fa14634'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:01.322363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-49c57ded-0f86-49e7-adfb-41e47fa14634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2ffd00c53e444ae84569429b4a04f44', 'request_id': 'req-49c57ded-0f86-49e7-adfb-41e47fa14634'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:01.335315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-49c57ded-0f86-49e7-adfb-41e47fa14634 admin admin] The token cache is being invalidate because project c2ffd00c53e444ae84569429b4a04f44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:37:01.335608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-49c57ded-0f86-49e7-adfb-41e47fa14634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:01.336279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-49c57ded-0f86-49e7-adfb-41e47fa14634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:01.337831 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3994/8016] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:37:01 2022] DELETE /identity/v3/projects/c2ffd00c53e444ae84569429b4a04f44 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:01.343164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82eb4e-0575-4cbf-972e-715207f0caef None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:01.343458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82eb4e-0575-4cbf-972e-715207f0caef None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:01.343737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82eb4e-0575-4cbf-972e-715207f0caef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:01.362439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0e82eb4e-0575-4cbf-972e-715207f0caef None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:01.363693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0e82eb4e-0575-4cbf-972e-715207f0caef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:01.428032 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4023/8017] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:37:01.435512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67013c1c-50db-4b73-b4b2-599c0bb8ed9f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:01.435912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67013c1c-50db-4b73-b4b2-599c0bb8ed9f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:01.436257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67013c1c-50db-4b73-b4b2-599c0bb8ed9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:01.457282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-67013c1c-50db-4b73-b4b2-599c0bb8ed9f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:01.458558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-67013c1c-50db-4b73-b4b2-599c0bb8ed9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:01.466879 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3995/8018] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:37:01.471231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82eb4e-0575-4cbf-972e-715207f0caef None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:01.481304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82eb4e-0575-4cbf-972e-715207f0caef None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:01.486042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b10b1-8987-4cc4-a0d0-9857e4a7bc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:01.486715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b10b1-8987-4cc4-a0d0-9857e4a7bc7d admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:01.487004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b10b1-8987-4cc4-a0d0-9857e4a7bc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:01.487304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b10b1-8987-4cc4-a0d0-9857e4a7bc7d admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:01.488026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b10b1-8987-4cc4-a0d0-9857e4a7bc7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:37:01.488339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b10b1-8987-4cc4-a0d0-9857e4a7bc7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:01.489975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b10b1-8987-4cc4-a0d0-9857e4a7bc7d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:01.504457 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4024/8019] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:37:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:37:01.508808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67013c1c-50db-4b73-b4b2-599c0bb8ed9f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:01.518484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67013c1c-50db-4b73-b4b2-599c0bb8ed9f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:01.523409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0176f-bc7f-4811-82fd-7b9f50ae8847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:01.524076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0176f-bc7f-4811-82fd-7b9f50ae8847 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:01.524365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0176f-bc7f-4811-82fd-7b9f50ae8847 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:01.524646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0176f-bc7f-4811-82fd-7b9f50ae8847 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:01.525358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0176f-bc7f-4811-82fd-7b9f50ae8847 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:01.527294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0176f-bc7f-4811-82fd-7b9f50ae8847 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:01.602916 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3996/8020] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:37:01 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:37:01.607302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b10b1-8987-4cc4-a0d0-9857e4a7bc7d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:01.618471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b10b1-8987-4cc4-a0d0-9857e4a7bc7d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:01.646412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:01.679301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62483b76-797c-4ec1-8736-c756c146f34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:01.680010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62483b76-797c-4ec1-8736-c756c146f34f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:01.680302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62483b76-797c-4ec1-8736-c756c146f34f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:01.680592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62483b76-797c-4ec1-8736-c756c146f34f admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:01.681314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62483b76-797c-4ec1-8736-c756c146f34f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:01.683297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62483b76-797c-4ec1-8736-c756c146f34f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:01.797650 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4025/8021] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:37:01 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:37:01.801900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0176f-bc7f-4811-82fd-7b9f50ae8847 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:01.812468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0176f-bc7f-4811-82fd-7b9f50ae8847 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:01.817066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea5d69-a2b4-4912-80cf-b79ec3683f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:01.817723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea5d69-a2b4-4912-80cf-b79ec3683f0c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:01.818047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea5d69-a2b4-4912-80cf-b79ec3683f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:01.820833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea5d69-a2b4-4912-80cf-b79ec3683f0c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:01.820833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea5d69-a2b4-4912-80cf-b79ec3683f0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:01.820833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea5d69-a2b4-4912-80cf-b79ec3683f0c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:01.827834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:01.845525 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3997/8022] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:37:01.850340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62483b76-797c-4ec1-8736-c756c146f34f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:01.860823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62483b76-797c-4ec1-8736-c756c146f34f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:01.865582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387ff82-d61c-4648-8c18-7c49ef532e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:01.866320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387ff82-d61c-4648-8c18-7c49ef532e81 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:01.866612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387ff82-d61c-4648-8c18-7c49ef532e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:01.866893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387ff82-d61c-4648-8c18-7c49ef532e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:01.867627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387ff82-d61c-4648-8c18-7c49ef532e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:01.870142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387ff82-d61c-4648-8c18-7c49ef532e81 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:01.880054 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4026/8023] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:37:01.884437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea5d69-a2b4-4912-80cf-b79ec3683f0c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:01.896824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea5d69-a2b4-4912-80cf-b79ec3683f0c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:01.901440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058cdef-5113-4313-a52b-5a714e2cf673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:01.902347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058cdef-5113-4313-a52b-5a714e2cf673 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:01.902635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058cdef-5113-4313-a52b-5a714e2cf673 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:01.902939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058cdef-5113-4313-a52b-5a714e2cf673 admin admin] PATH_INFO: `/v3/projects/9b50da35b3424021ad71774889dabb2d/users/5b10fdc16ad64679be36d3cb907fed0a/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:01.920395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9058cdef-5113-4313-a52b-5a714e2cf673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b50da35b3424021ad71774889dabb2d, user_id=5b10fdc16ad64679be36d3cb907fed0a, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:01.923192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9058cdef-5113-4313-a52b-5a714e2cf673 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:01.992241 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3998/8024] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:37:01 2022] PUT /identity/v3/projects/9b50da35b3424021ad71774889dabb2d/users/5b10fdc16ad64679be36d3cb907fed0a/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:02.002659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911687ac-b6a0-41e3-aff1-a5d3d521eeac None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:02.002951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911687ac-b6a0-41e3-aff1-a5d3d521eeac None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:02.003236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911687ac-b6a0-41e3-aff1-a5d3d521eeac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:02.029781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-911687ac-b6a0-41e3-aff1-a5d3d521eeac None None] MFA Rules not processed for user `5b10fdc16ad64679be36d3cb907fed0a`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:02.031032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-911687ac-b6a0-41e3-aff1-a5d3d521eeac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:02.100816 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4027/8025] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:37:02.111531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058cdef-5113-4313-a52b-5a714e2cf673 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:02.114086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9058cdef-5113-4313-a52b-5a714e2cf673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:02.174656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058cdef-5113-4313-a52b-5a714e2cf673 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:02.179869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b19ad-e954-4d7a-95b2-8d2424fd1741 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:02.180518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b19ad-e954-4d7a-95b2-8d2424fd1741 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:02.180803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b19ad-e954-4d7a-95b2-8d2424fd1741 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:02.181076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b19ad-e954-4d7a-95b2-8d2424fd1741 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:02.190322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b19ad-e954-4d7a-95b2-8d2424fd1741 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:02.191884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b19ad-e954-4d7a-95b2-8d2424fd1741 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:02.201853 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 3999/8026] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:37:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:37:12.096331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:12.197446 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=38,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:16.442262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911687ac-b6a0-41e3-aff1-a5d3d521eeac None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:16.506893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911687ac-b6a0-41e3-aff1-a5d3d521eeac None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:16.511981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6583d74-ad10-4d80-a72c-e9811f12190c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:16.512855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6583d74-ad10-4d80-a72c-e9811f12190c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:16.513142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6583d74-ad10-4d80-a72c-e9811f12190c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:16.514667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6583d74-ad10-4d80-a72c-e9811f12190c admin admin] PATH_INFO: `/v3/users/5b10fdc16ad64679be36d3cb907fed0a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:16.514667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6583d74-ad10-4d80-a72c-e9811f12190c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b10fdc16ad64679be36d3cb907fed0a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:16.516581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6583d74-ad10-4d80-a72c-e9811f12190c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:16.725063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a6583d74-ad10-4d80-a72c-e9811f12190c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b10fdc16ad64679be36d3cb907fed0a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:16.733109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a6583d74-ad10-4d80-a72c-e9811f12190c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b10fdc16ad64679be36d3cb907fed0a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:16.733758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a6583d74-ad10-4d80-a72c-e9811f12190c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b10fdc16ad64679be36d3cb907fed0a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:16.734433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a6583d74-ad10-4d80-a72c-e9811f12190c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b10fdc16ad64679be36d3cb907fed0a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:16.819001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a6583d74-ad10-4d80-a72c-e9811f12190c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b10fdc16ad64679be36d3cb907fed0a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:16.837868 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4028/8027] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:37:16 2022] DELETE /identity/v3/users/5b10fdc16ad64679be36d3cb907fed0a => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:16.842776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edbcc7-a601-4d8c-90c8-413f3bb5a571 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:16.843266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edbcc7-a601-4d8c-90c8-413f3bb5a571 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:16.843387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edbcc7-a601-4d8c-90c8-413f3bb5a571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:16.861965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-51edbcc7-a601-4d8c-90c8-413f3bb5a571 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:16.864919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-51edbcc7-a601-4d8c-90c8-413f3bb5a571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:16.924083 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4000/8028] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:37:16.934017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6583d74-ad10-4d80-a72c-e9811f12190c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:16.939534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a6583d74-ad10-4d80-a72c-e9811f12190c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:16.998040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6583d74-ad10-4d80-a72c-e9811f12190c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:17.001804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906876e-3aaf-4dbf-b67a-f69ebf373f06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:17.002526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906876e-3aaf-4dbf-b67a-f69ebf373f06 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:17.002817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906876e-3aaf-4dbf-b67a-f69ebf373f06 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:17.003097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906876e-3aaf-4dbf-b67a-f69ebf373f06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:17.013069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906876e-3aaf-4dbf-b67a-f69ebf373f06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:17.014554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906876e-3aaf-4dbf-b67a-f69ebf373f06 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:17.024906 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4029/8029] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:37:17.450171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edbcc7-a601-4d8c-90c8-413f3bb5a571 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:17.451702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-51edbcc7-a601-4d8c-90c8-413f3bb5a571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:17.474516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edbcc7-a601-4d8c-90c8-413f3bb5a571 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:17.482948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad186e59-46c0-4afd-a37d-8a12951ce02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:17.484218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad186e59-46c0-4afd-a37d-8a12951ce02a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:17.484846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad186e59-46c0-4afd-a37d-8a12951ce02a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:17.485206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad186e59-46c0-4afd-a37d-8a12951ce02a admin admin] PATH_INFO: `/v3/projects/9b50da35b3424021ad71774889dabb2d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:17.486621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad186e59-46c0-4afd-a37d-8a12951ce02a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b50da35b3424021ad71774889dabb2d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:17.488662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad186e59-46c0-4afd-a37d-8a12951ce02a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:17.626244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ad186e59-46c0-4afd-a37d-8a12951ce02a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b50da35b3424021ad71774889dabb2d', 'request_id': 'req-ad186e59-46c0-4afd-a37d-8a12951ce02a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:17.631804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ad186e59-46c0-4afd-a37d-8a12951ce02a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b50da35b3424021ad71774889dabb2d', 'request_id': 'req-ad186e59-46c0-4afd-a37d-8a12951ce02a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:17.733530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ad186e59-46c0-4afd-a37d-8a12951ce02a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b50da35b3424021ad71774889dabb2d', 'request_id': 'req-ad186e59-46c0-4afd-a37d-8a12951ce02a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:17.746033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ad186e59-46c0-4afd-a37d-8a12951ce02a admin admin] The token cache is being invalidate because project 9b50da35b3424021ad71774889dabb2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:37:17.746306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ad186e59-46c0-4afd-a37d-8a12951ce02a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:17.746912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ad186e59-46c0-4afd-a37d-8a12951ce02a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:17.748435 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4001/8030] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:37:17 2022] DELETE /identity/v3/projects/9b50da35b3424021ad71774889dabb2d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:17.753534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9978c0c-6701-4c15-88c4-e38046aaa7d3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:17.754042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9978c0c-6701-4c15-88c4-e38046aaa7d3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:17.754156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9978c0c-6701-4c15-88c4-e38046aaa7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:17.774447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e9978c0c-6701-4c15-88c4-e38046aaa7d3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:17.775718 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e9978c0c-6701-4c15-88c4-e38046aaa7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:17.796628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:17.840574 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4030/8031] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:37:17.953506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6078b5-944f-4648-8cb1-6b252be00099 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:17.954026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6078b5-944f-4648-8cb1-6b252be00099 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:17.954133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6078b5-944f-4648-8cb1-6b252be00099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:17.972458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ad6078b5-944f-4648-8cb1-6b252be00099 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:17.976033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ad6078b5-944f-4648-8cb1-6b252be00099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:17.982280 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4002/8032] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:37:17.986648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9978c0c-6701-4c15-88c4-e38046aaa7d3 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:17.997066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9978c0c-6701-4c15-88c4-e38046aaa7d3 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:18.001823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd857cc7-be30-4d13-a484-b34983f0a0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:18.002520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd857cc7-be30-4d13-a484-b34983f0a0d0 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:18.002808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd857cc7-be30-4d13-a484-b34983f0a0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:18.003091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd857cc7-be30-4d13-a484-b34983f0a0d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:18.003829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd857cc7-be30-4d13-a484-b34983f0a0d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:37:18.004139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd857cc7-be30-4d13-a484-b34983f0a0d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:18.005762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd857cc7-be30-4d13-a484-b34983f0a0d0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:18.017680 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4031/8033] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:37:18.021781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6078b5-944f-4648-8cb1-6b252be00099 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:18.031406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6078b5-944f-4648-8cb1-6b252be00099 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:18.036141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a303b16-c2e1-4044-a2b3-b397b18fe655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:18.036792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a303b16-c2e1-4044-a2b3-b397b18fe655 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:18.037081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a303b16-c2e1-4044-a2b3-b397b18fe655 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:18.037367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a303b16-c2e1-4044-a2b3-b397b18fe655 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:18.038110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a303b16-c2e1-4044-a2b3-b397b18fe655 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:18.040594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a303b16-c2e1-4044-a2b3-b397b18fe655 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:18.144220 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4003/8034] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:37:18 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:37:18.145411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd857cc7-be30-4d13-a484-b34983f0a0d0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:18.156112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd857cc7-be30-4d13-a484-b34983f0a0d0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:18.213349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9083b-1e44-4742-8744-9eeaa62313fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:18.214052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9083b-1e44-4742-8744-9eeaa62313fa admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:18.214341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9083b-1e44-4742-8744-9eeaa62313fa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:18.214619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d9083b-1e44-4742-8744-9eeaa62313fa admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:18.215558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9083b-1e44-4742-8744-9eeaa62313fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:18.217469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d9083b-1e44-4742-8744-9eeaa62313fa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:18.316460 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4032/8035] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:37:18 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:37:18.320813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a303b16-c2e1-4044-a2b3-b397b18fe655 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:18.331170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a303b16-c2e1-4044-a2b3-b397b18fe655 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:18.336066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75882c6-de5a-4cfa-92c1-eed2bfc3c43c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:18.337220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75882c6-de5a-4cfa-92c1-eed2bfc3c43c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:18.337503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75882c6-de5a-4cfa-92c1-eed2bfc3c43c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:18.337776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75882c6-de5a-4cfa-92c1-eed2bfc3c43c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:18.338544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75882c6-de5a-4cfa-92c1-eed2bfc3c43c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:18.340396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75882c6-de5a-4cfa-92c1-eed2bfc3c43c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:18.351010 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4004/8036] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:37:18.355495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9083b-1e44-4742-8744-9eeaa62313fa admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:18.365450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d9083b-1e44-4742-8744-9eeaa62313fa admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:18.370087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a193146-01eb-4892-b319-3b3068c0417b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:18.370740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a193146-01eb-4892-b319-3b3068c0417b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:18.371040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a193146-01eb-4892-b319-3b3068c0417b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:18.371300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a193146-01eb-4892-b319-3b3068c0417b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:18.372054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a193146-01eb-4892-b319-3b3068c0417b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:18.373635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a193146-01eb-4892-b319-3b3068c0417b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:18.383552 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4033/8037] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:37:18.387337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75882c6-de5a-4cfa-92c1-eed2bfc3c43c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:18.397224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75882c6-de5a-4cfa-92c1-eed2bfc3c43c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:18.402436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e079f-8d2d-43e1-9202-d1091f4a00f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:18.403304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e079f-8d2d-43e1-9202-d1091f4a00f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:18.403589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e079f-8d2d-43e1-9202-d1091f4a00f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:18.403869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e079f-8d2d-43e1-9202-d1091f4a00f1 admin admin] PATH_INFO: `/v3/projects/2d4864de54c14fad83350fc6c0854158/users/8dfd178b4a654597a524be7ff2e86db5/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:18.418689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e079f-8d2d-43e1-9202-d1091f4a00f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d4864de54c14fad83350fc6c0854158, user_id=8dfd178b4a654597a524be7ff2e86db5, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:18.421303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e079f-8d2d-43e1-9202-d1091f4a00f1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:18.493817 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4005/8038] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:37:18 2022] PUT /identity/v3/projects/2d4864de54c14fad83350fc6c0854158/users/8dfd178b4a654597a524be7ff2e86db5/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:18.506487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e5158-d3da-4fbb-97d6-067225ef4708 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:18.506785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e5158-d3da-4fbb-97d6-067225ef4708 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:18.507067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e5158-d3da-4fbb-97d6-067225ef4708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:18.534238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e01e5158-d3da-4fbb-97d6-067225ef4708 None None] MFA Rules not processed for user `8dfd178b4a654597a524be7ff2e86db5`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:18.535445 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e01e5158-d3da-4fbb-97d6-067225ef4708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:18.599870 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4034/8039] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:37:18.604404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e079f-8d2d-43e1-9202-d1091f4a00f1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:18.626391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:18.673077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e079f-8d2d-43e1-9202-d1091f4a00f1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:18.678622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0d469-9ae3-4e25-9fa0-0583713eccd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:18.679041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0d469-9ae3-4e25-9fa0-0583713eccd7 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:18.679301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0d469-9ae3-4e25-9fa0-0583713eccd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:18.679580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef0d469-9ae3-4e25-9fa0-0583713eccd7 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:18.680280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0d469-9ae3-4e25-9fa0-0583713eccd7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:18.682310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef0d469-9ae3-4e25-9fa0-0583713eccd7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:18.803284 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4006/8040] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:37:18 2022] POST /identity/v3/projects => generated 375 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:37:18.807513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e5158-d3da-4fbb-97d6-067225ef4708 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:18.818517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01e5158-d3da-4fbb-97d6-067225ef4708 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:18.831313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:18.876370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacff2dd-aaee-4cf1-81b0-2cc302ce653f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:18.877006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacff2dd-aaee-4cf1-81b0-2cc302ce653f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:18.877325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacff2dd-aaee-4cf1-81b0-2cc302ce653f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:18.877633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacff2dd-aaee-4cf1-81b0-2cc302ce653f admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:18.878406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacff2dd-aaee-4cf1-81b0-2cc302ce653f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:18.880378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacff2dd-aaee-4cf1-81b0-2cc302ce653f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:18.964260 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4035/8041] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:37:18 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:37:18.966541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0d469-9ae3-4e25-9fa0-0583713eccd7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:18.977018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef0d469-9ae3-4e25-9fa0-0583713eccd7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:18.981728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7fbe8-e768-446a-b946-d58c45530e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:18.982545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7fbe8-e768-446a-b946-d58c45530e56 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:18.982772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7fbe8-e768-446a-b946-d58c45530e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:18.982954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7fbe8-e768-446a-b946-d58c45530e56 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:18.983710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7fbe8-e768-446a-b946-d58c45530e56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:18.985296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7fbe8-e768-446a-b946-d58c45530e56 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:18.996996 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4007/8042] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:37:19.001296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacff2dd-aaee-4cf1-81b0-2cc302ce653f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:19.012490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacff2dd-aaee-4cf1-81b0-2cc302ce653f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:19.017290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f946d-1f74-4b42-a2ee-ae0c01409530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:19.018051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f946d-1f74-4b42-a2ee-ae0c01409530 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:19.018336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f946d-1f74-4b42-a2ee-ae0c01409530 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:19.018612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f946d-1f74-4b42-a2ee-ae0c01409530 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:19.019335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f946d-1f74-4b42-a2ee-ae0c01409530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:19.020926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f946d-1f74-4b42-a2ee-ae0c01409530 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:19.030870 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4036/8043] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:37:19.035236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7fbe8-e768-446a-b946-d58c45530e56 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:19.045328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7fbe8-e768-446a-b946-d58c45530e56 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:19.050029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81035fe-f14f-4166-8ae3-60f3052593ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:19.050912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81035fe-f14f-4166-8ae3-60f3052593ec admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:19.051200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81035fe-f14f-4166-8ae3-60f3052593ec admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:19.051483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81035fe-f14f-4166-8ae3-60f3052593ec admin admin] PATH_INFO: `/v3/projects/bf7811ef9a9a4b37afa48878b1aacf8f/users/d4e5aeb1bbe04e87b305613f20893991/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:19.066999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81035fe-f14f-4166-8ae3-60f3052593ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf7811ef9a9a4b37afa48878b1aacf8f, user_id=d4e5aeb1bbe04e87b305613f20893991, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:19.069772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81035fe-f14f-4166-8ae3-60f3052593ec admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:19.126718 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4008/8044] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:37:19 2022] PUT /identity/v3/projects/bf7811ef9a9a4b37afa48878b1aacf8f/users/d4e5aeb1bbe04e87b305613f20893991/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:19.137711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c0a55-aee1-4b96-a0e0-879d30e0d4c0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:19.138038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c0a55-aee1-4b96-a0e0-879d30e0d4c0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:19.138323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c0a55-aee1-4b96-a0e0-879d30e0d4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:19.164951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ad4c0a55-aee1-4b96-a0e0-879d30e0d4c0 None None] MFA Rules not processed for user `d4e5aeb1bbe04e87b305613f20893991`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:19.166216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ad4c0a55-aee1-4b96-a0e0-879d30e0d4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:19.232145 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4037/8045] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:37:19.236753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81035fe-f14f-4166-8ae3-60f3052593ec admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:19.299145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81035fe-f14f-4166-8ae3-60f3052593ec admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:19.303957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af74d63-704f-4123-91d9-f4f505c58039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:19.304626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af74d63-704f-4123-91d9-f4f505c58039 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:19.304919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af74d63-704f-4123-91d9-f4f505c58039 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:19.305803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af74d63-704f-4123-91d9-f4f505c58039 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:19.306546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af74d63-704f-4123-91d9-f4f505c58039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:19.308491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af74d63-704f-4123-91d9-f4f505c58039 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:19.409670 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4009/8046] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:37:19 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:37:19.413730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c0a55-aee1-4b96-a0e0-879d30e0d4c0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:19.425381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c0a55-aee1-4b96-a0e0-879d30e0d4c0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:19.484404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e9698-87d6-4872-bf4b-e38fbe75019b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:19.485083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e9698-87d6-4872-bf4b-e38fbe75019b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:19.485374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e9698-87d6-4872-bf4b-e38fbe75019b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:19.486327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1e9698-87d6-4872-bf4b-e38fbe75019b admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:19.487029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1e9698-87d6-4872-bf4b-e38fbe75019b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:19.488980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1e9698-87d6-4872-bf4b-e38fbe75019b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:19.622323 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4038/8047] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:37:19 2022] POST /identity/v3/users => generated 295 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:37:19.624343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af74d63-704f-4123-91d9-f4f505c58039 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:19.634713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af74d63-704f-4123-91d9-f4f505c58039 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:19.639239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca635469-bf0c-4908-9f9d-7d02d60cfe9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:19.640292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca635469-bf0c-4908-9f9d-7d02d60cfe9a admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:19.640292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca635469-bf0c-4908-9f9d-7d02d60cfe9a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:19.640491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca635469-bf0c-4908-9f9d-7d02d60cfe9a admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:19.641212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca635469-bf0c-4908-9f9d-7d02d60cfe9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:19.642857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca635469-bf0c-4908-9f9d-7d02d60cfe9a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:19.654577 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4010/8048] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:37:19.658645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e9698-87d6-4872-bf4b-e38fbe75019b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:19.669343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1e9698-87d6-4872-bf4b-e38fbe75019b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:19.674395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a360d-3568-41ad-a6d7-d03bd44934ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:19.675284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a360d-3568-41ad-a6d7-d03bd44934ba admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:19.675675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a360d-3568-41ad-a6d7-d03bd44934ba admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:19.675963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154a360d-3568-41ad-a6d7-d03bd44934ba admin admin] PATH_INFO: `/v3/projects/3478eaf9782e46a88c11f5952f503a85/users/90a53216078840c09adf64fff7c90c80/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:19.688214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:19.691916 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a360d-3568-41ad-a6d7-d03bd44934ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=3478eaf9782e46a88c11f5952f503a85, user_id=90a53216078840c09adf64fff7c90c80, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:19.694357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154a360d-3568-41ad-a6d7-d03bd44934ba admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:19.785347 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4039/8049] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:37:19 2022] PUT /identity/v3/projects/3478eaf9782e46a88c11f5952f503a85/users/90a53216078840c09adf64fff7c90c80/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:19.789427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca635469-bf0c-4908-9f9d-7d02d60cfe9a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:19.800614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca635469-bf0c-4908-9f9d-7d02d60cfe9a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:19.844392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:19.859551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e0da5e-6a91-4565-ba11-213ab6999cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:19.860549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0da5e-6a91-4565-ba11-213ab6999cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:19.860549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0da5e-6a91-4565-ba11-213ab6999cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:19.860801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e0da5e-6a91-4565-ba11-213ab6999cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:19.861493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0da5e-6a91-4565-ba11-213ab6999cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:19.863152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e0da5e-6a91-4565-ba11-213ab6999cb9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:19.872623 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4011/8050] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:37:19.876807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a360d-3568-41ad-a6d7-d03bd44934ba admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:19.887238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154a360d-3568-41ad-a6d7-d03bd44934ba admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:19.892023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84f4ca-183a-483b-ab66-898e47367f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:19.893003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84f4ca-183a-483b-ab66-898e47367f5d admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:19.893292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84f4ca-183a-483b-ab66-898e47367f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:19.893573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84f4ca-183a-483b-ab66-898e47367f5d admin admin] PATH_INFO: `/v3/projects/3478eaf9782e46a88c11f5952f503a85/users/90a53216078840c09adf64fff7c90c80/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:19.895621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84f4ca-183a-483b-ab66-898e47367f5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3478eaf9782e46a88c11f5952f503a85, user_id=90a53216078840c09adf64fff7c90c80, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:19.898406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84f4ca-183a-483b-ab66-898e47367f5d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:19.910544 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4040/8051] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:37:19 2022] PUT /identity/v3/projects/3478eaf9782e46a88c11f5952f503a85/users/90a53216078840c09adf64fff7c90c80/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:19.922378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78179f20-2591-43ec-9159-f85fa876c36f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:19.922675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78179f20-2591-43ec-9159-f85fa876c36f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:19.922964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78179f20-2591-43ec-9159-f85fa876c36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:19.949383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-78179f20-2591-43ec-9159-f85fa876c36f None None] MFA Rules not processed for user `90a53216078840c09adf64fff7c90c80`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:19.950632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-78179f20-2591-43ec-9159-f85fa876c36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:20.019957 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4012/8052] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:37:20.041503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84f4ca-183a-483b-ab66-898e47367f5d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:20.043311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dc84f4ca-183a-483b-ab66-898e47367f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:20.113783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84f4ca-183a-483b-ab66-898e47367f5d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:20.117664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371312e-16e9-4151-94ba-6ab1d5231cf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:20.118350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371312e-16e9-4151-94ba-6ab1d5231cf8 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:20.120162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371312e-16e9-4151-94ba-6ab1d5231cf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:20.120162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371312e-16e9-4151-94ba-6ab1d5231cf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:20.185072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371312e-16e9-4151-94ba-6ab1d5231cf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:20.186975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371312e-16e9-4151-94ba-6ab1d5231cf8 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:20.197895 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4041/8053] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:37:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:37:23.256699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78179f20-2591-43ec-9159-f85fa876c36f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:23.258164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-78179f20-2591-43ec-9159-f85fa876c36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:23.272795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78179f20-2591-43ec-9159-f85fa876c36f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:23.276563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d86e613-6160-4244-89ac-ee7063ce468c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:23.277211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86e613-6160-4244-89ac-ee7063ce468c service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:23.277499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86e613-6160-4244-89ac-ee7063ce468c service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:23.277806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d86e613-6160-4244-89ac-ee7063ce468c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:23.335978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d86e613-6160-4244-89ac-ee7063ce468c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:23.338251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d86e613-6160-4244-89ac-ee7063ce468c service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:23.347946 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4013/8054] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:37:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:37:25.084278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371312e-16e9-4151-94ba-6ab1d5231cf8 service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:25.097819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371312e-16e9-4151-94ba-6ab1d5231cf8 service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:25.101649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b3461-33b4-4066-ab28-b994359dd6e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:25.103023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b3461-33b4-4066-ab28-b994359dd6e6 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:25.103374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b3461-33b4-4066-ab28-b994359dd6e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:25.103741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b3461-33b4-4066-ab28-b994359dd6e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:25.113780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b3461-33b4-4066-ab28-b994359dd6e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:25.115282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5b3461-33b4-4066-ab28-b994359dd6e6 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:25.125743 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4042/8055] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:37:25 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:37:28.963633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f825f-1469-4fd4-b8d6-78e80af9b189 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:28.964109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f825f-1469-4fd4-b8d6-78e80af9b189 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:28.964228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f825f-1469-4fd4-b8d6-78e80af9b189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:28.982417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bb7f825f-1469-4fd4-b8d6-78e80af9b189 None None] MFA Rules not processed for user `b16d0e791edd4f119e646e7ced74bf74`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:28.983626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bb7f825f-1469-4fd4-b8d6-78e80af9b189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:29.040253 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4014/8056] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:37:35.121544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=62,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:35.340675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b3461-33b4-4066-ab28-b994359dd6e6 service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:35.402771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5b3461-33b4-4066-ab28-b994359dd6e6 service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:35.407290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2db450-ad4d-4f6a-a6d9-e61b217ef509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:35.408152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2db450-ad4d-4f6a-a6d9-e61b217ef509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:35.408437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2db450-ad4d-4f6a-a6d9-e61b217ef509 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:35.408716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2db450-ad4d-4f6a-a6d9-e61b217ef509 admin admin] PATH_INFO: `/v3/users/8dfd178b4a654597a524be7ff2e86db5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:35.409934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2db450-ad4d-4f6a-a6d9-e61b217ef509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dfd178b4a654597a524be7ff2e86db5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:35.411863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2db450-ad4d-4f6a-a6d9-e61b217ef509 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:35.593780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0c2db450-ad4d-4f6a-a6d9-e61b217ef509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dfd178b4a654597a524be7ff2e86db5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:35.600990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0c2db450-ad4d-4f6a-a6d9-e61b217ef509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dfd178b4a654597a524be7ff2e86db5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:35.601663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0c2db450-ad4d-4f6a-a6d9-e61b217ef509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dfd178b4a654597a524be7ff2e86db5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:35.602340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0c2db450-ad4d-4f6a-a6d9-e61b217ef509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dfd178b4a654597a524be7ff2e86db5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:35.670137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0c2db450-ad4d-4f6a-a6d9-e61b217ef509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dfd178b4a654597a524be7ff2e86db5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:35.687970 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4043/8057] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:37:35 2022] DELETE /identity/v3/users/8dfd178b4a654597a524be7ff2e86db5 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:35.692606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65237a-62cb-400d-9faf-3ac58d824e28 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:35.693062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65237a-62cb-400d-9faf-3ac58d824e28 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:35.693203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a65237a-62cb-400d-9faf-3ac58d824e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:35.711211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7a65237a-62cb-400d-9faf-3ac58d824e28 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:35.712418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7a65237a-62cb-400d-9faf-3ac58d824e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:35.717257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:35.771665 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4015/8058] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:37:35.781231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2db450-ad4d-4f6a-a6d9-e61b217ef509 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:35.782688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0c2db450-ad4d-4f6a-a6d9-e61b217ef509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:35.843572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2db450-ad4d-4f6a-a6d9-e61b217ef509 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:35.847454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989aa6e4-897a-4ef5-bd07-42546f398811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:35.848111 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989aa6e4-897a-4ef5-bd07-42546f398811 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:35.848394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989aa6e4-897a-4ef5-bd07-42546f398811 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:35.848673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989aa6e4-897a-4ef5-bd07-42546f398811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:35.858824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989aa6e4-897a-4ef5-bd07-42546f398811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:35.860263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989aa6e4-897a-4ef5-bd07-42546f398811 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:35.870594 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4044/8059] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:37:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:37:36.260710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65237a-62cb-400d-9faf-3ac58d824e28 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:36.265217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7a65237a-62cb-400d-9faf-3ac58d824e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:36.277344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a65237a-62cb-400d-9faf-3ac58d824e28 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:36.282947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:36.283977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:36.285156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:36.285598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa admin admin] PATH_INFO: `/v3/projects/2d4864de54c14fad83350fc6c0854158` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:36.287127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d4864de54c14fad83350fc6c0854158)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:36.289175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:36.368515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d4864de54c14fad83350fc6c0854158', 'request_id': 'req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:36.374038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d4864de54c14fad83350fc6c0854158', 'request_id': 'req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:36.424751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d4864de54c14fad83350fc6c0854158', 'request_id': 'req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:36.435748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa admin admin] The token cache is being invalidate because project 2d4864de54c14fad83350fc6c0854158 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:37:36.436026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:36.436653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:36.438232 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4016/8060] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:37:36 2022] DELETE /identity/v3/projects/2d4864de54c14fad83350fc6c0854158 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:36.442354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989aa6e4-897a-4ef5-bd07-42546f398811 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:36.452793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989aa6e4-897a-4ef5-bd07-42546f398811 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:36.472114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:36.510560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8da64e-7844-4cef-8d92-7101e5fa3a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:36.511454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8da64e-7844-4cef-8d92-7101e5fa3a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:36.511734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8da64e-7844-4cef-8d92-7101e5fa3a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:36.512009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8da64e-7844-4cef-8d92-7101e5fa3a64 admin admin] PATH_INFO: `/v3/users/d4e5aeb1bbe04e87b305613f20893991` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:36.513320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8da64e-7844-4cef-8d92-7101e5fa3a64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4e5aeb1bbe04e87b305613f20893991)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:36.515360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8da64e-7844-4cef-8d92-7101e5fa3a64 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:36.695749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dd8da64e-7844-4cef-8d92-7101e5fa3a64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4e5aeb1bbe04e87b305613f20893991'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:36.703207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dd8da64e-7844-4cef-8d92-7101e5fa3a64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4e5aeb1bbe04e87b305613f20893991'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:36.703897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dd8da64e-7844-4cef-8d92-7101e5fa3a64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4e5aeb1bbe04e87b305613f20893991'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:36.704518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dd8da64e-7844-4cef-8d92-7101e5fa3a64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4e5aeb1bbe04e87b305613f20893991'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:36.767691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dd8da64e-7844-4cef-8d92-7101e5fa3a64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4e5aeb1bbe04e87b305613f20893991'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:36.786381 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4045/8061] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:37:36 2022] DELETE /identity/v3/users/d4e5aeb1bbe04e87b305613f20893991 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:37.203152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:37.204582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:37.221609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba8ba0a-2e2d-45bb-8fff-1d124c2b14aa admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:37.278295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd425f7-b73f-4e52-9fb8-0a79cab32360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:37.279162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd425f7-b73f-4e52-9fb8-0a79cab32360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:37.279447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd425f7-b73f-4e52-9fb8-0a79cab32360 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:37.279722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd425f7-b73f-4e52-9fb8-0a79cab32360 admin admin] PATH_INFO: `/v3/projects/bf7811ef9a9a4b37afa48878b1aacf8f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:37.280889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd425f7-b73f-4e52-9fb8-0a79cab32360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf7811ef9a9a4b37afa48878b1aacf8f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:37.282865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd425f7-b73f-4e52-9fb8-0a79cab32360 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:37.375831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:37.385541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0bd425f7-b73f-4e52-9fb8-0a79cab32360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf7811ef9a9a4b37afa48878b1aacf8f', 'request_id': 'req-0bd425f7-b73f-4e52-9fb8-0a79cab32360'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:37.391138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0bd425f7-b73f-4e52-9fb8-0a79cab32360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf7811ef9a9a4b37afa48878b1aacf8f', 'request_id': 'req-0bd425f7-b73f-4e52-9fb8-0a79cab32360'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:37.487964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0bd425f7-b73f-4e52-9fb8-0a79cab32360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf7811ef9a9a4b37afa48878b1aacf8f', 'request_id': 'req-0bd425f7-b73f-4e52-9fb8-0a79cab32360'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:37.500470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0bd425f7-b73f-4e52-9fb8-0a79cab32360 admin admin] The token cache is being invalidate because project bf7811ef9a9a4b37afa48878b1aacf8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:37:37.501080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0bd425f7-b73f-4e52-9fb8-0a79cab32360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:37.501757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0bd425f7-b73f-4e52-9fb8-0a79cab32360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:37.503350 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4017/8062] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:37:37 2022] DELETE /identity/v3/projects/bf7811ef9a9a4b37afa48878b1aacf8f => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:37.507386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8da64e-7844-4cef-8d92-7101e5fa3a64 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:37.508914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dd8da64e-7844-4cef-8d92-7101e5fa3a64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:37.521937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8da64e-7844-4cef-8d92-7101e5fa3a64 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:37.578836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eec804-ad52-4937-aed5-a8e860b71b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:37.579799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eec804-ad52-4937-aed5-a8e860b71b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:37.580030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eec804-ad52-4937-aed5-a8e860b71b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:37.580312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eec804-ad52-4937-aed5-a8e860b71b9d admin admin] PATH_INFO: `/v3/users/90a53216078840c09adf64fff7c90c80` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:37.581515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eec804-ad52-4937-aed5-a8e860b71b9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=90a53216078840c09adf64fff7c90c80)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:37.583542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eec804-ad52-4937-aed5-a8e860b71b9d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:37.783343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:37.785973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-46eec804-ad52-4937-aed5-a8e860b71b9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90a53216078840c09adf64fff7c90c80'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:37.793505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-46eec804-ad52-4937-aed5-a8e860b71b9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90a53216078840c09adf64fff7c90c80'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:37.794170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-46eec804-ad52-4937-aed5-a8e860b71b9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90a53216078840c09adf64fff7c90c80'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:37.794808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-46eec804-ad52-4937-aed5-a8e860b71b9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90a53216078840c09adf64fff7c90c80'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:37.851948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-46eec804-ad52-4937-aed5-a8e860b71b9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90a53216078840c09adf64fff7c90c80'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:37.870098 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4046/8063] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:37:37 2022] DELETE /identity/v3/users/90a53216078840c09adf64fff7c90c80 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:38.340602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd425f7-b73f-4e52-9fb8-0a79cab32360 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:38.342074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0bd425f7-b73f-4e52-9fb8-0a79cab32360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:38.355322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd425f7-b73f-4e52-9fb8-0a79cab32360 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:38.413573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d572b-6eb3-4157-a269-b67f162dc297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:38.414506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d572b-6eb3-4157-a269-b67f162dc297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:38.414790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d572b-6eb3-4157-a269-b67f162dc297 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:38.415067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d572b-6eb3-4157-a269-b67f162dc297 admin admin] PATH_INFO: `/v3/projects/3478eaf9782e46a88c11f5952f503a85` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:38.416274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d572b-6eb3-4157-a269-b67f162dc297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3478eaf9782e46a88c11f5952f503a85)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:38.418221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d572b-6eb3-4157-a269-b67f162dc297 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:38.500354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-589d572b-6eb3-4157-a269-b67f162dc297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3478eaf9782e46a88c11f5952f503a85', 'request_id': 'req-589d572b-6eb3-4157-a269-b67f162dc297'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:38.506582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-589d572b-6eb3-4157-a269-b67f162dc297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3478eaf9782e46a88c11f5952f503a85', 'request_id': 'req-589d572b-6eb3-4157-a269-b67f162dc297'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:38.550763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-589d572b-6eb3-4157-a269-b67f162dc297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3478eaf9782e46a88c11f5952f503a85', 'request_id': 'req-589d572b-6eb3-4157-a269-b67f162dc297'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:38.563053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-589d572b-6eb3-4157-a269-b67f162dc297 admin admin] The token cache is being invalidate because project 3478eaf9782e46a88c11f5952f503a85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:37:38.563357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-589d572b-6eb3-4157-a269-b67f162dc297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:38.564047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-589d572b-6eb3-4157-a269-b67f162dc297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:37:38.565962 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4018/8064] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:37:38 2022] DELETE /identity/v3/projects/3478eaf9782e46a88c11f5952f503a85 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:38.571816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b26e7-be76-499c-a01c-e3cd004c6e7f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:38.572881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b26e7-be76-499c-a01c-e3cd004c6e7f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:38.573006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b26e7-be76-499c-a01c-e3cd004c6e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:38.591317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-041b26e7-be76-499c-a01c-e3cd004c6e7f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:38.592600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-041b26e7-be76-499c-a01c-e3cd004c6e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:38.656557 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4047/8065] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:37:38.671920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf2521-52f5-465d-ab75-ec608f95d8a3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:38.672218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf2521-52f5-465d-ab75-ec608f95d8a3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:38.672504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf2521-52f5-465d-ab75-ec608f95d8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:38.698807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3ecf2521-52f5-465d-ab75-ec608f95d8a3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:38.700475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3ecf2521-52f5-465d-ab75-ec608f95d8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:38.711133 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4019/8066] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:37:38.716582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b26e7-be76-499c-a01c-e3cd004c6e7f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:38.728170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b26e7-be76-499c-a01c-e3cd004c6e7f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:38.734391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51ded4-3ca4-4002-80cc-90742c08e4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:38.735109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51ded4-3ca4-4002-80cc-90742c08e4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:38.735401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51ded4-3ca4-4002-80cc-90742c08e4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:38.735687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51ded4-3ca4-4002-80cc-90742c08e4b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:38.736423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51ded4-3ca4-4002-80cc-90742c08e4b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:37:38.736737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51ded4-3ca4-4002-80cc-90742c08e4b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:38.738370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51ded4-3ca4-4002-80cc-90742c08e4b0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:38.751764 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4048/8067] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:37:38.756167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf2521-52f5-465d-ab75-ec608f95d8a3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:38.766222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf2521-52f5-465d-ab75-ec608f95d8a3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:38.770957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9a105-8613-48f6-aea9-a6c67b805625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:38.771605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9a105-8613-48f6-aea9-a6c67b805625 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:38.771892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9a105-8613-48f6-aea9-a6c67b805625 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:38.772211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9a105-8613-48f6-aea9-a6c67b805625 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:38.772947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9a105-8613-48f6-aea9-a6c67b805625 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:38.774973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9a105-8613-48f6-aea9-a6c67b805625 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:38.900722 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4020/8068] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:37:38 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:37:38.904914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51ded4-3ca4-4002-80cc-90742c08e4b0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:38.915150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51ded4-3ca4-4002-80cc-90742c08e4b0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:38.971802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b356c-a2fc-4238-8b6f-7405257f5788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:38.972471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b356c-a2fc-4238-8b6f-7405257f5788 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:38.972760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b356c-a2fc-4238-8b6f-7405257f5788 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:38.973039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b356c-a2fc-4238-8b6f-7405257f5788 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:38.973734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b356c-a2fc-4238-8b6f-7405257f5788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:38.975695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b356c-a2fc-4238-8b6f-7405257f5788 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:39.060965 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4049/8069] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:37:38 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:37:39.062899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9a105-8613-48f6-aea9-a6c67b805625 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:39.073398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9a105-8613-48f6-aea9-a6c67b805625 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:39.078526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d658ccc-1318-4d24-9d3f-d87565e2e335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:39.079188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d658ccc-1318-4d24-9d3f-d87565e2e335 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:39.079477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d658ccc-1318-4d24-9d3f-d87565e2e335 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:39.079760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d658ccc-1318-4d24-9d3f-d87565e2e335 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:39.080490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d658ccc-1318-4d24-9d3f-d87565e2e335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:39.082229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d658ccc-1318-4d24-9d3f-d87565e2e335 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:39.093581 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4021/8070] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:37:39.101040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b356c-a2fc-4238-8b6f-7405257f5788 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:39.112136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b356c-a2fc-4238-8b6f-7405257f5788 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:39.116989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bef42-9c7c-4966-b0b8-93ea98575ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:39.117685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bef42-9c7c-4966-b0b8-93ea98575ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:39.118018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bef42-9c7c-4966-b0b8-93ea98575ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:39.118299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bef42-9c7c-4966-b0b8-93ea98575ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:39.119032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bef42-9c7c-4966-b0b8-93ea98575ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:39.120698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bef42-9c7c-4966-b0b8-93ea98575ea6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:39.131434 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4050/8071] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:37:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:37:39.135354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d658ccc-1318-4d24-9d3f-d87565e2e335 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:39.145442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d658ccc-1318-4d24-9d3f-d87565e2e335 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:39.150065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92997a7-a4de-4cf8-a71f-5c275fabda8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:39.151021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92997a7-a4de-4cf8-a71f-5c275fabda8b admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:39.151256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92997a7-a4de-4cf8-a71f-5c275fabda8b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:39.152074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92997a7-a4de-4cf8-a71f-5c275fabda8b admin admin] PATH_INFO: `/v3/projects/9d028950b4364337a9960b608838a496/users/c9df3624a6c64cbd8383125ddf3c21ca/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:39.164081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:39.168926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92997a7-a4de-4cf8-a71f-5c275fabda8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d028950b4364337a9960b608838a496, user_id=c9df3624a6c64cbd8383125ddf3c21ca, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:39.171677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92997a7-a4de-4cf8-a71f-5c275fabda8b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:39.274947 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4022/8072] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:37:39 2022] PUT /identity/v3/projects/9d028950b4364337a9960b608838a496/users/c9df3624a6c64cbd8383125ddf3c21ca/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:39.284432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a4b23-b80b-47ff-aa6f-0697c36dbcdd None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:39.284728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a4b23-b80b-47ff-aa6f-0697c36dbcdd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:39.285008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a4b23-b80b-47ff-aa6f-0697c36dbcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:39.303289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a28a4b23-b80b-47ff-aa6f-0697c36dbcdd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:39.304531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a28a4b23-b80b-47ff-aa6f-0697c36dbcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:39.326405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:39.367200 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4051/8073] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:37:39.376837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92997a7-a4de-4cf8-a71f-5c275fabda8b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:39.378689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d92997a7-a4de-4cf8-a71f-5c275fabda8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:39.439739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92997a7-a4de-4cf8-a71f-5c275fabda8b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:39.444965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf822f6b-98b2-4eda-9153-3619f8c72e74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:39.444965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf822f6b-98b2-4eda-9153-3619f8c72e74 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:39.444965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf822f6b-98b2-4eda-9153-3619f8c72e74 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:39.444965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf822f6b-98b2-4eda-9153-3619f8c72e74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:39.454881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf822f6b-98b2-4eda-9153-3619f8c72e74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:39.456376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf822f6b-98b2-4eda-9153-3619f8c72e74 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:39.467279 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4023/8074] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:37:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:37:45.283452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa3f7c-15a9-40e5-9f08-3ce4610b9216 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:45.285174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa3f7c-15a9-40e5-9f08-3ce4610b9216 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:45.285174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa3f7c-15a9-40e5-9f08-3ce4610b9216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:45.314978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1eaa3f7c-15a9-40e5-9f08-3ce4610b9216 None None] MFA Rules not processed for user `c9df3624a6c64cbd8383125ddf3c21ca`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:45.316325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1eaa3f7c-15a9-40e5-9f08-3ce4610b9216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:45.395227 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4052/8075] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:37:45.399710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf822f6b-98b2-4eda-9153-3619f8c72e74 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:45.411173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf822f6b-98b2-4eda-9153-3619f8c72e74 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:45.416766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9b17-ecd7-401b-8d1b-ab1986592fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:45.417711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9b17-ecd7-401b-8d1b-ab1986592fc7 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:45.418029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9b17-ecd7-401b-8d1b-ab1986592fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:45.418483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb9b17-ecd7-401b-8d1b-ab1986592fc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:45.419001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb9b17-ecd7-401b-8d1b-ab1986592fc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:45.420918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb9b17-ecd7-401b-8d1b-ab1986592fc7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:45.547339 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4024/8076] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:37:45 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:37:45.551674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa3f7c-15a9-40e5-9f08-3ce4610b9216 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:45.562738 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa3f7c-15a9-40e5-9f08-3ce4610b9216 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:45.623309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4ff0b-0302-460e-a59b-741d5e0c19c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:45.624052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4ff0b-0302-460e-a59b-741d5e0c19c2 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:45.624338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4ff0b-0302-460e-a59b-741d5e0c19c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:45.624645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e4ff0b-0302-460e-a59b-741d5e0c19c2 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:45.625467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4ff0b-0302-460e-a59b-741d5e0c19c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:45.627449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e4ff0b-0302-460e-a59b-741d5e0c19c2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:45.700686 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4053/8077] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:37:45 2022] POST /identity/v3/users => generated 294 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:37:45.704781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9b17-ecd7-401b-8d1b-ab1986592fc7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:45.716148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb9b17-ecd7-401b-8d1b-ab1986592fc7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:45.721026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cba27-5229-4701-b965-c2e833f820a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:45.721727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cba27-5229-4701-b965-c2e833f820a7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:45.722046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cba27-5229-4701-b965-c2e833f820a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:45.722356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8cba27-5229-4701-b965-c2e833f820a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:45.723107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8cba27-5229-4701-b965-c2e833f820a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:45.724804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8cba27-5229-4701-b965-c2e833f820a7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:45.734787 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4025/8078] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:37:45.738786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4ff0b-0302-460e-a59b-741d5e0c19c2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:45.749346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e4ff0b-0302-460e-a59b-741d5e0c19c2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:45.754097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9cffe-37c6-44a4-bf96-9b12407039c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:45.754971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9cffe-37c6-44a4-bf96-9b12407039c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:45.755261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9cffe-37c6-44a4-bf96-9b12407039c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:45.755543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc9cffe-37c6-44a4-bf96-9b12407039c0 admin admin] PATH_INFO: `/v3/projects/548e808b546044df903a929dfb8a6900/users/fd831e5fee154f41aeacb1cf9196939e/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:45.770832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9cffe-37c6-44a4-bf96-9b12407039c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=548e808b546044df903a929dfb8a6900, user_id=fd831e5fee154f41aeacb1cf9196939e, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:45.773488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc9cffe-37c6-44a4-bf96-9b12407039c0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:45.852477 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4054/8079] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:37:45 2022] PUT /identity/v3/projects/548e808b546044df903a929dfb8a6900/users/fd831e5fee154f41aeacb1cf9196939e/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:45.857761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cba27-5229-4701-b965-c2e833f820a7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:45.868052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8cba27-5229-4701-b965-c2e833f820a7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:45.923240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101b1243-cac6-4c1f-be68-7ff4d0173dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:45.923901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b1243-cac6-4c1f-be68-7ff4d0173dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:45.924186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b1243-cac6-4c1f-be68-7ff4d0173dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:45.924463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101b1243-cac6-4c1f-be68-7ff4d0173dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:45.925177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b1243-cac6-4c1f-be68-7ff4d0173dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:45.926813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101b1243-cac6-4c1f-be68-7ff4d0173dc8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:45.937050 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4026/8080] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:37:45 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:37:45.941202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9cffe-37c6-44a4-bf96-9b12407039c0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:45.952556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc9cffe-37c6-44a4-bf96-9b12407039c0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:45.957526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480632bc-9f85-4fec-bccc-f568f57018d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:45.958464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480632bc-9f85-4fec-bccc-f568f57018d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:45.958750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480632bc-9f85-4fec-bccc-f568f57018d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:45.959034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480632bc-9f85-4fec-bccc-f568f57018d6 admin admin] PATH_INFO: `/v3/projects/548e808b546044df903a929dfb8a6900/users/fd831e5fee154f41aeacb1cf9196939e/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:45.961051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480632bc-9f85-4fec-bccc-f568f57018d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=548e808b546044df903a929dfb8a6900, user_id=fd831e5fee154f41aeacb1cf9196939e, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:45.963770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480632bc-9f85-4fec-bccc-f568f57018d6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:45.975257 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4055/8081] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:37:45 2022] PUT /identity/v3/projects/548e808b546044df903a929dfb8a6900/users/fd831e5fee154f41aeacb1cf9196939e/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:37:51.994923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d7005-70ef-4082-8486-bf10e0ea31b9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:51.995946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d7005-70ef-4082-8486-bf10e0ea31b9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:51.996064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d7005-70ef-4082-8486-bf10e0ea31b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:52.023918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-798d7005-70ef-4082-8486-bf10e0ea31b9 None None] MFA Rules not processed for user `fd831e5fee154f41aeacb1cf9196939e`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:52.025138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-798d7005-70ef-4082-8486-bf10e0ea31b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:52.080098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:52.119899 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4027/8082] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:37:52.134983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480632bc-9f85-4fec-bccc-f568f57018d6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:37:52.137350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-480632bc-9f85-4fec-bccc-f568f57018d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:52.149018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:52.219979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480632bc-9f85-4fec-bccc-f568f57018d6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:37:52.225927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ce5c1-2adb-4d7b-a07f-19ab19c7ce17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:37:52.226600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ce5c1-2adb-4d7b-a07f-19ab19c7ce17 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:52.226956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ce5c1-2adb-4d7b-a07f-19ab19c7ce17 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:52.227169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ce5c1-2adb-4d7b-a07f-19ab19c7ce17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:52.237245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ce5c1-2adb-4d7b-a07f-19ab19c7ce17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:37:52.238913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2ce5c1-2adb-4d7b-a07f-19ab19c7ce17 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:37:52.250497 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4056/8083] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:37:52 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:37:52.300646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96439af2-2133-482a-bf33-3f280a5f51bc None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:37:52.300956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96439af2-2133-482a-bf33-3f280a5f51bc None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:37:52.301238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96439af2-2133-482a-bf33-3f280a5f51bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:37:52.321467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-96439af2-2133-482a-bf33-3f280a5f51bc None None] MFA Rules not processed for user `c9df3624a6c64cbd8383125ddf3c21ca`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:37:52.322715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-96439af2-2133-482a-bf33-3f280a5f51bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:37:52.381399 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4028/8084] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:37:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:38:02.246061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=30 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:02.374583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=38 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:03.383197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ce5c1-2adb-4d7b-a07f-19ab19c7ce17 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:03.384693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3c2ce5c1-2adb-4d7b-a07f-19ab19c7ce17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:03.444485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2ce5c1-2adb-4d7b-a07f-19ab19c7ce17 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:03.448120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79626c-3f02-4e3d-880b-5eaa7b0a8cde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:03.448766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79626c-3f02-4e3d-880b-5eaa7b0a8cde service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:03.449052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79626c-3f02-4e3d-880b-5eaa7b0a8cde service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:03.449333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79626c-3f02-4e3d-880b-5eaa7b0a8cde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:03.451183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7b79626c-3f02-4e3d-880b-5eaa7b0a8cde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:03.467322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79626c-3f02-4e3d-880b-5eaa7b0a8cde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:03.468760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79626c-3f02-4e3d-880b-5eaa7b0a8cde service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:03.482749 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4057/8085] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:38:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:38:04.948863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96439af2-2133-482a-bf33-3f280a5f51bc None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:04.961105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96439af2-2133-482a-bf33-3f280a5f51bc None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:04.965014 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df4da6-2de7-48d7-a4a4-e731dbcd5dfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:04.965691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df4da6-2de7-48d7-a4a4-e731dbcd5dfc service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:04.966036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df4da6-2de7-48d7-a4a4-e731dbcd5dfc service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:04.966322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df4da6-2de7-48d7-a4a4-e731dbcd5dfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:04.968106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-41df4da6-2de7-48d7-a4a4-e731dbcd5dfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:04.984578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df4da6-2de7-48d7-a4a4-e731dbcd5dfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:04.986049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df4da6-2de7-48d7-a4a4-e731dbcd5dfc service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:05.000031 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4029/8086] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:38:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:38:05.115369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f2b47-9dd3-4d7c-9bd2-74a149b66bf5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:05.115872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f2b47-9dd3-4d7c-9bd2-74a149b66bf5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:05.115981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f2b47-9dd3-4d7c-9bd2-74a149b66bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:05.133743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-dc2f2b47-9dd3-4d7c-9bd2-74a149b66bf5 None None] MFA Rules not processed for user `c9df3624a6c64cbd8383125ddf3c21ca`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:38:05.134923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dc2f2b47-9dd3-4d7c-9bd2-74a149b66bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:05.141812 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4058/8087] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:38:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:38:14.992069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:15.128855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=29 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:35.366022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df4da6-2de7-48d7-a4a4-e731dbcd5dfc service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:35.437679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df4da6-2de7-48d7-a4a4-e731dbcd5dfc service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:35.449024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faba68-f1d3-400c-9165-2d192cbcb7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:35.449024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faba68-f1d3-400c-9165-2d192cbcb7c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:35.449024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faba68-f1d3-400c-9165-2d192cbcb7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:35.449024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6faba68-f1d3-400c-9165-2d192cbcb7c5 admin admin] PATH_INFO: `/v3/users/c9df3624a6c64cbd8383125ddf3c21ca` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:35.455321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6faba68-f1d3-400c-9165-2d192cbcb7c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9df3624a6c64cbd8383125ddf3c21ca)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:35.455321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6faba68-f1d3-400c-9165-2d192cbcb7c5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:35.653735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a6faba68-f1d3-400c-9165-2d192cbcb7c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9df3624a6c64cbd8383125ddf3c21ca'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:35.654364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a6faba68-f1d3-400c-9165-2d192cbcb7c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9df3624a6c64cbd8383125ddf3c21ca'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:35.661376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a6faba68-f1d3-400c-9165-2d192cbcb7c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9df3624a6c64cbd8383125ddf3c21ca'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:35.755789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a6faba68-f1d3-400c-9165-2d192cbcb7c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9df3624a6c64cbd8383125ddf3c21ca'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:35.756426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a6faba68-f1d3-400c-9165-2d192cbcb7c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9df3624a6c64cbd8383125ddf3c21ca'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:35.774991 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4030/8088] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:38:35 2022] DELETE /identity/v3/users/c9df3624a6c64cbd8383125ddf3c21ca => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:36.035750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f2b47-9dd3-4d7c-9bd2-74a149b66bf5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:36.048656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f2b47-9dd3-4d7c-9bd2-74a149b66bf5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:36.109731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:36.110641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:36.110927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:36.111235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21 admin admin] PATH_INFO: `/v3/projects/9d028950b4364337a9960b608838a496` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:36.112431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d028950b4364337a9960b608838a496)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:36.114420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:36.237064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d028950b4364337a9960b608838a496', 'request_id': 'req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:36.242473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d028950b4364337a9960b608838a496', 'request_id': 'req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:36.316122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d028950b4364337a9960b608838a496', 'request_id': 'req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:36.329125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21 admin admin] The token cache is being invalidate because project 9d028950b4364337a9960b608838a496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:38:36.329449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:36.330147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:36.331714 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4059/8089] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:38:36 2022] DELETE /identity/v3/projects/9d028950b4364337a9960b608838a496 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:36.335704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faba68-f1d3-400c-9165-2d192cbcb7c5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:36.337104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a6faba68-f1d3-400c-9165-2d192cbcb7c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:36.350755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6faba68-f1d3-400c-9165-2d192cbcb7c5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:36.407733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d95f51-a6ba-482c-a889-d9bd1181aca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:36.408608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d95f51-a6ba-482c-a889-d9bd1181aca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:36.408893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d95f51-a6ba-482c-a889-d9bd1181aca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:36.409260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d95f51-a6ba-482c-a889-d9bd1181aca1 admin admin] PATH_INFO: `/v3/users/fd831e5fee154f41aeacb1cf9196939e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:36.410511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d95f51-a6ba-482c-a889-d9bd1181aca1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd831e5fee154f41aeacb1cf9196939e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:36.412454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d95f51-a6ba-482c-a889-d9bd1181aca1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:36.434634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:36.627276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e9d95f51-a6ba-482c-a889-d9bd1181aca1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd831e5fee154f41aeacb1cf9196939e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:36.628112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e9d95f51-a6ba-482c-a889-d9bd1181aca1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd831e5fee154f41aeacb1cf9196939e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:36.635440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e9d95f51-a6ba-482c-a889-d9bd1181aca1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd831e5fee154f41aeacb1cf9196939e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:36.761868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e9d95f51-a6ba-482c-a889-d9bd1181aca1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd831e5fee154f41aeacb1cf9196939e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:36.762441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e9d95f51-a6ba-482c-a889-d9bd1181aca1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd831e5fee154f41aeacb1cf9196939e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:36.783364 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4031/8090] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:38:36 2022] DELETE /identity/v3/users/fd831e5fee154f41aeacb1cf9196939e => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:37.020152 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:37.021605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:37.035732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c41b69c-9de6-41ee-b8f5-52aa3e113b21 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:37.098469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ef476-5e19-4bc8-8626-d1dc688d9c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:37.099341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ef476-5e19-4bc8-8626-d1dc688d9c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:37.099628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ef476-5e19-4bc8-8626-d1dc688d9c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:37.099912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ef476-5e19-4bc8-8626-d1dc688d9c35 admin admin] PATH_INFO: `/v3/projects/548e808b546044df903a929dfb8a6900` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:37.101113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ef476-5e19-4bc8-8626-d1dc688d9c35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=548e808b546044df903a929dfb8a6900)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:37.103075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ef476-5e19-4bc8-8626-d1dc688d9c35 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:37.190204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:37.202337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fa1ef476-5e19-4bc8-8626-d1dc688d9c35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '548e808b546044df903a929dfb8a6900', 'request_id': 'req-fa1ef476-5e19-4bc8-8626-d1dc688d9c35'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:37.207896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fa1ef476-5e19-4bc8-8626-d1dc688d9c35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '548e808b546044df903a929dfb8a6900', 'request_id': 'req-fa1ef476-5e19-4bc8-8626-d1dc688d9c35'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:37.274262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fa1ef476-5e19-4bc8-8626-d1dc688d9c35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '548e808b546044df903a929dfb8a6900', 'request_id': 'req-fa1ef476-5e19-4bc8-8626-d1dc688d9c35'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:37.285564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fa1ef476-5e19-4bc8-8626-d1dc688d9c35 admin admin] The token cache is being invalidate because project 548e808b546044df903a929dfb8a6900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:38:37.285873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fa1ef476-5e19-4bc8-8626-d1dc688d9c35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:37.286503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-fa1ef476-5e19-4bc8-8626-d1dc688d9c35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:37.288204 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4060/8091] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:38:37 2022] DELETE /identity/v3/projects/548e808b546044df903a929dfb8a6900 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:37.293183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277080e0-f9a4-44dc-b4cd-af4ae8123e39 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:37.293476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277080e0-f9a4-44dc-b4cd-af4ae8123e39 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:37.293763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277080e0-f9a4-44dc-b4cd-af4ae8123e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:37.314679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-277080e0-f9a4-44dc-b4cd-af4ae8123e39 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:38:37.315931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-277080e0-f9a4-44dc-b4cd-af4ae8123e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:37.374916 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4032/8092] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:38:37.382562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e596d97-0dcf-4b1b-955e-c081680fb69b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:37.382861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e596d97-0dcf-4b1b-955e-c081680fb69b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:37.383154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e596d97-0dcf-4b1b-955e-c081680fb69b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:37.401365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2e596d97-0dcf-4b1b-955e-c081680fb69b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:38:37.402623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2e596d97-0dcf-4b1b-955e-c081680fb69b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:37.410627 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4061/8093] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:38:37.414763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277080e0-f9a4-44dc-b4cd-af4ae8123e39 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:37.425138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277080e0-f9a4-44dc-b4cd-af4ae8123e39 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:37.429734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7325b87-20e0-4576-adb4-150a2d3a4ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:37.430419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7325b87-20e0-4576-adb4-150a2d3a4ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:37.430708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7325b87-20e0-4576-adb4-150a2d3a4ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:37.430991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7325b87-20e0-4576-adb4-150a2d3a4ad5 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:37.431718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7325b87-20e0-4576-adb4-150a2d3a4ad5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:38:37.432029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7325b87-20e0-4576-adb4-150a2d3a4ad5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:37.433644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7325b87-20e0-4576-adb4-150a2d3a4ad5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:37.445649 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4033/8094] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:38:37.449881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e596d97-0dcf-4b1b-955e-c081680fb69b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:37.459814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e596d97-0dcf-4b1b-955e-c081680fb69b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:37.465119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba21cac-a183-4148-9c99-211b9ca22e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:37.465816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba21cac-a183-4148-9c99-211b9ca22e3a admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:37.466678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba21cac-a183-4148-9c99-211b9ca22e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:37.466955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba21cac-a183-4148-9c99-211b9ca22e3a admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:37.467679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba21cac-a183-4148-9c99-211b9ca22e3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:37.469630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba21cac-a183-4148-9c99-211b9ca22e3a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:37.563775 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4062/8095] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:38:37 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:38:37.567877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7325b87-20e0-4576-adb4-150a2d3a4ad5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:37.578048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7325b87-20e0-4576-adb4-150a2d3a4ad5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:37.635207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c19f15-6da7-4d62-8130-c3a894b6b720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:37.635873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c19f15-6da7-4d62-8130-c3a894b6b720 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:37.636199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c19f15-6da7-4d62-8130-c3a894b6b720 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:37.636483 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c19f15-6da7-4d62-8130-c3a894b6b720 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:37.637199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c19f15-6da7-4d62-8130-c3a894b6b720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:37.639170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c19f15-6da7-4d62-8130-c3a894b6b720 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:37.704836 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4034/8096] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:38:37 2022] POST /identity/v3/users => generated 290 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:38:37.709106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba21cac-a183-4148-9c99-211b9ca22e3a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:37.720405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba21cac-a183-4148-9c99-211b9ca22e3a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:37.725077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670570b-c5dc-4a24-a29a-048268ea8f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:37.725810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670570b-c5dc-4a24-a29a-048268ea8f00 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:37.726192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670570b-c5dc-4a24-a29a-048268ea8f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:37.726471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a670570b-c5dc-4a24-a29a-048268ea8f00 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:37.727199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670570b-c5dc-4a24-a29a-048268ea8f00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:37.728969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a670570b-c5dc-4a24-a29a-048268ea8f00 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:37.740195 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4063/8097] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:38:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:38:37.744106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c19f15-6da7-4d62-8130-c3a894b6b720 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:37.759110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c19f15-6da7-4d62-8130-c3a894b6b720 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:37.763953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc52d5-33c2-483a-bf57-2a21182d02b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:37.764599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc52d5-33c2-483a-bf57-2a21182d02b6 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:37.764883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc52d5-33c2-483a-bf57-2a21182d02b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:37.765159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc52d5-33c2-483a-bf57-2a21182d02b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:37.766037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc52d5-33c2-483a-bf57-2a21182d02b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:37.767630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc52d5-33c2-483a-bf57-2a21182d02b6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:37.778030 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4035/8098] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:38:37 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:38:37.786678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670570b-c5dc-4a24-a29a-048268ea8f00 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:37.797291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a670570b-c5dc-4a24-a29a-048268ea8f00 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:37.801963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0c145-9f2f-4cf6-a6ff-ca12813b8b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:37.802853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0c145-9f2f-4cf6-a6ff-ca12813b8b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:37.803116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0c145-9f2f-4cf6-a6ff-ca12813b8b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:37.803388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0c145-9f2f-4cf6-a6ff-ca12813b8b8a admin admin] PATH_INFO: `/v3/projects/5833e78f486c4a118608290040b31cfc/users/388458a7b3504bd08f3789f53cf9ea35/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:37.817939 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b0c145-9f2f-4cf6-a6ff-ca12813b8b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5833e78f486c4a118608290040b31cfc, user_id=388458a7b3504bd08f3789f53cf9ea35, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:37.820742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b0c145-9f2f-4cf6-a6ff-ca12813b8b8a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:37.919778 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4064/8099] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:38:37 2022] PUT /identity/v3/projects/5833e78f486c4a118608290040b31cfc/users/388458a7b3504bd08f3789f53cf9ea35/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:38:37.924892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35350ee4-be75-4879-8aac-f1638b11d134 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:37.925207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35350ee4-be75-4879-8aac-f1638b11d134 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:37.925478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35350ee4-be75-4879-8aac-f1638b11d134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:37.932341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:37.943284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-35350ee4-be75-4879-8aac-f1638b11d134 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:38:37.946339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-35350ee4-be75-4879-8aac-f1638b11d134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:38.002070 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4036/8100] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:38:38.011701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0c145-9f2f-4cf6-a6ff-ca12813b8b8a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:38.014200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-38b0c145-9f2f-4cf6-a6ff-ca12813b8b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:38.074418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b0c145-9f2f-4cf6-a6ff-ca12813b8b8a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:38.078834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f31de-f906-4e7d-83b3-31e4b87b59be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:38.079486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f31de-f906-4e7d-83b3-31e4b87b59be service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:38.079774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f31de-f906-4e7d-83b3-31e4b87b59be service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:38.080056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8f31de-f906-4e7d-83b3-31e4b87b59be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:38.090210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f31de-f906-4e7d-83b3-31e4b87b59be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:38.092310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8f31de-f906-4e7d-83b3-31e4b87b59be service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:38.101912 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4065/8101] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:38:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:38:43.255771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91013121-497f-4fd8-856b-89e1a3bb7afd None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:43.256341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91013121-497f-4fd8-856b-89e1a3bb7afd None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:43.256588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91013121-497f-4fd8-856b-89e1a3bb7afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:43.289476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-91013121-497f-4fd8-856b-89e1a3bb7afd None None] MFA Rules not processed for user `388458a7b3504bd08f3789f53cf9ea35`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:38:43.290923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-91013121-497f-4fd8-856b-89e1a3bb7afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:43.362089 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4037/8102] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:38:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:38:43.368457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f31de-f906-4e7d-83b3-31e4b87b59be service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:43.376517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8f31de-f906-4e7d-83b3-31e4b87b59be service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:43.381720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640a116-5cb1-419e-8154-0b96bffbdb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:43.382429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640a116-5cb1-419e-8154-0b96bffbdb63 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:43.382726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640a116-5cb1-419e-8154-0b96bffbdb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:43.382998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d640a116-5cb1-419e-8154-0b96bffbdb63 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:43.383882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640a116-5cb1-419e-8154-0b96bffbdb63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:43.385938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d640a116-5cb1-419e-8154-0b96bffbdb63 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:43.496867 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4066/8103] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:38:43 2022] POST /identity/v3/projects => generated 379 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:38:43.499424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91013121-497f-4fd8-856b-89e1a3bb7afd None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:43.511855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91013121-497f-4fd8-856b-89e1a3bb7afd None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:43.588992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba150a5-bef5-4e5a-8983-e54e5b47f7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:43.591272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba150a5-bef5-4e5a-8983-e54e5b47f7b0 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:43.591272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba150a5-bef5-4e5a-8983-e54e5b47f7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:43.591272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba150a5-bef5-4e5a-8983-e54e5b47f7b0 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:43.591272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba150a5-bef5-4e5a-8983-e54e5b47f7b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:43.593230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba150a5-bef5-4e5a-8983-e54e5b47f7b0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:43.647910 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4038/8104] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:38:43 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:38:43.652024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640a116-5cb1-419e-8154-0b96bffbdb63 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:43.662868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d640a116-5cb1-419e-8154-0b96bffbdb63 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:43.667807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91640397-1c04-46b3-aae1-c1dfac6178ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:43.668513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91640397-1c04-46b3-aae1-c1dfac6178ce admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:43.668814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91640397-1c04-46b3-aae1-c1dfac6178ce admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:43.669077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91640397-1c04-46b3-aae1-c1dfac6178ce admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:43.669810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91640397-1c04-46b3-aae1-c1dfac6178ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:43.671448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91640397-1c04-46b3-aae1-c1dfac6178ce admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:43.682634 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4067/8105] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:38:43.687467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba150a5-bef5-4e5a-8983-e54e5b47f7b0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:43.697631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba150a5-bef5-4e5a-8983-e54e5b47f7b0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:43.703208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e3921-9989-4845-8a6d-4e2261b85c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:43.704634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e3921-9989-4845-8a6d-4e2261b85c2a admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:43.704634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e3921-9989-4845-8a6d-4e2261b85c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:43.704971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e3921-9989-4845-8a6d-4e2261b85c2a admin admin] PATH_INFO: `/v3/projects/79c5ad0a00bc47f2b8196d9ebb7fe4a3/users/72c48b3dba0a4a63a37873ba42dc0e41/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:43.721726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e3921-9989-4845-8a6d-4e2261b85c2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=79c5ad0a00bc47f2b8196d9ebb7fe4a3, user_id=72c48b3dba0a4a63a37873ba42dc0e41, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:43.724627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e3921-9989-4845-8a6d-4e2261b85c2a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:43.803293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:43.808485 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4039/8106] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:38:43 2022] PUT /identity/v3/projects/79c5ad0a00bc47f2b8196d9ebb7fe4a3/users/72c48b3dba0a4a63a37873ba42dc0e41/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:43.813118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91640397-1c04-46b3-aae1-c1dfac6178ce admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:43.823221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91640397-1c04-46b3-aae1-c1dfac6178ce admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:43.847313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:43.880876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828f702-c50a-4046-9668-d952cfe12557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:43.881476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828f702-c50a-4046-9668-d952cfe12557 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:43.881776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828f702-c50a-4046-9668-d952cfe12557 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:43.882098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828f702-c50a-4046-9668-d952cfe12557 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:43.882829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828f702-c50a-4046-9668-d952cfe12557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:43.884439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828f702-c50a-4046-9668-d952cfe12557 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:43.894184 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4068/8107] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:38:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:38:43.898388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e3921-9989-4845-8a6d-4e2261b85c2a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:43.909130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e3921-9989-4845-8a6d-4e2261b85c2a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:43.913731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0c1c2-d642-4074-9be1-f4108ce5e014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:43.914833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0c1c2-d642-4074-9be1-f4108ce5e014 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:43.915117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0c1c2-d642-4074-9be1-f4108ce5e014 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:43.915396 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0c1c2-d642-4074-9be1-f4108ce5e014 admin admin] PATH_INFO: `/v3/projects/79c5ad0a00bc47f2b8196d9ebb7fe4a3/users/72c48b3dba0a4a63a37873ba42dc0e41/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:43.917369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0c1c2-d642-4074-9be1-f4108ce5e014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79c5ad0a00bc47f2b8196d9ebb7fe4a3, user_id=72c48b3dba0a4a63a37873ba42dc0e41, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:43.920654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0c1c2-d642-4074-9be1-f4108ce5e014 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:43.930920 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4040/8108] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:38:43 2022] PUT /identity/v3/projects/79c5ad0a00bc47f2b8196d9ebb7fe4a3/users/72c48b3dba0a4a63a37873ba42dc0e41/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:49.334795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828f702-c50a-4046-9668-d952cfe12557 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:49.336784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7828f702-c50a-4046-9668-d952cfe12557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:49.504750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828f702-c50a-4046-9668-d952cfe12557 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:49.618859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59629529-bcf9-4097-8fd0-d734830f78ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:49.620112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59629529-bcf9-4097-8fd0-d734830f78ef service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:49.620410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59629529-bcf9-4097-8fd0-d734830f78ef service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:49.620881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59629529-bcf9-4097-8fd0-d734830f78ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:49.628449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0c1c2-d642-4074-9be1-f4108ce5e014 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:49.631975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-59629529-bcf9-4097-8fd0-d734830f78ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:49.632764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f7c0c1c2-d642-4074-9be1-f4108ce5e014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:49.668022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59629529-bcf9-4097-8fd0-d734830f78ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:49.671466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59629529-bcf9-4097-8fd0-d734830f78ef service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:49.719062 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4069/8109] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:38:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:38:49.778130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0c1c2-d642-4074-9be1-f4108ce5e014 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:49.784935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b45ad-07f9-4efc-9b3d-25e896aa2861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:49.785952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b45ad-07f9-4efc-9b3d-25e896aa2861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:49.786248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b45ad-07f9-4efc-9b3d-25e896aa2861 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:49.786525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b45ad-07f9-4efc-9b3d-25e896aa2861 admin admin] PATH_INFO: `/v3/users/b16d0e791edd4f119e646e7ced74bf74` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:49.788019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b45ad-07f9-4efc-9b3d-25e896aa2861 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b16d0e791edd4f119e646e7ced74bf74)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:49.795079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b45ad-07f9-4efc-9b3d-25e896aa2861 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:50.082224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f0b45ad-07f9-4efc-9b3d-25e896aa2861 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b16d0e791edd4f119e646e7ced74bf74'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:50.083225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f0b45ad-07f9-4efc-9b3d-25e896aa2861 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b16d0e791edd4f119e646e7ced74bf74'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:50.098401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f0b45ad-07f9-4efc-9b3d-25e896aa2861 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b16d0e791edd4f119e646e7ced74bf74'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:50.161636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f0b45ad-07f9-4efc-9b3d-25e896aa2861 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b16d0e791edd4f119e646e7ced74bf74'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:50.162387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f0b45ad-07f9-4efc-9b3d-25e896aa2861 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b16d0e791edd4f119e646e7ced74bf74'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:50.192958 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4041/8110] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:38:49 2022] DELETE /identity/v3/users/b16d0e791edd4f119e646e7ced74bf74 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:50.631390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59629529-bcf9-4097-8fd0-d734830f78ef service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:50.655172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59629529-bcf9-4097-8fd0-d734830f78ef service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:50.741957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:50.783017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53741e47-9de5-4b05-9222-5e1aa026de77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:50.783017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53741e47-9de5-4b05-9222-5e1aa026de77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:50.783017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53741e47-9de5-4b05-9222-5e1aa026de77 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:50.783017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53741e47-9de5-4b05-9222-5e1aa026de77 admin admin] PATH_INFO: `/v3/projects/f9c9caba58394e1480acb632405111c5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:50.785729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53741e47-9de5-4b05-9222-5e1aa026de77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9c9caba58394e1480acb632405111c5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:50.787958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53741e47-9de5-4b05-9222-5e1aa026de77 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:50.995227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53741e47-9de5-4b05-9222-5e1aa026de77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9c9caba58394e1480acb632405111c5', 'request_id': 'req-53741e47-9de5-4b05-9222-5e1aa026de77'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:51.009289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53741e47-9de5-4b05-9222-5e1aa026de77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9c9caba58394e1480acb632405111c5', 'request_id': 'req-53741e47-9de5-4b05-9222-5e1aa026de77'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:51.116583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53741e47-9de5-4b05-9222-5e1aa026de77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9c9caba58394e1480acb632405111c5', 'request_id': 'req-53741e47-9de5-4b05-9222-5e1aa026de77'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:51.143405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53741e47-9de5-4b05-9222-5e1aa026de77 admin admin] The token cache is being invalidate because project f9c9caba58394e1480acb632405111c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:38:51.143405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53741e47-9de5-4b05-9222-5e1aa026de77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:51.143695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-53741e47-9de5-4b05-9222-5e1aa026de77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:51.146020 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4070/8111] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:38:50 2022] DELETE /identity/v3/projects/f9c9caba58394e1480acb632405111c5 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:51.154092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824445c-b897-45e6-8beb-4ea48383e760 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:51.154626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824445c-b897-45e6-8beb-4ea48383e760 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:51.154732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824445c-b897-45e6-8beb-4ea48383e760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:51.182532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4824445c-b897-45e6-8beb-4ea48383e760 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:38:51.184585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4824445c-b897-45e6-8beb-4ea48383e760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:51.282190 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4042/8112] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:38:51.285121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf1354-a8d6-4d08-97a8-b7e08345be05 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:51.285415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf1354-a8d6-4d08-97a8-b7e08345be05 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:51.285700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf1354-a8d6-4d08-97a8-b7e08345be05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:51.307745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a4cf1354-a8d6-4d08-97a8-b7e08345be05 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:38:51.309320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a4cf1354-a8d6-4d08-97a8-b7e08345be05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:51.319396 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4071/8113] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:38:51.323763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824445c-b897-45e6-8beb-4ea48383e760 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:51.334832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824445c-b897-45e6-8beb-4ea48383e760 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:51.340374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ff21b-14aa-4c52-9e75-dbae907fe581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:51.344674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ff21b-14aa-4c52-9e75-dbae907fe581 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:51.344674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ff21b-14aa-4c52-9e75-dbae907fe581 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:51.344674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ff21b-14aa-4c52-9e75-dbae907fe581 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:51.345194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ff21b-14aa-4c52-9e75-dbae907fe581 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:38:51.345660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ff21b-14aa-4c52-9e75-dbae907fe581 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:51.347577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ff21b-14aa-4c52-9e75-dbae907fe581 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:51.349487 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:51.362153 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4043/8114] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:38:51.367186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf1354-a8d6-4d08-97a8-b7e08345be05 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:51.378973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf1354-a8d6-4d08-97a8-b7e08345be05 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:51.385455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31eed6-b894-44f9-93f3-a622945e1339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:51.386181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31eed6-b894-44f9-93f3-a622945e1339 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:51.386486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31eed6-b894-44f9-93f3-a622945e1339 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:51.386809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31eed6-b894-44f9-93f3-a622945e1339 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:51.387540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31eed6-b894-44f9-93f3-a622945e1339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:51.391210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31eed6-b894-44f9-93f3-a622945e1339 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:51.398980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1546f7-693c-4069-92aa-d8274ae40da6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:51.399286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1546f7-693c-4069-92aa-d8274ae40da6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:51.399714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1546f7-693c-4069-92aa-d8274ae40da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:51.436384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-af1546f7-693c-4069-92aa-d8274ae40da6 None None] MFA Rules not processed for user `72c48b3dba0a4a63a37873ba42dc0e41`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:38:51.437101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-af1546f7-693c-4069-92aa-d8274ae40da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:51.501563 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4072/8115] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:38:51 2022] POST /identity/v3/projects => generated 367 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:38:51.507070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31eed6-b894-44f9-93f3-a622945e1339 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:51.538629 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4044/8116] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:38:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:38:51.551042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1546f7-693c-4069-92aa-d8274ae40da6 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:51.552706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-af1546f7-693c-4069-92aa-d8274ae40da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:51.605171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31eed6-b894-44f9-93f3-a622945e1339 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:51.612252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead3c4f-74fe-4337-8fac-6d2abf197691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:51.612975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead3c4f-74fe-4337-8fac-6d2abf197691 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:51.613640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead3c4f-74fe-4337-8fac-6d2abf197691 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:51.613788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead3c4f-74fe-4337-8fac-6d2abf197691 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:51.614631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead3c4f-74fe-4337-8fac-6d2abf197691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:51.616884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead3c4f-74fe-4337-8fac-6d2abf197691 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:51.651301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1546f7-693c-4069-92aa-d8274ae40da6 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:51.729927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa285b-7008-4ff2-9501-e6f98aeb23b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:51.730851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa285b-7008-4ff2-9501-e6f98aeb23b1 service glance] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:51.731106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa285b-7008-4ff2-9501-e6f98aeb23b1 service glance] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:51.731635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa285b-7008-4ff2-9501-e6f98aeb23b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:51.733619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-09fa285b-7008-4ff2-9501-e6f98aeb23b1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:51.744423 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4073/8117] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:38:51 2022] POST /identity/v3/users => generated 284 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:38:51.749012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead3c4f-74fe-4337-8fac-6d2abf197691 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:51.759393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead3c4f-74fe-4337-8fac-6d2abf197691 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:51.769666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85261a5-33ba-4704-9b75-6728d5604cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:51.770480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85261a5-33ba-4704-9b75-6728d5604cec admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:51.770994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85261a5-33ba-4704-9b75-6728d5604cec admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:51.772475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85261a5-33ba-4704-9b75-6728d5604cec admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:51.773230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85261a5-33ba-4704-9b75-6728d5604cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:51.774972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85261a5-33ba-4704-9b75-6728d5604cec admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:51.786171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:51.788727 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4074/8118] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:38:51.792771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85261a5-33ba-4704-9b75-6728d5604cec admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:51.803845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85261a5-33ba-4704-9b75-6728d5604cec admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:51.808773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa285b-7008-4ff2-9501-e6f98aeb23b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:51.809402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8503af6-35b3-406a-b493-1dfcf2d3ec02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:51.811224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8503af6-35b3-406a-b493-1dfcf2d3ec02 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:51.811351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa285b-7008-4ff2-9501-e6f98aeb23b1 service glance] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:51.811897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8503af6-35b3-406a-b493-1dfcf2d3ec02 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:51.812372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8503af6-35b3-406a-b493-1dfcf2d3ec02 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:51.813317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8503af6-35b3-406a-b493-1dfcf2d3ec02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:51.815346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8503af6-35b3-406a-b493-1dfcf2d3ec02 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:51.821691 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4045/8119] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:38:51 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:38:51.828452 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4075/8120] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:38:51.837360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa285b-7008-4ff2-9501-e6f98aeb23b1 service glance] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:51.861687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa285b-7008-4ff2-9501-e6f98aeb23b1 service glance] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:51.867369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec3ce7-a4c0-4da5-8c34-7aa1d2ff1e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:51.868443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec3ce7-a4c0-4da5-8c34-7aa1d2ff1e9b admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:51.868828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec3ce7-a4c0-4da5-8c34-7aa1d2ff1e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:51.869215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ec3ce7-a4c0-4da5-8c34-7aa1d2ff1e9b admin admin] PATH_INFO: `/v3/projects/38aa3dccb32f4115b7647185e4b0c76f/users/7537906a79424e3b954366c3b92092d4/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:51.888702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec3ce7-a4c0-4da5-8c34-7aa1d2ff1e9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=38aa3dccb32f4115b7647185e4b0c76f, user_id=7537906a79424e3b954366c3b92092d4, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:51.891859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ec3ce7-a4c0-4da5-8c34-7aa1d2ff1e9b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:51.968268 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4046/8121] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:38:51 2022] PUT /identity/v3/projects/38aa3dccb32f4115b7647185e4b0c76f/users/7537906a79424e3b954366c3b92092d4/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:51.978425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2fcdb-bcfb-475c-9925-515cf83ec60c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:51.979093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2fcdb-bcfb-475c-9925-515cf83ec60c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:51.979565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f2fcdb-bcfb-475c-9925-515cf83ec60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:52.014839 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e8f2fcdb-bcfb-475c-9925-515cf83ec60c None None] MFA Rules not processed for user `7537906a79424e3b954366c3b92092d4`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:38:52.016182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e8f2fcdb-bcfb-475c-9925-515cf83ec60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:52.041641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:52.131201 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4076/8122] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:38:51 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:38:52.135465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec3ce7-a4c0-4da5-8c34-7aa1d2ff1e9b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:52.221936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ec3ce7-a4c0-4da5-8c34-7aa1d2ff1e9b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:52.226719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad26b8-b003-411c-86a5-9a1a5e817c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:52.227422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad26b8-b003-411c-86a5-9a1a5e817c90 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:52.227709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad26b8-b003-411c-86a5-9a1a5e817c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:52.227981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad26b8-b003-411c-86a5-9a1a5e817c90 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:52.228724 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad26b8-b003-411c-86a5-9a1a5e817c90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:52.230793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ad26b8-b003-411c-86a5-9a1a5e817c90 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:52.328664 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4047/8123] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:38:52 2022] POST /identity/v3/projects => generated 367 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:38:52.332449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2fcdb-bcfb-475c-9925-515cf83ec60c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:52.345803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f2fcdb-bcfb-475c-9925-515cf83ec60c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:52.409547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8f608-5203-42d8-a790-5d9d036377d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:52.410269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8f608-5203-42d8-a790-5d9d036377d1 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:52.410560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8f608-5203-42d8-a790-5d9d036377d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:52.410842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8f608-5203-42d8-a790-5d9d036377d1 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:52.411558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8f608-5203-42d8-a790-5d9d036377d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:52.413521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb8f608-5203-42d8-a790-5d9d036377d1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:52.501239 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4077/8124] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:38:52 2022] POST /identity/v3/users => generated 290 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:38:52.505618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad26b8-b003-411c-86a5-9a1a5e817c90 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:52.516614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad26b8-b003-411c-86a5-9a1a5e817c90 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:52.521753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b44961-8c8d-48b1-b8d5-7c91ffe8f86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:52.522587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b44961-8c8d-48b1-b8d5-7c91ffe8f86d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:52.523101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b44961-8c8d-48b1-b8d5-7c91ffe8f86d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:52.523387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b44961-8c8d-48b1-b8d5-7c91ffe8f86d admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:52.524152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b44961-8c8d-48b1-b8d5-7c91ffe8f86d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:52.525789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b44961-8c8d-48b1-b8d5-7c91ffe8f86d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:52.537801 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4048/8125] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:38:52.541984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8f608-5203-42d8-a790-5d9d036377d1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:52.552843 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb8f608-5203-42d8-a790-5d9d036377d1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:52.567671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70b08a-728d-4c3a-a086-e5dfe84e4f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:52.568581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70b08a-728d-4c3a-a086-e5dfe84e4f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:52.568866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70b08a-728d-4c3a-a086-e5dfe84e4f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:52.569176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd70b08a-728d-4c3a-a086-e5dfe84e4f97 admin admin] PATH_INFO: `/v3/projects/a5f057271bda4d8cad6b1e0941e6fa40/users/45e76d3f65b94900a5e2ac39445aa796/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:52.586789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70b08a-728d-4c3a-a086-e5dfe84e4f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5f057271bda4d8cad6b1e0941e6fa40, user_id=45e76d3f65b94900a5e2ac39445aa796, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:52.589719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd70b08a-728d-4c3a-a086-e5dfe84e4f97 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:52.676871 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4078/8126] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:38:52 2022] PUT /identity/v3/projects/a5f057271bda4d8cad6b1e0941e6fa40/users/45e76d3f65b94900a5e2ac39445aa796/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:52.681057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b44961-8c8d-48b1-b8d5-7c91ffe8f86d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:52.691250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b44961-8c8d-48b1-b8d5-7c91ffe8f86d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:52.748286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e9a87-68ff-4086-bcad-c64c15df11cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:52.748983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e9a87-68ff-4086-bcad-c64c15df11cb admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:52.749709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e9a87-68ff-4086-bcad-c64c15df11cb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:52.749709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e9a87-68ff-4086-bcad-c64c15df11cb admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:52.750387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e9a87-68ff-4086-bcad-c64c15df11cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:52.752077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e9a87-68ff-4086-bcad-c64c15df11cb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:52.764962 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4049/8127] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:38:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:38:52.769128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70b08a-728d-4c3a-a086-e5dfe84e4f97 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:52.780558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd70b08a-728d-4c3a-a086-e5dfe84e4f97 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:52.785757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b561c4a-7a99-49a8-a6e0-a7cb3531257f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:52.786770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b561c4a-7a99-49a8-a6e0-a7cb3531257f admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:52.787218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b561c4a-7a99-49a8-a6e0-a7cb3531257f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:52.787501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b561c4a-7a99-49a8-a6e0-a7cb3531257f admin admin] PATH_INFO: `/v3/projects/a5f057271bda4d8cad6b1e0941e6fa40/users/45e76d3f65b94900a5e2ac39445aa796/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:52.789590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b561c4a-7a99-49a8-a6e0-a7cb3531257f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5f057271bda4d8cad6b1e0941e6fa40, user_id=45e76d3f65b94900a5e2ac39445aa796, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:52.792515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b561c4a-7a99-49a8-a6e0-a7cb3531257f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:52.805023 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4079/8128] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:38:52 2022] PUT /identity/v3/projects/a5f057271bda4d8cad6b1e0941e6fa40/users/45e76d3f65b94900a5e2ac39445aa796/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:52.826085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b652e-0609-4614-9ba8-f1e3d0ed2b7c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:52.826864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b652e-0609-4614-9ba8-f1e3d0ed2b7c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:52.826864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b652e-0609-4614-9ba8-f1e3d0ed2b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:52.856306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9d9b652e-0609-4614-9ba8-f1e3d0ed2b7c None None] MFA Rules not processed for user `45e76d3f65b94900a5e2ac39445aa796`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:38:52.857884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9d9b652e-0609-4614-9ba8-f1e3d0ed2b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:52.871698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:52.951781 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4050/8129] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:38:52 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:38:52.953089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b561c4a-7a99-49a8-a6e0-a7cb3531257f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:53.026116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b561c4a-7a99-49a8-a6e0-a7cb3531257f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:53.031498 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9637256-8a26-4437-970f-eaf10bf26ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:53.032160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9637256-8a26-4437-970f-eaf10bf26ce8 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:53.032449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9637256-8a26-4437-970f-eaf10bf26ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:53.032757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9637256-8a26-4437-970f-eaf10bf26ce8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:53.033481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9637256-8a26-4437-970f-eaf10bf26ce8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:53.035423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9637256-8a26-4437-970f-eaf10bf26ce8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:53.045685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:53.157150 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4080/8130] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:38:52 2022] POST /identity/v3/projects => generated 365 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:38:53.158537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b652e-0609-4614-9ba8-f1e3d0ed2b7c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:53.170675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b652e-0609-4614-9ba8-f1e3d0ed2b7c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:53.240547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e144919-62f7-48cd-96a5-9dd86debc799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:53.241294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e144919-62f7-48cd-96a5-9dd86debc799 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:53.241590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e144919-62f7-48cd-96a5-9dd86debc799 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:53.242224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e144919-62f7-48cd-96a5-9dd86debc799 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:53.242833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e144919-62f7-48cd-96a5-9dd86debc799 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:53.244857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e144919-62f7-48cd-96a5-9dd86debc799 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:53.354992 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4051/8131] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:38:53 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:38:53.358941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9637256-8a26-4437-970f-eaf10bf26ce8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:53.369243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9637256-8a26-4437-970f-eaf10bf26ce8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:53.374849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55754234-fbf7-4b81-a463-e4aa0212362a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:53.375511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55754234-fbf7-4b81-a463-e4aa0212362a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:53.383881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55754234-fbf7-4b81-a463-e4aa0212362a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:53.383881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55754234-fbf7-4b81-a463-e4aa0212362a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:53.383881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55754234-fbf7-4b81-a463-e4aa0212362a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:53.383881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55754234-fbf7-4b81-a463-e4aa0212362a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:53.389203 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4081/8132] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:38:53.393322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e144919-62f7-48cd-96a5-9dd86debc799 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:53.404473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e144919-62f7-48cd-96a5-9dd86debc799 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:53.409895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9f0a6-2591-414d-ba4e-f408dfbe5b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:53.410580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9f0a6-2591-414d-ba4e-f408dfbe5b58 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:53.410866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9f0a6-2591-414d-ba4e-f408dfbe5b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:53.411146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9f0a6-2591-414d-ba4e-f408dfbe5b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:53.411886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9f0a6-2591-414d-ba4e-f408dfbe5b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:53.413507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9f0a6-2591-414d-ba4e-f408dfbe5b58 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:53.424118 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4052/8133] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:38:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:38:53.428322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55754234-fbf7-4b81-a463-e4aa0212362a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:53.439680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55754234-fbf7-4b81-a463-e4aa0212362a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:53.444350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3828d1b-73e0-44e5-a831-438a30e196c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:53.445231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3828d1b-73e0-44e5-a831-438a30e196c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:53.445531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3828d1b-73e0-44e5-a831-438a30e196c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:53.445915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3828d1b-73e0-44e5-a831-438a30e196c1 admin admin] PATH_INFO: `/v3/projects/4959c8f58dd54b798eb2b4495ffd5fd6/users/188e609cc7c0480295ab4183531530a5/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:53.462340 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3828d1b-73e0-44e5-a831-438a30e196c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4959c8f58dd54b798eb2b4495ffd5fd6, user_id=188e609cc7c0480295ab4183531530a5, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:53.465349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3828d1b-73e0-44e5-a831-438a30e196c1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:53.527592 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4082/8134] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:38:53 2022] PUT /identity/v3/projects/4959c8f58dd54b798eb2b4495ffd5fd6/users/188e609cc7c0480295ab4183531530a5/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:38:53.537806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1392a02-a780-4dda-8e33-bec4e0a66d9d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:53.538087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1392a02-a780-4dda-8e33-bec4e0a66d9d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:53.538584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1392a02-a780-4dda-8e33-bec4e0a66d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:53.575867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a1392a02-a780-4dda-8e33-bec4e0a66d9d None None] MFA Rules not processed for user `188e609cc7c0480295ab4183531530a5`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:38:53.581145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a1392a02-a780-4dda-8e33-bec4e0a66d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:53.641856 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4053/8135] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:38:53 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:38:53.646828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3828d1b-73e0-44e5-a831-438a30e196c1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:53.728706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3828d1b-73e0-44e5-a831-438a30e196c1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:53.734141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dbcbf-669c-4e36-b5bf-bb6ab08d6345 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:53.734848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dbcbf-669c-4e36-b5bf-bb6ab08d6345 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:53.735136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dbcbf-669c-4e36-b5bf-bb6ab08d6345 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:53.735420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9dbcbf-669c-4e36-b5bf-bb6ab08d6345 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:53.736163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dbcbf-669c-4e36-b5bf-bb6ab08d6345 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:53.737803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9dbcbf-669c-4e36-b5bf-bb6ab08d6345 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:53.838331 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4083/8136] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:38:53 2022] POST /identity/v3/domains => generated 275 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:38:53.842704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1392a02-a780-4dda-8e33-bec4e0a66d9d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:53.853443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1392a02-a780-4dda-8e33-bec4e0a66d9d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:53.865937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:53.912757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06313558-40db-47ea-9526-1807a69da4e1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:53.913422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06313558-40db-47ea-9526-1807a69da4e1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:53.913697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06313558-40db-47ea-9526-1807a69da4e1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:53.914023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06313558-40db-47ea-9526-1807a69da4e1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:53.914787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06313558-40db-47ea-9526-1807a69da4e1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:53.916413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06313558-40db-47ea-9526-1807a69da4e1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:54.011970 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4054/8137] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:38:53 2022] POST /identity/v3/roles => generated 240 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:38:54.015731 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dbcbf-669c-4e36-b5bf-bb6ab08d6345 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:54.026585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9dbcbf-669c-4e36-b5bf-bb6ab08d6345 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:54.031834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b59fd5-4198-459d-b439-c4223521607c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:54.032750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b59fd5-4198-459d-b439-c4223521607c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:54.033042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b59fd5-4198-459d-b439-c4223521607c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:54.033328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b59fd5-4198-459d-b439-c4223521607c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/2f15996cada241fb961c9b61ee4bb3ab/users/188e609cc7c0480295ab4183531530a5/roles/8688fc49d19443df8092a0ff9667320b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:54.046013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b59fd5-4198-459d-b439-c4223521607c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=2f15996cada241fb961c9b61ee4bb3ab, user_id=188e609cc7c0480295ab4183531530a5, role_id=8688fc49d19443df8092a0ff9667320b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:54.048672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b59fd5-4198-459d-b439-c4223521607c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:54.120051 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4084/8138] 10.222.0.33 () {60 vars in 1479 bytes} [Sat May 28 21:38:54 2022] PUT /identity/v3/domains/2f15996cada241fb961c9b61ee4bb3ab/users/188e609cc7c0480295ab4183531530a5/roles/8688fc49d19443df8092a0ff9667320b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:54.124170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06313558-40db-47ea-9526-1807a69da4e1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:54.134943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06313558-40db-47ea-9526-1807a69da4e1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:54.194293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb204eee-f249-4e73-bbc9-dc8e5b294858 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:54.194980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb204eee-f249-4e73-bbc9-dc8e5b294858 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:54.195268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb204eee-f249-4e73-bbc9-dc8e5b294858 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:54.195545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb204eee-f249-4e73-bbc9-dc8e5b294858 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:54.196324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb204eee-f249-4e73-bbc9-dc8e5b294858 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:54.197956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb204eee-f249-4e73-bbc9-dc8e5b294858 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:54.288911 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4055/8139] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:38:54 2022] POST /identity/v3/domains => generated 275 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:38:54.293066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b59fd5-4198-459d-b439-c4223521607c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:54.304555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b59fd5-4198-459d-b439-c4223521607c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:54.368057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9e3b0-8dcf-43fd-9df8-62c7001dc00f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:54.369219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9e3b0-8dcf-43fd-9df8-62c7001dc00f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:54.369510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9e3b0-8dcf-43fd-9df8-62c7001dc00f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:54.369795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f9e3b0-8dcf-43fd-9df8-62c7001dc00f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:54.370600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9e3b0-8dcf-43fd-9df8-62c7001dc00f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:54.372251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f9e3b0-8dcf-43fd-9df8-62c7001dc00f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:54.498121 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4085/8140] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:38:54 2022] POST /identity/v3/roles => generated 239 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:38:54.503799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb204eee-f249-4e73-bbc9-dc8e5b294858 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:54.514806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb204eee-f249-4e73-bbc9-dc8e5b294858 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:54.519800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea14b47-66e7-40c1-b698-07ee57d3ca7c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:54.520459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea14b47-66e7-40c1-b698-07ee57d3ca7c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:54.523601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea14b47-66e7-40c1-b698-07ee57d3ca7c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:54.523601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea14b47-66e7-40c1-b698-07ee57d3ca7c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/groups` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:54.523601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea14b47-66e7-40c1-b698-07ee57d3ca7c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:54.523900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea14b47-66e7-40c1-b698-07ee57d3ca7c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:54.609131 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4056/8141] 10.222.0.33 () {60 vars in 1146 bytes} [Sat May 28 21:38:54 2022] POST /identity/v3/groups => generated 325 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:38:54.613656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9e3b0-8dcf-43fd-9df8-62c7001dc00f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:54.624991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f9e3b0-8dcf-43fd-9df8-62c7001dc00f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:54.630027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812d1df-e63e-4b21-9c2c-0ef5d6f3bcdf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:54.630925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812d1df-e63e-4b21-9c2c-0ef5d6f3bcdf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:54.631208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812d1df-e63e-4b21-9c2c-0ef5d6f3bcdf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:54.632973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812d1df-e63e-4b21-9c2c-0ef5d6f3bcdf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/groups/f8387dbfa94d494c9d32743b48a783a6/users/188e609cc7c0480295ab4183531530a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:54.637108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812d1df-e63e-4b21-9c2c-0ef5d6f3bcdf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=f8387dbfa94d494c9d32743b48a783a6, user_id=188e609cc7c0480295ab4183531530a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:54.638199 ncdv-master-1 devstack@keystone.service[103103]: WARNING py.warnings [None req-1812d1df-e63e-4b21-9c2c-0ef5d6f3bcdf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:add_user_to_group": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 28 21:38:54.638199 ncdv-master-1 devstack@keystone.service[103103]: warnings.warn(msg) May 28 21:38:54.638199 ncdv-master-1 devstack@keystone.service[103103]: May 28 21:38:54.639413 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812d1df-e63e-4b21-9c2c-0ef5d6f3bcdf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:54.654008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:54.759876 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4086/8142] 10.222.0.33 () {60 vars in 1359 bytes} [Sat May 28 21:38:54 2022] PUT /identity/v3/groups/f8387dbfa94d494c9d32743b48a783a6/users/188e609cc7c0480295ab4183531530a5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:54.763770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea14b47-66e7-40c1-b698-07ee57d3ca7c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:54.782731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea14b47-66e7-40c1-b698-07ee57d3ca7c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:54.850753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa7b5a4-3150-41ab-9d3a-3ec39f405c4d tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:54.851823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7b5a4-3150-41ab-9d3a-3ec39f405c4d tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:54.852397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7b5a4-3150-41ab-9d3a-3ec39f405c4d tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:54.852740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7b5a4-3150-41ab-9d3a-3ec39f405c4d tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/8e0a9c3fc159420cb26e8f2ddf714ede/groups/f8387dbfa94d494c9d32743b48a783a6/roles/c0860064a8b0488c978e57b37eb4b29d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:54.855348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa7b5a4-3150-41ab-9d3a-3ec39f405c4d tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=8e0a9c3fc159420cb26e8f2ddf714ede, group_id=f8387dbfa94d494c9d32743b48a783a6, role_id=c0860064a8b0488c978e57b37eb4b29d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:54.859155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa7b5a4-3150-41ab-9d3a-3ec39f405c4d tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:54.961042 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4057/8143] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:38:54 2022] PUT /identity/v3/domains/8e0a9c3fc159420cb26e8f2ddf714ede/groups/f8387dbfa94d494c9d32743b48a783a6/roles/c0860064a8b0488c978e57b37eb4b29d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:54.965580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812d1df-e63e-4b21-9c2c-0ef5d6f3bcdf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:54.977456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812d1df-e63e-4b21-9c2c-0ef5d6f3bcdf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:55.039155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6ff0e-3ff0-4dca-985b-e19cbd796c10 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:55.039949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6ff0e-3ff0-4dca-985b-e19cbd796c10 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:55.040243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6ff0e-3ff0-4dca-985b-e19cbd796c10 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:55.040549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6ff0e-3ff0-4dca-985b-e19cbd796c10 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:55.041295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6ff0e-3ff0-4dca-985b-e19cbd796c10 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:55.042951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6ff0e-3ff0-4dca-985b-e19cbd796c10 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:55.196531 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4087/8144] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:38:54 2022] POST /identity/v3/domains => generated 276 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:38:55.201253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa7b5a4-3150-41ab-9d3a-3ec39f405c4d tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:55.211538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa7b5a4-3150-41ab-9d3a-3ec39f405c4d tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:55.225944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:55.273740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad47d2-9ad9-467c-83eb-9fa6b856572f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:55.274738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad47d2-9ad9-467c-83eb-9fa6b856572f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:55.275028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad47d2-9ad9-467c-83eb-9fa6b856572f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:55.275326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad47d2-9ad9-467c-83eb-9fa6b856572f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:55.276182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad47d2-9ad9-467c-83eb-9fa6b856572f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:55.277821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad47d2-9ad9-467c-83eb-9fa6b856572f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:55.397300 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4058/8145] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:38:55 2022] POST /identity/v3/roles => generated 239 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:38:55.401707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6ff0e-3ff0-4dca-985b-e19cbd796c10 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:55.414265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6ff0e-3ff0-4dca-985b-e19cbd796c10 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:55.419441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b28d0f-2dc0-401e-9c98-d574d13cafb1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:55.420372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b28d0f-2dc0-401e-9c98-d574d13cafb1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:55.420665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b28d0f-2dc0-401e-9c98-d574d13cafb1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:55.420947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b28d0f-2dc0-401e-9c98-d574d13cafb1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/5eb5d611ae774703857b691aafa561de/users/188e609cc7c0480295ab4183531530a5/roles/598af41ca4c34da6b8ef96aababbeb94` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:55.435447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b28d0f-2dc0-401e-9c98-d574d13cafb1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=5eb5d611ae774703857b691aafa561de, user_id=188e609cc7c0480295ab4183531530a5, role_id=598af41ca4c34da6b8ef96aababbeb94)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:55.438195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b28d0f-2dc0-401e-9c98-d574d13cafb1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:55.558311 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4088/8146] 10.222.0.33 () {60 vars in 1479 bytes} [Sat May 28 21:38:55 2022] PUT /identity/v3/domains/5eb5d611ae774703857b691aafa561de/users/188e609cc7c0480295ab4183531530a5/roles/598af41ca4c34da6b8ef96aababbeb94 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:38:55.562634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad47d2-9ad9-467c-83eb-9fa6b856572f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:55.586690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad47d2-9ad9-467c-83eb-9fa6b856572f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:55.709449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b63985-cc90-4e11-a465-25c04a2e9a4f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:55.710311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b63985-cc90-4e11-a465-25c04a2e9a4f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:55.710666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b63985-cc90-4e11-a465-25c04a2e9a4f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:55.711018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b63985-cc90-4e11-a465-25c04a2e9a4f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:55.711848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b63985-cc90-4e11-a465-25c04a2e9a4f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_domain()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:55.714841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b63985-cc90-4e11-a465-25c04a2e9a4f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:55.835588 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4059/8147] 10.222.0.33 () {60 vars in 1148 bytes} [Sat May 28 21:38:55 2022] POST /identity/v3/domains => generated 276 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:38:55.839404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b28d0f-2dc0-401e-9c98-d574d13cafb1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:55.868832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b28d0f-2dc0-401e-9c98-d574d13cafb1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:55.941965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dec51-6cec-4c16-bde3-5125d84dd42c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:55.942556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dec51-6cec-4c16-bde3-5125d84dd42c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:55.942887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dec51-6cec-4c16-bde3-5125d84dd42c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:55.943159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68dec51-6cec-4c16-bde3-5125d84dd42c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:55.943913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68dec51-6cec-4c16-bde3-5125d84dd42c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:55.945555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68dec51-6cec-4c16-bde3-5125d84dd42c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:56.022389 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4089/8148] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:38:55 2022] POST /identity/v3/roles => generated 240 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:38:56.024870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b63985-cc90-4e11-a465-25c04a2e9a4f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:56.040080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b63985-cc90-4e11-a465-25c04a2e9a4f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:56.046789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55453118-d49c-4ec4-9cfb-68d83825e473 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:56.047683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55453118-d49c-4ec4-9cfb-68d83825e473 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:56.048123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55453118-d49c-4ec4-9cfb-68d83825e473 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:56.048561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55453118-d49c-4ec4-9cfb-68d83825e473 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/groups` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:56.049425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55453118-d49c-4ec4-9cfb-68d83825e473 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_group()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:56.051720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55453118-d49c-4ec4-9cfb-68d83825e473 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:56.166974 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4060/8149] 10.222.0.33 () {60 vars in 1146 bytes} [Sat May 28 21:38:56 2022] POST /identity/v3/groups => generated 326 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:38:56.171289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dec51-6cec-4c16-bde3-5125d84dd42c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:56.182621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68dec51-6cec-4c16-bde3-5125d84dd42c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:56.190016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c936213-ffe1-472d-a788-a0ce2abf2742 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:56.190949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c936213-ffe1-472d-a788-a0ce2abf2742 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:56.192509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c936213-ffe1-472d-a788-a0ce2abf2742 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:56.192509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c936213-ffe1-472d-a788-a0ce2abf2742 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/groups/3a1ee6f6a76a49cda69e0abb4b920296/users/188e609cc7c0480295ab4183531530a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:56.198877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c936213-ffe1-472d-a788-a0ce2abf2742 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:add_user_to_group(group_id=3a1ee6f6a76a49cda69e0abb4b920296, user_id=188e609cc7c0480295ab4183531530a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:56.201474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c936213-ffe1-472d-a788-a0ce2abf2742 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:56.292990 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4090/8150] 10.222.0.33 () {60 vars in 1359 bytes} [Sat May 28 21:38:56 2022] PUT /identity/v3/groups/3a1ee6f6a76a49cda69e0abb4b920296/users/188e609cc7c0480295ab4183531530a5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:56.297219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55453118-d49c-4ec4-9cfb-68d83825e473 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:56.308087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55453118-d49c-4ec4-9cfb-68d83825e473 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:56.368826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1942a-9bd8-4e2e-9452-2b9baa4f2231 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:56.369863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1942a-9bd8-4e2e-9452-2b9baa4f2231 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:56.370225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1942a-9bd8-4e2e-9452-2b9baa4f2231 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:56.370583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1942a-9bd8-4e2e-9452-2b9baa4f2231 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/36ccbcd3cd4145c8bf5b29746f6078ea/groups/3a1ee6f6a76a49cda69e0abb4b920296/roles/11ac4c51c5824024b59824436263551a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:56.373583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1942a-9bd8-4e2e-9452-2b9baa4f2231 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_grant(domain_id=36ccbcd3cd4145c8bf5b29746f6078ea, group_id=3a1ee6f6a76a49cda69e0abb4b920296, role_id=11ac4c51c5824024b59824436263551a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:56.376428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1942a-9bd8-4e2e-9452-2b9baa4f2231 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:56.469809 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4061/8151] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:38:56 2022] PUT /identity/v3/domains/36ccbcd3cd4145c8bf5b29746f6078ea/groups/3a1ee6f6a76a49cda69e0abb4b920296/roles/11ac4c51c5824024b59824436263551a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:56.473952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c936213-ffe1-472d-a788-a0ce2abf2742 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:56.518348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:56.568957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c936213-ffe1-472d-a788-a0ce2abf2742 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:56.662554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd14110-1b75-4390-98c5-f8189c142e21 tempest-TokensV3TestJSON-54433841 tempest-TokensV3TestJSON-54433841-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '188e609cc7c0480295ab4183531530a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4959c8f58dd54b798eb2b4495ffd5fd6', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:56.663247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd14110-1b75-4390-98c5-f8189c142e21 tempest-TokensV3TestJSON-54433841 tempest-TokensV3TestJSON-54433841-project] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:56.663536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd14110-1b75-4390-98c5-f8189c142e21 tempest-TokensV3TestJSON-54433841 tempest-TokensV3TestJSON-54433841-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:56.663820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd14110-1b75-4390-98c5-f8189c142e21 tempest-TokensV3TestJSON-54433841 tempest-TokensV3TestJSON-54433841-project] PATH_INFO: `/v3/auth/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:56.664470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd14110-1b75-4390-98c5-f8189c142e21 tempest-TokensV3TestJSON-54433841 tempest-TokensV3TestJSON-54433841-project] RBAC: Authorizing `identity:get_auth_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:56.665580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd14110-1b75-4390-98c5-f8189c142e21 tempest-TokensV3TestJSON-54433841 tempest-TokensV3TestJSON-54433841-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:56.745015 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4091/8152] 10.222.0.33 () {58 vars in 1142 bytes} [Sat May 28 21:38:56 2022] GET /identity/v3/auth/domains => generated 1170 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:38:56.749652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1942a-9bd8-4e2e-9452-2b9baa4f2231 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:56.780352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:56.832480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1942a-9bd8-4e2e-9452-2b9baa4f2231 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:56.837232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02e45b-ad57-4b9e-95b7-52165dd30258 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:56.837980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02e45b-ad57-4b9e-95b7-52165dd30258 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:56.838271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02e45b-ad57-4b9e-95b7-52165dd30258 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:56.838555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02e45b-ad57-4b9e-95b7-52165dd30258 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/36ccbcd3cd4145c8bf5b29746f6078ea/groups/3a1ee6f6a76a49cda69e0abb4b920296/roles/11ac4c51c5824024b59824436263551a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:56.841152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02e45b-ad57-4b9e-95b7-52165dd30258 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=36ccbcd3cd4145c8bf5b29746f6078ea, group_id=3a1ee6f6a76a49cda69e0abb4b920296, role_id=11ac4c51c5824024b59824436263551a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:56.843890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02e45b-ad57-4b9e-95b7-52165dd30258 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:56.849327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-eb02e45b-ad57-4b9e-95b7-52165dd30258 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invalidating the token cache because role 11ac4c51c5824024b59824436263551a was removed from group 3a1ee6f6a76a49cda69e0abb4b920296 on domain 36ccbcd3cd4145c8bf5b29746f6078ea. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:38:56.854450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-eb02e45b-ad57-4b9e-95b7-52165dd30258 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:56.854450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-eb02e45b-ad57-4b9e-95b7-52165dd30258 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:56.948910 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4062/8153] 10.222.0.33 () {58 vars in 1466 bytes} [Sat May 28 21:38:56 2022] DELETE /identity/v3/domains/36ccbcd3cd4145c8bf5b29746f6078ea/groups/3a1ee6f6a76a49cda69e0abb4b920296/roles/11ac4c51c5824024b59824436263551a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:56.953046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd14110-1b75-4390-98c5-f8189c142e21 tempest-TokensV3TestJSON-54433841 tempest-TokensV3TestJSON-54433841-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:56.954881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ebd14110-1b75-4390-98c5-f8189c142e21 tempest-TokensV3TestJSON-54433841 tempest-TokensV3TestJSON-54433841-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:56.970654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd14110-1b75-4390-98c5-f8189c142e21 tempest-TokensV3TestJSON-54433841 tempest-TokensV3TestJSON-54433841-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:57.002263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:57.028825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35daf29a-82eb-4b05-90a6-0b4e593976d7 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:57.029740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35daf29a-82eb-4b05-90a6-0b4e593976d7 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:57.030074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35daf29a-82eb-4b05-90a6-0b4e593976d7 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:57.030360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35daf29a-82eb-4b05-90a6-0b4e593976d7 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/groups/3a1ee6f6a76a49cda69e0abb4b920296/users/188e609cc7c0480295ab4183531530a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:57.032104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35daf29a-82eb-4b05-90a6-0b4e593976d7 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=3a1ee6f6a76a49cda69e0abb4b920296, user_id=188e609cc7c0480295ab4183531530a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:57.034069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35daf29a-82eb-4b05-90a6-0b4e593976d7 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:57.103952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-35daf29a-82eb-4b05-90a6-0b4e593976d7 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '188e609cc7c0480295ab4183531530a5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:57.196939 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4092/8154] 10.222.0.33 () {58 vars in 1343 bytes} [Sat May 28 21:38:56 2022] DELETE /identity/v3/groups/3a1ee6f6a76a49cda69e0abb4b920296/users/188e609cc7c0480295ab4183531530a5 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:57.200834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02e45b-ad57-4b9e-95b7-52165dd30258 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:57.211326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02e45b-ad57-4b9e-95b7-52165dd30258 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:57.269379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f9a29-c88a-4e41-9895-97de43c6d63c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:57.270287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f9a29-c88a-4e41-9895-97de43c6d63c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:57.271617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f9a29-c88a-4e41-9895-97de43c6d63c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:57.271617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3f9a29-c88a-4e41-9895-97de43c6d63c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/groups/3a1ee6f6a76a49cda69e0abb4b920296` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:57.272087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f9a29-c88a-4e41-9895-97de43c6d63c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:delete_group(group_id=3a1ee6f6a76a49cda69e0abb4b920296)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:57.274058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3f9a29-c88a-4e41-9895-97de43c6d63c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:57.417832 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4063/8155] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:38:57 2022] DELETE /identity/v3/groups/3a1ee6f6a76a49cda69e0abb4b920296 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:57.421963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35daf29a-82eb-4b05-90a6-0b4e593976d7 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:57.432892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35daf29a-82eb-4b05-90a6-0b4e593976d7 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:57.491645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8358a-5497-4ae2-be50-d6ecc3d987bf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:57.492519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8358a-5497-4ae2-be50-d6ecc3d987bf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:57.492808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8358a-5497-4ae2-be50-d6ecc3d987bf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:57.493093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8358a-5497-4ae2-be50-d6ecc3d987bf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/roles/11ac4c51c5824024b59824436263551a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:57.494645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8358a-5497-4ae2-be50-d6ecc3d987bf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:delete_role(role_id=11ac4c51c5824024b59824436263551a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:57.496286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8358a-5497-4ae2-be50-d6ecc3d987bf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:57.630043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e3f8358a-5497-4ae2-be50-d6ecc3d987bf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invalidating the token cache because role 11ac4c51c5824024b59824436263551a has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:38:57.630428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e3f8358a-5497-4ae2-be50-d6ecc3d987bf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:57.631219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e3f8358a-5497-4ae2-be50-d6ecc3d987bf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:57.633098 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4093/8156] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:38:57 2022] DELETE /identity/v3/roles/11ac4c51c5824024b59824436263551a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:57.637196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f9a29-c88a-4e41-9895-97de43c6d63c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:57.639392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3d3f9a29-c88a-4e41-9895-97de43c6d63c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:57.652203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3f9a29-c88a-4e41-9895-97de43c6d63c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:57.711253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3832c72-f65f-4fab-b125-683a9d28d674 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:57.712277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3832c72-f65f-4fab-b125-683a9d28d674 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:57.712560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3832c72-f65f-4fab-b125-683a9d28d674 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:57.712887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3832c72-f65f-4fab-b125-683a9d28d674 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/36ccbcd3cd4145c8bf5b29746f6078ea` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:57.714283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3832c72-f65f-4fab-b125-683a9d28d674 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=36ccbcd3cd4145c8bf5b29746f6078ea)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:57.716174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3832c72-f65f-4fab-b125-683a9d28d674 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:57.724783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:57.735916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f3832c72-f65f-4fab-b125-683a9d28d674 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '36ccbcd3cd4145c8bf5b29746f6078ea'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:57.736599 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f3832c72-f65f-4fab-b125-683a9d28d674 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '36ccbcd3cd4145c8bf5b29746f6078ea'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:57.835356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f3832c72-f65f-4fab-b125-683a9d28d674 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '36ccbcd3cd4145c8bf5b29746f6078ea'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:57.836092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f3832c72-f65f-4fab-b125-683a9d28d674 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '36ccbcd3cd4145c8bf5b29746f6078ea'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:57.838717 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4064/8157] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:38:57 2022] PATCH /identity/v3/domains/36ccbcd3cd4145c8bf5b29746f6078ea => generated 277 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:38:57.844215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8358a-5497-4ae2-be50-d6ecc3d987bf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:57.854623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8358a-5497-4ae2-be50-d6ecc3d987bf tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:57.911125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:57.911991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:57.912274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:57.912549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/36ccbcd3cd4145c8bf5b29746f6078ea` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:57.922963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=36ccbcd3cd4145c8bf5b29746f6078ea)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:57.924574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:57.941626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:57.947009 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '36ccbcd3cd4145c8bf5b29746f6078ea'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:57.950242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '36ccbcd3cd4145c8bf5b29746f6078ea'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.047155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36ccbcd3cd4145c8bf5b29746f6078ea', 'request_id': 'req-0b048365-fc13-4272-8959-1a73e4357e9e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.052555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36ccbcd3cd4145c8bf5b29746f6078ea', 'request_id': 'req-0b048365-fc13-4272-8959-1a73e4357e9e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.104163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36ccbcd3cd4145c8bf5b29746f6078ea', 'request_id': 'req-0b048365-fc13-4272-8959-1a73e4357e9e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.116822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] The token cache is being invalidate because project 36ccbcd3cd4145c8bf5b29746f6078ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:38:58.117063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.117693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.126491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '36ccbcd3cd4145c8bf5b29746f6078ea', 'request_id': 'req-0b048365-fc13-4272-8959-1a73e4357e9e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.127155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '36ccbcd3cd4145c8bf5b29746f6078ea', 'request_id': 'req-0b048365-fc13-4272-8959-1a73e4357e9e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.130726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '36ccbcd3cd4145c8bf5b29746f6078ea', 'request_id': 'req-0b048365-fc13-4272-8959-1a73e4357e9e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.136503 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4094/8158] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:38:57 2022] DELETE /identity/v3/domains/36ccbcd3cd4145c8bf5b29746f6078ea => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:58.140883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3832c72-f65f-4fab-b125-683a9d28d674 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:58.142547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f3832c72-f65f-4fab-b125-683a9d28d674 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:58.155738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3832c72-f65f-4fab-b125-683a9d28d674 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:58.214784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e7222-1bf3-4c97-b456-6b6c34ad9a33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:58.215742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e7222-1bf3-4c97-b456-6b6c34ad9a33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:58.216023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e7222-1bf3-4c97-b456-6b6c34ad9a33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:58.216301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e7222-1bf3-4c97-b456-6b6c34ad9a33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/5eb5d611ae774703857b691aafa561de/users/188e609cc7c0480295ab4183531530a5/roles/598af41ca4c34da6b8ef96aababbeb94` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:58.218954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e7222-1bf3-4c97-b456-6b6c34ad9a33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=5eb5d611ae774703857b691aafa561de, user_id=188e609cc7c0480295ab4183531530a5, role_id=598af41ca4c34da6b8ef96aababbeb94)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:58.221648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e7222-1bf3-4c97-b456-6b6c34ad9a33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:58.227340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e38e7222-1bf3-4c97-b456-6b6c34ad9a33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invalidating the token cache because role 598af41ca4c34da6b8ef96aababbeb94 was removed from user 188e609cc7c0480295ab4183531530a5 on domain 5eb5d611ae774703857b691aafa561de. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:38:58.227734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e38e7222-1bf3-4c97-b456-6b6c34ad9a33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.228523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e38e7222-1bf3-4c97-b456-6b6c34ad9a33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.291479 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4065/8159] 10.222.0.33 () {58 vars in 1463 bytes} [Sat May 28 21:38:58 2022] DELETE /identity/v3/domains/5eb5d611ae774703857b691aafa561de/users/188e609cc7c0480295ab4183531530a5/roles/598af41ca4c34da6b8ef96aababbeb94 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:38:58.296178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:58.297560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:58.310129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b048365-fc13-4272-8959-1a73e4357e9e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:58.368962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf50b9-94fa-4573-8f98-0c89290f0b91 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:58.369816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf50b9-94fa-4573-8f98-0c89290f0b91 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:58.370136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf50b9-94fa-4573-8f98-0c89290f0b91 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:58.370419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf50b9-94fa-4573-8f98-0c89290f0b91 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/roles/598af41ca4c34da6b8ef96aababbeb94` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:58.371925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf50b9-94fa-4573-8f98-0c89290f0b91 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:delete_role(role_id=598af41ca4c34da6b8ef96aababbeb94)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:58.374170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf50b9-94fa-4573-8f98-0c89290f0b91 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:58.450193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9bbf50b9-94fa-4573-8f98-0c89290f0b91 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invalidating the token cache because role 598af41ca4c34da6b8ef96aababbeb94 has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:38:58.450444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9bbf50b9-94fa-4573-8f98-0c89290f0b91 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.451132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9bbf50b9-94fa-4573-8f98-0c89290f0b91 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.453092 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4095/8160] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:38:58 2022] DELETE /identity/v3/roles/598af41ca4c34da6b8ef96aababbeb94 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:58.457295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e7222-1bf3-4c97-b456-6b6c34ad9a33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:58.458728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e38e7222-1bf3-4c97-b456-6b6c34ad9a33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:58.472308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e7222-1bf3-4c97-b456-6b6c34ad9a33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:58.532794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfc81f-2dd8-43db-bbf7-c9448576934c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:58.533678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfc81f-2dd8-43db-bbf7-c9448576934c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:58.534005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfc81f-2dd8-43db-bbf7-c9448576934c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:58.534290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfc81f-2dd8-43db-bbf7-c9448576934c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/5eb5d611ae774703857b691aafa561de` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:58.535638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfc81f-2dd8-43db-bbf7-c9448576934c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=5eb5d611ae774703857b691aafa561de)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:58.537295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfc81f-2dd8-43db-bbf7-c9448576934c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:58.557819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-36cfc81f-2dd8-43db-bbf7-c9448576934c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '5eb5d611ae774703857b691aafa561de'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.558564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-36cfc81f-2dd8-43db-bbf7-c9448576934c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '5eb5d611ae774703857b691aafa561de'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.644365 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-36cfc81f-2dd8-43db-bbf7-c9448576934c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '5eb5d611ae774703857b691aafa561de'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.645002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-36cfc81f-2dd8-43db-bbf7-c9448576934c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '5eb5d611ae774703857b691aafa561de'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.647563 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4066/8161] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:38:58 2022] PATCH /identity/v3/domains/5eb5d611ae774703857b691aafa561de => generated 277 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:38:58.651553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf50b9-94fa-4573-8f98-0c89290f0b91 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:58.653082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9bbf50b9-94fa-4573-8f98-0c89290f0b91 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:58.666901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf50b9-94fa-4573-8f98-0c89290f0b91 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:58.692002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:58.731379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:58.731379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:58.731379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:58.732241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/5eb5d611ae774703857b691aafa561de` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:58.743348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=5eb5d611ae774703857b691aafa561de)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:58.744979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:58.766930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '5eb5d611ae774703857b691aafa561de'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.770218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '5eb5d611ae774703857b691aafa561de'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.905583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eb5d611ae774703857b691aafa561de', 'request_id': 'req-22c508db-fe3f-42da-bc3a-7eb129734338'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:58.911052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eb5d611ae774703857b691aafa561de', 'request_id': 'req-22c508db-fe3f-42da-bc3a-7eb129734338'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:59.000024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eb5d611ae774703857b691aafa561de', 'request_id': 'req-22c508db-fe3f-42da-bc3a-7eb129734338'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:59.013231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] The token cache is being invalidate because project 5eb5d611ae774703857b691aafa561de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:38:59.013529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:59.014777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:59.023090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '5eb5d611ae774703857b691aafa561de', 'request_id': 'req-22c508db-fe3f-42da-bc3a-7eb129734338'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:59.023732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '5eb5d611ae774703857b691aafa561de', 'request_id': 'req-22c508db-fe3f-42da-bc3a-7eb129734338'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:59.027139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '5eb5d611ae774703857b691aafa561de', 'request_id': 'req-22c508db-fe3f-42da-bc3a-7eb129734338'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:59.033657 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4096/8162] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:38:58 2022] DELETE /identity/v3/domains/5eb5d611ae774703857b691aafa561de => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:59.037823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfc81f-2dd8-43db-bbf7-c9448576934c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:59.039366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-36cfc81f-2dd8-43db-bbf7-c9448576934c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:59.051977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfc81f-2dd8-43db-bbf7-c9448576934c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:59.077915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:UPDATE=2,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:59.111296 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40870cbb-19e4-40dd-ad81-eb1f1c1bc78e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:59.112304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40870cbb-19e4-40dd-ad81-eb1f1c1bc78e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:59.112657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40870cbb-19e4-40dd-ad81-eb1f1c1bc78e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:59.113004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40870cbb-19e4-40dd-ad81-eb1f1c1bc78e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/8e0a9c3fc159420cb26e8f2ddf714ede/groups/f8387dbfa94d494c9d32743b48a783a6/roles/c0860064a8b0488c978e57b37eb4b29d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:59.115666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40870cbb-19e4-40dd-ad81-eb1f1c1bc78e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=8e0a9c3fc159420cb26e8f2ddf714ede, group_id=f8387dbfa94d494c9d32743b48a783a6, role_id=c0860064a8b0488c978e57b37eb4b29d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:59.118529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40870cbb-19e4-40dd-ad81-eb1f1c1bc78e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:59.124154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-40870cbb-19e4-40dd-ad81-eb1f1c1bc78e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invalidating the token cache because role c0860064a8b0488c978e57b37eb4b29d was removed from group f8387dbfa94d494c9d32743b48a783a6 on domain 8e0a9c3fc159420cb26e8f2ddf714ede. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:38:59.129810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-40870cbb-19e4-40dd-ad81-eb1f1c1bc78e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:59.129810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-40870cbb-19e4-40dd-ad81-eb1f1c1bc78e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:59.212540 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4067/8163] 10.222.0.33 () {58 vars in 1466 bytes} [Sat May 28 21:38:59 2022] DELETE /identity/v3/domains/8e0a9c3fc159420cb26e8f2ddf714ede/groups/f8387dbfa94d494c9d32743b48a783a6/roles/c0860064a8b0488c978e57b37eb4b29d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:59.216987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:59.218447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:59.231680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c508db-fe3f-42da-bc3a-7eb129734338 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:59.289716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7689e7-58a6-46c7-b9d4-9d1b7a7f23f4 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:59.290698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7689e7-58a6-46c7-b9d4-9d1b7a7f23f4 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:59.290982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7689e7-58a6-46c7-b9d4-9d1b7a7f23f4 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:59.291257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7689e7-58a6-46c7-b9d4-9d1b7a7f23f4 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/groups/f8387dbfa94d494c9d32743b48a783a6/users/188e609cc7c0480295ab4183531530a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:59.292991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7689e7-58a6-46c7-b9d4-9d1b7a7f23f4 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:remove_user_from_group(group_id=f8387dbfa94d494c9d32743b48a783a6, user_id=188e609cc7c0480295ab4183531530a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:59.295135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7689e7-58a6-46c7-b9d4-9d1b7a7f23f4 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:59.299460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:59.383766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ad7689e7-58a6-46c7-b9d4-9d1b7a7f23f4 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _user_callback for event identity persist_revocation_event_for_user internal for {'resource_info': '188e609cc7c0480295ab4183531530a5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:59.447863 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4097/8164] 10.222.0.33 () {58 vars in 1343 bytes} [Sat May 28 21:38:59 2022] DELETE /identity/v3/groups/f8387dbfa94d494c9d32743b48a783a6/users/188e609cc7c0480295ab4183531530a5 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:59.452019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40870cbb-19e4-40dd-ad81-eb1f1c1bc78e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:59.453723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-40870cbb-19e4-40dd-ad81-eb1f1c1bc78e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:59.467073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40870cbb-19e4-40dd-ad81-eb1f1c1bc78e tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:59.524429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd4b36-f13a-4d79-a4c0-fe004d5b6e0f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:59.525297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd4b36-f13a-4d79-a4c0-fe004d5b6e0f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:59.525589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd4b36-f13a-4d79-a4c0-fe004d5b6e0f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:59.525938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd4b36-f13a-4d79-a4c0-fe004d5b6e0f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/groups/f8387dbfa94d494c9d32743b48a783a6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:59.527156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd4b36-f13a-4d79-a4c0-fe004d5b6e0f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:delete_group(group_id=f8387dbfa94d494c9d32743b48a783a6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:59.529091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd4b36-f13a-4d79-a4c0-fe004d5b6e0f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:59.665062 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4068/8165] 10.222.0.33 () {58 vars in 1226 bytes} [Sat May 28 21:38:59 2022] DELETE /identity/v3/groups/f8387dbfa94d494c9d32743b48a783a6 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:59.669212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7689e7-58a6-46c7-b9d4-9d1b7a7f23f4 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:59.679822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7689e7-58a6-46c7-b9d4-9d1b7a7f23f4 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:59.737336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f400aae-9876-426e-9798-eae5c02a9715 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:59.738162 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f400aae-9876-426e-9798-eae5c02a9715 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:59.738454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f400aae-9876-426e-9798-eae5c02a9715 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:59.738739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f400aae-9876-426e-9798-eae5c02a9715 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/roles/c0860064a8b0488c978e57b37eb4b29d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:59.740254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f400aae-9876-426e-9798-eae5c02a9715 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:delete_role(role_id=c0860064a8b0488c978e57b37eb4b29d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:59.741877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f400aae-9876-426e-9798-eae5c02a9715 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:59.862841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4f400aae-9876-426e-9798-eae5c02a9715 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invalidating the token cache because role c0860064a8b0488c978e57b37eb4b29d has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:38:59.863141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4f400aae-9876-426e-9798-eae5c02a9715 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:59.863930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4f400aae-9876-426e-9798-eae5c02a9715 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:59.865789 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4098/8166] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:38:59 2022] DELETE /identity/v3/roles/c0860064a8b0488c978e57b37eb4b29d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:38:59.875806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd4b36-f13a-4d79-a4c0-fe004d5b6e0f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:38:59.877646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-80bd4b36-f13a-4d79-a4c0-fe004d5b6e0f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:38:59.890701 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd4b36-f13a-4d79-a4c0-fe004d5b6e0f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:38:59.947361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c945e6-0473-4bda-badf-f30e734521a0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:38:59.948261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c945e6-0473-4bda-badf-f30e734521a0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:38:59.948546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c945e6-0473-4bda-badf-f30e734521a0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:38:59.948838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c945e6-0473-4bda-badf-f30e734521a0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/8e0a9c3fc159420cb26e8f2ddf714ede` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:38:59.950188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c945e6-0473-4bda-badf-f30e734521a0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=8e0a9c3fc159420cb26e8f2ddf714ede)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:38:59.951851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c945e6-0473-4bda-badf-f30e734521a0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:38:59.971361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:59.973214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-74c945e6-0473-4bda-badf-f30e734521a0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '8e0a9c3fc159420cb26e8f2ddf714ede'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:38:59.973974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-74c945e6-0473-4bda-badf-f30e734521a0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '8e0a9c3fc159420cb26e8f2ddf714ede'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:00.078976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-74c945e6-0473-4bda-badf-f30e734521a0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '8e0a9c3fc159420cb26e8f2ddf714ede'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:00.079630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-74c945e6-0473-4bda-badf-f30e734521a0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '8e0a9c3fc159420cb26e8f2ddf714ede'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:00.083865 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4069/8167] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:38:59 2022] PATCH /identity/v3/domains/8e0a9c3fc159420cb26e8f2ddf714ede => generated 276 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:39:00.086309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f400aae-9876-426e-9798-eae5c02a9715 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:00.097429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f400aae-9876-426e-9798-eae5c02a9715 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:00.154322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:00.155696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:00.155990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:00.156277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/8e0a9c3fc159420cb26e8f2ddf714ede` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:00.166956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=8e0a9c3fc159420cb26e8f2ddf714ede)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:00.168585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:00.191665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '8e0a9c3fc159420cb26e8f2ddf714ede'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:00.195020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '8e0a9c3fc159420cb26e8f2ddf714ede'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:00.219549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:00.321665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e0a9c3fc159420cb26e8f2ddf714ede', 'request_id': 'req-16946465-9cb8-489a-af1c-e2fbb74868d9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:00.328032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e0a9c3fc159420cb26e8f2ddf714ede', 'request_id': 'req-16946465-9cb8-489a-af1c-e2fbb74868d9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:00.407029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e0a9c3fc159420cb26e8f2ddf714ede', 'request_id': 'req-16946465-9cb8-489a-af1c-e2fbb74868d9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:00.421626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] The token cache is being invalidate because project 8e0a9c3fc159420cb26e8f2ddf714ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:00.421949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:00.422663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:00.435817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '8e0a9c3fc159420cb26e8f2ddf714ede', 'request_id': 'req-16946465-9cb8-489a-af1c-e2fbb74868d9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:00.436590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '8e0a9c3fc159420cb26e8f2ddf714ede', 'request_id': 'req-16946465-9cb8-489a-af1c-e2fbb74868d9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:00.440965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '8e0a9c3fc159420cb26e8f2ddf714ede', 'request_id': 'req-16946465-9cb8-489a-af1c-e2fbb74868d9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:00.446951 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4099/8168] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:39:00 2022] DELETE /identity/v3/domains/8e0a9c3fc159420cb26e8f2ddf714ede => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:00.451131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c945e6-0473-4bda-badf-f30e734521a0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:00.453177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-74c945e6-0473-4bda-badf-f30e734521a0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:00.466355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c945e6-0473-4bda-badf-f30e734521a0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:00.528155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee35e9-da5e-4549-a40e-eb9415c9f909 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:00.529595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee35e9-da5e-4549-a40e-eb9415c9f909 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:00.529595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee35e9-da5e-4549-a40e-eb9415c9f909 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:00.530445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ee35e9-da5e-4549-a40e-eb9415c9f909 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/2f15996cada241fb961c9b61ee4bb3ab/users/188e609cc7c0480295ab4183531530a5/roles/8688fc49d19443df8092a0ff9667320b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:00.532546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee35e9-da5e-4549-a40e-eb9415c9f909 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:revoke_grant(domain_id=2f15996cada241fb961c9b61ee4bb3ab, user_id=188e609cc7c0480295ab4183531530a5, role_id=8688fc49d19443df8092a0ff9667320b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:00.535312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ee35e9-da5e-4549-a40e-eb9415c9f909 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:00.540642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-87ee35e9-da5e-4549-a40e-eb9415c9f909 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invalidating the token cache because role 8688fc49d19443df8092a0ff9667320b was removed from user 188e609cc7c0480295ab4183531530a5 on domain 2f15996cada241fb961c9b61ee4bb3ab. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:00.540951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-87ee35e9-da5e-4549-a40e-eb9415c9f909 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:00.541733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-87ee35e9-da5e-4549-a40e-eb9415c9f909 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:00.634881 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4070/8169] 10.222.0.33 () {58 vars in 1463 bytes} [Sat May 28 21:39:00 2022] DELETE /identity/v3/domains/2f15996cada241fb961c9b61ee4bb3ab/users/188e609cc7c0480295ab4183531530a5/roles/8688fc49d19443df8092a0ff9667320b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:00.640711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:00.642252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:00.655381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16946465-9cb8-489a-af1c-e2fbb74868d9 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:00.715826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc59ff2-53ee-4b5d-b098-47456688d012 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:00.716687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc59ff2-53ee-4b5d-b098-47456688d012 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:00.716975 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc59ff2-53ee-4b5d-b098-47456688d012 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:00.717292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc59ff2-53ee-4b5d-b098-47456688d012 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/roles/8688fc49d19443df8092a0ff9667320b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:00.718795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc59ff2-53ee-4b5d-b098-47456688d012 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:delete_role(role_id=8688fc49d19443df8092a0ff9667320b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:00.720589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc59ff2-53ee-4b5d-b098-47456688d012 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:00.925008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ebc59ff2-53ee-4b5d-b098-47456688d012 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invalidating the token cache because role 8688fc49d19443df8092a0ff9667320b has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:00.925302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ebc59ff2-53ee-4b5d-b098-47456688d012 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:00.926030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ebc59ff2-53ee-4b5d-b098-47456688d012 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:00.927910 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4100/8170] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:39:00 2022] DELETE /identity/v3/roles/8688fc49d19443df8092a0ff9667320b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:00.932093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee35e9-da5e-4549-a40e-eb9415c9f909 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:00.933530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-87ee35e9-da5e-4549-a40e-eb9415c9f909 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:00.946699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ee35e9-da5e-4549-a40e-eb9415c9f909 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:01.004532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac12e3-a87c-4963-b345-9b46414fb531 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:01.005376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac12e3-a87c-4963-b345-9b46414fb531 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:01.005661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac12e3-a87c-4963-b345-9b46414fb531 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:01.005979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ac12e3-a87c-4963-b345-9b46414fb531 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/2f15996cada241fb961c9b61ee4bb3ab` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:01.007294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac12e3-a87c-4963-b345-9b46414fb531 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:update_domain(domain_id=2f15996cada241fb961c9b61ee4bb3ab)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:01.008901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ac12e3-a87c-4963-b345-9b46414fb531 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:01.033132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-63ac12e3-a87c-4963-b345-9b46414fb531 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity project disabled for {'resource_info': '2f15996cada241fb961c9b61ee4bb3ab'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:01.033888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-63ac12e3-a87c-4963-b345-9b46414fb531 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity project disabled for {'resource_info': '2f15996cada241fb961c9b61ee4bb3ab'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:01.142945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-63ac12e3-a87c-4963-b345-9b46414fb531 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity domain disabled for {'resource_info': '2f15996cada241fb961c9b61ee4bb3ab'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:01.143567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-63ac12e3-a87c-4963-b345-9b46414fb531 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity domain disabled for {'resource_info': '2f15996cada241fb961c9b61ee4bb3ab'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:01.146119 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4071/8171] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:39:00 2022] PATCH /identity/v3/domains/2f15996cada241fb961c9b61ee4bb3ab => generated 276 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:39:01.150225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc59ff2-53ee-4b5d-b098-47456688d012 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:01.153675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ebc59ff2-53ee-4b5d-b098-47456688d012 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:01.166653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc59ff2-53ee-4b5d-b098-47456688d012 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:01.220615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:01.227032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:01.227895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:01.228262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:01.230195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/2f15996cada241fb961c9b61ee4bb3ab` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:01.243934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:delete_domain(domain_id=2f15996cada241fb961c9b61ee4bb3ab)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:01.245687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:01.268389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _cleanup_identity_provider for event identity domain_deleted internal for {'resource_info': '2f15996cada241fb961c9b61ee4bb3ab'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:01.271715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _domain_deleted for event identity domain_deleted internal for {'resource_info': '2f15996cada241fb961c9b61ee4bb3ab'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:01.413911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f15996cada241fb961c9b61ee4bb3ab', 'request_id': 'req-202c5461-e76f-4862-b1c4-fc6cd4fec48b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:01.419530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f15996cada241fb961c9b61ee4bb3ab', 'request_id': 'req-202c5461-e76f-4862-b1c4-fc6cd4fec48b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:01.527611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f15996cada241fb961c9b61ee4bb3ab', 'request_id': 'req-202c5461-e76f-4862-b1c4-fc6cd4fec48b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:01.541628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] The token cache is being invalidate because project 2f15996cada241fb961c9b61ee4bb3ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:01.541978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:01.542675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:01.551479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity domain deleted for {'resource_info': '2f15996cada241fb961c9b61ee4bb3ab', 'request_id': 'req-202c5461-e76f-4862-b1c4-fc6cd4fec48b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:01.552221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _delete_domain_assignments for event identity domain deleted for {'resource_info': '2f15996cada241fb961c9b61ee4bb3ab', 'request_id': 'req-202c5461-e76f-4862-b1c4-fc6cd4fec48b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:01.555693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity domain deleted for {'resource_info': '2f15996cada241fb961c9b61ee4bb3ab', 'request_id': 'req-202c5461-e76f-4862-b1c4-fc6cd4fec48b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:01.561611 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4101/8172] 10.222.0.33 () {58 vars in 1229 bytes} [Sat May 28 21:39:01 2022] DELETE /identity/v3/domains/2f15996cada241fb961c9b61ee4bb3ab => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:01.567481 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac12e3-a87c-4963-b345-9b46414fb531 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:01.568992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-63ac12e3-a87c-4963-b345-9b46414fb531 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:01.582052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ac12e3-a87c-4963-b345-9b46414fb531 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:01.619382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:UPDATE=2,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:01.644962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689ccc5-c932-4a62-8e87-8affca5db37c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:01.645826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689ccc5-c932-4a62-8e87-8affca5db37c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:01.646560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689ccc5-c932-4a62-8e87-8affca5db37c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:01.647203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689ccc5-c932-4a62-8e87-8affca5db37c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:01.648463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689ccc5-c932-4a62-8e87-8affca5db37c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:01.650621 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689ccc5-c932-4a62-8e87-8affca5db37c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:01.661877 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4072/8173] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:01 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:39:01.665913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:01.667391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:01.682227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202c5461-e76f-4862-b1c4-fc6cd4fec48b tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:01.687512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5049a54-e9b8-45ad-9911-18a0426b4d8d tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:01.688546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5049a54-e9b8-45ad-9911-18a0426b4d8d tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:01.689302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5049a54-e9b8-45ad-9911-18a0426b4d8d tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:01.689302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5049a54-e9b8-45ad-9911-18a0426b4d8d tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/projects/38aa3dccb32f4115b7647185e4b0c76f/users/45e76d3f65b94900a5e2ac39445aa796/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:01.691611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5049a54-e9b8-45ad-9911-18a0426b4d8d tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_grant(project_id=38aa3dccb32f4115b7647185e4b0c76f, user_id=45e76d3f65b94900a5e2ac39445aa796, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:01.694406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5049a54-e9b8-45ad-9911-18a0426b4d8d tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:01.785049 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4102/8174] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:01 2022] PUT /identity/v3/projects/38aa3dccb32f4115b7647185e4b0c76f/users/45e76d3f65b94900a5e2ac39445aa796/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:01.789468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689ccc5-c932-4a62-8e87-8affca5db37c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:01.801466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689ccc5-c932-4a62-8e87-8affca5db37c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:01.857982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74226b58-a7c5-4896-8f17-6e4352d016f3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:01.858364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74226b58-a7c5-4896-8f17-6e4352d016f3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:01.858662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74226b58-a7c5-4896-8f17-6e4352d016f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:01.862163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea21fbd-1fe0-4b75-bf2a-0e234fc46d95 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:01.862937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea21fbd-1fe0-4b75-bf2a-0e234fc46d95 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:01.863295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea21fbd-1fe0-4b75-bf2a-0e234fc46d95 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:01.864150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea21fbd-1fe0-4b75-bf2a-0e234fc46d95 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/auth/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:01.864282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea21fbd-1fe0-4b75-bf2a-0e234fc46d95 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:get_auth_projects()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:01.865580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea21fbd-1fe0-4b75-bf2a-0e234fc46d95 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:01.876549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-74226b58-a7c5-4896-8f17-6e4352d016f3 None None] MFA Rules not processed for user `388458a7b3504bd08f3789f53cf9ea35`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:01.877787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-74226b58-a7c5-4896-8f17-6e4352d016f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:01.911489 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4073/8175] 10.222.0.33 () {58 vars in 1145 bytes} [Sat May 28 21:39:01 2022] GET /identity/v3/auth/projects => generated 824 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:39:01.915859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea21fbd-1fe0-4b75-bf2a-0e234fc46d95 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:01.928262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea21fbd-1fe0-4b75-bf2a-0e234fc46d95 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:01.936457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2949754-3cd6-4058-8299-8819183e3302 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:01.936457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2949754-3cd6-4058-8299-8819183e3302 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:01.936457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2949754-3cd6-4058-8299-8819183e3302 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:01.937242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2949754-3cd6-4058-8299-8819183e3302 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/projects/38aa3dccb32f4115b7647185e4b0c76f/users/45e76d3f65b94900a5e2ac39445aa796/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:01.938256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2949754-3cd6-4058-8299-8819183e3302 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:revoke_grant(project_id=38aa3dccb32f4115b7647185e4b0c76f, user_id=45e76d3f65b94900a5e2ac39445aa796, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:01.938658 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4103/8176] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:01 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:01.941002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2949754-3cd6-4058-8299-8819183e3302 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:01.946879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a2949754-3cd6-4058-8299-8819183e3302 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invalidating the token cache because role 7805b93b32f1472f8365936d4d57d49a was removed from user 45e76d3f65b94900a5e2ac39445aa796 on project 38aa3dccb32f4115b7647185e4b0c76f. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:01.947238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a2949754-3cd6-4058-8299-8819183e3302 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:01.948364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-a2949754-3cd6-4058-8299-8819183e3302 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:02.047480 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4074/8177] 10.222.0.33 () {58 vars in 1466 bytes} [Sat May 28 21:39:01 2022] DELETE /identity/v3/projects/38aa3dccb32f4115b7647185e4b0c76f/users/45e76d3f65b94900a5e2ac39445aa796/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:39:02.057941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74226b58-a7c5-4896-8f17-6e4352d016f3 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:02.073702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74226b58-a7c5-4896-8f17-6e4352d016f3 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:02.104312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:02.137374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c9667-a959-4a65-964c-227cc51dec58 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:02.138130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c9667-a959-4a65-964c-227cc51dec58 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:02.139780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c9667-a959-4a65-964c-227cc51dec58 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:02.139780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c9667-a959-4a65-964c-227cc51dec58 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:02.139985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c9667-a959-4a65-964c-227cc51dec58 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:02.141919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c9667-a959-4a65-964c-227cc51dec58 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:02.245099 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4104/8178] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:39:02 2022] POST /identity/v3/users => generated 321 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:02.246842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2949754-3cd6-4058-8299-8819183e3302 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:02.248834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a2949754-3cd6-4058-8299-8819183e3302 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:02.262522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2949754-3cd6-4058-8299-8819183e3302 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:02.268112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4692e-3b7e-4dba-88cd-d7585c447066 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:02.268767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4692e-3b7e-4dba-88cd-d7585c447066 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:02.269053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4692e-3b7e-4dba-88cd-d7585c447066 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:02.269343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4692e-3b7e-4dba-88cd-d7585c447066 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:02.270066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4692e-3b7e-4dba-88cd-d7585c447066 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:02.272004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4692e-3b7e-4dba-88cd-d7585c447066 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:02.351839 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4075/8179] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:39:02 2022] POST /identity/v3/projects => generated 362 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:02.359120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c9667-a959-4a65-964c-227cc51dec58 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:02.370037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c9667-a959-4a65-964c-227cc51dec58 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:02.441369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a422cb17-bbf0-40a6-89ef-949af2ed9043 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:02.443528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422cb17-bbf0-40a6-89ef-949af2ed9043 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:02.443528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422cb17-bbf0-40a6-89ef-949af2ed9043 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:02.443528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a422cb17-bbf0-40a6-89ef-949af2ed9043 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:02.443801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a422cb17-bbf0-40a6-89ef-949af2ed9043 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:02.445686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a422cb17-bbf0-40a6-89ef-949af2ed9043 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:02.563659 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4105/8180] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:39:02 2022] POST /identity/v3/projects => generated 364 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:02.566755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4692e-3b7e-4dba-88cd-d7585c447066 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:02.577824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4692e-3b7e-4dba-88cd-d7585c447066 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:02.628033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:02.637028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d95bb-844d-43e8-8e27-80e9b0659ee1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:02.637711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d95bb-844d-43e8-8e27-80e9b0659ee1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:02.638159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d95bb-844d-43e8-8e27-80e9b0659ee1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:02.638456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d95bb-844d-43e8-8e27-80e9b0659ee1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:02.639283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d95bb-844d-43e8-8e27-80e9b0659ee1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_role()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:02.640916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d95bb-844d-43e8-8e27-80e9b0659ee1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:02.683924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a422cb17-bbf0-40a6-89ef-949af2ed9043 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:02.691902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a422cb17-bbf0-40a6-89ef-949af2ed9043 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:02.728556 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4076/8181] 10.222.0.33 () {60 vars in 1142 bytes} [Sat May 28 21:39:02 2022] POST /identity/v3/roles => generated 240 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:39:02.733255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d95bb-844d-43e8-8e27-80e9b0659ee1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:02.743958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d95bb-844d-43e8-8e27-80e9b0659ee1 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:02.749415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d4c80-428e-4574-a9ad-80036e5042c2 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:02.750714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d4c80-428e-4574-a9ad-80036e5042c2 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:02.750714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d4c80-428e-4574-a9ad-80036e5042c2 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:02.750912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d4c80-428e-4574-a9ad-80036e5042c2 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/projects/14f343d725b8400889ab947cea2a072d/users/18d6098248fe4fa085fa131af11694ed/roles/32aabc3aed4b416ca3ac2aa439d9302f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:02.767362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d4c80-428e-4574-a9ad-80036e5042c2 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_grant(project_id=14f343d725b8400889ab947cea2a072d, user_id=18d6098248fe4fa085fa131af11694ed, role_id=32aabc3aed4b416ca3ac2aa439d9302f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:02.770061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d4c80-428e-4574-a9ad-80036e5042c2 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:02.783765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a422cb17-bbf0-40a6-89ef-949af2ed9043 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:02.790432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d377c-686e-4005-87ce-d79479e8c69c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:02.791096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d377c-686e-4005-87ce-d79479e8c69c service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:02.791387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d377c-686e-4005-87ce-d79479e8c69c service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:02.791668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d377c-686e-4005-87ce-d79479e8c69c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:02.793673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e53d377c-686e-4005-87ce-d79479e8c69c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:02.809115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d377c-686e-4005-87ce-d79479e8c69c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:02.810890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d377c-686e-4005-87ce-d79479e8c69c service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:02.824867 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4106/8182] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:39:02.869131 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4077/8183] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:02 2022] PUT /identity/v3/projects/14f343d725b8400889ab947cea2a072d/users/18d6098248fe4fa085fa131af11694ed/roles/32aabc3aed4b416ca3ac2aa439d9302f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:39:02.873420 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d377c-686e-4005-87ce-d79479e8c69c service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:02.885524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d377c-686e-4005-87ce-d79479e8c69c service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:02.921427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:02.954136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bf472c-ca52-4afb-bce1-a808cc8faad0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:02.955049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf472c-ca52-4afb-bce1-a808cc8faad0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:02.955335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf472c-ca52-4afb-bce1-a808cc8faad0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:02.955616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bf472c-ca52-4afb-bce1-a808cc8faad0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/projects/1ea401920d894a0baf4cdaf1dc9bb93d/users/18d6098248fe4fa085fa131af11694ed/roles/32aabc3aed4b416ca3ac2aa439d9302f` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:02.958149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf472c-ca52-4afb-bce1-a808cc8faad0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:create_grant(project_id=1ea401920d894a0baf4cdaf1dc9bb93d, user_id=18d6098248fe4fa085fa131af11694ed, role_id=32aabc3aed4b416ca3ac2aa439d9302f)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:02.960778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bf472c-ca52-4afb-bce1-a808cc8faad0 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:03.065397 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4107/8184] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:02 2022] PUT /identity/v3/projects/1ea401920d894a0baf4cdaf1dc9bb93d/users/18d6098248fe4fa085fa131af11694ed/roles/32aabc3aed4b416ca3ac2aa439d9302f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:03.069917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c97d2-f85b-400c-92e7-8094e60c2044 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:03.070387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c97d2-f85b-400c-92e7-8094e60c2044 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:03.070492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c97d2-f85b-400c-92e7-8094e60c2044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:03.098353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1e3c97d2-f85b-400c-92e7-8094e60c2044 None None] MFA Rules not processed for user `18d6098248fe4fa085fa131af11694ed`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:03.099543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1e3c97d2-f85b-400c-92e7-8094e60c2044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:03.102863 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4078/8185] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:03 2022] POST /identity/v3/auth/tokens => generated 334 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 21:39:03.107347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e145cbf-2b6f-4e4b-86b7-d1c9e265667b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:03.112589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e145cbf-2b6f-4e4b-86b7-d1c9e265667b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:03.112589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e145cbf-2b6f-4e4b-86b7-d1c9e265667b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:03.117567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8e145cbf-2b6f-4e4b-86b7-d1c9e265667b None None] MFA Rules not processed for user `18d6098248fe4fa085fa131af11694ed`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:03.118723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8e145cbf-2b6f-4e4b-86b7-d1c9e265667b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:03.179421 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4108/8186] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:03 2022] POST /identity/v3/auth/tokens => generated 3214 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) May 28 21:39:03.183727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c97d2-f85b-400c-92e7-8094e60c2044 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:03.251416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c97d2-f85b-400c-92e7-8094e60c2044 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:03.256471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66a20c-cd85-4ab3-9575-0b2617476aa6 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:03.257135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66a20c-cd85-4ab3-9575-0b2617476aa6 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:03.258797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66a20c-cd85-4ab3-9575-0b2617476aa6 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:03.258797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b66a20c-cd85-4ab3-9575-0b2617476aa6 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:03.266561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66a20c-cd85-4ab3-9575-0b2617476aa6 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:revoke_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:03.268563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b66a20c-cd85-4ab3-9575-0b2617476aa6 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:03.368446 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4079/8187] 10.222.0.33 () {56 vars in 1307 bytes} [Sat May 28 21:39:03 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:03.372683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c11af-a3a5-4bb6-934f-cf6c85f805d8 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:03.376120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c11af-a3a5-4bb6-934f-cf6c85f805d8 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:03.376120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4c11af-a3a5-4bb6-934f-cf6c85f805d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:03.383665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2a4c11af-a3a5-4bb6-934f-cf6c85f805d8 None None] MFA Rules not processed for user `18d6098248fe4fa085fa131af11694ed`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:03.384802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2a4c11af-a3a5-4bb6-934f-cf6c85f805d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:03.456776 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4109/8188] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:03 2022] POST /identity/v3/auth/tokens => generated 3215 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 406 bytes (1 switches on core 0) May 28 21:39:03.461153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66a20c-cd85-4ab3-9575-0b2617476aa6 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:03.471486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b66a20c-cd85-4ab3-9575-0b2617476aa6 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:03.476106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a6fbf-ab16-4331-929d-109227f8e48c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:03.476967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a6fbf-ab16-4331-929d-109227f8e48c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:03.477258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a6fbf-ab16-4331-929d-109227f8e48c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:03.477534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8a6fbf-ab16-4331-929d-109227f8e48c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/roles/32aabc3aed4b416ca3ac2aa439d9302f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:03.479082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a6fbf-ab16-4331-929d-109227f8e48c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:delete_role(role_id=32aabc3aed4b416ca3ac2aa439d9302f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:03.480797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8a6fbf-ab16-4331-929d-109227f8e48c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:03.698221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5d8a6fbf-ab16-4331-929d-109227f8e48c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invalidating the token cache because role 32aabc3aed4b416ca3ac2aa439d9302f has been removed. Role assignments for users will be recalculated and enforced accordingly the next time they authenticate or validate a token {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:03.698523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5d8a6fbf-ab16-4331-929d-109227f8e48c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:03.699193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5d8a6fbf-ab16-4331-929d-109227f8e48c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:03.701097 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4080/8189] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:39:03 2022] DELETE /identity/v3/roles/32aabc3aed4b416ca3ac2aa439d9302f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:03.705292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c11af-a3a5-4bb6-934f-cf6c85f805d8 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:03.717722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4c11af-a3a5-4bb6-934f-cf6c85f805d8 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:03.775022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da013faa-242b-48eb-a610-a9b4555ecb33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:03.775947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da013faa-242b-48eb-a610-a9b4555ecb33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:03.776260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da013faa-242b-48eb-a610-a9b4555ecb33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:03.776541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da013faa-242b-48eb-a610-a9b4555ecb33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/projects/1ea401920d894a0baf4cdaf1dc9bb93d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:03.777753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da013faa-242b-48eb-a610-a9b4555ecb33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1ea401920d894a0baf4cdaf1dc9bb93d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:03.780465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da013faa-242b-48eb-a610-a9b4555ecb33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:03.795370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:03.882320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da013faa-242b-48eb-a610-a9b4555ecb33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ea401920d894a0baf4cdaf1dc9bb93d', 'request_id': 'req-da013faa-242b-48eb-a610-a9b4555ecb33'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:03.887982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da013faa-242b-48eb-a610-a9b4555ecb33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ea401920d894a0baf4cdaf1dc9bb93d', 'request_id': 'req-da013faa-242b-48eb-a610-a9b4555ecb33'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:03.949444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da013faa-242b-48eb-a610-a9b4555ecb33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ea401920d894a0baf4cdaf1dc9bb93d', 'request_id': 'req-da013faa-242b-48eb-a610-a9b4555ecb33'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:03.961069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da013faa-242b-48eb-a610-a9b4555ecb33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] The token cache is being invalidate because project 1ea401920d894a0baf4cdaf1dc9bb93d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:03.961364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da013faa-242b-48eb-a610-a9b4555ecb33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:03.962035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-da013faa-242b-48eb-a610-a9b4555ecb33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:03.963628 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4110/8190] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:03 2022] DELETE /identity/v3/projects/1ea401920d894a0baf4cdaf1dc9bb93d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:03.968441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a6fbf-ab16-4331-929d-109227f8e48c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:03.969948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5d8a6fbf-ab16-4331-929d-109227f8e48c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:03.983629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8a6fbf-ab16-4331-929d-109227f8e48c tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:04.035712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:04.042930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fdb86-10b5-43ff-bb84-7b063929fc9f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:04.043797 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fdb86-10b5-43ff-bb84-7b063929fc9f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:04.044079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fdb86-10b5-43ff-bb84-7b063929fc9f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:04.044388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503fdb86-10b5-43ff-bb84-7b063929fc9f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/projects/1ea401920d894a0baf4cdaf1dc9bb93d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:04.051722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503fdb86-10b5-43ff-bb84-7b063929fc9f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1ea401920d894a0baf4cdaf1dc9bb93d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:04.053578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503fdb86-10b5-43ff-bb84-7b063929fc9f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:04.069141 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-503fdb86-10b5-43ff-bb84-7b063929fc9f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Could not find project: 1ea401920d894a0baf4cdaf1dc9bb93d.: keystone.exception.ProjectNotFound: Could not find project: 1ea401920d894a0baf4cdaf1dc9bb93d. May 28 21:39:04.069141 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:39:04.069141 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:39:04.069141 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:39:04.069141 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:39:04.069141 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:39:04.069141 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:39:04.069141 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:39:04.069141 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:39:04.069141 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:39:04.069141 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:39:04.069141 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:39:04.069141 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete May 28 21:39:04.069141 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( May 28 21:39:04.074518 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:39:04.074518 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:39:04.074518 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project May 28 21:39:04.074518 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) May 28 21:39:04.074518 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:39:04.074518 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:39:04.074518 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:39:04.074518 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:39:04.074518 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 1ea401920d894a0baf4cdaf1dc9bb93d. May 28 21:39:04.074518 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:39:04.074518 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4081/8191] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:03 2022] DELETE /identity/v3/projects/1ea401920d894a0baf4cdaf1dc9bb93d => generated 113 bytes in 103 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:39:04.075403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da013faa-242b-48eb-a610-a9b4555ecb33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:04.076297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-da013faa-242b-48eb-a610-a9b4555ecb33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:04.089819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da013faa-242b-48eb-a610-a9b4555ecb33 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:04.095490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10eea70-ba54-4ca5-96ce-39d330c0ff60 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:04.096354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10eea70-ba54-4ca5-96ce-39d330c0ff60 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:04.101631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10eea70-ba54-4ca5-96ce-39d330c0ff60 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:04.101631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10eea70-ba54-4ca5-96ce-39d330c0ff60 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/projects/14f343d725b8400889ab947cea2a072d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:04.101631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10eea70-ba54-4ca5-96ce-39d330c0ff60 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:delete_project(project_id=14f343d725b8400889ab947cea2a072d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:04.101631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10eea70-ba54-4ca5-96ce-39d330c0ff60 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:04.225805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a10eea70-ba54-4ca5-96ce-39d330c0ff60 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14f343d725b8400889ab947cea2a072d', 'request_id': 'req-a10eea70-ba54-4ca5-96ce-39d330c0ff60'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:04.231337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a10eea70-ba54-4ca5-96ce-39d330c0ff60 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14f343d725b8400889ab947cea2a072d', 'request_id': 'req-a10eea70-ba54-4ca5-96ce-39d330c0ff60'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:04.325341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a10eea70-ba54-4ca5-96ce-39d330c0ff60 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14f343d725b8400889ab947cea2a072d', 'request_id': 'req-a10eea70-ba54-4ca5-96ce-39d330c0ff60'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:04.337986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a10eea70-ba54-4ca5-96ce-39d330c0ff60 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] The token cache is being invalidate because project 14f343d725b8400889ab947cea2a072d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:04.338297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a10eea70-ba54-4ca5-96ce-39d330c0ff60 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:04.338986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a10eea70-ba54-4ca5-96ce-39d330c0ff60 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:04.349927 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4111/8192] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:04 2022] DELETE /identity/v3/projects/14f343d725b8400889ab947cea2a072d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:04.351204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fdb86-10b5-43ff-bb84-7b063929fc9f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:04.353111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-503fdb86-10b5-43ff-bb84-7b063929fc9f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:04.366833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503fdb86-10b5-43ff-bb84-7b063929fc9f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:04.430466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383060a-3553-496f-bb12-bb40ac34b0a3 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:04.431348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383060a-3553-496f-bb12-bb40ac34b0a3 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:04.431635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383060a-3553-496f-bb12-bb40ac34b0a3 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:04.431930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383060a-3553-496f-bb12-bb40ac34b0a3 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/users/18d6098248fe4fa085fa131af11694ed` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:04.433118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383060a-3553-496f-bb12-bb40ac34b0a3 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorizing `identity:delete_user(user_id=18d6098248fe4fa085fa131af11694ed)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:04.435119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383060a-3553-496f-bb12-bb40ac34b0a3 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:04.561288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b383060a-3553-496f-bb12-bb40ac34b0a3 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18d6098248fe4fa085fa131af11694ed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:04.561993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b383060a-3553-496f-bb12-bb40ac34b0a3 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18d6098248fe4fa085fa131af11694ed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:04.568930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b383060a-3553-496f-bb12-bb40ac34b0a3 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18d6098248fe4fa085fa131af11694ed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:04.645711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b383060a-3553-496f-bb12-bb40ac34b0a3 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18d6098248fe4fa085fa131af11694ed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:04.646364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b383060a-3553-496f-bb12-bb40ac34b0a3 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18d6098248fe4fa085fa131af11694ed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:04.664871 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4082/8193] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:39:04 2022] DELETE /identity/v3/users/18d6098248fe4fa085fa131af11694ed => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:04.670087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10eea70-ba54-4ca5-96ce-39d330c0ff60 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:04.680455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10eea70-ba54-4ca5-96ce-39d330c0ff60 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:04.741824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff88ebc-b47d-4a3b-8fae-fd853600b6d8 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:04.742809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff88ebc-b47d-4a3b-8fae-fd853600b6d8 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:04.743104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff88ebc-b47d-4a3b-8fae-fd853600b6d8 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:04.743388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff88ebc-b47d-4a3b-8fae-fd853600b6d8 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/36ccbcd3cd4145c8bf5b29746f6078ea` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:04.751140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff88ebc-b47d-4a3b-8fae-fd853600b6d8 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:39:04.762323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-0ff88ebc-b47d-4a3b-8fae-fd853600b6d8 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Could not find domain: 36ccbcd3cd4145c8bf5b29746f6078ea.: keystone.exception.DomainNotFound: Could not find domain: 36ccbcd3cd4145c8bf5b29746f6078ea. May 28 21:39:04.762323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:39:04.762323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 21:39:04.762323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 21:39:04.762323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:39:04.762323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:39:04.762323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:39:04.762323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:39:04.762323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 36ccbcd3cd4145c8bf5b29746f6078ea. May 28 21:39:04.762323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:39:04.762323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:39:04.762323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:39:04.762323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:39:04.762323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:39:04.762323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:39:04.762323 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:39:04.763805 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:39:04.763805 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:39:04.763805 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:39:04.763805 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:39:04.763805 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:39:04.763805 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:39:04.763805 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:39:04.763805 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch May 28 21:39:04.763805 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') May 28 21:39:04.763805 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:39:04.763805 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:39:04.763805 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:39:04.763805 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:39:04.763805 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:39:04.763805 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:39:04.763805 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:39:04.765124 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:39:04.765124 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:39:04.765124 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:39:04.765124 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:39:04.765124 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 21:39:04.765124 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:39:04.765124 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 21:39:04.765124 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:39:04.765124 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:39:04.765124 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:39:04.765124 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 21:39:04.765124 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:39:04.765124 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 21:39:04.765124 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 21:39:04.765124 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 21:39:04.765124 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 36ccbcd3cd4145c8bf5b29746f6078ea. May 28 21:39:04.765124 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:39:04.766631 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4112/8194] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:39:04 2022] PATCH /identity/v3/domains/36ccbcd3cd4145c8bf5b29746f6078ea => generated 112 bytes in 94 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:39:04.767910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383060a-3553-496f-bb12-bb40ac34b0a3 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:04.769330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b383060a-3553-496f-bb12-bb40ac34b0a3 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:04.784082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383060a-3553-496f-bb12-bb40ac34b0a3 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:04.788953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb937621-9b05-4ece-9510-7344aa904fea tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:04.789822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb937621-9b05-4ece-9510-7344aa904fea tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:04.790138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb937621-9b05-4ece-9510-7344aa904fea tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:04.790416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb937621-9b05-4ece-9510-7344aa904fea tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/5eb5d611ae774703857b691aafa561de` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:04.798150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb937621-9b05-4ece-9510-7344aa904fea tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:39:04.808049 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-cb937621-9b05-4ece-9510-7344aa904fea tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Could not find domain: 5eb5d611ae774703857b691aafa561de.: keystone.exception.DomainNotFound: Could not find domain: 5eb5d611ae774703857b691aafa561de. May 28 21:39:04.808049 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:39:04.808049 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 21:39:04.808049 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 21:39:04.808049 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:39:04.808049 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:39:04.808049 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:39:04.808049 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:39:04.808049 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 5eb5d611ae774703857b691aafa561de. May 28 21:39:04.808049 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:39:04.808049 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:39:04.808049 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:39:04.808049 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:39:04.808049 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:39:04.808049 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:39:04.808049 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:39:04.812360 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:39:04.812360 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:39:04.812360 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:39:04.812360 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:39:04.812360 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:39:04.812360 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:39:04.812360 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:39:04.812360 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch May 28 21:39:04.812360 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') May 28 21:39:04.812360 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:39:04.812360 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:39:04.812360 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:39:04.812360 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:39:04.812360 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:39:04.812360 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:39:04.812360 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:39:04.813605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:39:04.813605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:39:04.813605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:39:04.813605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:39:04.813605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 21:39:04.813605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:39:04.813605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 21:39:04.813605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:39:04.813605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:39:04.813605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:39:04.813605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 21:39:04.813605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:39:04.813605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 21:39:04.813605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 21:39:04.813605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 21:39:04.813605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 5eb5d611ae774703857b691aafa561de. May 28 21:39:04.813605 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:39:04.814924 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4083/8195] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:39:04 2022] PATCH /identity/v3/domains/5eb5d611ae774703857b691aafa561de => generated 112 bytes in 42 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:39:04.815016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff88ebc-b47d-4a3b-8fae-fd853600b6d8 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:04.824219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff88ebc-b47d-4a3b-8fae-fd853600b6d8 tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:04.829448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2003cef-1e71-4794-bc68-b678c2e60c2a tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:04.830365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2003cef-1e71-4794-bc68-b678c2e60c2a tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:04.830651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2003cef-1e71-4794-bc68-b678c2e60c2a tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:04.830933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2003cef-1e71-4794-bc68-b678c2e60c2a tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/8e0a9c3fc159420cb26e8f2ddf714ede` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:04.839691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2003cef-1e71-4794-bc68-b678c2e60c2a tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:39:04.850373 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-b2003cef-1e71-4794-bc68-b678c2e60c2a tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Could not find domain: 8e0a9c3fc159420cb26e8f2ddf714ede.: keystone.exception.DomainNotFound: Could not find domain: 8e0a9c3fc159420cb26e8f2ddf714ede. May 28 21:39:04.850373 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:39:04.850373 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 21:39:04.850373 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 21:39:04.850373 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:39:04.850373 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:39:04.850373 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:39:04.850373 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:39:04.850373 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 8e0a9c3fc159420cb26e8f2ddf714ede. May 28 21:39:04.850373 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:39:04.850373 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:39:04.850373 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:39:04.850373 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:39:04.850373 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:39:04.850373 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:39:04.850373 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:39:04.852572 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:39:04.852572 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:39:04.852572 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:39:04.852572 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:39:04.852572 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:39:04.852572 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:39:04.852572 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:39:04.852572 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch May 28 21:39:04.852572 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') May 28 21:39:04.852572 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:39:04.852572 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:39:04.852572 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:39:04.852572 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:39:04.852572 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:39:04.852572 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:39:04.852572 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:39:04.853801 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:39:04.853801 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:39:04.853801 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:39:04.853801 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:39:04.853801 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application with Lock( May 28 21:39:04.853801 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:39:04.853801 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self._enter() May 28 21:39:04.853801 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:39:04.853801 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:39:04.853801 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:39:04.853801 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.creator() May 28 21:39:04.853801 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:39:04.853801 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application created_value = creator( May 28 21:39:04.853801 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 21:39:04.853801 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 21:39:04.853801 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 8e0a9c3fc159420cb26e8f2ddf714ede. May 28 21:39:04.853801 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:39:04.855191 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4113/8196] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:39:04 2022] PATCH /identity/v3/domains/8e0a9c3fc159420cb26e8f2ddf714ede => generated 112 bytes in 39 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:39:04.856292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb937621-9b05-4ece-9510-7344aa904fea tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:04.867346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb937621-9b05-4ece-9510-7344aa904fea tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:04.872405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc3d53-b7ef-49d7-bba2-47ddcb75760f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '45e76d3f65b94900a5e2ac39445aa796', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a5f057271bda4d8cad6b1e0941e6fa40', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:04.873349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc3d53-b7ef-49d7-bba2-47ddcb75760f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] REQUEST_METHOD: `PATCH` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:04.873726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc3d53-b7ef-49d7-bba2-47ddcb75760f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:04.874108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc3d53-b7ef-49d7-bba2-47ddcb75760f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] PATH_INFO: `/v3/domains/2f15996cada241fb961c9b61ee4bb3ab` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:04.882376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc3d53-b7ef-49d7-bba2-47ddcb75760f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 28 21:39:04.893121 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-fedc3d53-b7ef-49d7-bba2-47ddcb75760f tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Could not find domain: 2f15996cada241fb961c9b61ee4bb3ab.: keystone.exception.DomainNotFound: Could not find domain: 2f15996cada241fb961c9b61ee4bb3ab. May 28 21:39:04.893121 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:39:04.893121 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 28 21:39:04.893121 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 28 21:39:04.893121 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:39:04.893121 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:39:04.893121 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:39:04.893121 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:39:04.893121 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 2f15996cada241fb961c9b61ee4bb3ab. May 28 21:39:04.893121 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:39:04.893121 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:39:04.893121 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:39:04.893121 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:39:04.893121 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:39:04.893121 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:39:04.893121 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:39:04.894486 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:39:04.894486 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:39:04.894486 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:39:04.894486 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:39:04.894486 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:39:04.894486 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:39:04.894486 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:39:04.894486 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 137, in patch May 28 21:39:04.894486 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:update_domain') May 28 21:39:04.894486 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 28 21:39:04.894486 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 28 21:39:04.894486 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 28 21:39:04.894486 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 28 21:39:04.894486 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:39:04.894486 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:39:04.894486 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 28 21:39:04.895873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 28 21:39:04.895873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 28 21:39:04.895873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.get_or_create( May 28 21:39:04.895873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 28 21:39:04.895873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application with Lock( May 28 21:39:04.895873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 28 21:39:04.895873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._enter() May 28 21:39:04.895873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 28 21:39:04.895873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 28 21:39:04.895873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 28 21:39:04.895873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.creator() May 28 21:39:04.895873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 28 21:39:04.895873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application created_value = creator( May 28 21:39:04.895873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 28 21:39:04.895873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 28 21:39:04.895873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: 2f15996cada241fb961c9b61ee4bb3ab. May 28 21:39:04.895873 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:39:04.897771 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4084/8197] 10.222.0.33 () {60 vars in 1248 bytes} [Sat May 28 21:39:04 2022] PATCH /identity/v3/domains/2f15996cada241fb961c9b61ee4bb3ab => generated 112 bytes in 39 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) May 28 21:39:04.898383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2003cef-1e71-4794-bc68-b678c2e60c2a tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:04.900137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b2003cef-1e71-4794-bc68-b678c2e60c2a tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:04.915142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:04.970030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2003cef-1e71-4794-bc68-b678c2e60c2a tempest-TokensV3TestJSON-457773323 tempest-TokensV3TestJSON-457773323-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:04.974768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831d9e3-286b-4ca2-ba0d-77bbbf181c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:04.975678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831d9e3-286b-4ca2-ba0d-77bbbf181c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:04.975937 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831d9e3-286b-4ca2-ba0d-77bbbf181c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:04.976219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831d9e3-286b-4ca2-ba0d-77bbbf181c2d admin admin] PATH_INFO: `/v3/users/7537906a79424e3b954366c3b92092d4` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:04.977454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831d9e3-286b-4ca2-ba0d-77bbbf181c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7537906a79424e3b954366c3b92092d4)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:04.979440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831d9e3-286b-4ca2-ba0d-77bbbf181c2d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:05.197088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2831d9e3-286b-4ca2-ba0d-77bbbf181c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7537906a79424e3b954366c3b92092d4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:05.204773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2831d9e3-286b-4ca2-ba0d-77bbbf181c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7537906a79424e3b954366c3b92092d4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:05.205513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2831d9e3-286b-4ca2-ba0d-77bbbf181c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7537906a79424e3b954366c3b92092d4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:05.206269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2831d9e3-286b-4ca2-ba0d-77bbbf181c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7537906a79424e3b954366c3b92092d4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:05.276563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2831d9e3-286b-4ca2-ba0d-77bbbf181c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7537906a79424e3b954366c3b92092d4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:05.298093 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4114/8198] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:39:04 2022] DELETE /identity/v3/users/7537906a79424e3b954366c3b92092d4 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:05.302681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58656121-038d-4bae-ae68-eea040d5df83 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:05.303146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58656121-038d-4bae-ae68-eea040d5df83 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:05.303248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58656121-038d-4bae-ae68-eea040d5df83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:05.321648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-58656121-038d-4bae-ae68-eea040d5df83 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:05.322929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-58656121-038d-4bae-ae68-eea040d5df83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:05.369289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:05.390518 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4085/8199] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:39:05.397911 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831d9e3-286b-4ca2-ba0d-77bbbf181c2d admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:05.399448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2831d9e3-286b-4ca2-ba0d-77bbbf181c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:05.469181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831d9e3-286b-4ca2-ba0d-77bbbf181c2d admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:05.473314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85008ef-ea01-4149-9b9b-1f426671ab75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:05.474056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85008ef-ea01-4149-9b9b-1f426671ab75 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:05.474349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85008ef-ea01-4149-9b9b-1f426671ab75 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:05.474671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85008ef-ea01-4149-9b9b-1f426671ab75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:05.485659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85008ef-ea01-4149-9b9b-1f426671ab75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:05.487171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85008ef-ea01-4149-9b9b-1f426671ab75 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:05.499645 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4115/8200] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:06.090630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58656121-038d-4bae-ae68-eea040d5df83 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:06.093533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-58656121-038d-4bae-ae68-eea040d5df83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:06.124121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58656121-038d-4bae-ae68-eea040d5df83 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:06.124121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a583c-5859-4ffa-8747-313c85837551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:06.124121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a583c-5859-4ffa-8747-313c85837551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:06.124121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a583c-5859-4ffa-8747-313c85837551 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:06.124563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3a583c-5859-4ffa-8747-313c85837551 admin admin] PATH_INFO: `/v3/projects/38aa3dccb32f4115b7647185e4b0c76f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:06.124563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a583c-5859-4ffa-8747-313c85837551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38aa3dccb32f4115b7647185e4b0c76f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:06.124563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3a583c-5859-4ffa-8747-313c85837551 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:06.300890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4a3a583c-5859-4ffa-8747-313c85837551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38aa3dccb32f4115b7647185e4b0c76f', 'request_id': 'req-4a3a583c-5859-4ffa-8747-313c85837551'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:06.306692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4a3a583c-5859-4ffa-8747-313c85837551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38aa3dccb32f4115b7647185e4b0c76f', 'request_id': 'req-4a3a583c-5859-4ffa-8747-313c85837551'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:06.396392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4a3a583c-5859-4ffa-8747-313c85837551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38aa3dccb32f4115b7647185e4b0c76f', 'request_id': 'req-4a3a583c-5859-4ffa-8747-313c85837551'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:06.412337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4a3a583c-5859-4ffa-8747-313c85837551 admin admin] The token cache is being invalidate because project 38aa3dccb32f4115b7647185e4b0c76f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:06.413301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4a3a583c-5859-4ffa-8747-313c85837551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:06.413573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4a3a583c-5859-4ffa-8747-313c85837551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:06.418376 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4086/8201] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:06 2022] DELETE /identity/v3/projects/38aa3dccb32f4115b7647185e4b0c76f => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:06.419823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85008ef-ea01-4149-9b9b-1f426671ab75 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:06.435580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85008ef-ea01-4149-9b9b-1f426671ab75 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:06.475574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:06.502849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68399414-b38d-430d-80e2-01189aa0a1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:06.504188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68399414-b38d-430d-80e2-01189aa0a1a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:06.504654 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68399414-b38d-430d-80e2-01189aa0a1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:06.504940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68399414-b38d-430d-80e2-01189aa0a1a1 admin admin] PATH_INFO: `/v3/users/45e76d3f65b94900a5e2ac39445aa796` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:06.509505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68399414-b38d-430d-80e2-01189aa0a1a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45e76d3f65b94900a5e2ac39445aa796)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:06.511536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68399414-b38d-430d-80e2-01189aa0a1a1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:06.751636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-68399414-b38d-430d-80e2-01189aa0a1a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45e76d3f65b94900a5e2ac39445aa796'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:06.761149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-68399414-b38d-430d-80e2-01189aa0a1a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45e76d3f65b94900a5e2ac39445aa796'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:06.761921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-68399414-b38d-430d-80e2-01189aa0a1a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45e76d3f65b94900a5e2ac39445aa796'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:06.762573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-68399414-b38d-430d-80e2-01189aa0a1a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45e76d3f65b94900a5e2ac39445aa796'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:06.870760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-68399414-b38d-430d-80e2-01189aa0a1a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45e76d3f65b94900a5e2ac39445aa796'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:06.895922 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4116/8202] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:39:06 2022] DELETE /identity/v3/users/45e76d3f65b94900a5e2ac39445aa796 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:07.439198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a583c-5859-4ffa-8747-313c85837551 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:07.440851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4a3a583c-5859-4ffa-8747-313c85837551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:07.464106 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3a583c-5859-4ffa-8747-313c85837551 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:07.544564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:07.545480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:07.545765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:07.546080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa admin admin] PATH_INFO: `/v3/projects/a5f057271bda4d8cad6b1e0941e6fa40` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:07.547304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5f057271bda4d8cad6b1e0941e6fa40)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:07.549755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:07.734186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5f057271bda4d8cad6b1e0941e6fa40', 'request_id': 'req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:07.755537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5f057271bda4d8cad6b1e0941e6fa40', 'request_id': 'req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:07.859201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5f057271bda4d8cad6b1e0941e6fa40', 'request_id': 'req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:07.871129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa admin admin] The token cache is being invalidate because project a5f057271bda4d8cad6b1e0941e6fa40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:07.874224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:07.874224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:07.874437 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4087/8203] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:07 2022] DELETE /identity/v3/projects/a5f057271bda4d8cad6b1e0941e6fa40 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:07.878237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68399414-b38d-430d-80e2-01189aa0a1a1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:07.879650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-68399414-b38d-430d-80e2-01189aa0a1a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:07.892966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68399414-b38d-430d-80e2-01189aa0a1a1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:07.952492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ee3cf-0e39-45f8-9a55-eedb84041dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:07.956604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ee3cf-0e39-45f8-9a55-eedb84041dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:07.956604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ee3cf-0e39-45f8-9a55-eedb84041dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:07.956604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ee3cf-0e39-45f8-9a55-eedb84041dc3 admin admin] PATH_INFO: `/v3/users/188e609cc7c0480295ab4183531530a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:07.956604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ee3cf-0e39-45f8-9a55-eedb84041dc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=188e609cc7c0480295ab4183531530a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:07.957281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ee3cf-0e39-45f8-9a55-eedb84041dc3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:08.117169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-075ee3cf-0e39-45f8-9a55-eedb84041dc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '188e609cc7c0480295ab4183531530a5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:08.125156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-075ee3cf-0e39-45f8-9a55-eedb84041dc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '188e609cc7c0480295ab4183531530a5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:08.126229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-075ee3cf-0e39-45f8-9a55-eedb84041dc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '188e609cc7c0480295ab4183531530a5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:08.126995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-075ee3cf-0e39-45f8-9a55-eedb84041dc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '188e609cc7c0480295ab4183531530a5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:08.128857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:08.211528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-075ee3cf-0e39-45f8-9a55-eedb84041dc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '188e609cc7c0480295ab4183531530a5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:08.230014 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4117/8204] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:39:07 2022] DELETE /identity/v3/users/188e609cc7c0480295ab4183531530a5 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:08.712680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:08.714286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:08.724556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:08.729054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7db49-a41a-4d00-aa33-f052f0cf8dfa admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:08.789847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c849842-72dd-4437-80a9-4678ca2a37c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:08.790742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c849842-72dd-4437-80a9-4678ca2a37c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:08.791086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c849842-72dd-4437-80a9-4678ca2a37c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:08.791375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c849842-72dd-4437-80a9-4678ca2a37c4 admin admin] PATH_INFO: `/v3/projects/4959c8f58dd54b798eb2b4495ffd5fd6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:08.792698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c849842-72dd-4437-80a9-4678ca2a37c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4959c8f58dd54b798eb2b4495ffd5fd6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:08.797003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c849842-72dd-4437-80a9-4678ca2a37c4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:08.921604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4c849842-72dd-4437-80a9-4678ca2a37c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4959c8f58dd54b798eb2b4495ffd5fd6', 'request_id': 'req-4c849842-72dd-4437-80a9-4678ca2a37c4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:08.927527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4c849842-72dd-4437-80a9-4678ca2a37c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4959c8f58dd54b798eb2b4495ffd5fd6', 'request_id': 'req-4c849842-72dd-4437-80a9-4678ca2a37c4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:09.024405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4c849842-72dd-4437-80a9-4678ca2a37c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4959c8f58dd54b798eb2b4495ffd5fd6', 'request_id': 'req-4c849842-72dd-4437-80a9-4678ca2a37c4'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:09.036781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4c849842-72dd-4437-80a9-4678ca2a37c4 admin admin] The token cache is being invalidate because project 4959c8f58dd54b798eb2b4495ffd5fd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:09.037078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4c849842-72dd-4437-80a9-4678ca2a37c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:09.037736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4c849842-72dd-4437-80a9-4678ca2a37c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:09.039351 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4088/8205] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:08 2022] DELETE /identity/v3/projects/4959c8f58dd54b798eb2b4495ffd5fd6 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:09.044692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848aa8dc-8f10-4980-b614-e6d480ac8226 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:09.045178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848aa8dc-8f10-4980-b614-e6d480ac8226 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:09.045277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848aa8dc-8f10-4980-b614-e6d480ac8226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:09.063973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-848aa8dc-8f10-4980-b614-e6d480ac8226 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:09.065210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-848aa8dc-8f10-4980-b614-e6d480ac8226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:09.125898 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4118/8206] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:09.134806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d91b95-63be-4951-821a-d41e3818e874 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:09.135099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d91b95-63be-4951-821a-d41e3818e874 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:09.135384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d91b95-63be-4951-821a-d41e3818e874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:09.155505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-81d91b95-63be-4951-821a-d41e3818e874 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:09.156710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-81d91b95-63be-4951-821a-d41e3818e874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:09.164868 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4089/8207] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:09.169277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848aa8dc-8f10-4980-b614-e6d480ac8226 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:09.179603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848aa8dc-8f10-4980-b614-e6d480ac8226 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:09.185144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36dd1a-8de0-4c52-b92c-f653e82c6cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:09.186373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36dd1a-8de0-4c52-b92c-f653e82c6cba admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:09.194830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36dd1a-8de0-4c52-b92c-f653e82c6cba admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:09.194830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36dd1a-8de0-4c52-b92c-f653e82c6cba admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:09.194830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36dd1a-8de0-4c52-b92c-f653e82c6cba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:39:09.194830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36dd1a-8de0-4c52-b92c-f653e82c6cba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:09.194830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36dd1a-8de0-4c52-b92c-f653e82c6cba admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:09.210092 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4119/8208] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:39:09.211234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d91b95-63be-4951-821a-d41e3818e874 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:09.221257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d91b95-63be-4951-821a-d41e3818e874 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:09.227541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf96012-b2f0-4ec2-b68a-1145819c5fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:09.228359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf96012-b2f0-4ec2-b68a-1145819c5fcb admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:09.228641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf96012-b2f0-4ec2-b68a-1145819c5fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:09.228993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf96012-b2f0-4ec2-b68a-1145819c5fcb admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:09.229781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf96012-b2f0-4ec2-b68a-1145819c5fcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:09.231811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf96012-b2f0-4ec2-b68a-1145819c5fcb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:09.313763 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4090/8209] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:39:09 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:09.314986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36dd1a-8de0-4c52-b92c-f653e82c6cba admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:09.324989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36dd1a-8de0-4c52-b92c-f653e82c6cba admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:09.397864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10554774-5a46-448f-853f-b2fe5bb53b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:09.398564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10554774-5a46-448f-853f-b2fe5bb53b78 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:09.398854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10554774-5a46-448f-853f-b2fe5bb53b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:09.399778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10554774-5a46-448f-853f-b2fe5bb53b78 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:09.400495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10554774-5a46-448f-853f-b2fe5bb53b78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:09.402463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10554774-5a46-448f-853f-b2fe5bb53b78 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:09.488489 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4120/8210] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:39:09 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:39:09.492388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf96012-b2f0-4ec2-b68a-1145819c5fcb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:09.503584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf96012-b2f0-4ec2-b68a-1145819c5fcb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:09.508434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913409b3-5cb4-4bee-98c2-142d2980a108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:09.509334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913409b3-5cb4-4bee-98c2-142d2980a108 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:09.509645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913409b3-5cb4-4bee-98c2-142d2980a108 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:09.509880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913409b3-5cb4-4bee-98c2-142d2980a108 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:09.510458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913409b3-5cb4-4bee-98c2-142d2980a108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:09.512081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913409b3-5cb4-4bee-98c2-142d2980a108 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:09.524750 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4091/8211] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:09.527345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10554774-5a46-448f-853f-b2fe5bb53b78 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:09.538615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10554774-5a46-448f-853f-b2fe5bb53b78 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:09.544822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98599e2c-b0d9-4d10-9f89-a4e901b6694a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:09.546015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98599e2c-b0d9-4d10-9f89-a4e901b6694a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:09.546700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98599e2c-b0d9-4d10-9f89-a4e901b6694a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:09.546897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98599e2c-b0d9-4d10-9f89-a4e901b6694a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:09.547820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98599e2c-b0d9-4d10-9f89-a4e901b6694a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:09.549758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98599e2c-b0d9-4d10-9f89-a4e901b6694a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:09.562019 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4121/8212] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:09.566117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913409b3-5cb4-4bee-98c2-142d2980a108 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:09.578094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913409b3-5cb4-4bee-98c2-142d2980a108 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:09.583226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6dadf-c70e-4b82-b039-284584c28878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:09.584249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6dadf-c70e-4b82-b039-284584c28878 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:09.584533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6dadf-c70e-4b82-b039-284584c28878 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:09.584811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd6dadf-c70e-4b82-b039-284584c28878 admin admin] PATH_INFO: `/v3/projects/18e8fa434e0d4e53a7bdbb631bbf00ea/users/cbc4ea3d814546a4b760ce1f64702ef9/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:09.600639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd6dadf-c70e-4b82-b039-284584c28878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18e8fa434e0d4e53a7bdbb631bbf00ea, user_id=cbc4ea3d814546a4b760ce1f64702ef9, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:09.606034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd6dadf-c70e-4b82-b039-284584c28878 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:09.712755 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4092/8213] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:09 2022] PUT /identity/v3/projects/18e8fa434e0d4e53a7bdbb631bbf00ea/users/cbc4ea3d814546a4b760ce1f64702ef9/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:09.718093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814650b4-39df-404a-9fa6-644815c8a4c5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:09.718422 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814650b4-39df-404a-9fa6-644815c8a4c5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:09.718700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814650b4-39df-404a-9fa6-644815c8a4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:09.753072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-814650b4-39df-404a-9fa6-644815c8a4c5 None None] MFA Rules not processed for user `cbc4ea3d814546a4b760ce1f64702ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:09.755564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-814650b4-39df-404a-9fa6-644815c8a4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:09.790081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:09.827737 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4122/8214] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:09 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:09.833156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6dadf-c70e-4b82-b039-284584c28878 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:09.844888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd6dadf-c70e-4b82-b039-284584c28878 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:09.849298 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfcf3b-3f05-4313-b72a-bebaf616e01e tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4ea3d814546a4b760ce1f64702ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18e8fa434e0d4e53a7bdbb631bbf00ea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:09.850199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfcf3b-3f05-4313-b72a-bebaf616e01e tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:09.850497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfcf3b-3f05-4313-b72a-bebaf616e01e tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:09.850782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cfcf3b-3f05-4313-b72a-bebaf616e01e tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] PATH_INFO: `/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/application_credentials` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:09.851623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfcf3b-3f05-4313-b72a-bebaf616e01e tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorizing `identity:create_application_credential(user_id=cbc4ea3d814546a4b760ce1f64702ef9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:09.852838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cfcf3b-3f05-4313-b72a-bebaf616e01e tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:09.883920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:10.052155 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4093/8215] 10.222.0.33 () {60 vars in 1314 bytes} [Sat May 28 21:39:09 2022] POST /identity/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/application_credentials => generated 920 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:10.053358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814650b4-39df-404a-9fa6-644815c8a4c5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:10.062930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814650b4-39df-404a-9fa6-644815c8a4c5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:10.069104 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cb344-5ab1-4a73-9009-60222c1272ec tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4ea3d814546a4b760ce1f64702ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18e8fa434e0d4e53a7bdbb631bbf00ea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:10.070120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cb344-5ab1-4a73-9009-60222c1272ec tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:10.074536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cb344-5ab1-4a73-9009-60222c1272ec tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:10.074536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cb344-5ab1-4a73-9009-60222c1272ec tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] PATH_INFO: `/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/access_rules/6ad1b3f4a21f48668731b0de90d2491a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:10.074536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2cb344-5ab1-4a73-9009-60222c1272ec tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorizing `identity:delete_access_rule(user_id=cbc4ea3d814546a4b760ce1f64702ef9, access_rule_id=6ad1b3f4a21f48668731b0de90d2491a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:10.074905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2cb344-5ab1-4a73-9009-60222c1272ec tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:10.134434 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application [None req-6a2cb344-5ab1-4a73-9009-60222c1272ec tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] May not delete access rule in use: keystone.exception.ForbiddenNotSecurity: May not delete access rule in use May 28 21:39:10.134434 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:39:10.134434 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 256, in delete_access_rule May 28 21:39:10.134434 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application session.delete(ref) May 28 21:39:10.134434 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 21:39:10.134434 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application next(self.gen) May 28 21:39:10.134434 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 28 21:39:10.134434 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application yield resource May 28 21:39:10.134434 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 28 21:39:10.134434 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application next(self.gen) May 28 21:39:10.134434 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 28 21:39:10.134434 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.session.rollback() May 28 21:39:10.134434 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 28 21:39:10.134434 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.force_reraise() May 28 21:39:10.134434 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 28 21:39:10.134434 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise self.value May 28 21:39:10.135871 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 28 21:39:10.135871 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 28 21:39:10.135871 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 28 21:39:10.135871 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application session.commit() May 28 21:39:10.135871 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 28 21:39:10.135871 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 28 21:39:10.135871 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 28 21:39:10.135871 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._prepare_impl() May 28 21:39:10.135871 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 28 21:39:10.135871 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.session.flush() May 28 21:39:10.135871 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 28 21:39:10.135871 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._flush(objects) May 28 21:39:10.135871 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 28 21:39:10.135871 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 28 21:39:10.135871 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 28 21:39:10.135871 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application compat.raise_( May 28 21:39:10.137098 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 28 21:39:10.137098 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception May 28 21:39:10.137098 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 28 21:39:10.137098 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application flush_context.execute() May 28 21:39:10.137098 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 28 21:39:10.137098 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rec.execute(self) May 28 21:39:10.137098 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 577, in execute May 28 21:39:10.137098 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.dependency_processor.process_deletes(uow, states) May 28 21:39:10.137098 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 552, in process_deletes May 28 21:39:10.137098 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self._synchronize( May 28 21:39:10.137098 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/dependency.py", line 610, in _synchronize May 28 21:39:10.137098 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application sync.clear(dest, self.mapper, self.prop.synchronize_pairs) May 28 21:39:10.137098 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/sync.py", line 86, in clear May 28 21:39:10.137098 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise AssertionError( May 28 21:39:10.137098 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application AssertionError: Dependency rule tried to blank-out primary key column 'application_credential_access_rule.access_rule_id' on instance '' May 28 21:39:10.137098 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:39:10.138281 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 28 21:39:10.138281 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:39:10.138281 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:39:10.138281 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:39:10.138281 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:39:10.138281 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:39:10.138281 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:39:10.138281 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:39:10.138281 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:39:10.138281 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:39:10.138281 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:39:10.138281 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:39:10.138281 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:39:10.138281 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 763, in delete May 28 21:39:10.138281 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application PROVIDERS.application_credential_api.delete_access_rule( May 28 21:39:10.138281 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:39:10.138281 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:39:10.140517 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/core.py", line 252, in delete_access_rule May 28 21:39:10.140517 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application self.driver.delete_access_rule(access_rule_id) May 28 21:39:10.140517 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/application_credential/backends/sql.py", line 258, in delete_access_rule May 28 21:39:10.140517 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application raise exception.ForbiddenNotSecurity( May 28 21:39:10.140517 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application keystone.exception.ForbiddenNotSecurity: May not delete access rule in use May 28 21:39:10.140517 ncdv-master-1 devstack@keystone.service[103103]: ERROR keystone.server.flask.application May 28 21:39:10.140517 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4123/8216] 10.222.0.33 () {58 vars in 1361 bytes} [Sat May 28 21:39:10 2022] DELETE /identity/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/access_rules/6ad1b3f4a21f48668731b0de90d2491a => generated 89 bytes in 83 msecs (HTTP/1.1 403) 5 headers in 183 bytes (2 switches on core 0) May 28 21:39:10.141105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfcf3b-3f05-4313-b72a-bebaf616e01e tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:10.150583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cfcf3b-3f05-4313-b72a-bebaf616e01e tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:10.155589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81674a18-06e5-4f51-8311-6305c9f52cb1 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4ea3d814546a4b760ce1f64702ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18e8fa434e0d4e53a7bdbb631bbf00ea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:10.156536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81674a18-06e5-4f51-8311-6305c9f52cb1 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:10.156960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81674a18-06e5-4f51-8311-6305c9f52cb1 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:10.157422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81674a18-06e5-4f51-8311-6305c9f52cb1 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] PATH_INFO: `/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/application_credentials/fbbf2a4c2bc54e2999e2cfa96d112c44` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:10.168983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81674a18-06e5-4f51-8311-6305c9f52cb1 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorizing `identity:delete_application_credential(user_id=cbc4ea3d814546a4b760ce1f64702ef9, application_credential_id=fbbf2a4c2bc54e2999e2cfa96d112c44)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:10.170923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81674a18-06e5-4f51-8311-6305c9f52cb1 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:10.286977 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4094/8217] 10.222.0.33 () {58 vars in 1394 bytes} [Sat May 28 21:39:10 2022] DELETE /identity/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/application_credentials/fbbf2a4c2bc54e2999e2cfa96d112c44 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:10.291065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cb344-5ab1-4a73-9009-60222c1272ec tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:10.301606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cb344-5ab1-4a73-9009-60222c1272ec tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:10.305866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6fbb7-ebe2-4d8b-98fc-ba36be1cce79 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4ea3d814546a4b760ce1f64702ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18e8fa434e0d4e53a7bdbb631bbf00ea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:10.306757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6fbb7-ebe2-4d8b-98fc-ba36be1cce79 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:10.307044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6fbb7-ebe2-4d8b-98fc-ba36be1cce79 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:10.307322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6fbb7-ebe2-4d8b-98fc-ba36be1cce79 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] PATH_INFO: `/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/access_rules` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:10.308566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6fbb7-ebe2-4d8b-98fc-ba36be1cce79 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorizing `identity:list_access_rules(user_id=cbc4ea3d814546a4b760ce1f64702ef9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:10.310100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6fbb7-ebe2-4d8b-98fc-ba36be1cce79 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:10.315326 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4124/8218] 10.222.0.33 () {58 vars in 1259 bytes} [Sat May 28 21:39:10 2022] GET /identity/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/access_rules => generated 369 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:39:10.321211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81674a18-06e5-4f51-8311-6305c9f52cb1 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:10.331345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81674a18-06e5-4f51-8311-6305c9f52cb1 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:10.339975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6138fa-ea33-4532-9ffe-1364dab2065a tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4ea3d814546a4b760ce1f64702ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18e8fa434e0d4e53a7bdbb631bbf00ea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:10.340861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6138fa-ea33-4532-9ffe-1364dab2065a tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:10.341172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6138fa-ea33-4532-9ffe-1364dab2065a tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:10.341437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6138fa-ea33-4532-9ffe-1364dab2065a tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] PATH_INFO: `/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/access_rules/6ad1b3f4a21f48668731b0de90d2491a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:10.343293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6138fa-ea33-4532-9ffe-1364dab2065a tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorizing `identity:delete_access_rule(user_id=cbc4ea3d814546a4b760ce1f64702ef9, access_rule_id=6ad1b3f4a21f48668731b0de90d2491a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:10.344682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6138fa-ea33-4532-9ffe-1364dab2065a tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:10.445667 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4095/8219] 10.222.0.33 () {58 vars in 1361 bytes} [Sat May 28 21:39:10 2022] DELETE /identity/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/access_rules/6ad1b3f4a21f48668731b0de90d2491a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:39:10.449652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6fbb7-ebe2-4d8b-98fc-ba36be1cce79 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:10.467820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6fbb7-ebe2-4d8b-98fc-ba36be1cce79 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:10.469808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d637b-536e-45c8-89dd-d838071d655e tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4ea3d814546a4b760ce1f64702ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18e8fa434e0d4e53a7bdbb631bbf00ea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:10.470727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d637b-536e-45c8-89dd-d838071d655e tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:10.471119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d637b-536e-45c8-89dd-d838071d655e tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:10.471602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d637b-536e-45c8-89dd-d838071d655e tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] PATH_INFO: `/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/access_rules` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:10.473142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d637b-536e-45c8-89dd-d838071d655e tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorizing `identity:list_access_rules(user_id=cbc4ea3d814546a4b760ce1f64702ef9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:10.474743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47d637b-536e-45c8-89dd-d838071d655e tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:10.480090 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4125/8220] 10.222.0.33 () {58 vars in 1259 bytes} [Sat May 28 21:39:10 2022] GET /identity/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/access_rules => generated 158 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:39:10.486122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6138fa-ea33-4532-9ffe-1364dab2065a tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:10.495712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6138fa-ea33-4532-9ffe-1364dab2065a tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:10.502689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171c2ff-8fb1-484c-bb06-fc681ed8dd26 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4ea3d814546a4b760ce1f64702ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18e8fa434e0d4e53a7bdbb631bbf00ea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:10.503572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171c2ff-8fb1-484c-bb06-fc681ed8dd26 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:10.503864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171c2ff-8fb1-484c-bb06-fc681ed8dd26 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:10.504146 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171c2ff-8fb1-484c-bb06-fc681ed8dd26 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] PATH_INFO: `/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/application_credentials` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:10.504894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171c2ff-8fb1-484c-bb06-fc681ed8dd26 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorizing `identity:create_application_credential(user_id=cbc4ea3d814546a4b760ce1f64702ef9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:10.506076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171c2ff-8fb1-484c-bb06-fc681ed8dd26 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:10.650734 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4096/8221] 10.222.0.33 () {60 vars in 1314 bytes} [Sat May 28 21:39:10 2022] POST /identity/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/application_credentials => generated 921 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:10.652075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d637b-536e-45c8-89dd-d838071d655e tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:10.662344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d637b-536e-45c8-89dd-d838071d655e tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:10.666661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dc18c-7695-444f-9680-6c3a1dfffaec tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4ea3d814546a4b760ce1f64702ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18e8fa434e0d4e53a7bdbb631bbf00ea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:10.667592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dc18c-7695-444f-9680-6c3a1dfffaec tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:10.668147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dc18c-7695-444f-9680-6c3a1dfffaec tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:10.668462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321dc18c-7695-444f-9680-6c3a1dfffaec tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] PATH_INFO: `/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/access_rules` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:10.670889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321dc18c-7695-444f-9680-6c3a1dfffaec tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorizing `identity:list_access_rules(user_id=cbc4ea3d814546a4b760ce1f64702ef9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:10.672553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321dc18c-7695-444f-9680-6c3a1dfffaec tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:10.678713 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4126/8222] 10.222.0.33 () {58 vars in 1259 bytes} [Sat May 28 21:39:10 2022] GET /identity/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/access_rules => generated 369 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:39:10.683842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171c2ff-8fb1-484c-bb06-fc681ed8dd26 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:10.693952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171c2ff-8fb1-484c-bb06-fc681ed8dd26 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:10.698884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7bcdc-9893-43f2-804d-a56c8ff5a34a tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4ea3d814546a4b760ce1f64702ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18e8fa434e0d4e53a7bdbb631bbf00ea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:10.699811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7bcdc-9893-43f2-804d-a56c8ff5a34a tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:10.700097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7bcdc-9893-43f2-804d-a56c8ff5a34a tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:10.700377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7bcdc-9893-43f2-804d-a56c8ff5a34a tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] PATH_INFO: `/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/application_credentials` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:10.701129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7bcdc-9893-43f2-804d-a56c8ff5a34a tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorizing `identity:create_application_credential(user_id=cbc4ea3d814546a4b760ce1f64702ef9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:10.702439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7bcdc-9893-43f2-804d-a56c8ff5a34a tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:10.752731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:INSERT=8,keystone:DELETE=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:10.891119 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4097/8223] 10.222.0.33 () {60 vars in 1314 bytes} [Sat May 28 21:39:10 2022] POST /identity/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/application_credentials => generated 920 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:10.892449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dc18c-7695-444f-9680-6c3a1dfffaec tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:10.903852 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321dc18c-7695-444f-9680-6c3a1dfffaec tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:10.920477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2293840-b531-48d6-98df-884204f8a4e4 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4ea3d814546a4b760ce1f64702ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18e8fa434e0d4e53a7bdbb631bbf00ea', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:10.924655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2293840-b531-48d6-98df-884204f8a4e4 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:10.925164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2293840-b531-48d6-98df-884204f8a4e4 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:10.925695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2293840-b531-48d6-98df-884204f8a4e4 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] PATH_INFO: `/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/access_rules/cc7b1e1f06534b158065a14156abb9bb` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:10.927238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2293840-b531-48d6-98df-884204f8a4e4 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorizing `identity:get_access_rule(user_id=cbc4ea3d814546a4b760ce1f64702ef9, access_rule_id=cc7b1e1f06534b158065a14156abb9bb)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:10.928936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2293840-b531-48d6-98df-884204f8a4e4 tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:10.941116 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4127/8224] 10.222.0.33 () {58 vars in 1358 bytes} [Sat May 28 21:39:10 2022] GET /identity/v3/users/cbc4ea3d814546a4b760ce1f64702ef9/access_rules/cc7b1e1f06534b158065a14156abb9bb => generated 229 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:39:10.951850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7bcdc-9893-43f2-804d-a56c8ff5a34a tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:11.021699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7bcdc-9893-43f2-804d-a56c8ff5a34a tempest-AccessRulesV3Test-533880812 tempest-AccessRulesV3Test-533880812-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:11.027814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb2e81-ca50-49ce-8842-58df7195717a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:11.028798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb2e81-ca50-49ce-8842-58df7195717a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:11.029112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb2e81-ca50-49ce-8842-58df7195717a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:11.029592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bb2e81-ca50-49ce-8842-58df7195717a admin admin] PATH_INFO: `/v3/users/cbc4ea3d814546a4b760ce1f64702ef9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:11.031376 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb2e81-ca50-49ce-8842-58df7195717a admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbc4ea3d814546a4b760ce1f64702ef9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:11.033385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bb2e81-ca50-49ce-8842-58df7195717a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:11.529175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-61bb2e81-ca50-49ce-8842-58df7195717a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbc4ea3d814546a4b760ce1f64702ef9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:11.529889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-61bb2e81-ca50-49ce-8842-58df7195717a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbc4ea3d814546a4b760ce1f64702ef9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:11.536738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-61bb2e81-ca50-49ce-8842-58df7195717a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbc4ea3d814546a4b760ce1f64702ef9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:11.604985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-61bb2e81-ca50-49ce-8842-58df7195717a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbc4ea3d814546a4b760ce1f64702ef9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:11.605687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-61bb2e81-ca50-49ce-8842-58df7195717a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbc4ea3d814546a4b760ce1f64702ef9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:11.846104 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4098/8225] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:39:10 2022] DELETE /identity/v3/users/cbc4ea3d814546a4b760ce1f64702ef9 => generated 0 bytes in 895 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:11.850471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ea8e2-a84b-4141-a065-77242cf53073 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:11.850888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ea8e2-a84b-4141-a065-77242cf53073 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:11.851030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4ea8e2-a84b-4141-a065-77242cf53073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:11.869433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ff4ea8e2-a84b-4141-a065-77242cf53073 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:11.870669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ff4ea8e2-a84b-4141-a065-77242cf53073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:11.929975 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4128/8226] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:11.939736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb2e81-ca50-49ce-8842-58df7195717a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:11.941227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-61bb2e81-ca50-49ce-8842-58df7195717a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:11.977951 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:12.003261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bb2e81-ca50-49ce-8842-58df7195717a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:12.007180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b181dadf-eebd-4f73-93db-6488fa2fa72f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:12.007829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181dadf-eebd-4f73-93db-6488fa2fa72f service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:12.008114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181dadf-eebd-4f73-93db-6488fa2fa72f service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:12.008390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b181dadf-eebd-4f73-93db-6488fa2fa72f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:12.017584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181dadf-eebd-4f73-93db-6488fa2fa72f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:12.038331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b181dadf-eebd-4f73-93db-6488fa2fa72f service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:12.038331 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4099/8227] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:12.447133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ea8e2-a84b-4141-a065-77242cf53073 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:12.448878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ff4ea8e2-a84b-4141-a065-77242cf53073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:12.463241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4ea8e2-a84b-4141-a065-77242cf53073 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:12.469000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c357d17-f300-4d60-88d3-4d1ee5bc338d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:12.469917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c357d17-f300-4d60-88d3-4d1ee5bc338d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:12.470211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c357d17-f300-4d60-88d3-4d1ee5bc338d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:12.470493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c357d17-f300-4d60-88d3-4d1ee5bc338d admin admin] PATH_INFO: `/v3/projects/18e8fa434e0d4e53a7bdbb631bbf00ea` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:12.471822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c357d17-f300-4d60-88d3-4d1ee5bc338d admin admin] RBAC: Authorizing `identity:delete_project(project_id=18e8fa434e0d4e53a7bdbb631bbf00ea)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:12.473776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c357d17-f300-4d60-88d3-4d1ee5bc338d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:12.596444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c357d17-f300-4d60-88d3-4d1ee5bc338d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18e8fa434e0d4e53a7bdbb631bbf00ea', 'request_id': 'req-4c357d17-f300-4d60-88d3-4d1ee5bc338d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:12.602052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c357d17-f300-4d60-88d3-4d1ee5bc338d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18e8fa434e0d4e53a7bdbb631bbf00ea', 'request_id': 'req-4c357d17-f300-4d60-88d3-4d1ee5bc338d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:12.664050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c357d17-f300-4d60-88d3-4d1ee5bc338d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18e8fa434e0d4e53a7bdbb631bbf00ea', 'request_id': 'req-4c357d17-f300-4d60-88d3-4d1ee5bc338d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:12.675423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c357d17-f300-4d60-88d3-4d1ee5bc338d admin admin] The token cache is being invalidate because project 18e8fa434e0d4e53a7bdbb631bbf00ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:12.675717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c357d17-f300-4d60-88d3-4d1ee5bc338d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:12.676350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4c357d17-f300-4d60-88d3-4d1ee5bc338d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:12.678012 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4129/8228] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:12 2022] DELETE /identity/v3/projects/18e8fa434e0d4e53a7bdbb631bbf00ea => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:12.683035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b95876-6aa5-4167-9460-de269b6b1e48 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:12.683326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b95876-6aa5-4167-9460-de269b6b1e48 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:12.683628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b95876-6aa5-4167-9460-de269b6b1e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:12.702674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-61b95876-6aa5-4167-9460-de269b6b1e48 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:12.703822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-61b95876-6aa5-4167-9460-de269b6b1e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:12.763442 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4100/8229] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:12.771882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c003c-0522-442e-8bf9-5194068a4923 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:12.776897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c003c-0522-442e-8bf9-5194068a4923 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:12.776897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c003c-0522-442e-8bf9-5194068a4923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:12.785081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:12.791377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-503c003c-0522-442e-8bf9-5194068a4923 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:12.792580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-503c003c-0522-442e-8bf9-5194068a4923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:12.800843 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4130/8230] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:12.804945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b95876-6aa5-4167-9460-de269b6b1e48 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:12.815323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b95876-6aa5-4167-9460-de269b6b1e48 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:12.820252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d4d96-cf24-409c-8e57-b9ae3cf92967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:12.820910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d4d96-cf24-409c-8e57-b9ae3cf92967 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:12.821199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d4d96-cf24-409c-8e57-b9ae3cf92967 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:12.821480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d4d96-cf24-409c-8e57-b9ae3cf92967 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:12.822251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d4d96-cf24-409c-8e57-b9ae3cf92967 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:39:12.822564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d4d96-cf24-409c-8e57-b9ae3cf92967 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:12.824173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d4d96-cf24-409c-8e57-b9ae3cf92967 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:12.836249 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4101/8231] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:39:12.840759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c003c-0522-442e-8bf9-5194068a4923 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:12.850908 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c003c-0522-442e-8bf9-5194068a4923 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:12.855552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0a0f4-ebb7-43b6-8944-3d069bfe92ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:12.856302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0a0f4-ebb7-43b6-8944-3d069bfe92ef admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:12.856578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0a0f4-ebb7-43b6-8944-3d069bfe92ef admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:12.856857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0a0f4-ebb7-43b6-8944-3d069bfe92ef admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:12.857548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0a0f4-ebb7-43b6-8944-3d069bfe92ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:12.859577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0a0f4-ebb7-43b6-8944-3d069bfe92ef admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:12.940785 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4131/8232] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:39:12 2022] POST /identity/v3/projects => generated 361 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:12.944616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d4d96-cf24-409c-8e57-b9ae3cf92967 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:12.958402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d4d96-cf24-409c-8e57-b9ae3cf92967 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:13.012912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34030cf-bb30-46a5-a289-809fe012ffe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:13.013588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34030cf-bb30-46a5-a289-809fe012ffe3 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:13.013901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34030cf-bb30-46a5-a289-809fe012ffe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:13.014186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34030cf-bb30-46a5-a289-809fe012ffe3 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:13.014884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34030cf-bb30-46a5-a289-809fe012ffe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:13.017457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34030cf-bb30-46a5-a289-809fe012ffe3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:13.121903 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4102/8233] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:39:12 2022] POST /identity/v3/users => generated 281 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:39:13.126775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0a0f4-ebb7-43b6-8944-3d069bfe92ef admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:13.137708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0a0f4-ebb7-43b6-8944-3d069bfe92ef admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:13.142521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bef4bd-9e53-4bbc-a033-19598a2576a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:13.149389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bef4bd-9e53-4bbc-a033-19598a2576a6 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:13.149389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bef4bd-9e53-4bbc-a033-19598a2576a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:13.149389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bef4bd-9e53-4bbc-a033-19598a2576a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:13.149389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bef4bd-9e53-4bbc-a033-19598a2576a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:13.149389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bef4bd-9e53-4bbc-a033-19598a2576a6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:13.156150 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4132/8234] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:13.160113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34030cf-bb30-46a5-a289-809fe012ffe3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:13.170507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34030cf-bb30-46a5-a289-809fe012ffe3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:13.175725 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89162032-409a-4248-bee3-dacaf8f42ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:13.176382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89162032-409a-4248-bee3-dacaf8f42ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:13.176669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89162032-409a-4248-bee3-dacaf8f42ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:13.176954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89162032-409a-4248-bee3-dacaf8f42ae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:13.177675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89162032-409a-4248-bee3-dacaf8f42ae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:13.179329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89162032-409a-4248-bee3-dacaf8f42ae2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:13.190079 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4103/8235] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:13.194346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bef4bd-9e53-4bbc-a033-19598a2576a6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:13.204903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bef4bd-9e53-4bbc-a033-19598a2576a6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:13.210212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd8a455-08c6-4d91-bc1f-5bd1502f734d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:13.211130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8a455-08c6-4d91-bc1f-5bd1502f734d admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:13.211510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8a455-08c6-4d91-bc1f-5bd1502f734d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:13.211795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd8a455-08c6-4d91-bc1f-5bd1502f734d admin admin] PATH_INFO: `/v3/projects/59e10d6dfc3e4268a5297ab1d6a50db4/users/0f5674eca2d94fd2b8177e1a49843f51/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:13.227885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd8a455-08c6-4d91-bc1f-5bd1502f734d admin admin] RBAC: Authorizing `identity:create_grant(project_id=59e10d6dfc3e4268a5297ab1d6a50db4, user_id=0f5674eca2d94fd2b8177e1a49843f51, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:13.230754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd8a455-08c6-4d91-bc1f-5bd1502f734d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:13.287678 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4133/8236] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:13 2022] PUT /identity/v3/projects/59e10d6dfc3e4268a5297ab1d6a50db4/users/0f5674eca2d94fd2b8177e1a49843f51/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:13.299315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995d7a7-cb47-4c6b-960b-a1114ae0f4b2 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:13.304507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995d7a7-cb47-4c6b-960b-a1114ae0f4b2 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:13.304507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995d7a7-cb47-4c6b-960b-a1114ae0f4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:13.330125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0995d7a7-cb47-4c6b-960b-a1114ae0f4b2 None None] MFA Rules not processed for user `0f5674eca2d94fd2b8177e1a49843f51`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:13.331575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0995d7a7-cb47-4c6b-960b-a1114ae0f4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:13.363044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:13.403742 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4104/8237] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:13.409966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fca37-e0cc-4eed-924a-7b7491db92b5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:13.410318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fca37-e0cc-4eed-924a-7b7491db92b5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:13.410610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fca37-e0cc-4eed-924a-7b7491db92b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:13.427773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8f5fca37-e0cc-4eed-924a-7b7491db92b5 None None] MFA Rules not processed for user `0f5674eca2d94fd2b8177e1a49843f51`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:13.429035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8f5fca37-e0cc-4eed-924a-7b7491db92b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:13.432234 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4134/8238] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:13 2022] POST /identity/v3/auth/tokens => generated 346 bytes in 23 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 21:39:13.437395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767aa70-d408-4761-8b60-68f35e0e8d07 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:13.437717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767aa70-d408-4761-8b60-68f35e0e8d07 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:13.438057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767aa70-d408-4761-8b60-68f35e0e8d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:13.454713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4767aa70-d408-4761-8b60-68f35e0e8d07 None None] MFA Rules not processed for user `0f5674eca2d94fd2b8177e1a49843f51`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:13.455914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4767aa70-d408-4761-8b60-68f35e0e8d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:13.459021 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4105/8239] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:13 2022] POST /identity/v3/auth/tokens => generated 346 bytes in 23 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 21:39:13.463120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fca37-e0cc-4eed-924a-7b7491db92b5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:13.475205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fca37-e0cc-4eed-924a-7b7491db92b5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:13.480243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b6001-67f6-4693-842b-6e4030ace18d tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5674eca2d94fd2b8177e1a49843f51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59e10d6dfc3e4268a5297ab1d6a50db4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:13.480243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b6001-67f6-4693-842b-6e4030ace18d tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] REQUEST_METHOD: `HEAD` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:13.480536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b6001-67f6-4693-842b-6e4030ace18d tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:13.480862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b6001-67f6-4693-842b-6e4030ace18d tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:13.487830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b6001-67f6-4693-842b-6e4030ace18d tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] RBAC: Authorizing `identity:check_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:13.489512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b6001-67f6-4693-842b-6e4030ace18d tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:13.495726 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4135/8240] 10.222.0.33 () {56 vars in 1263 bytes} [Sat May 28 21:39:13 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 358 bytes (0 switches on core 0) May 28 21:39:13.499563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767aa70-d408-4761-8b60-68f35e0e8d07 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:13.510461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767aa70-d408-4761-8b60-68f35e0e8d07 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:13.514550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16eb96e-df47-42b8-8184-e396b81fce46 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5674eca2d94fd2b8177e1a49843f51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59e10d6dfc3e4268a5297ab1d6a50db4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:13.515180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16eb96e-df47-42b8-8184-e396b81fce46 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:13.515490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16eb96e-df47-42b8-8184-e396b81fce46 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:13.515742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16eb96e-df47-42b8-8184-e396b81fce46 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:13.521723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16eb96e-df47-42b8-8184-e396b81fce46 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:13.523531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16eb96e-df47-42b8-8184-e396b81fce46 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:13.529895 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4106/8241] 10.222.0.33 () {56 vars in 1262 bytes} [Sat May 28 21:39:13 2022] GET /identity/v3/auth/tokens => generated 346 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 28 21:39:13.534344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b6001-67f6-4693-842b-6e4030ace18d tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:13.543342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b6001-67f6-4693-842b-6e4030ace18d tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:13.548049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda0705-a81a-49f6-a563-b75311ed012c tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5674eca2d94fd2b8177e1a49843f51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59e10d6dfc3e4268a5297ab1d6a50db4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:13.548682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda0705-a81a-49f6-a563-b75311ed012c tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:13.548971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda0705-a81a-49f6-a563-b75311ed012c tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:13.549257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda0705-a81a-49f6-a563-b75311ed012c tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:13.555294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda0705-a81a-49f6-a563-b75311ed012c tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:13.556798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda0705-a81a-49f6-a563-b75311ed012c tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:13.663997 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4136/8242] 10.222.0.33 () {56 vars in 1265 bytes} [Sat May 28 21:39:13 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:13.668010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16eb96e-df47-42b8-8184-e396b81fce46 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:13.677831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16eb96e-df47-42b8-8184-e396b81fce46 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:13.682528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59188bc2-d209-4e2c-9e4c-f8fb728d1d7a tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5674eca2d94fd2b8177e1a49843f51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59e10d6dfc3e4268a5297ab1d6a50db4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:13.683172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59188bc2-d209-4e2c-9e4c-f8fb728d1d7a tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] REQUEST_METHOD: `HEAD` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:13.683462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59188bc2-d209-4e2c-9e4c-f8fb728d1d7a tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:13.683745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59188bc2-d209-4e2c-9e4c-f8fb728d1d7a tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:13.685528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-59188bc2-d209-4e2c-9e4c-f8fb728d1d7a tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:13.698442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-59188bc2-d209-4e2c-9e4c-f8fb728d1d7a tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token May 28 21:39:13.698442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:39:13.698442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:39:13.698442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:39:13.698442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:39:13.698442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:39:13.698442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:39:13.698442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:39:13.698442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:39:13.698442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:39:13.698442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:39:13.698442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:39:13.698442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 283, in get May 28 21:39:13.698442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:check_token') May 28 21:39:13.698442 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call May 28 21:39:13.699727 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() May 28 21:39:13.699727 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data May 28 21:39:13.699727 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( May 28 21:39:13.699727 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:39:13.699727 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:39:13.699727 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token May 28 21:39:13.699727 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) May 28 21:39:13.699727 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token May 28 21:39:13.699727 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.check_revocation(token) May 28 21:39:13.699727 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:39:13.699727 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:39:13.699727 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation May 28 21:39:13.699727 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) May 28 21:39:13.699727 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:39:13.699727 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:39:13.699727 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 May 28 21:39:13.699727 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) May 28 21:39:13.700880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:39:13.700880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:39:13.700880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token May 28 21:39:13.700880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) May 28 21:39:13.700880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token May 28 21:39:13.700880 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:39:13.700880 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4107/8243] 10.222.0.33 () {56 vars in 1263 bytes} [Sat May 28 21:39:13 2022] HEAD /identity/v3/auth/tokens => generated 0 bytes in 32 msecs (HTTP/1.1 404) 5 headers in 183 bytes (0 switches on core 0) May 28 21:39:13.707148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46151644-b19d-4957-905a-a21e37cdd701 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:13.707514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46151644-b19d-4957-905a-a21e37cdd701 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:13.707886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46151644-b19d-4957-905a-a21e37cdd701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:13.725799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-46151644-b19d-4957-905a-a21e37cdd701 None None] MFA Rules not processed for user `0f5674eca2d94fd2b8177e1a49843f51`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:13.729398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-46151644-b19d-4957-905a-a21e37cdd701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:13.730201 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4137/8244] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:13 2022] POST /identity/v3/auth/tokens => generated 346 bytes in 24 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 28 21:39:13.734307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59188bc2-d209-4e2c-9e4c-f8fb728d1d7a tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:13.744237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59188bc2-d209-4e2c-9e4c-f8fb728d1d7a tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:13.748647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6788e2a-7c81-4a6f-97ba-1a706c1a6b28 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5674eca2d94fd2b8177e1a49843f51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59e10d6dfc3e4268a5297ab1d6a50db4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:13.752353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6788e2a-7c81-4a6f-97ba-1a706c1a6b28 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:13.752353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6788e2a-7c81-4a6f-97ba-1a706c1a6b28 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:13.752353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6788e2a-7c81-4a6f-97ba-1a706c1a6b28 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:13.756210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6788e2a-7c81-4a6f-97ba-1a706c1a6b28 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:13.758054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6788e2a-7c81-4a6f-97ba-1a706c1a6b28 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:13.764076 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4108/8245] 10.222.0.33 () {56 vars in 1262 bytes} [Sat May 28 21:39:13 2022] GET /identity/v3/auth/tokens => generated 346 bytes in 30 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 28 21:39:13.768539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46151644-b19d-4957-905a-a21e37cdd701 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:13.779987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46151644-b19d-4957-905a-a21e37cdd701 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:13.784810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd2efc-a712-4bbb-a014-d7f9ba3f89e7 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5674eca2d94fd2b8177e1a49843f51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59e10d6dfc3e4268a5297ab1d6a50db4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:13.785461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd2efc-a712-4bbb-a014-d7f9ba3f89e7 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:13.785751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd2efc-a712-4bbb-a014-d7f9ba3f89e7 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:13.786069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd2efc-a712-4bbb-a014-d7f9ba3f89e7 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:13.792295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd2efc-a712-4bbb-a014-d7f9ba3f89e7 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] RBAC: Authorizing `identity:revoke_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:13.793829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd2efc-a712-4bbb-a014-d7f9ba3f89e7 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:13.911045 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4138/8246] 10.222.0.33 () {56 vars in 1265 bytes} [Sat May 28 21:39:13 2022] DELETE /identity/v3/auth/tokens => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:39:13.914935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6788e2a-7c81-4a6f-97ba-1a706c1a6b28 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:13.924213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6788e2a-7c81-4a6f-97ba-1a706c1a6b28 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:13.928408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d88dfc-6cf0-4661-b1a5-acbad680d265 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f5674eca2d94fd2b8177e1a49843f51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '59e10d6dfc3e4268a5297ab1d6a50db4', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:13.929037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d88dfc-6cf0-4661-b1a5-acbad680d265 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:13.929326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d88dfc-6cf0-4661-b1a5-acbad680d265 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:13.929607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d88dfc-6cf0-4661-b1a5-acbad680d265 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:13.931438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-10d88dfc-6cf0-4661-b1a5-acbad680d265 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:13.943019 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-10d88dfc-6cf0-4661-b1a5-acbad680d265 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] Failed to validate token: keystone.exception.TokenNotFound: Failed to validate token May 28 21:39:13.943019 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:39:13.943019 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:39:13.943019 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:39:13.943019 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:39:13.943019 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:39:13.943019 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:39:13.943019 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:39:13.943019 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:39:13.943019 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:39:13.943019 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:39:13.943019 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:39:13.943019 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get May 28 21:39:13.943019 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') May 28 21:39:13.943019 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call May 28 21:39:13.944270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() May 28 21:39:13.944270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data May 28 21:39:13.944270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( May 28 21:39:13.944270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:39:13.944270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:39:13.944270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 146, in validate_token May 28 21:39:13.944270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self._is_valid_token(token, window_seconds=window_seconds) May 28 21:39:13.944270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 199, in _is_valid_token May 28 21:39:13.944270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application self.check_revocation(token) May 28 21:39:13.944270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:39:13.944270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:39:13.944270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 137, in check_revocation May 28 21:39:13.944270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.check_revocation_v3(token) May 28 21:39:13.944270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:39:13.944270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:39:13.944270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 134, in check_revocation_v3 May 28 21:39:13.944270 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application PROVIDERS.revoke_api.check_token(token_values) May 28 21:39:13.946003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:39:13.946003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:39:13.946003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/revoke/core.py", line 137, in check_token May 28 21:39:13.946003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.TokenNotFound(_('Failed to validate token')) May 28 21:39:13.946003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.TokenNotFound: Failed to validate token May 28 21:39:13.946003 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:39:13.946003 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4109/8247] 10.222.0.33 () {56 vars in 1262 bytes} [Sat May 28 21:39:13 2022] GET /identity/v3/auth/tokens => generated 80 bytes in 30 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 21:39:13.951190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd2efc-a712-4bbb-a014-d7f9ba3f89e7 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:14.014062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd2efc-a712-4bbb-a014-d7f9ba3f89e7 tempest-TokensV3Test-1737512926 tempest-TokensV3Test-1737512926-project] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:14.019481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8e836-9660-487b-9914-117a3e8e57e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:14.020378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8e836-9660-487b-9914-117a3e8e57e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:14.020669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8e836-9660-487b-9914-117a3e8e57e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:14.020949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a8e836-9660-487b-9914-117a3e8e57e1 admin admin] PATH_INFO: `/v3/users/0f5674eca2d94fd2b8177e1a49843f51` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:14.022590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a8e836-9660-487b-9914-117a3e8e57e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f5674eca2d94fd2b8177e1a49843f51)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:14.024703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a8e836-9660-487b-9914-117a3e8e57e1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:14.242198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:14.248101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c9a8e836-9660-487b-9914-117a3e8e57e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f5674eca2d94fd2b8177e1a49843f51'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:14.255397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c9a8e836-9660-487b-9914-117a3e8e57e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f5674eca2d94fd2b8177e1a49843f51'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:14.256062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c9a8e836-9660-487b-9914-117a3e8e57e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f5674eca2d94fd2b8177e1a49843f51'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:14.260700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c9a8e836-9660-487b-9914-117a3e8e57e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f5674eca2d94fd2b8177e1a49843f51'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:14.332330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-c9a8e836-9660-487b-9914-117a3e8e57e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f5674eca2d94fd2b8177e1a49843f51'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:14.351437 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4139/8248] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:39:13 2022] DELETE /identity/v3/users/0f5674eca2d94fd2b8177e1a49843f51 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:14.355955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944426c-0783-40b2-8a64-338dbb899176 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:14.356279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944426c-0783-40b2-8a64-338dbb899176 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:14.356561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944426c-0783-40b2-8a64-338dbb899176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:14.374780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c944426c-0783-40b2-8a64-338dbb899176 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:14.376526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c944426c-0783-40b2-8a64-338dbb899176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:14.461122 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4110/8249] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:14.488294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8e836-9660-487b-9914-117a3e8e57e1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:14.489771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c9a8e836-9660-487b-9914-117a3e8e57e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:14.553988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a8e836-9660-487b-9914-117a3e8e57e1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:14.557679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e84a6-9107-46aa-96ef-eee6a4a5dabc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:14.558485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e84a6-9107-46aa-96ef-eee6a4a5dabc service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:14.558671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e84a6-9107-46aa-96ef-eee6a4a5dabc service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:14.558969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e84a6-9107-46aa-96ef-eee6a4a5dabc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:14.569487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e84a6-9107-46aa-96ef-eee6a4a5dabc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:14.571091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e84a6-9107-46aa-96ef-eee6a4a5dabc service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:14.581369 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4140/8250] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:15.100093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944426c-0783-40b2-8a64-338dbb899176 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:15.100534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c944426c-0783-40b2-8a64-338dbb899176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:15.129005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944426c-0783-40b2-8a64-338dbb899176 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:15.133534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d9684-7270-4b18-b505-38e15e915f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:15.134442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d9684-7270-4b18-b505-38e15e915f48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:15.134726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d9684-7270-4b18-b505-38e15e915f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:15.135030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d9684-7270-4b18-b505-38e15e915f48 admin admin] PATH_INFO: `/v3/projects/59e10d6dfc3e4268a5297ab1d6a50db4` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:15.136384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d9684-7270-4b18-b505-38e15e915f48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59e10d6dfc3e4268a5297ab1d6a50db4)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:15.139304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133d9684-7270-4b18-b505-38e15e915f48 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:15.246910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-133d9684-7270-4b18-b505-38e15e915f48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59e10d6dfc3e4268a5297ab1d6a50db4', 'request_id': 'req-133d9684-7270-4b18-b505-38e15e915f48'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:15.253369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-133d9684-7270-4b18-b505-38e15e915f48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59e10d6dfc3e4268a5297ab1d6a50db4', 'request_id': 'req-133d9684-7270-4b18-b505-38e15e915f48'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:15.332604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-133d9684-7270-4b18-b505-38e15e915f48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59e10d6dfc3e4268a5297ab1d6a50db4', 'request_id': 'req-133d9684-7270-4b18-b505-38e15e915f48'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:15.345088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-133d9684-7270-4b18-b505-38e15e915f48 admin admin] The token cache is being invalidate because project 59e10d6dfc3e4268a5297ab1d6a50db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:15.345382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-133d9684-7270-4b18-b505-38e15e915f48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:15.346045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-133d9684-7270-4b18-b505-38e15e915f48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:15.347655 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4111/8251] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:15 2022] DELETE /identity/v3/projects/59e10d6dfc3e4268a5297ab1d6a50db4 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:15.355180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95758462-d070-4014-b5ae-1cec355c6467 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:15.355663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95758462-d070-4014-b5ae-1cec355c6467 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:15.355763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95758462-d070-4014-b5ae-1cec355c6467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:15.373355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-95758462-d070-4014-b5ae-1cec355c6467 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:15.374607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-95758462-d070-4014-b5ae-1cec355c6467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:15.434237 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4141/8252] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:15.442367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2dfd6c-2d13-4b3a-8476-226bfc699805 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:15.442667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2dfd6c-2d13-4b3a-8476-226bfc699805 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:15.448293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2dfd6c-2d13-4b3a-8476-226bfc699805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:15.449337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:15.462689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7e2dfd6c-2d13-4b3a-8476-226bfc699805 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:15.465492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7e2dfd6c-2d13-4b3a-8476-226bfc699805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:15.472703 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4112/8253] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:15.476849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95758462-d070-4014-b5ae-1cec355c6467 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:15.487168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95758462-d070-4014-b5ae-1cec355c6467 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:15.491888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b74093-9864-418f-b177-590fedc050e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:15.492556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b74093-9864-418f-b177-590fedc050e0 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:15.492844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b74093-9864-418f-b177-590fedc050e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:15.493127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b74093-9864-418f-b177-590fedc050e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:15.493986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b74093-9864-418f-b177-590fedc050e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:39:15.494301 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b74093-9864-418f-b177-590fedc050e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:15.495965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b74093-9864-418f-b177-590fedc050e0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:15.507868 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4142/8254] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:39:15.512264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2dfd6c-2d13-4b3a-8476-226bfc699805 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:15.521919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2dfd6c-2d13-4b3a-8476-226bfc699805 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:15.527170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fd25e-d622-446e-8bec-595adf54c757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:15.528616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fd25e-d622-446e-8bec-595adf54c757 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:15.528616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fd25e-d622-446e-8bec-595adf54c757 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:15.528616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fd25e-d622-446e-8bec-595adf54c757 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:15.529022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684fd25e-d622-446e-8bec-595adf54c757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:15.530950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684fd25e-d622-446e-8bec-595adf54c757 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:15.634247 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4113/8255] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:39:15 2022] POST /identity/v3/projects => generated 399 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:15.635375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b74093-9864-418f-b177-590fedc050e0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:15.647147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b74093-9864-418f-b177-590fedc050e0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:15.693858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:15.707605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1bb42f-ce04-40e1-b61f-cd65de8ac5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:15.709350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bb42f-ce04-40e1-b61f-cd65de8ac5ec admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:15.709350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bb42f-ce04-40e1-b61f-cd65de8ac5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:15.709577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1bb42f-ce04-40e1-b61f-cd65de8ac5ec admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:15.710302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1bb42f-ce04-40e1-b61f-cd65de8ac5ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:15.712238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1bb42f-ce04-40e1-b61f-cd65de8ac5ec admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:15.811581 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4143/8256] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:39:15 2022] POST /identity/v3/users => generated 306 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:15.816234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fd25e-d622-446e-8bec-595adf54c757 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:15.826598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fd25e-d622-446e-8bec-595adf54c757 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:15.832120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852796b-4dea-47a8-9f44-72e94d001792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:15.833013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852796b-4dea-47a8-9f44-72e94d001792 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:15.833154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852796b-4dea-47a8-9f44-72e94d001792 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:15.833322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852796b-4dea-47a8-9f44-72e94d001792 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:15.834112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852796b-4dea-47a8-9f44-72e94d001792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:15.835775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852796b-4dea-47a8-9f44-72e94d001792 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:15.846082 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4114/8257] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:15.850264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1bb42f-ce04-40e1-b61f-cd65de8ac5ec admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:15.860258 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1bb42f-ce04-40e1-b61f-cd65de8ac5ec admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:15.864873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f91a08-e6d2-41f5-b740-de1afcec74b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:15.865751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f91a08-e6d2-41f5-b740-de1afcec74b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:15.866074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f91a08-e6d2-41f5-b740-de1afcec74b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:15.866357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f91a08-e6d2-41f5-b740-de1afcec74b1 admin admin] PATH_INFO: `/v3/projects/fbd320882d39473f8457147fd23865e7/users/f7f0afa87e0f4c719dc91854e3c25875/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:15.881647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f91a08-e6d2-41f5-b740-de1afcec74b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbd320882d39473f8457147fd23865e7, user_id=f7f0afa87e0f4c719dc91854e3c25875, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:15.884351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f91a08-e6d2-41f5-b740-de1afcec74b1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:15.989548 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4144/8258] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:15 2022] PUT /identity/v3/projects/fbd320882d39473f8457147fd23865e7/users/f7f0afa87e0f4c719dc91854e3c25875/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:15.993653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852796b-4dea-47a8-9f44-72e94d001792 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:16.004404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852796b-4dea-47a8-9f44-72e94d001792 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:16.062894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe2920-4080-4dbc-ac42-1a177183511f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:16.064202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe2920-4080-4dbc-ac42-1a177183511f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:16.064517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe2920-4080-4dbc-ac42-1a177183511f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:16.064780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe2920-4080-4dbc-ac42-1a177183511f admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:16.065518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe2920-4080-4dbc-ac42-1a177183511f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:16.067248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe2920-4080-4dbc-ac42-1a177183511f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:16.081043 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4115/8259] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:15 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:16.085190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f91a08-e6d2-41f5-b740-de1afcec74b1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:16.096163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f91a08-e6d2-41f5-b740-de1afcec74b1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:16.102405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736dda2-3362-4ce9-b381-8fb0a44d444c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:16.103304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736dda2-3362-4ce9-b381-8fb0a44d444c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:16.103587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736dda2-3362-4ce9-b381-8fb0a44d444c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:16.103864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1736dda2-3362-4ce9-b381-8fb0a44d444c admin admin] PATH_INFO: `/v3/projects/fbd320882d39473f8457147fd23865e7/users/f7f0afa87e0f4c719dc91854e3c25875/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:16.105902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736dda2-3362-4ce9-b381-8fb0a44d444c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbd320882d39473f8457147fd23865e7, user_id=f7f0afa87e0f4c719dc91854e3c25875, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:16.108666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1736dda2-3362-4ce9-b381-8fb0a44d444c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:16.119532 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4145/8260] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:16 2022] PUT /identity/v3/projects/fbd320882d39473f8457147fd23865e7/users/f7f0afa87e0f4c719dc91854e3c25875/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:16.130220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b27907-f0bd-43a9-9de5-ba5e45da878b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:16.130716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b27907-f0bd-43a9-9de5-ba5e45da878b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:16.130816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b27907-f0bd-43a9-9de5-ba5e45da878b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:16.158064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-23b27907-f0bd-43a9-9de5-ba5e45da878b None None] MFA Rules not processed for user `f7f0afa87e0f4c719dc91854e3c25875`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:16.159421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-23b27907-f0bd-43a9-9de5-ba5e45da878b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:16.231167 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4116/8261] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:16 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:39:16.236121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736dda2-3362-4ce9-b381-8fb0a44d444c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:16.302986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1736dda2-3362-4ce9-b381-8fb0a44d444c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:16.307848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7691e-32ba-4089-9120-d295f7c059f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:16.308617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7691e-32ba-4089-9120-d295f7c059f6 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:16.308936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7691e-32ba-4089-9120-d295f7c059f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:16.309215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7691e-32ba-4089-9120-d295f7c059f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:16.309951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7691e-32ba-4089-9120-d295f7c059f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:16.311928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7691e-32ba-4089-9120-d295f7c059f6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:16.405885 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4146/8262] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:39:16 2022] POST /identity/v3/projects => generated 397 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:16.408116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b27907-f0bd-43a9-9de5-ba5e45da878b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:16.419177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b27907-f0bd-43a9-9de5-ba5e45da878b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:16.464763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:16.480985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2b19c-765a-4ee9-ac9e-1a499514bd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:16.481664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2b19c-765a-4ee9-ac9e-1a499514bd38 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:16.481981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2b19c-765a-4ee9-ac9e-1a499514bd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:16.482262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2b19c-765a-4ee9-ac9e-1a499514bd38 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:16.482982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2b19c-765a-4ee9-ac9e-1a499514bd38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:16.484959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2b19c-765a-4ee9-ac9e-1a499514bd38 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:16.616451 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4117/8263] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:39:16 2022] POST /identity/v3/users => generated 299 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:39:16.617711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7691e-32ba-4089-9120-d295f7c059f6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:16.628423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7691e-32ba-4089-9120-d295f7c059f6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:16.634324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b284a-d438-4e9e-958e-7350d10ac314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:16.634324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b284a-d438-4e9e-958e-7350d10ac314 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:16.634324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b284a-d438-4e9e-958e-7350d10ac314 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:16.634873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b284a-d438-4e9e-958e-7350d10ac314 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:16.635622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b284a-d438-4e9e-958e-7350d10ac314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:16.637244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746b284a-d438-4e9e-958e-7350d10ac314 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:16.650459 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4147/8264] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:16.652103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2b19c-765a-4ee9-ac9e-1a499514bd38 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:16.664202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2b19c-765a-4ee9-ac9e-1a499514bd38 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:16.668965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca76ee8-2719-457f-b0be-5878e42aeef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:16.669628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca76ee8-2719-457f-b0be-5878e42aeef6 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:16.670019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca76ee8-2719-457f-b0be-5878e42aeef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:16.670303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca76ee8-2719-457f-b0be-5878e42aeef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:16.671036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca76ee8-2719-457f-b0be-5878e42aeef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:16.672642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca76ee8-2719-457f-b0be-5878e42aeef6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:16.683302 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4118/8265] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:16.687405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b284a-d438-4e9e-958e-7350d10ac314 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:16.697371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746b284a-d438-4e9e-958e-7350d10ac314 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:16.701986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7760ac9-efa1-4484-b315-ffd608923a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:16.702859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7760ac9-efa1-4484-b315-ffd608923a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:16.703144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7760ac9-efa1-4484-b315-ffd608923a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:16.703423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7760ac9-efa1-4484-b315-ffd608923a0c admin admin] PATH_INFO: `/v3/projects/a1c5cc926f184969bffb1797aa0d6192/users/9e75f39e0db94241abc3c0e21c71868b/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:16.718932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7760ac9-efa1-4484-b315-ffd608923a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1c5cc926f184969bffb1797aa0d6192, user_id=9e75f39e0db94241abc3c0e21c71868b, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:16.722308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7760ac9-efa1-4484-b315-ffd608923a0c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:16.824018 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4148/8266] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:16 2022] PUT /identity/v3/projects/a1c5cc926f184969bffb1797aa0d6192/users/9e75f39e0db94241abc3c0e21c71868b/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:39:16.914851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e26230-4610-4a40-b96b-4cfdaede2334 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:16.915384 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e26230-4610-4a40-b96b-4cfdaede2334 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:16.915786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e26230-4610-4a40-b96b-4cfdaede2334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:16.962578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-42e26230-4610-4a40-b96b-4cfdaede2334 None None] MFA Rules not processed for user `9e75f39e0db94241abc3c0e21c71868b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:16.962578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-42e26230-4610-4a40-b96b-4cfdaede2334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:17.032345 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4119/8267] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:16 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:17.044374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7760ac9-efa1-4484-b315-ffd608923a0c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:17.046218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b7760ac9-efa1-4484-b315-ffd608923a0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:17.112290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:17.118878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7760ac9-efa1-4484-b315-ffd608923a0c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:17.122743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202df4ed-bb58-47f5-90f3-af1211a71869 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:17.123447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202df4ed-bb58-47f5-90f3-af1211a71869 service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:17.128825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202df4ed-bb58-47f5-90f3-af1211a71869 service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:17.128825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202df4ed-bb58-47f5-90f3-af1211a71869 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:17.193386 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202df4ed-bb58-47f5-90f3-af1211a71869 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:17.194853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202df4ed-bb58-47f5-90f3-af1211a71869 service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:17.204809 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4149/8268] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:17 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:17.848867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e26230-4610-4a40-b96b-4cfdaede2334 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:17.911409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e26230-4610-4a40-b96b-4cfdaede2334 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:17.916174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a1aca-ec52-45e5-a0e6-0d911d9e99ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:17.916991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a1aca-ec52-45e5-a0e6-0d911d9e99ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:17.917279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a1aca-ec52-45e5-a0e6-0d911d9e99ee admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:17.917561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640a1aca-ec52-45e5-a0e6-0d911d9e99ee admin admin] PATH_INFO: `/v3/users/f7f0afa87e0f4c719dc91854e3c25875` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:17.918794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a1aca-ec52-45e5-a0e6-0d911d9e99ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7f0afa87e0f4c719dc91854e3c25875)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:17.920742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640a1aca-ec52-45e5-a0e6-0d911d9e99ee admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:18.119549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-640a1aca-ec52-45e5-a0e6-0d911d9e99ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7f0afa87e0f4c719dc91854e3c25875'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:18.120250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-640a1aca-ec52-45e5-a0e6-0d911d9e99ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7f0afa87e0f4c719dc91854e3c25875'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:18.125915 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:18.127961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-640a1aca-ec52-45e5-a0e6-0d911d9e99ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7f0afa87e0f4c719dc91854e3c25875'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:18.256317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-640a1aca-ec52-45e5-a0e6-0d911d9e99ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7f0afa87e0f4c719dc91854e3c25875'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:18.256947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-640a1aca-ec52-45e5-a0e6-0d911d9e99ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7f0afa87e0f4c719dc91854e3c25875'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:18.275796 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4120/8269] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:39:17 2022] DELETE /identity/v3/users/f7f0afa87e0f4c719dc91854e3c25875 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:18.280798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ab530-0ffd-42ad-890c-8984888d39de None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:18.282196 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ab530-0ffd-42ad-890c-8984888d39de None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:18.283493 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ab530-0ffd-42ad-890c-8984888d39de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:18.301321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a35ab530-0ffd-42ad-890c-8984888d39de None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:18.302570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a35ab530-0ffd-42ad-890c-8984888d39de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:18.374874 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4150/8270] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:18.384775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a1aca-ec52-45e5-a0e6-0d911d9e99ee admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:18.386247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-640a1aca-ec52-45e5-a0e6-0d911d9e99ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:18.449178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640a1aca-ec52-45e5-a0e6-0d911d9e99ee admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:18.453788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5790457-5c85-4967-8862-7d11a3a3ccaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:18.455162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5790457-5c85-4967-8862-7d11a3a3ccaa service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:18.455437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5790457-5c85-4967-8862-7d11a3a3ccaa service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:18.455720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5790457-5c85-4967-8862-7d11a3a3ccaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:18.465753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5790457-5c85-4967-8862-7d11a3a3ccaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:18.467194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5790457-5c85-4967-8862-7d11a3a3ccaa service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:18.477981 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4121/8271] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:18.511960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ab530-0ffd-42ad-890c-8984888d39de None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:18.513424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a35ab530-0ffd-42ad-890c-8984888d39de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:18.596712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ab530-0ffd-42ad-890c-8984888d39de None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:18.602670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b804e-b7fe-4728-8bfe-beaa033d41f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:18.603335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b804e-b7fe-4728-8bfe-beaa033d41f5 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:18.603782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b804e-b7fe-4728-8bfe-beaa033d41f5 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:18.603997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b804e-b7fe-4728-8bfe-beaa033d41f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:18.605817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-be4b804e-b7fe-4728-8bfe-beaa033d41f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:18.622668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b804e-b7fe-4728-8bfe-beaa033d41f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:18.626183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4b804e-b7fe-4728-8bfe-beaa033d41f5 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:18.638721 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4151/8272] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:18.738403 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5790457-5c85-4967-8862-7d11a3a3ccaa service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:18.740012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a5790457-5c85-4967-8862-7d11a3a3ccaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:18.755966 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5790457-5c85-4967-8862-7d11a3a3ccaa service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:18.767717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f6690-dfc4-4ee7-b15d-99c0b9ea5049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:18.767717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f6690-dfc4-4ee7-b15d-99c0b9ea5049 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:18.767717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f6690-dfc4-4ee7-b15d-99c0b9ea5049 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:18.767717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4f6690-dfc4-4ee7-b15d-99c0b9ea5049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:18.768199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7f4f6690-dfc4-4ee7-b15d-99c0b9ea5049 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:18.778438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f6690-dfc4-4ee7-b15d-99c0b9ea5049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:18.779986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4f6690-dfc4-4ee7-b15d-99c0b9ea5049 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:18.793441 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4122/8273] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:18.957548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b804e-b7fe-4728-8bfe-beaa033d41f5 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:18.959040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-be4b804e-b7fe-4728-8bfe-beaa033d41f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:18.983031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4b804e-b7fe-4728-8bfe-beaa033d41f5 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:18.984283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:18.985318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:18.986125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:18.986411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e admin admin] PATH_INFO: `/v3/projects/fbd320882d39473f8457147fd23865e7` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:18.987612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbd320882d39473f8457147fd23865e7)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:18.989644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:19.012787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:19.122280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbd320882d39473f8457147fd23865e7', 'request_id': 'req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:19.128502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbd320882d39473f8457147fd23865e7', 'request_id': 'req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:19.222799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbd320882d39473f8457147fd23865e7', 'request_id': 'req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:19.236337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e admin admin] The token cache is being invalidate because project fbd320882d39473f8457147fd23865e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:19.236633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:19.237235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:19.238873 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4152/8274] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:18 2022] DELETE /identity/v3/projects/fbd320882d39473f8457147fd23865e7 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:19.242896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f6690-dfc4-4ee7-b15d-99c0b9ea5049 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:19.256250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4f6690-dfc4-4ee7-b15d-99c0b9ea5049 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:19.327276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffb806-aed5-4942-b699-2be445821257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:19.328155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffb806-aed5-4942-b699-2be445821257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:19.328433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffb806-aed5-4942-b699-2be445821257 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:19.328703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffb806-aed5-4942-b699-2be445821257 admin admin] PATH_INFO: `/v3/users/9e75f39e0db94241abc3c0e21c71868b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:19.330490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffb806-aed5-4942-b699-2be445821257 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e75f39e0db94241abc3c0e21c71868b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:19.332418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffb806-aed5-4942-b699-2be445821257 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:19.561710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-49ffb806-aed5-4942-b699-2be445821257 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e75f39e0db94241abc3c0e21c71868b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:19.562395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-49ffb806-aed5-4942-b699-2be445821257 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e75f39e0db94241abc3c0e21c71868b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:19.570287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-49ffb806-aed5-4942-b699-2be445821257 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e75f39e0db94241abc3c0e21c71868b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:19.651192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-49ffb806-aed5-4942-b699-2be445821257 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e75f39e0db94241abc3c0e21c71868b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:19.652037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-49ffb806-aed5-4942-b699-2be445821257 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e75f39e0db94241abc3c0e21c71868b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:19.653440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:19.670317 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4123/8275] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:39:19 2022] DELETE /identity/v3/users/9e75f39e0db94241abc3c0e21c71868b => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:20.099076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:20.100770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:20.114713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177ffa3-6b1d-4f7d-995e-a6a6453b234e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:20.173960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555219fe-d387-4b58-8e71-bc5d6b81f378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:20.174828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555219fe-d387-4b58-8e71-bc5d6b81f378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:20.175110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555219fe-d387-4b58-8e71-bc5d6b81f378 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:20.175382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555219fe-d387-4b58-8e71-bc5d6b81f378 admin admin] PATH_INFO: `/v3/projects/a1c5cc926f184969bffb1797aa0d6192` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:20.176578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555219fe-d387-4b58-8e71-bc5d6b81f378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1c5cc926f184969bffb1797aa0d6192)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:20.179029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555219fe-d387-4b58-8e71-bc5d6b81f378 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:20.294726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-555219fe-d387-4b58-8e71-bc5d6b81f378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1c5cc926f184969bffb1797aa0d6192', 'request_id': 'req-555219fe-d387-4b58-8e71-bc5d6b81f378'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:20.301456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-555219fe-d387-4b58-8e71-bc5d6b81f378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1c5cc926f184969bffb1797aa0d6192', 'request_id': 'req-555219fe-d387-4b58-8e71-bc5d6b81f378'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:20.371138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-555219fe-d387-4b58-8e71-bc5d6b81f378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1c5cc926f184969bffb1797aa0d6192', 'request_id': 'req-555219fe-d387-4b58-8e71-bc5d6b81f378'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:20.383391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-555219fe-d387-4b58-8e71-bc5d6b81f378 admin admin] The token cache is being invalidate because project a1c5cc926f184969bffb1797aa0d6192 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:20.383854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-555219fe-d387-4b58-8e71-bc5d6b81f378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:20.384527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-555219fe-d387-4b58-8e71-bc5d6b81f378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:20.386248 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4153/8276] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:20 2022] DELETE /identity/v3/projects/a1c5cc926f184969bffb1797aa0d6192 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:20.391837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102960bc-524c-4406-86df-e2e6940bc68a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:20.392341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102960bc-524c-4406-86df-e2e6940bc68a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:20.392565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102960bc-524c-4406-86df-e2e6940bc68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:20.411013 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-102960bc-524c-4406-86df-e2e6940bc68a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:20.412346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-102960bc-524c-4406-86df-e2e6940bc68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:20.487464 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4124/8277] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:39:20.495471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f9551-b2ee-4b67-8af8-e102bcb94213 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:20.495760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f9551-b2ee-4b67-8af8-e102bcb94213 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:20.496043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f9551-b2ee-4b67-8af8-e102bcb94213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:20.524326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-da5f9551-b2ee-4b67-8af8-e102bcb94213 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:20.525551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-da5f9551-b2ee-4b67-8af8-e102bcb94213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:20.534349 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4154/8278] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:20.538595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102960bc-524c-4406-86df-e2e6940bc68a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:20.548540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102960bc-524c-4406-86df-e2e6940bc68a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:20.553323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440929b3-63a5-41b4-9cb2-b36b05c312c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:20.554097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440929b3-63a5-41b4-9cb2-b36b05c312c2 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:20.554386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440929b3-63a5-41b4-9cb2-b36b05c312c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:20.554665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440929b3-63a5-41b4-9cb2-b36b05c312c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:20.555400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440929b3-63a5-41b4-9cb2-b36b05c312c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:39:20.555764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440929b3-63a5-41b4-9cb2-b36b05c312c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:20.557383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440929b3-63a5-41b4-9cb2-b36b05c312c2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:20.569293 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4125/8279] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:39:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:39:20.573586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f9551-b2ee-4b67-8af8-e102bcb94213 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:20.583518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f9551-b2ee-4b67-8af8-e102bcb94213 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:20.588223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a373f4-1dc5-48ab-947e-4cc3e4506c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:20.588887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a373f4-1dc5-48ab-947e-4cc3e4506c46 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:20.589175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a373f4-1dc5-48ab-947e-4cc3e4506c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:20.589711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a373f4-1dc5-48ab-947e-4cc3e4506c46 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:20.590462 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a373f4-1dc5-48ab-947e-4cc3e4506c46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:20.592378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a373f4-1dc5-48ab-947e-4cc3e4506c46 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:20.666787 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4155/8280] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:39:20 2022] POST /identity/v3/projects => generated 369 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:20.670950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440929b3-63a5-41b4-9cb2-b36b05c312c2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:20.681179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440929b3-63a5-41b4-9cb2-b36b05c312c2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:20.740189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38699b18-24d2-4dc2-bc16-eb14edfd39cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:20.740914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38699b18-24d2-4dc2-bc16-eb14edfd39cc admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:20.741202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38699b18-24d2-4dc2-bc16-eb14edfd39cc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:20.741484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38699b18-24d2-4dc2-bc16-eb14edfd39cc admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:20.742231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38699b18-24d2-4dc2-bc16-eb14edfd39cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:20.744191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38699b18-24d2-4dc2-bc16-eb14edfd39cc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:20.813347 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4126/8281] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:39:20 2022] POST /identity/v3/users => generated 285 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:20.817262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a373f4-1dc5-48ab-947e-4cc3e4506c46 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:20.827820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a373f4-1dc5-48ab-947e-4cc3e4506c46 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:20.833123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01fe76-ea43-4c2e-ac31-e188e35dd901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:20.833791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01fe76-ea43-4c2e-ac31-e188e35dd901 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:20.834112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01fe76-ea43-4c2e-ac31-e188e35dd901 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:20.834397 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d01fe76-ea43-4c2e-ac31-e188e35dd901 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:20.835207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01fe76-ea43-4c2e-ac31-e188e35dd901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:20.836822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d01fe76-ea43-4c2e-ac31-e188e35dd901 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:20.838974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:20.848750 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4156/8282] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:20.852877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38699b18-24d2-4dc2-bc16-eb14edfd39cc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:20.863619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38699b18-24d2-4dc2-bc16-eb14edfd39cc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:20.868424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a334060-ef89-4504-ab23-531baeebb239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:20.869125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334060-ef89-4504-ab23-531baeebb239 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:20.869409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334060-ef89-4504-ab23-531baeebb239 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:20.869688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a334060-ef89-4504-ab23-531baeebb239 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:20.870583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a334060-ef89-4504-ab23-531baeebb239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:20.872177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a334060-ef89-4504-ab23-531baeebb239 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:20.883061 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4127/8283] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:20.887271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01fe76-ea43-4c2e-ac31-e188e35dd901 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:20.897776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d01fe76-ea43-4c2e-ac31-e188e35dd901 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:20.902506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad4a3d-f9d1-475b-9bf0-93a44713e5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:20.903761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad4a3d-f9d1-475b-9bf0-93a44713e5c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:20.904265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad4a3d-f9d1-475b-9bf0-93a44713e5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:20.904958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad4a3d-f9d1-475b-9bf0-93a44713e5c8 admin admin] PATH_INFO: `/v3/projects/773aefd97d374cdc9497c9d39cbdd9bc/users/0d453d6d31f04dc7a69fb5805728588e/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:20.920095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad4a3d-f9d1-475b-9bf0-93a44713e5c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=773aefd97d374cdc9497c9d39cbdd9bc, user_id=0d453d6d31f04dc7a69fb5805728588e, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:20.923633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad4a3d-f9d1-475b-9bf0-93a44713e5c8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:20.977676 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4157/8284] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:20 2022] PUT /identity/v3/projects/773aefd97d374cdc9497c9d39cbdd9bc/users/0d453d6d31f04dc7a69fb5805728588e/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:20.989114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384da77-0ec6-4efd-8542-d6f695dae415 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:20.989342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384da77-0ec6-4efd-8542-d6f695dae415 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:20.989624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384da77-0ec6-4efd-8542-d6f695dae415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:21.017746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d384da77-0ec6-4efd-8542-d6f695dae415 None None] MFA Rules not processed for user `0d453d6d31f04dc7a69fb5805728588e`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:21.019015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d384da77-0ec6-4efd-8542-d6f695dae415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:21.051468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:21.084056 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4128/8285] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:20 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:21.088532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad4a3d-f9d1-475b-9bf0-93a44713e5c8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:21.150033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad4a3d-f9d1-475b-9bf0-93a44713e5c8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:21.154649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7785b6-d1fd-4d37-a6d6-675e04d70b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:21.155307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7785b6-d1fd-4d37-a6d6-675e04d70b67 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:21.155590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7785b6-d1fd-4d37-a6d6-675e04d70b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:21.155870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7785b6-d1fd-4d37-a6d6-675e04d70b67 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:21.156564 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7785b6-d1fd-4d37-a6d6-675e04d70b67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:21.158517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7785b6-d1fd-4d37-a6d6-675e04d70b67 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:21.246753 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4158/8286] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:39:21 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:21.250917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384da77-0ec6-4efd-8542-d6f695dae415 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:21.262361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384da77-0ec6-4efd-8542-d6f695dae415 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:21.320010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ce06f-daea-42b9-8674-90fb64d9ed91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:21.320696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ce06f-daea-42b9-8674-90fb64d9ed91 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:21.320979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ce06f-daea-42b9-8674-90fb64d9ed91 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:21.321254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ce06f-daea-42b9-8674-90fb64d9ed91 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:21.321977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ce06f-daea-42b9-8674-90fb64d9ed91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:21.323908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ce06f-daea-42b9-8674-90fb64d9ed91 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:21.402588 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4129/8287] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:39:21 2022] POST /identity/v3/users => generated 284 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:21.406475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7785b6-d1fd-4d37-a6d6-675e04d70b67 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:21.417825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7785b6-d1fd-4d37-a6d6-675e04d70b67 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:21.422879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba8fc6-86b6-411b-835a-6500473612da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:21.423555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba8fc6-86b6-411b-835a-6500473612da admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:21.423848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba8fc6-86b6-411b-835a-6500473612da admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:21.424132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba8fc6-86b6-411b-835a-6500473612da admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:21.425407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba8fc6-86b6-411b-835a-6500473612da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:21.427052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba8fc6-86b6-411b-835a-6500473612da admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:21.438322 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4159/8288] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:21.444716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ce06f-daea-42b9-8674-90fb64d9ed91 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:21.455003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ce06f-daea-42b9-8674-90fb64d9ed91 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:21.459970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d1a48-56a2-4434-b324-7b79a3877850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:21.460636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d1a48-56a2-4434-b324-7b79a3877850 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:21.460916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d1a48-56a2-4434-b324-7b79a3877850 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:21.461193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d1a48-56a2-4434-b324-7b79a3877850 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:21.461958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d1a48-56a2-4434-b324-7b79a3877850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:21.463559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d1a48-56a2-4434-b324-7b79a3877850 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:21.474213 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4130/8289] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:21.478437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba8fc6-86b6-411b-835a-6500473612da admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:21.489321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba8fc6-86b6-411b-835a-6500473612da admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:21.494074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de8208-f1ce-421c-9fe4-f71c222b2ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:21.495018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de8208-f1ce-421c-9fe4-f71c222b2ed0 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:21.495329 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de8208-f1ce-421c-9fe4-f71c222b2ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:21.495614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de8208-f1ce-421c-9fe4-f71c222b2ed0 admin admin] PATH_INFO: `/v3/projects/01dbb4b670484fbab39af96b69a0d941/users/d7fc6bd952c849f595d1dd8d8e21cffd/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:21.512444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de8208-f1ce-421c-9fe4-f71c222b2ed0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01dbb4b670484fbab39af96b69a0d941, user_id=d7fc6bd952c849f595d1dd8d8e21cffd, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:21.523318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de8208-f1ce-421c-9fe4-f71c222b2ed0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:21.647295 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4160/8290] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:21 2022] PUT /identity/v3/projects/01dbb4b670484fbab39af96b69a0d941/users/d7fc6bd952c849f595d1dd8d8e21cffd/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:39:21.657649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac24a2-30e1-4f8d-9618-a830c3991b17 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:21.657955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac24a2-30e1-4f8d-9618-a830c3991b17 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:21.658238 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac24a2-30e1-4f8d-9618-a830c3991b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:21.686943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3bac24a2-30e1-4f8d-9618-a830c3991b17 None None] MFA Rules not processed for user `d7fc6bd952c849f595d1dd8d8e21cffd`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:21.688159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3bac24a2-30e1-4f8d-9618-a830c3991b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:21.757618 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4131/8291] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:21 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:21.770382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de8208-f1ce-421c-9fe4-f71c222b2ed0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:21.772625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-48de8208-f1ce-421c-9fe4-f71c222b2ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:21.838028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de8208-f1ce-421c-9fe4-f71c222b2ed0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:21.842808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66116175-393e-44e3-9f53-34fccabda205 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:21.843508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66116175-393e-44e3-9f53-34fccabda205 service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:21.843797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66116175-393e-44e3-9f53-34fccabda205 service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:21.844080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66116175-393e-44e3-9f53-34fccabda205 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:21.887269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:21.906046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66116175-393e-44e3-9f53-34fccabda205 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:21.907727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66116175-393e-44e3-9f53-34fccabda205 service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:21.918178 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4161/8292] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:21 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:39:22.175027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac24a2-30e1-4f8d-9618-a830c3991b17 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:22.187177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac24a2-30e1-4f8d-9618-a830c3991b17 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:22.191412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39709212-ccb0-487d-a631-965f4c146cdd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:22.192059 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39709212-ccb0-487d-a631-965f4c146cdd service glance] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:22.192348 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39709212-ccb0-487d-a631-965f4c146cdd service glance] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:22.192752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39709212-ccb0-487d-a631-965f4c146cdd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:22.201229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39709212-ccb0-487d-a631-965f4c146cdd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:22.202706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39709212-ccb0-487d-a631-965f4c146cdd service glance] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:22.211747 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4132/8293] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:22 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:24.516970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66116175-393e-44e3-9f53-34fccabda205 service glance] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:24.519171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-66116175-393e-44e3-9f53-34fccabda205 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:24.615158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66116175-393e-44e3-9f53-34fccabda205 service glance] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:24.622404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38794ee-074e-4402-8592-658c7ddf85b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:24.622404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38794ee-074e-4402-8592-658c7ddf85b2 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:24.622404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38794ee-074e-4402-8592-658c7ddf85b2 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:24.622404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38794ee-074e-4402-8592-658c7ddf85b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:24.624088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f38794ee-074e-4402-8592-658c7ddf85b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:24.644314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38794ee-074e-4402-8592-658c7ddf85b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:24.646047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38794ee-074e-4402-8592-658c7ddf85b2 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:24.661284 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4162/8294] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:24.682821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39709212-ccb0-487d-a631-965f4c146cdd service glance] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:24.723962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:24.753735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39709212-ccb0-487d-a631-965f4c146cdd service glance] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:24.758989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cd7ff-3e8d-4ec6-bd8b-ea2bb720559d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:24.759866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cd7ff-3e8d-4ec6-bd8b-ea2bb720559d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:24.760144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cd7ff-3e8d-4ec6-bd8b-ea2bb720559d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:24.760427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cd7ff-3e8d-4ec6-bd8b-ea2bb720559d admin admin] PATH_INFO: `/v3/users/0d453d6d31f04dc7a69fb5805728588e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:24.761737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cd7ff-3e8d-4ec6-bd8b-ea2bb720559d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d453d6d31f04dc7a69fb5805728588e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:24.763669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cd7ff-3e8d-4ec6-bd8b-ea2bb720559d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:24.975235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-047cd7ff-3e8d-4ec6-bd8b-ea2bb720559d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d453d6d31f04dc7a69fb5805728588e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:24.976098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-047cd7ff-3e8d-4ec6-bd8b-ea2bb720559d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d453d6d31f04dc7a69fb5805728588e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:24.985677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-047cd7ff-3e8d-4ec6-bd8b-ea2bb720559d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d453d6d31f04dc7a69fb5805728588e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:25.121334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-047cd7ff-3e8d-4ec6-bd8b-ea2bb720559d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d453d6d31f04dc7a69fb5805728588e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:25.121944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-047cd7ff-3e8d-4ec6-bd8b-ea2bb720559d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d453d6d31f04dc7a69fb5805728588e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:25.140400 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4133/8295] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:39:24 2022] DELETE /identity/v3/users/0d453d6d31f04dc7a69fb5805728588e => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:25.144891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963111d-6b70-4f29-ae90-52ef80a4161c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:25.145186 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963111d-6b70-4f29-ae90-52ef80a4161c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:25.145541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963111d-6b70-4f29-ae90-52ef80a4161c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:25.162443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e963111d-6b70-4f29-ae90-52ef80a4161c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:25.163638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e963111d-6b70-4f29-ae90-52ef80a4161c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:25.226827 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4163/8296] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:25.237338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cd7ff-3e8d-4ec6-bd8b-ea2bb720559d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:25.238825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-047cd7ff-3e8d-4ec6-bd8b-ea2bb720559d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:25.300242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cd7ff-3e8d-4ec6-bd8b-ea2bb720559d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:25.304174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf970d-c4c7-4ec0-80ff-40a8bc7b72a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:25.304856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf970d-c4c7-4ec0-80ff-40a8bc7b72a0 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:25.305140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf970d-c4c7-4ec0-80ff-40a8bc7b72a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:25.305417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf970d-c4c7-4ec0-80ff-40a8bc7b72a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:25.316026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf970d-c4c7-4ec0-80ff-40a8bc7b72a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:25.318119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf970d-c4c7-4ec0-80ff-40a8bc7b72a0 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:25.328766 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4134/8297] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:25.812874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963111d-6b70-4f29-ae90-52ef80a4161c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:25.814407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e963111d-6b70-4f29-ae90-52ef80a4161c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:25.830247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963111d-6b70-4f29-ae90-52ef80a4161c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:25.836026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:25.836905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:25.837187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:25.837465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90 admin admin] PATH_INFO: `/v3/projects/773aefd97d374cdc9497c9d39cbdd9bc` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:25.838777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=773aefd97d374cdc9497c9d39cbdd9bc)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:25.841461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:25.988266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '773aefd97d374cdc9497c9d39cbdd9bc', 'request_id': 'req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:25.998412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '773aefd97d374cdc9497c9d39cbdd9bc', 'request_id': 'req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:26.000521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:26.114300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '773aefd97d374cdc9497c9d39cbdd9bc', 'request_id': 'req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:26.126303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90 admin admin] The token cache is being invalidate because project 773aefd97d374cdc9497c9d39cbdd9bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:26.126613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:26.127241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:26.128862 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4164/8298] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:25 2022] DELETE /identity/v3/projects/773aefd97d374cdc9497c9d39cbdd9bc => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:26.133069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf970d-c4c7-4ec0-80ff-40a8bc7b72a0 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:26.143240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf970d-c4c7-4ec0-80ff-40a8bc7b72a0 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:26.193217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:26.201703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c449731c-f9b2-4e05-89c1-b7da71675bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:26.202607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449731c-f9b2-4e05-89c1-b7da71675bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:26.202892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449731c-f9b2-4e05-89c1-b7da71675bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:26.203184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c449731c-f9b2-4e05-89c1-b7da71675bdb admin admin] PATH_INFO: `/v3/users/d7fc6bd952c849f595d1dd8d8e21cffd` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:26.204430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449731c-f9b2-4e05-89c1-b7da71675bdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7fc6bd952c849f595d1dd8d8e21cffd)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:26.206440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c449731c-f9b2-4e05-89c1-b7da71675bdb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:26.426906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c449731c-f9b2-4e05-89c1-b7da71675bdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7fc6bd952c849f595d1dd8d8e21cffd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:26.427578 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c449731c-f9b2-4e05-89c1-b7da71675bdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7fc6bd952c849f595d1dd8d8e21cffd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:26.434920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c449731c-f9b2-4e05-89c1-b7da71675bdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7fc6bd952c849f595d1dd8d8e21cffd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:26.515482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c449731c-f9b2-4e05-89c1-b7da71675bdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7fc6bd952c849f595d1dd8d8e21cffd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:26.516258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c449731c-f9b2-4e05-89c1-b7da71675bdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7fc6bd952c849f595d1dd8d8e21cffd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:26.535262 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4135/8299] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:39:26 2022] DELETE /identity/v3/users/d7fc6bd952c849f595d1dd8d8e21cffd => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:26.984877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:26.986407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:27.001736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89e9eb-3e69-439d-b103-dc2a3aa74b90 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:27.088601 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f82e6-ff86-4d9f-835b-ab925998439a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:27.089565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f82e6-ff86-4d9f-835b-ab925998439a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:27.089909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f82e6-ff86-4d9f-835b-ab925998439a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:27.090191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f82e6-ff86-4d9f-835b-ab925998439a admin admin] PATH_INFO: `/v3/projects/01dbb4b670484fbab39af96b69a0d941` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:27.091539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f82e6-ff86-4d9f-835b-ab925998439a admin admin] RBAC: Authorizing `identity:delete_project(project_id=01dbb4b670484fbab39af96b69a0d941)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:27.093470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f82e6-ff86-4d9f-835b-ab925998439a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:27.260276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-357f82e6-ff86-4d9f-835b-ab925998439a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01dbb4b670484fbab39af96b69a0d941', 'request_id': 'req-357f82e6-ff86-4d9f-835b-ab925998439a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:27.266044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-357f82e6-ff86-4d9f-835b-ab925998439a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01dbb4b670484fbab39af96b69a0d941', 'request_id': 'req-357f82e6-ff86-4d9f-835b-ab925998439a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:27.349675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-357f82e6-ff86-4d9f-835b-ab925998439a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01dbb4b670484fbab39af96b69a0d941', 'request_id': 'req-357f82e6-ff86-4d9f-835b-ab925998439a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:27.361584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-357f82e6-ff86-4d9f-835b-ab925998439a admin admin] The token cache is being invalidate because project 01dbb4b670484fbab39af96b69a0d941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:27.362404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-357f82e6-ff86-4d9f-835b-ab925998439a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:27.363133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-357f82e6-ff86-4d9f-835b-ab925998439a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:27.364696 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4165/8300] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:26 2022] DELETE /identity/v3/projects/01dbb4b670484fbab39af96b69a0d941 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:27.370827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e3db8-3bf5-4fba-b991-434d14d4be66 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:27.371322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e3db8-3bf5-4fba-b991-434d14d4be66 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:27.371423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4e3db8-3bf5-4fba-b991-434d14d4be66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:27.390169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-cd4e3db8-3bf5-4fba-b991-434d14d4be66 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:27.391412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cd4e3db8-3bf5-4fba-b991-434d14d4be66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:27.450848 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4136/8301] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:39:27.459402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500998d-2a12-43e0-acce-002c3e54f1bd None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:27.459695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500998d-2a12-43e0-acce-002c3e54f1bd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:27.460034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7500998d-2a12-43e0-acce-002c3e54f1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:27.479447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7500998d-2a12-43e0-acce-002c3e54f1bd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:27.480807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7500998d-2a12-43e0-acce-002c3e54f1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:27.489649 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4166/8302] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:27.493961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e3db8-3bf5-4fba-b991-434d14d4be66 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:27.504499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4e3db8-3bf5-4fba-b991-434d14d4be66 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:27.509173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef692d2a-e811-48c2-b8b3-319819bd59c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:27.509828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef692d2a-e811-48c2-b8b3-319819bd59c0 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:27.510136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef692d2a-e811-48c2-b8b3-319819bd59c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:27.510465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef692d2a-e811-48c2-b8b3-319819bd59c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:27.511144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef692d2a-e811-48c2-b8b3-319819bd59c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:39:27.511453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef692d2a-e811-48c2-b8b3-319819bd59c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:27.513063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef692d2a-e811-48c2-b8b3-319819bd59c0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:27.525737 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4137/8303] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:39:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:39:27.530260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500998d-2a12-43e0-acce-002c3e54f1bd None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:27.539715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7500998d-2a12-43e0-acce-002c3e54f1bd None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:27.544549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f063f-d664-4911-bb85-9e5d25b3f9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:27.545209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f063f-d664-4911-bb85-9e5d25b3f9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:27.545496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f063f-d664-4911-bb85-9e5d25b3f9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:27.545788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f063f-d664-4911-bb85-9e5d25b3f9b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:27.546546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f063f-d664-4911-bb85-9e5d25b3f9b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:27.548557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f063f-d664-4911-bb85-9e5d25b3f9b2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:27.660699 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4167/8304] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:39:27 2022] POST /identity/v3/projects => generated 367 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:27.664950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef692d2a-e811-48c2-b8b3-319819bd59c0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:27.675545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef692d2a-e811-48c2-b8b3-319819bd59c0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:27.732069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174dbf0e-97d9-4c61-b1fb-e21ea43d99c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:27.732844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174dbf0e-97d9-4c61-b1fb-e21ea43d99c4 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:27.733132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174dbf0e-97d9-4c61-b1fb-e21ea43d99c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:27.733414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174dbf0e-97d9-4c61-b1fb-e21ea43d99c4 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:27.734143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174dbf0e-97d9-4c61-b1fb-e21ea43d99c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:27.736082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174dbf0e-97d9-4c61-b1fb-e21ea43d99c4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:27.743391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:27.841076 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4138/8305] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:39:27 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:27.845325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f063f-d664-4911-bb85-9e5d25b3f9b2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:27.856720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f063f-d664-4911-bb85-9e5d25b3f9b2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:27.861394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c920e78-026d-43dc-81db-1e40e5f4d196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:27.866699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c920e78-026d-43dc-81db-1e40e5f4d196 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:27.866699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c920e78-026d-43dc-81db-1e40e5f4d196 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:27.866699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c920e78-026d-43dc-81db-1e40e5f4d196 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:27.866699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c920e78-026d-43dc-81db-1e40e5f4d196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:27.866699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c920e78-026d-43dc-81db-1e40e5f4d196 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:27.876805 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4168/8306] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:27.880832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174dbf0e-97d9-4c61-b1fb-e21ea43d99c4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:27.890994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174dbf0e-97d9-4c61-b1fb-e21ea43d99c4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:27.895832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c787a4-2183-4d3a-9a9e-fee0eac6d2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:27.896491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c787a4-2183-4d3a-9a9e-fee0eac6d2bc admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:27.896780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c787a4-2183-4d3a-9a9e-fee0eac6d2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:27.897058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c787a4-2183-4d3a-9a9e-fee0eac6d2bc admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:27.897810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c787a4-2183-4d3a-9a9e-fee0eac6d2bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:27.899454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c787a4-2183-4d3a-9a9e-fee0eac6d2bc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:27.910868 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4139/8307] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:27.914477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c920e78-026d-43dc-81db-1e40e5f4d196 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:27.925062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c920e78-026d-43dc-81db-1e40e5f4d196 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:27.930126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e28b75-27cf-4b29-a84a-505ad6fc03cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:27.930997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e28b75-27cf-4b29-a84a-505ad6fc03cd admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:27.931281 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e28b75-27cf-4b29-a84a-505ad6fc03cd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:27.931555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e28b75-27cf-4b29-a84a-505ad6fc03cd admin admin] PATH_INFO: `/v3/projects/03f003c9e52d4075ab46fad9146e91b0/users/28258d38b18949feb13ecec71b366265/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:27.946110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e28b75-27cf-4b29-a84a-505ad6fc03cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=03f003c9e52d4075ab46fad9146e91b0, user_id=28258d38b18949feb13ecec71b366265, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:27.948875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e28b75-27cf-4b29-a84a-505ad6fc03cd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:28.035804 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4169/8308] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:27 2022] PUT /identity/v3/projects/03f003c9e52d4075ab46fad9146e91b0/users/28258d38b18949feb13ecec71b366265/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:28.048179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b01e4-50b0-4e34-8bb7-d606b4d2d52c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:28.048474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b01e4-50b0-4e34-8bb7-d606b4d2d52c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:28.048755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b01e4-50b0-4e34-8bb7-d606b4d2d52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:28.081295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-053b01e4-50b0-4e34-8bb7-d606b4d2d52c None None] MFA Rules not processed for user `28258d38b18949feb13ecec71b366265`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:28.082602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-053b01e4-50b0-4e34-8bb7-d606b4d2d52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:28.148874 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4140/8309] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:28 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:28.163165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e28b75-27cf-4b29-a84a-505ad6fc03cd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:28.163165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b5e28b75-27cf-4b29-a84a-505ad6fc03cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:28.177475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:28.223694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e28b75-27cf-4b29-a84a-505ad6fc03cd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:28.227335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e84f7c1-e0e7-451b-921a-061f344a3ae0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:28.227980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84f7c1-e0e7-451b-921a-061f344a3ae0 service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:28.228333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84f7c1-e0e7-451b-921a-061f344a3ae0 service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:28.228619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e84f7c1-e0e7-451b-921a-061f344a3ae0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:28.237804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84f7c1-e0e7-451b-921a-061f344a3ae0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:28.239270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e84f7c1-e0e7-451b-921a-061f344a3ae0 service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:28.248067 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4170/8310] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:28 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:28.343246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b01e4-50b0-4e34-8bb7-d606b4d2d52c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:28.408272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b01e4-50b0-4e34-8bb7-d606b4d2d52c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:28.413534 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629dd28-9c5b-4d9a-9c72-ec83e3016145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:28.415960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629dd28-9c5b-4d9a-9c72-ec83e3016145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:28.415960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629dd28-9c5b-4d9a-9c72-ec83e3016145 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:28.415960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629dd28-9c5b-4d9a-9c72-ec83e3016145 admin admin] PATH_INFO: `/v3/users/28258d38b18949feb13ecec71b366265` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:28.416254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629dd28-9c5b-4d9a-9c72-ec83e3016145 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28258d38b18949feb13ecec71b366265)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:28.418255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f629dd28-9c5b-4d9a-9c72-ec83e3016145 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:28.598128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f629dd28-9c5b-4d9a-9c72-ec83e3016145 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28258d38b18949feb13ecec71b366265'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:28.598784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f629dd28-9c5b-4d9a-9c72-ec83e3016145 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28258d38b18949feb13ecec71b366265'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:28.605755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f629dd28-9c5b-4d9a-9c72-ec83e3016145 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28258d38b18949feb13ecec71b366265'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:28.698143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f629dd28-9c5b-4d9a-9c72-ec83e3016145 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28258d38b18949feb13ecec71b366265'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:28.698771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-f629dd28-9c5b-4d9a-9c72-ec83e3016145 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28258d38b18949feb13ecec71b366265'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:28.705035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:28.718043 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4141/8311] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:39:28 2022] DELETE /identity/v3/users/28258d38b18949feb13ecec71b366265 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:28.723124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595f675-2c3b-41d2-bc5e-e22703df0b17 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:28.723530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595f675-2c3b-41d2-bc5e-e22703df0b17 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:28.723817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595f675-2c3b-41d2-bc5e-e22703df0b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:28.740973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9595f675-2c3b-41d2-bc5e-e22703df0b17 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:28.742252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9595f675-2c3b-41d2-bc5e-e22703df0b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:28.805367 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4171/8312] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:28.815156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629dd28-9c5b-4d9a-9c72-ec83e3016145 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:28.816562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f629dd28-9c5b-4d9a-9c72-ec83e3016145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:28.878258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629dd28-9c5b-4d9a-9c72-ec83e3016145 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:28.881995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242938b1-d55c-41f5-86db-f91077f427fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:28.882645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242938b1-d55c-41f5-86db-f91077f427fc service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:28.882934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242938b1-d55c-41f5-86db-f91077f427fc service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:28.883218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242938b1-d55c-41f5-86db-f91077f427fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:28.892338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242938b1-d55c-41f5-86db-f91077f427fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:28.894330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242938b1-d55c-41f5-86db-f91077f427fc service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:28.904052 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4142/8313] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:29.310768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595f675-2c3b-41d2-bc5e-e22703df0b17 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:29.312218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9595f675-2c3b-41d2-bc5e-e22703df0b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:29.332758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595f675-2c3b-41d2-bc5e-e22703df0b17 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:29.332758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df331b-f563-4c4a-adc7-8ad0e9414514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:29.332758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df331b-f563-4c4a-adc7-8ad0e9414514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:29.332758 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df331b-f563-4c4a-adc7-8ad0e9414514 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:29.333991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df331b-f563-4c4a-adc7-8ad0e9414514 admin admin] PATH_INFO: `/v3/projects/03f003c9e52d4075ab46fad9146e91b0` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:29.334205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df331b-f563-4c4a-adc7-8ad0e9414514 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03f003c9e52d4075ab46fad9146e91b0)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:29.336761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df331b-f563-4c4a-adc7-8ad0e9414514 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:29.418339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f8df331b-f563-4c4a-adc7-8ad0e9414514 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03f003c9e52d4075ab46fad9146e91b0', 'request_id': 'req-f8df331b-f563-4c4a-adc7-8ad0e9414514'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:29.424389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f8df331b-f563-4c4a-adc7-8ad0e9414514 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03f003c9e52d4075ab46fad9146e91b0', 'request_id': 'req-f8df331b-f563-4c4a-adc7-8ad0e9414514'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:29.498163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f8df331b-f563-4c4a-adc7-8ad0e9414514 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03f003c9e52d4075ab46fad9146e91b0', 'request_id': 'req-f8df331b-f563-4c4a-adc7-8ad0e9414514'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:29.510751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f8df331b-f563-4c4a-adc7-8ad0e9414514 admin admin] The token cache is being invalidate because project 03f003c9e52d4075ab46fad9146e91b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:29.511045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f8df331b-f563-4c4a-adc7-8ad0e9414514 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:29.511716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f8df331b-f563-4c4a-adc7-8ad0e9414514 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:29.513400 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4172/8314] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:29 2022] DELETE /identity/v3/projects/03f003c9e52d4075ab46fad9146e91b0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:29.518773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02564e-1c2e-4bef-b011-713c4e05d8ce None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:29.519251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02564e-1c2e-4bef-b011-713c4e05d8ce None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:29.519633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02564e-1c2e-4bef-b011-713c4e05d8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:29.537397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2d02564e-1c2e-4bef-b011-713c4e05d8ce None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:29.538638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2d02564e-1c2e-4bef-b011-713c4e05d8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:29.599758 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4143/8315] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:29.607473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8f851-8e10-47cd-bf63-91620f821d3d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:29.607770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8f851-8e10-47cd-bf63-91620f821d3d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:29.608059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8f851-8e10-47cd-bf63-91620f821d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:29.626788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-22c8f851-8e10-47cd-bf63-91620f821d3d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:29.628022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-22c8f851-8e10-47cd-bf63-91620f821d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:29.636654 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4173/8316] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:29.640933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02564e-1c2e-4bef-b011-713c4e05d8ce None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:29.650892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02564e-1c2e-4bef-b011-713c4e05d8ce None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:29.655520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900b068-2908-4528-ba5f-a3d35eca5d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:29.656166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900b068-2908-4528-ba5f-a3d35eca5d0d admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:29.656448 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900b068-2908-4528-ba5f-a3d35eca5d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:29.657084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900b068-2908-4528-ba5f-a3d35eca5d0d admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:29.657812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900b068-2908-4528-ba5f-a3d35eca5d0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:39:29.662554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900b068-2908-4528-ba5f-a3d35eca5d0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:29.662554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900b068-2908-4528-ba5f-a3d35eca5d0d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:29.672117 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4144/8317] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:39:29.676699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8f851-8e10-47cd-bf63-91620f821d3d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:29.686097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8f851-8e10-47cd-bf63-91620f821d3d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:29.690786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d231cf6-f39a-41e1-8f6f-e26968a3cee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:29.691495 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d231cf6-f39a-41e1-8f6f-e26968a3cee9 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:29.692299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d231cf6-f39a-41e1-8f6f-e26968a3cee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:29.692586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d231cf6-f39a-41e1-8f6f-e26968a3cee9 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:29.693283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d231cf6-f39a-41e1-8f6f-e26968a3cee9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:29.695325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d231cf6-f39a-41e1-8f6f-e26968a3cee9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:29.814529 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4174/8318] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:39:29 2022] POST /identity/v3/projects => generated 357 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:29.822523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900b068-2908-4528-ba5f-a3d35eca5d0d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:29.832822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900b068-2908-4528-ba5f-a3d35eca5d0d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:29.889689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf2b3b-9555-4e84-a74e-d36e0bcbfc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:29.890400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf2b3b-9555-4e84-a74e-d36e0bcbfc74 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:29.890682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf2b3b-9555-4e84-a74e-d36e0bcbfc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:29.890990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf2b3b-9555-4e84-a74e-d36e0bcbfc74 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:29.891685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf2b3b-9555-4e84-a74e-d36e0bcbfc74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:29.893645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf2b3b-9555-4e84-a74e-d36e0bcbfc74 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:29.999950 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4145/8319] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:39:29 2022] POST /identity/v3/users => generated 279 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:30.004679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d231cf6-f39a-41e1-8f6f-e26968a3cee9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:30.015139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d231cf6-f39a-41e1-8f6f-e26968a3cee9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:30.020600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862f093-f4ec-498b-9775-893e93455074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:30.021275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862f093-f4ec-498b-9775-893e93455074 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:30.021567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862f093-f4ec-498b-9775-893e93455074 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:30.021870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e862f093-f4ec-498b-9775-893e93455074 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:30.025871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862f093-f4ec-498b-9775-893e93455074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:30.025871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e862f093-f4ec-498b-9775-893e93455074 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:30.029609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:30.035134 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4175/8320] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:30.038955 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf2b3b-9555-4e84-a74e-d36e0bcbfc74 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:30.048918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf2b3b-9555-4e84-a74e-d36e0bcbfc74 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:30.053693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21c7ab-fce7-4e76-88ba-fa5880ac107e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:30.054398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21c7ab-fce7-4e76-88ba-fa5880ac107e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:30.054683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21c7ab-fce7-4e76-88ba-fa5880ac107e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:30.054963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21c7ab-fce7-4e76-88ba-fa5880ac107e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:30.055685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21c7ab-fce7-4e76-88ba-fa5880ac107e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:30.057332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21c7ab-fce7-4e76-88ba-fa5880ac107e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:30.058662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:30.072337 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4146/8321] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:30.075008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862f093-f4ec-498b-9775-893e93455074 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:30.086118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e862f093-f4ec-498b-9775-893e93455074 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:30.092034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffc7aa-19bf-470b-83d7-69febb9703f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:30.095385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffc7aa-19bf-470b-83d7-69febb9703f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:30.095385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffc7aa-19bf-470b-83d7-69febb9703f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:30.095385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffc7aa-19bf-470b-83d7-69febb9703f2 admin admin] PATH_INFO: `/v3/projects/3306093318744a8980a3e9bb25772afb/users/3b94e96f9200494d9deacfa0ff851e2e/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:30.108267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffc7aa-19bf-470b-83d7-69febb9703f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3306093318744a8980a3e9bb25772afb, user_id=3b94e96f9200494d9deacfa0ff851e2e, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:30.111011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffc7aa-19bf-470b-83d7-69febb9703f2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:30.207373 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4176/8322] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:30 2022] PUT /identity/v3/projects/3306093318744a8980a3e9bb25772afb/users/3b94e96f9200494d9deacfa0ff851e2e/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:30.217745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec66235-08a8-4ce4-9ced-c3e81909ee58 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:30.218060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec66235-08a8-4ce4-9ced-c3e81909ee58 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:30.218341 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec66235-08a8-4ce4-9ced-c3e81909ee58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:30.245198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4ec66235-08a8-4ce4-9ced-c3e81909ee58 None None] MFA Rules not processed for user `3b94e96f9200494d9deacfa0ff851e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:30.246427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4ec66235-08a8-4ce4-9ced-c3e81909ee58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:30.311428 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4147/8323] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:30 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:30.315930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffc7aa-19bf-470b-83d7-69febb9703f2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:30.382797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffc7aa-19bf-470b-83d7-69febb9703f2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:30.387683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd5c6d-88ee-4361-ad2b-30a7e45a007a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:30.388342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd5c6d-88ee-4361-ad2b-30a7e45a007a admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:30.388639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd5c6d-88ee-4361-ad2b-30a7e45a007a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:30.388912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd5c6d-88ee-4361-ad2b-30a7e45a007a admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:30.389665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd5c6d-88ee-4361-ad2b-30a7e45a007a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:30.391618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd5c6d-88ee-4361-ad2b-30a7e45a007a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:30.473962 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4177/8324] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:39:30 2022] POST /identity/v3/projects => generated 357 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:30.475177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec66235-08a8-4ce4-9ced-c3e81909ee58 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:30.486421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec66235-08a8-4ce4-9ced-c3e81909ee58 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:30.545542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62ddd5-d5f8-40a1-96bd-b5ed09815ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:30.546264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62ddd5-d5f8-40a1-96bd-b5ed09815ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:30.546529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62ddd5-d5f8-40a1-96bd-b5ed09815ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:30.546811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df62ddd5-d5f8-40a1-96bd-b5ed09815ee3 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:30.547495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62ddd5-d5f8-40a1-96bd-b5ed09815ee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:30.549501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df62ddd5-d5f8-40a1-96bd-b5ed09815ee3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:30.654156 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4148/8325] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:39:30 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:30.658174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd5c6d-88ee-4361-ad2b-30a7e45a007a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:30.669796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd5c6d-88ee-4361-ad2b-30a7e45a007a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:30.674694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fb332-4828-4cf4-b25f-e323c2b33647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:30.675349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fb332-4828-4cf4-b25f-e323c2b33647 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:30.675669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fb332-4828-4cf4-b25f-e323c2b33647 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:30.676756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fb332-4828-4cf4-b25f-e323c2b33647 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:30.677519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fb332-4828-4cf4-b25f-e323c2b33647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:30.679211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fb332-4828-4cf4-b25f-e323c2b33647 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:30.690423 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4178/8326] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:30.696508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62ddd5-d5f8-40a1-96bd-b5ed09815ee3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:30.707056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df62ddd5-d5f8-40a1-96bd-b5ed09815ee3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:30.711929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e883c-d820-455e-b0a3-0f7bd982cae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:30.712823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e883c-d820-455e-b0a3-0f7bd982cae7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:30.713110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e883c-d820-455e-b0a3-0f7bd982cae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:30.713476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e883c-d820-455e-b0a3-0f7bd982cae7 admin admin] PATH_INFO: `/v3/projects/3c2c36d457f24362b3ebfd68d26f58c6/users/52ce4ff4043641b2bd33ed52d0f51a45/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:30.729356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e883c-d820-455e-b0a3-0f7bd982cae7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c2c36d457f24362b3ebfd68d26f58c6, user_id=52ce4ff4043641b2bd33ed52d0f51a45, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:30.732108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e883c-d820-455e-b0a3-0f7bd982cae7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:30.802134 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4149/8327] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:30 2022] PUT /identity/v3/projects/3c2c36d457f24362b3ebfd68d26f58c6/users/52ce4ff4043641b2bd33ed52d0f51a45/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:30.806294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fb332-4828-4cf4-b25f-e323c2b33647 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:30.816687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fb332-4828-4cf4-b25f-e323c2b33647 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:30.873682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7a56ca-b6f0-41f6-ac19-dedba9b04d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:30.874331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a56ca-b6f0-41f6-ac19-dedba9b04d29 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:30.874623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a56ca-b6f0-41f6-ac19-dedba9b04d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:30.874907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7a56ca-b6f0-41f6-ac19-dedba9b04d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:30.875645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a56ca-b6f0-41f6-ac19-dedba9b04d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:30.877254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7a56ca-b6f0-41f6-ac19-dedba9b04d29 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:30.890988 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4179/8328] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:30 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:30.892109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e883c-d820-455e-b0a3-0f7bd982cae7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:30.904270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e883c-d820-455e-b0a3-0f7bd982cae7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:30.909208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13916f5a-527d-4929-b4e0-33a6ba5251ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:30.910776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13916f5a-527d-4929-b4e0-33a6ba5251ea admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:30.911063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13916f5a-527d-4929-b4e0-33a6ba5251ea admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:30.911339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13916f5a-527d-4929-b4e0-33a6ba5251ea admin admin] PATH_INFO: `/v3/projects/3c2c36d457f24362b3ebfd68d26f58c6/users/52ce4ff4043641b2bd33ed52d0f51a45/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:30.913345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13916f5a-527d-4929-b4e0-33a6ba5251ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c2c36d457f24362b3ebfd68d26f58c6, user_id=52ce4ff4043641b2bd33ed52d0f51a45, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:30.916045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13916f5a-527d-4929-b4e0-33a6ba5251ea admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:30.927257 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4150/8329] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:30 2022] PUT /identity/v3/projects/3c2c36d457f24362b3ebfd68d26f58c6/users/52ce4ff4043641b2bd33ed52d0f51a45/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:39:30.938756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32a580-b70f-41e3-8b71-2d66e42ca930 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:30.939087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32a580-b70f-41e3-8b71-2d66e42ca930 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:30.939374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb32a580-b70f-41e3-8b71-2d66e42ca930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:30.966597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fb32a580-b70f-41e3-8b71-2d66e42ca930 None None] MFA Rules not processed for user `52ce4ff4043641b2bd33ed52d0f51a45`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:30.967739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fb32a580-b70f-41e3-8b71-2d66e42ca930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:31.002694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:31.038055 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4180/8330] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:30 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:31.043336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13916f5a-527d-4929-b4e0-33a6ba5251ea admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:31.054121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13916f5a-527d-4929-b4e0-33a6ba5251ea admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:31.059737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ed9eb-201a-47b1-90ef-9009c3298de4 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ce4ff4043641b2bd33ed52d0f51a45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2c36d457f24362b3ebfd68d26f58c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:31.060441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ed9eb-201a-47b1-90ef-9009c3298de4 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:31.060735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ed9eb-201a-47b1-90ef-9009c3298de4 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:31.061019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ed9eb-201a-47b1-90ef-9009c3298de4 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:31.061781 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ed9eb-201a-47b1-90ef-9009c3298de4 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:39:31.062144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ed9eb-201a-47b1-90ef-9009c3298de4 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:31.063811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ed9eb-201a-47b1-90ef-9009c3298de4 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:31.077540 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4151/8331] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:39:31.081756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32a580-b70f-41e3-8b71-2d66e42ca930 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:31.091785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb32a580-b70f-41e3-8b71-2d66e42ca930 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:31.096540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f0b8e-3f7d-42b6-9c54-3bb37692e32e tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ce4ff4043641b2bd33ed52d0f51a45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2c36d457f24362b3ebfd68d26f58c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:31.097190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f0b8e-3f7d-42b6-9c54-3bb37692e32e tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:31.097484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f0b8e-3f7d-42b6-9c54-3bb37692e32e tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:31.098391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15f0b8e-3f7d-42b6-9c54-3bb37692e32e tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:31.099095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f0b8e-3f7d-42b6-9c54-3bb37692e32e tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:31.101013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15f0b8e-3f7d-42b6-9c54-3bb37692e32e tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:31.212714 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4181/8332] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:39:31 2022] POST /identity/v3/projects => generated 349 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:31.217543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ed9eb-201a-47b1-90ef-9009c3298de4 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:31.229461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ed9eb-201a-47b1-90ef-9009c3298de4 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:31.249468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:31.288255 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80338ce5-be38-4936-90e7-607a91052401 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ce4ff4043641b2bd33ed52d0f51a45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2c36d457f24362b3ebfd68d26f58c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:31.288925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80338ce5-be38-4936-90e7-607a91052401 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:31.289212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80338ce5-be38-4936-90e7-607a91052401 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:31.289492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80338ce5-be38-4936-90e7-607a91052401 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:31.290260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80338ce5-be38-4936-90e7-607a91052401 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:39:31.292036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80338ce5-be38-4936-90e7-607a91052401 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:31.292294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80338ce5-be38-4936-90e7-607a91052401 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:31.305536 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4152/8333] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:39:31.315504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f0b8e-3f7d-42b6-9c54-3bb37692e32e tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:31.317780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a15f0b8e-3f7d-42b6-9c54-3bb37692e32e tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:31.393346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15f0b8e-3f7d-42b6-9c54-3bb37692e32e tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:31.398166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c809e8-ae36-4b3f-b386-7f8e1f99cbfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:31.398830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c809e8-ae36-4b3f-b386-7f8e1f99cbfb service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:31.399118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c809e8-ae36-4b3f-b386-7f8e1f99cbfb service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:31.399458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c809e8-ae36-4b3f-b386-7f8e1f99cbfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:31.419786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c809e8-ae36-4b3f-b386-7f8e1f99cbfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:31.421360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c809e8-ae36-4b3f-b386-7f8e1f99cbfb service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:31.434339 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4182/8334] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:31 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:31.755982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80338ce5-be38-4936-90e7-607a91052401 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:31.766152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80338ce5-be38-4936-90e7-607a91052401 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:31.771233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5c849-2daf-4e6b-a8a3-47383b4f30e2 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52ce4ff4043641b2bd33ed52d0f51a45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3c2c36d457f24362b3ebfd68d26f58c6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:31.772086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5c849-2daf-4e6b-a8a3-47383b4f30e2 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:31.772370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5c849-2daf-4e6b-a8a3-47383b4f30e2 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:31.772650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5c849-2daf-4e6b-a8a3-47383b4f30e2 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] PATH_INFO: `/v3/projects/2d4310e8e1254f0c90a10478af4cbc40` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:31.773873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b5c849-2daf-4e6b-a8a3-47383b4f30e2 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] RBAC: Authorizing `identity:delete_project(project_id=2d4310e8e1254f0c90a10478af4cbc40)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:31.775989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b5c849-2daf-4e6b-a8a3-47383b4f30e2 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:31.911140 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-44b5c849-2daf-4e6b-a8a3-47383b4f30e2 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d4310e8e1254f0c90a10478af4cbc40', 'request_id': 'req-44b5c849-2daf-4e6b-a8a3-47383b4f30e2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:31.916778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-44b5c849-2daf-4e6b-a8a3-47383b4f30e2 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d4310e8e1254f0c90a10478af4cbc40', 'request_id': 'req-44b5c849-2daf-4e6b-a8a3-47383b4f30e2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:31.961080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-44b5c849-2daf-4e6b-a8a3-47383b4f30e2 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d4310e8e1254f0c90a10478af4cbc40', 'request_id': 'req-44b5c849-2daf-4e6b-a8a3-47383b4f30e2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:31.974073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-44b5c849-2daf-4e6b-a8a3-47383b4f30e2 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] The token cache is being invalidate because project 2d4310e8e1254f0c90a10478af4cbc40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:31.974364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-44b5c849-2daf-4e6b-a8a3-47383b4f30e2 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:31.975004 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-44b5c849-2daf-4e6b-a8a3-47383b4f30e2 tempest-QuotasTest-2128728723 tempest-QuotasTest-2128728723-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:31.976566 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4153/8335] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:31 2022] DELETE /identity/v3/projects/2d4310e8e1254f0c90a10478af4cbc40 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:32.099278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c809e8-ae36-4b3f-b386-7f8e1f99cbfb service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:32.163231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c809e8-ae36-4b3f-b386-7f8e1f99cbfb service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:32.218828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf40308-163b-4f16-8c87-515542c2bffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:32.219705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf40308-163b-4f16-8c87-515542c2bffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:32.219989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf40308-163b-4f16-8c87-515542c2bffe admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:32.220267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf40308-163b-4f16-8c87-515542c2bffe admin admin] PATH_INFO: `/v3/users/3b94e96f9200494d9deacfa0ff851e2e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:32.221455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf40308-163b-4f16-8c87-515542c2bffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b94e96f9200494d9deacfa0ff851e2e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:32.223657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf40308-163b-4f16-8c87-515542c2bffe admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:32.225217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:32.415158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cbf40308-163b-4f16-8c87-515542c2bffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b94e96f9200494d9deacfa0ff851e2e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:32.422389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cbf40308-163b-4f16-8c87-515542c2bffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b94e96f9200494d9deacfa0ff851e2e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:32.423071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cbf40308-163b-4f16-8c87-515542c2bffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b94e96f9200494d9deacfa0ff851e2e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:32.423726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cbf40308-163b-4f16-8c87-515542c2bffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b94e96f9200494d9deacfa0ff851e2e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:32.492662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-cbf40308-163b-4f16-8c87-515542c2bffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b94e96f9200494d9deacfa0ff851e2e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:32.512413 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4183/8336] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:39:32 2022] DELETE /identity/v3/users/3b94e96f9200494d9deacfa0ff851e2e => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:32.517159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516249ec-cd82-4d22-80e4-9423c1e39b84 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:32.517453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516249ec-cd82-4d22-80e4-9423c1e39b84 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:32.517736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516249ec-cd82-4d22-80e4-9423c1e39b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:32.537163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-516249ec-cd82-4d22-80e4-9423c1e39b84 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:32.538484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-516249ec-cd82-4d22-80e4-9423c1e39b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:32.610336 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4154/8337] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:32.621515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf40308-163b-4f16-8c87-515542c2bffe admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:32.623242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cbf40308-163b-4f16-8c87-515542c2bffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:32.689523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf40308-163b-4f16-8c87-515542c2bffe admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:32.694140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134715db-34f8-4b5b-bb4c-dab11c065a4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:32.694856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134715db-34f8-4b5b-bb4c-dab11c065a4c service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:32.695144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134715db-34f8-4b5b-bb4c-dab11c065a4c service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:32.695456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134715db-34f8-4b5b-bb4c-dab11c065a4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:32.704943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134715db-34f8-4b5b-bb4c-dab11c065a4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:32.706436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134715db-34f8-4b5b-bb4c-dab11c065a4c service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:32.716427 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4184/8338] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:33.116513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516249ec-cd82-4d22-80e4-9423c1e39b84 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:33.118165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-516249ec-cd82-4d22-80e4-9423c1e39b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:33.131655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516249ec-cd82-4d22-80e4-9423c1e39b84 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:33.136333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:33.137212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:33.137505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:33.137787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b admin admin] PATH_INFO: `/v3/projects/3306093318744a8980a3e9bb25772afb` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:33.139033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3306093318744a8980a3e9bb25772afb)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:33.141036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:33.239052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:33.247437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3306093318744a8980a3e9bb25772afb', 'request_id': 'req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:33.252847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3306093318744a8980a3e9bb25772afb', 'request_id': 'req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:33.326520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3306093318744a8980a3e9bb25772afb', 'request_id': 'req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:33.337911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b admin admin] The token cache is being invalidate because project 3306093318744a8980a3e9bb25772afb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:33.338412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:33.339728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:33.341394 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4155/8339] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:33 2022] DELETE /identity/v3/projects/3306093318744a8980a3e9bb25772afb => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:33.345449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134715db-34f8-4b5b-bb4c-dab11c065a4c service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:33.355418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134715db-34f8-4b5b-bb4c-dab11c065a4c service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:33.411181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078693f-6bbe-49fe-90ae-bbcd2be1643b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:33.412039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078693f-6bbe-49fe-90ae-bbcd2be1643b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:33.412356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078693f-6bbe-49fe-90ae-bbcd2be1643b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:33.412631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078693f-6bbe-49fe-90ae-bbcd2be1643b admin admin] PATH_INFO: `/v3/users/52ce4ff4043641b2bd33ed52d0f51a45` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:33.413822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078693f-6bbe-49fe-90ae-bbcd2be1643b admin admin] RBAC: Authorizing `identity:delete_user(user_id=52ce4ff4043641b2bd33ed52d0f51a45)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:33.415779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078693f-6bbe-49fe-90ae-bbcd2be1643b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:33.438141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:33.608297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f078693f-6bbe-49fe-90ae-bbcd2be1643b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52ce4ff4043641b2bd33ed52d0f51a45'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:33.616405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f078693f-6bbe-49fe-90ae-bbcd2be1643b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52ce4ff4043641b2bd33ed52d0f51a45'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:33.617099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f078693f-6bbe-49fe-90ae-bbcd2be1643b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52ce4ff4043641b2bd33ed52d0f51a45'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:33.617724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f078693f-6bbe-49fe-90ae-bbcd2be1643b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52ce4ff4043641b2bd33ed52d0f51a45'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:33.692589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f078693f-6bbe-49fe-90ae-bbcd2be1643b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52ce4ff4043641b2bd33ed52d0f51a45'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:33.710766 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4185/8340] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:39:33 2022] DELETE /identity/v3/users/52ce4ff4043641b2bd33ed52d0f51a45 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:34.150768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:34.152430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:34.166435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34a49b-4b3d-4b7f-acba-3c47734ca59b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:34.225700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224246ff-4374-4687-bc27-dcf69cf70c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:34.226592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224246ff-4374-4687-bc27-dcf69cf70c1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:34.226876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224246ff-4374-4687-bc27-dcf69cf70c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:34.227151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224246ff-4374-4687-bc27-dcf69cf70c1d admin admin] PATH_INFO: `/v3/projects/3c2c36d457f24362b3ebfd68d26f58c6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:34.228363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224246ff-4374-4687-bc27-dcf69cf70c1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c2c36d457f24362b3ebfd68d26f58c6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:34.230442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224246ff-4374-4687-bc27-dcf69cf70c1d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:34.338601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-224246ff-4374-4687-bc27-dcf69cf70c1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c2c36d457f24362b3ebfd68d26f58c6', 'request_id': 'req-224246ff-4374-4687-bc27-dcf69cf70c1d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:34.344523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-224246ff-4374-4687-bc27-dcf69cf70c1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c2c36d457f24362b3ebfd68d26f58c6', 'request_id': 'req-224246ff-4374-4687-bc27-dcf69cf70c1d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:34.423758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-224246ff-4374-4687-bc27-dcf69cf70c1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c2c36d457f24362b3ebfd68d26f58c6', 'request_id': 'req-224246ff-4374-4687-bc27-dcf69cf70c1d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:34.437180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-224246ff-4374-4687-bc27-dcf69cf70c1d admin admin] The token cache is being invalidate because project 3c2c36d457f24362b3ebfd68d26f58c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:34.437474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-224246ff-4374-4687-bc27-dcf69cf70c1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:34.438167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-224246ff-4374-4687-bc27-dcf69cf70c1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:34.440416 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4156/8341] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:34 2022] DELETE /identity/v3/projects/3c2c36d457f24362b3ebfd68d26f58c6 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:34.446625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683b8b4-b3e2-48e2-bb38-4a480363c91b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:34.447126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683b8b4-b3e2-48e2-bb38-4a480363c91b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:34.447388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683b8b4-b3e2-48e2-bb38-4a480363c91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:34.466312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d683b8b4-b3e2-48e2-bb38-4a480363c91b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:34.467531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d683b8b4-b3e2-48e2-bb38-4a480363c91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:34.537882 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4186/8342] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:34.546082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f36503-a02d-4c9d-a116-88881cc738f3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:34.546377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f36503-a02d-4c9d-a116-88881cc738f3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:34.546657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f36503-a02d-4c9d-a116-88881cc738f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:34.565564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-11f36503-a02d-4c9d-a116-88881cc738f3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:34.569504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-11f36503-a02d-4c9d-a116-88881cc738f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:34.574851 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4157/8343] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:34.579295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683b8b4-b3e2-48e2-bb38-4a480363c91b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:34.589552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683b8b4-b3e2-48e2-bb38-4a480363c91b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:34.594309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de20b666-8909-4238-8545-7af72477de71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:34.594968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20b666-8909-4238-8545-7af72477de71 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:34.595256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20b666-8909-4238-8545-7af72477de71 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:34.595538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20b666-8909-4238-8545-7af72477de71 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:34.596277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de20b666-8909-4238-8545-7af72477de71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:39:34.596597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de20b666-8909-4238-8545-7af72477de71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:34.598233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de20b666-8909-4238-8545-7af72477de71 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:34.610392 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4187/8344] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:39:34.614780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f36503-a02d-4c9d-a116-88881cc738f3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:34.624783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f36503-a02d-4c9d-a116-88881cc738f3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:34.630408 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533a5b9-7c71-4093-997f-33bcde294ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:34.638589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533a5b9-7c71-4093-997f-33bcde294ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:34.638589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533a5b9-7c71-4093-997f-33bcde294ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:34.638589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533a5b9-7c71-4093-997f-33bcde294ef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:34.638589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533a5b9-7c71-4093-997f-33bcde294ef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:34.638589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533a5b9-7c71-4093-997f-33bcde294ef1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:34.768794 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4158/8345] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:39:34 2022] POST /identity/v3/projects => generated 385 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:34.773830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de20b666-8909-4238-8545-7af72477de71 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:34.784983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de20b666-8909-4238-8545-7af72477de71 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:34.844505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad89b47-d580-4fa2-8d99-7fefde507302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:34.845188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad89b47-d580-4fa2-8d99-7fefde507302 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:34.845472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad89b47-d580-4fa2-8d99-7fefde507302 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:34.846784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad89b47-d580-4fa2-8d99-7fefde507302 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:34.846784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad89b47-d580-4fa2-8d99-7fefde507302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:34.848558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad89b47-d580-4fa2-8d99-7fefde507302 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:34.977653 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4188/8346] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:39:34 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:34.981601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533a5b9-7c71-4093-997f-33bcde294ef1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:34.992203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533a5b9-7c71-4093-997f-33bcde294ef1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:34.996807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a772999-6485-45cc-a568-e882a3e2dc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:34.998742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a772999-6485-45cc-a568-e882a3e2dc62 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:34.998742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a772999-6485-45cc-a568-e882a3e2dc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:34.998742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a772999-6485-45cc-a568-e882a3e2dc62 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:34.998742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a772999-6485-45cc-a568-e882a3e2dc62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:35.000362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a772999-6485-45cc-a568-e882a3e2dc62 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:35.011335 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4159/8347] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:35.015209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad89b47-d580-4fa2-8d99-7fefde507302 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:35.026315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad89b47-d580-4fa2-8d99-7fefde507302 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:35.031054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e91c5-7ad2-474b-8845-aca9c8a258c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:35.031708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e91c5-7ad2-474b-8845-aca9c8a258c1 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:35.031992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e91c5-7ad2-474b-8845-aca9c8a258c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:35.032270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e91c5-7ad2-474b-8845-aca9c8a258c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:35.033027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e91c5-7ad2-474b-8845-aca9c8a258c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:35.034751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e91c5-7ad2-474b-8845-aca9c8a258c1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:35.045524 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4189/8348] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:35.050427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a772999-6485-45cc-a568-e882a3e2dc62 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:35.059716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a772999-6485-45cc-a568-e882a3e2dc62 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:35.064645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a4c7d-cf22-4bf3-8964-ed4a6049185d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:35.065221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a4c7d-cf22-4bf3-8964-ed4a6049185d admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:35.065509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a4c7d-cf22-4bf3-8964-ed4a6049185d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:35.065801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a4c7d-cf22-4bf3-8964-ed4a6049185d admin admin] PATH_INFO: `/v3/projects/083e6e6a88074e5cb51a6e63dfd486da/users/e17658837f1941939a14b07159b11dd6/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:35.082262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:35.085827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a4c7d-cf22-4bf3-8964-ed4a6049185d admin admin] RBAC: Authorizing `identity:create_grant(project_id=083e6e6a88074e5cb51a6e63dfd486da, user_id=e17658837f1941939a14b07159b11dd6, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:35.088558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a4c7d-cf22-4bf3-8964-ed4a6049185d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:35.160318 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4160/8349] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:35 2022] PUT /identity/v3/projects/083e6e6a88074e5cb51a6e63dfd486da/users/e17658837f1941939a14b07159b11dd6/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:35.171591 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44356482-9a61-49ca-b73c-f443c20b6124 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:35.171883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44356482-9a61-49ca-b73c-f443c20b6124 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:35.172161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44356482-9a61-49ca-b73c-f443c20b6124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:35.185237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:35.199834 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-44356482-9a61-49ca-b73c-f443c20b6124 None None] MFA Rules not processed for user `e17658837f1941939a14b07159b11dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:35.201042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-44356482-9a61-49ca-b73c-f443c20b6124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:35.265480 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4190/8350] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:35.269189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a4c7d-cf22-4bf3-8964-ed4a6049185d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:35.330861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a4c7d-cf22-4bf3-8964-ed4a6049185d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:35.336074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d18f0-fb07-42f4-b779-7d04e3455b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:35.336846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d18f0-fb07-42f4-b779-7d04e3455b84 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:35.337131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d18f0-fb07-42f4-b779-7d04e3455b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:35.337438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d18f0-fb07-42f4-b779-7d04e3455b84 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:35.338505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d18f0-fb07-42f4-b779-7d04e3455b84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:35.342242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d18f0-fb07-42f4-b779-7d04e3455b84 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:35.419225 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4161/8351] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:39:35 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:35.423468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44356482-9a61-49ca-b73c-f443c20b6124 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:35.434819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44356482-9a61-49ca-b73c-f443c20b6124 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:35.492183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1ccd9-7b8b-45a5-b24e-6cd00f79fb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:35.492866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1ccd9-7b8b-45a5-b24e-6cd00f79fb68 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:35.493175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1ccd9-7b8b-45a5-b24e-6cd00f79fb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:35.493459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f1ccd9-7b8b-45a5-b24e-6cd00f79fb68 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:35.494201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1ccd9-7b8b-45a5-b24e-6cd00f79fb68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:35.496137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f1ccd9-7b8b-45a5-b24e-6cd00f79fb68 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:35.565650 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4191/8352] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:39:35 2022] POST /identity/v3/users => generated 299 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:35.570326 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d18f0-fb07-42f4-b779-7d04e3455b84 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:35.580554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d18f0-fb07-42f4-b779-7d04e3455b84 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:35.585489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29335a9-b801-491a-9725-16516f2d68b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:35.586252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29335a9-b801-491a-9725-16516f2d68b1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:35.586541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29335a9-b801-491a-9725-16516f2d68b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:35.586825 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29335a9-b801-491a-9725-16516f2d68b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:35.587557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29335a9-b801-491a-9725-16516f2d68b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:35.589176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29335a9-b801-491a-9725-16516f2d68b1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:35.600459 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4162/8353] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:35.604707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1ccd9-7b8b-45a5-b24e-6cd00f79fb68 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:35.615551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f1ccd9-7b8b-45a5-b24e-6cd00f79fb68 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:35.620657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe3dae-13fe-4683-af92-e697b2b1cbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:35.621555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe3dae-13fe-4683-af92-e697b2b1cbae admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:35.621860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe3dae-13fe-4683-af92-e697b2b1cbae admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:35.622181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe3dae-13fe-4683-af92-e697b2b1cbae admin admin] PATH_INFO: `/v3/projects/b0411dca9a084774851dfbbd22c0a3b8/users/51f8519cd4dc44c9aa724bb5d9be190a/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:35.639860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe3dae-13fe-4683-af92-e697b2b1cbae admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0411dca9a084774851dfbbd22c0a3b8, user_id=51f8519cd4dc44c9aa724bb5d9be190a, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:35.642637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fe3dae-13fe-4683-af92-e697b2b1cbae admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:35.716436 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4192/8354] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:35 2022] PUT /identity/v3/projects/b0411dca9a084774851dfbbd22c0a3b8/users/51f8519cd4dc44c9aa724bb5d9be190a/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:35.721319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29335a9-b801-491a-9725-16516f2d68b1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:35.732965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29335a9-b801-491a-9725-16516f2d68b1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:35.793978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621d0fb-215f-4270-ac84-7e7033b00728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:35.794726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621d0fb-215f-4270-ac84-7e7033b00728 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:35.795167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621d0fb-215f-4270-ac84-7e7033b00728 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:35.795452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621d0fb-215f-4270-ac84-7e7033b00728 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:35.796179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621d0fb-215f-4270-ac84-7e7033b00728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:35.797779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621d0fb-215f-4270-ac84-7e7033b00728 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:35.813958 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4163/8355] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:39:35.815125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe3dae-13fe-4683-af92-e697b2b1cbae admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:35.826165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe3dae-13fe-4683-af92-e697b2b1cbae admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:35.830841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32789fa-0188-4bb9-a6b5-7868291f5d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:35.831747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32789fa-0188-4bb9-a6b5-7868291f5d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:35.832024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32789fa-0188-4bb9-a6b5-7868291f5d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:35.832461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32789fa-0188-4bb9-a6b5-7868291f5d02 admin admin] PATH_INFO: `/v3/projects/b0411dca9a084774851dfbbd22c0a3b8/users/51f8519cd4dc44c9aa724bb5d9be190a/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:35.834846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32789fa-0188-4bb9-a6b5-7868291f5d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0411dca9a084774851dfbbd22c0a3b8, user_id=51f8519cd4dc44c9aa724bb5d9be190a, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:35.837678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32789fa-0188-4bb9-a6b5-7868291f5d02 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:35.849691 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4193/8356] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:35 2022] PUT /identity/v3/projects/b0411dca9a084774851dfbbd22c0a3b8/users/51f8519cd4dc44c9aa724bb5d9be190a/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:35.861477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a032c-573a-4d67-b6b9-9a181bbb984b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:35.861868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a032c-573a-4d67-b6b9-9a181bbb984b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:35.862211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a032c-573a-4d67-b6b9-9a181bbb984b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:35.890779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-546a032c-573a-4d67-b6b9-9a181bbb984b None None] MFA Rules not processed for user `51f8519cd4dc44c9aa724bb5d9be190a`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:35.892113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-546a032c-573a-4d67-b6b9-9a181bbb984b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:35.941682 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:35.962275 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4164/8357] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:35.977769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32789fa-0188-4bb9-a6b5-7868291f5d02 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:35.979512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a32789fa-0188-4bb9-a6b5-7868291f5d02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:36.033464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:36.043030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32789fa-0188-4bb9-a6b5-7868291f5d02 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:36.044679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa50c6c2-426f-4efb-a40d-b617473fabe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:36.045307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50c6c2-426f-4efb-a40d-b617473fabe8 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:36.045594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50c6c2-426f-4efb-a40d-b617473fabe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:36.045907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50c6c2-426f-4efb-a40d-b617473fabe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:36.057274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50c6c2-426f-4efb-a40d-b617473fabe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:36.058816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa50c6c2-426f-4efb-a40d-b617473fabe8 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:36.069419 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4194/8358] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:38.485965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a032c-573a-4d67-b6b9-9a181bbb984b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:38.549306 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a032c-573a-4d67-b6b9-9a181bbb984b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:38.554806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcc3f4-b2f4-4f63-aec7-ec40c42b11f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:38.555713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcc3f4-b2f4-4f63-aec7-ec40c42b11f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:38.555996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcc3f4-b2f4-4f63-aec7-ec40c42b11f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:38.556273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcc3f4-b2f4-4f63-aec7-ec40c42b11f2 admin admin] PATH_INFO: `/v3/users/e17658837f1941939a14b07159b11dd6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:38.557514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbcc3f4-b2f4-4f63-aec7-ec40c42b11f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e17658837f1941939a14b07159b11dd6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:38.559508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbcc3f4-b2f4-4f63-aec7-ec40c42b11f2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:38.832251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2dbcc3f4-b2f4-4f63-aec7-ec40c42b11f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e17658837f1941939a14b07159b11dd6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:38.832910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2dbcc3f4-b2f4-4f63-aec7-ec40c42b11f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e17658837f1941939a14b07159b11dd6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:38.840322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2dbcc3f4-b2f4-4f63-aec7-ec40c42b11f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e17658837f1941939a14b07159b11dd6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:38.942141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2dbcc3f4-b2f4-4f63-aec7-ec40c42b11f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e17658837f1941939a14b07159b11dd6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:38.942784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2dbcc3f4-b2f4-4f63-aec7-ec40c42b11f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e17658837f1941939a14b07159b11dd6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:38.961163 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4165/8359] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:39:38 2022] DELETE /identity/v3/users/e17658837f1941939a14b07159b11dd6 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:38.966024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8bab1-2d8e-4d2c-8063-97b57a9d5b97 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:38.966517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8bab1-2d8e-4d2c-8063-97b57a9d5b97 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:38.966622 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8bab1-2d8e-4d2c-8063-97b57a9d5b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:38.985211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8cc8bab1-2d8e-4d2c-8063-97b57a9d5b97 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:38.986558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8cc8bab1-2d8e-4d2c-8063-97b57a9d5b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:39.045053 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4195/8360] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:39:39.059773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcc3f4-b2f4-4f63-aec7-ec40c42b11f2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:39.059773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2dbcc3f4-b2f4-4f63-aec7-ec40c42b11f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:39.121493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcc3f4-b2f4-4f63-aec7-ec40c42b11f2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:39.125281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbadc455-5cb0-40e4-a795-83764007fc1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:39.125971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadc455-5cb0-40e4-a795-83764007fc1c service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:39.126263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadc455-5cb0-40e4-a795-83764007fc1c service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:39.126545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbadc455-5cb0-40e4-a795-83764007fc1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:39.136206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbadc455-5cb0-40e4-a795-83764007fc1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:39.137683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbadc455-5cb0-40e4-a795-83764007fc1c service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:39.147904 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4166/8361] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:39.578400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8bab1-2d8e-4d2c-8063-97b57a9d5b97 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:39.580169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8cc8bab1-2d8e-4d2c-8063-97b57a9d5b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:39.595789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8bab1-2d8e-4d2c-8063-97b57a9d5b97 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:39.600943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4f0b0-b259-4711-a62e-a9e751562ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:39.601817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4f0b0-b259-4711-a62e-a9e751562ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:39.602132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4f0b0-b259-4711-a62e-a9e751562ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:39.602474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4f0b0-b259-4711-a62e-a9e751562ecd admin admin] PATH_INFO: `/v3/projects/083e6e6a88074e5cb51a6e63dfd486da` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:39.603745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4f0b0-b259-4711-a62e-a9e751562ecd admin admin] RBAC: Authorizing `identity:delete_project(project_id=083e6e6a88074e5cb51a6e63dfd486da)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:39.605552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4f0b0-b259-4711-a62e-a9e751562ecd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:39.748878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0fe4f0b0-b259-4711-a62e-a9e751562ecd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '083e6e6a88074e5cb51a6e63dfd486da', 'request_id': 'req-0fe4f0b0-b259-4711-a62e-a9e751562ecd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:39.754400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0fe4f0b0-b259-4711-a62e-a9e751562ecd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '083e6e6a88074e5cb51a6e63dfd486da', 'request_id': 'req-0fe4f0b0-b259-4711-a62e-a9e751562ecd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:39.823343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0fe4f0b0-b259-4711-a62e-a9e751562ecd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '083e6e6a88074e5cb51a6e63dfd486da', 'request_id': 'req-0fe4f0b0-b259-4711-a62e-a9e751562ecd'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:39.834719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0fe4f0b0-b259-4711-a62e-a9e751562ecd admin admin] The token cache is being invalidate because project 083e6e6a88074e5cb51a6e63dfd486da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:39.835007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0fe4f0b0-b259-4711-a62e-a9e751562ecd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:39.835664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0fe4f0b0-b259-4711-a62e-a9e751562ecd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:39.837313 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4196/8362] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:39 2022] DELETE /identity/v3/projects/083e6e6a88074e5cb51a6e63dfd486da => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:39.841513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbadc455-5cb0-40e4-a795-83764007fc1c service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:39.851503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbadc455-5cb0-40e4-a795-83764007fc1c service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:39.856215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:39.909703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed36f8c-485f-4ee8-914b-497836c8748e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:39.910616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed36f8c-485f-4ee8-914b-497836c8748e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:39.910896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed36f8c-485f-4ee8-914b-497836c8748e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:39.911166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed36f8c-485f-4ee8-914b-497836c8748e admin admin] PATH_INFO: `/v3/users/51f8519cd4dc44c9aa724bb5d9be190a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:39.912356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed36f8c-485f-4ee8-914b-497836c8748e admin admin] RBAC: Authorizing `identity:delete_user(user_id=51f8519cd4dc44c9aa724bb5d9be190a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:39.914499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed36f8c-485f-4ee8-914b-497836c8748e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:40.129368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2ed36f8c-485f-4ee8-914b-497836c8748e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51f8519cd4dc44c9aa724bb5d9be190a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:40.130057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2ed36f8c-485f-4ee8-914b-497836c8748e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51f8519cd4dc44c9aa724bb5d9be190a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:40.138288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2ed36f8c-485f-4ee8-914b-497836c8748e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51f8519cd4dc44c9aa724bb5d9be190a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:40.177618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2ed36f8c-485f-4ee8-914b-497836c8748e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51f8519cd4dc44c9aa724bb5d9be190a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:40.178272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2ed36f8c-485f-4ee8-914b-497836c8748e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51f8519cd4dc44c9aa724bb5d9be190a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:40.196908 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4167/8363] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:39:39 2022] DELETE /identity/v3/users/51f8519cd4dc44c9aa724bb5d9be190a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:40.448319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4f0b0-b259-4711-a62e-a9e751562ecd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:40.449759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0fe4f0b0-b259-4711-a62e-a9e751562ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:40.463147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4f0b0-b259-4711-a62e-a9e751562ecd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:40.524200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4716ef6b-7194-4f65-98fb-81bf8d6739cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:40.525110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716ef6b-7194-4f65-98fb-81bf8d6739cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:40.525395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716ef6b-7194-4f65-98fb-81bf8d6739cb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:40.525676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4716ef6b-7194-4f65-98fb-81bf8d6739cb admin admin] PATH_INFO: `/v3/projects/b0411dca9a084774851dfbbd22c0a3b8` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:40.526888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716ef6b-7194-4f65-98fb-81bf8d6739cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0411dca9a084774851dfbbd22c0a3b8)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:40.528798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4716ef6b-7194-4f65-98fb-81bf8d6739cb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:40.545588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:40.653002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4716ef6b-7194-4f65-98fb-81bf8d6739cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0411dca9a084774851dfbbd22c0a3b8', 'request_id': 'req-4716ef6b-7194-4f65-98fb-81bf8d6739cb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:40.658351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4716ef6b-7194-4f65-98fb-81bf8d6739cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0411dca9a084774851dfbbd22c0a3b8', 'request_id': 'req-4716ef6b-7194-4f65-98fb-81bf8d6739cb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:40.732137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4716ef6b-7194-4f65-98fb-81bf8d6739cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0411dca9a084774851dfbbd22c0a3b8', 'request_id': 'req-4716ef6b-7194-4f65-98fb-81bf8d6739cb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:40.743650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4716ef6b-7194-4f65-98fb-81bf8d6739cb admin admin] The token cache is being invalidate because project b0411dca9a084774851dfbbd22c0a3b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:39:40.743945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4716ef6b-7194-4f65-98fb-81bf8d6739cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:40.744588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4716ef6b-7194-4f65-98fb-81bf8d6739cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:39:40.746230 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4197/8364] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:39:40 2022] DELETE /identity/v3/projects/b0411dca9a084774851dfbbd22c0a3b8 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:40.751191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9197455-d228-48ae-afff-fee06ddb0ed5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:40.751491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9197455-d228-48ae-afff-fee06ddb0ed5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:40.751778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9197455-d228-48ae-afff-fee06ddb0ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:40.770103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f9197455-d228-48ae-afff-fee06ddb0ed5 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:40.771300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f9197455-d228-48ae-afff-fee06ddb0ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:40.831751 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4168/8365] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:40.840754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451add32-04c3-40bc-b5e1-129ed7f8d7b6 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:40.841042 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451add32-04c3-40bc-b5e1-129ed7f8d7b6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:40.841323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451add32-04c3-40bc-b5e1-129ed7f8d7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:40.860594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-451add32-04c3-40bc-b5e1-129ed7f8d7b6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:40.861824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-451add32-04c3-40bc-b5e1-129ed7f8d7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:40.870114 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4198/8366] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:40.874404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9197455-d228-48ae-afff-fee06ddb0ed5 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:40.884683 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9197455-d228-48ae-afff-fee06ddb0ed5 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:40.889577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d1ef7-8e61-4f7f-a84f-c9ad42a9c4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:40.890264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d1ef7-8e61-4f7f-a84f-c9ad42a9c4dc admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:40.890565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d1ef7-8e61-4f7f-a84f-c9ad42a9c4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:40.890927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d1ef7-8e61-4f7f-a84f-c9ad42a9c4dc admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:40.891648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d1ef7-8e61-4f7f-a84f-c9ad42a9c4dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:39:40.891958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d1ef7-8e61-4f7f-a84f-c9ad42a9c4dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:40.893555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d1ef7-8e61-4f7f-a84f-c9ad42a9c4dc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:40.906121 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4169/8367] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:39:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:39:40.910588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451add32-04c3-40bc-b5e1-129ed7f8d7b6 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:40.920581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451add32-04c3-40bc-b5e1-129ed7f8d7b6 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:40.925235 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245de860-3b39-4edb-8306-90891956ee1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:40.925914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245de860-3b39-4edb-8306-90891956ee1a admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:40.926198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245de860-3b39-4edb-8306-90891956ee1a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:40.926474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245de860-3b39-4edb-8306-90891956ee1a admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:40.927156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245de860-3b39-4edb-8306-90891956ee1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:40.929081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245de860-3b39-4edb-8306-90891956ee1a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:41.054073 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4199/8368] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:39:40 2022] POST /identity/v3/projects => generated 373 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:39:41.058207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d1ef7-8e61-4f7f-a84f-c9ad42a9c4dc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:41.069058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d1ef7-8e61-4f7f-a84f-c9ad42a9c4dc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:41.082418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:41.126933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e638e-ae8a-46d8-aec0-78d1b77e143e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:41.127617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e638e-ae8a-46d8-aec0-78d1b77e143e admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:41.127903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e638e-ae8a-46d8-aec0-78d1b77e143e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:41.128183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153e638e-ae8a-46d8-aec0-78d1b77e143e admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:41.128868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e638e-ae8a-46d8-aec0-78d1b77e143e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:41.130866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153e638e-ae8a-46d8-aec0-78d1b77e143e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:41.229882 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4170/8369] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:39:41 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:39:41.233722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245de860-3b39-4edb-8306-90891956ee1a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:41.244446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245de860-3b39-4edb-8306-90891956ee1a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:41.249120 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816eb5e-5e39-4e21-a219-373a509f776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:41.249778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816eb5e-5e39-4e21-a219-373a509f776b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:41.250130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816eb5e-5e39-4e21-a219-373a509f776b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:41.250450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816eb5e-5e39-4e21-a219-373a509f776b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:41.251187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816eb5e-5e39-4e21-a219-373a509f776b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:41.252795 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816eb5e-5e39-4e21-a219-373a509f776b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:41.263705 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4200/8370] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:41.267634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e638e-ae8a-46d8-aec0-78d1b77e143e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:41.278659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153e638e-ae8a-46d8-aec0-78d1b77e143e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:41.283319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e695d6-b79f-41ab-a529-a73df416bcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:41.284006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e695d6-b79f-41ab-a529-a73df416bcb7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:41.284295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e695d6-b79f-41ab-a529-a73df416bcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:41.284577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e695d6-b79f-41ab-a529-a73df416bcb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:41.285305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e695d6-b79f-41ab-a529-a73df416bcb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:41.286950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e695d6-b79f-41ab-a529-a73df416bcb7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:41.297172 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4171/8371] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:39:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:39:41.301202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816eb5e-5e39-4e21-a219-373a509f776b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:41.312298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816eb5e-5e39-4e21-a219-373a509f776b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:41.317620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef9f9d-918a-4faf-90e6-c2c612e3b21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:41.319035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef9f9d-918a-4faf-90e6-c2c612e3b21e admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:41.319035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef9f9d-918a-4faf-90e6-c2c612e3b21e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:41.319257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ef9f9d-918a-4faf-90e6-c2c612e3b21e admin admin] PATH_INFO: `/v3/projects/42b636fdc4204b3282fde6f9422a5afd/users/02eea1dae1314a53ba4958ea84b3b287/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:41.335325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef9f9d-918a-4faf-90e6-c2c612e3b21e admin admin] RBAC: Authorizing `identity:create_grant(project_id=42b636fdc4204b3282fde6f9422a5afd, user_id=02eea1dae1314a53ba4958ea84b3b287, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:41.338080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ef9f9d-918a-4faf-90e6-c2c612e3b21e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:41.429709 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4201/8372] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:39:41 2022] PUT /identity/v3/projects/42b636fdc4204b3282fde6f9422a5afd/users/02eea1dae1314a53ba4958ea84b3b287/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:39:41.440758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc3219-2f04-4e02-863a-b2f827e9b829 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:41.441050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc3219-2f04-4e02-863a-b2f827e9b829 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:41.441330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc3219-2f04-4e02-863a-b2f827e9b829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:41.468546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d1bc3219-2f04-4e02-863a-b2f827e9b829 None None] MFA Rules not processed for user `02eea1dae1314a53ba4958ea84b3b287`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:41.469763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d1bc3219-2f04-4e02-863a-b2f827e9b829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:41.534994 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4172/8373] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:41.544945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef9f9d-918a-4faf-90e6-c2c612e3b21e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:41.546737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c7ef9f9d-918a-4faf-90e6-c2c612e3b21e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:41.606736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ef9f9d-918a-4faf-90e6-c2c612e3b21e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:41.610433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712a45e-0cd8-4c9b-94a5-374ef5b00007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:41.611077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712a45e-0cd8-4c9b-94a5-374ef5b00007 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:41.611362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712a45e-0cd8-4c9b-94a5-374ef5b00007 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:41.611642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712a45e-0cd8-4c9b-94a5-374ef5b00007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:41.620576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8712a45e-0cd8-4c9b-94a5-374ef5b00007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:41.622038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8712a45e-0cd8-4c9b-94a5-374ef5b00007 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:41.630934 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4202/8374] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:51.334898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c730d-7c89-47ea-9f11-165c7262b24c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:51.339040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c730d-7c89-47ea-9f11-165c7262b24c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:51.339040 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c730d-7c89-47ea-9f11-165c7262b24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:51.354844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-514c730d-7c89-47ea-9f11-165c7262b24c None None] MFA Rules not processed for user `388458a7b3504bd08f3789f53cf9ea35`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:39:51.356250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-514c730d-7c89-47ea-9f11-165c7262b24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:51.413646 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4173/8375] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:39:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:39:51.626797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:51.816145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712a45e-0cd8-4c9b-94a5-374ef5b00007 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:51.837178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8712a45e-0cd8-4c9b-94a5-374ef5b00007 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:51.841757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4423340b-91ff-4fc9-8d07-4737307e07ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:51.845963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423340b-91ff-4fc9-8d07-4737307e07ed service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:51.845963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423340b-91ff-4fc9-8d07-4737307e07ed service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:51.845963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423340b-91ff-4fc9-8d07-4737307e07ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:51.845963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4423340b-91ff-4fc9-8d07-4737307e07ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:51.858434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4423340b-91ff-4fc9-8d07-4737307e07ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:51.859892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4423340b-91ff-4fc9-8d07-4737307e07ed service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:51.871126 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4203/8376] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:57.033792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c730d-7c89-47ea-9f11-165c7262b24c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:57.037482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-514c730d-7c89-47ea-9f11-165c7262b24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:57.082702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c730d-7c89-47ea-9f11-165c7262b24c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:57.088614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127dc35-e54f-4654-904d-99ba46368bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:57.089628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127dc35-e54f-4654-904d-99ba46368bae service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:57.090053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127dc35-e54f-4654-904d-99ba46368bae service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:57.091838 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127dc35-e54f-4654-904d-99ba46368bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:57.096468 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3127dc35-e54f-4654-904d-99ba46368bae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:57.112571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127dc35-e54f-4654-904d-99ba46368bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:57.116063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127dc35-e54f-4654-904d-99ba46368bae service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:57.128095 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4174/8377] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:39:58.709157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4423340b-91ff-4fc9-8d07-4737307e07ed service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:39:58.721286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4423340b-91ff-4fc9-8d07-4737307e07ed service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:39:58.724999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697ac43-e331-42ce-9ded-7c8df17c16fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:39:58.725655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697ac43-e331-42ce-9ded-7c8df17c16fa service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:39:58.725989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697ac43-e331-42ce-9ded-7c8df17c16fa service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:39:58.726272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697ac43-e331-42ce-9ded-7c8df17c16fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:39:58.728053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5697ac43-e331-42ce-9ded-7c8df17c16fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:39:58.802946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697ac43-e331-42ce-9ded-7c8df17c16fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:39:58.804403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697ac43-e331-42ce-9ded-7c8df17c16fa service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:39:58.824599 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4204/8378] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:39:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:40:00.187580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127dc35-e54f-4654-904d-99ba46368bae service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:00.189652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3127dc35-e54f-4654-904d-99ba46368bae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:00.205184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127dc35-e54f-4654-904d-99ba46368bae service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:00.211744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ad8de-424d-4a60-ad85-70ab98577c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:00.212637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ad8de-424d-4a60-ad85-70ab98577c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:00.213425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ad8de-424d-4a60-ad85-70ab98577c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:00.213706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ad8de-424d-4a60-ad85-70ab98577c2d admin admin] PATH_INFO: `/v3/users/8d033e99f9294e348b01c59a970931d5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:00.215290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ad8de-424d-4a60-ad85-70ab98577c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d033e99f9294e348b01c59a970931d5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:00.217349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ad8de-424d-4a60-ad85-70ab98577c2d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:00.229822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:00.412199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b73ad8de-424d-4a60-ad85-70ab98577c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d033e99f9294e348b01c59a970931d5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:00.412791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b73ad8de-424d-4a60-ad85-70ab98577c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d033e99f9294e348b01c59a970931d5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:00.420465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b73ad8de-424d-4a60-ad85-70ab98577c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d033e99f9294e348b01c59a970931d5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:00.513889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b73ad8de-424d-4a60-ad85-70ab98577c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d033e99f9294e348b01c59a970931d5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:00.514525 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b73ad8de-424d-4a60-ad85-70ab98577c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d033e99f9294e348b01c59a970931d5'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:00.532879 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4175/8379] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:40:00 2022] DELETE /identity/v3/users/8d033e99f9294e348b01c59a970931d5 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:40:00.819461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697ac43-e331-42ce-9ded-7c8df17c16fa service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:00.839515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697ac43-e331-42ce-9ded-7c8df17c16fa service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:00.906491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19d64c-e692-4ac4-907d-96aac772f5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:00.907392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19d64c-e692-4ac4-907d-96aac772f5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:00.907675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19d64c-e692-4ac4-907d-96aac772f5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:00.907952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19d64c-e692-4ac4-907d-96aac772f5e1 admin admin] PATH_INFO: `/v3/projects/6ed82f578e214948a9311cbdbb55a027` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:00.909137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19d64c-e692-4ac4-907d-96aac772f5e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ed82f578e214948a9311cbdbb55a027)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:00.911086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19d64c-e692-4ac4-907d-96aac772f5e1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:01.090473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-df19d64c-e692-4ac4-907d-96aac772f5e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ed82f578e214948a9311cbdbb55a027', 'request_id': 'req-df19d64c-e692-4ac4-907d-96aac772f5e1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:01.097019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-df19d64c-e692-4ac4-907d-96aac772f5e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ed82f578e214948a9311cbdbb55a027', 'request_id': 'req-df19d64c-e692-4ac4-907d-96aac772f5e1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:01.183744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-df19d64c-e692-4ac4-907d-96aac772f5e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ed82f578e214948a9311cbdbb55a027', 'request_id': 'req-df19d64c-e692-4ac4-907d-96aac772f5e1'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:01.185185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:01.196915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-df19d64c-e692-4ac4-907d-96aac772f5e1 admin admin] The token cache is being invalidate because project 6ed82f578e214948a9311cbdbb55a027 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:40:01.197713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-df19d64c-e692-4ac4-907d-96aac772f5e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:01.198427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-df19d64c-e692-4ac4-907d-96aac772f5e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:01.200157 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4205/8380] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:40:00 2022] DELETE /identity/v3/projects/6ed82f578e214948a9311cbdbb55a027 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:40:01.205469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3c782-56dd-416c-bc7d-19072d730adb None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:01.205971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3c782-56dd-416c-bc7d-19072d730adb None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:01.206154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3c782-56dd-416c-bc7d-19072d730adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:01.225672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-02f3c782-56dd-416c-bc7d-19072d730adb None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:01.227112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-02f3c782-56dd-416c-bc7d-19072d730adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:01.295116 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4176/8381] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:40:01.302661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb7696-9368-4418-aee1-e7eea5543324 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:01.302950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb7696-9368-4418-aee1-e7eea5543324 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:01.303226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb7696-9368-4418-aee1-e7eea5543324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:01.323962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0fdb7696-9368-4418-aee1-e7eea5543324 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:01.325256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0fdb7696-9368-4418-aee1-e7eea5543324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:01.333412 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4206/8382] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:40:01.341019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3c782-56dd-416c-bc7d-19072d730adb None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:01.350983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f3c782-56dd-416c-bc7d-19072d730adb None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:01.360484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8d0da-9cb4-4882-a831-090a026c88c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:01.361172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8d0da-9cb4-4882-a831-090a026c88c6 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:01.361464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8d0da-9cb4-4882-a831-090a026c88c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:01.366215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8d0da-9cb4-4882-a831-090a026c88c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:01.366215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8d0da-9cb4-4882-a831-090a026c88c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:40:01.366215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8d0da-9cb4-4882-a831-090a026c88c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:01.366215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8d0da-9cb4-4882-a831-090a026c88c6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:01.380670 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4177/8383] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:40:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:40:01.382180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb7696-9368-4418-aee1-e7eea5543324 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:01.392693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb7696-9368-4418-aee1-e7eea5543324 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:01.397558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706aa3f-7aae-41b7-a918-308134ee175a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:01.398374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706aa3f-7aae-41b7-a918-308134ee175a admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:01.398737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706aa3f-7aae-41b7-a918-308134ee175a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:01.399094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d706aa3f-7aae-41b7-a918-308134ee175a admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:01.399881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706aa3f-7aae-41b7-a918-308134ee175a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:01.401956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d706aa3f-7aae-41b7-a918-308134ee175a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:01.500610 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4207/8384] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:40:01 2022] POST /identity/v3/projects => generated 391 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:40:01.504687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8d0da-9cb4-4882-a831-090a026c88c6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:01.515029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8d0da-9cb4-4882-a831-090a026c88c6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:01.573024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df2055-064d-493c-836a-34e1838ae3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:01.573718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df2055-064d-493c-836a-34e1838ae3c1 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:01.574093 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df2055-064d-493c-836a-34e1838ae3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:01.574302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df2055-064d-493c-836a-34e1838ae3c1 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:01.574987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df2055-064d-493c-836a-34e1838ae3c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:01.576916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df2055-064d-493c-836a-34e1838ae3c1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:01.662815 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4178/8385] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:40:01 2022] POST /identity/v3/users => generated 296 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:40:01.667017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706aa3f-7aae-41b7-a918-308134ee175a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:01.677629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d706aa3f-7aae-41b7-a918-308134ee175a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:01.682841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a877e0b-c5f7-4b54-b2d4-bde5e0712b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:01.683585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a877e0b-c5f7-4b54-b2d4-bde5e0712b2a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:01.683878 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a877e0b-c5f7-4b54-b2d4-bde5e0712b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:01.690331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a877e0b-c5f7-4b54-b2d4-bde5e0712b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:01.691205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a877e0b-c5f7-4b54-b2d4-bde5e0712b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:01.692857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a877e0b-c5f7-4b54-b2d4-bde5e0712b2a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:01.704634 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4208/8386] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:40:01.708815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df2055-064d-493c-836a-34e1838ae3c1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:01.718310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:01.722412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df2055-064d-493c-836a-34e1838ae3c1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:01.727398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3964a-cbfb-4d66-b95c-3b8bd958f762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:01.728077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3964a-cbfb-4d66-b95c-3b8bd958f762 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:01.728539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3964a-cbfb-4d66-b95c-3b8bd958f762 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:01.728875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3964a-cbfb-4d66-b95c-3b8bd958f762 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:01.730039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3964a-cbfb-4d66-b95c-3b8bd958f762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:01.731668 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3964a-cbfb-4d66-b95c-3b8bd958f762 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:01.742706 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4179/8387] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:40:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:40:01.747972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a877e0b-c5f7-4b54-b2d4-bde5e0712b2a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:01.760352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a877e0b-c5f7-4b54-b2d4-bde5e0712b2a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:01.765511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b389520-cffe-443e-a2fc-ecfb0927a543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:01.766440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b389520-cffe-443e-a2fc-ecfb0927a543 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:01.766729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b389520-cffe-443e-a2fc-ecfb0927a543 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:01.767028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b389520-cffe-443e-a2fc-ecfb0927a543 admin admin] PATH_INFO: `/v3/projects/8376a726a998426195c974ba76d614e8/users/181b118771d64f09a894612ebcca32ac/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:01.783921 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b389520-cffe-443e-a2fc-ecfb0927a543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8376a726a998426195c974ba76d614e8, user_id=181b118771d64f09a894612ebcca32ac, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:01.787205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b389520-cffe-443e-a2fc-ecfb0927a543 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:01.881545 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4209/8388] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:40:01 2022] PUT /identity/v3/projects/8376a726a998426195c974ba76d614e8/users/181b118771d64f09a894612ebcca32ac/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:40:01.892256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e5455-f216-4b02-b275-99f6c1ee7495 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:01.892601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e5455-f216-4b02-b275-99f6c1ee7495 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:01.892890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e5455-f216-4b02-b275-99f6c1ee7495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:01.921616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-365e5455-f216-4b02-b275-99f6c1ee7495 None None] MFA Rules not processed for user `181b118771d64f09a894612ebcca32ac`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:01.922956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-365e5455-f216-4b02-b275-99f6c1ee7495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:01.992030 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4180/8389] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:40:02.003294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b389520-cffe-443e-a2fc-ecfb0927a543 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:02.005094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7b389520-cffe-443e-a2fc-ecfb0927a543 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:02.066171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b389520-cffe-443e-a2fc-ecfb0927a543 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:02.071672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3e62d-9746-43c2-aa51-86bbd77b39e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:02.071672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3e62d-9746-43c2-aa51-86bbd77b39e8 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:02.071672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3e62d-9746-43c2-aa51-86bbd77b39e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:02.072013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3e62d-9746-43c2-aa51-86bbd77b39e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:02.081599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3e62d-9746-43c2-aa51-86bbd77b39e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:02.083100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3e62d-9746-43c2-aa51-86bbd77b39e8 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:02.093608 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4210/8390] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:40:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:40:08.530466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20b3aa-9422-410b-893c-cee3831be2ca None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:08.530978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20b3aa-9422-410b-893c-cee3831be2ca None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:08.531107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad20b3aa-9422-410b-893c-cee3831be2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:08.584266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ad20b3aa-9422-410b-893c-cee3831be2ca None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:08.585419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ad20b3aa-9422-410b-893c-cee3831be2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:08.661629 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4181/8391] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:40:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:40:08.804391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3e62d-9746-43c2-aa51-86bbd77b39e8 service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:08.805900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7bb3e62d-9746-43c2-aa51-86bbd77b39e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:08.921943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3e62d-9746-43c2-aa51-86bbd77b39e8 service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:08.929664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a2b29-dfed-4319-89d9-ba7628194d82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:08.929664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a2b29-dfed-4319-89d9-ba7628194d82 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:08.929664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a2b29-dfed-4319-89d9-ba7628194d82 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:08.929664 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6a2b29-dfed-4319-89d9-ba7628194d82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:08.939500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a2b29-dfed-4319-89d9-ba7628194d82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:08.941033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6a2b29-dfed-4319-89d9-ba7628194d82 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:08.950011 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4211/8392] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:40:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:40:11.445795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434c242-823f-4d6d-b8ab-b77fc5a56044 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:11.446360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434c242-823f-4d6d-b8ab-b77fc5a56044 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:11.446460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434c242-823f-4d6d-b8ab-b77fc5a56044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:11.465337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b434c242-823f-4d6d-b8ab-b77fc5a56044 None None] MFA Rules not processed for user `388458a7b3504bd08f3789f53cf9ea35`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:11.466583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b434c242-823f-4d6d-b8ab-b77fc5a56044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:11.517705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:11.526209 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4182/8393] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:40:17.632547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a2b29-dfed-4319-89d9-ba7628194d82 service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:17.634232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0c6a2b29-dfed-4319-89d9-ba7628194d82 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:17.643728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:17.746783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6a2b29-dfed-4319-89d9-ba7628194d82 service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:17.749384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a3545-4654-4a52-8eae-dc57af5b7ded service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:17.750321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a3545-4654-4a52-8eae-dc57af5b7ded service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:17.750607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a3545-4654-4a52-8eae-dc57af5b7ded service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:17.751077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a3545-4654-4a52-8eae-dc57af5b7ded service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:17.753284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9a1a3545-4654-4a52-8eae-dc57af5b7ded service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:17.770996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a3545-4654-4a52-8eae-dc57af5b7ded service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:17.775005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1a3545-4654-4a52-8eae-dc57af5b7ded service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:17.790900 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4212/8394] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:40:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:40:22.663215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434c242-823f-4d6d-b8ab-b77fc5a56044 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:22.682325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434c242-823f-4d6d-b8ab-b77fc5a56044 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:22.689170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b14eac-abec-40c8-9c80-77e0b044b3f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:22.689987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b14eac-abec-40c8-9c80-77e0b044b3f5 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:22.690342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b14eac-abec-40c8-9c80-77e0b044b3f5 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:22.691068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b14eac-abec-40c8-9c80-77e0b044b3f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:22.693063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b6b14eac-abec-40c8-9c80-77e0b044b3f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:22.788783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b14eac-abec-40c8-9c80-77e0b044b3f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:22.790497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b14eac-abec-40c8-9c80-77e0b044b3f5 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:22.800451 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4183/8395] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:40:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:40:26.361674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a3545-4654-4a52-8eae-dc57af5b7ded service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:26.458982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a3545-4654-4a52-8eae-dc57af5b7ded service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:26.465173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215b969-c89d-44ed-8bff-36592c8cf63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:26.466254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215b969-c89d-44ed-8bff-36592c8cf63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:26.466677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215b969-c89d-44ed-8bff-36592c8cf63e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:26.467052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215b969-c89d-44ed-8bff-36592c8cf63e admin admin] PATH_INFO: `/v3/users/181b118771d64f09a894612ebcca32ac` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:26.468442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215b969-c89d-44ed-8bff-36592c8cf63e admin admin] RBAC: Authorizing `identity:delete_user(user_id=181b118771d64f09a894612ebcca32ac)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:26.470626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0215b969-c89d-44ed-8bff-36592c8cf63e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:26.751325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0215b969-c89d-44ed-8bff-36592c8cf63e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '181b118771d64f09a894612ebcca32ac'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:26.762343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0215b969-c89d-44ed-8bff-36592c8cf63e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '181b118771d64f09a894612ebcca32ac'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:26.764290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0215b969-c89d-44ed-8bff-36592c8cf63e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '181b118771d64f09a894612ebcca32ac'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:26.765955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0215b969-c89d-44ed-8bff-36592c8cf63e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '181b118771d64f09a894612ebcca32ac'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:26.867746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-0215b969-c89d-44ed-8bff-36592c8cf63e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '181b118771d64f09a894612ebcca32ac'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:26.892579 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4213/8396] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:40:26 2022] DELETE /identity/v3/users/181b118771d64f09a894612ebcca32ac => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:40:26.894269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4326cc-d336-4e35-8708-e8e09229380c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:26.895530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4326cc-d336-4e35-8708-e8e09229380c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:26.896001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4326cc-d336-4e35-8708-e8e09229380c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:26.918751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bf4326cc-d336-4e35-8708-e8e09229380c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:26.920016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bf4326cc-d336-4e35-8708-e8e09229380c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:27.002002 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4184/8397] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:40:27.012012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215b969-c89d-44ed-8bff-36592c8cf63e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:27.013594 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0215b969-c89d-44ed-8bff-36592c8cf63e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:27.030824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:27.100019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0215b969-c89d-44ed-8bff-36592c8cf63e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:27.106242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbf0c1-86d2-460e-b6e4-c44a5cc5fee5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:27.108388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbf0c1-86d2-460e-b6e4-c44a5cc5fee5 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:27.108765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbf0c1-86d2-460e-b6e4-c44a5cc5fee5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:27.109327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbf0c1-86d2-460e-b6e4-c44a5cc5fee5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:27.122085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cbf0c1-86d2-460e-b6e4-c44a5cc5fee5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:27.123957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cbf0c1-86d2-460e-b6e4-c44a5cc5fee5 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:27.138280 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4214/8398] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:40:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:40:27.838669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4326cc-d336-4e35-8708-e8e09229380c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:27.840256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bf4326cc-d336-4e35-8708-e8e09229380c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:27.863311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4326cc-d336-4e35-8708-e8e09229380c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:27.883886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e3eef-c05e-40f4-999d-404efdff0a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:27.885636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e3eef-c05e-40f4-999d-404efdff0a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:27.886116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e3eef-c05e-40f4-999d-404efdff0a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:27.890381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e3eef-c05e-40f4-999d-404efdff0a85 admin admin] PATH_INFO: `/v3/projects/8376a726a998426195c974ba76d614e8` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:27.890381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e3eef-c05e-40f4-999d-404efdff0a85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8376a726a998426195c974ba76d614e8)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:27.890381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e3eef-c05e-40f4-999d-404efdff0a85 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:28.006385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c39e3eef-c05e-40f4-999d-404efdff0a85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8376a726a998426195c974ba76d614e8', 'request_id': 'req-c39e3eef-c05e-40f4-999d-404efdff0a85'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:28.012969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c39e3eef-c05e-40f4-999d-404efdff0a85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8376a726a998426195c974ba76d614e8', 'request_id': 'req-c39e3eef-c05e-40f4-999d-404efdff0a85'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:28.118634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c39e3eef-c05e-40f4-999d-404efdff0a85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8376a726a998426195c974ba76d614e8', 'request_id': 'req-c39e3eef-c05e-40f4-999d-404efdff0a85'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:28.135079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c39e3eef-c05e-40f4-999d-404efdff0a85 admin admin] The token cache is being invalidate because project 8376a726a998426195c974ba76d614e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:40:28.135451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c39e3eef-c05e-40f4-999d-404efdff0a85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:28.136320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c39e3eef-c05e-40f4-999d-404efdff0a85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:28.138191 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4185/8399] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:40:27 2022] DELETE /identity/v3/projects/8376a726a998426195c974ba76d614e8 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:40:28.143015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45c57b-2744-4af0-9b2c-c8af58c8e2d0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:28.143586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45c57b-2744-4af0-9b2c-c8af58c8e2d0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:28.143586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45c57b-2744-4af0-9b2c-c8af58c8e2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:28.168694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ab45c57b-2744-4af0-9b2c-c8af58c8e2d0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:28.170458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ab45c57b-2744-4af0-9b2c-c8af58c8e2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:28.244488 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4215/8400] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:40:28.253226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e475bda-3b11-44bd-b3ac-1a5817bd6caa None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:28.253520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e475bda-3b11-44bd-b3ac-1a5817bd6caa None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:28.253805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e475bda-3b11-44bd-b3ac-1a5817bd6caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:28.273804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0e475bda-3b11-44bd-b3ac-1a5817bd6caa None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:28.277875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0e475bda-3b11-44bd-b3ac-1a5817bd6caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:28.283825 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4186/8401] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:40:28.288123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45c57b-2744-4af0-9b2c-c8af58c8e2d0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:28.298781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45c57b-2744-4af0-9b2c-c8af58c8e2d0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:28.303535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d9462-7123-42e3-a1a6-ed3e1ce18a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:28.304326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d9462-7123-42e3-a1a6-ed3e1ce18a00 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:28.304620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d9462-7123-42e3-a1a6-ed3e1ce18a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:28.304904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d9462-7123-42e3-a1a6-ed3e1ce18a00 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:28.305642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341d9462-7123-42e3-a1a6-ed3e1ce18a00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:40:28.306011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341d9462-7123-42e3-a1a6-ed3e1ce18a00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:28.308216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341d9462-7123-42e3-a1a6-ed3e1ce18a00 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:28.323861 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4216/8402] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:40:28.340993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e475bda-3b11-44bd-b3ac-1a5817bd6caa None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:28.340993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e475bda-3b11-44bd-b3ac-1a5817bd6caa None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:28.340993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63adb1-87d3-4e43-87c8-91de00c9ce63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:28.340993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63adb1-87d3-4e43-87c8-91de00c9ce63 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:28.341675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63adb1-87d3-4e43-87c8-91de00c9ce63 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:28.341675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63adb1-87d3-4e43-87c8-91de00c9ce63 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:28.342177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63adb1-87d3-4e43-87c8-91de00c9ce63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:28.344156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63adb1-87d3-4e43-87c8-91de00c9ce63 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:28.358498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:28.469898 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4187/8403] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:40:28 2022] POST /identity/v3/projects => generated 377 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:40:28.473751 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d9462-7123-42e3-a1a6-ed3e1ce18a00 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:28.484940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d9462-7123-42e3-a1a6-ed3e1ce18a00 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:28.604395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396c4b0-24d1-424d-b679-ca341b84b600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:28.605511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396c4b0-24d1-424d-b679-ca341b84b600 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:28.605985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396c4b0-24d1-424d-b679-ca341b84b600 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:28.606371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e396c4b0-24d1-424d-b679-ca341b84b600 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:28.607460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396c4b0-24d1-424d-b679-ca341b84b600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:28.610116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e396c4b0-24d1-424d-b679-ca341b84b600 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:28.744823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63adb1-87d3-4e43-87c8-91de00c9ce63 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:28.746079 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4217/8404] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:40:28 2022] POST /identity/v3/users => generated 289 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:40:28.756777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63adb1-87d3-4e43-87c8-91de00c9ce63 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:28.768771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14be98-8649-428a-96fa-fa186780e545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:28.769474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14be98-8649-428a-96fa-fa186780e545 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:28.769763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14be98-8649-428a-96fa-fa186780e545 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:28.770174 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14be98-8649-428a-96fa-fa186780e545 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:28.771587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14be98-8649-428a-96fa-fa186780e545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:28.772685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd14be98-8649-428a-96fa-fa186780e545 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:28.814786 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4188/8405] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:40:28.823283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396c4b0-24d1-424d-b679-ca341b84b600 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:28.839580 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e396c4b0-24d1-424d-b679-ca341b84b600 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:28.862430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc39e3-8032-4ff9-9149-6f7928436afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:28.863205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc39e3-8032-4ff9-9149-6f7928436afd admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:28.863728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc39e3-8032-4ff9-9149-6f7928436afd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:28.864991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc39e3-8032-4ff9-9149-6f7928436afd admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:28.865860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc39e3-8032-4ff9-9149-6f7928436afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:28.873783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc39e3-8032-4ff9-9149-6f7928436afd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:28.895768 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4218/8406] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:40:28.904314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14be98-8649-428a-96fa-fa186780e545 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:28.919793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14be98-8649-428a-96fa-fa186780e545 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:28.928945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260cde9-e947-4472-97d4-3773597964d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:28.930656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260cde9-e947-4472-97d4-3773597964d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:28.931314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260cde9-e947-4472-97d4-3773597964d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:28.931819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260cde9-e947-4472-97d4-3773597964d0 admin admin] PATH_INFO: `/v3/projects/1c3296bc45b64149bbc40007b1e01b28/users/a99614f1bbba4e6088d947e18e4f803e/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:28.954623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260cde9-e947-4472-97d4-3773597964d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c3296bc45b64149bbc40007b1e01b28, user_id=a99614f1bbba4e6088d947e18e4f803e, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:28.959519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260cde9-e947-4472-97d4-3773597964d0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:29.077447 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4189/8407] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:40:28 2022] PUT /identity/v3/projects/1c3296bc45b64149bbc40007b1e01b28/users/a99614f1bbba4e6088d947e18e4f803e/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:40:29.082455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15162666-a505-4f23-a59b-eb5bf17bc69c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:29.082736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15162666-a505-4f23-a59b-eb5bf17bc69c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:29.083021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15162666-a505-4f23-a59b-eb5bf17bc69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:29.118613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-15162666-a505-4f23-a59b-eb5bf17bc69c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:29.118613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-15162666-a505-4f23-a59b-eb5bf17bc69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:29.134099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:29.158619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260cde9-e947-4472-97d4-3773597964d0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:29.161333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9260cde9-e947-4472-97d4-3773597964d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:29.238573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260cde9-e947-4472-97d4-3773597964d0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:29.243265 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4219/8408] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:40:29.250001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14beefaf-0da2-4350-b535-7db3f103c4aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:29.250001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beefaf-0da2-4350-b535-7db3f103c4aa service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:29.250001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beefaf-0da2-4350-b535-7db3f103c4aa service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:29.250645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beefaf-0da2-4350-b535-7db3f103c4aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:29.253830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-14beefaf-0da2-4350-b535-7db3f103c4aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:29.274930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14beefaf-0da2-4350-b535-7db3f103c4aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:29.276721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14beefaf-0da2-4350-b535-7db3f103c4aa service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:29.278280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15162666-a505-4f23-a59b-eb5bf17bc69c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:29.280017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-15162666-a505-4f23-a59b-eb5bf17bc69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:29.307261 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4190/8409] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:40:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:40:29.316621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15162666-a505-4f23-a59b-eb5bf17bc69c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:29.320902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c38d66-91c7-4366-9fd1-1ec199fd446d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:29.324864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c38d66-91c7-4366-9fd1-1ec199fd446d service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:29.324864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c38d66-91c7-4366-9fd1-1ec199fd446d service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:29.324864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c38d66-91c7-4366-9fd1-1ec199fd446d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:29.333482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c38d66-91c7-4366-9fd1-1ec199fd446d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:29.335035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c38d66-91c7-4366-9fd1-1ec199fd446d service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:29.349647 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4220/8410] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:40:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:40:36.620245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14beefaf-0da2-4350-b535-7db3f103c4aa service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:36.621870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-14beefaf-0da2-4350-b535-7db3f103c4aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:36.746845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14beefaf-0da2-4350-b535-7db3f103c4aa service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:36.755553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981cf2e0-7fdb-4a95-b3d3-9e4aa7fb5b23 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:36.756681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cf2e0-7fdb-4a95-b3d3-9e4aa7fb5b23 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:36.757993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cf2e0-7fdb-4a95-b3d3-9e4aa7fb5b23 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:36.758387 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981cf2e0-7fdb-4a95-b3d3-9e4aa7fb5b23 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:36.760945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-981cf2e0-7fdb-4a95-b3d3-9e4aa7fb5b23 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:36.792370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981cf2e0-7fdb-4a95-b3d3-9e4aa7fb5b23 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:36.796005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981cf2e0-7fdb-4a95-b3d3-9e4aa7fb5b23 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:36.835221 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4191/8411] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:40:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:40:37.131624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9dcf3e-5de2-478a-825d-d3675d6bf543 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:37.132161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9dcf3e-5de2-478a-825d-d3675d6bf543 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:37.136678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9dcf3e-5de2-478a-825d-d3675d6bf543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:37.178877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1e9dcf3e-5de2-478a-825d-d3675d6bf543 None None] MFA Rules not processed for user `a99614f1bbba4e6088d947e18e4f803e`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:37.180210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1e9dcf3e-5de2-478a-825d-d3675d6bf543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:37.282835 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4221/8412] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:40:37.287757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981cf2e0-7fdb-4a95-b3d3-9e4aa7fb5b23 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:37.309963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981cf2e0-7fdb-4a95-b3d3-9e4aa7fb5b23 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:37.316065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd041141-9466-4762-94fb-c6714d35f42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:37.316747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd041141-9466-4762-94fb-c6714d35f42b admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:37.317034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd041141-9466-4762-94fb-c6714d35f42b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:37.317447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd041141-9466-4762-94fb-c6714d35f42b admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:37.318236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd041141-9466-4762-94fb-c6714d35f42b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:37.320205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd041141-9466-4762-94fb-c6714d35f42b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:37.439305 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4192/8413] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:40:37 2022] POST /identity/v3/projects => generated 381 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:40:37.444000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9dcf3e-5de2-478a-825d-d3675d6bf543 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:37.458091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9dcf3e-5de2-478a-825d-d3675d6bf543 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:37.528525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca99ea-156a-4385-82ed-8cc9e7c0386b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:37.529261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca99ea-156a-4385-82ed-8cc9e7c0386b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:37.529568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca99ea-156a-4385-82ed-8cc9e7c0386b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:37.529844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca99ea-156a-4385-82ed-8cc9e7c0386b admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:37.532661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca99ea-156a-4385-82ed-8cc9e7c0386b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:37.534873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca99ea-156a-4385-82ed-8cc9e7c0386b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:37.625312 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4222/8414] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:40:37 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:40:37.629263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd041141-9466-4762-94fb-c6714d35f42b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:37.651752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd041141-9466-4762-94fb-c6714d35f42b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:37.656728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aaf1c7-11ab-457b-a85b-87f585766803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:37.657441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaf1c7-11ab-457b-a85b-87f585766803 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:37.657731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaf1c7-11ab-457b-a85b-87f585766803 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:37.658555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaf1c7-11ab-457b-a85b-87f585766803 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:37.659311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aaf1c7-11ab-457b-a85b-87f585766803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:37.661252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aaf1c7-11ab-457b-a85b-87f585766803 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:37.674323 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4193/8415] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:40:37.679502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca99ea-156a-4385-82ed-8cc9e7c0386b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:37.691662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca99ea-156a-4385-82ed-8cc9e7c0386b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:37.697305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86365fdc-1d1d-4555-a524-602c80660a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:37.707769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86365fdc-1d1d-4555-a524-602c80660a98 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:37.707769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86365fdc-1d1d-4555-a524-602c80660a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:37.707769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86365fdc-1d1d-4555-a524-602c80660a98 admin admin] PATH_INFO: `/v3/projects/fe732730c5974bdba837cb2fc954fe79/users/8226817fe20248ca98ab3ca6cdff3627/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:37.716985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86365fdc-1d1d-4555-a524-602c80660a98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe732730c5974bdba837cb2fc954fe79, user_id=8226817fe20248ca98ab3ca6cdff3627, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:37.720024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86365fdc-1d1d-4555-a524-602c80660a98 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:37.824971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:37.830772 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4223/8416] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:40:37 2022] PUT /identity/v3/projects/fe732730c5974bdba837cb2fc954fe79/users/8226817fe20248ca98ab3ca6cdff3627/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:40:37.834881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aaf1c7-11ab-457b-a85b-87f585766803 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:37.846562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aaf1c7-11ab-457b-a85b-87f585766803 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:37.851619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:37.914616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5325e92-49ef-4d70-801c-b29c64475efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:37.915654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5325e92-49ef-4d70-801c-b29c64475efe admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:37.916042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5325e92-49ef-4d70-801c-b29c64475efe admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:37.916421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5325e92-49ef-4d70-801c-b29c64475efe admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:37.917269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5325e92-49ef-4d70-801c-b29c64475efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:37.919290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5325e92-49ef-4d70-801c-b29c64475efe admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:37.931150 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4194/8417] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:40:37 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:40:37.935770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86365fdc-1d1d-4555-a524-602c80660a98 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:37.951823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86365fdc-1d1d-4555-a524-602c80660a98 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:37.971406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7ef94-1dca-4f5f-b49b-0ff445fdf03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:37.972338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7ef94-1dca-4f5f-b49b-0ff445fdf03a admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:37.972626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7ef94-1dca-4f5f-b49b-0ff445fdf03a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:37.972918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7ef94-1dca-4f5f-b49b-0ff445fdf03a admin admin] PATH_INFO: `/v3/projects/fe732730c5974bdba837cb2fc954fe79/users/8226817fe20248ca98ab3ca6cdff3627/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:37.978886 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b7ef94-1dca-4f5f-b49b-0ff445fdf03a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe732730c5974bdba837cb2fc954fe79, user_id=8226817fe20248ca98ab3ca6cdff3627, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:37.982197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b7ef94-1dca-4f5f-b49b-0ff445fdf03a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:37.994882 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4224/8418] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:40:37 2022] PUT /identity/v3/projects/fe732730c5974bdba837cb2fc954fe79/users/8226817fe20248ca98ab3ca6cdff3627/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:40:39.792314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5325e92-49ef-4d70-801c-b29c64475efe admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:39.796271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a5325e92-49ef-4d70-801c-b29c64475efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:39.819687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5325e92-49ef-4d70-801c-b29c64475efe admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:39.955990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd13ed6-eb1a-469a-909d-519eaa68ed0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:39.956901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd13ed6-eb1a-469a-909d-519eaa68ed0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:39.957219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd13ed6-eb1a-469a-909d-519eaa68ed0d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:39.957495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd13ed6-eb1a-469a-909d-519eaa68ed0d admin admin] PATH_INFO: `/v3/users/02eea1dae1314a53ba4958ea84b3b287` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:39.959444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd13ed6-eb1a-469a-909d-519eaa68ed0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=02eea1dae1314a53ba4958ea84b3b287)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:39.961488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd13ed6-eb1a-469a-909d-519eaa68ed0d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:40.291766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1cd13ed6-eb1a-469a-909d-519eaa68ed0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02eea1dae1314a53ba4958ea84b3b287'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:40.292680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1cd13ed6-eb1a-469a-909d-519eaa68ed0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02eea1dae1314a53ba4958ea84b3b287'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:40.304500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1cd13ed6-eb1a-469a-909d-519eaa68ed0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02eea1dae1314a53ba4958ea84b3b287'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:40.386001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1cd13ed6-eb1a-469a-909d-519eaa68ed0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02eea1dae1314a53ba4958ea84b3b287'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:40.389795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1cd13ed6-eb1a-469a-909d-519eaa68ed0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02eea1dae1314a53ba4958ea84b3b287'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:40.429371 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4195/8419] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:40:39 2022] DELETE /identity/v3/users/02eea1dae1314a53ba4958ea84b3b287 => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:40:40.434715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b759a-306a-471f-9301-916f1d9dec4b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:40.435680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b759a-306a-471f-9301-916f1d9dec4b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:40.436200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b759a-306a-471f-9301-916f1d9dec4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:40.458787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2a2b759a-306a-471f-9301-916f1d9dec4b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:40.460555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2a2b759a-306a-471f-9301-916f1d9dec4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:40.551154 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4225/8420] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:40:40.562177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd13ed6-eb1a-469a-909d-519eaa68ed0d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:40.563759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1cd13ed6-eb1a-469a-909d-519eaa68ed0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:40.616615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:40.657608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd13ed6-eb1a-469a-909d-519eaa68ed0d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:40.662182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cb561-6c8c-436f-8df2-10b7301fdbe2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:40.663007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cb561-6c8c-436f-8df2-10b7301fdbe2 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:40.663962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cb561-6c8c-436f-8df2-10b7301fdbe2 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:40.664148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cb561-6c8c-436f-8df2-10b7301fdbe2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:40.675905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cb561-6c8c-436f-8df2-10b7301fdbe2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:40.678665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cb561-6c8c-436f-8df2-10b7301fdbe2 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:40.694796 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4196/8421] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:40:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:40:41.073040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b759a-306a-471f-9301-916f1d9dec4b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:41.074571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2a2b759a-306a-471f-9301-916f1d9dec4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:41.103561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b759a-306a-471f-9301-916f1d9dec4b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:41.124844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a04f94-e76b-491f-b7eb-437abbb59006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:41.124844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a04f94-e76b-491f-b7eb-437abbb59006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:41.124844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a04f94-e76b-491f-b7eb-437abbb59006 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:41.124844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a04f94-e76b-491f-b7eb-437abbb59006 admin admin] PATH_INFO: `/v3/projects/42b636fdc4204b3282fde6f9422a5afd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:41.125658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a04f94-e76b-491f-b7eb-437abbb59006 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42b636fdc4204b3282fde6f9422a5afd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:41.128497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a04f94-e76b-491f-b7eb-437abbb59006 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:41.268468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e0a04f94-e76b-491f-b7eb-437abbb59006 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42b636fdc4204b3282fde6f9422a5afd', 'request_id': 'req-e0a04f94-e76b-491f-b7eb-437abbb59006'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:41.278059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e0a04f94-e76b-491f-b7eb-437abbb59006 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42b636fdc4204b3282fde6f9422a5afd', 'request_id': 'req-e0a04f94-e76b-491f-b7eb-437abbb59006'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:41.378226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e0a04f94-e76b-491f-b7eb-437abbb59006 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42b636fdc4204b3282fde6f9422a5afd', 'request_id': 'req-e0a04f94-e76b-491f-b7eb-437abbb59006'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:41.394902 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e0a04f94-e76b-491f-b7eb-437abbb59006 admin admin] The token cache is being invalidate because project 42b636fdc4204b3282fde6f9422a5afd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:40:41.395529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e0a04f94-e76b-491f-b7eb-437abbb59006 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:41.397214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e0a04f94-e76b-491f-b7eb-437abbb59006 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:41.402293 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4226/8422] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:40:41 2022] DELETE /identity/v3/projects/42b636fdc4204b3282fde6f9422a5afd => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:40:41.412313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2eaceb-a6cd-4ee8-bb56-333361ba3e46 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:41.412313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2eaceb-a6cd-4ee8-bb56-333361ba3e46 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:41.412313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2eaceb-a6cd-4ee8-bb56-333361ba3e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:41.450812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4c2eaceb-a6cd-4ee8-bb56-333361ba3e46 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:41.452184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4c2eaceb-a6cd-4ee8-bb56-333361ba3e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:41.593726 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4197/8423] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:40:41.595080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a7769-9d7b-4f1d-abba-baeb15cb3b28 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:41.596561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a7769-9d7b-4f1d-abba-baeb15cb3b28 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:41.597020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a7769-9d7b-4f1d-abba-baeb15cb3b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:41.664115 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0f2a7769-9d7b-4f1d-abba-baeb15cb3b28 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:41.666366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0f2a7769-9d7b-4f1d-abba-baeb15cb3b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:41.696312 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4227/8424] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:40:41.707531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2eaceb-a6cd-4ee8-bb56-333361ba3e46 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:41.719916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2eaceb-a6cd-4ee8-bb56-333361ba3e46 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:41.726462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1499237-d6c7-49f3-895b-ae13d90b0676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:41.727368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1499237-d6c7-49f3-895b-ae13d90b0676 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:41.728374 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1499237-d6c7-49f3-895b-ae13d90b0676 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:41.728864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1499237-d6c7-49f3-895b-ae13d90b0676 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:41.730899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1499237-d6c7-49f3-895b-ae13d90b0676 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:40:41.731333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1499237-d6c7-49f3-895b-ae13d90b0676 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:41.733307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1499237-d6c7-49f3-895b-ae13d90b0676 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:41.749556 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4198/8425] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:40:41.754077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a7769-9d7b-4f1d-abba-baeb15cb3b28 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:41.768753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a7769-9d7b-4f1d-abba-baeb15cb3b28 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:41.773390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369436a5-2014-434a-b1da-63942b0dd5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:41.774457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369436a5-2014-434a-b1da-63942b0dd5e8 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:41.774821 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369436a5-2014-434a-b1da-63942b0dd5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:41.774999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369436a5-2014-434a-b1da-63942b0dd5e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:41.778391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369436a5-2014-434a-b1da-63942b0dd5e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:41.779172 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369436a5-2014-434a-b1da-63942b0dd5e8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:41.914215 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4228/8426] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:40:41 2022] POST /identity/v3/projects => generated 359 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:40:41.916052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1499237-d6c7-49f3-895b-ae13d90b0676 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:41.927884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1499237-d6c7-49f3-895b-ae13d90b0676 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:42.023651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bef573-d84c-4fd4-be03-b108c5ec4665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:42.025157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bef573-d84c-4fd4-be03-b108c5ec4665 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:42.025711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bef573-d84c-4fd4-be03-b108c5ec4665 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:42.026220 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bef573-d84c-4fd4-be03-b108c5ec4665 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:42.027086 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bef573-d84c-4fd4-be03-b108c5ec4665 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:42.029210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bef573-d84c-4fd4-be03-b108c5ec4665 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:42.141229 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4199/8427] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:40:41 2022] POST /identity/v3/users => generated 280 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:40:42.142543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369436a5-2014-434a-b1da-63942b0dd5e8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:42.161299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369436a5-2014-434a-b1da-63942b0dd5e8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:42.172658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05018948-7442-4d80-bd62-4b6778b99171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:42.173357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05018948-7442-4d80-bd62-4b6778b99171 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:42.173642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05018948-7442-4d80-bd62-4b6778b99171 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:42.174033 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05018948-7442-4d80-bd62-4b6778b99171 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:42.174709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05018948-7442-4d80-bd62-4b6778b99171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:42.176330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05018948-7442-4d80-bd62-4b6778b99171 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:42.245934 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4229/8428] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:40:42.247380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bef573-d84c-4fd4-be03-b108c5ec4665 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:42.282776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bef573-d84c-4fd4-be03-b108c5ec4665 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:42.298789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418c159-ffdc-42e0-9d6b-e959af09d1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:42.299367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418c159-ffdc-42e0-9d6b-e959af09d1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:42.299654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418c159-ffdc-42e0-9d6b-e959af09d1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:42.300367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418c159-ffdc-42e0-9d6b-e959af09d1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:42.305833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418c159-ffdc-42e0-9d6b-e959af09d1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:42.305833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418c159-ffdc-42e0-9d6b-e959af09d1c1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:42.327229 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4200/8429] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:40:42.330073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05018948-7442-4d80-bd62-4b6778b99171 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:42.348062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05018948-7442-4d80-bd62-4b6778b99171 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:42.357136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5895628-788e-47d4-9346-f10c3a9dbf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:42.358206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5895628-788e-47d4-9346-f10c3a9dbf55 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:42.358657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5895628-788e-47d4-9346-f10c3a9dbf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:42.359010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5895628-788e-47d4-9346-f10c3a9dbf55 admin admin] PATH_INFO: `/v3/projects/ed1ea4f454ee4cba84a5e5cc9b5a60e1/users/f350405c79b241e19a9229dff252cff7/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:42.389457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5895628-788e-47d4-9346-f10c3a9dbf55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed1ea4f454ee4cba84a5e5cc9b5a60e1, user_id=f350405c79b241e19a9229dff252cff7, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:42.392265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5895628-788e-47d4-9346-f10c3a9dbf55 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:42.471688 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4230/8430] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:40:42 2022] PUT /identity/v3/projects/ed1ea4f454ee4cba84a5e5cc9b5a60e1/users/f350405c79b241e19a9229dff252cff7/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:40:42.482134 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb394d4c-44f0-4efd-81f2-3f5734c60756 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:42.483345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb394d4c-44f0-4efd-81f2-3f5734c60756 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:42.483778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb394d4c-44f0-4efd-81f2-3f5734c60756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:42.540138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-eb394d4c-44f0-4efd-81f2-3f5734c60756 None None] MFA Rules not processed for user `f350405c79b241e19a9229dff252cff7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:42.540138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-eb394d4c-44f0-4efd-81f2-3f5734c60756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:42.581364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:42.747342 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4201/8431] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:42 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:40:42.759490 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5895628-788e-47d4-9346-f10c3a9dbf55 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:42.761610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b5895628-788e-47d4-9346-f10c3a9dbf55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:42.768629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:42.840176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5895628-788e-47d4-9346-f10c3a9dbf55 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:42.845372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61cb1e-06e5-46b1-aa77-07b161a5579c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:42.845372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61cb1e-06e5-46b1-aa77-07b161a5579c service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:42.845372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61cb1e-06e5-46b1-aa77-07b161a5579c service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:42.845372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61cb1e-06e5-46b1-aa77-07b161a5579c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:42.856179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61cb1e-06e5-46b1-aa77-07b161a5579c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:42.858284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61cb1e-06e5-46b1-aa77-07b161a5579c service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:42.869311 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4231/8432] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:40:42 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:40:42.917940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb394d4c-44f0-4efd-81f2-3f5734c60756 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:42.920820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-eb394d4c-44f0-4efd-81f2-3f5734c60756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:43.037521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb394d4c-44f0-4efd-81f2-3f5734c60756 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:43.042449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e513cde-37fd-490a-8df5-e96fc841d336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:43.043524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e513cde-37fd-490a-8df5-e96fc841d336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:43.043802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e513cde-37fd-490a-8df5-e96fc841d336 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:43.044132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e513cde-37fd-490a-8df5-e96fc841d336 admin admin] PATH_INFO: `/v3/users/13249a1c05fa43bd917b36aafb9db0d9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:43.053017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e513cde-37fd-490a-8df5-e96fc841d336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13249a1c05fa43bd917b36aafb9db0d9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:43.053017 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e513cde-37fd-490a-8df5-e96fc841d336 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:43.298506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1e513cde-37fd-490a-8df5-e96fc841d336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13249a1c05fa43bd917b36aafb9db0d9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:43.298506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1e513cde-37fd-490a-8df5-e96fc841d336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13249a1c05fa43bd917b36aafb9db0d9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:43.314884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1e513cde-37fd-490a-8df5-e96fc841d336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13249a1c05fa43bd917b36aafb9db0d9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:43.401941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1e513cde-37fd-490a-8df5-e96fc841d336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13249a1c05fa43bd917b36aafb9db0d9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:43.402773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1e513cde-37fd-490a-8df5-e96fc841d336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13249a1c05fa43bd917b36aafb9db0d9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:43.428474 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4202/8433] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:40:42 2022] DELETE /identity/v3/users/13249a1c05fa43bd917b36aafb9db0d9 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:40:43.962544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61cb1e-06e5-46b1-aa77-07b161a5579c service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:43.975742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61cb1e-06e5-46b1-aa77-07b161a5579c service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:44.102388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:44.103306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:44.103765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:44.104243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f admin admin] PATH_INFO: `/v3/projects/4b4e0b07528040a3aa0194c06fb942cb` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:44.105697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b4e0b07528040a3aa0194c06fb942cb)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:44.108500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:44.283091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b4e0b07528040a3aa0194c06fb942cb', 'request_id': 'req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:44.291262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b4e0b07528040a3aa0194c06fb942cb', 'request_id': 'req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:44.385252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b4e0b07528040a3aa0194c06fb942cb', 'request_id': 'req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:44.408470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f admin admin] The token cache is being invalidate because project 4b4e0b07528040a3aa0194c06fb942cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:40:44.409103 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:44.410665 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:44.414780 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4232/8434] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:40:43 2022] DELETE /identity/v3/projects/4b4e0b07528040a3aa0194c06fb942cb => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:40:44.419118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e513cde-37fd-490a-8df5-e96fc841d336 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:44.427652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1e513cde-37fd-490a-8df5-e96fc841d336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:44.442028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e513cde-37fd-490a-8df5-e96fc841d336 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:44.495757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:44.534696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501ca4fb-bcc6-4b7a-97c9-f258c6d2ab50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:44.535642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ca4fb-bcc6-4b7a-97c9-f258c6d2ab50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:44.537744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ca4fb-bcc6-4b7a-97c9-f258c6d2ab50 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:44.538072 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501ca4fb-bcc6-4b7a-97c9-f258c6d2ab50 admin admin] PATH_INFO: `/v3/users/bce779fb1eb547a5a077c6f1ac48d800` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:44.544063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ca4fb-bcc6-4b7a-97c9-f258c6d2ab50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bce779fb1eb547a5a077c6f1ac48d800)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:44.548612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501ca4fb-bcc6-4b7a-97c9-f258c6d2ab50 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:44.897225 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-501ca4fb-bcc6-4b7a-97c9-f258c6d2ab50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bce779fb1eb547a5a077c6f1ac48d800'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:44.897870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-501ca4fb-bcc6-4b7a-97c9-f258c6d2ab50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bce779fb1eb547a5a077c6f1ac48d800'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:44.906939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-501ca4fb-bcc6-4b7a-97c9-f258c6d2ab50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bce779fb1eb547a5a077c6f1ac48d800'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:45.017997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-501ca4fb-bcc6-4b7a-97c9-f258c6d2ab50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bce779fb1eb547a5a077c6f1ac48d800'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:45.018885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-501ca4fb-bcc6-4b7a-97c9-f258c6d2ab50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bce779fb1eb547a5a077c6f1ac48d800'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:45.051237 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4203/8435] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:40:44 2022] DELETE /identity/v3/users/bce779fb1eb547a5a077c6f1ac48d800 => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:40:45.458229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:45.458229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:45.478338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5c848-93eb-4f5c-adb0-f405ef23ec6f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:45.539027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:45.578270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41ed6fd-6bb6-4d61-b90d-b54631a930e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:45.579174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ed6fd-6bb6-4d61-b90d-b54631a930e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:45.579504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ed6fd-6bb6-4d61-b90d-b54631a930e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:45.579783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41ed6fd-6bb6-4d61-b90d-b54631a930e3 admin admin] PATH_INFO: `/v3/projects/e5d9e07b1e18471e9a5c0667719672f6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:45.581004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ed6fd-6bb6-4d61-b90d-b54631a930e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5d9e07b1e18471e9a5c0667719672f6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:45.583030 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41ed6fd-6bb6-4d61-b90d-b54631a930e3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:45.770185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b41ed6fd-6bb6-4d61-b90d-b54631a930e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5d9e07b1e18471e9a5c0667719672f6', 'request_id': 'req-b41ed6fd-6bb6-4d61-b90d-b54631a930e3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:45.777873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b41ed6fd-6bb6-4d61-b90d-b54631a930e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5d9e07b1e18471e9a5c0667719672f6', 'request_id': 'req-b41ed6fd-6bb6-4d61-b90d-b54631a930e3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:45.872010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b41ed6fd-6bb6-4d61-b90d-b54631a930e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5d9e07b1e18471e9a5c0667719672f6', 'request_id': 'req-b41ed6fd-6bb6-4d61-b90d-b54631a930e3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:45.889676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b41ed6fd-6bb6-4d61-b90d-b54631a930e3 admin admin] The token cache is being invalidate because project e5d9e07b1e18471e9a5c0667719672f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:40:45.890390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b41ed6fd-6bb6-4d61-b90d-b54631a930e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:45.891431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b41ed6fd-6bb6-4d61-b90d-b54631a930e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:45.893730 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4233/8436] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:40:45 2022] DELETE /identity/v3/projects/e5d9e07b1e18471e9a5c0667719672f6 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:40:45.898824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b8401-e65d-47a4-99fc-16c5e409a254 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:45.905763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b8401-e65d-47a4-99fc-16c5e409a254 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:45.905763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b8401-e65d-47a4-99fc-16c5e409a254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:45.929799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e75b8401-e65d-47a4-99fc-16c5e409a254 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:45.939941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e75b8401-e65d-47a4-99fc-16c5e409a254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:46.062632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9d957-e11c-4a79-aeee-5cc91b2520a1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:46.063010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9d957-e11c-4a79-aeee-5cc91b2520a1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:46.063254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e9d957-e11c-4a79-aeee-5cc91b2520a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:46.078121 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4204/8437] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:40:46.093195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f3b99-596c-4b14-bf41-c528c95ee177 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:46.093573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f3b99-596c-4b14-bf41-c528c95ee177 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:46.093894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f3b99-596c-4b14-bf41-c528c95ee177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:46.109078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a2e9d957-e11c-4a79-aeee-5cc91b2520a1 None None] MFA Rules not processed for user `8226817fe20248ca98ab3ca6cdff3627`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:46.110859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a2e9d957-e11c-4a79-aeee-5cc91b2520a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:46.119215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5c1f3b99-596c-4b14-bf41-c528c95ee177 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:46.120720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5c1f3b99-596c-4b14-bf41-c528c95ee177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:46.133355 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4205/8438] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:40:46.134999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f3b99-596c-4b14-bf41-c528c95ee177 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:46.145003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f3b99-596c-4b14-bf41-c528c95ee177 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:46.157677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1373e2e-8ff6-4da3-bcb0-3ebc19e020bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:46.158566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1373e2e-8ff6-4da3-bcb0-3ebc19e020bd admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:46.159126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1373e2e-8ff6-4da3-bcb0-3ebc19e020bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:46.159425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1373e2e-8ff6-4da3-bcb0-3ebc19e020bd admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:46.160180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1373e2e-8ff6-4da3-bcb0-3ebc19e020bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:40:46.160498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1373e2e-8ff6-4da3-bcb0-3ebc19e020bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:46.162178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1373e2e-8ff6-4da3-bcb0-3ebc19e020bd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:46.181422 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4206/8439] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:40:46.186848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1373e2e-8ff6-4da3-bcb0-3ebc19e020bd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:46.201417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1373e2e-8ff6-4da3-bcb0-3ebc19e020bd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:46.211674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534f4c5-7aef-4683-8461-6e53a2e3485c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:46.213231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534f4c5-7aef-4683-8461-6e53a2e3485c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:46.213516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534f4c5-7aef-4683-8461-6e53a2e3485c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:46.213800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534f4c5-7aef-4683-8461-6e53a2e3485c admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:46.214952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534f4c5-7aef-4683-8461-6e53a2e3485c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:46.217568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534f4c5-7aef-4683-8461-6e53a2e3485c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:46.217789 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4234/8440] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:40:46.233888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9d957-e11c-4a79-aeee-5cc91b2520a1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:46.238883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a2e9d957-e11c-4a79-aeee-5cc91b2520a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:46.321024 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4207/8441] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:40:46 2022] POST /identity/v3/projects => generated 363 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:40:46.328607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534f4c5-7aef-4683-8461-6e53a2e3485c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:46.362740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e9d957-e11c-4a79-aeee-5cc91b2520a1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:46.422358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534f4c5-7aef-4683-8461-6e53a2e3485c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:46.427908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df51e92-7783-4056-9bb4-daf064afcdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:46.428773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df51e92-7783-4056-9bb4-daf064afcdfd admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:46.429175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df51e92-7783-4056-9bb4-daf064afcdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:46.429553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df51e92-7783-4056-9bb4-daf064afcdfd admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:46.430413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df51e92-7783-4056-9bb4-daf064afcdfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:46.433082 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df51e92-7783-4056-9bb4-daf064afcdfd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:46.454894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda3171-0c3e-45a5-bf7e-19936c862d30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:46.455556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda3171-0c3e-45a5-bf7e-19936c862d30 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:46.455950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda3171-0c3e-45a5-bf7e-19936c862d30 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:46.456139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecda3171-0c3e-45a5-bf7e-19936c862d30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:46.519227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda3171-0c3e-45a5-bf7e-19936c862d30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:46.520848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecda3171-0c3e-45a5-bf7e-19936c862d30 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:46.532118 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4235/8442] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:40:46 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:40:46.567264 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4208/8443] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:40:46 2022] POST /identity/v3/users => generated 282 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:40:46.571734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda3171-0c3e-45a5-bf7e-19936c862d30 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:46.584426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:46.584879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecda3171-0c3e-45a5-bf7e-19936c862d30 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:46.591906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002207d-f740-4b26-980b-25ac0c79993e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:46.592593 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002207d-f740-4b26-980b-25ac0c79993e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:46.592988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002207d-f740-4b26-980b-25ac0c79993e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:46.593339 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002207d-f740-4b26-980b-25ac0c79993e admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:46.594619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0002207d-f740-4b26-980b-25ac0c79993e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:46.596326 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0002207d-f740-4b26-980b-25ac0c79993e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:46.612309 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4236/8444] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:40:46.616350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df51e92-7783-4056-9bb4-daf064afcdfd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:46.646012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df51e92-7783-4056-9bb4-daf064afcdfd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:46.664303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1d253-ae10-4c9c-aaf3-1378108f1331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:46.666239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1d253-ae10-4c9c-aaf3-1378108f1331 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:46.666613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1d253-ae10-4c9c-aaf3-1378108f1331 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:46.667141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1d253-ae10-4c9c-aaf3-1378108f1331 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:46.668535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1d253-ae10-4c9c-aaf3-1378108f1331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:46.670521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1d253-ae10-4c9c-aaf3-1378108f1331 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:46.687752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002207d-f740-4b26-980b-25ac0c79993e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:46.693541 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4209/8445] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:40:46.699883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0002207d-f740-4b26-980b-25ac0c79993e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:46.707375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556321a0-b884-4d87-a4de-cd30b1ad74c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:46.708534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556321a0-b884-4d87-a4de-cd30b1ad74c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:46.708871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556321a0-b884-4d87-a4de-cd30b1ad74c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:46.709210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556321a0-b884-4d87-a4de-cd30b1ad74c9 admin admin] PATH_INFO: `/v3/projects/02a82e6798e74a53b1bbde8ed8a9cf9f/users/283d56477a24406188f9b59f6d832ec3/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:46.732008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556321a0-b884-4d87-a4de-cd30b1ad74c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02a82e6798e74a53b1bbde8ed8a9cf9f, user_id=283d56477a24406188f9b59f6d832ec3, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:46.732008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556321a0-b884-4d87-a4de-cd30b1ad74c9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:46.821321 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4237/8446] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:40:46 2022] PUT /identity/v3/projects/02a82e6798e74a53b1bbde8ed8a9cf9f/users/283d56477a24406188f9b59f6d832ec3/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:40:46.826988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087937c8-b955-4734-af35-70e514eb3d38 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:46.827244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087937c8-b955-4734-af35-70e514eb3d38 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:46.827604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087937c8-b955-4734-af35-70e514eb3d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:46.855009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-087937c8-b955-4734-af35-70e514eb3d38 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:46.856470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-087937c8-b955-4734-af35-70e514eb3d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:46.962629 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4210/8447] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:40:46.972833 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556321a0-b884-4d87-a4de-cd30b1ad74c9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:46.974776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-556321a0-b884-4d87-a4de-cd30b1ad74c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:47.019304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087937c8-b955-4734-af35-70e514eb3d38 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:47.020850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-087937c8-b955-4734-af35-70e514eb3d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:47.055666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556321a0-b884-4d87-a4de-cd30b1ad74c9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:47.060924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173f83f-0c0c-407b-8a14-0849a804b505 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:47.061645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173f83f-0c0c-407b-8a14-0849a804b505 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:47.061958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173f83f-0c0c-407b-8a14-0849a804b505 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:47.062242 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173f83f-0c0c-407b-8a14-0849a804b505 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:47.082712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173f83f-0c0c-407b-8a14-0849a804b505 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:47.084025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c173f83f-0c0c-407b-8a14-0849a804b505 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:47.106048 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4238/8448] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:40:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:40:47.116000 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087937c8-b955-4734-af35-70e514eb3d38 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:47.121512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931716d-6e34-4788-952e-b13f6eb3140f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:47.122474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931716d-6e34-4788-952e-b13f6eb3140f service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:47.123054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931716d-6e34-4788-952e-b13f6eb3140f service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:47.123330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931716d-6e34-4788-952e-b13f6eb3140f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:47.125496 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0931716d-6e34-4788-952e-b13f6eb3140f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:47.141502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931716d-6e34-4788-952e-b13f6eb3140f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:47.143828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931716d-6e34-4788-952e-b13f6eb3140f service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:47.159004 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4211/8449] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:40:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:40:47.724653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173f83f-0c0c-407b-8a14-0849a804b505 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:47.726073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c173f83f-0c0c-407b-8a14-0849a804b505 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:47.749653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173f83f-0c0c-407b-8a14-0849a804b505 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:47.755211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1cb1a-ee93-4913-9a87-c7bd0669a2a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:47.755892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1cb1a-ee93-4913-9a87-c7bd0669a2a1 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:47.756169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1cb1a-ee93-4913-9a87-c7bd0669a2a1 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:47.756448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c1cb1a-ee93-4913-9a87-c7bd0669a2a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:47.758655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-34c1cb1a-ee93-4913-9a87-c7bd0669a2a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:47.777831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1cb1a-ee93-4913-9a87-c7bd0669a2a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:47.779549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c1cb1a-ee93-4913-9a87-c7bd0669a2a1 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:47.798573 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4239/8450] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:40:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:40:48.002227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:49.447263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:57.360159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb0fdd-794f-4167-a2dd-244a94614b14 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:57.365517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb0fdd-794f-4167-a2dd-244a94614b14 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:57.365517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb0fdd-794f-4167-a2dd-244a94614b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:57.419199 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c4bb0fdd-794f-4167-a2dd-244a94614b14 None None] MFA Rules not processed for user `283d56477a24406188f9b59f6d832ec3`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:57.420607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c4bb0fdd-794f-4167-a2dd-244a94614b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:57.455092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:57.527688 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4212/8451] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:57 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:40:57.531885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1cb1a-ee93-4913-9a87-c7bd0669a2a1 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:57.549174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c1cb1a-ee93-4913-9a87-c7bd0669a2a1 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:57.583056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614a60b-16ba-4446-b543-ed7778e672f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:57.585005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614a60b-16ba-4446-b543-ed7778e672f4 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:57.585678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614a60b-16ba-4446-b543-ed7778e672f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:57.586179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614a60b-16ba-4446-b543-ed7778e672f4 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:57.587181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614a60b-16ba-4446-b543-ed7778e672f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:57.598963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614a60b-16ba-4446-b543-ed7778e672f4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:57.723372 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4240/8452] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:40:57 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:40:57.727398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb0fdd-794f-4167-a2dd-244a94614b14 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:57.744779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb0fdd-794f-4167-a2dd-244a94614b14 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:57.757992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9d6a8-7f3a-404c-a2da-9389a504e030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:57.758707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9d6a8-7f3a-404c-a2da-9389a504e030 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:57.770054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9d6a8-7f3a-404c-a2da-9389a504e030 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:57.770054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9d6a8-7f3a-404c-a2da-9389a504e030 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:57.770054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9d6a8-7f3a-404c-a2da-9389a504e030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:57.770054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9d6a8-7f3a-404c-a2da-9389a504e030 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:57.785174 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4213/8453] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:40:57.789446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614a60b-16ba-4446-b543-ed7778e672f4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:57.818647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614a60b-16ba-4446-b543-ed7778e672f4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:57.830396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee7e64-8c2a-4a5d-bd1d-4ddbfb18b3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:57.832784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee7e64-8c2a-4a5d-bd1d-4ddbfb18b3d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:57.832784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee7e64-8c2a-4a5d-bd1d-4ddbfb18b3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:57.833797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee7e64-8c2a-4a5d-bd1d-4ddbfb18b3d9 admin admin] PATH_INFO: `/v3/system/users/1123a329780c47ef82208d41488e7ec9/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:57.857768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee7e64-8c2a-4a5d-bd1d-4ddbfb18b3d9 admin admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1123a329780c47ef82208d41488e7ec9, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:57.860048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee7e64-8c2a-4a5d-bd1d-4ddbfb18b3d9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:57.972927 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4241/8454] 10.222.0.33 () {60 vars in 1377 bytes} [Sat May 28 21:40:57 2022] PUT /identity/v3/system/users/1123a329780c47ef82208d41488e7ec9/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:40:57.975665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f769d-7484-4805-be43-3cb2faaf9916 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:57.976721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f769d-7484-4805-be43-3cb2faaf9916 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:57.977362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f769d-7484-4805-be43-3cb2faaf9916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:58.022154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-196f769d-7484-4805-be43-3cb2faaf9916 None None] MFA Rules not processed for user `1123a329780c47ef82208d41488e7ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:40:58.023642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-196f769d-7484-4805-be43-3cb2faaf9916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:58.129582 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4214/8455] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:40:57 2022] POST /identity/v3/auth/tokens => generated 2404 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 28 21:40:58.134344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee7e64-8c2a-4a5d-bd1d-4ddbfb18b3d9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:58.272117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee7e64-8c2a-4a5d-bd1d-4ddbfb18b3d9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:58.315514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75bb83d-25e9-4217-8fc4-71bbe2943798 None tempest-ImageQuotaTest-644300533-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123a329780c47ef82208d41488e7ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:58.317523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bb83d-25e9-4217-8fc4-71bbe2943798 None tempest-ImageQuotaTest-644300533-system-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:58.317523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bb83d-25e9-4217-8fc4-71bbe2943798 None tempest-ImageQuotaTest-644300533-system-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:58.317523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75bb83d-25e9-4217-8fc4-71bbe2943798 None tempest-ImageQuotaTest-644300533-system-admin] PATH_INFO: `/v3/services` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:58.317523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75bb83d-25e9-4217-8fc4-71bbe2943798 None tempest-ImageQuotaTest-644300533-system-admin] RBAC: Authorizing `identity:list_services()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:58.319246 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75bb83d-25e9-4217-8fc4-71bbe2943798 None tempest-ImageQuotaTest-644300533-system-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:58.328133 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4242/8456] 10.222.0.33 () {58 vars in 1109 bytes} [Sat May 28 21:40:58 2022] GET /identity/v3/services => generated 2191 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:40:58.332660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f769d-7484-4805-be43-3cb2faaf9916 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:58.419097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:58.450777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f769d-7484-4805-be43-3cb2faaf9916 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:58.486815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c48f19-211c-4c09-b459-ffbf2420e3fa None tempest-ImageQuotaTest-644300533-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123a329780c47ef82208d41488e7ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:58.487532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c48f19-211c-4c09-b459-ffbf2420e3fa None tempest-ImageQuotaTest-644300533-system-admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:58.488125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c48f19-211c-4c09-b459-ffbf2420e3fa None tempest-ImageQuotaTest-644300533-system-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:58.488125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c48f19-211c-4c09-b459-ffbf2420e3fa None tempest-ImageQuotaTest-644300533-system-admin] PATH_INFO: `/v3/limits` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:58.488862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c48f19-211c-4c09-b459-ffbf2420e3fa None tempest-ImageQuotaTest-644300533-system-admin] RBAC: Authorizing `identity:create_limits()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:58.490709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c48f19-211c-4c09-b459-ffbf2420e3fa None tempest-ImageQuotaTest-644300533-system-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:58.548334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-60c48f19-211c-4c09-b459-ffbf2420e3fa None tempest-ImageQuotaTest-644300533-system-admin] Unable to create a limit that has no corresponding registered limit.: keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. May 28 21:40:58.548334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:40:58.548334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:40:58.548334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:40:58.548334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:40:58.548334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:40:58.548334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:40:58.548334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:40:58.548334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:40:58.548334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:40:58.548334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:40:58.548334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:40:58.548334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/limits.py", line 110, in post May 28 21:40:58.548334 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application refs = PROVIDERS.unified_limit_api.create_limits(limits) May 28 21:40:58.555726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:40:58.555726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:40:58.555726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/core.py", line 116, in create_limits May 28 21:40:58.555726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.driver.create_limits(limits) May 28 21:40:58.555726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 28 21:40:58.555726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 28 21:40:58.555726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 260, in create_limits May 28 21:40:58.555726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application target = self._check_and_fill_registered_limit_id(limit) May 28 21:40:58.555726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/limit/backends/sql.py", line 249, in _check_and_fill_registered_limit_id May 28 21:40:58.555726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.NoLimitReference May 28 21:40:58.555726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.NoLimitReference: Unable to create a limit that has no corresponding registered limit. May 28 21:40:58.555726 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:40:58.555726 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4215/8457] 10.222.0.33 () {60 vars in 1125 bytes} [Sat May 28 21:40:58 2022] POST /identity/v3/limits => generated 124 bytes in 218 msecs (HTTP/1.1 403) 5 headers in 184 bytes (1 switches on core 0) May 28 21:40:58.556858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75bb83d-25e9-4217-8fc4-71bbe2943798 None tempest-ImageQuotaTest-644300533-system-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:58.558066 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:58.750175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75bb83d-25e9-4217-8fc4-71bbe2943798 None tempest-ImageQuotaTest-644300533-system-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:58.820991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b5ade-ecdd-430f-9adf-fb346817e70d None tempest-ImageQuotaTest-644300533-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123a329780c47ef82208d41488e7ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:58.822929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b5ade-ecdd-430f-9adf-fb346817e70d None tempest-ImageQuotaTest-644300533-system-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:58.823380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b5ade-ecdd-430f-9adf-fb346817e70d None tempest-ImageQuotaTest-644300533-system-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:58.823805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b5ade-ecdd-430f-9adf-fb346817e70d None tempest-ImageQuotaTest-644300533-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:58.826955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b5ade-ecdd-430f-9adf-fb346817e70d None tempest-ImageQuotaTest-644300533-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:58.826955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b5ade-ecdd-430f-9adf-fb346817e70d None tempest-ImageQuotaTest-644300533-system-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:58.835699 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4243/8458] 10.222.0.33 () {58 vars in 1136 bytes} [Sat May 28 21:40:58 2022] GET /identity/v3/registered_limits => generated 129 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:40:58.843309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c48f19-211c-4c09-b459-ffbf2420e3fa None tempest-ImageQuotaTest-644300533-system-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:59.094800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c48f19-211c-4c09-b459-ffbf2420e3fa None tempest-ImageQuotaTest-644300533-system-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:59.130849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:59.141949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48864f71-44d2-4327-8cf5-0c2fe883954b None tempest-ImageQuotaTest-644300533-system-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1123a329780c47ef82208d41488e7ec9', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:59.142706 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48864f71-44d2-4327-8cf5-0c2fe883954b None tempest-ImageQuotaTest-644300533-system-admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:59.143291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48864f71-44d2-4327-8cf5-0c2fe883954b None tempest-ImageQuotaTest-644300533-system-admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:59.143730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48864f71-44d2-4327-8cf5-0c2fe883954b None tempest-ImageQuotaTest-644300533-system-admin] PATH_INFO: `/v3/registered_limits` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:59.144584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48864f71-44d2-4327-8cf5-0c2fe883954b None tempest-ImageQuotaTest-644300533-system-admin] RBAC: Authorizing `identity:list_registered_limits()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:59.145791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48864f71-44d2-4327-8cf5-0c2fe883954b None tempest-ImageQuotaTest-644300533-system-admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:59.153889 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4216/8459] 10.222.0.33 () {58 vars in 1136 bytes} [Sat May 28 21:40:58 2022] GET /identity/v3/registered_limits => generated 129 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:40:59.544727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b5ade-ecdd-430f-9adf-fb346817e70d None tempest-ImageQuotaTest-644300533-system-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:40:59.546732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6c8b5ade-ecdd-430f-9adf-fb346817e70d None tempest-ImageQuotaTest-644300533-system-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:40:59.570570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b5ade-ecdd-430f-9adf-fb346817e70d None tempest-ImageQuotaTest-644300533-system-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:40:59.578087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07923cdb-d42e-4c61-b62d-2def429bdc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:40:59.579479 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07923cdb-d42e-4c61-b62d-2def429bdc4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:40:59.580055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07923cdb-d42e-4c61-b62d-2def429bdc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:40:59.580615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07923cdb-d42e-4c61-b62d-2def429bdc4c admin admin] PATH_INFO: `/v3/users/a99614f1bbba4e6088d947e18e4f803e` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:40:59.585470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07923cdb-d42e-4c61-b62d-2def429bdc4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a99614f1bbba4e6088d947e18e4f803e)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:40:59.587759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07923cdb-d42e-4c61-b62d-2def429bdc4c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:40:59.806741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-07923cdb-d42e-4c61-b62d-2def429bdc4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a99614f1bbba4e6088d947e18e4f803e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:59.814056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-07923cdb-d42e-4c61-b62d-2def429bdc4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a99614f1bbba4e6088d947e18e4f803e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:59.814909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-07923cdb-d42e-4c61-b62d-2def429bdc4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a99614f1bbba4e6088d947e18e4f803e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:59.815634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-07923cdb-d42e-4c61-b62d-2def429bdc4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a99614f1bbba4e6088d947e18e4f803e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:59.922739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-07923cdb-d42e-4c61-b62d-2def429bdc4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a99614f1bbba4e6088d947e18e4f803e'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:40:59.934055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:59.952152 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4244/8460] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:40:59 2022] DELETE /identity/v3/users/a99614f1bbba4e6088d947e18e4f803e => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:41:00.265933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48864f71-44d2-4327-8cf5-0c2fe883954b None tempest-ImageQuotaTest-644300533-system-admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:00.267479 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-48864f71-44d2-4327-8cf5-0c2fe883954b None tempest-ImageQuotaTest-644300533-system-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:00.384979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48864f71-44d2-4327-8cf5-0c2fe883954b None tempest-ImageQuotaTest-644300533-system-admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:00.391826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8f52f-9576-4e28-8cbd-08004205f2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:00.393153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8f52f-9576-4e28-8cbd-08004205f2e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:00.393628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8f52f-9576-4e28-8cbd-08004205f2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:00.394178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8f52f-9576-4e28-8cbd-08004205f2e1 admin admin] PATH_INFO: `/v3/projects/1c3296bc45b64149bbc40007b1e01b28` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:00.395779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8f52f-9576-4e28-8cbd-08004205f2e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c3296bc45b64149bbc40007b1e01b28)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:00.397901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8f52f-9576-4e28-8cbd-08004205f2e1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:00.561622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e3b8f52f-9576-4e28-8cbd-08004205f2e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c3296bc45b64149bbc40007b1e01b28', 'request_id': 'req-e3b8f52f-9576-4e28-8cbd-08004205f2e1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:00.571291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e3b8f52f-9576-4e28-8cbd-08004205f2e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c3296bc45b64149bbc40007b1e01b28', 'request_id': 'req-e3b8f52f-9576-4e28-8cbd-08004205f2e1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:00.690499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e3b8f52f-9576-4e28-8cbd-08004205f2e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c3296bc45b64149bbc40007b1e01b28', 'request_id': 'req-e3b8f52f-9576-4e28-8cbd-08004205f2e1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:00.710214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e3b8f52f-9576-4e28-8cbd-08004205f2e1 admin admin] The token cache is being invalidate because project 1c3296bc45b64149bbc40007b1e01b28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:41:00.710672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e3b8f52f-9576-4e28-8cbd-08004205f2e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:00.712519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e3b8f52f-9576-4e28-8cbd-08004205f2e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:00.718344 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4217/8461] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:41:00 2022] DELETE /identity/v3/projects/1c3296bc45b64149bbc40007b1e01b28 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:41:00.726984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07923cdb-d42e-4c61-b62d-2def429bdc4c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:00.752477 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07923cdb-d42e-4c61-b62d-2def429bdc4c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:00.829368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521489d2-af80-41a3-8af3-3f707d9a0153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:00.830722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521489d2-af80-41a3-8af3-3f707d9a0153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:00.833153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521489d2-af80-41a3-8af3-3f707d9a0153 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:00.833637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521489d2-af80-41a3-8af3-3f707d9a0153 admin admin] PATH_INFO: `/v3/users/8226817fe20248ca98ab3ca6cdff3627` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:00.835207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521489d2-af80-41a3-8af3-3f707d9a0153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8226817fe20248ca98ab3ca6cdff3627)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:00.837499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521489d2-af80-41a3-8af3-3f707d9a0153 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:01.120286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-521489d2-af80-41a3-8af3-3f707d9a0153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8226817fe20248ca98ab3ca6cdff3627'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:01.131163 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-521489d2-af80-41a3-8af3-3f707d9a0153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8226817fe20248ca98ab3ca6cdff3627'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:01.132008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-521489d2-af80-41a3-8af3-3f707d9a0153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8226817fe20248ca98ab3ca6cdff3627'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:01.132623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-521489d2-af80-41a3-8af3-3f707d9a0153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8226817fe20248ca98ab3ca6cdff3627'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:01.200720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-521489d2-af80-41a3-8af3-3f707d9a0153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8226817fe20248ca98ab3ca6cdff3627'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:01.233486 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4245/8462] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:41:00 2022] DELETE /identity/v3/users/8226817fe20248ca98ab3ca6cdff3627 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:41:02.021813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8f52f-9576-4e28-8cbd-08004205f2e1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:02.023441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e3b8f52f-9576-4e28-8cbd-08004205f2e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:02.046573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8f52f-9576-4e28-8cbd-08004205f2e1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:02.175046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90182918-30ad-4512-859b-8e384fbbe2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:02.176567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90182918-30ad-4512-859b-8e384fbbe2f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:02.176567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90182918-30ad-4512-859b-8e384fbbe2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:02.176567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90182918-30ad-4512-859b-8e384fbbe2f9 admin admin] PATH_INFO: `/v3/projects/fe732730c5974bdba837cb2fc954fe79` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:02.178142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90182918-30ad-4512-859b-8e384fbbe2f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe732730c5974bdba837cb2fc954fe79)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:02.180353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90182918-30ad-4512-859b-8e384fbbe2f9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:02.214948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:02.344417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-90182918-30ad-4512-859b-8e384fbbe2f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe732730c5974bdba837cb2fc954fe79', 'request_id': 'req-90182918-30ad-4512-859b-8e384fbbe2f9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:02.364837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-90182918-30ad-4512-859b-8e384fbbe2f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe732730c5974bdba837cb2fc954fe79', 'request_id': 'req-90182918-30ad-4512-859b-8e384fbbe2f9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:02.439748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-90182918-30ad-4512-859b-8e384fbbe2f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe732730c5974bdba837cb2fc954fe79', 'request_id': 'req-90182918-30ad-4512-859b-8e384fbbe2f9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:02.476590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-90182918-30ad-4512-859b-8e384fbbe2f9 admin admin] The token cache is being invalidate because project fe732730c5974bdba837cb2fc954fe79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:41:02.478391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-90182918-30ad-4512-859b-8e384fbbe2f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:02.479156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-90182918-30ad-4512-859b-8e384fbbe2f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:02.486233 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4218/8463] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:41:02 2022] DELETE /identity/v3/projects/fe732730c5974bdba837cb2fc954fe79 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:41:02.487230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fa3a9-8c55-4515-961c-4e5025c36bc1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:02.488770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fa3a9-8c55-4515-961c-4e5025c36bc1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:02.489592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fa3a9-8c55-4515-961c-4e5025c36bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:02.518735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ba1fa3a9-8c55-4515-961c-4e5025c36bc1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:41:02.520012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ba1fa3a9-8c55-4515-961c-4e5025c36bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:02.630576 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4246/8464] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:41:02.631709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a8bd7-44f2-4206-86df-fb205a8a071a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:02.632065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a8bd7-44f2-4206-86df-fb205a8a071a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:02.632473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a8bd7-44f2-4206-86df-fb205a8a071a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:02.678877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-763a8bd7-44f2-4206-86df-fb205a8a071a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:41:02.682513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-763a8bd7-44f2-4206-86df-fb205a8a071a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:02.696801 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4219/8465] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:41:02.698766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fa3a9-8c55-4515-961c-4e5025c36bc1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:02.706475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:02.713080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1fa3a9-8c55-4515-961c-4e5025c36bc1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:02.726935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591a524-8606-478e-8dfd-89b47c64378f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:02.729142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591a524-8606-478e-8dfd-89b47c64378f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:02.729705 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591a524-8606-478e-8dfd-89b47c64378f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:02.730229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591a524-8606-478e-8dfd-89b47c64378f admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:02.731309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591a524-8606-478e-8dfd-89b47c64378f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:41:02.732312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591a524-8606-478e-8dfd-89b47c64378f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:02.735324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591a524-8606-478e-8dfd-89b47c64378f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:02.764456 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4247/8466] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:41:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:41:02.764878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a8bd7-44f2-4206-86df-fb205a8a071a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:02.770485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a8bd7-44f2-4206-86df-fb205a8a071a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:02.781625 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfec32-2bd6-4289-bbff-2d2a9534ef59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:02.787361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfec32-2bd6-4289-bbff-2d2a9534ef59 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:02.787361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfec32-2bd6-4289-bbff-2d2a9534ef59 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:02.787361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfec32-2bd6-4289-bbff-2d2a9534ef59 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:02.787361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfec32-2bd6-4289-bbff-2d2a9534ef59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:02.787361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfec32-2bd6-4289-bbff-2d2a9534ef59 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:02.922613 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4220/8467] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:41:02 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:41:02.927688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591a524-8606-478e-8dfd-89b47c64378f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:02.944216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591a524-8606-478e-8dfd-89b47c64378f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:03.054960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd2a19-84e6-4580-a744-acf47a24000c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:03.056459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd2a19-84e6-4580-a744-acf47a24000c admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:03.056936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd2a19-84e6-4580-a744-acf47a24000c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:03.057393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd2a19-84e6-4580-a744-acf47a24000c admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:03.058374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd2a19-84e6-4580-a744-acf47a24000c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:03.061678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd2a19-84e6-4580-a744-acf47a24000c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:03.180034 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4248/8468] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:41:02 2022] POST /identity/v3/users => generated 290 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:41:03.189116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfec32-2bd6-4289-bbff-2d2a9534ef59 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:03.201253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfec32-2bd6-4289-bbff-2d2a9534ef59 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:03.207778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2148b1-b8cf-4aa0-a427-550a40d569a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:03.208700 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2148b1-b8cf-4aa0-a427-550a40d569a2 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:03.209600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2148b1-b8cf-4aa0-a427-550a40d569a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:03.210876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2148b1-b8cf-4aa0-a427-550a40d569a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:03.215167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2148b1-b8cf-4aa0-a427-550a40d569a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:03.216819 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2148b1-b8cf-4aa0-a427-550a40d569a2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:03.228477 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4221/8469] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:41:03.232655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd2a19-84e6-4580-a744-acf47a24000c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:03.242977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd2a19-84e6-4580-a744-acf47a24000c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:03.249385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e28a7-52fb-436a-b72b-b70b7b91362d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:03.250700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e28a7-52fb-436a-b72b-b70b7b91362d admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:03.250700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e28a7-52fb-436a-b72b-b70b7b91362d admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:03.253125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e28a7-52fb-436a-b72b-b70b7b91362d admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:03.253444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e28a7-52fb-436a-b72b-b70b7b91362d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:03.255596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e28a7-52fb-436a-b72b-b70b7b91362d admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:03.272213 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4249/8470] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:41:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:41:03.273197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2148b1-b8cf-4aa0-a427-550a40d569a2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:03.293204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2148b1-b8cf-4aa0-a427-550a40d569a2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:03.321367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb386db8-63e1-4b06-8c03-e59791c8d708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:03.322696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb386db8-63e1-4b06-8c03-e59791c8d708 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:03.323184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb386db8-63e1-4b06-8c03-e59791c8d708 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:03.323643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb386db8-63e1-4b06-8c03-e59791c8d708 admin admin] PATH_INFO: `/v3/projects/338603768c9143128533b10cfc00530e/users/ddfb9c65496f4c28bbfed14bb0a4529d/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:03.358085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb386db8-63e1-4b06-8c03-e59791c8d708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=338603768c9143128533b10cfc00530e, user_id=ddfb9c65496f4c28bbfed14bb0a4529d, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:03.360942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb386db8-63e1-4b06-8c03-e59791c8d708 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:03.451989 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4222/8471] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:41:03 2022] PUT /identity/v3/projects/338603768c9143128533b10cfc00530e/users/ddfb9c65496f4c28bbfed14bb0a4529d/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:41:03.457491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62a121-cd37-4f77-996e-ec1b8fb66be5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:03.458117 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62a121-cd37-4f77-996e-ec1b8fb66be5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:03.458561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62a121-cd37-4f77-996e-ec1b8fb66be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:03.480876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7d62a121-cd37-4f77-996e-ec1b8fb66be5 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:41:03.482218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7d62a121-cd37-4f77-996e-ec1b8fb66be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:03.557028 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4250/8472] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:41:03.567213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb386db8-63e1-4b06-8c03-e59791c8d708 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:03.569191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bb386db8-63e1-4b06-8c03-e59791c8d708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:03.636727 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb386db8-63e1-4b06-8c03-e59791c8d708 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:03.640751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69692f-d315-46a6-af59-e6bebd970481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:03.641417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69692f-d315-46a6-af59-e6bebd970481 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:03.641705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69692f-d315-46a6-af59-e6bebd970481 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:03.642115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69692f-d315-46a6-af59-e6bebd970481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:03.652427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69692f-d315-46a6-af59-e6bebd970481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:03.653959 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69692f-d315-46a6-af59-e6bebd970481 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:03.664120 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4223/8473] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:41:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:41:04.831722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62a121-cd37-4f77-996e-ec1b8fb66be5 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:04.833519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7d62a121-cd37-4f77-996e-ec1b8fb66be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:04.849336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62a121-cd37-4f77-996e-ec1b8fb66be5 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:04.855277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c8cdd1-80b7-4b94-b3c9-3c4c4f455e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:04.856265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8cdd1-80b7-4b94-b3c9-3c4c4f455e63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:04.856635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8cdd1-80b7-4b94-b3c9-3c4c4f455e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:04.856988 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c8cdd1-80b7-4b94-b3c9-3c4c4f455e63 admin admin] PATH_INFO: `/v3/users/283d56477a24406188f9b59f6d832ec3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:04.858346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8cdd1-80b7-4b94-b3c9-3c4c4f455e63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=283d56477a24406188f9b59f6d832ec3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:04.860443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c8cdd1-80b7-4b94-b3c9-3c4c4f455e63 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:05.109521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47c8cdd1-80b7-4b94-b3c9-3c4c4f455e63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '283d56477a24406188f9b59f6d832ec3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:05.117380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47c8cdd1-80b7-4b94-b3c9-3c4c4f455e63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '283d56477a24406188f9b59f6d832ec3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:05.118158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47c8cdd1-80b7-4b94-b3c9-3c4c4f455e63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '283d56477a24406188f9b59f6d832ec3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:05.118992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47c8cdd1-80b7-4b94-b3c9-3c4c4f455e63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '283d56477a24406188f9b59f6d832ec3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:05.192819 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-47c8cdd1-80b7-4b94-b3c9-3c4c4f455e63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '283d56477a24406188f9b59f6d832ec3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:05.195407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:05.215461 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4251/8474] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:41:04 2022] DELETE /identity/v3/users/283d56477a24406188f9b59f6d832ec3 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:41:05.481809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69692f-d315-46a6-af59-e6bebd970481 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:05.494213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69692f-d315-46a6-af59-e6bebd970481 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:05.553310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:05.563616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf64db-5756-4c83-9396-b08435379d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:05.567928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf64db-5756-4c83-9396-b08435379d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:05.567928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf64db-5756-4c83-9396-b08435379d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:05.567928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf64db-5756-4c83-9396-b08435379d84 admin admin] PATH_INFO: `/v3/projects/02a82e6798e74a53b1bbde8ed8a9cf9f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:05.567928 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf64db-5756-4c83-9396-b08435379d84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02a82e6798e74a53b1bbde8ed8a9cf9f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:05.568588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf64db-5756-4c83-9396-b08435379d84 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:05.769250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacfe2d-d558-4edb-a356-07ef31a62c24 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:05.769551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacfe2d-d558-4edb-a356-07ef31a62c24 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:05.770060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacfe2d-d558-4edb-a356-07ef31a62c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:05.786455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bcbf64db-5756-4c83-9396-b08435379d84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02a82e6798e74a53b1bbde8ed8a9cf9f', 'request_id': 'req-bcbf64db-5756-4c83-9396-b08435379d84'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:05.793993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bcbf64db-5756-4c83-9396-b08435379d84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02a82e6798e74a53b1bbde8ed8a9cf9f', 'request_id': 'req-bcbf64db-5756-4c83-9396-b08435379d84'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:05.802955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9dacfe2d-d558-4edb-a356-07ef31a62c24 None None] MFA Rules not processed for user `ddfb9c65496f4c28bbfed14bb0a4529d`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:41:05.804442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9dacfe2d-d558-4edb-a356-07ef31a62c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:05.872890 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4252/8475] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:41:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:41:05.877371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacfe2d-d558-4edb-a356-07ef31a62c24 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:05.878912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9dacfe2d-d558-4edb-a356-07ef31a62c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:05.879981 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bcbf64db-5756-4c83-9396-b08435379d84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02a82e6798e74a53b1bbde8ed8a9cf9f', 'request_id': 'req-bcbf64db-5756-4c83-9396-b08435379d84'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:05.895897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bcbf64db-5756-4c83-9396-b08435379d84 admin admin] The token cache is being invalidate because project 02a82e6798e74a53b1bbde8ed8a9cf9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:41:05.896455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bcbf64db-5756-4c83-9396-b08435379d84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:05.897185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bcbf64db-5756-4c83-9396-b08435379d84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:05.899002 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4224/8476] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:41:05 2022] DELETE /identity/v3/projects/02a82e6798e74a53b1bbde8ed8a9cf9f => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:41:05.905768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf64db-5756-4c83-9396-b08435379d84 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:05.907560 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bcbf64db-5756-4c83-9396-b08435379d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:05.952317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacfe2d-d558-4edb-a356-07ef31a62c24 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:05.958053 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791e68f-5e12-4d87-ba64-3dd8eb60305e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:05.958845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791e68f-5e12-4d87-ba64-3dd8eb60305e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:05.959237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791e68f-5e12-4d87-ba64-3dd8eb60305e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:05.959598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791e68f-5e12-4d87-ba64-3dd8eb60305e admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:05.960299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791e68f-5e12-4d87-ba64-3dd8eb60305e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:05.962312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791e68f-5e12-4d87-ba64-3dd8eb60305e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:05.988889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf64db-5756-4c83-9396-b08435379d84 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:05.994115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaecaac-9678-418d-9240-3b085bb628d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:05.995019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaecaac-9678-418d-9240-3b085bb628d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:05.995304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaecaac-9678-418d-9240-3b085bb628d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:05.995580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaecaac-9678-418d-9240-3b085bb628d5 admin admin] PATH_INFO: `/v3/users/1123a329780c47ef82208d41488e7ec9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:05.996796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaecaac-9678-418d-9240-3b085bb628d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1123a329780c47ef82208d41488e7ec9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:05.999097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaecaac-9678-418d-9240-3b085bb628d5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:06.077097 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4253/8477] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:41:05 2022] POST /identity/v3/projects => generated 377 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:41:06.081132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791e68f-5e12-4d87-ba64-3dd8eb60305e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:06.082801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0791e68f-5e12-4d87-ba64-3dd8eb60305e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:06.146700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791e68f-5e12-4d87-ba64-3dd8eb60305e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:06.152091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ba11c-5b48-4a95-8a30-c8596055f5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:06.152877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ba11c-5b48-4a95-8a30-c8596055f5fb admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:06.153171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ba11c-5b48-4a95-8a30-c8596055f5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:06.153459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ba11c-5b48-4a95-8a30-c8596055f5fb admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:06.154191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ba11c-5b48-4a95-8a30-c8596055f5fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:06.156150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683ba11c-5b48-4a95-8a30-c8596055f5fb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:06.278332 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4254/8478] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:41:06 2022] POST /identity/v3/users => generated 295 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:41:06.279887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ba11c-5b48-4a95-8a30-c8596055f5fb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:06.290790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ba11c-5b48-4a95-8a30-c8596055f5fb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:06.295656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086ac09-610f-4c46-b044-dc67e7e6e82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:06.298184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086ac09-610f-4c46-b044-dc67e7e6e82a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:06.298184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086ac09-610f-4c46-b044-dc67e7e6e82a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:06.298184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b086ac09-610f-4c46-b044-dc67e7e6e82a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:06.298184 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086ac09-610f-4c46-b044-dc67e7e6e82a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:06.299231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b086ac09-610f-4c46-b044-dc67e7e6e82a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:06.313234 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4255/8479] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:41:06.315913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086ac09-610f-4c46-b044-dc67e7e6e82a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:06.326155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b086ac09-610f-4c46-b044-dc67e7e6e82a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:06.328618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7aaecaac-9678-418d-9240-3b085bb628d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1123a329780c47ef82208d41488e7ec9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:06.329388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7aaecaac-9678-418d-9240-3b085bb628d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1123a329780c47ef82208d41488e7ec9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:06.331667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018222a-62d3-4940-acc0-e8e9c4c872a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:06.332683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018222a-62d3-4940-acc0-e8e9c4c872a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:06.333506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018222a-62d3-4940-acc0-e8e9c4c872a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:06.333797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018222a-62d3-4940-acc0-e8e9c4c872a6 admin admin] PATH_INFO: `/v3/projects/917c2022c8124f15b783cc9ea2881687/users/74d1f74153364269b0e639be85cd05a0/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:06.338002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7aaecaac-9678-418d-9240-3b085bb628d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1123a329780c47ef82208d41488e7ec9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:06.351410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018222a-62d3-4940-acc0-e8e9c4c872a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=917c2022c8124f15b783cc9ea2881687, user_id=74d1f74153364269b0e639be85cd05a0, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:06.355345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018222a-62d3-4940-acc0-e8e9c4c872a6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:06.421375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7aaecaac-9678-418d-9240-3b085bb628d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1123a329780c47ef82208d41488e7ec9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:06.430425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7aaecaac-9678-418d-9240-3b085bb628d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1123a329780c47ef82208d41488e7ec9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:06.442452 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4225/8480] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:41:05 2022] DELETE /identity/v3/users/1123a329780c47ef82208d41488e7ec9 => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:41:06.486663 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4256/8481] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:41:06 2022] PUT /identity/v3/projects/917c2022c8124f15b783cc9ea2881687/users/74d1f74153364269b0e639be85cd05a0/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:41:06.490741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaecaac-9678-418d-9240-3b085bb628d5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:06.492187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7aaecaac-9678-418d-9240-3b085bb628d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:06.554354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:06.561358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaecaac-9678-418d-9240-3b085bb628d5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:06.619209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39debe-f949-49ea-ba1c-48d1ca0090b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:06.619918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39debe-f949-49ea-ba1c-48d1ca0090b7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:06.620206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39debe-f949-49ea-ba1c-48d1ca0090b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:06.620488 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39debe-f949-49ea-ba1c-48d1ca0090b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:06.621226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39debe-f949-49ea-ba1c-48d1ca0090b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:06.622920 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39debe-f949-49ea-ba1c-48d1ca0090b7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:06.636708 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4226/8482] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:41:06 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:41:06.638078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018222a-62d3-4940-acc0-e8e9c4c872a6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:06.648868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018222a-62d3-4940-acc0-e8e9c4c872a6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:06.654520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b040d8-2df9-488f-9aa9-9b7009920b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:06.655418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b040d8-2df9-488f-9aa9-9b7009920b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:06.655704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b040d8-2df9-488f-9aa9-9b7009920b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:06.655989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b040d8-2df9-488f-9aa9-9b7009920b97 admin admin] PATH_INFO: `/v3/projects/917c2022c8124f15b783cc9ea2881687/users/74d1f74153364269b0e639be85cd05a0/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:06.658061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b040d8-2df9-488f-9aa9-9b7009920b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=917c2022c8124f15b783cc9ea2881687, user_id=74d1f74153364269b0e639be85cd05a0, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:06.660744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b040d8-2df9-488f-9aa9-9b7009920b97 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:06.675577 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4257/8483] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:41:06 2022] PUT /identity/v3/projects/917c2022c8124f15b783cc9ea2881687/users/74d1f74153364269b0e639be85cd05a0/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:41:06.994876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:07.415543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39debe-f949-49ea-ba1c-48d1ca0090b7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:07.417707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4d39debe-f949-49ea-ba1c-48d1ca0090b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:07.449532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39debe-f949-49ea-ba1c-48d1ca0090b7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:07.532052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410788c9-8eba-43e5-8517-1fd8e1bb16dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:07.532870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410788c9-8eba-43e5-8517-1fd8e1bb16dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:07.533165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410788c9-8eba-43e5-8517-1fd8e1bb16dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:07.533441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410788c9-8eba-43e5-8517-1fd8e1bb16dc admin admin] PATH_INFO: `/v3/projects/None` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:07.541965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410788c9-8eba-43e5-8517-1fd8e1bb16dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=None)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:07.543899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410788c9-8eba-43e5-8517-1fd8e1bb16dc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:07.557467 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application [None req-410788c9-8eba-43e5-8517-1fd8e1bb16dc admin admin] Could not find project: None.: keystone.exception.ProjectNotFound: Could not find project: None. May 28 21:41:07.557467 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application Traceback (most recent call last): May 28 21:41:07.557467 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 28 21:41:07.557467 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 28 21:41:07.557467 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 28 21:41:07.557467 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 28 21:41:07.557467 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 28 21:41:07.557467 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 28 21:41:07.557467 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 28 21:41:07.557467 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 28 21:41:07.557467 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 28 21:41:07.557467 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 28 21:41:07.557467 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 211, in delete May 28 21:41:07.557467 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application PROVIDERS.resource_api.delete_project( May 28 21:41:07.557467 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 28 21:41:07.557467 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 28 21:41:07.558794 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 478, in delete_project May 28 21:41:07.558794 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application project = self.driver.get_project(project_id) May 28 21:41:07.558794 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 28 21:41:07.558794 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 28 21:41:07.558794 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 28 21:41:07.558794 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 28 21:41:07.558794 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: None. May 28 21:41:07.558794 ncdv-master-1 devstack@keystone.service[103104]: ERROR keystone.server.flask.application May 28 21:41:07.562779 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4227/8484] 10.222.0.33 () {58 vars in 1148 bytes} [Sat May 28 21:41:07 2022] DELETE /identity/v3/projects/None => generated 85 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 28 21:41:07.781862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b040d8-2df9-488f-9aa9-9b7009920b97 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:07.783958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-60b040d8-2df9-488f-9aa9-9b7009920b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:07.901219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b040d8-2df9-488f-9aa9-9b7009920b97 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:08.228527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448d4248-e1f2-4165-a1ee-a81d49ac9fa0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:08.228527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d4248-e1f2-4165-a1ee-a81d49ac9fa0 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:08.228527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d4248-e1f2-4165-a1ee-a81d49ac9fa0 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:08.228527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448d4248-e1f2-4165-a1ee-a81d49ac9fa0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:08.229319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-448d4248-e1f2-4165-a1ee-a81d49ac9fa0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:08.229319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d4248-e1f2-4165-a1ee-a81d49ac9fa0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:08.229319 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448d4248-e1f2-4165-a1ee-a81d49ac9fa0 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:08.229319 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4258/8485] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:41:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:41:08.840645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13ba42-8332-4546-8efa-2975957ef191 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:08.841237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13ba42-8332-4546-8efa-2975957ef191 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:08.841680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13ba42-8332-4546-8efa-2975957ef191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:08.875404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-cf13ba42-8332-4546-8efa-2975957ef191 None None] MFA Rules not processed for user `74d1f74153364269b0e639be85cd05a0`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:41:08.877716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cf13ba42-8332-4546-8efa-2975957ef191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:08.891830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:08.946840 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4228/8486] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:41:08 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:41:08.957796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a53b28-1d39-4803-9f78-1eb048e52bbd None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:08.958316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a53b28-1d39-4803-9f78-1eb048e52bbd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:08.958430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a53b28-1d39-4803-9f78-1eb048e52bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:08.976236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-03a53b28-1d39-4803-9f78-1eb048e52bbd None None] MFA Rules not processed for user `ddfb9c65496f4c28bbfed14bb0a4529d`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:41:08.983849 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-03a53b28-1d39-4803-9f78-1eb048e52bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:09.045483 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4259/8487] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:41:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:41:09.055290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13ba42-8332-4546-8efa-2975957ef191 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:09.056748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-cf13ba42-8332-4546-8efa-2975957ef191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:09.131613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13ba42-8332-4546-8efa-2975957ef191 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:09.140571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9b9f2e-1c7a-418c-bf29-ca4142b6cf79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:09.140571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b9f2e-1c7a-418c-bf29-ca4142b6cf79 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:09.140571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b9f2e-1c7a-418c-bf29-ca4142b6cf79 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:09.140571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9b9f2e-1c7a-418c-bf29-ca4142b6cf79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:09.141009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6a9b9f2e-1c7a-418c-bf29-ca4142b6cf79 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:09.151930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b9f2e-1c7a-418c-bf29-ca4142b6cf79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:09.153539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9b9f2e-1c7a-418c-bf29-ca4142b6cf79 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:09.163279 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4229/8488] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:41:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:41:11.230275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a53b28-1d39-4803-9f78-1eb048e52bbd None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:11.258585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a53b28-1d39-4803-9f78-1eb048e52bbd None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:11.265938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c685b-a5ba-4a96-a56c-9c59fb521426 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:11.267179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c685b-a5ba-4a96-a56c-9c59fb521426 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:11.267501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c685b-a5ba-4a96-a56c-9c59fb521426 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:11.269197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c685b-a5ba-4a96-a56c-9c59fb521426 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:11.271309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c66c685b-a5ba-4a96-a56c-9c59fb521426 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:11.290554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c685b-a5ba-4a96-a56c-9c59fb521426 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:11.293039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c685b-a5ba-4a96-a56c-9c59fb521426 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:11.308718 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4260/8489] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:41:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:41:19.434303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=56 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:21.301577 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=86,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:24.561130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4858b55-0919-4488-8d4f-747e0788f181 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:24.561636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4858b55-0919-4488-8d4f-747e0788f181 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:24.561831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4858b55-0919-4488-8d4f-747e0788f181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:24.579837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c4858b55-0919-4488-8d4f-747e0788f181 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:41:24.581166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c4858b55-0919-4488-8d4f-747e0788f181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:24.639067 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4230/8490] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:41:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:41:24.791429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c685b-a5ba-4a96-a56c-9c59fb521426 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:24.792900 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c66c685b-a5ba-4a96-a56c-9c59fb521426 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:24.889970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c685b-a5ba-4a96-a56c-9c59fb521426 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:24.895861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66947cd9-0ee2-45b3-a493-db1099c6a74e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:24.896516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66947cd9-0ee2-45b3-a493-db1099c6a74e service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:24.896989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66947cd9-0ee2-45b3-a493-db1099c6a74e service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:24.897446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66947cd9-0ee2-45b3-a493-db1099c6a74e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:24.911765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66947cd9-0ee2-45b3-a493-db1099c6a74e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:24.914509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66947cd9-0ee2-45b3-a493-db1099c6a74e service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:24.932023 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4261/8491] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:41:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 21:41:34.508252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6289a39-1afd-4fcc-8a3a-6aab9e2bacd9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:34.509203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6289a39-1afd-4fcc-8a3a-6aab9e2bacd9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:34.509297 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6289a39-1afd-4fcc-8a3a-6aab9e2bacd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:34.529720 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b6289a39-1afd-4fcc-8a3a-6aab9e2bacd9 None None] MFA Rules not processed for user `388458a7b3504bd08f3789f53cf9ea35`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:41:34.531090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b6289a39-1afd-4fcc-8a3a-6aab9e2bacd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:34.589043 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4231/8492] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:41:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:41:34.922943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=32 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:36.755280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66947cd9-0ee2-45b3-a493-db1099c6a74e service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:36.766211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66947cd9-0ee2-45b3-a493-db1099c6a74e service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:36.770810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9211c00-c9e9-48ac-9de2-98752a3c34b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:36.772398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9211c00-c9e9-48ac-9de2-98752a3c34b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:36.772398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9211c00-c9e9-48ac-9de2-98752a3c34b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:36.772398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9211c00-c9e9-48ac-9de2-98752a3c34b4 admin admin] PATH_INFO: `/v3/users/ddfb9c65496f4c28bbfed14bb0a4529d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:36.773443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9211c00-c9e9-48ac-9de2-98752a3c34b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddfb9c65496f4c28bbfed14bb0a4529d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:36.776269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9211c00-c9e9-48ac-9de2-98752a3c34b4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:36.982280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b9211c00-c9e9-48ac-9de2-98752a3c34b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddfb9c65496f4c28bbfed14bb0a4529d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:36.993677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b9211c00-c9e9-48ac-9de2-98752a3c34b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddfb9c65496f4c28bbfed14bb0a4529d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:36.994496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b9211c00-c9e9-48ac-9de2-98752a3c34b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddfb9c65496f4c28bbfed14bb0a4529d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:36.996499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b9211c00-c9e9-48ac-9de2-98752a3c34b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddfb9c65496f4c28bbfed14bb0a4529d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:37.085065 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b9211c00-c9e9-48ac-9de2-98752a3c34b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddfb9c65496f4c28bbfed14bb0a4529d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:37.106248 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4262/8493] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:41:36 2022] DELETE /identity/v3/users/ddfb9c65496f4c28bbfed14bb0a4529d => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:41:37.362050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6289a39-1afd-4fcc-8a3a-6aab9e2bacd9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:37.380011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6289a39-1afd-4fcc-8a3a-6aab9e2bacd9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:37.491846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6041a6-fdec-4d4e-ba29-c8df82b39274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:37.494843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6041a6-fdec-4d4e-ba29-c8df82b39274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:37.495160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6041a6-fdec-4d4e-ba29-c8df82b39274 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:37.495419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6041a6-fdec-4d4e-ba29-c8df82b39274 admin admin] PATH_INFO: `/v3/projects/338603768c9143128533b10cfc00530e` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:37.496687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6041a6-fdec-4d4e-ba29-c8df82b39274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=338603768c9143128533b10cfc00530e)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:37.498657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6041a6-fdec-4d4e-ba29-c8df82b39274 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:37.611773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4e6041a6-fdec-4d4e-ba29-c8df82b39274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '338603768c9143128533b10cfc00530e', 'request_id': 'req-4e6041a6-fdec-4d4e-ba29-c8df82b39274'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:37.613733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:37.617540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4e6041a6-fdec-4d4e-ba29-c8df82b39274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '338603768c9143128533b10cfc00530e', 'request_id': 'req-4e6041a6-fdec-4d4e-ba29-c8df82b39274'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:37.685498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4e6041a6-fdec-4d4e-ba29-c8df82b39274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '338603768c9143128533b10cfc00530e', 'request_id': 'req-4e6041a6-fdec-4d4e-ba29-c8df82b39274'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:37.699729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4e6041a6-fdec-4d4e-ba29-c8df82b39274 admin admin] The token cache is being invalidate because project 338603768c9143128533b10cfc00530e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:41:37.700327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4e6041a6-fdec-4d4e-ba29-c8df82b39274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:37.701697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4e6041a6-fdec-4d4e-ba29-c8df82b39274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:37.704265 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4232/8494] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:41:37 2022] DELETE /identity/v3/projects/338603768c9143128533b10cfc00530e => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:41:37.709230 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9211c00-c9e9-48ac-9de2-98752a3c34b4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:37.710854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b9211c00-c9e9-48ac-9de2-98752a3c34b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:37.725888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9211c00-c9e9-48ac-9de2-98752a3c34b4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:37.786062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc47fc3-9998-42e2-8d97-8d2c0f336333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:37.788334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc47fc3-9998-42e2-8d97-8d2c0f336333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:37.788334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc47fc3-9998-42e2-8d97-8d2c0f336333 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:37.788334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc47fc3-9998-42e2-8d97-8d2c0f336333 admin admin] PATH_INFO: `/v3/users/74d1f74153364269b0e639be85cd05a0` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:37.789530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc47fc3-9998-42e2-8d97-8d2c0f336333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74d1f74153364269b0e639be85cd05a0)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:37.790961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc47fc3-9998-42e2-8d97-8d2c0f336333 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:38.008045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abc47fc3-9998-42e2-8d97-8d2c0f336333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74d1f74153364269b0e639be85cd05a0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:38.016782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abc47fc3-9998-42e2-8d97-8d2c0f336333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74d1f74153364269b0e639be85cd05a0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:38.017581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abc47fc3-9998-42e2-8d97-8d2c0f336333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74d1f74153364269b0e639be85cd05a0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:38.018297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abc47fc3-9998-42e2-8d97-8d2c0f336333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74d1f74153364269b0e639be85cd05a0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:38.113608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-abc47fc3-9998-42e2-8d97-8d2c0f336333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74d1f74153364269b0e639be85cd05a0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:38.129866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:38.143829 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4263/8495] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:41:37 2022] DELETE /identity/v3/users/74d1f74153364269b0e639be85cd05a0 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:41:38.699738 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6041a6-fdec-4d4e-ba29-c8df82b39274 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:38.702382 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4e6041a6-fdec-4d4e-ba29-c8df82b39274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:38.721047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6041a6-fdec-4d4e-ba29-c8df82b39274 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:38.835030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9953e4-147b-46ce-8722-bc84a8be0285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:38.837101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9953e4-147b-46ce-8722-bc84a8be0285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:38.837101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9953e4-147b-46ce-8722-bc84a8be0285 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:38.837101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9953e4-147b-46ce-8722-bc84a8be0285 admin admin] PATH_INFO: `/v3/projects/917c2022c8124f15b783cc9ea2881687` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:38.838442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9953e4-147b-46ce-8722-bc84a8be0285 admin admin] RBAC: Authorizing `identity:delete_project(project_id=917c2022c8124f15b783cc9ea2881687)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:38.840857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9953e4-147b-46ce-8722-bc84a8be0285 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:38.968786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e9953e4-147b-46ce-8722-bc84a8be0285 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '917c2022c8124f15b783cc9ea2881687', 'request_id': 'req-7e9953e4-147b-46ce-8722-bc84a8be0285'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:38.975422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e9953e4-147b-46ce-8722-bc84a8be0285 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '917c2022c8124f15b783cc9ea2881687', 'request_id': 'req-7e9953e4-147b-46ce-8722-bc84a8be0285'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:39.067708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e9953e4-147b-46ce-8722-bc84a8be0285 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '917c2022c8124f15b783cc9ea2881687', 'request_id': 'req-7e9953e4-147b-46ce-8722-bc84a8be0285'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:39.085388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e9953e4-147b-46ce-8722-bc84a8be0285 admin admin] The token cache is being invalidate because project 917c2022c8124f15b783cc9ea2881687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:41:39.085760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e9953e4-147b-46ce-8722-bc84a8be0285 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:39.087115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7e9953e4-147b-46ce-8722-bc84a8be0285 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:39.093893 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4233/8496] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:41:38 2022] DELETE /identity/v3/projects/917c2022c8124f15b783cc9ea2881687 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:41:48.702721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:49.082150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=57,keystone:DELETE=9,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:50.177400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc47fc3-9998-42e2-8d97-8d2c0f336333 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:50.178979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-abc47fc3-9998-42e2-8d97-8d2c0f336333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:50.246744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc47fc3-9998-42e2-8d97-8d2c0f336333 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:50.296926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411cd35-f3ea-433b-8721-d04ae63525a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:50.297611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411cd35-f3ea-433b-8721-d04ae63525a7 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:50.297943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411cd35-f3ea-433b-8721-d04ae63525a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:50.298244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411cd35-f3ea-433b-8721-d04ae63525a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:50.300342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7411cd35-f3ea-433b-8721-d04ae63525a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:50.371974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411cd35-f3ea-433b-8721-d04ae63525a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:50.373734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411cd35-f3ea-433b-8721-d04ae63525a7 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:50.387126 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4264/8497] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:41:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:41:55.554242 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9953e4-147b-46ce-8722-bc84a8be0285 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:55.556626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7e9953e4-147b-46ce-8722-bc84a8be0285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:55.624064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9953e4-147b-46ce-8722-bc84a8be0285 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:55.628704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253165e-7e55-489d-81d0-6f45140ce1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:55.629623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253165e-7e55-489d-81d0-6f45140ce1f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:55.629940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253165e-7e55-489d-81d0-6f45140ce1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:55.630221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253165e-7e55-489d-81d0-6f45140ce1f9 admin admin] PATH_INFO: `/v3/users/f350405c79b241e19a9229dff252cff7` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:55.631409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253165e-7e55-489d-81d0-6f45140ce1f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f350405c79b241e19a9229dff252cff7)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:55.634007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2253165e-7e55-489d-81d0-6f45140ce1f9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:55.829029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2253165e-7e55-489d-81d0-6f45140ce1f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f350405c79b241e19a9229dff252cff7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:55.829697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2253165e-7e55-489d-81d0-6f45140ce1f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f350405c79b241e19a9229dff252cff7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:55.836692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2253165e-7e55-489d-81d0-6f45140ce1f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f350405c79b241e19a9229dff252cff7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:55.894314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2253165e-7e55-489d-81d0-6f45140ce1f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f350405c79b241e19a9229dff252cff7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:55.894946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-2253165e-7e55-489d-81d0-6f45140ce1f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f350405c79b241e19a9229dff252cff7'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:55.913180 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4234/8498] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:41:55 2022] DELETE /identity/v3/users/f350405c79b241e19a9229dff252cff7 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:41:55.918416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425b1d3-481c-4aeb-8611-f1441fde5dd6 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:55.918928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425b1d3-481c-4aeb-8611-f1441fde5dd6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:55.919047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1425b1d3-481c-4aeb-8611-f1441fde5dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:55.936544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1425b1d3-481c-4aeb-8611-f1441fde5dd6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:41:55.937748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1425b1d3-481c-4aeb-8611-f1441fde5dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:55.997001 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4265/8499] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:41:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:41:56.006858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253165e-7e55-489d-81d0-6f45140ce1f9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:56.008321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2253165e-7e55-489d-81d0-6f45140ce1f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:56.068699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253165e-7e55-489d-81d0-6f45140ce1f9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:56.072531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1694ac5-d587-42aa-9799-aa8bee9f00e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:56.073240 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1694ac5-d587-42aa-9799-aa8bee9f00e1 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:56.073532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1694ac5-d587-42aa-9799-aa8bee9f00e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:56.073813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1694ac5-d587-42aa-9799-aa8bee9f00e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:56.083247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1694ac5-d587-42aa-9799-aa8bee9f00e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:56.084692 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1694ac5-d587-42aa-9799-aa8bee9f00e1 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:56.094559 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4235/8500] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:41:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:41:56.330325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425b1d3-481c-4aeb-8611-f1441fde5dd6 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:56.331785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1425b1d3-481c-4aeb-8611-f1441fde5dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:56.344964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1425b1d3-481c-4aeb-8611-f1441fde5dd6 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:56.350144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb524ad-26e6-4ce7-b7f9-d44aec500ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:56.355114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb524ad-26e6-4ce7-b7f9-d44aec500ef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:56.355114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb524ad-26e6-4ce7-b7f9-d44aec500ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:56.355114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb524ad-26e6-4ce7-b7f9-d44aec500ef4 admin admin] PATH_INFO: `/v3/projects/ed1ea4f454ee4cba84a5e5cc9b5a60e1` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:56.355114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb524ad-26e6-4ce7-b7f9-d44aec500ef4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed1ea4f454ee4cba84a5e5cc9b5a60e1)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:56.356735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb524ad-26e6-4ce7-b7f9-d44aec500ef4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:56.358348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:56.473763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3eb524ad-26e6-4ce7-b7f9-d44aec500ef4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed1ea4f454ee4cba84a5e5cc9b5a60e1', 'request_id': 'req-3eb524ad-26e6-4ce7-b7f9-d44aec500ef4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:56.479366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3eb524ad-26e6-4ce7-b7f9-d44aec500ef4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed1ea4f454ee4cba84a5e5cc9b5a60e1', 'request_id': 'req-3eb524ad-26e6-4ce7-b7f9-d44aec500ef4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:56.547317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3eb524ad-26e6-4ce7-b7f9-d44aec500ef4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed1ea4f454ee4cba84a5e5cc9b5a60e1', 'request_id': 'req-3eb524ad-26e6-4ce7-b7f9-d44aec500ef4'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:56.558926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3eb524ad-26e6-4ce7-b7f9-d44aec500ef4 admin admin] The token cache is being invalidate because project ed1ea4f454ee4cba84a5e5cc9b5a60e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:41:56.559217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3eb524ad-26e6-4ce7-b7f9-d44aec500ef4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:56.559860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3eb524ad-26e6-4ce7-b7f9-d44aec500ef4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:41:56.561402 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4266/8501] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:41:56 2022] DELETE /identity/v3/projects/ed1ea4f454ee4cba84a5e5cc9b5a60e1 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:41:56.566647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329690c-9c85-4a83-8f2a-e46169826d83 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:56.567123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329690c-9c85-4a83-8f2a-e46169826d83 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:56.571153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329690c-9c85-4a83-8f2a-e46169826d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:56.585372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d329690c-9c85-4a83-8f2a-e46169826d83 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:41:56.586601 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d329690c-9c85-4a83-8f2a-e46169826d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:56.604373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:56.647986 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4236/8502] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:41:56.656624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128a7b7-616c-4ca9-8291-c7a9b0d08dd0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:56.656944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128a7b7-616c-4ca9-8291-c7a9b0d08dd0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:56.657228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128a7b7-616c-4ca9-8291-c7a9b0d08dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:56.676595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9128a7b7-616c-4ca9-8291-c7a9b0d08dd0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:41:56.677820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9128a7b7-616c-4ca9-8291-c7a9b0d08dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:56.686504 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4267/8503] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:41:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:41:56.690549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329690c-9c85-4a83-8f2a-e46169826d83 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:56.700383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329690c-9c85-4a83-8f2a-e46169826d83 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:56.705099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996449a-441d-4f42-baa1-d6df635c27ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:56.705754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996449a-441d-4f42-baa1-d6df635c27ea admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:56.706070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996449a-441d-4f42-baa1-d6df635c27ea admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:56.706352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996449a-441d-4f42-baa1-d6df635c27ea admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:56.707078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996449a-441d-4f42-baa1-d6df635c27ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:41:56.707386 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996449a-441d-4f42-baa1-d6df635c27ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:56.708992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9996449a-441d-4f42-baa1-d6df635c27ea admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:56.721408 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4237/8504] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:41:56.724915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128a7b7-616c-4ca9-8291-c7a9b0d08dd0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:56.734799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128a7b7-616c-4ca9-8291-c7a9b0d08dd0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:56.739505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d2517-508a-4d95-8a7b-8460e8627010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:56.740149 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d2517-508a-4d95-8a7b-8460e8627010 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:56.740436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d2517-508a-4d95-8a7b-8460e8627010 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:56.740713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d2517-508a-4d95-8a7b-8460e8627010 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:56.741492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d2517-508a-4d95-8a7b-8460e8627010 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:56.743984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d2517-508a-4d95-8a7b-8460e8627010 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:56.811814 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4268/8505] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:41:56 2022] POST /identity/v3/projects => generated 381 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:41:56.815863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996449a-441d-4f42-baa1-d6df635c27ea admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:56.826511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996449a-441d-4f42-baa1-d6df635c27ea admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:56.883346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afe8f1-cada-44e2-a9c0-9b7a67ad34bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:56.883949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afe8f1-cada-44e2-a9c0-9b7a67ad34bd admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:56.884237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afe8f1-cada-44e2-a9c0-9b7a67ad34bd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:56.884524 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04afe8f1-cada-44e2-a9c0-9b7a67ad34bd admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:56.885204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04afe8f1-cada-44e2-a9c0-9b7a67ad34bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:56.887207 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04afe8f1-cada-44e2-a9c0-9b7a67ad34bd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:56.962827 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4238/8506] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:41:56 2022] POST /identity/v3/users => generated 291 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:41:56.969148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d2517-508a-4d95-8a7b-8460e8627010 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:56.980379 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d2517-508a-4d95-8a7b-8460e8627010 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:56.985240 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1bb8d-be59-48e5-a8d6-08969033637a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:56.985983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1bb8d-be59-48e5-a8d6-08969033637a admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:56.986272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1bb8d-be59-48e5-a8d6-08969033637a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:56.986555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1bb8d-be59-48e5-a8d6-08969033637a admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:56.987324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1bb8d-be59-48e5-a8d6-08969033637a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:56.988992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1bb8d-be59-48e5-a8d6-08969033637a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:57.000075 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4269/8507] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:41:57.003965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afe8f1-cada-44e2-a9c0-9b7a67ad34bd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:57.014206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04afe8f1-cada-44e2-a9c0-9b7a67ad34bd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:57.018740 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acf4bd6-3ae2-4726-9184-2ff53816b1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:57.019388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf4bd6-3ae2-4726-9184-2ff53816b1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:57.019673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf4bd6-3ae2-4726-9184-2ff53816b1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:57.019950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acf4bd6-3ae2-4726-9184-2ff53816b1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:57.020670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf4bd6-3ae2-4726-9184-2ff53816b1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:57.022419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acf4bd6-3ae2-4726-9184-2ff53816b1a5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:57.033431 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4239/8508] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:41:57.036883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1bb8d-be59-48e5-a8d6-08969033637a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:57.047644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1bb8d-be59-48e5-a8d6-08969033637a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:57.052300 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565ddb5-7454-4d2e-a7c3-4037cc2911f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:57.053183 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565ddb5-7454-4d2e-a7c3-4037cc2911f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:57.053473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565ddb5-7454-4d2e-a7c3-4037cc2911f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:57.053756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565ddb5-7454-4d2e-a7c3-4037cc2911f2 admin admin] PATH_INFO: `/v3/projects/a8d10175bdec4b54ad9993436291040b/users/6955d215d3974fcf964bd56a664c1770/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:57.068924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565ddb5-7454-4d2e-a7c3-4037cc2911f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d10175bdec4b54ad9993436291040b, user_id=6955d215d3974fcf964bd56a664c1770, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:57.071692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565ddb5-7454-4d2e-a7c3-4037cc2911f2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:57.182541 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4270/8509] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:41:57 2022] PUT /identity/v3/projects/a8d10175bdec4b54ad9993436291040b/users/6955d215d3974fcf964bd56a664c1770/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:41:57.193099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064961bb-b33a-4525-8638-d899a874ce82 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:57.193389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064961bb-b33a-4525-8638-d899a874ce82 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:57.193665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064961bb-b33a-4525-8638-d899a874ce82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:57.220794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-064961bb-b33a-4525-8638-d899a874ce82 None None] MFA Rules not processed for user `6955d215d3974fcf964bd56a664c1770`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:41:57.222021 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-064961bb-b33a-4525-8638-d899a874ce82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:57.286686 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4240/8510] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:41:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:41:57.296620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565ddb5-7454-4d2e-a7c3-4037cc2911f2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:41:57.298348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c565ddb5-7454-4d2e-a7c3-4037cc2911f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:41:57.359672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565ddb5-7454-4d2e-a7c3-4037cc2911f2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:41:57.363372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18e12dc-d2eb-432f-b50a-106b1fecb3f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:41:57.364023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e12dc-d2eb-432f-b50a-106b1fecb3f5 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:41:57.364305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e12dc-d2eb-432f-b50a-106b1fecb3f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:41:57.364583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e12dc-d2eb-432f-b50a-106b1fecb3f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:41:57.373781 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e12dc-d2eb-432f-b50a-106b1fecb3f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:41:57.375250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18e12dc-d2eb-432f-b50a-106b1fecb3f5 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:41:57.384421 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4271/8511] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:41:57 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:42:07.283710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:07.380086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:16.084505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064961bb-b33a-4525-8638-d899a874ce82 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:16.148693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064961bb-b33a-4525-8638-d899a874ce82 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:16.154010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19126a-d907-4990-82e6-8a887f4afc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:16.154989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19126a-d907-4990-82e6-8a887f4afc51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:16.156424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19126a-d907-4990-82e6-8a887f4afc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:16.156424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19126a-d907-4990-82e6-8a887f4afc51 admin admin] PATH_INFO: `/v3/users/6955d215d3974fcf964bd56a664c1770` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:16.156709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19126a-d907-4990-82e6-8a887f4afc51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6955d215d3974fcf964bd56a664c1770)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:16.158786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19126a-d907-4990-82e6-8a887f4afc51 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:16.332567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6f19126a-d907-4990-82e6-8a887f4afc51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6955d215d3974fcf964bd56a664c1770'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:16.333237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6f19126a-d907-4990-82e6-8a887f4afc51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6955d215d3974fcf964bd56a664c1770'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:16.341060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6f19126a-d907-4990-82e6-8a887f4afc51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6955d215d3974fcf964bd56a664c1770'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:16.386962 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6f19126a-d907-4990-82e6-8a887f4afc51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6955d215d3974fcf964bd56a664c1770'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:16.387574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6f19126a-d907-4990-82e6-8a887f4afc51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6955d215d3974fcf964bd56a664c1770'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:16.405558 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4241/8512] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:42:16 2022] DELETE /identity/v3/users/6955d215d3974fcf964bd56a664c1770 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:16.410249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54565912-6b86-47e8-931f-2c2a8990f3de None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:16.410736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54565912-6b86-47e8-931f-2c2a8990f3de None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:16.410846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54565912-6b86-47e8-931f-2c2a8990f3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:16.427964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-54565912-6b86-47e8-931f-2c2a8990f3de None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:16.429159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-54565912-6b86-47e8-931f-2c2a8990f3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:16.488159 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4272/8513] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:16.498632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19126a-d907-4990-82e6-8a887f4afc51 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:16.500070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6f19126a-d907-4990-82e6-8a887f4afc51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:16.560891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19126a-d907-4990-82e6-8a887f4afc51 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:16.564546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936c7d84-73a0-43ec-af6d-eec0e5c156c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:16.565215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c7d84-73a0-43ec-af6d-eec0e5c156c7 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:16.565497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c7d84-73a0-43ec-af6d-eec0e5c156c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:16.565775 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936c7d84-73a0-43ec-af6d-eec0e5c156c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:16.575515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c7d84-73a0-43ec-af6d-eec0e5c156c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:16.576961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936c7d84-73a0-43ec-af6d-eec0e5c156c7 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:16.586674 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4242/8514] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:42:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:42:16.829506 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54565912-6b86-47e8-931f-2c2a8990f3de None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:16.831095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-54565912-6b86-47e8-931f-2c2a8990f3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:16.850132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54565912-6b86-47e8-931f-2c2a8990f3de None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:16.850541 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18dfee8-fdc7-406c-9414-e67edf70e30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:16.851602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18dfee8-fdc7-406c-9414-e67edf70e30a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:16.851899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18dfee8-fdc7-406c-9414-e67edf70e30a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:16.852170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18dfee8-fdc7-406c-9414-e67edf70e30a admin admin] PATH_INFO: `/v3/projects/a8d10175bdec4b54ad9993436291040b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:16.853936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18dfee8-fdc7-406c-9414-e67edf70e30a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8d10175bdec4b54ad9993436291040b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:16.855895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18dfee8-fdc7-406c-9414-e67edf70e30a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:16.992649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b18dfee8-fdc7-406c-9414-e67edf70e30a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8d10175bdec4b54ad9993436291040b', 'request_id': 'req-b18dfee8-fdc7-406c-9414-e67edf70e30a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:16.998992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b18dfee8-fdc7-406c-9414-e67edf70e30a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8d10175bdec4b54ad9993436291040b', 'request_id': 'req-b18dfee8-fdc7-406c-9414-e67edf70e30a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:17.076935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b18dfee8-fdc7-406c-9414-e67edf70e30a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8d10175bdec4b54ad9993436291040b', 'request_id': 'req-b18dfee8-fdc7-406c-9414-e67edf70e30a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:17.089136 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b18dfee8-fdc7-406c-9414-e67edf70e30a admin admin] The token cache is being invalidate because project a8d10175bdec4b54ad9993436291040b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:42:17.089375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b18dfee8-fdc7-406c-9414-e67edf70e30a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:17.090045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-b18dfee8-fdc7-406c-9414-e67edf70e30a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:17.091868 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4273/8515] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:42:16 2022] DELETE /identity/v3/projects/a8d10175bdec4b54ad9993436291040b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:17.096885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f40a4-e89d-42e0-80f5-a3da1ce09c6d None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:17.097381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f40a4-e89d-42e0-80f5-a3da1ce09c6d None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:17.097381 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90f40a4-e89d-42e0-80f5-a3da1ce09c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:17.115655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d90f40a4-e89d-42e0-80f5-a3da1ce09c6d None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:17.116863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d90f40a4-e89d-42e0-80f5-a3da1ce09c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:17.134624 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:17.176859 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4243/8516] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:17.185905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb844960-1680-4d29-a504-a47e26390ceb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:17.186202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb844960-1680-4d29-a504-a47e26390ceb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:17.186496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb844960-1680-4d29-a504-a47e26390ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:17.204842 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-eb844960-1680-4d29-a504-a47e26390ceb None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:17.206089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eb844960-1680-4d29-a504-a47e26390ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:17.215026 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4274/8517] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:17.219478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f40a4-e89d-42e0-80f5-a3da1ce09c6d None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:17.230158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90f40a4-e89d-42e0-80f5-a3da1ce09c6d None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:17.235010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dc86e-ab16-43cd-a8c2-f6bb827816e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:17.235665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dc86e-ab16-43cd-a8c2-f6bb827816e6 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:17.238795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dc86e-ab16-43cd-a8c2-f6bb827816e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:17.238795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dc86e-ab16-43cd-a8c2-f6bb827816e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:17.238795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dc86e-ab16-43cd-a8c2-f6bb827816e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:42:17.238795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dc86e-ab16-43cd-a8c2-f6bb827816e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:17.239185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dc86e-ab16-43cd-a8c2-f6bb827816e6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:17.250878 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4244/8518] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:42:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:42:17.254913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb844960-1680-4d29-a504-a47e26390ceb None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:17.264381 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb844960-1680-4d29-a504-a47e26390ceb None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:17.269080 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33a7fd-2a9c-404f-8bea-599023dac074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:17.269739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33a7fd-2a9c-404f-8bea-599023dac074 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:17.270072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33a7fd-2a9c-404f-8bea-599023dac074 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:17.270358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33a7fd-2a9c-404f-8bea-599023dac074 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:17.271054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c33a7fd-2a9c-404f-8bea-599023dac074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:17.272977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c33a7fd-2a9c-404f-8bea-599023dac074 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:17.387351 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4275/8519] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:42:17 2022] POST /identity/v3/projects => generated 405 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:42:17.391580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dc86e-ab16-43cd-a8c2-f6bb827816e6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:17.402788 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dc86e-ab16-43cd-a8c2-f6bb827816e6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:17.460824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade2412-753c-45d8-9702-406f559992f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:17.461546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade2412-753c-45d8-9702-406f559992f1 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:17.461850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade2412-753c-45d8-9702-406f559992f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:17.462138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade2412-753c-45d8-9702-406f559992f1 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:17.463612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade2412-753c-45d8-9702-406f559992f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:17.465566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade2412-753c-45d8-9702-406f559992f1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:17.609070 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4245/8520] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:42:17 2022] POST /identity/v3/users => generated 303 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:42:17.613356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33a7fd-2a9c-404f-8bea-599023dac074 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:17.623960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33a7fd-2a9c-404f-8bea-599023dac074 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:17.628892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48e0af-8c2b-4ac2-b922-b03bbe42abdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:17.632516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48e0af-8c2b-4ac2-b922-b03bbe42abdb admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:17.632516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48e0af-8c2b-4ac2-b922-b03bbe42abdb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:17.632516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48e0af-8c2b-4ac2-b922-b03bbe42abdb admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:17.632516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48e0af-8c2b-4ac2-b922-b03bbe42abdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:17.632920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48e0af-8c2b-4ac2-b922-b03bbe42abdb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:17.643858 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4276/8521] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:42:17.648022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade2412-753c-45d8-9702-406f559992f1 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:17.658778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade2412-753c-45d8-9702-406f559992f1 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:17.663470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716afb2-1f37-4b20-81fa-1c24651cb6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:17.664201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716afb2-1f37-4b20-81fa-1c24651cb6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:17.664486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716afb2-1f37-4b20-81fa-1c24651cb6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:17.664833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716afb2-1f37-4b20-81fa-1c24651cb6d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:17.665592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716afb2-1f37-4b20-81fa-1c24651cb6d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:17.667282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716afb2-1f37-4b20-81fa-1c24651cb6d8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:17.677148 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4246/8522] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:42:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:42:17.681312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48e0af-8c2b-4ac2-b922-b03bbe42abdb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:17.691978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48e0af-8c2b-4ac2-b922-b03bbe42abdb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:17.697118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108b5aa-0d1e-413b-a357-1574c4b670b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:17.698685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108b5aa-0d1e-413b-a357-1574c4b670b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:17.698965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108b5aa-0d1e-413b-a357-1574c4b670b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:17.699268 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108b5aa-0d1e-413b-a357-1574c4b670b2 admin admin] PATH_INFO: `/v3/projects/0b8b282648974880bfff431ea1f90d87/users/3a991095f6cf4a32949c22ad83f424ed/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:17.715768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108b5aa-0d1e-413b-a357-1574c4b670b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8b282648974880bfff431ea1f90d87, user_id=3a991095f6cf4a32949c22ad83f424ed, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:17.718368 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108b5aa-0d1e-413b-a357-1574c4b670b2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:17.797943 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4277/8523] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:42:17 2022] PUT /identity/v3/projects/0b8b282648974880bfff431ea1f90d87/users/3a991095f6cf4a32949c22ad83f424ed/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:17.806934 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65e110-2040-4da1-b7ce-899fff54b648 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:17.807228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65e110-2040-4da1-b7ce-899fff54b648 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:17.807510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65e110-2040-4da1-b7ce-899fff54b648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:17.826198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5a65e110-2040-4da1-b7ce-899fff54b648 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:17.830133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5a65e110-2040-4da1-b7ce-899fff54b648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:17.888050 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4247/8524] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:17.898003 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108b5aa-0d1e-413b-a357-1574c4b670b2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:17.899686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3108b5aa-0d1e-413b-a357-1574c4b670b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:17.922200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:17.960936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108b5aa-0d1e-413b-a357-1574c4b670b2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:17.964989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4583e-81d6-4f6f-bdcc-050d32f4705c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:17.965649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4583e-81d6-4f6f-bdcc-050d32f4705c service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:17.965973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4583e-81d6-4f6f-bdcc-050d32f4705c service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:17.966262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4583e-81d6-4f6f-bdcc-050d32f4705c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:17.975447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4583e-81d6-4f6f-bdcc-050d32f4705c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:17.976896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4583e-81d6-4f6f-bdcc-050d32f4705c service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:17.986376 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4278/8525] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:42:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:42:19.348344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260d78f-1b24-4d78-9cc0-f7a179613794 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:19.348800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260d78f-1b24-4d78-9cc0-f7a179613794 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:19.348800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260d78f-1b24-4d78-9cc0-f7a179613794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:19.375429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6260d78f-1b24-4d78-9cc0-f7a179613794 None None] MFA Rules not processed for user `3a991095f6cf4a32949c22ad83f424ed`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:19.376666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6260d78f-1b24-4d78-9cc0-f7a179613794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:19.394466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:19.445622 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4248/8526] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:19 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:19.450859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4583e-81d6-4f6f-bdcc-050d32f4705c service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:19.461236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4583e-81d6-4f6f-bdcc-050d32f4705c service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:19.466797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b573f-b375-40a9-a517-8fc89dc63e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:19.467465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b573f-b375-40a9-a517-8fc89dc63e4f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:19.467788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b573f-b375-40a9-a517-8fc89dc63e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:19.468071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b573f-b375-40a9-a517-8fc89dc63e4f admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:19.468785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b573f-b375-40a9-a517-8fc89dc63e4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:19.470756 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b573f-b375-40a9-a517-8fc89dc63e4f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:19.582736 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4279/8527] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:42:19 2022] POST /identity/v3/projects => generated 405 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:42:19.586695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260d78f-1b24-4d78-9cc0-f7a179613794 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:19.598455 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260d78f-1b24-4d78-9cc0-f7a179613794 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:19.656319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da391ef-1ff9-4cdb-b03e-42c7776e65f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:19.656997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da391ef-1ff9-4cdb-b03e-42c7776e65f7 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:19.657289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da391ef-1ff9-4cdb-b03e-42c7776e65f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:19.657568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da391ef-1ff9-4cdb-b03e-42c7776e65f7 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:19.658314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da391ef-1ff9-4cdb-b03e-42c7776e65f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:19.660271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da391ef-1ff9-4cdb-b03e-42c7776e65f7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:19.798261 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4249/8528] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:42:19 2022] POST /identity/v3/users => generated 309 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:42:19.802346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b573f-b375-40a9-a517-8fc89dc63e4f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:19.813470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b573f-b375-40a9-a517-8fc89dc63e4f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:19.818070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaea9c5-1a54-4f7c-9eb3-17f38f44d8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:19.818727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaea9c5-1a54-4f7c-9eb3-17f38f44d8a9 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:19.819007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaea9c5-1a54-4f7c-9eb3-17f38f44d8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:19.819282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaea9c5-1a54-4f7c-9eb3-17f38f44d8a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:19.820008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaea9c5-1a54-4f7c-9eb3-17f38f44d8a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:19.821600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaea9c5-1a54-4f7c-9eb3-17f38f44d8a9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:19.832140 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4280/8529] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:42:19.836243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da391ef-1ff9-4cdb-b03e-42c7776e65f7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:19.846765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da391ef-1ff9-4cdb-b03e-42c7776e65f7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:19.851606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bf5b4-b3e0-4313-b234-1f950ad6c659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:19.852505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bf5b4-b3e0-4313-b234-1f950ad6c659 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:19.852787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bf5b4-b3e0-4313-b234-1f950ad6c659 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:19.853065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bf5b4-b3e0-4313-b234-1f950ad6c659 admin admin] PATH_INFO: `/v3/projects/07497726790d417fb8c005951ee94383/users/b1a21c82376546f9baa611501306aa25/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:19.868232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bf5b4-b3e0-4313-b234-1f950ad6c659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07497726790d417fb8c005951ee94383, user_id=b1a21c82376546f9baa611501306aa25, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:19.871722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bf5b4-b3e0-4313-b234-1f950ad6c659 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:19.963581 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4250/8530] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:42:19 2022] PUT /identity/v3/projects/07497726790d417fb8c005951ee94383/users/b1a21c82376546f9baa611501306aa25/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:19.967576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaea9c5-1a54-4f7c-9eb3-17f38f44d8a9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:19.978144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaea9c5-1a54-4f7c-9eb3-17f38f44d8a9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:20.033971 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ad75c4-d9d2-423a-a51b-9bd3536be0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:20.034642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad75c4-d9d2-423a-a51b-9bd3536be0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:20.034935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad75c4-d9d2-423a-a51b-9bd3536be0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:20.035217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad75c4-d9d2-423a-a51b-9bd3536be0a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:20.036045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad75c4-d9d2-423a-a51b-9bd3536be0a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:20.037648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ad75c4-d9d2-423a-a51b-9bd3536be0a4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:20.047861 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4281/8531] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:42:20.051969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bf5b4-b3e0-4313-b234-1f950ad6c659 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:20.063588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bf5b4-b3e0-4313-b234-1f950ad6c659 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:20.068480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3024a7-ae47-40f4-9086-ac1b9a32537e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:20.069370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3024a7-ae47-40f4-9086-ac1b9a32537e admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:20.069659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3024a7-ae47-40f4-9086-ac1b9a32537e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:20.069969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3024a7-ae47-40f4-9086-ac1b9a32537e admin admin] PATH_INFO: `/v3/projects/07497726790d417fb8c005951ee94383/users/b1a21c82376546f9baa611501306aa25/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:20.072029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3024a7-ae47-40f4-9086-ac1b9a32537e admin admin] RBAC: Authorizing `identity:create_grant(project_id=07497726790d417fb8c005951ee94383, user_id=b1a21c82376546f9baa611501306aa25, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:20.074826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3024a7-ae47-40f4-9086-ac1b9a32537e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:20.085564 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4251/8532] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:42:20 2022] PUT /identity/v3/projects/07497726790d417fb8c005951ee94383/users/b1a21c82376546f9baa611501306aa25/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:42:21.434308 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ddcf4-45cd-4220-b356-be816c62d0ac None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:21.434881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ddcf4-45cd-4220-b356-be816c62d0ac None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:21.434881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ddcf4-45cd-4220-b356-be816c62d0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:21.463034 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6f8ddcf4-45cd-4220-b356-be816c62d0ac None None] MFA Rules not processed for user `b1a21c82376546f9baa611501306aa25`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:21.465127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6f8ddcf4-45cd-4220-b356-be816c62d0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:21.540345 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4282/8533] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:21 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:23.151253 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3024a7-ae47-40f4-9086-ac1b9a32537e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:23.215765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3024a7-ae47-40f4-9086-ac1b9a32537e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:23.221509 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed479aeb-d1b9-4bc3-bead-92e57ed9d8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:23.222435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed479aeb-d1b9-4bc3-bead-92e57ed9d8ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:23.222718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed479aeb-d1b9-4bc3-bead-92e57ed9d8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:23.223066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed479aeb-d1b9-4bc3-bead-92e57ed9d8ff admin admin] PATH_INFO: `/v3/users/3a991095f6cf4a32949c22ad83f424ed` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:23.224281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed479aeb-d1b9-4bc3-bead-92e57ed9d8ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a991095f6cf4a32949c22ad83f424ed)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:23.226352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed479aeb-d1b9-4bc3-bead-92e57ed9d8ff admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:23.240083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:23.401369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ed479aeb-d1b9-4bc3-bead-92e57ed9d8ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a991095f6cf4a32949c22ad83f424ed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:23.401978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ed479aeb-d1b9-4bc3-bead-92e57ed9d8ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a991095f6cf4a32949c22ad83f424ed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:23.408924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ed479aeb-d1b9-4bc3-bead-92e57ed9d8ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a991095f6cf4a32949c22ad83f424ed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:23.490117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ed479aeb-d1b9-4bc3-bead-92e57ed9d8ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a991095f6cf4a32949c22ad83f424ed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:23.490737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ed479aeb-d1b9-4bc3-bead-92e57ed9d8ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a991095f6cf4a32949c22ad83f424ed'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:23.508492 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4252/8534] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:42:23 2022] DELETE /identity/v3/users/3a991095f6cf4a32949c22ad83f424ed => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:23.764823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ddcf4-45cd-4220-b356-be816c62d0ac None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:23.779259 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ddcf4-45cd-4220-b356-be816c62d0ac None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:23.789218 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:23.845324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a43c83-4cce-41f2-ac75-8e3886d980ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:23.846276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a43c83-4cce-41f2-ac75-8e3886d980ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:23.846569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a43c83-4cce-41f2-ac75-8e3886d980ed admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:23.846845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a43c83-4cce-41f2-ac75-8e3886d980ed admin admin] PATH_INFO: `/v3/projects/0b8b282648974880bfff431ea1f90d87` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:23.848086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a43c83-4cce-41f2-ac75-8e3886d980ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b8b282648974880bfff431ea1f90d87)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:23.850067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a43c83-4cce-41f2-ac75-8e3886d980ed admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:23.953621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-94a43c83-4cce-41f2-ac75-8e3886d980ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b8b282648974880bfff431ea1f90d87', 'request_id': 'req-94a43c83-4cce-41f2-ac75-8e3886d980ed'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:23.959253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-94a43c83-4cce-41f2-ac75-8e3886d980ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b8b282648974880bfff431ea1f90d87', 'request_id': 'req-94a43c83-4cce-41f2-ac75-8e3886d980ed'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:24.025358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-94a43c83-4cce-41f2-ac75-8e3886d980ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b8b282648974880bfff431ea1f90d87', 'request_id': 'req-94a43c83-4cce-41f2-ac75-8e3886d980ed'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:24.036910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-94a43c83-4cce-41f2-ac75-8e3886d980ed admin admin] The token cache is being invalidate because project 0b8b282648974880bfff431ea1f90d87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:42:24.037204 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-94a43c83-4cce-41f2-ac75-8e3886d980ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:24.037896 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-94a43c83-4cce-41f2-ac75-8e3886d980ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:24.039604 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4283/8535] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:42:23 2022] DELETE /identity/v3/projects/0b8b282648974880bfff431ea1f90d87 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:24.043699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed479aeb-d1b9-4bc3-bead-92e57ed9d8ff admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:24.045139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ed479aeb-d1b9-4bc3-bead-92e57ed9d8ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:24.058901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed479aeb-d1b9-4bc3-bead-92e57ed9d8ff admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:24.116952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbf299-81be-4424-b1d7-f7c7ea693a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:24.117882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbf299-81be-4424-b1d7-f7c7ea693a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:24.118178 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbf299-81be-4424-b1d7-f7c7ea693a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:24.118461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbf299-81be-4424-b1d7-f7c7ea693a15 admin admin] PATH_INFO: `/v3/users/b1a21c82376546f9baa611501306aa25` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:24.120193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbf299-81be-4424-b1d7-f7c7ea693a15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1a21c82376546f9baa611501306aa25)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:24.122184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbf299-81be-4424-b1d7-f7c7ea693a15 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:24.332441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-87bbf299-81be-4424-b1d7-f7c7ea693a15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1a21c82376546f9baa611501306aa25'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:24.333092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-87bbf299-81be-4424-b1d7-f7c7ea693a15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1a21c82376546f9baa611501306aa25'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:24.340523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-87bbf299-81be-4424-b1d7-f7c7ea693a15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1a21c82376546f9baa611501306aa25'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:24.581805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-87bbf299-81be-4424-b1d7-f7c7ea693a15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1a21c82376546f9baa611501306aa25'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:24.588491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-87bbf299-81be-4424-b1d7-f7c7ea693a15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1a21c82376546f9baa611501306aa25'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:24.601282 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4253/8536] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:42:24 2022] DELETE /identity/v3/users/b1a21c82376546f9baa611501306aa25 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:24.874168 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a43c83-4cce-41f2-ac75-8e3886d980ed admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:24.875764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-94a43c83-4cce-41f2-ac75-8e3886d980ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:24.889133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a43c83-4cce-41f2-ac75-8e3886d980ed admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:24.949019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ace13-e01b-4f18-9aed-e6be32ee95f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:24.949925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ace13-e01b-4f18-9aed-e6be32ee95f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:24.950217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ace13-e01b-4f18-9aed-e6be32ee95f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:24.950502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ace13-e01b-4f18-9aed-e6be32ee95f5 admin admin] PATH_INFO: `/v3/projects/07497726790d417fb8c005951ee94383` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:24.951708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ace13-e01b-4f18-9aed-e6be32ee95f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07497726790d417fb8c005951ee94383)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:24.953788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ace13-e01b-4f18-9aed-e6be32ee95f5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:25.091897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:25.095861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-876ace13-e01b-4f18-9aed-e6be32ee95f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07497726790d417fb8c005951ee94383', 'request_id': 'req-876ace13-e01b-4f18-9aed-e6be32ee95f5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:25.101287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-876ace13-e01b-4f18-9aed-e6be32ee95f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07497726790d417fb8c005951ee94383', 'request_id': 'req-876ace13-e01b-4f18-9aed-e6be32ee95f5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:25.181288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-876ace13-e01b-4f18-9aed-e6be32ee95f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07497726790d417fb8c005951ee94383', 'request_id': 'req-876ace13-e01b-4f18-9aed-e6be32ee95f5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:25.193442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-876ace13-e01b-4f18-9aed-e6be32ee95f5 admin admin] The token cache is being invalidate because project 07497726790d417fb8c005951ee94383 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:42:25.194626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-876ace13-e01b-4f18-9aed-e6be32ee95f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:25.194626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-876ace13-e01b-4f18-9aed-e6be32ee95f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:25.196028 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4284/8537] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:42:24 2022] DELETE /identity/v3/projects/07497726790d417fb8c005951ee94383 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:25.200914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203c70c-3e56-401c-a5ec-b7f65d76143f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:25.201212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203c70c-3e56-401c-a5ec-b7f65d76143f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:25.201673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203c70c-3e56-401c-a5ec-b7f65d76143f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:25.221864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d203c70c-3e56-401c-a5ec-b7f65d76143f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:25.221864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d203c70c-3e56-401c-a5ec-b7f65d76143f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:25.225354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:25.281339 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4254/8538] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:25.289145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03121afe-f43e-443b-8fa7-1f629221550a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:25.289468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03121afe-f43e-443b-8fa7-1f629221550a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:25.289757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03121afe-f43e-443b-8fa7-1f629221550a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:25.309501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-03121afe-f43e-443b-8fa7-1f629221550a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:25.314195 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-03121afe-f43e-443b-8fa7-1f629221550a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:25.318821 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4285/8539] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:25.322988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203c70c-3e56-401c-a5ec-b7f65d76143f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:25.334005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203c70c-3e56-401c-a5ec-b7f65d76143f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:25.339024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3204bf-3f18-4a38-b704-5680f7d946c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:25.339678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3204bf-3f18-4a38-b704-5680f7d946c0 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:25.340010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3204bf-3f18-4a38-b704-5680f7d946c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:25.340285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3204bf-3f18-4a38-b704-5680f7d946c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:25.341005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3204bf-3f18-4a38-b704-5680f7d946c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:42:25.341359 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3204bf-3f18-4a38-b704-5680f7d946c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:25.343092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3204bf-3f18-4a38-b704-5680f7d946c0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:25.356485 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4255/8540] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:42:25.360636 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03121afe-f43e-443b-8fa7-1f629221550a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:25.370522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03121afe-f43e-443b-8fa7-1f629221550a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:25.375269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5858ce-10cd-429e-b880-0a4d8df6dbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:25.375913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5858ce-10cd-429e-b880-0a4d8df6dbc9 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:25.376193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5858ce-10cd-429e-b880-0a4d8df6dbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:25.376468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5858ce-10cd-429e-b880-0a4d8df6dbc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:25.377177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5858ce-10cd-429e-b880-0a4d8df6dbc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:25.379105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5858ce-10cd-429e-b880-0a4d8df6dbc9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:25.470626 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4286/8541] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:42:25 2022] POST /identity/v3/projects => generated 361 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:42:25.474394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3204bf-3f18-4a38-b704-5680f7d946c0 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:25.485330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3204bf-3f18-4a38-b704-5680f7d946c0 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:25.540333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910ae00-cd6c-4080-b964-d6478319684e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:25.540998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910ae00-cd6c-4080-b964-d6478319684e admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:25.541983 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910ae00-cd6c-4080-b964-d6478319684e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:25.542263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3910ae00-cd6c-4080-b964-d6478319684e admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:25.542941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910ae00-cd6c-4080-b964-d6478319684e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:25.544867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3910ae00-cd6c-4080-b964-d6478319684e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:25.610623 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4256/8542] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:42:25 2022] POST /identity/v3/users => generated 281 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:42:25.614725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5858ce-10cd-429e-b880-0a4d8df6dbc9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:25.625219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5858ce-10cd-429e-b880-0a4d8df6dbc9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:25.630068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfb500-8df0-408e-a7b1-5d27b3d5b9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:25.630740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfb500-8df0-408e-a7b1-5d27b3d5b9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:25.631004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfb500-8df0-408e-a7b1-5d27b3d5b9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:25.631285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cfb500-8df0-408e-a7b1-5d27b3d5b9e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:25.632047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfb500-8df0-408e-a7b1-5d27b3d5b9e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:25.633670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cfb500-8df0-408e-a7b1-5d27b3d5b9e4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:25.644229 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4287/8543] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:42:25.648189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910ae00-cd6c-4080-b964-d6478319684e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:25.659431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3910ae00-cd6c-4080-b964-d6478319684e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:25.664674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75eed9-2ac8-43ae-a8a8-29b61f2bacd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:25.666958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75eed9-2ac8-43ae-a8a8-29b61f2bacd9 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:25.666958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75eed9-2ac8-43ae-a8a8-29b61f2bacd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:25.666958 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75eed9-2ac8-43ae-a8a8-29b61f2bacd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:25.667389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75eed9-2ac8-43ae-a8a8-29b61f2bacd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:25.669044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75eed9-2ac8-43ae-a8a8-29b61f2bacd9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:25.679508 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4257/8544] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:42:25.683645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfb500-8df0-408e-a7b1-5d27b3d5b9e4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:25.693808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cfb500-8df0-408e-a7b1-5d27b3d5b9e4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:25.698526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab40f37-f475-4ed4-ad0a-e9792aeed5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:25.699396 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab40f37-f475-4ed4-ad0a-e9792aeed5a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:25.699681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab40f37-f475-4ed4-ad0a-e9792aeed5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:25.699958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab40f37-f475-4ed4-ad0a-e9792aeed5a8 admin admin] PATH_INFO: `/v3/projects/cdee5829587b413abdb006249452b3b4/users/9004132307744bc89c57958caa4c860b/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:25.714844 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab40f37-f475-4ed4-ad0a-e9792aeed5a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdee5829587b413abdb006249452b3b4, user_id=9004132307744bc89c57958caa4c860b, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:25.717497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab40f37-f475-4ed4-ad0a-e9792aeed5a8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:25.810455 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4288/8545] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:42:25 2022] PUT /identity/v3/projects/cdee5829587b413abdb006249452b3b4/users/9004132307744bc89c57958caa4c860b/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:25.821627 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a21b6-22ba-4a6b-a445-a3f84dff243c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:25.822016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a21b6-22ba-4a6b-a445-a3f84dff243c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:25.822288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a21b6-22ba-4a6b-a445-a3f84dff243c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:25.849942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-5c8a21b6-22ba-4a6b-a445-a3f84dff243c None None] MFA Rules not processed for user `9004132307744bc89c57958caa4c860b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:25.851135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5c8a21b6-22ba-4a6b-a445-a3f84dff243c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:25.916467 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4258/8546] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:25.920998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab40f37-f475-4ed4-ad0a-e9792aeed5a8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:25.984549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab40f37-f475-4ed4-ad0a-e9792aeed5a8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:25.989267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19a687-50b3-40ad-be97-8fb39d5177dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:25.990039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19a687-50b3-40ad-be97-8fb39d5177dc admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:25.990331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19a687-50b3-40ad-be97-8fb39d5177dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:25.990610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e19a687-50b3-40ad-be97-8fb39d5177dc admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:25.991312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19a687-50b3-40ad-be97-8fb39d5177dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:25.993226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e19a687-50b3-40ad-be97-8fb39d5177dc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:26.101884 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4289/8547] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:42:25 2022] POST /identity/v3/projects => generated 365 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:42:26.103058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a21b6-22ba-4a6b-a445-a3f84dff243c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:26.110053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:26.116518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a21b6-22ba-4a6b-a445-a3f84dff243c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:26.173521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5be6a-7ac4-4da6-8d20-1a14e8a96673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:26.174236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5be6a-7ac4-4da6-8d20-1a14e8a96673 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:26.174665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5be6a-7ac4-4da6-8d20-1a14e8a96673 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:26.174808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5be6a-7ac4-4da6-8d20-1a14e8a96673 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:26.175474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5be6a-7ac4-4da6-8d20-1a14e8a96673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:26.177451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5be6a-7ac4-4da6-8d20-1a14e8a96673 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:26.278443 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4259/8548] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:42:26 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:42:26.282470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19a687-50b3-40ad-be97-8fb39d5177dc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:26.293100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e19a687-50b3-40ad-be97-8fb39d5177dc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:26.297739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a5575-9260-4f18-af02-04e0c731497e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:26.298432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a5575-9260-4f18-af02-04e0c731497e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:26.298717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a5575-9260-4f18-af02-04e0c731497e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:26.298996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a5575-9260-4f18-af02-04e0c731497e admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:26.299774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a5575-9260-4f18-af02-04e0c731497e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:26.301548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a5575-9260-4f18-af02-04e0c731497e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:26.312945 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4290/8549] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:42:26.317065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5be6a-7ac4-4da6-8d20-1a14e8a96673 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:26.327944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5be6a-7ac4-4da6-8d20-1a14e8a96673 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:26.332598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957ac62-5975-4e87-a5ef-79e23fe12484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:26.333516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957ac62-5975-4e87-a5ef-79e23fe12484 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:26.333804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957ac62-5975-4e87-a5ef-79e23fe12484 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:26.334117 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957ac62-5975-4e87-a5ef-79e23fe12484 admin admin] PATH_INFO: `/v3/projects/39194e155c7a4248b3c3d0c88e7c5d0f/users/c2978b0702b84003bf3488d51a9793cb/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:26.349290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957ac62-5975-4e87-a5ef-79e23fe12484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39194e155c7a4248b3c3d0c88e7c5d0f, user_id=c2978b0702b84003bf3488d51a9793cb, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:26.352144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957ac62-5975-4e87-a5ef-79e23fe12484 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:26.421430 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4260/8550] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:42:26 2022] PUT /identity/v3/projects/39194e155c7a4248b3c3d0c88e7c5d0f/users/c2978b0702b84003bf3488d51a9793cb/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:42:26.432614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a5575-9260-4f18-af02-04e0c731497e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:26.443054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a5575-9260-4f18-af02-04e0c731497e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:26.483540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:26.500051 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d60903-5cc3-4a6d-981d-16d43c26566b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:26.504920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d60903-5cc3-4a6d-981d-16d43c26566b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:26.504920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d60903-5cc3-4a6d-981d-16d43c26566b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:26.504920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d60903-5cc3-4a6d-981d-16d43c26566b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:26.504920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d60903-5cc3-4a6d-981d-16d43c26566b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:26.504920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d60903-5cc3-4a6d-981d-16d43c26566b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:26.514912 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4291/8551] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:42:26 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:42:26.519098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957ac62-5975-4e87-a5ef-79e23fe12484 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:26.529519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957ac62-5975-4e87-a5ef-79e23fe12484 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:26.534256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542c0e3-3ac2-4be0-8888-bd5011d34147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:26.535136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542c0e3-3ac2-4be0-8888-bd5011d34147 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:26.535425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542c0e3-3ac2-4be0-8888-bd5011d34147 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:26.535704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542c0e3-3ac2-4be0-8888-bd5011d34147 admin admin] PATH_INFO: `/v3/projects/39194e155c7a4248b3c3d0c88e7c5d0f/users/c2978b0702b84003bf3488d51a9793cb/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:26.537735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542c0e3-3ac2-4be0-8888-bd5011d34147 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39194e155c7a4248b3c3d0c88e7c5d0f, user_id=c2978b0702b84003bf3488d51a9793cb, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:26.540484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542c0e3-3ac2-4be0-8888-bd5011d34147 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:26.551455 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4261/8552] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:42:26 2022] PUT /identity/v3/projects/39194e155c7a4248b3c3d0c88e7c5d0f/users/c2978b0702b84003bf3488d51a9793cb/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:26.562173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f8044-5898-4f89-80c7-d5a493f90e7c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:26.562469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f8044-5898-4f89-80c7-d5a493f90e7c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:26.562753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7f8044-5898-4f89-80c7-d5a493f90e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:26.589229 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ef7f8044-5898-4f89-80c7-d5a493f90e7c None None] MFA Rules not processed for user `c2978b0702b84003bf3488d51a9793cb`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:26.590522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ef7f8044-5898-4f89-80c7-d5a493f90e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:26.657719 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4292/8553] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:26.669430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542c0e3-3ac2-4be0-8888-bd5011d34147 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:26.671159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7542c0e3-3ac2-4be0-8888-bd5011d34147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:26.741678 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542c0e3-3ac2-4be0-8888-bd5011d34147 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:26.746061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a8bfe-2114-4424-8b35-c2b749202aeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:26.746730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a8bfe-2114-4424-8b35-c2b749202aeb service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:26.747026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a8bfe-2114-4424-8b35-c2b749202aeb service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:26.747308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a8bfe-2114-4424-8b35-c2b749202aeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:26.756773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a8bfe-2114-4424-8b35-c2b749202aeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:26.761888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a8bfe-2114-4424-8b35-c2b749202aeb service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:26.768246 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4262/8554] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:42:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:42:27.678701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f8044-5898-4f89-80c7-d5a493f90e7c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:27.746922 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7f8044-5898-4f89-80c7-d5a493f90e7c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:27.751582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309b5a8-daed-436f-9dbf-65788bb639c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:27.752450 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309b5a8-daed-436f-9dbf-65788bb639c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:27.752732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309b5a8-daed-436f-9dbf-65788bb639c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:27.753011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309b5a8-daed-436f-9dbf-65788bb639c3 admin admin] PATH_INFO: `/v3/users/9004132307744bc89c57958caa4c860b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:27.754244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309b5a8-daed-436f-9dbf-65788bb639c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9004132307744bc89c57958caa4c860b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:27.756220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309b5a8-daed-436f-9dbf-65788bb639c3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:27.908436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e309b5a8-daed-436f-9dbf-65788bb639c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9004132307744bc89c57958caa4c860b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:27.915253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e309b5a8-daed-436f-9dbf-65788bb639c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9004132307744bc89c57958caa4c860b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:27.915914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e309b5a8-daed-436f-9dbf-65788bb639c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9004132307744bc89c57958caa4c860b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:27.916540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e309b5a8-daed-436f-9dbf-65788bb639c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9004132307744bc89c57958caa4c860b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:27.973998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-e309b5a8-daed-436f-9dbf-65788bb639c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9004132307744bc89c57958caa4c860b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:27.976175 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:27.992002 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4293/8555] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:42:27 2022] DELETE /identity/v3/users/9004132307744bc89c57958caa4c860b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:27.997138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d23d9-4347-41b7-8f04-df771c8baaf9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:28.001984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d23d9-4347-41b7-8f04-df771c8baaf9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:28.001984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d23d9-4347-41b7-8f04-df771c8baaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:28.016260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-163d23d9-4347-41b7-8f04-df771c8baaf9 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:28.017475 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-163d23d9-4347-41b7-8f04-df771c8baaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:28.063735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:28.078093 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4263/8556] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:28.087713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309b5a8-daed-436f-9dbf-65788bb639c3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:28.090325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e309b5a8-daed-436f-9dbf-65788bb639c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:28.148290 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309b5a8-daed-436f-9dbf-65788bb639c3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:28.152016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a6c0c-e517-42a2-9063-d2c1804f5c1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:28.152662 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a6c0c-e517-42a2-9063-d2c1804f5c1a service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:28.152948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a6c0c-e517-42a2-9063-d2c1804f5c1a service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:28.153227 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a6c0c-e517-42a2-9063-d2c1804f5c1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:28.162552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a6c0c-e517-42a2-9063-d2c1804f5c1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:28.163997 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a6c0c-e517-42a2-9063-d2c1804f5c1a service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:28.174538 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4294/8557] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:42:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:42:28.662831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d23d9-4347-41b7-8f04-df771c8baaf9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:28.676030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-163d23d9-4347-41b7-8f04-df771c8baaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:28.677343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d23d9-4347-41b7-8f04-df771c8baaf9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:28.682273 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:28.683204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:28.683469 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:28.683771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c admin admin] PATH_INFO: `/v3/projects/cdee5829587b413abdb006249452b3b4` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:28.684998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdee5829587b413abdb006249452b3b4)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:28.686969 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:28.794687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdee5829587b413abdb006249452b3b4', 'request_id': 'req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:28.800285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdee5829587b413abdb006249452b3b4', 'request_id': 'req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:28.895936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdee5829587b413abdb006249452b3b4', 'request_id': 'req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:28.908234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c admin admin] The token cache is being invalidate because project cdee5829587b413abdb006249452b3b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:42:28.908553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:28.909226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:28.910904 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4264/8558] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:42:28 2022] DELETE /identity/v3/projects/cdee5829587b413abdb006249452b3b4 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:28.914934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a6c0c-e517-42a2-9063-d2c1804f5c1a service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:28.924614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a6c0c-e517-42a2-9063-d2c1804f5c1a service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:28.983260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e08cf5-e81b-4d94-a593-a6452831d7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:28.984179 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e08cf5-e81b-4d94-a593-a6452831d7c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:28.984465 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e08cf5-e81b-4d94-a593-a6452831d7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:28.984744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e08cf5-e81b-4d94-a593-a6452831d7c2 admin admin] PATH_INFO: `/v3/users/c2978b0702b84003bf3488d51a9793cb` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:28.986099 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e08cf5-e81b-4d94-a593-a6452831d7c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2978b0702b84003bf3488d51a9793cb)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:28.988071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e08cf5-e81b-4d94-a593-a6452831d7c2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:29.194206 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f4e08cf5-e81b-4d94-a593-a6452831d7c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2978b0702b84003bf3488d51a9793cb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:29.202202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f4e08cf5-e81b-4d94-a593-a6452831d7c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2978b0702b84003bf3488d51a9793cb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:29.202906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f4e08cf5-e81b-4d94-a593-a6452831d7c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2978b0702b84003bf3488d51a9793cb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:29.207323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f4e08cf5-e81b-4d94-a593-a6452831d7c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2978b0702b84003bf3488d51a9793cb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:29.278948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-f4e08cf5-e81b-4d94-a593-a6452831d7c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2978b0702b84003bf3488d51a9793cb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:29.297467 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4295/8559] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:42:28 2022] DELETE /identity/v3/users/c2978b0702b84003bf3488d51a9793cb => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:29.724122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:29.725561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:29.740460 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8552d9f-dd1a-4851-a8c7-e0c6ec7af93c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:29.799686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15817258-5bdd-4389-adf8-b798eb958f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:29.800558 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15817258-5bdd-4389-adf8-b798eb958f6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:29.800842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15817258-5bdd-4389-adf8-b798eb958f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:29.801119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15817258-5bdd-4389-adf8-b798eb958f6a admin admin] PATH_INFO: `/v3/projects/39194e155c7a4248b3c3d0c88e7c5d0f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:29.802343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15817258-5bdd-4389-adf8-b798eb958f6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=39194e155c7a4248b3c3d0c88e7c5d0f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:29.804436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15817258-5bdd-4389-adf8-b798eb958f6a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:29.895664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-15817258-5bdd-4389-adf8-b798eb958f6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39194e155c7a4248b3c3d0c88e7c5d0f', 'request_id': 'req-15817258-5bdd-4389-adf8-b798eb958f6a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:29.901168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-15817258-5bdd-4389-adf8-b798eb958f6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39194e155c7a4248b3c3d0c88e7c5d0f', 'request_id': 'req-15817258-5bdd-4389-adf8-b798eb958f6a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:29.969771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-15817258-5bdd-4389-adf8-b798eb958f6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39194e155c7a4248b3c3d0c88e7c5d0f', 'request_id': 'req-15817258-5bdd-4389-adf8-b798eb958f6a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:29.981889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-15817258-5bdd-4389-adf8-b798eb958f6a admin admin] The token cache is being invalidate because project 39194e155c7a4248b3c3d0c88e7c5d0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:42:29.982176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-15817258-5bdd-4389-adf8-b798eb958f6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:29.982814 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-15817258-5bdd-4389-adf8-b798eb958f6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:29.984398 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4265/8560] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:42:29 2022] DELETE /identity/v3/projects/39194e155c7a4248b3c3d0c88e7c5d0f => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:29.990549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4117b0-1bd7-40ba-b04b-653200953e65 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:29.991037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4117b0-1bd7-40ba-b04b-653200953e65 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:29.991160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4117b0-1bd7-40ba-b04b-653200953e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:29.998515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:30.009342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-ae4117b0-1bd7-40ba-b04b-653200953e65 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:30.010640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ae4117b0-1bd7-40ba-b04b-653200953e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:30.072421 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4296/8561] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:30.081331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35181672-520e-4fb1-bd8b-64cabbe43340 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:30.081702 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35181672-520e-4fb1-bd8b-64cabbe43340 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:30.082084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35181672-520e-4fb1-bd8b-64cabbe43340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:30.090471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:30.101616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-35181672-520e-4fb1-bd8b-64cabbe43340 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:30.102852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-35181672-520e-4fb1-bd8b-64cabbe43340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:30.111548 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4266/8562] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:30.115589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4117b0-1bd7-40ba-b04b-653200953e65 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:30.125165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4117b0-1bd7-40ba-b04b-653200953e65 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:30.129684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c66719-04a1-48b5-b796-5cab16f6ee8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:30.130398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c66719-04a1-48b5-b796-5cab16f6ee8c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:30.130682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c66719-04a1-48b5-b796-5cab16f6ee8c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:30.130981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c66719-04a1-48b5-b796-5cab16f6ee8c admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:30.131706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c66719-04a1-48b5-b796-5cab16f6ee8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:42:30.132016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c66719-04a1-48b5-b796-5cab16f6ee8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:30.133620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c66719-04a1-48b5-b796-5cab16f6ee8c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:30.145469 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4297/8563] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:42:30.149596 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35181672-520e-4fb1-bd8b-64cabbe43340 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:30.159315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35181672-520e-4fb1-bd8b-64cabbe43340 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:30.164159 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a45cef-1eb0-430e-8632-1470ca2642cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:30.167248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a45cef-1eb0-430e-8632-1470ca2642cc admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:30.167248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a45cef-1eb0-430e-8632-1470ca2642cc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:30.167248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a45cef-1eb0-430e-8632-1470ca2642cc admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:30.167248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a45cef-1eb0-430e-8632-1470ca2642cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:30.168684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a45cef-1eb0-430e-8632-1470ca2642cc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:30.243741 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4267/8564] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:42:30 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:42:30.248088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c66719-04a1-48b5-b796-5cab16f6ee8c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:30.258724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c66719-04a1-48b5-b796-5cab16f6ee8c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:30.316766 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847ab61-2d4d-43f2-b780-72373d5688f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:30.317437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847ab61-2d4d-43f2-b780-72373d5688f7 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:30.317726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847ab61-2d4d-43f2-b780-72373d5688f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:30.318040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847ab61-2d4d-43f2-b780-72373d5688f7 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:30.318753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847ab61-2d4d-43f2-b780-72373d5688f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:30.320674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e847ab61-2d4d-43f2-b780-72373d5688f7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:30.421563 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4298/8565] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:42:30 2022] POST /identity/v3/users => generated 294 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:42:30.425705 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a45cef-1eb0-430e-8632-1470ca2642cc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:30.436009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a45cef-1eb0-430e-8632-1470ca2642cc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:30.440707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7412b-c7ba-45be-800e-48d3ae71c323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:30.442099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7412b-c7ba-45be-800e-48d3ae71c323 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:30.442099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7412b-c7ba-45be-800e-48d3ae71c323 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:30.442099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7412b-c7ba-45be-800e-48d3ae71c323 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:30.442802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7412b-c7ba-45be-800e-48d3ae71c323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:30.444399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7412b-c7ba-45be-800e-48d3ae71c323 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:30.455286 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4268/8566] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:42:30.459248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847ab61-2d4d-43f2-b780-72373d5688f7 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:30.471205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847ab61-2d4d-43f2-b780-72373d5688f7 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:30.476128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169422c-e738-4357-b183-14ee7554fa91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:30.476796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169422c-e738-4357-b183-14ee7554fa91 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:30.477087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169422c-e738-4357-b183-14ee7554fa91 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:30.477484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169422c-e738-4357-b183-14ee7554fa91 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:30.478263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169422c-e738-4357-b183-14ee7554fa91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:30.479872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169422c-e738-4357-b183-14ee7554fa91 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:30.489894 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4299/8567] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:42:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:42:30.493897 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7412b-c7ba-45be-800e-48d3ae71c323 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:30.504012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7412b-c7ba-45be-800e-48d3ae71c323 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:30.508728 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d508353-c58c-4783-bb6c-58f1bee0a7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:30.509593 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d508353-c58c-4783-bb6c-58f1bee0a7b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:30.509952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d508353-c58c-4783-bb6c-58f1bee0a7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:30.510743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d508353-c58c-4783-bb6c-58f1bee0a7b9 admin admin] PATH_INFO: `/v3/projects/03a42f3a81c04a2fb9ea15f1427b081c/users/15572513e234483ea5fef09966d04dd9/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:30.525500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d508353-c58c-4783-bb6c-58f1bee0a7b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03a42f3a81c04a2fb9ea15f1427b081c, user_id=15572513e234483ea5fef09966d04dd9, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:30.528883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d508353-c58c-4783-bb6c-58f1bee0a7b9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:30.609930 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4269/8568] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:42:30 2022] PUT /identity/v3/projects/03a42f3a81c04a2fb9ea15f1427b081c/users/15572513e234483ea5fef09966d04dd9/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:30.625203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb73ce9-0170-4fab-8f70-adcea5234e7f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:30.625496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb73ce9-0170-4fab-8f70-adcea5234e7f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:30.625776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb73ce9-0170-4fab-8f70-adcea5234e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:30.653859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-bdb73ce9-0170-4fab-8f70-adcea5234e7f None None] MFA Rules not processed for user `15572513e234483ea5fef09966d04dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:30.655121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bdb73ce9-0170-4fab-8f70-adcea5234e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:30.716202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:30.719738 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4300/8569] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:30.724429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d508353-c58c-4783-bb6c-58f1bee0a7b9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:30.789335 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d508353-c58c-4783-bb6c-58f1bee0a7b9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:30.794126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f009d6f-52cb-4624-bee9-0e80a1c1f7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:30.794783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f009d6f-52cb-4624-bee9-0e80a1c1f7c3 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:30.795068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f009d6f-52cb-4624-bee9-0e80a1c1f7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:30.795350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f009d6f-52cb-4624-bee9-0e80a1c1f7c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:30.796044 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f009d6f-52cb-4624-bee9-0e80a1c1f7c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:30.798041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f009d6f-52cb-4624-bee9-0e80a1c1f7c3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:30.913646 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4270/8570] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:42:30 2022] POST /identity/v3/projects => generated 383 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:42:30.917715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb73ce9-0170-4fab-8f70-adcea5234e7f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:30.928683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb73ce9-0170-4fab-8f70-adcea5234e7f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:30.985649 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a81c38-7844-4687-9b2a-a38b384eaa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:30.986261 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a81c38-7844-4687-9b2a-a38b384eaa44 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:30.986544 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a81c38-7844-4687-9b2a-a38b384eaa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:30.986823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a81c38-7844-4687-9b2a-a38b384eaa44 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:30.987540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a81c38-7844-4687-9b2a-a38b384eaa44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:30.989468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a81c38-7844-4687-9b2a-a38b384eaa44 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:31.095115 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4301/8571] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:42:30 2022] POST /identity/v3/users => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:42:31.098853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f009d6f-52cb-4624-bee9-0e80a1c1f7c3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:31.109717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f009d6f-52cb-4624-bee9-0e80a1c1f7c3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:31.114510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22334d-c537-44b3-aa6c-e3dd8ff8b4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:31.115337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22334d-c537-44b3-aa6c-e3dd8ff8b4f7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:31.116229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22334d-c537-44b3-aa6c-e3dd8ff8b4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:31.116229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a22334d-c537-44b3-aa6c-e3dd8ff8b4f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:31.116642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22334d-c537-44b3-aa6c-e3dd8ff8b4f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:31.118318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a22334d-c537-44b3-aa6c-e3dd8ff8b4f7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:31.129813 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4271/8572] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:42:31.133870 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a81c38-7844-4687-9b2a-a38b384eaa44 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:31.144531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a81c38-7844-4687-9b2a-a38b384eaa44 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:31.149255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f87ec-c615-4ca8-b7de-bb470e67d938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:31.150181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f87ec-c615-4ca8-b7de-bb470e67d938 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:31.150470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f87ec-c615-4ca8-b7de-bb470e67d938 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:31.150812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f87ec-c615-4ca8-b7de-bb470e67d938 admin admin] PATH_INFO: `/v3/projects/412e778a92bc43429d8a62ec518d76a5/users/d072de34a4ce462786b1682a54b89cc3/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:31.166087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f87ec-c615-4ca8-b7de-bb470e67d938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=412e778a92bc43429d8a62ec518d76a5, user_id=d072de34a4ce462786b1682a54b89cc3, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:31.169375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f87ec-c615-4ca8-b7de-bb470e67d938 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:31.256553 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4302/8573] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:42:31 2022] PUT /identity/v3/projects/412e778a92bc43429d8a62ec518d76a5/users/d072de34a4ce462786b1682a54b89cc3/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:31.260545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22334d-c537-44b3-aa6c-e3dd8ff8b4f7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:31.270853 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a22334d-c537-44b3-aa6c-e3dd8ff8b4f7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:31.328505 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288af432-626d-480e-a9e4-1ecb5b2a6353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:31.329224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288af432-626d-480e-a9e4-1ecb5b2a6353 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:31.329526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288af432-626d-480e-a9e4-1ecb5b2a6353 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:31.329811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288af432-626d-480e-a9e4-1ecb5b2a6353 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:31.330584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288af432-626d-480e-a9e4-1ecb5b2a6353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:31.332277 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288af432-626d-480e-a9e4-1ecb5b2a6353 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:31.342594 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4272/8574] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:42:31 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:42:31.346722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f87ec-c615-4ca8-b7de-bb470e67d938 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:31.358311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f87ec-c615-4ca8-b7de-bb470e67d938 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:31.363029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb45c3-300e-473a-b839-00aa192dc2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:31.363918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb45c3-300e-473a-b839-00aa192dc2c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:31.364207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb45c3-300e-473a-b839-00aa192dc2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:31.364488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb45c3-300e-473a-b839-00aa192dc2c9 admin admin] PATH_INFO: `/v3/projects/412e778a92bc43429d8a62ec518d76a5/users/d072de34a4ce462786b1682a54b89cc3/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:31.366792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb45c3-300e-473a-b839-00aa192dc2c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=412e778a92bc43429d8a62ec518d76a5, user_id=d072de34a4ce462786b1682a54b89cc3, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:31.369647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb45c3-300e-473a-b839-00aa192dc2c9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:31.380692 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4303/8575] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:42:31 2022] PUT /identity/v3/projects/412e778a92bc43429d8a62ec518d76a5/users/d072de34a4ce462786b1682a54b89cc3/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:31.390924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899787f-41f1-4766-97e1-f5c31d80e77e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:31.391325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899787f-41f1-4766-97e1-f5c31d80e77e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:31.391613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899787f-41f1-4766-97e1-f5c31d80e77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:31.410413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:31.418619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9899787f-41f1-4766-97e1-f5c31d80e77e None None] MFA Rules not processed for user `d072de34a4ce462786b1682a54b89cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:31.419886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9899787f-41f1-4766-97e1-f5c31d80e77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:31.488534 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4273/8576] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:31.499384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb45c3-300e-473a-b839-00aa192dc2c9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:31.501205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8edb45c3-300e-473a-b839-00aa192dc2c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:31.561217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb45c3-300e-473a-b839-00aa192dc2c9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:31.564846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a05add-112d-48e0-89d3-7327b55e8fc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:31.565610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a05add-112d-48e0-89d3-7327b55e8fc6 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:31.565927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a05add-112d-48e0-89d3-7327b55e8fc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:31.570737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a05add-112d-48e0-89d3-7327b55e8fc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:31.575264 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a05add-112d-48e0-89d3-7327b55e8fc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:31.576697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a05add-112d-48e0-89d3-7327b55e8fc6 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:31.586405 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4304/8577] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:42:31 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:42:31.719736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899787f-41f1-4766-97e1-f5c31d80e77e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:31.782830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9899787f-41f1-4766-97e1-f5c31d80e77e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:31.787609 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612d2a4-afba-408c-a7f3-ef56f93046b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:31.788485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612d2a4-afba-408c-a7f3-ef56f93046b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:31.788770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612d2a4-afba-408c-a7f3-ef56f93046b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:31.789046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3612d2a4-afba-408c-a7f3-ef56f93046b3 admin admin] PATH_INFO: `/v3/users/15572513e234483ea5fef09966d04dd9` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:31.790269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612d2a4-afba-408c-a7f3-ef56f93046b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15572513e234483ea5fef09966d04dd9)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:31.792899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3612d2a4-afba-408c-a7f3-ef56f93046b3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:31.961415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3612d2a4-afba-408c-a7f3-ef56f93046b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15572513e234483ea5fef09966d04dd9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:31.962039 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3612d2a4-afba-408c-a7f3-ef56f93046b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15572513e234483ea5fef09966d04dd9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:31.970057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3612d2a4-afba-408c-a7f3-ef56f93046b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15572513e234483ea5fef09966d04dd9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:32.015778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3612d2a4-afba-408c-a7f3-ef56f93046b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15572513e234483ea5fef09966d04dd9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:32.016442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-3612d2a4-afba-408c-a7f3-ef56f93046b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15572513e234483ea5fef09966d04dd9'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:32.034364 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4274/8578] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:42:31 2022] DELETE /identity/v3/users/15572513e234483ea5fef09966d04dd9 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:32.038986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711e0d5-3d57-454c-9b97-d16cc3ac17d1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:32.039376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711e0d5-3d57-454c-9b97-d16cc3ac17d1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:32.039794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2711e0d5-3d57-454c-9b97-d16cc3ac17d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:32.057944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2711e0d5-3d57-454c-9b97-d16cc3ac17d1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:32.059158 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2711e0d5-3d57-454c-9b97-d16cc3ac17d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:32.109461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:32.120795 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4305/8579] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:32.138428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612d2a4-afba-408c-a7f3-ef56f93046b3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:32.138428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3612d2a4-afba-408c-a7f3-ef56f93046b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:32.151536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:32.192626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3612d2a4-afba-408c-a7f3-ef56f93046b3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:32.196409 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a80bb3-a8ca-441c-9899-72ae76a6eaf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:32.197115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a80bb3-a8ca-441c-9899-72ae76a6eaf6 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:32.197436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a80bb3-a8ca-441c-9899-72ae76a6eaf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:32.198361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a80bb3-a8ca-441c-9899-72ae76a6eaf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:32.208251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a80bb3-a8ca-441c-9899-72ae76a6eaf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:32.209736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a80bb3-a8ca-441c-9899-72ae76a6eaf6 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:32.219428 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4275/8580] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:42:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:42:32.612501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711e0d5-3d57-454c-9b97-d16cc3ac17d1 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:32.614097 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2711e0d5-3d57-454c-9b97-d16cc3ac17d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:32.628348 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2711e0d5-3d57-454c-9b97-d16cc3ac17d1 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:32.632025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:32.632929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:32.633215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:32.633494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33 admin admin] PATH_INFO: `/v3/projects/03a42f3a81c04a2fb9ea15f1427b081c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:32.634734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03a42f3a81c04a2fb9ea15f1427b081c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:32.636715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:32.776715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03a42f3a81c04a2fb9ea15f1427b081c', 'request_id': 'req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:32.782526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03a42f3a81c04a2fb9ea15f1427b081c', 'request_id': 'req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:32.859307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03a42f3a81c04a2fb9ea15f1427b081c', 'request_id': 'req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:32.871815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33 admin admin] The token cache is being invalidate because project 03a42f3a81c04a2fb9ea15f1427b081c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:42:32.872102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:32.872776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:32.874402 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4306/8581] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:42:32 2022] DELETE /identity/v3/projects/03a42f3a81c04a2fb9ea15f1427b081c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:32.878699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a80bb3-a8ca-441c-9899-72ae76a6eaf6 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:32.888766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a80bb3-a8ca-441c-9899-72ae76a6eaf6 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:32.945988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbe691-07f3-4c62-bae3-6d14c903ea2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:32.946859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbe691-07f3-4c62-bae3-6d14c903ea2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:32.947141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbe691-07f3-4c62-bae3-6d14c903ea2d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:32.947417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbe691-07f3-4c62-bae3-6d14c903ea2d admin admin] PATH_INFO: `/v3/users/d072de34a4ce462786b1682a54b89cc3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:32.948592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbe691-07f3-4c62-bae3-6d14c903ea2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d072de34a4ce462786b1682a54b89cc3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:32.950642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbe691-07f3-4c62-bae3-6d14c903ea2d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:33.165051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c7dbe691-07f3-4c62-bae3-6d14c903ea2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd072de34a4ce462786b1682a54b89cc3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:33.165707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c7dbe691-07f3-4c62-bae3-6d14c903ea2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd072de34a4ce462786b1682a54b89cc3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:33.173413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c7dbe691-07f3-4c62-bae3-6d14c903ea2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd072de34a4ce462786b1682a54b89cc3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:33.238783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c7dbe691-07f3-4c62-bae3-6d14c903ea2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd072de34a4ce462786b1682a54b89cc3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:33.239431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c7dbe691-07f3-4c62-bae3-6d14c903ea2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd072de34a4ce462786b1682a54b89cc3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:33.257767 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4276/8582] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:42:32 2022] DELETE /identity/v3/users/d072de34a4ce462786b1682a54b89cc3 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:33.652296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:33.653793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:33.669694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d3a05-4ec0-49c3-853d-082afa5e7a33 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:33.730164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea65f2-9142-44d8-8b9d-cac6885fcd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:33.731029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea65f2-9142-44d8-8b9d-cac6885fcd0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:33.731316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea65f2-9142-44d8-8b9d-cac6885fcd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:33.731597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea65f2-9142-44d8-8b9d-cac6885fcd0a admin admin] PATH_INFO: `/v3/projects/412e778a92bc43429d8a62ec518d76a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:33.732856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea65f2-9142-44d8-8b9d-cac6885fcd0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=412e778a92bc43429d8a62ec518d76a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:33.734812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea65f2-9142-44d8-8b9d-cac6885fcd0a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:33.837507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-48ea65f2-9142-44d8-8b9d-cac6885fcd0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '412e778a92bc43429d8a62ec518d76a5', 'request_id': 'req-48ea65f2-9142-44d8-8b9d-cac6885fcd0a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:33.843322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-48ea65f2-9142-44d8-8b9d-cac6885fcd0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '412e778a92bc43429d8a62ec518d76a5', 'request_id': 'req-48ea65f2-9142-44d8-8b9d-cac6885fcd0a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:33.951315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-48ea65f2-9142-44d8-8b9d-cac6885fcd0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '412e778a92bc43429d8a62ec518d76a5', 'request_id': 'req-48ea65f2-9142-44d8-8b9d-cac6885fcd0a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:33.962873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-48ea65f2-9142-44d8-8b9d-cac6885fcd0a admin admin] The token cache is being invalidate because project 412e778a92bc43429d8a62ec518d76a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:42:33.963433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-48ea65f2-9142-44d8-8b9d-cac6885fcd0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:33.964074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-48ea65f2-9142-44d8-8b9d-cac6885fcd0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:42:33.965638 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4307/8583] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:42:33 2022] DELETE /identity/v3/projects/412e778a92bc43429d8a62ec518d76a5 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:33.970711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebc8f2-d16d-4965-b1e7-c04073ef3e85 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:33.971196 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebc8f2-d16d-4965-b1e7-c04073ef3e85 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:33.971302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ebc8f2-d16d-4965-b1e7-c04073ef3e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:33.989213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-50ebc8f2-d16d-4965-b1e7-c04073ef3e85 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:33.990750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-50ebc8f2-d16d-4965-b1e7-c04073ef3e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:34.023988 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:34.051481 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4277/8584] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:34.060694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a1df9-d49a-418d-b359-a7162c626070 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:34.060993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a1df9-d49a-418d-b359-a7162c626070 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:34.061310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a1df9-d49a-418d-b359-a7162c626070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:34.080130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4f4a1df9-d49a-418d-b359-a7162c626070 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:34.081352 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4f4a1df9-d49a-418d-b359-a7162c626070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:34.089732 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4308/8585] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:34.094087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebc8f2-d16d-4965-b1e7-c04073ef3e85 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:34.103834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ebc8f2-d16d-4965-b1e7-c04073ef3e85 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:34.109022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a245c-9517-4246-9d5b-60146c060287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:34.109679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a245c-9517-4246-9d5b-60146c060287 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:34.109994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a245c-9517-4246-9d5b-60146c060287 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:34.110274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a245c-9517-4246-9d5b-60146c060287 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:34.110999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a245c-9517-4246-9d5b-60146c060287 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:42:34.111351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a245c-9517-4246-9d5b-60146c060287 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:34.112956 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a245c-9517-4246-9d5b-60146c060287 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:34.125104 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4278/8586] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:42:34.129442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a1df9-d49a-418d-b359-a7162c626070 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:34.134061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:34.139449 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a1df9-d49a-418d-b359-a7162c626070 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:34.144252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9a41d-daf9-4a8f-86a7-43a6a0ef1b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:34.144907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9a41d-daf9-4a8f-86a7-43a6a0ef1b02 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:34.145194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9a41d-daf9-4a8f-86a7-43a6a0ef1b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:34.145475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e9a41d-daf9-4a8f-86a7-43a6a0ef1b02 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:34.146222 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9a41d-daf9-4a8f-86a7-43a6a0ef1b02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:34.151342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e9a41d-daf9-4a8f-86a7-43a6a0ef1b02 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:34.344726 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4309/8587] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:42:34 2022] POST /identity/v3/projects => generated 383 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:42:34.348782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a245c-9517-4246-9d5b-60146c060287 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:34.359323 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a245c-9517-4246-9d5b-60146c060287 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:34.415689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b79e27-63d7-4dd5-a8fa-d8cf73c4755a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:34.416857 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b79e27-63d7-4dd5-a8fa-d8cf73c4755a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:34.417151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b79e27-63d7-4dd5-a8fa-d8cf73c4755a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:34.417426 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b79e27-63d7-4dd5-a8fa-d8cf73c4755a admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:34.418160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b79e27-63d7-4dd5-a8fa-d8cf73c4755a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:34.420091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b79e27-63d7-4dd5-a8fa-d8cf73c4755a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:34.501757 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4279/8588] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:42:34 2022] POST /identity/v3/users => generated 292 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:42:34.513390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9a41d-daf9-4a8f-86a7-43a6a0ef1b02 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:34.524265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e9a41d-daf9-4a8f-86a7-43a6a0ef1b02 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:34.529527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c445758-544d-4cbb-b1f8-95fdccd98cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:34.530224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c445758-544d-4cbb-b1f8-95fdccd98cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:34.530515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c445758-544d-4cbb-b1f8-95fdccd98cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:34.530797 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c445758-544d-4cbb-b1f8-95fdccd98cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:34.531532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c445758-544d-4cbb-b1f8-95fdccd98cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:34.533148 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c445758-544d-4cbb-b1f8-95fdccd98cd1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:34.543844 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4310/8589] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:42:34.552932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b79e27-63d7-4dd5-a8fa-d8cf73c4755a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:34.563008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b79e27-63d7-4dd5-a8fa-d8cf73c4755a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:34.567648 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a822f-b9e9-4a01-a314-4f34211616f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:34.568303 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a822f-b9e9-4a01-a314-4f34211616f2 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:34.568592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a822f-b9e9-4a01-a314-4f34211616f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:34.568872 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a822f-b9e9-4a01-a314-4f34211616f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:34.569607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a822f-b9e9-4a01-a314-4f34211616f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:34.571257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a822f-b9e9-4a01-a314-4f34211616f2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:34.581125 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4280/8590] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:42:34.585144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c445758-544d-4cbb-b1f8-95fdccd98cd1 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:34.595898 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c445758-544d-4cbb-b1f8-95fdccd98cd1 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:34.600587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a7b0c-9e17-4ee0-8941-ecb28c832ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:34.601467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a7b0c-9e17-4ee0-8941-ecb28c832ada admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:34.601759 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a7b0c-9e17-4ee0-8941-ecb28c832ada admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:34.602074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a7b0c-9e17-4ee0-8941-ecb28c832ada admin admin] PATH_INFO: `/v3/projects/201bd6cbfd144d5c9ff30c81341773c6/users/a69e3e8e9a7c4eda80a8c110a62511e2/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:34.617630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a7b0c-9e17-4ee0-8941-ecb28c832ada admin admin] RBAC: Authorizing `identity:create_grant(project_id=201bd6cbfd144d5c9ff30c81341773c6, user_id=a69e3e8e9a7c4eda80a8c110a62511e2, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:34.620399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a7b0c-9e17-4ee0-8941-ecb28c832ada admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:34.690512 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4311/8591] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:42:34 2022] PUT /identity/v3/projects/201bd6cbfd144d5c9ff30c81341773c6/users/a69e3e8e9a7c4eda80a8c110a62511e2/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:34.700899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58f232-08b7-40e1-aca1-2dd55695f60b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:34.701197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58f232-08b7-40e1-aca1-2dd55695f60b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:34.701478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58f232-08b7-40e1-aca1-2dd55695f60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:34.728672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7a58f232-08b7-40e1-aca1-2dd55695f60b None None] MFA Rules not processed for user `a69e3e8e9a7c4eda80a8c110a62511e2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:34.730569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7a58f232-08b7-40e1-aca1-2dd55695f60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:34.794406 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4281/8592] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:34.798889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a7b0c-9e17-4ee0-8941-ecb28c832ada admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:34.861653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a7b0c-9e17-4ee0-8941-ecb28c832ada admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:34.866269 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825962f-dbdb-4e09-b65f-75268ce2a04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:34.869200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825962f-dbdb-4e09-b65f-75268ce2a04b admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:34.869200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825962f-dbdb-4e09-b65f-75268ce2a04b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:34.869200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825962f-dbdb-4e09-b65f-75268ce2a04b admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:34.869200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3825962f-dbdb-4e09-b65f-75268ce2a04b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:34.870017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3825962f-dbdb-4e09-b65f-75268ce2a04b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:34.952265 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4312/8593] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:42:34 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:42:34.956449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58f232-08b7-40e1-aca1-2dd55695f60b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:34.968430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58f232-08b7-40e1-aca1-2dd55695f60b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:34.998419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:35.024577 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c2de0-9ce2-4b8e-a15b-ceb66e7c018c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:35.025254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c2de0-9ce2-4b8e-a15b-ceb66e7c018c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:35.025544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c2de0-9ce2-4b8e-a15b-ceb66e7c018c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:35.026482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c2de0-9ce2-4b8e-a15b-ceb66e7c018c admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:35.027211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c2de0-9ce2-4b8e-a15b-ceb66e7c018c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:35.029183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c2de0-9ce2-4b8e-a15b-ceb66e7c018c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:35.123213 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4282/8594] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:42:34 2022] POST /identity/v3/users => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:42:35.127122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825962f-dbdb-4e09-b65f-75268ce2a04b admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:35.138265 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825962f-dbdb-4e09-b65f-75268ce2a04b admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:35.142925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6bde8-4cc5-418b-a3b4-f8eeca559079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:35.143708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6bde8-4cc5-418b-a3b4-f8eeca559079 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:35.143993 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6bde8-4cc5-418b-a3b4-f8eeca559079 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:35.144274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6bde8-4cc5-418b-a3b4-f8eeca559079 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:35.145026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6bde8-4cc5-418b-a3b4-f8eeca559079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:35.146666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6bde8-4cc5-418b-a3b4-f8eeca559079 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:35.157958 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4313/8595] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:42:35.161971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c2de0-9ce2-4b8e-a15b-ceb66e7c018c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:35.172343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c2de0-9ce2-4b8e-a15b-ceb66e7c018c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:35.177035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea592ff-5958-48d9-87df-e83fcd36be27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:35.177986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea592ff-5958-48d9-87df-e83fcd36be27 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:35.178276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea592ff-5958-48d9-87df-e83fcd36be27 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:35.178549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea592ff-5958-48d9-87df-e83fcd36be27 admin admin] PATH_INFO: `/v3/projects/7dd500c4a23c420a95cde6576edf7c5a/users/d6736b0b6d724f5fa2dab643eea2f8db/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:35.194497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea592ff-5958-48d9-87df-e83fcd36be27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dd500c4a23c420a95cde6576edf7c5a, user_id=d6736b0b6d724f5fa2dab643eea2f8db, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:35.197227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea592ff-5958-48d9-87df-e83fcd36be27 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:35.294712 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4283/8596] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:42:35 2022] PUT /identity/v3/projects/7dd500c4a23c420a95cde6576edf7c5a/users/d6736b0b6d724f5fa2dab643eea2f8db/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:35.298663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6bde8-4cc5-418b-a3b4-f8eeca559079 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:35.309260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6bde8-4cc5-418b-a3b4-f8eeca559079 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:35.365740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a660d-2a0e-49e5-b570-72a7cead28e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:35.366443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a660d-2a0e-49e5-b570-72a7cead28e1 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:35.371767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a660d-2a0e-49e5-b570-72a7cead28e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:35.371767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a660d-2a0e-49e5-b570-72a7cead28e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:35.371767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a660d-2a0e-49e5-b570-72a7cead28e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:35.371767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a660d-2a0e-49e5-b570-72a7cead28e1 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:35.379588 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4314/8597] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:42:35.383712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea592ff-5958-48d9-87df-e83fcd36be27 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:35.395187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea592ff-5958-48d9-87df-e83fcd36be27 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:35.399801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eeaefd-e6c8-409b-8bdb-c1ec4b8c91b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:35.400696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eeaefd-e6c8-409b-8bdb-c1ec4b8c91b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:35.402345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eeaefd-e6c8-409b-8bdb-c1ec4b8c91b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:35.402345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eeaefd-e6c8-409b-8bdb-c1ec4b8c91b4 admin admin] PATH_INFO: `/v3/projects/7dd500c4a23c420a95cde6576edf7c5a/users/d6736b0b6d724f5fa2dab643eea2f8db/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:35.403353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eeaefd-e6c8-409b-8bdb-c1ec4b8c91b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dd500c4a23c420a95cde6576edf7c5a, user_id=d6736b0b6d724f5fa2dab643eea2f8db, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:35.406327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eeaefd-e6c8-409b-8bdb-c1ec4b8c91b4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:35.417045 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4284/8598] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:42:35 2022] PUT /identity/v3/projects/7dd500c4a23c420a95cde6576edf7c5a/users/d6736b0b6d724f5fa2dab643eea2f8db/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:42:35.428112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f20e5-61eb-411f-a364-be2b39501b88 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:35.428409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f20e5-61eb-411f-a364-be2b39501b88 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:35.428693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606f20e5-61eb-411f-a364-be2b39501b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:35.456083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-606f20e5-61eb-411f-a364-be2b39501b88 None None] MFA Rules not processed for user `d6736b0b6d724f5fa2dab643eea2f8db`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:42:35.457293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-606f20e5-61eb-411f-a364-be2b39501b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:35.461245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:35.526646 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4315/8599] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:42:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:42:35.538739 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eeaefd-e6c8-409b-8bdb-c1ec4b8c91b4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:35.540435 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e1eeaefd-e6c8-409b-8bdb-c1ec4b8c91b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:35.600486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eeaefd-e6c8-409b-8bdb-c1ec4b8c91b4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:35.605048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d202d92-650c-424e-aee6-af4b01d7ecb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:35.606003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d202d92-650c-424e-aee6-af4b01d7ecb6 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:35.606286 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d202d92-650c-424e-aee6-af4b01d7ecb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:35.606588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d202d92-650c-424e-aee6-af4b01d7ecb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:35.662798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d202d92-650c-424e-aee6-af4b01d7ecb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:35.664170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d202d92-650c-424e-aee6-af4b01d7ecb6 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:35.673770 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4285/8600] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:42:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:42:45.547232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=27 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:45.669704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:47.439297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f20e5-61eb-411f-a364-be2b39501b88 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:42:47.440726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-606f20e5-61eb-411f-a364-be2b39501b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:42:47.467944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606f20e5-61eb-411f-a364-be2b39501b88 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:42:47.471764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afabc6d-e050-44e9-9825-48772350d1d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:42:47.472412 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afabc6d-e050-44e9-9825-48772350d1d3 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:42:47.472701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afabc6d-e050-44e9-9825-48772350d1d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:42:47.472985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afabc6d-e050-44e9-9825-48772350d1d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:42:47.482761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afabc6d-e050-44e9-9825-48772350d1d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:42:47.484203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afabc6d-e050-44e9-9825-48772350d1d3 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:42:47.493993 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4316/8601] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:42:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:42:57.489565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=11 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:43:44.098352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d202d92-650c-424e-aee6-af4b01d7ecb6 service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:44.194313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d202d92-650c-424e-aee6-af4b01d7ecb6 service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:44.199230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36450c-6136-4699-8283-1330daa3c5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:44.200104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36450c-6136-4699-8283-1330daa3c5c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:44.200383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36450c-6136-4699-8283-1330daa3c5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:44.200655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae36450c-6136-4699-8283-1330daa3c5c2 admin admin] PATH_INFO: `/v3/users/a69e3e8e9a7c4eda80a8c110a62511e2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:44.201880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae36450c-6136-4699-8283-1330daa3c5c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a69e3e8e9a7c4eda80a8c110a62511e2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:44.203810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae36450c-6136-4699-8283-1330daa3c5c2 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:44.381935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ae36450c-6136-4699-8283-1330daa3c5c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a69e3e8e9a7c4eda80a8c110a62511e2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:44.382555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ae36450c-6136-4699-8283-1330daa3c5c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a69e3e8e9a7c4eda80a8c110a62511e2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:44.389579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ae36450c-6136-4699-8283-1330daa3c5c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a69e3e8e9a7c4eda80a8c110a62511e2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:44.454173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ae36450c-6136-4699-8283-1330daa3c5c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a69e3e8e9a7c4eda80a8c110a62511e2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:44.454793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ae36450c-6136-4699-8283-1330daa3c5c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a69e3e8e9a7c4eda80a8c110a62511e2'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:44.472929 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4286/8602] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:43:44 2022] DELETE /identity/v3/users/a69e3e8e9a7c4eda80a8c110a62511e2 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:43:44.478189 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7be1ac-68e7-4c55-8a78-04c2f582f8b0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:44.478697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7be1ac-68e7-4c55-8a78-04c2f582f8b0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:44.478825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7be1ac-68e7-4c55-8a78-04c2f582f8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:44.528726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9a7be1ac-68e7-4c55-8a78-04c2f582f8b0 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:43:44.529983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9a7be1ac-68e7-4c55-8a78-04c2f582f8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:43:44.589269 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4317/8603] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:43:44.599703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36450c-6136-4699-8283-1330daa3c5c2 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:44.601147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ae36450c-6136-4699-8283-1330daa3c5c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:43:44.654794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:43:44.689721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae36450c-6136-4699-8283-1330daa3c5c2 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:44.693823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6f015-be5e-4eaf-aa32-787a06c7f1a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:44.694502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6f015-be5e-4eaf-aa32-787a06c7f1a6 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:44.694789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6f015-be5e-4eaf-aa32-787a06c7f1a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:44.695068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6f015-be5e-4eaf-aa32-787a06c7f1a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:44.704751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6f015-be5e-4eaf-aa32-787a06c7f1a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:44.706254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6f015-be5e-4eaf-aa32-787a06c7f1a6 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:44.716559 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4287/8604] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:43:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:43:45.141085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7be1ac-68e7-4c55-8a78-04c2f582f8b0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:45.142553 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9a7be1ac-68e7-4c55-8a78-04c2f582f8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:43:45.154627 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7be1ac-68e7-4c55-8a78-04c2f582f8b0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:45.159040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:45.159904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:45.160194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:45.160764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5 admin admin] PATH_INFO: `/v3/projects/201bd6cbfd144d5c9ff30c81341773c6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:45.166619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=201bd6cbfd144d5c9ff30c81341773c6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:45.166619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:45.282036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '201bd6cbfd144d5c9ff30c81341773c6', 'request_id': 'req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:45.287555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '201bd6cbfd144d5c9ff30c81341773c6', 'request_id': 'req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:45.372980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '201bd6cbfd144d5c9ff30c81341773c6', 'request_id': 'req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:45.384771 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5 admin admin] The token cache is being invalidate because project 201bd6cbfd144d5c9ff30c81341773c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:43:45.385067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:45.385699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:45.387319 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4318/8605] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:43:45 2022] DELETE /identity/v3/projects/201bd6cbfd144d5c9ff30c81341773c6 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:43:45.391419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6f015-be5e-4eaf-aa32-787a06c7f1a6 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:45.401310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6f015-be5e-4eaf-aa32-787a06c7f1a6 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:45.459226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edef098-a976-4427-8737-6b75ecb882f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:45.460007 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edef098-a976-4427-8737-6b75ecb882f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:45.460292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edef098-a976-4427-8737-6b75ecb882f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:45.460569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edef098-a976-4427-8737-6b75ecb882f5 admin admin] PATH_INFO: `/v3/users/d6736b0b6d724f5fa2dab643eea2f8db` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:45.461755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edef098-a976-4427-8737-6b75ecb882f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6736b0b6d724f5fa2dab643eea2f8db)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:45.463993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edef098-a976-4427-8737-6b75ecb882f5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:45.689921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5edef098-a976-4427-8737-6b75ecb882f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6736b0b6d724f5fa2dab643eea2f8db'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:45.690598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5edef098-a976-4427-8737-6b75ecb882f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6736b0b6d724f5fa2dab643eea2f8db'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:45.697508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5edef098-a976-4427-8737-6b75ecb882f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6736b0b6d724f5fa2dab643eea2f8db'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:45.775203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5edef098-a976-4427-8737-6b75ecb882f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6736b0b6d724f5fa2dab643eea2f8db'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:45.775831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5edef098-a976-4427-8737-6b75ecb882f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6736b0b6d724f5fa2dab643eea2f8db'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:45.794413 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4288/8606] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:43:45 2022] DELETE /identity/v3/users/d6736b0b6d724f5fa2dab643eea2f8db => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:43:46.245087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:46.246702 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:43:46.261965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba85aa9-4f65-4900-b096-ea9e112c7ab5 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:46.315698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:43:46.322307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c904f7-f3be-4011-8f2e-c13709468db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:46.323180 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c904f7-f3be-4011-8f2e-c13709468db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:46.323470 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c904f7-f3be-4011-8f2e-c13709468db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:46.323754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c904f7-f3be-4011-8f2e-c13709468db0 admin admin] PATH_INFO: `/v3/projects/7dd500c4a23c420a95cde6576edf7c5a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:46.324960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c904f7-f3be-4011-8f2e-c13709468db0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dd500c4a23c420a95cde6576edf7c5a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:46.326977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c904f7-f3be-4011-8f2e-c13709468db0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:46.435623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-31c904f7-f3be-4011-8f2e-c13709468db0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dd500c4a23c420a95cde6576edf7c5a', 'request_id': 'req-31c904f7-f3be-4011-8f2e-c13709468db0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:46.441651 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-31c904f7-f3be-4011-8f2e-c13709468db0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dd500c4a23c420a95cde6576edf7c5a', 'request_id': 'req-31c904f7-f3be-4011-8f2e-c13709468db0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:46.526945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-31c904f7-f3be-4011-8f2e-c13709468db0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dd500c4a23c420a95cde6576edf7c5a', 'request_id': 'req-31c904f7-f3be-4011-8f2e-c13709468db0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:46.538426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-31c904f7-f3be-4011-8f2e-c13709468db0 admin admin] The token cache is being invalidate because project 7dd500c4a23c420a95cde6576edf7c5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:43:46.539780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-31c904f7-f3be-4011-8f2e-c13709468db0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:46.539780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-31c904f7-f3be-4011-8f2e-c13709468db0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:43:46.541100 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4319/8607] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:43:46 2022] DELETE /identity/v3/projects/7dd500c4a23c420a95cde6576edf7c5a => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:43:46.546852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9443a3a-e6c3-4226-95af-f6a4206a85e8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:46.547361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9443a3a-e6c3-4226-95af-f6a4206a85e8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:46.547473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9443a3a-e6c3-4226-95af-f6a4206a85e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:46.594813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c9443a3a-e6c3-4226-95af-f6a4206a85e8 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:43:46.595938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c9443a3a-e6c3-4226-95af-f6a4206a85e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:43:46.607805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:43:46.654751 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4289/8608] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:43:46.662312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decaec84-4ec9-499c-aa6e-da85a7750075 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:46.662701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decaec84-4ec9-499c-aa6e-da85a7750075 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:46.663005 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decaec84-4ec9-499c-aa6e-da85a7750075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:46.682084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-decaec84-4ec9-499c-aa6e-da85a7750075 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:43:46.683291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-decaec84-4ec9-499c-aa6e-da85a7750075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:43:46.691519 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4320/8609] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:43:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:43:46.697120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9443a3a-e6c3-4226-95af-f6a4206a85e8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:46.706164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9443a3a-e6c3-4226-95af-f6a4206a85e8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:46.710860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4b73c-a549-4c0c-9421-6bf5459ba849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:46.711533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4b73c-a549-4c0c-9421-6bf5459ba849 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:46.711820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4b73c-a549-4c0c-9421-6bf5459ba849 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:46.712099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f4b73c-a549-4c0c-9421-6bf5459ba849 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:46.713109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4b73c-a549-4c0c-9421-6bf5459ba849 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:43:46.713422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4b73c-a549-4c0c-9421-6bf5459ba849 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:46.715135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f4b73c-a549-4c0c-9421-6bf5459ba849 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:46.726495 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4290/8610] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:43:46.730707 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decaec84-4ec9-499c-aa6e-da85a7750075 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:46.740016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decaec84-4ec9-499c-aa6e-da85a7750075 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:46.745322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e2b80-0673-441a-99ac-4fca3118c832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:46.750139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e2b80-0673-441a-99ac-4fca3118c832 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:46.750139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e2b80-0673-441a-99ac-4fca3118c832 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:46.750139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e2b80-0673-441a-99ac-4fca3118c832 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:46.750139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e2b80-0673-441a-99ac-4fca3118c832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:46.750139 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e2b80-0673-441a-99ac-4fca3118c832 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:46.866354 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4321/8611] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:43:46 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:43:46.870807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4b73c-a549-4c0c-9421-6bf5459ba849 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:46.880850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f4b73c-a549-4c0c-9421-6bf5459ba849 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:46.938275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea686ba-b423-4682-96ea-26e39fb571f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:46.939005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea686ba-b423-4682-96ea-26e39fb571f9 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:46.939331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea686ba-b423-4682-96ea-26e39fb571f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:46.939612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea686ba-b423-4682-96ea-26e39fb571f9 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:46.940309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea686ba-b423-4682-96ea-26e39fb571f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:46.942269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea686ba-b423-4682-96ea-26e39fb571f9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:47.037683 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4291/8612] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:43:46 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:43:47.042361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e2b80-0673-441a-99ac-4fca3118c832 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:47.052892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e2b80-0673-441a-99ac-4fca3118c832 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:47.058293 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5232f43-e46a-4eba-a7d9-aa66b7cffa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:47.058952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5232f43-e46a-4eba-a7d9-aa66b7cffa1e admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:47.059237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5232f43-e46a-4eba-a7d9-aa66b7cffa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:47.059517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5232f43-e46a-4eba-a7d9-aa66b7cffa1e admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:47.060249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5232f43-e46a-4eba-a7d9-aa66b7cffa1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:47.061865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5232f43-e46a-4eba-a7d9-aa66b7cffa1e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:47.073260 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4322/8613] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:43:47.077245 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea686ba-b423-4682-96ea-26e39fb571f9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:47.088023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea686ba-b423-4682-96ea-26e39fb571f9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:47.092630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c0d43-9038-4789-8615-1c400841a7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:47.093283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c0d43-9038-4789-8615-1c400841a7da admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:47.093562 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c0d43-9038-4789-8615-1c400841a7da admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:47.093861 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c0d43-9038-4789-8615-1c400841a7da admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:47.094591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c0d43-9038-4789-8615-1c400841a7da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:47.096312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5c0d43-9038-4789-8615-1c400841a7da admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:47.106736 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4292/8614] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:43:47.110746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5232f43-e46a-4eba-a7d9-aa66b7cffa1e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:47.121423 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5232f43-e46a-4eba-a7d9-aa66b7cffa1e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:47.126007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6ca08-8512-4560-a123-c6f6db646aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:47.126879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6ca08-8512-4560-a123-c6f6db646aa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:47.127164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6ca08-8512-4560-a123-c6f6db646aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:47.127444 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb6ca08-8512-4560-a123-c6f6db646aa9 admin admin] PATH_INFO: `/v3/projects/65965326d8ef4ea89a8a4245e147bee6/users/a436e059356f4840ac977fcc5db9d5b3/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:47.141782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6ca08-8512-4560-a123-c6f6db646aa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65965326d8ef4ea89a8a4245e147bee6, user_id=a436e059356f4840ac977fcc5db9d5b3, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:47.144466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb6ca08-8512-4560-a123-c6f6db646aa9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:47.255330 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4323/8615] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:43:47 2022] PUT /identity/v3/projects/65965326d8ef4ea89a8a4245e147bee6/users/a436e059356f4840ac977fcc5db9d5b3/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:43:47.266515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf795115-49aa-409d-a7a0-e1fc84f04572 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:47.266811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf795115-49aa-409d-a7a0-e1fc84f04572 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:47.267091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf795115-49aa-409d-a7a0-e1fc84f04572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:47.294900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bf795115-49aa-409d-a7a0-e1fc84f04572 None None] MFA Rules not processed for user `a436e059356f4840ac977fcc5db9d5b3`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:43:47.296124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bf795115-49aa-409d-a7a0-e1fc84f04572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:43:47.360882 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4293/8616] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:43:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:43:47.365328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6ca08-8512-4560-a123-c6f6db646aa9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:47.427234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb6ca08-8512-4560-a123-c6f6db646aa9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:47.432113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50edb8-13f4-45cf-9e1c-330ae2cad311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:47.432791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50edb8-13f4-45cf-9e1c-330ae2cad311 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:47.433081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50edb8-13f4-45cf-9e1c-330ae2cad311 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:47.433364 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50edb8-13f4-45cf-9e1c-330ae2cad311 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:47.434135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de50edb8-13f4-45cf-9e1c-330ae2cad311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:47.436140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de50edb8-13f4-45cf-9e1c-330ae2cad311 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:47.548259 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4324/8617] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:43:47 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:43:47.552514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf795115-49aa-409d-a7a0-e1fc84f04572 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:47.563264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf795115-49aa-409d-a7a0-e1fc84f04572 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:47.572180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:43:47.620061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1af95-c724-4d25-999c-7dbb12479364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:47.620754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1af95-c724-4d25-999c-7dbb12479364 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:47.621066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1af95-c724-4d25-999c-7dbb12479364 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:47.621340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1af95-c724-4d25-999c-7dbb12479364 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:47.622065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1af95-c724-4d25-999c-7dbb12479364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:47.624027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1af95-c724-4d25-999c-7dbb12479364 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:47.804286 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4294/8618] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:43:47 2022] POST /identity/v3/users => generated 293 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:43:47.808201 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50edb8-13f4-45cf-9e1c-330ae2cad311 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:47.818558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50edb8-13f4-45cf-9e1c-330ae2cad311 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:47.823691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070685a-34cc-42af-afc8-a65d742038bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:47.824349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070685a-34cc-42af-afc8-a65d742038bf admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:47.824633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070685a-34cc-42af-afc8-a65d742038bf admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:47.824910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070685a-34cc-42af-afc8-a65d742038bf admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:47.825747 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2070685a-34cc-42af-afc8-a65d742038bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:47.827385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2070685a-34cc-42af-afc8-a65d742038bf admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:47.829435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:43:47.839365 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4325/8619] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:43:47.848759 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1af95-c724-4d25-999c-7dbb12479364 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:47.853664 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1af95-c724-4d25-999c-7dbb12479364 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:47.858429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3fc98b-b3eb-415d-92d9-e86e974f38ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:47.859084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fc98b-b3eb-415d-92d9-e86e974f38ca admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:47.859375 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fc98b-b3eb-415d-92d9-e86e974f38ca admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:47.859661 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fc98b-b3eb-415d-92d9-e86e974f38ca admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:47.860392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3fc98b-b3eb-415d-92d9-e86e974f38ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:47.862037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3fc98b-b3eb-415d-92d9-e86e974f38ca admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:47.873104 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4295/8620] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:43:47.886581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070685a-34cc-42af-afc8-a65d742038bf admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:47.896694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070685a-34cc-42af-afc8-a65d742038bf admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:47.902519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bc41a-5878-4785-903c-b2e3bd00b06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:47.903394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bc41a-5878-4785-903c-b2e3bd00b06c admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:47.903678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bc41a-5878-4785-903c-b2e3bd00b06c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:47.903960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bc41a-5878-4785-903c-b2e3bd00b06c admin admin] PATH_INFO: `/v3/projects/e7caae5fa76f455ab74a2dfe7526aa74/users/8650bda735b2496cb67e1a020950c86d/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:47.918385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6bc41a-5878-4785-903c-b2e3bd00b06c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7caae5fa76f455ab74a2dfe7526aa74, user_id=8650bda735b2496cb67e1a020950c86d, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:47.921057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6bc41a-5878-4785-903c-b2e3bd00b06c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:47.986543 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4326/8621] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:43:47 2022] PUT /identity/v3/projects/e7caae5fa76f455ab74a2dfe7526aa74/users/8650bda735b2496cb67e1a020950c86d/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:43:47.997289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974b0b7-2876-4ddf-8c7e-2277315f0809 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:47.997594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974b0b7-2876-4ddf-8c7e-2277315f0809 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:47.997924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974b0b7-2876-4ddf-8c7e-2277315f0809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:48.025071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0974b0b7-2876-4ddf-8c7e-2277315f0809 None None] MFA Rules not processed for user `8650bda735b2496cb67e1a020950c86d`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:43:48.026340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0974b0b7-2876-4ddf-8c7e-2277315f0809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:43:48.091421 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4296/8622] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:43:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:43:48.096092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bc41a-5878-4785-903c-b2e3bd00b06c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:48.158840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bc41a-5878-4785-903c-b2e3bd00b06c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:48.163687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af4b0e-d957-4ed9-bed7-075a16b9befd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:48.164350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af4b0e-d957-4ed9-bed7-075a16b9befd admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:48.164642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af4b0e-d957-4ed9-bed7-075a16b9befd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:48.164927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75af4b0e-d957-4ed9-bed7-075a16b9befd admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:48.165618 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af4b0e-d957-4ed9-bed7-075a16b9befd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:48.167628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75af4b0e-d957-4ed9-bed7-075a16b9befd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:48.284182 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4327/8623] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:43:48 2022] POST /identity/v3/projects => generated 385 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:43:48.288310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974b0b7-2876-4ddf-8c7e-2277315f0809 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:48.299258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974b0b7-2876-4ddf-8c7e-2277315f0809 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:48.357060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f379e2f-d82d-40f4-b6da-262f985ccae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:48.357654 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f379e2f-d82d-40f4-b6da-262f985ccae4 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:48.357975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f379e2f-d82d-40f4-b6da-262f985ccae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:48.358257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f379e2f-d82d-40f4-b6da-262f985ccae4 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:48.358954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f379e2f-d82d-40f4-b6da-262f985ccae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:48.360898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f379e2f-d82d-40f4-b6da-262f985ccae4 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:48.458434 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4297/8624] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:43:48 2022] POST /identity/v3/users => generated 299 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:43:48.462850 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af4b0e-d957-4ed9-bed7-075a16b9befd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:48.473583 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75af4b0e-d957-4ed9-bed7-075a16b9befd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:48.479091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a0892-f325-401a-b82e-f14bc1e615c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:48.479755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a0892-f325-401a-b82e-f14bc1e615c3 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:48.480043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a0892-f325-401a-b82e-f14bc1e615c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:48.480333 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a0892-f325-401a-b82e-f14bc1e615c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:48.481062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a0892-f325-401a-b82e-f14bc1e615c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:48.482767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a0892-f325-401a-b82e-f14bc1e615c3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:48.495115 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4328/8625] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:43:48.499137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f379e2f-d82d-40f4-b6da-262f985ccae4 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:48.509143 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f379e2f-d82d-40f4-b6da-262f985ccae4 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:48.513673 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2c95c-6455-4f75-af95-ce3913e0f121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:48.514576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2c95c-6455-4f75-af95-ce3913e0f121 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:48.514862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2c95c-6455-4f75-af95-ce3913e0f121 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:48.515142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2c95c-6455-4f75-af95-ce3913e0f121 admin admin] PATH_INFO: `/v3/projects/278b026261444c54b2f4ef4b0ad09bc5/users/cff617bddf8d4384a68676b0c02f0cb6/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:48.523422 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:43:48.529741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2c95c-6455-4f75-af95-ce3913e0f121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=278b026261444c54b2f4ef4b0ad09bc5, user_id=cff617bddf8d4384a68676b0c02f0cb6, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:48.532443 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d2c95c-6455-4f75-af95-ce3913e0f121 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:48.614369 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4298/8626] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:43:48 2022] PUT /identity/v3/projects/278b026261444c54b2f4ef4b0ad09bc5/users/cff617bddf8d4384a68676b0c02f0cb6/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:43:48.618788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a0892-f325-401a-b82e-f14bc1e615c3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:48.628954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a0892-f325-401a-b82e-f14bc1e615c3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:48.685157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1caa9-81a1-4efa-b1d1-91e2204da94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:48.685828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1caa9-81a1-4efa-b1d1-91e2204da94b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:48.686144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1caa9-81a1-4efa-b1d1-91e2204da94b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:48.686427 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1caa9-81a1-4efa-b1d1-91e2204da94b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:48.687167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1caa9-81a1-4efa-b1d1-91e2204da94b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:48.688775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1caa9-81a1-4efa-b1d1-91e2204da94b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:48.699040 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4329/8627] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:43:48.702982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2c95c-6455-4f75-af95-ce3913e0f121 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:48.714471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2c95c-6455-4f75-af95-ce3913e0f121 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:48.719075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd80714-93b4-4b27-92b7-3f38517a6e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:48.719948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd80714-93b4-4b27-92b7-3f38517a6e5d admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:48.720232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd80714-93b4-4b27-92b7-3f38517a6e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:48.720510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd80714-93b4-4b27-92b7-3f38517a6e5d admin admin] PATH_INFO: `/v3/projects/278b026261444c54b2f4ef4b0ad09bc5/users/cff617bddf8d4384a68676b0c02f0cb6/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:48.722501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd80714-93b4-4b27-92b7-3f38517a6e5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=278b026261444c54b2f4ef4b0ad09bc5, user_id=cff617bddf8d4384a68676b0c02f0cb6, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:48.725160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd80714-93b4-4b27-92b7-3f38517a6e5d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:48.735843 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4299/8628] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:43:48 2022] PUT /identity/v3/projects/278b026261444c54b2f4ef4b0ad09bc5/users/cff617bddf8d4384a68676b0c02f0cb6/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:43:48.746327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e16b19-16fd-4fb4-86b0-5a402ca17b90 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:48.750647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e16b19-16fd-4fb4-86b0-5a402ca17b90 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:48.750647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e16b19-16fd-4fb4-86b0-5a402ca17b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:48.773073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-24e16b19-16fd-4fb4-86b0-5a402ca17b90 None None] MFA Rules not processed for user `cff617bddf8d4384a68676b0c02f0cb6`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:43:48.774820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-24e16b19-16fd-4fb4-86b0-5a402ca17b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:43:48.807071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:43:48.843387 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4330/8629] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:43:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:43:48.855205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd80714-93b4-4b27-92b7-3f38517a6e5d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:48.857266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4bd80714-93b4-4b27-92b7-3f38517a6e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:43:48.919495 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd80714-93b4-4b27-92b7-3f38517a6e5d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:48.923120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a653d9-65cf-4d2d-8bef-2dbf85656c49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:48.923761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a653d9-65cf-4d2d-8bef-2dbf85656c49 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:48.924050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a653d9-65cf-4d2d-8bef-2dbf85656c49 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:48.924329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a653d9-65cf-4d2d-8bef-2dbf85656c49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:48.981472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a653d9-65cf-4d2d-8bef-2dbf85656c49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:48.982957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a653d9-65cf-4d2d-8bef-2dbf85656c49 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:48.991940 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4300/8630] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:43:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:43:52.026901 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e16b19-16fd-4fb4-86b0-5a402ca17b90 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:52.039550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e16b19-16fd-4fb4-86b0-5a402ca17b90 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:52.043188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee690ecb-fc7e-4017-bc37-84ffe5518731 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:52.043918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee690ecb-fc7e-4017-bc37-84ffe5518731 service cinder] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:52.044209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee690ecb-fc7e-4017-bc37-84ffe5518731 service cinder] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:52.044519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee690ecb-fc7e-4017-bc37-84ffe5518731 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:52.102894 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee690ecb-fc7e-4017-bc37-84ffe5518731 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:52.104349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee690ecb-fc7e-4017-bc37-84ffe5518731 service cinder] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:52.113372 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4331/8631] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:43:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:43:53.758880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a653d9-65cf-4d2d-8bef-2dbf85656c49 service cinder] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:43:53.770248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a653d9-65cf-4d2d-8bef-2dbf85656c49 service cinder] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:43:53.774177 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a80321-510e-4169-a831-e4edd52fba54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afc08d15f24e46e29c34d618d955bd28', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:43:53.774829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a80321-510e-4169-a831-e4edd52fba54 service cinder] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:43:53.775118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a80321-510e-4169-a831-e4edd52fba54 service cinder] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:43:53.775398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a80321-510e-4169-a831-e4edd52fba54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:43:53.785230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a80321-510e-4169-a831-e4edd52fba54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:43:53.786709 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a80321-510e-4169-a831-e4edd52fba54 service cinder] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:43:53.797007 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4301/8632] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:43:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:44:02.109303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=42 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:44:03.791985 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=64,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:44:09.084078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee690ecb-fc7e-4017-bc37-84ffe5518731 service cinder] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:09.145737 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee690ecb-fc7e-4017-bc37-84ffe5518731 service cinder] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:09.150403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60bf40-313e-41da-9c11-6f80b1883bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:09.151270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60bf40-313e-41da-9c11-6f80b1883bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:09.151555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60bf40-313e-41da-9c11-6f80b1883bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:09.151862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60bf40-313e-41da-9c11-6f80b1883bf8 admin admin] PATH_INFO: `/v3/users/a436e059356f4840ac977fcc5db9d5b3` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:09.153084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60bf40-313e-41da-9c11-6f80b1883bf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a436e059356f4840ac977fcc5db9d5b3)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:09.155036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60bf40-313e-41da-9c11-6f80b1883bf8 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:09.323170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ac60bf40-313e-41da-9c11-6f80b1883bf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a436e059356f4840ac977fcc5db9d5b3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:09.331283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ac60bf40-313e-41da-9c11-6f80b1883bf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a436e059356f4840ac977fcc5db9d5b3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:09.336073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ac60bf40-313e-41da-9c11-6f80b1883bf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a436e059356f4840ac977fcc5db9d5b3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:09.336073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ac60bf40-313e-41da-9c11-6f80b1883bf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a436e059356f4840ac977fcc5db9d5b3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:09.406688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ac60bf40-313e-41da-9c11-6f80b1883bf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a436e059356f4840ac977fcc5db9d5b3'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:09.426123 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4332/8633] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:44:09 2022] DELETE /identity/v3/users/a436e059356f4840ac977fcc5db9d5b3 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:44:09.430583 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113c11e-c5e9-464f-812f-6fb9e1bea3d1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:09.431051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113c11e-c5e9-464f-812f-6fb9e1bea3d1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:09.431186 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113c11e-c5e9-464f-812f-6fb9e1bea3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:09.448108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8113c11e-c5e9-464f-812f-6fb9e1bea3d1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:44:09.449315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8113c11e-c5e9-464f-812f-6fb9e1bea3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:09.508188 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4302/8634] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:44:09.517891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60bf40-313e-41da-9c11-6f80b1883bf8 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:09.519286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ac60bf40-313e-41da-9c11-6f80b1883bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:09.579405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60bf40-313e-41da-9c11-6f80b1883bf8 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:09.583595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e57399-24c8-4ebc-ba51-e8f5c70d07b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:09.584260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e57399-24c8-4ebc-ba51-e8f5c70d07b7 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:09.584550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e57399-24c8-4ebc-ba51-e8f5c70d07b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:09.584836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e57399-24c8-4ebc-ba51-e8f5c70d07b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:09.595757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e57399-24c8-4ebc-ba51-e8f5c70d07b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:09.600358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e57399-24c8-4ebc-ba51-e8f5c70d07b7 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:09.607883 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4333/8635] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:44:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:44:10.050170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113c11e-c5e9-464f-812f-6fb9e1bea3d1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:10.052016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8113c11e-c5e9-464f-812f-6fb9e1bea3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:10.067369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113c11e-c5e9-464f-812f-6fb9e1bea3d1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:10.072300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:10.073254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:10.073612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:10.074015 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c admin admin] PATH_INFO: `/v3/projects/65965326d8ef4ea89a8a4245e147bee6` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:10.076293 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c admin admin] RBAC: Authorizing `identity:delete_project(project_id=65965326d8ef4ea89a8a4245e147bee6)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:10.078419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:10.180557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65965326d8ef4ea89a8a4245e147bee6', 'request_id': 'req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:10.186112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65965326d8ef4ea89a8a4245e147bee6', 'request_id': 'req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:10.252870 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65965326d8ef4ea89a8a4245e147bee6', 'request_id': 'req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:10.264595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c admin admin] The token cache is being invalidate because project 65965326d8ef4ea89a8a4245e147bee6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:44:10.264886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:10.265513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:10.267761 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4303/8636] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:44:10 2022] DELETE /identity/v3/projects/65965326d8ef4ea89a8a4245e147bee6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:44:10.271775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e57399-24c8-4ebc-ba51-e8f5c70d07b7 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:10.282029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e57399-24c8-4ebc-ba51-e8f5c70d07b7 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:10.301028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:44:10.339810 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca68101-c9b3-4fef-a6e7-5fb811f85dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:10.340687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca68101-c9b3-4fef-a6e7-5fb811f85dbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:10.341035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca68101-c9b3-4fef-a6e7-5fb811f85dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:10.341321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca68101-c9b3-4fef-a6e7-5fb811f85dbb admin admin] PATH_INFO: `/v3/users/8650bda735b2496cb67e1a020950c86d` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:10.342858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca68101-c9b3-4fef-a6e7-5fb811f85dbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8650bda735b2496cb67e1a020950c86d)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:10.348090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca68101-c9b3-4fef-a6e7-5fb811f85dbb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:10.598956 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-aca68101-c9b3-4fef-a6e7-5fb811f85dbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8650bda735b2496cb67e1a020950c86d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:10.606002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-aca68101-c9b3-4fef-a6e7-5fb811f85dbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8650bda735b2496cb67e1a020950c86d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:10.606674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-aca68101-c9b3-4fef-a6e7-5fb811f85dbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8650bda735b2496cb67e1a020950c86d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:10.607313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-aca68101-c9b3-4fef-a6e7-5fb811f85dbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8650bda735b2496cb67e1a020950c86d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:10.688903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-aca68101-c9b3-4fef-a6e7-5fb811f85dbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8650bda735b2496cb67e1a020950c86d'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:10.706639 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4334/8637] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:44:10 2022] DELETE /identity/v3/users/8650bda735b2496cb67e1a020950c86d => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:44:11.135742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:11.137210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:11.151321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea082f2-41ae-4c36-8ecc-02f4b0c6949c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:11.211179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15859a-15b5-4b60-95f1-4e499c4d0690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:11.212050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15859a-15b5-4b60-95f1-4e499c4d0690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:11.212658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15859a-15b5-4b60-95f1-4e499c4d0690 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:11.213019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15859a-15b5-4b60-95f1-4e499c4d0690 admin admin] PATH_INFO: `/v3/projects/e7caae5fa76f455ab74a2dfe7526aa74` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:11.214269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15859a-15b5-4b60-95f1-4e499c4d0690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7caae5fa76f455ab74a2dfe7526aa74)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:11.216203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15859a-15b5-4b60-95f1-4e499c4d0690 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:11.322842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:44:11.339799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0a15859a-15b5-4b60-95f1-4e499c4d0690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7caae5fa76f455ab74a2dfe7526aa74', 'request_id': 'req-0a15859a-15b5-4b60-95f1-4e499c4d0690'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:11.346089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0a15859a-15b5-4b60-95f1-4e499c4d0690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7caae5fa76f455ab74a2dfe7526aa74', 'request_id': 'req-0a15859a-15b5-4b60-95f1-4e499c4d0690'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:11.409164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0a15859a-15b5-4b60-95f1-4e499c4d0690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7caae5fa76f455ab74a2dfe7526aa74', 'request_id': 'req-0a15859a-15b5-4b60-95f1-4e499c4d0690'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:11.421346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0a15859a-15b5-4b60-95f1-4e499c4d0690 admin admin] The token cache is being invalidate because project e7caae5fa76f455ab74a2dfe7526aa74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:44:11.421649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0a15859a-15b5-4b60-95f1-4e499c4d0690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:11.422353 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0a15859a-15b5-4b60-95f1-4e499c4d0690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:11.423925 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4304/8638] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:44:11 2022] DELETE /identity/v3/projects/e7caae5fa76f455ab74a2dfe7526aa74 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:44:11.427958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca68101-c9b3-4fef-a6e7-5fb811f85dbb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:11.429366 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-aca68101-c9b3-4fef-a6e7-5fb811f85dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:11.442645 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca68101-c9b3-4fef-a6e7-5fb811f85dbb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:11.500517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c12ae7-86e7-466a-b2f2-d97824a5facc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:11.501938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c12ae7-86e7-466a-b2f2-d97824a5facc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:11.502228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c12ae7-86e7-466a-b2f2-d97824a5facc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:11.502509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c12ae7-86e7-466a-b2f2-d97824a5facc admin admin] PATH_INFO: `/v3/users/cff617bddf8d4384a68676b0c02f0cb6` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:11.503713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c12ae7-86e7-466a-b2f2-d97824a5facc admin admin] RBAC: Authorizing `identity:delete_user(user_id=cff617bddf8d4384a68676b0c02f0cb6)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:11.505680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c12ae7-86e7-466a-b2f2-d97824a5facc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:11.717546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:44:11.720365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10c12ae7-86e7-466a-b2f2-d97824a5facc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cff617bddf8d4384a68676b0c02f0cb6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:11.727869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10c12ae7-86e7-466a-b2f2-d97824a5facc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cff617bddf8d4384a68676b0c02f0cb6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:11.728600 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10c12ae7-86e7-466a-b2f2-d97824a5facc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cff617bddf8d4384a68676b0c02f0cb6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:11.729239 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10c12ae7-86e7-466a-b2f2-d97824a5facc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cff617bddf8d4384a68676b0c02f0cb6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:11.837156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-10c12ae7-86e7-466a-b2f2-d97824a5facc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cff617bddf8d4384a68676b0c02f0cb6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:11.855182 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4335/8639] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:44:11 2022] DELETE /identity/v3/users/cff617bddf8d4384a68676b0c02f0cb6 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:44:12.313343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15859a-15b5-4b60-95f1-4e499c4d0690 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:12.315600 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0a15859a-15b5-4b60-95f1-4e499c4d0690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:12.328313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15859a-15b5-4b60-95f1-4e499c4d0690 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:12.386482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcfb905-03b8-4c36-b6fe-80057063b37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:12.387346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfb905-03b8-4c36-b6fe-80057063b37e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:12.387679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfb905-03b8-4c36-b6fe-80057063b37e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:12.387939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcfb905-03b8-4c36-b6fe-80057063b37e admin admin] PATH_INFO: `/v3/projects/278b026261444c54b2f4ef4b0ad09bc5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:12.389113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfb905-03b8-4c36-b6fe-80057063b37e admin admin] RBAC: Authorizing `identity:delete_project(project_id=278b026261444c54b2f4ef4b0ad09bc5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:12.391091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcfb905-03b8-4c36-b6fe-80057063b37e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:12.495289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dfcfb905-03b8-4c36-b6fe-80057063b37e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '278b026261444c54b2f4ef4b0ad09bc5', 'request_id': 'req-dfcfb905-03b8-4c36-b6fe-80057063b37e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:12.500798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dfcfb905-03b8-4c36-b6fe-80057063b37e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '278b026261444c54b2f4ef4b0ad09bc5', 'request_id': 'req-dfcfb905-03b8-4c36-b6fe-80057063b37e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:12.566557 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dfcfb905-03b8-4c36-b6fe-80057063b37e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '278b026261444c54b2f4ef4b0ad09bc5', 'request_id': 'req-dfcfb905-03b8-4c36-b6fe-80057063b37e'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:12.578275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dfcfb905-03b8-4c36-b6fe-80057063b37e admin admin] The token cache is being invalidate because project 278b026261444c54b2f4ef4b0ad09bc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:44:12.578636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dfcfb905-03b8-4c36-b6fe-80057063b37e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:12.579774 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-dfcfb905-03b8-4c36-b6fe-80057063b37e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:44:12.581335 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4305/8640] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:44:12 2022] DELETE /identity/v3/projects/278b026261444c54b2f4ef4b0ad09bc5 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:44:12.587879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eaa73a-0d7b-4597-96ed-1f726537e34a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:12.592885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eaa73a-0d7b-4597-96ed-1f726537e34a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:12.592885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eaa73a-0d7b-4597-96ed-1f726537e34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:12.609100 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-74eaa73a-0d7b-4597-96ed-1f726537e34a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:44:12.610377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-74eaa73a-0d7b-4597-96ed-1f726537e34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:12.669111 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4336/8641] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:44:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:44:12.678208 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1c181-1f69-4d1c-93de-c06dc0b67419 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:12.678503 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1c181-1f69-4d1c-93de-c06dc0b67419 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:12.678787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a1c181-1f69-4d1c-93de-c06dc0b67419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:12.697122 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f5a1c181-1f69-4d1c-93de-c06dc0b67419 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:44:12.698424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f5a1c181-1f69-4d1c-93de-c06dc0b67419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:12.706956 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4306/8642] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:44:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:44:12.711473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eaa73a-0d7b-4597-96ed-1f726537e34a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:12.722171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eaa73a-0d7b-4597-96ed-1f726537e34a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:12.726820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e640580-6e7f-4523-a84f-01b8ce661535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:12.727475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e640580-6e7f-4523-a84f-01b8ce661535 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:12.727763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e640580-6e7f-4523-a84f-01b8ce661535 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:12.728045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e640580-6e7f-4523-a84f-01b8ce661535 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:12.728779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e640580-6e7f-4523-a84f-01b8ce661535 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:44:12.729095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e640580-6e7f-4523-a84f-01b8ce661535 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:12.730763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e640580-6e7f-4523-a84f-01b8ce661535 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:12.743042 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4337/8643] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:44:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:44:12.747361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1c181-1f69-4d1c-93de-c06dc0b67419 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:12.762574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a1c181-1f69-4d1c-93de-c06dc0b67419 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:12.767742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967dd6f-3603-4f2d-b30b-0bfb51d829fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:12.768477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967dd6f-3603-4f2d-b30b-0bfb51d829fd admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:12.772906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967dd6f-3603-4f2d-b30b-0bfb51d829fd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:12.772906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0967dd6f-3603-4f2d-b30b-0bfb51d829fd admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:12.772906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967dd6f-3603-4f2d-b30b-0bfb51d829fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:12.772906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0967dd6f-3603-4f2d-b30b-0bfb51d829fd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:12.844185 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4307/8644] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:44:12 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:44:12.849114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e640580-6e7f-4523-a84f-01b8ce661535 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:12.859441 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e640580-6e7f-4523-a84f-01b8ce661535 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:12.915683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65cf75-54eb-4431-bc4d-017f8a1c2708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:12.916286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65cf75-54eb-4431-bc4d-017f8a1c2708 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:12.916547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65cf75-54eb-4431-bc4d-017f8a1c2708 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:12.916827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65cf75-54eb-4431-bc4d-017f8a1c2708 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:12.917530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65cf75-54eb-4431-bc4d-017f8a1c2708 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:12.919626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e65cf75-54eb-4431-bc4d-017f8a1c2708 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:13.014014 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4338/8645] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:44:12 2022] POST /identity/v3/users => generated 287 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:44:13.018563 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967dd6f-3603-4f2d-b30b-0bfb51d829fd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:13.029511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0967dd6f-3603-4f2d-b30b-0bfb51d829fd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:13.034390 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32284a-b454-462a-99f9-66987b5369a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:13.035066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32284a-b454-462a-99f9-66987b5369a7 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:13.035389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32284a-b454-462a-99f9-66987b5369a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:13.035697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32284a-b454-462a-99f9-66987b5369a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:13.036431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32284a-b454-462a-99f9-66987b5369a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:13.038108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd32284a-b454-462a-99f9-66987b5369a7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:13.048732 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4308/8646] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:44:13.052655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65cf75-54eb-4431-bc4d-017f8a1c2708 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:13.062800 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e65cf75-54eb-4431-bc4d-017f8a1c2708 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:13.068410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e085f16-1b9a-46bf-807d-5c711723d603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:13.069087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e085f16-1b9a-46bf-807d-5c711723d603 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:13.069376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e085f16-1b9a-46bf-807d-5c711723d603 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:13.069657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e085f16-1b9a-46bf-807d-5c711723d603 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:13.070431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e085f16-1b9a-46bf-807d-5c711723d603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:13.072041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e085f16-1b9a-46bf-807d-5c711723d603 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:13.083659 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4339/8647] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:44:13.088108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32284a-b454-462a-99f9-66987b5369a7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:13.094756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:44:13.098612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32284a-b454-462a-99f9-66987b5369a7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:13.103362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0d4e9-8be0-43eb-9422-c0b7437b7fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:13.107136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0d4e9-8be0-43eb-9422-c0b7437b7fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:13.107136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0d4e9-8be0-43eb-9422-c0b7437b7fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:13.107136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0d4e9-8be0-43eb-9422-c0b7437b7fc7 admin admin] PATH_INFO: `/v3/projects/d7ae678d8e60475bb9b35abcb027a02d/users/854bc76149074ec5a4ba92bd42ae6a32/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:13.120209 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c0d4e9-8be0-43eb-9422-c0b7437b7fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ae678d8e60475bb9b35abcb027a02d, user_id=854bc76149074ec5a4ba92bd42ae6a32, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:13.122929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c0d4e9-8be0-43eb-9422-c0b7437b7fc7 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:13.186584 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4309/8648] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:44:13 2022] PUT /identity/v3/projects/d7ae678d8e60475bb9b35abcb027a02d/users/854bc76149074ec5a4ba92bd42ae6a32/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:44:13.197573 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c365757-daeb-453f-9b10-d6c88b4c6c5b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:13.197967 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c365757-daeb-453f-9b10-d6c88b4c6c5b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:13.198321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c365757-daeb-453f-9b10-d6c88b4c6c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:13.227857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9c365757-daeb-453f-9b10-d6c88b4c6c5b None None] MFA Rules not processed for user `854bc76149074ec5a4ba92bd42ae6a32`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:44:13.229302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9c365757-daeb-453f-9b10-d6c88b4c6c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:13.242565 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:44:13.295275 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4340/8649] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:44:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:44:13.300553 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0d4e9-8be0-43eb-9422-c0b7437b7fc7 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:13.364695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0d4e9-8be0-43eb-9422-c0b7437b7fc7 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:13.369399 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c265a-2464-471d-833f-462a9ac09847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:13.370094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c265a-2464-471d-833f-462a9ac09847 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:13.370378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c265a-2464-471d-833f-462a9ac09847 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:13.370655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c265a-2464-471d-833f-462a9ac09847 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:13.371342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c265a-2464-471d-833f-462a9ac09847 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:13.373257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c265a-2464-471d-833f-462a9ac09847 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:13.478319 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4310/8650] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:44:13 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:44:13.482250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c365757-daeb-453f-9b10-d6c88b4c6c5b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:13.493987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c365757-daeb-453f-9b10-d6c88b4c6c5b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:13.551866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ad523-f33a-42fc-aed7-a0c54d101faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:13.552551 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ad523-f33a-42fc-aed7-a0c54d101faa admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:13.552867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ad523-f33a-42fc-aed7-a0c54d101faa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:13.553114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ad523-f33a-42fc-aed7-a0c54d101faa admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:13.553808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ad523-f33a-42fc-aed7-a0c54d101faa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:13.555860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535ad523-f33a-42fc-aed7-a0c54d101faa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:13.632817 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4341/8651] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:44:13 2022] POST /identity/v3/users => generated 293 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:44:13.636742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c265a-2464-471d-833f-462a9ac09847 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:13.647373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c265a-2464-471d-833f-462a9ac09847 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:13.652850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33246623-c64f-4f9d-85de-b716a6de1d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:13.652850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33246623-c64f-4f9d-85de-b716a6de1d1b admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:13.653064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33246623-c64f-4f9d-85de-b716a6de1d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:13.653270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33246623-c64f-4f9d-85de-b716a6de1d1b admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:13.654037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33246623-c64f-4f9d-85de-b716a6de1d1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:13.655659 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33246623-c64f-4f9d-85de-b716a6de1d1b admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:13.667430 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4311/8652] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:44:13.671620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ad523-f33a-42fc-aed7-a0c54d101faa admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:13.682401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ad523-f33a-42fc-aed7-a0c54d101faa admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:13.687036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4783e1-a123-4dbd-a0bb-0cd4ff92b480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:13.687919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4783e1-a123-4dbd-a0bb-0cd4ff92b480 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:13.688862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4783e1-a123-4dbd-a0bb-0cd4ff92b480 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:13.689154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4783e1-a123-4dbd-a0bb-0cd4ff92b480 admin admin] PATH_INFO: `/v3/projects/8df8886ac9fc4edea72c766a9741177c/users/37fa89e4582545c59bf0e86403da46d9/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:13.703930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4783e1-a123-4dbd-a0bb-0cd4ff92b480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8df8886ac9fc4edea72c766a9741177c, user_id=37fa89e4582545c59bf0e86403da46d9, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:13.706727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4783e1-a123-4dbd-a0bb-0cd4ff92b480 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:13.775115 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4342/8653] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:44:13 2022] PUT /identity/v3/projects/8df8886ac9fc4edea72c766a9741177c/users/37fa89e4582545c59bf0e86403da46d9/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:44:13.779110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33246623-c64f-4f9d-85de-b716a6de1d1b admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:13.789805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33246623-c64f-4f9d-85de-b716a6de1d1b admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:13.850839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69ad3d-54da-4e92-90b0-e2675f935dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:13.851513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69ad3d-54da-4e92-90b0-e2675f935dcd admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:13.851796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69ad3d-54da-4e92-90b0-e2675f935dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:13.852071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69ad3d-54da-4e92-90b0-e2675f935dcd admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:13.852794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69ad3d-54da-4e92-90b0-e2675f935dcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:13.854532 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69ad3d-54da-4e92-90b0-e2675f935dcd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:13.865033 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4312/8654] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:44:13 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:44:13.869205 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4783e1-a123-4dbd-a0bb-0cd4ff92b480 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:13.879626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4783e1-a123-4dbd-a0bb-0cd4ff92b480 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:13.884639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca872a-5db6-4d82-a1fb-b447b4c7ec69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:13.885486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca872a-5db6-4d82-a1fb-b447b4c7ec69 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:13.885776 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca872a-5db6-4d82-a1fb-b447b4c7ec69 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:13.886170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca872a-5db6-4d82-a1fb-b447b4c7ec69 admin admin] PATH_INFO: `/v3/projects/8df8886ac9fc4edea72c766a9741177c/users/37fa89e4582545c59bf0e86403da46d9/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:13.888212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca872a-5db6-4d82-a1fb-b447b4c7ec69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8df8886ac9fc4edea72c766a9741177c, user_id=37fa89e4582545c59bf0e86403da46d9, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:13.890940 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca872a-5db6-4d82-a1fb-b447b4c7ec69 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:13.902002 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4343/8655] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:44:13 2022] PUT /identity/v3/projects/8df8886ac9fc4edea72c766a9741177c/users/37fa89e4582545c59bf0e86403da46d9/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:44:13.913536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f1511-c1dd-41c1-a08e-82464fef8a04 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:13.913866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f1511-c1dd-41c1-a08e-82464fef8a04 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:13.914147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f1511-c1dd-41c1-a08e-82464fef8a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:13.941136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-f36f1511-c1dd-41c1-a08e-82464fef8a04 None None] MFA Rules not processed for user `37fa89e4582545c59bf0e86403da46d9`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:44:13.942424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f36f1511-c1dd-41c1-a08e-82464fef8a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:13.982810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:44:14.011636 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4313/8656] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:44:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:44:14.022853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca872a-5db6-4d82-a1fb-b447b4c7ec69 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:14.024632 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-beca872a-5db6-4d82-a1fb-b447b4c7ec69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:14.089732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca872a-5db6-4d82-a1fb-b447b4c7ec69 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:14.093616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44d883-6b0d-448e-9e6b-74b8557540e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:14.094324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44d883-6b0d-448e-9e6b-74b8557540e6 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:14.098741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44d883-6b0d-448e-9e6b-74b8557540e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:14.098741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44d883-6b0d-448e-9e6b-74b8557540e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:14.116695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:44:14.153276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44d883-6b0d-448e-9e6b-74b8557540e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:14.154722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44d883-6b0d-448e-9e6b-74b8557540e6 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:14.164333 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4344/8657] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:44:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:44:16.730416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f1511-c1dd-41c1-a08e-82464fef8a04 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:16.743830 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f1511-c1dd-41c1-a08e-82464fef8a04 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:16.747594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cd197-db7a-466f-ba9a-171d2814d431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:16.748244 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cd197-db7a-466f-ba9a-171d2814d431 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:16.748528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cd197-db7a-466f-ba9a-171d2814d431 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:16.748808 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cd197-db7a-466f-ba9a-171d2814d431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:16.758419 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22cd197-db7a-466f-ba9a-171d2814d431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:16.759868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22cd197-db7a-466f-ba9a-171d2814d431 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:16.769971 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4314/8658] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:44:16 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:44:24.169996 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b857da2-7b28-439d-9996-f61634d7c392 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:24.170494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b857da2-7b28-439d-9996-f61634d7c392 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:24.170605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b857da2-7b28-439d-9996-f61634d7c392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:24.187951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0b857da2-7b28-439d-9996-f61634d7c392 None None] MFA Rules not processed for user `854bc76149074ec5a4ba92bd42ae6a32`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:44:24.189695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0b857da2-7b28-439d-9996-f61634d7c392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:24.196824 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4345/8659] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:44:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:44:24.237485 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cd197-db7a-466f-ba9a-171d2814d431 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:24.238904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e22cd197-db7a-466f-ba9a-171d2814d431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:24.308276 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cd197-db7a-466f-ba9a-171d2814d431 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:24.312644 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab03da9-f0ef-4219-a4ae-913173ce67c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:24.313766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab03da9-f0ef-4219-a4ae-913173ce67c7 service glance] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:24.314085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab03da9-f0ef-4219-a4ae-913173ce67c7 service glance] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:24.314358 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab03da9-f0ef-4219-a4ae-913173ce67c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:24.316892 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fab03da9-f0ef-4219-a4ae-913173ce67c7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:24.408800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab03da9-f0ef-4219-a4ae-913173ce67c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:24.410308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab03da9-f0ef-4219-a4ae-913173ce67c7 service glance] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:24.423773 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4315/8660] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:44:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:44:26.082657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b857da2-7b28-439d-9996-f61634d7c392 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:26.084328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0b857da2-7b28-439d-9996-f61634d7c392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:26.099195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab03da9-f0ef-4219-a4ae-913173ce67c7 service glance] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:26.100749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fab03da9-f0ef-4219-a4ae-913173ce67c7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:26.115344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab03da9-f0ef-4219-a4ae-913173ce67c7 service glance] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:26.119053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d522c-1ee6-4602-b78d-75782a7dcd57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:26.119698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d522c-1ee6-4602-b78d-75782a7dcd57 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:26.119980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d522c-1ee6-4602-b78d-75782a7dcd57 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:26.120257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d522c-1ee6-4602-b78d-75782a7dcd57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:26.122075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e76d522c-1ee6-4602-b78d-75782a7dcd57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:26.138908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d522c-1ee6-4602-b78d-75782a7dcd57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:26.140349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d522c-1ee6-4602-b78d-75782a7dcd57 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:26.156359 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4316/8661] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:44:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:44:26.177534 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b857da2-7b28-439d-9996-f61634d7c392 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:26.182658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188de53-75ec-4fef-a7ea-4cb1368f5b55 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:26.183312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188de53-75ec-4fef-a7ea-4cb1368f5b55 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:26.183615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188de53-75ec-4fef-a7ea-4cb1368f5b55 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:26.183925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4188de53-75ec-4fef-a7ea-4cb1368f5b55 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:26.185710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4188de53-75ec-4fef-a7ea-4cb1368f5b55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:26.200890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188de53-75ec-4fef-a7ea-4cb1368f5b55 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:26.202497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4188de53-75ec-4fef-a7ea-4cb1368f5b55 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:26.216749 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4346/8662] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:44:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:44:29.582056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d522c-1ee6-4602-b78d-75782a7dcd57 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:29.594708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d522c-1ee6-4602-b78d-75782a7dcd57 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:29.598923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6c580-1861-48e5-9a47-3a15e272df0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:29.599567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6c580-1861-48e5-9a47-3a15e272df0e service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:29.599846 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6c580-1861-48e5-9a47-3a15e272df0e service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:29.600119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6c580-1861-48e5-9a47-3a15e272df0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:29.601905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8fb6c580-1861-48e5-9a47-3a15e272df0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:29.614642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6c580-1861-48e5-9a47-3a15e272df0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:29.616100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6c580-1861-48e5-9a47-3a15e272df0e service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:29.626501 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4317/8663] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:44:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:44:30.460026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188de53-75ec-4fef-a7ea-4cb1368f5b55 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:30.461555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4188de53-75ec-4fef-a7ea-4cb1368f5b55 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:30.481398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4188de53-75ec-4fef-a7ea-4cb1368f5b55 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:30.486409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae592649-e9aa-4d1d-a538-e540d6607087 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:30.487077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae592649-e9aa-4d1d-a538-e540d6607087 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:30.487363 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae592649-e9aa-4d1d-a538-e540d6607087 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:30.487644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae592649-e9aa-4d1d-a538-e540d6607087 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:30.490167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ae592649-e9aa-4d1d-a538-e540d6607087 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:30.506772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae592649-e9aa-4d1d-a538-e540d6607087 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:30.508519 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae592649-e9aa-4d1d-a538-e540d6607087 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:30.523282 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4347/8664] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:44:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:44:35.951318 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6c580-1861-48e5-9a47-3a15e272df0e service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:44:35.967602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6c580-1861-48e5-9a47-3a15e272df0e service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:44:35.973119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e555e8-e505-40f5-9015-22f6c4ff6282 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:44:35.973755 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e555e8-e505-40f5-9015-22f6c4ff6282 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:44:35.974076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e555e8-e505-40f5-9015-22f6c4ff6282 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:44:35.974357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e555e8-e505-40f5-9015-22f6c4ff6282 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:44:35.976152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c1e555e8-e505-40f5-9015-22f6c4ff6282 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:44:35.991401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e555e8-e505-40f5-9015-22f6c4ff6282 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:44:35.992990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e555e8-e505-40f5-9015-22f6c4ff6282 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:44:35.997891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:44:36.007342 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4318/8665] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:44:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:44:40.518606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=63 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:44:46.042132 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:45:02.979661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae592649-e9aa-4d1d-a538-e540d6607087 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:45:02.991791 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae592649-e9aa-4d1d-a538-e540d6607087 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:45:02.995466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270437e-ac7b-4c43-aeb2-116f90218130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:45:02.999442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270437e-ac7b-4c43-aeb2-116f90218130 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:45:02.999442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270437e-ac7b-4c43-aeb2-116f90218130 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:45:02.999442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270437e-ac7b-4c43-aeb2-116f90218130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:45:02.999442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4270437e-ac7b-4c43-aeb2-116f90218130 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:45:03.011616 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270437e-ac7b-4c43-aeb2-116f90218130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:45:03.013056 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4270437e-ac7b-4c43-aeb2-116f90218130 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:45:03.024287 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4348/8666] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:45:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:45:13.017604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:45:19.911823 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d8bef-cfe4-4a62-885a-6d5ea59885bf None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:45:19.912336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d8bef-cfe4-4a62-885a-6d5ea59885bf None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:45:19.912461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40d8bef-cfe4-4a62-885a-6d5ea59885bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:45:19.929606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c40d8bef-cfe4-4a62-885a-6d5ea59885bf None None] MFA Rules not processed for user `854bc76149074ec5a4ba92bd42ae6a32`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:45:19.930852 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c40d8bef-cfe4-4a62-885a-6d5ea59885bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:45:19.938058 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4319/8667] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:45:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:45:20.251932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270437e-ac7b-4c43-aeb2-116f90218130 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:45:20.263584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270437e-ac7b-4c43-aeb2-116f90218130 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:45:20.267330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf25d4-4826-4e4e-968e-e120e3e41b8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:45:20.268346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf25d4-4826-4e4e-968e-e120e3e41b8a service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:45:20.268346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf25d4-4826-4e4e-968e-e120e3e41b8a service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:45:20.268613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf25d4-4826-4e4e-968e-e120e3e41b8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:45:20.270499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-49cf25d4-4826-4e4e-968e-e120e3e41b8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:45:20.283174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf25d4-4826-4e4e-968e-e120e3e41b8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:45:20.284598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf25d4-4826-4e4e-968e-e120e3e41b8a service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:45:20.296010 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4349/8668] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:45:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:45:24.516281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d8bef-cfe4-4a62-885a-6d5ea59885bf None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:45:24.533165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40d8bef-cfe4-4a62-885a-6d5ea59885bf None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:45:24.539973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62271be3-9c66-4878-bff2-da3663fe9751 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:45:24.540113 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62271be3-9c66-4878-bff2-da3663fe9751 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:45:24.540320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62271be3-9c66-4878-bff2-da3663fe9751 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:45:24.540806 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62271be3-9c66-4878-bff2-da3663fe9751 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:45:24.542639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-62271be3-9c66-4878-bff2-da3663fe9751 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:45:24.560126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62271be3-9c66-4878-bff2-da3663fe9751 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:45:24.562160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62271be3-9c66-4878-bff2-da3663fe9751 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:45:24.577976 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4320/8669] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:45:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:45:26.157557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf25d4-4826-4e4e-968e-e120e3e41b8a service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:45:26.174039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf25d4-4826-4e4e-968e-e120e3e41b8a service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:45:26.179686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5dcb38-e62c-47a3-b1ac-8eed96edc004 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:45:26.180332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5dcb38-e62c-47a3-b1ac-8eed96edc004 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:45:26.181752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5dcb38-e62c-47a3-b1ac-8eed96edc004 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:45:26.181752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5dcb38-e62c-47a3-b1ac-8eed96edc004 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:45:26.182748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2c5dcb38-e62c-47a3-b1ac-8eed96edc004 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:45:26.242663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5dcb38-e62c-47a3-b1ac-8eed96edc004 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:45:26.244294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5dcb38-e62c-47a3-b1ac-8eed96edc004 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:45:26.253929 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4350/8670] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:45:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:45:34.571456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=9 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:45:36.249146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=34 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:45:36.951029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62271be3-9c66-4878-bff2-da3663fe9751 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:45:36.965586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62271be3-9c66-4878-bff2-da3663fe9751 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:45:36.969761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968079c4-a58b-4c03-9370-c1770d7bd8e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:45:36.970603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968079c4-a58b-4c03-9370-c1770d7bd8e9 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:45:36.970890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968079c4-a58b-4c03-9370-c1770d7bd8e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:45:36.971164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968079c4-a58b-4c03-9370-c1770d7bd8e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:45:36.972948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-968079c4-a58b-4c03-9370-c1770d7bd8e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:45:37.066697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968079c4-a58b-4c03-9370-c1770d7bd8e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:45:37.068349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968079c4-a58b-4c03-9370-c1770d7bd8e9 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:45:37.081100 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4321/8671] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:45:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:45:40.591165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5dcb38-e62c-47a3-b1ac-8eed96edc004 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:45:40.595917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2c5dcb38-e62c-47a3-b1ac-8eed96edc004 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:45:40.609533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5dcb38-e62c-47a3-b1ac-8eed96edc004 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:45:40.614656 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690f029-e29f-4e07-9390-6ec75e37ad53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:45:40.615306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690f029-e29f-4e07-9390-6ec75e37ad53 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:45:40.615592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690f029-e29f-4e07-9390-6ec75e37ad53 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:45:40.615871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690f029-e29f-4e07-9390-6ec75e37ad53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:45:40.617691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d690f029-e29f-4e07-9390-6ec75e37ad53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:45:40.632505 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d690f029-e29f-4e07-9390-6ec75e37ad53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:45:40.634130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d690f029-e29f-4e07-9390-6ec75e37ad53 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:45:40.647364 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4351/8672] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:45:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:45:44.312824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968079c4-a58b-4c03-9370-c1770d7bd8e9 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:45:44.326816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968079c4-a58b-4c03-9370-c1770d7bd8e9 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:45:44.331880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fca14b-b945-4d05-a1eb-087803cc1c30 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:45:44.332527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fca14b-b945-4d05-a1eb-087803cc1c30 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:45:44.332811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fca14b-b945-4d05-a1eb-087803cc1c30 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:45:44.333089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fca14b-b945-4d05-a1eb-087803cc1c30 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:45:44.334926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-23fca14b-b945-4d05-a1eb-087803cc1c30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:45:44.350312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fca14b-b945-4d05-a1eb-087803cc1c30 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:45:44.352009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fca14b-b945-4d05-a1eb-087803cc1c30 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:45:44.365484 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4322/8673] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:45:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:45:44.420520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690f029-e29f-4e07-9390-6ec75e37ad53 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:45:44.422160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d690f029-e29f-4e07-9390-6ec75e37ad53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:45:44.437929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690f029-e29f-4e07-9390-6ec75e37ad53 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:45:44.442699 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847b63b-e474-48e9-b7dc-86adfb2697c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:45:44.446089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847b63b-e474-48e9-b7dc-86adfb2697c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:45:44.446089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847b63b-e474-48e9-b7dc-86adfb2697c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:45:44.446089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847b63b-e474-48e9-b7dc-86adfb2697c6 admin admin] PATH_INFO: `/v3/users/388458a7b3504bd08f3789f53cf9ea35` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:45:44.446089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847b63b-e474-48e9-b7dc-86adfb2697c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=388458a7b3504bd08f3789f53cf9ea35)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:45:44.447313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847b63b-e474-48e9-b7dc-86adfb2697c6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:45:44.620757 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7847b63b-e474-48e9-b7dc-86adfb2697c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '388458a7b3504bd08f3789f53cf9ea35'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:44.627748 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7847b63b-e474-48e9-b7dc-86adfb2697c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '388458a7b3504bd08f3789f53cf9ea35'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:44.628421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7847b63b-e474-48e9-b7dc-86adfb2697c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '388458a7b3504bd08f3789f53cf9ea35'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:44.629035 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7847b63b-e474-48e9-b7dc-86adfb2697c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '388458a7b3504bd08f3789f53cf9ea35'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:44.686970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7847b63b-e474-48e9-b7dc-86adfb2697c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '388458a7b3504bd08f3789f53cf9ea35'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:44.705347 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4352/8674] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:45:44 2022] DELETE /identity/v3/users/388458a7b3504bd08f3789f53cf9ea35 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:45:44.940431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fca14b-b945-4d05-a1eb-087803cc1c30 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:45:44.952345 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fca14b-b945-4d05-a1eb-087803cc1c30 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:45:45.009748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190de85-b7b4-4653-b1f4-11b5de97679a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:45:45.010575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190de85-b7b4-4653-b1f4-11b5de97679a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:45:45.010864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190de85-b7b4-4653-b1f4-11b5de97679a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:45:45.011145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4190de85-b7b4-4653-b1f4-11b5de97679a admin admin] PATH_INFO: `/v3/projects/5833e78f486c4a118608290040b31cfc` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:45:45.012328 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190de85-b7b4-4653-b1f4-11b5de97679a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5833e78f486c4a118608290040b31cfc)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:45:45.014317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4190de85-b7b4-4653-b1f4-11b5de97679a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:45:45.140065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4190de85-b7b4-4653-b1f4-11b5de97679a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5833e78f486c4a118608290040b31cfc', 'request_id': 'req-4190de85-b7b4-4653-b1f4-11b5de97679a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:45.146085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4190de85-b7b4-4653-b1f4-11b5de97679a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5833e78f486c4a118608290040b31cfc', 'request_id': 'req-4190de85-b7b4-4653-b1f4-11b5de97679a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:45.212791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4190de85-b7b4-4653-b1f4-11b5de97679a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5833e78f486c4a118608290040b31cfc', 'request_id': 'req-4190de85-b7b4-4653-b1f4-11b5de97679a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:45.224333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4190de85-b7b4-4653-b1f4-11b5de97679a admin admin] The token cache is being invalidate because project 5833e78f486c4a118608290040b31cfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:45:45.224628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4190de85-b7b4-4653-b1f4-11b5de97679a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:45.225268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-4190de85-b7b4-4653-b1f4-11b5de97679a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:45.226860 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4323/8675] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:45:44 2022] DELETE /identity/v3/projects/5833e78f486c4a118608290040b31cfc => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:45:45.231008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847b63b-e474-48e9-b7dc-86adfb2697c6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:45:45.232409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7847b63b-e474-48e9-b7dc-86adfb2697c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:45:45.246181 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847b63b-e474-48e9-b7dc-86adfb2697c6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:45:45.303082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d597be0-803f-427c-8cc3-72e30a1aab19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:45:45.303961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d597be0-803f-427c-8cc3-72e30a1aab19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:45:45.304251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d597be0-803f-427c-8cc3-72e30a1aab19 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:45:45.304536 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d597be0-803f-427c-8cc3-72e30a1aab19 admin admin] PATH_INFO: `/v3/users/72c48b3dba0a4a63a37873ba42dc0e41` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:45:45.305755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d597be0-803f-427c-8cc3-72e30a1aab19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72c48b3dba0a4a63a37873ba42dc0e41)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:45:45.307721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d597be0-803f-427c-8cc3-72e30a1aab19 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:45:45.479743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4d597be0-803f-427c-8cc3-72e30a1aab19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72c48b3dba0a4a63a37873ba42dc0e41'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:45.488232 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4d597be0-803f-427c-8cc3-72e30a1aab19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72c48b3dba0a4a63a37873ba42dc0e41'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:45.489187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4d597be0-803f-427c-8cc3-72e30a1aab19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72c48b3dba0a4a63a37873ba42dc0e41'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:45.489825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4d597be0-803f-427c-8cc3-72e30a1aab19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72c48b3dba0a4a63a37873ba42dc0e41'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:45.493864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:45:45.593763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-4d597be0-803f-427c-8cc3-72e30a1aab19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72c48b3dba0a4a63a37873ba42dc0e41'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:45.612938 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4353/8676] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:45:45 2022] DELETE /identity/v3/users/72c48b3dba0a4a63a37873ba42dc0e41 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:45:45.847325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190de85-b7b4-4653-b1f4-11b5de97679a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:45:45.848703 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4190de85-b7b4-4653-b1f4-11b5de97679a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:45:45.864048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4190de85-b7b4-4653-b1f4-11b5de97679a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:45:45.872704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:45:45.922656 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:45:45.924247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:45:45.924526 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:45:45.924803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d admin admin] PATH_INFO: `/v3/projects/79c5ad0a00bc47f2b8196d9ebb7fe4a3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:45:45.926118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=79c5ad0a00bc47f2b8196d9ebb7fe4a3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:45:45.929542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:45:46.040127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79c5ad0a00bc47f2b8196d9ebb7fe4a3', 'request_id': 'req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:46.045486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79c5ad0a00bc47f2b8196d9ebb7fe4a3', 'request_id': 'req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:46.125398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79c5ad0a00bc47f2b8196d9ebb7fe4a3', 'request_id': 'req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:46.136968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d admin admin] The token cache is being invalidate because project 79c5ad0a00bc47f2b8196d9ebb7fe4a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:45:46.137259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:46.137930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:45:46.139491 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4324/8677] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:45:45 2022] DELETE /identity/v3/projects/79c5ad0a00bc47f2b8196d9ebb7fe4a3 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:45:55.696494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:45:56.133877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:46:09.566928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d597be0-803f-427c-8cc3-72e30a1aab19 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:46:09.568574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4d597be0-803f-427c-8cc3-72e30a1aab19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:46:09.647141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d597be0-803f-427c-8cc3-72e30a1aab19 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:46:09.710661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b0adb-1041-4d1f-a340-bda9df70e302 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:46:09.711574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b0adb-1041-4d1f-a340-bda9df70e302 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:46:09.711853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b0adb-1041-4d1f-a340-bda9df70e302 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:46:09.712127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b0adb-1041-4d1f-a340-bda9df70e302 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:46:09.714027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-343b0adb-1041-4d1f-a340-bda9df70e302 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:46:09.728801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b0adb-1041-4d1f-a340-bda9df70e302 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:46:09.730440 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b0adb-1041-4d1f-a340-bda9df70e302 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:46:09.745126 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4354/8678] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:46:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:46:13.201302 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:46:13.202816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:46:13.224907 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8dcc24-d55c-44fe-b954-aa40b7369a3d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:46:13.230694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de383e50-54f3-4dde-aa6e-34ed8eec6e62 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:46:13.231594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de383e50-54f3-4dde-aa6e-34ed8eec6e62 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:46:13.231800 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de383e50-54f3-4dde-aa6e-34ed8eec6e62 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:46:13.232094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de383e50-54f3-4dde-aa6e-34ed8eec6e62 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:46:13.234030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-de383e50-54f3-4dde-aa6e-34ed8eec6e62 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:46:13.251058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de383e50-54f3-4dde-aa6e-34ed8eec6e62 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:46:13.252949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de383e50-54f3-4dde-aa6e-34ed8eec6e62 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:46:13.267757 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4325/8679] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:46:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:46:19.738727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=58 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:46:20.697595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d9eb1-bf4f-42f3-906e-9617ddddf5b3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:46:20.698027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d9eb1-bf4f-42f3-906e-9617ddddf5b3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:46:20.698313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d9eb1-bf4f-42f3-906e-9617ddddf5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:46:20.717626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b71d9eb1-bf4f-42f3-906e-9617ddddf5b3 None None] MFA Rules not processed for user `854bc76149074ec5a4ba92bd42ae6a32`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:46:20.718891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b71d9eb1-bf4f-42f3-906e-9617ddddf5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:46:20.778081 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4355/8680] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:46:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:46:21.094637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de383e50-54f3-4dde-aa6e-34ed8eec6e62 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:46:21.096062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-de383e50-54f3-4dde-aa6e-34ed8eec6e62 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:46:21.154514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de383e50-54f3-4dde-aa6e-34ed8eec6e62 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:46:21.158103 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d92c1-44c3-4c2f-8324-c3b7a08c47f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:46:21.158744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d92c1-44c3-4c2f-8324-c3b7a08c47f3 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:46:21.159055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d92c1-44c3-4c2f-8324-c3b7a08c47f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:46:21.159332 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d92c1-44c3-4c2f-8324-c3b7a08c47f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:46:21.161064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e35d92c1-44c3-4c2f-8324-c3b7a08c47f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:46:21.172929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d92c1-44c3-4c2f-8324-c3b7a08c47f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:46:21.174453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d92c1-44c3-4c2f-8324-c3b7a08c47f3 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:46:21.185697 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4326/8681] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:46:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:46:30.770469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=25 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:46:31.179635 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=32 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:47:16.828542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d9eb1-bf4f-42f3-906e-9617ddddf5b3 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:16.829825 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b71d9eb1-bf4f-42f3-906e-9617ddddf5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:16.898494 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d9eb1-bf4f-42f3-906e-9617ddddf5b3 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:16.902995 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38375d-4cec-4e80-b9f9-4f6ca2cd858e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:16.903862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38375d-4cec-4e80-b9f9-4f6ca2cd858e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:16.904146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38375d-4cec-4e80-b9f9-4f6ca2cd858e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:16.904480 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38375d-4cec-4e80-b9f9-4f6ca2cd858e admin admin] PATH_INFO: `/v3/users/854bc76149074ec5a4ba92bd42ae6a32` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:16.905697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38375d-4cec-4e80-b9f9-4f6ca2cd858e admin admin] RBAC: Authorizing `identity:delete_user(user_id=854bc76149074ec5a4ba92bd42ae6a32)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:16.907650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38375d-4cec-4e80-b9f9-4f6ca2cd858e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:17.041677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ac38375d-4cec-4e80-b9f9-4f6ca2cd858e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '854bc76149074ec5a4ba92bd42ae6a32'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:17.049137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ac38375d-4cec-4e80-b9f9-4f6ca2cd858e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '854bc76149074ec5a4ba92bd42ae6a32'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:17.049804 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ac38375d-4cec-4e80-b9f9-4f6ca2cd858e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '854bc76149074ec5a4ba92bd42ae6a32'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:17.050459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ac38375d-4cec-4e80-b9f9-4f6ca2cd858e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '854bc76149074ec5a4ba92bd42ae6a32'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:17.096711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ac38375d-4cec-4e80-b9f9-4f6ca2cd858e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '854bc76149074ec5a4ba92bd42ae6a32'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:17.114799 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4356/8682] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:47:16 2022] DELETE /identity/v3/users/854bc76149074ec5a4ba92bd42ae6a32 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:47:17.119515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acbc94-4cee-48be-b2f7-b5c71c685795 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:17.120048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acbc94-4cee-48be-b2f7-b5c71c685795 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:17.120173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0acbc94-4cee-48be-b2f7-b5c71c685795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:17.137950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c0acbc94-4cee-48be-b2f7-b5c71c685795 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:47:17.139187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c0acbc94-4cee-48be-b2f7-b5c71c685795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:17.197160 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4327/8683] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:47:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:47:17.207041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38375d-4cec-4e80-b9f9-4f6ca2cd858e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:17.208461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-ac38375d-4cec-4e80-b9f9-4f6ca2cd858e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:17.269174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38375d-4cec-4e80-b9f9-4f6ca2cd858e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:17.272805 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0fabf-24a0-4f10-b17d-77d30ac40ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:17.273792 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0fabf-24a0-4f10-b17d-77d30ac40ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:17.274113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0fabf-24a0-4f10-b17d-77d30ac40ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:17.274393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d0fabf-24a0-4f10-b17d-77d30ac40ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:17.284135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0fabf-24a0-4f10-b17d-77d30ac40ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:17.285569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d0fabf-24a0-4f10-b17d-77d30ac40ba4 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:17.294938 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4357/8684] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:47:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:47:17.548421 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acbc94-4cee-48be-b2f7-b5c71c685795 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:17.550984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c0acbc94-4cee-48be-b2f7-b5c71c685795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:17.565083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0acbc94-4cee-48be-b2f7-b5c71c685795 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:17.571313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f696e33-c9b8-4ec6-aced-38527be4ebcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:17.572355 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f696e33-c9b8-4ec6-aced-38527be4ebcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:17.572622 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f696e33-c9b8-4ec6-aced-38527be4ebcd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:17.572976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f696e33-c9b8-4ec6-aced-38527be4ebcd admin admin] PATH_INFO: `/v3/projects/d7ae678d8e60475bb9b35abcb027a02d` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:17.574690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f696e33-c9b8-4ec6-aced-38527be4ebcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7ae678d8e60475bb9b35abcb027a02d)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:17.577264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f696e33-c9b8-4ec6-aced-38527be4ebcd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:17.707054 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f696e33-c9b8-4ec6-aced-38527be4ebcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7ae678d8e60475bb9b35abcb027a02d', 'request_id': 'req-9f696e33-c9b8-4ec6-aced-38527be4ebcd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:17.712733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f696e33-c9b8-4ec6-aced-38527be4ebcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7ae678d8e60475bb9b35abcb027a02d', 'request_id': 'req-9f696e33-c9b8-4ec6-aced-38527be4ebcd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:17.788053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f696e33-c9b8-4ec6-aced-38527be4ebcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7ae678d8e60475bb9b35abcb027a02d', 'request_id': 'req-9f696e33-c9b8-4ec6-aced-38527be4ebcd'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:17.799652 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f696e33-c9b8-4ec6-aced-38527be4ebcd admin admin] The token cache is being invalidate because project d7ae678d8e60475bb9b35abcb027a02d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:47:17.804929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f696e33-c9b8-4ec6-aced-38527be4ebcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:17.804929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-9f696e33-c9b8-4ec6-aced-38527be4ebcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:17.804929 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4328/8685] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:47:17 2022] DELETE /identity/v3/projects/d7ae678d8e60475bb9b35abcb027a02d => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:47:17.806418 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0fabf-24a0-4f10-b17d-77d30ac40ba4 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:17.816872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d0fabf-24a0-4f10-b17d-77d30ac40ba4 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:17.836549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:47:17.876774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ccaeef-a646-427d-b17c-1403736a6d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:17.877646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ccaeef-a646-427d-b17c-1403736a6d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:17.877965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ccaeef-a646-427d-b17c-1403736a6d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:17.878299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ccaeef-a646-427d-b17c-1403736a6d05 admin admin] PATH_INFO: `/v3/users/37fa89e4582545c59bf0e86403da46d9` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:17.879606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ccaeef-a646-427d-b17c-1403736a6d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37fa89e4582545c59bf0e86403da46d9)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:17.882347 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ccaeef-a646-427d-b17c-1403736a6d05 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:18.035416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-88ccaeef-a646-427d-b17c-1403736a6d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37fa89e4582545c59bf0e86403da46d9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:18.042458 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-88ccaeef-a646-427d-b17c-1403736a6d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37fa89e4582545c59bf0e86403da46d9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:18.043118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-88ccaeef-a646-427d-b17c-1403736a6d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37fa89e4582545c59bf0e86403da46d9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:18.043728 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-88ccaeef-a646-427d-b17c-1403736a6d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37fa89e4582545c59bf0e86403da46d9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:18.124137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-88ccaeef-a646-427d-b17c-1403736a6d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37fa89e4582545c59bf0e86403da46d9'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:18.142866 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4358/8686] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:47:17 2022] DELETE /identity/v3/users/37fa89e4582545c59bf0e86403da46d9 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:47:18.518912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f696e33-c9b8-4ec6-aced-38527be4ebcd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:18.520477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9f696e33-c9b8-4ec6-aced-38527be4ebcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:18.534058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f696e33-c9b8-4ec6-aced-38527be4ebcd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:18.592776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c621973c-3b8a-42e1-b4bf-98063f83b815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:18.593567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621973c-3b8a-42e1-b4bf-98063f83b815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:18.593871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621973c-3b8a-42e1-b4bf-98063f83b815 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:18.594152 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c621973c-3b8a-42e1-b4bf-98063f83b815 admin admin] PATH_INFO: `/v3/projects/8df8886ac9fc4edea72c766a9741177c` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:18.595338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621973c-3b8a-42e1-b4bf-98063f83b815 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8df8886ac9fc4edea72c766a9741177c)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:18.597250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c621973c-3b8a-42e1-b4bf-98063f83b815 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:18.709567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:47:18.726783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c621973c-3b8a-42e1-b4bf-98063f83b815 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8df8886ac9fc4edea72c766a9741177c', 'request_id': 'req-c621973c-3b8a-42e1-b4bf-98063f83b815'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:18.732265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c621973c-3b8a-42e1-b4bf-98063f83b815 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8df8886ac9fc4edea72c766a9741177c', 'request_id': 'req-c621973c-3b8a-42e1-b4bf-98063f83b815'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:18.879893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c621973c-3b8a-42e1-b4bf-98063f83b815 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8df8886ac9fc4edea72c766a9741177c', 'request_id': 'req-c621973c-3b8a-42e1-b4bf-98063f83b815'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:18.891629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c621973c-3b8a-42e1-b4bf-98063f83b815 admin admin] The token cache is being invalidate because project 8df8886ac9fc4edea72c766a9741177c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:47:18.891926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c621973c-3b8a-42e1-b4bf-98063f83b815 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:18.892572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-c621973c-3b8a-42e1-b4bf-98063f83b815 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:18.894188 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4329/8687] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:47:18 2022] DELETE /identity/v3/projects/8df8886ac9fc4edea72c766a9741177c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:47:28.138517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:47:28.869075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85186b-50f3-4385-8405-3af2ab986c4c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:28.869336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85186b-50f3-4385-8405-3af2ab986c4c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:28.869624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85186b-50f3-4385-8405-3af2ab986c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:28.887917 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1c85186b-50f3-4385-8405-3af2ab986c4c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:47:28.889113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1c85186b-50f3-4385-8405-3af2ab986c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:28.949313 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4359/8688] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:47:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:47:28.969437 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5f94d-5ba5-412e-988b-374394387799 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:28.970083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5f94d-5ba5-412e-988b-374394387799 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:28.970083 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5f94d-5ba5-412e-988b-374394387799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:28.989236 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-9dd5f94d-5ba5-412e-988b-374394387799 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:47:28.990582 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9dd5f94d-5ba5-412e-988b-374394387799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:28.998513 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4330/8689] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:47:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:47:29.003752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85186b-50f3-4385-8405-3af2ab986c4c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:29.013424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85186b-50f3-4385-8405-3af2ab986c4c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:29.018588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8b008-5f8d-4376-bb79-24b193d68eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:29.019276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8b008-5f8d-4376-bb79-24b193d68eff admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:29.019556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8b008-5f8d-4376-bb79-24b193d68eff admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:29.019827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e8b008-5f8d-4376-bb79-24b193d68eff admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:29.020547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8b008-5f8d-4376-bb79-24b193d68eff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:47:29.020851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8b008-5f8d-4376-bb79-24b193d68eff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:29.022486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e8b008-5f8d-4376-bb79-24b193d68eff admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:29.033881 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4360/8690] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:47:29.045292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5f94d-5ba5-412e-988b-374394387799 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:29.054623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5f94d-5ba5-412e-988b-374394387799 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:29.059232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e5edb-1978-4019-80b7-018dc5a1f9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:29.059876 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e5edb-1978-4019-80b7-018dc5a1f9f8 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:29.060160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e5edb-1978-4019-80b7-018dc5a1f9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:29.060434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e5edb-1978-4019-80b7-018dc5a1f9f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:29.061130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e5edb-1978-4019-80b7-018dc5a1f9f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:29.063081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914e5edb-1978-4019-80b7-018dc5a1f9f8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:29.152998 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4331/8691] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:47:29 2022] POST /identity/v3/projects => generated 393 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:47:29.157086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8b008-5f8d-4376-bb79-24b193d68eff admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:29.167637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e8b008-5f8d-4376-bb79-24b193d68eff admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:29.223674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868f802-91d2-47b6-8023-21b74e3679fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:29.224342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868f802-91d2-47b6-8023-21b74e3679fe admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:29.224630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868f802-91d2-47b6-8023-21b74e3679fe admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:29.224907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d868f802-91d2-47b6-8023-21b74e3679fe admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:29.225606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868f802-91d2-47b6-8023-21b74e3679fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:29.227568 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d868f802-91d2-47b6-8023-21b74e3679fe admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:29.323255 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4361/8692] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:47:29 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:47:29.327431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e5edb-1978-4019-80b7-018dc5a1f9f8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:29.338327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e5edb-1978-4019-80b7-018dc5a1f9f8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:29.343010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c6f7f-da85-481e-bd87-61947757baf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:29.343665 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c6f7f-da85-481e-bd87-61947757baf9 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:29.343952 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c6f7f-da85-481e-bd87-61947757baf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:29.344232 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c6f7f-da85-481e-bd87-61947757baf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:29.344964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c6f7f-da85-481e-bd87-61947757baf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:29.347111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c6f7f-da85-481e-bd87-61947757baf9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:29.357758 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4332/8693] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:47:29.362237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868f802-91d2-47b6-8023-21b74e3679fe admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:29.372253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d868f802-91d2-47b6-8023-21b74e3679fe admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:29.376891 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a4112-131c-41ae-b4b0-9ac62d0ee754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:29.377542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a4112-131c-41ae-b4b0-9ac62d0ee754 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:29.377832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a4112-131c-41ae-b4b0-9ac62d0ee754 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:29.378143 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a4112-131c-41ae-b4b0-9ac62d0ee754 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:29.378874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a4112-131c-41ae-b4b0-9ac62d0ee754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:29.380463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a4112-131c-41ae-b4b0-9ac62d0ee754 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:29.390262 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4362/8694] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:47:29.394889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c6f7f-da85-481e-bd87-61947757baf9 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:29.407710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c6f7f-da85-481e-bd87-61947757baf9 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:29.410071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05694666-e112-411d-90d3-e05ec16420b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:29.410937 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05694666-e112-411d-90d3-e05ec16420b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:29.411219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05694666-e112-411d-90d3-e05ec16420b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:29.411492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05694666-e112-411d-90d3-e05ec16420b3 admin admin] PATH_INFO: `/v3/projects/1d7b02b506014c31ac4e33b324648ec1/users/50097382ed6e438ba79564a1dcee76c3/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:29.425948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05694666-e112-411d-90d3-e05ec16420b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d7b02b506014c31ac4e33b324648ec1, user_id=50097382ed6e438ba79564a1dcee76c3, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:29.428604 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05694666-e112-411d-90d3-e05ec16420b3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:29.489259 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4333/8695] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:47:29 2022] PUT /identity/v3/projects/1d7b02b506014c31ac4e33b324648ec1/users/50097382ed6e438ba79564a1dcee76c3/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:47:29.504046 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8a7e2-3708-493c-9524-3c2a7d448995 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:29.504510 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8a7e2-3708-493c-9524-3c2a7d448995 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:29.504796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8a7e2-3708-493c-9524-3c2a7d448995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:29.532550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-70e8a7e2-3708-493c-9524-3c2a7d448995 None None] MFA Rules not processed for user `50097382ed6e438ba79564a1dcee76c3`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:47:29.534555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-70e8a7e2-3708-493c-9524-3c2a7d448995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:29.589692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:47:29.599487 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4363/8696] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:47:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:47:29.604162 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05694666-e112-411d-90d3-e05ec16420b3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:29.665921 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05694666-e112-411d-90d3-e05ec16420b3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:29.670799 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a730b-043a-488f-beb2-4314012e6269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:29.671452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a730b-043a-488f-beb2-4314012e6269 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:29.671733 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a730b-043a-488f-beb2-4314012e6269 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:29.672036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a730b-043a-488f-beb2-4314012e6269 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:29.672747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a730b-043a-488f-beb2-4314012e6269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:29.674866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1a730b-043a-488f-beb2-4314012e6269 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:29.772625 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4334/8697] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:47:29 2022] POST /identity/v3/projects => generated 391 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:47:29.777138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8a7e2-3708-493c-9524-3c2a7d448995 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:29.788231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8a7e2-3708-493c-9524-3c2a7d448995 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:29.844500 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e7788-5a56-4637-a0ba-7de98f3db6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:29.845167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e7788-5a56-4637-a0ba-7de98f3db6bc admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:29.845452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e7788-5a56-4637-a0ba-7de98f3db6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:29.845730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179e7788-5a56-4637-a0ba-7de98f3db6bc admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:29.846454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e7788-5a56-4637-a0ba-7de98f3db6bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:29.848374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179e7788-5a56-4637-a0ba-7de98f3db6bc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:29.950026 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4364/8698] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:47:29 2022] POST /identity/v3/users => generated 302 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:47:29.951961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a730b-043a-488f-beb2-4314012e6269 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:29.963473 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a730b-043a-488f-beb2-4314012e6269 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:29.968219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812115d0-5ea5-4d2c-aa80-f17a963e4844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:29.968878 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812115d0-5ea5-4d2c-aa80-f17a963e4844 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:29.969167 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812115d0-5ea5-4d2c-aa80-f17a963e4844 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:29.969447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812115d0-5ea5-4d2c-aa80-f17a963e4844 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:29.970247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812115d0-5ea5-4d2c-aa80-f17a963e4844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:29.971866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812115d0-5ea5-4d2c-aa80-f17a963e4844 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:29.982511 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4335/8699] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:47:29.990076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e7788-5a56-4637-a0ba-7de98f3db6bc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:30.000355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179e7788-5a56-4637-a0ba-7de98f3db6bc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:30.005502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3ecb1-50d4-48be-9991-4dc03f01972f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:30.006409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3ecb1-50d4-48be-9991-4dc03f01972f admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:30.010044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3ecb1-50d4-48be-9991-4dc03f01972f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:30.010044 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b3ecb1-50d4-48be-9991-4dc03f01972f admin admin] PATH_INFO: `/v3/projects/5bd34cd977554e59b9f9974979d7afe5/users/3bdb15556d29491ab7de9d0724208ad0/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:30.022234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b3ecb1-50d4-48be-9991-4dc03f01972f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bd34cd977554e59b9f9974979d7afe5, user_id=3bdb15556d29491ab7de9d0724208ad0, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:30.024929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b3ecb1-50d4-48be-9991-4dc03f01972f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:30.119096 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4365/8700] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:47:29 2022] PUT /identity/v3/projects/5bd34cd977554e59b9f9974979d7afe5/users/3bdb15556d29491ab7de9d0724208ad0/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:47:30.123104 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812115d0-5ea5-4d2c-aa80-f17a963e4844 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:30.133826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812115d0-5ea5-4d2c-aa80-f17a963e4844 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:30.153688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:47:30.190598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26df5377-76b9-4dab-bf26-bb2bde6732dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:30.191321 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df5377-76b9-4dab-bf26-bb2bde6732dd admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:30.191607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df5377-76b9-4dab-bf26-bb2bde6732dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:30.191884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df5377-76b9-4dab-bf26-bb2bde6732dd admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:30.192613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df5377-76b9-4dab-bf26-bb2bde6732dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:30.194290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26df5377-76b9-4dab-bf26-bb2bde6732dd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:30.204255 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4336/8701] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:47:30.208330 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3ecb1-50d4-48be-9991-4dc03f01972f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:30.219517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b3ecb1-50d4-48be-9991-4dc03f01972f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:30.224154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390078e-d86d-45f3-af98-0305ff0065ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:30.225055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390078e-d86d-45f3-af98-0305ff0065ef admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:30.225328 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390078e-d86d-45f3-af98-0305ff0065ef admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:30.225607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390078e-d86d-45f3-af98-0305ff0065ef admin admin] PATH_INFO: `/v3/projects/5bd34cd977554e59b9f9974979d7afe5/users/3bdb15556d29491ab7de9d0724208ad0/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:30.227673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390078e-d86d-45f3-af98-0305ff0065ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bd34cd977554e59b9f9974979d7afe5, user_id=3bdb15556d29491ab7de9d0724208ad0, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:30.230394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390078e-d86d-45f3-af98-0305ff0065ef admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:30.241806 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4366/8702] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:47:30 2022] PUT /identity/v3/projects/5bd34cd977554e59b9f9974979d7afe5/users/3bdb15556d29491ab7de9d0724208ad0/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:47:30.253963 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51a16f-b8e3-48a1-9da3-86796c79f6b8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:30.254514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51a16f-b8e3-48a1-9da3-86796c79f6b8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:30.254990 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a51a16f-b8e3-48a1-9da3-86796c79f6b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:30.282721 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-0a51a16f-b8e3-48a1-9da3-86796c79f6b8 None None] MFA Rules not processed for user `3bdb15556d29491ab7de9d0724208ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:47:30.284003 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0a51a16f-b8e3-48a1-9da3-86796c79f6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:30.351497 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4337/8703] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:47:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:47:30.363895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f739e7d-68cc-4260-adeb-52530f7cc872 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:30.365595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f739e7d-68cc-4260-adeb-52530f7cc872 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:30.365595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f739e7d-68cc-4260-adeb-52530f7cc872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:30.382610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-0f739e7d-68cc-4260-adeb-52530f7cc872 None None] MFA Rules not processed for user `50097382ed6e438ba79564a1dcee76c3`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:47:30.383815 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0f739e7d-68cc-4260-adeb-52530f7cc872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:30.438453 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4367/8704] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:47:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:47:30.452843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51a16f-b8e3-48a1-9da3-86796c79f6b8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:30.454315 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0a51a16f-b8e3-48a1-9da3-86796c79f6b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:30.531453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a51a16f-b8e3-48a1-9da3-86796c79f6b8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:30.537597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a8c5a3-0a60-4712-a6a4-17dddedc7e4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:30.538282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8c5a3-0a60-4712-a6a4-17dddedc7e4b service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:30.538572 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8c5a3-0a60-4712-a6a4-17dddedc7e4b service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:30.538850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a8c5a3-0a60-4712-a6a4-17dddedc7e4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:30.546784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8c5a3-0a60-4712-a6a4-17dddedc7e4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:30.548388 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a8c5a3-0a60-4712-a6a4-17dddedc7e4b service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:30.558034 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4338/8705] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:47:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:47:40.431670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=71,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:47:40.553875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=86 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:47:41.549958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f739e7d-68cc-4260-adeb-52530f7cc872 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:41.551373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0f739e7d-68cc-4260-adeb-52530f7cc872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:41.570371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f739e7d-68cc-4260-adeb-52530f7cc872 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:41.575885 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06885f63-a3e2-4cfc-863d-b5dad95cfd2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:41.576530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06885f63-a3e2-4cfc-863d-b5dad95cfd2e service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:41.576811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06885f63-a3e2-4cfc-863d-b5dad95cfd2e service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:41.577085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06885f63-a3e2-4cfc-863d-b5dad95cfd2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:41.585556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06885f63-a3e2-4cfc-863d-b5dad95cfd2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:41.587193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06885f63-a3e2-4cfc-863d-b5dad95cfd2e service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:41.596780 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4368/8706] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:47:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:47:41.610339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c311fe-cd81-49da-b657-f25bdd209e67 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:41.618633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c311fe-cd81-49da-b657-f25bdd209e67 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:41.618633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c311fe-cd81-49da-b657-f25bdd209e67 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:41.618633 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4339/8707] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:47:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:47:41.618986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06885f63-a3e2-4cfc-863d-b5dad95cfd2e service nova] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:47:41.631539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06885f63-a3e2-4cfc-863d-b5dad95cfd2e service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:41.639715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06885f63-a3e2-4cfc-863d-b5dad95cfd2e service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:41.644277 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8ac98-bc5f-4b55-ac81-9f6a0322ede7 tempest-ServersAdminNegativeTestJSON-498023403 tempest-ServersAdminNegativeTestJSON-498023403-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdb15556d29491ab7de9d0724208ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bd34cd977554e59b9f9974979d7afe5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:41.645121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8ac98-bc5f-4b55-ac81-9f6a0322ede7 tempest-ServersAdminNegativeTestJSON-498023403 tempest-ServersAdminNegativeTestJSON-498023403-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:41.645404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8ac98-bc5f-4b55-ac81-9f6a0322ede7 tempest-ServersAdminNegativeTestJSON-498023403 tempest-ServersAdminNegativeTestJSON-498023403-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:41.645678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8ac98-bc5f-4b55-ac81-9f6a0322ede7 tempest-ServersAdminNegativeTestJSON-498023403 tempest-ServersAdminNegativeTestJSON-498023403-project-admin] PATH_INFO: `/v3/projects/5bd34cd977554e59b9f9974979d7afe5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:41.646905 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8ac98-bc5f-4b55-ac81-9f6a0322ede7 tempest-ServersAdminNegativeTestJSON-498023403 tempest-ServersAdminNegativeTestJSON-498023403-project-admin] RBAC: Authorizing `identity:get_project(project_id=5bd34cd977554e59b9f9974979d7afe5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:41.648681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8ac98-bc5f-4b55-ac81-9f6a0322ede7 tempest-ServersAdminNegativeTestJSON-498023403 tempest-ServersAdminNegativeTestJSON-498023403-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:41.650441 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4369/8708] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:47:41 2022] GET /identity/v3/projects/5bd34cd977554e59b9f9974979d7afe5 => generated 391 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:47:42.097507 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467f529-aa3d-4f01-823f-06b6f5d38084 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:42.101307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467f529-aa3d-4f01-823f-06b6f5d38084 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:42.101307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467f529-aa3d-4f01-823f-06b6f5d38084 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:42.101307 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4340/8709] 10.222.0.33 () {54 vars in 912 bytes} [Sat May 28 21:47:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:47:42.104094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8ac98-bc5f-4b55-ac81-9f6a0322ede7 tempest-ServersAdminNegativeTestJSON-498023403 tempest-ServersAdminNegativeTestJSON-498023403-project-admin] Authenticating service token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:365}} May 28 21:47:42.119589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8ac98-bc5f-4b55-ac81-9f6a0322ede7 tempest-ServersAdminNegativeTestJSON-498023403 tempest-ServersAdminNegativeTestJSON-498023403-project-admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:42.128355 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8ac98-bc5f-4b55-ac81-9f6a0322ede7 tempest-ServersAdminNegativeTestJSON-498023403 tempest-ServersAdminNegativeTestJSON-498023403-project-admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:42.132943 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b699d2-94e9-400f-b9fb-a07bb10ccc94 tempest-ServersAdminNegativeTestJSON-498023403 tempest-ServersAdminNegativeTestJSON-498023403-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3bdb15556d29491ab7de9d0724208ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5bd34cd977554e59b9f9974979d7afe5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:42.133803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b699d2-94e9-400f-b9fb-a07bb10ccc94 tempest-ServersAdminNegativeTestJSON-498023403 tempest-ServersAdminNegativeTestJSON-498023403-project-admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:42.134130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b699d2-94e9-400f-b9fb-a07bb10ccc94 tempest-ServersAdminNegativeTestJSON-498023403 tempest-ServersAdminNegativeTestJSON-498023403-project-admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:42.134419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b699d2-94e9-400f-b9fb-a07bb10ccc94 tempest-ServersAdminNegativeTestJSON-498023403 tempest-ServersAdminNegativeTestJSON-498023403-project-admin] PATH_INFO: `/v3/projects/5bd34cd977554e59b9f9974979d7afe5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:42.135669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b699d2-94e9-400f-b9fb-a07bb10ccc94 tempest-ServersAdminNegativeTestJSON-498023403 tempest-ServersAdminNegativeTestJSON-498023403-project-admin] RBAC: Authorizing `identity:get_project(project_id=5bd34cd977554e59b9f9974979d7afe5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:42.137454 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b699d2-94e9-400f-b9fb-a07bb10ccc94 tempest-ServersAdminNegativeTestJSON-498023403 tempest-ServersAdminNegativeTestJSON-498023403-project-admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:42.139203 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4370/8710] 10.222.0.33 () {58 vars in 1443 bytes} [Sat May 28 21:47:42 2022] GET /identity/v3/projects/5bd34cd977554e59b9f9974979d7afe5 => generated 391 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:47:46.289585 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467f529-aa3d-4f01-823f-06b6f5d38084 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:46.354573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467f529-aa3d-4f01-823f-06b6f5d38084 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:46.358998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20ba36-d8d1-4d95-9c2b-a06f3da8c3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:46.359863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20ba36-d8d1-4d95-9c2b-a06f3da8c3d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:46.360148 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20ba36-d8d1-4d95-9c2b-a06f3da8c3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:46.360424 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b20ba36-d8d1-4d95-9c2b-a06f3da8c3d5 admin admin] PATH_INFO: `/v3/users/50097382ed6e438ba79564a1dcee76c3` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:46.361613 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20ba36-d8d1-4d95-9c2b-a06f3da8c3d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50097382ed6e438ba79564a1dcee76c3)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:46.364203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b20ba36-d8d1-4d95-9c2b-a06f3da8c3d5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:46.486470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7b20ba36-d8d1-4d95-9c2b-a06f3da8c3d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50097382ed6e438ba79564a1dcee76c3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:46.487135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7b20ba36-d8d1-4d95-9c2b-a06f3da8c3d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50097382ed6e438ba79564a1dcee76c3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:46.494576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7b20ba36-d8d1-4d95-9c2b-a06f3da8c3d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50097382ed6e438ba79564a1dcee76c3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:46.564767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7b20ba36-d8d1-4d95-9c2b-a06f3da8c3d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50097382ed6e438ba79564a1dcee76c3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:46.565436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-7b20ba36-d8d1-4d95-9c2b-a06f3da8c3d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50097382ed6e438ba79564a1dcee76c3'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:46.585388 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4341/8711] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:47:46 2022] DELETE /identity/v3/users/50097382ed6e438ba79564a1dcee76c3 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:47:46.590170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c1b3e-c1b1-4dd6-a4a9-76b8fc8b4b88 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:46.590677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c1b3e-c1b1-4dd6-a4a9-76b8fc8b4b88 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:46.590793 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643c1b3e-c1b1-4dd6-a4a9-76b8fc8b4b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:46.609469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-643c1b3e-c1b1-4dd6-a4a9-76b8fc8b4b88 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:47:46.610681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-643c1b3e-c1b1-4dd6-a4a9-76b8fc8b4b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:46.668500 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4371/8712] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:47:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:47:46.684246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20ba36-d8d1-4d95-9c2b-a06f3da8c3d5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:46.685646 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7b20ba36-d8d1-4d95-9c2b-a06f3da8c3d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:46.745305 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b20ba36-d8d1-4d95-9c2b-a06f3da8c3d5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:46.749216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042cd27-de9b-4e3e-a341-29f368c6bd7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:46.749900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042cd27-de9b-4e3e-a341-29f368c6bd7f service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:46.750191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042cd27-de9b-4e3e-a341-29f368c6bd7f service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:46.750472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042cd27-de9b-4e3e-a341-29f368c6bd7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:46.759708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042cd27-de9b-4e3e-a341-29f368c6bd7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:46.761155 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042cd27-de9b-4e3e-a341-29f368c6bd7f service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:46.771229 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4342/8713] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:47:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:47:47.023960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c1b3e-c1b1-4dd6-a4a9-76b8fc8b4b88 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:47.025405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-643c1b3e-c1b1-4dd6-a4a9-76b8fc8b4b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:47.037873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643c1b3e-c1b1-4dd6-a4a9-76b8fc8b4b88 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:47.042535 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f797873-e44a-4cfa-ba1d-d711adab4dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:47.043407 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f797873-e44a-4cfa-ba1d-d711adab4dd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:47.043696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f797873-e44a-4cfa-ba1d-d711adab4dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:47.043978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f797873-e44a-4cfa-ba1d-d711adab4dd6 admin admin] PATH_INFO: `/v3/projects/1d7b02b506014c31ac4e33b324648ec1` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:47.045199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f797873-e44a-4cfa-ba1d-d711adab4dd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d7b02b506014c31ac4e33b324648ec1)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:47.047167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f797873-e44a-4cfa-ba1d-d711adab4dd6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:47.154673 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2f797873-e44a-4cfa-ba1d-d711adab4dd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d7b02b506014c31ac4e33b324648ec1', 'request_id': 'req-2f797873-e44a-4cfa-ba1d-d711adab4dd6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:47.162962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2f797873-e44a-4cfa-ba1d-d711adab4dd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d7b02b506014c31ac4e33b324648ec1', 'request_id': 'req-2f797873-e44a-4cfa-ba1d-d711adab4dd6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:47.271428 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2f797873-e44a-4cfa-ba1d-d711adab4dd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d7b02b506014c31ac4e33b324648ec1', 'request_id': 'req-2f797873-e44a-4cfa-ba1d-d711adab4dd6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:47.282877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2f797873-e44a-4cfa-ba1d-d711adab4dd6 admin admin] The token cache is being invalidate because project 1d7b02b506014c31ac4e33b324648ec1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:47:47.283167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2f797873-e44a-4cfa-ba1d-d711adab4dd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:47.283817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-2f797873-e44a-4cfa-ba1d-d711adab4dd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:47.285385 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4372/8714] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:47:47 2022] DELETE /identity/v3/projects/1d7b02b506014c31ac4e33b324648ec1 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:47:47.289471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042cd27-de9b-4e3e-a341-29f368c6bd7f service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:47.299275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042cd27-de9b-4e3e-a341-29f368c6bd7f service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:47.318257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:47:47.356262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8783ef96-4693-40e1-a5ca-fb1328066146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:47.357131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783ef96-4693-40e1-a5ca-fb1328066146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:47.357413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783ef96-4693-40e1-a5ca-fb1328066146 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:47.357689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8783ef96-4693-40e1-a5ca-fb1328066146 admin admin] PATH_INFO: `/v3/users/3bdb15556d29491ab7de9d0724208ad0` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:47.358905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8783ef96-4693-40e1-a5ca-fb1328066146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bdb15556d29491ab7de9d0724208ad0)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:47.367274 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8783ef96-4693-40e1-a5ca-fb1328066146 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:47.544217 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8783ef96-4693-40e1-a5ca-fb1328066146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bdb15556d29491ab7de9d0724208ad0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:47.544883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8783ef96-4693-40e1-a5ca-fb1328066146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bdb15556d29491ab7de9d0724208ad0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:47.552516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8783ef96-4693-40e1-a5ca-fb1328066146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bdb15556d29491ab7de9d0724208ad0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:47.627368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8783ef96-4693-40e1-a5ca-fb1328066146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bdb15556d29491ab7de9d0724208ad0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:47.628633 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-8783ef96-4693-40e1-a5ca-fb1328066146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bdb15556d29491ab7de9d0724208ad0'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:47.646420 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4343/8715] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:47:47 2022] DELETE /identity/v3/users/3bdb15556d29491ab7de9d0724208ad0 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:47:48.058509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f797873-e44a-4cfa-ba1d-d711adab4dd6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:48.063743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2f797873-e44a-4cfa-ba1d-d711adab4dd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:48.076957 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f797873-e44a-4cfa-ba1d-d711adab4dd6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:48.126927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:47:48.135682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ba5ca-13d4-4a2b-8ad6-3d79405fb61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:48.136595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ba5ca-13d4-4a2b-8ad6-3d79405fb61a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:48.136879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ba5ca-13d4-4a2b-8ad6-3d79405fb61a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:48.137159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ba5ca-13d4-4a2b-8ad6-3d79405fb61a admin admin] PATH_INFO: `/v3/projects/5bd34cd977554e59b9f9974979d7afe5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:48.138370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ba5ca-13d4-4a2b-8ad6-3d79405fb61a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bd34cd977554e59b9f9974979d7afe5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:48.140273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ba5ca-13d4-4a2b-8ad6-3d79405fb61a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:48.243010 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-884ba5ca-13d4-4a2b-8ad6-3d79405fb61a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bd34cd977554e59b9f9974979d7afe5', 'request_id': 'req-884ba5ca-13d4-4a2b-8ad6-3d79405fb61a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:48.248528 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-884ba5ca-13d4-4a2b-8ad6-3d79405fb61a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bd34cd977554e59b9f9974979d7afe5', 'request_id': 'req-884ba5ca-13d4-4a2b-8ad6-3d79405fb61a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:48.318953 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-884ba5ca-13d4-4a2b-8ad6-3d79405fb61a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bd34cd977554e59b9f9974979d7afe5', 'request_id': 'req-884ba5ca-13d4-4a2b-8ad6-3d79405fb61a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:48.330192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-884ba5ca-13d4-4a2b-8ad6-3d79405fb61a admin admin] The token cache is being invalidate because project 5bd34cd977554e59b9f9974979d7afe5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:47:48.330486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-884ba5ca-13d4-4a2b-8ad6-3d79405fb61a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:48.331613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-884ba5ca-13d4-4a2b-8ad6-3d79405fb61a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:47:48.333312 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4373/8716] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:47:48 2022] DELETE /identity/v3/projects/5bd34cd977554e59b9f9974979d7afe5 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:47:48.339579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d345d-7a60-447e-a14b-01daa0c69357 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:48.340076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d345d-7a60-447e-a14b-01daa0c69357 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:48.340205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d345d-7a60-447e-a14b-01daa0c69357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:48.358313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-925d345d-7a60-447e-a14b-01daa0c69357 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:47:48.359537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-925d345d-7a60-447e-a14b-01daa0c69357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:48.417469 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4344/8717] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:47:48.425696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4c372-604e-4caa-9cff-af11b165e7fd None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:48.426024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4c372-604e-4caa-9cff-af11b165e7fd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:48.426365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4c372-604e-4caa-9cff-af11b165e7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:48.445123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-82e4c372-604e-4caa-9cff-af11b165e7fd None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:47:48.446351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-82e4c372-604e-4caa-9cff-af11b165e7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:48.454338 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4374/8718] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:47:48.459368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d345d-7a60-447e-a14b-01daa0c69357 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:48.469439 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d345d-7a60-447e-a14b-01daa0c69357 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:48.474068 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79782c3d-ecfd-4c94-bfa9-56716e60b5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:48.479941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79782c3d-ecfd-4c94-bfa9-56716e60b5d8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:48.479941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79782c3d-ecfd-4c94-bfa9-56716e60b5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:48.479941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79782c3d-ecfd-4c94-bfa9-56716e60b5d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:48.479941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79782c3d-ecfd-4c94-bfa9-56716e60b5d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:47:48.479941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79782c3d-ecfd-4c94-bfa9-56716e60b5d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:48.479941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79782c3d-ecfd-4c94-bfa9-56716e60b5d8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:48.489473 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4345/8719] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:47:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 28 21:47:48.493652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4c372-604e-4caa-9cff-af11b165e7fd None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:48.503538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4c372-604e-4caa-9cff-af11b165e7fd None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:48.508185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b5a56-3488-4b60-bcb6-31dabf8ebd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:48.508829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b5a56-3488-4b60-bcb6-31dabf8ebd7f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:48.509110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b5a56-3488-4b60-bcb6-31dabf8ebd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:48.509384 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850b5a56-3488-4b60-bcb6-31dabf8ebd7f admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:48.510178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b5a56-3488-4b60-bcb6-31dabf8ebd7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:48.512078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850b5a56-3488-4b60-bcb6-31dabf8ebd7f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:48.617634 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4375/8720] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:47:48 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:47:48.626012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79782c3d-ecfd-4c94-bfa9-56716e60b5d8 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:48.636780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79782c3d-ecfd-4c94-bfa9-56716e60b5d8 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:48.663252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:47:48.694760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdfdc3-ebb9-43ee-b61e-f6328875636c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:48.695541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdfdc3-ebb9-43ee-b61e-f6328875636c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:48.695829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdfdc3-ebb9-43ee-b61e-f6328875636c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:48.696108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acdfdc3-ebb9-43ee-b61e-f6328875636c admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:48.696803 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdfdc3-ebb9-43ee-b61e-f6328875636c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:48.698821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acdfdc3-ebb9-43ee-b61e-f6328875636c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:48.782615 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4346/8721] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:47:48 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:47:48.786713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b5a56-3488-4b60-bcb6-31dabf8ebd7f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:48.797433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850b5a56-3488-4b60-bcb6-31dabf8ebd7f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:48.802130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd618772-4362-457c-940a-250e92f45b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:48.802780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd618772-4362-457c-940a-250e92f45b72 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:48.803063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd618772-4362-457c-940a-250e92f45b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:48.803338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd618772-4362-457c-940a-250e92f45b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:48.804171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd618772-4362-457c-940a-250e92f45b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:48.805773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd618772-4362-457c-940a-250e92f45b72 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:48.816279 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4376/8722] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:47:48.820077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdfdc3-ebb9-43ee-b61e-f6328875636c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:48.830464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acdfdc3-ebb9-43ee-b61e-f6328875636c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:48.837212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e41450-4174-49c9-a85d-2fcdcc122097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:48.837894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e41450-4174-49c9-a85d-2fcdcc122097 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:48.838184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e41450-4174-49c9-a85d-2fcdcc122097 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:48.838465 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e41450-4174-49c9-a85d-2fcdcc122097 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:48.839212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e41450-4174-49c9-a85d-2fcdcc122097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:48.840802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e41450-4174-49c9-a85d-2fcdcc122097 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:48.850853 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4347/8723] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:47:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:47:48.854915 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd618772-4362-457c-940a-250e92f45b72 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:48.865297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd618772-4362-457c-940a-250e92f45b72 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:48.869866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb466dd7-486a-4a25-a859-6a6b4185c8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:48.870691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb466dd7-486a-4a25-a859-6a6b4185c8aa admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:48.870978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb466dd7-486a-4a25-a859-6a6b4185c8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:48.871257 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb466dd7-486a-4a25-a859-6a6b4185c8aa admin admin] PATH_INFO: `/v3/projects/6bf011cdf83d49df98c2e932eda352cd/users/45bef914dfcc4fb49dfc30dc1acdd799/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:48.886112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb466dd7-486a-4a25-a859-6a6b4185c8aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bf011cdf83d49df98c2e932eda352cd, user_id=45bef914dfcc4fb49dfc30dc1acdd799, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:48.888760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb466dd7-486a-4a25-a859-6a6b4185c8aa admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:48.952753 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4377/8724] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:47:48 2022] PUT /identity/v3/projects/6bf011cdf83d49df98c2e932eda352cd/users/45bef914dfcc4fb49dfc30dc1acdd799/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:47:48.957910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c4a99-c761-44d9-b2c5-1326d38bdd04 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:48.958203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c4a99-c761-44d9-b2c5-1326d38bdd04 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:48.958480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c4a99-c761-44d9-b2c5-1326d38bdd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:48.976708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1d0c4a99-c761-44d9-b2c5-1326d38bdd04 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:47:48.977942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1d0c4a99-c761-44d9-b2c5-1326d38bdd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:49.038212 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4348/8725] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:47:49.047599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb466dd7-486a-4a25-a859-6a6b4185c8aa admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:49.049286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fb466dd7-486a-4a25-a859-6a6b4185c8aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:49.109365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb466dd7-486a-4a25-a859-6a6b4185c8aa admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:49.113614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108104d-ca1b-4c7b-b87c-dfda02a55f94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:49.114297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108104d-ca1b-4c7b-b87c-dfda02a55f94 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:49.114584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108104d-ca1b-4c7b-b87c-dfda02a55f94 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:49.114863 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108104d-ca1b-4c7b-b87c-dfda02a55f94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:49.124543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108104d-ca1b-4c7b-b87c-dfda02a55f94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:49.125474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108104d-ca1b-4c7b-b87c-dfda02a55f94 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:49.135651 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4378/8726] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:47:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:47:50.438679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7d9c2-fd3f-44aa-8134-2ca312c1c766 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:50.439237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7d9c2-fd3f-44aa-8134-2ca312c1c766 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:50.439237 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7d9c2-fd3f-44aa-8134-2ca312c1c766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:50.473414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ddb7d9c2-fd3f-44aa-8134-2ca312c1c766 None None] MFA Rules not processed for user `45bef914dfcc4fb49dfc30dc1acdd799`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:47:50.474667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ddb7d9c2-fd3f-44aa-8134-2ca312c1c766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:50.539578 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4349/8727] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:47:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:47:50.551461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308e008-2847-420d-930f-97b120dd83df None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:50.552039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308e008-2847-420d-930f-97b120dd83df None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:50.552039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308e008-2847-420d-930f-97b120dd83df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:50.573089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1308e008-2847-420d-930f-97b120dd83df None None] MFA Rules not processed for user `45bef914dfcc4fb49dfc30dc1acdd799`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:47:50.573089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1308e008-2847-420d-930f-97b120dd83df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:50.578145 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4379/8728] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:47:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:47:50.590035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7d9c2-fd3f-44aa-8134-2ca312c1c766 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:47:50.591508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ddb7d9c2-fd3f-44aa-8134-2ca312c1c766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:47:50.634416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:47:50.667914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb7d9c2-fd3f-44aa-8134-2ca312c1c766 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:47:50.673574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa8b152-19e8-4d54-941c-6d29c79967c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:47:50.674258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8b152-19e8-4d54-941c-6d29c79967c2 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:47:50.675078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8b152-19e8-4d54-941c-6d29c79967c2 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:47:50.675347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa8b152-19e8-4d54-941c-6d29c79967c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:47:50.683222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8b152-19e8-4d54-941c-6d29c79967c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:47:50.684890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa8b152-19e8-4d54-941c-6d29c79967c2 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:47:50.694458 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4350/8729] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:47:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:48:00.564287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:48:00.753432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=17 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:48:03.719721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308e008-2847-420d-930f-97b120dd83df None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:48:03.719721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1308e008-2847-420d-930f-97b120dd83df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:48:03.734530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308e008-2847-420d-930f-97b120dd83df None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:48:03.738202 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872dfe2d-e001-4927-af8b-e6f009ba674c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:48:03.738860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872dfe2d-e001-4927-af8b-e6f009ba674c service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:03.739128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872dfe2d-e001-4927-af8b-e6f009ba674c service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:03.739411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872dfe2d-e001-4927-af8b-e6f009ba674c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:03.741129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-872dfe2d-e001-4927-af8b-e6f009ba674c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:48:03.752979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872dfe2d-e001-4927-af8b-e6f009ba674c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:48:03.754985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872dfe2d-e001-4927-af8b-e6f009ba674c service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:48:03.765561 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4380/8730] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:48:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:48:03.916936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb46fe6-b509-4ef0-af26-47cdf10d532f None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:03.917547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb46fe6-b509-4ef0-af26-47cdf10d532f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:03.917547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb46fe6-b509-4ef0-af26-47cdf10d532f None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:03.918926 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4351/8731] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:48:03.923716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215df403-4685-477e-983d-6b0762c29591 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:03.924007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215df403-4685-477e-983d-6b0762c29591 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:03.924288 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215df403-4685-477e-983d-6b0762c29591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:03.972091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-215df403-4685-477e-983d-6b0762c29591 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:48:03.973224 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-215df403-4685-477e-983d-6b0762c29591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:48:04.040063 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4381/8732] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:48:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:48:04.061608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb46fe6-b509-4ef0-af26-47cdf10d532f None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:48:04.063406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fdb46fe6-b509-4ef0-af26-47cdf10d532f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:48:04.085932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb46fe6-b509-4ef0-af26-47cdf10d532f None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:48:04.092181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca2637-1844-4b79-aca6-79721309a68a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:48:04.092181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca2637-1844-4b79-aca6-79721309a68a service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:04.092181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca2637-1844-4b79-aca6-79721309a68a service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:04.092574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca2637-1844-4b79-aca6-79721309a68a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:04.101515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca2637-1844-4b79-aca6-79721309a68a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:48:04.103312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca2637-1844-4b79-aca6-79721309a68a service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:48:04.114110 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4352/8733] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:48:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:48:04.408501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215df403-4685-477e-983d-6b0762c29591 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:48:04.421127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215df403-4685-477e-983d-6b0762c29591 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:48:04.424918 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318bafa-b96f-4f4b-b127-d2a14dc47554 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:48:04.425586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318bafa-b96f-4f4b-b127-d2a14dc47554 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:04.425903 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318bafa-b96f-4f4b-b127-d2a14dc47554 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:04.426193 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318bafa-b96f-4f4b-b127-d2a14dc47554 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:04.427999 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e318bafa-b96f-4f4b-b127-d2a14dc47554 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:48:04.445344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318bafa-b96f-4f4b-b127-d2a14dc47554 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:48:04.446876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e318bafa-b96f-4f4b-b127-d2a14dc47554 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:48:04.462700 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4382/8734] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:48:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:48:14.109123 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:48:14.455517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=51 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:48:43.917816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca2637-1844-4b79-aca6-79721309a68a service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:48:43.929886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca2637-1844-4b79-aca6-79721309a68a service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:48:43.934589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087650fe-9fa1-4751-a990-c74c6c23b8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:48:43.935492 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087650fe-9fa1-4751-a990-c74c6c23b8d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:43.935779 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087650fe-9fa1-4751-a990-c74c6c23b8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:43.936057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087650fe-9fa1-4751-a990-c74c6c23b8d9 admin admin] PATH_INFO: `/v3/users/45bef914dfcc4fb49dfc30dc1acdd799` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:43.937261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087650fe-9fa1-4751-a990-c74c6c23b8d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45bef914dfcc4fb49dfc30dc1acdd799)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:48:43.939256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087650fe-9fa1-4751-a990-c74c6c23b8d9 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:48:44.113153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-087650fe-9fa1-4751-a990-c74c6c23b8d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45bef914dfcc4fb49dfc30dc1acdd799'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:48:44.113741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-087650fe-9fa1-4751-a990-c74c6c23b8d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45bef914dfcc4fb49dfc30dc1acdd799'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:48:44.121416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-087650fe-9fa1-4751-a990-c74c6c23b8d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45bef914dfcc4fb49dfc30dc1acdd799'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:48:44.179484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-087650fe-9fa1-4751-a990-c74c6c23b8d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45bef914dfcc4fb49dfc30dc1acdd799'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:48:44.180227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-087650fe-9fa1-4751-a990-c74c6c23b8d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45bef914dfcc4fb49dfc30dc1acdd799'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:48:44.197944 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4353/8735] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:48:43 2022] DELETE /identity/v3/users/45bef914dfcc4fb49dfc30dc1acdd799 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:48:44.434393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318bafa-b96f-4f4b-b127-d2a14dc47554 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:48:44.447307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318bafa-b96f-4f4b-b127-d2a14dc47554 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:48:44.505417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae185936-2fa0-4335-b36c-84201a1171de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:48:44.506336 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae185936-2fa0-4335-b36c-84201a1171de admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:44.506692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae185936-2fa0-4335-b36c-84201a1171de admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:44.506973 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae185936-2fa0-4335-b36c-84201a1171de admin admin] PATH_INFO: `/v3/projects/6bf011cdf83d49df98c2e932eda352cd` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:44.508147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae185936-2fa0-4335-b36c-84201a1171de admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bf011cdf83d49df98c2e932eda352cd)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:48:44.510106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae185936-2fa0-4335-b36c-84201a1171de admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:48:44.624150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ae185936-2fa0-4335-b36c-84201a1171de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bf011cdf83d49df98c2e932eda352cd', 'request_id': 'req-ae185936-2fa0-4335-b36c-84201a1171de'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:48:44.629931 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ae185936-2fa0-4335-b36c-84201a1171de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bf011cdf83d49df98c2e932eda352cd', 'request_id': 'req-ae185936-2fa0-4335-b36c-84201a1171de'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:48:44.698855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ae185936-2fa0-4335-b36c-84201a1171de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bf011cdf83d49df98c2e932eda352cd', 'request_id': 'req-ae185936-2fa0-4335-b36c-84201a1171de'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:48:44.710692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ae185936-2fa0-4335-b36c-84201a1171de admin admin] The token cache is being invalidate because project 6bf011cdf83d49df98c2e932eda352cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:48:44.710986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ae185936-2fa0-4335-b36c-84201a1171de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:48:44.711650 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-ae185936-2fa0-4335-b36c-84201a1171de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:48:44.713216 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4383/8736] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:48:44 2022] DELETE /identity/v3/projects/6bf011cdf83d49df98c2e932eda352cd => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:48:44.718394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308de37-8182-4b6e-a39b-fe3e1b9f3770 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:44.718689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308de37-8182-4b6e-a39b-fe3e1b9f3770 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:44.718971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308de37-8182-4b6e-a39b-fe3e1b9f3770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:44.737310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c308de37-8182-4b6e-a39b-fe3e1b9f3770 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:48:44.738575 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c308de37-8182-4b6e-a39b-fe3e1b9f3770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:48:44.798987 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4354/8737] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:48:44.808211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632aa48-6492-4412-8a25-5d5a4e4b857b None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:44.808499 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632aa48-6492-4412-8a25-5d5a4e4b857b None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:44.808782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e632aa48-6492-4412-8a25-5d5a4e4b857b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:44.826927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e632aa48-6492-4412-8a25-5d5a4e4b857b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:48:44.828122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e632aa48-6492-4412-8a25-5d5a4e4b857b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:48:44.836296 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4384/8738] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:48:44.840548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308de37-8182-4b6e-a39b-fe3e1b9f3770 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:48:44.850971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308de37-8182-4b6e-a39b-fe3e1b9f3770 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:48:44.855611 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a56a1c-ec27-47f0-bb75-2db3e1e58bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:48:44.856268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a56a1c-ec27-47f0-bb75-2db3e1e58bba admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:44.856559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a56a1c-ec27-47f0-bb75-2db3e1e58bba admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:44.856842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a56a1c-ec27-47f0-bb75-2db3e1e58bba admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:44.857574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a56a1c-ec27-47f0-bb75-2db3e1e58bba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:48:44.857923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a56a1c-ec27-47f0-bb75-2db3e1e58bba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:48:44.859550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a56a1c-ec27-47f0-bb75-2db3e1e58bba admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:48:44.871796 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4355/8739] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:48:44.875628 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632aa48-6492-4412-8a25-5d5a4e4b857b None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:48:44.885000 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e632aa48-6492-4412-8a25-5d5a4e4b857b None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:48:44.889567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb595f3-1bc0-4bb5-baea-a46e85a4cc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:48:44.890256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb595f3-1bc0-4bb5-baea-a46e85a4cc3f admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:44.890540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb595f3-1bc0-4bb5-baea-a46e85a4cc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:44.890817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb595f3-1bc0-4bb5-baea-a46e85a4cc3f admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:44.891518 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb595f3-1bc0-4bb5-baea-a46e85a4cc3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:48:44.893410 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb595f3-1bc0-4bb5-baea-a46e85a4cc3f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:48:44.995756 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4385/8740] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:48:44 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:48:44.999820 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a56a1c-ec27-47f0-bb75-2db3e1e58bba admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:48:45.010092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a56a1c-ec27-47f0-bb75-2db3e1e58bba admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:48:45.066084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e25d9a-0730-4fb0-aa45-439ca42264ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:48:45.066771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e25d9a-0730-4fb0-aa45-439ca42264ed admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:45.067058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e25d9a-0730-4fb0-aa45-439ca42264ed admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:45.067338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e25d9a-0730-4fb0-aa45-439ca42264ed admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:45.068045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e25d9a-0730-4fb0-aa45-439ca42264ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:48:45.070049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e25d9a-0730-4fb0-aa45-439ca42264ed admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:48:45.076710 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:48:45.118095 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4356/8741] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:48:44 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:48:45.122393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb595f3-1bc0-4bb5-baea-a46e85a4cc3f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:48:45.132845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb595f3-1bc0-4bb5-baea-a46e85a4cc3f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:48:45.137456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4da3f-0dfb-4743-8bb5-188f2becae18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:48:45.138155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4da3f-0dfb-4743-8bb5-188f2becae18 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:45.138443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4da3f-0dfb-4743-8bb5-188f2becae18 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:45.138724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4da3f-0dfb-4743-8bb5-188f2becae18 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:45.139453 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4da3f-0dfb-4743-8bb5-188f2becae18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:48:45.141090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4da3f-0dfb-4743-8bb5-188f2becae18 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:48:45.152125 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4386/8742] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:48:45.156111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e25d9a-0730-4fb0-aa45-439ca42264ed admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:48:45.166723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e25d9a-0730-4fb0-aa45-439ca42264ed admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:48:45.171804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2284e088-a7f4-42fc-b81d-58a5ef8b2a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:48:45.172458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284e088-a7f4-42fc-b81d-58a5ef8b2a1f admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:45.172745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284e088-a7f4-42fc-b81d-58a5ef8b2a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:45.173027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2284e088-a7f4-42fc-b81d-58a5ef8b2a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:45.173899 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284e088-a7f4-42fc-b81d-58a5ef8b2a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:48:45.175515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2284e088-a7f4-42fc-b81d-58a5ef8b2a1f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:48:45.185482 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4357/8743] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:48:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:48:45.189511 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4da3f-0dfb-4743-8bb5-188f2becae18 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:48:45.199393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4da3f-0dfb-4743-8bb5-188f2becae18 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:48:45.203914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2b006-b964-4259-b323-773f47b50fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:48:45.204782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2b006-b964-4259-b323-773f47b50fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:45.205064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2b006-b964-4259-b323-773f47b50fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:45.205343 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f2b006-b964-4259-b323-773f47b50fcb admin admin] PATH_INFO: `/v3/projects/ded5b434b72f4a2ba885f887baa3c35a/users/c44b88da38234871ad6732dab5858246/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:45.221316 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2b006-b964-4259-b323-773f47b50fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ded5b434b72f4a2ba885f887baa3c35a, user_id=c44b88da38234871ad6732dab5858246, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:48:45.224091 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f2b006-b964-4259-b323-773f47b50fcb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:48:45.295547 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4387/8744] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:48:45 2022] PUT /identity/v3/projects/ded5b434b72f4a2ba885f887baa3c35a/users/c44b88da38234871ad6732dab5858246/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:48:45.301029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3664d-9487-4758-a2af-020826f1b3e5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:45.301428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3664d-9487-4758-a2af-020826f1b3e5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:45.301716 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3664d-9487-4758-a2af-020826f1b3e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:45.319929 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-afe3664d-9487-4758-a2af-020826f1b3e5 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:48:45.321170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-afe3664d-9487-4758-a2af-020826f1b3e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:48:45.380645 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4358/8745] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:48:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:48:45.390585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2b006-b964-4259-b323-773f47b50fcb admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:48:45.392318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-08f2b006-b964-4259-b323-773f47b50fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:48:45.423835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:48:45.453552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f2b006-b964-4259-b323-773f47b50fcb admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:48:45.457671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e84a5-6c63-4b20-a5c0-e449cdfaab53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:48:45.458371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e84a5-6c63-4b20-a5c0-e449cdfaab53 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:45.458660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e84a5-6c63-4b20-a5c0-e449cdfaab53 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:45.458944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e84a5-6c63-4b20-a5c0-e449cdfaab53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:45.470068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e84a5-6c63-4b20-a5c0-e449cdfaab53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:48:45.471561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e84a5-6c63-4b20-a5c0-e449cdfaab53 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:48:45.485338 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4388/8746] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:48:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:48:46.753712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f97a9-4e06-4258-b737-be4427e9b084 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:46.754290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f97a9-4e06-4258-b737-be4427e9b084 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:46.754405 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f97a9-4e06-4258-b737-be4427e9b084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:46.782249 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-948f97a9-4e06-4258-b737-be4427e9b084 None None] MFA Rules not processed for user `c44b88da38234871ad6732dab5858246`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:48:46.783783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-948f97a9-4e06-4258-b737-be4427e9b084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:48:46.851500 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4359/8747] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:48:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:48:46.861098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540a5e2-f666-4d06-9a97-4b3a3c0a8293 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:46.861609 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540a5e2-f666-4d06-9a97-4b3a3c0a8293 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:46.861722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540a5e2-f666-4d06-9a97-4b3a3c0a8293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:46.879282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1540a5e2-f666-4d06-9a97-4b3a3c0a8293 None None] MFA Rules not processed for user `c44b88da38234871ad6732dab5858246`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:48:46.880481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1540a5e2-f666-4d06-9a97-4b3a3c0a8293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:48:46.887923 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4389/8748] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:48:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:48:46.899453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f97a9-4e06-4258-b737-be4427e9b084 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:48:46.900858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-948f97a9-4e06-4258-b737-be4427e9b084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:48:46.976308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f97a9-4e06-4258-b737-be4427e9b084 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:48:46.981858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc633da-4874-4ced-b1d8-2393115d6fe8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:48:46.982527 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc633da-4874-4ced-b1d8-2393115d6fe8 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:48:46.982813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc633da-4874-4ced-b1d8-2393115d6fe8 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:48:46.983087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc633da-4874-4ced-b1d8-2393115d6fe8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:48:46.987793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:48:46.992235 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc633da-4874-4ced-b1d8-2393115d6fe8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:48:46.993905 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc633da-4874-4ced-b1d8-2393115d6fe8 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:48:47.011430 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4360/8749] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:48:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:48:56.874459 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=17 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:48:57.043850 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:49:00.623981 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51c45d-9e37-4b32-883a-2ab6a361428e None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:49:00.624533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51c45d-9e37-4b32-883a-2ab6a361428e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:49:00.624533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51c45d-9e37-4b32-883a-2ab6a361428e None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:49:00.625981 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4390/8750] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:49:00.630763 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831ec7d-10b0-4a0e-b5c7-6225bae87ede None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:49:00.631213 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831ec7d-10b0-4a0e-b5c7-6225bae87ede None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:49:00.631453 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831ec7d-10b0-4a0e-b5c7-6225bae87ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:49:00.651294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c831ec7d-10b0-4a0e-b5c7-6225bae87ede None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:49:00.652444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c831ec7d-10b0-4a0e-b5c7-6225bae87ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:49:00.714627 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4361/8751] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:49:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:49:00.735159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51c45d-9e37-4b32-883a-2ab6a361428e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:49:00.736851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bb51c45d-9e37-4b32-883a-2ab6a361428e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:49:00.756775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51c45d-9e37-4b32-883a-2ab6a361428e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:49:00.761829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c53ba5-faf4-4785-8f3c-fc4bfb648443 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:49:00.762492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c53ba5-faf4-4785-8f3c-fc4bfb648443 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:49:00.762775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c53ba5-faf4-4785-8f3c-fc4bfb648443 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:49:00.763052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c53ba5-faf4-4785-8f3c-fc4bfb648443 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:49:00.771398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c53ba5-faf4-4785-8f3c-fc4bfb648443 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:49:00.773497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c53ba5-faf4-4785-8f3c-fc4bfb648443 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:49:00.783159 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4391/8752] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:49:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:49:10.705935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=29 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:49:10.779074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:49:24.972561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831ec7d-10b0-4a0e-b5c7-6225bae87ede None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:49:24.985180 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831ec7d-10b0-4a0e-b5c7-6225bae87ede None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:49:24.988734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22104db-840f-47dc-ae32-e397ec08523b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:49:24.989412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22104db-840f-47dc-ae32-e397ec08523b service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:49:24.989698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22104db-840f-47dc-ae32-e397ec08523b service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:49:24.990009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22104db-840f-47dc-ae32-e397ec08523b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:49:24.992499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d22104db-840f-47dc-ae32-e397ec08523b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:49:25.008325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22104db-840f-47dc-ae32-e397ec08523b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:49:25.009744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22104db-840f-47dc-ae32-e397ec08523b service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:49:25.023186 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4362/8753] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:49:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:49:26.195832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c53ba5-faf4-4785-8f3c-fc4bfb648443 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:49:26.207802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c53ba5-faf4-4785-8f3c-fc4bfb648443 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:49:26.212199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e183444-aa84-4b36-87bc-5366dc615602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:49:26.212835 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e183444-aa84-4b36-87bc-5366dc615602 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:49:26.213236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e183444-aa84-4b36-87bc-5366dc615602 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:49:26.213517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e183444-aa84-4b36-87bc-5366dc615602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:49:26.215310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8e183444-aa84-4b36-87bc-5366dc615602 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:49:26.232069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e183444-aa84-4b36-87bc-5366dc615602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:49:26.233579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e183444-aa84-4b36-87bc-5366dc615602 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:49:26.247235 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4392/8754] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:49:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:49:29.960783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22104db-840f-47dc-ae32-e397ec08523b service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:49:29.972680 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22104db-840f-47dc-ae32-e397ec08523b service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:49:29.984930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a0da5-42c4-44b4-a059-67393fcef3b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:49:29.984930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a0da5-42c4-44b4-a059-67393fcef3b3 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:49:29.984930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a0da5-42c4-44b4-a059-67393fcef3b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:49:29.984930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a0da5-42c4-44b4-a059-67393fcef3b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:49:29.985499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f24a0da5-42c4-44b4-a059-67393fcef3b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:49:29.992433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a0da5-42c4-44b4-a059-67393fcef3b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:49:29.993891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a0da5-42c4-44b4-a059-67393fcef3b3 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:49:30.004463 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4363/8755] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:49:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:49:36.074390 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e183444-aa84-4b36-87bc-5366dc615602 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:49:36.091154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e183444-aa84-4b36-87bc-5366dc615602 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:49:36.096724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0bb66-a262-4fe1-89f0-03092b10e204 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:49:36.097367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0bb66-a262-4fe1-89f0-03092b10e204 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:49:36.097657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0bb66-a262-4fe1-89f0-03092b10e204 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:49:36.097970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0bb66-a262-4fe1-89f0-03092b10e204 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:49:36.099933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a4d0bb66-a262-4fe1-89f0-03092b10e204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:49:36.117443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d0bb66-a262-4fe1-89f0-03092b10e204 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:49:36.119370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d0bb66-a262-4fe1-89f0-03092b10e204 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:49:36.134476 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4393/8756] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:49:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:49:38.030718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751363c-fafc-4c20-b105-509e0e11522f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:49:38.031264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751363c-fafc-4c20-b105-509e0e11522f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:49:38.031264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751363c-fafc-4c20-b105-509e0e11522f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:49:38.049075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e751363c-fafc-4c20-b105-509e0e11522f None None] MFA Rules not processed for user `c44b88da38234871ad6732dab5858246`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:49:38.050377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e751363c-fafc-4c20-b105-509e0e11522f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:49:38.057859 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4364/8757] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:49:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:49:38.692119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0bb66-a262-4fe1-89f0-03092b10e204 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:49:38.693619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a4d0bb66-a262-4fe1-89f0-03092b10e204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:49:38.711818 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0bb66-a262-4fe1-89f0-03092b10e204 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:49:38.717263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a486157-e0ad-43ff-bc69-7d3ef1aea6d4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:49:38.718119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a486157-e0ad-43ff-bc69-7d3ef1aea6d4 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:49:38.718478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a486157-e0ad-43ff-bc69-7d3ef1aea6d4 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:49:38.718832 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a486157-e0ad-43ff-bc69-7d3ef1aea6d4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:49:38.720691 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-0a486157-e0ad-43ff-bc69-7d3ef1aea6d4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:49:38.736959 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a486157-e0ad-43ff-bc69-7d3ef1aea6d4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:49:38.738452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a486157-e0ad-43ff-bc69-7d3ef1aea6d4 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:49:38.751450 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4394/8758] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:49:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:49:48.043549 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=15 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:49:48.744652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=18 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:49:51.352954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd25988-57b8-4452-988a-900ee583f8be None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:49:51.353498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd25988-57b8-4452-988a-900ee583f8be None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:49:51.353498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd25988-57b8-4452-988a-900ee583f8be None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:49:51.354854 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4365/8759] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:49:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:49:51.359883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e179c46-a803-40a2-b298-de7abc3950f0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:49:51.360315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e179c46-a803-40a2-b298-de7abc3950f0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:49:51.360526 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e179c46-a803-40a2-b298-de7abc3950f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:49:51.378695 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3e179c46-a803-40a2-b298-de7abc3950f0 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:49:51.379952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3e179c46-a803-40a2-b298-de7abc3950f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:49:51.388765 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4395/8760] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:49:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:49:51.409427 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd25988-57b8-4452-988a-900ee583f8be None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:49:51.426188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd25988-57b8-4452-988a-900ee583f8be None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:49:51.431168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0dc47-a139-4ecc-b0f1-e816b752de9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:49:51.431805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0dc47-a139-4ecc-b0f1-e816b752de9d service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:49:51.432092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0dc47-a139-4ecc-b0f1-e816b752de9d service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:49:51.432372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0dc47-a139-4ecc-b0f1-e816b752de9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:49:51.440821 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0dc47-a139-4ecc-b0f1-e816b752de9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:49:51.442518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0dc47-a139-4ecc-b0f1-e816b752de9d service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:49:51.452300 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4366/8761] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:49:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:50:01.373671 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:50:01.448114 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:50:05.994253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668c047-c727-4e64-9638-1d85d1e055e3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:05.994822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668c047-c727-4e64-9638-1d85d1e055e3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:05.994822 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668c047-c727-4e64-9638-1d85d1e055e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:06.029502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7668c047-c727-4e64-9638-1d85d1e055e3 None None] MFA Rules not processed for user `1b2a9ebcd3f84b758322a80c6e590de0`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:50:06.030725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7668c047-c727-4e64-9638-1d85d1e055e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:50:06.046917 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4396/8762] 10.222.0.33 () {58 vars in 1061 bytes} [Sat May 28 21:50:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:50:06.052754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0dc47-a139-4ecc-b0f1-e816b752de9d service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:50:06.062642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0dc47-a139-4ecc-b0f1-e816b752de9d service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:50:06.066281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fc184-00ba-4238-9c06-844aa6038b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:50:06.066931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fc184-00ba-4238-9c06-844aa6038b7c service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:06.067224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fc184-00ba-4238-9c06-844aa6038b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:06.067502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fc184-00ba-4238-9c06-844aa6038b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:06.069278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-df0fc184-00ba-4238-9c06-844aa6038b7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:50:06.081480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fc184-00ba-4238-9c06-844aa6038b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:50:06.082950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fc184-00ba-4238-9c06-844aa6038b7c service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:50:06.093889 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4367/8763] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:50:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:50:16.043965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=13 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:50:16.087726 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:50:21.322562 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668c047-c727-4e64-9638-1d85d1e055e3 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:50:21.324096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7668c047-c727-4e64-9638-1d85d1e055e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:50:21.343831 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7668c047-c727-4e64-9638-1d85d1e055e3 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:50:21.349399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea43fd-c428-4e9a-8d28-4d19a8f9831c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:50:21.352642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea43fd-c428-4e9a-8d28-4d19a8f9831c service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:21.352642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea43fd-c428-4e9a-8d28-4d19a8f9831c service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:21.352642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea43fd-c428-4e9a-8d28-4d19a8f9831c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:21.352642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e8ea43fd-c428-4e9a-8d28-4d19a8f9831c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:50:21.367116 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea43fd-c428-4e9a-8d28-4d19a8f9831c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:50:21.369273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea43fd-c428-4e9a-8d28-4d19a8f9831c service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:50:21.382592 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4397/8764] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:50:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:50:31.375584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:50:32.001493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fc184-00ba-4238-9c06-844aa6038b7c service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:50:32.023657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fc184-00ba-4238-9c06-844aa6038b7c service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:50:32.028434 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a7079-331b-4747-902b-0b3724228808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:50:32.029320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a7079-331b-4747-902b-0b3724228808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:32.029607 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a7079-331b-4747-902b-0b3724228808 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:32.029950 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a7079-331b-4747-902b-0b3724228808 admin admin] PATH_INFO: `/v3/users/c44b88da38234871ad6732dab5858246` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:32.031184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a7079-331b-4747-902b-0b3724228808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c44b88da38234871ad6732dab5858246)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:50:32.033169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a7079-331b-4747-902b-0b3724228808 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:50:32.190844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bd4a7079-331b-4747-902b-0b3724228808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c44b88da38234871ad6732dab5858246'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:50:32.191431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bd4a7079-331b-4747-902b-0b3724228808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c44b88da38234871ad6732dab5858246'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:50:32.198927 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bd4a7079-331b-4747-902b-0b3724228808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c44b88da38234871ad6732dab5858246'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:50:32.264666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bd4a7079-331b-4747-902b-0b3724228808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c44b88da38234871ad6732dab5858246'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:50:32.265291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-bd4a7079-331b-4747-902b-0b3724228808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c44b88da38234871ad6732dab5858246'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:50:32.283380 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4368/8765] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:50:32 2022] DELETE /identity/v3/users/c44b88da38234871ad6732dab5858246 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:50:32.529575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea43fd-c428-4e9a-8d28-4d19a8f9831c service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:50:32.540969 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea43fd-c428-4e9a-8d28-4d19a8f9831c service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:50:32.598994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db49268-ee08-423d-87b3-d7eb746898f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:50:32.599851 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db49268-ee08-423d-87b3-d7eb746898f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:32.600132 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db49268-ee08-423d-87b3-d7eb746898f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:32.600406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db49268-ee08-423d-87b3-d7eb746898f0 admin admin] PATH_INFO: `/v3/projects/ded5b434b72f4a2ba885f887baa3c35a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:32.601595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db49268-ee08-423d-87b3-d7eb746898f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ded5b434b72f4a2ba885f887baa3c35a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:50:32.603567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db49268-ee08-423d-87b3-d7eb746898f0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:50:32.714907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5db49268-ee08-423d-87b3-d7eb746898f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ded5b434b72f4a2ba885f887baa3c35a', 'request_id': 'req-5db49268-ee08-423d-87b3-d7eb746898f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:50:32.720405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5db49268-ee08-423d-87b3-d7eb746898f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ded5b434b72f4a2ba885f887baa3c35a', 'request_id': 'req-5db49268-ee08-423d-87b3-d7eb746898f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:50:32.788250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5db49268-ee08-423d-87b3-d7eb746898f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ded5b434b72f4a2ba885f887baa3c35a', 'request_id': 'req-5db49268-ee08-423d-87b3-d7eb746898f0'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:50:32.799469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5db49268-ee08-423d-87b3-d7eb746898f0 admin admin] The token cache is being invalidate because project ded5b434b72f4a2ba885f887baa3c35a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:50:32.799762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5db49268-ee08-423d-87b3-d7eb746898f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:50:32.800395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-5db49268-ee08-423d-87b3-d7eb746898f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:50:32.802563 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4398/8766] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:50:32 2022] DELETE /identity/v3/projects/ded5b434b72f4a2ba885f887baa3c35a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:50:32.807772 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879466f-7320-4bf2-ae36-b92e9ab326a1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:32.808065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879466f-7320-4bf2-ae36-b92e9ab326a1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:32.808346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879466f-7320-4bf2-ae36-b92e9ab326a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:32.827074 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1879466f-7320-4bf2-ae36-b92e9ab326a1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:50:32.828267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1879466f-7320-4bf2-ae36-b92e9ab326a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:50:32.886238 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4369/8767] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:50:32.893812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576781d-b6de-4803-abf1-7a0818f22ddb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:32.894131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576781d-b6de-4803-abf1-7a0818f22ddb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:32.894409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576781d-b6de-4803-abf1-7a0818f22ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:32.912365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-2576781d-b6de-4803-abf1-7a0818f22ddb None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:50:32.913552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2576781d-b6de-4803-abf1-7a0818f22ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:50:32.922233 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4399/8768] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:50:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:50:32.926449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879466f-7320-4bf2-ae36-b92e9ab326a1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:50:32.936911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879466f-7320-4bf2-ae36-b92e9ab326a1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:50:32.941449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6dc9e1-5fd6-45a5-b65d-3cbbbb6ad6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:50:32.942147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dc9e1-5fd6-45a5-b65d-3cbbbb6ad6cc admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:32.942420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dc9e1-5fd6-45a5-b65d-3cbbbb6ad6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:32.942730 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6dc9e1-5fd6-45a5-b65d-3cbbbb6ad6cc admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:32.943456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6dc9e1-5fd6-45a5-b65d-3cbbbb6ad6cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:50:32.943765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6dc9e1-5fd6-45a5-b65d-3cbbbb6ad6cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:50:32.945380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6dc9e1-5fd6-45a5-b65d-3cbbbb6ad6cc admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:50:32.958210 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4370/8769] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:50:32.962431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576781d-b6de-4803-abf1-7a0818f22ddb None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:50:32.971683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576781d-b6de-4803-abf1-7a0818f22ddb None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:50:32.976238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fceb05-bde2-4720-9922-30387e40babd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:50:32.976887 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fceb05-bde2-4720-9922-30387e40babd admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:32.977174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fceb05-bde2-4720-9922-30387e40babd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:32.977455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fceb05-bde2-4720-9922-30387e40babd admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:32.978220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fceb05-bde2-4720-9922-30387e40babd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:50:32.980661 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fceb05-bde2-4720-9922-30387e40babd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:50:33.064977 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4400/8770] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:50:32 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:50:33.073715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6dc9e1-5fd6-45a5-b65d-3cbbbb6ad6cc admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:50:33.084333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6dc9e1-5fd6-45a5-b65d-3cbbbb6ad6cc admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:50:33.130811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:50:33.140395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52ecc7-89cd-4461-8bce-b6c69e752c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:50:33.141052 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52ecc7-89cd-4461-8bce-b6c69e752c7c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:33.141338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52ecc7-89cd-4461-8bce-b6c69e752c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:33.145304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52ecc7-89cd-4461-8bce-b6c69e752c7c admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:33.145304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52ecc7-89cd-4461-8bce-b6c69e752c7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:50:33.145304 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b52ecc7-89cd-4461-8bce-b6c69e752c7c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:50:33.222748 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4371/8771] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:50:33 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:50:33.226722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fceb05-bde2-4720-9922-30387e40babd admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:50:33.237658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fceb05-bde2-4720-9922-30387e40babd admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:50:33.242638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533d387-d7fd-4cc2-8cd2-a1036661d033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:50:33.243298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533d387-d7fd-4cc2-8cd2-a1036661d033 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:33.243586 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533d387-d7fd-4cc2-8cd2-a1036661d033 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:33.243874 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533d387-d7fd-4cc2-8cd2-a1036661d033 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:33.245211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533d387-d7fd-4cc2-8cd2-a1036661d033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:50:33.246855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533d387-d7fd-4cc2-8cd2-a1036661d033 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:50:33.257214 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4401/8772] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:50:33.261049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52ecc7-89cd-4461-8bce-b6c69e752c7c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:50:33.271313 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52ecc7-89cd-4461-8bce-b6c69e752c7c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:50:33.276459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09115b69-3d44-4688-95ea-4a6f12f7b96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:50:33.277109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09115b69-3d44-4688-95ea-4a6f12f7b96c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:33.277394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09115b69-3d44-4688-95ea-4a6f12f7b96c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:33.277679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09115b69-3d44-4688-95ea-4a6f12f7b96c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:33.278480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09115b69-3d44-4688-95ea-4a6f12f7b96c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:50:33.283373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09115b69-3d44-4688-95ea-4a6f12f7b96c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:50:33.290866 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4372/8773] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:50:33.295279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533d387-d7fd-4cc2-8cd2-a1036661d033 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:50:33.305207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533d387-d7fd-4cc2-8cd2-a1036661d033 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:50:33.309816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324a84e-4436-49fc-a9f0-45efa349f6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:50:33.310741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324a84e-4436-49fc-a9f0-45efa349f6d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:33.311028 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324a84e-4436-49fc-a9f0-45efa349f6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:33.311313 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324a84e-4436-49fc-a9f0-45efa349f6d6 admin admin] PATH_INFO: `/v3/projects/77ee3a3a7c3c4288984836d79f10e732/users/aa3da8047b124549a05e86b438789dfb/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:33.325860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324a84e-4436-49fc-a9f0-45efa349f6d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77ee3a3a7c3c4288984836d79f10e732, user_id=aa3da8047b124549a05e86b438789dfb, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:50:33.328475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324a84e-4436-49fc-a9f0-45efa349f6d6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:50:33.410889 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4402/8774] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:50:33 2022] PUT /identity/v3/projects/77ee3a3a7c3c4288984836d79f10e732/users/aa3da8047b124549a05e86b438789dfb/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:50:33.415916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cccd98-3022-4740-ab50-5809779d3cb6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:33.416212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cccd98-3022-4740-ab50-5809779d3cb6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:33.417137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cccd98-3022-4740-ab50-5809779d3cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:33.434111 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-06cccd98-3022-4740-ab50-5809779d3cb6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:50:33.435316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-06cccd98-3022-4740-ab50-5809779d3cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:50:33.497046 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4373/8775] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:50:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:50:33.506176 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80ad8c-4684-4539-aa31-0d49549f6ecd None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:33.506468 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80ad8c-4684-4539-aa31-0d49549f6ecd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:33.506752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80ad8c-4684-4539-aa31-0d49549f6ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:33.524722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-eb80ad8c-4684-4539-aa31-0d49549f6ecd None None] MFA Rules not processed for user `1b2a9ebcd3f84b758322a80c6e590de0`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:50:33.526507 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-eb80ad8c-4684-4539-aa31-0d49549f6ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:50:33.547719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:50:33.578039 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4403/8776] 10.222.0.33 () {58 vars in 1061 bytes} [Sat May 28 21:50:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:50:33.584002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cccd98-3022-4740-ab50-5809779d3cb6 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:50:33.593957 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cccd98-3022-4740-ab50-5809779d3cb6 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:50:33.597552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d19e18-c7e5-4bfe-adc1-b1d7180fc925 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:50:33.598224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d19e18-c7e5-4bfe-adc1-b1d7180fc925 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:33.598510 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d19e18-c7e5-4bfe-adc1-b1d7180fc925 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:33.598783 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d19e18-c7e5-4bfe-adc1-b1d7180fc925 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:33.608589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d19e18-c7e5-4bfe-adc1-b1d7180fc925 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:50:33.610050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d19e18-c7e5-4bfe-adc1-b1d7180fc925 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:50:33.619530 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4374/8777] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:50:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:50:38.792167 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb39ec-2bcd-456b-b27d-626b98aeebf4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:38.793072 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb39ec-2bcd-456b-b27d-626b98aeebf4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:38.793530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbb39ec-2bcd-456b-b27d-626b98aeebf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:38.824335 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-7dbb39ec-2bcd-456b-b27d-626b98aeebf4 None None] MFA Rules not processed for user `aa3da8047b124549a05e86b438789dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:50:38.825561 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-7dbb39ec-2bcd-456b-b27d-626b98aeebf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:50:39.067116 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4404/8778] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:50:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:50:39.068060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d19e18-c7e5-4bfe-adc1-b1d7180fc925 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:50:39.068060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-52d19e18-c7e5-4bfe-adc1-b1d7180fc925 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:50:39.068060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d19e18-c7e5-4bfe-adc1-b1d7180fc925 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:50:39.068516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b6476-0cfc-4ace-90cd-c7ac25c74fef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:50:39.068516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b6476-0cfc-4ace-90cd-c7ac25c74fef service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:39.068516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b6476-0cfc-4ace-90cd-c7ac25c74fef service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:39.068516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b6476-0cfc-4ace-90cd-c7ac25c74fef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:39.068771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b6476-0cfc-4ace-90cd-c7ac25c74fef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:50:39.068771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b6476-0cfc-4ace-90cd-c7ac25c74fef service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:50:39.068771 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4375/8779] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:50:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:50:41.558856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb8516-238f-4348-8943-0da1d5d2374f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:41.559275 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb8516-238f-4348-8943-0da1d5d2374f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:41.559448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb8516-238f-4348-8943-0da1d5d2374f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:41.587299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3deb8516-238f-4348-8943-0da1d5d2374f None None] MFA Rules not processed for user `aa3da8047b124549a05e86b438789dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:50:41.588920 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3deb8516-238f-4348-8943-0da1d5d2374f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:50:41.600696 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4405/8780] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:50:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:50:43.379649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fca1a-90d5-43e5-a031-f636db10672e None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:43.380042 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fca1a-90d5-43e5-a031-f636db10672e None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:43.380246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fca1a-90d5-43e5-a031-f636db10672e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:43.399369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-978fca1a-90d5-43e5-a031-f636db10672e None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:50:43.400616 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-978fca1a-90d5-43e5-a031-f636db10672e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:50:43.442544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:50:43.476621 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4376/8781] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:50:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:50:43.489049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb8516-238f-4348-8943-0da1d5d2374f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:50:43.490605 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3deb8516-238f-4348-8943-0da1d5d2374f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:50:43.508589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb8516-238f-4348-8943-0da1d5d2374f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:50:43.513710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415642f0-7d60-4d5f-a419-25b48cdb5469 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:50:43.514402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415642f0-7d60-4d5f-a419-25b48cdb5469 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:43.514693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415642f0-7d60-4d5f-a419-25b48cdb5469 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:43.514977 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415642f0-7d60-4d5f-a419-25b48cdb5469 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:43.524262 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415642f0-7d60-4d5f-a419-25b48cdb5469 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:50:43.525935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415642f0-7d60-4d5f-a419-25b48cdb5469 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:50:43.535973 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4406/8782] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:50:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:50:53.527243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=14 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:50:53.531711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=52 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:50:57.809634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fca1a-90d5-43e5-a031-f636db10672e None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:50:57.835842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fca1a-90d5-43e5-a031-f636db10672e None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:50:57.842194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be8ea8-a12b-42b2-ac92-3e98158cfc0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:50:57.842910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be8ea8-a12b-42b2-ac92-3e98158cfc0d service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:50:57.843201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be8ea8-a12b-42b2-ac92-3e98158cfc0d service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:50:57.843482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be8ea8-a12b-42b2-ac92-3e98158cfc0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:50:57.845338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-09be8ea8-a12b-42b2-ac92-3e98158cfc0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:50:57.861151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be8ea8-a12b-42b2-ac92-3e98158cfc0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:50:57.862903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be8ea8-a12b-42b2-ac92-3e98158cfc0d service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:50:57.878201 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4377/8783] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:50:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:51:00.176166 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da8f63-bcaf-40f7-882b-55fd80d7a859 None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:51:00.176637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da8f63-bcaf-40f7-882b-55fd80d7a859 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:51:00.176637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65da8f63-bcaf-40f7-882b-55fd80d7a859 None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:51:00.178183 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4407/8784] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:51:00.182979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c89292-1fbf-4996-b602-7b343c36cf31 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:51:00.183367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c89292-1fbf-4996-b602-7b343c36cf31 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:51:00.184101 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c89292-1fbf-4996-b602-7b343c36cf31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:51:00.203544 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e8c89292-1fbf-4996-b602-7b343c36cf31 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:51:00.204760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e8c89292-1fbf-4996-b602-7b343c36cf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:51:00.213699 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4378/8785] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:51:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:51:00.234602 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da8f63-bcaf-40f7-882b-55fd80d7a859 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:51:00.236306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-65da8f63-bcaf-40f7-882b-55fd80d7a859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:51:00.256114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65da8f63-bcaf-40f7-882b-55fd80d7a859 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:51:00.261332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08f483-0aba-4fdb-95a0-77920e4ddf62 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:51:00.262074 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08f483-0aba-4fdb-95a0-77920e4ddf62 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:51:00.262367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08f483-0aba-4fdb-95a0-77920e4ddf62 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:51:00.262652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c08f483-0aba-4fdb-95a0-77920e4ddf62 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:51:00.271679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08f483-0aba-4fdb-95a0-77920e4ddf62 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:51:00.273365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c08f483-0aba-4fdb-95a0-77920e4ddf62 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:51:00.284037 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4408/8786] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:51:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:51:00.631893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d72a2f-97f7-43fd-94d9-631f5440b4b0 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:51:00.632184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d72a2f-97f7-43fd-94d9-631f5440b4b0 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:51:00.632464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d72a2f-97f7-43fd-94d9-631f5440b4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:51:00.650904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d5d72a2f-97f7-43fd-94d9-631f5440b4b0 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:51:00.652715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d5d72a2f-97f7-43fd-94d9-631f5440b4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:51:00.661571 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4379/8787] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:51:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:51:00.672679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08f483-0aba-4fdb-95a0-77920e4ddf62 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:51:00.685774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c08f483-0aba-4fdb-95a0-77920e4ddf62 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:51:00.690994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32965e72-860e-4dae-9ead-6c5a6ff625ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:51:00.691639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32965e72-860e-4dae-9ead-6c5a6ff625ba service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:51:00.691928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32965e72-860e-4dae-9ead-6c5a6ff625ba service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:51:00.692241 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32965e72-860e-4dae-9ead-6c5a6ff625ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:51:00.701278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32965e72-860e-4dae-9ead-6c5a6ff625ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:51:00.702919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32965e72-860e-4dae-9ead-6c5a6ff625ba service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:51:00.713069 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4409/8788] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:51:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:51:10.645589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=16 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:51:10.708934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=12 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:51:12.593121 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d72a2f-97f7-43fd-94d9-631f5440b4b0 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:51:12.606856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d72a2f-97f7-43fd-94d9-631f5440b4b0 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:51:12.611873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee096948-fdaf-4571-90b1-6baebde0ddfd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:51:12.612516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee096948-fdaf-4571-90b1-6baebde0ddfd service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:51:12.612801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee096948-fdaf-4571-90b1-6baebde0ddfd service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:51:12.613081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee096948-fdaf-4571-90b1-6baebde0ddfd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:51:12.614890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ee096948-fdaf-4571-90b1-6baebde0ddfd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:51:12.629634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee096948-fdaf-4571-90b1-6baebde0ddfd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:51:12.631264 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee096948-fdaf-4571-90b1-6baebde0ddfd service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:51:12.644487 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4380/8789] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:51:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:51:14.206243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32965e72-860e-4dae-9ead-6c5a6ff625ba service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:51:14.220203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32965e72-860e-4dae-9ead-6c5a6ff625ba service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:51:14.225314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b86a2-43bd-4f3a-98da-6ff560d36a0e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:51:14.226032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b86a2-43bd-4f3a-98da-6ff560d36a0e service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:51:14.226353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b86a2-43bd-4f3a-98da-6ff560d36a0e service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:51:14.226635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b86a2-43bd-4f3a-98da-6ff560d36a0e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:51:14.228433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-012b86a2-43bd-4f3a-98da-6ff560d36a0e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:51:14.242964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b86a2-43bd-4f3a-98da-6ff560d36a0e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:51:14.244545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b86a2-43bd-4f3a-98da-6ff560d36a0e service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:51:14.257673 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4410/8790] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:51:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:51:22.637761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:51:24.250860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:51:31.733848 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2fb28-323d-485e-b80c-2ccd155112ff None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:51:31.734839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2fb28-323d-485e-b80c-2ccd155112ff None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:51:31.735150 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2fb28-323d-485e-b80c-2ccd155112ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:51:31.752741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-09a2fb28-323d-485e-b80c-2ccd155112ff None None] MFA Rules not processed for user `aa3da8047b124549a05e86b438789dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:51:31.754081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-09a2fb28-323d-485e-b80c-2ccd155112ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:51:31.761036 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4381/8791] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:51:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:51:32.064787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b86a2-43bd-4f3a-98da-6ff560d36a0e service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:51:32.074803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b86a2-43bd-4f3a-98da-6ff560d36a0e service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:51:32.078323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e0eee-8036-430c-a455-ba049b890306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:51:32.078965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e0eee-8036-430c-a455-ba049b890306 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:51:32.079252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e0eee-8036-430c-a455-ba049b890306 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:51:32.079533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e0eee-8036-430c-a455-ba049b890306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:51:32.081318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f08e0eee-8036-430c-a455-ba049b890306 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:51:32.094031 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e0eee-8036-430c-a455-ba049b890306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:51:32.095469 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e0eee-8036-430c-a455-ba049b890306 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:51:32.106510 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4411/8792] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:51:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:51:41.747463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:51:42.099944 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:52:39.065770 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55a96a-340f-4166-8e63-5a5be6489758 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:52:39.066301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55a96a-340f-4166-8e63-5a5be6489758 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:52:39.066397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55a96a-340f-4166-8e63-5a5be6489758 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:52:39.067808 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4382/8793] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:52:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:52:39.072789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5d49b-8e4b-46a5-ae97-1c3aec2f8fe5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:52:39.073283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5d49b-8e4b-46a5-ae97-1c3aec2f8fe5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:52:39.073430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5d49b-8e4b-46a5-ae97-1c3aec2f8fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:52:39.092552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6fe5d49b-8e4b-46a5-ae97-1c3aec2f8fe5 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:52:39.101369 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6fe5d49b-8e4b-46a5-ae97-1c3aec2f8fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:52:39.102529 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4412/8794] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:52:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:52:39.122764 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55a96a-340f-4166-8e63-5a5be6489758 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:52:39.140594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55a96a-340f-4166-8e63-5a5be6489758 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:52:39.146263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cffc243-2d46-4b5c-b4a1-ecaf40833190 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:52:39.146900 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cffc243-2d46-4b5c-b4a1-ecaf40833190 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:52:39.147187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cffc243-2d46-4b5c-b4a1-ecaf40833190 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:52:39.147476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cffc243-2d46-4b5c-b4a1-ecaf40833190 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:52:39.156041 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cffc243-2d46-4b5c-b4a1-ecaf40833190 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:52:39.157697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cffc243-2d46-4b5c-b4a1-ecaf40833190 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:52:39.167712 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4383/8795] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:52:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:52:49.087344 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:52:49.163518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:52:57.880135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a451f4d-42b0-4402-bff3-9e0e406b20b4 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:52:57.880721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a451f4d-42b0-4402-bff3-9e0e406b20b4 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:52:57.881174 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a451f4d-42b0-4402-bff3-9e0e406b20b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:52:57.900375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5a451f4d-42b0-4402-bff3-9e0e406b20b4 None None] MFA Rules not processed for user `aa3da8047b124549a05e86b438789dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:52:57.901087 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5a451f4d-42b0-4402-bff3-9e0e406b20b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:52:57.908945 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4413/8796] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:52:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:53:07.894683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:53:45.823383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cffc243-2d46-4b5c-b4a1-ecaf40833190 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:53:45.842651 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cffc243-2d46-4b5c-b4a1-ecaf40833190 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:53:45.846330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87cf89-81b1-428e-a5fa-fe50359451dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:53:45.846973 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87cf89-81b1-428e-a5fa-fe50359451dc service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:53:45.847256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87cf89-81b1-428e-a5fa-fe50359451dc service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:53:45.847539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87cf89-81b1-428e-a5fa-fe50359451dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:53:45.849329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3e87cf89-81b1-428e-a5fa-fe50359451dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:53:45.893420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87cf89-81b1-428e-a5fa-fe50359451dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:53:45.894839 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87cf89-81b1-428e-a5fa-fe50359451dc service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:53:45.905431 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4384/8797] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:53:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:53:46.062570 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dafc1-95ae-42a7-ba36-bce819e168bd None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:53:46.063055 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dafc1-95ae-42a7-ba36-bce819e168bd None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:53:46.063272 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dafc1-95ae-42a7-ba36-bce819e168bd None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:53:46.064650 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4414/8798] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:53:46.069729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadcb14-fd4c-4118-ac9a-37ada36521b3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:53:46.070221 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadcb14-fd4c-4118-ac9a-37ada36521b3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:53:46.070511 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadcb14-fd4c-4118-ac9a-37ada36521b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:53:46.089314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ffadcb14-fd4c-4118-ac9a-37ada36521b3 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:53:46.090530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ffadcb14-fd4c-4118-ac9a-37ada36521b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:53:46.119300 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4385/8799] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:53:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:53:46.141966 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0dafc1-95ae-42a7-ba36-bce819e168bd None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:53:46.158698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0dafc1-95ae-42a7-ba36-bce819e168bd None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:53:46.164613 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f2845-5fb8-405e-985c-0f4c02230c8d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:53:46.168706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f2845-5fb8-405e-985c-0f4c02230c8d service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:53:46.168706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f2845-5fb8-405e-985c-0f4c02230c8d service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:53:46.168706 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f2845-5fb8-405e-985c-0f4c02230c8d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:53:46.174581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f2845-5fb8-405e-985c-0f4c02230c8d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:53:46.176153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f2845-5fb8-405e-985c-0f4c02230c8d service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:53:46.185820 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4415/8800] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:53:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:53:46.479471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadcb14-fd4c-4118-ac9a-37ada36521b3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:53:46.480995 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ffadcb14-fd4c-4118-ac9a-37ada36521b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:53:46.492517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffadcb14-fd4c-4118-ac9a-37ada36521b3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:53:46.496170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3395fd-f0f6-444d-a708-ccf5ce113f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:53:46.496813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3395fd-f0f6-444d-a708-ccf5ce113f4e service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:53:46.497100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3395fd-f0f6-444d-a708-ccf5ce113f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:53:46.497379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3395fd-f0f6-444d-a708-ccf5ce113f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:53:46.499128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5f3395fd-f0f6-444d-a708-ccf5ce113f4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:53:46.515639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3395fd-f0f6-444d-a708-ccf5ce113f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:53:46.517094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3395fd-f0f6-444d-a708-ccf5ce113f4e service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:53:46.531372 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4386/8801] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:53:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:53:56.181764 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:53:56.523908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=42 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:54:22.487437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2886442-5c47-4f99-a805-2d2ba6755629 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:54:22.487925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2886442-5c47-4f99-a805-2d2ba6755629 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:54:22.488337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2886442-5c47-4f99-a805-2d2ba6755629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:54:22.507365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b2886442-5c47-4f99-a805-2d2ba6755629 None None] MFA Rules not processed for user `aa3da8047b124549a05e86b438789dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:54:22.508582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b2886442-5c47-4f99-a805-2d2ba6755629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:54:22.515638 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4416/8802] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:54:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:54:22.824462 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3395fd-f0f6-444d-a708-ccf5ce113f4e service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:54:22.833188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3395fd-f0f6-444d-a708-ccf5ce113f4e service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:54:22.836817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0e8b2-bf8f-41c4-91ef-9134d611e2bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:54:22.837454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0e8b2-bf8f-41c4-91ef-9134d611e2bb service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:54:22.837747 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0e8b2-bf8f-41c4-91ef-9134d611e2bb service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:54:22.838073 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0e8b2-bf8f-41c4-91ef-9134d611e2bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:54:22.839841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4ab0e8b2-bf8f-41c4-91ef-9134d611e2bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:54:22.851735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab0e8b2-bf8f-41c4-91ef-9134d611e2bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:54:22.853165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab0e8b2-bf8f-41c4-91ef-9134d611e2bb service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:54:22.864017 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4387/8803] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:54:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:54:25.463930 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2886442-5c47-4f99-a805-2d2ba6755629 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:54:25.475889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2886442-5c47-4f99-a805-2d2ba6755629 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:54:25.477587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb4a25-e0f8-4dd4-a243-8edc680043b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:54:25.478378 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb4a25-e0f8-4dd4-a243-8edc680043b0 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:54:25.478663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb4a25-e0f8-4dd4-a243-8edc680043b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:54:25.478949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb4a25-e0f8-4dd4-a243-8edc680043b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:54:25.480749 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c6eb4a25-e0f8-4dd4-a243-8edc680043b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:54:25.518094 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb4a25-e0f8-4dd4-a243-8edc680043b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:54:25.519582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb4a25-e0f8-4dd4-a243-8edc680043b0 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:54:25.533672 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4417/8804] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:54:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:54:26.646914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0e8b2-bf8f-41c4-91ef-9134d611e2bb service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:54:26.657685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0e8b2-bf8f-41c4-91ef-9134d611e2bb service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:54:26.661689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d420de9-2852-4d0b-887e-07f82578857e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:54:26.662594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d420de9-2852-4d0b-887e-07f82578857e service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:54:26.663030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d420de9-2852-4d0b-887e-07f82578857e service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:54:26.663592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d420de9-2852-4d0b-887e-07f82578857e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:54:26.666171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9d420de9-2852-4d0b-887e-07f82578857e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:54:26.688010 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d420de9-2852-4d0b-887e-07f82578857e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:54:26.689494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d420de9-2852-4d0b-887e-07f82578857e service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:54:26.708351 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4388/8805] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:54:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:54:35.526070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=17 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:54:36.699612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=12 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:55:09.961639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb4a25-e0f8-4dd4-a243-8edc680043b0 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:55:09.970481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb4a25-e0f8-4dd4-a243-8edc680043b0 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:55:09.974703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7065975-f93d-4dd7-b65d-3044427b5133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:55:09.975345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7065975-f93d-4dd7-b65d-3044427b5133 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:55:09.975629 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7065975-f93d-4dd7-b65d-3044427b5133 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:55:09.975913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7065975-f93d-4dd7-b65d-3044427b5133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:55:09.978178 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d7065975-f93d-4dd7-b65d-3044427b5133 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:55:09.991909 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7065975-f93d-4dd7-b65d-3044427b5133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:55:09.993345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7065975-f93d-4dd7-b65d-3044427b5133 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:55:10.004581 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4418/8806] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:55:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:55:15.453352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eda449-95f5-4a2d-8f16-4ff6bc60f6b3 None None] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:55:15.453933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eda449-95f5-4a2d-8f16-4ff6bc60f6b3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:55:15.454287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eda449-95f5-4a2d-8f16-4ff6bc60f6b3 None None] PATH_INFO: `/` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:55:15.455724 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4389/8807] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 21:55:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 28 21:55:15.460644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec06aa-5c95-4ee1-8772-fe4143267a76 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:55:15.461086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec06aa-5c95-4ee1-8772-fe4143267a76 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:55:15.461217 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec06aa-5c95-4ee1-8772-fe4143267a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:55:15.501904 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-12ec06aa-5c95-4ee1-8772-fe4143267a76 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:55:15.503126 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-12ec06aa-5c95-4ee1-8772-fe4143267a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:55:15.511809 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4419/8808] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 21:55:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:55:15.532531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eda449-95f5-4a2d-8f16-4ff6bc60f6b3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:55:15.549224 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eda449-95f5-4a2d-8f16-4ff6bc60f6b3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:55:15.554520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6252a-91e3-4a83-9e09-a5adb0bf6600 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:55:15.555166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6252a-91e3-4a83-9e09-a5adb0bf6600 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:55:15.555477 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6252a-91e3-4a83-9e09-a5adb0bf6600 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:55:15.555760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6252a-91e3-4a83-9e09-a5adb0bf6600 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:55:15.564401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6252a-91e3-4a83-9e09-a5adb0bf6600 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:55:15.566028 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6252a-91e3-4a83-9e09-a5adb0bf6600 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:55:15.575751 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4390/8809] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:55:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:55:25.496617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=18 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:55:25.571675 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:55:31.446899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec06aa-5c95-4ee1-8772-fe4143267a76 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:55:31.456635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec06aa-5c95-4ee1-8772-fe4143267a76 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:55:31.460287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b8df7-76d8-4520-af10-1cb125ee668a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:55:31.461225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b8df7-76d8-4520-af10-1cb125ee668a service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:55:31.461516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b8df7-76d8-4520-af10-1cb125ee668a service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:55:31.461801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b8df7-76d8-4520-af10-1cb125ee668a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:55:31.463635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-766b8df7-76d8-4520-af10-1cb125ee668a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:55:31.476803 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b8df7-76d8-4520-af10-1cb125ee668a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:55:31.478283 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766b8df7-76d8-4520-af10-1cb125ee668a service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:55:31.490419 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4420/8810] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:55:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:55:32.003930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6252a-91e3-4a83-9e09-a5adb0bf6600 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:55:32.005561 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-acb6252a-91e3-4a83-9e09-a5adb0bf6600 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:55:32.024782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6252a-91e3-4a83-9e09-a5adb0bf6600 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:55:32.030268 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8319729-51be-4854-a74a-c8eb71baf318 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:55:32.030906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8319729-51be-4854-a74a-c8eb71baf318 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:55:32.031212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8319729-51be-4854-a74a-c8eb71baf318 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:55:32.031515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8319729-51be-4854-a74a-c8eb71baf318 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:55:32.033316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e8319729-51be-4854-a74a-c8eb71baf318 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:55:32.048489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8319729-51be-4854-a74a-c8eb71baf318 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:55:32.050262 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8319729-51be-4854-a74a-c8eb71baf318 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:55:32.064916 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4391/8811] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:55:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:55:39.839598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b8df7-76d8-4520-af10-1cb125ee668a service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:55:39.855687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b8df7-76d8-4520-af10-1cb125ee668a service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:55:39.861872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64ac0f-85e1-42da-97dc-a4584db3561f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:55:39.862513 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64ac0f-85e1-42da-97dc-a4584db3561f service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:55:39.862801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64ac0f-85e1-42da-97dc-a4584db3561f service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:55:39.863083 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64ac0f-85e1-42da-97dc-a4584db3561f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:55:39.870992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f64ac0f-85e1-42da-97dc-a4584db3561f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:55:39.872641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f64ac0f-85e1-42da-97dc-a4584db3561f service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:55:39.882435 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4421/8812] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:55:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:55:41.291256 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8319729-51be-4854-a74a-c8eb71baf318 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:55:41.305204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8319729-51be-4854-a74a-c8eb71baf318 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:55:41.311055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e865c-01bb-4588-ae5e-e67d0b7cbc64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:55:41.311699 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e865c-01bb-4588-ae5e-e67d0b7cbc64 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:55:41.312135 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e865c-01bb-4588-ae5e-e67d0b7cbc64 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:55:41.314749 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e865c-01bb-4588-ae5e-e67d0b7cbc64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:55:41.315246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c87e865c-01bb-4588-ae5e-e67d0b7cbc64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:55:41.330594 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e865c-01bb-4588-ae5e-e67d0b7cbc64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:55:41.332211 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87e865c-01bb-4588-ae5e-e67d0b7cbc64 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:55:41.345409 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4392/8813] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:55:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:55:49.878169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=12 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:55:51.338508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=12 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:56:01.161961 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64ac0f-85e1-42da-97dc-a4584db3561f service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:01.171284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64ac0f-85e1-42da-97dc-a4584db3561f service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:01.175421 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45fa4e-3f86-4858-9fe1-f18f47cc53bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:01.176123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45fa4e-3f86-4858-9fe1-f18f47cc53bc service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:01.176446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45fa4e-3f86-4858-9fe1-f18f47cc53bc service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:01.176729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45fa4e-3f86-4858-9fe1-f18f47cc53bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:01.185950 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45fa4e-3f86-4858-9fe1-f18f47cc53bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:01.187426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45fa4e-3f86-4858-9fe1-f18f47cc53bc service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:01.197440 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4422/8814] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:56:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:56:04.718977 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e865c-01bb-4588-ae5e-e67d0b7cbc64 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:04.733895 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e865c-01bb-4588-ae5e-e67d0b7cbc64 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:04.739284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f4bb4-5683-4b21-88fb-dfbb46e0a044 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:04.739944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f4bb4-5683-4b21-88fb-dfbb46e0a044 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:04.740233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f4bb4-5683-4b21-88fb-dfbb46e0a044 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:04.740515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f4bb4-5683-4b21-88fb-dfbb46e0a044 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:04.742402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-305f4bb4-5683-4b21-88fb-dfbb46e0a044 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:04.757060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f4bb4-5683-4b21-88fb-dfbb46e0a044 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:04.758696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f4bb4-5683-4b21-88fb-dfbb46e0a044 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:04.773094 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4393/8815] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:56:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 21:56:04.841779 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45fa4e-3f86-4858-9fe1-f18f47cc53bc service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:04.852630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45fa4e-3f86-4858-9fe1-f18f47cc53bc service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:04.857361 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67421eb3-7092-493d-864e-9e2c07895c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:04.858270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67421eb3-7092-493d-864e-9e2c07895c44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:04.858588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67421eb3-7092-493d-864e-9e2c07895c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:04.858879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67421eb3-7092-493d-864e-9e2c07895c44 admin admin] PATH_INFO: `/v3/users/aa3da8047b124549a05e86b438789dfb` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:04.860086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67421eb3-7092-493d-864e-9e2c07895c44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa3da8047b124549a05e86b438789dfb)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:04.862073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67421eb3-7092-493d-864e-9e2c07895c44 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:05.057978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-67421eb3-7092-493d-864e-9e2c07895c44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa3da8047b124549a05e86b438789dfb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:05.065729 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-67421eb3-7092-493d-864e-9e2c07895c44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa3da8047b124549a05e86b438789dfb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:05.066859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-67421eb3-7092-493d-864e-9e2c07895c44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa3da8047b124549a05e86b438789dfb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:05.067484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-67421eb3-7092-493d-864e-9e2c07895c44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa3da8047b124549a05e86b438789dfb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:05.147207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-67421eb3-7092-493d-864e-9e2c07895c44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa3da8047b124549a05e86b438789dfb'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:05.164863 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4423/8816] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:56:04 2022] DELETE /identity/v3/users/aa3da8047b124549a05e86b438789dfb => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:56:05.397175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f4bb4-5683-4b21-88fb-dfbb46e0a044 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:05.409896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f4bb4-5683-4b21-88fb-dfbb46e0a044 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:05.477170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb9c73b-be12-4aa8-ac49-b1cc5e9ed2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:05.478062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9c73b-be12-4aa8-ac49-b1cc5e9ed2d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:05.478346 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9c73b-be12-4aa8-ac49-b1cc5e9ed2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:05.478653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb9c73b-be12-4aa8-ac49-b1cc5e9ed2d6 admin admin] PATH_INFO: `/v3/projects/77ee3a3a7c3c4288984836d79f10e732` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:05.479828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9c73b-be12-4aa8-ac49-b1cc5e9ed2d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77ee3a3a7c3c4288984836d79f10e732)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:05.481745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb9c73b-be12-4aa8-ac49-b1cc5e9ed2d6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:05.606394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fbb9c73b-be12-4aa8-ac49-b1cc5e9ed2d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77ee3a3a7c3c4288984836d79f10e732', 'request_id': 'req-fbb9c73b-be12-4aa8-ac49-b1cc5e9ed2d6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:05.612737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fbb9c73b-be12-4aa8-ac49-b1cc5e9ed2d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77ee3a3a7c3c4288984836d79f10e732', 'request_id': 'req-fbb9c73b-be12-4aa8-ac49-b1cc5e9ed2d6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:05.678597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fbb9c73b-be12-4aa8-ac49-b1cc5e9ed2d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77ee3a3a7c3c4288984836d79f10e732', 'request_id': 'req-fbb9c73b-be12-4aa8-ac49-b1cc5e9ed2d6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:05.702587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fbb9c73b-be12-4aa8-ac49-b1cc5e9ed2d6 admin admin] The token cache is being invalidate because project 77ee3a3a7c3c4288984836d79f10e732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:56:05.702869 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fbb9c73b-be12-4aa8-ac49-b1cc5e9ed2d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:05.703757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-fbb9c73b-be12-4aa8-ac49-b1cc5e9ed2d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:05.705319 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4394/8817] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:56:05 2022] DELETE /identity/v3/projects/77ee3a3a7c3c4288984836d79f10e732 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:56:05.710345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1b913-e3a0-4c5a-9ffb-c8b113742230 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:05.710642 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1b913-e3a0-4c5a-9ffb-c8b113742230 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:05.710949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1b913-e3a0-4c5a-9ffb-c8b113742230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:05.759532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-75e1b913-e3a0-4c5a-9ffb-c8b113742230 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:56:05.760895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-75e1b913-e3a0-4c5a-9ffb-c8b113742230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:05.821427 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4424/8818] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:56:05.830098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc1705-c9e5-4546-b664-3d0ba894764b None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:05.830536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc1705-c9e5-4546-b664-3d0ba894764b None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:05.830691 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc1705-c9e5-4546-b664-3d0ba894764b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:05.848643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-36bc1705-c9e5-4546-b664-3d0ba894764b None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:56:05.849883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-36bc1705-c9e5-4546-b664-3d0ba894764b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:05.858067 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4395/8819] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:56:05.862678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1b913-e3a0-4c5a-9ffb-c8b113742230 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:05.872404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1b913-e3a0-4c5a-9ffb-c8b113742230 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:05.877039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de5319-24a1-4825-a067-ebb2ea7ce9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:05.877725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de5319-24a1-4825-a067-ebb2ea7ce9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:05.878057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de5319-24a1-4825-a067-ebb2ea7ce9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:05.878341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de5319-24a1-4825-a067-ebb2ea7ce9f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:05.879081 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de5319-24a1-4825-a067-ebb2ea7ce9f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:56:05.879399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de5319-24a1-4825-a067-ebb2ea7ce9f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:05.881011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de5319-24a1-4825-a067-ebb2ea7ce9f4 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:05.892626 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4425/8820] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:56:05.896787 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc1705-c9e5-4546-b664-3d0ba894764b None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:05.906063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc1705-c9e5-4546-b664-3d0ba894764b None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:05.911246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4315bf-b67f-4bd2-a502-19289583442a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:05.911916 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4315bf-b67f-4bd2-a502-19289583442a admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:05.912198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4315bf-b67f-4bd2-a502-19289583442a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:05.912476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4315bf-b67f-4bd2-a502-19289583442a admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:05.913171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4315bf-b67f-4bd2-a502-19289583442a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:05.915110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4315bf-b67f-4bd2-a502-19289583442a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:06.005630 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4396/8821] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:56:05 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:56:06.009709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de5319-24a1-4825-a067-ebb2ea7ce9f4 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:06.020858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de5319-24a1-4825-a067-ebb2ea7ce9f4 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:06.042040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:56:06.078970 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574e73e-bd38-4320-a3a1-5c0538504198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:06.079679 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574e73e-bd38-4320-a3a1-5c0538504198 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:06.083357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574e73e-bd38-4320-a3a1-5c0538504198 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:06.083357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574e73e-bd38-4320-a3a1-5c0538504198 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:06.083357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574e73e-bd38-4320-a3a1-5c0538504198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:06.083638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574e73e-bd38-4320-a3a1-5c0538504198 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:06.184952 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4426/8822] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:56:06 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:56:06.186440 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4315bf-b67f-4bd2-a502-19289583442a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:06.196948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4315bf-b67f-4bd2-a502-19289583442a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:06.202024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c5dbc-689b-4ed1-baa3-b5c711d420aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:06.202729 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c5dbc-689b-4ed1-baa3-b5c711d420aa admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:06.203023 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c5dbc-689b-4ed1-baa3-b5c711d420aa admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:06.203566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c5dbc-689b-4ed1-baa3-b5c711d420aa admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:06.204311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c5dbc-689b-4ed1-baa3-b5c711d420aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:06.206657 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937c5dbc-689b-4ed1-baa3-b5c711d420aa admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:06.216875 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4397/8823] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:56:06.220848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574e73e-bd38-4320-a3a1-5c0538504198 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:06.231554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574e73e-bd38-4320-a3a1-5c0538504198 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:06.236298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed9d215-5a30-4ecb-a00b-dbf2a5d967c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:06.236962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9d215-5a30-4ecb-a00b-dbf2a5d967c7 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:06.237248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9d215-5a30-4ecb-a00b-dbf2a5d967c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:06.237531 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed9d215-5a30-4ecb-a00b-dbf2a5d967c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:06.238317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9d215-5a30-4ecb-a00b-dbf2a5d967c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:06.239932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed9d215-5a30-4ecb-a00b-dbf2a5d967c7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:06.249831 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4427/8824] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:56:06.253817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c5dbc-689b-4ed1-baa3-b5c711d420aa admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:06.269780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c5dbc-689b-4ed1-baa3-b5c711d420aa admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:06.269780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccd7ca-8dd2-4316-89d1-0b7d25c80704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:06.270314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccd7ca-8dd2-4316-89d1-0b7d25c80704 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:06.270587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccd7ca-8dd2-4316-89d1-0b7d25c80704 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:06.270898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eccd7ca-8dd2-4316-89d1-0b7d25c80704 admin admin] PATH_INFO: `/v3/projects/22e09bcdc3974f888d75cc14a7cd098a/users/65d8da19cff64475bd6cb5ce24ae530b/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:06.285948 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eccd7ca-8dd2-4316-89d1-0b7d25c80704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22e09bcdc3974f888d75cc14a7cd098a, user_id=65d8da19cff64475bd6cb5ce24ae530b, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:06.288719 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eccd7ca-8dd2-4316-89d1-0b7d25c80704 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:06.290904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:56:06.387922 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4398/8825] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:56:06 2022] PUT /identity/v3/projects/22e09bcdc3974f888d75cc14a7cd098a/users/65d8da19cff64475bd6cb5ce24ae530b/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:56:06.393624 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b6cb5-2f19-4ca9-8d54-26d47755daaf None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:06.393945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b6cb5-2f19-4ca9-8d54-26d47755daaf None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:06.394234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b6cb5-2f19-4ca9-8d54-26d47755daaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:06.413007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d92b6cb5-2f19-4ca9-8d54-26d47755daaf None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:56:06.414252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d92b6cb5-2f19-4ca9-8d54-26d47755daaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:06.472846 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4428/8826] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:56:06.482566 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccd7ca-8dd2-4316-89d1-0b7d25c80704 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:06.484301 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6eccd7ca-8dd2-4316-89d1-0b7d25c80704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:06.541325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eccd7ca-8dd2-4316-89d1-0b7d25c80704 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:06.544964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13e565-b4e9-4abb-a312-16cea0368cc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:06.545606 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13e565-b4e9-4abb-a312-16cea0368cc9 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:06.545923 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13e565-b4e9-4abb-a312-16cea0368cc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:06.546205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13e565-b4e9-4abb-a312-16cea0368cc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:06.555263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13e565-b4e9-4abb-a312-16cea0368cc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:06.556698 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13e565-b4e9-4abb-a312-16cea0368cc9 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:06.567092 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4399/8827] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:56:07.831382 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a2e3b-19b1-45e3-b511-4201923328ab None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:07.831848 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a2e3b-19b1-45e3-b511-4201923328ab None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:07.832008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a2e3b-19b1-45e3-b511-4201923328ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:07.860486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-e65a2e3b-19b1-45e3-b511-4201923328ab None None] MFA Rules not processed for user `65d8da19cff64475bd6cb5ce24ae530b`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:56:07.861733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e65a2e3b-19b1-45e3-b511-4201923328ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:07.930875 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4429/8828] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:56:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:56:07.942540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a51e38-d2b0-469f-bf8f-5ef37623d568 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:07.943046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a51e38-d2b0-469f-bf8f-5ef37623d568 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:07.943139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a51e38-d2b0-469f-bf8f-5ef37623d568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:07.960926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-c7a51e38-d2b0-469f-bf8f-5ef37623d568 None None] MFA Rules not processed for user `65d8da19cff64475bd6cb5ce24ae530b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:56:07.962075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c7a51e38-d2b0-469f-bf8f-5ef37623d568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:07.968911 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4400/8829] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:56:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:56:07.983978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a2e3b-19b1-45e3-b511-4201923328ab None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:07.985392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e65a2e3b-19b1-45e3-b511-4201923328ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:08.021135 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:56:08.061243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65a2e3b-19b1-45e3-b511-4201923328ab None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:08.067845 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35280bcc-32e7-4981-9630-11bab4a76b0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:08.068482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35280bcc-32e7-4981-9630-11bab4a76b0d service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:08.068703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35280bcc-32e7-4981-9630-11bab4a76b0d service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:08.069130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35280bcc-32e7-4981-9630-11bab4a76b0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:08.083502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35280bcc-32e7-4981-9630-11bab4a76b0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:08.085122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35280bcc-32e7-4981-9630-11bab4a76b0d service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:08.094439 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4430/8830] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:56:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:56:09.713432 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a51e38-d2b0-469f-bf8f-5ef37623d568 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:09.714979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c7a51e38-d2b0-469f-bf8f-5ef37623d568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:09.791259 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a51e38-d2b0-469f-bf8f-5ef37623d568 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:09.796254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f132e-0761-46d1-bcab-5e9485d63659 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:09.796908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f132e-0761-46d1-bcab-5e9485d63659 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:09.797191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f132e-0761-46d1-bcab-5e9485d63659 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:09.797470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f132e-0761-46d1-bcab-5e9485d63659 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:09.799898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3a1f132e-0761-46d1-bcab-5e9485d63659 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:09.811299 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f132e-0761-46d1-bcab-5e9485d63659 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:09.813265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1f132e-0761-46d1-bcab-5e9485d63659 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:09.824524 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4401/8831] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:56:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:56:13.495814 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35280bcc-32e7-4981-9630-11bab4a76b0d service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:13.497349 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-35280bcc-32e7-4981-9630-11bab4a76b0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:13.515615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35280bcc-32e7-4981-9630-11bab4a76b0d service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:13.521320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23b000f-1aed-435e-a8af-bd5e632937dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:13.522637 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b000f-1aed-435e-a8af-bd5e632937dd service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:13.522925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b000f-1aed-435e-a8af-bd5e632937dd service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:13.523209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b000f-1aed-435e-a8af-bd5e632937dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:13.525036 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f23b000f-1aed-435e-a8af-bd5e632937dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:13.540670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23b000f-1aed-435e-a8af-bd5e632937dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:13.542411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23b000f-1aed-435e-a8af-bd5e632937dd service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:13.555786 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4431/8832] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:56:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:56:19.555096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f132e-0761-46d1-bcab-5e9485d63659 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:19.568339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f132e-0761-46d1-bcab-5e9485d63659 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:19.574193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddfb9d-f44a-4cc1-b599-b298f2260857 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:19.574843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddfb9d-f44a-4cc1-b599-b298f2260857 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:19.575128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddfb9d-f44a-4cc1-b599-b298f2260857 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:19.575404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddfb9d-f44a-4cc1-b599-b298f2260857 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:19.577164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f7ddfb9d-f44a-4cc1-b599-b298f2260857 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:19.592372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ddfb9d-f44a-4cc1-b599-b298f2260857 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:19.594126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ddfb9d-f44a-4cc1-b599-b298f2260857 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:19.607293 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4402/8833] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:56:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:56:21.306095 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017d309-1ffa-4d1d-bfc3-30b9a9b2e31d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:21.306647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017d309-1ffa-4d1d-bfc3-30b9a9b2e31d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:21.306754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017d309-1ffa-4d1d-bfc3-30b9a9b2e31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:21.325770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c017d309-1ffa-4d1d-bfc3-30b9a9b2e31d None None] MFA Rules not processed for user `65d8da19cff64475bd6cb5ce24ae530b`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:56:21.326963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c017d309-1ffa-4d1d-bfc3-30b9a9b2e31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:21.337067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddfb9d-f44a-4cc1-b599-b298f2260857 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:21.337685 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4432/8834] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:56:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:56:21.338588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-f7ddfb9d-f44a-4cc1-b599-b298f2260857 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:21.355389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddfb9d-f44a-4cc1-b599-b298f2260857 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:21.360750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48105fbf-1230-4d7d-8582-a5474b9a1352 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:21.361658 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48105fbf-1230-4d7d-8582-a5474b9a1352 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:21.362047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48105fbf-1230-4d7d-8582-a5474b9a1352 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:21.362398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48105fbf-1230-4d7d-8582-a5474b9a1352 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:21.364436 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-48105fbf-1230-4d7d-8582-a5474b9a1352 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:21.379689 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48105fbf-1230-4d7d-8582-a5474b9a1352 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:21.381311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48105fbf-1230-4d7d-8582-a5474b9a1352 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:21.403653 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4403/8835] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:56:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:56:21.974558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017d309-1ffa-4d1d-bfc3-30b9a9b2e31d None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:21.989949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017d309-1ffa-4d1d-bfc3-30b9a9b2e31d None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:21.995827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b4524-a413-485d-93ae-6f2a12326bb0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:21.996563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b4524-a413-485d-93ae-6f2a12326bb0 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:21.996855 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b4524-a413-485d-93ae-6f2a12326bb0 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:21.997145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5b4524-a413-485d-93ae-6f2a12326bb0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:21.999015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bc5b4524-a413-485d-93ae-6f2a12326bb0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:22.011857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b4524-a413-485d-93ae-6f2a12326bb0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:22.013588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5b4524-a413-485d-93ae-6f2a12326bb0 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:22.027468 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4433/8836] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:56:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:56:31.388105 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=1,keystone:SELECT=75 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:56:32.020299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=34 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:56:43.538115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48105fbf-1230-4d7d-8582-a5474b9a1352 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:43.548718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48105fbf-1230-4d7d-8582-a5474b9a1352 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:43.553442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb0b199-dd27-4ae5-a407-ac98f733bc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:43.554428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0b199-dd27-4ae5-a407-ac98f733bc72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:43.554713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0b199-dd27-4ae5-a407-ac98f733bc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:43.554991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb0b199-dd27-4ae5-a407-ac98f733bc72 admin admin] PATH_INFO: `/v3/users/65d8da19cff64475bd6cb5ce24ae530b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:43.556183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0b199-dd27-4ae5-a407-ac98f733bc72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65d8da19cff64475bd6cb5ce24ae530b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:43.558163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb0b199-dd27-4ae5-a407-ac98f733bc72 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:43.754805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1cb0b199-dd27-4ae5-a407-ac98f733bc72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65d8da19cff64475bd6cb5ce24ae530b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:43.755513 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1cb0b199-dd27-4ae5-a407-ac98f733bc72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65d8da19cff64475bd6cb5ce24ae530b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:43.762687 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1cb0b199-dd27-4ae5-a407-ac98f733bc72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65d8da19cff64475bd6cb5ce24ae530b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:43.849731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1cb0b199-dd27-4ae5-a407-ac98f733bc72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65d8da19cff64475bd6cb5ce24ae530b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:43.854026 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-1cb0b199-dd27-4ae5-a407-ac98f733bc72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65d8da19cff64475bd6cb5ce24ae530b'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:43.868926 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4404/8837] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:56:43 2022] DELETE /identity/v3/users/65d8da19cff64475bd6cb5ce24ae530b => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:56:44.110860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b4524-a413-485d-93ae-6f2a12326bb0 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:44.123129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5b4524-a413-485d-93ae-6f2a12326bb0 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:44.181254 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f136d8-b20c-4458-96b1-d2677be3397b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:44.182161 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f136d8-b20c-4458-96b1-d2677be3397b admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:44.182452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f136d8-b20c-4458-96b1-d2677be3397b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:44.182733 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f136d8-b20c-4458-96b1-d2677be3397b admin admin] PATH_INFO: `/v3/projects/22e09bcdc3974f888d75cc14a7cd098a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:44.183955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f136d8-b20c-4458-96b1-d2677be3397b admin admin] RBAC: Authorizing `identity:delete_project(project_id=22e09bcdc3974f888d75cc14a7cd098a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:44.185948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f136d8-b20c-4458-96b1-d2677be3397b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:44.298043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-58f136d8-b20c-4458-96b1-d2677be3397b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22e09bcdc3974f888d75cc14a7cd098a', 'request_id': 'req-58f136d8-b20c-4458-96b1-d2677be3397b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:44.303634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-58f136d8-b20c-4458-96b1-d2677be3397b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22e09bcdc3974f888d75cc14a7cd098a', 'request_id': 'req-58f136d8-b20c-4458-96b1-d2677be3397b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:44.371868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-58f136d8-b20c-4458-96b1-d2677be3397b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22e09bcdc3974f888d75cc14a7cd098a', 'request_id': 'req-58f136d8-b20c-4458-96b1-d2677be3397b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:44.383311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-58f136d8-b20c-4458-96b1-d2677be3397b admin admin] The token cache is being invalidate because project 22e09bcdc3974f888d75cc14a7cd098a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:56:44.383604 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-58f136d8-b20c-4458-96b1-d2677be3397b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:44.384225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-58f136d8-b20c-4458-96b1-d2677be3397b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:56:44.385858 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4434/8838] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:56:44 2022] DELETE /identity/v3/projects/22e09bcdc3974f888d75cc14a7cd098a => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:56:44.390859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0aa9b-efc5-4816-b477-6458d5944d6a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:44.391184 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0aa9b-efc5-4816-b477-6458d5944d6a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:44.391471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb0aa9b-efc5-4816-b477-6458d5944d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:44.409881 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-7fb0aa9b-efc5-4816-b477-6458d5944d6a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:56:44.411094 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-7fb0aa9b-efc5-4816-b477-6458d5944d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:44.470154 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4405/8839] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:56:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:56:44.478746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b24bb-2569-472b-9a16-bad79f8836c6 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:44.479043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b24bb-2569-472b-9a16-bad79f8836c6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:44.479331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b24bb-2569-472b-9a16-bad79f8836c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:44.497558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8f2b24bb-2569-472b-9a16-bad79f8836c6 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:56:44.498808 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8f2b24bb-2569-472b-9a16-bad79f8836c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:44.506678 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4435/8840] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:56:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:56:44.511012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0aa9b-efc5-4816-b477-6458d5944d6a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:44.520766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb0aa9b-efc5-4816-b477-6458d5944d6a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:44.525926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab999c-4934-48b3-8f9c-6e9cca68a9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:44.526581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab999c-4934-48b3-8f9c-6e9cca68a9ea admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:44.526868 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab999c-4934-48b3-8f9c-6e9cca68a9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:44.527149 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab999c-4934-48b3-8f9c-6e9cca68a9ea admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:44.527877 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab999c-4934-48b3-8f9c-6e9cca68a9ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:56:44.528192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab999c-4934-48b3-8f9c-6e9cca68a9ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:44.529827 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab999c-4934-48b3-8f9c-6e9cca68a9ea admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:44.541854 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4406/8841] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:56:44.546102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b24bb-2569-472b-9a16-bad79f8836c6 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:44.555482 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b24bb-2569-472b-9a16-bad79f8836c6 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:44.560169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4223e8a-e12b-4fa0-aacf-0b9695bf8714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:44.560827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4223e8a-e12b-4fa0-aacf-0b9695bf8714 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:44.561114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4223e8a-e12b-4fa0-aacf-0b9695bf8714 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:44.561401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4223e8a-e12b-4fa0-aacf-0b9695bf8714 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:44.562141 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4223e8a-e12b-4fa0-aacf-0b9695bf8714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:44.564062 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4223e8a-e12b-4fa0-aacf-0b9695bf8714 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:44.647306 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4436/8842] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:56:44 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:56:44.652079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab999c-4934-48b3-8f9c-6e9cca68a9ea admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:44.662115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab999c-4934-48b3-8f9c-6e9cca68a9ea admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:44.718713 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cecef6-e3f1-48b2-b9ac-03f32973b506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:44.719385 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cecef6-e3f1-48b2-b9ac-03f32973b506 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:44.719674 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cecef6-e3f1-48b2-b9ac-03f32973b506 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:44.719970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cecef6-e3f1-48b2-b9ac-03f32973b506 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:44.720647 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cecef6-e3f1-48b2-b9ac-03f32973b506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:44.722628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cecef6-e3f1-48b2-b9ac-03f32973b506 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:44.729142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:56:44.794765 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4407/8843] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:56:44 2022] POST /identity/v3/users => generated 284 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:56:44.798675 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4223e8a-e12b-4fa0-aacf-0b9695bf8714 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:44.809140 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4223e8a-e12b-4fa0-aacf-0b9695bf8714 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:44.813783 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9acf03a-df9a-4e74-b2f9-ed5641a8c6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:44.814486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acf03a-df9a-4e74-b2f9-ed5641a8c6f6 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:44.814773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acf03a-df9a-4e74-b2f9-ed5641a8c6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:44.815054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9acf03a-df9a-4e74-b2f9-ed5641a8c6f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:44.815796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9acf03a-df9a-4e74-b2f9-ed5641a8c6f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:44.817392 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9acf03a-df9a-4e74-b2f9-ed5641a8c6f6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:44.828554 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4437/8844] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:56:44.832404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cecef6-e3f1-48b2-b9ac-03f32973b506 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:44.843163 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cecef6-e3f1-48b2-b9ac-03f32973b506 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:44.848154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e9a57-bec0-4e96-9822-c10d6ff5fa03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:44.848801 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e9a57-bec0-4e96-9822-c10d6ff5fa03 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:44.849084 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e9a57-bec0-4e96-9822-c10d6ff5fa03 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:44.849360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e9a57-bec0-4e96-9822-c10d6ff5fa03 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:44.850109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e9a57-bec0-4e96-9822-c10d6ff5fa03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:44.851837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e9a57-bec0-4e96-9822-c10d6ff5fa03 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:44.862497 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4408/8845] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:56:44.866297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9acf03a-df9a-4e74-b2f9-ed5641a8c6f6 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:44.876276 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9acf03a-df9a-4e74-b2f9-ed5641a8c6f6 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:44.881007 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2816b-ebac-474a-8697-53de22170665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:44.881910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2816b-ebac-474a-8697-53de22170665 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:44.882703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2816b-ebac-474a-8697-53de22170665 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:44.882979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2816b-ebac-474a-8697-53de22170665 admin admin] PATH_INFO: `/v3/projects/abc040f067fc41ed805b45c6d3160022/users/5c5ee86d551742dfa7ebb14bbd521a5b/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:44.897060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2816b-ebac-474a-8697-53de22170665 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abc040f067fc41ed805b45c6d3160022, user_id=5c5ee86d551742dfa7ebb14bbd521a5b, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:44.900557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2816b-ebac-474a-8697-53de22170665 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:44.989149 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4438/8846] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:56:44 2022] PUT /identity/v3/projects/abc040f067fc41ed805b45c6d3160022/users/5c5ee86d551742dfa7ebb14bbd521a5b/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:56:45.047660 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7d9cc-132e-426a-82e5-700262c041b8 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:45.047946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7d9cc-132e-426a-82e5-700262c041b8 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:45.048223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7d9cc-132e-426a-82e5-700262c041b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:45.075161 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-e1b7d9cc-132e-426a-82e5-700262c041b8 None None] MFA Rules not processed for user `5c5ee86d551742dfa7ebb14bbd521a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:56:45.076366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e1b7d9cc-132e-426a-82e5-700262c041b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:45.140295 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4409/8847] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:56:45 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:56:45.150927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2816b-ebac-474a-8697-53de22170665 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:45.152723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5ce2816b-ebac-474a-8697-53de22170665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:45.183788 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:56:45.210057 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2816b-ebac-474a-8697-53de22170665 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:45.213947 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143abd07-b399-43e9-894a-34cdcf8aa79e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:45.214603 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143abd07-b399-43e9-894a-34cdcf8aa79e service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:45.214890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143abd07-b399-43e9-894a-34cdcf8aa79e service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:45.215169 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143abd07-b399-43e9-894a-34cdcf8aa79e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:45.224020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143abd07-b399-43e9-894a-34cdcf8aa79e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:45.225464 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143abd07-b399-43e9-894a-34cdcf8aa79e service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:45.234250 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4439/8848] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:56:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:56:55.137404 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:56:55.268796 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=14 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:56:59.375342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7d9cc-132e-426a-82e5-700262c041b8 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:56:59.377038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e1b7d9cc-132e-426a-82e5-700262c041b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:59.462056 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7d9cc-132e-426a-82e5-700262c041b8 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:56:59.468214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22e4f7-659f-4693-8336-ef6b53894b72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:56:59.469088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22e4f7-659f-4693-8336-ef6b53894b72 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:56:59.469392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22e4f7-659f-4693-8336-ef6b53894b72 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:56:59.469746 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22e4f7-659f-4693-8336-ef6b53894b72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:56:59.472099 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6b22e4f7-659f-4693-8336-ef6b53894b72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:56:59.490097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22e4f7-659f-4693-8336-ef6b53894b72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:56:59.491777 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b22e4f7-659f-4693-8336-ef6b53894b72 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:56:59.510085 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4410/8849] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:56:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:57:02.948617 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143abd07-b399-43e9-894a-34cdcf8aa79e service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:03.012735 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143abd07-b399-43e9-894a-34cdcf8aa79e service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:03.017370 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5f6b6-56c9-4140-be58-443055c9a5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:03.018279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5f6b6-56c9-4140-be58-443055c9a5ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:03.018567 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5f6b6-56c9-4140-be58-443055c9a5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:03.018877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5f6b6-56c9-4140-be58-443055c9a5ac admin admin] PATH_INFO: `/v3/users/5c5ee86d551742dfa7ebb14bbd521a5b` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:03.020085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5f6b6-56c9-4140-be58-443055c9a5ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c5ee86d551742dfa7ebb14bbd521a5b)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:03.022058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5f6b6-56c9-4140-be58-443055c9a5ac admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:03.206002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9fb5f6b6-56c9-4140-be58-443055c9a5ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c5ee86d551742dfa7ebb14bbd521a5b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:03.213501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9fb5f6b6-56c9-4140-be58-443055c9a5ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c5ee86d551742dfa7ebb14bbd521a5b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:03.216145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9fb5f6b6-56c9-4140-be58-443055c9a5ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c5ee86d551742dfa7ebb14bbd521a5b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:03.216145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9fb5f6b6-56c9-4140-be58-443055c9a5ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c5ee86d551742dfa7ebb14bbd521a5b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:03.363107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9fb5f6b6-56c9-4140-be58-443055c9a5ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c5ee86d551742dfa7ebb14bbd521a5b'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:03.381583 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4440/8850] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:57:02 2022] DELETE /identity/v3/users/5c5ee86d551742dfa7ebb14bbd521a5b => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:57:03.386750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83121c9d-a0ef-43f9-aa29-fdfd61dab345 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:03.387197 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83121c9d-a0ef-43f9-aa29-fdfd61dab345 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:03.387317 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83121c9d-a0ef-43f9-aa29-fdfd61dab345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:03.404690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-83121c9d-a0ef-43f9-aa29-fdfd61dab345 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:57:03.405931 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-83121c9d-a0ef-43f9-aa29-fdfd61dab345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:03.464370 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4411/8851] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:57:03.473829 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5f6b6-56c9-4140-be58-443055c9a5ac admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:03.475266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9fb5f6b6-56c9-4140-be58-443055c9a5ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:03.533256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5f6b6-56c9-4140-be58-443055c9a5ac admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:03.536980 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cfbd0-d0f6-4033-9bb4-92fcb5e56e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:03.537630 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cfbd0-d0f6-4033-9bb4-92fcb5e56e54 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:03.537952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cfbd0-d0f6-4033-9bb4-92fcb5e56e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:03.538237 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cfbd0-d0f6-4033-9bb4-92fcb5e56e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:03.547303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cfbd0-d0f6-4033-9bb4-92fcb5e56e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:03.548745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cfbd0-d0f6-4033-9bb4-92fcb5e56e54 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:03.558840 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4441/8852] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:57:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:57:03.817166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83121c9d-a0ef-43f9-aa29-fdfd61dab345 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:03.820516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-83121c9d-a0ef-43f9-aa29-fdfd61dab345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:03.839339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83121c9d-a0ef-43f9-aa29-fdfd61dab345 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:03.844445 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6dedb-1a91-402f-94fd-afd1faea86b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:03.845619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6dedb-1a91-402f-94fd-afd1faea86b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:03.846080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6dedb-1a91-402f-94fd-afd1faea86b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:03.846478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6dedb-1a91-402f-94fd-afd1faea86b6 admin admin] PATH_INFO: `/v3/projects/abc040f067fc41ed805b45c6d3160022` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:03.848047 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6dedb-1a91-402f-94fd-afd1faea86b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abc040f067fc41ed805b45c6d3160022)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:03.850089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6dedb-1a91-402f-94fd-afd1faea86b6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:03.946662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-63f6dedb-1a91-402f-94fd-afd1faea86b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abc040f067fc41ed805b45c6d3160022', 'request_id': 'req-63f6dedb-1a91-402f-94fd-afd1faea86b6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:03.952096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-63f6dedb-1a91-402f-94fd-afd1faea86b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abc040f067fc41ed805b45c6d3160022', 'request_id': 'req-63f6dedb-1a91-402f-94fd-afd1faea86b6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:04.003540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-63f6dedb-1a91-402f-94fd-afd1faea86b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abc040f067fc41ed805b45c6d3160022', 'request_id': 'req-63f6dedb-1a91-402f-94fd-afd1faea86b6'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:04.015060 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-63f6dedb-1a91-402f-94fd-afd1faea86b6 admin admin] The token cache is being invalidate because project abc040f067fc41ed805b45c6d3160022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:57:04.015350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-63f6dedb-1a91-402f-94fd-afd1faea86b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:04.016064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-63f6dedb-1a91-402f-94fd-afd1faea86b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:04.017715 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4412/8853] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:57:03 2022] DELETE /identity/v3/projects/abc040f067fc41ed805b45c6d3160022 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:57:04.023101 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ed047-843c-43d2-8d75-a557eb1b602a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:04.023558 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ed047-843c-43d2-8d75-a557eb1b602a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:04.023659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ed047-843c-43d2-8d75-a557eb1b602a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:04.041809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-556ed047-843c-43d2-8d75-a557eb1b602a None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:57:04.043125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-556ed047-843c-43d2-8d75-a557eb1b602a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:04.060897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:57:04.106142 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4442/8854] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:57:04.114102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743e391-2279-4146-891b-11dd85c83669 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:04.116431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743e391-2279-4146-891b-11dd85c83669 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:04.116431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3743e391-2279-4146-891b-11dd85c83669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:04.133142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-3743e391-2279-4146-891b-11dd85c83669 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:57:04.134383 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3743e391-2279-4146-891b-11dd85c83669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:04.142284 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4413/8855] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:57:04.146539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ed047-843c-43d2-8d75-a557eb1b602a None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:04.161801 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ed047-843c-43d2-8d75-a557eb1b602a None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:04.167165 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158ad5f-1a06-4e32-9b39-ddec1162871f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:04.167824 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158ad5f-1a06-4e32-9b39-ddec1162871f admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:04.168110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158ad5f-1a06-4e32-9b39-ddec1162871f admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:04.168389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158ad5f-1a06-4e32-9b39-ddec1162871f admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:04.169121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158ad5f-1a06-4e32-9b39-ddec1162871f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:57:04.169430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158ad5f-1a06-4e32-9b39-ddec1162871f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:04.171059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158ad5f-1a06-4e32-9b39-ddec1162871f admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:04.182577 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4443/8856] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:57:04.186910 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743e391-2279-4146-891b-11dd85c83669 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:04.196212 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3743e391-2279-4146-891b-11dd85c83669 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:04.201133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7482b-a2b6-4e90-9eac-32302fe190e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:04.201812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7482b-a2b6-4e90-9eac-32302fe190e6 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:04.202133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7482b-a2b6-4e90-9eac-32302fe190e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:04.202415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7482b-a2b6-4e90-9eac-32302fe190e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:04.203418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e7482b-a2b6-4e90-9eac-32302fe190e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:04.205491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e7482b-a2b6-4e90-9eac-32302fe190e6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:04.208914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:57:04.307777 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4414/8857] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:57:04 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:57:04.311929 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158ad5f-1a06-4e32-9b39-ddec1162871f admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:04.322159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158ad5f-1a06-4e32-9b39-ddec1162871f admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:04.379219 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289005be-1af4-41b1-8405-cca10864016c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:04.379890 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289005be-1af4-41b1-8405-cca10864016c admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:04.380177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289005be-1af4-41b1-8405-cca10864016c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:04.380457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289005be-1af4-41b1-8405-cca10864016c admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:04.381159 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289005be-1af4-41b1-8405-cca10864016c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:04.383125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289005be-1af4-41b1-8405-cca10864016c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:04.449354 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4444/8858] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:57:04 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:57:04.453798 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7482b-a2b6-4e90-9eac-32302fe190e6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:04.464972 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7482b-a2b6-4e90-9eac-32302fe190e6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:04.470024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e5bf2-94d8-449b-8b33-11bba97abef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:04.470694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e5bf2-94d8-449b-8b33-11bba97abef3 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:04.470980 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e5bf2-94d8-449b-8b33-11bba97abef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:04.471260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e5bf2-94d8-449b-8b33-11bba97abef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:04.471987 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e5bf2-94d8-449b-8b33-11bba97abef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:04.473620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e5bf2-94d8-449b-8b33-11bba97abef3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:04.483686 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4415/8859] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:57:04.487548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289005be-1af4-41b1-8405-cca10864016c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:04.497631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289005be-1af4-41b1-8405-cca10864016c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:04.502367 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba475be-f8fc-446f-be7a-d9f70dc4f1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:04.503025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba475be-f8fc-446f-be7a-d9f70dc4f1b7 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:04.503311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba475be-f8fc-446f-be7a-d9f70dc4f1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:04.503597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba475be-f8fc-446f-be7a-d9f70dc4f1b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:04.504327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba475be-f8fc-446f-be7a-d9f70dc4f1b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:04.505952 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba475be-f8fc-446f-be7a-d9f70dc4f1b7 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:04.516346 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4445/8860] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:57:04.520266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e5bf2-94d8-449b-8b33-11bba97abef3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:04.530873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e5bf2-94d8-449b-8b33-11bba97abef3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:04.535766 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3943cc9-ed63-41c0-9bf0-a1c85afcfb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:04.536650 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3943cc9-ed63-41c0-9bf0-a1c85afcfb3f admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:04.536944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3943cc9-ed63-41c0-9bf0-a1c85afcfb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:04.537257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3943cc9-ed63-41c0-9bf0-a1c85afcfb3f admin admin] PATH_INFO: `/v3/projects/693cb04182424591981814361e8a387b/users/82401877f986403b9ffedf3887824f00/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:04.552001 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3943cc9-ed63-41c0-9bf0-a1c85afcfb3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=693cb04182424591981814361e8a387b, user_id=82401877f986403b9ffedf3887824f00, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:04.554754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3943cc9-ed63-41c0-9bf0-a1c85afcfb3f admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:04.620341 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4416/8861] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:57:04 2022] PUT /identity/v3/projects/693cb04182424591981814361e8a387b/users/82401877f986403b9ffedf3887824f00/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:57:04.630424 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075ec8c-776b-4573-809b-41afc8916364 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:04.630888 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075ec8c-776b-4573-809b-41afc8916364 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:04.631177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075ec8c-776b-4573-809b-41afc8916364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:04.658777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d075ec8c-776b-4573-809b-41afc8916364 None None] MFA Rules not processed for user `82401877f986403b9ffedf3887824f00`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:57:04.659987 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d075ec8c-776b-4573-809b-41afc8916364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:04.724033 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4446/8862] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:57:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:57:04.733829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3943cc9-ed63-41c0-9bf0-a1c85afcfb3f admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:04.735533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e3943cc9-ed63-41c0-9bf0-a1c85afcfb3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:04.792189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3943cc9-ed63-41c0-9bf0-a1c85afcfb3f admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:04.795824 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73571024-e96c-4237-9b1f-cf42dfccc3e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:04.796464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73571024-e96c-4237-9b1f-cf42dfccc3e9 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:04.796753 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73571024-e96c-4237-9b1f-cf42dfccc3e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:04.797033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73571024-e96c-4237-9b1f-cf42dfccc3e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:04.805887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73571024-e96c-4237-9b1f-cf42dfccc3e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:04.807336 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73571024-e96c-4237-9b1f-cf42dfccc3e9 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:04.816206 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4417/8863] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:57:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:57:10.566516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075ec8c-776b-4573-809b-41afc8916364 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:10.586001 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:57:10.629846 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075ec8c-776b-4573-809b-41afc8916364 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:10.635039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c47f28-112a-469b-b8a0-b9f269b5a8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:10.635919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c47f28-112a-469b-b8a0-b9f269b5a8bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:10.636209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c47f28-112a-469b-b8a0-b9f269b5a8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:10.636485 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c47f28-112a-469b-b8a0-b9f269b5a8bc admin admin] PATH_INFO: `/v3/users/82401877f986403b9ffedf3887824f00` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:10.637681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c47f28-112a-469b-b8a0-b9f269b5a8bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=82401877f986403b9ffedf3887824f00)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:10.639687 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c47f28-112a-469b-b8a0-b9f269b5a8bc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:10.803322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a7c47f28-112a-469b-b8a0-b9f269b5a8bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82401877f986403b9ffedf3887824f00'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:10.810655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a7c47f28-112a-469b-b8a0-b9f269b5a8bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82401877f986403b9ffedf3887824f00'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:10.811351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a7c47f28-112a-469b-b8a0-b9f269b5a8bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82401877f986403b9ffedf3887824f00'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:10.811968 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a7c47f28-112a-469b-b8a0-b9f269b5a8bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82401877f986403b9ffedf3887824f00'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:10.894619 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a7c47f28-112a-469b-b8a0-b9f269b5a8bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82401877f986403b9ffedf3887824f00'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:10.913578 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4447/8864] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:57:10 2022] DELETE /identity/v3/users/82401877f986403b9ffedf3887824f00 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:57:10.918786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed467c47-d406-4b5c-8dfb-a36536ffaa13 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:10.919239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed467c47-d406-4b5c-8dfb-a36536ffaa13 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:10.919406 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed467c47-d406-4b5c-8dfb-a36536ffaa13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:10.936536 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-ed467c47-d406-4b5c-8dfb-a36536ffaa13 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:57:10.937744 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ed467c47-d406-4b5c-8dfb-a36536ffaa13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:10.997503 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4418/8865] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:57:11.007198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c47f28-112a-469b-b8a0-b9f269b5a8bc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:11.008578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a7c47f28-112a-469b-b8a0-b9f269b5a8bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:11.067377 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c47f28-112a-469b-b8a0-b9f269b5a8bc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:11.071432 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4798124-7855-491d-9213-3c1a7def2438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:11.072121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4798124-7855-491d-9213-3c1a7def2438 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:11.072411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4798124-7855-491d-9213-3c1a7def2438 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:11.072698 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4798124-7855-491d-9213-3c1a7def2438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:11.082560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4798124-7855-491d-9213-3c1a7def2438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:11.084011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4798124-7855-491d-9213-3c1a7def2438 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:11.094216 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4448/8866] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:57:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:57:11.318734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed467c47-d406-4b5c-8dfb-a36536ffaa13 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:11.320307 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ed467c47-d406-4b5c-8dfb-a36536ffaa13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:11.333603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed467c47-d406-4b5c-8dfb-a36536ffaa13 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:11.339138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99237d6a-09a4-4000-9b03-fe1ec68e5766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:11.350908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99237d6a-09a4-4000-9b03-fe1ec68e5766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:11.350908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99237d6a-09a4-4000-9b03-fe1ec68e5766 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:11.350908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99237d6a-09a4-4000-9b03-fe1ec68e5766 admin admin] PATH_INFO: `/v3/projects/693cb04182424591981814361e8a387b` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:11.350908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99237d6a-09a4-4000-9b03-fe1ec68e5766 admin admin] RBAC: Authorizing `identity:delete_project(project_id=693cb04182424591981814361e8a387b)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:11.350908 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99237d6a-09a4-4000-9b03-fe1ec68e5766 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:11.454715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:57:11.455979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-99237d6a-09a4-4000-9b03-fe1ec68e5766 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '693cb04182424591981814361e8a387b', 'request_id': 'req-99237d6a-09a4-4000-9b03-fe1ec68e5766'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:11.461430 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-99237d6a-09a4-4000-9b03-fe1ec68e5766 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '693cb04182424591981814361e8a387b', 'request_id': 'req-99237d6a-09a4-4000-9b03-fe1ec68e5766'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:11.540548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-99237d6a-09a4-4000-9b03-fe1ec68e5766 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '693cb04182424591981814361e8a387b', 'request_id': 'req-99237d6a-09a4-4000-9b03-fe1ec68e5766'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:11.552291 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-99237d6a-09a4-4000-9b03-fe1ec68e5766 admin admin] The token cache is being invalidate because project 693cb04182424591981814361e8a387b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:57:11.552587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-99237d6a-09a4-4000-9b03-fe1ec68e5766 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:11.553231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-99237d6a-09a4-4000-9b03-fe1ec68e5766 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:57:11.554826 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4419/8867] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:57:11 2022] DELETE /identity/v3/projects/693cb04182424591981814361e8a387b => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:57:11.561188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e79e0-e507-45e8-a01c-fadec57b7694 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:11.561481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e79e0-e507-45e8-a01c-fadec57b7694 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:11.561762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e79e0-e507-45e8-a01c-fadec57b7694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:11.580426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d85e79e0-e507-45e8-a01c-fadec57b7694 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:57:11.581606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d85e79e0-e507-45e8-a01c-fadec57b7694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:11.615611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:57:11.642123 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4449/8868] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:57:11.649933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa15ae-7cc2-4e78-9b16-708fdb8999e1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:11.650261 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa15ae-7cc2-4e78-9b16-708fdb8999e1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:11.650555 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa15ae-7cc2-4e78-9b16-708fdb8999e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:11.668856 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a8aa15ae-7cc2-4e78-9b16-708fdb8999e1 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:57:11.670109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a8aa15ae-7cc2-4e78-9b16-708fdb8999e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:11.678593 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4420/8869] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:57:11.683307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e79e0-e507-45e8-a01c-fadec57b7694 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:11.692925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e79e0-e507-45e8-a01c-fadec57b7694 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:11.698122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625af67f-cab0-4769-ab6e-cee5e9b24a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:11.698773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625af67f-cab0-4769-ab6e-cee5e9b24a1c admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:11.699061 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625af67f-cab0-4769-ab6e-cee5e9b24a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:11.699341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625af67f-cab0-4769-ab6e-cee5e9b24a1c admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:11.700068 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625af67f-cab0-4769-ab6e-cee5e9b24a1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:57:11.700376 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625af67f-cab0-4769-ab6e-cee5e9b24a1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:11.702018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625af67f-cab0-4769-ab6e-cee5e9b24a1c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:11.714268 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4450/8870] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:57:11.718389 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa15ae-7cc2-4e78-9b16-708fdb8999e1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:11.728266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa15ae-7cc2-4e78-9b16-708fdb8999e1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:11.733053 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2499c-350b-47b9-9b6c-12c36099d97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:11.733686 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2499c-350b-47b9-9b6c-12c36099d97c admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:11.734008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2499c-350b-47b9-9b6c-12c36099d97c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:11.734292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2499c-350b-47b9-9b6c-12c36099d97c admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:11.734986 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2499c-350b-47b9-9b6c-12c36099d97c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:11.737187 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a2499c-350b-47b9-9b6c-12c36099d97c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:11.862422 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4421/8871] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:57:11 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:57:11.866512 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625af67f-cab0-4769-ab6e-cee5e9b24a1c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:11.878151 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625af67f-cab0-4769-ab6e-cee5e9b24a1c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:11.936017 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040c7a1-311d-4e1c-b30b-1487102ae11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:11.936693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040c7a1-311d-4e1c-b30b-1487102ae11c admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:11.936983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040c7a1-311d-4e1c-b30b-1487102ae11c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:11.937266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9040c7a1-311d-4e1c-b30b-1487102ae11c admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:11.938004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040c7a1-311d-4e1c-b30b-1487102ae11c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:11.939949 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9040c7a1-311d-4e1c-b30b-1487102ae11c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:12.037460 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4451/8872] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:57:11 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:57:12.042285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2499c-350b-47b9-9b6c-12c36099d97c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:12.053078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2499c-350b-47b9-9b6c-12c36099d97c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:12.058718 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3f983-a62a-41d3-9703-5d4dc390ddcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:12.059380 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3f983-a62a-41d3-9703-5d4dc390ddcd admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:12.059666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3f983-a62a-41d3-9703-5d4dc390ddcd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:12.059943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3f983-a62a-41d3-9703-5d4dc390ddcd admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:12.060677 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3f983-a62a-41d3-9703-5d4dc390ddcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:12.062337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd3f983-a62a-41d3-9703-5d4dc390ddcd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:12.073786 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4422/8873] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:57:12.078372 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040c7a1-311d-4e1c-b30b-1487102ae11c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:12.088817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9040c7a1-311d-4e1c-b30b-1487102ae11c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:12.093882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0780d0c1-fa8d-4919-801f-935ff91f45fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:12.094549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780d0c1-fa8d-4919-801f-935ff91f45fb admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:12.094836 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780d0c1-fa8d-4919-801f-935ff91f45fb admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:12.095138 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780d0c1-fa8d-4919-801f-935ff91f45fb admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:12.095872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780d0c1-fa8d-4919-801f-935ff91f45fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:12.097475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0780d0c1-fa8d-4919-801f-935ff91f45fb admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:12.109189 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4452/8874] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:57:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:57:12.113418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3f983-a62a-41d3-9703-5d4dc390ddcd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:12.124194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd3f983-a62a-41d3-9703-5d4dc390ddcd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:12.128941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1098ec-ca95-4a62-b13e-a10489607fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:12.131954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1098ec-ca95-4a62-b13e-a10489607fcd admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:12.131954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1098ec-ca95-4a62-b13e-a10489607fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:12.131954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1098ec-ca95-4a62-b13e-a10489607fcd admin admin] PATH_INFO: `/v3/projects/f1c74667a82a49b6b4334c53eabb842f/users/b99fe7794c22452484990b7e783f9a43/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:12.144911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1098ec-ca95-4a62-b13e-a10489607fcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1c74667a82a49b6b4334c53eabb842f, user_id=b99fe7794c22452484990b7e783f9a43, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:12.147645 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1098ec-ca95-4a62-b13e-a10489607fcd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:12.232264 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4423/8875] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:57:12 2022] PUT /identity/v3/projects/f1c74667a82a49b6b4334c53eabb842f/users/b99fe7794c22452484990b7e783f9a43/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:57:12.237393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874d274-c8c1-4293-9b7d-02e8ee0c14ea None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:12.237682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874d274-c8c1-4293-9b7d-02e8ee0c14ea None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:12.237992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874d274-c8c1-4293-9b7d-02e8ee0c14ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:12.255516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3874d274-c8c1-4293-9b7d-02e8ee0c14ea None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:57:12.256711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3874d274-c8c1-4293-9b7d-02e8ee0c14ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:12.314717 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4453/8876] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:57:12.324590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1098ec-ca95-4a62-b13e-a10489607fcd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:12.326366 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-9f1098ec-ca95-4a62-b13e-a10489607fcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:12.383874 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1098ec-ca95-4a62-b13e-a10489607fcd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:12.387543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4856eeb-3342-4f80-892c-b19317b9a299 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:12.388193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4856eeb-3342-4f80-892c-b19317b9a299 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:12.388482 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4856eeb-3342-4f80-892c-b19317b9a299 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:12.388768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4856eeb-3342-4f80-892c-b19317b9a299 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:12.398027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4856eeb-3342-4f80-892c-b19317b9a299 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:12.399458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4856eeb-3342-4f80-892c-b19317b9a299 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:12.409605 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4424/8877] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:57:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:57:17.571563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ee2ab-e7c2-4d6b-a79d-7dbc8089a4c0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:17.572599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ee2ab-e7c2-4d6b-a79d-7dbc8089a4c0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:17.572811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ee2ab-e7c2-4d6b-a79d-7dbc8089a4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:17.602986 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8f1ee2ab-e7c2-4d6b-a79d-7dbc8089a4c0 None None] MFA Rules not processed for user `b99fe7794c22452484990b7e783f9a43`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:57:17.604302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8f1ee2ab-e7c2-4d6b-a79d-7dbc8089a4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:17.642084 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:57:17.674961 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4454/8878] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:57:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:57:17.679531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4856eeb-3342-4f80-892c-b19317b9a299 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:17.690537 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4856eeb-3342-4f80-892c-b19317b9a299 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:17.695501 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00989be1-ca24-4304-9fee-ec2122815a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:17.696165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00989be1-ca24-4304-9fee-ec2122815a77 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:17.696612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00989be1-ca24-4304-9fee-ec2122815a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:17.696891 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00989be1-ca24-4304-9fee-ec2122815a77 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:17.698285 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00989be1-ca24-4304-9fee-ec2122815a77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:17.701190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00989be1-ca24-4304-9fee-ec2122815a77 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:17.800601 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4425/8879] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:57:17 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:57:17.801742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ee2ab-e7c2-4d6b-a79d-7dbc8089a4c0 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:17.816529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ee2ab-e7c2-4d6b-a79d-7dbc8089a4c0 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:17.884860 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b831f7-57a2-4e2d-8df2-773694409b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:17.885830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b831f7-57a2-4e2d-8df2-773694409b6c admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:17.886285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b831f7-57a2-4e2d-8df2-773694409b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:17.886660 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b831f7-57a2-4e2d-8df2-773694409b6c admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:17.887566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b831f7-57a2-4e2d-8df2-773694409b6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:17.889907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b831f7-57a2-4e2d-8df2-773694409b6c admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:17.970012 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4455/8880] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:57:17 2022] POST /identity/v3/users => generated 299 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:57:17.970289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00989be1-ca24-4304-9fee-ec2122815a77 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:17.985061 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00989be1-ca24-4304-9fee-ec2122815a77 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:17.990412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e4386-adf5-4766-b29f-ab74498f0e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:17.991119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e4386-adf5-4766-b29f-ab74498f0e0c admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:17.991410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e4386-adf5-4766-b29f-ab74498f0e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:17.993166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39e4386-adf5-4766-b29f-ab74498f0e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:17.993166 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e4386-adf5-4766-b29f-ab74498f0e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:17.994636 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39e4386-adf5-4766-b29f-ab74498f0e0c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:18.005629 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4426/8881] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:57:18.010110 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b831f7-57a2-4e2d-8df2-773694409b6c admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:18.020621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b831f7-57a2-4e2d-8df2-773694409b6c admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:18.025556 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539e301-93cc-4ee2-81e4-41f61f9d0998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:18.026474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539e301-93cc-4ee2-81e4-41f61f9d0998 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:18.026772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539e301-93cc-4ee2-81e4-41f61f9d0998 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:18.027069 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539e301-93cc-4ee2-81e4-41f61f9d0998 admin admin] PATH_INFO: `/v3/projects/0b9831209c1a4f9399fdf76a22aff020/users/3c318e1e73f443bf9f63dbf49988034c/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:18.042527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3539e301-93cc-4ee2-81e4-41f61f9d0998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9831209c1a4f9399fdf76a22aff020, user_id=3c318e1e73f443bf9f63dbf49988034c, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:18.045471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3539e301-93cc-4ee2-81e4-41f61f9d0998 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:18.104132 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4456/8882] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:57:18 2022] PUT /identity/v3/projects/0b9831209c1a4f9399fdf76a22aff020/users/3c318e1e73f443bf9f63dbf49988034c/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:57:18.108022 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e4386-adf5-4766-b29f-ab74498f0e0c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:18.121347 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39e4386-adf5-4766-b29f-ab74498f0e0c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:18.163164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:57:18.180033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b8257-1600-4136-be0f-21fe6bfbdb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:18.180708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b8257-1600-4136-be0f-21fe6bfbdb1e admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:18.180994 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b8257-1600-4136-be0f-21fe6bfbdb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:18.182417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b8257-1600-4136-be0f-21fe6bfbdb1e admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:18.182592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b8257-1600-4136-be0f-21fe6bfbdb1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:18.184200 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b8257-1600-4136-be0f-21fe6bfbdb1e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:18.194277 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4427/8883] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 21:57:18.198082 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539e301-93cc-4ee2-81e4-41f61f9d0998 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:18.208761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539e301-93cc-4ee2-81e4-41f61f9d0998 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:18.213596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833047c-74cd-45e7-9847-f21ab0e163ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:18.214503 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833047c-74cd-45e7-9847-f21ab0e163ab admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:18.214786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833047c-74cd-45e7-9847-f21ab0e163ab admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:18.215067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833047c-74cd-45e7-9847-f21ab0e163ab admin admin] PATH_INFO: `/v3/projects/0b9831209c1a4f9399fdf76a22aff020/users/3c318e1e73f443bf9f63dbf49988034c/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:18.217156 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833047c-74cd-45e7-9847-f21ab0e163ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b9831209c1a4f9399fdf76a22aff020, user_id=3c318e1e73f443bf9f63dbf49988034c, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:18.219856 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833047c-74cd-45e7-9847-f21ab0e163ab admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:18.230692 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4457/8884] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:57:18 2022] PUT /identity/v3/projects/0b9831209c1a4f9399fdf76a22aff020/users/3c318e1e73f443bf9f63dbf49988034c/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:57:24.492684 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f0434-bcf5-467f-a226-81f5daa18154 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:24.493191 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f0434-bcf5-467f-a226-81f5daa18154 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:24.493278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f0434-bcf5-467f-a226-81f5daa18154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:24.521792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6a4f0434-bcf5-467f-a226-81f5daa18154 None None] MFA Rules not processed for user `3c318e1e73f443bf9f63dbf49988034c`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:57:24.523085 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6a4f0434-bcf5-467f-a226-81f5daa18154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:24.593027 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4428/8885] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:57:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:57:24.620107 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07e88f-a8f0-4ccf-b8aa-50c77105d5a1 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:24.620522 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07e88f-a8f0-4ccf-b8aa-50c77105d5a1 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:24.620784 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07e88f-a8f0-4ccf-b8aa-50c77105d5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:24.668598 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8e07e88f-a8f0-4ccf-b8aa-50c77105d5a1 None None] MFA Rules not processed for user `224c1a0427c942e5bfef2bac75648d80`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:57:24.669746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8e07e88f-a8f0-4ccf-b8aa-50c77105d5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:24.732449 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4458/8886] 10.222.0.33 () {58 vars in 1059 bytes} [Sat May 28 21:57:24 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:57:24.738530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f0434-bcf5-467f-a226-81f5daa18154 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:24.748484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f0434-bcf5-467f-a226-81f5daa18154 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:24.752055 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32561e2f-1b9f-4e0f-9fbf-73fcdb7ab0a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:24.752690 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32561e2f-1b9f-4e0f-9fbf-73fcdb7ab0a2 service glance] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:24.752976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32561e2f-1b9f-4e0f-9fbf-73fcdb7ab0a2 service glance] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:24.753251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32561e2f-1b9f-4e0f-9fbf-73fcdb7ab0a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:24.810158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32561e2f-1b9f-4e0f-9fbf-73fcdb7ab0a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:24.811615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32561e2f-1b9f-4e0f-9fbf-73fcdb7ab0a2 service glance] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:24.820599 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4429/8887] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:57:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:57:25.095447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbe232-9499-420c-91fc-520c0b7b381e None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:25.095741 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbe232-9499-420c-91fc-520c0b7b381e None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:25.096025 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbe232-9499-420c-91fc-520c0b7b381e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:25.102478 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:57:25.153360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-4bcbe232-9499-420c-91fc-520c0b7b381e None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:57:25.154615 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4bcbe232-9499-420c-91fc-520c0b7b381e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:25.223595 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4459/8888] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:57:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:57:25.386050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e5fd3-d5c0-405e-b605-9bec027b070f None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:25.386576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e5fd3-d5c0-405e-b605-9bec027b070f None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:25.386893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e5fd3-d5c0-405e-b605-9bec027b070f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:25.435391 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-db8e5fd3-d5c0-405e-b605-9bec027b070f None None] MFA Rules not processed for user `a01bba888a644eb18fc457aff578aa4b`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:57:25.436612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-db8e5fd3-d5c0-405e-b605-9bec027b070f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:25.470792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=100 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:57:25.514042 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4430/8889] 10.222.0.33 () {58 vars in 1039 bytes} [Sat May 28 21:57:25 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:57:25.520037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbe232-9499-420c-91fc-520c0b7b381e None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:25.532730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbe232-9499-420c-91fc-520c0b7b381e None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:25.537813 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d8f34-61f3-4fcf-a953-30d8c1deea0e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:25.538496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d8f34-61f3-4fcf-a953-30d8c1deea0e service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:25.538785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d8f34-61f3-4fcf-a953-30d8c1deea0e service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:25.539142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d8f34-61f3-4fcf-a953-30d8c1deea0e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:25.547935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d8f34-61f3-4fcf-a953-30d8c1deea0e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:25.549545 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d8f34-61f3-4fcf-a953-30d8c1deea0e service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:25.558684 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4460/8890] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:57:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:57:35.493251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324adf79-3a49-46e7-a31c-7b884fc12cfa None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:35.493804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324adf79-3a49-46e7-a31c-7b884fc12cfa None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:35.493804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324adf79-3a49-46e7-a31c-7b884fc12cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:35.511541 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-324adf79-3a49-46e7-a31c-7b884fc12cfa None None] MFA Rules not processed for user `b99fe7794c22452484990b7e783f9a43`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:57:35.512754 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-324adf79-3a49-46e7-a31c-7b884fc12cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:35.519991 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4431/8891] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:57:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:57:35.555200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=44 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:57:35.776599 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d8f34-61f3-4fcf-a953-30d8c1deea0e service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:35.778200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-4c1d8f34-61f3-4fcf-a953-30d8c1deea0e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:35.835128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d8f34-61f3-4fcf-a953-30d8c1deea0e service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:35.839318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62858dff-a0e8-4ec3-b051-1d47d048275d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:35.839960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62858dff-a0e8-4ec3-b051-1d47d048275d service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:35.840245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62858dff-a0e8-4ec3-b051-1d47d048275d service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:35.840520 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62858dff-a0e8-4ec3-b051-1d47d048275d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:35.842374 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-62858dff-a0e8-4ec3-b051-1d47d048275d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:35.855373 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62858dff-a0e8-4ec3-b051-1d47d048275d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:35.857106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62858dff-a0e8-4ec3-b051-1d47d048275d service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:35.867958 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4461/8892] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:57:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:57:39.038379 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed977d3-9b7a-42d5-ba92-97b35535bdce None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:39.039172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed977d3-9b7a-42d5-ba92-97b35535bdce None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:39.039457 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed977d3-9b7a-42d5-ba92-97b35535bdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:39.057057 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-eed977d3-9b7a-42d5-ba92-97b35535bdce None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:57:39.058320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-eed977d3-9b7a-42d5-ba92-97b35535bdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:57:39.066848 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4432/8893] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 21:57:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:57:39.081145 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62858dff-a0e8-4ec3-b051-1d47d048275d service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:57:39.093877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62858dff-a0e8-4ec3-b051-1d47d048275d service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:57:39.098897 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a14d3-aa4f-4931-81cd-765f98adca43 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:57:39.099576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a14d3-aa4f-4931-81cd-765f98adca43 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:57:39.099869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a14d3-aa4f-4931-81cd-765f98adca43 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:57:39.100153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a14d3-aa4f-4931-81cd-765f98adca43 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:57:39.108768 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a14d3-aa4f-4931-81cd-765f98adca43 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:57:39.110439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a14d3-aa4f-4931-81cd-765f98adca43 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:57:39.118616 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4462/8894] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:57:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:57:49.051592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=22 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:57:49.115304 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=32 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:58:02.768265 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed977d3-9b7a-42d5-ba92-97b35535bdce None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:58:02.778417 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed977d3-9b7a-42d5-ba92-97b35535bdce None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:58:02.782257 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311606a-c819-44a5-963f-d78fd1b38787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:58:02.782906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311606a-c819-44a5-963f-d78fd1b38787 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:58:02.783192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311606a-c819-44a5-963f-d78fd1b38787 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:58:02.783474 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311606a-c819-44a5-963f-d78fd1b38787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:58:02.793226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311606a-c819-44a5-963f-d78fd1b38787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:58:02.794715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2311606a-c819-44a5-963f-d78fd1b38787 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:58:02.804288 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4433/8895] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:58:02 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:58:12.800049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:59:22.588700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3236f-00c8-4b21-bbb4-4cd0e20a0d6f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:22.589306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3236f-00c8-4b21-bbb4-4cd0e20a0d6f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:22.589306 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d3236f-00c8-4b21-bbb4-4cd0e20a0d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:22.645078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-60d3236f-00c8-4b21-bbb4-4cd0e20a0d6f None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:59:22.646380 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-60d3236f-00c8-4b21-bbb4-4cd0e20a0d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:22.732677 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4463/8896] 10.222.0.15 () {58 vars in 1015 bytes} [Sat May 28 21:59:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 21:59:22.747704 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e91585-e6cb-48c1-ba3e-a1f706638fa5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:22.748632 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e91585-e6cb-48c1-ba3e-a1f706638fa5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:22.749157 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e91585-e6cb-48c1-ba3e-a1f706638fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:22.794030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-61e91585-e6cb-48c1-ba3e-a1f706638fa5 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:59:22.795463 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-61e91585-e6cb-48c1-ba3e-a1f706638fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:22.809823 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4434/8897] 10.222.0.33 () {58 vars in 1057 bytes} [Sat May 28 21:59:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:59:22.815984 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3236f-00c8-4b21-bbb4-4cd0e20a0d6f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:22.827309 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d3236f-00c8-4b21-bbb4-4cd0e20a0d6f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:22.833043 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f8c61-f58f-4ef2-94e4-22af40342d3d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:22.833709 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f8c61-f58f-4ef2-94e4-22af40342d3d service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:22.834037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f8c61-f58f-4ef2-94e4-22af40342d3d service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:22.834320 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f8c61-f58f-4ef2-94e4-22af40342d3d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:22.844284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f8c61-f58f-4ef2-94e4-22af40342d3d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:22.845965 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94f8c61-f58f-4ef2-94e4-22af40342d3d service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:22.857673 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4464/8898] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:59:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:59:24.203760 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f143f7-041b-4570-9620-760a3bf4a015 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:24.204344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f143f7-041b-4570-9620-760a3bf4a015 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:24.204344 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f143f7-041b-4570-9620-760a3bf4a015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:24.223672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a8f143f7-041b-4570-9620-760a3bf4a015 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:59:24.224889 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a8f143f7-041b-4570-9620-760a3bf4a015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:24.233593 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4435/8899] 10.222.0.41 () {58 vars in 1015 bytes} [Sat May 28 21:59:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:59:24.245760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d230f-8d63-4270-aa25-ba282ceacd9c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:24.246212 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d230f-8d63-4270-aa25-ba282ceacd9c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:24.246401 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d230f-8d63-4270-aa25-ba282ceacd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:24.265715 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-783d230f-8d63-4270-aa25-ba282ceacd9c None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:59:24.267291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-783d230f-8d63-4270-aa25-ba282ceacd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:24.275737 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4465/8900] 10.222.0.33 () {58 vars in 1057 bytes} [Sat May 28 21:59:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:59:24.281489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f143f7-041b-4570-9620-760a3bf4a015 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:24.292812 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f143f7-041b-4570-9620-760a3bf4a015 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:24.298126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660530b-a719-4eed-aeb1-4c0589492809 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:24.298802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660530b-a719-4eed-aeb1-4c0589492809 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:24.299092 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660530b-a719-4eed-aeb1-4c0589492809 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:24.299370 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9660530b-a719-4eed-aeb1-4c0589492809 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:24.308539 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660530b-a719-4eed-aeb1-4c0589492809 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:24.310193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9660530b-a719-4eed-aeb1-4c0589492809 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:24.320743 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4436/8901] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:59:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:59:27.060683 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d230f-8d63-4270-aa25-ba282ceacd9c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:27.071123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d230f-8d63-4270-aa25-ba282ceacd9c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:27.074876 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f515f3-d7ba-47ed-93e5-53eac30ee4c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:27.075787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f515f3-d7ba-47ed-93e5-53eac30ee4c2 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:27.076076 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f515f3-d7ba-47ed-93e5-53eac30ee4c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:27.076358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f515f3-d7ba-47ed-93e5-53eac30ee4c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:27.078273 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-88f515f3-d7ba-47ed-93e5-53eac30ee4c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:27.151403 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f515f3-d7ba-47ed-93e5-53eac30ee4c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:27.152869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f515f3-d7ba-47ed-93e5-53eac30ee4c2 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:27.166499 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4466/8902] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:59:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:59:28.336736 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660530b-a719-4eed-aeb1-4c0589492809 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:28.346586 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9660530b-a719-4eed-aeb1-4c0589492809 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:28.350287 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf0bf5-59aa-4e0b-a954-9cec1da1009b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:28.350925 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf0bf5-59aa-4e0b-a954-9cec1da1009b service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:28.351214 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf0bf5-59aa-4e0b-a954-9cec1da1009b service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:28.351497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf0bf5-59aa-4e0b-a954-9cec1da1009b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:28.353311 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-64cf0bf5-59aa-4e0b-a954-9cec1da1009b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:28.369282 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf0bf5-59aa-4e0b-a954-9cec1da1009b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:28.370750 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf0bf5-59aa-4e0b-a954-9cec1da1009b service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:28.384793 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4437/8903] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:59:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:59:30.779433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a4910-7606-41fa-892c-7861171e92d6 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:30.779910 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a4910-7606-41fa-892c-7861171e92d6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:30.780011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3a4910-7606-41fa-892c-7861171e92d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:30.797190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-df3a4910-7606-41fa-892c-7861171e92d6 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:59:30.798429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-df3a4910-7606-41fa-892c-7861171e92d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:30.805958 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4467/8904] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 21:59:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:59:30.846369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf0bf5-59aa-4e0b-a954-9cec1da1009b service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:30.857810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf0bf5-59aa-4e0b-a954-9cec1da1009b service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:30.862880 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338a6cb-0752-484e-822a-39644ebd9b6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:30.863546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338a6cb-0752-484e-822a-39644ebd9b6e service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:30.863832 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338a6cb-0752-484e-822a-39644ebd9b6e service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:30.864112 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338a6cb-0752-484e-822a-39644ebd9b6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:30.872229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338a6cb-0752-484e-822a-39644ebd9b6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:30.873924 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338a6cb-0752-484e-822a-39644ebd9b6e service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:30.882264 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4438/8905] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 21:59:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 21:59:39.632550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a4910-7606-41fa-892c-7861171e92d6 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:39.667794 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=100 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:59:39.696582 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3a4910-7606-41fa-892c-7861171e92d6 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:39.701932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddfd6c-b01a-496f-b8fc-b246777da3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:39.702736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddfd6c-b01a-496f-b8fc-b246777da3c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:39.703024 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddfd6c-b01a-496f-b8fc-b246777da3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:39.703307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ddfd6c-b01a-496f-b8fc-b246777da3c2 admin admin] PATH_INFO: `/v3/users/b99fe7794c22452484990b7e783f9a43` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:39.704525 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ddfd6c-b01a-496f-b8fc-b246777da3c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b99fe7794c22452484990b7e783f9a43)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:39.706515 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ddfd6c-b01a-496f-b8fc-b246777da3c2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:39.884669 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-49ddfd6c-b01a-496f-b8fc-b246777da3c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b99fe7794c22452484990b7e783f9a43'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:39.892278 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-49ddfd6c-b01a-496f-b8fc-b246777da3c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b99fe7794c22452484990b7e783f9a43'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:39.895877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-49ddfd6c-b01a-496f-b8fc-b246777da3c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b99fe7794c22452484990b7e783f9a43'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:39.895877 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-49ddfd6c-b01a-496f-b8fc-b246777da3c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b99fe7794c22452484990b7e783f9a43'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:39.971657 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-49ddfd6c-b01a-496f-b8fc-b246777da3c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b99fe7794c22452484990b7e783f9a43'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:39.990967 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4468/8906] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:59:39 2022] DELETE /identity/v3/users/b99fe7794c22452484990b7e783f9a43 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:59:40.232792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338a6cb-0752-484e-822a-39644ebd9b6e service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:40.244461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338a6cb-0752-484e-822a-39644ebd9b6e service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:40.303758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15f91d-2af0-4dbc-824f-f591791c94ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:40.304655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15f91d-2af0-4dbc-824f-f591791c94ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:40.304944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15f91d-2af0-4dbc-824f-f591791c94ef admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:40.305226 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15f91d-2af0-4dbc-824f-f591791c94ef admin admin] PATH_INFO: `/v3/projects/f1c74667a82a49b6b4334c53eabb842f` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:40.306992 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15f91d-2af0-4dbc-824f-f591791c94ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1c74667a82a49b6b4334c53eabb842f)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:40.308919 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15f91d-2af0-4dbc-824f-f591791c94ef admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:40.435188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ca15f91d-2af0-4dbc-824f-f591791c94ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1c74667a82a49b6b4334c53eabb842f', 'request_id': 'req-ca15f91d-2af0-4dbc-824f-f591791c94ef'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:40.440912 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ca15f91d-2af0-4dbc-824f-f591791c94ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1c74667a82a49b6b4334c53eabb842f', 'request_id': 'req-ca15f91d-2af0-4dbc-824f-f591791c94ef'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:40.508322 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ca15f91d-2af0-4dbc-824f-f591791c94ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1c74667a82a49b6b4334c53eabb842f', 'request_id': 'req-ca15f91d-2af0-4dbc-824f-f591791c94ef'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:40.520064 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ca15f91d-2af0-4dbc-824f-f591791c94ef admin admin] The token cache is being invalidate because project f1c74667a82a49b6b4334c53eabb842f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:59:40.520361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ca15f91d-2af0-4dbc-824f-f591791c94ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:40.521032 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-ca15f91d-2af0-4dbc-824f-f591791c94ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:40.522732 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4439/8907] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:59:40 2022] DELETE /identity/v3/projects/f1c74667a82a49b6b4334c53eabb842f => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:59:40.526668 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddfd6c-b01a-496f-b8fc-b246777da3c2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:40.528192 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-49ddfd6c-b01a-496f-b8fc-b246777da3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:40.542314 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ddfd6c-b01a-496f-b8fc-b246777da3c2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:40.598841 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d04211-ea4c-4f50-aa0f-14283e188a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:40.599711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d04211-ea4c-4f50-aa0f-14283e188a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:40.600038 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d04211-ea4c-4f50-aa0f-14283e188a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:40.600312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d04211-ea4c-4f50-aa0f-14283e188a98 admin admin] PATH_INFO: `/v3/users/3c318e1e73f443bf9f63dbf49988034c` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:40.601516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d04211-ea4c-4f50-aa0f-14283e188a98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c318e1e73f443bf9f63dbf49988034c)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:40.603983 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d04211-ea4c-4f50-aa0f-14283e188a98 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:40.779006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a4d04211-ea4c-4f50-aa0f-14283e188a98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c318e1e73f443bf9f63dbf49988034c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:40.784532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:59:40.786554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a4d04211-ea4c-4f50-aa0f-14283e188a98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c318e1e73f443bf9f63dbf49988034c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:40.787253 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a4d04211-ea4c-4f50-aa0f-14283e188a98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c318e1e73f443bf9f63dbf49988034c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:40.787879 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a4d04211-ea4c-4f50-aa0f-14283e188a98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c318e1e73f443bf9f63dbf49988034c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:40.865058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a4d04211-ea4c-4f50-aa0f-14283e188a98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c318e1e73f443bf9f63dbf49988034c'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:40.883615 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4469/8908] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 21:59:40 2022] DELETE /identity/v3/users/3c318e1e73f443bf9f63dbf49988034c => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:59:41.153179 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15f91d-2af0-4dbc-824f-f591791c94ef admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:41.156546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ca15f91d-2af0-4dbc-824f-f591791c94ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:41.170855 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15f91d-2af0-4dbc-824f-f591791c94ef admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:41.208352 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:59:41.230953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09ba7b-6914-480e-a378-0111bc29b1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:41.231793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09ba7b-6914-480e-a378-0111bc29b1a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:41.232081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09ba7b-6914-480e-a378-0111bc29b1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:41.232361 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09ba7b-6914-480e-a378-0111bc29b1a1 admin admin] PATH_INFO: `/v3/projects/0b9831209c1a4f9399fdf76a22aff020` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:41.233545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09ba7b-6914-480e-a378-0111bc29b1a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b9831209c1a4f9399fdf76a22aff020)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:41.235514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09ba7b-6914-480e-a378-0111bc29b1a1 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:41.362826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0e09ba7b-6914-480e-a378-0111bc29b1a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b9831209c1a4f9399fdf76a22aff020', 'request_id': 'req-0e09ba7b-6914-480e-a378-0111bc29b1a1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:41.369091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0e09ba7b-6914-480e-a378-0111bc29b1a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b9831209c1a4f9399fdf76a22aff020', 'request_id': 'req-0e09ba7b-6914-480e-a378-0111bc29b1a1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:41.436466 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0e09ba7b-6914-480e-a378-0111bc29b1a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b9831209c1a4f9399fdf76a22aff020', 'request_id': 'req-0e09ba7b-6914-480e-a378-0111bc29b1a1'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:41.447758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0e09ba7b-6914-480e-a378-0111bc29b1a1 admin admin] The token cache is being invalidate because project 0b9831209c1a4f9399fdf76a22aff020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 21:59:41.448048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0e09ba7b-6914-480e-a378-0111bc29b1a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:41.448685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-0e09ba7b-6914-480e-a378-0111bc29b1a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 21:59:41.450526 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4440/8909] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 21:59:41 2022] DELETE /identity/v3/projects/0b9831209c1a4f9399fdf76a22aff020 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:59:41.455659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc205489-3431-426a-8826-b0450530933c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:41.460059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc205489-3431-426a-8826-b0450530933c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:41.460059 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc205489-3431-426a-8826-b0450530933c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:41.474155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-bc205489-3431-426a-8826-b0450530933c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:59:41.475408 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bc205489-3431-426a-8826-b0450530933c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:41.535856 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4470/8910] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:59:41.544063 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f7e23-d48d-479d-b46d-d7ea647e40a3 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:41.544363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f7e23-d48d-479d-b46d-d7ea647e40a3 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:41.544649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42f7e23-d48d-479d-b46d-d7ea647e40a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:41.564591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a42f7e23-d48d-479d-b46d-d7ea647e40a3 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:59:41.565817 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a42f7e23-d48d-479d-b46d-d7ea647e40a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:41.574123 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4441/8911] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:59:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:59:41.583353 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc205489-3431-426a-8826-b0450530933c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:41.593578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc205489-3431-426a-8826-b0450530933c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:41.598155 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01951f-9d25-45b0-9a75-fc4989cc42b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:41.598838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01951f-9d25-45b0-9a75-fc4989cc42b3 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:41.599122 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01951f-9d25-45b0-9a75-fc4989cc42b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:41.599399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c01951f-9d25-45b0-9a75-fc4989cc42b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:41.600128 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01951f-9d25-45b0-9a75-fc4989cc42b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 21:59:41.600437 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01951f-9d25-45b0-9a75-fc4989cc42b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:41.602067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c01951f-9d25-45b0-9a75-fc4989cc42b3 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:41.614103 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4471/8912] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 21:59:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 21:59:41.623198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f7e23-d48d-479d-b46d-d7ea647e40a3 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:41.632756 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42f7e23-d48d-479d-b46d-d7ea647e40a3 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:41.637515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da0d2c-5b23-4065-af56-93adfc371446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:41.638229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da0d2c-5b23-4065-af56-93adfc371446 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:41.638523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da0d2c-5b23-4065-af56-93adfc371446 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:41.638804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91da0d2c-5b23-4065-af56-93adfc371446 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:41.639731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da0d2c-5b23-4065-af56-93adfc371446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:41.641431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91da0d2c-5b23-4065-af56-93adfc371446 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:41.783038 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4442/8913] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:59:41 2022] POST /identity/v3/projects => generated 391 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:59:41.785608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01951f-9d25-45b0-9a75-fc4989cc42b3 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:41.796342 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c01951f-9d25-45b0-9a75-fc4989cc42b3 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:41.853762 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824de3a-7446-46b2-833e-9d625a3a06ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:41.854473 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824de3a-7446-46b2-833e-9d625a3a06ad admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:41.854760 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824de3a-7446-46b2-833e-9d625a3a06ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:41.855041 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824de3a-7446-46b2-833e-9d625a3a06ad admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:41.855744 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824de3a-7446-46b2-833e-9d625a3a06ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:41.857659 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824de3a-7446-46b2-833e-9d625a3a06ad admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:41.937535 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4472/8914] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:59:41 2022] POST /identity/v3/users => generated 296 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:59:41.941454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da0d2c-5b23-4065-af56-93adfc371446 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:41.952940 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91da0d2c-5b23-4065-af56-93adfc371446 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:41.957849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f054e7a-beb0-48a9-b8d5-67e6e1e91641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:41.958523 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f054e7a-beb0-48a9-b8d5-67e6e1e91641 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:41.958836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f054e7a-beb0-48a9-b8d5-67e6e1e91641 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:41.959124 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f054e7a-beb0-48a9-b8d5-67e6e1e91641 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:41.959845 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f054e7a-beb0-48a9-b8d5-67e6e1e91641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:41.961486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f054e7a-beb0-48a9-b8d5-67e6e1e91641 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:41.972248 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4443/8915] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:59:41.976305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824de3a-7446-46b2-833e-9d625a3a06ad admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:41.986942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824de3a-7446-46b2-833e-9d625a3a06ad admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:41.991740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0647323-0481-4e59-9221-3ebd76c25ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:41.992406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0647323-0481-4e59-9221-3ebd76c25ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:41.992701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0647323-0481-4e59-9221-3ebd76c25ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:41.992985 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0647323-0481-4e59-9221-3ebd76c25ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:41.993723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0647323-0481-4e59-9221-3ebd76c25ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:41.995375 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0647323-0481-4e59-9221-3ebd76c25ec2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:42.005312 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4473/8916] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:59:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:59:42.010069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f054e7a-beb0-48a9-b8d5-67e6e1e91641 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:42.020195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f054e7a-beb0-48a9-b8d5-67e6e1e91641 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:42.024743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27086ffb-4aa7-4b5b-a7c8-0fcad9ad15f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:42.026329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27086ffb-4aa7-4b5b-a7c8-0fcad9ad15f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:42.026612 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27086ffb-4aa7-4b5b-a7c8-0fcad9ad15f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:42.026886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27086ffb-4aa7-4b5b-a7c8-0fcad9ad15f5 admin admin] PATH_INFO: `/v3/projects/38d3db1f6443461f90a5be23358b6816/users/56b376ad4a314418bca5fcbf8a8ca429/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:42.041333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27086ffb-4aa7-4b5b-a7c8-0fcad9ad15f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38d3db1f6443461f90a5be23358b6816, user_id=56b376ad4a314418bca5fcbf8a8ca429, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:42.044038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27086ffb-4aa7-4b5b-a7c8-0fcad9ad15f5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:42.108848 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4444/8917] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:59:42 2022] PUT /identity/v3/projects/38d3db1f6443461f90a5be23358b6816/users/56b376ad4a314418bca5fcbf8a8ca429/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:59:42.124889 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05a5ed-fea2-4592-a9c7-79ce5d7013d3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:42.125209 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05a5ed-fea2-4592-a9c7-79ce5d7013d3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:42.125501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05a5ed-fea2-4592-a9c7-79ce5d7013d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:42.152809 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-fd05a5ed-fea2-4592-a9c7-79ce5d7013d3 None None] MFA Rules not processed for user `56b376ad4a314418bca5fcbf8a8ca429`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:59:42.154075 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fd05a5ed-fea2-4592-a9c7-79ce5d7013d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:42.179529 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:59:42.219326 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4474/8918] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:59:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:59:42.223778 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27086ffb-4aa7-4b5b-a7c8-0fcad9ad15f5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:42.280038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:59:42.284945 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27086ffb-4aa7-4b5b-a7c8-0fcad9ad15f5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:42.289641 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6e54e-c786-4cbe-a92a-f7a07a144da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:42.290364 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6e54e-c786-4cbe-a92a-f7a07a144da5 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:42.298757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6e54e-c786-4cbe-a92a-f7a07a144da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:42.298757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a6e54e-c786-4cbe-a92a-f7a07a144da5 admin admin] PATH_INFO: `/v3/projects` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:42.298757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6e54e-c786-4cbe-a92a-f7a07a144da5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:42.298757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a6e54e-c786-4cbe-a92a-f7a07a144da5 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:42.375637 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4445/8919] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 21:59:42 2022] POST /identity/v3/projects => generated 393 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 21:59:42.379625 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05a5ed-fea2-4592-a9c7-79ce5d7013d3 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:42.391133 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05a5ed-fea2-4592-a9c7-79ce5d7013d3 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:42.447713 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74a3f5-bea9-4b59-b3fb-2931bde05833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:42.448409 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74a3f5-bea9-4b59-b3fb-2931bde05833 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:42.448694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74a3f5-bea9-4b59-b3fb-2931bde05833 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:42.448978 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d74a3f5-bea9-4b59-b3fb-2931bde05833 admin admin] PATH_INFO: `/v3/users` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:42.449680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74a3f5-bea9-4b59-b3fb-2931bde05833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:42.451945 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d74a3f5-bea9-4b59-b3fb-2931bde05833 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:42.531328 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4475/8920] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 21:59:42 2022] POST /identity/v3/users => generated 303 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 21:59:42.535267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6e54e-c786-4cbe-a92a-f7a07a144da5 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:42.546338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a6e54e-c786-4cbe-a92a-f7a07a144da5 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:42.551062 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486affd4-dc9b-4b8e-b2ac-b72676dab385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:42.551714 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486affd4-dc9b-4b8e-b2ac-b72676dab385 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:42.551996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486affd4-dc9b-4b8e-b2ac-b72676dab385 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:42.552272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486affd4-dc9b-4b8e-b2ac-b72676dab385 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:42.553043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486affd4-dc9b-4b8e-b2ac-b72676dab385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:42.554688 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486affd4-dc9b-4b8e-b2ac-b72676dab385 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:42.566435 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4446/8921] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:59:42.570474 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74a3f5-bea9-4b59-b3fb-2931bde05833 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:42.581096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d74a3f5-bea9-4b59-b3fb-2931bde05833 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:42.585725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cd44f-7dfd-4ea0-982c-aa1a75a13e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:42.586644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cd44f-7dfd-4ea0-982c-aa1a75a13e98 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:42.586933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cd44f-7dfd-4ea0-982c-aa1a75a13e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:42.587211 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2cd44f-7dfd-4ea0-982c-aa1a75a13e98 admin admin] PATH_INFO: `/v3/projects/3bccf296472843d1bbbf379ef2e52e70/users/e289ee1f8bda46e19f327972d9152b59/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:42.603093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2cd44f-7dfd-4ea0-982c-aa1a75a13e98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bccf296472843d1bbbf379ef2e52e70, user_id=e289ee1f8bda46e19f327972d9152b59, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:42.606331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2cd44f-7dfd-4ea0-982c-aa1a75a13e98 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:42.682726 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4476/8922] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:59:42 2022] PUT /identity/v3/projects/3bccf296472843d1bbbf379ef2e52e70/users/e289ee1f8bda46e19f327972d9152b59/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 21:59:42.686666 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486affd4-dc9b-4b8e-b2ac-b72676dab385 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:42.697171 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486affd4-dc9b-4b8e-b2ac-b72676dab385 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:42.753542 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0660d75-2597-4d2a-90b3-8222eaa965d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:42.754231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0660d75-2597-4d2a-90b3-8222eaa965d8 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:42.754519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0660d75-2597-4d2a-90b3-8222eaa965d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:42.754796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0660d75-2597-4d2a-90b3-8222eaa965d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:42.755518 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0660d75-2597-4d2a-90b3-8222eaa965d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:42.757110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0660d75-2597-4d2a-90b3-8222eaa965d8 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:42.767109 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4447/8923] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 21:59:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 21:59:42.771571 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cd44f-7dfd-4ea0-982c-aa1a75a13e98 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:42.785933 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2cd44f-7dfd-4ea0-982c-aa1a75a13e98 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:42.786700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85640a8-1776-4ed2-b340-f736a9322097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:42.787579 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85640a8-1776-4ed2-b340-f736a9322097 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:42.787859 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85640a8-1776-4ed2-b340-f736a9322097 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:42.788137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85640a8-1776-4ed2-b340-f736a9322097 admin admin] PATH_INFO: `/v3/projects/3bccf296472843d1bbbf379ef2e52e70/users/e289ee1f8bda46e19f327972d9152b59/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:42.790173 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85640a8-1776-4ed2-b340-f736a9322097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bccf296472843d1bbbf379ef2e52e70, user_id=e289ee1f8bda46e19f327972d9152b59, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:42.793430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85640a8-1776-4ed2-b340-f736a9322097 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:42.803814 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4477/8924] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 21:59:42 2022] PUT /identity/v3/projects/3bccf296472843d1bbbf379ef2e52e70/users/e289ee1f8bda46e19f327972d9152b59/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 21:59:42.814280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fb408-1b8a-4e89-94a1-724f14d0a895 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:42.814576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fb408-1b8a-4e89-94a1-724f14d0a895 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:42.814860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fb408-1b8a-4e89-94a1-724f14d0a895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:42.840831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-1d8fb408-1b8a-4e89-94a1-724f14d0a895 None None] MFA Rules not processed for user `e289ee1f8bda46e19f327972d9152b59`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:59:42.842067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1d8fb408-1b8a-4e89-94a1-724f14d0a895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:42.909333 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4448/8925] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:59:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:59:42.925798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85640a8-1776-4ed2-b340-f736a9322097 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:42.927597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d85640a8-1776-4ed2-b340-f736a9322097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:43.004496 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85640a8-1776-4ed2-b340-f736a9322097 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:43.010114 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc139a0f-e646-40d0-b182-e6434e5309b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:43.010763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc139a0f-e646-40d0-b182-e6434e5309b0 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:43.011049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc139a0f-e646-40d0-b182-e6434e5309b0 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:43.011325 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc139a0f-e646-40d0-b182-e6434e5309b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:43.024119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:59:43.068194 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc139a0f-e646-40d0-b182-e6434e5309b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:43.071131 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc139a0f-e646-40d0-b182-e6434e5309b0 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:43.079556 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4478/8926] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:59:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:59:48.287327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fb408-1b8a-4e89-94a1-724f14d0a895 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:48.288909 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1d8fb408-1b8a-4e89-94a1-724f14d0a895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:48.347075 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fb408-1b8a-4e89-94a1-724f14d0a895 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:48.350615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ef355-16e0-45d3-8366-22d94f24f86b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:48.351271 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ef355-16e0-45d3-8366-22d94f24f86b service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:48.351559 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ef355-16e0-45d3-8366-22d94f24f86b service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:48.351843 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ef355-16e0-45d3-8366-22d94f24f86b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:48.357997 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:59:48.361882 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ef355-16e0-45d3-8366-22d94f24f86b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:48.363392 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ef355-16e0-45d3-8366-22d94f24f86b service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:48.373788 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4449/8927] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:59:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 21:59:51.687864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707b5c3-ff5c-40b4-822f-31998f2d4522 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:51.688327 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707b5c3-ff5c-40b4-822f-31998f2d4522 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:51.688417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707b5c3-ff5c-40b4-822f-31998f2d4522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:51.712402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-d707b5c3-ff5c-40b4-822f-31998f2d4522 None None] MFA Rules not processed for user `56b376ad4a314418bca5fcbf8a8ca429`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 21:59:51.714481 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d707b5c3-ff5c-40b4-822f-31998f2d4522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:51.723585 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4479/8928] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 21:59:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 21:59:52.114630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ef355-16e0-45d3-8366-22d94f24f86b service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 21:59:52.116127 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a07ef355-16e0-45d3-8366-22d94f24f86b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:52.128281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ef355-16e0-45d3-8366-22d94f24f86b service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 21:59:52.132147 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9856f43-2fe4-4910-a410-4fb5f0f7e67d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 21:59:52.137398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9856f43-2fe4-4910-a410-4fb5f0f7e67d service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 21:59:52.137398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9856f43-2fe4-4910-a410-4fb5f0f7e67d service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 21:59:52.137398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9856f43-2fe4-4910-a410-4fb5f0f7e67d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 21:59:52.137398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-c9856f43-2fe4-4910-a410-4fb5f0f7e67d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 21:59:52.149991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9856f43-2fe4-4910-a410-4fb5f0f7e67d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 21:59:52.151556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9856f43-2fe4-4910-a410-4fb5f0f7e67d service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 21:59:52.165797 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4450/8929] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 21:59:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:00:01.706927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=23 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:00:02.156272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=9 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:00:12.961676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707b5c3-ff5c-40b4-822f-31998f2d4522 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:00:12.982029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707b5c3-ff5c-40b4-822f-31998f2d4522 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:00:12.986085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a2188-5dae-4781-8e94-6a45bb02f267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:00:12.986727 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a2188-5dae-4781-8e94-6a45bb02f267 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:00:12.987012 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a2188-5dae-4781-8e94-6a45bb02f267 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:00:12.987291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a2188-5dae-4781-8e94-6a45bb02f267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:00:12.989164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-375a2188-5dae-4781-8e94-6a45bb02f267 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:00:13.001430 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a2188-5dae-4781-8e94-6a45bb02f267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:00:13.002912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a2188-5dae-4781-8e94-6a45bb02f267 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:00:13.013650 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4480/8930] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:00:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:00:23.007270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=10 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:00:57.841717 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0678d0-7fd5-43ad-8511-a1656fc03246 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:00:57.842349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0678d0-7fd5-43ad-8511-a1656fc03246 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:00:57.842349 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0678d0-7fd5-43ad-8511-a1656fc03246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:00:57.869030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-bf0678d0-7fd5-43ad-8511-a1656fc03246 None None] MFA Rules not processed for user `56b376ad4a314418bca5fcbf8a8ca429`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:00:57.871319 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-bf0678d0-7fd5-43ad-8511-a1656fc03246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:00:57.908289 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4451/8931] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:00:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:00:57.926767 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa08c1e-3acb-43c3-8f94-f30fa4136fb3 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:00:57.927289 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa08c1e-3acb-43c3-8f94-f30fa4136fb3 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:00:57.927393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa08c1e-3acb-43c3-8f94-f30fa4136fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:00:57.976563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-3aa08c1e-3acb-43c3-8f94-f30fa4136fb3 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:00:57.985853 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3aa08c1e-3acb-43c3-8f94-f30fa4136fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:00:58.012963 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4481/8932] 10.222.0.33 () {58 vars in 1058 bytes} [Sat May 28 22:00:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:00:58.798833 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70239d2d-14f2-4022-821c-7664fc60268a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:00:58.799129 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70239d2d-14f2-4022-821c-7664fc60268a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:00:58.800504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70239d2d-14f2-4022-821c-7664fc60268a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:00:58.819865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-70239d2d-14f2-4022-821c-7664fc60268a None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:00:58.821410 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-70239d2d-14f2-4022-821c-7664fc60268a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:00:58.831737 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4452/8933] 10.222.0.33 () {58 vars in 1058 bytes} [Sat May 28 22:00:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:01:02.173354 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc2ae9-ed87-4153-bc05-7ab0a094f75c None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:01:02.173786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc2ae9-ed87-4153-bc05-7ab0a094f75c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:01:02.174118 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fc2ae9-ed87-4153-bc05-7ab0a094f75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:01:02.199446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-19fc2ae9-ed87-4153-bc05-7ab0a094f75c None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:01:02.200840 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-19fc2ae9-ed87-4153-bc05-7ab0a094f75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:01:02.213585 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4482/8934] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 22:01:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:01:02.225182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70239d2d-14f2-4022-821c-7664fc60268a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:01:02.236045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70239d2d-14f2-4022-821c-7664fc60268a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:01:02.240327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffefbf4-313b-44ef-82ca-226c2d8a3ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:01:02.241153 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffefbf4-313b-44ef-82ca-226c2d8a3ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:01:02.241442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffefbf4-313b-44ef-82ca-226c2d8a3ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:01:02.241723 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffefbf4-313b-44ef-82ca-226c2d8a3ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:01:02.258789 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffefbf4-313b-44ef-82ca-226c2d8a3ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:01:02.260449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffefbf4-313b-44ef-82ca-226c2d8a3ca8 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:01:02.273341 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4453/8935] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:01:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:01:04.623533 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c84502-a60c-4ea2-98f0-ed14378cf778 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:01:04.624089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c84502-a60c-4ea2-98f0-ed14378cf778 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:01:04.624089 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c84502-a60c-4ea2-98f0-ed14378cf778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:01:04.642555 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-10c84502-a60c-4ea2-98f0-ed14378cf778 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:01:04.643770 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-10c84502-a60c-4ea2-98f0-ed14378cf778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:01:04.653555 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4483/8936] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 22:01:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:01:04.664693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffefbf4-313b-44ef-82ca-226c2d8a3ca8 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:01:04.673947 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffefbf4-313b-44ef-82ca-226c2d8a3ca8 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:01:04.677605 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca88c63-7bff-4b1c-a3f1-b4bf79c6df01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:01:04.678280 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca88c63-7bff-4b1c-a3f1-b4bf79c6df01 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:01:04.678569 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca88c63-7bff-4b1c-a3f1-b4bf79c6df01 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:01:04.678849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca88c63-7bff-4b1c-a3f1-b4bf79c6df01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:01:04.689096 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca88c63-7bff-4b1c-a3f1-b4bf79c6df01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:01:04.690581 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca88c63-7bff-4b1c-a3f1-b4bf79c6df01 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:01:04.701681 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4454/8937] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:01:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:01:07.349433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c84502-a60c-4ea2-98f0-ed14378cf778 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:01:07.361004 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c84502-a60c-4ea2-98f0-ed14378cf778 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:01:07.366539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab409a-b36a-4bd8-a70f-9f95aa463225 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:01:07.367185 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab409a-b36a-4bd8-a70f-9f95aa463225 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:01:07.367471 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab409a-b36a-4bd8-a70f-9f95aa463225 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:01:07.367750 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab409a-b36a-4bd8-a70f-9f95aa463225 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:01:07.369572 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9fab409a-b36a-4bd8-a70f-9f95aa463225 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:01:07.384826 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab409a-b36a-4bd8-a70f-9f95aa463225 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:01:07.386443 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab409a-b36a-4bd8-a70f-9f95aa463225 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:01:07.399834 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4484/8938] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:01:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:01:07.487984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a52ba-770b-43bf-a45a-94e42dea2660 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:01:07.488393 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a52ba-770b-43bf-a45a-94e42dea2660 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:01:07.488540 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8a52ba-770b-43bf-a45a-94e42dea2660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:01:07.525181 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-8c8a52ba-770b-43bf-a45a-94e42dea2660 None None] MFA Rules not processed for user `1b2a9ebcd3f84b758322a80c6e590de0`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:01:07.526407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8c8a52ba-770b-43bf-a45a-94e42dea2660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:01:07.533375 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4455/8939] 10.222.0.41 () {58 vars in 1015 bytes} [Sat May 28 22:01:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:01:07.544694 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab409a-b36a-4bd8-a70f-9f95aa463225 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:01:07.554688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab409a-b36a-4bd8-a70f-9f95aa463225 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:01:07.558350 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77021f6c-c792-4709-a6aa-c0f820cb5738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:01:07.558991 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77021f6c-c792-4709-a6aa-c0f820cb5738 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:01:07.559284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77021f6c-c792-4709-a6aa-c0f820cb5738 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:01:07.559569 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77021f6c-c792-4709-a6aa-c0f820cb5738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:01:07.566924 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77021f6c-c792-4709-a6aa-c0f820cb5738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:01:07.568395 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77021f6c-c792-4709-a6aa-c0f820cb5738 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:01:07.576508 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4485/8940] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:01:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:01:11.948070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a52ba-770b-43bf-a45a-94e42dea2660 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:01:11.966156 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8a52ba-770b-43bf-a45a-94e42dea2660 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:01:11.973423 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6169a7-a889-4072-a95a-bfabdee8b38a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:01:11.974110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6169a7-a889-4072-a95a-bfabdee8b38a service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:01:11.974395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6169a7-a889-4072-a95a-bfabdee8b38a service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:01:11.974672 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6169a7-a889-4072-a95a-bfabdee8b38a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:01:11.977016 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-0b6169a7-a889-4072-a95a-bfabdee8b38a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:01:11.993564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6169a7-a889-4072-a95a-bfabdee8b38a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:01:11.995970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6169a7-a889-4072-a95a-bfabdee8b38a service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:01:12.012350 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4456/8941] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:01:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:01:17.572782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=31 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:01:22.004491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=33 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:02:34.125394 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e158bac-e557-41f2-9a8c-fc018185ad56 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:02:34.134015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e158bac-e557-41f2-9a8c-fc018185ad56 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:02:34.134015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e158bac-e557-41f2-9a8c-fc018185ad56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:02:34.155399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9e158bac-e557-41f2-9a8c-fc018185ad56 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:02:34.157134 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9e158bac-e557-41f2-9a8c-fc018185ad56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:02:34.167875 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4486/8942] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 22:02:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:02:34.182531 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6169a7-a889-4072-a95a-bfabdee8b38a service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:02:34.194968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6169a7-a889-4072-a95a-bfabdee8b38a service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:02:34.201984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06900ce0-9c51-46aa-8198-45c9755867d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:02:34.202715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06900ce0-9c51-46aa-8198-45c9755867d6 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:02:34.203338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06900ce0-9c51-46aa-8198-45c9755867d6 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:02:34.205206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06900ce0-9c51-46aa-8198-45c9755867d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:02:34.215911 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06900ce0-9c51-46aa-8198-45c9755867d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:02:34.217512 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06900ce0-9c51-46aa-8198-45c9755867d6 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:02:34.229031 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4457/8943] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:02:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 22:02:36.476294 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589bace-b4c6-4103-b791-b42b7d69dbd0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:02:36.476742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589bace-b4c6-4103-b791-b42b7d69dbd0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:02:36.476868 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a589bace-b4c6-4103-b791-b42b7d69dbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:02:36.495231 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-a589bace-b4c6-4103-b791-b42b7d69dbd0 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:02:36.498047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a589bace-b4c6-4103-b791-b42b7d69dbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:02:36.506472 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4487/8944] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 22:02:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:02:36.518873 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06900ce0-9c51-46aa-8198-45c9755867d6 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:02:36.530734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06900ce0-9c51-46aa-8198-45c9755867d6 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:02:36.536901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa488f0-3712-41e4-be94-c32a2e6885ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:02:36.537545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa488f0-3712-41e4-be94-c32a2e6885ba service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:02:36.537831 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa488f0-3712-41e4-be94-c32a2e6885ba service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:02:36.538130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa488f0-3712-41e4-be94-c32a2e6885ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:02:36.548711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa488f0-3712-41e4-be94-c32a2e6885ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:02:36.555395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa488f0-3712-41e4-be94-c32a2e6885ba service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:02:36.562465 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4458/8945] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:02:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:02:41.010714 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf035fd-c517-4aef-a6e8-8bde24291e30 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:02:41.011190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf035fd-c517-4aef-a6e8-8bde24291e30 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:02:41.011299 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf035fd-c517-4aef-a6e8-8bde24291e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:02:41.029899 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-dcf035fd-c517-4aef-a6e8-8bde24291e30 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:02:41.031113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dcf035fd-c517-4aef-a6e8-8bde24291e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:02:41.104913 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4488/8946] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 22:02:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:02:41.119623 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa488f0-3712-41e4-be94-c32a2e6885ba service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:02:41.121035 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-1fa488f0-3712-41e4-be94-c32a2e6885ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:02:41.137707 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa488f0-3712-41e4-be94-c32a2e6885ba service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:02:41.143859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a523ad9-10e0-430c-a64c-b225206657e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:02:41.144516 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a523ad9-10e0-430c-a64c-b225206657e2 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:02:41.144863 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a523ad9-10e0-430c-a64c-b225206657e2 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:02:41.145142 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a523ad9-10e0-430c-a64c-b225206657e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:02:41.154234 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a523ad9-10e0-430c-a64c-b225206657e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:02:41.155834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a523ad9-10e0-430c-a64c-b225206657e2 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:02:41.166769 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4459/8947] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:02:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:02:49.555020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d592a45-fe33-4cbe-9f5c-3c54dc6f8b52 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:02:49.555549 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d592a45-fe33-4cbe-9f5c-3c54dc6f8b52 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:02:49.555663 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d592a45-fe33-4cbe-9f5c-3c54dc6f8b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:02:49.573303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-9d592a45-fe33-4cbe-9f5c-3c54dc6f8b52 None None] MFA Rules not processed for user `56b376ad4a314418bca5fcbf8a8ca429`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:02:49.574532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9d592a45-fe33-4cbe-9f5c-3c54dc6f8b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:02:49.581384 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4489/8948] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:02:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:02:49.913499 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e24ddb-f99c-4741-83fe-5e5e9c1eb614 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:02:49.914081 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e24ddb-f99c-4741-83fe-5e5e9c1eb614 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:02:49.914254 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e24ddb-f99c-4741-83fe-5e5e9c1eb614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:02:49.940067 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-17e24ddb-f99c-4741-83fe-5e5e9c1eb614 None None] MFA Rules not processed for user `1b2a9ebcd3f84b758322a80c6e590de0`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:02:49.941551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-17e24ddb-f99c-4741-83fe-5e5e9c1eb614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:02:49.949778 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4460/8949] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 22:02:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 22:02:49.966389 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d592a45-fe33-4cbe-9f5c-3c54dc6f8b52 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:02:49.982754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d592a45-fe33-4cbe-9f5c-3c54dc6f8b52 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:02:49.988962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123b1ba-ba62-4d94-81d8-8131b34da73d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:02:49.989560 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b1ba-ba62-4d94-81d8-8131b34da73d service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:02:49.989972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b1ba-ba62-4d94-81d8-8131b34da73d service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:02:49.990318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b1ba-ba62-4d94-81d8-8131b34da73d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:02:50.004557 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123b1ba-ba62-4d94-81d8-8131b34da73d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:02:50.006869 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123b1ba-ba62-4d94-81d8-8131b34da73d service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:02:50.015901 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4490/8950] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:02:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 22:02:52.471811 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e24ddb-f99c-4741-83fe-5e5e9c1eb614 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:02:52.485791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e24ddb-f99c-4741-83fe-5e5e9c1eb614 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:02:52.491887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a3b84-41cc-4c22-92a1-87dbec737d6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:02:52.492631 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a3b84-41cc-4c22-92a1-87dbec737d6e service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:02:52.493012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a3b84-41cc-4c22-92a1-87dbec737d6e service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:02:52.493369 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a3b84-41cc-4c22-92a1-87dbec737d6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:02:52.495279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4b0a3b84-41cc-4c22-92a1-87dbec737d6e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:02:52.507991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0a3b84-41cc-4c22-92a1-87dbec737d6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:02:52.509630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0a3b84-41cc-4c22-92a1-87dbec737d6e service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:02:52.529667 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4461/8951] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:02:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:03:00.012466 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=48 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:03:02.516316 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=27 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:03:04.026063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416eebe4-d918-41a6-a773-2b5d1fba63f5 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:03:04.026575 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416eebe4-d918-41a6-a773-2b5d1fba63f5 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:03:04.026670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416eebe4-d918-41a6-a773-2b5d1fba63f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:03:04.045491 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-416eebe4-d918-41a6-a773-2b5d1fba63f5 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:03:04.046722 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-416eebe4-d918-41a6-a773-2b5d1fba63f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:03:04.056840 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4491/8952] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 22:03:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:03:04.069793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a3b84-41cc-4c22-92a1-87dbec737d6e service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:03:04.083494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a3b84-41cc-4c22-92a1-87dbec737d6e service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:03:04.089120 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da07a9d-adc4-4704-b83d-4f10da8b88e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:03:04.089761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da07a9d-adc4-4704-b83d-4f10da8b88e3 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:03:04.090080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da07a9d-adc4-4704-b83d-4f10da8b88e3 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:03:04.090362 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da07a9d-adc4-4704-b83d-4f10da8b88e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:03:04.101210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da07a9d-adc4-4704-b83d-4f10da8b88e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:03:04.102844 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da07a9d-adc4-4704-b83d-4f10da8b88e3 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:03:04.113271 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4462/8953] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:03:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:03:14.040016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:03:14.108976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:03:37.001225 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a85811-9102-47ed-b6d8-cbc4d0064878 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:03:37.001954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a85811-9102-47ed-b6d8-cbc4d0064878 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:03:37.001954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a85811-9102-47ed-b6d8-cbc4d0064878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:03:37.022018 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-72a85811-9102-47ed-b6d8-cbc4d0064878 None None] MFA Rules not processed for user `1b2a9ebcd3f84b758322a80c6e590de0`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:03:37.023317 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-72a85811-9102-47ed-b6d8-cbc4d0064878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:03:37.030762 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4492/8954] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 22:03:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:03:37.042637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da07a9d-adc4-4704-b83d-4f10da8b88e3 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:03:37.051933 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da07a9d-adc4-4704-b83d-4f10da8b88e3 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:03:37.055570 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3fbbc-814f-4c77-9610-3c170df67bb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:03:37.056203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3fbbc-814f-4c77-9610-3c170df67bb3 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:03:37.056514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3fbbc-814f-4c77-9610-3c170df67bb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:03:37.056795 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3fbbc-814f-4c77-9610-3c170df67bb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:03:37.065556 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3fbbc-814f-4c77-9610-3c170df67bb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:03:37.067551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3fbbc-814f-4c77-9610-3c170df67bb3 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:03:37.079087 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4463/8955] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:03:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:03:37.250109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18521f7-caf6-420b-94a7-6d6e209dd84c None None] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:03:37.250404 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18521f7-caf6-420b-94a7-6d6e209dd84c None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:03:37.250680 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18521f7-caf6-420b-94a7-6d6e209dd84c None None] PATH_INFO: `/` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:03:37.252063 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4493/8956] 10.222.0.33 () {54 vars in 921 bytes} [Sat May 28 22:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 28 22:03:37.256975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d41b69-356a-47c0-ac08-af0c4575aaa6 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:03:37.257275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d41b69-356a-47c0-ac08-af0c4575aaa6 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:03:37.257564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d41b69-356a-47c0-ac08-af0c4575aaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:03:37.275671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a6d41b69-356a-47c0-ac08-af0c4575aaa6 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:03:37.276938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a6d41b69-356a-47c0-ac08-af0c4575aaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:03:37.285701 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4464/8957] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 22:03:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:03:37.306223 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18521f7-caf6-420b-94a7-6d6e209dd84c None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:03:37.308646 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a18521f7-caf6-420b-94a7-6d6e209dd84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:03:37.325282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18521f7-caf6-420b-94a7-6d6e209dd84c None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:03:37.331016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1d785-e313-4a08-8b05-30e25e0f459b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:03:37.331689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1d785-e313-4a08-8b05-30e25e0f459b service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:03:37.331974 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1d785-e313-4a08-8b05-30e25e0f459b service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:03:37.332267 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1d785-e313-4a08-8b05-30e25e0f459b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:03:37.340964 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1d785-e313-4a08-8b05-30e25e0f459b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:03:37.342746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1d785-e313-4a08-8b05-30e25e0f459b service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:03:37.352521 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4494/8958] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:03:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:03:37.663206 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ea09c-15e3-4c23-b532-339fd676dc82 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:03:37.663530 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ea09c-15e3-4c23-b532-339fd676dc82 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:03:37.663816 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ea09c-15e3-4c23-b532-339fd676dc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:03:37.682743 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-fe0ea09c-15e3-4c23-b532-339fd676dc82 None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:03:37.683946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-fe0ea09c-15e3-4c23-b532-339fd676dc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:03:37.693438 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4465/8959] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 22:03:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:03:37.704739 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1d785-e313-4a08-8b05-30e25e0f459b service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:03:37.714726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1d785-e313-4a08-8b05-30e25e0f459b service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:03:37.718318 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0804333-3e08-4841-9fbe-905457883a19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:03:37.718951 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0804333-3e08-4841-9fbe-905457883a19 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:03:37.719448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0804333-3e08-4841-9fbe-905457883a19 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:03:37.719631 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0804333-3e08-4841-9fbe-905457883a19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:03:37.730927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0804333-3e08-4841-9fbe-905457883a19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:03:37.732387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0804333-3e08-4841-9fbe-905457883a19 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:03:37.743155 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4495/8960] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:03:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:03:37.803342 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ea09c-15e3-4c23-b532-339fd676dc82 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:03:37.814711 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ea09c-15e3-4c23-b532-339fd676dc82 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:03:37.819836 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f15c1-2b8e-4fad-9fcf-1105648ba094 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:03:37.820470 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f15c1-2b8e-4fad-9fcf-1105648ba094 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:03:37.820758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f15c1-2b8e-4fad-9fcf-1105648ba094 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:03:37.821037 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f15c1-2b8e-4fad-9fcf-1105648ba094 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:03:37.823091 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-114f15c1-2b8e-4fad-9fcf-1105648ba094 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:03:37.834401 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f15c1-2b8e-4fad-9fcf-1105648ba094 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:03:37.835993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114f15c1-2b8e-4fad-9fcf-1105648ba094 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:03:37.846299 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4466/8961] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:03:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:03:47.738540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=15 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:03:47.842110 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=18 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:03:52.415608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d739b-2672-40d6-b961-04e5e8722826 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:03:52.416171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d739b-2672-40d6-b961-04e5e8722826 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:03:52.416171 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d739b-2672-40d6-b961-04e5e8722826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:03:52.455550 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-6a0d739b-2672-40d6-b961-04e5e8722826 None None] MFA Rules not processed for user `1b2a9ebcd3f84b758322a80c6e590de0`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:03:52.456689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6a0d739b-2672-40d6-b961-04e5e8722826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:03:52.482785 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4496/8962] 10.222.0.15 () {58 vars in 1015 bytes} [Sat May 28 22:03:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:03:52.494231 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f15c1-2b8e-4fad-9fcf-1105648ba094 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:03:52.514138 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114f15c1-2b8e-4fad-9fcf-1105648ba094 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:03:52.517932 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ee0b0-b8ee-4369-94c7-eaf1907e12dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:03:52.522158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ee0b0-b8ee-4369-94c7-eaf1907e12dc service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:03:52.522158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ee0b0-b8ee-4369-94c7-eaf1907e12dc service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:03:52.522158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ee0b0-b8ee-4369-94c7-eaf1907e12dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:03:52.526203 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ee0b0-b8ee-4369-94c7-eaf1907e12dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:03:52.527667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ee0b0-b8ee-4369-94c7-eaf1907e12dc service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:03:52.535485 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4467/8963] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:03:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:04:02.480337 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=21 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:04:02.532029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=11 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:04:27.174867 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d739b-2672-40d6-b961-04e5e8722826 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:04:27.208581 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d739b-2672-40d6-b961-04e5e8722826 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:04:27.214129 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bdd451-23db-40be-8a7c-a4a6f31263f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:04:27.214790 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdd451-23db-40be-8a7c-a4a6f31263f8 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:04:27.215078 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdd451-23db-40be-8a7c-a4a6f31263f8 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:04:27.215357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bdd451-23db-40be-8a7c-a4a6f31263f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:04:27.217704 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-f6bdd451-23db-40be-8a7c-a4a6f31263f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:04:27.230093 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bdd451-23db-40be-8a7c-a4a6f31263f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:04:27.231693 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bdd451-23db-40be-8a7c-a4a6f31263f8 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:04:27.242431 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4497/8964] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:04:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:04:29.193991 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ee0b0-b8ee-4369-94c7-eaf1907e12dc service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:04:29.206223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ee0b0-b8ee-4369-94c7-eaf1907e12dc service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:04:29.211565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef7522-4ca1-4282-99c5-51f6864cf04b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:04:29.212216 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef7522-4ca1-4282-99c5-51f6864cf04b service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:04:29.212500 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef7522-4ca1-4282-99c5-51f6864cf04b service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:04:29.212776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef7522-4ca1-4282-99c5-51f6864cf04b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:04:29.214592 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e5ef7522-4ca1-4282-99c5-51f6864cf04b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:04:29.242337 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef7522-4ca1-4282-99c5-51f6864cf04b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:04:29.244312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef7522-4ca1-4282-99c5-51f6864cf04b service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:04:29.255891 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4468/8965] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:04:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:04:34.548077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664c3ba-6ec7-448d-8f4a-0c262181e369 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:04:34.548595 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664c3ba-6ec7-448d-8f4a-0c262181e369 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:04:34.548774 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664c3ba-6ec7-448d-8f4a-0c262181e369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:04:34.567812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1664c3ba-6ec7-448d-8f4a-0c262181e369 None None] MFA Rules not processed for user `b02348c479904bac893fa8d71d077550`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:04:34.569016 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1664c3ba-6ec7-448d-8f4a-0c262181e369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:04:34.579116 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4498/8966] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 22:04:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 22:04:34.591944 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef7522-4ca1-4282-99c5-51f6864cf04b service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:04:34.602478 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef7522-4ca1-4282-99c5-51f6864cf04b service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:04:34.607708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1821d-dbaa-4ddc-a045-47bd6fa7cf19 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:04:34.608354 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1821d-dbaa-4ddc-a045-47bd6fa7cf19 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:04:34.608639 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1821d-dbaa-4ddc-a045-47bd6fa7cf19 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:04:34.608917 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1821d-dbaa-4ddc-a045-47bd6fa7cf19 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:04:34.617333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1821d-dbaa-4ddc-a045-47bd6fa7cf19 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:04:34.618961 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1821d-dbaa-4ddc-a045-47bd6fa7cf19 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:04:34.629465 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4469/8967] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:04:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:04:43.278160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ebb4d-d1f3-4a86-a0a4-6432152f4e90 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:04:43.278160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ebb4d-d1f3-4a86-a0a4-6432152f4e90 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:04:43.278745 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ebb4d-d1f3-4a86-a0a4-6432152f4e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:04:43.297982 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-8c4ebb4d-d1f3-4a86-a0a4-6432152f4e90 None None] MFA Rules not processed for user `56b376ad4a314418bca5fcbf8a8ca429`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:04:43.299539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8c4ebb4d-d1f3-4a86-a0a4-6432152f4e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:04:43.306668 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4499/8968] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:04:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 28 22:04:43.322284 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1821d-dbaa-4ddc-a045-47bd6fa7cf19 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:04:43.342745 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1821d-dbaa-4ddc-a045-47bd6fa7cf19 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:04:43.349020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765b9c8-8b57-40db-a7e9-2d4f6e3ca69f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:04:43.349757 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765b9c8-8b57-40db-a7e9-2d4f6e3ca69f service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:04:43.350151 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765b9c8-8b57-40db-a7e9-2d4f6e3ca69f service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:04:43.350529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765b9c8-8b57-40db-a7e9-2d4f6e3ca69f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:04:43.360786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765b9c8-8b57-40db-a7e9-2d4f6e3ca69f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:04:43.362694 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765b9c8-8b57-40db-a7e9-2d4f6e3ca69f service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:04:43.372401 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4470/8969] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:04:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:04:53.291538 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=21 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:04:53.367849 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=21 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:04:56.336086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ebb4d-d1f3-4a86-a0a4-6432152f4e90 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:04:56.345508 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ebb4d-d1f3-4a86-a0a4-6432152f4e90 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:04:56.349050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7e1e7-91be-4ca9-987c-2a758618aa49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:04:56.349689 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7e1e7-91be-4ca9-987c-2a758618aa49 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:04:56.350112 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7e1e7-91be-4ca9-987c-2a758618aa49 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:04:56.350398 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7e1e7-91be-4ca9-987c-2a758618aa49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:04:56.359726 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7e1e7-91be-4ca9-987c-2a758618aa49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:04:56.361157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7e1e7-91be-4ca9-987c-2a758618aa49 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:04:56.370720 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4500/8970] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:04:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:05:06.366492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:06:12.291020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765b9c8-8b57-40db-a7e9-2d4f6e3ca69f service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:06:12.300708 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765b9c8-8b57-40db-a7e9-2d4f6e3ca69f service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:06:12.305260 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cceb69-c684-492b-811c-d5eb52da26a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:06:12.306522 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cceb69-c684-492b-811c-d5eb52da26a6 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:06:12.306890 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cceb69-c684-492b-811c-d5eb52da26a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:06:12.307248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cceb69-c684-492b-811c-d5eb52da26a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:06:12.315554 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cceb69-c684-492b-811c-d5eb52da26a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:06:12.317173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cceb69-c684-492b-811c-d5eb52da26a6 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:06:12.326215 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4471/8971] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:06:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:06:12.661431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7e1e7-91be-4ca9-987c-2a758618aa49 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:06:12.670324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7e1e7-91be-4ca9-987c-2a758618aa49 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:06:12.674674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19de45e-bca3-4782-bdc0-25ea359de4c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:06:12.675542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19de45e-bca3-4782-bdc0-25ea359de4c4 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:06:12.677109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19de45e-bca3-4782-bdc0-25ea359de4c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:06:12.677399 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19de45e-bca3-4782-bdc0-25ea359de4c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:06:12.692170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19de45e-bca3-4782-bdc0-25ea359de4c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:06:12.693754 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19de45e-bca3-4782-bdc0-25ea359de4c4 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:06:12.706269 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4501/8972] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:06:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:06:13.888834 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cceb69-c684-492b-811c-d5eb52da26a6 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:06:13.897576 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cceb69-c684-492b-811c-d5eb52da26a6 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:06:13.901472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c825308-4f4b-488f-aec9-e000f821ef30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:06:13.902079 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c825308-4f4b-488f-aec9-e000f821ef30 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:06:13.902368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c825308-4f4b-488f-aec9-e000f821ef30 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:06:13.902649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c825308-4f4b-488f-aec9-e000f821ef30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:06:13.913107 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c825308-4f4b-488f-aec9-e000f821ef30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:06:13.914587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c825308-4f4b-488f-aec9-e000f821ef30 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:06:13.926006 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4472/8973] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:06:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:06:22.700537 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:06:23.921529 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=12 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:06:28.285666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28aa4a-5eba-461e-b9a4-ed2c0e36c9ee None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:06:28.286266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28aa4a-5eba-461e-b9a4-ed2c0e36c9ee None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:06:28.286527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28aa4a-5eba-461e-b9a4-ed2c0e36c9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:06:28.309524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-aa28aa4a-5eba-461e-b9a4-ed2c0e36c9ee None None] MFA Rules not processed for user `56b376ad4a314418bca5fcbf8a8ca429`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:06:28.310820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-aa28aa4a-5eba-461e-b9a4-ed2c0e36c9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:06:28.323217 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4502/8974] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:06:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:06:38.303703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:07:32.127350 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c825308-4f4b-488f-aec9-e000f821ef30 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:32.211629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c825308-4f4b-488f-aec9-e000f821ef30 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:32.217201 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed6e20-32e3-4d82-a548-e6f45a25b92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:32.218108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed6e20-32e3-4d82-a548-e6f45a25b92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:32.218420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed6e20-32e3-4d82-a548-e6f45a25b92e admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:32.218697 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ed6e20-32e3-4d82-a548-e6f45a25b92e admin admin] PATH_INFO: `/v3/users/56b376ad4a314418bca5fcbf8a8ca429` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:32.219913 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed6e20-32e3-4d82-a548-e6f45a25b92e admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b376ad4a314418bca5fcbf8a8ca429)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:32.221974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ed6e20-32e3-4d82-a548-e6f45a25b92e admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:32.382810 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d6ed6e20-32e3-4d82-a548-e6f45a25b92e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b376ad4a314418bca5fcbf8a8ca429'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:32.383471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d6ed6e20-32e3-4d82-a548-e6f45a25b92e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b376ad4a314418bca5fcbf8a8ca429'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:32.390428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d6ed6e20-32e3-4d82-a548-e6f45a25b92e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b376ad4a314418bca5fcbf8a8ca429'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:32.444198 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d6ed6e20-32e3-4d82-a548-e6f45a25b92e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b376ad4a314418bca5fcbf8a8ca429'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:32.444769 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-d6ed6e20-32e3-4d82-a548-e6f45a25b92e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b376ad4a314418bca5fcbf8a8ca429'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:32.462953 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4473/8975] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 22:07:32 2022] DELETE /identity/v3/users/56b376ad4a314418bca5fcbf8a8ca429 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 22:07:32.469214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ccf37-c6f4-4f4d-8e2c-ff02ce82daee None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:32.469720 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ccf37-c6f4-4f4d-8e2c-ff02ce82daee None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:32.469823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ccf37-c6f4-4f4d-8e2c-ff02ce82daee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:32.506626 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-398ccf37-c6f4-4f4d-8e2c-ff02ce82daee None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:07:32.507811 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-398ccf37-c6f4-4f4d-8e2c-ff02ce82daee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:07:32.567136 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4503/8976] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:07:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:07:32.577685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed6e20-32e3-4d82-a548-e6f45a25b92e admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:32.579160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d6ed6e20-32e3-4d82-a548-e6f45a25b92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:07:32.635742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ed6e20-32e3-4d82-a548-e6f45a25b92e admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:32.639851 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b435de3-7a3f-45d3-9ee8-7b9eb4353b9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:32.640490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b435de3-7a3f-45d3-9ee8-7b9eb4353b9a service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:32.640776 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b435de3-7a3f-45d3-9ee8-7b9eb4353b9a service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:32.641058 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b435de3-7a3f-45d3-9ee8-7b9eb4353b9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:32.650109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b435de3-7a3f-45d3-9ee8-7b9eb4353b9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:32.651548 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b435de3-7a3f-45d3-9ee8-7b9eb4353b9a service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:32.661647 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4474/8977] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:07:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:07:32.868331 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ccf37-c6f4-4f4d-8e2c-ff02ce82daee None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:32.869769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-398ccf37-c6f4-4f4d-8e2c-ff02ce82daee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:07:32.885050 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ccf37-c6f4-4f4d-8e2c-ff02ce82daee None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:32.891710 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:32.892590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:32.892907 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:32.893150 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03 admin admin] PATH_INFO: `/v3/projects/38d3db1f6443461f90a5be23358b6816` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:32.894436 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38d3db1f6443461f90a5be23358b6816)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:32.896360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:32.986371 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38d3db1f6443461f90a5be23358b6816', 'request_id': 'req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:32.992144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38d3db1f6443461f90a5be23358b6816', 'request_id': 'req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:33.055260 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38d3db1f6443461f90a5be23358b6816', 'request_id': 'req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:33.067400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03 admin admin] The token cache is being invalidate because project 38d3db1f6443461f90a5be23358b6816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 22:07:33.067692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:33.068596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:33.070193 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4504/8978] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 22:07:32 2022] DELETE /identity/v3/projects/38d3db1f6443461f90a5be23358b6816 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 22:07:33.074170 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b435de3-7a3f-45d3-9ee8-7b9eb4353b9a service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:33.080935 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:07:33.084767 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b435de3-7a3f-45d3-9ee8-7b9eb4353b9a service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:33.140033 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e236a75-c00f-46b9-af33-9b4850f328e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:33.140906 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e236a75-c00f-46b9-af33-9b4850f328e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:33.141188 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e236a75-c00f-46b9-af33-9b4850f328e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:33.141471 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e236a75-c00f-46b9-af33-9b4850f328e6 admin admin] PATH_INFO: `/v3/users/e289ee1f8bda46e19f327972d9152b59` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:33.142681 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e236a75-c00f-46b9-af33-9b4850f328e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e289ee1f8bda46e19f327972d9152b59)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:33.144643 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e236a75-c00f-46b9-af33-9b4850f328e6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:33.314887 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5e236a75-c00f-46b9-af33-9b4850f328e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e289ee1f8bda46e19f327972d9152b59'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:33.315589 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5e236a75-c00f-46b9-af33-9b4850f328e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e289ee1f8bda46e19f327972d9152b59'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:33.323100 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5e236a75-c00f-46b9-af33-9b4850f328e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e289ee1f8bda46e19f327972d9152b59'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:33.395538 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5e236a75-c00f-46b9-af33-9b4850f328e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e289ee1f8bda46e19f327972d9152b59'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:33.396251 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5e236a75-c00f-46b9-af33-9b4850f328e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e289ee1f8bda46e19f327972d9152b59'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:33.415508 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4475/8979] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 22:07:33 2022] DELETE /identity/v3/users/e289ee1f8bda46e19f327972d9152b59 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 22:07:33.840032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:33.842221 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:07:33.855358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7e5a2-8a8a-44b0-984c-4e321de9ff03 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:33.914011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7626ebb2-3932-48c5-8057-fb084b18fda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:33.914866 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626ebb2-3932-48c5-8057-fb084b18fda6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:33.915153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626ebb2-3932-48c5-8057-fb084b18fda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:33.915429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7626ebb2-3932-48c5-8057-fb084b18fda6 admin admin] PATH_INFO: `/v3/projects/3bccf296472843d1bbbf379ef2e52e70` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:33.916606 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626ebb2-3932-48c5-8057-fb084b18fda6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bccf296472843d1bbbf379ef2e52e70)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:33.918677 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7626ebb2-3932-48c5-8057-fb084b18fda6 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:33.926864 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:07:34.040584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7626ebb2-3932-48c5-8057-fb084b18fda6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bccf296472843d1bbbf379ef2e52e70', 'request_id': 'req-7626ebb2-3932-48c5-8057-fb084b18fda6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:34.046554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7626ebb2-3932-48c5-8057-fb084b18fda6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bccf296472843d1bbbf379ef2e52e70', 'request_id': 'req-7626ebb2-3932-48c5-8057-fb084b18fda6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:34.124639 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7626ebb2-3932-48c5-8057-fb084b18fda6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bccf296472843d1bbbf379ef2e52e70', 'request_id': 'req-7626ebb2-3932-48c5-8057-fb084b18fda6'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:34.136807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7626ebb2-3932-48c5-8057-fb084b18fda6 admin admin] The token cache is being invalidate because project 3bccf296472843d1bbbf379ef2e52e70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 22:07:34.137106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7626ebb2-3932-48c5-8057-fb084b18fda6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:34.137743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-7626ebb2-3932-48c5-8057-fb084b18fda6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:07:34.139351 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4505/8980] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 22:07:33 2022] DELETE /identity/v3/projects/3bccf296472843d1bbbf379ef2e52e70 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 22:07:34.144829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01f7c6-fc9b-47af-84e5-058baacc05a7 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:34.145294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01f7c6-fc9b-47af-84e5-058baacc05a7 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:34.145377 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01f7c6-fc9b-47af-84e5-058baacc05a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:34.192822 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4d01f7c6-fc9b-47af-84e5-058baacc05a7 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:07:34.194331 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4d01f7c6-fc9b-47af-84e5-058baacc05a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:07:34.245413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:07:34.254357 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4476/8981] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:07:34.262701 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6934c-8802-455b-8651-5ea56a4dfa7f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:34.262992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6934c-8802-455b-8651-5ea56a4dfa7f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:34.263274 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6934c-8802-455b-8651-5ea56a4dfa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:34.281284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-28d6934c-8802-455b-8651-5ea56a4dfa7f None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:07:34.282516 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-28d6934c-8802-455b-8651-5ea56a4dfa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:07:34.290495 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4506/8982] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:07:34.297574 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01f7c6-fc9b-47af-84e5-058baacc05a7 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:34.307456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01f7c6-fc9b-47af-84e5-058baacc05a7 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:34.313480 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0d3e1-6f3d-4df0-9acc-e18cc6060967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:34.314176 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0d3e1-6f3d-4df0-9acc-e18cc6060967 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:34.314467 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0d3e1-6f3d-4df0-9acc-e18cc6060967 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:34.314748 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e0d3e1-6f3d-4df0-9acc-e18cc6060967 admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:34.315476 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e0d3e1-6f3d-4df0-9acc-e18cc6060967 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 22:07:34.315786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e0d3e1-6f3d-4df0-9acc-e18cc6060967 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:34.317444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e0d3e1-6f3d-4df0-9acc-e18cc6060967 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:34.331042 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4477/8983] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 22:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 22:07:34.335486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6934c-8802-455b-8651-5ea56a4dfa7f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:34.345554 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6934c-8802-455b-8651-5ea56a4dfa7f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:34.350360 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82363f-a81c-4daf-8ab0-94a5957579dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:34.351020 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82363f-a81c-4daf-8ab0-94a5957579dc admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:34.351307 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82363f-a81c-4daf-8ab0-94a5957579dc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:34.351589 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82363f-a81c-4daf-8ab0-94a5957579dc admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:34.352311 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82363f-a81c-4daf-8ab0-94a5957579dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:34.354296 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82363f-a81c-4daf-8ab0-94a5957579dc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:34.440049 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4507/8984] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 22:07:34 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 22:07:34.444154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0d3e1-6f3d-4df0-9acc-e18cc6060967 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:34.454866 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e0d3e1-6f3d-4df0-9acc-e18cc6060967 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:34.511069 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28201641-d47c-4394-a92c-48d85f7f0527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:34.511735 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28201641-d47c-4394-a92c-48d85f7f0527 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:34.512050 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28201641-d47c-4394-a92c-48d85f7f0527 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:34.512329 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28201641-d47c-4394-a92c-48d85f7f0527 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:34.513025 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28201641-d47c-4394-a92c-48d85f7f0527 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:34.514979 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28201641-d47c-4394-a92c-48d85f7f0527 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:34.578988 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4478/8985] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 22:07:34 2022] POST /identity/v3/users => generated 286 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 22:07:34.583303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82363f-a81c-4daf-8ab0-94a5957579dc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:34.593761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82363f-a81c-4daf-8ab0-94a5957579dc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:34.598411 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764a156-1926-479e-893c-237c44ca3dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:34.599067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764a156-1926-479e-893c-237c44ca3dbe admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:34.599356 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764a156-1926-479e-893c-237c44ca3dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:34.599633 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6764a156-1926-479e-893c-237c44ca3dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:34.600400 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764a156-1926-479e-893c-237c44ca3dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:34.602022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6764a156-1926-479e-893c-237c44ca3dbe admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:34.613292 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4508/8986] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 22:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 22:07:34.617248 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28201641-d47c-4394-a92c-48d85f7f0527 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:34.627960 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28201641-d47c-4394-a92c-48d85f7f0527 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:34.632620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e53814c-331a-4fe6-8413-e9f1e05c6ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:34.633269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53814c-331a-4fe6-8413-e9f1e05c6ac0 admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:34.633552 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53814c-331a-4fe6-8413-e9f1e05c6ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:34.633826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e53814c-331a-4fe6-8413-e9f1e05c6ac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:34.634580 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53814c-331a-4fe6-8413-e9f1e05c6ac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:34.636173 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e53814c-331a-4fe6-8413-e9f1e05c6ac0 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:34.646588 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4479/8987] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 22:07:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 22:07:34.650607 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764a156-1926-479e-893c-237c44ca3dbe admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:34.660542 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6764a156-1926-479e-893c-237c44ca3dbe admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:34.665292 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ebe00-fc0c-4a62-878c-f6c17f87baea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:34.666207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ebe00-fc0c-4a62-878c-f6c17f87baea admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:34.666487 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ebe00-fc0c-4a62-878c-f6c17f87baea admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:34.666773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ebe00-fc0c-4a62-878c-f6c17f87baea admin admin] PATH_INFO: `/v3/projects/4ef8feba0c8e45fdb82f09eeab276061/users/31c2c3ecbc7148859a7c530deef8ab95/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:34.681548 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ebe00-fc0c-4a62-878c-f6c17f87baea admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ef8feba0c8e45fdb82f09eeab276061, user_id=31c2c3ecbc7148859a7c530deef8ab95, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:34.684310 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ebe00-fc0c-4a62-878c-f6c17f87baea admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:34.772795 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4509/8988] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 22:07:34 2022] PUT /identity/v3/projects/4ef8feba0c8e45fdb82f09eeab276061/users/31c2c3ecbc7148859a7c530deef8ab95/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 22:07:34.777965 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47f46e-7511-4fdb-a5f3-a9b433632f0c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:34.778266 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47f46e-7511-4fdb-a5f3-a9b433632f0c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:34.778550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47f46e-7511-4fdb-a5f3-a9b433632f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:34.796885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-6a47f46e-7511-4fdb-a5f3-a9b433632f0c None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:07:34.798136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6a47f46e-7511-4fdb-a5f3-a9b433632f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:07:34.856381 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4480/8989] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:07:34.866563 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ebe00-fc0c-4a62-878c-f6c17f87baea admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:34.868263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-bf8ebe00-fc0c-4a62-878c-f6c17f87baea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:07:34.926147 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ebe00-fc0c-4a62-878c-f6c17f87baea admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:34.929787 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19c66f-e0ed-4bfb-9a60-ed652a37a6b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:34.930461 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19c66f-e0ed-4bfb-9a60-ed652a37a6b4 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:34.930746 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19c66f-e0ed-4bfb-9a60-ed652a37a6b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:34.931023 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19c66f-e0ed-4bfb-9a60-ed652a37a6b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:34.940566 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f19c66f-e0ed-4bfb-9a60-ed652a37a6b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:34.942021 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f19c66f-e0ed-4bfb-9a60-ed652a37a6b4 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:34.951747 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4510/8990] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:07:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:07:40.345416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570df8a5-d4c7-43e5-bfcb-e86d4ee08eb1 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:40.346051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570df8a5-d4c7-43e5-bfcb-e86d4ee08eb1 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:40.346051 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570df8a5-d4c7-43e5-bfcb-e86d4ee08eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:40.378813 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-570df8a5-d4c7-43e5-bfcb-e86d4ee08eb1 None None] MFA Rules not processed for user `31c2c3ecbc7148859a7c530deef8ab95`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:07:40.380108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-570df8a5-d4c7-43e5-bfcb-e86d4ee08eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:07:40.445751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:07:40.455115 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4481/8991] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:07:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:07:40.464914 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19c66f-e0ed-4bfb-9a60-ed652a37a6b4 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:40.475837 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19c66f-e0ed-4bfb-9a60-ed652a37a6b4 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:40.480799 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca52f33-6a7b-4612-95a4-f43690406c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:40.481472 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca52f33-6a7b-4612-95a4-f43690406c85 admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:40.482438 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca52f33-6a7b-4612-95a4-f43690406c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:40.482719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca52f33-6a7b-4612-95a4-f43690406c85 admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:40.483431 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca52f33-6a7b-4612-95a4-f43690406c85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:40.485357 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca52f33-6a7b-4612-95a4-f43690406c85 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:40.608405 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4511/8992] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 22:07:40 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 28 22:07:40.612669 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570df8a5-d4c7-43e5-bfcb-e86d4ee08eb1 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:40.624269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570df8a5-d4c7-43e5-bfcb-e86d4ee08eb1 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:40.686936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6118f-abad-4a4f-8c22-b5ed27c99f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:40.687617 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6118f-abad-4a4f-8c22-b5ed27c99f68 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:40.687903 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6118f-abad-4a4f-8c22-b5ed27c99f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:40.688183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b6118f-abad-4a4f-8c22-b5ed27c99f68 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:40.688884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b6118f-abad-4a4f-8c22-b5ed27c99f68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:40.690898 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b6118f-abad-4a4f-8c22-b5ed27c99f68 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:40.796020 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4482/8993] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 22:07:40 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 22:07:40.798146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca52f33-6a7b-4612-95a4-f43690406c85 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:40.809058 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca52f33-6a7b-4612-95a4-f43690406c85 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:40.814742 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb73a6-5497-49e6-b4ad-a856bcd51e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:40.815517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb73a6-5497-49e6-b4ad-a856bcd51e51 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:40.815871 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb73a6-5497-49e6-b4ad-a856bcd51e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:40.816250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb73a6-5497-49e6-b4ad-a856bcd51e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:40.817063 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb73a6-5497-49e6-b4ad-a856bcd51e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:40.819125 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb73a6-5497-49e6-b4ad-a856bcd51e51 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:40.832660 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4512/8994] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 22:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 28 22:07:40.837141 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6118f-abad-4a4f-8c22-b5ed27c99f68 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:40.848696 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b6118f-abad-4a4f-8c22-b5ed27c99f68 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:40.853588 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5a1b1-71fe-49a8-8696-5ae87ac4e01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:40.854520 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5a1b1-71fe-49a8-8696-5ae87ac4e01d admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:40.854841 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5a1b1-71fe-49a8-8696-5ae87ac4e01d admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:40.855125 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff5a1b1-71fe-49a8-8696-5ae87ac4e01d admin admin] PATH_INFO: `/v3/projects/44d5836dc89442619d209da7b7e1c3b2/users/b71d46b517584c3fa0531443612f1528/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:40.871667 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5a1b1-71fe-49a8-8696-5ae87ac4e01d admin admin] RBAC: Authorizing `identity:create_grant(project_id=44d5836dc89442619d209da7b7e1c3b2, user_id=b71d46b517584c3fa0531443612f1528, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:40.874431 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff5a1b1-71fe-49a8-8696-5ae87ac4e01d admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:40.937063 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4483/8995] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 22:07:40 2022] PUT /identity/v3/projects/44d5836dc89442619d209da7b7e1c3b2/users/b71d46b517584c3fa0531443612f1528/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 22:07:40.941417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb73a6-5497-49e6-b4ad-a856bcd51e51 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:40.947119 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:07:40.953256 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb73a6-5497-49e6-b4ad-a856bcd51e51 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:41.012587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4438462-84cf-42c6-b940-6c8b7d28cc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:41.013960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4438462-84cf-42c6-b940-6c8b7d28cc0b admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:41.014425 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4438462-84cf-42c6-b940-6c8b7d28cc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:41.014717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4438462-84cf-42c6-b940-6c8b7d28cc0b admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:41.015457 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4438462-84cf-42c6-b940-6c8b7d28cc0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:41.017077 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4438462-84cf-42c6-b940-6c8b7d28cc0b admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:41.029452 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4513/8996] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 22:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 22:07:41.033642 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5a1b1-71fe-49a8-8696-5ae87ac4e01d admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:41.044535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff5a1b1-71fe-49a8-8696-5ae87ac4e01d admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:41.049309 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f51cbb-7398-4e51-ad98-677be379c0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:41.050333 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f51cbb-7398-4e51-ad98-677be379c0bb admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:41.050619 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f51cbb-7398-4e51-ad98-677be379c0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:41.050894 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f51cbb-7398-4e51-ad98-677be379c0bb admin admin] PATH_INFO: `/v3/projects/44d5836dc89442619d209da7b7e1c3b2/users/b71d46b517584c3fa0531443612f1528/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:41.052946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f51cbb-7398-4e51-ad98-677be379c0bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=44d5836dc89442619d209da7b7e1c3b2, user_id=b71d46b517584c3fa0531443612f1528, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:41.055731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f51cbb-7398-4e51-ad98-677be379c0bb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:41.072992 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4484/8997] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 22:07:41 2022] PUT /identity/v3/projects/44d5836dc89442619d209da7b7e1c3b2/users/b71d46b517584c3fa0531443612f1528/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 22:07:46.573188 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac1c44-4791-438c-86cb-e98bcdf46ea9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:46.573655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac1c44-4791-438c-86cb-e98bcdf46ea9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:46.573755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ac1c44-4791-438c-86cb-e98bcdf46ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:46.601827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-31ac1c44-4791-438c-86cb-e98bcdf46ea9 None None] MFA Rules not processed for user `b71d46b517584c3fa0531443612f1528`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:07:46.603220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-31ac1c44-4791-438c-86cb-e98bcdf46ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:07:46.673800 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4514/8998] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:07:46 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:07:46.690790 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f51cbb-7398-4e51-ad98-677be379c0bb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:46.692618 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-05f51cbb-7398-4e51-ad98-677be379c0bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:07:46.789768 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f51cbb-7398-4e51-ad98-677be379c0bb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:46.795974 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74fd798-f215-4b6a-a3f7-9c01fc27efab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:46.796626 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fd798-f215-4b6a-a3f7-9c01fc27efab service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:46.796946 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fd798-f215-4b6a-a3f7-9c01fc27efab service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:46.797227 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74fd798-f215-4b6a-a3f7-9c01fc27efab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:46.832493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:07:46.855953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fd798-f215-4b6a-a3f7-9c01fc27efab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:46.858119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74fd798-f215-4b6a-a3f7-9c01fc27efab service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:46.869873 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4485/8999] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:07:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:07:49.436013 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5da8bf-e784-4f8b-ab06-3c0faa9d59b6 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:49.436502 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5da8bf-e784-4f8b-ab06-3c0faa9d59b6 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:49.436596 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5da8bf-e784-4f8b-ab06-3c0faa9d59b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:49.456587 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-5a5da8bf-e784-4f8b-ab06-3c0faa9d59b6 None None] MFA Rules not processed for user `31c2c3ecbc7148859a7c530deef8ab95`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:07:49.457823 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-5a5da8bf-e784-4f8b-ab06-3c0faa9d59b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:07:49.465531 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4515/9000] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:07:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:07:49.776519 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce9d73-d3e3-4a67-9563-1f11533b2f14 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:49.777020 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce9d73-d3e3-4a67-9563-1f11533b2f14 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:49.777288 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce9d73-d3e3-4a67-9563-1f11533b2f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:49.796300 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-4bce9d73-d3e3-4a67-9563-1f11533b2f14 None None] MFA Rules not processed for user `1b2a9ebcd3f84b758322a80c6e590de0`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:07:49.797545 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4bce9d73-d3e3-4a67-9563-1f11533b2f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:07:49.851451 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4486/9001] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 22:07:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:07:49.864298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5da8bf-e784-4f8b-ab06-3c0faa9d59b6 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:49.874434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5da8bf-e784-4f8b-ab06-3c0faa9d59b6 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:49.878215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0a135-a772-4437-ac26-33edc99c9ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:49.878872 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0a135-a772-4437-ac26-33edc99c9ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:49.879247 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0a135-a772-4437-ac26-33edc99c9ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:49.880203 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0a135-a772-4437-ac26-33edc99c9ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:49.887919 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0a135-a772-4437-ac26-33edc99c9ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:49.892590 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0a135-a772-4437-ac26-33edc99c9ed4 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:49.898009 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4516/9002] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:07:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:07:52.460045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce9d73-d3e3-4a67-9563-1f11533b2f14 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:52.461454 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-4bce9d73-d3e3-4a67-9563-1f11533b2f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:07:52.546343 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce9d73-d3e3-4a67-9563-1f11533b2f14 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:52.551320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca881f-6e8c-455a-b6a9-eaf7f34c315d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:52.551975 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca881f-6e8c-455a-b6a9-eaf7f34c315d service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:52.552289 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca881f-6e8c-455a-b6a9-eaf7f34c315d service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:52.552568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca881f-6e8c-455a-b6a9-eaf7f34c315d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:52.554394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-25ca881f-6e8c-455a-b6a9-eaf7f34c315d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:07:52.566533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca881f-6e8c-455a-b6a9-eaf7f34c315d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:52.568722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca881f-6e8c-455a-b6a9-eaf7f34c315d service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:52.579558 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4487/9003] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:07:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:07:58.415286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0a135-a772-4437-ac26-33edc99c9ed4 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:07:58.428913 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0a135-a772-4437-ac26-33edc99c9ed4 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:07:58.437948 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0aca2-829b-40f3-88cd-de90a7b20873 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:07:58.438736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0aca2-829b-40f3-88cd-de90a7b20873 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:07:58.439123 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0aca2-829b-40f3-88cd-de90a7b20873 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:07:58.439623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0aca2-829b-40f3-88cd-de90a7b20873 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:07:58.442936 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-17d0aca2-829b-40f3-88cd-de90a7b20873 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:07:58.457777 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0aca2-829b-40f3-88cd-de90a7b20873 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:07:58.459712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0aca2-829b-40f3-88cd-de90a7b20873 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:07:58.474028 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4517/9004] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:07:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:08:02.574597 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=72 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:08:06.468169 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca881f-6e8c-455a-b6a9-eaf7f34c315d service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:08:06.477281 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca881f-6e8c-455a-b6a9-eaf7f34c315d service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:08:06.481118 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6fff5-1830-4eeb-929c-5dce40ade396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:08:06.481751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6fff5-1830-4eeb-929c-5dce40ade396 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:08:06.482088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6fff5-1830-4eeb-929c-5dce40ade396 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:08:06.482367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6fff5-1830-4eeb-929c-5dce40ade396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:08:06.491444 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6fff5-1830-4eeb-929c-5dce40ade396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:08:06.492867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6fff5-1830-4eeb-929c-5dce40ade396 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:08:06.502261 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4488/9005] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:08:06 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:08:08.416071 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0aca2-829b-40f3-88cd-de90a7b20873 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:08:08.428098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0aca2-829b-40f3-88cd-de90a7b20873 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:08:08.433817 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95978e97-8cd9-4264-b6db-8102ec174b84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:08:08.434521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95978e97-8cd9-4264-b6db-8102ec174b84 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:08:08.434928 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95978e97-8cd9-4264-b6db-8102ec174b84 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:08:08.435215 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95978e97-8cd9-4264-b6db-8102ec174b84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:08:08.437740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-95978e97-8cd9-4264-b6db-8102ec174b84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:08:08.450684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95978e97-8cd9-4264-b6db-8102ec174b84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:08:08.452351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95978e97-8cd9-4264-b6db-8102ec174b84 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:08:08.463518 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4518/9006] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:08:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:08:16.498164 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:08:18.458865 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=83 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:09:29.183514 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6fff5-1830-4eeb-929c-5dce40ade396 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:09:29.184943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-89d6fff5-1830-4eeb-929c-5dce40ade396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:09:29.199363 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6fff5-1830-4eeb-929c-5dce40ade396 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:09:29.204360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013264b-dc0c-4188-89a3-de54e23f15b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:09:29.204998 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013264b-dc0c-4188-89a3-de54e23f15b2 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:09:29.205290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013264b-dc0c-4188-89a3-de54e23f15b2 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:09:29.205573 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013264b-dc0c-4188-89a3-de54e23f15b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:09:29.207294 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-3013264b-dc0c-4188-89a3-de54e23f15b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:09:29.218278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013264b-dc0c-4188-89a3-de54e23f15b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:09:29.219901 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3013264b-dc0c-4188-89a3-de54e23f15b2 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:09:29.230188 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4489/9007] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:09:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:09:31.575780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95978e97-8cd9-4264-b6db-8102ec174b84 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:09:31.587255 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95978e97-8cd9-4264-b6db-8102ec174b84 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:09:31.592295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096531c4-a6b5-47be-8c1c-6583a391c306 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:09:31.592935 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096531c4-a6b5-47be-8c1c-6583a391c306 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:09:31.593228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096531c4-a6b5-47be-8c1c-6583a391c306 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:09:31.593509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096531c4-a6b5-47be-8c1c-6583a391c306 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:09:31.595322 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-096531c4-a6b5-47be-8c1c-6583a391c306 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:09:31.606614 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096531c4-a6b5-47be-8c1c-6583a391c306 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:09:31.608208 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096531c4-a6b5-47be-8c1c-6583a391c306 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:09:31.619382 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4519/9008] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:09:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:09:39.226108 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:09:41.614576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:10:16.024867 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013264b-dc0c-4188-89a3-de54e23f15b2 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:10:16.037402 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013264b-dc0c-4188-89a3-de54e23f15b2 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:10:16.043564 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840736e7-0c3f-4f15-baf7-5e6580f31b28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:10:16.044218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840736e7-0c3f-4f15-baf7-5e6580f31b28 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:10:16.044504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840736e7-0c3f-4f15-baf7-5e6580f31b28 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:10:16.044782 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840736e7-0c3f-4f15-baf7-5e6580f31b28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:10:16.046629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-840736e7-0c3f-4f15-baf7-5e6580f31b28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:10:16.065233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840736e7-0c3f-4f15-baf7-5e6580f31b28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:10:16.067308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840736e7-0c3f-4f15-baf7-5e6580f31b28 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:10:16.080436 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4490/9009] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:10:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:10:16.371270 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459430bd-1190-45cf-b9b3-acfcd337e64d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:10:16.371763 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459430bd-1190-45cf-b9b3-acfcd337e64d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:10:16.371873 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459430bd-1190-45cf-b9b3-acfcd337e64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:10:16.430434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-459430bd-1190-45cf-b9b3-acfcd337e64d None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:10:16.431906 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-459430bd-1190-45cf-b9b3-acfcd337e64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:10:16.496486 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4520/9010] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 22:10:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:10:16.634791 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840736e7-0c3f-4f15-baf7-5e6580f31b28 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:10:16.636210 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-840736e7-0c3f-4f15-baf7-5e6580f31b28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:10:16.724840 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840736e7-0c3f-4f15-baf7-5e6580f31b28 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:10:16.729939 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9da1d-53a8-4d91-ba73-c35bfcec6548 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:10:16.730587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9da1d-53a8-4d91-ba73-c35bfcec6548 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:10:16.730875 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9da1d-53a8-4d91-ba73-c35bfcec6548 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:10:16.731154 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9da1d-53a8-4d91-ba73-c35bfcec6548 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:10:16.739229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9da1d-53a8-4d91-ba73-c35bfcec6548 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:10:16.740885 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9da1d-53a8-4d91-ba73-c35bfcec6548 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:10:16.749428 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4491/9011] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 22:10:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 22:10:26.493492 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=44 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:10:26.746185 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=47 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:10:30.703782 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6faba-10fa-463b-adf0-65346166becb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:10:30.704345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6faba-10fa-463b-adf0-65346166becb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:10:30.704345 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6faba-10fa-463b-adf0-65346166becb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:10:30.723197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-80b6faba-10fa-463b-adf0-65346166becb None None] MFA Rules not processed for user `cfdebb5e6b8d4e2d86f0a03a30c21671`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:10:30.726688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-80b6faba-10fa-463b-adf0-65346166becb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:10:30.734562 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4521/9012] 10.222.0.33 () {58 vars in 1018 bytes} [Sat May 28 22:10:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:10:30.745761 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bca519-a286-44b2-9d08-539851447ce5 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:10:30.751243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bca519-a286-44b2-9d08-539851447ce5 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:10:30.751243 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bca519-a286-44b2-9d08-539851447ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:10:30.795999 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-67bca519-a286-44b2-9d08-539851447ce5 None None] MFA Rules not processed for user `224c1a0427c942e5bfef2bac75648d80`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:10:30.797222 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-67bca519-a286-44b2-9d08-539851447ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:10:30.858099 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4492/9013] 10.222.0.33 () {58 vars in 1059 bytes} [Sat May 28 22:10:30 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:10:30.864067 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6faba-10fa-463b-adf0-65346166becb None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:10:30.873249 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6faba-10fa-463b-adf0-65346166becb None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:10:30.876838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fb43f-2d66-4e01-90c4-3babf47906c8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:10:30.877475 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fb43f-2d66-4e01-90c4-3babf47906c8 service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:10:30.877761 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fb43f-2d66-4e01-90c4-3babf47906c8 service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:10:30.878085 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255fb43f-2d66-4e01-90c4-3babf47906c8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:10:30.888234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fb43f-2d66-4e01-90c4-3babf47906c8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:10:30.889670 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255fb43f-2d66-4e01-90c4-3babf47906c8 service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:10:30.900909 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4522/9014] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:10:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:10:31.092066 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bca519-a286-44b2-9d08-539851447ce5 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:10:31.093491 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-67bca519-a286-44b2-9d08-539851447ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:10:31.113930 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bca519-a286-44b2-9d08-539851447ce5 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:10:31.117418 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef95a2e-2c1b-4973-9901-d0273f2240b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:10:31.118088 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef95a2e-2c1b-4973-9901-d0273f2240b4 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:10:31.118371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef95a2e-2c1b-4973-9901-d0273f2240b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:10:31.118655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef95a2e-2c1b-4973-9901-d0273f2240b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:10:31.120394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-aef95a2e-2c1b-4973-9901-d0273f2240b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:10:31.129438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef95a2e-2c1b-4973-9901-d0273f2240b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:10:31.130968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef95a2e-2c1b-4973-9901-d0273f2240b4 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:10:31.139386 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4493/9015] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:10:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:10:31.217546 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fb43f-2d66-4e01-90c4-3babf47906c8 service glance] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:10:31.226946 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255fb43f-2d66-4e01-90c4-3babf47906c8 service glance] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:10:31.230578 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c5be7-49da-48fe-9977-1554bff53d2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:10:31.231207 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c5be7-49da-48fe-9977-1554bff53d2d service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:10:31.231497 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c5be7-49da-48fe-9977-1554bff53d2d service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:10:31.231778 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c5be7-49da-48fe-9977-1554bff53d2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:10:31.233524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-833c5be7-49da-48fe-9977-1554bff53d2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:10:31.246620 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c5be7-49da-48fe-9977-1554bff53d2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:10:31.248047 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c5be7-49da-48fe-9977-1554bff53d2d service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:10:31.259368 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4523/9016] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:10:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:10:31.320637 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef95a2e-2c1b-4973-9901-d0273f2240b4 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:10:31.333190 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef95a2e-2c1b-4973-9901-d0273f2240b4 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:10:31.338858 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f36312-b52d-4197-9adb-1fd5ae173f8b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:10:31.339490 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f36312-b52d-4197-9adb-1fd5ae173f8b service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:10:31.339773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f36312-b52d-4197-9adb-1fd5ae173f8b service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:10:31.340048 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f36312-b52d-4197-9adb-1fd5ae173f8b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:10:31.341792 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-11f36312-b52d-4197-9adb-1fd5ae173f8b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:10:31.353628 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f36312-b52d-4197-9adb-1fd5ae173f8b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:10:31.355398 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f36312-b52d-4197-9adb-1fd5ae173f8b service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:10:31.365091 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4494/9017] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:10:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:10:35.374883 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1c9fa-11ee-432d-be3d-d17c162de37d None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:10:35.375393 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1c9fa-11ee-432d-be3d-d17c162de37d None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:10:35.375501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1c9fa-11ee-432d-be3d-d17c162de37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:10:35.394243 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-c9b1c9fa-11ee-432d-be3d-d17c162de37d None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:10:35.395460 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-c9b1c9fa-11ee-432d-be3d-d17c162de37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:10:35.403295 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4524/9018] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 22:10:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:10:35.415012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f36312-b52d-4197-9adb-1fd5ae173f8b service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:10:35.426918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f36312-b52d-4197-9adb-1fd5ae173f8b service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:10:35.431954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fab43c-6ea7-43d6-b28f-2ba264e4f7a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:10:35.432638 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fab43c-6ea7-43d6-b28f-2ba264e4f7a7 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:10:35.432926 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fab43c-6ea7-43d6-b28f-2ba264e4f7a7 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:10:35.433230 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fab43c-6ea7-43d6-b28f-2ba264e4f7a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:10:35.442043 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fab43c-6ea7-43d6-b28f-2ba264e4f7a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:10:35.443634 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fab43c-6ea7-43d6-b28f-2ba264e4f7a7 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:10:35.453266 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4495/9019] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 22:10:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 22:10:45.387724 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=18 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:10:45.448815 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=60 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:10:49.439002 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083cff7-64d8-4dfd-bba1-da8d636c163f None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:10:49.439504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083cff7-64d8-4dfd-bba1-da8d636c163f None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:10:49.439504 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083cff7-64d8-4dfd-bba1-da8d636c163f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:10:49.464807 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-b083cff7-64d8-4dfd-bba1-da8d636c163f None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:10:49.468098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-b083cff7-64d8-4dfd-bba1-da8d636c163f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:10:49.478274 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4525/9020] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 22:10:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:10:49.491734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fab43c-6ea7-43d6-b28f-2ba264e4f7a7 service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:10:49.506521 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fab43c-6ea7-43d6-b28f-2ba264e4f7a7 service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:10:49.513095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9b76e-5d1e-4d79-ae8b-7b627ee12902 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:10:49.513864 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9b76e-5d1e-4d79-ae8b-7b627ee12902 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:10:49.514239 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9b76e-5d1e-4d79-ae8b-7b627ee12902 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:10:49.514859 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a9b76e-5d1e-4d79-ae8b-7b627ee12902 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:10:49.528373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9b76e-5d1e-4d79-ae8b-7b627ee12902 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:10:49.530126 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a9b76e-5d1e-4d79-ae8b-7b627ee12902 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:10:49.540014 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4496/9021] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 22:10:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 22:10:59.455514 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:10:59.536018 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:11:24.956015 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083cff7-64d8-4dfd-bba1-da8d636c163f None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:11:24.965530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083cff7-64d8-4dfd-bba1-da8d636c163f None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:11:24.969228 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440abc5-5abd-4a38-8316-0df8c419c845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:11:24.970419 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440abc5-5abd-4a38-8316-0df8c419c845 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:11:24.970708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440abc5-5abd-4a38-8316-0df8c419c845 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:11:24.970990 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440abc5-5abd-4a38-8316-0df8c419c845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:11:24.972881 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-2440abc5-5abd-4a38-8316-0df8c419c845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:11:24.982279 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440abc5-5abd-4a38-8316-0df8c419c845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:11:24.984233 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2440abc5-5abd-4a38-8316-0df8c419c845 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:11:24.992014 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4526/9022] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:11:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:11:34.988552 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:12:02.965065 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9b76e-5d1e-4d79-ae8b-7b627ee12902 service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:12:02.975270 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a9b76e-5d1e-4d79-ae8b-7b627ee12902 service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:12:02.979145 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537fc7d-96a8-4691-81f9-93977d5142be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:12:02.979793 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537fc7d-96a8-4691-81f9-93977d5142be service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:12:02.980077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537fc7d-96a8-4691-81f9-93977d5142be service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:12:02.980351 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537fc7d-96a8-4691-81f9-93977d5142be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:12:02.982308 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a537fc7d-96a8-4691-81f9-93977d5142be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:12:02.996653 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537fc7d-96a8-4691-81f9-93977d5142be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:12:02.998290 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537fc7d-96a8-4691-81f9-93977d5142be service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:12:03.019627 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4497/9023] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:12:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:12:04.214719 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440abc5-5abd-4a38-8316-0df8c419c845 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:12:04.224692 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2440abc5-5abd-4a38-8316-0df8c419c845 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:12:04.230708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffcb72-2280-4623-833e-81098a199d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:12:04.231324 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffcb72-2280-4623-833e-81098a199d17 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:12:04.231736 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffcb72-2280-4623-833e-81098a199d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:12:04.232027 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffcb72-2280-4623-833e-81098a199d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:12:04.234048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-73ffcb72-2280-4623-833e-81098a199d17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:12:04.248780 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffcb72-2280-4623-833e-81098a199d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:12:04.250266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffcb72-2280-4623-833e-81098a199d17 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:12:04.263375 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4527/9024] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:12:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:12:09.285598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0a272-0a15-483f-8dfa-d797511f3fc9 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:12:09.288172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0a272-0a15-483f-8dfa-d797511f3fc9 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:12:09.288172 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0a272-0a15-483f-8dfa-d797511f3fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:12:09.304327 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-48a0a272-0a15-483f-8dfa-d797511f3fc9 None None] MFA Rules not processed for user `31c2c3ecbc7148859a7c530deef8ab95`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:12:09.305528 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-48a0a272-0a15-483f-8dfa-d797511f3fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:12:09.313267 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4498/9025] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:12:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:12:14.256838 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:12:19.299533 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=13 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:12:22.173717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffcb72-2280-4623-833e-81098a199d17 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:12:22.183532 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffcb72-2280-4623-833e-81098a199d17 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:12:22.187912 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d2056-079e-4931-8563-da6a461c8c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:12:22.188686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d2056-079e-4931-8563-da6a461c8c14 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:12:22.189113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d2056-079e-4931-8563-da6a461c8c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:12:22.189658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d2056-079e-4931-8563-da6a461c8c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:12:22.191989 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-3f8d2056-079e-4931-8563-da6a461c8c14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:12:22.203647 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d2056-079e-4931-8563-da6a461c8c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:12:22.205332 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d2056-079e-4931-8563-da6a461c8c14 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:12:22.214718 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4528/9026] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:12:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:12:32.211447 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:14:34.164031 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0a272-0a15-483f-8dfa-d797511f3fc9 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:14:34.227070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0a272-0a15-483f-8dfa-d797511f3fc9 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:14:34.232215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67db47-67d3-4aed-b1ed-1ee96d9a7e2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:14:34.232943 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67db47-67d3-4aed-b1ed-1ee96d9a7e2d service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:14:34.233804 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67db47-67d3-4aed-b1ed-1ee96d9a7e2d service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:14:34.234131 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67db47-67d3-4aed-b1ed-1ee96d9a7e2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:14:34.242805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67db47-67d3-4aed-b1ed-1ee96d9a7e2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:14:34.246078 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67db47-67d3-4aed-b1ed-1ee96d9a7e2d service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:14:34.254359 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4499/9027] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:14:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:14:34.587509 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d2056-079e-4931-8563-da6a461c8c14 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:14:34.610892 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d2056-079e-4931-8563-da6a461c8c14 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:14:34.616049 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ad053-06f2-4d0d-8e62-e0896047d502 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:14:34.618127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ad053-06f2-4d0d-8e62-e0896047d502 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:14:34.618127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ad053-06f2-4d0d-8e62-e0896047d502 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:14:34.618127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ad053-06f2-4d0d-8e62-e0896047d502 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:14:34.626434 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ad053-06f2-4d0d-8e62-e0896047d502 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:14:34.628415 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ad053-06f2-4d0d-8e62-e0896047d502 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:14:34.638323 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4529/9028] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:14:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:14:44.250102 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=27 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:14:44.634160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=11 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:16:04.884671 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67db47-67d3-4aed-b1ed-1ee96d9a7e2d service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:04.890603 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-af67db47-67d3-4aed-b1ed-1ee96d9a7e2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:04.919441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67db47-67d3-4aed-b1ed-1ee96d9a7e2d service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:04.925008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e4eed-8a53-42fe-9ab3-1cc69a8262ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:04.925655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e4eed-8a53-42fe-9ab3-1cc69a8262ba service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:04.926046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e4eed-8a53-42fe-9ab3-1cc69a8262ba service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:04.926330 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e4eed-8a53-42fe-9ab3-1cc69a8262ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:04.934183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e4eed-8a53-42fe-9ab3-1cc69a8262ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:04.935773 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e4eed-8a53-42fe-9ab3-1cc69a8262ba service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:04.944997 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4500/9029] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:16:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:16:07.227146 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ad053-06f2-4d0d-8e62-e0896047d502 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:07.241263 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ad053-06f2-4d0d-8e62-e0896047d502 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:07.249266 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e474a6-a5ff-4c4a-b602-b144dc22d3f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:07.250405 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e474a6-a5ff-4c4a-b602-b144dc22d3f1 service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:07.250769 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e474a6-a5ff-4c4a-b602-b144dc22d3f1 service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:07.251153 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e474a6-a5ff-4c4a-b602-b144dc22d3f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:07.261608 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e474a6-a5ff-4c4a-b602-b144dc22d3f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:07.263467 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e474a6-a5ff-4c4a-b602-b144dc22d3f1 service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:07.276577 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4530/9030] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:16:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:16:11.496029 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e4eed-8a53-42fe-9ab3-1cc69a8262ba service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:11.505591 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e4eed-8a53-42fe-9ab3-1cc69a8262ba service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:11.509219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5726107-d739-474f-89bb-65ccf7e435cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:11.510447 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5726107-d739-474f-89bb-65ccf7e435cc service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:11.510742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5726107-d739-474f-89bb-65ccf7e435cc service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:11.511024 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5726107-d739-474f-89bb-65ccf7e435cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:11.571971 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5726107-d739-474f-89bb-65ccf7e435cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:11.573964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5726107-d739-474f-89bb-65ccf7e435cc service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:11.583542 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4501/9031] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:16:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:16:14.980455 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e474a6-a5ff-4c4a-b602-b144dc22d3f1 service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:14.992882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e474a6-a5ff-4c4a-b602-b144dc22d3f1 service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:14.998334 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc94964-af3c-4e24-bae3-2f0ada65884d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:14.999321 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc94964-af3c-4e24-bae3-2f0ada65884d service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:14.999611 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc94964-af3c-4e24-bae3-2f0ada65884d service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:14.999958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc94964-af3c-4e24-bae3-2f0ada65884d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:15.002102 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-6dc94964-af3c-4e24-bae3-2f0ada65884d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:15.023048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc94964-af3c-4e24-bae3-2f0ada65884d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:15.026271 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc94964-af3c-4e24-bae3-2f0ada65884d service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:15.037930 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4531/9032] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:16:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:16:18.843936 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5726107-d739-474f-89bb-65ccf7e435cc service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:18.853968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5726107-d739-474f-89bb-65ccf7e435cc service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:18.859279 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d56050-ef20-488a-9441-3c5c3dcb7417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:18.860356 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d56050-ef20-488a-9441-3c5c3dcb7417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:18.860807 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d56050-ef20-488a-9441-3c5c3dcb7417 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:18.861089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d56050-ef20-488a-9441-3c5c3dcb7417 admin admin] PATH_INFO: `/v3/users/31c2c3ecbc7148859a7c530deef8ab95` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:18.862334 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d56050-ef20-488a-9441-3c5c3dcb7417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c2c3ecbc7148859a7c530deef8ab95)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:18.864267 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d56050-ef20-488a-9441-3c5c3dcb7417 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:19.051550 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b9d56050-ef20-488a-9441-3c5c3dcb7417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c2c3ecbc7148859a7c530deef8ab95'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:19.052223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b9d56050-ef20-488a-9441-3c5c3dcb7417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c2c3ecbc7148859a7c530deef8ab95'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:19.061168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b9d56050-ef20-488a-9441-3c5c3dcb7417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c2c3ecbc7148859a7c530deef8ab95'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:19.128565 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b9d56050-ef20-488a-9441-3c5c3dcb7417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c2c3ecbc7148859a7c530deef8ab95'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:19.129115 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-b9d56050-ef20-488a-9441-3c5c3dcb7417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c2c3ecbc7148859a7c530deef8ab95'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:19.147579 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4502/9033] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 22:16:18 2022] DELETE /identity/v3/users/31c2c3ecbc7148859a7c530deef8ab95 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 22:16:19.390621 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc94964-af3c-4e24-bae3-2f0ada65884d service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:19.403641 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc94964-af3c-4e24-bae3-2f0ada65884d service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:19.463365 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91cfacd-d81f-45f2-b03e-b1997d79cf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:19.464732 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cfacd-d81f-45f2-b03e-b1997d79cf21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:19.465011 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cfacd-d81f-45f2-b03e-b1997d79cf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:19.465287 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91cfacd-d81f-45f2-b03e-b1997d79cf21 admin admin] PATH_INFO: `/v3/projects/4ef8feba0c8e45fdb82f09eeab276061` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:19.466484 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91cfacd-d81f-45f2-b03e-b1997d79cf21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ef8feba0c8e45fdb82f09eeab276061)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:19.468416 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91cfacd-d81f-45f2-b03e-b1997d79cf21 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:19.568142 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d91cfacd-d81f-45f2-b03e-b1997d79cf21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ef8feba0c8e45fdb82f09eeab276061', 'request_id': 'req-d91cfacd-d81f-45f2-b03e-b1997d79cf21'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:19.573827 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d91cfacd-d81f-45f2-b03e-b1997d79cf21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ef8feba0c8e45fdb82f09eeab276061', 'request_id': 'req-d91cfacd-d81f-45f2-b03e-b1997d79cf21'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:19.708830 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d91cfacd-d81f-45f2-b03e-b1997d79cf21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ef8feba0c8e45fdb82f09eeab276061', 'request_id': 'req-d91cfacd-d81f-45f2-b03e-b1997d79cf21'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:19.721105 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d91cfacd-d81f-45f2-b03e-b1997d79cf21 admin admin] The token cache is being invalidate because project 4ef8feba0c8e45fdb82f09eeab276061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 22:16:19.721426 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d91cfacd-d81f-45f2-b03e-b1997d79cf21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:19.722090 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-d91cfacd-d81f-45f2-b03e-b1997d79cf21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:19.723648 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4532/9034] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 22:16:19 2022] DELETE /identity/v3/projects/4ef8feba0c8e45fdb82f09eeab276061 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 22:16:19.727796 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d56050-ef20-488a-9441-3c5c3dcb7417 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:19.729258 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b9d56050-ef20-488a-9441-3c5c3dcb7417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:19.742888 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d56050-ef20-488a-9441-3c5c3dcb7417 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:19.778192 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:16:19.801229 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2947a8d-9741-48f1-bea1-6a37decfb173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:19.802202 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2947a8d-9741-48f1-bea1-6a37decfb173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:19.802494 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2947a8d-9741-48f1-bea1-6a37decfb173 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:19.802780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2947a8d-9741-48f1-bea1-6a37decfb173 admin admin] PATH_INFO: `/v3/users/b71d46b517584c3fa0531443612f1528` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:19.803996 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2947a8d-9741-48f1-bea1-6a37decfb173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b71d46b517584c3fa0531443612f1528)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:19.805982 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2947a8d-9741-48f1-bea1-6a37decfb173 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:19.993450 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e2947a8d-9741-48f1-bea1-6a37decfb173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b71d46b517584c3fa0531443612f1528'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:19.994378 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e2947a8d-9741-48f1-bea1-6a37decfb173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b71d46b517584c3fa0531443612f1528'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:20.002038 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e2947a8d-9741-48f1-bea1-6a37decfb173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b71d46b517584c3fa0531443612f1528'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:20.071395 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e2947a8d-9741-48f1-bea1-6a37decfb173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b71d46b517584c3fa0531443612f1528'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:20.072027 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-e2947a8d-9741-48f1-bea1-6a37decfb173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b71d46b517584c3fa0531443612f1528'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:20.091580 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4503/9035] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 22:16:19 2022] DELETE /identity/v3/users/b71d46b517584c3fa0531443612f1528 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 22:16:20.325252 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91cfacd-d81f-45f2-b03e-b1997d79cf21 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:20.326962 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-d91cfacd-d81f-45f2-b03e-b1997d79cf21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:20.341238 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91cfacd-d81f-45f2-b03e-b1997d79cf21 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:20.398298 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f03ead-438d-405d-befa-5ce903055e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:20.399164 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f03ead-438d-405d-befa-5ce903055e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:20.399448 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f03ead-438d-405d-befa-5ce903055e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:20.399725 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f03ead-438d-405d-befa-5ce903055e19 admin admin] PATH_INFO: `/v3/projects/44d5836dc89442619d209da7b7e1c3b2` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:20.400895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f03ead-438d-405d-befa-5ce903055e19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44d5836dc89442619d209da7b7e1c3b2)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:20.403463 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f03ead-438d-405d-befa-5ce903055e19 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:20.409653 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:16:20.533060 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-77f03ead-438d-405d-befa-5ce903055e19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44d5836dc89442619d209da7b7e1c3b2', 'request_id': 'req-77f03ead-438d-405d-befa-5ce903055e19'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:20.538672 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-77f03ead-438d-405d-befa-5ce903055e19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44d5836dc89442619d209da7b7e1c3b2', 'request_id': 'req-77f03ead-438d-405d-befa-5ce903055e19'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:20.602623 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-77f03ead-438d-405d-befa-5ce903055e19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44d5836dc89442619d209da7b7e1c3b2', 'request_id': 'req-77f03ead-438d-405d-befa-5ce903055e19'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:20.614391 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-77f03ead-438d-405d-befa-5ce903055e19 admin admin] The token cache is being invalidate because project 44d5836dc89442619d209da7b7e1c3b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 22:16:20.614685 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-77f03ead-438d-405d-befa-5ce903055e19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:20.615383 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-77f03ead-438d-405d-befa-5ce903055e19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:16:20.617132 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4533/9036] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 22:16:20 2022] DELETE /identity/v3/projects/44d5836dc89442619d209da7b7e1c3b2 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 28 22:16:20.622752 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf845b8-1e20-441d-91a7-ff5afdb54836 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:20.623045 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf845b8-1e20-441d-91a7-ff5afdb54836 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:20.623325 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf845b8-1e20-441d-91a7-ff5afdb54836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:20.641357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-adf845b8-1e20-441d-91a7-ff5afdb54836 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:16:20.642587 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-adf845b8-1e20-441d-91a7-ff5afdb54836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:20.703611 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4504/9037] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:16:20.711248 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d39f7-088a-4cd2-867c-4cc069083edb None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:20.711634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d39f7-088a-4cd2-867c-4cc069083edb None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:20.711925 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d39f7-088a-4cd2-867c-4cc069083edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:20.730385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-944d39f7-088a-4cd2-867c-4cc069083edb None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:16:20.732130 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-944d39f7-088a-4cd2-867c-4cc069083edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:20.739728 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4534/9038] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:16:20.743741 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf845b8-1e20-441d-91a7-ff5afdb54836 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:20.754160 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf845b8-1e20-441d-91a7-ff5afdb54836 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:20.758784 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cce75f-dce2-4e34-811f-e6b1345e76cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:20.759441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cce75f-dce2-4e34-811f-e6b1345e76cd admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:20.761989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cce75f-dce2-4e34-811f-e6b1345e76cd admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:20.761989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cce75f-dce2-4e34-811f-e6b1345e76cd admin admin] PATH_INFO: `/v3/domains` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:20.761989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cce75f-dce2-4e34-811f-e6b1345e76cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=103104) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 28 22:16:20.761989 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cce75f-dce2-4e34-811f-e6b1345e76cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:20.763219 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cce75f-dce2-4e34-811f-e6b1345e76cd admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:20.774866 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4505/9039] 10.222.0.33 () {58 vars in 1152 bytes} [Sat May 28 22:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 28 22:16:20.778884 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d39f7-088a-4cd2-867c-4cc069083edb None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:20.790652 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d39f7-088a-4cd2-867c-4cc069083edb None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:20.793040 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77230f3-efff-4db9-958c-8bc212e4b96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:20.793688 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77230f3-efff-4db9-958c-8bc212e4b96e admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:20.794006 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77230f3-efff-4db9-958c-8bc212e4b96e admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:20.794285 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77230f3-efff-4db9-958c-8bc212e4b96e admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:20.794979 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77230f3-efff-4db9-958c-8bc212e4b96e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:20.796958 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77230f3-efff-4db9-958c-8bc212e4b96e admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:20.913320 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4535/9040] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 22:16:20 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 22:16:20.917484 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cce75f-dce2-4e34-811f-e6b1345e76cd admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:20.928090 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cce75f-dce2-4e34-811f-e6b1345e76cd admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:20.970630 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:16:20.986461 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ea4c0-75f2-4937-8743-b1a3c5a100e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:20.987165 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ea4c0-75f2-4937-8743-b1a3c5a100e3 admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:20.987452 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ea4c0-75f2-4937-8743-b1a3c5a100e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:20.987734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ea4c0-75f2-4937-8743-b1a3c5a100e3 admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:20.988433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ea4c0-75f2-4937-8743-b1a3c5a100e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:20.990498 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ea4c0-75f2-4937-8743-b1a3c5a100e3 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:21.078156 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4506/9041] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 22:16:20 2022] POST /identity/v3/users => generated 288 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 22:16:21.087177 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77230f3-efff-4db9-958c-8bc212e4b96e admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:21.092523 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77230f3-efff-4db9-958c-8bc212e4b96e admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:21.097250 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8da9a-8e1f-4432-aabb-af0bb343daf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:21.097934 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8da9a-8e1f-4432-aabb-af0bb343daf2 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:21.098220 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8da9a-8e1f-4432-aabb-af0bb343daf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:21.098610 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8da9a-8e1f-4432-aabb-af0bb343daf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:21.099351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8da9a-8e1f-4432-aabb-af0bb343daf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:21.100954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8da9a-8e1f-4432-aabb-af0bb343daf2 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:21.111265 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4536/9042] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 22:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 22:16:21.118506 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ea4c0-75f2-4937-8743-b1a3c5a100e3 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:21.129183 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ea4c0-75f2-4937-8743-b1a3c5a100e3 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:21.134136 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339013c0-0572-49b7-b19b-4a912ebe19ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:21.134949 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339013c0-0572-49b7-b19b-4a912ebe19ad admin admin] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:21.135579 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339013c0-0572-49b7-b19b-4a912ebe19ad admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:21.135862 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339013c0-0572-49b7-b19b-4a912ebe19ad admin admin] PATH_INFO: `/v3/roles` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:21.136595 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339013c0-0572-49b7-b19b-4a912ebe19ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:21.138250 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339013c0-0572-49b7-b19b-4a912ebe19ad admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:21.148119 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4507/9043] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 22:16:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 22:16:21.157678 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8da9a-8e1f-4432-aabb-af0bb343daf2 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:21.167708 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8da9a-8e1f-4432-aabb-af0bb343daf2 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:21.172893 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105efd30-38be-4035-b963-521ecffebbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:21.173772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105efd30-38be-4035-b963-521ecffebbd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:21.174088 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105efd30-38be-4035-b963-521ecffebbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:21.174385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105efd30-38be-4035-b963-521ecffebbd9 admin admin] PATH_INFO: `/v3/projects/c9313da576ba44f4a4fdb41a1306a1c2/users/ef1a345c9a0b4a3287ac4110fcf90e50/roles/c0c09ba97c694896ae14fceb6982b1a5` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:21.189932 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105efd30-38be-4035-b963-521ecffebbd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9313da576ba44f4a4fdb41a1306a1c2, user_id=ef1a345c9a0b4a3287ac4110fcf90e50, role_id=c0c09ba97c694896ae14fceb6982b1a5)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:21.192658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105efd30-38be-4035-b963-521ecffebbd9 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:21.282449 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4537/9044] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 22:16:21 2022] PUT /identity/v3/projects/c9313da576ba44f4a4fdb41a1306a1c2/users/ef1a345c9a0b4a3287ac4110fcf90e50/roles/c0c09ba97c694896ae14fceb6982b1a5 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 22:16:21.287429 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a9014-b40c-425f-9106-7f8fc9066f46 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:21.287722 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a9014-b40c-425f-9106-7f8fc9066f46 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:21.288006 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a9014-b40c-425f-9106-7f8fc9066f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:21.306139 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-483a9014-b40c-425f-9106-7f8fc9066f46 None None] MFA Rules not processed for user `3e371762e9974f2484669486a37339c7`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:16:21.307357 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-483a9014-b40c-425f-9106-7f8fc9066f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:21.365901 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4508/9045] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:16:21.376048 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105efd30-38be-4035-b963-521ecffebbd9 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:21.377802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-105efd30-38be-4035-b963-521ecffebbd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:21.446032 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105efd30-38be-4035-b963-521ecffebbd9 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:21.449711 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23a010-ee86-42bd-83ee-9c49f22d0972 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:21.450388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23a010-ee86-42bd-83ee-9c49f22d0972 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:21.450676 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23a010-ee86-42bd-83ee-9c49f22d0972 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:21.450955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23a010-ee86-42bd-83ee-9c49f22d0972 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:21.459998 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c23a010-ee86-42bd-83ee-9c49f22d0972 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:21.461429 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c23a010-ee86-42bd-83ee-9c49f22d0972 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:21.471332 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4538/9046] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:16:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:16:26.847517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2b0d6-30ca-40bb-b8f8-0dca55e41c2c None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:26.848034 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2b0d6-30ca-40bb-b8f8-0dca55e41c2c None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:26.848128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2b0d6-30ca-40bb-b8f8-0dca55e41c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:26.878371 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-d9b2b0d6-30ca-40bb-b8f8-0dca55e41c2c None None] MFA Rules not processed for user `ef1a345c9a0b4a3287ac4110fcf90e50`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:16:26.879751 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d9b2b0d6-30ca-40bb-b8f8-0dca55e41c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:26.954321 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4509/9047] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:16:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:16:26.958941 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23a010-ee86-42bd-83ee-9c49f22d0972 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:26.969789 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23a010-ee86-42bd-83ee-9c49f22d0972 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:26.974681 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d95c80-bfd6-47bc-8474-201b3adb99fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:26.975585 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d95c80-bfd6-47bc-8474-201b3adb99fc admin admin] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:26.976052 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d95c80-bfd6-47bc-8474-201b3adb99fc admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:26.976521 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d95c80-bfd6-47bc-8474-201b3adb99fc admin admin] PATH_INFO: `/v3/projects` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:26.977244 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d95c80-bfd6-47bc-8474-201b3adb99fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:26.979226 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d95c80-bfd6-47bc-8474-201b3adb99fc admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:26.990734 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:16:27.091277 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4539/9048] 10.222.0.33 () {60 vars in 1152 bytes} [Sat May 28 22:16:26 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 22:16:27.095425 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2b0d6-30ca-40bb-b8f8-0dca55e41c2c None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:27.108002 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2b0d6-30ca-40bb-b8f8-0dca55e41c2c None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:27.162009 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:16:27.172189 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928302b-a461-402c-8383-d14d6c035cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:27.172938 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928302b-a461-402c-8383-d14d6c035cdb admin admin] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:27.173158 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928302b-a461-402c-8383-d14d6c035cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:27.173438 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2928302b-a461-402c-8383-d14d6c035cdb admin admin] PATH_INFO: `/v3/users` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:27.174168 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928302b-a461-402c-8383-d14d6c035cdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:27.176272 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2928302b-a461-402c-8383-d14d6c035cdb admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:27.277565 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4510/9049] 10.222.0.33 () {60 vars in 1143 bytes} [Sat May 28 22:16:27 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 28 22:16:27.292666 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d95c80-bfd6-47bc-8474-201b3adb99fc admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:27.303109 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d95c80-bfd6-47bc-8474-201b3adb99fc admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:27.307696 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be59e6f-af06-4eca-afab-7ec399b018f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:27.308358 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be59e6f-af06-4eca-afab-7ec399b018f0 admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:27.308648 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be59e6f-af06-4eca-afab-7ec399b018f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:27.308927 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be59e6f-af06-4eca-afab-7ec399b018f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:27.309655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be59e6f-af06-4eca-afab-7ec399b018f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:27.311315 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be59e6f-af06-4eca-afab-7ec399b018f0 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:27.321485 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4540/9050] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 22:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 22:16:27.326144 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928302b-a461-402c-8383-d14d6c035cdb admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:27.336456 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2928302b-a461-402c-8383-d14d6c035cdb admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:27.341953 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143df2d-45b7-4f24-ac07-910b1b778b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:27.342837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143df2d-45b7-4f24-ac07-910b1b778b8c admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:27.343128 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143df2d-45b7-4f24-ac07-910b1b778b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:27.343414 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143df2d-45b7-4f24-ac07-910b1b778b8c admin admin] PATH_INFO: `/v3/projects/e4f22ffb71b747c2b9209b18128a1042/users/ef7e26f10e184cb5b47306c1e1fbb82a/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:27.358275 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143df2d-45b7-4f24-ac07-910b1b778b8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4f22ffb71b747c2b9209b18128a1042, user_id=ef7e26f10e184cb5b47306c1e1fbb82a, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:27.361019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143df2d-45b7-4f24-ac07-910b1b778b8c admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:27.429733 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4511/9051] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 22:16:27 2022] PUT /identity/v3/projects/e4f22ffb71b747c2b9209b18128a1042/users/ef7e26f10e184cb5b47306c1e1fbb82a/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 22:16:27.433682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be59e6f-af06-4eca-afab-7ec399b018f0 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:27.444755 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be59e6f-af06-4eca-afab-7ec399b018f0 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:27.501446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10bfcf3-e1ce-4c0a-bb4a-aae7f39121dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:27.502154 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bfcf3-e1ce-4c0a-bb4a-aae7f39121dd admin admin] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:27.502442 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bfcf3-e1ce-4c0a-bb4a-aae7f39121dd admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:27.502723 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10bfcf3-e1ce-4c0a-bb4a-aae7f39121dd admin admin] PATH_INFO: `/v3/roles` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:27.503452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10bfcf3-e1ce-4c0a-bb4a-aae7f39121dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:27.505039 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10bfcf3-e1ce-4c0a-bb4a-aae7f39121dd admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:27.515297 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4541/9052] 10.222.0.33 () {58 vars in 1121 bytes} [Sat May 28 22:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 28 22:16:27.519451 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143df2d-45b7-4f24-ac07-910b1b778b8c admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:27.530205 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143df2d-45b7-4f24-ac07-910b1b778b8c admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:27.534918 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3273ac-8736-41ac-a05c-610af2994cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:27.535802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3273ac-8736-41ac-a05c-610af2994cd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:27.536089 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3273ac-8736-41ac-a05c-610af2994cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:27.536373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3273ac-8736-41ac-a05c-610af2994cd6 admin admin] PATH_INFO: `/v3/projects/e4f22ffb71b747c2b9209b18128a1042/users/ef7e26f10e184cb5b47306c1e1fbb82a/roles/7805b93b32f1472f8365936d4d57d49a` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:27.538415 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3273ac-8736-41ac-a05c-610af2994cd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4f22ffb71b747c2b9209b18128a1042, user_id=ef7e26f10e184cb5b47306c1e1fbb82a, role_id=7805b93b32f1472f8365936d4d57d49a)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:27.541097 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3273ac-8736-41ac-a05c-610af2994cd6 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:27.551451 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4512/9053] 10.222.0.33 () {60 vars in 1482 bytes} [Sat May 28 22:16:27 2022] PUT /identity/v3/projects/e4f22ffb71b747c2b9209b18128a1042/users/ef7e26f10e184cb5b47306c1e1fbb82a/roles/7805b93b32f1472f8365936d4d57d49a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 22:16:33.294476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2f976-dbc2-4efb-b7f3-4fba322713d0 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:33.295019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2f976-dbc2-4efb-b7f3-4fba322713d0 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:33.295019 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2f976-dbc2-4efb-b7f3-4fba322713d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:33.323812 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-46b2f976-dbc2-4efb-b7f3-4fba322713d0 None None] MFA Rules not processed for user `ef7e26f10e184cb5b47306c1e1fbb82a`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:16:33.325597 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-46b2f976-dbc2-4efb-b7f3-4fba322713d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:33.397756 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4542/9054] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:16:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:16:33.413567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3273ac-8736-41ac-a05c-610af2994cd6 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:33.415433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-ac3273ac-8736-41ac-a05c-610af2994cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:33.494130 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3273ac-8736-41ac-a05c-610af2994cd6 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:33.500314 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7727ff1-d056-4d01-9939-0b5357b39470 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:33.500978 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7727ff1-d056-4d01-9939-0b5357b39470 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:33.501269 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7727ff1-d056-4d01-9939-0b5357b39470 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:33.501551 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7727ff1-d056-4d01-9939-0b5357b39470 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:33.564223 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7727ff1-d056-4d01-9939-0b5357b39470 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:33.565860 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7727ff1-d056-4d01-9939-0b5357b39470 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:33.576283 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4513/9055] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:16:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:16:36.130346 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f35df-8e36-42af-9dcb-d65bc4d2c0b9 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:36.131199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f35df-8e36-42af-9dcb-d65bc4d2c0b9 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:36.131524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f35df-8e36-42af-9dcb-d65bc4d2c0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:36.150297 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-cb5f35df-8e36-42af-9dcb-d65bc4d2c0b9 None None] MFA Rules not processed for user `ef1a345c9a0b4a3287ac4110fcf90e50`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:16:36.151547 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cb5f35df-8e36-42af-9dcb-d65bc4d2c0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:36.161020 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4543/9056] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:16:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:16:36.497712 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7727ff1-d056-4d01-9939-0b5357b39470 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:36.577598 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:16:36.589407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7727ff1-d056-4d01-9939-0b5357b39470 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:36.595228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c936ad-95f4-49e7-a8a5-e99b0073b48d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:36.596742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c936ad-95f4-49e7-a8a5-e99b0073b48d service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:36.596883 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c936ad-95f4-49e7-a8a5-e99b0073b48d service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:36.597076 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c936ad-95f4-49e7-a8a5-e99b0073b48d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:36.599133 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-65c936ad-95f4-49e7-a8a5-e99b0073b48d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:36.614070 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c936ad-95f4-49e7-a8a5-e99b0073b48d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:36.615584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c936ad-95f4-49e7-a8a5-e99b0073b48d service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:36.626091 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4514/9057] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:16:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:16:37.125305 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f35df-8e36-42af-9dcb-d65bc4d2c0b9 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:37.127245 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-cb5f35df-8e36-42af-9dcb-d65bc4d2c0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:37.246312 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:16:37.276942 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f35df-8e36-42af-9dcb-d65bc4d2c0b9 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:37.284752 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0eddcc-8c00-4a45-b0ac-02bd3bbbc4a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:37.286086 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0eddcc-8c00-4a45-b0ac-02bd3bbbc4a6 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:37.286584 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0eddcc-8c00-4a45-b0ac-02bd3bbbc4a6 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:37.287070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0eddcc-8c00-4a45-b0ac-02bd3bbbc4a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:37.290214 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fe0eddcc-8c00-4a45-b0ac-02bd3bbbc4a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:37.305785 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0eddcc-8c00-4a45-b0ac-02bd3bbbc4a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:37.308113 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0eddcc-8c00-4a45-b0ac-02bd3bbbc4a6 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:37.322707 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4544/9058] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:16:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:16:40.299071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c936ad-95f4-49e7-a8a5-e99b0073b48d service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:40.300489 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-65c936ad-95f4-49e7-a8a5-e99b0073b48d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:40.310884 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c936ad-95f4-49e7-a8a5-e99b0073b48d service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:40.314826 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433fa6d-0e17-4b68-a480-3716ec9eac08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:40.315459 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433fa6d-0e17-4b68-a480-3716ec9eac08 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:40.315742 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433fa6d-0e17-4b68-a480-3716ec9eac08 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:40.316019 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433fa6d-0e17-4b68-a480-3716ec9eac08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:40.317731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-d433fa6d-0e17-4b68-a480-3716ec9eac08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:40.327049 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d433fa6d-0e17-4b68-a480-3716ec9eac08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:40.328458 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d433fa6d-0e17-4b68-a480-3716ec9eac08 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:40.336161 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4515/9059] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:16:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:16:47.368291 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=7 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:16:47.884127 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0eddcc-8c00-4a45-b0ac-02bd3bbbc4a6 service placement] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:47.885667 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-fe0eddcc-8c00-4a45-b0ac-02bd3bbbc4a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:47.901820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0eddcc-8c00-4a45-b0ac-02bd3bbbc4a6 service placement] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:47.908539 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913a726-0bd3-4813-bc69-e1df0f7a18ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:47.909190 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913a726-0bd3-4813-bc69-e1df0f7a18ae service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:47.909476 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913a726-0bd3-4813-bc69-e1df0f7a18ae service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:47.909954 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913a726-0bd3-4813-bc69-e1df0f7a18ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:47.911686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e913a726-0bd3-4813-bc69-e1df0f7a18ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:16:47.924960 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913a726-0bd3-4813-bc69-e1df0f7a18ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:47.926588 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e913a726-0bd3-4813-bc69-e1df0f7a18ae service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:47.937271 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4545/9060] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:16:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:16:50.105794 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433fa6d-0e17-4b68-a480-3716ec9eac08 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:16:50.114809 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433fa6d-0e17-4b68-a480-3716ec9eac08 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:16:50.118670 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116f8b0-4978-484d-a8ec-883f8eca2d56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:16:50.119320 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116f8b0-4978-484d-a8ec-883f8eca2d56 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:16:50.119608 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116f8b0-4978-484d-a8ec-883f8eca2d56 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:16:50.119893 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116f8b0-4978-484d-a8ec-883f8eca2d56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:16:50.129693 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116f8b0-4978-484d-a8ec-883f8eca2d56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:16:50.131182 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116f8b0-4978-484d-a8ec-883f8eca2d56 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:16:50.141588 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4516/9061] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:16:50 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:16:57.932697 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:17:00.136571 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=19 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:18:47.618108 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913a726-0bd3-4813-bc69-e1df0f7a18ae service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:18:47.619576 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-e913a726-0bd3-4813-bc69-e1df0f7a18ae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:18:47.676197 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913a726-0bd3-4813-bc69-e1df0f7a18ae service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:18:47.679926 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec057ae-7d00-4140-aba0-61dc51fb6fd8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '224c1a0427c942e5bfef2bac75648d80', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:18:47.684452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec057ae-7d00-4140-aba0-61dc51fb6fd8 service glance] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:18:47.684452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec057ae-7d00-4140-aba0-61dc51fb6fd8 service glance] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:18:47.684452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec057ae-7d00-4140-aba0-61dc51fb6fd8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:18:47.684452 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8ec057ae-7d00-4140-aba0-61dc51fb6fd8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:18:47.709284 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec057ae-7d00-4140-aba0-61dc51fb6fd8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:18:47.710820 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec057ae-7d00-4140-aba0-61dc51fb6fd8 service glance] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:18:47.722387 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4546/9062] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:18:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:18:52.247116 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88257e87-df57-4700-b409-c51855a214da None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:18:52.247614 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88257e87-df57-4700-b409-c51855a214da None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:18:52.247715 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88257e87-df57-4700-b409-c51855a214da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:18:52.265339 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-88257e87-df57-4700-b409-c51855a214da None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:18:52.266590 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-88257e87-df57-4700-b409-c51855a214da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:18:52.324075 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4517/9063] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 22:18:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:18:52.452658 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec057ae-7d00-4140-aba0-61dc51fb6fd8 service glance] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:18:52.454121 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8ec057ae-7d00-4140-aba0-61dc51fb6fd8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:18:52.530303 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec057ae-7d00-4140-aba0-61dc51fb6fd8 service glance] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:18:52.536064 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5079b-6a11-4f64-be06-80610178e52e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:18:52.536721 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5079b-6a11-4f64-be06-80610178e52e service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:18:52.537008 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5079b-6a11-4f64-be06-80610178e52e service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:18:52.537286 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5079b-6a11-4f64-be06-80610178e52e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:18:52.546717 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5079b-6a11-4f64-be06-80610178e52e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:18:52.548341 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa5079b-6a11-4f64-be06-80610178e52e service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:18:52.556614 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4547/9064] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 22:18:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 22:19:02.317312 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=25 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:19:02.553280 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=63 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:19:02.798080 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4254423-6ff3-45ac-9a7e-94f3e529c438 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:19:02.798372 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4254423-6ff3-45ac-9a7e-94f3e529c438 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:19:02.798655 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4254423-6ff3-45ac-9a7e-94f3e529c438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:19:02.817472 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-a4254423-6ff3-45ac-9a7e-94f3e529c438 None None] MFA Rules not processed for user `ef1a345c9a0b4a3287ac4110fcf90e50`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:19:02.824193 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-a4254423-6ff3-45ac-9a7e-94f3e529c438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:19:02.826284 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4518/9065] 10.222.0.33 () {58 vars in 957 bytes} [Sat May 28 22:19:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:19:03.433854 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5079b-6a11-4f64-be06-80610178e52e service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:19:03.443198 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa5079b-6a11-4f64-be06-80610178e52e service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:19:03.446773 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4d5a3-209f-403b-98e3-0292a3b3f150 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:19:03.447433 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4d5a3-209f-403b-98e3-0292a3b3f150 service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:19:03.447716 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4d5a3-209f-403b-98e3-0292a3b3f150 service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:19:03.447994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f4d5a3-209f-403b-98e3-0292a3b3f150 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:19:03.449740 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-17f4d5a3-209f-403b-98e3-0292a3b3f150 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:19:03.459026 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4d5a3-209f-403b-98e3-0292a3b3f150 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:19:03.460446 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f4d5a3-209f-403b-98e3-0292a3b3f150 service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:19:03.468666 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4548/9066] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:19:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:19:03.872394 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4254423-6ff3-45ac-9a7e-94f3e529c438 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:19:03.884886 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4254423-6ff3-45ac-9a7e-94f3e529c438 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:19:03.890442 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc81927-261a-470e-af47-88331f78b934 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:19:03.891095 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc81927-261a-470e-af47-88331f78b934 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:19:03.892865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc81927-261a-470e-af47-88331f78b934 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:19:03.892865 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc81927-261a-470e-af47-88331f78b934 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:19:03.893515 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-6cc81927-261a-470e-af47-88331f78b934 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:19:03.896282 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4d5a3-209f-403b-98e3-0292a3b3f150 service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:19:03.907199 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f4d5a3-209f-403b-98e3-0292a3b3f150 service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:19:03.912857 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c4eed-40d7-4f9e-81ba-3b64f50d248c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:19:03.920417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c4eed-40d7-4f9e-81ba-3b64f50d248c service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:19:03.920417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c4eed-40d7-4f9e-81ba-3b64f50d248c service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:19:03.920417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c4eed-40d7-4f9e-81ba-3b64f50d248c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:19:03.920417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-028c4eed-40d7-4f9e-81ba-3b64f50d248c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:19:03.923416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc81927-261a-470e-af47-88331f78b934 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:19:03.925036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc81927-261a-470e-af47-88331f78b934 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:19:03.931037 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c4eed-40d7-4f9e-81ba-3b64f50d248c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:19:03.933144 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c4eed-40d7-4f9e-81ba-3b64f50d248c service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:19:03.936227 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4519/9067] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:19:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:19:03.946667 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4549/9068] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:19:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 28 22:19:06.231416 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a584474-7faa-4003-b4e4-93f703a1af45 None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:19:06.231879 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a584474-7faa-4003-b4e4-93f703a1af45 None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:19:06.231967 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a584474-7faa-4003-b4e4-93f703a1af45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:19:06.249780 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-2a584474-7faa-4003-b4e4-93f703a1af45 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:19:06.251071 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-2a584474-7faa-4003-b4e4-93f703a1af45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:19:06.258986 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4520/9069] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 22:19:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:19:06.271816 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c4eed-40d7-4f9e-81ba-3b64f50d248c service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:19:06.284187 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c4eed-40d7-4f9e-81ba-3b64f50d248c service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:19:06.289684 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b69b29-13cc-4d01-8b78-f7ce1ec1b79c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:19:06.291524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b69b29-13cc-4d01-8b78-f7ce1ec1b79c service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:19:06.291524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b69b29-13cc-4d01-8b78-f7ce1ec1b79c service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:19:06.291524 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b69b29-13cc-4d01-8b78-f7ce1ec1b79c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:19:06.299938 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b69b29-13cc-4d01-8b78-f7ce1ec1b79c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:19:06.301775 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b69b29-13cc-4d01-8b78-f7ce1ec1b79c service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:19:06.311475 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4550/9070] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 22:19:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 22:19:15.176904 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a584474-7faa-4003-b4e4-93f703a1af45 None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:19:15.187233 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a584474-7faa-4003-b4e4-93f703a1af45 None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:19:15.190837 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0a3df-4204-4ff5-8773-e539f6467aa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:19:15.191508 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0a3df-4204-4ff5-8773-e539f6467aa0 service neutron] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:19:15.191847 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0a3df-4204-4ff5-8773-e539f6467aa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:19:15.192119 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b0a3df-4204-4ff5-8773-e539f6467aa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:19:15.193954 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-54b0a3df-4204-4ff5-8773-e539f6467aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:19:15.203497 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0a3df-4204-4ff5-8773-e539f6467aa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:19:15.204941 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b0a3df-4204-4ff5-8773-e539f6467aa0 service neutron] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:19:15.212934 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4521/9071] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:19:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:19:16.307191 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=18 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:19:25.209400 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=23 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:19:38.173963 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b69b29-13cc-4d01-8b78-f7ce1ec1b79c service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:19:38.186098 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b69b29-13cc-4d01-8b78-f7ce1ec1b79c service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:19:38.191236 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb08887-6098-460d-9327-d8bfb9ce5b69 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:19:38.191880 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb08887-6098-460d-9327-d8bfb9ce5b69 service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:19:38.192170 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb08887-6098-460d-9327-d8bfb9ce5b69 service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:19:38.192456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb08887-6098-460d-9327-d8bfb9ce5b69 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:19:38.194302 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dfb08887-6098-460d-9327-d8bfb9ce5b69 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:19:38.206488 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb08887-6098-460d-9327-d8bfb9ce5b69 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:19:38.208137 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb08887-6098-460d-9327-d8bfb9ce5b69 service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:19:38.219328 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4551/9072] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:19:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:19:41.555175 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0a3df-4204-4ff5-8773-e539f6467aa0 service neutron] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:19:41.556615 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-54b0a3df-4204-4ff5-8773-e539f6467aa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:19:41.572368 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b0a3df-4204-4ff5-8773-e539f6467aa0 service neutron] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:19:41.577535 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2ef98-6976-4a42-bb18-657c283a47cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:19:41.578215 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2ef98-6976-4a42-bb18-657c283a47cb service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:19:41.578504 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2ef98-6976-4a42-bb18-657c283a47cb service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:19:41.578786 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2ef98-6976-4a42-bb18-657c283a47cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:19:41.580620 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-8ef2ef98-6976-4a42-bb18-657c283a47cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:19:41.592098 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2ef98-6976-4a42-bb18-657c283a47cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:19:41.593695 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef2ef98-6976-4a42-bb18-657c283a47cb service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:19:41.603705 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4522/9073] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:19:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:19:48.221387 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=6 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:19:51.599842 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=6 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:20:57.197530 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48647107-f3de-474f-bfb7-e5b5f84b9d1a None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:20:57.198073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48647107-f3de-474f-bfb7-e5b5f84b9d1a None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:20:57.198182 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48647107-f3de-474f-bfb7-e5b5f84b9d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:20:57.257527 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-48647107-f3de-474f-bfb7-e5b5f84b9d1a None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:20:57.258802 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-48647107-f3de-474f-bfb7-e5b5f84b9d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:20:57.276160 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4552/9074] 10.222.0.33 () {58 vars in 1009 bytes} [Sat May 28 22:20:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:20:57.316397 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2ef98-6976-4a42-bb18-657c283a47cb service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:20:57.337765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2ef98-6976-4a42-bb18-657c283a47cb service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:20:57.343247 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e2cb4-a29a-4508-83da-c81398160444 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:20:57.343968 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e2cb4-a29a-4508-83da-c81398160444 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:20:57.344263 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e2cb4-a29a-4508-83da-c81398160444 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:20:57.344547 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9e2cb4-a29a-4508-83da-c81398160444 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:20:57.353942 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e2cb4-a29a-4508-83da-c81398160444 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:20:57.355543 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9e2cb4-a29a-4508-83da-c81398160444 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:20:57.363722 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4523/9075] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 22:20:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 22:21:07.273092 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=22 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:21:07.360338 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=10 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:21:08.107798 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a833fe4-10c0-407b-85b0-f2d6bbd63156 None None] REQUEST_METHOD: `POST` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:21:08.108096 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a833fe4-10c0-407b-85b0-f2d6bbd63156 None None] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:21:08.108385 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a833fe4-10c0-407b-85b0-f2d6bbd63156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:21:08.127417 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.auth.core [None req-1a833fe4-10c0-407b-85b0-f2d6bbd63156 None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:21:08.128634 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-1a833fe4-10c0-407b-85b0-f2d6bbd63156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:21:08.135956 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4553/9076] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 22:21:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:21:08.147829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e2cb4-a29a-4508-83da-c81398160444 service swift] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:21:08.158413 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9e2cb4-a29a-4508-83da-c81398160444 service swift] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:21:08.164011 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f02334b-f8c6-4ea8-9aa6-8edfe8750a53 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:21:08.164679 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02334b-f8c6-4ea8-9aa6-8edfe8750a53 service swift] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:21:08.164970 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02334b-f8c6-4ea8-9aa6-8edfe8750a53 service swift] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:21:08.165252 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f02334b-f8c6-4ea8-9aa6-8edfe8750a53 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:21:08.173360 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02334b-f8c6-4ea8-9aa6-8edfe8750a53 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:21:08.175005 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f02334b-f8c6-4ea8-9aa6-8edfe8750a53 service swift] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:21:08.184020 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4524/9077] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 22:21:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 28 22:21:09.943323 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a833fe4-10c0-407b-85b0-f2d6bbd63156 None None] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:21:09.957638 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a833fe4-10c0-407b-85b0-f2d6bbd63156 None None] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:21:09.964216 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eeda15-a5f6-49b5-a8ce-50a6937323ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:21:09.964875 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeda15-a5f6-49b5-a8ce-50a6937323ab service nova] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:21:09.965160 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeda15-a5f6-49b5-a8ce-50a6937323ab service nova] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:21:09.965439 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeda15-a5f6-49b5-a8ce-50a6937323ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:21:09.971655 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-82eeda15-a5f6-49b5-a8ce-50a6937323ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:21:09.986858 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eeda15-a5f6-49b5-a8ce-50a6937323ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:21:09.988765 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eeda15-a5f6-49b5-a8ce-50a6937323ab service nova] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:21:10.001532 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4554/9078] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:21:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:21:13.810584 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6960c3d-afcb-416b-b782-466bc747f08a None None] REQUEST_METHOD: `POST` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:21:13.811077 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6960c3d-afcb-416b-b782-466bc747f08a None None] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:21:13.811195 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6960c3d-afcb-416b-b782-466bc747f08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:21:13.829502 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.auth.core [None req-b6960c3d-afcb-416b-b782-466bc747f08a None None] MFA Rules not processed for user `c1c1ea9ba4c740b3b7ed67c8335bfad2`. Rule list: `[]` (Enabled: `True`). {{(pid=103104) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 28 22:21:13.830731 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-b6960c3d-afcb-416b-b782-466bc747f08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:21:13.837938 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4525/9079] 10.222.0.33 () {58 vars in 967 bytes} [Sat May 28 22:21:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 28 22:21:13.851574 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eeda15-a5f6-49b5-a8ce-50a6937323ab service nova] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:21:13.863054 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eeda15-a5f6-49b5-a8ce-50a6937323ab service nova] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:21:13.868743 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b3b54-0feb-4d80-ac08-1eb7aabcf9d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a01bba888a644eb18fc457aff578aa4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:21:13.869435 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b3b54-0feb-4d80-ac08-1eb7aabcf9d4 service swift] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:21:13.869703 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b3b54-0feb-4d80-ac08-1eb7aabcf9d4 service swift] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:21:13.870029 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b3b54-0feb-4d80-ac08-1eb7aabcf9d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:21:13.878124 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b3b54-0feb-4d80-ac08-1eb7aabcf9d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:21:13.879700 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b3b54-0feb-4d80-ac08-1eb7aabcf9d4 service swift] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:21:13.888528 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4555/9080] 10.222.0.33 () {60 vars in 1384 bytes} [Sat May 28 22:21:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 28 22:21:18.341871 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6960c3d-afcb-416b-b782-466bc747f08a None None] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:21:18.355310 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6960c3d-afcb-416b-b782-466bc747f08a None None] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:21:18.360765 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6a435-6a3c-4dfe-942b-a41ae7a1e882 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cfdebb5e6b8d4e2d86f0a03a30c21671', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'ResellerAdmin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:21:18.361449 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6a435-6a3c-4dfe-942b-a41ae7a1e882 service nova] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:21:18.361734 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6a435-6a3c-4dfe-942b-a41ae7a1e882 service nova] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:21:18.362046 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6a435-6a3c-4dfe-942b-a41ae7a1e882 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:21:18.367771 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-88d6a435-6a3c-4dfe-942b-a41ae7a1e882 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:21:18.375685 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6a435-6a3c-4dfe-942b-a41ae7a1e882 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:21:18.377283 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6a435-6a3c-4dfe-942b-a41ae7a1e882 service nova] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:21:18.387842 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4526/9081] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:21:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:21:21.790862 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b3b54-0feb-4d80-ac08-1eb7aabcf9d4 service swift] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:21:21.810882 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b3b54-0feb-4d80-ac08-1eb7aabcf9d4 service swift] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:21:21.814753 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a9afb-4381-4b09-ab0c-ed43898e683a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1b2a9ebcd3f84b758322a80c6e590de0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:21:21.815501 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a9afb-4381-4b09-ab0c-ed43898e683a service neutron] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:21:21.815861 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a9afb-4381-4b09-ab0c-ed43898e683a service neutron] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:21:21.816210 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a9afb-4381-4b09-ab0c-ed43898e683a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:21:21.891644 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a9afb-4381-4b09-ab0c-ed43898e683a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:21:21.893106 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a9afb-4381-4b09-ab0c-ed43898e683a service neutron] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:21:21.912418 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4556/9082] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:21:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:21:23.727486 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6a435-6a3c-4dfe-942b-a41ae7a1e882 service nova] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:21:23.739228 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6a435-6a3c-4dfe-942b-a41ae7a1e882 service nova] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:21:23.744546 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7045f-3d58-4ac6-ba8c-a12670e9eab7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:21:23.745204 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7045f-3d58-4ac6-ba8c-a12670e9eab7 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:21:23.745493 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7045f-3d58-4ac6-ba8c-a12670e9eab7 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:21:23.745805 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7045f-3d58-4ac6-ba8c-a12670e9eab7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:21:23.747976 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-e5d7045f-3d58-4ac6-ba8c-a12670e9eab7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:21:23.760036 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7045f-3d58-4ac6-ba8c-a12670e9eab7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:21:23.761629 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7045f-3d58-4ac6-ba8c-a12670e9eab7 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:21:23.775223 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4527/9083] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:21:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:21:27.206895 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a9afb-4381-4b09-ab0c-ed43898e683a service neutron] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:21:27.217014 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a9afb-4381-4b09-ab0c-ed43898e683a service neutron] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:21:27.221712 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5479e-99af-4a5d-9739-1b3a2a4b182a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:21:27.222640 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5479e-99af-4a5d-9739-1b3a2a4b182a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:21:27.222923 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5479e-99af-4a5d-9739-1b3a2a4b182a admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:21:27.223200 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd5479e-99af-4a5d-9739-1b3a2a4b182a admin admin] PATH_INFO: `/v3/users/ef1a345c9a0b4a3287ac4110fcf90e50` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:21:27.224388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5479e-99af-4a5d-9739-1b3a2a4b182a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef1a345c9a0b4a3287ac4110fcf90e50)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:21:27.226362 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd5479e-99af-4a5d-9739-1b3a2a4b182a admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:21:27.387592 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dfd5479e-99af-4a5d-9739-1b3a2a4b182a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef1a345c9a0b4a3287ac4110fcf90e50'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:27.394406 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dfd5479e-99af-4a5d-9739-1b3a2a4b182a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef1a345c9a0b4a3287ac4110fcf90e50'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:27.395073 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dfd5479e-99af-4a5d-9739-1b3a2a4b182a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef1a345c9a0b4a3287ac4110fcf90e50'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:27.395682 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dfd5479e-99af-4a5d-9739-1b3a2a4b182a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef1a345c9a0b4a3287ac4110fcf90e50'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:27.458955 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-dfd5479e-99af-4a5d-9739-1b3a2a4b182a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef1a345c9a0b4a3287ac4110fcf90e50'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:27.477028 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4557/9084] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 22:21:27 2022] DELETE /identity/v3/users/ef1a345c9a0b4a3287ac4110fcf90e50 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 22:21:27.715993 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7045f-3d58-4ac6-ba8c-a12670e9eab7 service placement] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:21:27.730568 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7045f-3d58-4ac6-ba8c-a12670e9eab7 service placement] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:21:27.788087 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a0d52-68ce-4cd3-9cad-3a947547b299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:21:27.789517 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a0d52-68ce-4cd3-9cad-3a947547b299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:21:27.789802 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a0d52-68ce-4cd3-9cad-3a947547b299 admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:21:27.790109 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a0d52-68ce-4cd3-9cad-3a947547b299 admin admin] PATH_INFO: `/v3/projects/c9313da576ba44f4a4fdb41a1306a1c2` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:21:27.791295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a0d52-68ce-4cd3-9cad-3a947547b299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9313da576ba44f4a4fdb41a1306a1c2)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:21:27.793218 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a0d52-68ce-4cd3-9cad-3a947547b299 admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:21:27.890964 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5b3a0d52-68ce-4cd3-9cad-3a947547b299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9313da576ba44f4a4fdb41a1306a1c2', 'request_id': 'req-5b3a0d52-68ce-4cd3-9cad-3a947547b299'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:27.896420 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5b3a0d52-68ce-4cd3-9cad-3a947547b299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9313da576ba44f4a4fdb41a1306a1c2', 'request_id': 'req-5b3a0d52-68ce-4cd3-9cad-3a947547b299'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:27.966012 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5b3a0d52-68ce-4cd3-9cad-3a947547b299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9313da576ba44f4a4fdb41a1306a1c2', 'request_id': 'req-5b3a0d52-68ce-4cd3-9cad-3a947547b299'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:27.990737 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5b3a0d52-68ce-4cd3-9cad-3a947547b299 admin admin] The token cache is being invalidate because project c9313da576ba44f4a4fdb41a1306a1c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 22:21:27.991030 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5b3a0d52-68ce-4cd3-9cad-3a947547b299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:27.991662 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-5b3a0d52-68ce-4cd3-9cad-3a947547b299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:27.993259 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4528/9085] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 22:21:27 2022] DELETE /identity/v3/projects/c9313da576ba44f4a4fdb41a1306a1c2 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 22:21:27.997295 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5479e-99af-4a5d-9739-1b3a2a4b182a admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:21:27.998730 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-dfd5479e-99af-4a5d-9739-1b3a2a4b182a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:21:28.011828 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd5479e-99af-4a5d-9739-1b3a2a4b182a admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:21:28.015338 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:21:28.070070 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9523f9-9cf0-4129-9c05-1052847d2f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:21:28.071157 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9523f9-9cf0-4129-9c05-1052847d2f12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:21:28.071612 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9523f9-9cf0-4129-9c05-1052847d2f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:21:28.072251 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9523f9-9cf0-4129-9c05-1052847d2f12 admin admin] PATH_INFO: `/v3/users/ef7e26f10e184cb5b47306c1e1fbb82a` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:21:28.073540 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9523f9-9cf0-4129-9c05-1052847d2f12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef7e26f10e184cb5b47306c1e1fbb82a)` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:21:28.075517 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9523f9-9cf0-4129-9c05-1052847d2f12 admin admin] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:21:28.272234 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9a9523f9-9cf0-4129-9c05-1052847d2f12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef7e26f10e184cb5b47306c1e1fbb82a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:28.279686 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9a9523f9-9cf0-4129-9c05-1052847d2f12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef7e26f10e184cb5b47306c1e1fbb82a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:28.280351 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9a9523f9-9cf0-4129-9c05-1052847d2f12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef7e26f10e184cb5b47306c1e1fbb82a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:28.280994 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9a9523f9-9cf0-4129-9c05-1052847d2f12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef7e26f10e184cb5b47306c1e1fbb82a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:28.373486 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.notifications [None req-9a9523f9-9cf0-4129-9c05-1052847d2f12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef7e26f10e184cb5b47306c1e1fbb82a'} {{(pid=103103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:28.392041 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4558/9086] 10.222.0.33 () {58 vars in 1223 bytes} [Sat May 28 22:21:27 2022] DELETE /identity/v3/users/ef7e26f10e184cb5b47306c1e1fbb82a => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 22:21:28.664412 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a0d52-68ce-4cd3-9cad-3a947547b299 admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:21:28.665984 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-5b3a0d52-68ce-4cd3-9cad-3a947547b299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:21:28.679411 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a0d52-68ce-4cd3-9cad-3a947547b299 admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:21:28.711407 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:21:28.736732 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94940546-4bb3-481d-83b6-d09e9ef6a41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e371762e9974f2484669486a37339c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a7d6b819de6044b98f2c78098814b8f2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:21:28.737602 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94940546-4bb3-481d-83b6-d09e9ef6a41a admin admin] REQUEST_METHOD: `DELETE` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:21:28.737914 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94940546-4bb3-481d-83b6-d09e9ef6a41a admin admin] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:21:28.738194 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94940546-4bb3-481d-83b6-d09e9ef6a41a admin admin] PATH_INFO: `/v3/projects/e4f22ffb71b747c2b9209b18128a1042` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:21:28.739373 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94940546-4bb3-481d-83b6-d09e9ef6a41a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4f22ffb71b747c2b9209b18128a1042)` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:21:28.741295 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94940546-4bb3-481d-83b6-d09e9ef6a41a admin admin] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:21:28.881340 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94940546-4bb3-481d-83b6-d09e9ef6a41a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4f22ffb71b747c2b9209b18128a1042', 'request_id': 'req-94940546-4bb3-481d-83b6-d09e9ef6a41a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:28.887464 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94940546-4bb3-481d-83b6-d09e9ef6a41a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4f22ffb71b747c2b9209b18128a1042', 'request_id': 'req-94940546-4bb3-481d-83b6-d09e9ef6a41a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:28.965649 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94940546-4bb3-481d-83b6-d09e9ef6a41a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4f22ffb71b747c2b9209b18128a1042', 'request_id': 'req-94940546-4bb3-481d-83b6-d09e9ef6a41a'} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:28.981433 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94940546-4bb3-481d-83b6-d09e9ef6a41a admin admin] The token cache is being invalidate because project e4f22ffb71b747c2b9209b18128a1042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=103104) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 28 22:21:28.981758 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94940546-4bb3-481d-83b6-d09e9ef6a41a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:28.982441 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.notifications [None req-94940546-4bb3-481d-83b6-d09e9ef6a41a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=103104) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 28 22:21:28.984003 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4529/9087] 10.222.0.33 () {58 vars in 1232 bytes} [Sat May 28 22:21:28 2022] DELETE /identity/v3/projects/e4f22ffb71b747c2b9209b18128a1042 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 28 22:21:38.388045 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=50,keystone:DELETE=10,keystone:INSERT=1 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:21:38.974137 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:24:40.181022 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9523f9-9cf0-4129-9c05-1052847d2f12 admin admin] Authenticating user token {{(pid=103103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:24:40.182456 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-9a9523f9-9cf0-4129-9c05-1052847d2f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:24:40.298543 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9523f9-9cf0-4129-9c05-1052847d2f12 admin admin] Validating token access rules against request {{(pid=103103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:24:40.356786 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e82b8f2-5983-4892-9d91-8bc1cd0e760a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:24:40.357388 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82b8f2-5983-4892-9d91-8bc1cd0e760a service placement] REQUEST_METHOD: `GET` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:24:40.357674 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82b8f2-5983-4892-9d91-8bc1cd0e760a service placement] SCRIPT_NAME: `/identity` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:24:40.357992 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e82b8f2-5983-4892-9d91-8bc1cd0e760a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:24:40.359772 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.fernet_utils [None req-8e82b8f2-5983-4892-9d91-8bc1cd0e760a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:24:40.371402 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82b8f2-5983-4892-9d91-8bc1cd0e760a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:24:40.372972 ncdv-master-1 devstack@keystone.service[103103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e82b8f2-5983-4892-9d91-8bc1cd0e760a service placement] RBAC: Authorization granted {{(pid=103103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:24:40.382734 ncdv-master-1 devstack@keystone.service[103103]: [pid: 103103|app: 0|req: 4559/9088] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:24:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:24:48.531829 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94940546-4bb3-481d-83b6-d09e9ef6a41a admin admin] Authenticating user token {{(pid=103104) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 28 22:24:48.533278 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-94940546-4bb3-481d-83b6-d09e9ef6a41a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:24:48.570896 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94940546-4bb3-481d-83b6-d09e9ef6a41a admin admin] Validating token access rules against request {{(pid=103104) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 28 22:24:48.576367 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898c6dd1-abd9-4e28-adeb-1f2867fbcfd6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b02348c479904bac893fa8d71d077550', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=103104) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 28 22:24:48.577008 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c6dd1-abd9-4e28-adeb-1f2867fbcfd6 service placement] REQUEST_METHOD: `GET` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 28 22:24:48.577292 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c6dd1-abd9-4e28-adeb-1f2867fbcfd6 service placement] SCRIPT_NAME: `/identity` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 28 22:24:48.577567 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898c6dd1-abd9-4e28-adeb-1f2867fbcfd6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=103104) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 28 22:24:48.579324 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.fernet_utils [None req-898c6dd1-abd9-4e28-adeb-1f2867fbcfd6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=103104) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 28 22:24:48.591246 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c6dd1-abd9-4e28-adeb-1f2867fbcfd6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 28 22:24:48.592828 ncdv-master-1 devstack@keystone.service[103104]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898c6dd1-abd9-4e28-adeb-1f2867fbcfd6 service placement] RBAC: Authorization granted {{(pid=103104) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 28 22:24:48.603028 ncdv-master-1 devstack@keystone.service[103104]: [pid: 103104|app: 0|req: 4530/9089] 10.222.0.33 () {60 vars in 1365 bytes} [Sat May 28 22:24:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 28 22:24:50.378635 ncdv-master-1 devstack@keystone.service[103103]: DEBUG dbcounter [-] [103103] Writing DB stats keystone:SELECT=80 {{(pid=103103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:24:58.598428 ncdv-master-1 devstack@keystone.service[103104]: DEBUG dbcounter [-] [103104] Writing DB stats keystone:SELECT=17 {{(pid=103104) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}